JP2004241879A - Apparatus and method for image processing, recording medium, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To suppress an udnerflow of a VBV buffer by using the VBV buffer at a maximum. <P>SOLUTION: A VBV buffer underflow determination part 42 obtains information on the picture type, the number of macro blocks (MB), the number of slices, and the format of an inputted picture and supplies the information to an arithmetic part 43 and makes the arithmetic part 43 calculate a margin and stores it into a margin size memory 44. The VBV buffer underflow determination part 42 reads the margin out of the margin size memory 44 according to the inputted picture type, compares the sum of the margin and a maximum quantity of bits generated in one picture with the occupation quantity of the VBV buffer to determine whether the VBV buffer is in an underflow state, and sends a command for minimum parameter processing when determining that the VBV buffer underflow is caused. This invention is applicable to an MPEG encoder. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

【００６９】
また、クロマフォーマットが４：２：２のとき、ＤＣ成分ＧＢ＿ＤＣは、以下の式（８）のように演算される。
【００７０】

【００７１】
さらに、スライス内の第１ＭＢのビット量と最終ＭＢのビット量の和ＧＢ＿ＳＭＢは、ＩＴＵ−Ｔ Ｒｅｃ．Ｈ．２６２（２０００Ｅ）により規定されるＭＰＥＧ２の場合、以下の式（９）のように定義される。
【００７２】

【００７３】
ここで、ｍａｃｒｏｂｌｏｃｋ＿ｅｓｃａｐｅ＿Ｆは第１ＭＢのＭＢアドレス拡張用のビット量を、ｍａｃｒｏｂｌｏｃｋ＿ａｄｄｒｅｓｓ＿ｉｎｃｒｅｍｅｎｔ＿Ｆは第１ＭＢの現ＭＢアドレスと前ＭＢアドレスの差を、ｑ＿ｓｃａｌｅ＿ｃｏｄｅは第１ＭＢのｑスケールコードを、ｍａｃｒｏｂｌｏｃｋ＿ｔｙｐｅ＿Ｆは第１ＭＢのＭＢ符号化タイプを、ｆｒａｍｅ（ｏｒ ｆｉｅｌｄ）＿ｍｏｔｉｏｎ＿ｔｙｐｅ＿Ｆは、フレーム構造の（または、フィールド構造の）動き補償タイプを、ｍｏｔｉｏｎ＿ｖｅｃｔｏｒ＿Ｆは、第１ＭＢの動きベクトルのビット量を示しており、＿Ｌが＿Ｆに代えて付されているものは、対応するそれぞれの最終ＭＢのビット量を示している。
【００７４】
ＩＴＵ−Ｔ Ｒｅｃ．Ｈ．２６２（２０００Ｅ）により規定されるＭＰＥＧ２の場合、ｍａｃｒｏｂｌｏｃｋ＿ｅｓｃａｐｅ＿Ｆは０ビットに、ｍａｃｒｏｂｌｏｃｋ＿ａｄｄｒｅｓｓ＿ｉｎｃｒｅｍｅｎｔ＿Ｆは１ビットに、ｑ＿ｓｃａｌｅ＿ｃｏｄｅは５ビットに（ｓｌｉｃｅの第１ＭＢはｑ＿ｓｃａｌｅ＿ｃｏｄｅが必要となる）、ｍａｃｒｏｂｌｏｃｋ＿ｔｙｐｅ＿Ｆは６ビットに（ＩＴＵ−Ｔ Ｒｅｃ．Ｈ．２６２（２０００Ｅ）の第１２２頁の Ｔａｂｌｅ Ｂ．２乃至Ｂ．４より、ｆｒａｍｅ＿ｍｏｔｉｏｎ＿ｔｙｐｅ＿Ｆまたはｆｉｅｌｄ＿ｍｏｔｉｏｎ＿ｔｙｐｅ＿Ｆは２ビットに、ｍｏｔｉｏｎ＿ｖｅｃｔｏｒ＿Ｆは３ビット（ｍｏｔｉｏｎ＿ｖｅｒｔｉｃａｌ＿ｆｉｅｌｄ＿ｓｅｌｅｃｔ［０］［ｓ］、ｍｏｔｉｏｎ＿ｃｏｄｅ［ｒ］［ｓ］［０］、および、ｍｏｔｉｏｎ＿ｃｏｄｅ［ｒ］［ｓ］［１］がそれぞれ１ビットずつ）に、ｍａｃｒｏｂｌｏｃｋ＿ｅｓｃａｐｅ＿Ｌは１１ビットに、ｍａｃｒｏｂｌｏｃｋ＿ａｄｄｒｅｓｓ＿ｉｎｃｒｅｍｅｎｔ＿Ｌは８ビットに、ｍａｃｒｏｂｌｏｃｋ＿ｔｙｐｅ＿Ｌは３ビットに、ｆｒａｍｅ＿ｍｏｔｉｏｎ＿ｔｙｐｅ＿Ｌ、または、ｆｉｅｌｄ＿ｍｏｔｉｏｎ＿ｔｙｐｅ＿Ｌは２ビットに、ｍｏｔｉｏｎ＿ｖｅｃｔｏｒ＿Ｌは３ビットに（ｍｏｔｉｏｎ＿ｖｅｒｔｉｃａｌ＿ｆｉｅｌｄ＿ｓｅｌｅｃｔ［０］［ｓ］、ｍｏｔｉｏｎ＿ｃｏｄｅ［ｒ］［ｓ］［０］および、ｍｏｔｉｏｎ＿ｃｏｄｅ［ｒ］［ｓ］［１］が１ビットずつ）それぞれ、規定されている（ＩＴＵ−Ｔ Ｒｅｃ．Ｈ．２６２（２０００Ｅ）の第３３，３４頁 ６．２．５章、６．２．５．１章、６．２．５．２章参照）。
【００７５】
このため、ＩＴＵ−Ｔ Ｒｅｃ．Ｈ．２６２（２０００Ｅ）により規定されるＭＰＥＧ２の場合、スライス内の第１ＭＢのビット量と最終ＭＢのビット量の和ＧＢ＿ＳＭＢは、以下の式（１０）のように求められる。
【００７６】

【００７７】
式（１０）より、スライス内の第１ＭＢのビット量と最終ＭＢのビット量の和ＧＢ＿ＳＭＢは、４４ビットの固定値となる。
【００７８】
以上をまとめると、例えば、ＩＴＵ−Ｔ Ｒｅｃ．Ｈ．２６２（２０００Ｅ）により規定されるＭＰＥＧ２の場合、クロマフォーマットが、４：２：０、および、４：２：２であるとき、Ｉピクチャのマージンは、式（１）に基づいて、以下の式（１１），式（１２）で、それぞれ求められることになる。
【００７９】
Ｍａｒｇｉｎ＿Ｉｐｉｃｔｕｒｅ＝９３３６α＋３８β＋１４９γ ・・・（１１）
Ｍａｒｇｉｎ＿Ｉｐｉｃｔｕｒｅ＝１２４０８α＋３８β＋１９９γ ・・・（１２）
【００８０】
また、同様にして、Ｂピクチャ、または、Ｐピクチャのマージンは、クロマフォーマットが、４：２：０、および、４：２：２であるとき、式（２）に基づいて、以下の式（１３），式（１４）で、それぞれ求められることになる。
【００８１】

【００８２】このように、マージンは、クロマフォーマットが確定すると、ハードウェアにより制御できないＭＢ数α、スライス数β、および、ＭＢ数γに基づいて、算出することが可能となる。ここで、ＭＢ数αは、ハードウェア固有の値として設定されるため、その値は、予め設定される。
【００８３】
また、演算部４３は、Ｉピクチャ、並びに、Ｂピクチャ、または、Ｐピクチャ毎に、クロマフォーマットに対応した係数を予め内蔵するメモリに記憶しており、ピクチャタイプ決定部１１より入力されるクロマフォーマットとピクチャタイプの情報に基づいてその係数を読み出すと共に、同時にピクチャタイプ決定部１１より送信されてくるスライス数、および、ＭＢ数の情報に基づいて、マージンを計算し、マージンサイズメモリ４４に記憶させる。
【００８４】
１つのストリームにおいては、スライス数やＭＢ数は変化しないため、演算部４３は、Ｉピクチャ、並びに、Ｐピクチャ、または、Ｂピクチャが最初に入力されたときにのみ、マージンの計算を行ってマージンサイズメモリ４４に記憶させる。ＶＢＶアンダーフロー判定部４２は、上述のように演算されてサイズメモリ４４に記憶されているマージンを用いてＶＢＶアンダーフローの有無を判定する。
【００８５】
次に、図８のフローチャートを参照して、レート制御部３によるＶＢＶバッファアンダーフロー監視処理について説明する。
【００８６】
ステップＳ１において、レート制御部３のＶＢＶバッファアンダーフロー判定部４２は、ピクチャタイプ決定部１１より入力されてくるピクチャタイプ、ＭＢ数、スライス数、および、クロマフォーマットの情報を取得する。
【００８７】
ステップＳ２において、ＶＢＶバッファアンダーフロー判定部４２は、マージンサイズメモリ４４に、既に、マージンサイズが演算されているか否かを判定する。このとき、ＶＢＶバッファアンダーフロー判定部４２は、ステップＳ１において、取得したピクチャタイプのマージンサイズが既に演算されて、記憶されているか否かを判定する。
【００８８】
ステップＳ２において、マージンサイズが演算されていないと判定された場合、ステップＳ３において、ＶＢＶバッファアンダーフロー判定部４２は、演算部４３に取得したピクチャタイプ、ＭＢ数、スライス数、および、クロマフォーマットの情報を供給し、上述の式（１）、または、式（２）の演算を実行させ、演算結果をマージンサイズメモリ４４に記憶させる。
【００８９】
ステップＳ４において、ＶＢＶバッファアンダーフロー判定部４２は、入力された画像がＩピクチャであるか否かを判定し、例えば、Ｉピクチャであると判定された場合、その処理は、ステップＳ５に進む。
【００９０】
ステップＳ５において、ＶＢＶバッファアンダーフロー判定部４２は、マージンサイズメモリ４４に記憶されているＩピクチャのマージンサイズを読み出す。
【００９１】
ステップＳ６において、ＶＢＶバッファアンダーフロー判定部４２は、ＶＢＶバッファの占有量が、引き出されるピクチャの最大発生ビット量ＭａｘＧｅｎＢｉｔとマージンとの和よりも大きいか否かを判定し、例えば、ＶＢＶバッファの占有量が、引き出されるピクチャの最大発生ビット量ＭａｘＧｅｎＢｉｔとマージンとの和よりも大きくない、すなわち、ＶＢＶバッファの占有量が、引き出されるピクチャの最大発生ビット量ＭａｘＧｅｎＢｉｔとマージンとの和よりも小さいと判定された場合、その処理は、ステップＳ７に進む。
【００９２】
ステップＳ７において、ＶＢＶバッファアンダーフロー判定部４２は、ＶＢＶバッファアンダーフローが発生するとみなし、重み付け量子化部１５に対して、最少パラメータ処理を実行するように指令する。
【００９３】
ステップＳ８において、ＶＢＶバッファアンダーフロー判定部４２は、次のピクチャが存在するか否かを判定し、次のピクチャが存在する場合、その処理は、ステップＳ１に戻り、次のピクチャが存在しない場合、ステップＳ９において、マージンサイズメモリ４４をリセットして、その処理を終了する。
【００９４】
また、ステップＳ２において、マージンサイズが演算されている、すなわち、Ｉピクチャが入力された場合、Ｉピクチャのマージンサイズが、また、Ｐピクチャ、または、Ｂピクチャのマージンサイズが入力された場合、Ｐピクチャ、または、Ｂピクチャのマージンサイズが、既に、演算されているとき、ステップＳ３の処理は、スキップされる。
【００９５】
さらに、ステップＳ４において、Ｉピクチャではない、すなわち、Ｂピクチャ、または、Ｐピクチャであった場合、ステップＳ１０において、ＶＢＶバッファアンダーフロー判定部４２は、マージンサイズメモリ４４に記憶されているＢピクチャ、または、Ｐピクチャのマージンサイズを読み出す。
【００９６】
また、ステップＳ６において、ＶＢＶバッファの占有量が、引き出されるピクチャの最大発生ビット量ＭａｘＧｅｎＢｉｔとマージンとの和よりも大きいと判定された場合、ステップＳ７の処理はスキップされ、その処理は、ステップＳ８に進む。
【００９７】
すなわち、以上の処理をまとめると、ステップＳ３により、演算部４３が、Ｉピクチャ、もしくは、Ｐピクチャ、または、Ｂピクチャのマージンサイズを演算する。このとき、演算部４３は、上述のように、クロマフォーマットに対応する係数ＧＢ＿ＭＢ、ＧＢ＿ＨＤ、ＧＢ＿ＤＣを予め記憶しているので、ステップＳ１の処理で取得されるクロマフォーマットとピクチャタイプの情報に応じて選択し、さらに、予め設定されている、発生してしまうＭＢ数αと、ステップＳ１の処理で取得されたスライス数β、および、ＭＢ数γを用いて、式（１）、または、式（２）を演算してマージンサイズを演算する。
【００９８】
ただし、ステップＳ２の処理により、マージンサイズメモリ４４に記憶されたピクチャタイプのマージンサイズは、ステップＳ３がスキップされることにより、それ以降の処理においては、演算されないことになるので、１ストリームの処理については、ピクチャタイプ毎に１回だけ演算されることになる。
【００９９】
ステップＳ５，Ｓ１０において、ピクチャタイプ毎にマージンサイズが読み出される。そして、ステップＳ６において、最大発生ビット量ＭｅｘＧｅｎＢｉｔと読み出されたマージンサイズの和が、ＶＢＶバッファの占有量と比較され、ＶＢＶバッファの占有量が、引き出されるピクチャの最大発生ビット量ＭａｘＧｅｎＢｉｔとマージンの和よりも小さい場合、最少パラメータ処理がなされるように指令が出される。
【０１００】
結果として、これまで、経験的に設定されていたマージンが、ストリーム毎に発生される最大のビット量として演算されて設定されることになるので、過不足のないマージン設定が可能となるため、ＶＢＶバッファを最大限利用することができ、ＶＢＶバッファのアンダーフローの発生をほぼ１００％抑制することが可能となる。
【０１０１】
上述した一連の処理は、ハードウェアにより実行させることもできるが、ソフトウェアにより実行させることもできる。一連の処理をソフトウェアにより実行させる場合には、そのソフトウェアを構成するプログラムが、専用のハードウェアに組み込まれているコンピュータ、または、各種のプログラムをインストールすることで、各種の機能を実行させることが可能な、例えば汎用のパーソナルコンピュータなどに記録媒体からインストールされる。
【０１０２】
図９は、エンコーダをソフトウェアにより実現する場合のパーソナルコンピュータの一実施の形態の構成を示している。パーソナルコンピュータのＣＰＵ１０１は、パーソナルコンピュータの全体の動作を制御する。また、ＣＰＵ１０１は、バス１０４および入出力インタフェース１０５を介してユーザからキーボードやマウスなどからなる入力部１０６から指令が入力されると、それに対応してＲＯＭ（Ｒｅａｄ Ｏｎｌｙ Ｍｅｍｏｒｙ）１０２に格納されているプログラムを実行する。あるいはまた、ＣＰＵ１０１は、ドライブ１１０に接続された磁気ディスク１１１、光ディスク１１２、光磁気ディスク１１３、または半導体メモリ１１４から読み出され、記憶部１０８にインストールされたプログラムを、ＲＡＭ（Ｒａｎｄｏｍ Ａｃｃｅｓｓ Ｍｅｍｏｒｙ）１０３にロードして実行する。これにより、上述したエンコーダの機能が、ソフトウェアにより実現されている。さらに、ＣＰＵ１０１は、通信部１０９を制御して、外部と通信し、データの授受を実行する。
【０１０３】
プログラムが記録されている記録媒体は、図９に示すように、コンピュータとは別に、ユーザにプログラムを提供するために配布される、プログラムが記録されている磁気ディスク１１１（フレキシブルディスクを含む）、光ディスク１１２（ＣＤ−ＲＯＭ（Ｃｏｍｐａｃｔ Ｄｉｓｋ−Ｒｅａｄ Ｏｎｌｙ Ｍｅｍｏｒｙ），ＤＶＤ（Ｄｉｇｉｔａｌ Ｖｅｒｓａｔｉｌｅ Ｄｉｓｋ）を含む）、光磁気ディスク１１３（ＭＤ（Ｍｉｎｉ−Ｄｉｓｃ）を含む）、もしくは半導体メモリ１１４などよりなるパッケージメディアにより構成されるだけでなく、コンピュータに予め組み込まれた状態でユーザに提供される、プログラムが記録されているＲＯＭ１０２や、記憶部１０８に含まれるハードディスクなどで構成される。
【０１０４】
尚、本明細書において、記録媒体に記録されるプログラムを記述するステップは、記載された順序に沿って時系列的に行われる処理は、もちろん、必ずしも時系列的に処理されなくとも、並列的あるいは個別に実行される処理を含むものである。
【０１０５】
【発明の効果】
本発明によれば、ＶＢＶバッファを最大限利用することができ、ＶＢＶバッファアンダーフローをほぼ１００％抑制することが可能となる。
【図面の簡単な説明】
【図１】ＶＢＶバッファを説明する図である。
【図２】ＶＢＶバッファアンダーフローを説明する図である。
【図３】ＶＢＶバッファのマージンを説明する図である。
【図４】ＶＢＶバッファのマージンを説明する図である。
【図５】本発明を適用したエンコーダの構成を示すブロック図である。
【図６】図５のエンコーダによる処理を説明する図である。
【図７】図５のレート制御部の構成を示すブロック図である。
【図８】図７のレート制御部によるＶＢＶバッファアンダーフロー監視処理を説明するフローチャートである。
【図９】媒体を説明する図である。
【符号の説明】
１ 基本符号化処理部， ２ エンコーダ送出バッファ，３ レート制御部，１１ ピクチャタイプ決定部， １２ 走査変換部， １４ ＤＣＴ変換部， １５ 重み付け量子化部， １６ 可変長符号化部， ２０ モード判定部， ４１ 制御部， ４２ ＶＢＶバッファアンダーフロー判定部， ４３ 演算部，４４ マージンサイズメモリ[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image processing apparatus and method, a recording medium, and a program, and more particularly to an image processing apparatus that can suppress a VBV (Video Buffer Verifier) buffer underflow when image information is compressed by an MPEG (Moving Picture Experts Group) method. The present invention relates to an image processing apparatus and method, a recording medium, and a program.
[0002]
[Prior art]
2. Description of the Related Art In recent years, image compression / decompression techniques based on the MPEG system have been widely used.
[0003]
An MPEG bit stream is generated by compressing an image by the MPEG method, and the generated MPEG bit stream is transmitted to a receiving-side device via a transmission path. At this time, the transmitting device that compresses the image by the MPEG method and transmits the image is monitoring whether or not the MPEG bit stream to be transmitted is sufficiently reproducible by the receiving device. Sending.
[0004]
More specifically, as shown in FIG. 1, the transmitting device, that is, the encoder, virtually reproduces the state of the buffer used when the receiving device (decoder) receives the MPEG bit stream using a VBV buffer. Then, encoding is performed while monitoring to prevent occurrence of buffer overflow or buffer underflow.
[0005]
That is, the occupancy of the VBV buffer changes as shown in FIG. Here, in FIG. 2, the vertical axis indicates the occupancy of the VBV buffer, and the horizontal axis indicates time. Therefore, the slope of the straight line in FIG. 2 indicates the bit rate. When the VBV buffer receives an MPEG bit stream up to its maximum capacity (Max1 in the figure), the occupancy becomes state S1. When an MPEG bit stream for one picture (I picture (intra picture), P picture (forward predicted picture), or B picture (bidirectional predicted picture)) is extracted from the VBV buffer in this state S1 for decoding. , The state changes from the state S1 to the state S2. If the amount of generation for one picture does not exceed the lower limit of the VBV buffer, the decoder can receive all data for one picture and can decode normally.
[0006]
Subsequently, after the time T (= 1 / frame rate) elapses, when the MPEG bit stream is put into the VBV buffer, the occupation amount becomes the state S3. Here, when the MPEG bit stream for one picture is extracted from the VBV buffer, when the data amount (L1 + L2) in the figure is the data amount for one picture, in the state S3, only the data for the data amount L1 in the figure is used. The MPEG bit stream is not stored in the VBV buffer, and as shown in state S4, the state of the MPEG bit stream is such that the data amount L2 is insufficient for the MPEG bit stream for one picture, and the MPEG bit stream is interrupted. become. For this reason, a decoder serving as a device on the stream receiving side cannot decode one picture in the middle. A state in which the amount of data stored in the VBV buffer is less than one picture and decoding becomes impossible is called a VBV buffer underflow.
[0007]
The encoder controls the stream so that the amount of one picture generated is kept within the maximum amount of generated bits (MaxGenBit = L1) at the time of the state S3 so that the VBV buffer underflow does not occur. It is not encoded.
[0008]
One method of encoding so as not to cause VBV buffer underflow (encoding while controlling the stream so that the amount of one picture generated is kept within the maximum amount of generated bits (MaxGenBit = L1)) is one frame bit. In order to suppress the generation amount, it has been proposed to perform a compression process (hereinafter, referred to as a minimum parameter process) by minimizing a parameter generated by encoding in units of macroblocks (MB) (minimum parameter). The minimum parameter processing differs depending on the picture type. In the case of an I picture, processing is performed using only DC (Direct Current) components of all MBs, and in the case of a B picture or a P picture, all MBs are skipped. Then, a process for making a macro block (skipped MB) is executed. Here, the skipped macroblock is data composed of only the first macroblock header (MB header) of each slice and the last MB header of each slice in each picture.
[0009]
In addition, there is a technique that enables encoding at a target bit rate while stabilizing the image quality of a reproduced image (for example, see Patent Document 1).
[0010]
[Patent Document 1]
JP-A-10-155152
[0011]
[Problems to be solved by the invention]
However, as described above, even if the generated bit amount is minimized by the minimum parameter processing, the generated bit amount cannot be reduced to 0, so that the occupation amount of the VBV buffer is reduced to the data amount L1 shown in FIG. Even if the compression processing is performed using the minimum number of parameters, the entire amount of generated bits for one picture does not completely fit within the maximum amount of generated bits L1. There was a risk that L2 would occur.
[0012]
Therefore, as shown in FIG. 3, by providing a margin in the VBV buffer, the generated bit amount exceeds the margin + maximum generated bit amount by comparing the margin + maximum generated bit amount with the VBV buffer occupation amount. In such a case, a method has been proposed in which the processing with the minimum parameters as described above is executed to avoid VBV buffer underflow.
[0013]
That is, as shown in FIG. 4, the maximum capacity Max2 (= Max1 + margin) of the VBV buffer is set. Then, in state S1 ', the MPEG bit stream is stored up to the maximum capacity Max2, and when data of one picture is extracted from this state, the state changes to state S2'. In the same manner as described above, when the state S3 'is reached after the lapse of the time T and the data for one picture is extracted again, the occupation amount of the VBV buffer is changed to the data amount up to the margin level. Since L1 is less than the data amount of the MPEG bit stream for one picture as described above, compression processing is performed with the minimum number of parameters. At this time, since the margin is expected, the compression processing is performed with the minimum number of parameters. Since the data amount L2 is absorbed by the margin, as a result, VBV buffer underflow does not occur.
[0014]
By the way, the above-described setting of the margin is performed by obtaining a minimum parameter from a plurality of predetermined setting images in advance, that is, by a so-called tuning process, obtaining an amount of generated bits generated by the minimum parameter, and setting the maximum bit amount as the margin. It is so. However, such a setting of the margin by the tuning process may not be able to cope with the data amount of the MPEG bit stream of an unknown screen which is not included in a plurality of known setting screens. Due to the data amount of the MPEG bit stream of the image, the generated bit amount of the minimum parameter may be larger than the value set as the margin, and as a result, the occurrence of the VBV buffer underflow cannot always be suppressed. was there.
[0015]
In addition, since the margin varies depending on the image used for tuning, the margin may be set larger than necessary. As a result, although the VBV buffer underflow can be suppressed, the set VBV There was a possibility that the capacity of the buffer could not be used to the maximum.
[0016]
The present invention has been made in view of such a situation, and aims to maximize the use of a VBV buffer and suppress the VBV buffer underflow.
[0017]
[Means for Solving the Problems]
An image processing apparatus according to the present invention includes: a determination unit configured to determine a picture type of an input image; a calculation unit configured to calculate a margin of a VBV buffer for each picture type based on the image; and a picture type calculated by the calculation unit. A storage unit for storing a margin of another VBV buffer; a compression unit for compressing an image by the MPEG method; image information compressed by the compression unit; a picture type determined by the determination unit; and a margin of the VBV buffer. An underflow detection means for detecting the presence or absence of a VBV buffer underflow, and a control means for controlling the compression means so as to compress the image with a minimum number of parameters when the underflow detection means detects the presence of the VBV buffer underflow. It is characterized by having.
[0018]
The calculating means may calculate a margin of each VBV buffer when the picture type is an I picture and a P picture or a B picture based on an image.
[0019]
The margin of the VBV buffer can be set to a minimum bit amount generated when the control unit controls the compression unit to compress the image with the minimum number of parameters.
[0020]
When the picture type is an I picture, all the macroblocks are DC components, and when the picture type is a P picture or a B picture, all the macroblocks are skipped macroblocks. Can be controlled so as to compress with a minimum parameter.
[0021]
The image processing method according to the present invention includes a determining step of determining a picture type of an input image, a calculating step of calculating a margin of a VBV buffer for each picture type based on the image, and a calculating step. A storage step of storing a VBV buffer margin for each picture type, a compression step of compressing an image by the MPEG method, image information compressed in the processing of the compression step, a picture type determined in the processing of the determination step, and VBV In the process of the underflow detection step for detecting the presence or absence of the VBV buffer underflow based on the buffer margin and the processing of the underflow detection step, if the presence of the VBV buffer underflow is detected, the image is compressed with the minimum number of parameters. Control switch that controls the processing of the compression step Tsu, characterized in that it comprises a flop.
[0022]
A recording medium program according to the present invention includes a determining step of determining a picture type of an input image, a calculating step of calculating a margin of a VBV buffer for each picture type based on the image, and a compression method of compressing the image by the MPEG method. An underflow detection step for detecting the presence or absence of a VBV buffer underflow based on the step, image information compressed in the processing of the compression step, the picture type determined in the processing of the determination step, and a margin of the VBV buffer; A control step of controlling the processing of the compression step so as to compress the image with a minimum number of parameters when the VBV buffer underflow is detected in the processing of the flow detection step.
[0023]
The program according to the present invention includes a determining step of determining a picture type of an input image, a calculating step of calculating a margin of a VBV buffer for each picture type based on the image, and a compressing step of compressing the image by the MPEG method. An underflow detection step for detecting the presence or absence of a VBV buffer underflow based on the image information compressed in the compression step processing, the picture type determined in the determination step processing, and a margin of the VBV buffer; In the above processing, when the presence of the VBV buffer underflow is detected, the computer is caused to execute a processing including a control step of controlling the processing of the compression step so as to compress the image with the minimum number of parameters.
[0024]
In the image processing apparatus and method and the program according to the present invention, the picture type of the input image is determined, the margin of the VBV buffer is calculated for each picture type based on the image, and the calculated VBV for each picture type is calculated. The buffer margin is stored, the image is compressed by the MPEG method, and the presence or absence of the VBV buffer underflow is detected based on the compressed image information, the determined picture type, and the VBV buffer margin, and the VBV buffer underflow is detected. When the presence is detected, control is performed so that the image is compressed with the minimum number of parameters.
[0025]
BEST MODE FOR CARRYING OUT THE INVENTION
FIG. 5 shows a configuration of an embodiment of the basic encoding processing unit 1 of the encoder to which the present invention is applied.
[0026]
The basic encoding processing unit 1 of the encoder in FIG. 5 encodes the image data by the MPEG method, and outputs the encoded data to the encoder transmission buffer 2. The rate control unit 3 controls a quantization width q_scale when the basic encoding processing unit 1 encodes image data by the MPEG method.
[0027]
Here, the outline of the encoding process of the MPEG system will be described with reference to FIG. Each picture of the image data to be encoded is divided into a plurality of macroblocks and encoded. One macro block is composed of 16 × 16 pixel block data (further divided into four 8 × 8 pixel blocks which are basic encoding processing units) for luminance, and two 8 × 8 pixel blocks which are basic encoding processing units for color difference. The data is divided into data of × 8 pixel blocks, and these blocks are encoded. In one macroblock, the coding method and quantization width of the blocks in the macroblock are determined.
[0028]
A slice is a data unit including a plurality of macroblocks, and one picture is composed of a plurality of slices. A picture is encoded by an I-picture (intra-picture) in which the picture itself is encoded as it is, a P-picture to be encoded after predicting a motion from a temporally past picture, or a temporally past picture. , And a B picture to be coded after predicting motion from both or one of the future pictures.
[0029]
The arrangement of each I, P, and B picture in FIG. 6 is a typical example, and a P picture three frames ahead is predicted and coded using the first I picture, and each B picture included therebetween is predicted. Are predicted and coded from both I and P pictures. Therefore, an I picture is encoded first, then a P picture is encoded, and then a B picture is encoded. For this reason, the order of each I, P, B picture along the original time progress is changed, and then these pictures are coded.
[0030]
Further, a GOP (group of pictures) including a plurality of pictures starting from an I picture is formed, and one video sequence is formed by an arbitrary number of GOPs.
[0031]
The picture type determination unit 11 determines and determines the picture type of each picture of the input image, rearranges the encoding order, outputs the rearranged state to the scan conversion unit 12 and the motion detection unit 21. I do. Further, the picture type determination unit 11 outputs the determined picture type, the number of MBs (macroblocks), the number of slices, and the information of the chroma format of the picture to the rate control unit 3.
[0032]
The scan converter 12 converts the rearranged pictures into macroblocks, which are units to be coded, and sequentially outputs these macroblocks to the subtractor 13. The subtractor 13 obtains a difference between the macroblock input from the scan conversion unit 12 and the prediction data from the prediction unit 19 with motion compensation, obtains this difference as a prediction error, and sends the difference to a DCT (Discrete Cosine Transform) conversion unit 14. Output.
[0033]
The DCT transform unit 14 performs a DCT transform on the prediction error in units of 8 × 8 pixel blocks based on the mode decision result from the mode decision unit 20, and weights each transform coefficient obtained by the DCT transform into a weighted quantization unit 15. Output to
[0034]
The weighting quantization unit 15 quantizes each transform coefficient based on the quantization width q_scale input from the rate control unit 3 and quantizes the obtained quantization data into a variable-length encoding unit 16 and an inverse quantization unit. 17 is output. Further, when the weighting / quantizing unit 15 receives an instruction for encoding with the minimum parameters from the rate control unit 3, the weighting / quantizing unit 15 executes the encoding process with the minimum parameters as described above. Only a P picture or a B picture is coded by a skipped macroblock to reduce the coding amount.
[0035]
The variable-length coding unit 16 performs variable-length coding on the quantized data based on the mode determination result from the mode determining unit 20 and the motion vector from the motion detecting unit 21 to convert the compression-coded image data. The compressed and encoded image data is temporarily stored in the encoder transmission buffer 2 for transmission at a desired transmission rate and output. More specifically, the variable length encoding unit 16 sends an MB end timing signal to the rate control unit 3 each time encoding of each macroblock ends, and every time encoding of one picture starts, The picture start timing signal is sent to the rate control unit 3.
[0036]
The inverse quantization unit 17 inversely quantizes the image data that has been weighted and quantized, and outputs the image data to the inverse DCT transform unit 18. The inverse DCT transform unit 18 performs an inverse DCT transform on the input dequantized image data, and outputs the image data to the adder 22.
[0037]
The adder 22 adds the prediction data input from the prediction unit with motion compensation 19 and the image data subjected to the inverse DCT transformation, and outputs the result to the prediction unit with motion compensation 19. The prediction unit 19 with motion compensation includes the reproduced image data to which the prediction data input from the adder 22 has been added, the motion vector input from the motion detection unit 21, and the mode input from the mode determination unit 20. Based on the determination result, it forms prediction data indicating a picture whose motion has been predicted, and outputs the prediction data to the subtractor 13 and the adder 22.
[0038]
The motion detection processing unit 21 calculates a motion vector of an image for each macroblock, inputs the motion vector to the prediction unit with motion compensation 19, and outputs the motion vector to the mode determination unit 20.
[0039]
The mode determination unit 20 determines a motion compensation prediction mode based on the motion vector input from the motion detection unit 21 and outputs the mode to the DCT transform unit 14, the variable length encoding unit 16, and the prediction unit 19 with motion compensation. I do.
[0040]
The rate control unit 3 determines the above-described margin size of the VBV buffer based on the picture type, the number of MBs, the number of slices, and the chroma format information input from the picture type determination unit 11, and The presence / absence of VBV buffer underflow is determined based on the bit generation amount of the bit stream output from the bit stream 16, and the above-described processing with the minimum parameters is performed on the weighting quantization unit 15 in accordance with the VBV buffer underflow determination result. Output a command to
[0041]
Next, a detailed configuration of the rate control unit 3 in FIG. 5 will be described with reference to the block diagram in FIG.
[0042]
Each time the MB end timing signal is input, the control unit 41 obtains the actual bit generation amount per macroblock based on the bit stream from the variable length coding unit 16, and adds it as the occupation amount of the VBV buffer. In addition, the quantization width q_scale is obtained for each macroblock based on the occupation amount of the VBV buffer, and supplied to the VBV buffer underflow determination unit 42. The data is supplied to the quantization unit processing unit 15.
[0043]
More specifically, based on the input MB end timing signal and picture start timing signal, the control unit 41 determines the actual number of MBs per macroblock based on the MB count value at each MB end timing signal. The bit generation amount is obtained and added as the occupation amount of the VBV buffer.
[0044]
The VBV buffer underflow determination unit 42 controls the calculation unit 43 based on the picture type, the number of MBs, and the number of slices input from the picture type determination unit 11 to set the maximum value of the VBV buffer. A margin corresponding to the I picture and the B or P picture is calculated and stored in the margin size memory 44. The VBV buffer underflow determination unit 42 sets the maximum value (the maximum value Max2 shown in FIG. 4) of the VBV buffer based on the margin size stored in the margin size memory 44, and is obtained by the control unit 41. The occupancy of the VBV buffer is monitored, and it is determined whether or not a VBV buffer underflow occurs. Then, the VBV buffer underflow determination unit 42 outputs the underflow determination result to the weight quantization unit 15 when approaching the state of the VBV buffer underflow according to the determination result.
[0045]
Next, a method of calculating the margin size by the calculation unit 43 will be described.
[0046]
As the margin size, it is sufficient to set the generated bit amount that always occurs by the compression method using the minimum parameter. Therefore, in the case of an I picture, the amount of generated bits represented by the following equation (1) is calculated as the margin Margin_Ipicture.
[0047]
Margin_Ipicture = α × GB_MB + β × GB_HD + γ × GB_DC (1)
[0048]
Here, GB_MB indicates the bit amount generated by pipeline delay in the chip, GB_HD indicates the bit amount generated by the header, and GB_DC indicates the bit amount generated by the DC component in the I picture. . α indicates the number of macroblocks (a constant depending on hardware) that cannot be controlled and is generated (until the CPU that controls the bit generation amount detects the generated bit amount for each 1 MB). (The number of MBs that can be generated (cannot be controlled) within the time period). β indicates the number of slices, and γ indicates the number of macroblocks.
[0049]
On the other hand, in the case of a B picture or a P picture, the generated bit amount represented by the following equation (2) is calculated as a margin Margin_B / Picture.
[0050]
Margin_B / Picture = α × GB_MB + β × GB_HD + βGB_SMB (2)
[0051]
Here, GB_SMB indicates the sum of the bit amount of the first MB and the bit amount of the last MB in the slice.
[0052]
For example, when the chroma format is a: b: c, GB_MB is obtained as follows.
[0053]
GB_MB = (maximum bit amount of DCT coefficient) × (a + b + c)
× (number of pixels in one block) + (number of bits generated in MB header) (3)
[0054]
Here, the maximum bit amount of the DCT coefficient is, for example, as defined in ITU-T Rec. H. 262 (2000E) is the maximum value of First_DCT_coefficient (first non-zero DCT coefficient of non-intra block (inter block)) or Subsequent_DCT_coefficients (subsequent DCT coefficient). Note that the above-mentioned maximum value is a value of an inter macro block, but is larger than a value of an intra macro block. Therefore, the value may be used in all cases. ITU-T Rec. H. In H.262 (2000E), First_DCT_coefficient is defined as 2 to 24 bits, and Subsequent_DCT_coefficients is 3 to 24 bits, so that it may be 24 bits (36th of ITU-T Rec. H.262 (2000E)). See page 6.2.6).
[0055]
The number of pixels in one block is 64 in MPEG2 because it is 8 pixels × 8 pixels.
[0056]
Further, the amount of bits generated in the MB header is, for example, ITU-T Rec. H. In the case of MPEG2 defined by H.262 (2000E), macroblock_address_increment (difference between the current MB address and the previous MB address), quantizer_scale_code (MB quantization scale code), marker_bit (marker), macroblock type (macroblock type_ipalp_alp_ipal) (Space-time weighting code for up-sampling), frame_motion_type (motion compensation type of frame structure), dct_type (DCT type (frame or field), motion_vertical_field_select [0] [s] (selection information of reference field used for prediction), motion_ver physical_field_select [1] [s] (reference field selection information used for prediction), motion_code [0] [s] [0] (basic difference motion vector when the motion vector is motion_vector (0, s)), motion_residual [ 0] [s] [0] (residual vector when the motion vector is motion_vector (0, s)), dmvector [0] (dual-prime difference vector when the motion vector is motion_vector (0, s)) ), Motion_code [0] [s] [1] (basic difference motion vector when the motion vector is motion_vector (0, s)), motion_residual [0] [s] [1] (motion vector is mo residual_vector (0, s), dmvector [1] (dual-prime difference vector when the motion vector is motion_vector (0, s)), motion_code [1] [s] [0] (Basic difference motion vector when motion vector is motion_vector (1, s)), motion_residual [1] [s] [0] (residual vector when motion vector is motion_vector (1, s)), dmvector [0] (dual-prime difference vector when the motion vector is motion_vector (1, s)), motion_code [1] [s] [1] (when the motion vector is motion_vector (1, s)) This difference motion vector), motion_residual [1] [s] [1] (residual vector when the motion vector is motion_vector (1, s)), and dmvector [1] (motion vector is motion_vector (1, s)) This is the sum of the respective bit amounts of the dual-difference vector for a given case), coded_block_pattern_420 (code block pattern), and coded_block_pattern_2 (code block pattern). Note that s is a parameter that indicates forward prediction when it is 0, and that indicates bidirectional prediction when it is 1. Since s is any parameter, it is displayed as s.
[0057]
ITU-T Rec. H. In the case of H.262 (2000E), macroblock_address_increment (the difference between the current MB address and the previous MB address) is 1 bit, quantizer_scale_code (MB quantization scale code) is 5 bits, marker_bit (marker) is 1 bit, macroblock type (macroblock type). ) Is 9 bits, spatial_temporal_weight_code (spatio-temporal weighting code for up-sampling) is 2 bits, frame_motion_type (frame structure motion compensation type) is 2 bits, dct_type (DCT type (frame or field) is 1 bit, motion_version_motion_version_motion_version_motion_version_motion_version). ] [S] (reference used for prediction Field_selection information is 1 bit, motion_vertical_field_select [1] [s] (reference field selection information used for prediction) is 1 bit, motion_code [0] [s] [0] (motion vector is motion_vector (0, s)) Is 11 bits, motion_residual [0] [s] [0] (residual vector when the motion vector is motion_vector (0, s)) is 8 bits, and dmvector [0] ( When the motion vector is motion_vector (0, s), the difference vector for dual prime is 2 bits, and motion_code [0] [s] [1] (when the motion vector is motion_vector (0, s)). Is 11 bits, motion_residual [0] [s] [1] (residual vector when the motion vector is motion_vector (0, s)) is 8 bits, and dmvector [1] (motion vector is The motion_vector (0, s) is a dual-prime difference vector when the motion vector is (0, s), and the motion_code [1] [s] [0] is a basic difference motion vector when the motion vector is the motion_vector (1, s). 11 bits, motion_residual [1] [s] [0] (residual vector when the motion vector is motion_vector (1, s)) is 8 bits, and dmvector [0] (motion vector is motion_vector (1, s)). Motion_code [1] [s] [1] (the basic difference motion vector when the motion vector is motion_vector (1, s)) is 11 bits, and motion_residual [1]. [S] [1] (residual vector when the motion vector is motion_vector (1, s)) is 11 bits, and dmvector [1] (dual-prime difference when the motion vector is motion_vector (1, s)) Vector) is defined as 2 bits, coded_block_pattern_420 (code block pattern) is defined as 9 bits, and coded_block_pattern_2 (code block pattern) is defined as 6 bits. In it may be a 120-bit (ITU-T Rec. H. 262 (2000E), pp. 33-36, see section 6.2.5).
[0058]
The bit amount GB_MB generated by the pipeline delay in the chip is, for example, ITU-TRec. H. In the case of MPEG2 defined by H.262 (2000E), when the chroma format is 4: 2: 0, it is obtained as in the following equation (4).
[0059]
GB_MB = 24 × (4 + 2 + 0) × 64 + 120 = 9336 (bits) (4)
[0060]
When the chroma format is 4: 2: 2, it is obtained as in the following equation (5).
[0061]
GB_MB = 24 × (4 + 2 + 2) × 64 + 120 = 12408 (bits) (5)
[0062]
Further, since GM_HD is the bit amount of the header, for example, ITU-T Rec. H. In the case of MPEG2 defined by H.262 (2000E) (assuming that Vertical_size ≦ 1088), slice_start_code (slice start code + slice vertical position), q_scale_code (quantization scale code), and extra_bit_slice (extension bit of slice) And so on. In addition, ITU-T Rec. H. In H.262 (2000E), slice_start_code is specified as 32 bits, q_scale_code is specified as 5 bits, and extra_bit_slice is specified as 1 bit. Therefore, the total bit amount, GM_HD = 38 bits, is a fixed value of the header bit amount GB_MB. (Refer to page 32, chapter 6.2.4 of ITU-T Rec. H.262 (2000E)).
[0063]
GB_DC is defined in ITU-T Rec. H. In the case of MPEG2 defined by H.262 (2000E), when the chroma format is a: b: c, the picture is an I picture, so that picture_coding_type = I picture, and the following equation (6) is assumed on the assumption that no motion vector exists. Is defined as
[0064]
GB_DC = macroblock_address_increment + macroblock_mode
+ Dct_dc_size_luminance × a
+ Dct_dc_diff_luminance × a
+ Dct_dc_size_chrominance × (b + c)
+ Dct_dc_diff_chrominance × (b + c)
+ End_of_block × (a + b + c) (6)
[0065]
Here, macroblock_address_increment is the difference between the current MB address and the previous MB address, and macroblock_mode is the sum of macroblock_type and dct_type, and indicates the coding type of the macroblock and the DCT type, respectively.
[0066]
In addition, dct_dc_size_luminance is the DCT luminance DC coefficient difference size, dct_dc_diff_luminance is the DCT luminance DC coefficient difference value, dct_dc_size_chrominance is the DCT color difference DC coefficient difference size, dct_dc_diff_DCF difference value is the DCT difference_diff_DCDCiffiffDCNdiffDCNdDC_diffDCiffiffDCNdiffDCiffTDCiffonDCiff difference DCF In-block DCT coefficient end flags are shown.
[0067]
For example, ITU-T Rec. H. 262 (2000E) For MPEG2 defined by, macroblock_address_increment to 1 bit, Macroblock_mode to 2 bits, Dct_dc_size_luminance in 9 bits, Dct_dc_diff_luminance to 11 bits, dct_dc_size_ chrominance in 10 bits, Dct_dc_diff_chrominance to 11 bits, END_OF_BLOCK Are defined as 4 bits (ITU-T Rec. H.262 (2000E), pages 33, 34, 36, chapters 6.2.5, 6.2.5.1, 6.2.6). Section). For this reason, ITU-T Rec. H. In the case of MPEG2 defined by H.262 (2000E), when the chroma format is 4: 2: 0, the DC component GB_DC is calculated as in the following equation (7).
[0068]

[0069]
When the chroma format is 4: 2: 2, the DC component GB_DC is calculated as in the following equation (8).
[0070]

[0071]
Further, the sum GB_SMB of the bit amount of the first MB and the bit amount of the last MB in the slice is ITU-T Rec. H. In the case of MPEG2 defined by H.262 (2000E), it is defined as the following equation (9).
[0072]

[0073]
Here, macroblock_escape_F is the amount of bits for expanding the MB address of the first MB, macroblock_address_increment_F is the difference between the current MB address and the previous MB address of the first MB, q_scale_code is the q-scale code of the first MB, and macroblock_type1F_MB code is the macroblock_type1F_MB code. Frame_or_field_motion_type_F indicates a frame-structured (or field-structured) motion compensation type, and motion_vector_F indicates a bit amount of the first MB motion vector. _L is replaced with _F. Indicates the bit amount of each corresponding final MB.
[0074]
ITU-T Rec. H. In the case of MPEG2 defined by H.262 (2000E), macroblock_escape_F is 0 bit, macroblock_address_increment_F is 1 bit, q_scale_code is 5 bits (the first MB of slice requires q_scale_code, and _crob is _crob, and the _crob is _crob, and the _crob is _macro, and the t_macro is the _crob, and the t_macro is the _crob. From Tables B.2 to B.4 on page 122 of T Rec.H. 262 (2000E), frame_motion_type_F or field_motion_type_F is 2 bits, and motion_vector [motion_vector [motion_vector_motion_vector_motion_icon_motion_icon_motion_icon_motion_vector_motion_motion_vector_motion_motion []] is described. s] [0] Macroblock_escape_L is 11 bits, macroblock_address_increment_L is 8 bits, macroblock_type_L_ is 3 bits, macroblock_type_type_L is 3 bits, and frame_motion_type_L_bit is frame_motion_type_L_. The three bits (motion_vertical_field_select [0] [s], motion_code [r] [s] [0], and motion_code [r] [s] [1] are specified one bit each (ITU-T Rec) H.262 (2000E), pages 33 and 34 6.2.5. Chapters, 6.2.5.1 and 6.2.5.2).
[0075]
For this reason, ITU-T Rec. H. In the case of MPEG2 defined by H.262 (2000E), the sum GB_SMB of the bit amount of the first MB and the bit amount of the last MB in the slice is obtained as in the following equation (10).
[0076]

[0077]
From Expression (10), the sum GB_SMB of the bit amount of the first MB and the bit amount of the last MB in the slice is a fixed value of 44 bits.
[0078]
To summarize the above, for example, ITU-T Rec. H. In the case of MPEG2 defined by H.262 (2000E), when the chroma formats are 4: 2: 0 and 4: 2: 2, the margin of the I picture is calculated by the following equation based on the equation (1). (11) and Expression (12), respectively.
[0079]
Margin_Ipicture = 9336α + 38β + 149γ (11)
Margin_Ipicture = 12408α + 38β + 199γ (12)
[0080]
Similarly, when the chroma format is 4: 2: 0 and 4: 2: 2, the margin of a B picture or a P picture is calculated based on the following equation (2) based on equation (2). 13) and equation (14).
[0081]

As described above, when the chroma format is determined, the margin can be calculated based on the number of MBs α, the number of slices β, and the number of MBs γ that cannot be controlled by hardware. Here, since the number of MBs α is set as a value unique to hardware, the value is set in advance.
[0083]
The arithmetic unit 43 stores in advance a coefficient corresponding to a chroma format in a built-in memory for each I picture, B picture, or P picture, and stores the chroma format input from the picture type determination unit 11. Based on the information on the number of slices and the number of MBs transmitted from the picture type determination unit 11 at the same time, a margin is calculated and stored in the margin size memory 44. .
[0084]
Since the number of slices and the number of MBs do not change in one stream, the arithmetic unit 43 calculates the margin only when an I picture, a P picture, or a B picture is first input, and performs the margin calculation. The data is stored in the size memory 44. The VBV underflow determination unit 42 determines whether there is a VBV underflow using the margin calculated in the above-described manner and stored in the size memory 44.
[0085]
Next, VBV buffer underflow monitoring processing by the rate control unit 3 will be described with reference to the flowchart of FIG.
[0086]
In step S1, the VBV buffer underflow determination unit 42 of the rate control unit 3 acquires the picture type, the number of MBs, the number of slices, and the chroma format information input from the picture type determination unit 11.
[0087]
In step S2, the VBV buffer underflow determination unit 42 determines whether the margin size has already been calculated in the margin size memory 44. At this time, in step S1, the VBV buffer underflow determination unit 42 determines whether or not the margin size of the obtained picture type has already been calculated and stored.
[0088]
If it is determined in step S2 that the margin size has not been calculated, in step S3, the VBV buffer underflow determination unit 42 causes the calculation unit 43 to acquire the picture type, the number of MBs, the number of slices, and the The information is supplied, and the calculation of the above-described formula (1) or (2) is executed, and the calculation result is stored in the margin size memory 44.
[0089]
In step S4, the VBV buffer underflow determination unit 42 determines whether the input image is an I picture. For example, if it is determined that the input image is an I picture, the process proceeds to step S5.
[0090]
In step S5, the VBV buffer underflow determination unit 42 reads the margin size of the I picture stored in the margin size memory 44.
[0091]
In step S6, the VBV buffer underflow determination unit 42 determines whether or not the occupation amount of the VBV buffer is larger than the sum of the maximum generated bit amount MaxGenBit of the picture to be extracted and the margin. It is determined that the amount is not larger than the sum of the maximum generated bit amount MaxGenBit and the margin of the extracted picture, that is, the occupancy of the VBV buffer is smaller than the sum of the maximum generated bit amount MaxGenBit and the margin of the extracted picture. If so, the process proceeds to step S7.
[0092]
In step S7, the VBV buffer underflow determination unit 42 considers that a VBV buffer underflow has occurred, and instructs the weighting quantization unit 15 to execute the minimum parameter processing.
[0093]
In step S8, the VBV buffer underflow determination unit 42 determines whether or not the next picture exists. If the next picture exists, the process returns to step S1, and if the next picture does not exist. In step S9, the margin size memory 44 is reset, and the process ends.
[0094]
In step S2, when the margin size is calculated, that is, when an I picture is input, the margin size of the I picture is input, and when the margin size of a P picture or B picture is input, P When the margin size of the picture or the B picture has already been calculated, the processing of step S3 is skipped.
[0095]
Further, in step S4, if the picture is not an I picture, that is, if it is a B picture or a P picture, in step S10, the VBV buffer underflow determination unit 42 outputs the B picture stored in the margin size memory 44, Alternatively, the margin size of the P picture is read.
[0096]
If it is determined in step S6 that the occupancy of the VBV buffer is greater than the sum of the maximum generated bit amount MaxGenBit of the picture to be extracted and the margin, the process of step S7 is skipped, and the process proceeds to step S8. Proceed to.
[0097]
That is, when the above processes are summarized, in step S3, the calculation unit 43 calculates the margin size of the I picture, the P picture, or the B picture. At this time, since the arithmetic unit 43 previously stores the coefficients GB_MB, GB_HD, and GB_DC corresponding to the chroma format, as described above, the arithmetic unit 43 responds to the chroma format and the picture type information obtained in the process of step S1. The formula (1) or the formula (1) is selected by using the selected number of MBs α to be generated and the number of slices β and the number of MBs γ acquired in the process of step S1. 2) is calculated to calculate the margin size.
[0098]
However, the margin size of the picture type stored in the margin size memory 44 by the processing of step S2 is not calculated in the subsequent processing because step S3 is skipped, so that processing of one stream is performed. Is calculated only once for each picture type.
[0099]
In steps S5 and S10, a margin size is read for each picture type. Then, in step S6, the sum of the maximum generated bit amount MexGenBit and the read margin size is compared with the occupation amount of the VBV buffer, and the occupation amount of the VBV buffer is compared with the maximum generated bit amount MaxGenBit of the picture to be extracted and the margin. If less than the sum, a command is issued to perform the minimum parameter processing.
[0100]
As a result, the margin which has been set empirically until now is calculated and set as the maximum bit amount generated for each stream. The VBV buffer can be used to the maximum, and the occurrence of underflow of the VBV buffer can be suppressed almost 100%.
[0101]
The series of processes described above can be executed by hardware, but can also be executed by software. When a series of processing is executed by software, a program constituting the software may be executed by a computer built into dedicated hardware or by installing various programs to execute various functions. It is installed from a recording medium in a possible personal computer or the like, for example.
[0102]
FIG. 9 shows a configuration of an embodiment of a personal computer when the encoder is realized by software. The CPU 101 of the personal computer controls the entire operation of the personal computer. When a user inputs a command from an input unit 106 including a keyboard, a mouse, and the like via a bus 104 and an input / output interface 105, the CPU 101 stores the command in a ROM (Read Only Memory) 102 in response to the command. Execute the program. Alternatively, the CPU 101 reads a program read from the magnetic disk 111, the optical disk 112, the magneto-optical disk 113, or the semiconductor memory 114 connected to the drive 110 and installed in the storage unit 108, and stores the program in a RAM (Random Access Memory) 103. And run it. Thereby, the function of the encoder described above is realized by software. Further, the CPU 101 controls the communication unit 109 to communicate with the outside and exchange data.
[0103]
As shown in FIG. 9, the recording medium on which the program is recorded is a magnetic disk 111 (including a flexible disk) on which the program is recorded, which is distributed separately from the computer to provide the program to the user. An optical disk 112 (including a compact disk-only disk (CD-ROM), a digital versatile disk (DVD)), a magneto-optical disk 113 (including a mini-disk (MD)), or a package medium including a semiconductor memory 114 In addition to the configuration, the configuration includes a ROM 102 storing a program and a hard disk included in the storage unit 108, which are provided to a user in a state of being incorporated in a computer in advance.
[0104]
In this specification, a step of describing a program recorded on a recording medium is performed in a time-series manner in the order described. Alternatively, the processing includes individually executed processing.
[0105]
【The invention's effect】
According to the present invention, the VBV buffer can be used to the maximum extent, and the VBV buffer underflow can be suppressed by almost 100%.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating a VBV buffer.
FIG. 2 is a diagram illustrating a VBV buffer underflow.
FIG. 3 is a diagram illustrating a margin of a VBV buffer.
FIG. 4 is a diagram illustrating a margin of a VBV buffer.
FIG. 5 is a block diagram showing a configuration of an encoder to which the present invention is applied.
FIG. 6 is a diagram for explaining processing by the encoder of FIG. 5;
FIG. 7 is a block diagram illustrating a configuration of a rate control unit in FIG. 5;
FIG. 8 is a flowchart illustrating a VBV buffer underflow monitoring process by the rate control unit in FIG. 7;
FIG. 9 is a diagram illustrating a medium.
[Explanation of symbols]
Reference Signs List 1 basic encoding processing unit, 2 encoder transmission buffer, 3 rate control unit, 11 picture type determination unit, 12 scan conversion unit, 14 DCT conversion unit, 15 weighted quantization unit, 16 variable length coding unit, 20 mode determination unit , 41 control unit, 42 VBV buffer underflow determination unit, 43 calculation unit, 44 margin size memory

Claims (7)

In an image processing apparatus for compressing and transmitting an image by the MPEG method,
Determining means for determining a picture type of the input image;
Calculating means for calculating a margin of a VBV buffer for each picture type based on the image;
Storage means for storing a margin of the VBV buffer for each picture type calculated by the calculation means;
Compression means for compressing the image in an MPEG format;
Underflow detection means for detecting the presence or absence of a VBV buffer underflow based on the image information compressed by the compression means, the picture type determined by the determination means, and a margin of the VBV buffer;
An image processing apparatus, comprising: a control unit that controls the compression unit so as to compress the image with a minimum number of parameters when the underflow detection unit detects the presence of the VBV buffer underflow. 2. The method according to claim 1, wherein the calculating unit calculates, based on the image, a margin of each VBV buffer when the picture type is an I picture and a P picture or a B picture. 3. The image processing apparatus according to claim 1. 2. The image according to claim 1, wherein the margin of the VBV buffer is a minimum bit amount generated when the control unit controls the compression unit to compress the image with a minimum number of parameters. Processing equipment. When the picture type is an I picture, the control unit sets all macroblocks to DC components, and when the picture type is a P picture or a B picture, sets all macroblocks to skipped macroblocks. 2. The image processing apparatus according to claim 1, wherein the compression unit is controlled to compress the image with a minimum number of parameters. In an image processing method of an image processing apparatus for compressing and transmitting an image by an MPEG method,
A determining step of determining a picture type of the input image;
A calculating step of calculating a margin of a VBV buffer for each of the picture types based on the image;
A storage step of storing a margin of the VBV buffer for each picture type calculated in the processing of the calculation step;
A compression step of compressing the image in an MPEG format;
An underflow detection step of detecting the presence or absence of a VBV buffer underflow based on the image information compressed in the processing of the compression step, the picture type determined in the processing of the determination step, and a margin of the VBV buffer;
A control step of controlling the processing of the compression step so as to compress the image with a minimum number of parameters when the VBV buffer underflow is detected in the processing of the underflow detection step. Processing method. In a program that causes a computer to perform image processing for compressing and transmitting an image by the MPEG method,
A determining step of determining a picture type of the input image;
A calculating step of calculating a margin of a VBV buffer for each of the picture types based on the image;
A compression step of compressing the image in an MPEG format;
An underflow detection step of detecting the presence or absence of a VBV buffer underflow based on the image information compressed in the processing of the compression step, the picture type determined in the processing of the determination step, and a margin of the VBV buffer;
And a control step of controlling the processing of the compression step so as to compress the image with a minimum number of parameters when the VBV buffer underflow is detected in the processing of the underflow detection step. A recording medium on which a readable program is recorded. In a program that causes a computer to perform image processing for compressing and transmitting an image by the MPEG method,
A determining step of determining a picture type of the input image;
A calculating step of calculating a margin of a VBV buffer for each of the picture types based on the image;
A compression step of compressing the image in an MPEG format;
An underflow detection step of detecting the presence or absence of a VBV buffer underflow based on the image information compressed in the processing of the compression step, the picture type determined in the processing of the determination step, and a margin of the VBV buffer;
If the underflow detection step detects that the VBV buffer underflow is present, the computer causes the computer to execute processing including a control step of controlling the processing of the compression step so as to compress the image with a minimum number of parameters. A program characterized by the following.

Images