JP2004072144A - Encoder and encoding method, program, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To limit the minimum value of a virtual buffer in response to a bit rate when an image pattern change is close to the value of ME residual that presence of a scene change is discriminated. <P>SOLUTION: A flowchart explaining a minimum value limit process 3 includes: a step S62 of discriminating whether or not a bit rate value is smaller than a threshold value G when a relation of ME_info-avg>E holds in step S61; a step S63 of limiting a minimum value of the virtual buffer to a first value when the bit rate is smaller than the threshold value G; a step S64 of discriminating whether or not the bit rate value is smaller than a threshold value H when the bit rate value is greater than the threshold value G; a step S65 of limiting the minimum value of the virtual buffer to a second value smaller than the case with Step S63 when the bit rate value is smaller than the threshold value H; and no minimum value is limited because a difference between a present picture and a preceding picture is a degree of not needing consideration of the minimum value of a quantization index in the case that the relation of ME_info-avg>E does not hold and because there is a sufficient margin for the rate control when the bit rate value is greater than the threshold value H. <P>COPYRIGHT: (C)2004,JPO

Description

・・・（１０）
【００８４】
また、式（１０）において、式（１１）が成り立つ。
【数２】

・・・（１１）
【００８５】
画像の符号化難易度が易しいものから難しいものへのシーンチェンジ、および、難しいものから易しいものへのシーンチェンジの、双方に対して仮想バッファ調整を行ってしまった場合、難しいものから易しいものへのシーンチェンジでは、エンコードに余裕があるはずの易画像においてわざわざ画質を悪くしてしまう結果となる場合がある。また、難しいものから易しいものへのシーンチェンジであっても、その変化の大きさ、あるいは、シーンチェンジ後の画像の難易度によっては、仮想バッファの調整を行うほうがよい場合がある。しかしながら、ＭＥ残差情報のみでは、シーンチェンジの有無を判定することはできるが、シーンチェンジの内容が、易しいものから難しいものへのシーンチェンジであるか、あるいは、難しいものから易しいものへのシーンチェンジであるかを判定することができない。
【００８６】
そこで、イントラＡＣ算出部９１が、イントラＡＣを算出し、現在のイントラＡＣの値を示すｍａｄ＿ｉｎｆｏと、一つ前のイントラＡＣの値を示すｐｒｅｖ＿ｍａｄ＿ｉｎｆｏとを、発生符号量制御部９２に出力することにより、発生符号量制御部９２は、シーンチェンジの状態を判定して、仮想バッファ調整を行うか否かを判断することができる。
【００８７】
発生符号量制御部９２は、バッファ８２に格納される可変長符号化データの蓄積状態を常時監視しており、蓄積状態を表す占有量情報を基に量子化ステップサイズを決定するようになされている。
【００８８】
また、発生符号量制御部９２は、イントラスライス部分の目標発生符号量データＴｐｉよりも実際に発生したマクロブロックの発生符号量データＢ（ｊ）が多い場合、発生符号量を減らすために量子化ステップサイズを大きくし、また、目標発生符号量データＴｐｉよりも実際の発生符号量データＢ（ｊ）が少ない場合、発生符号量を増やすために量子化ステップサイズを小さくするようになされている。
【００８９】
更に、発生符号量制御部９２は、インタースライス部分の場合も同様に、目標発生符号量データＴｐｐよりも実際に発生したマクロブロックの発生符号量データＢ（ｊ）が多い場合、発生符号量を減らすために量子化ステップサイズを大きくし、また、目標発生符号量データＴｐｐよりも実際の発生符号量データＢ（ｊ）が少ない場合、発生符号量を増やすために量子化ステップサイズを小さくするようになされている。
【００９０】
すなわち、発生符号量制御部９２は、デコーダ側に設けられたＶＢＶバッファに格納された可変長符号化データの蓄積状態の推移を想定することにより、図９に示されるように、ｊ番目のマクロブロックにおける仮想バッファのバッファ占有量ｄ（ｊ）を次の式（１２）によって表し、また、ｊ＋１番目のマクロブロックにおける仮想バッファのバッファ占有量ｄ（ｊ＋１）を次の式（１３）によって表し、（１２）式から（１３）式を減算することにより、ｊ＋１番目のマクロブロックにおける仮想バッファのバッファ占有量ｄ（ｊ＋１）を次の式（１４）として変形することができる。
【００９１】
ｄ（ｊ）＝ｄ（０）＋Ｂ（ｊ−１）−｛Ｔ×（ｊ−１）／ＭＢｃｎｔ｝・・・（１２）
【００９２】
ここで、ｄ（０）は初期バッファ容量、Ｂ（ｊ）は、ｊ番目のマクロブロックにおける符号化発生ビット数、ＭＢｃｎｔは、ピクチャ内のマクロブロック数、そして、Ｔは、ピクチャ単位の目標発生符号量である。
【００９３】
ｄ（ｊ＋１）＝ｄ（０）＋Ｂ（ｉ）−（Ｔ×ｊ）／ＭＢｃｎｔ・・・（１３）
【００９４】
ｄ（ｊ＋１）＝ｄ（ｊ）＋｛Ｂ（ｊ）−Ｂ（ｊ−１）｝−Ｔ／ＭＢｃｎｔ・・・（１４）
【００９５】
続いて、発生符号量制御部９２は、ピクチャ内のマクロブロックがイントラスライス部分とインタースライス部分とに分かれているため、図１０に示されるように、イントラスライス部分のマクロブロックとインタースライス部分の各マクロブロックに割り当てる目標発生符号量ＴｐｉおよびＴｐｐをそれぞれ個別に設定する。
【００９６】
グラフにおいて、マクロブロックのカウント数が０乃至ｓ、および、ｔ乃至ｅｎｄの間にあるとき、次の式（１５）に、インタースライスの目標発生符号量Ｔｐｐを代入することにより、インタースライス部分におけるバッファ占有量ｄ（ｊ＋１）を得ることができる。
【００９７】
ｄ（ｊ＋１）＝ｄ（ｊ）＋｛Ｂ（ｊ）−Ｂ（ｊ−１）｝−Ｔｐｐ／｛ＭＢｃｎｔ−（ｔ−ｓ）｝・・・（１５）
【００９８】
また、マクロブロックのカウント数がｓ乃至ｔの間にあるときに、次の式（１６）に、イントラスライスの目標発生符号量Ｔｐｉを代入することにより、イントラスライス部分におけるバッファ占有量ｄ（ｊ＋１）を得ることができる。
【００９９】
ｄ（ｊ＋１）＝ｄ（ｊ）＋｛Ｂ（ｊ）−Ｂ（ｊ−１）｝−Ｔｐｉ／（ｔ−ｓ）・・・（１６）
【０１００】
従って、発生符号量制御部９２は、イントラスライス部分およびインタースライス部分におけるバッファ占有量ｄ（ｊ＋１）、および、式（１７）に示される定数ｒを、式（１８）に代入することにより、マクロブロック（ｊ＋１）の量子化インデックスデータＱ（ｊ＋１）を算出し、これを量子化部７５に供給する。
【０１０１】
ｒ＝（２×ｂｒ）／ｐｒ　・・・（１７）
Ｑ（ｊ＋１）＝ｄ（ｊ＋１）×（３１／ｒ）　・・・（１８）
ここで、ｂｒは、ビットレートであり、ｐｒは、ピクチャレートである。
【０１０２】
量子化部７５は、量子化インデックスデータＱ（ｊ＋１）に基づいて、次のマクロブロックにおけるイントラスライスまたはインタースライスに応じた量子化ステップサイズを決定し、量子化ステップサイズによってＤＣＴ係数データを量子化する。
【０１０３】
これにより、量子化部７５は、１つ前のピクチャのイントラスライス部分およびインタースライス部分における実際の発生符号量データＢ（ｊ）に基づいて算出された、次のピクチャのイントラスライス部分およびインタースライス部分における目標発生符号量ＴｐｐおよびＴｐｉにとって最適な量子化ステップサイズによって、ＤＣＴ係数データを量子化することができる。
【０１０４】
かくして、量子化部７５では、バッファ８２のデータ占有量に応じて、バッファ８２がオーバーフローまたはアンダーフローしないように量子化し得るとともに、デコーダ側のＶＢＶバッファがオーバーフロー、またはアンダーフローしないように量子化した量子化ＤＣＴ係数データを生成することができる。
【０１０５】
例えば、従来の技術として上述した、特願平１１−２０５８０３では、通常のフィードバック型の量子化制御を行いながら、次に符号化処理するピクチャの絵柄が大きく変化する場合には、フィードバック型の量子化制御を止め、ＭＥ残差算出部９５から供給されるＭＥ残差情報に基づいて、仮想バッファの初期バッファ容量ｄ（０）を初期化し、新たな初期バッファ容量ｄ（０）を基に、イントラスライスおよびインタースライスごとに量子化インデックスデータＱ（ｊ＋１）を新たに算出するようになされている。
【０１０６】
しかしながら、従来における場合のように、ＭＥ残差のみで仮想バッファ調整を行うか否かを判定してしまうと、画像難易度が易しいものから難しいものに変わった場合、および難しいものから簡単なものに変わった場合の双方に対して、仮想バッファ調整を行ってしまう。すなわち、画像難易度が難しいものから簡単なものに変わった場合では、エンコードに余裕があるはずの簡単な画像において、わざわざ画質を悪くしてしまう結果となる。
【０１０７】
そこで、図８のビデオエンコーダ６１においては、例えば、イントラＡＣ算出部９１によって算出されるイントラＡＣなどの情報を用いて、画像難易度が易しいものから難しいものに変わるシーンチェンジの時にのみ、仮想バッファ調整を行うようにすることにより、簡単な画像での画質の劣化を防ぐようにすることができる。
【０１０８】
すなわち、発生符号量制御部９２は、通常のフィードバック型の量子化制御を行いながら、次に符号化処理するピクチャの絵柄が大きく変化する場合には、フィードバック型の量子化制御を止め、ＭＥ残差算出部９５から供給されるＭＥ残差情報ＭＥ＿ｉｎｆｏ、並びに、イントラＡＣ算出部９１から供給される、ｐｒｅｖ＿ｍａｄ＿ｉｎｆｏおよびｍａｄ＿ｉｎｆｏを基に、仮想バッファの初期バッファ容量ｄ（０）を初期化するか否かを判断し、仮想バッファの初期バッファ容量ｄ（０）を初期化する場合は、ＭＥ残差算出部９５から供給されるＭＥ残差情報ＭＥ＿ｉｎｆｏに基づいて、仮想バッファの初期バッファ容量ｄ（０）を初期化する。仮想バッファの初期バッファ容量ｄ（０）の初期化については、式（２）乃至式（５）を用いて説明した従来における場合と同様である。
【０１０９】
そして、発生符号量制御部９２は、新たな初期バッファ容量ｄ（０）を基に、イントラスライスおよびインタースライスごとに、式（１２）乃至式（１８）を用いて、量子化インデックスデータＱ（ｊ＋１）を新たに算出し、量子化部７５に供給する。
【０１１０】
更に、発生符号量制御部９２は、次に符号化処理するピクチャの絵柄の変化が、シーンチェンジと判断されるほど大きくなく、しかしながら、シーンチェンジと判断されるのに近い場合、必要に応じて、仮想バッファの最小値をある一定の値より大きくなるように制限する。これにより、量子化インデックスの最小値も制限されるので、図７を用いて説明したような状況において、画像スキップの発生を抑制することができる。
【０１１１】
図１１のフローチャートを参照して、イントラＡＣなどの画像難易度情報を用いて、シーンチェンジは、簡単な画像から難しい画像への変化であるか否かの判定を導入して仮想バッファの調整を行う、仮想バッファ更新処理について説明する。
【０１１２】
ステップＳ２１において、発生符号量制御部９２は、ＭＥ残差算出部９５から、ＭＥ残差情報ＭＥ＿ｉｎｆｏ　を取得する。
【０１１３】
ステップＳ２２において、発生符号量制御部９２は、取得されたＭＥ残差情報から、ＭＥ残差情報の平均値ａｖｇを減算し、ＭＥ＿ｉｎｆｏ−ａｖｇ　＞　Ｄであるか否か、すなわち、算出された値が、所定の閾値Ｄよりも大きいか否かが判断される。ＭＥ残差情報の平均値ａｖｇは、後述するステップＳ２６において更新される値であり、上述した式（１）で示される。なお、所定の閾値Ｄは、画質を検討しながらチューニングされる性質の値である。
【０１１４】
ステップＳ２２において、算出された値は、所定の閾値Ｄより大きいと判断された場合、現在のピクチャにおける絵柄と、１つ前のピクチャにおける絵柄との差が大きい、すなわち、シーンチェンジがあったと判断されるので、ステップＳ２３において、発生符号量制御部９２は、イントラＡＣ算出部９１から取得される、このシーンチェンジの後のイントラＡＣの値であるｍａｄ＿ｉｎｆｏと、このシーンチェンジの前のイントラＡＣの値であるｐｒｅｖ＿ｍａｄ＿ｉｎｆｏとを比較し、ｍａｄ＿ｉｎｆｏ　＞　ｐｒｅｖ＿ｍａｄ＿ｉｎｆｏであるか否かを判断する。
【０１１５】
ステップＳ２３において、ｍａｄ＿ｉｎｆｏ　＞　ｐｒｅｖ＿ｍａｄ＿ｉｎｆｏではないと判断された場合、このシーンチェンジは、難しい画像から、簡単な画像へのシーンチェンジであるので、処理は、ステップＳ２６に進む。
【０１１６】
ステップＳ２３において、ｍａｄ＿ｉｎｆｏ　＞　ｐｒｅｖ＿ｍａｄ＿ｉｎｆｏであると判断された場合、このシーンチェンジは、簡単な画像から、難しい画像へのシーンチェンジであるので、ステップＳ２４において、発生符号量制御部９２は、図１を用いて説明した従来における場合と同様の処理により、仮想バッファの初期バッファ容量ｄ（０）の更新を行う。
【０１１７】
すなわち、発生符号量制御部９２は、上述した式（２）、式（３）、式（４）および式（５）に基づいて、仮想バッファの初期バッファ容量ｄ（０）を算出し、仮想バッファを更新する。
【０１１８】
ステップＳ２２において、算出された値は、所定の閾値Ｄより小さいと判断された場合、現在のピクチャにおける絵柄と、１つ前のピクチャにおける絵柄との差があまり無い、すなわちシーンチェンジがなかったと判断されるので、ステップＳ２５において、図１２を用いて後述する最小値制限処理１、図１３を用いて後述する最小値制限処理２、または、図１４を用いて後述する最小値制限処理３のうちのいずれかの処理が実行される。
【０１１９】
ステップＳ２３において、ｍａｄ＿ｉｎｆｏ　＞　ｐｒｅｖ＿ｍａｄ＿ｉｎｆｏではないと判断された場合、ステップＳ２４の処理の終了後、もしくは、ステップＳ２５の処理の終了後、ステップＳ２６において、発生符号量制御部９２は、次に供給されるピクチャに備えて、ＭＥ残差情報の平均値ａｖｇを、上述した式（１）により更新し、処理は、ステップＳ２１に戻り、それ以降の処理が繰り返される。
【０１２０】
図１１のフローチャートを用いて説明した処理により、イントラＡＣを用いて、画像難易度が易しいものから難しいものに変更されるシーンチェンジの時にのみ仮想バッファ調整を行うようにしたので、エンコードに余裕があるはずの簡単な画像において、更に画質を悪くしてしまうことを防ぐことができる。
【０１２１】
更に、シーンチェンジが検出されなかった場合は、図１２を用いて後述する最小値制限処理１、図１３を用いて後述する最小値制限処理２、または、図１４を用いて後述する最小値制限処理３のうちのいずれかの処理が実行される。
【０１２２】
次に、図１２のフローチャートを参照して、最小値制限処理１について説明する。
【０１２３】
ステップＳ４１において、発生符号量制御部９２は、図１１のステップＳ２１において取得したＭＥ残差情報ＭＥ＿ｉｎｆｏを基に、ステップＳ２２において算出したＭＥ＿ｉｎｆｏ−ａｖｇの値を所定の閾値Ｅと比較して、ＭＥ＿ｉｎｆｏ−ａｖｇ＞Ｅであるか否かを判断する。ここで、閾値Ｅは、図１１のステップＳ２２の閾値Ｄよりも小さな値である。
【０１２４】
ステップＳ４１において、ＭＥ＿ｉｎｆｏ−ａｖｇ＞Ｅであると判断された場合、ステップＳ４２において、発生符号量制御部９２は、仮想バッファの最小値を制限させるための制御信号を生成し、量子化部７５に供給する。量子化部７５は、発生符号量制御部９２から供給された制御信号に基づいて、前の画像が簡単なものであっても、仮想バッファのバッファ容量が、少なくとも予め定められた所定の値より大きくなるように制限して、式（１２）乃至式（１８）を用いて説明した処理により量子化インデックスを算出する。算出された量子化インデックスも、最小値が制限されたものとなる。ステップＳ４２の処理の終了後、処理は、図１１のステップＳ２６の処理に戻る。
【０１２５】
すなわち、図１１のステップＳ２２において、ＭＥ＿ｉｎｆｏ−ａｖｇ＞Ｄではないと判断され、ステップＳ４１において、ＭＥ＿ｉｎｆｏ−ａｖｇ＞Ｅであると判断された場合、図１３に示されるように、ＭＥ＿ｉｎｆｏ−ａｖｇの値が閾値Ｄより大きい場合のシーンチェンジ領域９６と、ＭＥ＿ｉｎｆｏ−ａｖｇの値が閾値Ｅより小さい場合の通常領域９８に挟まれた中間領域α９７に、ＭＥ＿ｉｎｆｏ−ａｖｇが存在する（Ｄ＞ＭＥ＿ｉｎｆｏ−ａｖｇ＞Ｅが成立する）。
【０１２６】
中間領域α９７においては、シーンチェンジとは判断されないが、前の画像を基にフィードバックされる量子化インデックスの値が非常に小さなものであった場合、図７を用いて説明したように、画面の途中までに、与えられたビット量を消費してしまい、符号量が足らなくなり、画像スキップが発生してしまう恐れがある。従って、ステップＳ４２において、仮想バッファの値の最小値を制限することにより、量子化インデックスの最小値も制限されるので、画像スキップの発生を抑制することができる。
【０１２７】
ステップＳ４１において、ＭＥ＿ｉｎｆｏ−ａｖｇ＞Ｅではないと判断された場合、前の画像との差は、量子化インデックスの最小値を考慮する必要がない程度であると判断されるので、最小値制限は行われず、処理は、図１１のステップＳ２６の処理に戻る。
【０１２８】
図１２を用いて説明した処理により、シーンチェンジとは判断されないが、従来の方法では画像スキップ（画面下端に前のピクチャが残ってしまうという現象）が発生してしまう恐れがある画像を、画面下端までエンコードすることが可能となる。
【０１２９】
ところで、ビットレートが高い画像をエンコードする場合のレート制御は、ビットレートが低い画像をエンコードする場合と比較して、余裕がある。レート制御に十分な余裕があるにもかかわらず、仮想バッファ（量子化インデックス）の最小値を制限してしまうと、画質が劣化してしまう。
【０１３０】
図１４のフローチャートを参照して、ＭＥ残差に加えて、ビットレートの値を参照して、最小値制限を行うか否かを判断する最小値制限処理２について説明する。
【０１３１】
ステップＳ５１において、発生符号量制御部９２は、図１１のステップＳ２１において取得したＭＥ残差情報ＭＥ＿ｉｎｆｏを基に、ステップＳ２２において算出したＭＥ＿ｉｎｆｏ−ａｖｇの値を、所定の閾値Ｅと比較して、ＭＥ＿ｉｎｆｏ−ａｖｇ＞Ｅであるか否かを判断する。ここで、閾値Ｅは、図１１のステップＳ２２の閾値Ｄよりも小さな値である。
【０１３２】
ステップＳ５１において、ＭＥ＿ｉｎｆｏ−ａｖｇ＞Ｅであると判断された場合、ステップＳ５２において、発生符号量制御部９２は、目標符号量算出部９４から供給されたビットレートの値は、所定の閾値Ｆより小さいか否かを判断する。ここで、閾値Ｆは、符号化する場合にレート制御に余裕が発生するか否かを判断するための値である。
【０１３３】
ステップＳ５２において、ビットレートの値は、所定の閾値Ｆより小さいと判断された場合、ステップＳ５３において、発生符号量制御部９２は、仮想バッファの最小値を制限させるための制御信号を生成し、量子化部７５に供給する。量子化部７５は、発生符号量制御部９２から供給された制御信号に基づいて、前の画像が簡単なものであっても、仮想バッファのバッファ容量が、少なくとも予め定められた所定の値より大きくなるように制限して、式（１２）乃至式（１８）を用いて説明した処理により量子化インデックスを算出する。算出された量子化インデックスも、最小値が制限されたものとなる。ステップＳ５３の処理の終了後、処理は、図１１のステップＳ２６の処理に戻る。
【０１３４】
ステップＳ５１において、ＭＥ＿ｉｎｆｏ−ａｖｇ＞Ｅではないと判断された場合、前の画像との差は、量子化インデックスの最小値を考慮する必要がない程度であると判断され、ステップＳ５２において、ビットレートの値は、所定の閾値Ｆより大きいと判断された場合、レート制御には余裕があると判断されるので、最小値制限は行われず、処理は、図１１のステップＳ２６の処理に戻る。
【０１３５】
図１４のフローチャートを用いて説明した処理により、レート制御に十分な余裕があるにもかかわらず、仮想バッファ（量子化インデックス）の最小値を制限して、画質が劣化してしまうことを防いで、必要な場合にのみ、仮想バッファ（量子化インデックス）の最小値を制限し、画像スキップ（画面下端に前のピクチャが残ってしまうという現象）の発生を抑制することができる。
【０１３６】
更に、ビットレートの値に応じて、仮想バッファ（量子化インデックス）の最小値を複数設定可能なようにすることにより、更に、画質の安定を図るようにすることができる。
【０１３７】
次に、図１５のフローチャートを参照して、ビットレートの値に応じて、仮想バッファ（量子化インデックス）の複数の最小値を設定することができる最小値制限処理３について説明する。
【０１３８】
ステップＳ６１において、発生符号量制御部９２は、図１１のステップＳ２１において取得したＭＥ残差情報ＭＥ＿ｉｎｆｏを基に、ステップＳ２２において算出したＭＥ＿ｉｎｆｏ−ａｖｇの値を、所定の閾値Ｅと比較して、ＭＥ＿ｉｎｆｏ−ａｖｇ＞Ｅであるか否かを判断する。ここで、閾値Ｅは、図１１のステップＳ２２の閾値Ｄよりも小さな値である。
【０１３９】
ステップＳ６１において、ＭＥ＿ｉｎｆｏ−ａｖｇ＞Ｅであると判断された場合、ステップＳ６２において、発生符号量制御部９２は、目標符号量算出部９４から供給されたビットレートの値は、所定の閾値Ｇより小さいか否かを判断する。ここで、閾値Ｇは、符号化する場合にレート制御に余裕が発生するか否かを判断するための値である。また、ビットレートが小さい場合の符号化において、最小値の制限を厳密に行うことにより、より効果的に、画質の劣化を防いで、画像スキップの発生を抑制することができるので、閾値Ｇは、図１４を用いて説明した処理における閾値Ｆより小さな値とすると好適である。
【０１４０】
ステップＳ６２において、ビットレートの値は、所定の閾値Ｇより小さいと判断された場合、ステップＳ６３において、発生符号量制御部９２は、仮想バッファの最小値を、予め定められた第１の値に制限させるための制御信号を生成し、量子化部７５に供給する。量子化部７５は、発生符号量制御部９２から供給された制御信号に基づいて、前の画像が簡単なものであっても、仮想バッファのバッファ容量が少なくとも第１の値より大きくなるように制限して、式（１２）乃至式（１８）を用いて説明した処理により量子化インデックスを算出する。算出された量子化インデックスも、第１の値により最小値が制限されたものとなる。ステップＳ６３の処理の終了後、処理は、図１１のステップＳ２６の処理に戻る。
【０１４１】
ステップＳ６２において、ビットレートの値は、所定の閾値Ｇより大きいと判断された場合、ステップＳ６４において、発生符号量制御部９２は、目標符号量算出部９４から供給されたビットレートの値は、所定の閾値Ｈより小さいか否かを判断する。ここで、レート制御に余裕が発生するか否かを判断するための閾値Ｈは、閾値Ｇよりも大きい値であり、例えば、図１４を用いて説明した処理における閾値Ｆと同等の値とすると好適である。
【０１４２】
ステップＳ６４において、ビットレートの値は、所定の閾値Ｈより小さいと判断された場合、ステップＳ６５において、発生符号量制御部９２は、仮想バッファの最小値を、ステップＳ６３における第１の値よりも小さい、予め定められた第２の値に制限させるための制御信号を生成し、量子化部７５に供給する。量子化部７５は、発生符号量制御部９２から供給された制御信号に基づいて、前の画像が簡単なものであっても、仮想バッファのバッファ容量が少なくとも第２の値より大きくなるように制限して、式（１２）乃至式（１８）を用いて説明した処理により量子化インデックスを算出する。算出された量子化インデックスも、第２の値により最小値が制限されたものとなる。ステップＳ６５の処理の終了後、処理は、図１１のステップＳ２６の処理に戻る。
【０１４３】
ステップＳ６１において、ＭＥ＿ｉｎｆｏ−ａｖｇ＞Ｅではないと判断された場合、前の画像との差は、量子化インデックスの最小値を考慮する必要がない程度であると判断されるので、最小値制限は行われず、処理は、図１１のステップＳ２６の処理に戻る。また、ステップＳ６４において、ビットレートの値は、所定の閾値Ｈより大きいと判断された場合、レート制御には充分な余裕があると判断されるので、最小値制限は行われず、処理は、図１１のステップＳ２６の処理に戻る。
【０１４４】
以上説明した処理により、画像の変更量が、シーンチェンジ検出の閾値付近である場合においても、符号化処理が困難な画像を検出して、仮想バッファ（量子化インデックス）の最小値を制限したり、更に、レート制御が困難な画像を検出して、そのビットレートに対応した仮想バッファ（量子化インデックス）の最小値を設定するようにすることが可能になる。
【０１４５】
このことにより、ローディレイモード、低ビットレートであっても、画像スキップ（画面下端に前のピクチャが残ってしまうという現象）の発生を抑制して、安定した画質を得ることができる。
【０１４６】
また、上述の実施の形態においては、ローディレイコーディングとしてナンバ０乃至１１の各フレーム画像を全てＰピクチャとし、例えば、横４５マクロブロック、縦２４マクロブロックの画枠サイズの中でフレーム画像の上段から縦２マクロブロックおよび横４５マクロブロック分の領域を１つのイントラスライス部分、他を全てインタースライス部分として設定するようにした場合について述べたが、本発明はこれに限らず、例えば、イントラスライス部分を縦１マクロブロック、横４５マクロブロック分の領域とするなど、他の種々の大きさの領域で形成するようにしても良い。
【０１４７】
また、ここでは、ローディレイエンコードを行う場合を例として説明したが、本発明は、例えば、１５フレームを、フレーム内符号化画像（以下、Ｉピクチャと称する）、フレーム間順方向予測符号化画像（以下、Ｐピクチャと称する）、もしくは、双方向予測符号化画像（以下、Ｂピクチャと称する）の３つの画像タイプのうちのいずれの画像タイプとして処理するかを指定し、指定されたフレーム画像の画像タイプ（Ｉピクチャ、Ｐピクチャ、あるいは、Ｂピクチャ）に応じて、フレーム画像を符号化するような場合にも適用可能である。
【０１４８】
更に、上述の実施の形態においては、本発明をＭＰＥＧ方式によって圧縮符号化する符号化装置としてのビデオエンコーダ６１に適用するようにした場合について述べたが、本発明はこれに限らず、他の種々の画像圧縮方式による符号化装置に適用するようにしても良い。
【０１４９】
上述した一連の処理は、ハードウエアにより実行させることもできるが、ソフトウエアにより実行させることもできる。この場合、例えば、ビデオエンコーダ６１は、図１６に示されるようなパーソナルコンピュータ１０１により構成される。
【０１５０】
図１６において、ＣＰＵ１１１は、ＲＯＭ１１２に記憶されているプログラム、または記憶部１１８からＲＡＭ１１３にロードされたプログラムに従って、各種の処理を実行する。ＲＡＭ１１３にはまた、ＣＰＵ１１１が各種の処理を実行する上において必要なデータなども適宜記憶される。
【０１５１】
ＣＰＵ１１１、ＲＯＭ１１２、およびＲＡＭ１１３は、バス１１４を介して相互に接続されている。このバス１１４にはまた、入出力インタフェース１１５も接続されている。
【０１５２】
入出力インタフェース１１５には、キーボード、マウスなどよりなる入力部１１６、ディスプレイやスピーカなどよりなる出力部１１７、ハードディスクなどより構成される記憶部１１８、モデム、ターミナルアダプタなどより構成される通信部１１９が接続されている。通信部１１９は、インターネットを含むネットワークを介しての通信処理を行う。
【０１５３】
入出力インタフェース１１５にはまた、必要に応じてドライブ１２０が接続され、磁気ディスク１３１、光ディスク１３２、光磁気ディスク１３３、あるいは、半導体メモリ１３４などが適宜装着され、それらから読み出されたコンピュータプログラムが、必要に応じて記憶部１１８にインストールされる。
【０１５４】
一連の処理をソフトウエアにより実行させる場合には、そのソフトウエアを構成するプログラムが、専用のハードウエアに組み込まれているコンピュータ、または、各種のプログラムをインストールすることで、各種の機能を実行することが可能な、例えば汎用のパーソナルコンピュータなどに、ネットワークや記録媒体からインストールされる。
【０１５５】
この記録媒体は、図１６に示されるように、装置本体とは別に、ユーザにプログラムを供給するために配布される、プログラムが記憶されている磁気ディスク１３１（フロッピディスクを含む）、光ディスク１３２（ＣＤ−ＲＯＭ（ＣｏｍｐａｃｔＤｉｓｋ−Ｒｅａｄ　Ｏｎｌｙ　Ｍｅｍｏｒｙ），ＤＶＤ（Ｄｉｇｉｔａｌ　Ｖｅｒｓａｔｉｌｅ　Ｄｉｓｋ）を含む）、光磁気ディスク１３３（ＭＤ（Ｍｉｎｉ−Ｄｉｓｋ）（商標）を含む）、もしくは半導体メモリ１３４などよりなるパッケージメディアにより構成されるだけでなく、装置本体に予め組み込まれた状態でユーザに供給される、プログラムが記憶されているＲＯＭ１１２や、記憶部１１８に含まれるハードディスクなどで構成される。
【０１５６】
なお、本明細書において、記録媒体に記憶されるプログラムを記述するステップは、含む順序に沿って時系列的に行われる処理はもちろん、必ずしも時系列的に処理されなくとも、並列的あるいは個別に実行される処理をも含むものである。
【０１５７】
【発明の効果】
本発明によれば、画像データをエンコードすることができる。
また、本発明によれば、シーンチェンジが起こっていない場合であっても、その画像の変化量が、シーンチェンジが起こったと判断される値に近い場合には、仮想バッファの最小値を制限するようにしたので、量子化インデックスはある一定の数値以上に制限され、シーンチェンジ時に画像が劣化してしまうのを防ぐようにすることができる。
【図面の簡単な説明】
【図１】ＭＰＥＧ２方式によって映像データを圧縮符号化する場合、および圧縮符号化された画像データを復号する場合の処理について説明する図である。
【図２】ＶＢＶバッファについて説明する図である。
【図３】ローディレイコーディングについて説明する図である。
【図４】ＶＢＶバッファについて説明する図である。
【図５】シーンチェンジについて説明する図である。
【図６】従来の仮想バッファ更新処理について説明するフローチャートである。
【図７】シーンチェンジではなく画像スキップが発生する場合について説明する図である。
【図８】本発明を適用したビデオエンコーダの構成を示すブロック図である。
【図９】仮想バッファのバッファ占有量について説明する図である。
【図１０】イントラスライスおよびインタースライス毎の、仮想バッファのバッファ占有量について説明する図である。
【図１１】本発明を適用した仮想バッファ更新処理について説明するフローチャートである。
【図１２】最小値制限処理１について説明するフローチャートである。
【図１３】シーンチェンジ領域と通常領域との間に存在する中間領域αについて説明する図である。
【図１４】最小値制限処理２について説明するフローチャートである。
【図１５】最小値制限処理３について説明するフローチャートである。
【図１６】パーソナルコンピュータの構成を示すブロック図である。
【符号の説明】
６１　ビデオエンコーダ，　７１　前処理部，　７２　演算部，　７３　動きベクトル検出部，　７４　ＤＣＴ部，　７５　量子化部，　７７　ＶＬＣ部，　７８　逆量子化部，　７９　逆ＤＣＴ部，　８０　演算部，　８１　動き補償部，　８２　バッファ，　８３　量子化制御部，　８４　フレームメモリ，　９１イントラＡＣ算出部，　９２　発生符号量制御部，　９３　ＧＣ算出部，　９４　目標符号量算出部，　９５　ＭＥ残差算出部，　９７　中間領域α[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an encoding device, an encoding method, a program, and a recording medium, and particularly to an encoding device, an encoding method, a program, and a recording medium that are suitable for performing low-delay coding.
[0002]
[Prior art]
In recent years, various compression-encoding methods have been proposed as methods for reducing the amount of information by compressing video data and audio data, and MPEG2 (Moving) is a typical method.
Picture Experts Group Phase 2).
[0003]
With reference to FIG. 1, a description will be given of a process in a case where the video data is compression-encoded by the MPEG2 system and a process in a case where the compression-encoded image data is decoded.
[0004]
The encoder 1 on the transmission side converts the frame images 11 of numbers 0 to 11 into an intra-coded image (hereinafter, referred to as an I-picture), an inter-frame forward prediction-coded image (hereinafter, referred to as a P-picture), or It specifies which of the three image types of bidirectional predictive coded image (hereinafter, referred to as B picture) to process, and specifies the image type of the specified frame image (I picture, P picture, or , B pictures), reordering is performed to rearrange the frame images in the order of encoding, and the encoding process is performed on each frame image in that order to generate an encoded frame 12, and the decoder 2 To be transmitted.
[0005]
After decoding the frame image encoded by the encoder 1, the decoder 2 on the receiving side reorders again, restores the image frame to the original order, restores the frame image 13, and displays the reproduced image. .
[0006]
In the encoder 1, since the encoding process is performed after the reordering, the frame image of the number 2 must be encoded before the encoding process of the frame image of the number 0. Hereinafter, this will be referred to as a reordering delay).
[0007]
Also, since the decoder 2 performs reordering after decoding, the frame image of number 2 must be decoded before the frame image of number 0 is decoded and displayed, and the reordering delay is accordingly reduced. Will happen.
[0008]
As described above, since reordering is performed in both the encoder 1 and the decoder 2, a reordering delay of three frames occurs between the encoding of the image data and the display of the reproduced image.
[0009]
When coded data compressed and coded according to the MPEG2 system is transmitted, coded data transmitted from the compression coding apparatus on the transmission side is converted into a video STD (System Target Decoder) buffer (a so-called VBV) on the reception side. (Video Buffer Verifier) buffer for each picture.
[0010]
As shown in FIG. 2, the VBV buffer has a fixed buffer size (capacity), and encoded data is sequentially stored in the VBV buffer for each picture. In this case, each of the coded data of the I picture, the P picture, and the B picture is stored in the VBV buffer at a fixed transmission rate, and is extracted by the decoder at the decoding timing when storage is completed (one frame cycle). . Since an I picture has a larger amount of encoded data than a B picture, it requires more time than a B picture to be stored in a VBV buffer.
[0011]
At this time, when the encoded data is stored in the VBV buffer of the decoder 2 and when the encoded data is extracted from the VBV buffer, the encoder 1 on the data transmission side causes overflow and underflow in the VBV buffer. In order to prevent this, it is necessary to control (rate control) the generated code amount of the encoded data generated based on the buffer occupancy of the VBV buffer. However, since the generated code amount of the I picture necessary for updating the screen is large, a long transmission time of the image data of the I picture is required, and this time is delayed.
[0012]
When performing real-time transmission that requires real-time properties, such as image data from videophones and video conferences, as described above, if a delay due to the transmission time or a reordering delay occurs, the There is a time lag between receiving the transmitted encoded data on the receiving side and displaying the reproduced image. On the other hand, in order to reduce such a delay, in the MPEG2 system, a method called low delay coding (Low Delay Coding) for reducing the delay time to 150 [ms] or less is prepared by the standard.
[0013]
In low-delay coding, only P-pictures are used without using B-pictures that cause reordering delay and I-pictures with a large amount of generated code, and these P-pictures are used as intra-slices consisting of several slices. , And the remaining slices are divided into inter-slices, so that encoding can be performed without reordering.
[0014]
The intra slice is an image portion in which the image data of the slice portion is intra-coded, and the inter slice is the difference data between the image data of the slice portion and the reference image data of the same region in the previous frame image. Image portion.
[0015]
In the low-delay coding, for example, as shown in FIG. 3, the encoder 1 sets all the frame images 11 of numbers 0 to 11 to P-pictures and, for example, within an image frame size of 45 horizontal macroblocks and 24 vertical macroblots. , An area corresponding to two vertical macroblocks and 45 horizontal macroblocks from the top of the frame image of number 0 is set as an intra slice I0, and all other areas are set as inter slices P0.
[0016]
Then, in the frame image of the next number 1, the encoder 1 sets the intra slice I1 in the area having the same area at a position following the intra slice I0 of the frame image of the number 0, and sets all other inter slices as the inter slice. Set to P1. Hereinafter, similarly, an intra slice and an inter slice are set for each frame image, and an intra slice I11 and an inter slice P11 are also set for the frame image of the last number 11.
[0017]
The encoder 1 encodes the intra slices I0 to I11 of each frame image as it is as transmission data, and encodes the other interslices P0 to P11 based on difference data from a reference image in the same region of the previous frame image. (However, at the start of encoding, there is no frame image before becoming the reference image of the inter slice P0, so this is not limited only at the start of encoding.) By repeating the same encoding process for the frame image of number 0 to the frame image of number 11, the encoder 1 encodes the image data of the entire screen in one P-picture, and Can be generated.
[0018]
In this case, the image data sizes of the intra slices I0 to I11 in each frame image are all uniform, and naturally the image data sizes of the inter slices P0 to P11 are also uniform. It becomes a fixed fixed rate.
[0019]
As a result, as shown in FIG. 4, all the frame images of the P picture become coded data having the same generated code amount. The transitions are all the same. As a result, the encoder 1 on the transmission side can easily control the generated code amount of the encoded data without causing an underflow and an overflow in the VBV buffer of the decoder 2, and can control the I-picture having a large generated code amount. Problems caused by such a delay or a reordering delay can be solved, and a reproduced image can be displayed without delay.
[0020]
By the way, in the compression encoding apparatus having the above-described configuration, the intra slices I0 to I11 are encoded as transmission data as they are, and the inter slices P0 to P11 are difference data from the reference image in the same area in the previous frame image. , The actual amount of generated code when the image data portions of the intra slices I0 to I11 are compression-coded is large, and the actual amount of code generated when the image data portions of the inter slices P0 to P11 are compression-coded. The generated code amount is reduced.
[0021]
However, the generated code amount for the entire picture is specified, but the generated code amount allocated to each of the intra slices I0 to I11 and the inter slices P0 to P11 is not specified. In other words, even for an image portion in which the amount of generated codes when encoding is performed as in intra-slices I0 to I11, an image in which the amount of generated codes is not so large when encoded as in interslices P0 to P11. The generated code amount is evenly allocated to the data portion.
[0022]
Therefore, the generated code amount allocated to the intra slices I0 to I11 having a large data amount is small, and the generated code amount allocated to the inter slices P0 to P11 having a small data amount may be large. However, there is a problem in that the image as the whole picture is distorted.
[0023]
Specifically, as shown in FIG. 5, when an image 32 having a high encoding difficulty of an image is present after an image 31 having a low encoding difficulty of the image, an image 31 having a low encoding difficulty is present. Is an image that is easy to encode, so the Q scale is small. However, according to the conventional method, encoding is started with a small Q scale for the subsequent image 32 having a high image coding difficulty. Therefore, a phenomenon occurs in which a given bit amount is consumed in the middle of the screen, and the previous picture remains at the lower end of the screen. This phenomenon has an effect until the intra slice appears next at the problem occurrence location at the lower end of the screen.
[0024]
In order to solve this problem, the present applicant has disclosed in Japanese Patent Application Laid-Open No. H11-205803 an encoding apparatus capable of generating encoded data capable of reproducing a high-quality image on a decoder side even in a low delay mode. An encoding method is proposed.
[0025]
That is, when performing the normal feedback type quantization control to determine the optimal quantization step size for each of the intra slices and the inter slices and perform the quantization control, the next picture is the same as the previous picture and the picture. When a greatly different scene change occurs, instead of using the quantization index data Q (j + 1) calculated based on the immediately preceding picture, based on the ME residual information of the picture to be encoded, By updating the initial buffer capacity d (0) of the virtual buffer, the quantization index data Q (j + 1) is newly calculated. Thus, even when a scene change occurs, the optimal quantization step size is determined for each of the intra slices and the inter slices, and quantization control is performed.
[0026]
The ME residual is calculated for each picture, and is a total value of luminance difference values between a previous picture and a next picture. Therefore, when the ME residual information indicates a large value, it indicates that the picture of the immediately preceding picture is largely different from the picture of the next picture to be coded (so-called scene change).
[0027]
This encoding method will be described with reference to the flowchart in FIG.
[0028]
In step S1, for example, ME residual information obtained when a motion vector is detected is obtained. The obtained ME residual information is defined as ME_info.
[0029]
In step S2, the average value avg of the ME residual information is subtracted from the acquired ME residual information, and it is determined whether or not the calculated value is larger than a predetermined threshold D. The average value avg of the ME residual information is a value updated in step S4 described later, and is represented by the following equation (1).
[0030]
avg = 1/2 (avg + ME_info) (1)
[0031]
In step S2, when it is determined that the calculated value is smaller than the predetermined threshold D, it is determined that there is not much difference between the picture in the current picture and the picture in the immediately preceding picture, that is, that there is no scene change. Therefore, the process proceeds to step S4.
[0032]
In step S2, when it is determined that the calculated value is greater than the predetermined threshold D, it is determined that the difference between the pattern in the current picture and the pattern in the immediately preceding picture is large, that is, a scene change has occurred. Therefore, in step S3, the initial buffer capacity d (0) of the virtual buffer is calculated based on the equations (2), (3), (4), and (5), and the virtual buffer is updated. Is done.
[0033]
X representing the difficulty GC (Global Complexity) of an image in picture units is represented by the following equation (2).
X = T × Q (2)
Here, T is a generated code amount in a picture unit, and Q is an average value of a quantization step size in a picture unit.
[0034]
When the difficulty X of an image in picture units is assumed to be equal to the ME residual information ME_info, that is, when the following equation (3) is satisfied, the quantization index data Q of the entire picture is expressed by the equation ( 4).
[0035]
X = ME_info (3)
Q = {d (0) × 31} / {2 × (br / pr)} (4)
Here, br is a bit rate, and pr is a picture rate.
[0036]
Then, the initial buffer capacity d (0) of the virtual buffer in Expression (4) is represented by the following Expression (5).
d (0) = 2 × {(ME_info × br / pr) / 31 × T} (5)
[0037]
By substituting the initial buffer capacity d (0) of this virtual buffer again into equation (4), the quantization index data Q of the entire picture is calculated.
[0038]
When it is determined in step S2 that the calculated value is smaller than the predetermined threshold value D, or after the process of step S3 ends, in step S4, the ME residual information is prepared for the next picture to be supplied. The average value avg of is calculated and updated by the above equation (1), the process returns to step S1, and the subsequent processes are repeated.
[0039]
According to the processing described with reference to the flowchart of FIG. 6, when a scene change occurs in which the next picture has a pattern greatly different from that of the immediately preceding picture, based on the ME residual information ME_info of the picture to be coded. Thus, the initial buffer capacity d (0) of the virtual buffer is updated, and the quantization index data Q (j + 1) is newly calculated based on this value. The optimal quantization step size is determined for each case.
[0040]
[Problems to be solved by the invention]
However, when the method described in JP-A-11-205803 is used, the same encoding is performed even when the scene changes from an image with high encoding difficulty (difficult) to an image with low encoding difficulty (easy). Since the processing is performed, the image quality is adversely affected.
[0041]
Specifically, virtual buffer adjustment is performed for both the case where the scene changes from an easy image to a difficult image and the case where the scene changes to an easy image, so that the scene is changed from a difficult image to an easy image. In the case of a change, there is a case where the image quality is degraded in an image having a low encoding difficulty, which should have a margin for encoding.
[0042]
For example, as shown in FIGS. 7A to 7C, a simple image (an image that is easy to encode) is configured in a part of an area 41 that is a somewhat complicated image (an image that is difficult to encode). 7A to 7C, the region 42 is not determined to be a scene change depending on the size of the region 42. Sometimes.
[0043]
In FIG. 7B, the quantization index is set to a small value because the easy-to-encode area 42 has a certain size on the screen. However, in the next frame shown in FIG. 7C, the portion of the region 42 that is easy to encode becomes small and the portion of the region 41 that is difficult to encode becomes large, but it is not determined that a scene change has occurred. , A small quantization index based on the previous image (FIG. 7B) results in encoding being performed. Therefore, in the encoding of the image shown in FIG. 7C, a given bit amount is consumed by the middle of the screen (the code amount for encoding the lower end of the image becomes insufficient), and the area 43 at the lower end of the screen is An image skip occurs in which a picture remains.
[0044]
The present invention has been made in view of such a situation, and in intra-slice encoding, it is possible to improve the image quality according to the situation not only at the time of a scene change but also at the time of an image change close to a scene change. Is to do so.
[0045]
[Means for Solving the Problems]
The encoding apparatus according to the present invention includes: a first detection unit configured to detect a change in a picture between a first picture that is a previous picture and a second picture that is a picture to be encoded next; Setting means for setting a minimum value of the initial buffer capacity of the virtual buffer based on a detection result by the first detecting means; and determining means for determining quantization index data using the value of the initial buffer capacity of the virtual buffer. A quantization means for performing quantization based on the quantization index data determined by the determination means, and an encoding means for encoding the quantized coefficient data quantized by the quantization means. And
[0046]
The setting means may set the minimum value of the initial buffer capacity of the virtual buffer when the change amount of the picture detected by the first detecting means is within a predetermined range.
[0047]
The image processing apparatus may further include a second detection unit that detects a bit rate of the frame image, wherein the setting unit is configured such that a change amount of the pattern detected by the first detection unit is within a predetermined range. If the value of the bit rate is smaller than a predetermined value, the minimum value of the initial buffer capacity of the virtual buffer can be set.
[0048]
The setting unit may cause the setting unit to compare the bit rate with a plurality of threshold values, and set a plurality of minimum values of the initial buffer capacity of the virtual buffer based on the comparison result.
[0049]
The first detecting means may calculate an index indicating a difference between the picture of the first picture and the picture of the second picture, and may detect a change in the picture based on the index.
[0050]
All of the frame images may be inter-frame forward prediction encoded images.
[0051]
The encoding method according to the present invention includes a detecting step of detecting a change in a picture between a first picture which is a previous picture and a second picture which is a picture to be next encoded, and a detecting step. A setting step of setting a minimum value of the initial buffer capacity of the virtual buffer based on a detection result by the processing; a determining step of determining quantization index data using the value of the initial buffer capacity of the virtual buffer; and a determining step. A quantization step of performing quantization based on the quantization index data determined by the processing, and an encoding step of encoding the quantized coefficient data quantized by the processing of the quantization step. And
[0052]
The program recorded on the recording medium of the present invention includes a detecting step of detecting a change in a picture between a first picture as a previous picture and a second picture as a picture to be encoded next. A setting step of setting a minimum value of the initial buffer capacity of the virtual buffer based on a detection result obtained by the processing of the detecting step; and a determining step of determining quantization index data using the value of the initial buffer capacity of the virtual buffer. And a quantization step of performing quantization based on the quantization index data determined by the processing of the determination step, and an encoding step of encoding the quantized coefficient data quantized by the processing of the quantization step. It is characterized by including.
[0053]
A program according to the present invention includes a detecting step of detecting a change in a picture between a first picture which is a previous picture and a second picture which is a picture to be encoded next, and the processing of the detecting step. A setting step of setting the minimum value of the initial buffer capacity of the virtual buffer based on the detection result; a determining step of determining the quantization index data using the value of the initial buffer capacity of the virtual buffer; The method includes a quantization step of performing quantization based on the determined quantization index data, and an encoding step of encoding quantized coefficient data quantized by the processing of the quantization step. .
[0054]
In the encoding device, the encoding method, and the program according to the present invention, a change in a pattern between a first picture that is a previous picture and a second picture that is a picture to be encoded next is detected, Based on the detection result, the minimum value of the initial buffer capacity of the virtual buffer is set, the quantization index data is determined using the value of the initial buffer capacity of the virtual buffer, and based on the determined quantization index data, The quantization is performed, and the quantized coefficient data is encoded.
[0055]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0056]
FIG. 8 is a block diagram showing a configuration of the video encoder 61.
[0057]
The video encoder 61 encodes image data by a low delay coding method using all P pictures. The pre-processing unit 71 of the video encoder 61 receives input of image data supplied from the outside.
[0058]
The preprocessing unit 71 divides each frame image of the sequentially input image data (all P pictures in this case) into a macroblock composed of a luminance signal of 16 pixels × 16 lines and a color difference signal corresponding to the luminance signal. Then, this is supplied as macroblock data to the calculation unit 72, the motion vector detection unit 73, and the intra AC calculation unit 91 of the quantization control unit 83.
[0059]
The motion vector detection unit 73 receives the input of the macroblock data, calculates the motion vector of each macroblock based on the macroblock data and the reference image data stored in the frame memory 84, and calculates the motion vector as the motion vector data. , To the motion compensator 81.
[0060]
The arithmetic unit 72 performs the intra mode for the intra slices I0 to I11 and the inter mode for the inter slices P0 to P11 based on the image type of each macro block with respect to the macroblock data supplied from the preprocessing unit 71. Motion compensation is performed in the forward prediction mode.
[0061]
Here, the intra mode is a method in which a frame image to be encoded is directly used as transmission data, and the forward prediction mode is a method in which a prediction residual between the frame image to be encoded and a past reference image is transmitted data. It is a method. In the video encoder 61, encoding is performed using only P pictures and divided into intra slices I0 to I11 and inter slices P0 to P11.
[0062]
First, when the macroblock data is one of the intra slices I0 to I11, the macroblock data is processed in the intra mode. That is, the arithmetic unit 72 sends the macroblock of the input macroblock data to the DCT (Discrete Cosine Transform: discrete cosine transform) unit 74 as arithmetic data as it is. The DCT unit 74 converts the input operation data into DCT coefficients by performing a DCT conversion process, and sends the converted data to the quantization unit 75 as DCT coefficient data.
[0063]
The quantization unit 75 performs a quantization process on the input DCT coefficient data based on the quantization index data Q (j + 1) supplied from the generated code amount control unit 92, and generates VLC as the quantized DCT coefficient data. (Variable Length Code; variable length coding) unit 77 and inverse quantization unit 78. Here, the quantization unit 75 adjusts the quantization step size in the quantization process according to the quantization index data Q (j + 1) supplied from the generated code amount control unit 92, thereby reducing the generated code amount. Have been made to control.
[0064]
The quantized DCT coefficient data sent to the inverse quantization unit 78 undergoes an inverse quantization process with the same quantization step size as the quantization unit 75, and is sent to the inverse DCT unit 79 as DCT coefficient data. The inverse DCT unit 79 performs an inverse DCT process on the supplied DCT coefficient data, and the generated operation data is sent to the operation unit 80 and stored in the frame memory 84 as reference image data.
[0065]
If the macroblock data is one of the inter slices P0 to P11, the calculation unit 72 performs a motion compensation process on the macroblock data in the forward prediction mode.
[0066]
The motion compensation unit 81 performs motion compensation on the reference image data stored in the frame memory 84 according to the motion vector data, and calculates forward prediction image data. The calculation unit 72 performs a subtraction process on the macroblock data using the forward prediction image data supplied from the motion compensation unit 81.
[0067]
That is, in the forward prediction mode, the motion compensation unit 81 reads the reference image data by shifting the read address of the frame memory 84 according to the motion vector data, and uses the read reference image data as the forward prediction image data. It is supplied to the arithmetic unit 80. The operation unit 72 subtracts the forward prediction image data from the supplied macroblock data to obtain difference data as a prediction residual. Then, the arithmetic unit 72 sends the difference data to the DCT unit 74.
[0068]
Further, the forward prediction image data from the motion compensation unit 81 is supplied to the operation unit 80, and the operation unit 80 adds the forward prediction image data to the operation data supplied from the inverse DCT unit. , And locally reproduces the reference image data, and outputs it to the frame memory 84 for storage.
[0069]
Thus, the image data input to the video encoder 61 undergoes motion compensation prediction processing, DCT processing, and quantization processing, and is supplied to the VLC unit 77 as quantized DCT coefficient data. The VLC unit 77 performs a variable-length encoding process on the quantized DCT coefficient data based on a predetermined conversion table, sends the resulting variable-length encoded data to the buffer 82, and performs encoding for each macro block. The generated code amount data B (j) indicating the number of generated bits is sent to the generated code amount control unit 92 of the quantization control unit 83 and the GC (Global Complexity) calculation unit 93, respectively.
[0070]
The GC calculation unit 93 sequentially accumulates the generated code amount data B (j) for each macro block, and when all the generated code amount data B (j) for one picture are accumulated, the GC calculation unit 93 generates the generated code amount data B (j). The generated code amount for one picture is calculated by cumulatively adding the code amount data B (j).
[0071]
Then, the GC calculation unit 93 calculates the product of the generated code amount of the intra slice portion of one picture and the average value of the quantization step size in the intra slice portion, using the following equation (6). Thus, GC data Xi representing the difficulty of the image in the intra slice portion (hereinafter, referred to as GC) is obtained, and supplied to the target code amount calculation section 94.
[0072]
Xi = (Ti / Ni) × Qi (6)
Here, Ti is the generated code amount of the intra slice, Ni is the number of intra slices, and Qi is the average value of the quantization step size of the intra slice.
[0073]
At the same time, the GC calculation unit 93 uses the following equation (7) to calculate the generated code amount of the inter-slice portion of one picture and the average value of the quantization step size in the inter-slice portion. By calculating the product, the GC data Xp in the inter-slice part is obtained, and this is supplied to the target code amount calculation unit 94.
[0074]
Xp = (Tp / Np) × Qp (7)
Here, Tp is the generated code amount of the inter slice, Np is the number of inter slices, and Qp is the average value of the quantization step size of the inter slice.
[0075]
Based on the GC data Xi supplied from the GC calculation unit 93, the target code amount calculation unit 94 calculates the target generated code amount data Tpi of the intra slice portion in the next picture using the following equation (8). At the same time, based on the GC data Xp supplied from the GC calculation unit 93, the target generated code amount data Tpp of the inter slice portion in the next picture is calculated based on the following equation (9), and the calculated target generated code Tpp is calculated. The amount data Tpi and Tpp are sent to the generated code amount control unit 92, respectively.
[0076]
Tpi = {(Ni × Xi) / (Np × Xp) + (Np × Xi)} × Xp (8)
[0077]
Tpp = {(Np × Xp) / (Np × Xp) + (Ni × Xi)} × Xp (9)
[0078]
Further, the target code amount calculation unit 94 receives an input of the bit rate value input by the user using the operation input unit 85, and supplies the input to the code amount generation unit 92.
[0079]
The ME residual calculation unit 95 calculates ME residual information ME_info based on the input macroblock data, and outputs it to the generated code amount control unit 92. Here, the ME residual information ME_info is calculated on a picture basis, and is a total value of luminance difference values between a previous picture and a next picture. Therefore, when the ME residual information ME_info indicates a large value, it indicates that the picture of the immediately preceding picture is largely different from the picture of the next picture to be coded (so-called scene change).
[0080]
If the picture of the previous picture is different from the picture of the picture to be encoded next, the quantum generated based on the target generated code amount data Tpi and Tpp calculated using the image data of the previous picture. It is not appropriate to determine the quantization step size of the quantization unit 75 based on the quantization index data Q (j + 1). Therefore, when a scene change occurs, the target generated code amount data Tpi and Tpp may be newly calculated and re-calculated.
[0081]
The intra AC calculation unit 91 calculates the intra AC (intra AC), and outputs mad_info indicating the current value of the intra AC and prev_mad_info indicating the value of the immediately previous intra AC to the generated code amount control unit 92. I do.
[0082]
Intra AC is a parameter defined as the sum of variances with video data for each DCT block in the DCT processing unit in the MPEG system, and indicates the complexity of the video, the difficulty of the picture pattern of the video, and the Correlated with data volume. That is, the intra AC is a total sum in the screen of the absolute value sum of the pixel value of each pixel minus the average value of the pixel value of each block in DCT block units. Intra AC is represented by the following equation (10).
[0083]
(Equation 1)

... (10)
[0084]
In the equation (10), the equation (11) is satisfied.
(Equation 2)

... (11)
[0085]
If the virtual buffer adjustment is performed for both the scene change from easy to difficult image coding and the scene change from difficult to easy image coding, it will change from difficult to easy. In the scene change described above, the image quality may be degraded in an easy image for which there is room for encoding. In addition, even in the case of a scene change from a difficult to an easy one, it may be better to adjust the virtual buffer depending on the magnitude of the change or the difficulty of the image after the scene change. However, the presence / absence of a scene change can be determined only by ME residual information. Change cannot be determined.
[0086]
Therefore, the intra AC calculation unit 91 calculates the intra AC, and outputs to the generated code amount control unit 92 the mad_info indicating the current value of the intra AC and the prev_mad_info indicating the value of the immediately previous intra AC. Accordingly, the generated code amount control unit 92 can determine the state of the scene change and determine whether to perform the virtual buffer adjustment.
[0087]
The generated code amount control unit 92 constantly monitors the accumulation state of the variable-length encoded data stored in the buffer 82, and determines the quantization step size based on the occupation amount information indicating the accumulation state. I have.
[0088]
When the generated code amount data B (j) of the macroblock actually generated is larger than the target generated code amount data Tpi of the intra slice portion, the generated code amount control unit 92 performs quantization to reduce the generated code amount. When the step size is increased and the actual generated code amount data B (j) is smaller than the target generated code amount data Tpi, the quantization step size is reduced to increase the generated code amount.
[0089]
Further, similarly, in the case of the inter-slice portion, when the generated code amount data B (j) of the macroblock actually generated is larger than the target generated code amount data Tpp, the generated code amount control unit 92 determines the generated code amount. In order to increase the generated code amount, if the actual generated code amount data B (j) is smaller than the target generated code amount data Tpp, the quantization step size is decreased. It has been made.
[0090]
That is, the generated code amount control unit 92 assumes the transition of the accumulation state of the variable-length encoded data stored in the VBV buffer provided on the decoder side, as shown in FIG. The buffer occupancy d (j) of the virtual buffer in the block is represented by the following equation (12), and the buffer occupancy d (j + 1) of the virtual buffer in the j + 1-th macroblock is represented by the following equation (13): By subtracting equation (13) from equation (12), the buffer occupancy d (j + 1) of the virtual buffer in the (j + 1) th macroblock can be transformed into the following equation (14).
[0091]
d (j) = d (0) + B (j−1) − {T × (j−1) / MBcnt} (12)
[0092]
Here, d (0) is the initial buffer capacity, B (j) is the number of bits generated in the j-th macroblock, MBcnt is the number of macroblocks in the picture, and T is the target generation in picture units. The code amount.
[0093]
d (j + 1) = d (0) + B (i) − (T × j) / MBcnt (13)
[0094]
d (j + 1) = d (j) + {B (j) -B (j-1)}-T / MBcnt (14)
[0095]
Subsequently, since the macroblock in the picture is divided into an intra-slice part and an inter-slice part, the generated code amount control unit 92 determines whether the macro block in the intra-slice part and the Target generated code amounts Tpi and Tpp to be assigned to each macroblock are individually set.
[0096]
In the graph, when the number of macroblocks is between 0 and s and between t and end, the target generated code amount Tpp of the interslice is substituted into the following equation (15), so that The buffer occupancy d (j + 1) can be obtained.
[0097]
d (j + 1) = d (j) + {B (j) -B (j-1)}-Tpp / {MBcnt- (ts)} (15)
[0098]
Also, when the count number of the macroblock is between s and t, the target generated code amount Tpi of the intra slice is substituted into the following equation (16), whereby the buffer occupation amount d (j + 1) in the intra slice portion is obtained. ) Can be obtained.
[0099]
d (j + 1) = d (j) + {B (j) -B (j-1)}-Tpi / (ts) (16)
[0100]
Therefore, the generated code amount control unit 92 substitutes the buffer occupancy d (j + 1) in the intra-slice part and the inter-slice part and the constant r shown in the equation (17) into the equation (18), thereby obtaining the macro The quantization index data Q (j + 1) of the block (j + 1) is calculated and supplied to the quantization unit 75.
[0101]
r = (2 × br) / pr (17)
Q (j + 1) = d (j + 1) × (31 / r) (18)
Here, br is a bit rate, and pr is a picture rate.
[0102]
The quantization unit 75 determines a quantization step size corresponding to an intra slice or an inter slice in the next macroblock based on the quantization index data Q (j + 1), and quantizes the DCT coefficient data according to the quantization step size. I do.
[0103]
Thereby, the quantization unit 75 calculates the intra-slice part and the inter-slice of the next picture, which are calculated based on the actual generated code amount data B (j) in the intra-slice part and the inter-slice part of the previous picture. The DCT coefficient data can be quantized by the optimal quantization step size for the target generated code amounts Tpp and Tpi in the portion.
[0104]
Thus, the quantization unit 75 can quantize the buffer 82 so as not to overflow or underflow, and quantize the decoder-side VBV buffer so as not to overflow or underflow in accordance with the data occupancy of the buffer 82. Quantized DCT coefficient data can be generated.
[0105]
For example, in Japanese Patent Application No. 11-205803 described above as a conventional technique, when a picture of a picture to be encoded next changes greatly while performing normal feedback-type quantization control, a feedback-type quantum control is performed. The initialization control is stopped, the initial buffer capacity d (0) of the virtual buffer is initialized based on the ME residual information supplied from the ME residual calculating unit 95, and based on the new initial buffer capacity d (0), The quantization index data Q (j + 1) is newly calculated for each of the intra slice and the inter slice.
[0106]
However, as in the conventional case, when it is determined whether or not to perform the virtual buffer adjustment using only the ME residual, when the image difficulty changes from easy to difficult, and from difficult to simple, In both cases, the virtual buffer adjustment is performed. That is, when the image difficulty is changed from a difficult one to a simple one, the result is that the image quality is degraded in a simple image for which there is room for encoding.
[0107]
Therefore, in the video encoder 61 of FIG. 8, for example, using information such as the intra AC calculated by the intra AC calculator 91, the virtual buffer is used only when a scene change in which the image difficulty changes from easy to difficult occurs. By performing the adjustment, it is possible to prevent the image quality from deteriorating in a simple image.
[0108]
In other words, the generated code amount control unit 92 performs the normal feedback-type quantization control and stops the feedback-type quantization control when the picture of the next picture to be coded greatly changes, and stops the ME remaining. Whether to initialize the initial buffer capacity d (0) of the virtual buffer based on the ME residual information ME_info supplied from the difference calculation unit 95 and prev_mad_info and mad_info supplied from the intra AC calculation unit 91 Is determined, and the initial buffer capacity d (0) of the virtual buffer is initialized based on the ME residual information ME_info supplied from the ME residual calculator 95. Is initialized. The initialization of the initial buffer capacity d (0) of the virtual buffer is the same as in the conventional case described using the equations (2) to (5).
[0109]
Then, based on the new initial buffer capacity d (0), the generated code amount control unit 92 uses the equations (12) to (18) to calculate the quantized index data Q ( j + 1) is newly calculated and supplied to the quantization unit 75.
[0110]
Further, the generated code amount control unit 92 determines that the change of the picture of the picture to be coded next is not large enough to be judged as a scene change, but is close to being judged as a scene change. , The minimum value of the virtual buffer is limited to be larger than a certain value. As a result, the minimum value of the quantization index is also limited, so that it is possible to suppress the occurrence of image skipping in the situation described with reference to FIG.
[0111]
Referring to the flowchart of FIG. 11, using image difficulty information such as intra AC, a scene change is introduced to determine whether or not a change from a simple image to a difficult image is performed to adjust the virtual buffer. The virtual buffer update process to be performed will be described.
[0112]
In step S21, the generated code amount control unit 92 acquires the ME residual information ME_info from the ME residual calculation unit 95.
[0113]
In step S22, the generated code amount control unit 92 subtracts the average value avg of the ME residual information from the acquired ME residual information, and determines whether or not ME_info-avg> D, that is, the calculated value. Is larger than a predetermined threshold value D. The average value avg of the ME residual information is a value updated in step S26 described later, and is represented by the above-described equation (1). Note that the predetermined threshold value D is a value of a property that is tuned while considering image quality.
[0114]
If it is determined in step S22 that the calculated value is larger than the predetermined threshold D, it is determined that the difference between the picture in the current picture and the picture in the immediately preceding picture is large, that is, that a scene change has occurred. Therefore, in step S23, the generated code amount control unit 92 obtains the value of the intra AC after the scene change, mad_info, acquired from the intra AC calculation unit 91, and the value of the intra AC before the scene change. The value is compared with prev_mad_info, and it is determined whether or not mad_info> prev_mad_info.
[0115]
If it is determined in step S23 that mad_info is not greater than prev_mad_info, this scene change is a scene change from a difficult image to a simple image, and the process proceeds to step S26.
[0116]
If it is determined in step S23 that mad_info> prev_mad_info, this scene change is a scene change from a simple image to a difficult image, and thus in step S24, the generated code amount control unit 92 determines that FIG. The initial buffer capacity d (0) of the virtual buffer is updated by the same processing as in the conventional case described above.
[0117]
That is, the generated code amount control unit 92 calculates the initial buffer capacity d (0) of the virtual buffer based on the above equations (2), (3), (4) and (5), and Update the buffer.
[0118]
In step S22, when it is determined that the calculated value is smaller than the predetermined threshold D, it is determined that there is not much difference between the picture in the current picture and the picture in the immediately preceding picture, that is, there is no scene change. Therefore, in step S25, a minimum value restriction process 1 described later with reference to FIG. 12, a minimum value restriction process 2 described later with reference to FIG. 13, or a minimum value restriction process 3 described later with reference to FIG. Is performed.
[0119]
If it is determined in step S23 that mad_info> prev_mad_info is not satisfied, after the processing in step S24 or the processing in step S25 is completed, in step S26, the generated code amount control unit 92 is supplied next. In preparation for a picture, the average value avg of the ME residual information is updated by the above equation (1), the process returns to step S21, and the subsequent processes are repeated.
[0120]
According to the processing described using the flowchart of FIG. 11, the virtual buffer adjustment is performed only at the time of a scene change in which the image difficulty is changed from easy to difficult by using the intra AC. In a simple image that should exist, it is possible to prevent the image quality from further deteriorating.
[0121]
Further, if a scene change is not detected, a minimum value restriction process 1 described later with reference to FIG. 12, a minimum value restriction process 2 described later with reference to FIG. 13, or a minimum value restriction process 2 described later with reference to FIG. One of the processes 3 is executed.
[0122]
Next, the minimum value restriction processing 1 will be described with reference to the flowchart of FIG.
[0123]
In step S41, the generated code amount control unit 92 compares the value of ME_info-avg calculated in step S22 with a predetermined threshold value E based on the ME residual information ME_info acquired in step S21 of FIG. Determine whether −avg> E. Here, the threshold value E is a value smaller than the threshold value D in step S22 in FIG.
[0124]
When it is determined in step S41 that ME_info-avg> E, in step S42, the generated code amount control unit 92 generates a control signal for limiting the minimum value of the virtual buffer, and sends the control signal to the quantization unit 75. Supply. Based on the control signal supplied from the generated code amount control unit 92, the quantization unit 75 sets the buffer capacity of the virtual buffer to at least a predetermined value even if the previous image is simple. The quantization index is calculated by performing the processing described with reference to Expressions (12) to (18) while limiting the quantization index to be larger. The calculated quantization index also has a minimum value restricted. After the processing in step S42 ends, the processing returns to the processing in step S26 in FIG.
[0125]
That is, in step S22 of FIG. 11, it is determined that ME_info-avg> D is not satisfied, and in step S41, it is determined that ME_info-avg> E, the value of ME_info-avg is determined as shown in FIG. ME_info-avg exists in the scene change area 96 when the value of the threshold value is larger than the threshold value D, and in the intermediate area α97 between the normal area 98 when the value of the ME_info-avg value is smaller than the threshold value E (D>ME_info-avg>). E holds).
[0126]
In the intermediate region α97, it is not determined that a scene change has occurred, but if the value of the quantization index fed back based on the previous image is very small, as described with reference to FIG. There is a possibility that a given bit amount is consumed halfway, the code amount becomes insufficient, and image skipping occurs. Accordingly, in step S42, by limiting the minimum value of the value of the virtual buffer, the minimum value of the quantization index is also limited, so that the occurrence of image skipping can be suppressed.
[0127]
If it is determined in step S41 that ME_info-avg> E is not satisfied, it is determined that the difference from the previous image is such that it is not necessary to consider the minimum value of the quantization index. The process returns to step S26 in FIG. 11 without being performed.
[0128]
An image that is not determined to be a scene change by the processing described with reference to FIG. 12 but is likely to cause image skipping (a phenomenon that a previous picture remains at the lower end of the screen) in the conventional method is displayed on the screen. It is possible to encode to the lower end.
[0129]
By the way, rate control when encoding an image with a high bit rate has more margin than encoding an image with a low bit rate. If the minimum value of the virtual buffer (quantization index) is limited even though there is a sufficient margin for the rate control, the image quality deteriorates.
[0130]
With reference to the flowchart of FIG. 14, a description will be given of a minimum value restriction process 2 that determines whether to perform the minimum value restriction by referring to the bit rate value in addition to the ME residual.
[0131]
In step S51, the generated code amount control unit 92 compares the value of ME_info-avg calculated in step S22 with a predetermined threshold E based on the ME residual information ME_info acquired in step S21 of FIG. It is determined whether or not ME_info-avg> E. Here, the threshold value E is a value smaller than the threshold value D in step S22 in FIG.
[0132]
When it is determined in step S51 that ME_info-avg> E, in step S52, the generated code amount control unit 92 determines that the value of the bit rate supplied from the target code amount calculation unit 94 is smaller than a predetermined threshold F. Determine whether it is small. Here, the threshold value F is a value for determining whether or not a margin occurs in the rate control when encoding.
[0133]
If it is determined in step S52 that the value of the bit rate is smaller than the predetermined threshold F, in step S53, the generated code amount control unit 92 generates a control signal for limiting the minimum value of the virtual buffer, This is supplied to the quantization unit 75. Based on the control signal supplied from the generated code amount control unit 92, the quantization unit 75 sets the buffer capacity of the virtual buffer to at least a predetermined value even if the previous image is simple. The quantization index is calculated by performing the processing described with reference to Expressions (12) to (18) while limiting the quantization index to be larger. The calculated quantization index also has a minimum value restricted. After the end of the process in the step S53, the process returns to the process in the step S26 in FIG.
[0134]
If it is determined in step S51 that ME_info-avg> E is not satisfied, it is determined that the difference from the previous image is such that it is not necessary to consider the minimum value of the quantization index. Is determined to be larger than the predetermined threshold value F, it is determined that there is room for the rate control, so that the minimum value is not limited, and the process returns to the process of step S26 in FIG.
[0135]
By the processing described with reference to the flowchart of FIG. 14, the minimum value of the virtual buffer (quantization index) is limited to prevent the image quality from deteriorating even though the rate control has sufficient margin. Only when necessary, it is possible to limit the minimum value of the virtual buffer (quantization index) and suppress the occurrence of image skipping (a phenomenon that the previous picture remains at the lower end of the screen).
[0136]
Furthermore, by making it possible to set a plurality of minimum values of the virtual buffer (quantization index) according to the value of the bit rate, the image quality can be further stabilized.
[0137]
Next, with reference to the flowchart of FIG. 15, a description will be given of a minimum value restriction process 3 that can set a plurality of minimum values of the virtual buffer (quantization index) according to the value of the bit rate.
[0138]
In step S61, the generated code amount control unit 92 compares the value of ME_info-avg calculated in step S22 with a predetermined threshold E based on the ME residual information ME_info acquired in step S21 of FIG. It is determined whether or not ME_info-avg> E. Here, the threshold value E is a value smaller than the threshold value D in step S22 in FIG.
[0139]
If it is determined in step S61 that ME_info-avg> E, in step S62, the generated code amount control unit 92 sets the value of the bit rate supplied from the target code amount calculation unit 94 to be smaller than a predetermined threshold G. Determine whether it is small. Here, the threshold value G is a value for determining whether or not there is a margin in rate control when encoding. In addition, in encoding when the bit rate is low, by strictly limiting the minimum value, it is possible to more effectively prevent the image quality from deteriorating and suppress the occurrence of image skipping. It is preferable that the value be smaller than the threshold value F in the processing described with reference to FIG.
[0140]
If it is determined in step S62 that the value of the bit rate is smaller than the predetermined threshold value G, in step S63, the generated code amount control unit 92 sets the minimum value of the virtual buffer to a predetermined first value. A control signal for limiting is generated and supplied to the quantization unit 75. Based on the control signal supplied from the generated code amount control unit 92, the quantization unit 75 sets the buffer capacity of the virtual buffer to be at least larger than the first value even if the previous image is simple. With the restriction, the quantization index is calculated by the processing described using equations (12) to (18). The calculated quantization index is also the one whose minimum value is limited by the first value. After the processing in step S63 ends, the processing returns to the processing in step S26 in FIG.
[0141]
In step S62, when it is determined that the value of the bit rate is greater than the predetermined threshold G, in step S64, the generated code amount control unit 92 sets the bit rate value supplied from the target code amount calculation unit 94 to: It is determined whether it is smaller than a predetermined threshold value H. Here, the threshold value H for determining whether or not a margin occurs in the rate control is a value larger than the threshold value G. For example, assuming that the threshold value H is equal to the threshold value F in the processing described with reference to FIG. It is suitable.
[0142]
When it is determined in step S64 that the value of the bit rate is smaller than the predetermined threshold value H, in step S65, the generated code amount control unit 92 sets the minimum value of the virtual buffer to be smaller than the first value in step S63. A small control signal for restricting to a predetermined second value is generated and supplied to the quantization unit 75. Based on the control signal supplied from the generated code amount control unit 92, the quantization unit 75 controls the buffer capacity of the virtual buffer to be at least larger than the second value even if the previous image is simple. With the restriction, the quantization index is calculated by the processing described using equations (12) to (18). The minimum value of the calculated quantization index is also limited by the second value. After the processing in step S65 ends, the processing returns to the processing in step S26 in FIG.
[0143]
If it is determined in step S61 that ME_info-avg> E is not satisfied, it is determined that the difference from the previous image is such that it is not necessary to consider the minimum value of the quantization index. The process returns to step S26 in FIG. 11 without being performed. If it is determined in step S64 that the value of the bit rate is larger than the predetermined threshold value H, it is determined that there is sufficient margin for the rate control. It returns to the process of step S26 of FIG.
[0144]
According to the above-described processing, even when the image change amount is near the scene change detection threshold, an image that is difficult to encode is detected, and the minimum value of the virtual buffer (quantization index) is limited. Furthermore, it is possible to detect an image for which rate control is difficult and to set the minimum value of the virtual buffer (quantization index) corresponding to the bit rate.
[0145]
Thus, even in the low delay mode and the low bit rate, image skipping (a phenomenon in which the previous picture remains at the lower end of the screen) is suppressed, and stable image quality can be obtained.
[0146]
Further, in the above-described embodiment, all the frame images of numbers 0 to 11 are set as P pictures as the low delay coding, and, for example, the upper part of the frame image in the image frame size of 45 horizontal macroblocks and 24 vertical macroblocks Has described the case where the area for 2 vertical macroblocks and 45 horizontal macroblocks is set as one intra-slice part, and the others are all set as inter-slice parts. However, the present invention is not limited to this. The portion may be formed of an area of various other sizes, such as an area for one vertical macroblock and 45 horizontal macroblocks.
[0147]
Also, here, the case where low delay encoding is performed has been described as an example. However, the present invention may be applied to, for example, encoding 15 frames into an intra-frame coded image (hereinafter, referred to as an I-picture) and an inter-frame forward prediction coded image. (Hereinafter, referred to as a P picture) or a bidirectional predictive coded image (hereinafter, referred to as a B picture), which one of the three image types is to be processed. It is also applicable to the case where a frame image is encoded according to the image type (I picture, P picture, or B picture).
[0148]
Further, in the above-described embodiment, a case has been described in which the present invention is applied to the video encoder 61 as an encoding device that performs compression and encoding by the MPEG method. However, the present invention is not limited to this, and other embodiments are applicable. You may make it apply to the encoding apparatus by various image compression systems.
[0149]
The series of processes described above can be executed by hardware, but can also be executed by software. In this case, for example, the video encoder 61 is configured by a personal computer 101 as shown in FIG.
[0150]
In FIG. 16, a CPU 111 executes various processes according to a program stored in a ROM 112 or a program loaded from a storage unit 118 into a RAM 113. The RAM 113 also appropriately stores data necessary for the CPU 111 to execute various processes.
[0151]
The CPU 111, the ROM 112, and the RAM 113 are mutually connected via a bus 114. The bus 114 is also connected to an input / output interface 115.
[0152]
The input / output interface 115 includes an input unit 116 including a keyboard and a mouse, an output unit 117 including a display and a speaker, a storage unit 118 including a hard disk, and a communication unit 119 including a modem, a terminal adapter, and the like. It is connected. The communication unit 119 performs communication processing via a network including the Internet.
[0153]
A drive 120 is connected to the input / output interface 115 as necessary, and a magnetic disk 131, an optical disk 132, a magneto-optical disk 133, a semiconductor memory 134, or the like is appropriately mounted, and a computer program read out from these is loaded. Are installed in the storage unit 118 as needed.
[0154]
When a series of processing is executed by software, a program constituting the software executes various functions by installing a computer built in dedicated hardware or installing various programs. For example, it is installed in a general-purpose personal computer or the like from a network or a recording medium.
[0155]
As shown in FIG. 16, this recording medium is distributed separately from the apparatus main body to supply the program to the user, and is stored on a magnetic disk 131 (including a floppy disk) and an optical disk 132 (including a floppy disk) storing the program. It is configured by a package medium including a CD-ROM (Compact Disk-Read Only Memory), a DVD (including a Digital Versatile Disk), a magneto-optical disk 133 (including an MD (Mini-Disk) (trademark)), a semiconductor memory 134, or the like. In addition to this, it is configured with a ROM 112 storing a program, a hard disk included in a storage unit 118, and the like, which are supplied to a user in a state of being incorporated in the apparatus main body in advance.
[0156]
In the present specification, the steps of describing a program stored in a recording medium include, in addition to the processing performed in chronological order in the order in which the program is included, the processing is not necessarily performed in chronological order, but may be performed in parallel or individually. This includes the processing to be executed.
[0157]
【The invention's effect】
According to the present invention, image data can be encoded.
Further, according to the present invention, even when no scene change has occurred, the minimum value of the virtual buffer is limited when the amount of change in the image is close to the value determined to have caused the scene change. As a result, the quantization index is limited to a certain value or more, so that it is possible to prevent the image from deteriorating at the time of a scene change.
[Brief description of the drawings]
FIG. 1 is a diagram for explaining processing when video data is compression-encoded by the MPEG2 system and processing when decoding compression-encoded image data.
FIG. 2 is a diagram illustrating a VBV buffer.
FIG. 3 is a diagram illustrating low delay coding.
FIG. 4 is a diagram illustrating a VBV buffer.
FIG. 5 is a diagram illustrating a scene change.
FIG. 6 is a flowchart illustrating a conventional virtual buffer update process.
FIG. 7 is a diagram illustrating a case where an image skip occurs instead of a scene change.
FIG. 8 is a block diagram illustrating a configuration of a video encoder to which the present invention has been applied.
FIG. 9 is a diagram illustrating the buffer occupancy of a virtual buffer.
FIG. 10 is a diagram illustrating buffer occupancy of a virtual buffer for each intra slice and inter slice.
FIG. 11 is a flowchart illustrating a virtual buffer update process to which the present invention has been applied.
FIG. 12 is a flowchart illustrating minimum value restriction processing 1;
FIG. 13 is a diagram illustrating an intermediate area α existing between a scene change area and a normal area.
FIG. 14 is a flowchart illustrating minimum value restriction processing 2;
FIG. 15 is a flowchart illustrating a minimum value restriction process 3;
FIG. 16 is a block diagram illustrating a configuration of a personal computer.
[Explanation of symbols]
61 video encoder, 71 preprocessing unit, 72 operation unit, 73 motion vector detection unit, 74 DCT unit, 75 quantization unit, 77 VLC unit, 78 inverse quantization unit, 79 inverse DCT unit, 80 operation unit, 81 motion compensation Unit, 82 buffer, 83 quantization control unit, 84 frame memory, 91 intra AC calculation unit, 92 generated code amount control unit, 93 GC calculation unit, 94 target code amount calculation unit, 95 ME residual calculation unit, 97 intermediate area α

Claims (9)

First detecting means for detecting a change in a picture between a first picture which is a previous picture and a second picture which is a picture to be subjected to next encoding in a coding apparatus for coding a frame image. When,
Setting means for setting a minimum value of the initial buffer capacity of the virtual buffer based on a detection result by the first detecting means;
Using the value of the initial buffer capacity of the virtual buffer, determining means for determining quantization index data,
Based on the quantization index data determined by the determination means, quantization means for performing quantization,
An encoding unit that encodes the quantized coefficient data quantized by the quantization unit. The setting means sets a minimum value of the initial buffer capacity of the virtual buffer when a change amount of the picture detected by the first detecting means is within a predetermined range. Item 2. The encoding device according to Item 1. A second detection unit that detects a bit rate of the frame image,
The setting means, when the change amount of the pattern detected by the first detection means is within a predetermined range, and the value of the bit rate is smaller than a predetermined value, 2. The encoding apparatus according to claim 1, wherein a minimum value of the initial buffer capacity is set. 4. The code according to claim 3, wherein the setting unit compares the bit rate with a plurality of thresholds, and sets a plurality of minimum values of the initial buffer capacity of the virtual buffer based on a comparison result. 5. Device. The first detection means calculates an index indicating a difference between the picture of the first picture and the picture of the second picture, and detects a change in the picture based on the index. The encoding device according to claim 1. The encoding apparatus according to claim 1, wherein the frame images are all inter-frame forward prediction encoded images. In an encoding method of an encoding device that encodes a frame image,
A detecting step of detecting a change in a picture between a first picture which is a previous picture and a second picture which is a picture to be encoded next;
A setting step of setting a minimum value of the initial buffer capacity of the virtual buffer based on a detection result obtained by the processing of the detection step;
Using the value of the initial buffer capacity of the virtual buffer, determining the quantization index data,
A quantization step of performing quantization based on the quantization index data determined by the processing of the determination step;
An encoding step of encoding the quantized coefficient data quantized by the processing of the quantization step. A program for an encoding device that encodes a frame image,
A detecting step of detecting a change in a picture between a first picture which is a previous picture and a second picture which is a picture to be encoded next;
A setting step of setting a minimum value of the initial buffer capacity of the virtual buffer based on a detection result obtained by the processing of the detection step;
Using the value of the initial buffer capacity of the virtual buffer, determining the quantization index data,
A quantization step of performing quantization based on the quantization index data determined by the processing of the determination step;
A coding step of coding the quantized coefficient data quantized by the processing of the quantization step. A computer-executable program that controls an encoding device that encodes a frame image,
A detecting step of detecting a change in a picture between a first picture which is a previous picture and a second picture which is a picture to be encoded next;
A setting step of setting a minimum value of the initial buffer capacity of the virtual buffer based on a detection result obtained by the processing of the detection step;
Using the value of the initial buffer capacity of the virtual buffer, determining the quantization index data,
A quantization step of performing quantization based on the quantization index data determined by the processing of the determination step;
A coding step of coding the quantized coefficient data quantized by the processing of the quantization step.

Images