JP4621415B2 - Moving picture coding method, moving picture coding apparatus, and moving picture coding processing program - Google Patents

Description

【００５５】
で表される。
【００５６】
また、Ｄは誤差に急に大きな変動が起こった場合、それに対応して符号量の制御量を大きくするものである。これを示したものが図３であり、変動量Ｄが正の方向に所定量よりも大きくなると、符号量を減らす方向に修正され、変動量Ｄが負の方向に所定量よりも大きくなると、符号量を増やす方向に修正される。なお、ｎ番目のフレームにおける生成符号量の変動量Ｄ（ｎ）は、
Ｄ（ｎ）＝Ｂ（ｎ）−Ｂ（ｎ−１） （３）
で表すことができる。
【００５７】
これは、急激な変化が起こった場合に、それをより速く修正するためのものであるが、この符号を逆にすれば、急激な変化に対する過剰な反応を抑制する目的で使用する事もできる。
【００５８】
たとえば、変動量Ｄが急に小さくなった場合（これは、動きが止まったときに相当する）は、Ｐ，Ｉの制御だけでは、より高画質側への符号化パラメータ修正が行われるが、これは本来無意味である。同時に、高画質側へ符号化パラメータが振られた状態においては、次に動きがあると、符号量が急増し、画質劣化を引き起こす。そのため、Ｄ成分が、符号化パラメータの高画質側への振られすぎを抑止する。
【００５９】
逆に、たとえば、変動量が急に大きくなった場合（これは、シーンが変わったり動きが激しくなったときに相当する）は、Ｐ，Ｉの制御だけでは、より低画質側への符号化パラメータ修正が行われるが、これも適切ではない。また、低画質側へ符号化パラメータが振られた状態においては、次に動きが少なくなったとき、符号量が急減し、画質劣化を引き起こす。そのためＤ成分が、符号化パラメータの低画質側への振られすぎを抑止する。
【００６０】
このように、ＰＩＤ制御は基本的には、上の３つのパラメータ、すなわち、レート誤差Ｅ（ｎ）を引数とする関数Ｆ_E（Ｅ）、生成符号の変動量Ｄ（ｎ）を引数とする関数Ｆ_D（Ｄ）、レート誤差の積算値を引数とする関数Ｆ_S（Ｓ）の和であるが、本発明では、それに加えて、生成符号量そのものも制御パラメータとして用いる。すなわち、生成符号量を引数とする関数をＦ_B（Ｂ）とすれば、符号化パラメータの更新量ΔＰは、
ΔＰ＝Ｆ_B（Ｂ）×｛Ｆ_E（Ｅ）＋Ｆ_D（Ｄ）＋Ｆ_S（Ｓ）｝ （４）
と表すことができる。
【００６１】
この（４）式におけるＦ_B（Ｂ）は、符号化パラメータ更新重み係数（以下では単に更新重み係数という）としての意味を有し、生成符号量Ｂに対して図４に示すように、生成符号量Ｂがある下限値ＴＨ１以下である場合とある上限値ＴＨ２以上である場合は、ともにＦ_B（Ｂ）＝０である。したがって、生成符号量Ｂがある下限値ＴＨ１以下の場合とある上限値ＴＨ２以上の場合は、符号化パラメータΔＰは０となり、符号化パラメータの更新がなされない。
【００６２】
本発明では、この生成符号量Ｂがある下限値ＴＨ１以下である場合は冗長符号領域として作動し、生成符号量Ｂがある上限値ＴＨ２以上である場合はフレーム間引き領域として作動する。また、生成符号量Ｂが下限値ＴＨ１と上限値ＴＨ２との間は、更新重み係数Ｆ_B（Ｂ）の大きさに応じた符号化パラメータ更新処理がなされる。
【００６３】
このように、状況に応じたフレーム間引き処理と符号冗長化処理が行われる。
すなわち、画質を短時間に低画質側へ変化させないと言う処理のために、符号量が急激に多くなる状況（たとえば、動きの少ないシーンから急激に動きの激しいシーンに移り変わる状況）に対しては、その動きの激しい画像に移り変わったときの符号量の大きいフレームについてはフレーム間引き処理で対応する。このとき、符号量の大きいフレームに対応するデータは廃棄され、「その前のフレームを繰り返す」と言うフレーム間引き符号が生成され、動きの激しくなる直前の符号がそのまま用いられる。つまり、動きの大きくなったデータを低いフレームレートで符号化するよりは、それ以前のデータを継続させてトータル符号量の余裕をつくり、符号量の余裕が生じた時点で、動きの大きいデータをフレームレートを落とさずに通常のフレームレートで送るというような制御を行うものである。これは、動きの大きいデータは、より高いフレームレートで送るほうが心理的に高画質に見えるということを利用したものである。
【００６４】
また、画質を短時間に高画質側へ変化させないと言う処理のために符号量が少なくなる状況に対しては、符号冗長化処理で対応する。これは、たとえば、量子化において、量子化係数が１２で、１、２，３と符号化できる係数を、量子化係数が２で、６、１２、１８と符号化するような方法である。これらは、量子化係数と係数の積として、何れも１２、２４，３６を表すが、後者の方が桁数が大きく、より冗長な表現となる。
【００６５】
以下、本発明の具体的な実施形態について説明する。
【００６６】
図５は本発明の実施形態１の構成を説明する図であり、動画データ中より１フレーム分のフレームデータを取得するフレームデータ取得手段１と、フレームデータの符号化を制御するための符号化パラメータを記憶する符号化パラメータ記憶手段２と、この符号化パラメータを用いてフレームデータを符号化するフレームデータ符号化手段３と、符号化データ出力を出力する符号化データ出力手段４と、目標符号量を記憶する目標符号量記憶手段５と、フレームデータ符号化手段３によって生成された符号量（これをここでは生成符号量と呼ぶ）を取得する生成符号量取得手段６と、取得された生成符号量と目標符号量の誤差を算出するレート誤差算出手段７と、そのレート誤差を積算するレート誤差積算手段８と、前フレームデータに対応する符号量を記憶する前フレーム符号量記憶手段９と、この前フレーム符号量と生成符号量取得手段６で取得された現フレームの生成符号量との差を変動量として算出する変動量算出手段１０と、上述のレート誤差とレート誤差積算値と変動量とを入力して目標符号量を更新する目標符号量更新手段１１と、前フレーム符号量とレート誤差とレート誤差積算値と変動量を入力して符号化パラメータを更新する符号化パラメータ更新手段１２とを有している。
【００６７】
このような構成において、ある１フレーム分の符号化処理についてその処理手順を図６のフローチャートを参照しながら説明する。フレームデータ取得手段１は動画データからあるｎ番目のフレームのフレームデータを取得し（ステップＳ１）、その取得したフレームデータをフレームデータ符号化手段３が符号化を行い（ステップＳ２）、その生成符号量を生成符号量取得手段６が取得する（ステップＳ３）。
【００６８】
次に、レート誤差算出手段７が、生成符号量取得手段６で取得された生成符号量と目標符号量記憶手段５に記憶された目標符号量とを用いて、レート誤差Ｅ（ｎ）を上述の（１）式によって算出し、また、レート誤差積算手段８が、（２）式を用いてレート誤差積算値Ｓ（ｎ）を算出し、さらに、変動量算出手段１０が、前フレーム符号量記憶手段９に記憶された前フレーム符号量と現フレーム符号量（生成符号量取得手段６で取得された生成符号量）とから変動量Ｄ（ｎ）を算出する（ステップＳ４）。
【００６９】
そして、これら算出されたレート誤差値、レート誤差積算値、変動量に基づいて、符号化パラメータの更新および目標符号量の更新を行う。
【００７０】
たとえば、符号化パラメータ更新について説明すると、この符号化パラメータ更新は符号化パラメータ更新手段１１がこれらレート誤差値、レート誤差積算値、変動量から得られた符号化パラメータ更新量ΔＰによって符号化パラメータの更新を行う。この符号化パラメータ更新量ΔＰは、前述した（４）式によって求められるものである。なお、（４）式における右辺の｛Ｆ_E（Ｅ）＋Ｆ_D（Ｄ）＋Ｆ_S（Ｓ）｝の項はいわゆるＰＩＤ制御に相当するもので、この｛Ｆ_E（Ｅ）＋Ｆ_D（Ｄ）＋Ｆ_S（Ｓ）｝の項によって得られた値を符号化パラメータの更新量として用いて符号化パラメータの制御を行うことも状況によっては可能となる場合もあるが、本発明では、（４）式に示されるように、｛Ｆ_E（Ｅ）＋Ｆ_D（Ｄ）＋Ｆ_S（Ｓ）｝の項に生成符号量Ｂを引数とする関数Ｆ_B（Ｂ）を乗算することによって、符号化パラメータ更新量ΔＰを算出するようにしている。
【００７１】
このようにして、符号化パラメータ更新がなされ（ステップＳ５）、その更新された符号化パラメータを用いて符号化データを符号化しその符号化データを出力（ステップＳ６）し、さらに、（４）式の計算結果に基づいて目標符号量を更新する（ステップＳ７）。
【００７２】
図７は本発明の実施形態２を説明する構成図であり、図５で示した実施形態１の構成に符号化パラメータ抑止手段１３を追加したもので、状況に応じて符号化パラメータ更新手段での符号化パラメータ更新処理を抑制するような動作を可能とするものである。この図７に示す構成は図５で示した構成に符号化パラメータ抑止手段１３を追加しただけであり、その他は図５と同じであるので、同一構成要素には同一符号を付すことでそれらの説明は省略する。
【００７３】
符号化パラメータ抑止手段１３は、フレームデータ符号化手段３が符号化パラメータ記憶手段２に記憶された符号化パラメータを用いて符号化することによって生成された符号量が所定値を下回ったときに、符号化パラメータ更新手段１２の符号化パラメータ更新処理を抑止する動作を行うものである。これは、生成符号量が所定量（たとえば、図４に予め設定された下限値ＴＨ１）よりも小さい場合、高画質画側に符号化パラメータが振られすぎるのを防止するためである。
【００７４】
この図７の構成に対応する処理手順を示すものが図８のフローチャートである。この図８において、ステップＳ１１からステップＳ１４は図６のステップＳ１からステップＳ４と同じ処理であるので、ここでは、それ以降の処理について説明する。
【００７５】
図８のステップＳ１４では図６と同様に、レート誤差値、レート誤差積算値、変動量が算出されると、生成符号量（生成符号量取得手段６で生成された現フレームの生成符号量）が所定値以下か否かを判定し（ステップＳ１５）、生成符号量が所定値以下でなければ、図６のステップＳ５と同様の処理によって符号化パラメータの更新（ステップＳ１６）を行ったのちに、符号化データを出力し（ステップＳ１７）、目標符号量更新を行う（ステップＳ１８）。
【００７６】
一方、生成符号量が所定値以下であった場合は、符号化パラメータ更新を行わずに、ステップＳ１７の符号化データ出力の処理に入る。
【００７７】
このように、生成符号量が所定値以下となった場合は、符号化パラメータ抑止手段１３が符号化パラメ―タ更新手段１２に対して符号化パラメータ更新処理を抑止するように動作する。それによって、生成符号量が所定値（たとえば、図４に示す下限値ＴＨ１）よりも小さくなったような場合でも、符号化パラメータが高画質側に振られ過ぎるのを防止することができる。
【００７８】
図９は本発明の実施形態３を説明する構成図であり、図７で示した実施形態２の構成に冗長符号生成手段１４とフレーム間引き符号生成手段１５が追加されており、状況に応じて冗長符合生成処理またはフレーム間引き処理を行うようにしたものである。この図９に示す構成は図７で示した構成に冗長符号生成手段１４とフレーム間引き符号生成手段１５を追加しただけであり、その他は図４および図６と同じであるので、同一構成要素には同一符号を付すことでそれらの説明は省略する。
【００７９】
冗長符号生成手段１４は、レート誤差積算値が所定値を下回ったときに、冗長符号を生成するものであり、また、フレーム間引き符号生成手段１５は、レート誤差積算値が所定値を上回った時に、フレーム間引き符号を生成するものである。
【００８０】
この図９の構成に対応する処理手順を示すものが図１０のフローチャートである。この図１０において、ステップＳ２１からステップＳ２６は図８のステップＳ１１からステップＳ１６と同じ処理であるので、それ以降の処理について説明する。
【００８１】
ステップＳ２７において、レート誤差積算値が所定値以下か否かを判定し、レート誤差積算値が所定値（予め設定された下限値であり、これは図４のＴＨ１に相当する）以下であれば、符号化データを冗長化する処理を行う（ステップＳ２８）。また、レート誤差積算値が所定値以下でなければ、レート誤差積算値が所定値（予め設定された上限値であり、これは図４のＴＨ２に相当する）以上であるかを判定し（ステップＳ２９）、レート誤差積算値が所定値以上であれば符号化データをフレームデータ間引き符号に置換する（ステップＳ３０）。
【００８２】
また、レート誤差積算値がある範囲以内（図４において予め設定された下限値ＴＨ１から予め設定された上限値ＴＨ２の間）であれば、符号化データの冗長化や符号化データをフレーム間引き符号に置換するといった処理を行うことなく、通常の符号化データ出力、つまり、ステップＳ２６までの処理において制御された符号化パラメータを用いた符号化データ出力を行い（ステップＳ３１）、さらに、目標符号量更新を行う（ステップＳ３２）。
【００８３】
図１１は上述した実施形態３における符号化処理を説明する図であり、横軸に時間をとり、縦軸に積算符号量をとって、それぞれのフレームごとの符号量（太線で示す）の変化の一例を示すものである。また、細かい破線は目標とすべき平均ビットレート（目標値という）Ａを表し、荒い破線は下限値ＴＨ１と上限値ＴＨ２を表している。この下限値ＴＨ１と上限値ＴＨ２は図４における下限値ＴＨ１と上限値ＴＨ２に対応している。
【００８４】
この図１１の例に示すように、時間軸ｔに沿って、個々のフレームで生成される符号量が目標値Ａよりも大きくなれば、次のフレームの符号量を小さくするように符号化パラメータ制御を行い、逆に、あるフレームにおける符号量が目標値よりも小さくくなれば、次のフレームの符号量を大きくするように符号化パラメータ制御を行うというように、目標値Ａに近づくような符号化パラメータ制御を行う。
【００８５】
そして、たとえば、あるフレーム（図１１においては４番目のフレームＦ４）において符号量が上限値ＴＨ２以上となると、フレーム間引き処理を行って、そのフレームＦ４の符号化データは送らずに、次のフレームＦ５で２フレーム分の符号量を割り当てて送る。また、６番目のフレームＦ６のように符号量が下限値ＴＨ１以下となると、そのフレームＦ６において冗長符号化処理を行い、符号量を増やす処理を行う。
【００８６】
このような本発明の実施形態３の符号化処理と前述した各従来技術（従来技術１〜５）とを対比させるために、図１２から図１４にこれら従来技術による符号化処理を示す。これら図１２から図１４は図１１と同じく、横軸に時間、縦軸に積算符号量をとっている。
【００８７】
図１２は従来技術１（非特許文献１として挙げた技術）、従来技術２（特開平５−４１８６０号公報に記載の技術）、従来技術３（特開平１０−１０８１９７号公報に記載の技術）に対応する符号化処理を示すもので、これら従来技術１、従来技術２，従来技術３は、この図１２からもわかるように、ビットレートの変動をできるだけ少なくすることのみの制御であり、画像品質の時間的変動を抑えるというような概念はない。
【００８８】
また、図１３は従来技術４（特開平１１−２１５４９７号公報に記載の技術）に対応する符号化処理を示すもので、この図１３による処理は１パスでの符号化処理であり、この図１３で示す従来技術４の処理が図１２と大きく異なるのは、符号量が大きく変動した場合に、それを目標値Ａに近づけるような合わせ込みを行う処理に入り、目標値Ａに近づけるための合わせ込みを試行錯誤的に行う点である。
【００８９】
すなわち、この従来技術４の場合、符号化が可能か否かを判断する判断線を符号量の上限値ＴＨ２としての判断線（図示の荒い破線）を設定し、符号量がこの判断線を越えると、符号化パラメータを変更しながら複数回の試行錯誤的な調整を行って目標値に近づくような合わせ込み処理を行う必要がある。
【００９０】
これに対して、本発明では、試行錯誤的な合わせ込みを行うのではなく、符号量が上限値（図１１におけるＴＨ２）以上であれば、フレーム間引きを行い、符号量が下限値（図１１におけるＴＨ１）以下であれば、冗長化処理を行って符号量増やす処理を行うというように、本発明ではどのような場合であっても、常に１回しか符号化を行わないのが特徴であり、それによって、処理の高速化が図れる。
【００９１】
また、図１４は従来技術５（特開２０００−２３１５５号公報に記載の技術）に対応する符号化処理を示すもので、この従来技術５では、たとえば、複雑な画像データのように符号量が多い場合、要求されたフレームレートでは処理が不可能という判断を行い、そのような場合は、１フレーム分の間引きを行うというように予めフレームレートを落とす設定を行う。しかし、それでも、予測を超えた大きな符号量が発生したというような場合には、２フレーム分を間引くというように、状況によってはフレームレートの設定を変更するというようなフレームレート制御を行う。
【００９２】
これに対して、本発明はフレームレートを変更するという概念ではなく、前述したように、たとえば、符号量が上限値以上であれば、そのフレームを落とすという考え方である。
【００９３】
確かに、符号量が増大したときには、フレームを間引くという点だけを見れば、本発明は従来技術５と共通点はあるが、この従来技術５では、画像の動きが激しくなって符号量が急激に増大したような場合、その増大した符号量を送って、その増えすぎた符号量をリカバリするために次のフレームの間引き行うというようなフレームレート制御が行われるが、これでは、本来、動きが激しく、多くのフレームレートが必要である場合であっても、その部分でフレームレートが低くなるため、動きにギクシャク感が現れ、見た目に自然な動くの画像が得られないという問題がある。
【００９４】
これに対して、本発明では、動きが多くなったフレームそのものを間引き、動きの激しくなる直前の動きの少なかったフレームをそのまま継続して用い、送ることのできる符号量に余裕が生じたところで、動きの多い部分は、できるだけその動きの多い部分に対応した符号量を、フレームレートを落とすことなく送るという考え方であり、これによって、動きの激しい部分で、フレームレートの減少によって生じるギクシャク感が現れにくくなる。
【００９５】
すなわち、本発明では、たとえば、動きが少ないシーンから動きが急激に多くなるシーンに変わるときに、符号量の急激な増大に対処するために、まずは、動きが急激に大きくなる直前の画像をそのまま用い、送ることのできる符号量に余裕ができた時点で、動きの多い画像に対応する符号量をフレームレートを少なくすることなく送るようにしている。これによって、動きの小さいシーンから動きの激しいシーンに移るような場合、動きの激しいシーンで見た目に自然な動きの画像を生成することができる。
【００９６】
なお、本発明は上述の各実施形態に限られるものではなく、本発明の要旨を逸脱しない範囲で種々変形実施可能となるものである。また、本発明は以上説明した本発明を実現するための処理手順が記述された処理プログラムを作成し、その処理プログラムをフロッピィディスク、光ディスク、ハードディスクなどの記録媒体に記録させておくこともでき、本発明は、その処理プログラムの記録された記録媒体をも含むものである。また、ネットワークから当該処理プログラムを得るようにしてもよい。
【００９７】
【発明の効果】
以上説明したように、本発明によれば、符号化パラメータが短時間に変化することで画質が短時間に変化することを防ぎ、これによって、見た目に自然な動きの画像を生成することができる。たとえば、動きが多い、あるいは高周波成分が多いなど高ビットレートが要求される部分において、符号化パラメータが短時間に低画質側へ振られ過ぎるのを防ぐことができ、これによって、このような動きが多い、あるいは高周波成分が多いシーンから動きの少ない、あるいは高周波成分の少ないシーンに移った時に、画質の劣化を防ぐことができる。また、逆に、動きが少ない、あるいは高周波成分が少ないなど低ビットレートで十分な部分において、符号化パラメータが、短時間に高画質側へ振られ過ぎるのを防ぐことができ、これによって、このような動きの少ない、あるいは高周波成分が少ないシーンから動きが多いあるいは高周波成分が多いシーンに移ったときに、符号量が急増してレート制御が困難となるのを防ぐことができる。
【図面の簡単な説明】
【図１】 本発明の基本的な処理を説明するための図であり、レート誤差がプラスまたはマイナス方向に発生した場合の符号量制御を説明する図である。
【図２】 本発明の基本的な処理を説明するための図であり、レート誤差積算値がプラスまたはマイナス方向にそれぞれ所定量を超えたときの符号量制御を説明する図である。
【図３】 本発明の基本的な処理を説明するための図であり、生成符号の変動量がプラスまたはマイナス方向にそれぞれ所定量を超えたときの符号量制御を説明する図である。
【図４】 生成符号量と符号パラメータ更新重みの関係を示す図である。
【図５】 本発明の実施形態１を説明する構成図である。
【図６】 実施形態１の処理手順を説明するフローチャートである。
【図７】 本発明の実施形態２を説明する構成図である。
【図８】 実施形態２の処理手順を説明するフローチャートである。
【図９】 本発明の実施形態３を説明する構成図である。
【図１０】 実施形態３の処理手順を説明するフローチャートである。
【図１１】 本発明の実施形態３における符号化処理を説明する図である。
【図１２】 従来技術１，２，３の符号化処理を説明する図である。
【図１３】 従来技術４の符号化処理を説明する図である。
【図１４】 従来技術５の符号化処理を説明する図である。
【図１５】 従来技術１の符号化処理を説明する構成図である。
【図１６】 図１５で示した従来技術１におけるバッファ１０６の使用量に対する符号化パラメータ制御を説明する図である。
【図１７】 図１６で示した動作を横軸にバッファ１０６の使用量、縦軸に符号化パラメータをとって示した図である。
【符号の説明】
１ フレームデータ取得手段
２ 符号化パラメータ記憶手段
３ フレームデータ符号化手段
４ 符号化データ出力手段
５ 目標符号量記憶手段
６ 生成符号量取得手段
７ レート誤差算出手段
８ レート誤差積算手段
９ 前フレーム符号量記憶手段
１０ 変動量算出手段
１１ 目標符号量更新手段
１２ 符号化パラメータ更新手段
１３ 符号化パラメータ更新抑止手段
１４ 冗長符号生成手段
１５ フレーム間引き符号生成手段
ＴＨ１ 下限値
ＴＨ２ 上限値[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a moving image encoding method, a moving image encoding apparatus, and a moving image encoding processing program.
[0002]
[Prior art]
There are roughly two types of video encoding methods. One of them is a method in which encoding parameters are fixed and data is encoded by a single encoding process. This is characterized by a high speed because the encoding process is performed once and a small temporal variation in image quality because the code parameters are fixed.
[0003]
On the other hand, in this method, the amount of data generated by the encoding process completely depends on the input data and may vary greatly. Therefore, there is a problem in application to a transmission system in which the amount of transmission data per unit time is limited and a storage system in which the total data amount is limited. In order to solve this, it is necessary to control the amount of data generated by the encoding process.
[0004]
Another method of video encoding is to solve the above-mentioned problem, and the temporal average value of the amount of data generated by the encoding process, that is, the average bit rate is made constant by some method. Is. This technique is called rate control. This method can also be roughly divided into two types. This rate control will be described below.
[0005]
One of the rate control methods is rate control based on code amount allocation, in which a target value of code amount is set in advance for each frame, and a plurality of parameters are set while changing encoding parameters, that is, image quality to be encoded. In this method, encoding is performed with the encoding parameter that is closest to the set code amount, and the average bit rate is controlled to a value close to a predetermined value.
[0006]
Another method is rate control by adapting the code amount, which does not set the target code amount for each frame, but sets multiple frames or an average bit rate in a certain time range. This is a method of performing some control so as to approach the bit rate. Conventional techniques in this field are described in, for example, IEEE Transaction On Communications, COM-32 (1984-3), P.M. There is a technique described in 225-232, “Scene Adaptive Coder” (this is referred to as Prior Art 1).
[0007]
FIG. 15 shows the configuration of this prior art 1, and information amount compression means 101, statistic calculation means 102, quantization means 103, code amount prediction means 104, encoding means 105, buffer 106, buffer occupancy. The configuration has a quantity detection means 107.
[0008]
In such a configuration, the data encoded by the encoding means 105 is temporarily stored in a buffer 106 that performs a first-in first-out (FIFO) operation, and from this buffer 106 at a target bit rate. Data is read and output.
[0009]
Here, the encoding parameter for controlling the encoding process is given as a function of the usage amount of the buffer 106. That is, as the amount of use of the buffer 106 increases, the encoded data amount decreases in order to prevent the buffer 106 from overflowing, that is, the encoded image quality is lowered (to the lower image quality side). Update parameters are updated. Conversely, when the amount of use of the buffer 106 decreases, the encoding parameter is updated so that the amount of encoded data increases in order to prevent buffer underflow, that is, the encoded image quality is improved.
[0010]
FIG. 16 is a diagram for explaining the encoding parameter control for the usage amount of the buffer 106, and the outline of the encoding parameter control will be described with reference to FIG. In FIG. 16, white portions indicate the free space of the buffer 6, and portions other than the hatched pattern, vertical line pattern, horizontal line pattern, and the like indicate portions where encoded data is accumulated. ing.
[0011]
As shown in FIG. 16A, when the usage amount of the buffer 106 increases, the encoding parameter is updated to the low image quality side, and thereafter, as shown in FIGS. 16B and 16C, the encoded data is further increased. Is stored, the encoding parameter is further updated to the lower image quality side. Note that the encoded data stored in the buffer 106 is sequentially output from the output side as necessary.
[0012]
Then, as shown in FIG. 16 (d), when the amount of use of the buffer 106 decreases, the encoding parameter is updated on the high image quality side, and then, as shown in FIG. Once accumulated, the encoding parameter is updated to the low image quality side.
[0013]
FIG. 17 is a graph showing the operation shown in FIG. 16, with the horizontal axis representing the usage amount of the buffer 106 and the vertical axis representing the encoding parameter. In FIG. 17, the buffer usage amounts (a), (b), (c), and (d) shown on the horizontal axis correspond to (a), (b), (c), and (d) in FIG. . T indicates a threshold value.
[0014]
As described above, the prior art 1 performs the encoding parameter control on the usage amount of the buffer 106 so that the temporal average value of the encoded data becomes constant, which is a simple and effective method. However, there are some problems. As an improvement of these, Japanese Patent Laid-Open No. 5-41860 “Image Compression Encoding Device” (hereinafter referred to as Prior Art 2), Japanese Patent Application Laid-Open No. 10-108197 “Image Encoding Device, Image Encoding Control Method, and Image Encoding Control” Medium recording program for use "(hereinafter referred to as prior art 3), JP-A-11-215497" variable bit rate encoding image data compression system "(hereinafter referred to as prior art 4), JP 2000-23155" moving image " Encoding device "(hereinafter referred to as prior art 5).
[0015]
Prior art 2 prevents the image quality from changing at the beginning and end of the same image when the buffer is relatively small. That is, when encoding certain frame data, the amount of buffer usage increases as the encoding progresses, and the encoding parameter changes in the direction of lowering the image quality. As a result, the image quality at the end tends to be lower than that at the beginning of the frame data.
[0016]
Therefore, in this prior art 2, first, a certain statistic about the entire image is examined, thereby determining the complexity of the image and adjusting the encoding parameters.
As a result, the encoded image quality is prevented from changing greatly in the same image, and an encoding process in which the bit rate is controlled is realized.
[0017]
Further, in the conventional technique 3, at the same bit rate, an input image with a large amount of high-frequency components or an intense movement is more likely to cause coding distortion than an image with a small amount of them. This improves the conspicuousness of blocks that are processing units.
[0018]
For this reason, in this conventional example, the temporal pre-filter and spatial pre-filter are controlled in accordance with the degree of difficulty in encoding the input image, such as the amount of motion and the number of high-frequency components, to limit the high-frequency and motion components. Is what you do. As a result, the input data is converted into data that can be easily encoded, and the amount of encoded data can be reduced. As a result, encoding distortion is reduced and rate control is performed.
[0019]
The prior art 4 improves the fact that a very large amount of calculation is required for the determination processing such as the complexity of the image and the difficulty of encoding. In this method, the complexity and the encoding difficulty of the input data are estimated using the code amount obtained from the previously encoded data, and the encoding parameters are determined based on the estimation. Also, when the amount of encoded data generated using the encoding parameter is larger than expected, the encoding parameter is adjusted by performing multiple trials and errors while changing the encoding parameter. It is. Thereby, rate control is performed.
[0020]
Further, the prior art 5 determines whether or not the bit rate, the image size, and the frame rate specified first are appropriate. At this time, image data such as CIF, QCIF, and SQCIF, and average correlation data such as a frame rate and a bit rate are previously stored, thereby correcting inappropriate encoding parameters. In other words, the parameters that do not cause a major failure are corrected in advance, and then it is determined whether or not encoding is possible with the set encoding parameters for each frame data of the newly set frame rate. If not possible, the frame data is skipped if not possible, and at the same time the encoding parameters including the frame rate are modified. As described above, rate control is performed by controlling the encoding parameters including the change of the frame rate.
[0021]
[Patent Document 1]
Japanese Patent Laid-Open No. 5-41860
[Patent Document 2]
JP-A-10-108197
[Patent Document 3]
JP 11-215497 A
[Patent Document 4]
JP 2000-23155 A
[Non-Patent Document 1]
IEEE Transaction On Communications, COM-32 (1984-3), p. 225-232, "Scene Adaptive Coder"
[0022]
[Problems to be solved by the invention]
However, since the process of the above-described conventional technique 1 (the technique described in Non-Patent Document 1) simply changes the encoding parameter corresponding to the remaining amount of the output buffer, the image quality changes in a short time. This may lead to deterioration of the image quality as a whole.
[0023]
Further, in the conventional technique 2 (the technique described in Patent Document 1), there is a problem that the amount of calculation is large because a statistic relating to the entire image must be obtained. In addition, since processing cannot be performed unless the entire image is once stored, there is a problem that a large amount of memory is required and a delay occurs. In addition, in a parameter that reduces the code amount, a large coding distortion may occur. Furthermore, there is a possibility that the encoding parameter changes greatly in a short time and the image quality changes accordingly.
[0024]
Further, in the prior art 3 (the technique described in Patent Document 2), it is possible to reduce the coding distortion by limiting the high frequency component and the motion component. However, this also means that the original image quality is degraded. In addition, pre-processing for the entire image is required, and a large amount of memory and processing are required, causing a delay. In addition, there is a possibility that the encoding parameter changes greatly in a short time and the image quality changes in a short time.
[0025]
Further, in the prior art 4 (the technique described in Patent Document 3), a trial and error is eventually performed a plurality of times in a frame where the prediction from before is not suitable, such as a part where the scene changes greatly or a part where there is a lot of movement. Therefore, the processing amount is not reduced. Even if the code amount can be reduced, this greatly deteriorates the image quality of the frame. On the other hand, there is a problem that no countermeasure is taken into account when the motion is very small, that is, when the amount of generated code is very small.
[0026]
In the prior art 5 (the technique described in Patent Document 4), the correlation data regarding the image size, the frame rate, and the bit rate possessed in advance is only average, for example, at a certain image size. However, it cannot deal with individual situations such as data with a lot of movement and data with a very small movement. In the end, a parameter called frame rate variation is added, but it is substantially the same as the method of the IEEE literature. That is, there is a possibility that the encoding parameter changes greatly in a short time and the image quality changes.
[0027]
Therefore, the present invention prevents the image quality from changing in a short time by changing the coding parameter in a short time, so that, for example, when moving from a scene with small motion to a scene with heavy motion, It is an object of the present invention to provide a moving image encoding method, a moving image encoding apparatus, and a moving image encoding processing program that can generate an image with a simple motion.
[0028]
[Means for Solving the Problems]
In order to achieve the above object, a moving image encoding method of the present invention includes a frame data acquisition step for acquiring frame data for one frame from moving image data, and an encoding parameter for controlling encoding of the frame data. Encoding parameter storing step for storing, frame data encoding step for encoding frame data using the encoding parameter, and encoded data output step for outputting encoded encoded data In the method, a target code amount storage step for storing a target code amount, a generated code amount acquisition step for acquiring a code amount generated by the frame data encoding step, and a code amount acquired by the generated code amount acquisition step And the target code amount as a rate error. A rate error integration step of integrating the rate error calculated in the rate error calculation step to calculate a rate error integrated value, a code amount acquired in the generated code amount acquisition step, and a code amount of the previous frame data A variation amount calculating step for calculating a difference between the variation amount, a rate error and a rate error integrated value as inputs, a target code amount update step for updating the target code amount, and the variation amount An encoding parameter update step for inputting the rate error and the rate error integrated value to update the encoding parameter, and using the encoding parameter stored in the encoding parameter storage step, The frame data encoding step is characterized by encoding by one encoding process.
[0029]
In such a moving image encoding method, the encoding parameter update step includes a function value using the rate error as an argument, a function value using the rate error integrated value as an argument, and a function value using the variation amount as an argument. A value obtained by multiplying the sum of the three by the function value having the code amount acquired in the generated code amount acquisition step as an argument is used as an encoding parameter update amount, and based on this encoding parameter update amount It is preferable to update the encoding parameters.
[0030]
In this moving image encoding method, the frame data encoding step encodes the frame data acquired in the frame data acquisition step using the encoding parameter stored in the encoding parameter storage step. It is preferable to provide a coding parameter update inhibiting step that inhibits the coding parameter updating process of the coding parameter updating step when the generated code amount falls below a predetermined value.
[0031]
In this moving image encoding method, the frame data encoding step encodes the frame data acquired in the frame data acquisition step using the encoding parameter stored in the encoding parameter storage step. It is preferable to provide a frame decimation code generation step for generating a frame decimation code when the rate error integrated value calculated based on the generated code amount exceeds a predetermined value.
[0032]
In this moving image encoding method, the frame data encoding step encodes the frame data acquired in the frame data acquisition step using the encoding parameter stored in the encoding parameter storage step. It is preferable to provide a redundant code generation step for generating a redundant code when the rate error integrated value calculated based on the generated code amount falls below a predetermined value.
[0033]
In addition, the moving image encoding apparatus of the present invention includes a frame data acquisition unit that acquires frame data for one frame from the moving image data, and an encoding parameter storage unit that stores an encoding parameter for controlling the encoding of the frame data. And a frame data encoding unit that encodes frame data using the encoding parameter, and an encoded data output unit that outputs the encoded data. A target code amount storage unit for storing; a generated code amount acquisition unit for acquiring a code amount generated by the frame data encoding unit; a code amount acquired by the generated code amount acquisition unit; and the target code amount Rate error calculation means for calculating the error as a rate error value, and the rate error calculated by the rate error calculation means is integrated. Rate error integration means for calculating the rate error integrated value, previous frame code amount storage means for storing the code amount of the previous frame data, acquisition of the code amount stored in the previous frame code amount storage means and the generated code amount A fluctuation amount calculation means for calculating a difference from the code amount acquired by the means as a fluctuation amount; and a target code amount update for updating the target code amount by inputting the fluctuation amount, a rate error, and a rate error integrated value Encoding parameter update means for inputting the fluctuation amount, rate error, and rate error integrated value to update the encoding parameter, and the encoding parameter stored in the encoding parameter storage means The frame data is encoded by a single encoding process by the frame data encoding means.
[0034]
In such a moving image encoding apparatus, the encoding parameter update means includes a function value having the rate error as an argument, a function value having the rate error integrated value as an argument, and a function value having the variation amount as an argument. A value obtained by multiplying the sum of the three by the function value having the code amount acquired by the generated code amount acquisition means as an argument is used as an encoding parameter update amount, and based on this encoding parameter update amount It is preferable to update the encoding parameters.
[0035]
In the moving image encoding apparatus, the frame data encoding unit generates the frame data acquired by the frame data acquisition unit using the encoding parameter stored in the encoding parameter storage unit. It is preferable to provide an encoding parameter update inhibiting unit that inhibits the encoding parameter updating process of the encoding parameter updating unit when the code amount falls below a predetermined value.
[0036]
In the moving image encoding apparatus, the frame data encoding unit generates the frame data acquired by the frame data acquisition unit using the encoding parameter stored in the encoding parameter storage unit. It is preferable to provide a frame decimation code generating means for generating a frame decimation code when the rate error integrated value calculated based on the code amount exceeds a predetermined value.
[0037]
In the moving image encoding apparatus, the frame data encoding unit generates the frame data acquired by the frame data acquisition unit using the encoding parameter stored in the encoding parameter storage unit. It is preferable to provide redundant code generating means for generating a redundant code when the rate error integrated value calculated based on the generated code amount falls below a predetermined value.
[0038]
The moving image encoding program of the present invention includes a frame data acquisition step for acquiring frame data for one frame from the moving image data, and an encoding parameter storage step for storing an encoding parameter for controlling the encoding of the frame data. A moving picture code in a moving picture coding processing program comprising: a frame data coding step for coding frame data using the coding parameter; and a coded data output step for outputting the coded data. The conversion processing program is acquired by the target code amount storage step for storing the target code amount, the generated code amount acquisition step for acquiring the code amount generated by the frame data encoding step, and the generated code amount acquisition step. The error between the code amount and the target code amount is the rate error value. A rate error calculation step to be calculated; a rate error integration step in which the rate errors calculated in the rate error calculation step are integrated to calculate a rate error integrated value; and the code amount acquired in the generated code amount acquisition step A fluctuation amount calculating step for calculating a difference from a code amount of frame data as a fluctuation amount; a target code amount updating step for updating the target code amount by inputting the fluctuation amount, a rate error, and a rate error integrated value; An encoding parameter update step of inputting the fluctuation amount, rate error, and rate error integrated value to update the encoding parameter, and using the encoding parameter stored in the encoding parameter storage step The frame data is encoded by a single encoding process by the frame data encoding step. To have.
[0039]
In such a moving image encoding program, the encoding parameter update step includes a function value using the rate error as an argument, a function value using the rate error integrated value as an argument, and a function value using the variation amount as an argument. A value obtained by multiplying the sum of the three by the function value having the code amount acquired in the generated code amount acquisition step as an argument is used as an encoding parameter update amount, and based on this encoding parameter update amount It is preferable to update the encoding parameter.
[0040]
In the moving image encoding program, the frame data encoding step uses the encoding parameter stored in the encoding parameter storage step to encode the frame data acquired in the frame data acquisition step. It is preferable to provide a coding parameter update inhibiting step that inhibits the coding parameter updating process of the coding parameter updating step when the generated code amount falls below a predetermined value.
[0041]
In the moving image encoding program, the frame data encoding step uses the encoding parameter stored in the encoding parameter storage step to encode the frame data acquired in the frame data acquisition step. It is preferable to provide a frame decimation code generation step for generating a frame decimation code when the rate error integrated value calculated based on the generated code amount exceeds a predetermined value.
[0042]
In the moving image encoding program, the frame data encoding step uses the encoding parameter stored in the encoding parameter storage step to encode the frame data acquired in the frame data acquisition step. It is preferable to provide a redundant code generation step for generating a redundant code when the rate error integrated value calculated based on the generated code amount falls below a predetermined value.
[0043]
Thus, in the present invention, a PID control method is used for encoding parameter control, but it is not PID control itself. That is, basically, a function F having a rate error E (n) as an argument. _E (E), a function F having a generated code variation D (n) as an argument _D (D) A function F with an integrated value of rate error as an argument _S Although encoding parameter control is performed based on the sum of (S)), the generated code amount itself is also used as a control parameter.
[0044]
In addition, a process for suppressing the encoding parameter update process is added when the code amount falls below a predetermined value in a scene with a small motion or a small amount of high frequency components. Also, a code redundancy process is added. This prevents the coding parameter from being overwhelmed in the high-quality side in a short time, and when the scene moves from a small motion or low frequency component to a scene with a lot of motion or high frequency component. Therefore, it is possible to prevent the rate control from becoming difficult due to a rapid increase in the code amount. Further, underflow of data can be prevented by the code redundancy or processing.
[0045]
In addition to this, frame thinning is performed when the amount of codes is too large.
[0046]
In the frame thinning performed by the present invention, for example, a frame when moving to a scene with a lot of motion or a high frequency component is thinned out, and a frame with a small amount of motion immediately before the amount of motion is continuously used. This is the idea of sending a code amount corresponding to a portion where the motion is intense as much as possible without reducing the frame rate, thereby preventing a decrease in the frame rate in the portion where the motion is intense.
[0047]
That is, in the present invention, for example, when changing from a scene with little movement to a scene with a lot of movement, in order to cope with a sudden increase in the amount of code, first, the sign immediately before the movement suddenly increases is continued. A small frame decimation code is sent to indicate that it will be used. As a result, when there is a margin in the amount of code that can be sent, a code corresponding to an image with a lot of motion is sent without reducing the frame rate. This corresponds to the fact that for data with a large amount of motion, it is more psychological to send data at a normal frame rate after thinning the frame than to continue sending data at a low frame rate. Thus, when moving from a scene with small motion to a scene with high motion, it is possible to generate a more natural motion image in the scene with high motion.
[0048]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below. The contents described in this embodiment are descriptions of the moving picture coding method and the moving picture coding apparatus of the present invention, and also include specific processing contents of the moving picture coding processing program of the present invention.
[0049]
First, the basic concept of the present invention will be described. The present invention uses a so-called PID control method for moving picture coding parameter control for rate control. However, it is not a simple PID control that is generally performed conventionally.
[0050]
PID control itself is effective when the condition variation is not so large, but in moving picture coding, the condition is likely to change greatly. For this reason, if the PID control is simply executed, the bit rate may break down. In order to cope with this, frame rate thinning processing is added when the amount of code is too large, and code redundancy processing is added when the amount of code is too small, and optimal rate control is realized by operating them in cooperation.
[0051]
Here, P in the PID control means proportionality, the parameter is changed corresponding to a simple difference from the target bit rate, I means integration, and the parameter is changed with respect to the integral value of the above difference, D means differentiation, and the parameter is changed with respect to the differential value of the difference.
[0052]
That is, as a basic part, if an error occurs in the plus or minus direction, P is controlled to pull it back. This is shown in FIG. 1. When the rate error E increases in the positive direction, the code amount is corrected to decrease. When the rate error E increases in the negative direction, the code amount is corrected to increase. The Note that the rate error E (n) in a certain nth frame is B (n) as the generated code amount and TB (n) as the target code amount.
E (n) = B (n) -TB (n) (1)
It is represented by
[0053]
I controls the integrated value of the rate error. In other words, even if the rate error itself is small, if it continues for a long time, the error between the code amount and the target value is integrated as a total, so that the control amount of the code amount is increased. This is similar to the concept of the buffer that performs the FIFO operation of the IEEE document of Prior Art 1. FIG. 2 shows the code amount control for the rate error integrated value. When the rate error integrated value S becomes larger than a predetermined amount in the positive direction, the code amount is corrected in a direction to reduce the code amount. When S is larger than a predetermined amount in the negative direction, the code amount is corrected to increase. The rate error integrated value S (n) in the nth frame is
[0054]
[Expression 1]

[0055]
It is represented by
[0056]
Further, D is a code amount control amount that is increased in response to a sudden large fluctuation in error. This is shown in FIG. 3, and when the fluctuation amount D is larger than the predetermined amount in the positive direction, the code amount is corrected to decrease, and when the fluctuation amount D is larger than the predetermined amount in the negative direction, It is corrected in the direction of increasing the code amount. Note that the variation D (n) of the generated code amount in the nth frame is
D (n) = B (n) -B (n-1) (3)
Can be expressed as
[0057]
This is intended to correct a rapid change when a sudden change occurs. However, if this sign is reversed, it can also be used to suppress an excessive reaction to the sudden change. .
[0058]
For example, when the fluctuation amount D suddenly decreases (this corresponds to when the movement stops), the encoding parameter correction to the higher image quality side is performed only by the control of P and I. This is essentially meaningless. At the same time, in the state in which the encoding parameter is set to the high image quality side, the next movement causes the code amount to increase rapidly, causing image quality degradation. For this reason, the D component prevents the encoding parameter from being excessively shaken toward the high image quality side.
[0059]
On the other hand, for example, when the amount of fluctuation suddenly increases (this corresponds to a scene change or movement), encoding to the lower image quality side is possible only by controlling P and I. Parameter corrections are made, but this is also not appropriate. In addition, in a state where the encoding parameter is set to the low image quality side, when the motion is next reduced, the code amount is drastically reduced, causing image quality deterioration. For this reason, the D component prevents the encoding parameter from being excessively shifted to the low image quality side.
[0060]
As described above, the PID control basically has the above three parameters, that is, the function F having the rate error E (n) as an argument. _E (E), a function F having a generated code variation D (n) as an argument _D (D) A function F with an integrated value of rate error as an argument _S In the present invention, the generated code amount itself is also used as a control parameter. That is, a function with the generated code amount as an argument is F _B Assuming (B), the update amount ΔP of the encoding parameter is
ΔP = F _B (B) x {F _E (E) + F _D (D) + F _S (S)} (4)
It can be expressed as.
[0061]
F in this equation (4) _B (B) has a meaning as an encoding parameter update weight coefficient (hereinafter simply referred to as an update weight coefficient), and the generated code quantity B has a lower limit value TH1 as shown in FIG. If the value is less than or equal to the upper limit value TH2, both F _B (B) = 0. Therefore, when the generated code amount B is less than a certain lower limit value TH1 or more than a certain upper limit value TH2, the coding parameter ΔP is 0, and the coding parameter is not updated.
[0062]
In the present invention, when the generated code amount B is less than a certain lower limit value TH1, it operates as a redundant code region, and when the generated code amount B is greater than a certain upper limit value TH2, it operates as a frame thinning region. Further, when the generated code amount B is between the lower limit value TH1 and the upper limit value TH2, the update weight coefficient F _B Encoding parameter update processing corresponding to the size of (B) is performed.
[0063]
In this way, frame thinning processing and code redundancy processing according to the situation are performed.
In other words, because of the process of not changing the image quality to the low image quality side in a short time, for situations where the amount of code suddenly increases (for example, a situation where there is a sudden transition from a scene with little motion to a scene with rapid motion) A frame with a large code amount when the image is changed to a picture with intense motion is dealt with by frame thinning processing. At this time, data corresponding to a frame having a large code amount is discarded, a frame thinning code “repeating the previous frame” is generated, and the code immediately before the motion becomes intense is used as it is. In other words, rather than encoding data with increased motion at a low frame rate, the previous data is continued to create a margin for the total code amount. Control is performed such that transmission is performed at a normal frame rate without reducing the frame rate. This takes advantage of the fact that data with a large amount of motion appears psychologically higher when sent at a higher frame rate.
[0064]
In addition, the code redundancy processing is used to deal with a situation where the amount of code decreases due to the process of not changing the image quality to the high image quality side in a short time. This is, for example, a method in which a quantization coefficient of 12 and a coefficient that can be encoded as 1, 2, 3 are encoded as 6, 12, 18 with a quantization coefficient of 2 in quantization. These represent 12, 24, and 36 as products of quantized coefficients and coefficients, but the latter has a larger number of digits and is more redundant.
[0065]
Hereinafter, specific embodiments of the present invention will be described.
[0066]
FIG. 5 is a diagram for explaining the configuration of the first embodiment of the present invention. Frame data acquisition means 1 for acquiring frame data for one frame from moving image data, and encoding for controlling the encoding of the frame data Encoding parameter storage means 2 for storing parameters, frame data encoding means 3 for encoding frame data using the encoding parameters, encoded data output means 4 for outputting encoded data output, target code A target code amount storage unit 5 for storing the amount, a generated code amount acquisition unit 6 for acquiring a code amount generated by the frame data encoding unit 3 (this is referred to as a generated code amount here), and an acquired generation Corresponding to the rate error calculating means 7 for calculating the error between the code amount and the target code amount, the rate error integrating means 8 for integrating the rate error, and the previous frame data Previous frame code amount storage means 9 for storing a code amount, and a fluctuation amount calculation means for calculating a difference between the previous frame code amount and the generated code amount of the current frame acquired by the generated code amount acquisition means 6 as a fluctuation amount 10, the target code amount updating means 11 for updating the target code amount by inputting the rate error, the rate error integrated value, and the variation amount, the previous frame code amount, the rate error, the rate error integrated value, and the variation amount. Coding parameter updating means 12 for inputting and updating the coding parameters.
[0067]
In such a configuration, an encoding process for one frame will be described with reference to the flowchart of FIG. The frame data acquisition unit 1 acquires frame data of an nth frame from the moving image data (step S1), the frame data encoding unit 3 encodes the acquired frame data (step S2), and the generated code The generated code amount acquisition means 6 acquires the amount (step S3).
[0068]
Next, the rate error calculation unit 7 uses the generated code amount acquired by the generated code amount acquisition unit 6 and the target code amount stored in the target code amount storage unit 5 to calculate the rate error E (n) as described above. The rate error integration means 8 calculates the rate error integration value S (n) using the expression (2), and the fluctuation amount calculation means 10 further calculates the previous frame code amount. The fluctuation amount D (n) is calculated from the previous frame code amount and the current frame code amount (generated code amount acquired by the generated code amount acquiring means 6) stored in the storage means 9 (step S4).
[0069]
Based on the calculated rate error value, rate error integrated value, and fluctuation amount, the encoding parameter is updated and the target code amount is updated.
[0070]
For example, the encoding parameter update will be described. In this encoding parameter update, the encoding parameter update means 11 uses the encoding parameter update amount ΔP obtained from the rate error value, the rate error integrated value, and the fluctuation amount to change the encoding parameter. Update. This encoding parameter update amount ΔP is obtained by the above-described equation (4). In addition, {F on the right side in the equation (4) _E (E) + F _D (D) + F _S The term (S)} corresponds to so-called PID control, and this {F _E (E) + F _D (D) + F _S Although it may be possible to control the encoding parameter using the value obtained by the item (S)} as the update amount of the encoding parameter depending on the situation, in the present invention, the expression (4) As shown, {F _E (E) + F _D (D) + F _S The function F with the generated code amount B as an argument in the term (S)} _B By multiplying (B), the encoding parameter update amount ΔP is calculated.
[0071]
In this way, the encoding parameter is updated (step S5), the encoded data is encoded using the updated encoding parameter, and the encoded data is output (step S6). of The target code amount is updated based on the calculation result (step S7).
[0072]
FIG. 7 is a block diagram for explaining the second embodiment of the present invention. The coding parameter inhibiting means 13 is added to the structure of the first embodiment shown in FIG. This enables an operation that suppresses the encoding parameter update process. The configuration shown in FIG. 7 is the same as that shown in FIG. 5 except that the encoding parameter suppression means 13 is added, and the others are the same as those in FIG. Description is omitted.
[0073]
When the code amount generated by the frame data encoding unit 3 using the encoding parameter stored in the encoding parameter storage unit 2 falls below a predetermined value, the encoding parameter suppression unit 13 The operation of suppressing the encoding parameter update processing of the encoding parameter update means 12 is performed. This is to prevent the encoding parameter from being excessively applied to the high-quality image side when the generated code amount is smaller than a predetermined amount (for example, the lower limit value TH1 preset in FIG. 4).
[0074]
FIG. 8 is a flowchart showing a processing procedure corresponding to the configuration of FIG. In FIG. 8, step S11 to step S14 are the same processing as step S1 to step S4 of FIG.
[0075]
In step S14 of FIG. 8, when the rate error value, the rate error integrated value, and the fluctuation amount are calculated, the generated code amount (the generated code amount of the current frame generated by the generated code amount acquisition unit 6) is calculated as in FIG. Is not more than a predetermined value (step S15). If the generated code amount is not less than the predetermined value, the encoding parameter is updated (step S16) by the same process as step S5 in FIG. The encoded data is output (step S17), and the target code amount is updated (step S18).
[0076]
On the other hand, if the generated code amount is equal to or smaller than the predetermined value, the encoded parameter output process of step S17 is started without updating the encoding parameter.
[0077]
As described above, when the generated code amount is equal to or less than the predetermined value, the encoding parameter suppression unit 13 operates so as to suppress the encoding parameter update process with respect to the encoding parameter update unit 12. Thereby, even when the generated code amount becomes smaller than a predetermined value (for example, the lower limit value TH1 shown in FIG. 4), it is possible to prevent the encoding parameter from being excessively shifted to the high image quality side.
[0078]
FIG. 9 is a block diagram for explaining Embodiment 3 of the present invention. Redundant code generation means 14 and frame decimation code generation means 15 are added to the structure of Embodiment 2 shown in FIG. A redundant code generation process or a frame decimation process is performed. The configuration shown in FIG. 9 is the same as the configuration shown in FIG. 7 except that redundant code generation means 14 and frame decimation code generation means 15 are added. Are denoted by the same reference numerals, and description thereof is omitted.
[0079]
The redundant code generating means 14 generates a redundant code when the rate error integrated value falls below a predetermined value, and the frame decimation code generating means 15 when the rate error integrated value exceeds a predetermined value. The frame decimation code is generated.
[0080]
The flowchart of FIG. 10 shows a processing procedure corresponding to the configuration of FIG. In FIG. 10, steps S21 to S26 are the same as steps S11 to S16 in FIG. 8, and therefore the subsequent processing will be described.
[0081]
In step S27, it is determined whether the rate error integrated value is equal to or less than a predetermined value. If the rate error integrated value is equal to or less than a predetermined value (a preset lower limit value, which corresponds to TH1 in FIG. 4). Then, a process of making the encoded data redundant is performed (step S28). If the rate error integrated value is not equal to or less than the predetermined value, it is determined whether the rate error integrated value is equal to or greater than a predetermined value (a preset upper limit value, which corresponds to TH2 in FIG. 4) (step S29) If the rate error integrated value is equal to or greater than a predetermined value, the encoded data is replaced with a frame data thinning code (step S30).
[0082]
Further, if the rate error integrated value is within a certain range (between the preset lower limit value TH1 and the preset upper limit value TH2 in FIG. 4), the encoded data is made redundant or the encoded data is converted into a frame decimation code. Without performing the process of substituting for, the normal encoded data output, that is, the encoded data output using the encoding parameters controlled in the process up to step S26 (step S31), and the target code amount Update is performed (step S32).
[0083]
FIG. 11 is a diagram for explaining the encoding process according to the third embodiment described above. The time is plotted on the horizontal axis and the integrated code amount is plotted on the vertical axis, and the change in the code amount (indicated by a bold line) for each frame is shown. An example is shown. A fine broken line represents an average bit rate (referred to as a target value) A to be targeted, and a rough broken line represents a lower limit value TH1 and an upper limit value TH2. The lower limit value TH1 and the upper limit value TH2 correspond to the lower limit value TH1 and the upper limit value TH2 in FIG.
[0084]
As shown in the example of FIG. 11, when the code amount generated in each frame is larger than the target value A along the time axis t, the encoding parameter is set so as to reduce the code amount of the next frame. On the contrary, when the code amount in a certain frame becomes smaller than the target value, the encoding parameter control is performed so as to increase the code amount of the next frame. Encoding parameter control is performed.
[0085]
Then, for example, when the code amount exceeds the upper limit value TH2 in a certain frame (fourth frame F4 in FIG. 11), the frame decimation process is performed, and the encoded data of the frame F4 is not sent, and the next frame In F5, the code amount for 2 frames is allocated and sent. When the code amount becomes equal to or lower than the lower limit value TH1 as in the sixth frame F6, the redundant encoding process is performed in the frame F6, and the code amount is increased.
[0086]
In order to compare the encoding process of the third embodiment of the present invention with the above-described conventional techniques (conventional techniques 1 to 5), FIGS. 12 to 14 show the encoding processes according to these conventional techniques. 12 to 14, like FIG. 11, the horizontal axis represents time, and the vertical axis represents the accumulated code amount.
[0087]
12 shows prior art 1 (technique cited as non-patent document 1), prior art 2 (technique described in JP-A-5-41860), and prior art 3 (technique described in JP-A-10-108197). The prior art 1, the prior art 2 and the prior art 3 are control only for reducing the fluctuation of the bit rate as much as possible, as can be seen from FIG. There is no concept of reducing quality fluctuations over time.
[0088]
FIG. 13 shows an encoding process corresponding to prior art 4 (the technique described in Japanese Patent Laid-Open No. 11-215497). The process shown in FIG. 13 is an encoding process in one pass. The processing of prior art 4 shown in FIG. 13 is significantly different from that in FIG. 12 when the code amount greatly fluctuates, the processing is performed to perform adjustment to bring it closer to the target value A. This is the point of fitting by trial and error.
[0089]
That is, in the case of this prior art 4, a determination line (rough broken line in the figure) is set as a determination amount upper limit value TH2 as a determination line for determining whether or not encoding is possible, and the amount of code exceeds this determination line. In addition, it is necessary to perform a trial and error adjustment a plurality of times while changing the encoding parameter to perform a fitting process that approaches the target value.
[0090]
In contrast, in the present invention, trial and error matching is not performed, but if the code amount is equal to or greater than the upper limit (TH2 in FIG. 11), frame thinning is performed, and the code amount is lower than the lower limit (FIG. 11). In the present invention, the encoding is always performed only once in any case, such as the processing for increasing the code amount by performing the redundancy processing. As a result, the processing speed can be increased.
[0091]
FIG. 14 shows an encoding process corresponding to the prior art 5 (the technique described in Japanese Patent Laid-Open No. 2000-23155). In this prior art 5, for example, the code amount is as in complex image data. In many cases, it is determined that the processing cannot be performed at the requested frame rate. In such a case, a setting for decreasing the frame rate is performed in advance such that one frame is thinned out. However, if a large code amount exceeding the prediction still occurs, frame rate control is performed such that the frame rate setting is changed depending on the situation, such as thinning out two frames.
[0092]
In contrast, the present invention is not based on the concept of changing the frame rate, but as described above, for example, if the code amount is equal to or greater than the upper limit value, the idea is to drop the frame.
[0093]
Certainly, when the code amount is increased, the present invention has similarities with the prior art 5 only by looking at the point that the frame is thinned out. However, in this prior art 5, the motion of the image becomes intense and the code amount is abrupt. In such a case, the frame rate control is performed such that the increased code amount is sent and the next frame is thinned out in order to recover the excessive code amount. However, even when a large number of frame rates are required, there is a problem in that the frame rate becomes low at that portion, so that a sense of jerkyness appears in the movement, and an image that moves visually cannot be obtained.
[0094]
On the other hand, in the present invention, the frame itself with increased movement is thinned out, and the frame with less movement immediately before the movement becomes intense is used as it is, where there is a margin in the amount of code that can be sent, The idea is to send the amount of code that corresponds to the part with the most movement for the part with much movement without reducing the frame rate. As a result, the part with a lot of movement shows the jerky feeling caused by the decrease in the frame rate. It becomes difficult.
[0095]
That is, in the present invention, for example, in order to cope with a sudden increase in the code amount when changing from a scene with little movement to a scene with a lot of movement, first, the image immediately before the movement is suddenly increased is used as it is. When there is a margin in the amount of code that can be used and sent, the amount of code corresponding to an image with much motion is sent without reducing the frame rate. As a result, when moving from a scene with small motion to a scene with intense motion, it is possible to generate an image that looks natural in the scene with intense motion.
[0096]
The present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present invention. Further, the present invention can create a processing program in which processing procedures for realizing the present invention described above are described, and the processing program can be recorded on a recording medium such as a floppy disk, an optical disk, or a hard disk, The present invention also includes a recording medium on which the processing program is recorded. Further, the processing program may be obtained from a network.
[0097]
【The invention's effect】
As described above, according to the present invention, it is possible to prevent the image quality from changing in a short time due to a change in the encoding parameter in a short time, thereby generating an image with a natural motion to the eye. . For example, in a portion where a high bit rate is required, such as a lot of movement or a lot of high frequency components, it is possible to prevent the encoding parameter from being excessively moved to the low image quality side in a short time. Deterioration of image quality can be prevented when moving from a scene with many high frequencies or high frequency components to a scene with little movement or few high frequency components. On the other hand, it is possible to prevent the encoding parameter from being excessively shifted to the high image quality in a short time in a portion where a low bit rate is sufficient, such as a small amount of motion or a high frequency component. It is possible to prevent the rate control from becoming difficult due to a sudden increase in the amount of codes when moving from a scene with a small amount of motion or a high frequency component to a scene with a large amount of motion or a high frequency component.
[Brief description of the drawings]
FIG. 1 is a diagram for explaining basic processing of the present invention, and is a diagram for explaining code amount control when a rate error occurs in a positive or negative direction.
FIG. 2 is a diagram for explaining basic processing of the present invention, and is a diagram for explaining code amount control when a rate error integrated value exceeds a predetermined amount in the plus or minus direction, respectively.
FIG. 3 is a diagram for explaining basic processing of the present invention, and is a diagram for explaining code amount control when a variation amount of a generated code exceeds a predetermined amount in the plus or minus direction, respectively.
FIG. 4 is a diagram illustrating a relationship between a generated code amount and a code parameter update weight.
FIG. 5 is a configuration diagram illustrating Embodiment 1 of the present invention.
FIG. 6 is a flowchart illustrating a processing procedure according to the first embodiment.
FIG. 7 is a configuration diagram illustrating Embodiment 2 of the present invention.
FIG. 8 is a flowchart illustrating a processing procedure according to the second embodiment.
FIG. 9 is a configuration diagram illustrating Embodiment 3 of the present invention.
FIG. 10 is a flowchart illustrating a processing procedure according to the third embodiment.
FIG. 11 is a diagram illustrating an encoding process according to Embodiment 3 of the present invention.
FIG. 12 is a diagram for explaining encoding processing of prior arts 1, 2, and 3.
[Fig. 13] Fig. 13 is a diagram for describing an encoding process of Conventional Technology 4.
FIG. 14 is a diagram for explaining an encoding process according to a conventional technique 5;
FIG. 15 is a configuration diagram illustrating an encoding process according to the conventional technique 1;
FIG. 16 is a diagram for explaining encoding parameter control with respect to the usage amount of the buffer 106 in the prior art 1 shown in FIG. 15;
FIG. 17 is a diagram showing the operation shown in FIG. 16 with the amount of buffer 106 used on the horizontal axis and the encoding parameter on the vertical axis.
[Explanation of symbols]
1 Frame data acquisition means
2 Coding parameter storage means
3 Frame data encoding means
4 Encoded data output means
5 Target code amount storage means
6 Generated code amount acquisition means
7 Rate error calculation means
8 Rate error integration means
9 Previous frame code amount storage means
10 Fluctuation amount calculation means
11 Target code amount update means
12 Encoding parameter update means
13 Encoding parameter update inhibiting means
14 Redundant code generation means
15 Frame decimation code generation means
TH1 lower limit
TH2 upper limit

Claims (15)

Frame data acquisition step for acquiring frame data for one frame from moving image data, an encoding parameter storage step for storing an encoding parameter for controlling encoding of the frame data, and frame data using the encoding parameter In a moving image encoding method having a frame data encoding step for encoding the encoded data and an encoded data output step for outputting the encoded data,
A target code amount storage step for storing the target code amount;
A generation code amount acquisition step of acquiring a code amount generated by the frame data encoding step;
A rate error calculation step of calculating an error between the code amount acquired by the generated code amount acquisition step and the target code amount as a rate error;
A rate error integration step of integrating the rate error calculated in the rate error calculation step to calculate a rate error integration value;
A fluctuation amount calculating step of calculating a difference between the code amount acquired in the generated code amount acquisition step and the code amount of the previous frame data as a fluctuation amount;
A target code amount update step of inputting the fluctuation amount, rate error, and rate error integrated value to update the target code amount;
An encoding parameter update step for updating the encoding parameter by inputting the fluctuation amount, rate error, and rate error integrated value;
And encoding the frame data in a single encoding process in the frame data encoding step using the encoding parameter stored in the encoding parameter storage step. The encoding parameter updating step includes the generation of a function value having the rate error as an argument, a function value having the rate error integrated value as an argument, and a function value having the variation amount as an argument. A value obtained by multiplying the function value having the code amount acquired in the code amount acquisition step as an argument is set as the encoding parameter update amount, and the encoding parameter is updated based on the encoding parameter update amount. The moving image encoding method according to claim 1, wherein: The amount of code generated by encoding the frame data acquired in the frame data acquisition step using the encoding parameter stored in the encoding parameter storage step is a predetermined value. 3. The moving picture coding method according to claim 1, further comprising a coding parameter update inhibiting step for inhibiting the coding parameter updating process of the coding parameter updating step when the number is below. Calculated based on the code amount generated by encoding the frame data acquired in the frame data acquisition step using the encoding parameter stored in the encoding parameter storage step in the frame data encoding step. 4. The moving picture encoding method according to claim 1, further comprising a frame decimation code generation step for generating a frame decimation code when the integrated rate error value exceeds a predetermined value. Calculated based on the code amount generated by encoding the frame data acquired in the frame data acquisition step using the encoding parameter stored in the encoding parameter storage step in the frame data encoding step. 5. The moving picture coding method according to claim 1, further comprising a redundant code generation step for generating a redundant code when the integrated rate error value falls below a predetermined value. Frame data acquisition means for acquiring frame data for one frame from moving image data, encoding parameter storage means for storing an encoding parameter for controlling encoding of the frame data, and frame data using the encoding parameter In a moving image encoding apparatus having frame data encoding means for encoding the encoded data and encoded data output means for outputting the encoded data,
Target code amount storage means for storing the target code amount;
Generated code amount acquisition means for acquiring the code amount generated by the frame data encoding means;
Rate error calculating means for calculating an error between the code amount acquired by the generated code amount acquiring means and the target code amount as a rate error value;
Rate error integration means for integrating the rate errors calculated by the rate error calculation means and calculating a rate error integrated value;
Previous frame code amount storage means for storing the code amount of the previous frame data;
Variation amount calculating means for calculating a difference between the code amount stored in the previous frame code amount storage means and the code amount acquired by the generated code amount acquisition means as a variation amount;
Target code amount updating means for inputting the fluctuation amount, rate error, and rate error integrated value to update the target code amount;
Coding parameter updating means for inputting the fluctuation amount, rate error, and rate error integrated value to update the coding parameter;
And encoding the frame data in one encoding process by the frame data encoding means using the encoding parameters stored in the encoding parameter storage means. The encoding parameter update means generates the sum of three values of a function value having the rate error as an argument, a function value having the rate error integrated value as an argument, and a function value having the variation as an argument. The value obtained by multiplying the function value having the code amount acquired by the code amount acquisition means as an argument is used as the encoding parameter update amount, and the encoding parameter is updated based on the encoding parameter update amount. The moving picture coding apparatus according to claim 6. The amount of code generated by encoding the frame data acquired by the frame data acquisition unit using the encoding parameter stored in the encoding parameter storage unit is set to a predetermined value. 8. The moving picture encoding apparatus according to claim 6, further comprising: an encoding parameter update inhibiting unit that inhibits an encoding parameter updating process of the encoding parameter updating unit when the number is lower. Calculated based on the amount of code generated by encoding the frame data acquired by the frame data acquisition unit using the encoding parameter stored in the encoding parameter storage unit by the frame data encoding unit. 9. The moving picture coding apparatus according to claim 6, further comprising: a frame decimation code generating unit that generates a frame decimation code when the integrated rate error value exceeds a predetermined value. Based on the generated code amount generated by encoding the frame data acquired by the frame data acquisition unit using the encoding parameter stored in the encoding parameter storage unit by the frame data encoding unit. 10. The moving picture coding apparatus according to claim 6, further comprising a redundant code generation unit that generates a redundant code when the calculated rate error integrated value falls below a predetermined value. Frame data acquisition step for acquiring frame data for one frame from moving image data, an encoding parameter storage step for storing an encoding parameter for controlling encoding of the frame data, and frame data using the encoding parameter In the moving image encoding processing program having the frame data encoding step for encoding the encoded data and the encoded data output step for outputting the encoded data, the moving image encoding processing program includes:
A target code amount storage step for storing the target code amount;
A generation code amount acquisition step of acquiring a code amount generated by the frame data encoding step;
A rate error calculation step of calculating an error between the code amount acquired by the generated code amount acquisition step and the target code amount as a rate error value;
A rate error integration step of integrating the rate error calculated in the rate error calculation step to calculate a rate error integration value;
A fluctuation amount calculating step of calculating a difference between the code amount acquired in the generated code amount acquisition step and the code amount of the previous frame data as a fluctuation amount;
A target code amount update step of inputting the fluctuation amount, rate error, and rate error integrated value to update the target code amount;
An encoding parameter update step for updating the encoding parameter by inputting the fluctuation amount, rate error, and rate error integrated value;
A moving picture encoding processing program characterized by encoding frame data in one encoding process by the frame data encoding step using the encoding parameter stored by the encoding parameter storing step . The encoding parameter updating step includes the generation of a function value having the rate error as an argument, a function value having the rate error integrated value as an argument, and a function value having the variation amount as an argument. A value obtained by multiplying the function value having the code amount acquired in the code amount acquisition step as an argument is set as the encoding parameter update amount, and the encoding parameter is updated based on the encoding parameter update amount. The moving image encoding processing program according to claim 11. The amount of code generated by encoding the frame data acquired in the frame data acquisition step using the encoding parameter stored in the encoding parameter storage step is a predetermined value. 13. The moving image coding processing program according to claim 11 or 12, further comprising a coding parameter update inhibiting step that inhibits the coding parameter updating process of the coding parameter updating step when the number is below. Calculated based on the code amount generated by encoding the frame data acquired in the frame data acquisition step using the encoding parameter stored in the encoding parameter storage step in the frame data encoding step. 14. The moving image coding processing program according to claim 11, further comprising a frame decimation code generation step for generating a frame decimation code when the integrated rate error value exceeds a predetermined value. Calculated based on the code amount generated by encoding the frame data acquired in the frame data acquisition step using the encoding parameter stored in the encoding parameter storage step in the frame data encoding step. 15. The moving image encoding processing program according to claim 11, further comprising a redundant code generation step for generating a redundant code when the integrated rate error value falls below a predetermined value.

Images