JP4243472B2 - Image coding apparatus, image coding method, and image coding program - Google Patents

Description

【００９２】
・・・（２）
この（２）式において、ＶＡＲは、変換ブロック（ＤＣＴブロック）における入力画素の分散、ｂｌｏｃｋ＿ｃｏｕｎｔは、マクロブロックを構成する変換ブロック（ＤＣＴブロック）の個数を示すものである。
【００９３】
この（２）式に示すように、予測誤差値Ｄは、マクロブロック内の変換ブロック（ＤＣＴブロック）の入力画素の分散を求め、この入力画素の分散の平均を取ったものである。
【００９４】
これによれば、予測誤差値演算手段３ａで、入力画素（画像データ）を所定の大きさに分割した変換ブロックを直流係数に基づいて符号化する符号化モードにおける予測誤差値Ｄが、変換ブロックの入力画素の分散平均とされる。つまり、変換ブロック毎に分散が求められ、ハイブリッド符号化ブロックを構成する変換ブロックの平均を予測誤差値として演算しており、この予測誤差値Ｄが符号化効率を示す評価値の算出に利用され、この評価値に基づいて符号化モードが選択され、この符号化モードが符号化画面割当符号量（符号化ビットレート）の増減によって変化するので、符号化ビットレートが低くなっても、最適な符号化モードが選択され、符号化した画像データの画質劣化を抑制することができる。
【００９５】
（イントラマクロブロックの変換係数割当符号量の演算方法について）
次に、図６を参照して、イントラマクロブロックの変換係数割当符号量の演算方法について説明する（適宜図１参照）。イントラマクロブロックを符号化モード決定部３で処理し、変換係数割当符号量を演算する場合、図６に示すように、変換係数割当符号量演算手段３ｂ内において、符号化モードおよび入力画素が符号量演算機能３ｂ１に入力され、符号化モード符号量および直流係数符号量が求められる。続いて、これら符号化モード符号量および直流係数符号量と、ハイブリッド符号化ブロック割当符号量とが変換係数割当符号量演算機能３ｂ２に入力され、変換係数割当符号量が求められ、出力される。この変換係数割当符号量を変換係数割当符号量Ｔ_Cとすると、以下に示す式によって演算される。
【００９６】
【数２】

【００９７】
・・・（３）
この（３）式において、Ｔ_Mはハイブリッド符号化ブロック割当符号量（マクロブロック割当符号量）、Ｓ_MMは符号化モード（マクロブロックモード）の発生符号量、Ｓ_DCは、変換ブロック（ＤＣＴブロック）の直流係数（イントラＤＣ係数）の発生符号量、ｂｌｏｃｋ＿ｃｏｕｎｔは、マクロブロックを構成する変換ブロック（ＤＣＴブロック）の個数を示すものである。
【００９８】
この（３）式に示すように、変換係数割当符号量Ｔ_Cは、ハイブリッド符号化ブロック割当符号量（マクロブロック割当符号量）Ｔ_Mから符号化モード（マクロブロックモード）の発生符号量Ｓ_MMと、変換ブロック（ＤＣＴブロック）の直流係数（イントラＤＣ係数）の発生符号量Ｓ_DCとを減算したものである。
【００９９】
これによれば、変換係数割当符号量演算手段３ｂで、入力画素（画像データ）を所定の大きさに分割した変換ブロックを直流係数に基づいて符号化する符号化モードにおける変換係数割当符号量Ｔ_Cが、ハイブリッド符号化ブロック割当符号量Ｔ_Mと変換ブロックを直流係数により符号化する際に生じる発生符号量Ｓ_DCとに基づいて算出される。この変換係数割当符号量Ｔ_Cが符号化効率を示す評価値の算出に利用され、この評価値に基づいて符号化モードが選択され、この符号化モードが符号化画面割当符号量（符号化ビットレート）の増減によって変化するので、符号化ビットレートが低くなっても、最適な符号化モードが選択され、符号化した画像データの画質劣化を抑制することができる。
【０１００】
（インターマクロブロックの予測誤差値の計算方法について［平均二乗誤差］）次に、図７を参照して、インターマクロブロックの予測誤差値の演算方法について説明する（適宜図１参照）。インターマクロブロックを符号化モード決定部３で処理し、予測誤差値を演算する場合、図７に示すように、予測誤差値演算手段３ａに符号化モード、動きベクトル、入力画素および参照画素が入力され、予測誤差値演算手段３ａ内において、平均二乗誤差が演算され、この平均二乗誤差が予測誤差値として出力される。この予測誤差値を予測誤差値Ｄとすると、以下に示す式によって演算される。
【０１０１】
【数３】

【０１０２】
・・・（４）
この（４）式において、Ｍ₁（ｉ，ｊ）は、ハイブリッド符号化ブロック（マクロブロック）の入力画素と、動き補償による参照画素の平均二乗誤差である。Ｘｍ、ｎは現フレーム／フィールドにおける水平ｍ、垂直ｎの位置の画素、Ｘｍ＋ｉ，ｎ＋ｊは参照フレーム／フィールドにおける水平ｍ＋ｉ，垂直ｎ＋ｊの位置の画素である。この（４）式に示すように、予測誤差値Ｄは、入力画素と参照画素との誤差を二乗した値をハイブリッド符号化ブロックで平均したものである。
【０１０３】
これによれば、予測誤差値演算手段３ａで、動き補償により入力画素（画像データ）を符号化する符号化モードにおける予測誤差値Ｄが、ハイブリッド符号化する際のハイブリッド符号化ブロックの入力画素と参照画素との誤差を二乗した値をハイブリッド符号化ブロックで平均した平均二乗誤差とされる。平均二乗誤差の算出は、入力画素と参照画素との誤差を二乗した値を演算することで、画素上におけるエネルギの算出に相当し、直交変換を用いた変換符号化における周波数上のエネルギと同次元で優劣が比較できることになり、予測誤差値の精度が向上する。この予測誤差値Ｄが符号化効率を示す評価値の算出に利用され、この評価値に基づいて符号化モードが選択され、この符号化モードが符号化画面割当符号量（符号化ビットレート）の増減によって変化するので、符号化ビットレートが低くなっても、最適な符号化モードが選択され、符号化した画像データの画質劣化を抑制することができる。
【０１０４】
（インターマクロブロックの予測誤差値の計算方法について［平均絶対値］）
次に、図８を参照して、インターマクロブロックの予測誤差値の演算方法について説明する（適宜図１参照）。インターマクロブロックを符号化モード決定部３で処理し、予測誤差値を演算する場合、図８に示すように、予測誤差値演算手段３ａに符号化モード、入力画素および参照画素が入力され、予測誤差値演算手段３ａ内において、平均絶対値誤差が演算され、この平均絶対値誤差が予測誤差値として出力される。この予測誤差値を予測誤差値Ｄとすると、以下に示す式によって演算される。
【０１０５】
【数４】

【０１０６】
・・・（５）
この（５）式において、Ｍ₂（ｉ，ｊ）は、ハイブリッド符号化ブロック（マクロブロック）の入力画素と、動き補償による参照画素の平均絶対値誤差である。Ｘｍ、ｎは現フレーム／フィールドにおける水平ｍ、垂直ｎの位置の画素、Ｘｍ＋ｉ，ｎ＋ｊは参照フレーム／フィールドにおける水平ｍ＋ｉ，垂直ｎ＋ｊの位置の画素である。この（５）式に示すように、予測誤差値Ｄは、入力画素と参照画素の平均絶対値誤差である。
【０１０７】
これによれば、予測誤差値演算手段３ａで、動き補償により入力画素（画像データ）を符号化する符号化モードにおける予測誤差値Ｄが、ハイブリッド符号化する際のハイブリッド符号化ブロックの入力画素と参照画素との誤差の絶対値をハイブリッド符号化ブロックで平均した値とされる。絶対値の平均を取るだけの平均絶対値誤差を予測誤差値とすることで、予測誤差値演算手段３ａの演算量が軽減され、処理速度が向上する。この予測誤差値Ｄが符号化効率を示す評価値の算出に利用され、この評価値に基づいて符号化モードが選択され、この符号化モードが符号化画面割当符号量（符号化ビットレート）の増減によって変化するので、符号化ビットレートが低くなっても、最適な符号化モードが選択され、符号化した画像データの画質劣化を抑制することができる。
【０１０８】
（インターマクロブロックの予測誤差値の計算方法について［相互相関係数］）次に、図９を参照して、インターマクロブロックの予測誤差値の演算方法について説明する（適宜図１参照）。インターマクロブロックを符号化モード決定部３で処理し、予測誤差値を演算する場合、図９に示すように、予測誤差値演算手段３ａに符号化モード、入力画素および参照画素が入力され、予測誤差値演算手段３ａ内において、相互相関係数が演算され、この相互相関係数が予測誤差値として出力される。この予測誤差値を予測誤差値Ｄとすると、以下に示す式によって演算される。
【０１０９】
【数５】

【０１１０】
・・・（６）
この（６）式において、Ｍ₃（ｉ，ｊ）は、ハイブリッド符号化ブロック（マクロブロック）の入力画素と、動き補償による参照画素の相互相関係数である。Ｘｍ、ｎは現フレーム／フィールドにおける水平ｍ、垂直ｎの位置の画素、Ｘｍ＋ｉ，ｎ＋ｊは参照フレーム／フィールドにおける水平ｍ＋ｉ，垂直ｎ＋ｊの位置の画素である。この（６）式に示すように、予測誤差値Ｄは、入力画素と参照画素の相互相関係数である。
【０１１１】
これによれば、予測誤差値演算手段３ａで、動き補償により入力画素（画像データ）を符号化する符号化モードにおける予測誤差値Ｄが、ハイブリッド符号化する際のハイブリッド符号化ブロックの入力画素と参照画素との相互相関係数とされる。入力画素と参照画素とを比較して、相関係数を演算した相互相関係数を予測誤差値としているので、予測誤差値の精度が向上する。この予測誤差値Ｄが符号化効率を示す評価値の算出に利用され、この評価値に基づいて符号化モードが選択され、この符号化モードが符号化画面割当符号量（符号化ビットレート）の増減によって変化するので、符号化ビットレートが低くなっても、最適な符号化モードが選択され、符号化した画像データの画質劣化を抑制することができる。
【０１１２】
（インターマクロブロックのＤＣＴ係数割当符号量の計算方法について）
次に、図１０を参照して、インターマクロブロックの変換係数割当符号量の演算方法について説明する（適宜図１参照）。インターマクロブロックを符号化モード決定部３で処理し、変換係数割当符号量を演算する場合、図１０に示すように、変換係数割当符号量演算手段３ｂ内において、符号化モードおよび動きベクトルが符号量演算機能３ｂ１に入力され、符号化モード符号量および動きベクトル符号量が求められる。続いて、これら符号化モード符号量および動きベクトル符号量とハイブリッド符号化ブロック割当符号量とが変換係数割当符号量演算機能３ｂ２に入力され、変換係数割当符号量が求められ、出力される。この変換係数割当符号量を変換係数割当符号量Ｔ_Cとすると、以下に示す式によって演算される。
【０１１３】
Ｔ_C＝Ｔ_M−Ｓ_MM−Ｓ_MV
【０１１４】
・・・（７）
この（７）式において、Ｔ_Mはハイブリッド符号化ブロック割当符号量（マクロブロック割当符号量）、Ｓ_MMは符号化モード（マクロブロックモード）の発生符号量、Ｓ_MVは、動きベクトルの発生符号量を示すものである。
【０１１５】
この（７）式に示すように、変換係数割当符号量Ｔ_Cは、ハイブリッド符号化ブロック割当符号量（マクロブロック割当符号量）Ｔ_Mから符号化モード（マクロブロックモード）の発生符号量Ｓ_MMと、動きベクトルの発生符号量Ｓ_MVとを減算したものである。
【０１１６】
これによれば、変換係数割当符号量演算手段３ｂで、動き補償により画像データを符号化する符号化モードにおける変換係数割当符号量Ｔ_Cが、ハイブリッド符号化ブロック割当符号量Ｔ_Mと、符号化モード（マクロブロックモード）の発生符号量Ｓ_MMと、動き補償に供される動きベクトルにより符号化する際に生じる発生符号量Ｓ_MVとに基づいて算出される。この変換係数割当符号量Ｔ_Cが符号化効率を示す評価値の算出に利用され、この評価値に基づいて符号化モードが選択され、この符号化モードが符号化画面割当符号量（符号化ビットレート）の増減によって変化するので、符号化ビットレートが低くなっても、最適な符号化モードが選択され、符号化した画像データの画質劣化を抑制することができる。
【０１１７】
（ＦＰＦＤの決定方法について［その１］）
次に、図１１を参照して、ハイブリッド符号化ブロック（マクロブロック）において符号化する変換ブロックを任意に選択する、または、一意に選択する画面符号化モードであるＦＰＦＤ（符号化パラメータ）について説明する。この図１１は画面全体の符号化効率を演算しＦＰＦＤ（符号化パラメータ）を決定する場合について説明した図である。
【０１１８】
この図１１に示すように、符号化効率評価値演算手段３ｅの符号化画面合計機能３ｅ１にハイブリッド符号化ブロックの変換係数割当符号量が入力され、符号化画面合計機能３ｅ２にハイブリッド符号化ブロックの予測誤差値が入力される。符号化画面合計機能３ｅ１から符号化画面の変換係数割当符号量が、符号化画面合計機能３ｅ２から符号化画面の予測誤差値が出力され、符号化効率計算機能３ｅ３に入力され、この符号化効率計算機能３ｅ３でＦＰＦＤ符号化効率を表す評価値（符号化画面の平均の符号化率を表す評価値）が計算され、計算されたＦＰＦＤ符号化効率を表す評価値（符号化画面の平均の符号化率を表す評価値）が符号化モード選択手段３ｆに出力される。このＦＰＦＤ符号化効率を表す評価値（符号化画面の平均の符号化率を表す評価値）を評価値Ｅ_Fとすると、以下に示す式によって演算される。
【０１１９】
【数６】

【０１２０】
・・・（８）
この（８）式において、Ｄは予測誤差値、Ｔ _Ｃ は変換係数符号量、ＭＢ＿ｃｏｕｎｔは符号化画面を構成するハイブリッド符号化ブロック（マクロブロック）の個数を示すものである。
【０１２１】
この（８）式に示すように、符号化画面の符号化効率を表す評価値Ｅ_Ｆは、各ハイブリッド符号化ブロック（マクロブロック）の予測誤差値Ｄを符号化画面毎に合計したものを、変換係数符号量Ｔ _Ｃ を符号化画面毎に合計したもので除算したものである。そして、この符号化画面平均の画面の符号化率を表す評価値Ｅ_Ｆが最大となるＦＰＦＤ（符号化パラメータ）が符号化モード選択手段３ｆで選択される。
【０１２２】
これによれば、符号化効率評価値演算手段３ｅで演算された評価値Ｅ_Fが最大となるＦＰＦＤ（符号化パラメータ）が符号化モード選択手段３ｆで選択され、ＦＰＦＤ（符号化パラメータ）が符号化画面割当符号量（符号化ビットレート）の増減によって変化するので、符号化ビットレートが低くなっても、最適な符号化モードが選択され、符号化した画像データの画質劣化を抑制することができる。
【０１２３】
（ＦＰＦＤの決定方法について［その２］）
次に、図１２を参照して、ハイブリッド符号化ブロック（マクロブロック）において符号化する変換ブロックを任意に選択する、または、一意に選択する画面符号化モードであるＦＰＦＤ（符号化パラメータ）の別の決定方法について説明する。この図１２はハイブリッド符号化ブロック毎に符号化効率を演算しＦＰＦＤ（符号化パラメータ）を決定する場合について説明した図である。
【０１２４】
この図１２に示すように、符号化効率評価値演算手段３ｅの符号化画面平均機能３ｅ４にハイブリッド符号化ブロックの符号化効率が入力され、この符号化画面平均機能３ｅ４でＦＰＦＤ符号化効率を表す評価値（符号化画面の平均の符号化率を表す評価値）が計算され、計算されたＦＰＦＤ符号化効率を表す評価値（符号化画面の平均の符号化率を表す評価値）が符号化モード選択手段３ｆに出力される。このＦＰＦＤ符号化効率を表す評価値（符号化画面の平均の符号化率を表す評価値）を評価値Ｅ_Fとすると、以下に示す式によって演算される。
【０１２５】
【数７】

【０１２６】
・・・（９）
この（９）式において、Ｅ _ＭＭ は各ハイブリッド符号化ブロック（マクロブロック）の符号化効率を表す評価値、ＭＢ＿ｃｏｕｎｔは符号化画面を構成するハイブリッド符号化ブロック（マクロブロック）の個数を示すものである。
【０１２７】
この（９）式に示すように、符号化画面平均の画面の符号化率を表す評価値Ｅ_Ｆは、各ハイブリッド符号化ブロック（マクロブロック）の符号化効率を表す評価値Ｅ _ＭＭ の合計をハイブリッド符号化ブロック（マクロブロック）の個数で除算したものである。そして、この符号化画面平均の画面の符号化率を表す評価値Ｅ_Ｆが最大となるＦＰＦＤ（符号化パラメータ）が符号化モード選択手段３ｆで選択される。
【０１２８】
これによれば、符号化効率評価値演算手段３ｅで演算された評価値Ｅ_Fが最大となるＦＰＦＤ（符号化パラメータ）が符号化モード選択手段３ｆで選択されるので、ＦＰＦＤ（符号化パラメータ）が符号化画面割当符号量（符号化ビットレート）の増減によって変化するので、符号化ビットレートが低くなっても、最適な符号化モードが選択され、符号化した画像データの画質劣化を抑制することができる。
【０１２９】
（マクロブロック割当符号量の演算方法について）
次に、図１３を参照して、ハイブリッド符号化ブロック（マクロブロック）の割当符号量の演算方法について説明する。図１３は、バッファ制御部２５とハイブリッド符号化ブロック割当符号量演算手段３ｄとによってハイブリッド符号化ブロック（マクロブロック）の符号量を演算する仕方について説明した図である。
【０１３０】
この図１３に示すように、バッファ制御部２５から符号化画面毎に割り当てられる符号量である符号化画面割当符号量がハイブリッド符号化ブロック割当符号量演算手段３ｄに入力され、このハイブリッド符号化ブロック割当符号量演算手段３ｄで各ハイブリッド符号化ブロックに割り当てられるハイブリッド符号化ブロック割当符号量が演算される。このハイブリッド符号化ブロック割当符号量をハイブリッド符号化ブロック割当符号量Ｔ_Mとすると、以下に示す式によって演算される。
【０１３１】
【数８】

【０１３２】
・・・（１０）
この（１０）式において、Ｔ_Pはバッファ制御部２５から与えられる符号化画面割当符号量、ＭＢ＿ｃｏｕｎｔは符号化画面を構成するハイブリッド符号化ブロック（マクロブロック）の個数を示すものである。
【０１３３】
この（１０）式に示すように、ハイブリッド符号化ブロック割当符号量Ｔ_Mは、バッファ制御部２５から与えられる符号化画面割当符号量Ｔ_Pをハイブリッド符号化ブロック（マクロブロック）の個数で除算したものである。
【０１３４】
これによれば、バッファ制御部２５から与えられる符号化画面割当符号量Ｔ_Pの増減によって、ハイブリッド符号化ブロック割当符号量Ｔ_Mも増減するので、符号化ビットレートが低くなっても、最適な符号化モードが選択され、符号化した画像データの画質劣化を抑制することができる。
【０１３５】
（画像符号化装置の効果について）
最後に、画像符号化装置１と、従来技術の段落で説明したＴＭ５とについて、符号化シミュレーションを行った結果、画像符号化装置１で確認された顕著な効果について説明する。なお、バッファ制御部２５の構成はＴＭ５（エンコーダ、従来技術の説明参照）と同様とした。
【０１３６】
ＩＴＥハイビジョン標準動画像“Ｅｕｒｏｐｅａｎ Ｍａｒｋｅｔ”、“Ｗｈａｌｅ Ｓｈｏｗ”、“Ｇｒｅｅｎ Ｌｅａｖｅｓ”について、画像符号化装置１における符号化ビットレートを１１，１５Ｍｂｐｓとし、ＴＭ５における符号化ビットレートを１１，１５，２２Ｍｂｐｓとして、ＰＳＮＲ（Ｐｅａｋ Ｓｉｇｎａｌ ｔｏ Ｎｏｉｓｅ Ｒａｔｉｏ）の比較を行った。
【０１３７】
すると、同一ビットレートのＴＭ５と比較した結果、いずれの画像（シーン）においても１〜３ｄＢ程度の画質の改善を確認することができ、画像符号化装置１の符号化ビットレートが１５ＭｂｐｓにおけるＰＳＮＲは、ＴＭ５の符号化ビットレート２２Ｍｂｐｓ以上のＰＳＮＲであることを確認することができた。
【０１３８】
また、画像符号化装置１によって符号化されたビットストリームを視聴した視聴者によって、視覚的にも全ての画像（シーン）で解像度感が向上し、特に、動きの速い映像では画質改善効果が高いことが確認された。
【０１３９】
さらに、１５Ｍｂｐｓで符号化したビットストリームについて、画像符号化装置１における変換係数符号量と動きベクトル符号量との比率と、ＴＭ５における変換係数符号量と動きベクトル符号量との比率とを比較すると、いずれの画像（シーン）においても動きベクトル符号量が１／５〜１／６に低減され、変換係数符号量が増加していることが確認でき、変換係数に符号量が多く割り当てられることで、画質改善できたことが確認できた。
【０１４０】
以上、一実施形態に基づいて本発明を説明したが、本発明はこれに限定されるものではない。
【０１４１】
例えば、画像符号化装置１の各構成の処理を一つずつの過程ととらえた画像符号化方法とみなすこともできる。また、画像符号化装置１の各構成の処理を汎用的なコンピュータ言語で記述した画像符号化プログラムとみなすこともできる。これらの場合、画像符号化装置１と同様の効果を得ることができ、さらに、画像符号化プログラムの場合、当該プログラムをネットワークに配信したり、或いは当該プログラムを記憶できる記憶媒体に記憶させて流通させたりすることも可能である。
【０１４２】
請求項１、３、４記載の発明によれば、画像データを符号化する際の誤差値と、変換係数に割り当てられる変換係数割当符号量とが演算される。誤差値と、変換係数割当符号量とに基づいて評価値が演算され、この評価値に基づいた符号化モードが選択され、選択された符号化モードにより、画像データが符号化される。このため、変換係数割当符号量が符号化ビットレートの増減によって変化するので、符号化ビットレートが低くなっても、最適な符号化モードが選択され、符号化した画像データの画質劣化を抑制することができる。
【０１４３】
また、請求項１、３、４記載の発明によれば、画像データを所定の大きさに分割したハイブリッド変換ブロックに含まれる変換ブロック毎に分散を求め、ハイブリッド符号化ブロックを構成する変換ブロックの分散の平均を、イントラ符号化モードにおける誤差値として得ることができると共に、この誤差値が利用され、最適な符号化モードが選択されるので、符号化した画像データの画質劣化を抑制することができる。
【０１４４】
また、請求項１、３、４記載の発明によれば、入力画素と参照画素の誤差を二乗した値を、ハイブリッド符号化ブロックで平均した平均二乗誤差をインター符号化モードにおける誤差値とすることにより、直交変換を用いた変換符号化における周波数上のエネルギと同次元で優劣が比較できると共に、この誤差値が利用されて、最適な符号化モードが選択されるので、符号化した画像データの画質劣化を抑制することができる。
【０１４５】
また、請求項１、３、４記載の発明によれば、入力画素と参照画素の誤差の絶対値をハイブリッド符号化ブロックで平均した平均絶対値誤差をインター符号化モードにおける誤差値とすることにより、予測誤差値演算手段における演算量を軽減することができると共に、この誤差値が利用され、最適な符号化モードが選択されるので、符号化した画像データの画質劣化を抑制することができる。
【０１４６】
また、請求項１、３、４記載の発明によれば、入力画素と参照画素とを比較して、相関係数を演算した相互相関係数によって、精度の向上したインター符号化モードにおける誤差値を得ることができると共に、この誤差値が利用され、最適な符号化モードが選択されるので、符号化した画像データの画質劣化を抑制することができる。
【０１４７】
請求項２記載の発明によれば、ハイブリッド符号化ブロックの割当符号量が、画像データに含まれる画面を構成するハイブリッド符号化ブロック数で演算された符号量とされ、符号化ビットレートの増減によって、ハイブリッド符号化ブロックの割当符号量も増減するので、符号化ビットレートが低くなっても、最適な符号化モードが選択され、符号化した画像データの画質劣化を抑制することができる。
【図面の簡単な説明】
【図１】本発明による一実施の形態である画像符号化装置のブロック図である。
【図２】画像符号化装置の動作を説明したフローチャートである。
【図３】符号化モード決定部の動作を説明したフローチャートである。
【図４】符号化モードの決定方法について説明した図である。
【図５】イントラマクロブロックの予測誤差値の演算方法について説明した図である。
【図６】イントラマクロブロックの変換係数割当符号量の演算方法について説明した図である。
【図７】インターマクロブロックの予測誤差値の演算方法（平均二乗誤差）について説明した図である。
【図８】インターマクロブロックの予測誤差値の演算方法（平均絶対値誤差）について説明した図である。
【図９】インターマクロブロックの予測誤差値の演算方法（相互相関係数）について説明した図である。
【図１０】インターマクロブロックの変換係数割当符号量の演算方法について説明した図である。
【図１１】ＦＰＦＤの決定方法（符号化画面全体）について説明した図である。
【図１２】ＦＰＦＤの決定方法（ハイブリッド符号化ブロック）について説明した図である。
【図１３】ハイブリッド符号化ブロック（マクロブロック）割当符号量の演算方法について説明した図である。
【図１４】従来の画像符号化装置のブロック図である。
【符号の説明】
１ 画像符号化装置
３ 符号化モード決定部
３ａ 予測誤差値演算手段
３ｂ 変換係数割当符号量演算手段
３ｃ 符号化モード・動きベクトル発生手段
３ｄ ハイブリッド符号化ブロック割当符号量演算手段
３ｅ 符号化効率評価値演算手段
３ｆ 符号化モード選択手段
５ 動き補償予測部
７ 減算器
９ ＤＣＴ部
１１ 量子化器
１３ 逆量子化器
１５ ＩＤＣＴ部
１７ 加算器
１９ フレームメモリ
２１ 可変長符号部
２３ バッファ
２５ バッファ制御部[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image encoding device, an image encoding method, and an image encoding program for encoding an image by hybrid encoding combining predictive encoding and transform encoding.
[0002]
[Prior art]
In general, as a method of encoding image data such as a moving image, a moving image by hybrid encoding combining prediction encoding for performing motion compensation prediction using a motion vector and transform encoding using a DCT (orthogonal cosine transform) coefficient. Image coding methods are widely used.
[0003]
In this moving picture coding system, there is MPEG-2 video (see Non-Patent Document 1) as a moving picture coding used for digital television broadcasting or the like. TM5 (Test Model 5, ISO / IEC JTC1 / SC29 / WG11 / N0400, MPEG93 / 475) can be cited as a moving picture encoding method compliant with the MPEG-2 video. This is an encoder that realizes the MPEG-2 video standard in the encoding system.
[0004]
Here, the MPEG-2 video will be described in detail.
In MPEG-2 video, in addition to a progressive sequence in which all the frame samples constituting the screen indicate the same time instant, an interless sequence in which alternate scan lines of the frame indicate different moments in time is supported. Therefore, it is standardized so that frame processing or field processing can be adaptively selected hierarchically. One frame consists of two fields. In addition, when encoding one frame as one screen or when encoding two fields as two screens, it is determined that the initial screen (original image) is to be encoded. This is called a structured screen structure (image structure). One frame encoded as one screen is referred to as a frame image, and each of two fields encoded as two screens is referred to as a field image. These frame images and field images are adaptive in units of frames (arbitrary). Can be selected.
[0005]
When the frame image is encoded, the DCT type of the frame DCT or the field DCT can be used properly. The initial image is a macroblock (luminance component 16 pixels × 16 lines and color difference components corresponding to this area). Further, the macroblock is divided into fine blocks (both luminance and chrominance components are each 8 pixels × 8 lines) and DCT-coded according to each DCT type. When frame DCT is used for the DCT type, the fine block is composed of alternating lines from two fields, and when the field DCT is used for the DCT type, the fine block is one line from the two fields. Consists of.
[0006]
The macroblock includes an intra macroblock that encodes the original image using information of only the macroblock, and an inter macroblock that encodes the original image based on prediction of the previously encoded image. There is. In addition, prediction types in predictive coding are roughly classified into two types: field prediction and frame prediction. Field prediction uses a previously encoded field and performs prediction independently for the top field and the bottom field. In frame prediction, prediction is performed from a previously encoded frame.
[0007]
In the field image, all prediction types are field prediction, and in the frame image, the prediction type can be selected for each macroblock for field prediction or frame prediction. However, when the FPFD (Frame Pred Frame Dct) flag is set to “1” in the frame image, only the frame DCT and the frame prediction are used, and in the field image, the FPFD flag is set to “0”.
[0008]
In MPEG-2 video, picture types in which an initial image is encoded include I, P, and B (I picture, P picture, and B picture). An I picture is an image encoded using only its own information, and is composed only of intra macroblocks. A P picture is an image that has been encoded using motion prediction compensation from a reference image in the past (before encoding the P picture), and includes an intra macro block and an inter macro block for forward prediction predicted from the past. Has been. A B picture is an image that is encoded using motion prediction compensation from a reference image in the past (before the B picture is encoded) and in the future (after the B picture is encoded), and is predicted from an intra macroblock and the past. Forward prediction, backward prediction predicted from the future, and inter-macroblocks of bidirectional prediction predicted from the past and the future.
[0009]
By the way, only the prediction types other than the prediction types in the above-described predictive encoding will be described until the supplement, for example, there are prediction types such as “dual prime” and “16 × 8 motion compensation”. In “dual prime”, one motion vector in the original image and a difference vector smaller than this motion vector are used. In “16 × 8 motion compensation”, two sets of motion vectors of the upper and lower parts of the 16 × 8 region are used for each macroblock, and “16 × 8 motion compensation” is used only for field images.
[0010]
By the way, in this MPEG-2 video-compliant moving image coding system, determination of a motion vector and a macroblock mode is performed as follows.
[0011]
In this moving image coding method, first, for each prediction type, the absolute value prediction errors of the motion vectors of the selection candidates are compared, and the motion vector that minimizes the absolute value prediction error is selected. Next, the mean square prediction error of each prediction type is compared, and the prediction type that minimizes the mean square prediction error is selected. Finally, the macroblock input error variance is compared with the mean square prediction error of the selected prediction type. If the variance of the input error is smaller, the intra macroblock is less than the mean square prediction error. , Select each intra macroblock of the selected prediction type.
[0012]
Buffer control in transform coding is performed by adjusting the DCT coefficient generation code amount based on the quantized value of the DCT coefficient obtained by feedback control from the buffer occupancy.
[0013]
For reference, FIG. 14 shows a conventional image encoding apparatus that encodes image data by this moving image encoding method. In this image encoding device 101, the encoding mode determination unit 103 determines how to encode image data (input pixels) input from video input. The encoding mode determination unit 103 calculates a prediction error value from the input pixel and the reference pixel (pixel input before the input pixel) stored in the frame memory, and based on the prediction error value The encoding mode is determined.
[0014]
[Non-Patent Document 1]
ISO / IEC13818-2 1995 (Video compression standard)
[0015]
[Problems to be solved by the invention]
However, in the conventional MPEG-2 video-compliant moving image coding method, the coding bit rate is not considered in the selection criteria of the motion vector, the prediction type, and the macroblock mode, so the same image data (video) can be used. For example, regardless of the encoding bit rate, the generated code amounts generated for the information related to the motion vector and the macroblock mode are substantially equal. When the encoding bit rate is lowered, the ratio of the generated code amount such as a motion vector tends to increase. As a result, the allocated code amount to the DCT coefficient decreases. There is a problem that the image quality of the processed image data deteriorates.
[0016]
Accordingly, an object of the present invention is to solve the problems of the conventional techniques described above, and image coding that can suppress deterioration in the image quality of the coded image data even when the coding bit rate is lowered in hybrid coding. An apparatus, an image encoding method, and an image encoding program are provided.
[0017]
[Means for Solving the Problems]
  In order to achieve the above-described object, the present invention has the following configuration.
  Claim 1InThe described image encoding device is an image encoding device that encodes image data by hybrid encoding that combines predictive encoding using motion compensated prediction based on motion vectors and transform encoding using orthogonal transform. TheThe prediction error value calculating means, the transform coefficient allocation code amount calculating means, the encoding efficiency evaluation value calculating means, the encoding mode selecting means, and the encoding means are provided.
[0018]
  According to such a configuration, the image encoding device uses the prediction error value calculation means to encode an error value when the image data is encoded by predictive encoding.As for the hybrid coding block obtained by dividing the image data into a predetermined size, the error value D in the intra coding mode ₁ Is calculated by averaging the variances of the input pixels of the transform block included in the hybrid coding block, and an error value D in the inter coding mode is calculated. ₂ By the mean square error, mean absolute value error, or cross-correlation coefficient between the input pixel of the hybrid coding block and the reference pixel of the block.Operate.
  ContinuedThe image encoding apparatus divides the image data into a predetermined size by the transform coefficient allocation code amount calculation means.Hybrid coding blockabout,SaidTransform coefficient allocation code amount in intra coding modeT _C1 Is generated from the hybrid coding block allocation code amount allocated to the hybrid coding block at the time of the hybrid encoding, and the generated code amount of the intra coding modeIncluded in the hybrid coding blockGenerated code amount of DC coefficient of transform blockSum ofObtained by subtracting, Transform coefficient allocation code amount T in the inter coding mode _C2 From the hybrid coding block allocation code amount, the generated code amount of the inter coding mode, and the generated code amount of the motion vectorFind by subtracting.
  MaIn addition, the image encoding device uses the encoding efficiency evaluation value calculation means,Evaluation value E indicating coding efficiency in the intra coding mode ₁ The error value D calculated by the prediction error value calculation means ₁ And the transform coefficient allocation code amount T calculated by the transform coefficient allocation code amount calculation means. _C1 And an evaluation value E indicating the coding efficiency in the inter coding mode. ₂ The error value D calculated by the prediction error value calculation means ₂ And the transform coefficient allocation code amount T calculated by the transform coefficient allocation code amount calculation means. _C2 The evaluation value indicating the coding efficiency is obtained by calculating the product of.
  SoThen, the image encoding device selects the encoding mode in which the evaluation value calculated by the encoding efficiency evaluation value calculating unit is maximized by the encoding mode selecting unit..
  SoThen, the image encoding device encodes the image data by the encoding unit in the encoding mode selected by the encoding mode selection unit.
[0019]
In other words, the motion vector and the generated code amount of the macro block mode (inter macro block or intra macro block) when encoding the image data calculated by the prediction error value calculating unit and the transform coefficient allocation code amount calculating unit are controlled. The evaluation value is calculated based on the prediction error value and the transform coefficient allocation code amount, and the evaluation value is input to the encoding mode selection unit, and the encoding mode selection unit selects the encoding mode. . In the coding efficiency evaluation value calculation means, for example, the product of the reciprocal of the prediction error value and the transform coefficient allocation code amount is calculated, and in the coding mode selection means, the evaluation value calculated by the coding efficiency evaluation value calculation means The largest of is selected.
[0020]
Also, in this encoding mode selection means, the motion vector and macroblock mode that maximize the evaluation value of the encoding efficiency for the selection candidate motion vector and macroblock mode are selected. For inter-macroblocks, the total generated code amount of the motion vector and macroblock mode information is obtained, and the subtraction from the macroblock allocation code amount is used as the DCT coefficient allocation code amount. The reciprocal of the motion vector prediction error and the DCT coefficient allocation code The value obtained by integrating the quantity is taken as the evaluation value. For the intra macroblock, the total generated code amount of the intra DC coefficient (intra direct current coefficient) and the macroblock mode information is obtained, and the subtraction from the macroblock allocation code amount is set as the DCT coefficient allocation code amount. The sum of the reciprocal of the variance and the DCT coefficient allocation code amount is taken as the evaluation value.
[0031]
  Claim2The image encoding device according to claim 11In the image encoding device described above, the encoding unit divides the screen allocation code amount allocated when encoding the screen included in the image data by the number of hybrid encoding blocks constituting the screen. The allocation code amount of the hybrid coding block at the time of the hybrid coding is obtained.
[0032]
  According to such a configuration,The image encoding device performs hybrid encoding by dividing the screen allocation code amount allocated when encoding the screen included in the image data by the number of hybrid encoding blocks constituting the screen by the encoding means. The allocated code amount of the hybrid coding block at that time is obtained.
[0033]
  Claim3The image encoding method described in 1 is obtained by converting predictive encoding for predicting an encoding target pixel based on encoded pixels and an image signal in which an encoding target image composed of the encoding target pixels is continuous. An image encoding method for encoding image data by hybrid encoding combined with transform encoding for quantizing a transform coefficient and variable-length encoding, comprising: a prediction error value calculating step; and a transform coefficient allocation code amount It includes a calculation step, a coding efficiency evaluation value calculation step, and a coding step.
[0034]
  According to this method, in the prediction error value calculation step, an error value when image data is encoded by predictive encoding.As for the hybrid coding block obtained by dividing the image data into a predetermined size, the error value D in the intra coding mode ₁ Is calculated by averaging the variances of the input pixels of the transform block included in the hybrid coding block, and an error value D in the inter coding mode is calculated. ₂ By the mean square error, mean absolute value error, or cross-correlation coefficient between the input pixel of the hybrid coding block and the reference pixel of the block.In the transform coefficient allocation code amount calculation step, the image data is divided into a predetermined size.Hybrid coding blockabout,SaidTransform coefficient allocation code amount in intra coding modeT _C1 Is generated from the hybrid coding block allocation code amount allocated to the hybrid coding block at the time of the hybrid encoding, and the generated code amount of the intra coding modeIncluded in the hybrid coding blockGenerated code amount of DC coefficient of transform blockSum ofObtained by subtracting, Transform coefficient allocation code amount T in the inter coding mode _C2 From the hybrid coding block allocation code amount, the generated code amount of the inter coding mode, and the generated code amount of the motion vectorFind by subtracting.
  SoIn the encoding efficiency evaluation value calculation step,Evaluation value E indicating coding efficiency in the intra coding mode ₁ The error value D calculated by the prediction error value calculation means ₁ And the transform coefficient allocation code amount T calculated by the transform coefficient allocation code amount calculation means. _C1 And an evaluation value E indicating the coding efficiency in the inter coding mode. ₂ The error value D calculated by the prediction error value calculation means ₂ And the transform coefficient allocation code amount T calculated by the transform coefficient allocation code amount calculation means. _C2 To calculate an evaluation value indicating the encoding efficiency, and in the encoding step, select an encoding mode in which the evaluation value calculated in the encoding efficiency evaluation value calculation step is maximum, The image data is encoded according to the selected encoding mode.
[0035]
  Claim4The image encoding program described in the above describes a computer for encoding image data by hybrid encoding combining prediction encoding using motion compensated prediction based on motion vectors and transform encoding using orthogonal transform. , A prediction error value calculation unit, a transform coefficient allocation code amount calculation unit, a coding efficiency evaluation value calculation unit, a coding mode selection unit, and a coding unit.
[0036]
  According to this configuration, the image encoding program uses the prediction error value calculation means to encode the error value when the image data is encoded by predictive encoding.As for the hybrid coding block obtained by dividing the image data into a predetermined size, the error value D in the intra coding mode ₁ Is calculated by averaging the variances of the input pixels of the transform block included in the hybrid coding block, and an error value D in the inter coding mode is calculated. ₂ By the mean square error, mean absolute value error, or cross-correlation coefficient between the input pixel of the hybrid coding block and the reference pixel of the block.And the image data is divided into a predetermined size by the transform coefficient allocation code amount calculation means.Hybrid coding blockabout,AboveTransform coefficient allocation code amount in intra coding modeT _C1 Is generated from the hybrid coding block allocation code amount allocated to the hybrid coding block at the time of the hybrid encoding, and the generated code amount of the intra coding modeIncluded in the hybrid coding blockGenerated code amount of DC coefficient of transform blockSum ofObtained by subtracting, Transform coefficient allocation code amount T in the inter coding mode _C2 From the hybrid coding block allocation code amount, the generated code amount of the inter coding mode, and the generated code amount of the motion vectorFind by subtracting.
  SoThen, by the coding efficiency evaluation value calculation means,Evaluation value E indicating coding efficiency in the intra coding mode ₁ The error value D calculated by the prediction error value calculation means ₁ And the transform coefficient allocation code amount T calculated by the transform coefficient allocation code amount calculation means. _C1 And an evaluation value E indicating the coding efficiency in the inter coding mode. ₂ The error value D calculated by the prediction error value calculation means ₂ And the transform coefficient allocation code amount T calculated by the transform coefficient allocation code amount calculation means. _C2 To calculate an evaluation value indicating the encoding efficiency, and the encoding mode selection unit selects an encoding mode that maximizes the evaluation value calculated by the encoding efficiency evaluation value calculation unit, The image data is encoded by the encoding means in the encoding mode selected by the encoding mode selection means.
[0041]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.
(Configuration of image encoding device)
FIG. 1 is a block diagram of an image encoding device. As shown in FIG. 1, an image encoding device 1 outputs an input pixel (image data) input from a video input as an encoded bit stream, and includes an encoding mode determination unit 3 and a motion compensated prediction. Unit 5, subtractor 7, DCT unit 9, quantizer 11, inverse quantizer 13, IDCT unit 15, adder 17, frame memory 19, variable length coding unit 21, buffer 23 and a buffer control unit 25.
[0042]
This image encoding device 1 is adapted to MPEG-2 video encoding (ISO / IEC13818-2), and an input pixel (image data) to be encoded is converted into a picture type (I picture, P picture, B). Coding is performed based on a combination of a coding mode (macroblock mode) and a motion vector that can be selected by a picture) and an image structure (frame image, field image). This encoding mode (macroblock mode) is determined by a combination of an intra / inter macroblock, a prediction direction, a prediction type, and a DCT type.
[0043]
The encoding mode determination unit 3 determines an encoding mode which is an encoding method for encoding image data, and includes a prediction error value calculation unit 3a, a transform coefficient allocation code amount calculation unit 3b, and an encoding mode. A motion vector generation unit 3c, a hybrid coding block allocation code amount calculation unit 3d, a coding efficiency evaluation value calculation unit 3e, and a coding mode selection unit 3f are provided. The encoding mode determination unit 3 has a determination function for determining whether to handle an intra macroblock or an inter macroblock. Based on the determination result of the determination function, the intra macroblock is determined. When the inter-macroblock is handled, the information input / output is slightly different in each of the means 3a to 3f. The details will be described in the description of the operation of the encoding mode determination unit 3.
[0044]
The prediction error value calculation means 3a is configured to generate input pixels (image data) from video input, reference pixels (previous image data) held (stored) in the frame memory 19, and encoding mode / motion vector generation. A prediction error value is calculated based on the encoding mode input from the means 3c (in the case of an inter macro block, a motion vector is also input). That is, the prediction error value calculation means 3a repeatedly calculates the prediction error value according to the number of encoding modes generated by the encoding mode / motion vector generation means 3c. The prediction error value indicates the difference between the input pixel and the reference pixel. For example, when the motion of the subject drawn by the continuous input pixel is intense (fast motion video), the prediction error value becomes large. .
[0045]
  Note that if it is determined by the determination function of the encoding mode determination unit 3 that an intra macroblock is handled, this prediction error is performed.Difference value operatorIn the stage 3a, the input pixel and the encoding mode are input, and when it is determined that the inter macroblock is handled (when it is not determined that the intra macroblock is handled), the input pixel, the reference pixel, the encoding mode, and the motion A vector is input. Further, the calculation method of the prediction error value will be described later (the prediction error value calculation method of the intra macroblock shown in FIG. 5 will be described below with respect to the prediction error value of the inter macroblock shown in FIGS. 7, 8, and 9). For the calculation method, see).
[0046]
The transform coefficient allocation code amount calculation means 3b includes input pixels (image data) from a video input, a hybrid coding block allocation code amount input from the hybrid coding block allocation code amount calculation means 3d, and a coding mode / motion. Based on the coding mode input from the vector generating means 3c (in the case of an inter macroblock, a motion vector is also input), the transform coefficient allocation code amount is calculated. That is, the transform coefficient allocation code amount calculation unit 3b repeatedly calculates the transform coefficient allocation code amount according to the number of encoding modes generated by the encoding mode / motion vector generation unit 3c. The transform coefficient allocation code amount is a code amount allocated to transform coefficients, that is, DCT coefficients.
[0047]
The transform coefficient allocation code amount calculation means 3b includes a code amount calculation function 3b1 (see FIG. 6 or 10) for calculating a code amount from input information, a hybrid coding block allocation code amount, and a code amount. Based on the code amount calculated by the calculation function 3b1, it has a transform coefficient allocation code amount calculation function 3b2 (see FIG. 6 or FIG. 10).
[0048]
If it is determined by the determination function of the encoding mode determination unit 3 that an intra macroblock is handled, the input pixel and the encoding mode are input to the transform coefficient allocation code amount calculation unit 3b, and the inter macroblock is converted into an inter macroblock. When it is determined that it is handled (when it is not determined that an intra macroblock is handled), an encoding mode and a motion vector are input. When it is determined that the intra macroblock is handled, the code amount calculation function 3b1 (see FIG. 6 or 10) of the transform coefficient allocation code amount calculation means 3b outputs the encoding mode code amount and the DC coefficient code amount. When it is determined that the inter macroblock is handled (when it is not determined that the intra macroblock is handled), the code amount calculation function 3b1 (see FIG. 6 or FIG. 10) of the transform coefficient allocation code amount calculation means 3b is The coding mode code amount and the motion vector code amount are output.
[0049]
The coding mode code amount is a code amount assigned for each coding mode, the DC coefficient code amount is a code amount assigned to the DC component of the input pixel, and the motion vector code amount is a motion vector in the input pixel. This is the amount of code allocated for each. Further, the calculation method of the transform coefficient allocation code amount will be described later (for the calculation method of the transform coefficient allocation code amount of the intra macroblock shown in FIG. 6, the transform coefficient allocation code amount of the inter macroblock shown in FIG. For the calculation method, see).
[0050]
The encoding mode / motion vector generating means 3c generates an encoding mode and a motion vector. The encoding mode is to be selected by the encoding mode selection means 3f, and includes a macroblock type (intra macroblock or intermacroblock), a prediction direction (forward prediction, backward prediction, bidirectional prediction), prediction type ( Field prediction or frame prediction) and DCT type (field DCT or frame DCT) are combined.
[0051]
As for the macro block type, an intra macro block is encoded by information of only a macro block, and an inter macro block predicts current image data from image data (encoded image data) encoded immediately before. It is to be encoded.
[0052]
With regard to the prediction direction, forward prediction is to predict from the past (frame or field immediately before encoding), backward prediction is to predict from the future (frame or field immediately after encoding), Bidirectional prediction is encoding (predicting) a macroblock to be encoded from the past and the future.
[0053]
With regard to the prediction type, field prediction is performed by independently predicting the top field (odd line) and the bottom field (even line) using the previously encoded field, and the frame prediction is encoded immediately before. Prediction is performed from the frame.
[0054]
Regarding the DCT type, the image data is divided into hybrid coding blocks (macroblock: luminance component 16 pixels × 16 lines and spatially corresponding color difference components), and further converted blocks (block: both luminance components and color difference components are 8 pixels × Field DCT is a line obtained from one of two fields to form the block, and frame DCT is a line obtained alternately from two fields. It constitutes a conversion block.
[0055]
The hybrid encoding block allocation code amount calculation means 3d divides the encoding screen allocation code amount output from the buffer control unit 25 by the number of hybrid encoding blocks (the number of macroblocks) to obtain each hybrid encoding block (macroblock). ) Is calculated and output to the transform coefficient allocation code amount calculation means 3b.
[0056]
The coding efficiency evaluation value calculation unit 3e is configured to code image data based on the prediction error value calculated by the prediction error value calculation unit 3a and the transform coefficient allocation code amount calculated by the transform coefficient allocation code amount calculation unit 3b. An evaluation value indicating the conversion efficiency is calculated, and this evaluation value is output to the encoding mode selection means 3f. In this embodiment, the coding efficiency evaluation value calculation means 3e calculates the product of the reciprocal of the prediction error value and the transform coefficient allocation code amount as the evaluation value.
[0057]
  Also, this coding efficiency evaluationvalueThe calculation means 3e includes coding screen summation functions 3e1 and 3e2 for summing up the transform coefficient allocation code amount or the prediction error value of the screen to be coded from the transform coefficient allocation code amount or the prediction error value of the hybrid coding block. Encoding efficiency calculation function 3e3 (see FIG. 11 and FIG. 12) for calculating an evaluation value based on the sum of transform coefficient allocation code amounts of screens to be calculated and the sum of prediction error values of screens to be encoded, It has a coding screen average function 3e4 that counts the coding efficiency of the hybrid coding block for each coding screen and outputs the average of the coding efficiency for each coding screen as the FPFD coding efficiency. The evaluation value calculation method will be described later.
[0058]
The encoding mode selection unit 3f selects an encoding mode based on the evaluation value calculated and output by the encoding efficiency evaluation value calculation unit 3e, and sends the selected encoding mode information to the motion compensation prediction unit 5. Output. In this embodiment, the encoding mode selection unit 3f selects the maximum (maximum evaluation value) among the evaluation values calculated and output by the encoding efficiency evaluation value calculation unit 3e.
[0059]
The encoding mode selection means 3f includes an encoding mode for arbitrarily selecting a hybrid encoding block (optional selection encoding mode) and an encoding mode for uniquely selecting a hybrid encoding block (unique selection encoding mode). And a selection function for selecting. This selection function compares the average value of the product of the reciprocal of the prediction error value of each hybrid encoding block constituting the encoding target image to be encoded and the transform coefficient allocation code amount, and this average value is maximized. A coding mode (an arbitrary coding mode or a unique selection coding mode) is selected. That is, in this case, the encoding mode selection unit 3f does not refer to the evaluation value calculated and output by the encoding efficiency evaluation value calculation unit 3e, but from the prediction error value calculation unit 3a and the transform coefficient allocation code amount calculation unit 3b. The prediction error value and the transform coefficient allocation code amount are obtained, and the average value of the products of the reciprocal of the prediction error value of each hybrid coding block and the transform coefficient allocation code amount is calculated and compared.
[0060]
That is, the selection function of the encoding mode selection means 3f is to determine the FPFD when the frame image is structured. When FPFD = 0, the prediction type can be arbitrarily selected from three types of frame prediction, field prediction, and dual prime prediction for each hybrid coding block (macroblock) (optional coding mode), and the DCT type is Any one of the two types of frame DCT and field DCT can be selected. When FPFD = 1, all hybrid coding blocks (macroblocks) on the coding screen (one screen) are uniquely selected with a prediction type of frame prediction and a DCT type of frame DCT (unique selection coding mode). It is. Note that the FPFD can be set for each encoded screen.
[0061]
The motion compensation prediction unit 5 holds the input pixel (image data) input from the video input and the frame memory 19 based on the encoding mode information output from the encoding mode selection unit 3 f of the encoding mode determination unit 3. A motion compensated prediction signal (an interframe prediction error signal in which motion is compensated in units of macroblocks) related to a P picture and a B picture from a reference pixel that is stored (stored), and a subtracter 7 and an adder 17 is output.
[0062]
The subtracter 7 outputs compensated prediction signal subtraction pixel data obtained by subtracting the motion compensation prediction signal from the input pixel input from the video input to the DCT unit 9.
[0063]
The DCT unit 9 divides the compensated prediction signal subtraction pixel data into transform block units, and based on the coding mode information output from the coding mode selection means 3f, DCT obtained by applying frame DCT and field DCT to the transform block The coefficient value is output to the quantizer 11. Note that the direct current component (intra DC) of the transform block of the intra macroblock is quantized with an accuracy of 8 to 11 bits, and the alternating current component is quantized with a quantization value given from the buffer control unit 25. Also, the conversion block of the inter macro block is quantized by the quantization value given from the buffer control unit 25 for both the DC component and the AC component.
[0064]
The quantizer 11 converts the DCT coefficient value output from the DCT unit 9 into a quantized value based on the encoded screen allocation code amount output from the buffer control unit 25, and the inverse quantizer 13 And it outputs to the variable-length encoding part 21.
[0065]
The inverse quantizer 13 inversely quantizes the quantized value quantized by the quantizer 11 to generate an original DCT coefficient value and outputs it to the IDCT unit 15.
[0066]
The IDCT unit 15 inversely converts the DCT coefficient value output from the inverse quantizer 13 into compensated prediction signal subtracted pixel data output from the subtractor 7 and outputs it to the adder 17.
[0067]
The adder 17 outputs a pixel (image data) obtained by adding the motion compensation prediction signal generated by the motion compensation prediction unit 5 to the compensated prediction signal subtraction pixel data output from the IDCT unit 15 to the frame memory 19 as a reference pixel. To do.
[0068]
The frame memory 19 stores the reference pixels output from the adder 17 in units of frames, and when a new input pixel is input from the video input, the prediction error value calculation means 3a of the encoding mode determination unit 3 and A reference pixel is output to the motion compensation prediction unit 5.
[0069]
The variable length code unit 21 is a two-dimensional variable in which one variable length code is assigned to the combination of the number of coefficient values 0 and the next coefficient value to the quantized value output from the quantizer 11. Long coding is performed to generate a bit stream.
[0070]
  Buffer 23 is a variable length markPartThe bit stream encoded in 21 is temporarily stored and then output to the outside.
[0071]
Based on the bit stream output from the buffer 23, the buffer control unit 25 calculates a coding screen allocation code amount that is a code amount that can be allocated to one screen (one frame), and performs coding mode determination unit 3 and quantum Is output to the generator 11. The encoded screen allocation code amount corresponds to the encoded bit rate. Also, the motion compensation prediction unit 5, the subtractor 7, the DCT unit 9, the quantizer 11, the inverse quantizer 13, the IDCT unit 15, the adder 17, the frame memory 19, the variable length coding unit 21, the buffer 23, and buffer control The unit 25 corresponds to the encoding means described in the claims.
[0072]
According to the image encoding device 1, the prediction error value calculation unit 3a of the encoding mode determination unit 3 calculates a prediction error value when encoding an input pixel (image data), and the prediction error value is encoded. Is output to the efficiency evaluation value calculation means 3e. Further, the transform coefficient allocation code amount calculation means 3b calculates the transform coefficient allocation code amount allocated to the transform coefficient when the input pixel (image data) is encoded. The encoding efficiency evaluation value calculation means 3e calculates the product of the reciprocal of the prediction error value and the transform coefficient allocation code amount as the evaluation value, and the encoding mode selection means 3f calculates the maximum value of this evaluation value ( The encoding mode based on the maximum evaluation value) is selected. Then, motion compensation prediction unit 5, subtractor 7, DCT unit 9, quantizer 11, inverse quantizer 13, IDCT unit 15, adder 17, frame memory 19, variable length coding unit 21, buffer 23 and buffer control The unit 25 encodes the image data in the encoding mode selected by the encoding mode selection unit 3f. Therefore, a coding mode is selected based on an evaluation value that maximizes the product of the reciprocal of the prediction error value and the transform coefficient allocation code amount, and this encoding mode is the encoded screen allocation code amount (encoding bit rate). ), The optimum encoding mode is selected even when the encoding bit rate is low, and deterioration of the image quality of the encoded image data can be suppressed.
[0073]
(Operation of image coding apparatus)
Next, the operation of the image coding apparatus 1 will be described with reference to the flowchart shown in FIG. 2 (see FIG. 1 as appropriate). First, input pixels (image data) are input from the video input, and the input pixels (image data) are input to the encoding mode determination unit 3, the motion compensation prediction unit 5, and the subtractor 7 (S1).
[0074]
Subsequently, the motion compensated prediction signal output from the motion compensated prediction unit 5 is subtracted from the input pixel input to the subtractor 7 and input to the DCT unit 9 as compensated prediction signal subtracted pixel data. The DCT unit 9 sequentially converts the DCT coefficient value to the quantized value, the quantizer 11 to the quantized value, the inverse quantizer 13 to the DCT coefficient value, and the IDCT unit 15 to the compensated prediction signal subtracted pixel data sequentially. The Then, the adder 17 adds the motion compensated prediction signal to the compensated prediction signal subtraction pixel data, and holds (accumulates) it in the frame memory 19 as the previous input pixel (reference pixel). Through this frame memory 19, reference pixels are input to the prediction error value calculation means 3a and the motion compensation prediction unit 5 of the encoding mode determination unit 3 (S2).
[0075]
The quantized value converted by the quantizer 11 is input to the variable length coding unit 21, a bit stream is generated by the variable length coding unit 21, and input to the buffer control unit 25. Based on the input bitstream, the encoded screen allocation code amount is calculated, and this encoded screen allocation code amount is input to the encoding mode determination unit 3 and the quantizer 11 (S3).
[0076]
Then, each process (refer to the description of the operation of the coding mode determination unit 3 for details) is performed inside the coding mode determination unit 3, and coding mode (macroblock mode) information is input to the motion compensation prediction unit 5. (S4). Then, based on the coding mode (macroblock mode) information, the input pixel (image data) is stored in the motion compensation prediction unit 5, the subtractor 7, the DCT unit 9, the quantizer 11, the variable length coding unit 21, and the buffer 23. Then, it is output to the outside as a bit stream (S5).
[0077]
(Operation of coding mode determination unit)
Next, the operation of the encoding mode determination unit 3 of the image encoding device 1 will be described with reference to the flowchart shown in FIG. First, it is determined whether or not an intra macroblock is handled (S11). Alternatively, this S11 may determine whether to handle an inter macro block.
[0078]
When it is determined that the intra macroblock is handled (S11, Yes), the input pixel is input to the prediction error value calculation unit 3a and the transform coefficient allocation code amount calculation unit 3b (S12). Also, the encoding mode (information relating to the encoding mode to be selected) generated by the encoding mode / motion vector generating means 3c is the prediction error value calculating means 3a, the transform coefficient allocation code amount calculating means 3b, and the encoding mode selection. It is input to the means 3f (S13).
[0079]
Further, the hybrid coding block allocation code amount calculation means 3d calculates a hybrid coding block allocation code amount based on the coding screen allocation code amount input from the buffer control unit 25, and this hybrid coding block allocation code The amount is output to the transform coefficient allocation code amount calculation means 3b (S14).
[0080]
Then, the prediction error value calculation unit 3a calculates a prediction error value based on the input pixel and the encoding mode, and outputs the prediction error value to the encoding efficiency evaluation value calculation unit 3e (S15). In addition, the transform coefficient allocation code amount calculation unit 3b, based on the input pixel and the encoding mode, encodes the coding mode code amount in the coding mode generated by the coding mode / motion vector generation unit 3c and the direct current of the input pixel. A DC coefficient code amount, which is a code amount when the component is encoded, is calculated. Further, the transform coefficient allocation code amount calculation means 3b is based on the coding mode code amount and DC coefficient code amount and the hybrid coding block allocation code amount calculated by the hybrid coding block allocation code amount calculation means 3d. Thus, the transform coefficient allocation code amount is calculated and output to the encoding efficiency evaluation value calculation means 3e (S16).
[0081]
Then, in the coding efficiency evaluation value calculation means 3e, the code is calculated based on the prediction error value calculated by the prediction error value calculation means 3a and the transform coefficient allocation code amount calculated by the transform coefficient allocation code amount calculation means 3b. An evaluation value indicating the conversion efficiency is calculated, and this evaluation value is output to the encoding mode selection means 3f (S17). Although not shown in the flowchart shown in FIG. 3, in the operation of the coding efficiency evaluation value calculation means 3e, when the FPFD flag is “1”, that is, when only frame prediction and frame DCT are used. Then, the FPFD encoding efficiency (an evaluation value indicating the encoding efficiency for each encoding screen) in this FPFD is calculated. When the coding efficiency of each hybrid coding block is input, the FPFD coding efficiency (an evaluation value indicating the coding efficiency for each coding screen) is calculated on an average for each coding screen.
[0082]
Thereafter, the encoding mode selection means 3f selects an encoding mode (becomes encoding mode information) based on the evaluation value calculated by the encoding efficiency evaluation value calculation means 3e. It is output to the motion compensation prediction unit 5 (S18). Although not shown in the flowchart shown in FIG. 3, a coding mode (optional coding mode) in which a hybrid coding block is arbitrarily selected by a selection function of the coding mode selection unit 3f, and a hybrid coding block When a coding mode (unique selection coding mode) is selected, the average value of the product of the reciprocal of the prediction error value of each hybrid coding block and the transform coefficient allocation code amount is compared. The encoding mode with the maximum average value is selected.
[0083]
If it is not determined in S11 that the intra macroblock is handled (S11, No), the input pixel is input to the prediction error value calculation unit 3a (S19). Also, the coding mode (information related to the coding mode to be selected) and the motion vector generated by the coding mode / motion vector generating means 3c are transferred to the prediction error value calculating means 3a and the transform coefficient allocation code amount calculating means 3b. The encoding mode (information regarding the encoding mode to be selected) is input to the encoding mode selection unit 3f (S20).
[0084]
Further, the hybrid coding block allocation code amount calculation means 3d calculates a hybrid coding block allocation code amount based on the coding screen allocation code amount input from the buffer control unit 25, and this hybrid coding block allocation code The amount is output to the transform coefficient allocation code amount calculation means 3b (S21).
[0085]
The prediction error value calculation unit 3a calculates a prediction error value based on the input pixel, the reference pixel, the encoding mode, and the motion vector, and outputs the prediction error value to the encoding efficiency evaluation value calculation unit 3e. (S22). In addition, the transform coefficient allocation code amount calculation means 3b, based on the coding mode and motion vector, the coding mode code amount, the input pixel and the reference in the coding mode generated by the coding mode / motion vector generation means 3c. A motion vector code amount that is a code amount based on a motion vector obtained from a pixel is calculated. Further, in the transform coefficient allocation code amount calculation means 3b, based on the coding mode code amount and motion vector code amount and the hybrid coding block allocation code amount calculated by the hybrid coding block allocation code amount calculation means 3d. The transform coefficient allocation code amount is calculated and output to the encoding efficiency evaluation value calculation means 3e (S23).
[0086]
Then, in the coding efficiency evaluation value calculation means 3e, the code is calculated based on the prediction error value calculated by the prediction error value calculation means 3a and the transform coefficient allocation code amount calculated by the transform coefficient allocation code amount calculation means 3b. An evaluation value indicating the conversion efficiency is calculated, and this evaluation value is output to the encoding mode selection means 3f (S17). Thereafter, the encoding mode selection means 3f selects an encoding mode (becomes encoding mode information) based on the evaluation value calculated by the encoding efficiency evaluation value calculation means 3e. It is output to the motion compensation prediction unit 5 (S18).
[0087]
(Encoding mode [macroblock mode], determination of motion vector)
Next, with reference to FIG. 4, a description will be given of how to determine a coding mode [macroblock mode] and a motion vector. As shown in FIG. 4, when the prediction error value and the transform coefficient allocation code amount are input to the encoding efficiency evaluation value calculation unit 3e, the evaluation value is calculated and the evaluation value is output to the encoding mode selection unit 3f. Is done.
[0088]
The evaluation value in the coding efficiency evaluation value calculation means 3e is the evaluation value E_MM, Transform coefficient allocation code amount (DCT coefficient allocation code amount) T_CWhen the prediction error value D is assumed, this evaluation value E_MMIs calculated by the following equation.
E_MM= T_C/ D (1)
[0089]
As shown in the equation (1), the evaluation value E_MMIs the reciprocal of the prediction error value D and the transform coefficient allocation code amount (DCT coefficient allocation code amount) T_CCan be represented by the product of. That is, in this encoding mode selection means 3f, the evaluation value E calculated by the equation (1)._MMA combination of a coding mode (macroblock mode) and a motion vector that maximizes the value is selected.
[0090]
(Calculation method of prediction error value of intra macroblock)
Next, a calculation method of a prediction error value of an intra macroblock will be described with reference to FIG. 5 (see FIG. 1 as appropriate). When an intra macroblock is processed by the encoding mode determination unit 3 and a prediction error value is calculated, as shown in FIG. 5, the encoding mode and the input pixel are input to the prediction error value calculation means 3a, and the prediction error value calculation is performed. Within the means 3a, a transform block pixel variance is obtained, and this transform block pixel variance is output as a prediction error value. When this prediction error value is a prediction error value D, the calculation is performed according to the following equation.
[0091]
[Expression 1]

[0092]
... (2)
In this equation (2), VAR represents the variance of input pixels in the transform block (DCT block), and block_count represents the number of transform blocks (DCT blocks) constituting the macro block.
[0093]
As shown in the equation (2), the prediction error value D is obtained by calculating the variance of the input pixels of the transform block (DCT block) in the macroblock and taking the average of the variances of the input pixels.
[0094]
According to this, the prediction error value D in the encoding mode in which the prediction error value calculation unit 3a encodes the conversion block obtained by dividing the input pixel (image data) into a predetermined size based on the DC coefficient is converted into the conversion block. Is the average of the variance of the input pixels. That is, variance is obtained for each transform block, and the average of transform blocks constituting the hybrid coding block is calculated as a prediction error value, and this prediction error value D is used to calculate an evaluation value indicating coding efficiency. The encoding mode is selected based on the evaluation value, and the encoding mode changes depending on the increase / decrease of the encoding screen allocation code amount (encoding bit rate). Therefore, even if the encoding bit rate decreases, the optimal An encoding mode is selected, and image quality deterioration of encoded image data can be suppressed.
[0095]
(Intra macroblock transform coefficient allocation code amount calculation method)
Next, with reference to FIG. 6, a method for calculating the transform coefficient allocation code amount of the intra macroblock will be described (see FIG. 1 as appropriate). When the intra macroblock is processed by the encoding mode determination unit 3 and the transform coefficient allocation code amount is calculated, the encoding mode and the input pixel are encoded in the transform coefficient allocation code amount calculation means 3b as shown in FIG. It is input to the quantity calculation function 3b1, and the coding mode code quantity and the DC coefficient code quantity are obtained. Subsequently, the coding mode code amount, the DC coefficient code amount, and the hybrid coding block allocation code amount are input to the transform coefficient allocation code amount calculation function 3b2, and the transform coefficient allocation code amount is obtained and output. This transform coefficient allocation code amount is converted into a transform coefficient allocation code amount T._CThen, it is calculated by the following formula.
[0096]
[Expression 2]

[0097]
... (3)
In this equation (3), T_MIs a hybrid coding block allocation code amount (macroblock allocation code amount), S_MMIs the amount of generated code in the coding mode (macroblock mode), S_DCRepresents the amount of generated code of the DC coefficient (intra DC coefficient) of the transform block (DCT block), and block_count represents the number of transform blocks (DCT blocks) constituting the macro block.
[0098]
As shown in the equation (3), the transform coefficient allocation code amount T_CIs a hybrid coding block allocation code amount (macroblock allocation code amount) T_MTo encoding mode (macroblock mode) generated code amount S_MMAnd the generated code amount S of the DC coefficient (intra DC coefficient) of the transform block (DCT block)_DCAre subtracted.
[0099]
According to this, the transform coefficient assignment code amount T in the coding mode in which the transform block obtained by dividing the input pixel (image data) into a predetermined size is coded based on the DC coefficient by the transform coefficient assignment code amount calculation means 3b._CIs the hybrid coding block allocation code amount T_MAnd a generated code amount S generated when the transform block is encoded with a DC coefficient._DCBased on the above. This conversion coefficient allocation code amount T_CIs used to calculate an evaluation value indicating encoding efficiency, and an encoding mode is selected based on the evaluation value, and this encoding mode changes according to an increase / decrease in the encoding screen allocation code amount (encoding bit rate). Even when the encoding bit rate is lowered, the optimum encoding mode is selected, and deterioration of the image quality of the encoded image data can be suppressed.
[0100]
(Regarding Calculation Method of Intermacroblock Prediction Error Value [Mean Square Error]) Next, a calculation method of the prediction error value of the intermacroblock will be described with reference to FIG. 7 (see FIG. 1 as appropriate). When the inter macroblock is processed by the encoding mode determination unit 3 and the prediction error value is calculated, as shown in FIG. 7, the encoding mode, the motion vector, the input pixel, and the reference pixel are input to the prediction error value calculation unit 3a. Then, a mean square error is calculated in the prediction error value calculation means 3a, and this mean square error is output as a prediction error value. When this prediction error value is a prediction error value D, the calculation is performed according to the following equation.
[0101]
[Equation 3]

[0102]
... (4)
In this equation (4), M₁(I, j) is the mean square error between the input pixel of the hybrid coding block (macroblock) and the reference pixel due to motion compensation. Xm and n are pixels at the position of horizontal m and vertical n in the current frame / field, and Xm + i and n + j are pixels at the position of horizontal m + i and vertical n + j in the reference frame / field. As shown in the equation (4), the prediction error value D is an average of values obtained by squaring the error between the input pixel and the reference pixel in the hybrid coding block.
[0103]
According to this, the prediction error value D in the encoding mode in which the prediction error value calculation unit 3a encodes the input pixel (image data) by motion compensation is used as the input pixel of the hybrid encoding block at the time of hybrid encoding. A value obtained by squaring the error from the reference pixel is an average square error obtained by averaging the values in the hybrid coding block. The calculation of the mean square error corresponds to the calculation of energy on the pixel by calculating a value obtained by squaring the error between the input pixel and the reference pixel, and is the same as the energy on the frequency in transform coding using orthogonal transform. Superiority and inferiority can be compared in dimension, and the accuracy of the prediction error value is improved. The prediction error value D is used to calculate an evaluation value indicating the encoding efficiency, and an encoding mode is selected based on the evaluation value. The encoding mode is an encoding screen allocation code amount (encoding bit rate). Since it changes depending on the increase / decrease, even when the encoding bit rate is lowered, the optimum encoding mode is selected, and the image quality deterioration of the encoded image data can be suppressed.
[0104]
(Inter macro block prediction error value calculation method [average absolute value])
Next, with reference to FIG. 8, a method for calculating the prediction error value of the inter macroblock will be described (see FIG. 1 as appropriate). When the inter-macroblock is processed by the encoding mode determination unit 3 and the prediction error value is calculated, as shown in FIG. 8, the encoding mode, the input pixel, and the reference pixel are input to the prediction error value calculation unit 3a. In the error value calculation means 3a, an average absolute value error is calculated, and this average absolute value error is output as a prediction error value. When this prediction error value is a prediction error value D, the calculation is performed according to the following equation.
[0105]
[Expression 4]

[0106]
... (5)
In this equation (5), M₂(I, j) is an average absolute value error between the input pixel of the hybrid coding block (macro block) and the reference pixel by motion compensation. Xm and n are pixels at the position of horizontal m and vertical n in the current frame / field, and Xm + i and n + j are pixels at the position of horizontal m + i and vertical n + j in the reference frame / field. As shown in the equation (5), the prediction error value D is an average absolute value error between the input pixel and the reference pixel.
[0107]
  According to this, the prediction error value D in the encoding mode in which the prediction error value calculation unit 3a encodes the input pixel (image data) by motion compensation is used as the input pixel of the hybrid encoding block at the time of hybrid encoding. The absolute value of the error from the reference pixel is a value obtained by averaging the hybrid coding block. Prediction error value calculation means by taking the average absolute value error that only takes the average of absolute values as the prediction error value3aThe calculation amount is reduced, and the processing speed is improved. The prediction error value D is used to calculate an evaluation value indicating the encoding efficiency, and an encoding mode is selected based on the evaluation value. The encoding mode is an encoding screen allocation code amount (encoding bit rate). Since it changes depending on the increase / decrease, even when the encoding bit rate is lowered, the optimum encoding mode is selected, and the image quality deterioration of the encoded image data can be suppressed.
[0108]
(Regarding Calculation Method of Inter-macroblock Prediction Error Value [Cross-Correlation Coefficient]) Next, the calculation method of the inter-macroblock prediction error value will be described with reference to FIG. 9 (see FIG. 1 as appropriate). When the inter-macroblock is processed by the encoding mode determination unit 3 and the prediction error value is calculated, the encoding mode, the input pixel, and the reference pixel are input to the prediction error value calculation unit 3a as shown in FIG. In the error value calculation means 3a, a cross-correlation coefficient is calculated, and this cross-correlation coefficient is output as a prediction error value. When this prediction error value is a prediction error value D, the calculation is performed according to the following equation.
[0109]
[Equation 5]

[0110]
... (6)
In this equation (6), M_Three(I, j) is a cross-correlation coefficient between the input pixel of the hybrid coding block (macroblock) and the reference pixel by motion compensation. Xm and n are pixels at the position of horizontal m and vertical n in the current frame / field, and Xm + i and n + j are pixels at the position of horizontal m + i and vertical n + j in the reference frame / field. As shown in the equation (6), the prediction error value D is a cross-correlation coefficient between the input pixel and the reference pixel.
[0111]
According to this, the prediction error value D in the encoding mode in which the prediction error value calculation unit 3a encodes the input pixel (image data) by motion compensation is used as the input pixel of the hybrid encoding block at the time of hybrid encoding. The cross correlation coefficient with the reference pixel is used. Since the cross correlation coefficient obtained by comparing the input pixel and the reference pixel and calculating the correlation coefficient is used as the prediction error value, the accuracy of the prediction error value is improved. The prediction error value D is used to calculate an evaluation value indicating the encoding efficiency, and an encoding mode is selected based on the evaluation value. The encoding mode is an encoding screen allocation code amount (encoding bit rate). Since it changes depending on the increase / decrease, even when the encoding bit rate is lowered, the optimum encoding mode is selected, and the image quality deterioration of the encoded image data can be suppressed.
[0112]
(Regarding calculation method of DCT coefficient allocation code amount of inter macro block)
Next, with reference to FIG. 10, a method of calculating the transform coefficient allocation code amount of the inter macroblock will be described (see FIG. 1 as appropriate). When the inter-macroblock is processed by the encoding mode determination unit 3 and the transform coefficient allocation code amount is calculated, the encoding mode and the motion vector are encoded in the transform coefficient allocation code amount calculation means 3b as shown in FIG. It is input to the quantity calculation function 3b1, and the encoding mode code quantity and motion vector code quantity are obtained. Subsequently, the coding mode code amount, the motion vector code amount, and the hybrid coding block allocation code amount are input to the transform coefficient allocation code amount calculation function 3b2, and the transform coefficient allocation code amount is obtained and output. The transform coefficient allocation code amount is converted into the transform coefficient allocation code amount T._CThen, it is calculated by the following formula.
[0113]
T_C= T_M-S_MM-S_MV
[0114]
... (7)
In this equation (7), T_MIs a hybrid coding block allocation code amount (macroblock allocation code amount), S_MMIs the amount of generated code in the coding mode (macroblock mode), S_MVIndicates the generated code amount of the motion vector.
[0115]
As shown in the equation (7), the transform coefficient allocation code amount T_CIs a hybrid coding block allocation code amount (macroblock allocation code amount) T_MTo encoding mode (macroblock mode) generated code amount S_MMAnd the generated code amount S of the motion vector_MVAre subtracted.
[0116]
According to this, the transform coefficient allocation code amount calculation means 3b performs transform coefficient allocation code amount T in a coding mode in which image data is encoded by motion compensation._CIs the hybrid coding block allocation code amount T_MAnd the generated code amount S in the coding mode (macroblock mode)_MMAnd a generated code amount S generated when encoding with a motion vector used for motion compensation_MVBased on the above. This conversion coefficient allocation code amount T_CIs used to calculate an evaluation value indicating encoding efficiency, and an encoding mode is selected based on the evaluation value, and this encoding mode changes according to an increase / decrease in the encoding screen allocation code amount (encoding bit rate). Even when the encoding bit rate is lowered, the optimum encoding mode is selected, and deterioration of the image quality of the encoded image data can be suppressed.
[0117]
(How to determine FPFD [part 1])
Next, with reference to FIG. 11, FPFD (encoding parameter) which is a screen encoding mode for arbitrarily selecting or uniquely selecting a transform block to be encoded in a hybrid encoding block (macro block) will be described. To do. FIG. 11 is a diagram illustrating a case where the encoding efficiency of the entire screen is calculated to determine the FPFD (encoding parameter).
[0118]
As shown in FIG. 11, the transform coefficient allocation code amount of the hybrid coding block is input to the coding screen total function 3e1 of the coding efficiency evaluation value calculation means 3e, and the hybrid coding block of the hybrid coding block is input to the coding screen total function 3e2. A prediction error value is input. From the coding screen total function 3e1, the coding coefficient allocation code amount of the coding screen is output, and the prediction error value of the coding screen is output from the coding screen total function 3e2, and is input to the coding efficiency calculation function 3e3. An evaluation value representing the FPFD coding efficiency (evaluation value representing the average coding rate of the coding screen) is calculated by the calculation function 3e3, and the evaluation value representing the calculated FPFD coding efficiency (the average code of the coding screen) (Evaluation value representing the conversion rate) is output to the encoding mode selection means 3f. An evaluation value representing this FPFD coding efficiency (an evaluation value representing the average coding rate of the coding screen) is used as an evaluation value E._FThen, it is calculated by the following formula.
[0119]
[Formula 6]

[0120]
... (8)
  In this equation (8),D is the prediction error value, T _C Is transform coefficient code amount, MB_count indicates the number of hybrid coding blocks (macroblocks) constituting the coding screen.
[0121]
  As shown in this equation (8), an evaluation value E representing the coding efficiency of the coding screen._FFor each hybrid coding block (macroblock)The sum of the prediction error values D for each coding screen is calculated as the transform coefficient code amount T. _C Total for each encoding screenDivided by. Then, an evaluation value E representing the coding rate of the screen of the coding screen average_FThe FPFD (encoding parameter) that maximizes is selected by the encoding mode selection means 3f.
[0122]
According to this, the evaluation value E calculated by the encoding efficiency evaluation value calculation means 3e._FIs selected by the encoding mode selection means 3f, and the FPFD (encoding parameter) changes according to the increase / decrease of the encoding screen allocation code amount (encoding bit rate). Even when the rate is lowered, an optimal encoding mode is selected, and deterioration in image quality of the encoded image data can be suppressed.
[0123]
(How to determine FPFD [2])
Next, referring to FIG. 12, another transform block to be encoded in a hybrid encoding block (macroblock) is arbitrarily selected, or another FPFD (encoding parameter) that is a screen encoding mode that is uniquely selected. The determination method will be described. FIG. 12 is a diagram for explaining a case where FPFD (encoding parameter) is determined by calculating encoding efficiency for each hybrid encoding block.
[0124]
As shown in FIG. 12, the encoding efficiency of the hybrid coding block is input to the encoding screen average function 3e4 of the encoding efficiency evaluation value calculation means 3e, and the FPFD encoding efficiency is expressed by this encoding screen average function 3e4. An evaluation value (an evaluation value representing the average coding rate of the coded screen) is calculated, and an evaluation value representing the calculated FPFD coding efficiency (an evaluation value representing the average coding rate of the coded screen) is encoded. It is output to the mode selection means 3f. An evaluation value representing this FPFD coding efficiency (an evaluation value representing the average coding rate of the coding screen) is used as an evaluation value E._FThen, it is calculated by the following formula.
[0125]
[Expression 7]

[0126]
... (9)
  In this equation (9),E _MM Is an evaluation value indicating the coding efficiency of each hybrid coding block (macroblock), MB_count indicates the number of hybrid coding blocks (macroblocks) constituting the coding screen.
[0127]
  As shown in the equation (9), an evaluation value E representing the coding rate of the screen of the coding screen average._FFor each hybrid coding block (macroblock)Evaluation value E indicating coding efficiency _MM The total number of hybrid coding blocks (macroblocks)Divided by. Then, an evaluation value E representing the coding rate of the screen of the coding screen average_FThe FPFD (encoding parameter) that maximizes is selected by the encoding mode selection means 3f.
[0128]
According to this, the evaluation value E calculated by the encoding efficiency evaluation value calculation means 3e._FSince the FPFD (encoding parameter) that maximizes the value is selected by the encoding mode selection means 3f, the FPFD (encoding parameter) changes with the increase / decrease of the encoding screen allocation code amount (encoding bit rate). Even when the encoding bit rate is lowered, the optimum encoding mode is selected, and the image quality deterioration of the encoded image data can be suppressed.
[0129]
(About calculation method of macroblock allocation code amount)
Next, with reference to FIG. 13, a method for calculating the allocated code amount of the hybrid coding block (macroblock) will be described. FIG. 13 is a diagram for explaining how to calculate the code amount of the hybrid coding block (macroblock) by the buffer control unit 25 and the hybrid coding block allocation code amount calculation means 3d.
[0130]
As shown in FIG. 13, an encoded screen allocation code amount, which is a code amount allocated to each encoded screen, is input from the buffer control unit 25 to the hybrid encoding block allocation code amount calculation means 3d. The allocated code amount calculation means 3d calculates the hybrid encoded block allocated code amount allocated to each hybrid encoded block. This hybrid coding block allocation code amount is converted to a hybrid coding block allocation code amount T._MThen, it is calculated by the following formula.
[0131]
[Equation 8]

[0132]
... (10)
In this equation (10), T_PIndicates the encoded screen allocation code amount given from the buffer control unit 25, and MB_count indicates the number of hybrid encoded blocks (macroblocks) constituting the encoded screen.
[0133]
As shown in the equation (10), the hybrid coding block allocation code amount T_MIs the encoded screen allocation code amount T given from the buffer control unit 25_PIs divided by the number of hybrid coding blocks (macroblocks).
[0134]
According to this, the encoded screen allocation code amount T given from the buffer control unit 25._PIncrease / decrease in the hybrid coding block allocation code amount T_MTherefore, even when the encoding bit rate is lowered, the optimum encoding mode is selected, and the image quality deterioration of the encoded image data can be suppressed.
[0135]
(Regarding the effect of the image encoding device)
Finally, the remarkable effects confirmed by the image encoding device 1 as a result of performing the encoding simulation on the image encoding device 1 and TM5 described in the paragraph of the prior art will be described. The configuration of the buffer control unit 25 is the same as that of TM5 (encoder, see the description of the prior art).
[0136]
For the ITE high-definition standard moving images “European Market”, “Whole Show”, and “Green Leaves”, the encoding bit rate in the image encoding device 1 is 11, 15 Mbps, and the encoding bit rate in TM5 is 11, 15, 22 Mbps. PSNR (Peak Signal to Noise Ratio) was compared.
[0137]
Then, as a result of comparison with TM5 of the same bit rate, improvement in image quality of about 1 to 3 dB can be confirmed in any image (scene), and the PSNR when the encoding bit rate of the image encoding device 1 is 15 Mbps is It was confirmed that the PSNR was a TM5 encoding bit rate of 22 Mbps or more.
[0138]
In addition, the viewer who has viewed the bitstream encoded by the image encoding device 1 visually improves the resolution in all images (scenes), and in particular, the image quality improvement effect is high for fast moving images. It was confirmed.
[0139]
Further, for a bitstream encoded at 15 Mbps, when comparing the ratio between the transform coefficient code amount and the motion vector code amount in the image encoding device 1 and the ratio between the transform coefficient code amount and the motion vector code amount in TM5, In any image (scene), the motion vector code amount is reduced to 1/5 to 1/6, it can be confirmed that the transform coefficient code amount is increased, and a large amount of code amount is assigned to the transform coefficient. It was confirmed that the image quality was improved.
[0140]
As mentioned above, although this invention was demonstrated based on one Embodiment, this invention is not limited to this.
[0141]
For example, it can be regarded as an image encoding method in which the processing of each component of the image encoding device 1 is regarded as one process. The processing of each component of the image encoding device 1 can be regarded as an image encoding program described in a general-purpose computer language. In these cases, the same effects as those of the image encoding device 1 can be obtained. Further, in the case of an image encoding program, the program is distributed to a network or stored in a storage medium that can store the program. It is also possible to make it.
[0142]
  Claims 1, 3,4According to the described invention, the error value at the time of encoding the image data and the transform coefficient allocation code amount allocated to the transform coefficient are calculated. An evaluation value is calculated based on the error value and the transform coefficient allocation code amount, an encoding mode based on the evaluation value is selected, and image data is encoded using the selected encoding mode. For this reason, since the transform coefficient allocation code amount changes as the coding bit rate increases / decreases, the optimum coding mode is selected even when the coding bit rate is low, and image quality deterioration of the coded image data is suppressed. be able to.
[0143]
  Also,Claim1, 3, 4According to the described invention, the image data is divided into a predetermined size.Included in hybrid transform blockObtain the variance for each transform block and configure a hybrid coding blockconversionThe average of the block variance isIntra coding modeSince this error value is used and an optimal encoding mode is selected, image quality deterioration of encoded image data can be suppressed.
[0144]
  Also,Claim1, 3, 4According to the described invention, the mean square error obtained by averaging the value obtained by squaring the error between the input pixel and the reference pixel by the hybrid coding block is obtained.In inter coding modeBy using the error value, it is possible to compare the superiority and inferiority in the same dimension as the energy on the frequency in transform coding using orthogonal transform, and this error value is used to select the optimum coding mode. It is possible to suppress image quality deterioration of the converted image data.
[0145]
  Also,Claim1, 3, 4According to the described invention, the average absolute value error obtained by averaging the absolute value of the error between the input pixel and the reference pixel by the hybrid coding block is calculated.In inter coding modeBy using the error value, it is possible to reduce the amount of calculation in the prediction error value calculation means, and since this error value is used and the optimum encoding mode is selected, the image quality of the encoded image data is deteriorated. Can be suppressed.
[0146]
  Also,Claim1, 3, 4According to the described invention, the accuracy is improved by the cross-correlation coefficient obtained by comparing the input pixel and the reference pixel and calculating the correlation coefficient.In inter coding modeAn error value can be obtained, and since this error value is used and an optimal encoding mode is selected, image quality deterioration of encoded image data can be suppressed.
[0147]
  Claim2According to the described invention, the allocated code amount of the hybrid coding block is the code amount calculated by the number of hybrid coding blocks constituting the screen included in the image data. Since the allocated code amount of the encoded block also increases / decreases, even when the encoding bit rate is lowered, the optimal encoding mode is selected, and deterioration of the image quality of the encoded image data can be suppressed.
[Brief description of the drawings]
FIG. 1 is a block diagram of an image encoding apparatus according to an embodiment of the present invention.
FIG. 2 is a flowchart illustrating an operation of the image encoding device.
FIG. 3 is a flowchart illustrating an operation of an encoding mode determination unit.
FIG. 4 is a diagram for explaining a method for determining an encoding mode.
FIG. 5 is a diagram illustrating a calculation method of a prediction error value of an intra macroblock.
[Fig. 6] Fig. 6 is a diagram for describing a method for calculating a transform coefficient allocation code amount of an intra macroblock.
FIG. 7 is a diagram for explaining a calculation method (mean square error) of a prediction error value of an inter macroblock.
FIG. 8 is a diagram illustrating a calculation method (average absolute value error) of a prediction error value of an inter macroblock.
FIG. 9 is a diagram illustrating a calculation method (cross-correlation coefficient) of a prediction error value of an inter macroblock.
[Fig. 10] Fig. 10 is a diagram for describing a method of calculating a transform coefficient allocation code amount of an inter macroblock.
FIG. 11 is a diagram for explaining a FPFD determination method (entire encoding screen).
FIG. 12 is a diagram for explaining a FPFD determination method (hybrid coding block);
FIG. 13 is a diagram for explaining a method of calculating a hybrid coding block (macroblock) allocation code amount.
FIG. 14 is a block diagram of a conventional image encoding device.
[Explanation of symbols]
1 Image encoding device
3 Coding mode decision unit
3a Prediction error value calculation means
3b Transform coefficient allocation code amount calculation means
3c Coding mode / motion vector generating means
3d hybrid coding block allocation code amount calculation means
3e Coding efficiency evaluation value calculation means
3f encoding mode selection means
5 Motion compensation prediction unit
7 Subtractor
9 DCT section
11 Quantizer
13 Inverse quantizer
15 IDCT section
17 Adder
19 frame memory
21 Variable length code part
23 Buffer
25 Buffer controller

Claims (4)

An image encoding device that encodes image data by hybrid encoding combining prediction encoding using motion compensated prediction based on motion vectors and transform encoding using orthogonal transform,
Examples error value when encoding the image data by predictive coding, hybrid coding blocks obtained by dividing the image data into a predetermined size, the error value D ₁ in the intra-coding mode, the hybrid coding was calculated by the average of the variance of the input pixels of the transformation block included in the block, the error value D ₂ in the inter coding mode, the mean square error between the reference pixel of the input pixel and the block of the hybrid coding blocks, the average absolute A prediction error value calculation means for calculating a value error or a cross-correlation coefficient;
Wherein the hybrid coding blocks obtained by dividing the image data into a predetermined size, the intra transform coefficients assigned code amount T _C1 in the coding mode, the hybrid coding hybrid coding blocks allocated to hybrid coding blocks when reduction Obtained by subtracting the generated code amount of the intra coding mode and the sum of the generated code amounts of the DC coefficients of the transform blocks included in the hybrid coding block from the allocated code amount, and converting in the inter coding mode Transform coefficient allocation code amount calculation means for obtaining the coefficient allocation code amount T _C2 by subtracting the generated code amount of the inter coding mode and the generated code amount of the motion vector from the hybrid encoded block allocation code amount When,
As the evaluation value E ₁ indicating the coding efficiency in the intra coding mode, the reciprocal number of the error value D ₁ calculated by the prediction error value calculation unit and the transform coefficient allocation code amount calculated by the transform coefficient allocation code amount calculation unit The product of T _C1 is calculated, and the evaluation value E ₂ indicating the coding efficiency in the inter coding mode is used as the evaluation value E ₂ , the reciprocal of the error value D ₂ calculated by the prediction error value calculation means, and the transform coefficient allocation code amount calculation Coding efficiency evaluation value calculation means for calculating an evaluation value indicating coding efficiency by calculating a product of the transform coefficient allocation code amount _TC2 calculated by the means;
Encoding mode selection means for selecting an encoding mode in which the evaluation value calculated by the encoding efficiency evaluation value calculation means is maximum;
Encoding means for encoding the image data according to the encoding mode selected by the encoding mode selection means;
An image encoding device comprising: The encoding unit is configured to divide a screen allocation code amount allocated when encoding a screen included in the image data by the number of hybrid encoding blocks constituting the screen, thereby performing the hybrid encoding. The image coding apparatus according to claim 1, wherein an allocated code amount of the hybrid coding block is obtained. Predictive coding for predicting the encoding target pixel based on the encoded pixels, and quantizing a conversion coefficient obtained by converting an image signal in which the encoding target image composed of the encoding target pixels is continuous, variable length An image encoding method for encoding image data by hybrid encoding combined with transform encoding for encoding,
For the hybrid coding block obtained by dividing the image data into a predetermined size, the error value D ₁ in the intra coding mode is used as the error value when the image data is coded by the predictive coding. was calculated by the average of the variance of the input pixels of the transformation block included in the block, the error value D ₂ in the inter coding mode, the mean square error between the reference pixel of the input pixel and the block of the hybrid coding blocks, the average absolute A prediction error value calculation step for calculating by a value error or a cross-correlation coefficient;
Wherein the hybrid coding blocks obtained by dividing the image data into a predetermined size, the intra transform coefficients assigned code amount T _C1 in the coding mode, the hybrid coding hybrid coding blocks allocated to hybrid coding blocks when reduction The conversion in the inter coding mode is obtained by subtracting the generated code quantity in the intra coding mode and the sum of the generated code quantities of the DC coefficients of the transform blocks included in the hybrid coding block from the allocated code amount. the coefficients assigned code amount T _C2, from the hybrid coding block allocated code quantity, the generated code amount of the inter coding mode, the transform coefficients assigned code amount calculating step of obtaining by subtracting the amount of code generated for the motion vector When,
As the evaluation value E ₁ indicating the coding efficiency in the intra coding mode, the reciprocal of the error value D ₁ calculated by the prediction error value calculating unit and the transform coefficient allocation code amount calculated by the transform coefficient allocation code amount calculating unit The product of T _C1 is calculated, and the evaluation value E ₂ indicating the coding efficiency in the inter coding mode is used as the reciprocal of the error value D ₂ calculated by the prediction error value calculation means, and the conversion coefficient allocation code amount calculation A coding efficiency evaluation value calculating step for calculating an evaluation value indicating the coding efficiency by calculating a product of the transform coefficient allocation code amount T _C2 calculated by the means ;
An encoding step of selecting an encoding mode in which the evaluation value calculated in the encoding efficiency evaluation value calculating step is maximum, and encoding the image data by the selected encoding mode;
An image encoding method comprising: In order to encode image data by hybrid encoding combining predictive encoding using motion compensated prediction with motion vectors and transform encoding using orthogonal transform,
For the hybrid coding block obtained by dividing the image data into a predetermined size, the error value D ₁ in the intra coding mode is used as the error value when the image data is coded by the predictive coding. was calculated by the average of the variance of the input pixels of the transformation block included in the block, the error value D ₂ in the inter coding mode, the mean square error between the reference pixel of the input pixel and the block of the hybrid coding blocks, the average absolute Prediction error value calculation means for calculating by value error or cross-correlation coefficient,
Wherein the hybrid coding blocks obtained by dividing the image data into a predetermined size, the intra transform coefficients assigned code amount T _C1 in the coding mode, the hybrid coding hybrid coding blocks allocated to hybrid coding blocks when reduction The conversion in the inter coding mode is obtained by subtracting the generated code quantity in the intra coding mode and the sum of the generated code quantities of the DC coefficients of the transform blocks included in the hybrid coding block from the allocated code amount. Transform coefficient allocation code amount calculation means for obtaining the coefficient allocation code amount T _C2 by subtracting the generated code amount of the inter coding mode and the generated code amount of the motion vector from the hybrid encoded block allocation code amount ,
As the evaluation value E ₁ indicating the coding efficiency in the intra coding mode, the reciprocal of the error value D ₁ calculated by the prediction error value calculating unit and the transform coefficient allocation code amount calculated by the transform coefficient allocation code amount calculating unit The product of T _C1 is calculated, and the evaluation value E ₂ indicating the coding efficiency in the inter coding mode is used as the reciprocal of the error value D ₂ calculated by the prediction error value calculation means, and the conversion coefficient allocation code amount calculation Coding efficiency evaluation value calculating means for calculating an evaluation value indicating the coding efficiency by calculating a product of the transform coefficient allocation code amount T _C2 calculated by the means;
An encoding mode selecting means for selecting an encoding mode in which the evaluation value calculated by the encoding efficiency evaluation value calculating means is maximized;
Encoding means for encoding the image data according to the encoding mode selected by the encoding mode selection means;
An image encoding program that functions as a computer program.

Images