JP3983657B2 - Image processing apparatus and image processing method - Google Patents

Description

【００５０】
ここで、Ｙ，Ｃｂ，Ｃｒ成分の分解レベルｎのサブバンドＬＬの量子化ステップ数（Δｂ）は、（図７の（ａ）〜（ｃ）の図表に示す分解レベルｎのサブバンドＬＬの正規化分母の値）／（図８の（ａ）〜（ｃ）の図表に示す分解レベルｎのサブバンドＬＬの高速用の重み係数の値）を算出することで求められる。なお、当該演算により求められた量子化ステップ数（Δｂ）の値を図９の（ａ）〜（ｃ）に示す。
【００５１】
以下、同様の手順でＹ，Ｃｂ，Ｃｒ各成分毎に分解レベルｎのサブバンドＨＬの全ウェーブレット係数に対して量子化を行い（ステップＳ２３）、Ｙ，Ｃｂ，Ｃｒ各成分毎に分解レベルｎのサブバンドＬＨの全ウェーブレット係数に対して量子化を行い（ステップＳ２４）、Ｙ，Ｃｂ，Ｃｒ各成分毎に分割レベルｎのサブバンドＬＨの全ウェーブレット係数に対して量子化を行い（ステップＳ２４）、最後に、Ｙ，Ｃｂ，Ｃｒ各成分毎に分解レベルｎのサブバンドＨＨの全ウェーブレット係数に対して量子化を行う（ステップＳ２５）。
【００５２】
分解レベルを表す変数ｎの値が１でない場合には（ステップＳ２６でＮＯ）、係数ｎより１を減算した後（ステップＳ２７）、上記ステップＳ２３〜Ｓ２５の処理を実行する。他方、係数ｎの値が１の場合には（ステップＳ２６でＹＥＳ）、量子化処理を終了して図５のフローチャートにリターンする。
【００５３】
なお、中速用量子化ステップを使用した量子化処理（図５、ステップＳ１８）は、上述した高速用量子化ステップを使用した量子化処理の内、ハードディスク７から読み出す重み係数のデータが図８の（ａ）〜（ｃ）の図表に示すデータの代わりに図１０の（ａ）〜（ｃ）の図表に示すデータになるだけで、量子化ステップ数の算出方法などについては、全く同じである。このため、重複した説明は省く。
【００５４】
即ち、中速用量子化ステップを使用した量子化処理（ステップＳ１８）で用いる数式は、上記「数１」と同じであるが、上記量子化ステップ数（Δｂ）は、（図７の（ａ）〜（ｃ）に示す正規化分母の値）／（図１０の（ａ）〜（ｃ）に示す中速用の重み係数）を演算して求められる図１１の（ａ）〜（ｃ）に示す各値になる。
【００５５】
同様に、低速用量子化ステップを使用した量子化処理（図５、ステップＳ１９）は、上述した高速用量子化ステップを使用した量子化処理の内、ハードディスク７から読み出す重み係数のデータが図８の（ａ）〜（ｃ）の図表に示すデータの代わりに図１２の（ａ）〜（ｃ）の図表に示すデータになるだけで、量子化ステップ数の算出方法などについては、全く同じであるため、ここでの重複した説明は省く。
【００５６】
即ち、低速用量子化ステップを使用した量子化処理（ステップＳ１９）で用いる数式も、上記「数１」と同じであるが、上記量子化ステップ数（Δｂ）は、（図７の（ａ）〜（ｃ）に示す正規化分母の値）／（図１２の（ａ）〜（ｃ）に示す中速用の重み係数）を演算して求められる図１３の（ａ）〜（ｃ）に示す各値になる。
【００５７】
上述した量子化ステップ（Δｂ）の算出時に用いる図８、図１０、図１２の各速度別の重み成分は、低速になる程、再生画像の解像度に影響する高域の分解レベル及びサブバンドに対する重みを大きくし、高速に移動している画像程、くし型部分の量子化を抑えるように設定してある。このように、被写体の移動速度に応じて適切な量子化を行うことで、再生画像の画質低下を抑えつつ、圧縮率を高めることができる。
【００５８】
更には、図８、図１０、図１２の各速度別の重み成分のかわりに、分解レベル２まで特にＬＨ成分を重視して量子化を抑える図１４、図１６、図１８の重み成分を用いても良い。この場合の量子化ステップ数（Δｂ）は、図１５，図１７、図１９に示す値になる。これにより、くし型部分の再現性をより良好にすることができる。
【００５９】
（３）実施の形態２
実施の形態２における画像処理装置は、基本的な構成は実施の形態１に係る画像処理装置１０と同じであり、以下、同じ構成物には同じ参照番号を用いて説明する。実施の形態２に係る画像処理装置は、実施の形態１に係る画像処理装置１０を制御するＣＰＵ１の実行する符号化処理において、２次元離散ウェーブレット変換により得られるウェーブレット係数を、被写体の移動速度に応じて決まるテーブルに従いビットプレーン単位で削減する（当該ビットプレーンのデータを全て０に置きかえる）ことを特徴とする。より詳しくは、ノンインターレース画像内の被写体の移動速度の増加に応じて、当該ノンインターレース画像内に現れるくし型の画像のずれの再現性を高めるため、画像のエッジ部分の再現性に影響を与える高周波成分（１ＨＬ、１ＬＨ等）のウェーブレット係数のビットプレーンの削除枚数を少なくする。
【００６０】
図２０は、実施の形態２における画像処理装置（図示せず）のＣＰＵ１が実行する符号化処理のフローチャートである。まず、ＲＡＭ３の第１（又は、第２）画像データ領域３ａ（又は３ｂ）に書き込みの完了したノンインターレース画像の画像データを読み出す（ステップＳ３０）。読み出したノンインターレース画像の画像データをＹ（輝度）、Ｃｂ，Ｃｒ（色差）の３つの信号に変換（いわゆる色変換）する（ステップＳ３１）。
【００６１】
色変換により得られるＹ，Ｃｂ，Ｃｒの３つの信号に対して、それぞれ周波数変換として、２次元離散ウェーブレット変換を実行する（ステップＳ３２）。当該変換により得られるＹ，Ｃｂ，Ｃｒ各成分のウェーブレット係数に対してＪＰＥＧ２０００に準拠したスカラ量子化を行う（ステップＳ３３）。
【００６２】
当該量子化後のＹ，Ｃｂ，Ｃｒ各成分のウェーブレット係数をサブバンド単位でビットプレーンに分解し、フレーム内の被写体の移動速度に対応して、サブバンド単位で個別に、再現画像に与える影響の少ないビットプレーンのデータを削除するデータ削減処理を実行する（ステップＳ３４）。なお、当該処理の内容については後に詳しく説明する。
【００６３】
上記速度対応データ削減処理の後、ＪＰＥＧ２０００に準拠したエントロピー符号化処理（係数モデリング処理及び算術符号化処理により成る）を実行する（ステップＳ３５）。ＣＰＵ１によりＲＡＭ３の第１符号データ領域３ｃ又は第２符号データ領域３ｄに書き込まれる符号データをハードディスク７に保存する（ステップＳ３６）。
【００６４】
未だ処理すべきフレームの画像データが存在する場合には（ステップＳ３７でＮＯ）、上記ステップＳ３０に戻り、前回、画像データの読み出しを行った画像データ領域とは異なる側の画像データ領域（３ｂ又は３ａ）に書き込まれたノンインターレース画像の画像データを読み出す。ビデオカメラ８による撮影を止め、全てのフレーム（図２に示すフィールドｎ−１及びフィールドｎよりなるフレーム）のノンインターレース画像の画像データの符号化処理が完了した場合には（ステップＳ３７でＹＥＳ）、符号化処理を終了する。
【００６５】
図２１は、速度対応データ削減処理（図２０、ステップＳ３４）のフローチャートである。まず、１ＬＨのサブバンドのウェーブレット係数の絶対値の和ｓｕｍ１ＬＨを算出する（ステップＳ４０）。次に、１ＨＬのサブバンドのウェーブレット係数の絶対値の和ｓｕｍ１ＨＬを算出する（ステップＳ４１）。引き続き、２ＬＨのサブバンドのウェーブレット係数の絶対値の和ｓｕｍ２ＬＨを算出する（ステップＳ４２）。２ＨＬのサブバンドのウェーブレット係数の絶対値の和ｓｕｍ２ＨＬを算出する（ステップＳ４３）。（ｓｕｍ１ＬＨ／ｓｕｍ１ＨＬ）／（ｓｕｍ２ＬＨ／ｓｕｍ２ＬＨ）の演算より変数ｓｐｅｅｄを算出する（ステップＳ４４）。
【００６６】
上記ステップＳ４４で算出される変数ｓｐｅｅｄは、実施の形態１の欄でも説明したように、フレーム内の被写体の移動速度に比例した値であると解される。そこで、変数ｓｐｅｅｄが実験的に求められたしきい値ｔｈ３よりも大きい場合（ステップＳ４５でＹＥＳ）、フレーム内の被写体の移動速度は高速であると判断して、高速用データ削減処理を実行する（ステップＳ４６）。具体的には、図２２に示すように、Ｙ，Ｃｂ，Ｃｒの各成分の量子化後のウェーブレット係数をサブバンド毎にビットプレーンに展開し、図２３の（ａ）〜（ｃ）に表示するビットプレーンの枚数分だけ、各分解レベルのサブバンドの最下位のビットプレーンからデータを破棄する（データを０にする）。例えば、Ｙ成分のウェーブレット係数では、分解レベル１のＨＨ成分、即ち１ＨＨのサブバンドの最下位のビットプレーン１枚分のデータを破棄する。
【００６７】
また、変数ｓｐｅｅｄの値がしきい値ｔｈ３以下であるが、しきい値ｔｈ４よりも大きい場合には（ステップＳ４７でＹＥＳ）、フレーム内の被写体の移動速度は低速であると判断して、中速用データ削減処理を実行する（ステップＳ４８）。具体的には、Ｙ，Ｃｂ，Ｃｒの各成分の量子化後のウェーブレット係数をサブバンド毎にビットプレーンに展開し、図２４の（ａ）〜（ｃ）に表示する枚数分だけ、各分解レベルのサブバンドの最下位のビットプレーンからデータを破棄する。例えば、Ｙ成分のウェーブレット係数では、分解レベル１のＨＬ，ＬＨ，ＨＨ成分、即ち１ＨＬ，１ＬＨ，１ＨＨのサブバンドの最下位のビットプレーン２枚，２枚，３枚分のデータを破棄する。
【００６８】
他方、変数ｓｐｅｅｄの値がしきい値ｔｈ３以下の場合には（ステップＳ４９でＮＯ）、低速用データ削減処理を実行する（ステップＳ４９）。具体的には、Ｙ，Ｃｂ，Ｃｒの各成分の量子化後のウェーブレット係数をサブバンド毎にビットプレーンに展開し、図２５の（ａ）〜（ｃ）に表示する枚数分だけ、各分解レベルのサブバンドの最下位のビットプレーンからデータを破棄する。例えば、Ｙ成分のウェーブレット係数では、分解レベル１のＨＬ，ＬＨ，ＨＨ成分、即ち１ＨＬ，１ＬＨ，１ＨＨのサブバンドの最下位のビットプレーン５枚，５枚，８枚分のデータを破棄し、更に、分解レベル２のＨＬ，ＬＨ，ＨＨ成分、即ち２ＨＬ，２ＬＨ，２ＨＨのサブバンドの最下位のビットプレーン１枚，１枚，２枚分のデータを破棄する。
【００６９】
上記ステップＳ４６，Ｓ４８，Ｓ４９の何れかの処理終了後、図２０に示したメインルーチンにリターンする。
【００７０】
以上に説明するように実施の形態２に係る画像処理装置では、ノンインターレース画像内の被写体の移動速度の増加に応じて、当該ノンインターレース画像内に現れるくし型の画像のずれの再現性を維持するため、高周波成分（１ＨＬ、１ＬＨ等）のウェーブレット係数のビットプレーンの削除枚数を少なくする。これにより、再現画像の画質劣化を防止しつつ、良好なデータ削減を実現する。
【００７１】
なお、図２３、図２４、図２５の各速度別のビットトランケートを行う代わりに、分解レベル２まで特にＬＨ成分を重視して量子化を抑える図２６、図２７、図２８の図表に示す内容のビットトレンケートを実行しても良い。これにより、良好な圧縮率を保持しつつも、特に高速で移動している被写体に現れるくし型の画像のずれの再現性を向上させることができる。
【００７２】
（４）実施の形態３
実施の形態３における画像処理装置は、基本的な構成は実施の形態１に係る画像処理装置１０と同じであり、以下、同じ構成物には同じ参照番号を用いて説明する。実施の形態３に係る画像処理装置は、予定符号量の設定画面（図３４を参照）を介して設定された予定符号量になるまで、フレーム内の被写体の移動速度に応じて最終的に不要なエントロピー符号を破棄することによりデータの削減を行う。当該エントロピー符号の破棄は、重要度の低いものから画質制御単位の係数のビットプレーン（例えば、ＪＰＥＧ２０００では、サブバンドやコードブロックの係数のビットプレーン）単位で行う。この際、ノンインターレース画像内の被写体の移動速度の増加に伴い、画像のエッジ部分の再現性に影響を与える，高周波数帯域の係数をエントロピー符号化して得られる符号データの下位ビットデータを破棄する量が少なくなるように重要度を調節する。
【００７３】
図２９は、実施の形態３に係る画像処理装置のＣＰＵの実行する符号化処理のフローチャートである。まず、ＲＡＭ３の第１（又は、第２）画像データ領域３ａ（又は３ｂ）に書き込みの完了したノンインターレース画像の画像データを読み出す（ステップＳ６０）。読み出したノンインターレース画像の画像データをＹ（輝度）、Ｃｂ，Ｃｒ（色差）の３つの信号に変換（いわゆる色変換）する（ステップＳ６１）。
【００７４】
色変換により得られるＹ，Ｃｂ，Ｃｒの３つの信号に対して、それぞれ周波数変換として、２次元離散ウェーブレット変換を実行する（ステップＳ６２）。当該変換により得られるＹ，Ｃｂ，Ｃｒ各成分のウェーブレット係数に対してＪＰＥＧ２０００に準拠したスカラ量子化を行う（ステップＳ６３）。スカラ量子化後のＹ，Ｃｂ，Ｃｒ成分のウェーブレット係数にＪＰＥＧ２０００に準拠したエントロピー符号化処理（係数モデリング処理及び算術符号化処理より成る）を実行する（ステップＳ６４）。
【００７５】
符号量が予定値に収まるように、符号化処理により得られる符号データの画質制御単位である３２×３２画素マトリクスで成るコードブロックのビットプレーンのデータを重要度の低いものから順に破棄する（データの値を０に置きかえる）データ削減処理を実行する（ステップＳ６５）。なお、当該処理の内容については後に詳しく説明する。
【００７６】
上記速度対応データ削減処理の後、ＣＰＵ１によりＲＡＭ３の第１符号データ領域３ｃ又は第２符号データ領域３ｄに書き込まれる符号データをハードディスク７に保存する（ステップＳ６６）。
【００７７】
未だ処理すべきフレームの画像データが存在する場合には（ステップＳ６７でＮＯ）、上記ステップＳ６０に戻り、前回画像データの読み出しを行った画像データ領域とは異なる側の第２又は第１画像データ領域（３ｂ又は３ａ）に書き込まれたノンインターレース画像の画像データを読み出す。ビデオカメラ８により撮影を止め、全てのフレーム（図２に示すフィールドｎ−１及びフィールドｎよりなるフレーム）のノンインターレース画像の符号化処理が完了した場合には（ステップＳ６７でＹＥＳ）、符号化処理を終了する。
【００７８】
図３０は、速度対応データ削減処理（図２９、ステップＳ６５）のフローチャートである。まず、図３１に示すように、符号データを３２×３２画素マトリクスよりなるｎ個（例えば、１０２４×１０２４画素の画像データの場合、１０２４個である。）のコードブロックに分割する（ステップＳ７０）。コードブロックを特定する変数ＣＢを１に設定する（ステップＳ７１）。１ＬＨのサブバンドのコードブロックＣＢの符号データの絶対値の和ｓｕｍ１ＬＨを算出する（ステップＳ７２）。１ＨＬのサブバンドのコードブロックＣＢの符号データの絶対値の和ｓｕｍ１ＨＬを算出する（ステップＳ７３）。２ＬＨのサブバンドのコードブロックＣＢの符号データの絶対値の和ｓｕｍ２ＬＨを算出する（ステップＳ７４）。２ＨＬのサブバンドのコードブロックＣＢの符号データの絶対値の和ｓｕｍ２ＨＬを算出する（ステップＳ７５）。（ｓｕｍ１ＬＨ／ｓｕｍ１ＨＬ）／(ｓｕｍ２ＬＨ／ｓｕｍ２ＨＬ）の演算より変数ｓｐｅｅｄを算出する（ステップＳ７６）。
【００７９】
上記ステップＳ７６で算出される変数ｓｐｅｅｄは、実施の形態１の欄でも説明したように、フレーム内の被写体の移動速度に比例した値であると解される。そこで変数ｓｐｅｅｄの値が実験的に求められるしきい値ｔｈ５よりも大きい場合（ステップＳ７７でＹＥＳ）、フレーム内の被写体が高速で移動していると判断し、図８に示した高速用の視覚重みを用いて符号の重用度Ｇ（ＣＢ）を算出する（ステップＳ７８）。当該符号の重要度Ｇ（ＣＢ）は、図３２に示すように、３２×３２ビットのコードブロックの１６枚の各ビットプレーンについて求める。なお、当該処理の内容については後に詳しく説明する。
【００８０】
一方、変数ｓｐｅｅｄの値が実験的に求められるしきい値ｔｈ５以下であるが、しきい値ｔｈ６以下ではない場合（ステップＳ７９でＮＯ）、フレーム内の被写体が中速で移動していると判断し、図１０に示した中速用の視覚重みを用いて符号の重要度Ｇ（ＣＢ）算出処理を実行する（ステップＳ８０）。当該符号の重要度Ｇ（ＣＢ）は、３２×３２ビットのコードブロックの１６枚の各ビットプレーンについて求める。
【００８１】
変数ｓｐｅｅｄの値が実験的に求められるしきい値ｔｈ６以下の場合（ステップＳ７９でＮＯ）、図１２に示した低速用の視覚重みを用いた符号の重要度Ｇ（ＣＢ）の算出処理を実行する（ステップＳ８１）。当該符号の重要度Ｇ（ＣＢ）は、３２×３２ビットのコードブロックの１６枚の各ビットプレーンについて求める。
【００８２】
ＣＢ≠ｎの場合（ステップＳ８２でＮＯ）、変数ＣＢに１を加算して（ステップＳ８３）、上記ステップＳ７２に戻る。他方、ＣＢ＝ｎの場合（ステップＳ８２でＹＥＳ）、図３３に示すフローチャートのステップＳ８４に進み、ｎ個のコードブロックの１６枚のビットプレーン、即ち全部でｎ×１６枚のビットプレーンについて求められた符号の重要度Ｇ（ＣＢ）の値を、コードブロックＣＢ及び当該コードブロックのＬＳＢからのビットプレーンの枚数ｍの情報と共に、低いものから順に並べ替える（ソートする）（ステップＳ８４）。
【００８３】
上記ソート処理の後、ディスプレイ６に図３４に示す予定符号量ＤＳ（Data Sizeの略）設定画面５０を表示させる（ステップＳ８５）。
【００８４】
図３２に示すように、設定画面５０内に設けられている予定符号量の数値設定欄５１内にキーボード４を用いて予定符号量（例えば１５０ｋｂｉｔ等）を設定し、設定キー５２がマウス５によりクリックされた場合（ステップＳ８６でＹＥＳ）、現状で最もＬＳＢ側に位置する全ビットプレーンの内、最も重要度Ｇ（ＣＢ）ｍの低いコードブロックＣＢのビットプレーン１枚分のデータを破棄する（ステップＳ８７）。データ破棄した後の符号量が上記設定画面５０において設定された予定符号量ＤＳ以下の場合（ステップＳ８８でＮＯ）、ステップＳ８７に戻り、そのときに最もＬＳＢ側に位置する全ビットプレーンの内、最も重要度Ｇ（ＣＢ）の低いコードブロックＣＢのビットプレーン１枚のデータを破棄する。データ破棄後の符号量が予定符号量ＤＳに満たない値になった場合（ステップＳ８８でＹＥＳ）、速度対応データ削減処理を終了してメインルーチンにリターンする。
【００８５】
図３５は、高速用の視覚重みを用いた符号の重要度Ｇ（ＣＢ）ｍの算出処理（図３０、ステップＳ７８）のフローチャートである。まず、１つのコードブロックを構成するビットプレーンの枚数（本例では１６枚）を表す変数ｍの値を１に設定する（ステップＳ９０）。コードブロックＣＢ（ＣＢは１〜ｎ）のＬＳＢから数えてｍ枚目のビットプレーンのデータの重要度Ｇ（ＣＢ）ｍを求める（ステップＳ９１）。
【００８６】
上記重要度Ｇ（ＣＢ）ｍは、（コードブロックＣＢのｍ枚目のビットプレーンのデータを全て破棄した場合の量子化誤差の増分×視覚重み係数）／（ビットプレーン内にある有効な符号の総量）の演算式で求められるが、フローチャートに示すように、｛２＾（ｍ−１）−２＾（ｍ−２）｝×（ＬＳＢから数えてｍ枚目のビットプレーンに含まれる有効な符号の数）×視覚重み係数]／（ビットプレーン内にある有効な符号の和）の演算式で求めることができる。
【００８７】
重要度Ｇ（ＣＢ）ｍが上記演算式に近似できる理由を以下に述べておく。上記のコードブロックＣＢのｍ枚目のビットプレーンのデータを全て破棄した場合の量子化誤差の増分の算出方法は、種々存在し、数学的に厳密な方法は、例えば、「２００２年にKluwer出版社より発行されたD.S.Taubuman and M.W.Marcelinによる著、“JPEG2000: Image Compression Fundamentals, Standards and Practice”」に記載されている。しかし、本画像処理装置１０では、“ＬＳＢ（最下位ビット）から数えてｍ枚目のビットプレーンの符号を破棄した場合の１ウェーブレット係数あたりの量子化誤差”を２＾（ｍ−１）で近似する。ビットプレーンをＬＳＢから１枚破棄することは、誤差的観点からは係数を２で割ることと等価であり、その誤差は確率的には２＾（ｍ−１）だからである。
【００８８】
よって、“コードブロックＣＢのＬＳＢから数えてｍ枚目のビットプレーンの符号を破棄した場合の量子化誤差の増分”は、｛２＾（ｍ−１）−２＾（ｍ−２）｝×（ＬＳＢから数えてｍ枚目のビットプレーンに含まれる有効な係数の数）と近似することができる。従って、コードブロックＣＢのＬＳＢから数えてｍ枚目のビットプレーンの重要度Ｇ（ＣＢ）ｍは、｛２＾（ｍ−１）−２＾（ｍ−２）｝×（ＬＳＢから数えてｍ枚目のビットプレーンに含まれる有効な符号の数）×視覚重み係数]／（ビットプレーン内にある有効な符号の和）の演算式に近似することができるのである。
【００８９】
上記重要度Ｇ（ＣＢ）ｍ算出後において、変数ｍの値が１６でない場合（ステップＳ９２でＮＯ）、変数ｍに１を加算してステップＳ９１に戻り、コードブロックＣＢの次のビットプレーンの重要度Ｇ（ＣＢ）ｍを求める。変数ｍが１６の場合（ステップＳ９２でＹＥＳ）、コードブロックＣＢの全１６枚のビットプレーンの重要度の算出が完了したと判断して処理を終了しリターンする。
【００９０】
なお、ビットプレーンのデータを破棄する順序を、各コードブロックＣＢの最もＬＳＢ側に位置する有効なビットプレーンであって最も重要度Ｇ（ＣＢ）ｍの低いものから順とする以外、ラグランジュの未定乗数法を用いて決めても良い。ＪＰＥＧ２０００におけるラグランジュの未定乗数法の実装法についても上述した“D.S.Taubman等による著、“JPEG2000: Image Compression Fundamentals, Standards and Practice”に詳しく説明されている。
【００９１】
中速用の視覚重みを用いた符号の重要度Ｇ（ＣＢ）ｍの算出処理（図３０、ステップＳ８０）及び低速用の視覚重みを用いた符号の重要度Ｇ（ＣＢ）ｍの算出処理（ステップＳ８１）の処理内容は、それぞれ符号の重要度Ｇ（ＣＢ）ｍを算出する際に用いる視覚重みの係数を図１０及び図１２に示す図表の内容に換えるだけであり、それ以外の処理内容は全く同じであるため、ここでの重複した説明は省く。
【００９２】
なお、高速、中速、低速用の視覚重み係数として、図８、図１０、図１２の各速度別の重み成分のかわりに、分解レベル２まで特にＬＨ成分を重視して量子化を抑える図１４、図１６、図１８の重み成分を用いても良い。これにより、くし型部分の再現性をより良好にすることができる。
【００９３】
なお、図３５のステップＳ９１に示す重要度Ｇ（ＣＢ）ｍの算出式の代わりに、（コードブロックＣＢのｍ枚目のビットプレーンのデータを全て破棄した場合の量子化誤差の増分×視覚重み係数／マスキング係数）／（ビットプレーン内にある有効な符号の総量）の算出式を用いても良い。ここで、マスキング係数とは、（コードブロックＣＢに含まれる係数の絶対値の和／コードブロックＣＢに含まれる係数）＾αで定義される係数であり、上記αは実験の結果、０＜α≦１であり、一例としてα＝１を採用する。
【００９４】
更には、上記マスキング係数の代わりに、（コードブロックＣＢの１ＬＨのサブバンドの係数の絶対値の和）／（コードブロックＣＢの１ＨＬのサブバンドの係数の絶対値の和）又は、（コードブロックＣＢの１ＬＨのサブバンドの符号量）／（コードブロックＣＢの１ＨＬのサブバンドの符号量）の演算式により求められる係数をノンインターレース画像に表れるくし型の指標値として用いることも考えられる。
【００９５】
以上に説明するように、実施の形態３に係る画像処理装置では、エントロピー符号化処理御に、画質制御単位であるコードブロックのビットプレーン単位で重要度を調べ、予定の符号量ＤＳ未満になるまで、重要度の低いビットプレーンのデータから順に破棄するが、この際、ノンインターレース画像内の被写体の移動速度の増加に伴い、画像のエッジ部分の再現性を劣化させ得る箇所である，高周波数帯域の係数をエントロピー符号化して得られる符号データの下位ビットデータを破棄する量が少なくなるように上記重要度を調節する。これにより、再生画像に最も影響を与えることなく良好な画像圧縮を実現することができる。
【００９６】
【発明の効果】
本発明の画像処理装置又は画像処理方法によれば、連続する２つのフィールドのインターレース画像から生成されるノンインターレース画像（フレーム）を処理対象とする画像処理装置であって、フレーム内の被写体の移動速度の増加に対して、人の視覚特性を考慮して、ノンインターレース画像に表れるくし型の画像のずれの再現性を高めるように、データ削減処理（量子化によるデータ削減を含む）を実行することができる。
【図面の簡単な説明】
【図１】 実施の形態１に係る画像処理装置の全体構成図である。
【図２】 ビデオカメラで撮影されるインターレース画像の様子を示す図である。
【図３】 （ａ）〜（ｄ）は、ノンインターレース画像からノンインターレース画像を形成する際に生じるくし型のずれについて説明するための図である。
【図４】 ＣＰＵの実行する符号化処理のフローチャートである。
【図５】 速度対応量子処理のフローチャートである。
【図６】 高速用量子化ステップを使用した量子化処理のフローチャートである。
【図７】 （ａ）〜（ｃ）は、量子化ステップを求める際に用いるＹ，Ｃｂ，Ｃｒ各成分の分解レベル５〜１に対する各サブバンドの正規化分母を示す表である。
【図８】 （ａ）〜（ｃ）は、量子化ステップを求める際に用いるＹ，Ｃｂ，Ｃｒ各成分の分解レベル５〜１に対する各サブバンドの高速用の重み係数を示す表である。
【図９】 （ａ）〜（ｃ）は、Ｙ，Ｃｂ，Ｃｒ各成分の分解レベル５〜１に対する各サブバンドの高速用量子化ステップを示す表である。
【図１０】 （ａ）〜（ｃ）は、量子化ステップを求める際に用いるＹ，Ｃｂ，Ｃｒ各成分の分解レベル５〜１に対する各サブバンドの中速用の重み係数を示す表である。
【図１１】 （ａ）〜（ｃ）は、Ｙ，Ｃｂ，Ｃｒ各成分の分解レベル５〜１に対する各サブバンドの中速用量子化ステップを示す表である。
【図１２】 （ａ）〜（ｃ）は、量子化ステップを求める際に用いるＹ，Ｃｂ，Ｃｒ各成分の分解レベル５〜１に対する各サブバンドの低速用の重み係数を示す表である。
【図１３】 （ａ）〜（ｃ）は、Ｙ，Ｃｂ，Ｃｒ各成分の分解レベル５〜１に対する各サブバンドの低速用量子化ステップを示す表である。
【図１４】 （ａ）〜（ｃ）は、量子化ステップを求める際に用いるＹ，Ｃｂ，Ｃｒ各成分の分解レベル５〜１に対する各サブバンドの高速用の重み係数を示す表である。
【図１５】 （ａ）〜（ｃ）は、Ｙ，Ｃｂ，Ｃｒ各成分の分解レベル５〜１に対する各サブバンドの高速用量子化ステップを示す表である。
【図１６】 （ａ）〜（ｃ）は、量子化ステップを求める際に用いるＹ，Ｃｂ，Ｃｒ各成分の分解レベル５〜１に対する各サブバンドの中速用の重み係数を示す表である。
【図１７】 （ａ）〜（ｃ）は、Ｙ，Ｃｂ，Ｃｒ各成分の分解レベル５〜１に対する各サブバンドの中速用量子化ステップを示す表である。
【図１８】 （ａ）〜（ｃ）は、量子化ステップを求める際に用いるＹ，Ｃｂ，Ｃｒ各成分の分解レベル５〜１に対する各サブバンドの低速用の重み係数を示す表である。
【図１９】 （ａ）〜（ｃ）は、Ｙ，Ｃｂ，Ｃｒ各成分の分解レベル５〜１に対する各サブバンドの低速用量子化ステップを示す表である。
【図２０】 実施の形態２の画像処理装置のＣＰＵが実行する符号化処理のフローチャートである。
【図２１】 速度対応データ削減処理のフローチャートである。
【図２２】 Ｙ成分の高速用ビットトランケーションの内容を図式的に示す図である。
【図２３】 （ａ）〜（ｃ）は、Ｙ，Ｃｂ，Ｃｒ各成分の分解レベル５〜１に対する各サブバンドの高速用ビットトランケート数を示す表である。
【図２４】 （ａ）〜（ｃ）は、Ｙ，Ｃｂ，Ｃｒ各成分の分解レベル５〜１に対する各サブバンドの中速用ビットトランケート数を示す表である。
【図２５】 （ａ）〜（ｃ）は、Ｙ，Ｃｂ，Ｃｒ各成分の分解レベル５〜１に対する各サブバンドの低速用ビットトランケート数を示す表である。
【図２６】 （ａ）〜（ｃ）は、別の重み係数を用いた場合のＹ，Ｃｂ，Ｃｒ各成分の分解レベル５〜１に対する各サブバンドの高速用ビットトランケート数を示す表である。
【図２７】 （ａ）〜（ｃ）は、別の重み係数を用いた場合のＹ，Ｃｂ，Ｃｒ各成分の分解レベル５〜１に対する各サブバンドの中速用ビットトランケート数を示す表である。
【図２８】 （ａ）〜（ｃ）は、別の重み係数を用いた場合のＹ，Ｃｂ，Ｃｒ各成分の分解レベル５〜１に対する各サブバンドの低速用ビットトランケート数を示す表である。
【図２９】 実施の形態３に係る画像処理装置のＣＰＵの実行する符号化処理のフローチャートである。
【図３０】 速度対応データ削減処理のフローチャートである。
【図３１】 符号データをｎ個のコードブロックに分割した状態を示す図である。
【図３２】 コードブロックをビットプレーンに分解した状態を示す図である。
【図３３】 速度対応データ削減処理のフローチャートの続きである。
【図３４】 ディスプレイに表示される予定符号量ＤＳの設定画面である。
【図３５】 高速用の視覚重みを用いた符号の重要度の算出処理のフローチャートである。
【符号の説明】
１ ＣＰＵ、２ ＲＯＭ、３ ＲＡＭ、３ａ 第１画像データ領域、３ｂ 第２画像データ領域、３ｃ 第１符号データ領域、３ｄ 第２符号データ領域、４ キーボード、５ マウス、６ ディスプレイ、７ ハードディスク、８ ビデオカメラ、１０ 画像処理装置。[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus that performs compression encoding of image data using wavelet transform.
[0002]
[Prior art]
Conventionally, various image processing apparatuses are known that convert image data into frequency domain coefficients by DCT transform or two-dimensional discrete wavelet transform, quantize the coefficients for each frequency, and entropy code the quantized coefficients. ing. For example, the quantization step size used in the above quantization is changed to an image quality control unit (for example, each subband unit when using wavelet transform) for the purpose of increasing the amount of data compression while preventing deterioration of the playback image. An image processing apparatus has been proposed (see, for example, Patent Document 1).
[0003]
[Patent Document 1]
Japanese Patent Laid-Open No. 08-186816
[0004]
[Problems to be solved by the invention]
In the image processing apparatus disclosed in Patent Document 1, two-dimensional discrete wavelet transform is used, but the quantization step size when quantizing the wavelet coefficients after transformation is set to subbands of LH, HL, and HH with respect to the luminance signal. The color difference signal is increased in the order of HL, LH, and HH subbands.
[0005]
The content of the quantization step size adjustment processing is obtained based on experimental data using the “Mobile calendar” which is an MPEG standard image, and is specified based on the characteristics of a general halftone image. Not a thing.
[0006]
Further, in a non-interlaced image composed of two fields of interlaced images that are continuously captured by the video camera every 1/60 seconds, a comb-shaped pixel shift occurs according to the moving speed of the subject in the horizontal direction. The image processing apparatus does not consider that the amount of displacement of the comb-shaped pixels varies depending on the moving speed of the subject. Therefore, depending on the moving speed of the subject, the original single vertical line becomes two. Or the outline of the reproduced image fluctuates from side to side and the image quality is greatly degraded.
[0007]
In particular, the present invention is an image processing apparatus for processing non-interlaced images generated from interlaced images of continuous fields, while preventing deterioration in the quality of a reproduced image according to the moving speed of a subject in a frame. An object of the present invention is to provide an image processing apparatus that realizes good compression encoding of image data.
[0008]
[Means for Solving the Problems]
A first image processing apparatus according to the present invention is a data reduction unit that reduces the amount of code data in an image processing apparatus that encodes image data of a non-interlaced image composed of two consecutive interlaced images into code data. In addition, the present invention is characterized by comprising data reduction means for reducing the reduction amount of code data at a portion that degrades the reproducibility of the edge portion of the image as the moving speed of the subject in the non-interlaced image increases. As data deletion means, (a) subband conversion means for performing subband conversion on the image data, and the subband conversion means 1LH The sum of the absolute values of the subband coefficients 1HL The calculation means for calculating the value divided by the sum of the absolute values of the subband coefficients, and the amount of reduction of the code data in the portion that degrades the reproducibility of the edge portion of the image as the value calculated by the calculation means increases A mode provided with data reduction means for reducing, (b) subband conversion means for performing subband conversion on the image data, and generated by the subband conversion means 1LH The sum of the absolute values of the subband coefficients 1HL The value divided by the sum of the absolute values of the subband coefficients is 2LH The sum of the absolute values of the subband coefficients 2HL A calculation means for calculating the value divided by the sum of the absolute values of the subband coefficients, and a portion that deteriorates the reproducibility of the edge portion of the image as the value calculated by the calculation means increases. (C) a subband conversion unit that performs subband conversion on the image data; and a subband generated by the subband conversion unit is divided into a plurality of rectangles ( Dividing means for dividing into (code block) and divided by the dividing means 1LH The sum of the absolute values of the coefficients contained in one rectangle of the subband is 1HL A calculation unit that calculates a value divided by the sum of absolute values of coefficients included in a rectangle at the same position as the rectangle of the subband, and the reproducibility of the edge portion of the image as the value calculated by the calculation unit increases. (D) a subband conversion unit that performs subband conversion on the image data, and the subband conversion unit. Generated Dividing means for dividing the subband into a plurality of rectangles (code blocks), and dividing by the dividing means 1LH The sum of the absolute values of the coefficients contained in one rectangle of the subband is 1HL The value divided by the sum of the absolute values of the coefficients contained in the rectangle at the same position as the rectangle of the subband, 2LH The sum of the absolute values of the coefficients included in the rectangle at the same position as the rectangle of the subband, 2HL The sum of the absolute values of the coefficients contained in the rectangle at the same position as the rectangle of the subband Divided by the value , A calculation means for calculating the divided value, and a data reduction means for reducing the amount of reduction of the code data at a portion that deteriorates the reproducibility of the edge portion of the image as the value calculated by the calculation means increases. (E) subband conversion means for performing subband conversion on the image data, division means for dividing the subband generated by the subband conversion means into a plurality of rectangles (code blocks), and the division means Divided 1LH The amount of code when the coefficient included in one rectangle of the subband is encoded, 1HL A calculation unit that calculates a value divided by a code amount when a coefficient included in a rectangle at the same position as the rectangle of the subband is encoded; and an edge portion of the image as the value calculated by the calculation unit increases There is a form provided with a data reduction means for reducing the amount of reduction of the code data at the location where the reproducibility of the image is degraded.
[0009]
According to a second image processing apparatus of the present invention, in the first image processing apparatus, the data reduction unit is configured to reduce the interlaced image of the two fields as the moving speed of the subject in the noninterlaced image increases. It is characterized in that the amount of code data to be reduced is reduced at a portion that deteriorates the reproducibility of a comb-shaped image shift occurring between them.
[0010]
According to a third image processing apparatus of the present invention, the first or second image processing apparatus performs frequency conversion on the image data of the non-interlaced image, quantizes a coefficient for each frequency obtained by the conversion, and performs quantization. An image processing apparatus that generates code data by entropy encoding a later coefficient, wherein the data reduction unit reduces code data by making the number of quantization steps used when performing the quantization larger than a standard value. And the value of the number of quantization steps used when quantizing the coefficient of the high frequency band is reduced as the moving speed of the subject in the non-interlaced image increases.
[0011]
In the fourth image processing apparatus of the present invention, the first or second image processing apparatus frequency-converts the image data of the non-interlaced image, quantizes the coefficient for each frequency obtained by the conversion, and after the quantization Is an image processing device that generates code data by entropy encoding the coefficients of the data, wherein the data reduction means reduces the code data by discarding the lower-order bit data of the coefficients after the quantization, As the moving speed of the subject increases, the amount of discarding the lower bit data of the coefficient in the high frequency band is reduced.
[0012]
According to a fifth image processing apparatus of the present invention, the first or second image processing apparatus performs frequency conversion on the image data of the non-interlaced image, quantizes a coefficient for each frequency obtained by the conversion, and performs quantization. An image processing apparatus that generates code data by entropy-encoding a later coefficient, wherein the data reduction unit performs data reduction by discarding lower-order bit data of the code data, and moves a subject in a non-interlaced image The amount of discarding the lower bit data of the code data is reduced as the speed increases.
[0013]
According to a sixth image processing apparatus of the present invention, in any one of the third to fifth image processing apparatuses, a two-dimensional discrete wavelet transform is performed as a frequency transform.
[0014]
A first image processing method of the present invention is a data reduction process for reducing the amount of code data in an image processing method for encoding image data of a non-interlace image composed of two consecutive interlace images into code data. In addition, the present invention is characterized in that it includes a data reduction step of reducing the amount of code data to be reduced at a location that degrades the reproducibility of the edge portion of the image as the moving speed of the subject in the non-interlaced image increases. The data deletion process includes (a) a subband conversion process for performing subband conversion on the image data, and the subband conversion process. 1LH The sum of the absolute values of the subband coefficients 1HL A calculation step for calculating a value divided by the sum of absolute values of subband coefficients, and a reduction amount of code data in a portion that deteriorates the reproducibility of the edge portion of the image as the value calculated in the calculation step increases. (B) a subband conversion process for performing subband conversion on the image data, and the subband conversion process. 1LH The sum of the absolute values of the subband coefficients 1HL The value divided by the sum of the absolute values of the subband coefficients is 2LH The sum of the absolute values of the subband coefficients 2HL A calculation step of calculating the value divided by the sum of the absolute values of the subband coefficients, and a point where the reproducibility of the edge portion of the image is deteriorated as the value calculated in the calculation step increases. (C) a subband conversion process for performing subband conversion on the image data, and subbands generated in the subband conversion process are divided into a plurality of rectangles ( Divided into code blocks) and divided in the dividing step 1LH The sum of the absolute values of the coefficients contained in one rectangle of the subband is 1HL A calculation step of calculating a value divided by the sum of absolute values of coefficients included in a rectangle at the same position as the rectangle of the subband, and the reproducibility of the edge portion of the image as the value calculated by the calculation means increases (D) a subband conversion process for performing subband conversion on the image data, and the subband conversion process. Generated Dividing the subband into a plurality of rectangles (code blocks) 1LH The sum of the absolute values of the coefficients contained in one rectangle of the subband is 1HL The value divided by the sum of the absolute values of the coefficients contained in the rectangle at the same position as the rectangle of the subband, 2LH The sum of the absolute values of the coefficients included in the rectangle at the same position as the rectangle of the subband, 2HL The sum of the absolute values of the coefficients contained in the rectangle at the same position as the rectangle of the subband Divided by the value A calculation step for calculating the divided value, and a data reduction step for reducing a reduction amount of code data at a portion that deteriorates the reproducibility of the edge portion of the image as the value calculated in the calculation step increases. (E) a subband conversion process for performing subband conversion on the image data, and the subband conversion process. Generated Dividing the subband into a plurality of rectangles (code blocks) 1LH The amount of code when the coefficient included in one rectangle of the subband is encoded, 1HL A calculation step of calculating a value divided by a code amount when a coefficient included in a rectangle at the same position as the rectangle of the subband is encoded, and an edge portion of the image as the value calculated in the calculation step increases There is a form provided with a data reduction process for reducing the amount of reduction of the code data at a location that deteriorates the reproducibility of the data.
[0015]
According to a second image processing method of the present invention, in the first image processing method, in the data reduction step, as the moving speed of the subject in the non-interlaced image increases, the interlaced image of the two fields is increased. It is characterized in that the amount of code data to be reduced is reduced at a portion that deteriorates the reproducibility of a comb-shaped image shift occurring between them.
[0016]
According to a third image processing method of the present invention, the first or second image processing method performs frequency conversion on the image data of the non-interlaced image, quantizes a coefficient for each frequency obtained by the conversion, and performs quantization. An image processing method for generating code data by entropy encoding a later coefficient, wherein the data reduction step reduces code data by making the number of quantization steps used when performing the quantization larger than a standard value. And the value of the number of quantization steps used when quantizing the coefficient of the high frequency band is reduced as the moving speed of the subject in the non-interlaced image increases.
[0017]
According to a fourth image processing method of the present invention, the first or second image processing method performs frequency conversion on the image data of the non-interlaced image, quantizes a coefficient for each frequency obtained by the conversion, and performs quantization. An image processing method for entropy-encoding a later coefficient to generate code data, wherein the data reduction step reduces the code data by discarding the lower-order bit data of the quantized coefficient to generate a non-interlaced image The amount of discarding the lower bit data of the coefficient in the high frequency band is reduced with an increase in the moving speed of the subject within.
[0018]
According to a fifth image processing method of the present invention, the first or second image processing method performs frequency conversion on the image data of the non-interlaced image, quantizes a coefficient for each frequency obtained by the conversion, and performs quantization. An image processing method for entropy encoding a later coefficient to generate code data, wherein the data reduction step discards the lower bit data of the code data and performs data reduction to move a subject in a non-interlaced image The amount of discarding the lower bit data of the code data is reduced as the speed increases.
[0019]
A sixth image processing method of the present invention is any one of the third to fifth image processing methods described above, wherein two-dimensional discrete wavelet transform is performed as frequency transform.
[0020]
DETAILED DESCRIPTION OF THE INVENTION
(1) Summary of the invention
The image processing apparatus according to the present invention targets a non-interlaced image (frame) generated from an interlaced image of two consecutive fields, converts the image data of the non-interlaced image into image-frequency coefficients, An image processing apparatus that quantizes the coefficient for each frequency and entropy-encodes the quantized coefficient in consideration of human visual characteristics with respect to an increase in moving speed of a subject in a frame. Data reduction processing (including data reduction by quantization) so as to improve the reproducibility of the shift of the comb-shaped image appearing in the image, that is, to reduce the amount of code data reduction in the portion that deteriorates the reproducibility of the edge portion of the image ). Specifically, the data reduction process is executed by the following three methods. Note that the image processing apparatuses according to the first to third embodiments that execute the data reduction processing of these three methods will be described in detail later.
[0021]
In the first method of data reduction processing, non-interlaced image data to be processed is converted into frequency domain coefficients by frequency conversion, for example, DCT conversion in JPEG or two-dimensional discrete wavelet conversion in JPEG2000. Data reduction is performed using quantization performed on each coefficient. At this time, as the moving speed of the subject in the frame increases, the value of the number of quantization steps used when quantizing the coefficient in the high frequency band is decreased. The quantized coefficients are entropy encoded. The first method is employed in the image processing apparatus according to the first embodiment described below.
[0022]
In the second method of data reduction processing, frequency conversion is performed on the data of the non-interlaced image to be processed, the coefficient for each frequency obtained by the conversion is quantized, and then the coefficient for each frequency after the quantization is performed. Are divided into image quality control units (for example, subbands and code blocks correspond to this in JPEG2000), and finally necessary portions (for example, for example, of the divided coefficients according to the moving speed of the subject in the frame). In JPEG2000, data reduction is performed by performing entropy coding of coefficients only for necessary sub-band unit bit planes or code block unit bit planes. At this time, as the moving speed of the subject in the non-interlaced image increases, the amount of discarding the lower bit data of the high frequency band coefficient is reduced. The second method is employed in the image processing apparatus according to the second embodiment described below.
[0023]
In the third method of data reduction processing, frequency conversion is performed on the data of the non-interlaced image to be processed, the coefficient for each frequency obtained by the conversion is quantized, and the coefficient for each frequency after quantization is entropy encoded. Thereafter, data reduction is performed by finally discarding unnecessary entropy codes in accordance with the moving speed of the subject in the frame. Discarding the entropy code is performed in units of bit planes of coefficients of image quality control units from the least important ones (for example, in JPEG 2000, bit planes of coefficients of subbands and code blocks). At this time, as the moving speed of the subject in the non-interlaced image increases, the importance is adjusted so that the amount of discarding the lower bit data of the code data obtained by entropy encoding the coefficient in the high frequency band is reduced. The third method is employed in the image processing apparatus according to the third embodiment described below.
[0024]
(2) Embodiment 1
The image processing apparatus 10 according to the first embodiment executes a coding process compliant with JPEG2000, and sets a quantization step size for scalar quantization performed on a wavelet coefficient obtained by two-dimensional discrete wavelet transform that is frequency transform. The image quality control unit is changed in units of subbands according to the moving speed of the subject in the frame. More specifically, the edge portion of the image is reproduced so that the reproducibility of the shift of the comb-shaped image appearing in the non-interlaced image increases as the moving speed of the subject in the non-interlaced image to be processed increases. The value of the number of quantization steps used when quantizing the coefficient of the high frequency band which is a place where the performance can be degraded is reduced.
[0025]
FIG. 1 is a diagram illustrating a configuration of the image processing apparatus 10. The image processing apparatus 10 is a compression encoding processing apparatus for image data that conforms to JPEG2000. The central processing unit (hereinafter referred to as a CPU) 1 is a ROM 2 that stores a control program, and an encoding process is executed. It comprises a RAM 3 used as a work memory, a keyboard 4, a mouse 5, a display 6, a hard disk 7, and a video camera 8.
[0026]
In the first image data area 3 a and the second image data area 3 b of the RAM 3, image data of non-interlaced images formed by interlaced images of two fields continuously captured by the video camera 8 are alternately controlled by the CPU 1. Is written to.
[0027]
More specifically, as shown in FIG. 2, among the interlace images of fields 0 to n that are continuously read in units of 1/60 seconds, for example, the interlace image A of field 0 ((a) of FIG. (See) is written in the first image data area 3a of the RAM 3, and then the interlaced image of the next field 1 (see FIG. 1) read after 1/60 seconds alternately in an area where data is not yet written in units of one line. 3 (see (b)) is added so as to supplement the image data. Thereby, the image data of the non-interlaced image as shown in FIG. 3C is formed in the first image data area 3a of the RAM 3.
[0028]
Similarly, the CPU 1 writes the image data of the interlaced images A and B of the fields 2 and 3 in the second image data area 3b to form image data of a non-interlaced image. The image data of the non-interlaced image written in the first image data area 3a is until the writing of the image data of the next non-interlaced image to be formed in the second image data area 3b is completed (about 1/30 seconds). In the meantime, it is encoded by the CPU 1.
[0029]
The CPU 1 writes code data generated by the encoding process in the first code data area 3c, and stores the code data written in the first code data area 3c in the hard disk 7 upon completion of the encoding process.
[0030]
On the other hand, when the writing of the image data of the non-interlaced image is completed in the second image data area 3b, the CPU 1 writes the code data generated by the encoding process in the second code data area 3d and performs the encoding process. The code data written in the second code data area 3d upon completion is stored in the hard disk 7.
[0031]
The CPU 1 once records all the non-interlaced image data before encoding written in the first image data area 3a and the second image data area 3b of the RAM 3 on the hard disk 7, and then records the recorded non-interlaced data. Image data of an image may be read out and sequentially encoded.
[0032]
3A to 3D are diagrams for explaining a phenomenon that occurs when a non-interlaced image is generated from an interlaced image obtained by the video camera 8, and a principle of determination of a moving speed that uses the phenomenon. . As shown in FIG. 3A, in the interlaced format, after scanning a line of one pixel (scan line indicated by a solid line), the line of the pixel immediately below (scan line indicated by a dotted line) is skipped and two pixels below. Are scanned again (scanning lines indicated by solid lines).
[0033]
After the image data of the interlaced image A is written in the first or second image data area 3a or 3b of the RAM 3, the video camera 8 immediately takes the pixel line that was not scanned last time as shown in FIG. (Scanning line indicated by a solid line in FIG. 3B) is scanned. Thereby, an interlaced image B is photographed. The image data of the scanned interlaced image B is not scanned by the CPU 1 in the first or second image data area 3a or 3b of the RAM 3 in which the data of the interlaced image A of the previous field is written. Writing is performed so as to supplement the image data of the line (the line indicated by the dotted line in FIG. 3A).
[0034]
During shooting, 1/60 seconds elapse after scanning a line of the interlaced image A in the field and then scanning a line of pixels immediately below the interlaced image B of the next field. As can be seen by comparing the interlaced image A shown in FIG. 3 (a) and the interlaced image B shown in FIG. 3 (b), the subject 15 moves rightward (naturally in the leftward direction) during the 1/60 second. There is also a move. For this reason, as shown in FIG. 3C, comb-shaped shifts of several pixels occur at both ends of the non-interlaced image formed in the first or second image data area 3a or 3b of the RAM 3. .
[0035]
The comb-shaped displacement amount L shown in FIGS. 3C and 3D becomes longer in proportion to the moving speed of the subject 15 in the interlaced image. As shown in FIG. 3 (d), the coefficient value of the 1LH subband obtained by two-dimensional discrete wavelet transform of the image data of the non-interlaced image having the comb-shaped shift is the horizontal edge component E1. The total increases, that is, in proportion to the moving speed of the subject 15 within the interlaced image. The coefficient value of the 1HL subband increases in proportion to the sum of the vertical edge components E2, but the subject movement follows the empirical rule that most subjects move in the horizontal direction in general shooting. Treated as being almost unchanged by speed. As will be described below, the image processing apparatus 10 determines the moving speed of the subject in the interlaced image using the above characteristics of the coefficient value of the 1LH subband.
[0036]
FIG. 4 is a flowchart of the main routine of the encoding process executed by the CPU 1. First, image data of a non-interlaced image that has been written to the first (or second) image data area 3a (or 3b) of the RAM 3 is read (step S1). The read image data of the non-interlaced image is converted into three signals of Y (luminance), Cb, and Cr (color difference) (so-called color conversion) (step S2).
[0037]
A two-dimensional discrete wavelet transform is executed as a frequency transform for each of the three signals Y, Cb, and Cr obtained by the color transform (step S3). A speed-corresponding quantization process is performed on the wavelet coefficients of the Y, Cb, and Cr components obtained by the conversion (step S4). The contents of the speed-corresponding quantization process will be described later in detail.
[0038]
An entropy encoding process including a coefficient modeling process and an arithmetic encoding process defined in JPEG 2000 is executed for each data of Y, Cb, and Cr components after the speed-corresponding quantization (step S5). After entropy encoding, the code data written in the first (or second) code data area 3c (or 3d) of the RAM 3 is stored in the hard disk 7 (step S6).
[0039]
If there is still image data of a frame to be processed (NO in step S7), the process returns to step S1, and the image data area (3b or 3b) on the side different from the image data area from which the image data was previously read is returned. The image data of the non-interlaced image written in 3a) is read out. When shooting by the video camera 8 is stopped and the encoding process of the image data of the non-interlaced image of the last frame (the frame composed of the field n-1 and the field n shown in FIG. 2) is completed (YES in step S7), The process is terminated.
[0040]
FIG. 5 is a flowchart of the velocity corresponding quantization (FIG. 4, step S4). Here, the moving speed of the subject in the frame is determined based on the subband 1HL coefficient obtained by the decomposition level 1 wavelet transform, and different quantization steps are performed for each subband, which is an image quality control unit, based on the determination result. Identifies and performs quantization using the identified quantization step. More specifically, as the moving speed of the subject increases, the number of quantization steps with respect to the coefficient of the high frequency component is reduced, thereby improving the reproducibility of the shift of the comb image. As a result, it is possible to prevent one vertical line in the original image from becoming two lines in the reproduced image and the image from being shaken left and right. In addition, by optimizing the quantization step used according to the speed for each image quality control unit, it is possible to achieve good data reduction while preventing deterioration in the image quality of the reproduced image.
[0041]
First, a sum sum1LH of absolute values of wavelet coefficients of a subband of 1LH is calculated (step S10). Next, a sum sum1HL of absolute values of wavelet coefficients of the 1HL subband is calculated (step S11). The sum sum2LH of the absolute values of the wavelet coefficients of the 2LH subband is calculated (step S12). The sum sum2HL of the absolute values of the wavelet coefficients of the 2HL subband is calculated (step S13). The variable speed is calculated by the operation of (sum1LH / sum1HL) / (sum2LH / sum2HL) (step S14).
[0042]
Here, sum1LH / sum1HL has a coefficient of 1HL, while the value of the coefficient of 1LH increases in proportion to the increase in the amount of edge in the horizontal direction of the image, that is, the movement speed of the subject within the frame. The value takes a value proportional to the vertical edge amount of the image, and from experience, the subject moves only in the horizontal direction in most cases, and therefore takes a relatively stable value. Reflects the amount of movement (movement speed) of the subject per unit time. Further, since the wavelet coefficients (2LH, 2HL, 2LL) at the decomposition level 2 are values obtained by calculating the above-described comb-shaped shift in units of lines for two pixels, they are relatively stable values regardless of the moving speed of the subject. It is thought that. Therefore, the mathematical formula executed in step S14 normalizes (sum1LH / sum1HL) with the vertical and horizontal ratios (sum2LH / sum2HL) of high frequencies other than combs included in the original image. For this reason, the variable speed can be considered to accurately reflect the moving speed of the subject within the frame.
[0043]
If the variable speed is larger than the experimentally obtained threshold value th1 (YES in step S15), a quantization process using the high-speed quantization step is executed (step S16). The quantization process is performed on the wavelet coefficients of the Y, Cb, and Cr components in each subband unit (LL, HL, LH, and HH) at each decomposition level (1 to 5).
[0044]
If the variable speed is equal to or less than the threshold value th1 (NO in step S15) but greater than the threshold value th2 (however, th2 <th1) (YES in step S16), medium speed quantization A quantization process using the steps is executed (step S18).
[0045]
If the variable speed is less than or equal to the threshold th2 (NO in step S17), a quantization process using the low-speed quantization step is executed (step S19).
[0046]
After executing any one of the above steps S16, S18, S19, the process returns to the main routine.
[0047]
FIG. 6 is a flowchart of the quantization process (step S16 in FIG. 5) using the high-speed quantization step. First, the value of the variable n representing the decomposition level is set to 5 (step S20). Normalized denominator data corresponding to the decomposition levels 5 to 1 of the Y, Cb, and Cr components shown in FIGS. 7A to 7C from the hard disk 7, and FIGS. 8A to 8C shown in FIG. The high-speed weight data corresponding to the decomposition levels 5 to 1 of the Y, Cb, and Cr components is read out to the work area (the free area shown in FIG. 1) of the RAM 3 (step S21).
[0048]
First, quantization is sequentially performed for all wavelet coefficients of the subband LL of decomposition level n of Y, Cb, and Cr components (step S22). Specifically, the quantization values “q” of the wavelet coefficients of the subband LL at the decomposition level n are sequentially calculated for each component of Y, Cb, and Cr using the following “Equation 1”, and the operation of the RAM 3 is performed. Save to area.
[0049]
[Expression 1]

[0050]
Here, the number of quantization steps (Δb) of the subband LL of the decomposition level n of the Y, Cb, and Cr components (the subband LL of the decomposition level n shown in the charts of (a) to (c) of FIG. Normalized denominator value) / (value of weighting factor for high speed of subband LL of decomposition level n shown in the chart of FIGS. 8A to 8C). The values of the number of quantization steps (Δb) obtained by the calculation are shown in (a) to (c) of FIG.
[0051]
Subsequently, quantization is performed on all wavelet coefficients of the subband HL at the decomposition level n for each Y, Cb, Cr component in the same procedure (step S23), and the decomposition level n is performed for each Y, Cb, Cr component. Quantization is performed on all wavelet coefficients of the subband LH (step S24), and quantization is performed on all wavelet coefficients of the subband LH of the division level n for each of Y, Cb, and Cr components (step S24). Finally, quantization is performed on all wavelet coefficients of the subband HH at the decomposition level n for each of the Y, Cb, and Cr components (step S25).
[0052]
If the value of the variable n representing the decomposition level is not 1 (NO in step S26), 1 is subtracted from the coefficient n (step S27), and then the processes of steps S23 to S25 are executed. On the other hand, if the value of the coefficient n is 1 (YES in step S26), the quantization process is terminated and the process returns to the flowchart of FIG.
[0053]
In the quantization process using the medium speed quantization step (FIG. 5, step S18), the weight coefficient data read from the hard disk 7 is the same as the quantization process using the high speed quantization step described above. The data shown in the charts of (a) to (c) of FIG. 10 is used instead of the data shown in the charts of (a) to (c) of FIG. is there. For this reason, a duplicate description is omitted.
[0054]
That is, the mathematical expression used in the quantization process (step S18) using the medium-speed quantization step is the same as the above “Expression 1”, but the quantization step number (Δb) is ((a of FIG. ) To (c) (normalized denominator values) / (medium speed weighting factors shown in FIGS. 10 (a) to 10 (c)) and (a) to (c) in FIG. It becomes each value shown in.
[0055]
Similarly, in the quantization process using the low-speed quantization step (FIG. 5, step S19), the weight coefficient data read from the hard disk 7 is the same as the quantization process using the high-speed quantization step described above. The data shown in the charts of (a) to (c) of FIG. 12 is used instead of the data shown in the charts of (a) to (c) of FIG. Because of this, duplicate explanations are omitted here.
[0056]
That is, the mathematical expression used in the quantization process using the low-speed quantization step (step S19) is also the same as the above “Equation 1”, but the quantization step number (Δb) is ((a) in FIG. 7). (A) to (c) in FIG. 13 obtained by calculating (normalized denominator value shown in (c)) / (medium speed weight coefficient shown in (a) to (c) in FIG. 12). It becomes each value shown.
[0057]
The weight component for each speed in FIGS. 8, 10, and 12 used when calculating the quantization step (Δb) described above corresponds to the high-frequency decomposition level and subband that affect the resolution of the reproduced image as the speed decreases. The image is set to increase the weight and suppress the quantization of the comb portion as the image moves faster. As described above, by performing appropriate quantization according to the moving speed of the subject, it is possible to increase the compression rate while suppressing deterioration in the quality of the reproduced image.
[0058]
Furthermore, instead of the weight components for each speed shown in FIGS. 8, 10, and 12, the weight components shown in FIGS. 14, 16, and 18 that suppress quantization by emphasizing LH components up to decomposition level 2 are used. May be. In this case, the number of quantization steps (Δb) has the values shown in FIGS. Thereby, the reproducibility of a comb-shaped part can be made more favorable.
[0059]
(3) Embodiment 2
The basic configuration of the image processing apparatus according to the second embodiment is the same as that of the image processing apparatus 10 according to the first embodiment. Hereinafter, the same components will be described using the same reference numerals. The image processing apparatus according to the second embodiment uses the wavelet coefficient obtained by the two-dimensional discrete wavelet transform as the moving speed of the subject in the encoding process performed by the CPU 1 that controls the image processing apparatus 10 according to the first embodiment. According to a table determined accordingly, it is reduced in units of bit planes (all data of the bit planes are replaced with 0). More specifically, as the moving speed of the subject in the non-interlaced image increases, the reproducibility of the shift of the comb-shaped image appearing in the non-interlaced image is enhanced, thereby affecting the reproducibility of the edge portion of the image. Reduce the number of deleted bit planes of wavelet coefficients of high frequency components (1HL, 1LH, etc.).
[0060]
FIG. 20 is a flowchart of the encoding process executed by CPU 1 of the image processing apparatus (not shown) in the second embodiment. First, image data of a non-interlaced image that has been written to the first (or second) image data area 3a (or 3b) of the RAM 3 is read (step S30). The read image data of the non-interlaced image is converted into three signals of Y (luminance), Cb, and Cr (color difference) (so-called color conversion) (step S31).
[0061]
A two-dimensional discrete wavelet transform is executed as a frequency transform for each of the three signals Y, Cb, and Cr obtained by the color transform (step S32). Scalar quantization based on JPEG2000 is performed on the wavelet coefficients of the Y, Cb, and Cr components obtained by the conversion (step S33).
[0062]
The wavelet coefficients of the respective Y, Cb, and Cr components after the quantization are decomposed into bit planes in subband units, and the influence on the reproduced image individually in subband units corresponding to the moving speed of the subject in the frame. A data reduction process for deleting data of bit planes with a small number is executed (step S34). The details of the processing will be described in detail later.
[0063]
After the speed corresponding data reduction process, an entropy encoding process (consisting of a coefficient modeling process and an arithmetic encoding process) compliant with JPEG2000 is executed (step S35). The CPU 1 stores the code data written in the first code data area 3c or the second code data area 3d of the RAM 3 in the hard disk 7 (step S36).
[0064]
If there is still image data of a frame to be processed (NO in step S37), the process returns to step S30, and the image data area (3b or 3b) on the side different from the image data area from which image data was previously read is returned. The image data of the non-interlaced image written in 3a) is read out. When shooting by the video camera 8 is stopped and the encoding processing of the image data of the non-interlaced image of all the frames (the frame consisting of the field n-1 and the field n shown in FIG. 2) is completed (YES in step S37). Then, the encoding process ends.
[0065]
FIG. 21 is a flowchart of the speed correspondence data reduction process (FIG. 20, step S34). First, the sum sum1LH of the absolute values of the wavelet coefficients of the 1LH subband is calculated (step S40). Next, the sum sum1HL of the absolute values of the wavelet coefficients of the 1HL subband is calculated (step S41). Subsequently, the sum sum2LH of the absolute values of the wavelet coefficients of the 2LH subband is calculated (step S42). The sum sum2HL of the absolute values of the wavelet coefficients of the 2HL subband is calculated (step S43). The variable speed is calculated from the calculation of (sum1LH / sum1HL) / (sum2LH / sum2LH) (step S44).
[0066]
The variable speed calculated in step S44 is understood to be a value proportional to the moving speed of the subject in the frame, as described in the first embodiment. Therefore, when the variable speed is larger than the experimentally obtained threshold value th3 (YES in step S45), it is determined that the moving speed of the subject in the frame is high, and high-speed data reduction processing is executed. (Step S46). Specifically, as shown in FIG. 22, the wavelet coefficients after the quantization of each component of Y, Cb, and Cr are developed on the bit plane for each subband and displayed in (a) to (c) of FIG. Data is discarded from the lowest bit plane of the subbands at each decomposition level by the number of bit planes to be performed (data is set to 0). For example, in the wavelet coefficient of the Y component, the data for one HH component at the decomposition level 1, that is, the lowest bit plane of the subband of 1HH is discarded.
[0067]
If the value of the variable speed is equal to or smaller than the threshold th3 but greater than the threshold th4 (YES in step S47), it is determined that the moving speed of the subject in the frame is low, and A speed data reduction process is executed (step S48). Specifically, the quantized wavelet coefficients of each component of Y, Cb, and Cr are developed on the bit plane for each subband, and each decomposition is performed for the number of sheets displayed in (a) to (c) of FIG. Discard data from the lowest bitplane of the level subband. For example, in the wavelet coefficient of the Y component, the data for the decomposition level 1 HL, LH, and HH components, that is, the lowest two bit planes of the subbands of 1HL, 1LH, and 1HH are discarded.
[0068]
On the other hand, if the value of the variable speed is equal to or smaller than the threshold th3 (NO in step S49), low-speed data reduction processing is executed (step S49). Specifically, the quantized wavelet coefficients of each component of Y, Cb, and Cr are developed on the bit plane for each subband, and each decomposition is performed for the number of sheets displayed in (a) to (c) of FIG. Discard data from the lowest bitplane of the level subband. For example, in the wavelet coefficients of the Y component, the data of the decomposition level 1 HL, LH, HH components, that is, the lowest 5 bit planes of the subbands of 1HL, 1LH, 1HH, 5 pieces, 8 pieces are discarded, Furthermore, HL, LH, and HH components of decomposition level 2, that is, data for one, one, and two least significant bit planes of 2HL, 2LH, and 2HH subbands are discarded.
[0069]
After completion of any of the steps S46, S48, S49, the process returns to the main routine shown in FIG.
[0070]
As described above, in the image processing apparatus according to the second embodiment, the reproducibility of the shift of the comb-shaped image appearing in the non-interlaced image is maintained as the moving speed of the subject in the non-interlaced image increases. Therefore, the number of deleted bit planes of wavelet coefficients of high frequency components (1HL, 1LH, etc.) is reduced. As a result, it is possible to achieve good data reduction while preventing image quality deterioration of the reproduced image.
[0071]
The contents shown in the charts of FIGS. 26, 27, and 28 are shown in FIGS. 26, 27, and 28 to suppress quantization by focusing on the LH component up to the decomposition level 2 instead of performing the bit truncation for each speed in FIGS. 23, 24, and 25. You may execute bit-trenching. Thereby, it is possible to improve the reproducibility of the shift of the comb-shaped image that appears particularly in the subject moving at high speed while maintaining a good compression rate.
[0072]
(4) Embodiment 3
The basic configuration of the image processing apparatus according to the third embodiment is the same as that of the image processing apparatus 10 according to the first embodiment. Hereinafter, the same components will be described using the same reference numerals. The image processing apparatus according to the third embodiment is finally unnecessary depending on the moving speed of the subject in the frame until the scheduled code amount set via the scheduled code amount setting screen (see FIG. 34) is reached. The data is reduced by discarding the entropy code. Discarding the entropy code is performed in units of bit planes of coefficients of image quality control units from the least important ones (for example, in JPEG 2000, bit planes of coefficients of subbands and code blocks). At this time, as the moving speed of the subject in the non-interlaced image increases, the low-order bit data of the code data obtained by entropy encoding the coefficient of the high frequency band that affects the reproducibility of the edge portion of the image is discarded. Adjust the importance so that the amount is small.
[0073]
FIG. 29 is a flowchart of the encoding process executed by the CPU of the image processing apparatus according to the third embodiment. First, image data of a non-interlaced image that has been written to the first (or second) image data area 3a (or 3b) of the RAM 3 is read (step S60). The read image data of the non-interlaced image is converted into three signals of Y (luminance), Cb, and Cr (color difference) (so-called color conversion) (step S61).
[0074]
A two-dimensional discrete wavelet transform is executed as a frequency transform for each of the three signals Y, Cb, and Cr obtained by the color transform (step S62). Scalar quantization based on JPEG2000 is performed on the wavelet coefficients of the Y, Cb, and Cr components obtained by the conversion (step S63). Entropy coding processing (consisting of coefficient modeling processing and arithmetic coding processing) based on JPEG2000 is executed on the wavelet coefficients of the Y, Cb, and Cr components after scalar quantization (step S64).
[0075]
The bit plane data of the code block consisting of a 32 × 32 pixel matrix, which is the image quality control unit of the code data obtained by the encoding process, is discarded in order of decreasing importance so that the code amount falls within the predetermined value (data The data reduction process is executed (replace the value of 0 with 0) (step S65). The details of the processing will be described in detail later.
[0076]
After the speed-corresponding data reduction process, the CPU 1 stores the code data written in the first code data area 3c or the second code data area 3d of the RAM 3 in the hard disk 7 (step S66).
[0077]
If there is still image data of a frame to be processed (NO in step S67), the process returns to step S60, and the second or first image data on the side different from the image data area from which the previous image data was read out. The image data of the non-interlaced image written in the area (3b or 3a) is read out. When shooting is stopped by the video camera 8 and the encoding process of the non-interlaced image of all the frames (the frame composed of the field n-1 and the field n shown in FIG. 2) is completed (YES in step S67), the encoding is performed. End the process.
[0078]
FIG. 30 is a flowchart of the speed correspondence data reduction process (FIG. 29, step S65). First, as shown in FIG. 31, the code data is divided into n code blocks of a 32 × 32 pixel matrix (for example, 1024 in the case of image data of 1024 × 1024 pixels) (step S70). . A variable CB for specifying a code block is set to 1 (step S71). The sum sum1LH of the absolute values of the code data of the code block CB of the 1LH subband is calculated (step S72). The sum sum1HL of the absolute values of the code data of the code block CB of the 1HL subband is calculated (step S73). The sum sum2LH of the absolute values of the code data of the code block CB of the 2LH subband is calculated (step S74). The sum sum2HL of the absolute values of the code data of the code block CB of the 2HL subband is calculated (step S75). The variable speed is calculated from the calculation of (sum1LH / sum1HL) / (sum2LH / sum2HL) (step S76).
[0079]
It is understood that the variable speed calculated in step S76 is a value proportional to the moving speed of the subject in the frame, as described in the first embodiment. Therefore, if the value of the variable speed is larger than the experimentally obtained threshold value th5 (YES in step S77), it is determined that the subject in the frame is moving at high speed, and the visual for high speed shown in FIG. Using the weight, the code importance G (CB) is calculated (step S78). The importance G (CB) of the code is obtained for each of 16 bit planes of a 32 × 32 bit code block as shown in FIG. The details of the processing will be described in detail later.
[0080]
On the other hand, if the value of the variable speed is less than or equal to the experimentally obtained threshold value th5 but not less than the threshold value th6 (NO in step S79), it is determined that the subject in the frame is moving at medium speed. Then, the sign importance G (CB) calculation process is executed using the medium-speed visual weight shown in FIG. 10 (step S80). The importance G (CB) of the code is obtained for each of 16 bit planes of a 32 × 32 bit code block.
[0081]
When the value of the variable speed is equal to or less than the threshold value th6 obtained experimentally (NO in step S79), the calculation process of the sign importance G (CB) using the low-speed visual weight shown in FIG. 12 is executed. (Step S81). The importance G (CB) of the code is obtained for each of 16 bit planes of a 32 × 32 bit code block.
[0082]
If CB ≠ n (NO in step S82), 1 is added to variable CB (step S83), and the process returns to step S72. On the other hand, if CB = n (YES in step S82), the process proceeds to step S84 in the flowchart shown in FIG. 33, and 16 bit planes of n code blocks, that is, n × 16 bit planes in total are obtained. The values of the importance G (CB) of the codes are rearranged (sorted) in order from the lowest together with the information of the number m of bit planes from the code block CB and the LSB of the code block (step S84).
[0083]
After the sorting process, the scheduled code amount DS (abbreviation of Data Size) setting screen 50 shown in FIG. 34 is displayed on the display 6 (step S85).
[0084]
As shown in FIG. 32, a scheduled code amount (for example, 150 kbit) is set using the keyboard 4 in the numerical value setting field 51 of the planned code amount provided in the setting screen 50, and the setting key 52 is moved by the mouse 5. If the button is clicked (YES in step S86), the data for one bit plane of the code block CB having the lowest importance G (CB) m among all the bit planes currently positioned on the LSB side is discarded ( Step S87). When the code amount after the data is discarded is equal to or less than the scheduled code amount DS set in the setting screen 50 (NO in step S88), the process returns to step S87, and among all the bit planes located closest to the LSB side at that time, The data of one bit plane of the code block CB having the lowest importance G (CB) is discarded. When the code amount after the data discard becomes a value less than the scheduled code amount DS (YES in step S88), the speed-corresponding data reduction process is terminated and the process returns to the main routine.
[0085]
FIG. 35 is a flowchart of the process of calculating the importance G (CB) m of the code using the visual weight for high speed (FIG. 30, step S78). First, the value of the variable m representing the number of bit planes constituting one code block (16 in this example) is set to 1 (step S90). The importance G (CB) m of the data of the m-th bit plane counted from the LSB of the code block CB (CB is 1 to n) is obtained (step S91).
[0086]
The importance G (CB) m is (increment of quantization error when all the data of the mth bit plane of the code block CB are discarded × visual weighting factor) / (effective code in the bit plane) As shown in the flowchart, {2 ^ (m−1) −2 ^ (m−2)} × (effective number included in the m-th bit plane counted from the LSB). The number of codes) × visual weighting coefficient] / (sum of effective codes in the bit plane) can be obtained by an arithmetic expression.
[0087]
The reason why the importance degree G (CB) m can be approximated to the above arithmetic expression will be described below. There are various methods for calculating the increment of quantization error when all the data of the m-th bit plane of the code block CB are discarded. For example, a mathematically exact method is described in “Kluwer Publishing in 2002”. It is described in “JPEG2000: Image Compression Fundamentals, Standards and Practice” by DSTaubuman and MW Marcelin issued by the company. However, in the present image processing apparatus 10, “quantization error per one wavelet coefficient when the code of the mth bit plane counted from the least significant bit (LSB) is discarded” is 2 ^ (m−1). Approximate. Discarding one bit plane from the LSB is equivalent to dividing the coefficient by 2 from an error viewpoint, and the error is 2 ^ (m-1) in terms of probability.
[0088]
Therefore, “increase in quantization error when the code of the mth bit plane counted from the LSB of the code block CB is discarded” is {2 ^ (m−1) −2 ^ (m−2)} ×. (The number of effective coefficients included in the m-th bit plane counted from the LSB). Therefore, the importance G (CB) m of the m-th bit plane counted from the LSB of the code block CB is {2 ^ (m−1) −2 ^ (m−2)} × (m counted from the LSB. It can be approximated to an arithmetic expression of (the number of effective codes included in the first bit plane) × visual weighting factor] / (sum of effective codes in the bit plane).
[0089]
If the value of the variable m is not 16 after the importance G (CB) m is calculated (NO in step S92), 1 is added to the variable m and the process returns to step S91 to determine the importance of the next bit plane of the code block CB. Degree G (CB) m is obtained. If the variable m is 16 (YES in step S92), it is determined that the calculation of the importance levels of all 16 bit planes of the code block CB has been completed, and the process ends and returns.
[0090]
It should be noted that the order of discarding the bit plane data is determined in the order of the Lagrange undetermined except that the order is the effective bit plane located on the most LSB side of each code block CB and having the lowest importance G (CB) m. You may decide using a multiplier method. The implementation method of Lagrange's undetermined multiplier method in JPEG2000 is also described in detail in the above-mentioned book “DSPEGubman et al.,“ JPEG2000: Image Compression Fundamentals, Standards and Practice ”.
[0091]
Code importance G (CB) m calculation processing using medium-speed visual weight (FIG. 30, step S80) and code importance G (CB) m calculation processing using low-speed visual weight ( The processing content of step S81) is merely to change the visual weighting coefficient used when calculating the significance G (CB) m of each code to the content of the chart shown in FIGS. 10 and 12, and the other processing content Are exactly the same, so a duplicate description is omitted here.
[0092]
In addition, as a visual weighting factor for high speed, medium speed, and low speed, a diagram that suppresses quantization by placing emphasis on the LH component up to decomposition level 2 instead of the weight component for each speed in FIGS. 8, 10, and 12. 14, 16 and 18 may be used. Thereby, the reproducibility of a comb-shaped part can be made more favorable.
[0093]
35, instead of the calculation formula for importance G (CB) m shown in step S91 in FIG. 35, (quantization error increment x visual weight when all data of m-th bit plane of code block CB are discarded) A calculation formula of (coefficient / masking coefficient) / (total amount of effective codes in the bit plane) may be used. Here, the masking coefficient is a coefficient defined by (sum of absolute values of coefficients included in code block CB / coefficient included in code block CB) ^ α, where α is a result of an experiment, 0 <α ≦ 1, and α = 1 is adopted as an example.
[0094]
Further, instead of the masking coefficient, (sum of absolute values of coefficients of 1LH subband of code block CB) / (sum of absolute values of coefficients of 1HL subband of code block CB) or (code block It is also conceivable to use a coefficient obtained from the arithmetic expression of CB 1LH subband code amount) / (code block CB 1HL subband code amount) as a comb-type index value appearing in a non-interlaced image.
[0095]
As described above, in the image processing apparatus according to the third embodiment, in the entropy encoding process, the importance is checked in the bit plane unit of the code block that is the image quality control unit, and becomes less than the scheduled code amount DS. Until the bit plane data is less important, the data is discarded in order. At this time, the reproducibility of the edge portion of the image can be degraded as the moving speed of the subject in the non-interlaced image increases. The importance is adjusted so that the amount of discarding the lower-order bit data of the code data obtained by entropy encoding the band coefficient is reduced. As a result, it is possible to achieve good image compression without affecting the reproduced image most.
[0096]
【The invention's effect】
According to the image processing apparatus or the image processing method of the present invention, an image processing apparatus that processes a non-interlaced image (frame) generated from an interlaced image of two consecutive fields, the subject moving within the frame Execute data reduction processing (including data reduction by quantization) so as to improve the reproducibility of the shift of comb-shaped images that appear in non-interlaced images in consideration of human visual characteristics as the speed increases. be able to.
[Brief description of the drawings]
FIG. 1 is an overall configuration diagram of an image processing apparatus according to a first embodiment.
FIG. 2 is a diagram illustrating a state of an interlaced image captured by a video camera.
FIGS. 3A to 3D are diagrams for explaining a comb-shaped shift that occurs when a non-interlaced image is formed from a non-interlaced image. FIGS.
FIG. 4 is a flowchart of encoding processing executed by a CPU.
FIG. 5 is a flowchart of speed-corresponding quantum processing.
FIG. 6 is a flowchart of a quantization process using a high-speed quantization step.
FIGS. 7A to 7C are tables showing normalized denominators of subbands with respect to decomposition levels 5 to 1 of Y, Cb, and Cr components used when obtaining a quantization step.
FIGS. 8A to 8C are tables showing high-speed weighting coefficients for each subband with respect to decomposition levels 5 to 1 of Y, Cb, and Cr components used when obtaining a quantization step.
9A to 9C are tables showing high-speed quantization steps for each subband with respect to decomposition levels 5 to 1 of Y, Cb, and Cr components.
FIGS. 10A to 10C are tables showing weighting factors for medium speeds of subbands for decomposition levels 5 to 1 of Y, Cb, and Cr components used in obtaining a quantization step. .
FIGS. 11A to 11C are tables showing medium-speed quantization steps of subbands for decomposition levels 5 to 1 of Y, Cb, and Cr components, respectively.
FIGS. 12A to 12C are tables showing low-speed weighting coefficients for each subband with respect to decomposition levels 5 to 1 of Y, Cb, and Cr components used when obtaining a quantization step.
FIGS. 13A to 13C are tables showing low-speed quantization steps for each subband with respect to decomposition levels 5 to 1 of Y, Cb, and Cr components.
FIGS. 14A to 14C are tables showing high-speed weighting coefficients for each subband with respect to decomposition levels 5 to 1 of Y, Cb, and Cr components used when obtaining a quantization step.
FIGS. 15A to 15C are tables showing high-speed quantization steps for each subband with respect to decomposition levels 5 to 1 of Y, Cb, and Cr components;
FIGS. 16A to 16C are tables showing weight coefficients for medium speeds of subbands for decomposition levels 5 to 1 of Y, Cb, and Cr components used when obtaining a quantization step; .
FIGS. 17A to 17C are tables showing medium-speed quantization steps for each subband with respect to decomposition levels 5 to 1 of Y, Cb, and Cr components;
FIGS. 18A to 18C are tables showing weight coefficients for low speed of each subband with respect to decomposition levels 5 to 1 of Y, Cb, and Cr components used when obtaining a quantization step.
19A to 19C are tables showing low-speed quantization steps for each subband with respect to decomposition levels 5 to 1 of Y, Cb, and Cr components.
FIG. 20 is a flowchart of an encoding process executed by the CPU of the image processing apparatus according to the second embodiment.
FIG. 21 is a flowchart of speed correspondence data reduction processing;
FIG. 22 is a diagram schematically showing the content of high-speed bit truncation of the Y component.
FIGS. 23A to 23C are tables showing the number of high-speed bit truncations in each subband with respect to decomposition levels 5 to 1 of Y, Cb, and Cr components.
FIGS. 24A to 24C are tables showing the number of bit truncations for medium speed for each subband with respect to decomposition levels 5 to 1 of each component of Y, Cb, and Cr.
FIGS. 25A to 25C are tables showing the number of low-speed bit truncations in each subband with respect to decomposition levels 5 to 1 of Y, Cb, and Cr components.
FIGS. 26A to 26C are tables showing the number of high-speed bit truncations in each subband with respect to decomposition levels 5 to 1 of Y, Cb, and Cr components when different weighting factors are used. .
FIGS. 27A to 27C are tables showing the number of bit truncations for medium speed for each subband with respect to decomposition levels 5 to 1 of Y, Cb, and Cr components when different weighting factors are used. is there.
FIGS. 28A to 28C are tables showing the number of low-speed bit truncations in each subband with respect to decomposition levels 5 to 1 of Y, Cb, and Cr components when different weighting factors are used. .
FIG. 29 is a flowchart of an encoding process executed by the CPU of the image processing apparatus according to the third embodiment.
FIG. 30 is a flowchart of a speed correspondence data reduction process.
FIG. 31 is a diagram illustrating a state in which code data is divided into n code blocks.
FIG. 32 is a diagram illustrating a state where a code block is disassembled into bit planes.
FIG. 33 is a continuation of the flowchart of the speed correspondence data reduction process.
FIG. 34 is a setting screen for a scheduled code amount DS displayed on the display;
FIG. 35 is a flowchart of code importance calculation processing using high-speed visual weights.
[Explanation of symbols]
1 CPU, 2 ROM, 3 RAM, 3a 1st image data area, 3b 2nd image data area, 3c 1st code data area, 3d 2nd code data area, 4 keyboard, 5 mouse, 6 display, 7 hard disk, 8 Video camera, 10 image processing device.

Claims (20)

In an image processing apparatus that encodes image data of a non-interlaced image composed of two consecutive interlaced images into code data, data reduction means for reducing the amount of the code data,
Subband conversion means for performing subband conversion on the image data;
Calculating means for calculating a value obtained by dividing the sum of absolute values of the coefficients of the 1LH subband generated by the subband converting means by the sum of absolute values of the coefficients of the 1HL subband;
Data reduction means for reducing the amount of reduction of code data at locations where the reproducibility of the edge portion of the image is degraded as the value calculated by the calculation means increases,
An image processing apparatus comprising: In an image processing apparatus that encodes image data of a non-interlaced image composed of two consecutive interlaced images into code data, data reduction means for reducing the amount of the code data,
Subband conversion means for performing subband conversion on the image data;
Wherein the absolute value divided by the sum of the absolute value coefficients 1HL subband sum of coefficients of subbands 1LH subband generated by the converting means, the absolute value 2HL subband sum of the coefficients of 2LH subbands A calculation means for calculating the value divided by the sum of the absolute values of the coefficients of
Data reduction means for reducing the amount of reduction in code data at locations where the reproducibility of the edge portion of the image is degraded as the value calculated by the calculation means increases,
An image processing apparatus comprising: In an image processing apparatus that encodes image data of a non-interlaced image composed of two consecutive interlaced images into code data, data reduction means for reducing the amount of the code data,
Subband conversion means for performing subband conversion on the image data;
Division means for dividing the subband generated by the subband conversion means into a plurality of rectangles (code blocks);
A value obtained by dividing the sum of the absolute values of the coefficients included in one rectangle of the 1LH subband divided by the dividing unit by the sum of the absolute values of the coefficients included in the rectangle at the same position as the rectangle of the 1HL subband. Calculating means for calculating
Data reduction means for reducing the amount of reduction in code data at locations where the reproducibility of the edge portion of the image is degraded as the value calculated by the calculation means increases,
An image processing apparatus comprising: In an image processing apparatus that encodes image data of a non-interlaced image composed of two consecutive interlaced images into code data, data reduction means for reducing the amount of the code data,
Subband conversion means for performing subband conversion on the image data;
Division means for dividing the subband generated by the subband conversion means into a plurality of rectangles (code blocks);
A value obtained by dividing the sum of the absolute values of the coefficients included in one rectangle of the 1LH subband divided by the dividing unit by the sum of the absolute values of the coefficients included in the rectangle at the same position as the rectangle of the 1HL subband. Is obtained by dividing the sum of the absolute values of the coefficients included in the rectangle at the same position as the rectangle of the 2LH subband by the sum of the absolute values of the coefficients included in the rectangle at the same position as the rectangle of the 2HL subband , A calculating means for calculating the divided value;
Data reduction means for reducing the amount of reduction in code data at locations where the reproducibility of the edge portion of the image is degraded as the value calculated by the calculation means increases,
An image processing apparatus comprising: In an image processing apparatus that encodes image data of a non-interlaced image composed of two consecutive interlaced images into code data, data reduction means for reducing the amount of the code data,
Subband conversion means for performing subband conversion on the image data;
Division means for dividing the subband generated by the subband conversion means into a plurality of rectangles (code blocks);
When the coefficient included in one rectangle of the 1LH subband divided by the dividing unit is encoded, the code amount included in the rectangle at the same position as the rectangle of the 1HL subband is encoded. A calculation means for calculating a value divided by the code amount;
Data reduction means for reducing the amount of reduction in code data at locations where the reproducibility of the edge portion of the image is degraded as the value calculated by the calculation means increases,
An image processing apparatus comprising: In the image processing device according to any one of claims 1 to 5,
The data reduction means is configured to reduce the reproducibility of the shift of the comb image generated between the interlaced images of the two fields as the moving speed of the subject in the non-interlaced image increases. An image processing apparatus that reduces the amount of reduction. The image processing apparatus according to claim 1, frequency-converts the image data of the non-interlaced image, quantizes the coefficient for each frequency obtained by the conversion, and entropy-encodes the quantized coefficient. An image processing device for generating code data
The data reduction means reduces the code data by making the number of quantization steps used when performing the quantization larger than a standard value, and increases as the moving speed of the subject in the non-interlaced image increases. An image processing apparatus that reduces the value of the number of quantization steps used when quantizing a frequency band coefficient. The image processing apparatus according to claim 1, frequency-converts the image data of the non-interlaced image, quantizes the coefficient for each frequency obtained by the conversion, and entropy-encodes the quantized coefficient. An image processing device for generating code data
The data reduction means reduces the code data by discarding the low-order bit data of the coefficient after the quantization, and the low-order bits of the high frequency band coefficient as the subject moving speed in the non-interlaced image increases. An image processing apparatus that reduces the amount of data discarded. The image processing apparatus according to claim 1, frequency-converts the image data of the non-interlaced image, quantizes the coefficient for each frequency obtained by the conversion, and entropy-encodes the quantized coefficient. An image processing device for generating code data
The data reduction means performs data reduction by discarding the lower bit data of the code data, and reduces the amount of discard of the lower bit data of the code data as the moving speed of the subject in the non-interlaced image increases. An image processing apparatus.   The image processing device according to claim 7, wherein the image processing device performs two-dimensional discrete wavelet transform as frequency transform. In an image processing method for encoding image data of a non-interlaced image composed of two consecutive interlaced images into code data, a data reduction step for reducing the amount of code data,
A subband conversion step for performing subband conversion on the image data;
A calculation step of calculating a value obtained by dividing the sum of the absolute values of the coefficients of the 1LH subband generated in the subband conversion step by the sum of the absolute values of the coefficients of the 1HL subband;
As the value calculated in the calculation step increases, a data reduction step for reducing the amount of code data reduction in a portion that degrades the reproducibility of the edge portion of the image,
An image processing method comprising: In an image processing method for encoding image data of a non-interlaced image composed of two consecutive interlaced images into code data, a data reduction step for reducing the amount of code data,
A subband conversion step for performing subband conversion on the image data;
Wherein the absolute value divided by the sum of the absolute value coefficients 1HL subband sum of coefficients of the sub-band transform 1LH subband generated in step, an absolute value 2HL subband sum of the coefficients of 2LH subbands A calculation step of calculating the value divided by the sum of the absolute values of the coefficients of
As the value calculated in the calculation step increases, a data reduction step of reducing the amount of code data reduction in a portion that degrades the reproducibility of the edge portion of the image,
An image processing method comprising: In an image processing method for encoding image data of a non-interlaced image composed of two consecutive interlaced images into code data, a data reduction step for reducing the amount of code data,
A subband conversion step for performing subband conversion on the image data;
A division step of dividing the subband generated in the subband conversion step into a plurality of rectangles (code blocks);
A value obtained by dividing the sum of the absolute values of the coefficients included in one rectangle of the 1LH subband divided in the dividing step by the sum of the absolute values of the coefficients included in the rectangle at the same position as the rectangle of the 1HL subband. A calculation step of calculating
A data reduction step of reducing the amount of code data reduction at a location that degrades the reproducibility of the edge portion of the image with an increase in the value calculated by the calculation means;
An image processing method comprising: In an image processing method for encoding image data of a non-interlaced image composed of two consecutive interlaced images into code data, a data reduction step for reducing the amount of code data,
A subband conversion step for performing subband conversion on the image data;
A division step of dividing the subband into a plurality of rectangles (code blocks) in the subband conversion step;
A value obtained by dividing the sum of the absolute values of the coefficients included in one rectangle of the 1LH subband divided in the dividing step by the sum of the absolute values of the coefficients included in the rectangle at the same position as the rectangle of the 1HL subband. Is obtained by dividing the sum of the absolute values of the coefficients included in the rectangle at the same position as the rectangle of the 2LH subband by the sum of the absolute values of the coefficients included in the rectangle of the same position as the rectangle of the 2HL subband. A calculation step to calculate,
As the value calculated in the calculation step increases, a data reduction step of reducing the amount of code data reduction in a portion that degrades the reproducibility of the edge portion of the image,
An image processing method comprising: In an image processing method for encoding image data of a non-interlaced image composed of two consecutive interlaced images into code data, a data reduction step for reducing the amount of code data,
A subband conversion step for performing subband conversion on the image data;
A division step of dividing the subband generated in the subband conversion step into a plurality of rectangles (code blocks);
When the coefficient included in one rectangle of the 1LH subband divided in the dividing step is encoded, the amount of code included in the rectangle at the same position as the rectangle of the 1HL subband is encoded. A calculation step of calculating a value divided by the code amount;
As the value calculated in the calculation step increases, a data reduction step of reducing the amount of code data reduction in a portion that degrades the reproducibility of the edge portion of the image,
An image processing method comprising: In the image processing method in any one of Claims 11-15,
In the data reduction step, as the moving speed of the subject in the non-interlaced image increases, the code data of the portion that deteriorates the reproducibility of the shift of the comb-shaped image generated between the interlaced images of the two fields. An image processing method that reduces the amount of reduction. The image processing method according to any one of claims 11 to 16, wherein the image data of the non-interlaced image is frequency-converted, the coefficient for each frequency obtained by the conversion is quantized, and the quantized coefficient is entropy-coded. Image processing method for generating code data
The data reduction step reduces the code data by increasing the number of quantization steps used when performing the quantization above the standard value, and increases with the movement speed of the subject in the non-interlaced image. An image processing method for reducing the number of quantization steps used when quantizing a frequency band coefficient. The image processing method according to any one of claims 11 to 16, wherein the image data of the non-interlaced image is frequency-converted, the coefficient for each frequency obtained by the conversion is quantized, and the quantized coefficient is entropy-coded. Image processing method for generating code data
The data reduction step reduces the code data by discarding the low-order bit data of the coefficient after the quantization, and the low-order bits of the high frequency band coefficient as the subject moving speed in the non-interlaced image increases. An image processing method that reduces the amount of data discarded. The image processing method according to any one of claims 11 to 16, wherein the image data of the non-interlaced image is frequency-converted, the coefficient for each frequency obtained by the conversion is quantized, and the quantized coefficient is entropy-coded. Image processing method for generating code data
The data reduction step performs data reduction by discarding the lower bit data of the code data, and the amount of discard of the lower bit data of the code data is reduced as the moving speed of the subject in the non-interlaced image increases. Image processing method.   20. The image processing method according to claim 11, wherein two-dimensional discrete wavelet transform is performed as frequency transform.

Images