JP2005039456A - Image processing method and apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing method and apparatus for suppressing a gradation jump and a deviation in a color tone even when the image processing apparatus applies the processing of enhancing the distribution of the lightness to a digital image. <P>SOLUTION: The image processing method includes: an extracting step of extracting a lightness component from image data; a scale conversion step of obtaining a distribution of the lightness component at a comparatively greater scale; an improvement step of using the lightness component and an output of the scale conversion step to improve the lightness component distribution of the image data; and a gradation compression step of discriminating whether or not the image data rebuilt up in response to the improved lightness component exist within a prescribed color range and carrying out gradation compression processing when the color range is at the outside of the prescribed range. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

【００３２】
但し、Ｙ’（ｘ，ｙ）、Ｆ_ｎ（ｘ，ｙ）、ｗ_ｎ、ｎ、γはそれぞれ改善された輝度成分の出力、ガウシアン関数、尺度間の重み、尺度を表すパラメータ、改善の度合いを表すパラメータである。また、＊は積和演算を表す。
【００３３】
なお、尺度間の重みは尺度の標準偏差を調整することで省略可能（単純な平均に置き換わる）であること、また、（式１）のように対数変換された値を出力するよりも、逆変換（ｅｘｐ演算）により元の輝度単位に戻した方が、改善された画像データの画質として好ましいことが分かっている。従って、以下の（式２）に示した出力を改善された輝度成分とすることがより好ましい。
【外２】

【００３４】
但し、Ａｖｇは平均値演算を表す。
【００３５】
（式２）の代わりに、以下に示す（式３）としてもよい。
【外３】

【００３６】
なお、複数尺度でのスケール変換出力の平均値演算をＳ１０３のスケール変換の処理で行い、複数尺度でのスケール変換出力の平均値をスケール変換された輝度成分の分布としてもよい。
【００３７】
この輝度変換をハードウェアで構成する場合には、例えば、平均値演算回路、ルックアップテーブルを作成する回路、テーブル記憶部、テーブル参照回路（ガンマ変換の部分）、除算回路によって構成できる。なお、平均値演算回路はスケール変換手段に設けてもよい。
【００３８】
ここで、Ｓ１０４におけるパラメータ調整方法の一例をＳ１０５における（式３）で輝度変換を行う場合について説明する。
【００３９】
まず、画像データの輝度成分を所定の輝度値の範囲に分けて輝度ヒストグラムを作成する。そして、暗い方から積算したヒストグラムの頻度の全サンプリングに対する割合が所定の割合になる輝度値（この輝度値をＹ_０とする）を求める。このとき、求めた輝度値Ｙ_０が所定の輝度値（この輝度値をＹ_１とする。但し、Ｙ_０≦Ｙ_１）になるようなγを改善の度合いを表すパラメータとする。なお、このγは、（式３）の分母の［ ］内をＹ（ｘ，ｙ）とほぼ等しいと仮定すると、以下に示す（式４）により求めることができる。
【外４】

【００４０】
なお、本実施形態では、画像データの輝度成分をもとにパラメータを自動的に調整するようにしたが、画像データのＲＧＢの画素値をもとにパラメータを自動的に調整するようにしてもよい。また、パラメータを入力インターフェースによって変更可能とし、変更したパラメータの値にしたがって輝度分布を変換し、画像データを再構成し、改善後の画像データをディスプレイに表示するようにして、対話的に調整するようにしてもよい。
【００４１】
以上の例では、従来技術１をもとに処理する画像データに応じて最適に明るさの分布の改善が行う方法について説明したが、以下の例では、従来技術２をもとに処理する画像データに応じて最適に明るさの分布の改善が行う方法について説明する。なお、上記実施形態との差異は主にＳ１０５での画像データの輝度成分の分布を改善する処理であるので、この部分を中心に説明する。
【００４２】
従来技術２によれば、異なる尺度の輝度成分を用いて輝度変換後のハロの発生度合いを評価し、ハロ発生による画質の不具合がないように最適な尺度を決定し、輝度変換を行うようにしている。これに本発明を適用した場合の例を以下の（式５）に示す。
【外５】

【００４３】
但し、Ｖ（ｘ，ｙ）は複数のスケール変換した輝度成分から選択された輝度成分、ａは輝度成分の強度を調整するパラメータ、ｅは輝度成分の改善の度合いを表すパラメータである。なお、輝度成分改善処理における調整方法の変更に伴い、Ｓ１０４での改善パラメータの調整方法も変更すべきであることはいうまでもない。
【００４４】
また、従来技術２にもとづいて説明した例において、輝度成分の改善の度合いとともに画像の輝度成分そのものを調整するようにしたが、このように本発明と画像の輝度成分そのものを調整する処理と組み合わせてもよいことはいうまでもない。
【００４５】
また、Ｓ１０５の処理において、（式３）の分母の［ ］内の値として、輝度成分に対してａｃｍ Ｔｒａｎｓａｃｔｉｏｎｓ ｏｎ Ｇｒａｐｈｉｃｓ， ＪＵＬＹ ２００２， Ｖｏｌ．２１， Ｎｏ．３ 中の”Ｆａｓｔ Ｂｉｌａｔｅｒａｌ Ｆｉｌｔｅｒｉｎｇ ｆｏｒ ｔｈｅ Ｄｉｓｐｌａｙ ｏｆ Ｈｉｇｈ−Ｄｙｎａｍｉｃ−Ｒａｎｇｅ Ｉｍａｇｅｓ”と題するＤｕｒａｎｄらの報告にあるようなＢｉｌａｔｅｒａｌ Ｆｉｌｔｅｒｉｎｇを適用してもよい。この場合、Ｓ１０３のスケール変換処理であらかじめ輝度成分にＢｉｌａｔｅｒａｌ Ｆｉｌｔｅｒｉｎｇ処理を行う。
【００４６】
次に、改善された輝度成分とＳ１０２で変換された色成分Ｘ（ｘ，ｙ）、Ｚ（ｘ，ｙ）とを合成し、画像データを再構成する。（Ｓ１０６）
画像データの再構成についての詳細アルゴリズムを図３に示す。
【００４７】
まず、画像データをＲＧＢのデータに変換する。（Ｓ６１）
ここで、再構成後の画像データの色ができるだけ変化しないように、色成分を輝度成分の変更にしたがって修正する。例えば、色成分Ｘ（ｘ，ｙ）、Ｚ（ｘ，ｙ）にそれぞれ輝度成分の変更前後の比Ｙ’（ｘ，ｙ）／Ｙ（ｘ，ｙ）を乗算する。そして、Ｘ、Ｙ、ＺのデータからＲＧＢのデータを求める。ここでの処理はＳ１０２における処理の逆変換である。したがって、３行３列のマトリクス演算および逆ガンマ変換の処理を行う。なお、Ｓ１０２で輝度成分を抽出する方法としてＲＧＢからＹＣｂＣｒ変換など別方式を用いた場合には、本処理において対応する逆変換の処理を行うべきであることはいうまでもない。
【００４８】
次に、ＲＧＢ画像データに対し、最大値を求める。（Ｓ６２）
次に、求めた最大値が１を越えるかどうか判定する。（Ｓ６３）
求めた最大値が１を越えない場合は、ＲＧＢデータがｓＲＧＢの範囲の画像データで表現できるので、ＲＧＢ各８ビットに量子化し、出力する。（Ｓ６６）
このように、Ｓ６６では、輝度改善されたデータが所定の色範囲を越えた際に、輝度改善されたデータの色範囲に応じて階調圧縮を行う。
【００４９】
一方、求めた最大値が１を越える場合は、ＲＧＢデータがｓＲＧＢの範囲の画像データで表現できないので、１を越えた画素値を１に置き換える方式などでＲＧＢ各８ビットに量子化すると階調とびや色調ずれの原因となる。そこで、ＲＧＢ画像データに対して階調圧縮を行うべく、まずは階調圧縮のパラメータを決定し（Ｓ６４）、パラメータに基づいて階調圧縮を行う。（Ｓ６５）
階調圧縮は、ＲＧＢデータの最大値をｐ（ｐ＞１）とすると、画素値ｐがｓＲＧＢ色空間での最大値１になるように行う。例えば、階調圧縮前のＲＧＢデータをそれぞれＲ１、Ｇ１、Ｂ１、階調圧縮後のＲＧＢデータをそれぞれＲ２、Ｇ２、Ｂ２とするとき、
Ｒ２＝ｋ・Ｒ１、Ｇ２＝ｋ・Ｇ１、Ｂ２＝ｋ・Ｂ１ （式６）
にて、画像中の全ＲＧＢデータについて階調圧縮する。但し、ｋは階調圧縮のパラメータで、ｋ＝１／ｐである。但し、ｐが所定の値（ｐｍ＞１とする）よりも大きい場合には、階調圧縮の度合いが大きくなりすぎ、画像全体が暗くなってしまうので、その場合はｋ＝１／ｐｍを用いて階調圧縮を行う。
【００５０】
また、上記の例では階調圧縮を１次変換で行ったが、２次変換で行うようにしてもよい。例えば、ＲＧＢデータの最大値をｐ（ｐ＞１）とすると、画素値ｐがｓＲＧＢ色空間での最大値１になるように行う。また、画素値が１の場合に所定の値、例えば、１と１／ｐとの平均値になるように定めると、
Ｒ２＝（ｋ２・Ｒ１＋ｋ１）Ｒ１ 、
Ｇ２＝（ｋ２・Ｇ１＋ｋ１）Ｇ１ 、
Ｂ２＝（ｋ２・Ｂ１＋ｋ１）Ｂ１ （式７）
にて、画像中の全ＲＧＢデータについて階調圧縮する。但し、ｋ２、ｋ１は階調圧縮のパラメータで、ｋ２＝（ｐ−１）／（２ｐ・（ｐ＋１））、ｋ１＝（ｐ＋１）／２ｐ−ｋ２である。但し、ｐが所定の値（ｐｍ＞１とする）よりも大きい場合には、階調圧縮の度合いが大きくなりすぎ、画像全体が暗くなってしまうので、その場合はｋ２＝（ｐｍ−１）／（２ｐｍ・（ｐｍ＋１））、ｋ１＝（ｐｍ＋１）／２ｐｍ−ｋ２を用いて階調圧縮を行う。
【００５１】
その他、階調圧縮を以下の（式８）に示すようなγ変換による方式で行ってもよい。
【００５２】
Ｒ２＝（ｋ３・Ｒ１）^γ 、
Ｇ２＝（ｋ３・Ｇ１）^γ 、
Ｂ２＝（ｋ３・Ｂ１）^γ （式８）
但し、ｋ３、γは階調圧縮のパラメータである。
【００５３】
以上、ＲＧＢデータの最大値が１を越える場合は、階調圧縮を行い、ＲＧＢデータがｓＲＧＢの範囲の画像データで表現できるように変換して、ＲＧＢ各８ビットに量子化し、出力する。（Ｓ６６）
なお、Ｓ６２〜Ｓ６５で説明した階調圧縮による階調とび、色調のずれの抑制処理はＳ１０５における輝度改善処理にも適用することができる。例えば、（式３）によって輝度変換を行った前後の輝度分布の範囲を比較してみると、輝度変換前の輝度分布の範囲は０〜１の範囲内であるのに対し、輝度変換後の輝度分布の範囲は１を越える可能性がある。このような場合、輝度分布を圧縮して０〜１の範囲に収めることでＳ１０６におけるＲＧＢ変換後の画像データにおいて前もってＲＧＢデータが１を越える確率を抑えることができる。例えば、Ｓ１０５の処理と同様に、改善処理後の輝度分布の最大値を求め、最大値が１を越える場合に対して、（式６）〜（式８）に示したような同形式の方法にてパラメータを決定、輝度圧縮を行うようにすればよい。
【００５４】
以上説明したような階調圧縮や輝度圧縮による階調とび、色調のずれの抑制処理を行うと改善処理で明るく改善された画像が暗くなってしまう。このような場合には階調圧縮や輝度圧縮による階調とび、色調のずれの抑制処理を行った画像データに対して元の画像の平均的な明るさが維持できるように明るさの補正を行うようにすればよい。例えば、元画像の輝度平均値をＹ_ｏｒｇ、輝度平均値Ｙ_ｏｒｇに対して（式６）〜（式８）に示したような同形式の方法にて輝度圧縮を行った場合の出力値をＹ_ｃｏｍｐとするとき、明るさの補正によってＹ_ｃｏｍｐをＹ_ｏｒｇに戻すようなパラメータγ０を（式９）のように求めることができる。
【外６】

【００５５】
そして、求まったパラメータγ０に従い、階調圧縮や輝度圧縮による階調とび、色調のずれの抑制処理を行ったＲＧＢ各画像データに対してＲ^{γ ０} 、Ｇ^{γ ０}、 Ｂ^{γ ０}を出力するようなガンマ補正を行うようにすればよい。但し、γ０の値が小さすぎると画像のうち必要以上に明るくなりすぎる領域が生じてしまうので所定の範囲になるように制限する方がよい。
【００５６】
また、以上説明した階調圧縮や輝度圧縮による階調とび、色調のずれの抑制処理は画像データに輝度改善処理を施した場合の階調とびや色調のずれを抑制することが目的であるので、輝度改善処理を行った後の画像データを所定の色空間に収めたときに階調とびや色調のずれが目立つかどうかを判定して、階調とびや色調のずれが目立つと判断した画像に対してのみ階調とび、色調のずれの抑制処理を行うようにしてもよい。例えば、輝度改善処理を行った後、所定の色空間に収めた画像データと元の画像データの画素ごとの色相の変化を算出し、色相の変化が所定量以上の画素の全画素に対する比率が所定の値を超える場合に色調のずれが目立つと判断するようにする。
【００５７】
また、以上説明した例においては、処理する画像データとしてＲＧＢ各８ビットのデータを想定して説明したが、ＲＧＢ各１６ビットの画像データから最適なＲＧＢ各８ビットの画像データを再生する場合においても適用できる。
【００５８】
（実施形態２）
図４に実施形態２にかかる画像処理システムの構成を示す。実施形態１と同一の構成については実施形態１と同様の符号を付け、説明を割愛する。
【００５９】
実施形態１では輝度改善結果が色域を越える場合は、輝度改善結果に対して階調圧縮を行っていた、これに対して本実施形態では輝度改善で用いるパラメータを変更し、再度、輝度改善を行う。
【００６０】
５は、画像再生手段であり、輝度改善手段４から出力された改善された輝度成分と輝度抽出手段２から出力された色成分とを合成し、画像データを再構成する。画像再生手段５はまた、ＲＧＢ変換手段５１、飽和度判定手段５２、符号化手段２３より構成される。ＲＧＢ変換手段５１は改善された輝度成分と色成分とを合成し、ＲＧＢ画像データに変換する。飽和度判定手段２１は変換されたＲＧＢ画像データの色の飽和の度合いを判定する。符号化手段２３はＲＧＢ画像データを所定形式の画像データとして符号化する。
【００６１】
２２は、パラメータ調整手段であり、輝度抽出手段２から出力された画像データの輝度成分、および一旦輝度改善を行った画像データの飽和度から輝度改善手段４で処理する改善の度合いを画像データに応じて最適にするようにパラメータの調整を行う。
【００６２】
次に、本実施形態の画像処理システムの動作を汎用の計算機で実行するアプリケーションプログラムのアルゴリズムを説明する。
【００６３】
Ｓ１０１〜Ｓ１０６では、実施形態１（図２）と同様に輝度改善処理を行う。
【００６４】
Ｓ１０６で得られたＲＧＢデータに対して、飽和度の判定を行う。（Ｓ２０７）
Ｓ１０６でＲＧＢに変換した画像データのＲＧＢ各成分に対し、１を越える（飽和する）かを判定し、飽和する成分がある画素数（飽和度）をカウントする。そして飽和する画素数が所定の値を越えるかどうか判定する。この処理は、実施形態１と同様に画像データが所定の色範囲内にあるか否かを判定する処理に相当する。
【００６５】
飽和度を判定する手段としては、上記のような単純にデータが飽和する画素数をカウントする方法の他に、飽和する量を次の（式１０）の左辺Ｓのように定量化し、その値を所定値と比較するようにしてもよい。
【００６６】
Ｓ＝Σ｛ｐｓ（Ｒ−１）＋ｐｓ（Ｇ−１）＋ｐｓ（Ｂ−１）｝（式１０）
但し、ｘ＜０のときｐｓ（ｘ）＝０、ｘ＞＝０のときｐｓ（ｘ）＝ｘ
Ｒ、Ｇ、Ｂは各画素のＲＧＢデータ
また、以上の飽和度は画像の総画素数で正規化してもよい。
【００６７】
Ｓ２０７で飽和度が所定の値を越える場合は、輝度改善後のＲＧＢデータの飽和度が所定の範囲に収まるように輝度改善の度合いを弱める必要があると判断し、処理をＳ１０４に移し、Ｓ１０４で決定した改善パラメータを所定量だけ弱めるようにする。例えば、（式４）で示したＹ_１の代わりに（Ｙ_０＋Ｙ_１）／２、等を用いるようにしてγを求める。
【００６８】
一方、Ｓ２０７で飽和度が所定の値を越えない場合は、Ｓ１０６で変換したＲＧＢデータをＲＧＢ各８ビットに量子化し、出力する。（Ｓ２０８）このとき、ＲＧＢデータが１を越える画素は１に置き換えた後、ＲＧＢ各８ビットに量子化する。
【００６９】
本実施形態では輝度改善処理を施した後の画像データの飽和度が所定の値を越えた場合に目立ってくる色の飽和をパラメータ調整手段において輝度改善の度合いを小さくすることで抑制するようにしたが、抑制後の飽和度を再度判定して、所定の値を越えているような場合には輝度改善処理を適用しないようにしてもよい。
【００７０】
また、以上説明した例においては、処理する画像データとしてＲＧＢ各８ビットのデータを想定して説明したが、ＲＧＢ各１６ビットの画像データから最適なＲＧＢ各８ビットの画像データを再生する場合においても適用できる。
【００７１】
【発明の効果】
本発明によれば、明るさの分布を改善する処理を施した際に生じる階調とびや色調のずれの発生を抑えることができる。
【図面の簡単な説明】
【図１】実施形態１にかかる画像処理システムの構成を示す図。
【図２】実施形態１にかかる画像処理システムにおけるアプリケーションプログラムのアルゴリズムを示す図。
【図３】実施形態１にかかる画像処理システムにおける画像データ再構成のアルゴリズムを示す図。
【図４】実施形態２にかかる画像処理システムの構成を示す図。
【図５】実施形態２にかかる画像処理システムにおけるアプリケーションプログラムのアルゴリズムを示す図。[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an improvement in the brightness distribution of a digital image.
[0002]
[Prior art]
Conventionally, as a method of taking a photograph with appropriate brightness, a method of measuring the average brightness of a scene to be taken and controlling the shutter speed, aperture value, etc. of the camera is known. There is also known an exposure control method based on a so-called evaluation photometry method that divides a scene into predetermined regions and weights the luminance measured for each region to obtain an average exposure by obtaining an average luminance.
[0003]
However, in a so-called backlight scene where the brightness of the main subject to be photographed is significantly darker than the brightness of the background, the main subject portion is inevitably dark in the photographed image. In order to take a photograph with appropriate brightness in such a backlight scene, it is necessary to set the exposure of the camera in advance so that the photograph is taken brighter than the average photograph. However, such an exposure correction operation is not only cumbersome but also requires skill to properly set the camera. Even if exposure correction is appropriately performed on the main subject, the background portion becomes too bright.
[0004]
An object of the present invention is to obtain an image with appropriate brightness even in a scene such as backlight where it is difficult to appropriately determine the brightness of the image.
[0005]
In order to achieve the object of the present invention, in the analog photographic technique, a print with appropriate brightness can be obtained by performing a so-called dodging process in a dark room. In order to easily realize such a dodging process, it is desirable to realize the dodging process in digital image processing.
[0006]
As a method for realizing such processing, for example, IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 6, NO. 7, JULY 1997 includes a report by Johnson et al. (Referred to as Prior Art 1) entitled “A Multiscale Retinex for Bridging the Gap Between Color Images and the Human Observation of Scenes”. This is to process the difference between the logarithmically transformed component of the digital image and the low frequency component of the logarithmically transformed component, thereby darkening the bright component in the low frequency region of the digital image and brightening the dark component in the low frequency region. Thus, the image is to be improved.
[0007]
Also, acm Transactions on Graphics, JULY 2002, Vol. 21, no. Also in the report of Reinhard et al. Entitled “Photographic Tone Production for Digital Images” (referred to as Prior Art 2), the brightness component of the digital image and its low-frequency component are used in the digital image processing. A method for obtaining a special effect has been proposed.
[0008]
[Non-Patent Document 1]
“A Multiscale Retinex for Bridging the Gap Between Color Images and the Human Observation of Scenes”, IEEE TRANSACTIONS ON IMAGESING, VOL. 6, NO. 7, JULY 1997
[Non-Patent Document 2]
“Photographic Tone Production for Digital Images” acm Transactions on Graphics, JULY 2002, Vol. 21, no. 3
[0009]
[Problems to be solved by the invention]
However, in the above-described conventional example, when the image data obtained by performing the improvement process on the brightness of the digital image to be processed does not fall within the predetermined color space range, the color space of the color outside the color space is included in the image data. There is a problem that gradation jumps and color tone shifts occur because the output is replaced with the color of the boundary region.
[0010]
SUMMARY OF THE INVENTION An object of the present invention is to solve the above-mentioned problems and suppress the occurrence of gradation jumps and color tone deviations even when a process for improving the brightness distribution of a digital image to be processed is performed.
[0011]
[Means for Solving the Problems]
In order to achieve the above object, the present invention has the following configuration.
[0012]
The invention described in claim 1 includes an extraction step for extracting a brightness component from image data, a scale conversion step for obtaining a distribution of the brightness component on a relatively large scale, and the brightness component and the scale conversion step. And using the output to improve the brightness component distribution of the image data, and determine whether the image data reconstructed according to the improved brightness component is within a predetermined color range A gradation compression step of performing gradation compression processing when image data outside the predetermined color range is included.
[0013]
The invention described in claim 7 includes an extraction step of extracting a brightness component from image data, a scale conversion step for obtaining a distribution of the brightness component on a relatively large scale, and the brightness component and the scale conversion step. An improvement step for improving the brightness distribution of the image data using the output, a determination step for determining the degree of saturation of the improved image data, and an improvement of the improvement step according to the determined degree of saturation And an adjusting step for adjusting the degree.
[0014]
DETAILED DESCRIPTION OF THE INVENTION
(Embodiment 1)
FIG. 1 shows the configuration of the image processing system of this embodiment.
[0015]
In the figure, reference numeral 1 denotes image input means for inputting digital image data (hereinafter referred to as image data) to the image processing system. For example, it is composed of a digital camera, a scanner and the like.
[0016]
Reference numeral 2 denotes a luminance extraction unit that extracts a luminance component and a color component from the image data input by the image input unit 1. Reference numeral 3 denotes a scale conversion unit that obtains a distribution on a relatively large scale of the luminance component of the image data output from the luminance extraction unit 2. Reference numeral 4 denotes luminance improvement means, which uses the luminance component distribution of the image data output from the luminance extraction means 2 and the luminance component distribution on a relatively large scale output from the scale conversion means 3 to calculate the luminance component of the image data. Improve distribution.
[0017]
Reference numeral 5 denotes an image reproduction unit that combines the improved luminance component output from the luminance improvement unit 4 and the color component output from the luminance extraction unit 2 to reconstruct the image data. The image reproduction means 5 is also composed of an RGB conversion means 51, a color gamut determination means 52, a compression parameter adjustment means 53, and a gradation compression means 54. The RGB conversion means 51 combines the improved luminance component and color component, and converts them into RGB image data. The color gamut determining means 52 determines whether or not the converted RGB image data fits in a predetermined color space. The compression parameter adjustment unit 53 obtains a compression parameter for performing gradation compression according to the color gamut of the RGB image data when the RGB image data does not fit in a predetermined color space. The gradation compression means 54 performs gradation compression on the RGB image data using the compression parameters, and encodes the image data as a predetermined color space.
[0018]
Reference numeral 6 denotes a parameter adjusting unit that adjusts parameters so as to optimize the degree of improvement processed by the luminance improving unit 4 from the luminance component of the image data output from the luminance extracting unit 2 in accordance with the image data.
[0019]
The image processing system configured as described above can also be executed on a general-purpose computer by an application program. Hereinafter, in this specification, an image processing system that is mainly executed in an application program will be described, and a configuration of the image processing system in FIG. 1 will be supplementarily described.
[0020]
FIG. 2 shows an algorithm of an application program for executing the operation of the image processing system in this embodiment by a general-purpose computer.
[0021]
First, when the application program is activated, the user inputs the file name of the image data and reads the image data into the storage unit of the computer (S101).
[0022]
The image data read here is, for example, an M × N two-dimensional array composed of 8-bit pixels (where M is the number of horizontal pixels and N is the number of vertical pixels), and R, G, B, 3 Consists of two faces. The image data is R, G, B, R (x, y), G (x, y), and B (x, y) (where (x, y) is an integer representing a pixel position, 1 ≦ x ≦ M, 1 ≦ y ≦ N). At this time, if the image data is compressed by a method such as JPEG, the image data is decompressed according to a predetermined decompression method to obtain image data composed of RGB pixels.
[0023]
Next, a luminance component is extracted based on the RGB pixels constituting the image data (S102).
[0024]
For example, the luminance component is extracted by assuming that the pixel component of RGB is data in the sRGB color space described in IEC 61966-2-1, and performing gamma conversion according to the method described in IEC 61966-2-1. Conversion into CIE1931XYZ is performed by a matrix operation of 3 rows and 3 columns. Here, if the converted XYZ data is X (x, y), Y (x, y), and Z (x, y), Y (x, y) is a luminance component to be extracted. When this luminance extraction is configured by hardware, for example, it can be configured by a table reference circuit (gamma conversion part) using a lookup table and a matrix operation circuit.
[0025]
Note that as a method of extracting the luminance component, the above-described processing may be simplified and extraction may be performed only by matrix calculation. Also, conversion from RGB to YCbCr, conversion from RGB to L ^* a ^* b ^* , or conversion from RGB to HSV may be used.
[0026]
Next, a distribution of luminance components on a relatively large scale is obtained from the extracted luminance components (S103).
[0027]
In order to obtain a luminance component distribution on a large scale, for example, as in the prior art 1, a product-sum operation of the extracted luminance component and a Gaussian function is performed and output. (However, in the prior art 1, the product-sum operation is performed directly on each RGB pixel of the image data instead of the luminance component of the image data.) Here, in order to improve the image quality of the improved image data, It is more preferable to perform a product-sum operation with a plurality of Gaussian functions having different standard deviations to obtain a distribution of luminance components on a plurality of scales. The process for obtaining the luminance component distribution on a large scale as described above is hereinafter referred to as scale conversion. When this scale conversion is configured by hardware, for example, it can be configured by a product-sum operation circuit.
[0028]
Obtaining a luminance component on a large scale is equivalent to obtaining a low-frequency component of the luminance component. Further, obtaining the luminance component with a plurality of scales means obtaining frequency components of different luminance components. Further, the luminance component of a large scale may be an average value of a plurality of pixels including the target pixel.
[0029]
Next, a parameter for determining the degree of the luminance distribution to be improved is adjusted from the extracted luminance component. (S104) The parameter adjustment is related to the process of improving the luminance distribution, and will be described in detail later.
[0030]
Next, the distribution of the luminance component of the image data is improved using the distribution of the luminance component of the image data and the scaled luminance component (S105).
[0031]
As an example of processing, according to the method based on the prior art 1, each distribution of the luminance component and the scaled luminance component is logarithmically converted and the difference is output. Furthermore, the weighted average of the differential outputs at different scales is taken as an improved luminance component. However, in this method, since the degree of improvement cannot be adjusted according to the image, the logarithm conversion output of the scaled luminance component is multiplied by a coefficient. This coefficient is a parameter for adjusting the degree of improvement. The output of the improved luminance component based on the processing described above is as shown in (Equation 1) below.
[Outside 1]

[0032]
However, Y ′ (x, y), F _n (x, y), w _n , n, and γ are the output of the improved luminance component, the Gaussian function, the weight between the scales, the parameter representing the scale, and the degree of improvement, respectively. Is a parameter that represents * Represents a product-sum operation.
[0033]
Note that the weights between scales can be omitted by replacing the standard deviation of the scales (replaced with a simple average), and the inverse of the output of logarithmically transformed values as in (Equation 1). It has been found that returning to the original luminance unit by conversion (exp calculation) is preferable as improved image data image quality. Therefore, it is more preferable that the output shown in the following (Formula 2) is an improved luminance component.
[Outside 2]

[0034]
However, Avg represents average value calculation.
[0035]
Instead of (Formula 2), the following (Formula 3) may be used.
[Outside 3]

[0036]
Note that the average value calculation of the scale conversion output at a plurality of scales may be performed by the scale conversion processing of S103, and the average value of the scale conversion output at the plurality of scales may be used as the distribution of the luminance component after the scale conversion.
[0037]
When the luminance conversion is configured by hardware, it can be configured by, for example, an average value calculation circuit, a circuit for creating a lookup table, a table storage unit, a table reference circuit (gamma conversion part), and a division circuit. The average value calculation circuit may be provided in the scale conversion means.
[0038]
Here, an example of the parameter adjustment method in S104 will be described in the case of performing luminance conversion in (Equation 3) in S105.
[0039]
First, a luminance histogram is created by dividing the luminance component of the image data into a predetermined luminance value range. Then, a luminance value (this luminance value is set to Y ₀ ) is obtained in which the ratio of the histogram frequency accumulated from the darker side to the total sampling is a predetermined ratio. At this time, γ such that the obtained luminance value Y ₀ is a predetermined luminance value (this luminance value is Y ₁ , where Y ₀ ≦ Y ₁ ) is a parameter representing the degree of improvement. Assuming that the value in [] of the denominator of (Expression 3) is almost equal to Y (x, y), γ can be obtained by (Expression 4) shown below.
[Outside 4]

[0040]
In this embodiment, the parameters are automatically adjusted based on the luminance component of the image data. However, the parameters may be automatically adjusted based on the RGB pixel values of the image data. Good. In addition, the parameters can be changed by the input interface, the luminance distribution is converted according to the changed parameter value, the image data is reconstructed, and the improved image data is displayed on the display and adjusted interactively. You may do it.
[0041]
In the above example, the method for optimally improving the brightness distribution according to the image data processed based on the prior art 1 has been described. However, in the following example, the image processed based on the prior art 2 is processed. A method of optimally improving the brightness distribution according to data will be described. Note that the difference from the above embodiment is mainly the process of improving the distribution of the luminance component of the image data in S105, so this portion will be mainly described.
[0042]
According to Prior Art 2, the degree of occurrence of halo after luminance conversion is evaluated using luminance components of different scales, an optimal scale is determined so that there is no image quality defect due to halo generation, and luminance conversion is performed. ing. An example of applying the present invention to this is shown in the following (Formula 5).
[Outside 5]

[0043]
Here, V (x, y) is a luminance component selected from a plurality of scale-converted luminance components, a is a parameter for adjusting the intensity of the luminance component, and e is a parameter representing the degree of improvement of the luminance component. Needless to say, the adjustment method of the improvement parameter in S104 should be changed along with the change of the adjustment method in the luminance component improvement processing.
[0044]
In the example described based on the prior art 2, the luminance component itself of the image is adjusted together with the degree of improvement of the luminance component. However, the present invention is combined with the processing for adjusting the luminance component of the image itself as described above. Needless to say, it may be.
[0045]
Further, in the process of S105, as a value in [] of the denominator of (Equation 3), acm Transactions on Graphics, JULY 2002, Vol. 21, no. 3 may be applied as described in the report of Durand et al. Entitled "Fast Bilateral Filtering for the Display of High-Dynamic-Range Images". In this case, the Bilatal Filtering process is performed on the luminance component in advance in the scale conversion process of S103.
[0046]
Next, the improved luminance component and the color components X (x, y) and Z (x, y) converted in S102 are combined to reconstruct the image data. (S106)
A detailed algorithm for the reconstruction of the image data is shown in FIG.
[0047]
First, image data is converted into RGB data. (S61)
Here, the color component is corrected according to the change of the luminance component so that the color of the reconstructed image data does not change as much as possible. For example, the color components X (x, y) and Z (x, y) are multiplied by the ratio Y ′ (x, y) / Y (x, y) before and after the change of the luminance component, respectively. Then, RGB data is obtained from the X, Y, and Z data. The process here is an inverse transformation of the process in S102. Accordingly, a matrix operation of 3 rows and 3 columns and inverse gamma conversion are performed. Needless to say, if another method such as RGB to YCbCr conversion is used as the method of extracting the luminance component in S102, the corresponding inverse conversion process should be performed in this process.
[0048]
Next, the maximum value is obtained for the RGB image data. (S62)
Next, it is determined whether or not the obtained maximum value exceeds 1. (S63)
If the obtained maximum value does not exceed 1, RGB data can be expressed by image data in the range of sRGB, so that RGB is quantized to 8 bits and output. (S66)
As described above, in S66, when the data with improved luminance exceeds a predetermined color range, gradation compression is performed according to the color range of the data with improved luminance.
[0049]
On the other hand, when the obtained maximum value exceeds 1, the RGB data cannot be represented by image data in the range of sRGB, and therefore, when the pixel value exceeding 1 is replaced with 1, RGB is quantized to 8 bits for each RGB. It causes skipping and color shift. Therefore, in order to perform gradation compression on the RGB image data, first, gradation compression parameters are determined (S64), and gradation compression is performed based on the parameters. (S65)
The gradation compression is performed so that the pixel value p becomes the maximum value 1 in the sRGB color space, where the maximum value of RGB data is p (p> 1). For example, when RGB data before gradation compression is R1, G1, and B1, and RGB data after gradation compression is R2, G2, and B2, respectively.
R2 = k · R1, G2 = k · G1, B2 = k · B1 (Formula 6)
The gradation compression is performed on all RGB data in the image. However, k is a gradation compression parameter, and k = 1 / p. However, when p is larger than a predetermined value (pm> 1), the degree of gradation compression becomes too large and the entire image becomes dark. In this case, k = 1 / pm is used. To perform gradation compression.
[0050]
In the above example, gradation compression is performed by primary conversion, but it may be performed by secondary conversion. For example, if the maximum value of RGB data is p (p> 1), the pixel value p is set to the maximum value 1 in the sRGB color space. Further, when the pixel value is 1, it is determined to be a predetermined value, for example, an average value of 1 and 1 / p.
R2 = (k2 · R1 + k1) R1,
G2 = (k2 · G1 + k1) G1,
B2 = (k2 · B1 + k1) B1 (Formula 7)
The gradation compression is performed on all RGB data in the image. However, k2 and k1 are parameters for gradation compression, and are k2 = (p−1) / (2p · (p + 1)) and k1 = (p + 1) / 2p−k2. However, when p is larger than a predetermined value (pm> 1), the degree of gradation compression becomes too large and the entire image becomes dark. In this case, k2 = (pm−1) / (2 pm· (pm + 1)), k1 = (pm + 1) / 2 pm−k2 is used to perform gradation compression.
[0051]
In addition, gradation compression may be performed by a method using γ conversion as shown in (Expression 8) below.
[0052]
R2 = (k3 · R1) ^γ ,
G2 = (k3 · G1) ^γ ,
B2 = (k3 · B1) ^γ (Formula 8)
However, k3 and γ are gradation compression parameters.
[0053]
As described above, when the maximum value of RGB data exceeds 1, gradation compression is performed, the RGB data is converted so that it can be represented by image data in the range of sRGB, quantized to 8 bits for each RGB, and output. (S66)
Note that the gradation skipping and tone shift suppression processing by gradation compression described in S62 to S65 can also be applied to the luminance improvement processing in S105. For example, comparing the range of the luminance distribution before and after performing the luminance conversion according to (Equation 3), the range of the luminance distribution before the luminance conversion is in the range of 0 to 1, whereas the range after the luminance conversion is The range of the luminance distribution may exceed 1. In such a case, the probability that the RGB data exceeds 1 in advance in the image data after RGB conversion in S106 can be suppressed by compressing the luminance distribution to fall within the range of 0 to 1. For example, as in the process of S105, the maximum value of the luminance distribution after the improvement process is obtained, and the method of the same form as shown in (Expression 6) to (Expression 8) when the maximum value exceeds 1 The parameters may be determined at, and the luminance may be compressed.
[0054]
If the gradation skipping and tone shift suppression processing as described above is performed by gradation compression or luminance compression, the image that has been brightened and improved by the improvement processing becomes dark. In such a case, the brightness correction is performed so that the average brightness of the original image can be maintained with respect to the image data that has been subjected to the gradation skip due to gradation compression or luminance compression and the process of suppressing the shift in color tone. You just have to do it. For example, the luminance average value of the original image is Y _org , and the output value when the luminance compression is performed on the luminance average value Y _org by a method of the same format as shown in (Expression 6) to (Expression 8). When Y _comp is obtained, a parameter γ0 that returns Y _comp to Y _org by correcting the brightness can be obtained as in (Equation 9).
[Outside 6]

[0055]
Then, according to the obtained parameter γ0, R ^{γ 0} , G ^{γ 0} , and B ^{γ 0} are output with respect to each RGB image data that has been subjected to gradation skipping by gradation compression or luminance compression and subjected to tone shift suppression processing. Gamma correction should be performed. However, if the value of γ0 is too small, an area that is excessively brighter than necessary is generated in the image, so it is better to limit the range to a predetermined range.
[0056]
In addition, the above-described gradation skip and tone shift suppression processing by tone compression and luminance compression is intended to suppress the tone skip and tone shift when the luminance improvement processing is performed on image data. An image that has been judged to have a noticeable gradation jump or tone shift when the image data after the brightness improvement processing is stored in a predetermined color space. It is also possible to perform processing for suppressing gradation deviation and tone deviation only for the above. For example, after performing the luminance improvement processing, the hue change for each pixel of the image data stored in the predetermined color space and the original image data is calculated, and the ratio of the pixels whose hue change is a predetermined amount or more to all the pixels is calculated. When a predetermined value is exceeded, it is determined that a color shift is conspicuous.
[0057]
Further, in the example described above, the description has been made assuming that 8-bit RGB data is processed as image data to be processed. However, when reproducing optimal 8-bit RGB image data from 16-bit RGB image data, Is also applicable.
[0058]
(Embodiment 2)
FIG. 4 shows the configuration of an image processing system according to the second embodiment. The same components as those in the first embodiment are denoted by the same reference numerals as those in the first embodiment, and the description thereof is omitted.
[0059]
In the first embodiment, when the brightness improvement result exceeds the color gamut, gradation compression is performed on the brightness improvement result. On the other hand, in this embodiment, the parameters used for the brightness improvement are changed and the brightness improvement is performed again. I do.
[0060]
Reference numeral 5 denotes an image reproduction unit that combines the improved luminance component output from the luminance improvement unit 4 and the color component output from the luminance extraction unit 2 to reconstruct the image data. The image reproduction means 5 is also composed of RGB conversion means 51, saturation determination means 52, and encoding means 23. The RGB conversion means 51 combines the improved luminance component and color component, and converts them into RGB image data. The saturation determination means 21 determines the degree of color saturation of the converted RGB image data. The encoding means 23 encodes RGB image data as image data of a predetermined format.
[0061]
Reference numeral 22 denotes a parameter adjusting unit, which determines the degree of improvement processed by the luminance improving unit 4 from the luminance component of the image data output from the luminance extracting unit 2 and the saturation of the image data once improved in luminance to the image data. The parameters are adjusted so as to be optimized accordingly.
[0062]
Next, an algorithm of an application program that executes the operation of the image processing system of the present embodiment with a general-purpose computer will be described.
[0063]
In S101 to S106, luminance improvement processing is performed in the same manner as in the first embodiment (FIG. 2).
[0064]
The degree of saturation is determined for the RGB data obtained in S106. (S207)
In step S106, it is determined whether each RGB component of the image data converted into RGB exceeds 1 (saturates), and the number of pixels (saturation degree) having a saturated component is counted. Then, it is determined whether or not the number of saturated pixels exceeds a predetermined value. This process corresponds to a process for determining whether the image data is within a predetermined color range, as in the first embodiment.
[0065]
As means for determining the degree of saturation, in addition to the method of simply counting the number of pixels in which data is saturated, the amount of saturation is quantified as the left side S of the following (Equation 10), and the value May be compared with a predetermined value.
[0066]
S = Σ {ps (R−1) + ps (G−1) + ps (B−1)} (Formula 10)
However, when x <0, ps (x) = 0, and when x> = 0, ps (x) = x
R, G, and B are RGB data of each pixel, and the above saturation may be normalized by the total number of pixels of the image.
[0067]
If the degree of saturation exceeds a predetermined value in S207, it is determined that the degree of luminance improvement needs to be weakened so that the degree of saturation of the RGB data after luminance improvement falls within a predetermined range, and the process proceeds to S104. The improvement parameter determined in step 1 is weakened by a predetermined amount. For example, γ is obtained by using (Y ₀ + Y ₁ ) / 2, etc., instead of Y ₁ shown in (Expression 4).
[0068]
On the other hand, if the saturation does not exceed a predetermined value in S207, the RGB data converted in S106 is quantized into 8 bits for each RGB and output. (S208) At this time, pixels whose RGB data exceeds 1 are replaced with 1, and then quantized to 8 bits for each of RGB.
[0069]
In the present embodiment, color saturation that is noticeable when the saturation of image data after the luminance improvement processing exceeds a predetermined value is suppressed by reducing the degree of luminance improvement in the parameter adjustment means. However, the degree of saturation after the suppression may be determined again, and the luminance improvement process may not be applied when the predetermined value is exceeded.
[0070]
Further, in the example described above, the description has been made assuming that 8-bit RGB data is processed as image data to be processed. However, when reproducing optimal 8-bit RGB image data from 16-bit RGB image data, Is also applicable.
[0071]
【The invention's effect】
According to the present invention, it is possible to suppress occurrence of gradation jumps and color tone shifts that occur when processing for improving the brightness distribution is performed.
[Brief description of the drawings]
FIG. 1 is a diagram showing a configuration of an image processing system according to a first embodiment.
FIG. 2 is a view showing an algorithm of an application program in the image processing system according to the first embodiment.
FIG. 3 is a view showing an algorithm for image data reconstruction in the image processing system according to the first embodiment;
FIG. 4 is a diagram showing a configuration of an image processing system according to a second embodiment.
FIG. 5 is a view showing an algorithm of an application program in the image processing system according to the second embodiment.

Claims (11)

An extraction process for extracting brightness components from the image data;
A scale conversion step for obtaining a distribution on a relatively large scale of the brightness component;
Using the brightness component and the output of the scale conversion step to improve the brightness component distribution of the image data;
It is determined whether or not the image data reconstructed according to the improved brightness component is within a predetermined color range, and if it includes image data outside the predetermined color range, gradation compression processing is performed. An image processing method comprising: a gradation compression step. The image processing method according to claim 1, wherein the gradation compression step adjusts a degree of gradation compression according to a color range of a target image. The image processing method according to claim 1, wherein the gradation compression step adjusts the degree of gradation compression based on a maximum value in the image data of the target image. 4. The image processing method according to claim 1, wherein the gradation compression step adjusts the degree of gradation compression by a degree equal to or less than a predetermined amount. 5. The improvement process determines whether the brightness component distribution to be improved falls within a predetermined range, and compresses the brightness component when the brightness component outside the range is included. An image processing method according to any one of the above. The image processing method according to claim 1, wherein brightness of the image data subjected to the gradation compression processing is corrected. An extraction process for extracting brightness components from the image data;
A scale conversion step for obtaining a distribution on a relatively large scale of the brightness component;
An improvement step of improving the brightness distribution of the image data using the brightness component and the output of the scale conversion step;
A determination step of determining a degree of saturation of the improved image data;
An image processing method comprising: an adjustment step of adjusting an improvement degree of the improvement step according to the determined degree of saturation. The program for implement | achieving the image processing method in any one of Claims 1 thru | or 7 using a computer. Luminance extraction means for extracting luminance components from the image data;
A scale conversion means for obtaining a distribution on a relatively large scale of the luminance component;
Brightness improvement means for improving the brightness distribution of the image data using the brightness component and the output of the scale conversion means;
In an image processing apparatus comprising image reproduction means for reproducing image data using the output of the luminance improvement means as a new image luminance distribution,
The image reproduction means determines whether the image data to be reproduced fits in a predetermined color space. If the image data to be reproduced includes a color outside the color space, the image reproduction means performs gradation compression to perform image data within the color space. An image processing apparatus for reproducing the image. Luminance extraction means for extracting luminance components from the image data;
A scale conversion means for obtaining a distribution on a relatively large scale of the luminance component;
Brightness improvement means for improving the brightness distribution of the image data using the brightness component and the output of the scale conversion means;
In an image processing apparatus comprising image reproduction means for reproducing image data using the output of the luminance improvement means as a new image luminance distribution,
The luminance improvement means determines whether the luminance distribution to be improved falls within a predetermined range, and when the luminance distribution to be improved includes luminance outside the range, luminance compression is performed to improve the luminance distribution within the range. An image processing apparatus. Luminance extraction means for extracting luminance components from the image data;
A scale conversion means for obtaining a distribution on a relatively large scale of the luminance component;
Brightness improvement means for improving the brightness distribution of the image data using the brightness component and the output of the scale conversion means;
In an image processing apparatus comprising image reproduction means for reproducing image data using the output of the luminance improvement means as a new image luminance distribution,
The image reproduction means determines the degree of saturation of image data to be reproduced, and adjusts the degree of improvement of the luminance improvement means to be small when the degree of saturation exceeds a predetermined value. apparatus.

Images