JP4017312B2 - Image processing method, image processing apparatus, and recording medium - Google Patents

Description

【００３５】
この係数配列では、特に強いシャープネス強調を掛けたときに、画像のエッジに不自然な輪郭が発生し易い。そこで、そのような欠点を少なくするために、本発明では式（７）に示したような正規分布型(Gaussian)のぼけ関数を用いたアンシャープマスクを用いるのが好ましい。
G(x,y)＝(1/2πσ²)exp[−(x² + y²)/２σ²] (７）
ここで、σ² は正規分布関数の広がりを表すパラメータであり、マスクの端ｘ＝ｘ₁ における値とマスクの中心ｘ＝０における値の比、
G(x₁,0)/G(0,0)＝exp[−x₁ ²/２σ²] （８）
が０．１〜１．０となるように調節することによって、３×３のアンシャープマスクのシャープさを所望のものとすることができる。式（８）の値を１．０に近い値にすると、式（５）の中央のラプラシアンフィルタとほぼ同じマスクを作ることができる。
マスクのシャープさを変更するには、この他にマスクの大きさを変更する方法があり、たとえば５×５、７×７、９×９等のマスクを用いることによって、シャープネス強調された画像の空間周波数を大幅に変更することができる。
【００３６】
また、マスクの関数形としても、上記の正規分布型以外のもの、たとえば、下記式（９）のような指数関数型のマスクを用いることができる。
E(x,y)＝exp[−(x² + y²)^1/2／ａ] （９）
ここで、ａは式（８）のσ² と同様にアンシャープマスクの広がりを表すパラメータであり、マスクの端の値とマスクの中心値の比、
E(x₁,0)/E(0,0)＝exp[−x₁／ａ] （１０）
が０．１〜１．０となるように調節することによって、３×３のアンシャープマスクのシャープさを所望のものとすることができる。式（１０）に、E(x₁,0)/E(0,0)＝０.3としたときの式（９）の指数関数のマスクの数値例を示す。
0.18 0.30 0.18
0.30 1.00 0.30 （１１）
0.18 0.30 0.18
このマスクから、アンシャープマスクの１例を計算すると、次式（１２）のようになる。
−0.12 −0.22 −0.12
−0.21 2.32 −0.21 （１２）
−0.12 −0.21 −0.12
【００３７】
このようなアンシャープマスクを用いて、原画像データＩ₀(x,y)からシャープネス強調画像データＩ_S (x,y）を求めることができる。なお、本発明に用いられるアンシャープマスクおよびシャープネス強調方法は、上述したものに限定されるわけではなく、この他の従来公知のアンシャープマスクや空間周波数フィルタリング等によるシャープネス強調方法を適用可能なことはもちろんである。
図４（ａ）には、エッジ成分が支配的な領域Ｅ₁ および領域Ｅ₂ と、ノイズモトルのある領域Ａや領域Ｂや領域Ｃを含むノイズ領域とを有する原画像データＩ₀ の一次元プロファイル波形の一例が示され、このプロファイル波形に対してシャープネス強調を施すことによって、図４（ｂ）に示すように、領域Ｅ₁ および領域Ｅ₂ のようにエッジ成分の強い領域においてエッジの信号が強調されるとともに、領域Ａ、領域Ｂおよび領域Ｃを含むノイズ領域についてもノイズ成分が強調されるのが判る。
【００３８】
次に、平滑化工程（ステップ１０２）について説明する。
平滑化を行う方法としては、実空間領域の処理と空間周波数領域の処理を挙げることができる。実空間領域処理では、隣接する画素全体の和を求め平均値を計算してその値に置き換える方法、各画素に重み係数、たとえば正規分布型の関数を掛けて平均値を求める方法、メディアンフィルタのような非線型な処理を行う方法等の種々の方法がある。一方、空間周波数領域の処理では、ローパスフィルタを掛ける方法がある。たとえば、重み係数を用いる平均化の方法では下記式（１３）を挙げることができる。なお、ここで、(x,y) 等は、画像内の注目画素の位置座標を表す。
【数１】

【００３９】
ただし、ｎは平均化のマスクサイズ、ｗ(x,y) は重み係数である。ｗ(x,y) ＝１．０とすると、単純平均となる。
本発明では、実空間領域処理の中で、正規分布型の重み係数を掛けて平均値を求める方法を用いるが、これに限定されない。この時、処理のマスクとしては、下記のようなｎ×ｎ画素のマスクを用いるのが好ましい。具体的には３×３から５×５、７×７、９×９程度のものを用いるのが好ましい。

【００４０】
９×９画素のマスクの一例を示す。この式（１５）では中心の値を１．０に正規化した値で示しているが、実際の処理ではマスク全体の和が１．０になるようにする。
0.09 0.15 0.22 0.28 0.30 0.28 0.22 0.15 0.09
0.15 0.26 0.38 0.47 0.51 0.47 0.38 0.26 0.15
0.22 0.38 0.55 0.69 0.74 0.69 0.55 0.38 0.22
0.28 0.47 0.69 0.86 0.93 0.86 0.69 0.47 0.28
0.30 0.51 0.74 0.93 1.00 0.93 0.74 0.51 0.30 （１５）
0.28 0.47 0.69 0.86 0.93 0.86 0.69 0.47 0.28
0.22 0.38 0.55 0.69 0.74 0.69 0.55 0.38 0.22
0.15 0.26 0.38 0.47 0.51 0.47 0.38 0.26 0.15
0.09 0.15 0.22 0.28 0.30 0.28 0.22 0.15 0.09
【００４１】
このようなマスクを用いて、原画像データＩ₀(x,y)から平滑化画像データＩ_AV（x,y)を求めることができる。なお、本発明に用いられる平滑化方法としては、上述した種々の方法に限定されるわけではなく、従来公知の平滑化方法はいずれも適用可能なことはいうまでもない。
図４（ｃ）には、図４（ａ）に示されるプロファイル波形が、平滑化処理によって処理された結果が示され、ノイズモトルのある領域Ａや領域Ｂや領域Ｃを含むノイズ領域のノイズ成分が抑制され滑らかになるとともに、エッジ成分が支配的な領域Ｅ₁ および領域Ｅ₂ についても、エッジ信号が抑制され滑らかなプロファイル波形となっていることが判る。
【００４２】
次いで、エッジ・ノイズ混在成分の抽出工程（ステップ１０４）について説明する。
平滑化工程（ステップ１０２）で得られた平滑画像データＩ_AV(x,y) を、下記式（１６）に従って、シャープネス強調工程（ステップ１００）で得られたシャープネス強調画像データＩ_S から減算し、シャープネス強調されたエッジとノイズの混在する微細構造データである混在成分ΔＩ_EG(x,y) を抽出する。
ΔＩ_EG(x,y）＝Ｉ₀ (x,y) −Ｉ_AV(x,y) （１６）
図４（ｄ）には、図４（ｂ）および（ｃ）で示されるプロファイル波形を用いてエッジ・ノイズ混在成分の抽出工程によって得られる混在成分ΔＩ_EG(x,y) のプロファイル波形が抽出されている。
【００４３】
次に、エッジ検出工程（ステップ１０６）について説明する。
ここでは、一例として局所分散方式によるエッジ検出を代表例として説明するが、本発明はこれに限定される訳ではない。
【００４４】
エッジ検出を行う際に先ず、以下のような前処理による濃度変換を行う。このような前処理を行うのは、カラー画像データを構成するＲ画像データとＧ画像データとＢ画像データで相関のないノイズを減少させ以降で行うエッジ検出の際の精度を向上するためである。すなわち、式（１７）に示したように、原画像データＩ₀(x,y)のＲ，Ｇ，Ｂの３色の濃度値Ｄ_R ，Ｄ_G ，Ｄ_B に重み係数ｒ，ｇ，ｂを掛けて視覚濃度(Visual density)Ｄ_V に変換する。
Ｄ_V ＝（ｒＤ_R ＋ｇＤ_G ＋ｂＤ_B ）／（ｒ＋ｇ＋ｂ) （１７）
重み係数としては、例えば、ｒ：ｇ：ｂ＝４：５：１のような値を用いる。この変換を行うのは、Ｒ，Ｇ，Ｂで相関の無いノイズを減少させ、エッジ検出の精度を向上させるためである。前処理の配列の大きさの範囲は５×５、あるいは７×７画素程度のものを用いるのがよいが、それは、後述する所定の配列内の画像濃度の変動を、配列内で小さい配列、例えば、３×３程度の配列を用いて、移動しながら計算するためである。
【００４５】
なお、エッジ検出における重み係数ｒ，ｇ，ｂは以下のようにして求めることができる。
重み係数については、視覚で観察したときに目立つ（これは、分光的な視感度分布に対応するという見方もあるが）、すなわち寄与の大きい色の画像データの重み係数が大きいという考えに基づいて最適な値に設定するのが好ましい。一般には、視覚評価実験等に基づいて経験的な重み係数が求められており、下記のような値が一般的な知見として知られている（公知文献としては、野口高史、「心理対応の良い粒状評価法）、日本写真学会誌，５７（６），４１５（１９９４）があり、色によって異なるが、下記の比に近い数値が示されている）。
ｒ：ｇ：ｂ＝３：６：１
ｒ：ｇ：ｂ＝４：５：１ （１８）
ｒ：ｇ：ｂ＝２：７：１
ここで、係数の比ｒ：ｇ：ｂとして好ましい値の範囲を規定するとすれば、r ＋ｇ＋ｂ＝１０．０でｂを１．０としたときに、ｇの値として、
ｇ＝５．０〜７．０
の範囲の値が好ましい。ただし、ｒ＝１０．０−ｂ−ｇである。
【００４６】
次ぎに、エッジ検出工程（ステップ１０６）の局所分散によるエッジ検出について説明する。
エッジの検出は、上記視覚濃度Ｄ_V の画像データからｎ_E ×ｎ_E 画素の配列を移動しつつ、配列内の画像濃度変動を式（１９）を用いて、その位置毎の局所的な標準偏差である局所分散σを順次注目画素(x,y) ごとに計算することによって、画像中の被写体エッジの検出を行う。画素配列の大きさ（ｎ_E ×ｎ_E ）は、検出精度および計算負荷を考慮して適宜決めればよいが、例えば３×３、あるいは５×５程度の大きさを用いるのが好ましい。
【００４７】
【数２】

ただし、注目画素位置をx 、y とし、Ｄ_V (x+i-n_E /2 -1/2, y+j-n_E /2-1/2) は局所分散σ(x,y）を計算するｎ_E ×ｎ_E の画素配列の濃度で、< Ｄ_V (x,y) > はその配列の平均濃度で、
【数３】

である。
【００４８】
原画像データＩ_O （ｘ，ｙ）から、上記式（１９）に示した局所分散σ(x,y) を計算し、被写体画像のエッジ強度Ｅ_O (x,y) を求めるには、下記式（２１）のような指数関数で表した式を用いる。
Ｅ_O (x,y) ＝１− exp [−σ(x,y）／ａ_E ] （２１）
ただし、ａ_E は局所分散σ(x,y）の値をエッジ強度に変換する際の係数であって、エッジ強度Ｅ_O ＝０．５に割り付ける局所分散σ(x,y）の閾値σ_T とすると、
ａ_E ＝−σ_T ／ log_e (0.5) （２２）
である。σ_T の値は、ノイズと被写体輪郭の信号の大きさによって適切な値にする必要があるが、各色８ｂｉｔ（２５６階調）のカラー画像では、１０〜１００の範囲の値が好ましい。この変換は、ルックアップテーブルとして作成しておくと、変換に要する計算時間を短縮することができる。
【００４９】
エッジ強度Ｅ_O (x,y) を求める変換式としては、上記式に限定されるものではなく、他の式を用いることもできる。たとえば、下記式のようなガウシャン型の関数を用いてもよい。
Ｅ_O (x,y) ＝１− exp｛− [σ(x,y)]² ／ａ_E1 ² ｝ （２３）
ただし、ａ_E1はσ(x,y) からＥ_O (x,y) に変換する際の係数で、Ｅ_O (x,y) ＝０．５に割り付ける局所分散σ(x,y) の閾値をσ_T とすると、
ａ_E1 ² ＝−σ_T ² ／ log_e (0.5) （２４）
である。σ_T の値は、各色８ｂｉｔ（２５６階調）のカラー画像では、１０〜１００の範囲の値が好ましい。
【００５０】
また、このエッジ強度データＥ_O (x,y) は、以下に示す式（２５）のように、最大の局所分散データσ_Max で正規化され、０以上１以下の正規化されたエッジ強度データＥ_O (x,y) を得てもよい。
Ｅ₀ (x,y) ＝ σ(x,y) ／σ_Max （２５）
ここで、σ_max は、局所分散データσ (x,y)の最大値で、σ (x,y)を正規化するための定数である。σ_Max の決定方法は、式（１９）で求めた画像全体の局所分散データσ (x,y)から下記式（２６）のように最大値を求める。
σ_Max ＝Ｍａｘ｛σ (x,y)｝ （２６）
【００５１】
また、画像全体から求まる最大値を用いず、画像の一部分、例えば画像の重要被写体のある確率の高い画像の中央部分の特定範囲、あるいは全画像から間引いた画像データ（原画像データの１／４〜１／１０程度）から上記式（２５）や（２６）を用いて最大値σ_Max を求めてもよい。この場合、画像中央部分の特定範囲の画像データや間引いた画像データは、画像処理を施して処理画像データを得る前に予め粗い画素密度で得ることのできる各種処理条件調整用原画像データ（プレスキャン画像データ）を用いてもよいし、原画像データから抜き出してもよい。
より好ましくは、σ_Max は、局所分散データσ (x,y)を大きい値から順番に並べた際の上位５〜１０％以内に含まれる値の平均値，例えば上位１０％以内に含まれる値の平均値＜σ (x,y)＞_Max10%をσ_Max とし、σ (x,y)がこのσ_Max を超える場合、すべてσ_Max に置き換える。この場合、平均値は、画像全体の平均値でも、重要被写体が撮影される場合の多い中央部分の所定の範囲の平均値でも、あるいは、間引いた画像データの平均値であってもよい。
【００５２】
ところで、本発明におけるエッジ検出法としては、上記局所分散方式のエッジ検出法に限定されるわけではなく、他のエッジ検出法も利用可能である。上記局所分散方式以外のエッジ検出法には、一次微分や二次微分に基づく方法があり、それぞれに、更に幾つかの方法がある。
まず、空間的な一次微分に基づく方法としては、下記の２つのオペレータがある。
差分型エッジ抽出オペレータとして、 Prewittのオペレータ、 Sobelのオペレータ、 Robertsのオペレータなどがある。 Robertsのオペレータは下記式で表わすことができる。
g(i,j)＝｛[f(i,j) - f(i+l,j+l)]²＋[f(i+l,j) - f(i,j+l)]²｝^1/2
テンプレート型オペレータとして、８方向のエッジパターンに相当する３×３テンプレートを用いる Robinson のオペレータや Kirshのオペレータがある。
次に、空間的な二次微分に基づく方法としては、ラプラシアンを用いた方法がある。この場合、雑音を強調してしまうので、先ず正規分布型のぼかし処理をしてからエッジ検出する方法が良く用いられる。
【００５３】
図４（ｅ）には、図４（ａ）に示す原画像データＩ₀ からエッジ検出工程で求められたエッジ強度データＥ₀ の波形を示している。図４（ａ）に示すエッジ強度データＥ₀ において、領域Ｅ₁ およびＥ₂ の付近でエッジ強度データＥ₀ の値が大きくなっていることがわかる。
【００５４】
次に、ノイズ領域重付け係数の計算工程（ステップ１０８）を説明する。
エッジ検出工程（ステップ１０６）で得られるエッジ強度データＥ_O (x,y) を用いて、下記式（２７）に従って、エッジ領域の重み付け係数Ｗ_E (x,y) を求めた後、ノイズ領域の重み付け係数Ｗ_G (x,y) を、下記式（２８）に従って求める。
Ｗ_E (x,y) ＝１−α_E ＋α_E Ｅ₀ (x,y) （２７）
Ｗ_G (x,y) ＝１−Ｗ_E (x,y) （２８）
【００５５】
ここで、α_E はエッジ領域とノイズ領域の重み付けを設定する定数であり、オペレータが０以上１以下の任意の値を設定することができる。
エッジ領域の重み付け係数Ｗ_E (x,y) は、１−α_E 以上１以下の値となり、Ｗ_E (x,y) が大きいほど、その画素位置で被写体のエッジ領域である確率が高いと判断され、Ｗ_E (x,y) が小さいほど、その画素位置でノイズ領域である確率が高いと判断される。正規化されたエッジ強度データＥ₀ (x,y) の値が１．０に近いエッジ領域では、α_E の値にかかわらずエッジ領域の重み付け係数Ｗ_E (x,y) が１．０に近い大きい値となり、ノイズ領域の重み付け係数Ｗ_G (x,y) は最も小さくなる。一方、正規化されたエッジ強度データＥ₀ (x,y) が１．０より小さくなるに連れ、エッジ領域の重み付け係数Ｗ_E (x,y) は最小値１−α_E に近づき、ノイズ領域の重み付けデータＷ_G (x,y) は最大値α_E に近づく。このようなα_E は一定の値にデフォルト値として予め設定しておき、オペレータがこのデフォルト値によって画像処理された処理画像を見ながらα_E の値を必要に応じて調整するようにしてもよい。
なお、α_E が１の場合、正規化されたエッジ強度データＥ_O (x,y) 自身がエッジ領域の重み付け係数Ｗ_E (x,y) となり、１．０−Ｅ_O (x,y) がノイズ領域の重み付け係数Ｗ_G (x,y) となる。
【００５６】
図４（ｆ）には、図４（ｅ）で示されるエッジ強度データから求まるノイズ領域の重み付け係数Ｗ_G (x,y) の波形を示している。領域Ｅ₁ およびＥ₂ の付近でノイズ領域の重み付け係数Ｗ_G (x,y) の値が小さくなっている。なお、領域Ｅ₁ およびＥ₂ の付近におけるノイズ領域の重み付け係数Ｗ_G (x,y) の値は上記式（２７）で設定されるエッジ領域とノイズ領域の重み付けを設定する定数α_E によって定まる。
【００５７】
次に、ノイズデータの識別・分離工程工程（ステップ１１０）について説明する。エッジ・ノイズ混在成分の抽出工程（ステップ１０４）で得られたシャープネス強調されたエッジ成分およびノイズ成分の混在する混在成分ΔＩ_EG(x,y) に、下記式（２９）で示されるように、ノイズ重付け係数計算工程（ステップ１０８）で得られたノイズ領域の重み付け係数Ｗ_G (x,y) を乗算して、ノイズデータＧ₀ (x,y) を求める。
Ｇ₀ (x,y) ＝ΔＩ_EG(x,y) × Ｗ_G (x,y) （２９）
図４（ｇ）には、図４（ｄ）に示される混在成分ΔＩ_EG(x,y) と図４（ｆ）に示されるノイズ領域の重み付け係数Ｗ_G (x,y) を乗算して得られるノイズデータＧ₀ (x,y) のプロファイル波形が示され、エッジ成分が支配的な領域Ｅ₁ や領域Ｅ₂ のエッジの信号成分が除去されていることがわかる。
【００５８】
次に、ノイズモトル抑制分布演算工程（ステップ１１２）について説明する。まず、原画像のノイズ領域の画素におけるノイズ抑制の広がりを表すノイズ抑制分布関数ｓ（ｒ）を式（１）のように設定する。ここでｒは、点Ｐをノイズ抑制の広がりの中心位置（原点）とする座標(x,y) に位置する注目画素の原点Ｐからの距離である。
ｓ（ｒ）＝ ｅｘｐ（−ｒ／ａ） （１）
すなわち、ｓ(x,y) ＝ ｅｘｐ（−（ｘ² ＋ｙ² ）^(1/2) ／ａ） と表される。
【００５９】
ここで、上式（１）中のａは、原画像の画素におけるノイズ抑制の広がりの範囲を調整する抑制範囲定数であり、予め与えられ、あるいは入力により設定される。抑制範囲定数ａが入力により設定される場合、ノイズ抑制を及ぼす範囲の境界の位置、すなわち抑制距離ｒ_s とこの位置での値ｓ_min を定めることによって設定される。すなわち、抑制距離ｒ_s での値ｓ_min から下記式（３０）によって抑制範囲定数ａを計算して求める。
ａ ＝ −ｒ_s ／ｌｏｇ（ｓ_min ） （３０）
【００６０】
ここで、ｓ_min の値は０．１以上０．５以下であるのが好ましく、抑制距離ｒ_s の値は１以上１５以下の範囲の値であることが好ましい。特に、フィルムに記録された画像をスキャナ等で読み込んで原画像データを得る場合、抑制距離ｒ_s の値の設定は、フィルムフォーマット（フィルムの１コマの画像サイズ）やフィルムの種類や感度によって異なるフィルムの粒状（粒状モトル）の粗さ、およびフィルムを走査してディジタル画像データを得るスキャナの走査アパーチャサイズに応じて適宜変更することが好ましい。すなわち、粗い粒状（粒状モトルも大きい）の場合、抑制距離ｒ_s の値を大きくし、逆に細かい粒状の場合、抑制距離ｒ_s の値を小さくする。また、スキャナの走査アパーチャサイズに応じて適宜変更するのは、同じフィルムであっても、スキャナの走査アパーチャが１５μｍ□の場合と１０μｍ□とした場合、抑制範囲定数ａの値が同じでも、実際の画像密度が１．５倍、１０μｍ□の方の画像の画素密度が高くなり、粒状の粗さが変化するからである。
【００６１】
また、この抑制距離ｒ_s は、平滑化工程（ステップ１０２）における平均化のマスクサイズを大きくするとステップ１１０で得られるノイズデータＧ₀ に含まれるノイズモトルの成分も大きくぼけて広がるため、このノイズモトルの成分を抑制するために抑制距離ｒ_s を大きくする必要がある。それゆえ、平滑化工程（ステップ１０２）における平均化のためのマスクサイズに応じて、抑制距離ｒ_s も適宜変更するとよい。
平滑化工程における平均化のためのマスクサイズは、３画素×３画素程度を最小マスクサイズとし、１１画素×１１画素程度を最大マスクサイズとするが、抑制範囲定数ａは、このマスクサイズより１．５倍〜２倍程度大きめの値とし、例えば、マスクサイズが３×３の場合、抑制距離ｒ_s の値を５程度とするのが好ましい。
【００６２】
このようなノイズ抑制分布関数ｓ（ｒ）として式（２）のような関数を設定してもよい。
ｓ（ｒ）＝ ｅｘｐ（−ｒ² ／ａ² ） （２）
ここで、ｒは、ノイズ抑制の広がりの中心位置からの距離である。
この場合、抑制範囲定数ａは、抑制距離ｒ_s とその位置での値ｓ_min を用いて下記式（３１）によって求める。
ａ ＝ ｒ_s ／［−ｌｏｇ（Ｓ_min ）］^(1/2) （３１）
あるいは、ノイズ抑制分布関数ｓ（ｒ）として下記式（３）のような関数を設定してもよい。
ｓ（ｒ）＝ ｒｅｃｔ（ｒ／ａ） （３）
ここで、ｒｅｃｔ（ｒ／ａ）は、値が１の矩形関数でであり、ｒは、ノイズ抑制の広がりの中心位置からの距離である。
この場合、抑制範囲定数ａは、設定する抑制距離ｒ_s とする。また式（２）や式（３）においてｓ_min ＝１とすることで式（１）と同一のノイズ抑制分布関数ｓ（ｒ）を得ることができる。
【００６３】
このようなノイズ抑制分布関数ｓ（ｒ）は、下記式（３２）に従って、０以上１以下のノイズ抑制分布関数ｓ₁ (x,y) に正規化される。
ｓ₁ (x,y) ＝ ｓ(x,y) ／Ｓ （３２）
ここで、Ｓは、下記式（３３）によって求まる積分値である。
【数４】

図４（ｈ）には、式（１）を用いて得られるノイズ抑制分布関数ｓ(x,y) を式（３３）に従って正規化したノイズ抑制分布関数ｓ₁ (x,y) の一例が示される。
【００６４】
次に、この正規化されたノイズ抑制分布関数ｓ₁(x,y)とノイズデータＧ₀ (x,y) とを用い、下記式（３４）のようにノイズ抑制の及ぼす範囲において畳み込み積分を行い、ノイズモトル抑制分布Ｍ(x,y) を算出する。
【数５】

ここで算出するノイズモトル抑制分布Ｍ(x,y) は、ある注目画素位置(x,y) におけるノイズの、他の画素から受ける抑制の程度を畳み込み積分によって求めるものであって、原画像データＩ₀ がフィルム画像からスキャナ等で読み取られた画像データである場合の、フィルム感光層中のハロゲン化銀粒子の現像時に生成される現像抑制物質の拡散分布に相当するもので、ノイズデータＧ₀ の変動を感光材料の粒子の分布と見なして、原画像データＩ₀ に含まれる、感光材料の銀粒子を中心にして生成される現像抑制物質の拡散分布に相当するノイズモトル抑制分布Ｍ(x,y) を求めるものである。
【００６５】
図４に示す例で説明すると、図４（ｇ）に示されるノイズデータＧ₀ と図４（ｈ）に示されるノイズ抑制分布関数ｓ₁(x,y)を用い、式（３４）に従って算出することで、図４（ｉ）に示されるようなノイズモトル抑制分布Ｍ(x,y) を得る。図４（ｉ）では、図中領域Ａや領域Ｂや領域Ｃに示される部分でノイズモトル抑制分布の値が大きくなり、この領域Ａ、領域Ｂおよび領域Ｃの部分では、銀塩感光材料にの場合、銀塩感光材料で生成される現像抑制物質の濃度が高くなっていることを意味するものである。
このようにしてノイズモトル抑制分布Ｍ(x,y) が算出される。
【００６６】
次に、ノイズモトル抑制成分演算工程（ステップ１１４）を説明する。
ノイズモトル抑制成分演算工程では、ステップ１１２で演算されて算出されたノイズモトル抑制分布Ｍ(x,y) を下記式（３５）で示すように、ステップ１１０で求めたノイズデータＧ₀ に乗算することで、ノイズ抑制成分画像データＧ₁ (x,y) を演算する。
Ｇ₁ (x,y) ＝ Ｇ₀(x,y) × Ｍ(x,y) （３５）
【００６７】
ノイズモトル抑制分布Ｍ(x,y) をノイズデータＧ₀ に乗算するのは、銀塩感光材料の場合、銀塩感光材料中で生成される現像抑制物質と銀粒子の作用に応じて抑制することを考慮するためである。図４に示す例では、図４（ｊ）にノイズ抑制成分画像データＧ₁ (x,y) が示される。図４（ｊ）に示されるノイズ抑制成分画像データＧ₁ (x,y) を図４（ｇ）に示されるノイズデータＧ₀ と比較すると、領域Ａや領域Ｂや領域Ｃでは、それ以外の領域、例えば領域Ｅ₁ や領域Ｅ₂ に対して、値が相対的に大きくなっていることがわかる。
【００６８】
最後に、ノイズ抑制画像演算工程（ステップ１１６）について説明する。
ノイズ抑制画像演算工程では、ステップ１１４で求めたノイズ抑制成分画像データＧ₁ (x,y) を、下記式（３６）で示すように、ノイズ抑制係数αを用いて変倍した後、ステップ１００で求めたシャープネス強調画像データＩ_S から減算し処理画像データＩ₁ ’を求める。
Ｉ₁'(x,y) ＝ Ｉ₀(x,y) − α×Ｇ₁(x,y) （３６）
ここで、ノイズ抑制係数αは、ノイズ抑制の程度を制御する設定可能なパラメータであり、適宜設定入力される。あるいは、予めデフォルト設定値を設け、必要に応じて変更するものであってもよい。
その後、画像出力装置１６に適した画像データの変換を行って処理画像データＩ₁ ’から処理画像データＩ₁ を得る。
【００６９】
図４に示す例では、図４（ｋ）に示すような処理画像データＩ₁ ^' のプロファイル波形を得ることができ、領域Ａや領域Ｂや領域Ｃ等のノイズモトルの領域ではノイズが大きく除去され、それ以外のノイズ領域でもノイズが適切に除去されていることがわかる。しかも、図４（ａ）に示される原画像データＩ₀ と比較して、領域Ｅ₁ や領域Ｅ₂ のエッジ領域はシャープネス強調されている。
【００７０】
上記画像処理方法は、写真感光材料等における粒状成分のように、粒状成分が一定の画素領域に影響を及ぼす、すなわち、粒状成分が他の画素の粒状成分に影響を及ぼす場合、粒状抑制のための画像処理として有効であるが、また、上記画像処理法は、他の画素のノイズ成分の影響を受けないＣＣＤやＭＯＳ撮像素子等を利用するデジタルカメラ等においても、他の画素の画像データを利用して傷欠陥補正等を行うことから、効果的に適用することができる。ＣＣＤ撮像素子としては、特開平１０−１３６３９１号に記載のハニカム配列のものにも適用される。
【００７１】
なお、ノイズ抑制のための画像処理方法として、特開平１１−２５０２４６号公報では、画像のエッジ強度から求めた圧縮係数をノイズ成分に乗じてノイズを抑制する方法が提案されているが、ノイズ領域では一定の圧縮係数が掛かるのでノイズの振幅が比例的に圧縮される。また、ノイズの振幅が大きくなる程圧縮率が高くなる圧縮係数を採用することで、すなわち、ノイズの揺らぎの大きさ（振幅）に非線形変換を施すことで、ノイズの揺らぎを上記公報に記載される画像処理方法より均一な方向にすることもできる。しかし、本発明の画像処理方法は、上記２つの方法のようにノイズの振幅に対して抑制処理を施すのと異なり、ノイズモトルのようにノイズ成分の揺らぎが空間的に大きく形成されるノイズ領域、例えば図４（ａ）で示される領域Ａや領域Ｂや領域Ｃに対してノイズ抑制を施す処理であり、ノイズの揺らぎが大きい場所でより大きく抑制し、ノイズの揺らぎを均一化することができる。
【００７２】
本発明の画像処理方法は以上のように説明される。
このような画像処理方法は、回路やハードウェアから成る上述した画像処理装置として構成してもよいし、あるいは、ソフトウェアとしてコンピュータの中で機能を発揮するようなプログラムであってもよく、この場合、上記方法を実行するためのプログラムを記録したコンピュータが読み取り可能な記録媒体、ＣＤ＝ＲＯＭ等として提供するものであってもよい。
【００７３】
このような本発明の画像処理方法を、銀塩カラー写真、例えば３５ｍｍカラーネガフィルムに撮影した写真画像に適用したところ、粒状とシャープネスともに、一見して判る程の顕著な改善効果を得ることができた。
特に、粒状について感光材料の微粒子化による粒状改良に匹敵する処理効果を持つため、従来の平均化や揺らぎの減少に基づく各種の粒状除去処理法の欠点であった「ぼけ粒状」的な不自然さや違和感がなくなった。また、シャープネスについては、上記粒状抑制と組み合わせることにより、粒状を悪化させずに、従来のアンシャープネスマスクやラプラシアンフィルタより大幅な強調効果が得られた。
また、デジタルスチルカメラで撮影した画像にも適用したところ、上記銀塩カラー写真によるカラー画像と同様に顕著な画質向上効果が得られた。
【００７４】
以上、本発明の画像処理方法、画像処理装置および記録媒体について詳細に説明したが、本発明は上記実施例に限定はされず、本発明の要旨を逸脱しない範囲において、各種の改良および変更を行ってもよいのはもちろんである。
【００７５】
【発明の効果】
以上、詳細に説明したように、本発明によれば、先ず原画像をシャープネス強調することによって、被写体画像とノイズの双方を鮮鋭化しておき、その画像から被写体輪郭とノイズ成分を抽出し、ノイズ成分を選択的に除去するので、ノイズがぼけて視覚的に不快に見える大きなむらを形成せず、コントラストの低い画像信号をノイズと誤認することもなく、ノイズ除去領域とシャープネス強調領域の境界に不自然なアーチファクトも形成しない。さらに、ノイズ成分の濃度揺らぎが空間的に粗い領域を抽出し選択的に大きく除去することによってノイズ成分を抑制しているので、ノイズモトル等のような大きく粗いノイズ成分を処理画像が含むことも少なく（濃度揺らぎが小さい）、従って、濃度揺らぎが均一化され、空間的にも細かいノイズにすることができ、視覚的にも自然なノイズの抑制を実現することができる。
【００７６】
また、本発明によると、ノイズが粒状である場合、粒状はシャープネス強調され、かつ、空間的に揺らぎの大きい粒状モトル等の粒状成分が除去・抑制され、粒状パターンが微細化、均一化されるので、銀塩写真の感光材料では微粒子乳剤を用いた時に得られるような細かい粒状となり、平滑化を用いた従来法の欠点であるぼけ粒状のような視覚的な違和感や不快感の無い自然な粒状抑制効果が得られる。また、本発明の画像処理法を銀塩カラー写真感光材料に適用することにより、従来の粒状抑制処理方法の欠点であった、いわゆる「ぼけ粒状」的な不自然さや違和感がなく、粒状が改善され、極めて顕著な改善効果を得ることができ、産業上大きな効果が得られる。
【図面の簡単な説明】
【図１】 本発明に係る画像処理装置を組み込んだ、カラー写真画像を読み取り、ノイズ抑制とシャープネス強調の画像処理を行い、出力装置でカラー画像を出力するシステムの一実施例を示すブロック図である。
【図２】 本発明に係る画像処理装置の、ノイズ抑制とシャープネス強調の画像処理を行う画像処理部の一実施例を示すブロック図である。
【図３】 本発明の画像処理方法の一実施例を示すブロック図である。
【図４】 （ａ）〜（ｋ）は、本発明の画像処理方法によって画像データが処理される一例を示す図である。
【符号の説明】
１０ カラー画像再生システム
１２ 画像入力装置
１４ 画像処理装置
１６ 画像出力装置
１８ 色・調子処理部
２０ ノイズ抑制画像処理部
２２ 画像モニタ・画像処理パラメータ設定部
２６ 平滑化処理部
２８ エッジ・ノイズ混在成分抽出部
３０ エッジ検出部
３２ ノイズ領域重み係数演算部
３４ ノイズ成分識別分離部
３６ ノイズ抑制分布関数設定部
３８ ノイズモトル抑制分布演算部
４０ ノイズ抑制成分演算部
４２ ノイズ抑制演算処理部[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing method, apparatus, and recording medium for digital images that suppress noise components such as graininess of digital images and enhance sharpness.
[0002]
[Prior art]
In a system where a digital image obtained by scanning a silver halide photograph with an image input scanner or a digital image taken with a digital still camera is processed and output by an image output printer, the output image is scanned by a scanner. In order to recover from this, sharpness enhancement by a Laplacian filter or an unsharp mask (USM) has been conventionally performed. However, since there is a side effect that noise (noise) such as granularity deteriorates while improving the sharpness of the image, in a noisy image such as granularity, only modest sharpness enhancement can be performed within a range where granular deterioration is allowed, It was difficult to improve the image quality over the original image.
[0003]
Several digital image processing methods have been proposed to remove noise graininess and enhance sharpness in digital images. However, since a method of averaging or blurring is used as a method of removing graininess, a blurred grain pattern However, it is not suitable for an aesthetic image such as a photograph.
[0004]
In images from photographs, prints, televisions, various copying machines, etc., sharpness deterioration due to optical systems such as cameras, graininess and sharpness deterioration inherent to photographic photosensitive materials, or original images such as photographs and prints can be input with an image input device. Various methods have been devised as image processing methods for suppressing noise or enhancing sharpness in order to recover noise (noise) and sharpness degradation added in digitization. For example, in the conventional image processing method, smoothing or coring is used as the granular removal processing method, and unsharp masking (USM), Laplacian, or high-pass filter processing is used as the sharpness enhancement processing method. It is used. However, in these conventional grain removal processing methods, it is not desirable that suppressing the grain causes unnatural discomfort artifacts or suppresses the fine structure of the image that should not be suppressed together with the grain. Had drawbacks.
[0005]
For example, Japanese Patent Publication Nos. 57-500311 and 57-500334 and “Digital Sharpening Method of Unsharp and Grainy Photo Images”, International Conference on Electronic Image Processing, July 1982, pp. 179-183 (PG Powell and BEBayer, `` A Method for the Digital Enhancement of Unsharp, Grainy Photographic Images '', Proceedingus of the International Conference on Electronic Image Processing, Jul. 26-28, 1982, pp. 179-183) In the Powell and Bayer et al. Processing method, a smoothing processing method (low-pass filter) is used as a graininess suppression method, and a processing method using an unsharp mask (high-pass filter) is used as a sharpness enhancement method. The smoothing process is a process for suppressing grain by smoothing a signal by multiplying a signal value of n × n pixels by a weight such as a Gaussian type. In the sharpness enhancement processing, first, a differential value in the direction from the center pixel to the surrounding pixels is obtained using an image signal of m × m pixels, and if the value is smaller than a set threshold value, it is regarded as grain or noise and coring is performed. The sharpness enhancement is performed by taking the sum of the differential values larger than the remaining threshold value after removal by processing, multiplying by a constant of 1.0 or more, and adding to the smoothed signal.
[0006]
In this processing method, since the granular pattern is blurred, the contrast of the granular pattern is lowered, but a large uneven pattern composed of large settlements (granular mottle) of particles constituting the granularity becomes visually noticeable. Therefore, there is a drawback that it looks like an unpleasant granularity. In addition, since the granularity and the image are identified with the set threshold (coring processing), the low-contrast image signal is erroneously recognized as granular and is suppressed or removed together with the granularity. Discontinuity occurs at the boundary of the image, and unnatural artifacts are seen in the image. In particular, this defect appears in a fine image such as a lawn and a carpet, and an image in which a texture such as a fabric is depicted, and this is a visually unnatural and undesirable artifact.
[0007]
[Problems to be solved by the invention]
By the way, in the conventional grain suppression / sharpness enhancement image processing method described above, sharpness is enhanced with an unsharp mask, grain is blurred or smoothed, and a method of suppressing graininess (noise) signal and contour signal from the original image is used. Are processed at the signal level, the contour signal is sharpened and the smooth area is suppressed by granularity, so that small signals are processed as granular. There is a problem in that image signals such as texture and hair are suppressed together with graininess, resulting in a visually unpleasant image as an image processing artifact. That is, in such a conventional method, the image is blurred by using averaging as a method of suppressing graininess, and the blurred granular pattern (“blurred granularity”) becomes smaller as the fluctuation of the density in the image and the graininess is better. On the other hand, a granular pattern with a small amount of density fluctuation but blurred but spread is visually recognized as an unpleasant pattern, especially for portraits and other faces and skin, walls and sky There was a problem of being conspicuous with a uniform subject.
[0008]
In the conventional method of separating the granular (noise, noise) area and the outline area from the original image at the signal level, the outline area and the flat area are identified from the difference signal between the original image and the blurred image, and each area is unsharp. By processing using different coefficients such as mask and Laplacian, graininess is suppressed in the flat area while graininess is suppressed in the outline area without enhancing the sharpness and blurring the edges. There is a problem that discontinuity occurs at the boundary because recognition / separation is uniformly performed at a signal level that is a threshold value.
Furthermore, in such a conventional method, an unsharp mask or Laplacian is used as an edge enhancement or sharpness enhancement method. However, a border such as a McKee line is likely to occur at the contour / edge portion of the image, which is visually inconvenient. There was a problem of giving a natural impression.
These problems related to noise suppression and sharpness enhancement are not unique to silver halide photography, but also include shot noise and electrical noise when shooting images with a digital still camera. It occurs as a problem of various noise suppression and sharpness enhancement.
[0009]
The present invention has been made in view of the current state of the prior art described above. In images of silver salt photographs, digital still camera images, printing, televisions, various copying machines, etc., blur due to cameras, granularity of photographic photosensitive materials. Noise (sharpness) and sharpness degradation inherent in original images such as blurring, noise added when digitizing the original image with an image input device, and shot noise and sharpness when shooting with a digital still camera When performing processing to recover the degradation, the above-mentioned problems of the prior art, that is, the noise is blurred and large unevenness appears visually unpleasant, and the image signal has a low contrast. Is grainy or misidentified as noise, and is suppressed or removed, and the boundary between the noise removal area and sharpness enhancement area becomes discontinuous, resulting in an image Digital image processing method that suppresses noise and enhances image sharpness without causing a problem that natural artifacts can be seen, an image processing apparatus that performs the image processing method, and a computer that executes the method are readable It is an object to provide a simple recording medium.
[0010]
[Means for Solving the Problems]
In order to achieve the above object, the present invention performs sharpness enhancement processing on original image data, and creates sharpness enhanced image data in which noise included in the image is sharpened together with the image,
Smoothing the original image data to create smoothed image data;
Subtracting the smoothed image data from the sharpness-enhanced image data to create mixed image data in which the sharpness-enhanced subject image edge and the sharpness-enhanced noise are mixed,
Obtaining edge strength data for performing edge detection from the original image data to identify a subject edge region and a noise region;
From this edge strength data, the noise area weighting coefficient indicating the degree of the noise area is obtained,
Multiplying the mixed image data by the weighting coefficient of the noise region to obtain noise data,
Set a noise suppression distribution function representing the spread of noise suppression in the original image data, perform a convolution integration of the noise data and this noise suppression distribution function to obtain a noise suppression distribution,
By multiplying the noise data by the noise suppression distribution, noise suppression component image data is calculated,
Noise suppression of a digital image, wherein the noise suppression component image data is scaled and subtracted from the sharpness-enhanced image data to create an image that selectively suppresses noise components in the noise region of the original image An image processing method is provided.
[0011]
Here, the noise suppression distribution function is preferably a monotonically decreasing function that has a maximum value at the center position of the noise suppression spread in the pixels of the original image and decreases as the distance from the center position decreases. The noise suppression distribution function is preferably a rectangular function having a constant value in a range in which noise suppression is performed in pixels of the original image.
[0012]
The range of the noise suppression distribution function that exerts the noise suppression in the pixels of the original image is preferably a region in the range of 1 to 15 in terms of the number of pixels, and the noise suppression distribution function is the original image. It is preferable to set this by determining the position of the boundary of the range in which noise suppression is performed in this pixel and the value at this position.
[0013]
When r is a distance from the center position of the noise suppression spread in the pixels of the original image and a is a suppression range constant that determines the range of the noise suppression spread in the pixels of the original image, The function s (r) represented by 1) is preferable,
s (r) = exp (−r / a) (1)
s (r) = exp (−r²/ A²(2)
s (r) = rect (r / a) (3)
Here, rect (r / a) in equation (3) is a rectangular function having a value of 1.
[0014]
Furthermore, in order to achieve the above object, the present invention includes a sharpness enhancement processing unit that performs sharpness enhancement processing on original image data and creates sharpness enhanced image data in which noise included in the image is sharpened together with the image,
Performing a smoothing process on the original image data to create a smoothed image data; and
Subtracting the smoothed image data created by the smoothing processing unit from the sharpness-enhanced image data to create mixed image data in which sharpness-enhanced subject image edges and sharpness-enhanced noise are mixed A noise mixed component extraction unit;
An edge detection unit for performing edge detection from the original image data to obtain edge intensity data for identifying a subject edge region and a noise region;
A noise region weighting factor calculation unit for obtaining a weighting factor of a noise region indicating the degree of the noise region from the edge intensity data obtained by the edge detection unit;
A noise component identification / separation unit that obtains noise data by multiplying the mixed image data created by the edge / noise mixed component extraction unit by a weighting coefficient of the noise region obtained by the noise region weighting factor calculation unit;
A noise suppression distribution function setting unit for setting a noise suppression distribution function representing a spread of noise suppression in the original image data;
A noise suppression distribution calculation unit that obtains a noise suppression distribution by performing convolution integration of the noise data determined by the noise component identification separation unit and the noise suppression distribution function set by the noise suppression distribution function setting unit;
A noise suppression component calculation unit that calculates noise suppression component image data by multiplying the noise suppression distribution determined in the noise suppression distribution calculation unit by the noise data calculated by the noise component identification separation unit;
A noise suppression calculation processing unit that obtains processed image data by scaling and subtracting the noise suppression component image data calculated by the noise suppression component calculation unit from the sharpness-enhanced image data. An image processing apparatus for noise suppression is provided.
[0015]
In order to achieve the above object, the present invention provides:
A procedure for performing sharpness enhancement processing on the original image data and creating sharpness enhancement image data in which noise included in the image is sharpened together with the image,
A procedure for performing smoothing processing on the original image data to create smoothed image data;
Subtracting the smoothed image data from the sharpness-enhanced image data to create mixed image data in which the edge of the sharpened image and the noise of the subject image are enhanced.
A procedure for performing edge detection from the original image data to obtain edge intensity data for identifying a subject edge region and a noise region;
A procedure for obtaining a weighting coefficient of a noise region indicating a degree of the noise region from the edge strength data,
A procedure for obtaining noise data by multiplying the mixed image data by a weighting coefficient of the noise region;
Setting a noise suppression distribution function representing the spread of noise suppression in the original image data, obtaining a noise suppression distribution by performing convolution integration of the noise data and the noise suppression distribution function;
Calculating noise suppression component image data by multiplying the noise data by the noise suppression distribution;
A method for causing a computer to execute a procedure for scaling and subtracting the noise suppression component image data from the sharpness-enhanced image data to create an image in which a noise component in a noise region of an original image is selectively suppressed. A computer-readable recording medium recording a program to be recorded is provided.
[0016]
Here, the noise is not only the grain attributed to the particles of the photosensitive material contained in the image data obtained by reading a photographic film using a silver salt photosensitive material with a film scanner, but also silver salt photosensitive. It also includes a wide range of noise included in image data obtained using various image pickup tubes and image pickup elements such as CCD and MOS such as digital still cameras without using materials.
[0017]
The above invention
-Detect edges from the original image, obtain edge strength, consider areas with low edge strength as noise areas, calculate noise area weighting coefficients to divide the noise area from edge areas,
-Create a sharpness-enhanced image and a smoothed image from the original image, and subtract the smoothed image from the sharpness-enhanced image to obtain a mixed component of sharpness-enhanced edges and sharpness-enhanced noise,
・ From the noise component obtained by multiplying the noise area weighting coefficient by the edge and noise mixture component, obtain the noise suppression distribution using the noise suppression distribution function, and calculate the noise suppression component from this noise suppression distribution and the above noise component. Seeking
-By subtracting the noise suppression component from the sharpness-enhanced image,
For example, when the original image data is read from a photographic film using a silver salt photosensitive material using a film scanner, it is a large colony of photosensitive material particles, and the granular mottle that forms noise components is larger (granularity). The grain is strongly suppressed in a portion where the grain is coarse, and the grain is weakly suppressed or suppressed in a portion where the granular mottle is small.
As a result, noise fluctuations such as graininess can be suppressed and uniformed, and a high-quality image can be obtained in which a spatially fluctuating part such as a granular mottle is selectively suppressed in the noise region of the image. .
When the original image data is obtained using an image sensor such as a digital still camera, it is generated by photon fluctuations of incident light, noise inherent to each photosensor in the image sensor, or an electric circuit. Various noises such as thermal noise (noise), quantization noise, etc. appear as pixel unit signals or density fluctuations in the image, but noise fluctuations are spatially sparse or dense, Since the noise motor is formed in the same way as the granular motor of the silver halide photosensitive material, the noise can be made uniform by suppressing strongly where the noise motor is large and weakly suppressing where the noise motor is small. A good image can be obtained.
[0018]
That is, in the noise suppression image processing method of the present invention, noise components are identified, large and rough noises such as noise motors in the noise components are suppressed more strongly, and those with less noise are suppressed or not suppressed. By performing this process, a process of making the fluctuation of the image density due to noise small and uniform (a variation in fluctuation magnitude is reduced) is performed. When the noise component is a granular component, the granular pattern is not subjected to smoothing processing as in the past, and the large granular mottle is not noticeable, so that it can be obtained when a fine grain emulsion is used in a silver salt photographic material. It can be made into fine granularity (spatially fine and granular with small amplitude).
[0019]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an image processing apparatus that performs an image processing method of the present invention will be described in detail based on a preferred embodiment shown in the accompanying drawings.
“Noise” in the present invention refers to noise in general as described above. In addition, “noise” is simply referred to as noise in an electronic imaging system such as a digital still camera, but in silver halide photographic light-sensitive materials, it is generally referred to as granular rather than noise. Therefore, in the following description, when describing a silver salt photographic light-sensitive material as an example, “grain” is used instead of “noise”. The “noise motor” described later includes a granular motor in a silver halide photographic material as well as a noise motor in which noise fluctuation forms a spatially dense portion in an electronic imaging system such as a digital still camera.
[0020]
FIG. 1 is a block diagram of a color image reproduction system that incorporates an image processing apparatus according to the present invention, reads a color image, performs the image processing method of the present invention, and outputs a color image. FIG. 2 is a block diagram of an embodiment of an image processing apparatus for performing the image processing method according to the present invention. FIG. 3 is a flowchart showing an example of the processing algorithm of the image processing method of the present invention. In the following description, image data obtained from a silver salt color photographic image as a digital image will be described as a representative example.
[0021]
As shown in FIG. 1, the color image reproduction system 10 obtains digital input image data by reading a color image such as a color photographic image (a film image such as a color negative film or a color reversal film or a photographed image such as a digital camera). The image input device 12 and the input image data input from the image input device 12 are subjected to the required image processing and the image processing for noise suppression and sharpness enhancement of the digital image of the present invention to obtain processed image data I₁Image processing apparatus 14 for obtaining the processed image data I output from the image processing apparatus 14₁And an image output device 16 for outputting a color image such as a print image.
[0022]
The image input device 12 is for creating digital color image data and outputting it as input image data to the image processing device 14. For example, a color (or monochrome) negative film or a color (or monochrome) reversal film is used. A film scanner device that reads digital color film images to create digital image data, and a reflective document scanner device that reads digital color reflection document images such as printed matter and reflection print images to create digital image data. In the present invention, a digital camera, an electronic still camera, a video camera that directly shoots a subject to create digital image data, or a recording medium that stores digital image data created by these, such as smart media, Digital image data by driving semiconductor memory such as Memory Stick, PC card, magnetic recording media such as FD and Zip, magneto-optical recording media such as MO and MD, and optical recording media such as CD-ROM and Photo-CD A driver for reading, a display device such as a CRT monitor and a liquid crystal monitor for reading these digital image data and displaying a soft copy image, and an image processing for processing all or part of the read or displayed digital image data PC, WS, etc. It may be.
[0023]
The image output device 16 processes the processed image data I output from the image processing device 14 as final processed image data.₁Digital photo printers, copiers, and electronic devices that output color hard copy images such as reflection print images and reflection original images. Image output devices such as digital color printers of various systems such as photographs, laser printers, ink jets, thermal sublimation types, TAs, display devices such as TVs, CRT monitors, and liquid crystal monitors that display as soft copy images, PCs, WSs, etc. A computer etc. can be mentioned.
[0024]
The image processing device 14 adjusts the color and tone (gradation) of the input image data from the image input device 12 to output the desired color and tone to the image output device 16 to perform original image data I_OA color / tone processing unit 18 for generating the original image data I processed by the color / tone processing unit 18_OThe most characteristic part of the present invention is the processing of image data I by implementing the image processing method for noise suppression and sharpness enhancement of the digital image of the present invention.₁A noise suppression / sharpness-enhanced image processing unit 20 for generating the image, an image monitor for displaying a reproduced image based on the image data whose color and tone reproducibility are adjusted, and various required image processing and image processing of the present invention. An image monitor / image processing parameter setting unit 22 including an image processing parameter setting unit for setting parameters for the purpose.
[0025]
Here, the color / tone processing unit 18 performs color conversion or color so that the reproducibility of the color and tone (gradation) of the input image data input from the image input device 12 is properly reproduced in the image output device 16. Original image data I for performing the image processing method of the present invention by performing correction (including gradation conversion or correction)._OThe processing performed here includes, for example, color (gray) conversion and correction, gradation correction, density (brightness) correction, saturation correction, magnification conversion, density dynamic range compression / expansion And various other processes.
[0026]
The image monitor / image processing parameter setting unit 22 includes an image monitor and an image processing parameter setting unit. The image monitor / image processing parameter setting unit 22 displays an input image on the image monitor based on the input image data input from the image input device 12. A mouse or a keyboard (not shown) used for various image processing parameters used by the color / tone processing unit 18 and the noise suppression image processing unit 20 for performing the image processing method of the present invention on the input image data (for example, by GUI) For setting by a data input device such as Here, the parameters to be set include correction coefficients, conversion coefficients, magnifications, and the like used in the various processes described above, and various coefficients and constants necessary for carrying out the image processing method of the present invention described in detail later. And so on.
[0027]
A noise suppression image processing unit (hereinafter simply referred to as a main image processing unit) 20 that implements the image processing method of the present invention includes original image data I created by the color / tone processing unit 18._OIn addition, processed image data I, which is final processed image data for performing image processing for noise suppression and sharpness enhancement, which is a feature of the present invention, and outputting the processed image data to the image output device 16 is performed.₁Is for creating.
[0028]
Here, the main image processing unit 20, as shown in FIG._OSharpness-enhanced image data I obtained by performing sharpness-enhancement processing and sharpening noise (noise) including graininess included in the image._SSharpness enhancement processing unit 24 for creating image data I and original image data I_OSmoothed image data I_AVA smoothing processing unit 26 for generating the original image data I₀To smoothed image data I_AVThe mixed image data ΔI in which the edge and noise of the subject image are mixed_EGEdge / noise mixed component extraction unit 28 for generating the original image data I_OEdge strength data E for detecting the edge of the subject image from the image and identifying the subject edge region and the noise region_OEdge detecting unit 30 for obtaining the edge strength data E_OTo noise area weighting factor W_GThe mixed image data ΔI obtained by the noise region weighting coefficient calculation unit 32 and the edge / noise mixed component extraction unit 28._EGThe noise weighting coefficient W obtained by the noise weighting coefficient calculator 32_GIs multiplied by the noise data G in the noise area₀Noise component identification / separation unit 34 for obtaining the original image data I_ONoise suppression distribution function s representing the spread of noise suppression in₁The noise data G obtained by the noise suppression distribution function setting unit 36 for calculating and setting₀And the noise suppression distribution function s set by the noise suppression distribution function setting unit 36₁And a noise motor suppression distribution calculation unit (equivalent to the noise suppression distribution calculation unit in the present invention) 38 obtained by calculating a noise motor suppression distribution (corresponding to the noise suppression distribution in the present invention) M by performing convolution integration with Noise data G obtained by the noise component identification / separation unit 34 based on the noise motor suppression distribution M obtained by the unit 38₀Is multiplied by the noise suppression component image data G₁The noise suppression component calculation unit 40 for calculating the noise suppression component image data G calculated by the noise suppression component calculation unit 40₁And the sharpness enhancement image data I obtained by the sharpness enhancement processing unit 24_SProcessed image data I suitable for the image output device 16₁And a noise suppression calculation processing unit 42 for converting to.
[0029]
The noise suppression / sharpness enhancement image processing unit 20 shown in FIG. 2 is basically configured as described above.
Next, the image processing method of the present invention will be outlined based on the operation of the processing unit 20 with reference to the flowchart showing the processing algorithm of the image processing method of the present invention shown in FIG.
[0030]
In this embodiment, as shown in FIG. 3, first, for each pixel, original image data I_OThen, sharpness enhancement processing unit 24 performs sharpness enhancement processing (step 100), and sharpness enhancement image data I_SIs smoothed by the smoothing processing unit 26 (step 102), and the smoothed image data I_AVAnd the mixed image data ΔI in which the edge and noise are sharpened and sharpened in the edge / noise mixed component extraction unit 28._EGIs extracted (step 104).
[0031]
On the other hand, in the edge detection unit 30, the original image data I_OEdge strength data E for identifying a subject edge area and a noise area from_OEdge detection is performed (step 106), and the noise area weighting coefficient calculation unit 32 performs noise area weighting coefficient W._GIs calculated and obtained (step 108).
Further, the noise component identification / separation unit 34 identifies and separates noise data (step 110). That is, the mixed image data ΔI_EGThe noise weighting coefficient W obtained by the noise weighting coefficient calculator 32_GMultiplied by the noise data G₀Ask for.
[0032]
Next, in the noise suppression distribution function setting unit 36, the original image data I₀Noise suppression distribution function representing the spread of noise suppression in₁In the noise motor suppression distribution calculation unit 38, the noise data G₀And this noise suppression distribution function s₁The noise motor suppression distribution M is obtained by performing convolution integration with (step 112), and the noise suppression component calculation unit 40 converts the noise motor suppression distribution M into the noise data G.₀Is multiplied by the noise suppression component image data G₁Is calculated (the noise motor suppression distribution is calculated) (step 114). In the noise suppression calculation processing unit 42, the sharpness enhancement image data I obtained by the sharpness enhancement processing unit 24 is calculated._SNoise suppression component image data G calculated previously from₁Is scaled and subtracted (a noise-suppressed image is calculated), and further, if necessary, converted into image data suitable for the image output device 16 to process image data I₁Is obtained (step 116).
[0033]
Next, each process described above of the image processing method of the present invention will be described in detail.
First, the sharpness enhancement step (step 100) will be described.
Here, as a method of enhancing the sharpness of an image, unsharp masking (USM) or Laplacian is well known. Also in the present invention, by using these, the sharpness of the image can be enhanced if the sharpness degradation of the image is slight.
[0034]
The unsharp mask is the original image data I₀From (x, y) (assuming the pixel position of interest is x and y), I₀Image I averaged or blurred (x, y)_av  Edge enhancement component I obtained by subtracting x, y)₀(x, y) -I_av(x, y) is multiplied by a coefficient a to obtain original image data I₀By adding to (x, y), the sharpness-enhanced image I is given by the following equation (4)._SThis is a method for obtaining (x, y).
I_S(x, y) = I₀(x, y) + a [I₀(x, y) -I_av  x, y)] (4)
Here, a is a constant for adjusting the degree of sharpness enhancement.
Laplacian is the original image data I₀Second derivative of (x, y) (Laplacian) ▽²I₀(x, y) is the original image data I₀A method of enhancing sharpness by subtracting from (x, y), and is expressed by the following equation.
I_S(x, y) = I₀(x, y) − ▽²I₀(x, y) (5)
As a specific example of sharpness enhancement by Laplacian, a 3 × 3 coefficient array such as the following formula (6) is often used.

[0035]
In this coefficient arrangement, an unnatural contour is likely to occur at the edge of the image when particularly strong sharpness enhancement is applied. Therefore, in order to reduce such a drawback, in the present invention, it is preferable to use an unsharp mask using a normal distribution type (Gaussian) blur function as shown in Expression (7).
G (x, y) = (1 / 2πσ²) exp [− (x²+ y²) / 2σ²] (7)
Where σ²Is a parameter representing the spread of the normal distribution function, and the edge x = x of the mask₁The ratio of the value at x to the value at the mask center x = 0,
G (x₁, 0) / G (0,0) = exp [−x₁ ²/ 2σ²] (8)
Is adjusted to 0.1 to 1.0, the desired sharpness of the 3 × 3 unsharp mask can be obtained. When the value of Expression (8) is set to a value close to 1.0, a mask substantially the same as the Laplacian filter at the center of Expression (5) can be made.
In order to change the sharpness of the mask, there is another method of changing the size of the mask. For example, by using a mask of 5 × 5, 7 × 7, 9 × 9, etc., the sharpness-enhanced image can be changed. The spatial frequency can be changed significantly.
[0036]
Also, as a function form of the mask, other than the above normal distribution type, for example, an exponential function type mask such as the following formula (9) can be used.
E (x, y) = exp [− (x²+ y²)^1/2/ A] (9)
Where a is σ in equation (8)²Is a parameter that represents the spread of the unsharp mask, as well as the ratio of the mask edge value to the mask center value,
E (x₁, 0) / E (0,0) = exp [−x₁/ A] (10)
Is adjusted to 0.1 to 1.0, the desired sharpness of the 3 × 3 unsharp mask can be obtained. In equation (10), E (x₁, 0) / E (0,0) = 0.3, an example of a numerical value of the mask of the exponential function of equation (9) is shown.
0.18 0.30 0.18
0.30 1.00 0.30 (11)
0.18 0.30 0.18
When an example of an unsharp mask is calculated from this mask, the following equation (12) is obtained.
−0.12 −0.22 −0.12
−0.21 2.32 −0.21 (12)
−0.12 −0.21 −0.12
[0037]
Using such an unsharp mask, the original image data I₀Sharpness weighted image data I from (x, y)_S(x, y) can be obtained. The unsharp mask and sharpness enhancement method used in the present invention are not limited to those described above, and other conventionally known unsharp masks and sharpness enhancement methods such as spatial frequency filtering can be applied. Of course.
FIG. 4A shows a region E where the edge component is dominant.₁And region E₂And original image data I having a noise region including a region A, a region B, and a region C having a noise motor₀An example of a one-dimensional profile waveform is shown. By applying sharpness enhancement to the profile waveform, as shown in FIG.₁And region E₂It can be seen that the edge signal is emphasized in a region having a strong edge component, and the noise component is also enhanced in the noise region including the regions A, B, and C.
[0038]
Next, the smoothing step (step 102) will be described.
Examples of the smoothing method include real space domain processing and spatial frequency domain processing. In real space region processing, the sum of all adjacent pixels is calculated and the average value is calculated and replaced with that value. The average value is calculated by multiplying each pixel by a weighting coefficient, for example, a normal distribution type function. There are various methods such as a method of performing such non-linear processing. On the other hand, in processing in the spatial frequency domain, there is a method of applying a low-pass filter. For example, in the averaging method using the weighting coefficient, the following equation (13) can be given. Here, (x, y) etc. represent the position coordinates of the target pixel in the image.
[Expression 1]

[0039]
Here, n is an averaging mask size, and w (x, y) is a weighting factor. When w (x, y) = 1.0, a simple average is obtained.
In the present invention, a method of obtaining an average value by multiplying a normal distribution type weighting coefficient in real space region processing is used, but the present invention is not limited to this. At this time, it is preferable to use a mask of n × n pixels as described below as a processing mask. Specifically, it is preferable to use those of about 3 × 3 to 5 × 5, 7 × 7, 9 × 9.

[0040]
An example of a 9 × 9 pixel mask is shown. In this equation (15), the center value is shown as a value normalized to 1.0, but in the actual processing, the sum of the entire mask is set to 1.0.
0.09 0.15 0.22 0.28 0.30 0.28 0.22 0.15 0.09
0.15 0.26 0.38 0.47 0.51 0.47 0.38 0.26 0.15
0.22 0.38 0.55 0.69 0.74 0.69 0.55 0.38 0.22
0.28 0.47 0.69 0.86 0.93 0.86 0.69 0.47 0.28
0.30 0.51 0.74 0.93 1.00 0.93 0.74 0.51 0.30 (15)
0.28 0.47 0.69 0.86 0.93 0.86 0.69 0.47 0.28
0.22 0.38 0.55 0.69 0.74 0.69 0.55 0.38 0.22
0.15 0.26 0.38 0.47 0.51 0.47 0.38 0.26 0.15
0.09 0.15 0.22 0.28 0.30 0.28 0.22 0.15 0.09
[0041]
Using such a mask, the original image data I₀Smoothed image data I from (x, y)_AV(X, y) can be obtained. It should be noted that the smoothing method used in the present invention is not limited to the various methods described above, and it goes without saying that any of the conventionally known smoothing methods can be applied.
FIG. 4C shows the result of the profile waveform shown in FIG. 4A processed by the smoothing process, and the noise component of the noise region including the region A, the region B, and the region C having the noise motor. Is suppressed and smoothed, and the region E where the edge component is dominant₁And region E₂Also, it can be seen that the edge signal is suppressed and the profile waveform is smooth.
[0042]
Next, the extraction process (step 104) of the edge / noise mixed component will be described.
Smooth image data I obtained in the smoothing step (step 102)_AV(x, y) is the sharpness-enhanced image data I obtained in the sharpness enhancement step (step 100) according to the following equation (16)_SMixed component ΔI which is fine structure data in which edges and noise are mixed by subtracting from sharpened edges_EGExtract (x, y).
ΔI_EG(x, y) = I₀(x, y) −I_AV(x, y) (16)
FIG. 4D shows a mixed component ΔI obtained by the edge / noise mixed component extraction process using the profile waveforms shown in FIGS. 4B and 4C._EGA profile waveform of (x, y) is extracted.
[0043]
Next, the edge detection process (step 106) will be described.
Here, edge detection by the local dispersion method is described as a representative example as an example, but the present invention is not limited to this.
[0044]
When performing edge detection, first, density conversion is performed by the following preprocessing. Such pre-processing is performed in order to improve the accuracy of edge detection performed thereafter by reducing uncorrelated noise in the R image data, G image data, and B image data constituting the color image data. . That is, as shown in the equation (17), the original image data I₀Density value D of three colors of R, G, B of (x, y)_R, D_G, D_BIs multiplied by weighting factors r, g, and b to create a visual density D_VConvert to
D_V= (RD_R+ GD_G+ BD_B) / (R + g + b) (17)
For example, a value such as r: g: b = 4: 5: 1 is used as the weighting coefficient. This conversion is performed in order to reduce uncorrelated noise in R, G, and B and improve the accuracy of edge detection. The range of the size of the preprocessing array is preferably about 5 × 5 or 7 × 7 pixels. This is because the variation in image density in the predetermined array described later is a small array in the array, For example, the calculation is performed while moving using an array of about 3 × 3.
[0045]
Note that the weighting factors r, g, and b in edge detection can be obtained as follows.
The weighting factor is conspicuous when visually observed (although this may correspond to a spectral luminous intensity distribution), that is, based on the idea that the weighting factor of image data with a large contribution is large. It is preferable to set an optimal value. In general, empirical weighting factors are obtained based on visual evaluation experiments and the like, and the following values are known as general knowledge (as well-known literature, Takashi Noguchi, “Good psychological response” Granular evaluation method), Journal of the Japan Photography Society, 57 (6), 415 (1994), which differs depending on the color, but shows a numerical value close to the following ratio).
r: g: b = 3: 6: 1
r: g: b = 4: 5: 1 (18)
r: g: b = 2: 7: 1
Here, if a preferable range of values is defined as the ratio r: g: b of the coefficient, when r + g + b = 10.0 and b is 1.0,
g = 5.0 to 7.0
A value in the range of is preferred. However, r = 10.0−b−g.
[0046]
Next, edge detection by local dispersion in the edge detection step (step 106) will be described.
Edge detection is performed using the visual density D described above._VN from the image data of_E× n_EWhile moving the pixel array, the local variance σ, which is the local standard deviation for each position, is calculated for each pixel of interest (x, y) sequentially using the equation (19). Thus, the subject edge in the image is detected. Pixel array size (n_E× n_E) May be appropriately determined in consideration of detection accuracy and calculation load, but it is preferable to use a size of about 3 × 3 or 5 × 5, for example.
[0047]
[Expression 2]

However, the target pixel position is x, y, and D_V(x + i-n_E/ 2 -1/2, y + j-n_E/ 2-1 / 2) is n to calculate the local variance σ (x, y)_E× n_E<D_V(x, y)> is the average concentration of the sequence,
[Equation 3]

It is.
[0048]
Original image data I_OFrom (x, y), the local variance σ (x, y) shown in the above equation (19) is calculated, and the edge intensity E of the subject image is calculated._OIn order to obtain (x, y), an expression represented by an exponential function such as the following expression (21) is used.
E_O(x, y) = 1−exp [−σ (x, y) / a_E] (21)
However, a_EIs a coefficient for converting the value of the local variance σ (x, y) into edge strength, and the edge strength E_O= Threshold of local variance σ (x, y) to be assigned to 0.5_TThen,
a_E= −σ_T/ Log_e(0.5) (22)
It is. σ_TThe value needs to be an appropriate value depending on the noise and the signal level of the subject outline. However, a value in the range of 10 to 100 is preferable for a color image of 8 bits (256 gradations). If this conversion is created as a lookup table, the calculation time required for the conversion can be shortened.
[0049]
Edge strength E_OThe conversion equation for obtaining (x, y) is not limited to the above equation, and other equations can also be used. For example, a Gaussian function such as the following equation may be used.
E_O(x, y) = 1−exp {− [σ (x, y)]²/ A_E1 ²} (23)
However, a_E1Is E from σ (x, y)_OCoefficient for conversion to (x, y), E_OThe threshold of local variance σ (x, y) assigned to (x, y) = 0.5 is σ_TThen,
a_E1 ²= −σ_T ²/ Log_e(0.5) (24)
It is. σ_TThe value of is preferably in the range of 10 to 100 in a color image of 8 bits (256 gradations) for each color.
[0050]
The edge strength data E_O(x, y) is the maximum local variance data σ as shown in the following equation (25)._MaxNormalized edge intensity data E normalized by 0 and 1 or less and 1 or less_O(x, y) may be obtained.
E₀(x, y) = σ (x, y) / σ_Max                      (25)
Where σ_maxIs a constant for normalizing σ (x, y) with the maximum value of the local variance data σ (x, y). σ_MaxIs determined from the local dispersion data σ (x, y) of the entire image obtained by the equation (19) as in the following equation (26).
σ_Max= Max {σ (x, y)} (26)
[0051]
Further, without using the maximum value obtained from the entire image, a part of the image, for example, a specific range of the central part of the image with a high probability of having an important subject of the image, or image data thinned out from the entire image (1/4 of the original image data). ˜1 / 10) to the maximum value σ using the above formulas (25) and (26)_MaxYou may ask for. In this case, the image data of a specific range in the central portion of the image or the thinned image data is subjected to various processing condition adjustment original image data (pressed) that can be obtained in advance with a coarse pixel density before image processing is performed to obtain processed image data. Can image data) or may be extracted from the original image data.
More preferably, σ_MaxIs the average value of the values included in the top 5-10% when the local variance data σ (x, y) are arranged in order from the largest value, for example, the average value of the values included in the top 10% <σ ( x, y)>_Max10%Σ_MaxΣ (x, y) is this σ_Max, All σ_MaxReplace with In this case, the average value may be an average value of the entire image, an average value of a predetermined range in a central portion where an important subject is often photographed, or an average value of thinned image data.
[0052]
By the way, the edge detection method in the present invention is not limited to the above-described local dispersion type edge detection method, and other edge detection methods can also be used. As edge detection methods other than the local dispersion method, there are methods based on primary differentiation and secondary differentiation, and there are several methods for each.
First, there are the following two operators as a method based on the spatial first derivative.
Examples of differential edge extraction operators include Prewitt operator, Sobel operator, and Roberts operator. The Roberts operator can be expressed as:
g (i, j) = {[f (i, j)-f (i + l, j + l)]²+ [F (i + l, j)-f (i, j + l)]²}^1/2
As template type operators, there are Robinson operators and Kirsh operators that use 3 × 3 templates corresponding to edge patterns in 8 directions.
Next, as a method based on a spatial second derivative, there is a method using Laplacian. In this case, since noise is emphasized, a method of performing edge distribution after first performing a normal distribution type blurring process is often used.
[0053]
FIG. 4E shows the original image data I shown in FIG.₀Edge strength data E obtained from the edge detection process₀The waveform is shown. Edge strength data E shown in FIG.₀In region E₁And E₂Edge strength data E near₀It can be seen that the value of increases.
[0054]
Next, the noise area weighting coefficient calculation step (step 108) will be described.
Edge strength data E obtained in the edge detection step (step 106)_O(x, y) is used to weight the edge region weighting factor W according to the following equation (27)._EAfter obtaining (x, y), the noise weighting factor W_G(x, y) is determined according to the following equation (28).
W_E(x, y) = 1-α_E+ Α_EE₀(x, y) (27)
W_G(x, y) = 1-W_E(x, y) (28)
[0055]
Where α_EIs a constant for setting the weighting of the edge area and the noise area, and an operator can set an arbitrary value between 0 and 1.
Edge area weighting factor W_E(x, y) is 1-α_EThe value is 1 or less and W_EAs (x, y) is larger, it is determined that the probability of being the edge region of the subject at that pixel position is higher, and W_EIt is determined that the smaller the (x, y), the higher the probability of being a noise region at that pixel position. Normalized edge strength data E₀In the edge region where the value of (x, y) is close to 1.0, α_EThe edge area weighting factor W regardless of the value of_E(x, y) is a large value close to 1.0, and the noise region weighting coefficient W_G(x, y) is the smallest. On the other hand, normalized edge strength data E₀As (x, y) becomes smaller than 1.0, the edge region weighting coefficient W_E(x, y) is the minimum value 1-α_EWeighting data W for the noise area_G(x, y) is the maximum value α_EGet closer to. Such α_EIs preset to a constant value as a default value, and the operator sees the processed image that has been image-processed with this default value._EYou may make it adjust the value of as needed.
Α_EIs 1, normalized edge strength data E_O(x, y) is the edge area weighting factor W_E(x, y) and 1.0-E_O(x, y) is the noise domain weighting factor W_G(x, y)
[0056]
In FIG. 4F, the weighting coefficient W of the noise region obtained from the edge intensity data shown in FIG._GThe waveform of (x, y) is shown. Region E₁And E₂The weighting factor W of the noise area near_GThe value of (x, y) is small. Region E₁And E₂Weighting factor W of the noise region in the vicinity of_GThe value of (x, y) is a constant α that sets the weighting of the edge region and the noise region set by the above equation (27)._EIt depends on.
[0057]
Next, the noise data identification / separation process (step 110) will be described. The mixed component ΔI in which the sharpness-enhanced edge component and the noise component mixed in the extraction step of the edge / noise mixed component (step 104) are mixed._EGIn (x, y), as shown by the following equation (29), the weighting factor W of the noise region obtained in the noise weighting factor calculation step (step 108)._GMultiply (x, y) to obtain noise data G₀Find (x, y).
G₀(x, y) = ΔI_EG(x, y) × W_G(x, y) (29)
FIG. 4G shows the mixed component ΔI shown in FIG._EG(x, y) and the noise region weighting coefficient W shown in FIG._GNoise data G obtained by multiplying (x, y)₀Region E where the profile waveform of (x, y) is shown and the edge component is dominant₁And region E₂It can be seen that the signal component of the edge of the signal is removed.
[0058]
Next, the noise motor suppression distribution calculation step (step 112) will be described. First, a noise suppression distribution function s (r) representing the spread of noise suppression in the pixels in the noise area of the original image is set as in Expression (1). Here, r is the distance from the origin P of the pixel of interest located at the coordinates (x, y) with the point P as the center position (origin) of the noise suppression spread.
s (r) = exp (−r / a) (1)
That is, s (x, y) = exp (− (x²+ Y²)^(1/2)/ A).
[0059]
Here, a in the above equation (1) is a suppression range constant for adjusting the range of noise suppression spread in the pixels of the original image, and is given in advance or set by input. When the suppression range constant a is set by input, the position of the boundary of the range that exerts noise suppression, that is, the suppression distance r_sAnd the value s at this position_minIt is set by determining. That is, the suppression distance r_sValue at_minFrom the following equation (30), the suppression range constant a is calculated and obtained.
a = −r_s/ Log (s_min(30)
[0060]
Where s_minIs preferably 0.1 or more and 0.5 or less, and the suppression distance r is_sIs preferably in the range of 1 to 15. In particular, when the original image data is obtained by reading an image recorded on a film with a scanner or the like, the suppression distance r_sThe value of is set by the film format (image size of one frame of film), the roughness of the film grain (granular motor), which varies depending on the type and sensitivity of the film, and the scanning of the scanner that scans the film to obtain digital image data. It is preferable to change appropriately according to the aperture size. That is, in the case of coarse granularity (the granular mottle is also large), the suppression distance r_sIn the case of fine particles, the suppression distance r is increased._sReduce the value of. In addition, even if the same film is used, the scanning aperture size of the scanner is changed to 15 μm □ and 10 μm □ when the scanning aperture size of the scanner is 10 μm □. This is because the pixel density of the image of 1.5 μm is 10 times larger and the pixel density of the 10 μm square is higher, and the graininess changes.
[0061]
Moreover, this suppression distance r_sThe noise data G obtained in step 110 when the mask size for averaging in the smoothing step (step 102) is increased.₀Since the noise motor component included in is greatly blurred and spreads, the suppression distance r is suppressed in order to suppress the noise motor component._sNeed to be larger. Therefore, the suppression distance r depends on the mask size for averaging in the smoothing step (step 102)._sMay be changed as appropriate.
The mask size for averaging in the smoothing step is about 3 pixels × 3 pixels as the minimum mask size and about 11 pixels × 11 pixels as the maximum mask size, but the suppression range constant a is 1 from this mask size. For example, when the mask size is 3 × 3, the suppression distance r is about 5 to 2 times larger._sThe value of is preferably about 5.
[0062]
A function like Formula (2) may be set as such a noise suppression distribution function s (r).
s (r) = exp (−r²/ A²(2)
Here, r is the distance from the center position of the noise suppression spread.
In this case, the suppression range constant a is the suppression distance r._sAnd the value s at that position_minUsing the following equation (31).
a = r_s/ [-Log (S_min]]^(1/2)              (31)
Or you may set a function like following formula (3) as noise suppression distribution function s (r).
s (r) = rect (r / a) (3)
Here, rect (r / a) is a rectangular function having a value of 1, and r is a distance from the center position of the noise suppression spread.
In this case, the suppression range constant a is the suppression distance r to be set._sAnd Further, in the equations (2) and (3), s_minBy setting = 1, the same noise suppression distribution function s (r) as in the expression (1) can be obtained.
[0063]
Such a noise suppression distribution function s (r) is a noise suppression distribution function s of 0 to 1 in accordance with the following equation (32).₁Normalized to (x, y).
s₁(x, y) = s (x, y) / S (32)
Here, S is an integral value obtained by the following equation (33).
[Expression 4]

FIG. 4 (h) shows a noise suppression distribution function s obtained by normalizing the noise suppression distribution function s (x, y) obtained using the equation (1) according to the equation (33).₁An example of (x, y) is shown.
[0064]
Next, this normalized noise suppression distribution function s₁(x, y) and noise data G₀Using (x, y), convolution integration is performed in a range where noise suppression is effected as in the following equation (34) to calculate a noise motor suppression distribution M (x, y).
[Equation 5]

The noise motor suppression distribution M (x, y) calculated here is for determining the degree of suppression received from other pixels of noise at a certain target pixel position (x, y) by convolution integration.₀Corresponds to the diffusion distribution of the development inhibiting substance generated during the development of the silver halide grains in the film photosensitive layer when the image data is read from the film image by a scanner or the like, and the noise data G₀Is regarded as the particle distribution of the photosensitive material, and the original image data I₀The noise motor suppression distribution M (x, y) corresponding to the diffusion distribution of the development suppressing substance generated around the silver particles of the photosensitive material is obtained.
[0065]
In the example shown in FIG. 4, the noise data G shown in FIG.₀And the noise suppression distribution function s shown in FIG.₁By using (x, y) and calculating according to the equation (34), a noise motor suppression distribution M (x, y) as shown in FIG. 4I is obtained. In FIG. 4 (i), the value of the noise motor suppression distribution becomes large in the portions indicated by region A, region B, and region C in the drawing, and in the regions A, B, and C, the silver salt photosensitive material In this case, it means that the concentration of the development inhibiting substance produced by the silver salt photosensitive material is high.
In this way, the noise motor suppression distribution M (x, y) is calculated.
[0066]
Next, the noise motor suppression component calculation step (step 114) will be described.
In the noise motor suppression component calculation step, the noise data G calculated in step 110 is calculated by calculating the noise motor suppression distribution M (x, y) calculated in step 112 by the following equation (35).₀Is multiplied by the noise suppression component image data G₁Calculate (x, y).
G₁(x, y) = G₀(x, y) x M (x, y) (35)
[0067]
Noise motor suppression distribution M (x, y)₀In the case of a silver salt light-sensitive material, the multiplication is performed in consideration of the suppression in accordance with the action of the development inhibitor and silver particles produced in the silver salt light-sensitive material. In the example shown in FIG. 4, the noise suppression component image data G is shown in FIG.₁(x, y) is shown. Noise suppression component image data G shown in FIG.₁(x, y) is the noise data G shown in FIG.₀As compared with the region A, the region B, and the region C, other regions, for example, the region E₁And region E₂It can be seen that the value is relatively large.
[0068]
Finally, the noise suppression image calculation step (step 116) will be described.
In the noise suppression image calculation process, the noise suppression component image data G obtained in step 114 is displayed.₁(x, y) is scaled using the noise suppression coefficient α as shown by the following formula (36), and then the sharpness weighted image data I obtained in step 100 is obtained._SProcessed image data I subtracted from₁'
I₁'(x, y) = I₀(x, y) − α × G₁(x, y) (36)
Here, the noise suppression coefficient α is a settable parameter for controlling the degree of noise suppression, and is appropriately set and input. Alternatively, a default setting value may be provided in advance and changed as necessary.
Thereafter, image data conversion suitable for the image output device 16 is performed to process the image data I₁'To processed image data I₁Get.
[0069]
In the example shown in FIG. 4, the processed image data I as shown in FIG.₁ ^'It can be seen that noise is largely removed in the area of the noise motor such as the area A, the area B, and the area C, and noise is appropriately removed in other noise areas. Moreover, the original image data I shown in FIG.₀Compared to region E₁And region E₂The edge region is emphasized with sharpness.
[0070]
The above image processing method is used to suppress granularity when the granular component affects a certain pixel area, such as a granular component in a photographic photosensitive material, that is, when the granular component affects the granular component of another pixel. However, the above image processing method is also useful for image data of other pixels even in a digital camera or the like using a CCD or MOS image sensor that is not affected by noise components of other pixels. Since it is used to perform defect correction and the like, it can be effectively applied. The CCD image pickup device is also applied to a honeycomb array described in JP-A-10-136391.
[0071]
As an image processing method for suppressing noise, Japanese Patent Application Laid-Open No. 11-250246 proposes a method for suppressing noise by multiplying a noise component by a compression coefficient obtained from the edge strength of an image. Since a certain compression coefficient is applied, the noise amplitude is proportionally compressed. In addition, by adopting a compression coefficient that increases the compression ratio as the noise amplitude increases, that is, by applying non-linear transformation to the magnitude (amplitude) of the noise fluctuation, the noise fluctuation is described in the above publication. The direction can be more uniform than the image processing method. However, unlike the two methods described above, the image processing method of the present invention is different from performing suppression processing on the amplitude of noise, unlike a noise motor, a noise region in which noise component fluctuations are spatially formed, For example, it is a process of performing noise suppression on the region A, the region B, and the region C shown in FIG. 4A, and can be more greatly suppressed at a place where the noise fluctuation is large, and the noise fluctuation can be made uniform. .
[0072]
The image processing method of the present invention is described as above.
Such an image processing method may be configured as the above-described image processing apparatus including a circuit or hardware, or may be a program that performs functions in a computer as software. A computer-readable recording medium recording a program for executing the above method may be provided as a CD = ROM or the like.
[0073]
When the image processing method of the present invention is applied to a silver salt color photograph, for example, a photographic image taken on a 35 mm color negative film, it is possible to obtain a remarkable improvement effect that can be seen at a glance both in graininess and sharpness. It was.
In particular, it has a processing effect comparable to the granular improvement by making the photosensitive material finer with respect to the granularity, so that the “blurred granular” unnaturalness that was a drawback of various granular removal processing methods based on the conventional averaging and reduction of fluctuations The feeling of discomfort disappeared. In addition, with regard to sharpness, by combining with the above-described granular suppression, a significant enhancement effect was obtained compared to conventional unsharpness masks and Laplacian filters without deteriorating the granularity.
In addition, when applied to an image taken with a digital still camera, a remarkable image quality improvement effect was obtained as in the case of a color image by the silver salt color photograph.
[0074]
The image processing method, the image processing apparatus, and the recording medium of the present invention have been described in detail above. However, the present invention is not limited to the above embodiments, and various improvements and modifications can be made without departing from the gist of the present invention. Of course you can go.
[0075]
【The invention's effect】
As described above in detail, according to the present invention, the original image is first sharpened to sharpen both the subject image and noise, and the subject contour and noise components are extracted from the image, and the noise is extracted. Since the components are selectively removed, the noise is blurred and there is no large unevenness that appears visually unpleasant, and the low-contrast image signal is not mistaken for noise. It also does not form unnatural artifacts. Furthermore, since the noise component is suppressed by extracting and selectively removing a spatially rough area where the density fluctuation of the noise component is large, the processed image rarely includes a large and rough noise component such as a noise motor. (The density fluctuation is small). Therefore, the density fluctuation can be made uniform, the noise can be made fine in space, and the natural noise suppression can be realized visually.
[0076]
In addition, according to the present invention, when noise is granular, the granularity is sharpened and granular components such as a granular mottle with a large spatial fluctuation are removed and suppressed, and the granular pattern is made finer and uniform. Therefore, the silver halide photographic light-sensitive material has a fine granularity as obtained when using a fine-grain emulsion, and has no natural discomfort and unpleasant feeling like the blurred granularity, which is a disadvantage of the conventional method using smoothing. Grain suppression effect is obtained. In addition, by applying the image processing method of the present invention to a silver salt color photographic light-sensitive material, there is no so-called “blurred grain” unnaturalness and uncomfortable feeling, which is a disadvantage of the conventional graininess suppression processing method, and the graininess is improved. Therefore, an extremely remarkable improvement effect can be obtained, and a large industrial effect can be obtained.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an embodiment of a system that incorporates an image processing apparatus according to the present invention, reads a color photographic image, performs image processing for noise suppression and sharpness enhancement, and outputs a color image with an output device. is there.
FIG. 2 is a block diagram illustrating an embodiment of an image processing unit that performs image processing for noise suppression and sharpness enhancement in the image processing apparatus according to the present invention.
FIG. 3 is a block diagram showing an embodiment of an image processing method of the present invention.
4A to 4K are diagrams illustrating an example in which image data is processed by the image processing method of the present invention. FIG.
[Explanation of symbols]
10 Color image playback system
12 Image input device
14 Image processing device
16 Image output device
18 color / tone processing section
20 Noise suppression image processing unit
22 Image Monitor / Image Processing Parameter Setting Unit
26 Smoothing processor
28 Edge / Noise Mixed Component Extraction Unit
30 Edge detector
32 Noise area weighting factor calculator
34 Noise component identification and separation unit
36 Noise suppression distribution function setting section
38 Noise motor suppression distribution calculator
40 Noise suppression component calculation unit
42 Noise suppression calculation processing unit

Claims (8)

Sharpness enhancement processing is performed on the original image data, and sharpness-enhanced image data that sharpens the noise contained in the image is created together with the image.
Smoothing the original image data to create smoothed image data;
Subtracting the smoothed image data from the sharpness-enhanced image data to create mixed image data in which the edge of the sharpened image and subject noise are mixed,
Obtaining edge strength data for performing edge detection from the original image data to identify a subject edge region and a noise region;
From this edge strength data, the noise area weighting coefficient indicating the degree of the noise area is obtained,
Multiplying the mixed image data by the weighting coefficient of the noise region to obtain noise data,
Set a noise suppression distribution function representing the spread of noise suppression in the original image data, perform a convolution integration of the noise data and this noise suppression distribution function to obtain a noise suppression distribution,
By multiplying the noise data by the noise suppression distribution, noise suppression component image data is calculated,
Noise suppression of a digital image, wherein the noise suppression component image data is scaled and subtracted from the sharpness-enhanced image data to create an image that selectively suppresses noise components in the noise region of the original image Image processing method for. 2. The image processing according to claim 1, wherein the noise suppression distribution function is a monotonously decreasing function that has a maximum value at a center position of a noise suppression spread in a pixel of an original image and decreases as the distance from the center position increases. Method. The image processing method according to claim 1, wherein the noise suppression distribution function is a rectangular function having a constant value in a range in which noise suppression is performed in pixels of an original image. 4. The image processing method according to claim 2, wherein a range in which the noise suppression distribution function exerts the noise suppression in pixels of the original image is an area having a number of pixels in a range of 1 to 15. 5. The image processing method according to claim 2, wherein the noise suppression distribution function is set by determining a position of a boundary of a range in which noise suppression is performed in pixels of the original image and a value at this position. When r is a distance from the center position of the noise suppression spread in the pixels of the original image and a is a suppression range constant that determines the range of the noise suppression spread in the pixels of the original image, The image processing method according to claim 5, which is a function s (r) represented by any one of 1) to (3).
s (r) = exp (−r / a) (1)
s (r) = exp (−r ² / a ² ) (2)
s (r) = rect (r / a) (3)
Here, rect (r / a) in equation (3) is a rectangular function having a value of 1. A sharpness enhancement processing unit that performs sharpness enhancement processing on the original image data and creates sharpness enhancement image data in which noise included in the image is sharpened together with the image;
Performing a smoothing process on the original image data to create a smoothed image data; and
Edge / A noise mixed component extraction unit;
An edge detection unit for performing edge detection from the original image data to obtain edge intensity data for identifying a subject edge region and a noise region;
A noise region weighting factor calculation unit for obtaining a weighting factor of a noise region indicating the degree of the noise region from the edge intensity data obtained by the edge detection unit;
A noise component identification / separation unit that obtains noise data by multiplying the mixed image data created by the edge / noise mixed component extraction unit by a weighting coefficient of the noise region obtained by the noise region weighting factor calculation unit;
A noise suppression distribution function setting unit for setting a noise suppression distribution function representing a spread of noise suppression in the original image data;
A noise suppression distribution calculation unit that obtains a noise suppression distribution by performing convolution integration of the noise data determined by the noise component identification separation unit and the noise suppression distribution function set by the noise suppression distribution function setting unit;
A noise suppression component calculation unit that calculates noise suppression component image data by multiplying the noise suppression distribution determined in the noise suppression distribution calculation unit by the noise data calculated by the noise component identification separation unit;
A noise suppression calculation processing unit that obtains processed image data by scaling and subtracting the noise suppression component image data calculated by the noise suppression component calculation unit from the sharpness-enhanced image data. An image processing device for noise suppression. A procedure for performing sharpness enhancement processing on the original image data and creating sharpness enhancement image data in which noise included in the image is sharpened together with the image,
A procedure for performing smoothing processing on the original image data to create smoothed image data;
Subtracting the smoothed image data from the sharpness-enhanced image data to create mixed image data in which the edge of the sharpened image and the noise of the subject image are enhanced.
A procedure for performing edge detection from the original image data to obtain edge intensity data for identifying a subject edge region and a noise region;
A procedure for obtaining a weighting coefficient of a noise region indicating a degree of the noise region from the edge strength data,
A procedure for obtaining noise data by multiplying the mixed image data by a weighting coefficient of the noise region;
Setting a noise suppression distribution function representing the spread of noise suppression in the original image data, obtaining a noise suppression distribution by performing convolution integration of the noise data and the noise suppression distribution function;
Calculating noise suppression component image data by multiplying the noise data by the noise suppression distribution;
A method for causing a computer to execute a procedure for scaling and subtracting the noise suppression component image data from the sharpness-enhanced image data to create an image in which a noise component in a noise region of an original image is selectively suppressed. A computer-readable recording medium on which a program to be recorded is recorded.

Images