JP3804880B2 - Image processing method for noise suppression and sharpness enhancement of digital images - Google Patents

Description

【００２４】
この係数配列では、特に強いシャープネス強調を掛けたときに、画像のエッジに不自然な輪郭が発生し易い。そこで、そのような欠点を少なくするために、本発明では式（９）に示したような正規分布型(Gaussian)のぼけ関数を用いたアンシャープマスクを用いるのが好ましい。
G(x,y)＝(1/2πσ²)exp[−(x² + y²)/２σ²] (９）
ここで、σ² は正規分布関数の広がりを表すパラメータであり、マスクの端ｘ＝ｘ₁ における値とマスクの中心ｘ＝０における値の比、
G(x₁,0)/G(0,0)＝exp[−x₁ ²/２σ²] （１０）
が０．１〜１．０となるように調節することによって、３×３のアンシャープマスクのシャープさを所望のものとすることができる。式（１０）の値を１．０に近い値にすると、式（８）の中央のラプラシアンフィルタとほぼ同じマスクを作ることができる。
マスクのシャープさを変更するには、この他にマスクの大きさを変更する方法があり、たとえば５×５、７×７、９×９等のマスクを用いることによって、シャープネス強調の空間周波数域の大幅な変更が可能となる。
【００２５】
また、マスクの関数形としても、上記の正規分布型以外のもの、たとえば、下記式（１１）のような指数関数型のマスクを用いることができる。
E(x,y)＝exp[−(x² + y²)^1/2／ａ] （１１）
ここで、ａは式（９）のσ² と同様にアンシャープマスクの広がりを表すパラメータであり、マスクの端の値とマスクの中心値の比、
E(x₁,0)/E(0,0)＝exp[−x₁／ａ] （１２）
が０．１〜１．０となるように調節することによって、３×３のアンシャープマスクのシャープさを所望のものとすることができる。式（１３）に、E(x₁,0)/E(0,0)＝０.3としたときの式（１１）の指数関数のマスクの数値例を示す。
0.18 0.30 0.18
0.30 1.00 0.30 （１３）
0.18 0.30 0.18
このマスクから、アンシャープマスクの１例を計算すると、次式（１４）のようになる。
−0.12 −0.22 −0.12
−0.21 2.32 −0.21 （１４）
−0.12 −0.21 −0.12
【００２６】
このようなアンシャープマスクを用いて、原画像Ｉ₀(x,y)からシャープネス強調画像Ｉ_S (x,y）を求めることができる。なお、本発明に用いられるアンシャープマスクおよびシャープネス強調方法は、上述したものに限定されるわけではなく、この他の従来公知のアンシャープマスクや空間周波数フィルタリング等によるシャープネス強調方法を適用可能なことはもちろんである。
【００２７】
２）次に平滑化工程について説明する。
平滑化を行う方法としては、実空間領域の処理と空間周波数領域の処理を挙げることができる。実空間領域処理では、隣接する画素全体の和を求め平均値を計算してその値に置き換える方法、各画素に重み係数、たとえば正規分布型の関数を掛けて平均値を求める方法、メディアンフィルタのような非線型な処理を行う方法等の種々の方法がある。一方、空間周波数領域の処理では、ローパスフィルタを掛ける方法がある。たとえば、重み係数を用いる平均化の方法では下記式（１５）を挙げることができる。
【００２８】
【数１】

【００２９】
ただし、ｎは平均化のマスクサイズ、ｗは重み係数である。ｗ＝１．０とすると、単純平均となる。
本発明では、実空間領域処理の中で、正規分布型の重み係数を掛けて平均値を求める方法を用いることにするが、これに限定されない。この時、処理のマスクとしては、下記のようなｎ×ｎ画素のマスクを用いるのが好ましい。具体的には３×３から５×５、７×７、９×９程度のものを用いるのが好ましい。

【００３０】
式（１７）に９×９画素のマスクの一例を示す。この式（１７）では中心の値を１．０に正規化した値で示しているが、実際の処理ではマスク全体の和が１．０になるようにする。
0.09 0.15 0.22 0.28 0.30 0.28 0.22 0.15 0.09
0.15 0.26 0.38 0.47 0.51 0.47 0.38 0.26 0.15
0.22 0.38 0.55 0.69 0.74 0.69 0.55 0.38 0.22
0.28 0.47 0.69 0.86 0.93 0.86 0.69 0.47 0.28
0.30 0.51 0.74 0.93 1.00 0.93 0.74 0.51 0.30 （１７）
0.28 0.47 0.69 0.86 0.93 0.86 0.69 0.47 0.28
0.22 0.38 0.55 0.69 0.74 0.69 0.55 0.38 0.22
0.15 0.26 0.38 0.47 0.51 0.47 0.38 0.26 0.15
0.09 0.15 0.22 0.28 0.30 0.28 0.22 0.15 0.09
【００３１】
このようなマスクを用いて、原画像Ｉ₀(x,y)から平滑化画像Ｉ_AV（x,y)を求めることができる。なお、本発明に用いられる平滑化方法としては、上述した種々の方法に限定されるわけではなく、従来公知の平滑化方法はいずれも適用可能なことはいうまでもない。
【００３２】
３）次いで、粒状とエッジの混在成分の抽出工程について説明する。
こうして得られたシャープネス強調画像Ｉ_S (x,y）と平滑画像Ｉ_AV(x,y) から、両者の差を計算し粒状とエッジとの混在成分ΔＩ_EG(x,y) として抽出する。
ΔＩ_EG(x,y）＝Ｉ_S (x,y) −Ｉ_AV(x,y) （１８）
【００３３】
４）エッジ検出およびエッジ領域と粒状領域の重み付け係数の算出工程について説明する。ここでは、一例として局所分散方式によるエッジ検出を代表例として説明するが、本発明はこれに限定されるわけではない。
【００３４】
▲１▼前処理：濃度変換、
エッジ検出を行う際に先ず、式（１９）に示したように、原画像Ｉ₀(x,y)のＲ，Ｇ，Ｂの３色の濃度値Ｄ_R ，Ｄ_G ，Ｄ_B に重み係数ｒ，ｇ，ｂを掛けて視覚濃度(Visual density)Ｄ_V に変換する。
Ｄ_V ＝（ｒＤ_R ＋ｇＤ_G ＋ｂＤ_B ）／（ｒ＋ｇ＋ｂ) （１９）
重み係数としては、例えば、ｒ：ｇ：ｂ＝４：５：１のような値を用いる。この変換を行うのは、Ｒ，Ｇ，Ｂで相関の無い粒状やノイズを減少させ、エッジ検出の精度を向上させるためである。前処理の配列の大きさは範囲は５×５、あるいは７×７画素程度のものを用いるのがよいが、それは、次の処理▲２▼で配列内の画像濃度の変動を、配列内で小さい配列、例えば、図２に示すように、３×３程度の配列を用いて、移動しながら計算するためである。
【００３５】
なお、エッジ検出における重み係数ｒ，ｇ，ｂは以下のようにして求めることができる。
重み係数については、視覚で観察したときに目立つ（これは、分光的な視感度分布に対応するという見方もあるが）、すなわち寄与の大きい色の画像データの重み係数が大きいという考えに基づいて最適な値に設定するのが好ましい。一般には、視覚評価実験等に基づいて経験的な重み係数が求められており、下記のような値が一般的な知見として知られている（公知文献としては、野口高史、「心理対応の良い粒状評価法）、日本写真学会誌，５７（６），４１５（１９９４）があり、色によって異なるが、下記の比に近い数値が示されている）。
ｒ：ｇ：ｂ＝３：６：１
ｒ：ｇ：ｂ＝４：５：１
ｒ：ｇ：ｂ＝２：７：１
ここで、係数の比ｒ：ｇ：ｂとして好ましい値の範囲を規定するとすれば、r ＋ｇ＋ｂ＝１０．０でｂを１．０としたときに、ｇの値として、
ｇ＝５．０〜７．０
の範囲の値が好ましい。ただし、ｒ＝１０．０−ｂ−ｇである。
【００３６】
▲２▼局所分散によるエッジ検出、
エッジの検出は、図２に示すように、上記視覚濃度Ｄ_V の画像データからｎ_E ×ｎ_E 画素の配列を移動しつつ、配列内の画像濃度変動を式（２０）を用いて、その位置毎の局所的な標準偏差σを順次局所分散として計算することによって、画像中の被写体エッジの検出を行う。画素配列の大きさ（ｎ_E ×ｎ_E ）は、検出精度および計算負荷を考慮して適宜決めればよいが、例えば３×３、あるいは５×５程度の大きさを用いるのが好ましい。
【００３７】
【数２】

ただし、Ｄ_ijは局所分散を計算するｎ_E ×ｎ_E の画素配列の濃度で、＜Ｄ＞はその配列の平均濃度で、
【数３】

である。
【００３８】
▲３▼エッジ領域と粒状領域の重み付け係数の算出、
上記式（２０）および（２１）に示す局所分散σ(x,y) から、エッジ領域の重み付けＷ_E (x,y) と粒状領域の重み付けＷ_G (x,y) を求めるために下記のような指数関数を用いた式（２２）および（２３）を用いた。
Ｗ_E (x,y) ＝１− exp [−σ(x,y）／ａ_E ] （２２）
Ｗ_G (x,y) ＝１−Ｗ_E (x,y） （２３）
ただし、ａ_E は局所分散σ(x,y）の値を重み付けＷ_E (x,y) に変換する際の係数で、Ｗ_E ＝０．５に割り付けるσ(x,y）の閾値σ_T とすると、
ａ_E ＝−σ_T ／ log_e (0.5)
である。σ_T の値は、粒状と被写体輪郭の信号の大きさによって適切な値にする必要があるが、各色８ｂｉｔ（２５６階調）のカラー画像では１０〜１００の範囲の値が好ましい。この変換は、LUT(Look up table)として作成しておくと、変換に要する計算時間を短縮することができる。
【００３９】
エッジ領域の重み付けＷ_E (x,y) を求める変換式としては、上記式に限定されるものではなく、他の式を用いることもできる。たとえば、下記のようなガウシャン型の関数を用いても良い。
Ｗ_E (x,y) ＝１− exp｛− [σ(x,y)]² ／ａ_E1 ² ｝
ただし、ａ_E1はσ(x,y) からＷ_E (x,y) に変換する際の係数で、Ｗ_E ＝０．５に割り付けるσ(x,y) の閾値をσ_T とすると、
ａ_E1 ² ＝−σ_T ² ／ log_e (0.5)
である。σ_T の値は、各色８ｂｉｔ（２５６階調）のカラー画像では１０〜１００の範囲の値が好ましい。
【００４０】
ところで、本発明においてエッジ検出法としては、上記局所分散方式のエッジ検出法に限定されるわけではなく、他のエッジ検出法も利用可能である。上記局所分散方式以外のエッジ検出法には、一次微分や二次微分に基づく方法があり、それぞれに、更に幾つかの方法がある。
まず、空間的な一次微分に基づく方法としては、下記の２つのオペレータがある。
差分型エッジ抽出オペレータとして、 Prewittのオペレータ、 Sobelのオペレータ、 Robertsのオペレータなどがある。 Robertsのオペレータは下記式で表わすことができる。
g(i,j)＝｛[f(i,j) - f(i+l,j+l)]²＋[f(i+l,j) - f(i,j+l)]²｝^1/2
テンプレート型オペレータとして、８方向のエッジパターンに相当する３×３テンプレートを用いる Robinson のオペレータや Kirshのオペレータがある。
次に、空間的な二次微分に基づく方法としては、ラプラシアンを用いた方法がある。この場合、雑音を強調してしまうので、先ず正規分布型のぼかし処理をしてからエッジ検出する方法が良く用いられる。
【００４１】
５）次に、エッジと粒状の識別・分離工程における粒状とエッジの重み付けについて説明する。
ここでは、粒状成分とエッジ成分を識別し、分離するには、粒状とエッジの特徴を利用する。先ず、空間的な領域では、粒状はフィルム全体すなわち画像全体にあるが、被写体の輪郭やエッジの部分よりも平坦な部分で目立つ。一方、エッジは画像中の主として被写体の輪郭部分と被写体表面の微細構造のある部分にある。また、濃度領域では、図３に示すように、粒状は主として撮影に用いた写真感光材料の粒状で構成されているので、図中点線で示されるように、濃度差は小さいものが多いが、エッジは被写体のコントラストに依存しており、画像によって大きく異なるが図中実線で示されるように、濃度差は微小なものから非常に大きいものまで変化に富んだものとなっている。
【００４２】
６）次に、粒状とエッジの識別・分離工程について説明する。
粒状とエッジを識別・分離するために、先ず両者の空間的な特徴を利用して、粒状とエッジの領域分割を行う。原画像から検出した被写体のエッジを用いて求めた粒状領域の重み付けＷ_G (x,y) を、粒状とエッジの混在濃度値ΔＩ_EG(x,y) に掛けることによって、エッジ情報を減少させ、粒状情報の比率の高い信号ΔＩ_G (x,y) にすることができる。
ΔＩ_G (x,y) ＝Ｗ_G (x,y) ×ΔＩ_EG(x,y) （２４）
【００４３】
次に、濃度領域での特徴を利用して、粒状とエッジの画像情報の分離を行う。図３に示したように、濃度差の小さい信号は主として粒状成分で、エッジ信号も幾分混在し、濃度差の大きい信号は主としてエッジ成分で、濃度差の大きめの粒状成分が混在しているので、濃度差の大小を用いて粒状とエッジの分離を行うことができる。粒状成分Ｇ(x,y) の分離は、下記の式（２５）で表す非線形変換のＬＵＴを用いて行う。粒状成分の分離に用いられる非線形変換ＬＵＴの一例を図４に示す。
Ｇ(x,y) ＝ＬＵＴ (ΔＤ(x,y)) （２５）
ただし、ＬＵＴは
ＬＵＴ (ΔＤ）＝ΔＤ× exp [−（ΔＤ)²／ａ_G ²] （２６）
で、ａ_G ² は粒状の濃度変動の閾値Ｇ_T から決まる定数で、
ａ_G ² ＝−Ｇ_T ²/ log_e (1/2) （２７）
である。
【００４４】
ここで、粒状の濃度変動の閾値Ｇ_T は、粒状とエッジの混在濃度変動ΔＩ_EG(x,y) の中で、この値以下の濃度変動は粒状であると見做すものであるが、式（２６）と図４から容易に判るように、この閾値を境にｏｎ／ｏｆｆ的に分離するのではなく、濃度変動が大きくなるにつれて徐々に小さくなるＬＵＴ形状に従って、分離する粒状が減少していくようにしている。従って、粒状と共にエッジも混入するが、その割合も徐々に減少する。
このような非線型変換ＬＵＴを非線型変換関数ＮＬＧとして表わし、エッジ・粒状混在成分をΔＩ_EG（x,y)で表わすと、粒状成分Ｇ₀(x,y)は、上記式（２４）および（２５）より、
Ｇ₀(x,y)＝ＮＬＧ｛ΔＩ_EG(x,y) ×Ｗ_G (x,y) ｝ （２８）
として表わすことができる。こうして粒状成分Ｇ₀(x,y)を求めることができる。
なお、本工程で行なうエッジ成分の算出は後述する。
【００４５】
ところで、粒状の識別閾値Ｇ_T の値は、処理する画像の粒状やノイズの大きさとシャープネス強調処理の程度によって、最適な値を選択するのが好ましい。粒状の識別は、シャープネス強調処理を行った画像で行うので、その粒状は元の画像の粒状がシャープネス強調処理でシャープになり、かつ濃度変動が大きくなった粒状である。したがって、粒状抑制処理を行う際に、周辺のｎ×ｎ画素の濃度変動を参照して、シャープネス強調処理後の粒状の大きさをＲＭＳ粒状度σ等の物理値で表し、それに基づいて粒状の識別閾値Ｇ_T を決めることになる。以下に、その決定方法について説明する。
【００４６】
カラー写真感光材料の粒状は、通常、マイクロデンシトメータを用いて、４８μφの測定開口を用いてＲＭＳ粒状度で測定されており、一般的あるいは典型的なカラーネガフィルム、例えばSuper G ACE 100, 200, 400, 800（いずれも富士写真フイルム社製）などでは４〜５の値（ＲＭＳ粒状度σ₄₈を１０００倍した値で表示したもの）となっている。このフィルムを開口面積Ａでスキャニングすることによってディジタル化すると、その開口面積でのフィルムの粒状度σ_scは、良く知られたSelwynの粒状度の式Ｓ＝σ√Ａを用いて、上記４８μφの開口で測定したＲＭＳ粒状度σ₄₈から次式（２９）で換算することができる。
σ_sc＝σ₄₈√Ａ₄₈／√Ａ_sc （２９）
ここで、Ａ₄₈は４８μφの開口の面積である。たとえば、フィルムの粒状度が４で、ディジタル化のスキャニング開口を１２μφ（開口面積はＡ₁₂）とすると、
σ_sc＝σ₄₈√Ａ₄₈／√Ａ₁₂＝０．０１６ （３０）
となる。ただし、いずれの場合も光学系とスキャニング開口によるぼけは同じとする。
【００４７】
シャープネス強調を行ったときに粒状度σ_scがｐ倍に大きくなったとすると、
σ_sc’＝ｐσ_sc （３１）
となる。
粒状の識別閾値Ｇ_T の値は、処理すべき画像の粒状度に比例する値、すなわちＧ_T ＝ｋ_G σ_sc’が好ましい。ただし、ｋ_G は定数で、１．０〜３．０の値が好ましい。Ｇ_T の値をσより大きくすればするほど粒状はより完全に識別できるようになる反面、粒状の濃度変動に近い低コントラストの被写体情報が粒状として誤認される確率が高まる。逆にσより小さくすると被写体情報は誤認されにくくなるが、粒状の中で濃度変動の大きい粒状が捕らえられなくなってしまい、画像の中に粗い粒状が残ることになる。
【００４８】
７）次に、粒状モトルの細分化工程について説明する。
粒状パターンは画像中で細かい濃度変動から成っているが、その濃度変動は濃度値の振幅の変動と空間的な変動の大きさに分けて考えることができる。粒状抑制の画像処理の目的は、この粒状を視覚的に目立たないようにすることである。そのためには、上記の濃度振幅と空間的な大きさの両方を小さくすると粒状を抑制することができるが、振幅も大きさもある程度以下に小さくなると視覚的な改良効果が認識できなくなる。しかし、ディジタル画像では、空間的には画素の大きさが最小単位となり、濃度振幅ではデータの濃度分解能（たとえば、８bit の画像データならば、１bit の濃度差）が最小単位となるので、物理的にこれ以下にはできない。
【００４９】
粒状モトルは空間的に大きいものから小さいものまであり、小さいモトルでは画素単位で細かく信号や濃度が変動するが、大きいモトルでは周辺の幾つかの画素に跨がって信号や濃度が変動する。画素単位で変動する信号については、それ以上細かくすることはできないが、複数の画素に跨がる大きい粒状モトルについては、小さくすることによって視覚的に目立たないようにできる。
粒状モトルを小さくする処理は、先ず粒状Ｇ₀(x,y)を検出し、その粒状Ｇ₀(x,y)に細分化するための乱数や格子等の細かいパターンから成る細分化マスクＲ(x,y) を掛けて、粒状抑制成分、
Ｇ₁(x,y)＝Ｒ(x,y) ×Ｇ₀(x,y) （３２）
を求める。次に、粒状を抑制したい画像Ｉ_S (x,y) から、下記式（３３）のように引くことによって、粒状を抑制することができる。
Ｉ₁₀(x,y) ＝Ｉ_s (x,y) −αＧ₁(x,y) （３３）
【００５０】
以上の粒状抑制処理の工程は、以下のように、粒状細分化マスクの生成工程と粒状の細分化工程の二段階に分けることができる。
▲１▼粒状細分化マスクの生成工程、
マスクとしては、乱数パターンの他に網点状パターン（通常の二次元の網点、一次元の万線スクリーン、ＦＭスクリーン等）や誤差拡散のようなパターンでも良いがモアレ等のアーティファクトの無い乱数パターンが好ましい。
乱数も一様乱数、正規乱数、Poisson 乱数、二項乱数等があるが、自然界の揺らぎ現象に近い正規乱数が好ましい。また、自然界の揺らぎを最も良く表していると言われている１／ｆ揺らぎもこの目的に適している。
【００５１】
乱数の生成は、次式のように、画像の各画素に対して一個の乱数を発生させる。
Ｒ(x,y) ＝Ｒａｎ(i) （３４）
ただし、Ｒａｎ(i) は乱数、ｉ＝１，２,..., Ｎで、Ｎは処理する画像の総画素数である。正規乱数の確率密度分布は、下記式（３５）のようになり、平均値μのとき分散がσ² となる。粒状の細分化マスクの特性は、この乱数の平均値μの値を調節することによって、粒状の細分化の程度、特にマスクの振幅の変動量を制御することができる。μの値としては１０〜１０００程度の範囲のものが好ましい。
【００５２】
【数４】

【００５３】
▲２▼粒状の濃度変動パターンの細分化
検出した粒状データＧ₀(x,y)に上記の乱数Ｒ(x,y) を掛けて、細分化粒状データ（粒状抑制成分）、Ｇ₁(x,y)を計算する。なお、ここで、Ｒ_G (x,y) は正規乱数を表わす。
G₁(x,y)＝ R_G (x,y) × G₀(x,y) （３６）
こうして、細分化粒状成分Ｇ₁(x,y)を求めることができる。
【００５４】
８）次に、最後の粒状抑制およびエッジ強調工程における粒状の抑制・除去工程について説明する。
シャープネス強調した画像Ｉ_S (x,y) から、次式（３７）のように式（３６）で得られた細分化粒状成分Ｇ₁(x,y)を引くことによって、粒状を抑制する。
Ｉ₁₀(x,y) ＝Ｉ_s (x,y) −αＧ₁(x,y) （３７）
ただし、αは粒状抑制の程度を調節する係数である。この処理により、シャープネス強調によって悪化した粒状を抑制することができる。ここでの粒状抑制は粒状モトルのような粗いパターンが除去され、細分化されたパターンのみが残り、視覚的には微細化、あるいは微粒子化されたような粒状となる。
【００５５】
９）最後に、エッジ強調工程について説明する。
エッジ・粒状混在成分の抽出工程において得られた式（１８）で示した粒状成分とエッジ成分の混在成分ΔＩ_EG(x,y) にエッジ領域の重み付けＷ_E (x,y）を乗じて、エッジ強調成分Ｅ₁(x,y)を計算する。
Ｅ₁(x,y)＝Ｗ_E (x,y) ×ΔＩ_EG(x,y) （３８）
こうして得られたエッジ強調成分Ｅ₁(x,y)を上記式（３７）で得られた粒状抑制画像Ｉ₁₀(x,y）に加えることにより、最終的な処理画像Ｉ₁(x,y)を求める。ただし、βはエッジ強調成分を付加する程度を調節する係数である。
Ｉ₁(x,y)＝ I₁₀(x,y)+βＥ₁(x,y) （３９）
シャープネス強調は式（９）で行う処理でほぼ目的を達しているが、このエッジ強調を行う意味は、粒状抑制処理によって粒状と共に若干抑制されるエッジ成分を修復するために行うものであり、通常は僅かな量で良い。
【００５６】
こうして、原画像Ｉ₀(x,y)から粒状などのノイズが抑制されかつ十分にシャープネスが強調された最終処理画像Ｉ₁(x,y)を得ることができる。なお、エッジ強調工程で求めたエッジ強調成分Ｅ₁(x,y)を先の工程で求めておき、粒状抑制とエッジ強調とを同時に行ってもよい。この場合には、式（３７）および（３９）から、下記式（４０）として表わすことができる。
Ｉ₁(x,y)＝Ｉ_S (x,y) −αＧ₁(x,y)＋βＥ₁(x,y) （４０）
本発明の画像処理方法は基本的に以上のように構成される。
【００５７】
【実施例】
本発明のディジタル画像のノイズ抑制およびシャープネス強調のための画像処理方法を実施例に基づいて具体的に説明する。
３５ｍｍカラーネガフィルム FUJICOLOR SUPER G ACE 400 (富士写真フィルム社製）に撮影された写真画像をスキャナ SCANNER & IMAGE PROCESSOR SP-1000（画素数：２７６０×１８４０、富士写真フイルム社製）で読み取って、Ｒ，Ｇ，Ｂ各色８ｂｉｔでディジタル化した画像を原画像として、図１に示すフローに従って本発明の画像処理方法を行った。
【００５８】
ここで、シャープネス強調処理は、式（９）で示されるガウシアン型アンシャープマスキングを行ない、アンシャープマスクとしては、下記式のような３×３のマスクを用いた。
−０．５０ −０．８２ −０．５０
−０．８２ ６．２７ −０．８２
−０．５０ −０．８２ −０．５０
【００５９】
平滑化処理は、正規分布型の関数を掛けて平均値を求める方法を用い、処理マスクとしては上記式（１７）で表わされる９×９画素のマスクをマスク全体の和が１．０になるようにして用い、ローパスフィルタ処理と同様の効果を得た。
エッジ検出は図２に示す３×３画素の配列を用い、上記式（２０）による局所分散方式で行い、エッジ領域の重み付け係数Ｗ_E (x,y) は上記式（２２）を用いて非線形変換を行った。Ｗ_E を計算する際のエッジ局所分散の閾値σ_T の値は３０とした。
【００６０】
粒状とエッジの識別・分離処理では、図４に示す非線形変換ＬＵＴを用い、粒状の識別閾値Ｇ_T の値は５０（８ｂｉｔの０〜２５５の階調ステップ単位で、濃度値の約１００倍）とした。粒状モトル細分化処理では、ガウシアン乱数を用いた。また、最終処理画像を得るための上記式（４０）のパラメータαおよびβの値は、それぞれα＝０．７およびβ＝０．３とした。
【００６１】
このように本発明の画像処理方法が適用された画像の３次元濃度プロファイルを原画像Ｉ₀ から最終処理画像Ｉ₁ までにわたって１つのシーンについて図５（ａ）〜図９（ｂ）に示す。図５（ａ）は原画像Ｉ₀ 、図５（ｂ）は被写体エッジＥ₀ の検出画像、図６（ａ）はシャープネス強調画像Ｉ_S 、図６（ｂ）は平滑化画像Ｉ_AV、図７（ａ）は粒状・エッジ混在成分ΔＩ_EG、図７（ｂ）は粒状領域の重み付けデータＷ_G 、図８（ａ）は粒状・エッジ混在成分から分離された粒状成分Ｇ₀ 、図８（ｂ）は細分化粒状成分Ｇ₁ 、図９（ａ）はエッジ領域の重み付けデータＷ_E 、図９（ｂ）は最終処理画像Ｉ₁ の３次元濃度プロファイルを示す。
【００６２】
これらの図から明らかなように、図９（ｂ）に示す最終処理画像Ｉ₁ は、図５（ａ）に示す原画像Ｉ₀ に比べ、エッジ部が図６（ａ）に示すシャープネス強調画像Ｉ_S とほぼ同程度にシャープに強調されているにもかかわらず、粒状はシャープネス強調画像Ｉ_S に比べ格段に原画像Ｉ₀ とほぼ同程度まで抑制されていることがわかる。すなわち、本発明の画像処理方法が、粒状のみを抑制し、エッジ部のみをシャープネス強調した高品質な画像であることがわかる。
【００６３】
また、本発明の画像処理方法を主な銀塩カラー写真感光材料、すなわち、カラーフィルムおよび黒白フィルムに撮影した写真画像（３５ｍｍ、新写真システムＡＰＳ（アドバンスフォトシステム）、ＬＦパノラマ、インスタント）等に適用したところ、粒状とシャープネス共に一見して判る程の顕著な改善効果を得ることができた。
特に粒状については感材の微粒子化による粒状改良に匹敵する処理効果を持つため、従来の平均化や揺らぎの減少に基づく各種の粒状除去処理法の欠点であった「ぼけ粒状」的な不自然さや違和感はなかった。また、シャープネスについては、上記の粒状抑制と組み合わせることにより、従来のアンシャープマスクやラプラシアンフィルタよりかなり大幅な強調効果が得られた。
【００６４】
本発明に係るディジタル画像のノイズ抑制およびシャープネス強調のための画像処理方法について実施例を挙げて詳細に説明したが、本発明はこれに限定されず、本発明の要旨を逸脱しない範囲において、種々の改良および設計の変更を行ってよいことはもちろんである。
【００６５】
【発明の効果】
以上詳述したように、本発明によれば、原画像からシャープネス強調して得られたシャープネス強調画像のシャープネス強調された粒状成分を識別し、その粒状成分にさらにランダムな変調もしくは微細規則パターンによる変調を掛けた微細化粒状成分をシャープネス強調画像から差し引く方法で粒状を抑制しているので、元の粒状より空間的に細かく且つ濃淡変化の小さい、視覚的にも自然な粒状抑制を実現することができる。また、本発明によれば、粒状はシャープネス強調され、且つ空間的に微細化されるので、銀塩写真感材では微粒子乳剤を用いた時に得られるような細かい粒状となり、平滑化を用いた従来法の欠点であるぼけ粒状のような視覚的な違和感や不快感の無い自然な粒状抑制効果が得られる。
また、本発明法を銀塩カラー写真感光材料へ適用することにより、従来の粒状抑制シャープネス強調処理法の欠点であった、いわゆる「ぼけ粒状」的な不自然さや違和感がなく、粒状抑制とシャープネス強調とが同時に改善され、極めて顕著な改善効果を得ることができ、産業上大きな効果を奏する。
【図面の簡単な説明】
【図１】 本発明に係るディジタル画像のノイズ抑制およびシャープネス強調のための画像処理方法の一実施例を示すフローチャートである。
【図２】 本発明の画像処理方法の局所分散方式によるエッジ検出における正方画素配列の移動の一例を説明する説明図である。
【図３】 本発明の画像処理方法の粒状とエッジの識別・分離を説明するのに用いられる粒状とエッジの濃度差成分の頻度分布の特徴の一例を示す説明図である。
【図４】 本発明の画像処理方法において粒状成分の分離に用いられるルックアップテーブル（ＬＵＴ）関数の一例を示す説明図である。
【図５】 （ａ）および（ｂ）は、それぞれ本発明の実施例で得られたあるシーンの原画像およびエッジ検出画像の３次元濃度プロファイルの一例を示す図である。
【図６】 （ａ）および（ｂ）は、それぞれ本発明の実施例で得られたあるシーンのシャープネス強調画像および平滑化画像の３次元濃度プロファイルの一例を示す図である。
【図７】 （ａ）および（ｂ）は、それぞれ本発明の実施例で得られたあるシーンの粒状・エッジ混在成分および粒状領域の重み付けデータの３次元濃度プロファイルの一例を示す図である。
【図８】 （ａ）および（ｂ）は、それぞれ本発明の実施例で得られたあるシーンの粒状成分および細分化粒状成分の３次元濃度プロファイルの一例を示す図である。
【図９】 （ａ）および（ｂ）は、それぞれ本発明の実施例で得られたシーンのエッジ領域の重み付けデータおよび最終処理画像の３次元濃度プロファイルの一例を示す図である。[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image processing method for suppressing noise and sharpness enhancement of a digital image that suppresses noise (noise) such as graininess of the digital image and enhances the sharpness of the digital image.
[0002]
[Prior art]
Digital images, such as photos recorded by an image input scanner and output by an image output printer, have significant sharpness degradation due to the scanner and printer. To recover from this, a Laplacian filter or unsharp mask ( USM) sharpness enhancement is performed. However, since the sharpness of the image is improved and there is a side effect that noise such as granularity is deteriorated, in a granular image, only moderate sharpness enhancement can be performed within a range where granular deterioration is allowed, which is higher than that of the original image. It was difficult to improve the image quality.
[0003]
Several digital image processing methods have been proposed for removing noise graininess and enhancing sharpness in digital images. However, since a method of averaging or blurring is used as a method for removing graininess, a blurred grain pattern is used. However, it is not suitable for an aesthetic image such as a photograph.
[0004]
In images from photographs, prints, televisions, various copying machines, etc., sharpness deterioration due to optical systems such as cameras, graininess and sharpness deterioration inherent to photographic photosensitive materials, or original images such as photographs and prints can be input with an image input device. Various methods have been devised as image processing methods for suppressing noise or enhancing sharpness in order to recover noise (noise) and sharpness degradation added when digitizing. For example, in the conventional image processing method, smoothing and coring methods are used as the granular removal processing method, and unsharp mask (USM), Laplacian, or high-pass filter processing is used as the sharpness enhancement processing method. It is used. However, in these conventional grain removal processing methods, it is not desirable that suppressing the grain causes unnatural discomfort artifacts or suppresses the fine structure of the image that should not be suppressed together with the grain. Had drawbacks.
[0005]
  For example, Japanese Patent Publication Nos. 57-500311 and 57-500334 and “Digital Sharpening Method of Unsharp and Grainy Photo Images”, International Conference on Electronic Image Processing, July 1982, pp. 179-183 (PG Powell and BEBayer, “A Method for the Digital Enhancement of Unsharp, Grainy Photographic Images”,Proceedings The processing method of Powell and Buyer et al. disclosed in the International Conference on Electronic Image Processing, Jul. 26-28, 1982, pp. 179-183) uses a smoothing method (low-pass filter) as a granular suppression method. As a sharpness enhancement method, a processing method using an unsharp mask (high-pass filter) is used. The smoothing process is a process for suppressing grain by smoothing a signal by multiplying a signal value of n × n pixels by a weight such as a Gaussian type. In the sharpness enhancement process, first, a differential value in the direction from the center pixel to the surrounding pixels is obtained using an image signal of m × m pixels. The sharpness enhancement is performed by taking the sum of the differential values larger than the remaining threshold value after removal by processing, multiplying by a constant of 1.0 or more, and adding to the smoothed signal.
[0006]
In this processing method, since the granular pattern is blurred, the contrast of the granular pattern is lowered, but a large uneven pattern made up of particles (mottle) of particles constituting the granularity becomes visually noticeable. , Has the disadvantage of appearing as an unpleasant grain. In addition, since the granularity and the image are identified with the set threshold (coring processing), the low-contrast image signal is erroneously recognized as granular and is suppressed or removed together with the granularity. Discontinuity occurs at the boundary of the image, and unnatural artifacts are seen in the image. In particular, this defect appears in a fine image such as a lawn and a carpet, and an image in which a texture such as a fabric is depicted, and this is a visually unnatural and undesirable artifact.
[0007]
[Problems to be solved by the invention]
By the way, in the conventional grain suppression / sharpness enhancement image processing method described above, sharpness is enhanced with an unsharp mask, grain is blurred or smoothed, and a method of suppressing graininess (noise) signal and contour signal from the original image is used. Are processed at the signal level, the contour signal is sharpened and the smooth area is suppressed by granularity, so that small signals are processed as granular. There is a problem in that image signals such as texture and hair are suppressed together with graininess, resulting in a visually unpleasant image as an image processing artifact. That is, in such a conventional method, blurring or averaging is used as a granularity suppression method, and the blurred granular pattern appears as if the density fluctuation has become smaller and the graininess has improved, but conversely the density fluctuation. Although the amount is small, the blurred and widened granular pattern is visually recognized as an unpleasant pattern, and in particular, there is a problem that it stands out on the face and skin of portrait pictures and uniform subjects such as walls and sky .
[0008]
  In the conventional method of separating the granular (noise) region and the contour region from the original image at the signal level, the contour region and the flat region are identified from the difference signal between the original image and the blurred image, and an unsharp mask or By processing using different coefficients such as Laplacian, grain is suppressed in the flat area, while sharpness is emphasized in the outline area to suppress grain without blurring the edge. There is a problem that discontinuity occurs at the boundary because separation is performed uniformly at a signal level that becomes a threshold value.
  Furthermore, in such a conventional method, as an edge enhancement or sharpness enhancement method, an unsharp mask orLaplacianIs used, but the Mackey line is applied to the contour and edge of the image.The border likeThere is a problem that it tends to occur and gives a visually unnatural impression.
[0009]
The present invention has been made in view of the current state of the prior art described above. In an image of a photograph, printing, television, electronic still photograph, various copying machines, etc., a camera blur, a photographic photosensitive material grain or blur, etc. When performing processing for recovering noise and sharpness degradation inherent in the original image, or noise and sharpness degradation added when the original image is digitized by the image input device, Problems, that is, when grain suppression by smoothing is performed, graininess is blurred and large unevenness looks visually unpleasant, image signals with low contrast are mistakenly recognized as grain, and are suppressed or removed The boundary between the removal area and the sharpness enhancement area becomes discontinuous and the image is not unnatural. And an object thereof is to provide an image processing method for noise suppression and sharpness enhancement of a digital image for performing processing for emphasizing Punesu.
[0010]
[Means for Solving the Problems]
In order to achieve the above object, the present invention performs sharpness enhancement processing on original image data to create sharpness enhancement image data,
Smoothing the original image data to create smoothed image data;
Subtracting this smoothed image data from the sharpness-enhanced image data to extract a mixed component of edges and noise, and performing edge detection from the original image data to obtain weighted data of edge regions and noise regions,
After performing weighting using the weighting data of this noise region to the mixed component of the edge and noise, the noise component is separated by performing nonlinear transformation,
The obtained noise component is subdivided to obtain a subdivided noise component,
The edge enhancement component is obtained by performing weighting using the weighting data of the edge region to the mixed component of the edge and noise,
In order to suppress noise and enhance sharpness in a digital image, the processed image data is obtained by scaling and removing the subdivided noise component from the sharpness weighted image data and scaling and adding the edge weighted component. An image processing method is provided.
[0011]
Here, the sharpness enhancement processing is preferably Gaussian unsharp mask processing, and the smoothing processing is preferably Gaussian blur mask processing. The non-linear conversion for separating the noise component is preferably performed by a function that continuously decreases as a Gaussian function as the density fluctuation becomes larger than a threshold value of noise density fluctuation. Further, it is preferable that the subdividing process is a noise motor subdividing process.
[0012]
Further, it is preferable that the subdividing process is a process of multiplying the noise component by a random number. Moreover, it is preferable that the edge detection for obtaining the weighted data is based on a local dispersion method.
[0013]
[Action]
An image processing method for noise suppression and sharpness enhancement of a digital image according to the present invention performs sharpness enhancement processing on an original image before processing, sharpens the image, and includes granular / noise (noise) contained in the image. ) Are both sharpened, the edge and flat part of the image are divided into areas, the flat part is regarded as a granular area, a granular signal is detected, and the granularity is subdivided and selectively removed. is there. In this way, according to the present invention, it is possible to obtain a sharpness-enhanced image at the edge portion and a high-quality image in which the granularity is suppressed in the granular region.
Further, in the granularity suppression method of the present invention, a granular component with sharpness enhancement is identified, and the granularity is suppressed by a method of subtracting a finer granular component obtained by further applying random modulation to the granular component from the sharpness enhancement image. Thus, it is possible to realize a grain that is spatially finer than the original grain and has a small change in shading. Since the grain is sharply emphasized and spatially refined, the grain can be made as fine as obtained when a fine grain emulsion is used in a silver salt photographic material.
[0014]
DETAILED DESCRIPTION OF THE INVENTION
An image processing method for noise suppression and sharpness enhancement of a digital image according to the present invention will be described in detail based on a preferred embodiment shown in the accompanying drawings.
[0015]
FIG. 1 is a flowchart showing an example of a processing algorithm of the image processing method of the present invention. In the following description, granularity will be described as a representative example of noise of a digital image.
In the present invention, as shown in FIG.₀To sharpness-enhanced image I_SAnd smoothed image I_AVTo extract a mixed component of edge and grain (noise), that is, fine image data ΔI of edge / grain mixed_EGCreate On the other hand, the subject edge E in the original image directly from the original image₀And weighting data W of the edge area and the granular area_EAnd W_GAsk for.
[0016]
Thereafter, these weighting data W_EAnd W_GIs used to obtain the fine image data ΔI of the edge / grain mixture previously obtained._EGTo separate edges and grain by non-linear transformation (NLG). That is, the edge / grain mixed component ΔI_EGGranular area weighting data W_GAnd weighting by multiplying by the granular component G including the edge information of the granular region₀This granular component G₀Is refined into fine particles by a mottle subdivision process, and the subdivided granular component, that is, the granular suppression component G₁Create On the other hand, the edge component is an edge / grain mixed component ΔI._EGEdge region weighting data W_EAnd weighting the edge enhancement component E of the edge region₁Create
Finally, the sharpness-enhanced image I obtained first_SGranular suppression component G prepared in this way₁Is subtracted by a constant multiple, for example, α, and the edge enhancement component E is removed.₁Is added by multiplying by a constant multiple, for example, β times, to thereby add sharpness to the edge region of the image and process image I in which the flat region is grain-suppressed.₁Is created.
[0017]
The feature of the method of the present invention is that the granularity suppression process and the sharpness enhancement process are distributed to the fuzzy by suppressing the granularity by the micronization method and identifying the image edge (contour) and the granular area by non-linear transformation. It is. That is, the boundary between the granular area and the edge area is not on / off, and both areas overlap and the ratio gradually changes, so the boundary does not stand out unnaturally, It will be very natural. In the method of the present invention, the parameter for determining the degree of granular suppression can be automatically set based on the root mean square (RMS) of the granularity / edge component density fluctuation ΔD.
Further, the algorithm of the present invention that suppresses granularity / noise and enhances the sharpness can be processed on digitized image data using a computer or a dedicated processing device.
[0018]
Next, each step of the image processing method of the present invention will be briefly described with reference to FIG.
1) Sharpness enhancement process
Using Gaussian unsharp mask (Gaussian USM), the original image I₀In addition to the recovery of blur in the image input / output system, the sharpness enhancement image I_SCreate
2) Smoothing process and edge / grain mixed component extraction process,
For example, the original image I using averaging or a blur mask₀Smoothed image I_AVCreate sharpness-enhanced image I_SFrom the following formula (1), fine image data ΔI mixed with edge and grain_EGCreate
ΔI_EG  = I_S  -I_AV                  (1)
[0019]
3) a step of calculating a weighting coefficient between the edge region and the granular region;
Original image I₀Directly from the original image I, for example by local dispersion₀Inside subject edge E₀And this edge E₀Is used to weight the edge area and the granular area (data) W_EAnd W_GAsk for. Where W_E+ W_G= 1.0, so the granular area weighting data W_GThe edge area weighting data is 1.0-W._GMay be used or vice versa.
4) Edge / grain identification / separation process,
Granular area weighting data W_GAnd edge area weighting data W_EAnd the edge / grain mixed fine image data ΔI_EGTo separate edges and grain by a non-linear transformation function (NLG). Granular component G₀And edge component E₁Can be obtained by the following equations (2) and (3), but the edge component E₁Is the edge enhancement component E₁As required.
G₀= NLG (ΔI_EG× W_G(2)
E₁= ΔI_EG× W_E                          (3)
[0020]
5) Granular mottle subdivision process,
Granular component G₀Is a Gaussian random number R expressed by the following equation (4):_GThe fine particles are refined by the mottle subdivision process using₁Is created.
G₁= Φ (G_0,R_G(4)
6) Grain suppression and edge enhancement (sharpness enhancement) step (final processing image calculation step),
Sharpness-weighted image I obtained earlier_SAnd the granular suppression component G obtained here₁And edge enhancement component E₁To the original image I₀Processed image I with sharpness emphasis on the edge area of the image, and granular area and other flat areas with graininess suppressed.₁Can be obtained. Here, α and β are parameters for adjusting the strength of processing.
I₁= I_S-ΑG₁+ ΒE₁                    (5)
[0021]
By the way, in the above-described example, 4) the granular component G in the edge and granular identification / separation step₀And edge enhancement component E₁However, the present invention is not limited to this, and in this identification / separation step, the particulate component G₀Edge enhancement component E₁May be created in the final 6) edge enhancement step.
In the above-described example, the granular suppression component G is used in the final granular suppression and edge enhancement process.₁And edge enhancement component E₁And the sharpness-enhanced image I_SHowever, the present invention is not limited to this, and subtraction and addition may be performed separately.
[0022]
The target image in the image processing method of the present invention is not particularly limited, but includes not only hard copy images such as photographs, prints, electronic still photographs, and various copying machines, but also televisions, computer CRTs, liquid crystals, and the like. It may be a soft copy image displayed on the display device.
In the above description, the granularity is described as a representative example of noise to be suppressed in these images. However, the present invention is not limited to this, and is caused by blurring by a camera, granularity or blurring of a photographic photosensitive material, and the like. Noise inherent to the original image to be read, or noise added when these original images are read by an image input device such as a scanner and converted into a digital image, or a digital image shot with a video camera, electronic still camera or digital camera Any noise may be used as long as it is a noise to be suppressed, such as noise mixed when the image is converted.
[0023]
Next, each step of the image processing method of the present invention will be described in detail.
1) First, the sharpness enhancement step will be described.
Here, as a method for enhancing the sharpness of an image, an unsharp mask (Unsharp mask, USM) or a Laplacian is well known. Also in the present invention, by using these, the sharpness of the image can be enhanced if the sharpness degradation of the image is slight.
The unsharp mask is the original image I₀From (x, y), I₀(x, y) averaged or blurred image <I₀Edge enhancement component I obtained by subtracting (x, y)>₀(x, y)-<I₀(x, y)> multiplied by coefficient a₀By adding to (x, y), the sharpness enhanced image I_SThis is a method for obtaining (x, y).
I_S(x, y) = I₀(x, y) + a [I₀(x, y)-<I₀(x, y)>] (6)
Here, a is a constant for adjusting the degree of sharpness enhancement, and x and y indicate the position of the target pixel in the image.
Laplacian is the image I₀Second derivative of (x, y) (Laplacian) ▽²I₀A method of enhancing sharpness by subtracting (x, y) from the original image, and is expressed by the following equation.
I_S(x, y) = I₀(x, y) − ▽²I₀(x, y) (7)
As a specific example of sharpness enhancement by Laplacian, the following 3 × 3 coefficient array is often used.

[0024]
In this coefficient arrangement, an unnatural contour is likely to occur at the edge of the image when particularly strong sharpness enhancement is applied. Therefore, in order to reduce such a drawback, in the present invention, it is preferable to use an unsharp mask using a normal distribution type (Gaussian) blur function as shown in Equation (9).
G (x, y) = (1 / 2πσ²) exp [− (x²+ y²) / 2σ²] (9)
Where σ²Is a parameter representing the spread of the normal distribution function, and the edge x = x of the mask₁The ratio of the value at x to the value at the mask center x = 0,
G (x₁, 0) / G (0,0) = exp [−x₁ ²/ 2σ²] (10)
Is adjusted to 0.1 to 1.0, the desired sharpness of the 3 × 3 unsharp mask can be obtained. When the value of Expression (10) is set to a value close to 1.0, a mask almost the same as the Laplacian filter at the center of Expression (8) can be made.
In order to change the sharpness of the mask, there is another method of changing the size of the mask. For example, by using a mask of 5 × 5, 7 × 7, 9 × 9, etc., a spatial frequency region for sharpness enhancement. Can be changed drastically.
[0025]
Further, as the function form of the mask, other than the above normal distribution type, for example, an exponential function type mask as shown in the following formula (11) can be used.
E (x, y) = exp [− (x² + y²)^1/2/ A] (11)
Where a is σ in equation (9)²Is a parameter that represents the spread of the unsharp mask, as well as the ratio of the mask edge value to the mask center value,
E (x₁, 0) / E (0,0) = exp [−x₁/ A] (12)
Is adjusted to 0.1 to 1.0, the desired sharpness of the 3 × 3 unsharp mask can be obtained. In Equation (13), E (x₁, 0) / E (0,0) = 0.3 shows an example of the numerical value of the mask of the exponential function of equation (11).
0.18 0.30 0.18
0.30 1.00 0.30 (13)
0.18 0.30 0.18
When an example of an unsharp mask is calculated from this mask, the following equation (14) is obtained.
−0.12 −0.22 −0.12
−0.21 2.32 −0.21 (14)
−0.12 −0.21 −0.12
[0026]
Using such an unsharp mask, the original image I₀Sharpness-enhanced image I from (x, y)_S(x, y) can be obtained. The unsharp mask and sharpness enhancement method used in the present invention are not limited to those described above, and other conventionally known unsharp masks and sharpness enhancement methods such as spatial frequency filtering can be applied. Of course.
[0027]
2) Next, the smoothing process will be described.
Examples of the smoothing method include real space domain processing and spatial frequency domain processing. In real space region processing, the sum of all adjacent pixels is calculated and the average value is calculated and replaced with that value. The average value is calculated by multiplying each pixel by a weighting coefficient, for example, a normal distribution type function. There are various methods such as a method of performing such non-linear processing. On the other hand, in processing in the spatial frequency domain, there is a method of applying a low-pass filter. For example, in the averaging method using the weighting coefficient, the following equation (15) can be given.
[0028]
[Expression 1]

[0029]
Here, n is an averaging mask size, and w is a weighting factor. When w = 1.0, a simple average is obtained.
In the present invention, a method of obtaining an average value by multiplying a normal distribution type weighting coefficient in real space region processing is used, but the present invention is not limited to this. At this time, it is preferable to use a mask of n × n pixels as described below as a processing mask. Specifically, it is preferable to use those of about 3 × 3 to 5 × 5, 7 × 7, 9 × 9.

[0030]
Expression (17) shows an example of a 9 × 9 pixel mask. In this equation (17), the center value is shown as a value normalized to 1.0, but in the actual processing, the sum of the entire mask is set to 1.0.
0.09 0.15 0.22 0.28 0.30 0.28 0.22 0.15 0.09
0.15 0.26 0.38 0.47 0.51 0.47 0.38 0.26 0.15
0.22 0.38 0.55 0.69 0.74 0.69 0.55 0.38 0.22
0.28 0.47 0.69 0.86 0.93 0.86 0.69 0.47 0.28
0.30 0.51 0.74 0.93 1.00 0.93 0.74 0.51 0.30 (17)
0.28 0.47 0.69 0.86 0.93 0.86 0.69 0.47 0.28
0.22 0.38 0.55 0.69 0.74 0.69 0.55 0.38 0.22
0.15 0.26 0.38 0.47 0.51 0.47 0.38 0.26 0.15
0.09 0.15 0.22 0.28 0.30 0.28 0.22 0.15 0.09
[0031]
Using such a mask, the original image I₀Smoothed image I from (x, y)_AV(X, y) can be obtained. It should be noted that the smoothing method used in the present invention is not limited to the various methods described above, and it goes without saying that any of the conventionally known smoothing methods can be applied.
[0032]
3) Next, the extraction process of the mixed component of grain and edge will be described.
Sharpness-enhanced image I thus obtained_S(x, y) and smooth image I_AVFrom (x, y), the difference between the two is calculated and the mixed component ΔI between the grain and the edge_EGExtract as (x, y).
ΔI_EG(x, y) = I_S(x, y) −I_AV(x, y) (18)
[0033]
4) The process of calculating the edge detection and the weighting coefficient of the edge area and the granular area will be described. Here, edge detection by the local dispersion method is described as a representative example as an example, but the present invention is not limited to this.
[0034]
(1) Pretreatment: density conversion,
When performing edge detection, first, as shown in Equation (19), the original image I₀Density value D of three colors of R, G, B of (x, y)_R, D_G, D_BIs multiplied by weighting factors r, g, and b to create a visual density D_VConvert to
D_V= (RD_R+ GD_G+ BD_B) / (R + g + b) (19)
For example, a value such as r: g: b = 4: 5: 1 is used as the weighting coefficient. This conversion is performed in order to reduce grain and noise having no correlation in R, G, and B, and improve the accuracy of edge detection. The size of the pre-processing array is preferably about 5 × 5 or 7 × 7 pixels. However, in the next process (2), the variation in image density in the array This is because the calculation is performed while moving using a small array, for example, an array of about 3 × 3 as shown in FIG.
[0035]
Note that the weighting factors r, g, and b in edge detection can be obtained as follows.
The weighting factor is conspicuous when visually observed (although this may correspond to the spectral visibility distribution), that is, based on the idea that the weighting factor of image data with a large contribution is large. It is preferable to set an optimal value. In general, empirical weighting factors are obtained based on visual evaluation experiments and the like, and the following values are known as general knowledge (as well-known literature, Takashi Noguchi, “Good psychological response” Grain evaluation method), Journal of the Japan Photography Society, 57 (6), 415 (1994), which differs depending on the color, but shows a numerical value close to the following ratio).
r: g: b = 3: 6: 1
r: g: b = 4: 5: 1
r: g: b = 2: 7: 1
Here, if a preferable range of values is specified as the ratio r: g: b of the coefficient, when r + g + b = 10.0 and b is 1.0,
g = 5.0 to 7.0
A value in the range of is preferred. However, r = 10.0−b−g.
[0036]
(2) Edge detection by local dispersion,
As shown in FIG. 2, the edge is detected by the visual density D._VN from the image data of_E× n_EDetection of the subject edge in the image by moving the pixel array and calculating the local standard deviation σ for each position as local variance sequentially using the equation (20) for the image density variation in the array. I do. Pixel array size (n_E× n_E) May be appropriately determined in consideration of detection accuracy and calculation load, but it is preferable to use a size of about 3 × 3 or 5 × 5, for example.
[0037]
[Expression 2]

However, D_ijN computes the local variance_E× n_E<D> is the average density of the array,
[Equation 3]

It is.
[0038]
(3) Calculation of weighting coefficients for edge area and granular area,
From the local variance σ (x, y) shown in the above equations (20) and (21), the weight W of the edge region_E(x, y) and granular area weight W_GIn order to obtain (x, y), the following equations (22) and (23) using an exponential function were used.
W_E(x, y) = 1−exp [−σ (x, y) / a_E] (22)
W_G(x, y) = 1-W_E(x, y) (23)
However, a_EIs the weighting W of the local variance σ (x, y)_ECoefficient for conversion to (x, y), W_E= Σ (x, y) threshold σ assigned to 0.5_TThen,
a_E= −σ_T/ Log_e(0.5)
It is. σ_TThe value needs to be an appropriate value depending on the size of the granularity and the signal of the subject outline, but a value in the range of 10 to 100 is preferable for a color image of 8 bits (256 gradations). If this conversion is created as a LUT (Look up table), the calculation time required for the conversion can be shortened.
[0039]
Edge area weighting W_EThe conversion equation for obtaining (x, y) is not limited to the above equation, and other equations can also be used. For example, the following Gaussian function may be used.
W_E(x, y) = 1−exp {− [σ (x, y)]²/ A_E1 ²}
However, a_E1Is W from σ (x, y)_ECoefficient for conversion to (x, y), W_E= Σ (x, y) threshold assigned to 0.5_TThen,
a_E1 ²= −σ_T ²/ Log_e(0.5)
It is. σ_TThe value is preferably in the range of 10 to 100 for each color image of 8 bits (256 gradations).
[0040]
By the way, in the present invention, the edge detection method is not limited to the above-described local dispersion type edge detection method, and other edge detection methods can also be used. As edge detection methods other than the local dispersion method, there are methods based on primary differentiation and secondary differentiation, and there are several methods for each.
First, there are the following two operators as a method based on the spatial first derivative.
Examples of differential edge extraction operators include Prewitt operator, Sobel operator, and Roberts operator. The Roberts operator can be expressed as:
g (i, j) = {[f (i, j)-f (i + l, j + l)]²+ [F (i + l, j)-f (i, j + l)]²}^1/2
As template type operators, there are Robinson operators and Kirsh operators that use 3 × 3 templates corresponding to edge patterns in 8 directions.
Next, as a method based on a spatial second derivative, there is a method using Laplacian. In this case, since noise is emphasized, a method of first performing a normal distribution type blurring process and then detecting an edge is often used.
[0041]
5) Next, the weighting of the grain and the edge in the edge and grain identification / separation process will be described..
  Here, the granular and edge features are used to identify and separate the granular and edge components. First, in the spatial region, the graininess is present in the entire film, that is, the entire image, but is conspicuous in a portion flatter than the contour or edge portion of the subject. On the other hand, the edge is mainly in the contour portion of the subject and the fine portion of the subject surface in the image. Further, in the density region, as shown in FIG. 3, since the grain is mainly composed of the grain of the photographic photosensitive material used for photographing, the density difference is often small as shown by the dotted line in the figure. The edge depends on the contrast of the subject and varies greatly depending on the image. However, as shown by the solid line in the figure, the density difference varies from a very small one to a very large one.
[0042]
6) Next, the grain / edge identification / separation process will be described.
In order to identify and separate the grain and the edge, first, using the spatial characteristics of both, the grain and edge are divided into regions. The weight W of the granular area obtained by using the edge of the subject detected from the original image_G(x, y) is the mixed density value ΔI of grain and edge_EGBy multiplying by (x, y), the edge information is reduced, and the signal ΔI with a high ratio of granular information_Gcan be (x, y).
ΔI_G(x, y) = W_G(x, y) × ΔI_EG(x, y) (24)
[0043]
Next, the image information of the grain and the edge is separated using the feature in the density region. As shown in FIG. 3, signals with small density differences are mainly granular components, and edge signals are also mixed somewhat, signals with large density differences are mainly edge components, and granular components with larger density differences are mixed. Therefore, the granularity and the edge can be separated using the magnitude of the density difference. Separation of the granular component G (x, y) is performed using a non-linear transformation LUT expressed by the following equation (25). An example of the non-linear transformation LUT used for separating the granular components is shown in FIG.
G (x, y) = LUT (ΔD (x, y)) (25)
However, the LUT is
LUT (ΔD) = ΔD × exp [− (ΔD)²/ A_G ²] (26)
And a_G ²Is the threshold G of the granular density fluctuation_TIs a constant determined from
a_G ²= -G_T ²/ log_e(1/2) (27)
It is.
[0044]
Here, the threshold G of granular density fluctuation_TIs the mixed density variation ΔI between the grain and the edge_EGIn (x, y), the density fluctuation below this value is considered to be granular, but as is easily understood from Equation (26) and FIG. In this case, the granularity to be separated decreases according to the LUT shape that gradually decreases as the density fluctuation increases. Therefore, the edges are mixed together with the grain, but the ratio gradually decreases.
Such a non-linear conversion LUT is expressed as a non-linear conversion function NLG, and an edge / grain mixed component is represented by ΔI._EGExpressed by (x, y), the granular component G₀(x, y) is obtained from the above equations (24) and (25).
G₀(x, y) = NLG {ΔI_EG(x, y) × W_G(x, y)} (28)
Can be expressed as Thus, the granular component G₀(x, y) can be obtained.
The calculation of the edge component performed in this step will be described later.
[0045]
By the way, granular discrimination threshold G_TIt is preferable to select an optimal value depending on the granularity of the image to be processed, the size of noise, and the degree of sharpness enhancement processing. Grain identification is performed on an image that has been subjected to sharpness enhancement processing. Therefore, the granularity is that in which the granularity of the original image has been sharpened by sharpness enhancement processing and the density fluctuation has increased. Therefore, when performing the granularity suppression processing, the density variation of the surrounding n × n pixels is referred to, and the size of the granularity after the sharpness enhancement processing is expressed by a physical value such as RMS granularity σ, and the granularity is Identification threshold G_TWill be decided. The determination method will be described below.
[0046]
The granularity of a color photographic light-sensitive material is usually measured with an RMS granularity using a micro-densitometer and a measurement aperture of 48 μφ, and a general or typical color negative film such as Super G ACE 100, 200 is used. , 400, 800 (both manufactured by Fuji Photo Film Co., Ltd.), etc., values between 4 and 5 (RMS granularity σ₄₈Is displayed with a value multiplied by 1000). When this film is digitized by scanning with an opening area A, the granularity σ of the film at the opening area_scIs the RMS granularity σ measured at the 48 μφ aperture using the well-known Selwyn granularity equation S = σ√A.₄₈Can be converted by the following equation (29).
σ_sc= Σ₄₈√A₄₈/ √A_sc                            (29)
Where A₄₈Is the area of the opening of 48 μφ. For example, the granularity of the film is 4, and the digitized scanning aperture is 12 μφ (the aperture area is A₁₂)
σ_sc= Σ₄₈√A₄₈/ √A₁₂= 0.016 (30)
It becomes. In either case, the blur caused by the optical system and the scanning aperture is the same.
[0047]
Granularity σ when sharpness enhancement is performed_scSuppose that is p times larger
σ_sc′ = Pσ_sc                                      (31)
It becomes.
Granular discrimination threshold G_TIs a value proportional to the granularity of the image to be processed, ie G_T= K_Gσ_sc'Is preferred. Where k_GIs a constant, and a value of 1.0 to 3.0 is preferred. G_TThe larger the value of σ is, the more the granularity can be identified. On the other hand, the probability that the low-contrast subject information close to the granular density fluctuation is misidentified as a granularity increases. On the other hand, if it is smaller than σ, subject information is less likely to be misidentified, but particles having a large density fluctuation cannot be captured in the particles, and coarse particles remain in the image.
[0048]
7) Next, the subdivision process of the granular mottle will be described.
The granular pattern is composed of fine density fluctuations in the image. The density fluctuations can be considered by dividing them into the amplitude fluctuation of the density value and the magnitude of the spatial fluctuation. The purpose of image processing for grain suppression is to make this grain visually inconspicuous. To that end, if both the density amplitude and the spatial size are reduced, the graininess can be suppressed. However, if the amplitude and size are reduced to a certain extent, the visual improvement effect cannot be recognized. However, in a digital image, the pixel size is the smallest unit spatially, and the density resolution (for example, 1-bit density difference for 8-bit image data) is the smallest unit for density amplitude. No less than this.
[0049]
The granular mottles are spatially large to small, and in small mottles, the signal and density fluctuate finely in units of pixels, but in large mottles, the signal and density fluctuate across several neighboring pixels. A signal that varies in units of pixels cannot be made finer, but a large granular motor that spans a plurality of pixels can be made visually inconspicuous by making it small.
The process to make the granular mottle smaller is first the granular G₀(x, y) is detected and its granular G₀Multiplying a subdivision mask R (x, y) consisting of fine patterns such as random numbers and lattices to subdivide into (x, y)
G₁(x, y) = R (x, y) x G₀(x, y) (32)
Ask for. Next, I want to suppress grain I_SBy subtracting from (x, y) as shown in the following formula (33), graininess can be suppressed.
I_Ten(x, y) = I_s(x, y) -αG₁(x, y) (33)
[0050]
The process of the above granular suppression process can be divided into two stages, a granular subdivision mask generation process and a granular subdivision process, as follows.
(1) Granule segmentation mask generation process,
As a mask, in addition to a random number pattern, a halftone dot pattern (a normal two-dimensional halftone dot, a one-dimensional line screen, an FM screen, etc.) or a pattern such as error diffusion may be used. A pattern is preferred.
There are uniform random numbers, normal random numbers, Poisson random numbers, binomial random numbers, and the like, but normal random numbers close to the natural fluctuation phenomenon are preferable. Further, 1 / f fluctuation, which is said to best represent the fluctuation in the natural world, is also suitable for this purpose.
[0051]
The random number is generated by generating one random number for each pixel of the image as shown in the following equation.
R (x, y) = Ran (i) (34)
Here, Ran (i) is a random number, i = 1, 2,..., N, and N is the total number of pixels of the image to be processed. The probability density distribution of normal random numbers is as shown in the following formula (35), and the variance is σ when the average value μ is²It becomes. As the characteristics of the granular subdivision mask, the degree of granular subdivision, in particular, the amount of variation in the amplitude of the mask can be controlled by adjusting the average value μ of the random numbers. The value of μ is preferably in the range of about 10 to 1000.
[0052]
[Expression 4]

[0053]
(2) Subdivision of granular density fluctuation pattern
Detected granular data G₀Multiply (x, y) by the above random number R (x, y) to obtain subdivided granular data (granular suppression component), G₁Calculate (x, y). Here, R_G(x, y) represents a normal random number.
G₁(x, y) = R_G(x, y) × G₀(x, y) (36)
Thus, the finely divided granular component G₁(x, y) can be obtained.
[0054]
8) Next, the grain suppression / removal process in the final grain suppression and edge enhancement process will be described.
Sharpness-enhanced image I_SFrom (x, y), the subdivided granular component G obtained by the equation (36) as the following equation (37):₁Grain is suppressed by subtracting (x, y).
I_Ten(x, y) = I_s(x, y) -αG₁(x, y) (37)
Here, α is a coefficient for adjusting the degree of granular suppression. By this processing, the graininess deteriorated by sharpness enhancement can be suppressed. In the grain suppression, a coarse pattern such as a granular mottle is removed, and only the subdivided pattern remains, and the grain is visually refined or finely grained.
[0055]
9) Finally, the edge enhancement process will be described.
The mixed component ΔI of the granular component and the edge component represented by the equation (18) obtained in the extraction step of the edge / grain mixed component_EG(x, y) is the edge area weight W_EMultiply by (x, y) to obtain edge enhancement component E₁Calculate (x, y).
E₁(x, y) = W_E(x, y) × ΔI_EG(x, y) (38)
Edge enhancement component E thus obtained₁(x, y) is the granularity suppression image I obtained by the above equation (37)._TenBy adding to (x, y), the final processed image I₁Find (x, y). Here, β is a coefficient that adjusts the degree to which the edge enhancement component is added.
I₁(x, y) = I_Ten(x, y) + βE₁(x, y) (39)
Sharpness enhancement is almost achieved by the processing performed by equation (9). The meaning of performing edge enhancement is to repair edge components that are slightly suppressed together with the granularity by the granularity suppression processing. A small amount is sufficient.
[0056]
Thus, the original image I₀Final processed image I in which noise such as graininess is suppressed from (x, y) and sharpness is sufficiently enhanced₁(x, y) can be obtained. The edge enhancement component E obtained in the edge enhancement step₁(x, y) may be obtained in the previous step, and grain suppression and edge enhancement may be performed simultaneously. In this case, from the equations (37) and (39), it can be expressed as the following equation (40).
I₁(x, y) = I_S(x, y) -αG₁(x, y) + βE₁(x, y) (40)
The image processing method of the present invention is basically configured as described above.
[0057]
【Example】
An image processing method for noise suppression and sharpness enhancement of a digital image according to the present invention will be specifically described based on examples.
A photographic image taken on a 35 mm color negative film FUJICOLOR SUPER G ACE 400 (manufactured by Fuji Photo Film Co., Ltd.) is read with a scanner SCANNER & IMAGE PROCESSOR SP-1000 (pixel number: 2760 × 1840, manufactured by Fuji Photo Film Co., Ltd.). The image processing method of the present invention was performed according to the flow shown in FIG. 1 using an image digitized with 8 bits for each color of G and B as an original image.
[0058]
Here, the sharpness enhancement processing performed Gaussian unsharp masking represented by Expression (9), and a 3 × 3 mask represented by the following expression was used as the unsharp mask.
-0.50 -0.82 -0.50
-0.82 6.27 -0.82
-0.50 -0.82 -0.50
[0059]
The smoothing process uses a method of obtaining an average value by multiplying a normal distribution type function. As a process mask, a 9 × 9 pixel mask represented by the above equation (17) is set to 1.0. Thus, the same effect as the low-pass filter processing was obtained.
Edge detection is performed using the 3 × 3 pixel array shown in FIG. 2 by the local dispersion method according to the above equation (20)._E(x, y) was nonlinearly transformed using the above equation (22). W_EThreshold of edge local variance when calculating_TThe value of was 30.
[0060]
In the granularity / edge identification / separation process, the granularity discrimination threshold G is used using the nonlinear transformation LUT shown in FIG._TThe value was set to 50 (8-bit gradation step unit of 0 to 255, about 100 times the density value). Gaussian random numbers were used in the granular mottle subdivision process. In addition, the values of the parameters α and β in the above equation (40) for obtaining the final processed image are α = 0.7 and β = 0.3, respectively.
[0061]
Thus, the three-dimensional density profile of the image to which the image processing method of the present invention is applied is the original image I.₀To final processed image I₁One scene is shown in FIGS. 5A to 9B. FIG. 5A shows the original image I.₀FIG. 5B shows the subject edge E.₀FIG. 6A shows a sharpness-enhanced image I._SFIG. 6B shows a smoothed image I._AVFIG. 7A shows a grain / edge mixed component ΔI._EGFIG. 7B shows the weighted data W of the granular area._GFIG. 8A shows the granular component G separated from the granular / edge mixed component.₀FIG. 8B shows the subdivided granular component G.₁FIG. 9A shows the weighting data W of the edge region._EFIG. 9B shows the final processed image I.₁3 shows a three-dimensional concentration profile.
[0062]
As is clear from these figures, the final processed image I shown in FIG.₁Is the original image I shown in FIG.₀The sharpness-enhanced image I shown in FIG._SThe graininess is sharpness-enhanced image I in spite of being sharply enhanced almost as much as_SCompared to the original image I₀It can be seen that the level is suppressed to approximately the same level. That is, it can be seen that the image processing method of the present invention is a high-quality image in which only the grain is suppressed and only the edge portion is sharpened.
[0063]
In addition, the image processing method of the present invention is applied to main silver salt color photographic light-sensitive materials, that is, photographic images (35 mm, new photographic system APS (advanced photo system), LF panorama, instant) taken on color films and black-and-white films. As a result, it was possible to obtain a remarkable improvement effect that can be understood at a glance in terms of both graininess and sharpness.
In particular, the granularity has a processing effect comparable to the granular improvement by making the photosensitive material finer, so it is a “blurred granular” unnaturalness that has been a drawback of various granular removal processing methods based on conventional averaging and reduction of fluctuations. There was no sense of incongruity. In addition, with regard to sharpness, when combined with the above-described granular suppression, a considerably greater emphasis effect was obtained than conventional unsharp masks and Laplacian filters.
[0064]
The image processing method for noise suppression and sharpness enhancement of a digital image according to the present invention has been described in detail with reference to examples, but the present invention is not limited thereto, and various modifications can be made without departing from the spirit of the present invention. Of course, improvements and design changes may be made.
[0065]
【The invention's effect】
As described above in detail, according to the present invention, a sharpness-enhanced granular component of the sharpness-enhanced image obtained by enhancing the sharpness from the original image is identified, and the granular component is further subjected to random modulation or a fine regular pattern. Since granularity is suppressed by subtracting the modulated granular component from the sharpness-enhanced image, it is possible to realize visually natural granularity suppression that is spatially finer than the original granularity and that has less variation in shading. Can do. In addition, according to the present invention, the grain is sharpened and spatially refined, so the silver salt photographic material has a fine grain as obtained when a fine grain emulsion is used, and the conventional smoothing is used. A natural grain suppression effect without visual discomfort and discomfort such as blurred grain, which is a drawback of the law, can be obtained.
In addition, by applying the method of the present invention to a silver salt color photographic light-sensitive material, there is no so-called “blurred grain” unnaturalness and uncomfortable feeling, which is a disadvantage of the conventional grain-suppressed sharpness enhancement processing method. The emphasis is improved at the same time, and an extremely remarkable improvement effect can be obtained, which has a large industrial effect.
[Brief description of the drawings]
FIG. 1 is a flowchart showing an embodiment of an image processing method for noise suppression and sharpness enhancement of a digital image according to the present invention.
FIG. 2 is an explanatory diagram for explaining an example of movement of a square pixel array in edge detection by the local dispersion method of the image processing method of the present invention.
FIG. 3 is an explanatory diagram showing an example of characteristics of a frequency distribution of density difference components between a grain and an edge used for explaining identification / separation of the grain and an edge in the image processing method of the present invention.
FIG. 4 is an explanatory diagram showing an example of a look-up table (LUT) function used for separation of granular components in the image processing method of the present invention.
FIGS. 5A and 5B are diagrams showing examples of a three-dimensional density profile of an original image and an edge detection image of a certain scene obtained in an example of the present invention, respectively.
FIGS. 6A and 6B are diagrams showing examples of a three-dimensional density profile of a sharpness-enhanced image and a smoothed image of a scene obtained in an example of the present invention, respectively.
FIGS. 7A and 7B are diagrams showing an example of a three-dimensional density profile of a granular / edge mixed component of a scene and weighted data of a granular region obtained in an example of the present invention, respectively.
FIGS. 8A and 8B are diagrams showing examples of three-dimensional density profiles of a granular component and a subdivided granular component of a scene obtained in an example of the present invention, respectively.
FIGS. 9A and 9B are diagrams showing an example of the weight data of the edge region of the scene and the three-dimensional density profile of the final processed image obtained in the example of the present invention, respectively.

Claims (3)

Create sharpness-enhanced image data by performing sharpness-enhancement processing on the original image data,
Smoothing the original image data to create smoothed image data;
Subtracting this smoothed image data from the sharpness-enhanced image data to extract a mixed component of edges and noise,
Perform edge detection from the original image data to obtain weighted data of the edge area and noise area,
After performing weighting using the weighting data of this noise area to the mixed component of the edge and noise, the noise component is separated by performing nonlinear transformation,
The obtained noise component is subdivided to obtain a subdivided noise component,
The edge enhancement component is obtained by performing weighting using the weighting data of the edge region to the mixed component of the edge and noise,
In order to suppress noise and enhance sharpness in a digital image, the processed image data is obtained by scaling and removing the subdivided noise component from the sharpness weighted image data and scaling and adding the edge weighted component. Image processing method. 2. The digital image noise according to claim 1, wherein the non-linear conversion for separating the noise component is performed by a function that continuously decreases exponentially as the density fluctuation becomes larger than a threshold value of noise density fluctuation. Image processing method for suppression and sharpness enhancement. The image processing method for noise suppression and sharpness enhancement of a digital image according to claim 1, wherein the subdividing process is a process of multiplying the noise component by a random number.

Images