JP2004318693A - Image processing method, image processor, and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To satisfactorily remove an influence of any defective part from image information. <P>SOLUTION: Image information acquired from an image acquiring part 14 includes the image information of a visible area and an infrared area. An image processing part 11 acquires the image defective area of any scratch or trash from the image information of the infrared area, and corrects the image information of the visible area based on the correction value of the infrared ray, evaluates the validity/invalidity of the correction result, and executes various processing to the image defective area of the visible area based on the evaluation result. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

【００９１】
（鮮鋭性強調、あるいは平滑化フィルタ）
鮮鋭性強調、あるいは平滑化フィルタの場合、中央部５＊５画素の情報を用いて、下記の計算結果を得る。

但し、ｆｉｌｄａｔ［１］〜［７］は、所定の定数である。
【００９２】
そして、ＦＰ１に、下記の制限を設ける。

【００９３】
そして、下記の式により、新たな中央画素値、ｐ５５’を得る
ｐ５５’＝ｐ５５＋ＦＰ１
【００９４】
この処理例は、銀塩フィルムからの画像処理で特に好ましい処理例で、５＊５画素のエッジ強調フィルタの実施例である。ｄｉｖｄａｔを大きくすることによって鮮鋭性強調フィルタの効きが弱くなり、小さくすることによって、鮮鋭性強調フィルタの効きが強くなる。上側制限値、下側制限値を小さく設定すると、ごま塩（独立点の）ノイズが極端に強調される不具合を軽減し、滑らかな調子再現が得られ、制限値を大きく又は制限を設けなければ、自然なエッジ強調効果が得られる。
【００９５】
また、ｆｉｌｄａｔの値を変えることにより、平滑化フィルタとしても機能する。閾値の設定が必要ない場合は、下記の式が利用可能で、平滑化フィルタが簡単に設計できる。
【００９６】
中央部５＊５画素の情報を用いて、下記の計算結果を得、新たな中央画素値、ｐ５５’を得る。

但し、ｆｉｌｄａｔ［１］〜［７］は、所定の定数である。
これら各種パラメータを補正量として定義することができ、目的に応じ、領域ごとに変更可能でである。
【００９７】
（バンドカットフィルタ）
バンドカットフィルタの場合、９＊９画素の画像を用いて、下記の計算結果を得る。

【００９８】
そして、ＦＰ２に下記の制限を設ける。

そして、下記の式で、新たな中央画素値、ｐ５５’を得る
ｐ５５’ ＝ ｐ５５ ＋ ＦＰ２
【００９９】
この処理例は、９＊９画素の、バンドカットフィルタの実施例である。閾値２を大きくすると、ターゲットとなる空間周波数帯域の信号除去効果が大きくなり、信号の低周波変動が強く押さえられます。これら各種パラメータを補正量として定義することができ、また、周辺データの参照範囲を変えて特性の調整ができ、領域ごと、あるいは機種ごとに変更可能である。
【０１００】
（ノイズ補正）
ノイズ補正において、画像の番号付けを、下記の表２に示すように行う。表２は、画像情報の画素の信号値及びその画素の位置関係を示す表である。但し、Ｐ：中央画素、Ｘ、Ｙ：周辺画素である。
【表２】

【０１０１】
Ｘ方向に付いて、一番内側の組み合わせについて、
ａｂｓ（Ｘ１’＋Ｘ１−２＊Ｐ）＜閾値
を満たす場合、Ｘ１、Ｘ１’を平均化するデータ群に加える。
そして、一つ外側、その次・・と、上記判定式が成り立たなくなるところまで、または、あらかじめ初期値として指定された最大半径（例えば４画素）まで繰り返す。
【０１０２】
また、Ｙ方向についても同様に繰り返す。
そして、データ群に加えられたデータと、中央画素Ｐとの単純平均とを求め、新しい中央画素Ｐ’とする。
【０１０３】
上記方式で、閾値を大きく設定すると、ノイズ除去効果が大きくなり、一方、細かなディテールは消えてしまう場合がある。また、「あらかじめ初期値として指定された最大半径」を切り替えて、どの程度の大きさのノイズまで除去できるか変化させる。この例では上記の２つのパラメータがあり、領域ごとに設定値を変えることができる。
本実施の形態のステップＳ１３９では、その他のノイズフィルタの一例として、モルフォロジー処理の一形態として知られている、収縮処理（エロージョン、クロージング。以下、クロージングとする）を実施し、孤立点の除去を行う手法を利用している。
【０１０４】
傷ゴミ候補領域検出処理の続きを説明する。ノイズ処理された赤外差分データは、赤外画像上の傷ゴミ領域を示し、その領域を可視画像に適用するために、赤外画像情報の傷ゴミ領域拡張処理がなされる（ステップＳ１３Ａ）。赤外画像情報上の傷ゴミ領域に、可視画像情報上の傷ゴミ領域が確実に含まれるように所定量拡張される。傷ゴミ領域拡張処理は、モルフォロジー処理の一形態として知られている、膨張処理（ダイレーション、オープニング。以下、オープニングとする）を必要量実施する。前述のノイズフィルタに平滑化や、他のノイズフィルタを利用した場合は、信号値の再２値化の際、閾値の調整を行っても、傷ゴミ領域拡張処理と同様の処理が可能である。
【０１０５】
そして、拡張された傷ゴミ領域に含まれない画素に対応する可視画像情報の画素は正常画素とされ（ステップＳ１３Ｂ）、傷ゴミ候補領域検出処理が終了される。可視画像情報の正常画素でない画素領域が、可視画像情報の傷ゴミ候補領域とされる。傷ゴミ候補領域検出処理の終了後、次のステップ（図４のステップＳ１４）に移行される。
【０１０６】
なお、傷ゴミ候補領域の決定法は１方法に限られるものではない。本実施の形態のように赤外画像を用いるほかにも、反射原稿スキャナで表面反射画像を採取し、その表面不連続性から傷ゴミ候補領域を求めてもよい。この手法を行う反射原稿スキャナについては後述する。その他、透過原稿の場合に、照射光源の光質を切り替えて、例えば透過光と反射光、あるいは、フィルムに概平行光束を照射する集光光源と、拡散ボックスを利用して、柔らかな光をフィルムに当てる拡散光源を交互に切り替え、画像採取して、その差から傷ゴミ候補領域を検出してもよい。
【０１０７】
次に、図７を参照して、図４の傷ゴミ処理中のステップＳ１４の傷ゴミ領域決定処理を説明する。図７は、傷ゴミ領域決定処理を示すフローチャートである。傷ゴミ領域決定処理は、可視画像情報の傷ゴミ候補領域から正常画素を除外して傷ゴミ領域を決定する処理である。
【０１０８】
図７に示すように、先ず、ステップＳ１３１と同様に可視画像情報が取得される（ステップＳ１４１）。その取得された可視画像情報から一つの画素が注目画素として抽出される（ステップＳ１４２）。そして、注目画素が可視画像情報の傷ゴミ候補領域内にあるか否かが判別される（ステップＳ１４３）。注目画素が可視画像情報の傷ゴミ候補領域内にある場合（ステップＳ１４３；ＹＥＳ）、注目画素の近傍領域の正常画素が抽出される（ステップＳ１４４）。そして、その抽出された正常画素と注目画素との関連が評価される（ステップＳ１４５）。そして、注目画素と近傍の正常画素との関連性が高いか否かが判別される（ステップＳ１４６）。
【０１０９】
注目画素と、近傍の正常画素との関連評価は様々なものが考えられる。例えば、近傍の正常画素と注目画素との信号値差の絶対値が、所定閾値内に収まる画素を抽出し、その数が所定数以上存在すれば正常画素とすることができる。また、前記所定閾値は固定値でも良いが、例えば画像処理後段にノイズフィルタが機能している場合、その強度（ノイズフィルタが閾値を保持している場合は、閾値の大きさ）に応じて決定してもよい。このようにすると、微小な傷の影響は後段のノイズフィルタが消去するので、傷ゴミ処理をここで掛ける必要は無く、傷ゴミ処理が必要な領域を最低限に押さえる事ができ、画像処理能力が向上する。
【０１１０】
関連性が高くない場合（ステップＳ１４６；ＮＯ）、その注目画素は傷ゴミ領域内であると決定される（ステップＳ１４７）。そして、ステップＳ１４２において全ての画素が抽出されたか否かが判別される（ステップＳ１４８）。全ての画素が抽出された場合（ステップＳ１４８；ＹＥＳ）、傷ゴミ領域決定処理は終了される。
【０１１１】
注目画素が可視画像情報の傷ゴミ候補領域内にない場合（ステップＳ１４３；ＮＯ）、又は関連性が高い場合（ステップＳ１４６；ＹＥＳ）、その注目画素は正常画素とされ（ステップＳ１４９）、ステップＳ１４８に移行される。全ての画素が抽出されていない場合（ステップＳ１４８；ＮＯ）、ステップＳ１４２に移行される。傷ゴミ領域決定処理の終了後、次のステップ（図４のステップＳ１５）に移行される。
【０１１２】
次に、図８を参照して、図４の傷ゴミ処理中のステップＳ１１〜Ｓ１５に対応する傷ゴミ補正処理の一実施例としての第１の傷ゴミ補正処理を説明する。図８は、第１の傷ゴミ補正処理を示すフローチャートである。先ず、ステップＳ１５１〜Ｓ１５９は、図６の傷ゴミ候補領域検出処理のステップＳ１３１〜Ｓ１３９と同様である。そして、ノイズ処理された赤外差分データ上の傷、ゴミ部分が傷ゴミ領域として設定される（ステップＳ１６０）。ここでは、図６のステップＳ１３Ａのような領域拡張は行わない。
【０１１３】
そして、ステップＳ１６０で求めた傷ゴミ領域について、赤外差分データを用いて赤外画像情報上の傷、ゴミの影響を除去するための補正値が算出される（ステップＳ１６１）。そして、図６及び図７に示す傷ゴミ候補領域検出処理及び傷ゴミ領域決定処理が実行される（ステップＳ１６２）。
【０１１４】
そして、可視画像情報上の１つの画素が注目画素として抽出される（ステップＳ１６３）。そして、その注目画素が可視画像情報の傷ゴミ領域内であるか否かが判別される（ステップＳ１６４）。注目画素が可視画像情報の傷ゴミ領域内でない場合（ステップＳ１６４；ＮＯ）、ステップＳ１６９に移行される。注目画素が可視画像情報の傷ゴミ領域内である場合（ステップＳ１６４；ＹＥＳ）、注目画素の近傍領域に存在する傷ゴミ領域内の赤外補正値（ステップＳ１６１で求められた赤外画像情報の補正値）が複数抽出される（ステップＳ１６５）。
【０１１５】
そして、抽出された複数の赤外補正値の代表値が算出される（ステップＳ１６６）。具体的には、複数の赤外補正値の単純平均、各補正値の注目画素からの距離に基づいた重み付け平均、あるいは抽出された赤外補正値が多い場合には統計的処理によって異常データを排除し、残ったデータから前記平均などの処理がなされる。
【０１１６】
そして、算出された代表値が注目画素の傷ゴミ補正値に設定される（ステップＳ１６７）。そして、設定された傷ゴミ補正値を用いて、可視画像情報の傷ゴミ補正がなされる（ステップＳ１６８）。そして、ステップＳ１６８において全画素が抽出されたか否かが判別される（ステップＳ１６９）。全画素が抽出された場合（ステップＳ１６９；ＹＥＳ）、第１の傷ゴミ補正処理は終了される。全画素が抽出されていない場合（ステップＳ１６９；ＮＯ）、ステップＳ１６３に移行され、未抽出の画素が次の注目画素として抽出される。第１の傷ゴミ補正処理の終了後、次のステップ（図４のステップＳ１６）に移行される。
【０１１７】
第１の傷ゴミ補正処理によれば、赤外の傷ゴミ領域と、可視の傷ゴミ領域とを別途定め、その結合を近傍信号を元に行うので、赤外画像と可視画像とに若干のずれがあっても、またさらに、そのずれが画面局部各々で不連続に異なっていても、異常補正の危険が少ない補正が可能な手法である。
【０１１８】
次に、図９を参照して、図４の傷ゴミ処理中のステップＳ１５に対応する傷ゴミ補正処理の一実施例である第２の傷ゴミ補正処理を説明する。図９は、第２のゴミ補正処理を示すフローチャートである。傷ゴミ補正処理及び第１の傷ゴミ補正処理は、赤外画像情報から、傷、ゴミの影響による信号の減衰率を求め、この逆数を、対応する可視画像情報に掛けるというものである。この考え方は、赤外画像情報が傷、ゴミから受ける影響と、可視画像情報が受ける影響とが、１：１の関係にあるという前提で成り立っている。
【０１１９】
しかし、実際の画像処理システムに於いては、このような状況は存在せず、主には以下の各要素によって関係の比率が変化する。
１：撮像系のＭＴＦ（Ｍｏｄｕｌａｔｉｏｎ Ｔｒａｎｓｆｅｒ Ｆｕｎｃｔｉｏｎ：正弦波入力像に対する出力像のコントラスト比）が色光によってそれぞれ異なる。
２：撮像系のフレア特性が、色光によってそれぞれ異なる。
３：信号処理系のコントラスト特性が、色光によってそれぞれ異なる。
【０１２０】
したがって、これら影響を考慮して、赤外画像情報が受ける影響から、可視画像情報が受けているであろう影響を算出する修正値を定めて、補正値を修正する必要がある。簡単にいうと、修正値をＢＧＲ各々の色光について所定の定数として設定しても良いが、前述関係の比率は、その発生要因からも傷ゴミ領域の形態によって変化するものであることは明白であり、以下説明するように、傷ゴミ領域サイズに対応した補正値として定める事が望ましい。
【０１２１】
先ず、取得画像情報からＢ（青）画像を示すＢ画像情報、Ｇ（緑）画像を示すＧ画像情報及びＲ（赤）画像を示すＲ画像情報のうちの何れか一つが選択され（選択色画像情報とする）、その選択色画像情報中の画素の一つが注目画素として抽出される（ステップＳ２５１）。
【０１２２】
そして、注目画素が可視画像情報の傷ゴミ領域内にあるか否かが判別される（ステップＳ２５２）。注目画素が可視画像情報の傷ゴミ領域内でない場合（ステップＳ２５２；ＮＯ）、ステップＳ２５８に移行される。傷ゴミ領域内の画素である場合（ステップＳ２５２；ＹＥＳ）、注目画素について、所属する傷ゴミ領域の大きさを示す領域サイズ情報が取得される（ステップＳ２５３）。例えば、注目画素を中心に、縦、横、斜めの４方向に線分を取り、その最短線分を領域サイズ情報とする。後述する図１１を例にとれば、領域サイズ情報は、水平方向の４画素となる。また、図１１に示すように領域の平均的大きさに応じて、検索方向の数を増加させてもよい。
【０１２３】
そして、傷ゴミ領域内の注目画素の赤外補正値が算出される（ステップＳ２５４）。例えば、図８の第１の傷ゴミ補正処理などを参考に、次式により赤外画像情報から赤外画像補正値が算出される。
（赤外画像補正値）＝（赤外基準データ）／（赤外画素値、又はノイズ処理を施した赤外画素値）
但し、信号単位は、透過エネルギー量とする。
【０１２４】
そして、注目画素の周辺領域が設定され、その周辺領域内の選択色画像情報の信号の平均値と、注目画素の信号の平均値とから、仮の可視画像補正値が算出される（ステップＳ２５５）。ここで、周辺領域を後述する図１５に示すごとく定義して、先ほど求めた領域サイズ情報を設定した線分と重なる周辺領域内の画素が抽出され、その画素値の加算平均が取られる。この加算平均と注目画素の可視信号値とから、次式により可視画像補正値が取得される。
（可視画像補正値）＝（加算平均値）／（注目画素の可視信号値）
但し、信号単位は、透過エネルギー量とする。
【０１２５】
そして、ステップＳ２５５で算出された領域サイズ情報と、ステップＳ２５４で算出された可視画像情報とから、仮の修正値が算出される。例えば、赤外画像補正値と可視画像補正値から、次式により仮の修正値が求められる。
（仮の修正値）＝（可視画像補正値）／（赤外画像補正値）
【０１２６】
そして、領域サイズ情報と、仮の修正値との関係がプロット図にプロットされる（ステップＳ２５７）。プロット図は、プログラム上のプロット図であり、コンピュータプログラム実行時に仮想的に生成されるものである。そして、現在選択されている選択色画像情報について、全画素が注目画素として抽出されたか否かが判別される（ステップＳ２５８）。全画素が抽出されていない場合（ステップＳ２５８；ＮＯ）、ステップＳ２５１に移行され、同一選択色画像情報内の未抽出の画素、あるいは未抽出の色画像情報内の未抽出の画素が注目画素として抽出される。
【０１２７】
全画素が抽出された場合（ステップＳ２５７；ＹＥＳ）、現在選択されている選択画像のプロット図を元に代表曲線が生成される（ステップＳ２５９）。代表曲線生成の具体的手法は限定しないが、各種の統計手法を用い、異常点が除去でき、誤差最小曲線を求めればよい。以上の手順を複数の色光（ＲＧＢ）それぞれについて実施し、代表曲線を求める。本実施の形態のように、代表曲線を複数（ＢＧＲ）の色光各々に付いて求めれば、サイズ情報に対応した修正値を簡単に求めることができる。
【０１２８】
以上のように、代表曲線は、領域サイズ情報と修正値との関連関数として設定される（ステップＳ２６０）。そして、代表曲線が全ての選択色画像情報（Ｂ，Ｇ，Ｒ）について生成されたか否かが判別され（ステップＳ２６１）。代表曲線が全て生成されていない場合（ステップＳ２６１；ＮＯ）、ステップＳ２５１に移行され、全ての代表曲線の生成が完了するまで繰り返される。
【０１２９】
ここまでの処理が終了し、全ての代表曲線が生成された場合（ステップＳ２６１；ＹＥＳ）、次のブロック（のフロー）に移行される。先ず、可視画像情報内の一画素が注目画素として抽出される（ステップＳ２６２）。そして、注目画素が可視画像情報の傷ゴミ領域内にあるか否かが判別される（ステップＳ２６３）。注目画素が傷ゴミ領域内にない場合（ステップＳ２６３；ＮＯ）、ステップＳ２６９に移行される。
【０１３０】
注目画素が傷ゴミ領域内にある場合（ステップＳ２６３；ＹＥＳ）、ステップＳ２５３と同様にして、注目画素がある傷ゴミ領域の領域サイズ情報が取得される（ステップＳ２６４）。そして、注目画素のＢ信号、Ｇ信号、Ｒ信号の何れか一つが選択され、ステップＳ２５４と同様にして、その色信号（選択色信号とする）の赤外画像補正値が算出される（ステップＳ２６５）。そして、ステップＳＳ２６０において設定された代表曲線を用いて、ステップＳ２６４において取得された領域サイズ情報に対応する修正値が取得される（ステップＳ２６６）。
【０１３１】
そして、ステップＳ２６５において算出された赤外画像補正値と、ステップＳ２６６において取得された修正値との積が生成される（ステップＳ２６７）。その積を用いて、選択色信号に対応する色画像情報が補正される（ステップＳ２６８）。このステップＳ２６５〜Ｓ２６８の処理が全ての色信号（ＢＧＲ）について繰り返し実行される。
【０１３２】
そして、注目画素が、可視画像情報内の全画素から抽出されて、ステップＳ２６２〜Ｓ２６８の処理が全画素について実施されたか否かが判別される（ステップＳ２６９）。全画素について実施された場合（ステップＳ２６９；ＹＥＳ）、第２の傷ゴミ補正処理が終了される。全画素について実施されていない場合（ステップＳ２６９；ＮＯ）、ステップＳ２６２に移行され、ステップＳ２６２〜Ｓ２６９の処理が繰り返される。第２の傷ゴミ補正処理の終了後、次のステップ（図４のステップＳ１６）に移行される。
【０１３３】
なお、第２の傷ゴミ補正処理では、代表曲線を色画像情報ごとに求めるとしているが、状況に応じて、システム共通のデータとしてあらかじめ求めてもよい。さらに、ある程度以上の長さ（領域サイズは、領域最短幅のため、形態的には細長い傷、ゴミ領域ということになる）を有する傷ゴミ領域各々について独立に代表曲線を求めることもできる。
【０１３４】
次に、図１０〜図１３を参照して、図４の傷ゴミ処理中のステップＳ１８の傷ゴミ補間処理に用いられる好ましい一例を説明する。図１０は、傷ゴミ補間処理を示すフローチャートである。図１１は、注目画素の補間処理候補選択を示す図である。図１２は、注目画素の補間処理候補選択の３つの様態例を示す図である。図１３は、注目画素の２つの方向特性例を示す図である。
【０１３５】
先ず、傷ゴミ補正された可視画像情報から一つの画素が注目画素として抽出される（ステップＳ１８１）。そして、注目画素が可視画像情報の傷ゴミ領域内にあるか否かが判別される（ステップＳ１８２）。注目画素が傷ゴミ領域内にない場合（ステップＳ１８２；ＮＯ）、ステップＳ１８９に移行される。
【０１３６】
注目画素が傷ゴミ領域内にある場合（ステップＳ１８２；ＹＥＳ）、ＢＧＲのうちの一つの色画像情報が選択され、その色画像情報において、注目画素を中心に対向する正常画素対が傷ゴミ補間の候補データとして抽出される（ステップＳ１８３）。例えば、図１１に示すように、縦、横、斜めの４方向に対向する正常画素が抽出される。また、図１２（ａ）〜（ｃ）に示すように、傷ゴミ領域サイズ（対向画素の距離）に応じて抽出する方向数を増減してもよい。図１２では、対向画素間距離が大きくなるにつれて、方向数も増加されている。
【０１３７】
そして、抽出された対向画素同士の関連評価がなされる（ステップＳ１８４）。例えば、対向画素同士の信号値の差分データにより評価され、その差分データが小さいほど関連性が高くなる。そして、最も関連性の高い方向が抽出される（ステップＳ１８５）。ステップＳ１８３〜Ｓ１８５について、全ての色画像情報（ＲＧＢ画像情報）について繰り返し実行される。
【０１３８】
そして、ＢＧＲ画像情報それぞれで抽出された方向の関係が評価される（ステップＳ１８６）。その評価結果に基づき、補間元の画素を抽出する方向が定められ、その方向の画素対が抽出される（ステップＳ１８７）。例えば、図１３（ａ）に示すように、Ｇ及びＲ画像情報から求めた方向が同一で、Ｇ画像情報から求めた方向に重み付け６がなされ、Ｒ画像情報から求めた方向に重み付け３がなされ、その方向と垂直なＢ画像情報から求めた方向に重み付け１がなされる。この場合、ＢＧＲから求めた方向は重み付けが考慮されて、補間元の画素の取得方向が図の方向に決定される。ＢＧＲの補間方向を一致させないと補間後の画素に不自然な色がつく可能性があるからである。また、図１３（ｂ）に示すように各方向の重み付けに指向性が見られない場合、補間元の画素取得の方向性を特定の方向とせずに補間されるとしてもよい。
【０１３９】
上述の重み付けは、視感度を元に、具体的には、ＮＴＳＣ方式の色変換で用いるＢＧＲ画像の重み付けを元にしている。例えば、青は観察者の目に対する感度が低いために重み付けが低く設定され、緑は観察者の目に対する感度が高いために重み付けが高く設定されている。しかしこれに限るものではなく、ベクトル決定時の信頼度情報を別途求める構成でもよい。その信頼度情報は、例えば複数方向のなかで、当該方向を選んだ際に、選ばれなかった他のグループの選択指標との格差を、統計的に処理して信頼度を求めればよい。
【０１４０】
そして、抽出された画素対を用いて、注目画素の補間後の信号値が推定され、その推定された画素値に注目画素の信号値が置き換えられて補間される（ステップＳ１８８）。そして、ステップＳ１８１において、可視画像情報の全ての画素が注目画素として抽出されたか否かが判別される（ステップＳ１８９）。全ての画素が抽出された場合（ステップＳ１８９；ＹＥＳ）、傷ゴミ補間処理が終了される。全ての画素が抽出されていない場合（ステップＳ１８９；ＮＯ）、ステップＳ１８１に移行され、未抽出の画素が次の注目画素として抽出される。傷ゴミ補間処理の終了後、次のステップ（図３のステップＳ２）に移行される。
【０１４１】
次に、図１４及び図１５を参照して、図４の傷ゴミ処理中のステップＳ１５〜Ｓ１８に対応する傷ゴミ補正補間処理を説明する。図１４は、傷ゴミ補正補間処理を示すフローチャートである。図１５は、傷ゴミ領域及びその周辺領域を示す図である。
【０１４２】
先ず、既に求められた可視画像情報の傷ゴミ領域を用いて、可視画像情報の傷ゴミ補正前の傷ゴミ領域特性値が算出され、傷ゴミ領域の近傍画素の領域が周辺領域とされる（ステップＳ２８１）。例えば、図１５に示すように、近傍画素の所定範囲を１ピクセルとして、傷ゴミ領域の近傍の正常画素の一部を周辺領域とする。特性値は、所定画素の所定距離内にある傷ゴミ領域内の画素データを用いて算出した特性値であり、ステップＳ２８１では傷ゴミ領域内の全画素について算出される。傷ゴミ領域特性値は、各画素を中心とした、所定距離内にある傷ゴミ領域内の画素データを用いて算出した特性値であり、例えば、所定距離内で傷ゴミ領域内の各画素の画素値の平均値とされる。また、傷ゴミ領域特性値の算出は統計的手法によってもよい。
【０１４３】
そして、可視画像情報に傷ゴミ補正処理が実施される（ステップＳ２８２）。例えば、図９に示した赤外画像補正値を用いた傷ゴミ補正処理である。そして、可視画像情報内の一画素が注目画素として抽出される（ステップＳ２８３）。そして、その抽出された注目画素が傷ゴミ領域内にあるか否かが判別される（ステップＳ２８４）。
【０１４４】
注目画素が傷ゴミ領域内にない場合（ステップＳ２８４；ＮＯ）、ステップＳ２８９に移行される。注目画素が傷ゴミ領域内にある場合（ステップＳ２８４；ＹＥＳ）、注目画素に対応する、傷ゴミ補正後の傷ゴミ領域特性値と、周辺領域特性値とが算出される（ステップＳ２８５）。傷ゴミ領域特性値は、注目画素を中心とした、所定距離内にある傷ゴミ領域内の画素データを用いて算出した特性値である。周辺領域特性値は、注目画素を中心とした、所定距離内にある周辺領域の画素データを用いて算出した特性値とである。それぞれの特性値は、前述のように、所定距離内にある各領域内の画素の画素値の平均値や、統計的手法により算出される。
【０１４５】
そして、ステップＳ２８１において算出された補正前の傷ゴミ領域特性値の注目画素に対応する値、ステップＳ２８５において算出された補正後の傷ゴミ領域特性値及び周辺領域特性値とから、補正前と補正後との特性値間格差（傷ゴミ領域内特性値−周辺領域特性値）がそれぞれ算出され、補正前の特性値間格差と補正後の特性値間格差との関係が評価される（ステップＳ２８６）。そして、補正前の特性値間格差に比べて補正後の特性値間格差が十分縮小された否かが判別される（ステップＳ２８７）。例えば、補正後の特性値間格差が補正前の特性値間格差よりも所定値以上低下したか否かにより判別される。評価基準は、実際に複数サンプル画像で評価を行い、好ましく調整すればよい。
【０１４６】
補正後の特性値間格差が十分縮小された場合（ステップＳ２８７；ＹＥＳ）、ステップＳ２８９に移行される。補正後の特性値間格差が十分縮小されていない場合（ステップＳ２８７；ＮＯ）、注目画素について傷ゴミ補間処理が実施される（ステップＳ２８８）。例えば、図１０に示す傷ゴミ補間処理である。そして、ステップＳ２８３において可視画像情報内の全画素が注目画素として抽出されたか否かが判別される（ステップＳ２８９）。全画素が抽出された場合（ステップＳ２８９；ＹＥＳ）、傷ゴミ補正補間処理が終了される。全画素が抽出されていない場合（ステップＳ２８９；ＮＯ）、ステップＳ２８３に移行され、未抽出の画素が次の抽出画素として抽出される。傷ゴミ補正補間処理の終了後、次のステップ（図３のステップＳ２）に移行される。
【０１４７】
以上の傷ゴミ補正補間処理は、自動的に補正評価されるものである。なお、傷ゴミ補正補間処理では、傷ゴミ処理内の傷ゴミ補正処理を先に行い、その後、傷ゴミ補間処理を実施している。この際、傷ゴミ補間処理を実施するにあたって、傷ゴミ補正処理により補正された画素を正常画素として扱う事が望ましい。即ち、傷ゴミ補間処理の必要な領域近傍に、傷ゴミ補正処理が必要な領域が存在する場合に、まず傷ゴミ補正処理によって補正した画素を正常画素とすることによって、傷ゴミ補間処理に必要な周辺情報を有効に増大させる事が可能になる。この手法は、例えば、非常に広い傷ゴミ領域が存在する場合に、非常に有効な技術となる。
【０１４８】
ここで、傷ゴミ補正処理及び傷ゴミ補間処理の実施の具体例を３つ説明する。
（第１の例）
まず、非常に広い傷、ゴミ領域を赤外画像情報を用いて認識する。さらに、該領域内のテクスチャ構造を赤外画像情報を用いて評価し、均質部と非均質部とに分割する。均質部は、傷、ゴミの影響が安定して現れているところであり、この部分を赤外画像情報を元に傷ゴミ補正処理する。非均質部は、前述の補正された画素も正常画素と考えて、傷ゴミ補間処理を実施する。
【０１４９】
この傷ゴミ補正処理と傷ゴミ補間処理との組み合わせでは、安定領域で赤外画像を用いた補正処理が実施されるので、広い面積に渡って略良好な補正結果が得られる。また、２次効果として、赤外画像情報が可視画像情報と正確に位置対応が取れていなくても、安定補正領域であることから補正性能が劣化することが無く、補正性能劣化の可能性がある非均質領域は、補間処理を実施するので、副作用の発生が少ない処理結果が得られる。
【０１５０】
（第２の例）
まず、非常に広い傷ゴミ領域を、赤外画像情報を用いて認識する。さらに、該領域内のテクスチャ構造を、赤外画像情報を用いて評価し、均質部と非均質部とに分割する。均質部は、傷、ゴミの影響が安定して現れているところであり、可視画像情報のみで補正する事が簡単な領域である。非均質部は、可視画像情報による補正が比較的難しい部分であり、赤外画像情報と可視画像情報との十分な位置整合を取った上で、傷ゴミ補正処理を実施する。非均質部を傷ゴミ補正処理後、可視画像情報を用いた均質部の補間、または補正処理を実施する。
【０１５１】
この傷ゴミ補正処理と傷ゴミ補間処理との組み合わせでは、赤外画像情報と可視画像情報との十分な位置整合を取る必要があるが、それが可能な状態であれば、非均質領域の傷ゴミ補正が効果的に行える。また、均質領域は可視による補正処理のしやすい領域であり、実際の画像情報を元に処理を行うので、強力な補正結果が得られる。可視画像情報を用いた傷ゴミ補正処理の例は、後述する。
【０１５２】
また、さらに、傷ゴミ補間処理を実施する際に、周辺領域を大局的に調査し、その調査結果に基づいて傷ゴミ補間処理する場合も有効である。この場合は、傷ゴミ補正処理する傷ゴミ領域と、傷ゴミ補間処理する傷ゴミ領域とが接していない場合でも効果を発揮する。
【０１５３】
第１及び第２の例によれば、赤外画像情報の画像欠陥領域内のテクスチャ構造の特徴に応じて、可視画像情報に対して好ましい補正方法を選択して実行できる。また、例えば、画像情報取得元の画像記録層の欠損による画像欠損が発生し、赤外画像情報による信号強度の補正が逆補正となってしまう不具合にも、好ましい補正を選択して実行し、有効に対処できる。また、赤外画像情報の残存収差や、赤外画像情報と可視画像情報との間に色ずれ（位置ずれ）が存在してもこれに対応した良好な画像処理が可能になる。
【０１５４】
（第３の例）
先ず、補正処理を行う傷ゴミ領域について、傷ゴミ補正処理する（正常画素に分類しなおす）。その正常画素のみを対象に像構造を解析する。例えば、画像を複数の空間周波数帯域に分割し、分析する解像度変換の手法としてウェーブレット変換を用いることができる。以下、図１６〜図２４を参照して、ウェーブレット変換、二項ウェーブレット変換について簡単に説明する。
【０１５５】
画像の局所部位毎に周波数帯域の分割を行い、周波数帯域別に抑制・強調を行なう効率的な方法として、ウエーブレット変換を応用する技術が知られている。ウエーブレット変換の詳細については、例えば”Ｗａｖｅｌｅｔ ａｎｄ Ｆｉｌｔｅｒ Ｂａｎｋｓ” ｂｙ Ｇ． Ｓｔｒａｎｇ ＆ Ｔ． Ｎｇｕｙｅｎ， Ｗｅｌｌｅｓｌｅｙ−Ｃａｍｂｒｉｄｇｅ Ｐｒｅｓｓ（邦訳 「ウエーブレット解析とフィルタバンク」， Ｇ．ストラング・Ｔ．グエン共著， 培風館 ）や、”Ａ ｗａｖｅｌｅｔ ｔｏｕｒ ｏｆ ｓｉｇｎａｌ ｐｒｏｃｅｓｓｉｎｇ ２ｅｄ．” ｂｙ Ｓ．Ｍａｌｌａｔ， Ａｃａｄｅｍｉｃ Ｐｒｅｓｓ に記載されているが、ここでは概要を説明する。ここで挙げた文献を第１のウエーブレット文献とする。
【０１５６】
ウェーブレット変換とは、図１６に例示されるような有限範囲で振動するウェーブレット関数（下記式（１））を用いて、入力信号ｆ（ｘ）に対するウェーブレット変換係数〈ｆ、ψ_ａ，ｂ〉を、下記式（２）のように求めることにより、下記式（３）で示されるウェーブレット関数の総和に分解する変換である。
【数１】

【数２】

【数３】

【０１５７】
上記式（１）〜（３）において、ａはウェーブレット関数のスケールを表し、ｂはウェーブレット関数の位置を示す。図１６に例示するように、スケールａの値が大きいほどウェーブレット関数ψ_ａ，ｂ（ｘ）の周波数は小さくなり、位置ｂの値に従ってウェーブレット関数ψ_ａ，ｂ（ｘ）が振動する位置が移動する。従って、上記式（３）は、入力信号ｆ（ｘ）を、種々のスケールと位置を有するウェーブレット関数ψ_ａ，ｂ（ｘ）の総和に分解することを意味している。
【０１５８】
上述のような変換を可能にするウェーブレット関数は多くのものが知られているが、画像処理分野では計算が高速な直交ウェーブレット（ｏｒｔｈｏｇｏｎａｌ ｗａｖｅｌｅｔ）変換、双直交ウェーブレット（ｂｉｏｒｔｈｏｇｏｎａｌ ｗａｖｅｌｅｔ）変換が広く用いられている。以下、直交ウェーブレット・双直交ウェーブレットの変換計算の概要を説明する。
【０１５９】
直交ウェーブレット変換及び双直交ウェーブレット変換のウェーブレット関数は、下記式（４）のように定義される。
【数４】

但し、ｉは自然数である。
【０１６０】
式（４）と式（１）を比べると、直交ウェーブレット変換、双直交ウェーブレット変換においては、スケールａの値が２のｉ乗で離散的に定義され、また位置ｂの最小移動単位が２^ｉで離散的に定義されていることがわかる。このｉの値はレベルと呼ばれる。
【０１６１】
レベルｉを有限な上限Ｎまでに制限すると、入力信号ｆ（ｘ）は、下記式（５）〜（７）のように表される。
【数５】

【数６】

【数７】

【０１６２】
式（５）の第２項は、レベル１のウェーブレット関数ψ_１，ｊ（ｘ）の総和で表せない残差の低周波数帯域成分を、レベル１のスケーリング関数φ_１，ｊ（ｘ）の総和で表したものである。スケーリング関数はウェーブレット関数に対応して適切なものが用いられる（第１のウエーブレット文献を参照）。式（５）に示すレベル１のウェーブレット変換によって、入力信号ｆ（ｘ）＝Ｓ_０は、レベル１の高周波数帯域成分Ｗ_１と低周波数帯域成分Ｓ_１に信号分解されたことになる。
【０１６３】
ウェーブレット関数ψ_ｉ，ｊ（ｘ）の最小移動単位は２^ｉゆえ、入力信号Ｓ_０の信号量に対して、高周波数帯域成分Ｗ_１と低周波数帯域成分Ｓ_１の信号量は各々１／２となり、高周波数帯域成分Ｗ_１と低周波数帯域成分Ｓ_１の信号量の総和は、入力信号Ｓ_０の信号量と等しくなる。レベル１の低周波数帯域成分Ｓ_１は、式（６）でレベル２の高周波数帯域成分Ｗ_２と低周波数帯域成分Ｓ_２に分解され、以下同様にレベルＮ迄の変換を繰り返すことで、入力信号Ｓ_０は、式（７）に示すように、レベル１〜Ｎの高周波数帯域成分の総和と、レベルＮの低周波数帯域成分の和に分解される。
【０１６４】
ここで、式（６）で示す１レベルのウェーブレット変換は、図１７に示すようなフィルタ処理で計算できることが知られている。図１７において、ＬＰＦはローパスフィルタ、ＨＰＦはハイパスフィルタを示している。ローパスフィルタＬＰＦとハイパスフィルタＨＰＦのフィルタ係数は、ウェーブレット関数に応じて適切に定められる（第１のウエーブレット文献を参照）。図１７において、２↓は、信号を１つおきに間引くダウンサンプリングを示す。
【０１６５】
図１７に示すように、入力信号Ｓ_ｎ−１を、ローパスフィルタＬＰＦとハイパスフィルタＨＰＦで処理して、信号を１つおきに間引くことにより、入力信号Ｓ_{ｎ− １}を、高周波帯域成分Ｗ_ｎと低周波帯域成分Ｓ_ｎに分解することができる。
【０１６６】
画像信号のような２次元信号における１レベルのウェーブレット変換は、図１８に示すようなフィルタ処理で計算される。図１８において、ＬＰＦｘ、ＨＰＦｘ、２↓ｘはｘ方向の処理を示し、ＬＰＦｙ、ＨＰＦｙ、２↓ｙは、ｙ方向の処理を示す。まず、入力信号Ｓ_ｎ−１をｘ方向のローパスフィルタＬＰＦｘ、ハイパスフィルタＨＰＦｘによりフィルタ処理を行い、ｘ方向にダウンサンプリングする。これにより、入力信号Ｓ_ｎ−１は、低周波帯域成分ＳＸ_ｎと高周波帯域成分ＷＸ_ｎに分解される。低周波帯域成分ＳＸ_ｎと高周波帯域成分ＷＸ_ｎの各々に対して、ｙ方向のローパスフィルタＬＰＦｙ、ハイパスフィルタＨＰＦｙによるフィルタ処理を行い、ｙ方向にダウンサンプリングする。
【０１６７】
この１レベルのウェーブレット変換により、低周波数帯域成分Ｓ_ｎ−１は、３つの高周波数帯域成分Ｗｖ_ｎ、Ｗｈ_ｎ、Ｗｄ_ｎと、１つの低周波数帯域成分Ｓ_ｎに分解される。分解で生成されるＷｖ_ｎ、Ｗｈ_ｎ、Ｗｄ_ｎ、Ｓ_ｎの各々の信号量は、分解前のＳ_ｎ−１に比べて縦横ともに１／２となるため、分解後の４成分の信号量の総和は、分解前のＳ_ｎ−１の信号と等しくなる。
【０１６８】
入力信号Ｓ_０が３レベルのウェーブレット変換で信号分解される過程を模式的に図１９に示す。図１９に示すように、レベル数が大きくなるにつれて、ダウンサンプリングにより画像信号が間引かれ、分解画像が小さくなっていくことがわかる。
【０１６９】
また、図２０に示すように、分解によって生成したＷｖ_ｎ， Ｗｈ_ｎ， Ｗｄ_ｎ， Ｓ_ｎに対し、フィルタ処理で計算されるウェーブレット逆変換を施すことにより、分解前の信号Ｓ_ｎ−１を完全再構成できることが知られている。図２０において、ＬＰＦ’は、逆変換用のローパスフィルタ、ＨＰＦ’は、逆変換用のハイパスフィルタを示している。また、２↑は、信号に１つおきにゼロを挿入するアップサンプリング処理を示す。また、ＬＰＦ’ｘ、ＨＰＦ’ｘ、２↑ｘは、ｘ方向の処理を示し、ＬＰＦ’ｙ、ＨＰＦ’ｙ、２↑ｙはｙ方向の処理を示す。
【０１７０】
図２０に示すように、Ｓ_ｎをｙ方向にアップサンプリング処理及びローパスフィルタＬＰＦ’_ｙによるフィルタ処理を施すことにより得られる信号と、Ｗｈ_ｎをｙ方向におけるアップサンプリング処理及びハイパスフィルタＨＰＦ’_ｙによるフィルタ処理を施すことにより得られる信号とを加算してＳＸ_ｎを得る。これと同様にして、Ｗｖ_ｎとＷｄ_ｎからＷＸ_ｎを生成する。
【０１７１】
さらに、ＳＸ_ｎをｘ方向においてアップサンプリング処理及びローパスフィルタＬＰＦ’_ｘによるフィルタ処理を施すことにより得られる信号と、ＷＸ_ｎをｘ方向においてアップサンプリング処理及びハイパスフィルタＨＰＦ’_ｘによるフィルタ処理を施すことにより得られる信号とを加算することにより、分解前の信号Ｓ_ｎ−１を再構成することができる。
【０１７２】
ウェーブレット逆変換の際に用いられるフィルタは、直交ウェーブレット変換の場合には変換する際に用いた係数と同じ係数のフィルタが使用される。双直交ウェーブレット変換の場合には、変換に用いた係数とは異なる係数のフィルタが逆変換の際に使用される（第１のウェーブレット文献を参照）。
【０１７３】
ウェーブレット変換によって抽出された画像劣化を補正するには、以下の方法を用いることができる。例えば、ブロックノイズを補正するのであれば、輝度信号と色差信号の一方又は双方について、画像劣化画素に隣接する非画像劣化画素の高周波帯域成分信号を用いて画像劣化画素の補間処理をすればよい。
【０１７４】
また、モスキートノイズやリンギングを補正する場合も、輝度信号と色差信号の一方又は双方について、画像劣化画素に隣接する非画像劣化画素の高周波帯域成分信号を用いて画像劣化画素を補間処理をすればよい。特に、輝度信号については、画像劣化画素の信号値の１階微分値を０にするか、エッジの急峻さに応じて画像劣化画素の信号値の１階微分値を減衰させるようにすればよい。
【０１７５】
なお、このような１階微分値を連続させることを目的とする高周波帯域成分に関わる処理を、輝度信号と色差信号の一方に適用するか双方に適用するかは、補正を施す対象となる画像劣化の特性に応じて選択されることが好ましい。
【０１７６】
次に、二項ウェーブレット変換について説明する。二項ウェーブレット（Ｄｙａｄｉｃ Ｗａｖｅｌｅｔ）変換については、”Ｓｉｎｇｕｌａｒｉｔｙ ｄｅｔｅｃｔｉｏｎ ａｎｄ ｐｒｏｃｅｓｓｉｎｇ ｗｉｔｈ ｗａｖｅｌｅｔｓ” ｂｙ Ｓ．Ｍａｌｌａｔ ａｎｄ Ｗ．Ｌ．Ｈｗａｎｇ， ＩＥＥＥ Ｔｒａｎｓ． Ｉｎｆｏｒｍ． Ｔｈｅｏｒｙ ３８ ６１７ （１９９２） や ”Ｃｈａｒａｃｔｅｒｉｚａｔｉｏｎ ｏｆ ｓｉｇｎａｌｓ ｆｒｏｍ ｍｕｌｔｉｓｃａｌｅ ｅｄｇｅｓ” ｂｙ Ｓ．Ｍａｌｌａｔ ａｎｄ Ｓ．Ｚｈｏｎｇ， ＩＥＥＥ Ｔｒａｎｓ． Ｐａｔｔｅｒｎ Ａｎａｌ． Ｍａｃｈｉｎｅ Ｉｎｔｅｌ． １４ ７１０ （１９９２） や ”Ａ ｗａｖｅｌｅｔ ｔｏｕｒ ｏｆ ｓｉｇｎａｌ ｐｒｏｃｅｓｓｉｎｇ ２ｅｄ．” ｂｙ Ｓ．Ｍａｌｌａｔ， Ａｃａｄｅｍｉｃ Ｐｒｅｓｓ に詳細な説明があるが、以下にその概要を説明する。ここで挙げた文献を第２のウェーブレット文献とする。
【０１７７】
二項ウェーブレット変換で用いられるウェーブレット関数は下記式（８）のように定義される。
【数８】

但し、ｉは自然数である。
【０１７８】
直交ウェーブレット変換、双直交ウェーブレット変換のウェーブレット関数は、上述のように、レベルｉにおける位置の最小移動単位が２^ｉで離散的に定義されていたのに対し、二項ウェーブレット変換は、レベルｉにかかわらず位置の最小移動単位が一定である。この相違により、二項ウェーブレット変換は、下記の特徴を有する。
【０１７９】
第一の特徴として、下記式（９）に示す１レベルの二項ウェーブレット変換で生成する、高周波数帯域成分Ｗ_ｉと低周波数帯域成分Ｓ_ｉの各々の信号量は、変換前の信号Ｓ_ｉ−１と同一である。
【数９】

【０１８０】
二項ウェーブレット変換の第二の特徴として、スケーリング関数φ_ｉ，ｊ（ｘ）とウェーブレット関数ψ_ｉ，ｊ（ｘ）の間に、下記の関係式（１０）が成立する。
【数１０】

従って、二項ウェーブレット変換で生成する、高周波数帯域成分Ｗ_ｉは、低周波数帯域成分Ｓ_ｉの１階微分（勾配）で表される。
【０１８１】
二項ウェーブレット変換の第三の特徴として、ウェーブレット変換のレベルｉに応じて定められた係数γ_ｉ（上述の二項ウェーブレットに関する参考文献を参照）を高周波数帯域成分に乗じたＷ_ｉ・γ_ｉ（以下、これを補正済高周波数帯域成分と称す）について、入力信号の信号変化の特異性（ｓｉｎｇｕｌａｒｉｔｙ）に応じて、該変換後の補正済高周波数帯域成分Ｗ_ｉ・γ_ｉの信号強度のレベル間の関係が一定の法則に従う。
【０１８２】
図２１に、入力信号Ｓ_０の波形と、ウェーブレット変換により得られる各レベルの補正済高周波数帯域成分の波形を示す。図２１において、（ａ）は入力信号Ｓ_０を示し、（ｂ）はレベル１の二項ウェーブレット変換により得られる補正済高周波数帯域成分Ｗ_１・γ_１を示し、（ｃ）はレベル２の二項ウェーブレット変換により得られる補正済高周波数帯域成分Ｗ_２・γ_２を示し、（ｄ）はレベル３の二項ウェーブレット変換により得られる補正済高周波数帯域成分Ｗ_３・γ_３を示し、（ｅ）はレベル４の二項ウェーブレット変換により得られる補正済高周波数帯域成分Ｗ・γ_４を示す。
【０１８３】
各レベルにおける信号強度の変化を見ると、（ａ）において、”１”や”４”に示すなだらかな（微分可能な）信号変化に対応する補正済高周波数帯域成分Ｗ_ｉ・γ_ｉは、（ｂ）→（ｅ）に示すようにレベル数ｉが増大するほど信号強度が増大する。
【０１８４】
入力信号Ｓ_０において、”２”に示すステップ状の信号変化に対応する補正済高周波数帯域成分Ｗ_ｉ・γ_ｉは、レベル数ｉに関わらず信号強度が一定となる。入力信号Ｓ_０において、”３”に示すδ関数状の信号変化に対応する補正済高周波数帯域成分Ｗ_ｉ・γ_ｉは、（ｂ）→（ｅ）に示すように、レベル数ｉが増大するほど信号強度が減少する。
【０１８５】
二項ウェーブレット変換における第四の特徴として、画像信号のような２次元信号における１レベルの二項ウェーブレット変換の方法は、上述の直交ウェーブレット変換や双直交ウェーブレット変換と異なり、図２２に示す方法で行われる。
【０１８６】
図２２に示すように、１レベルの二項ウェーブレット変換により、入力信号Ｓ_ｎ−１を、ｘ方向のローパスフィルタ及びｙ方向のローパスフィルタで処理することにより、低周波数帯域成分Ｓ_ｎが得られる。また、入力信号Ｓ_ｎ−１を、ｘ方向のハイパスフィルタで処理することにより、高周波数帯域成分Ｗｘ_ｎが得られる。更に、入力信号Ｓ_ｎ−１を、ｙ方向のハイパスフィルタで処理することにより、もう一つの高周波数帯域成分Ｗｙ_ｎが得られる。
【０１８７】
このように、１レベルの二項ウェーブレット変換により、入力信号Ｓ_ｎ−１は、２つの高周波数帯域成分Ｗｘ_ｎ、Ｗｙ_ｎと、１つの低周波数帯域成分Ｓ_ｎに分解される。２つの高周波数帯域成分Ｗｘ_ｎ、Ｗｙ_ｎは、低周波数帯域成分Ｓ_ｎの２次元における変化ベクトルＶ_ｎのｘ成分とｙ成分に相当する。変化ベクトルＶ_ｎの大きさＭ_ｎと偏角Ａ_ｎは下記式（１１）及び（１２）で与えられる。
【数１１】

【数１２】

【０１８８】
また二項ウェーブレット変換で得られた２つの高周波数帯域成分Ｗｘ_ｎ、Ｗｙ_ｎと１つの低周波数帯域成分Ｓ_ｎに、図２３に示す二項ウェーブレット逆変換を施すことにより、変換前の信号Ｓ_ｎ−１を再構成することができる。すなわち、Ｓ_ｎをｘ方向のローパスフィルタＬＰＦ_ｘ及びｙ方向のローパスフィルタＬＰＦ_ｙで処理することにより得られる信号と、Ｗｘ_ｎをｘ方向のハイパスフィルタＨＰＦ_ｘ及びｙ方向のローパスフィルタＬＰＦ_ｙで処理することにより得られる信号と、Ｗｙ_ｎをｘ方向のローパスフィルタＬＰＦ_ｘ及びハイパスフィルタＨＰＦ_ｙで処理することにより得られる信号と、を加算することによって、二項ウェーブレット変換前の信号Ｓ_ｎ−１を得ることができる。
【０１８９】
次に、図２４のブロック図に基づいて、入力信号Ｓ_０に対する二項ウェーブレット変換から、画像劣化が補正された信号Ｓ_０’を得るまでの方法について説明する。
【０１９０】
入力信号Ｓ_０に対するレベル１の二項ウェーブレット変換によって、入力信号Ｓ_０は、２つの高周波数帯域成分Ｗｘ_１、Ｗｙ_１と低周波数帯域成分Ｓ_１に分解される。レベル２のウェーブレット変換によって、レベル１の二項ウェーブレット変換で得られた低周波数帯域成分Ｓ_１は、更に２つの高周波数帯域成分Ｗｘ_２、Ｗｙ_２と低周波数帯域成分Ｓ_２に分解される。この様な分解操作をレベルｎまで繰り返すことにより、入力信号Ｓ_０は、複数の高周波数帯域成分Ｗｘ_１、Ｗｘ_２、…、Ｗｘ_ｎ、Ｗｙ_１、Ｗｙ_２、…、Ｗｙ_ｎと、１つの低周波数帯域成分Ｓ_ｎとに分解される。
【０１９１】
このようにして得られた高周波数帯域成分Ｗｘ_１、Ｗｘ_２、…、Ｗｘ_ｎ、Ｗｙ_１、Ｗｙ_２、…、Ｗｙ_ｎ、低周波数帯域成分Ｓ_ｎから、画像劣化が抽出され、画像劣化に対する補正処理が行われ、補正された高周波数帯域成分Ｗｘ_１’、Ｗｘ_２’、…、Ｗｘ_ｎ’、Ｗｙ_１’、Ｗｙ_２’、…、Ｗｙ_ｎ’、低周波数帯域成分Ｓ_ｎ’が得られる。
【０１９２】
そして、これら高周波数帯域成分Ｗｘ_１’、Ｗｘ_２’、…、Ｗｘ_ｎ’、Ｗｙ_１’、Ｗｙ_２’、…、Ｗｙ_ｎ’、低周波数帯域成分Ｓ_ｎ’に、二項ウェーブレット逆変換が施される。すなわち、補正後のレベルｎにおける２つの高周波数帯域成分Ｗｘ_ｎ’、Ｗｙ_ｎ’と低周波数帯域成分Ｓ_ｎ’から、補正されたレベルｎ−１の低周波数帯域成分Ｓ_ｎ−１’が構成される。このような操作を繰り返し、補正後のレベル２における２つの高周波数帯域成分Ｗｘ_２’、Ｗｙ_２’と低周波数帯域成分Ｓ_２’から、補正されたレベル１の低周波数帯域成分Ｓ_１’が構成される。この低周波数帯域成分Ｓ_１’と、レベル１における２つの高周波数帯域成分Ｗｘ_１’、Ｗｙ_１’から、補正後の画像信号Ｓ_０’が構成される。
【０１９３】
なお、図２４において用いられる各フィルタのフィルタ係数は二項ウェーブレット変換に応じて適切に定められる（第２のウェーブレット文献参照）。また二項ウェーブレット変換においては、レベル毎に用いるフィルタのフィルタ係数が異なる。レベルｎにおいて使用するフィルタ係数は、レベル１のフィルタの各係数の間に２^ｎ−１−１個のゼロを挿入したものが用いられる（第２のウェーブレット文献参照）。
【０１９４】
二項ウェーブレット変換によって抽出された画像劣化を補正するには、以下の方法を用いることができる。輝度信号や色差信号の１階微分値を連続させるように、二項ウェーブレット変換によって得られた高周波数帯域成分（Ｗｘ_ｎ， Ｗｙ_ｎ）から算出される変化ベクトルＶ_ｎの大きさＭ_ｎ（式１１参照）と偏角Ａ_ｎ（式１２参照）の一方又は双方を補正するのが好ましい。より詳細には、画像劣化画素の輝度信号と色差信号の一方又は双方について、変化ベクトルＶ_ｎの大きさＭ_ｎや偏角Ａ_ｎの値を、画像劣化画素に隣接する非画像劣化画素の変化ベクトルの大きさや偏角の値を用いて補間処理をすればよい。なお、補間処理の方法は、当業界で知られる手法が選択され、上述の方法に限定されるものではない。
【０１９５】
以上述べたような、例えば２項ウェーブレット変換を用いて画像変換し、式（１１）で示される、変化ベクトルの大きさ（信号値勾配情報）と、式（１２）で示される変化ベクトルの方向に基づいて、傷ゴミ補間方法を決定できる。例えば、画像情報に傷ゴミ補正処理及び傷ゴミ補間処理を順に行い、傷ゴミ領域に含まれない正常画素が有する信号値勾配情報に基づいて、傷ゴミ領域の周辺領域の画像構造を評価し、その評価結果に基づいて補間処理を行う構成である。この場合、可能な範囲でエッジなどの画像構造に関する情報が復元でき、補間処理に用いるアルゴリズムがより効果的に機能し、より確実な補間処理を実行して補間結果を得ることができる。
【０１９６】
この際、多重解像度変換の分解レベルが上がるにつれ、変化ベクトルは大局的な構造を示すものとなり、補間結果にも大局的構造を反映させる事が可能となる。また、前述のように、あらかじめ補正処理によって正常画素領域の占める割合を増大させているから、より正確な大局的構造を評価する事が可能である。
【０１９７】
また、上述のウェーブレット変換は非常に処理量が多いものであるが、画像に関し非常に有用な情報を得ることができる。被写体パターン抽出や、覆い焼きなどの各種マスキング処理など、高度な処理にたいして非常に有用な技術であるので、これら処理を行う、高度な画像処理システムに於いては、既にこれら目的で用いられたウェーブレット変換の各種情報を、本実施の形態の傷ゴミ処理に用いることができるので大きな処理量も問題とはならないであろう。
【０１９８】
次に、図２５を参照して、図３の傷ゴミ処理中のステップＳ１５の一実施例としての第３の傷ゴミ補正処理を説明する。図２５は、第３の傷ゴミ補正処理を示すフローチャートである。第３の傷ゴミ補正処理は、可視画像情報を用いて補正処理を行う手法の一例である。
【０１９９】
先ず、既に求められた可視画像情報上の傷ゴミ領域の近傍画素の領域が周辺領域とされる（ステップＳ３５１）。例えば、図１５に示すように、近傍画素の所定範囲を１ピクセルとする。そして、可視画像情報内の一画素が注目画素として抽出される（ステップＳ３５２）。そして、その抽出された注目画素が傷ゴミ領域内にあるか否かが判別される（ステップＳ３５３）。
【０２００】
注目画素が傷ゴミ領域内にない場合（ステップＳ３５３；ＮＯ）、ステップＳ３５８に移行される。注目画素が傷ゴミ領域内にある場合（ステップＳ３５３；ＹＥＳ）、注目画素における傷ゴミ領域特性値が算出される（ステップＳ３５４）。例えば、注目画素から所定領域内の傷ゴミ領域内の画素の特性値の平均値が算出される。そして、注目画素における周辺領域特性値が算出される（ステップＳ３５５）。周辺領域特性値は、注目画素を中心とした、所定距離内にある周辺領域内の画素値を用いて算出した特性値であり、例えば、所定距離内で周辺領域内の各画素の画素値の平均値とされる。また、傷ゴミ領域特性値や周辺領域特性値の算出は、例えば異常データを除外し、その後最頻値を求めるなど、統計的手法によってもよい。
【０２０１】
そして、算出された傷ゴミ領域特性値と周辺領域特性値との差である特性値間格差に基づいて、可視画像補正値が傷ゴミ補正値として算出される（ステップＳ３５６）。その算出された傷ゴミ補正値（可視画像補正値）を用いて、注目画素に傷ゴミ補正がなされる（ステップＳ３５７）。そして、ステップＳ３５２において可視画像情報内の全画素が注目画素として抽出されたか否かが判別される（ステップＳ３５８）。全画素が抽出された場合（ステップＳ３５８；ＹＥＳ）、第３の傷ゴミ補正処理が終了される。全画素が抽出されていない場合（ステップＳ３５８；ＮＯ）、ステップＳ３５２に移行され、未抽出の画素が次の抽出画素として抽出される。第３の傷ゴミ補正処理の終了後、次のステップ（図４のステップＳ１６）に移行される。
【０２０２】
一般的に、傷、ゴミの影響は、その及ぼす領域の形態は様々であるが、一つの領域内の隣接する各ピクセルが受けている影響の度合いには特に重視すべき大きな差は無く、第３の傷ゴミ補正処理によって、視覚的に好ましい補正結果が得られる。また、第３の傷ゴミ補正処理によれば、実際に必要な画像（ここでは可視画像情報）のみを用いて傷ゴミ領域の補正が実施できるので、例えば、赤外画像情報と可視画像情報とのＭＴＦ特性やフレア特性に大きな差が有った場合にも補正過剰や不足の生じない補正性能が実現できる。さらに、可視画像情報が欠損しているような場合（例えば、フィルム上に損傷が存在する場合）でも、逆補正の危険が少ない、欠陥の影響を目立たせない、良好な処理結果が得られる。
【０２０３】
次に、図２６及び図２７を参照して、図３の画像処理中のステップＳ５の拡大縮小処理と、ステップＳ７の鮮鋭性粒状性補正処理とを説明する。図２６は、拡大縮小処理を示すフローチャートである。図２７は、鮮鋭性粒状性補正処理を示すフローチャートである。
【０２０４】
先に、図２６を参照して拡大縮小処理を説明する。拡大縮小処理は、その手法によって処理結果が異なり、特に、画像情報の平滑化作用に大きな差がある。先ず、可視画像情報中の拡大／縮小に必要な格子点座標が計算される（ステップＳ５１）。格子点は、例えば、拡大すべき画素と画素との間に補完するための中間点である。そして、その計算された格子点の周囲４画素が抽出される（ステップＳ５２）。周囲４画素は、格子点の補間の計算に用いられる。そして、周囲４画素が正常画素のみであるか否かが判別される（ステップＳ５３）。
【０２０５】
周囲４画素が正常画素のみでない場合（ステップＳ５３；ＮＯ）、周囲４画素が傷ゴミ領域内のみに存在するか否かが判別される（ステップＳ５４）。周囲４画素が傷ゴミ領域内のみに存在しない場合（ステップＳ５４；ＮＯ）、その周辺４画素は、正常画素及び傷ゴミ領域が混在する領域としての混在領域にあるとされる（ステップＳ５５）。そして、混在領域内の画素同士は信号の格差が大きいとみなすことができ、平滑性の強い補間方式を利用して、注目画素に対して拡大縮小が実行される（ステップＳ５６）。平滑性の強い補間方式によって、傷ゴミ処理境界の目立ちにくい画像処理結果が得られる。
【０２０６】
周囲４画素が正常画素のみである場合（ステップＳ５３；ＹＥＳ）、又は周囲４画素が傷ゴミ領域内のみに存在する場合（ステップＳ５４；ＹＥＳ）、周囲４画素が正常画素領域又は傷ゴミ領域の何れか一方に含まれ、その領域内画素同士は信号の格差が小さいとみなすことができ、平滑性の弱い補間方式を利用して、注目画素に対して拡大縮小が実行される（ステップＳ５７）。
【０２０７】
そして、ステップＳ５１において可視画像情報全面の拡大／縮小に必要な全格子点が抽出（計算）されたか否かが判別される（ステップＳ５８）。全格子点が抽出された場合（ステップＳ５８；ＹＥＳ）、拡大縮小処理が終了される。全格子点が抽出されていない場合（ステップＳ５８；ＮＯ）、ステップＳ５１に移行され、未抽出の格子点が次の格子点として計算される。拡大縮小処理の終了後、次のステップ（図３のステップＳ６）に移行される。
【０２０８】
拡大縮小処理における補間方法については、例えば、特開２００２−２６２０９４に記載の補間方法を用いることができる。その中で、変倍時のモアレなどが発生しやすいわずかな変倍率の拡大／縮小には、平滑化効果の大きな近傍９点の画素データを用いた線形補間方法が適用され、その他には、平滑化効果の比較的小さな、近傍４点の画素データを用いた補間方法が適用されている。即ち、本実施の形態において、前者を傷ゴミ補正領域及び正常画素領域の混在領域個所に適用し（ステップＳ５６に対応）、後者をその他の領域に適用すること（ステップＳ５７に対応）で好ましい結果が得られる。
【０２０９】
次に、図２７を参照して鮮鋭性粒状性補正処理を説明する。先ず、可視画像情報中の一画素が注目画素として抽出される（ステップＳ７１）。そして、注目画素が正常画素であるか否かが判別される（ステップＳ７２）。注目画素が正常画素である場合（ステップＳ７２；ＹＥＳ）、鮮鋭性強調が「強」で、粒状性補正が「中」に設定されて、その設定値に基づいて注目画素に鮮鋭性強調及び粒状性補正がなされる（ステップＳ７３）。
【０２１０】
そして、ステップＳ７１において可視画像情報内の全画素が注目画素として抽出されたか否かが判別される（ステップＳ７４）。全画素が抽出された場合（ステップＳ７４；ＹＥＳ）、鮮鋭性粒状性補正処理が終了される。全画素が抽出されていない場合（ステップＳ７４；ＮＯ）、ステップＳ７１に移行され、未抽出の画素が次の抽出画素として抽出される。鮮鋭性粒状性補正処理の終了後、次のステップ（図３のステップＳ８）に移行される。
【０２１１】
注目画素が正常画素でない場合（ステップＳ７２；ＮＯ）、注目画素が周辺領域にあるか否かが判別される（ステップＳ７５）。周辺領域は、可視画像情報中の傷ゴミ処理された領域と、されていない領域との境界に存在する領域である。注目画素が周辺領域内である場合（ステップＳ７５；ＹＥＳ）、鮮鋭性強調が「中」で、粒状性補正が「強」に設定されて、その設定値に基づいて注目画素に鮮鋭性強調及び粒状性補正がなされ（ステップＳ７６）、ステップＳ７４に移行される。周辺領域の鮮鋭性強調を強くすると、傷ゴミ領域の境界の境目が見やすくなる危険があるため、若干の加減がなされる。あわせて粒状性補正がやや強くされる（粒状が抑制される）。
【０２１２】
注目画素が周辺領域内でない場合（ステップＳ７５；ＮＯ）、注目画素が傷ゴミ領域にあるとされる（ステップＳ７７）。注目画素が傷ゴミ領域にある場合に、正常画素と異なる補正をすることで、全体的に均質化された、傷ゴミ処理の痕跡が目立ちにくい処理結果が得られる。傷ゴミ領域に付いてはさらに、その処理方法によって場合分けすることが好ましい。よって、注目画素に傷ゴミ補正処理がなされているか、傷ゴミ補間処理がなされているかが判別される（ステップＳ７８）。傷ゴミ補正処理がなされている場合（ステップＳ７８；補正処理）、鮮鋭性強調が「弱」で、粒状性補正が「強」に設定されて、その設定値に基づいて注目画素に鮮鋭性強調及び粒状性補正がなされ（ステップＳ７９）、ステップＳ７４に移行される。例えば、傷ゴミ補正処理がなされた場合は、減衰した信号を増強復元するので、ノイズ抑制のため、当該領域の粒状性補正を通常より強くする（粒状抑制を強く掛ける）。あわせて、鮮鋭性強調の程度は弱くする。
【０２１３】
傷ゴミ補間処理がなされている場合（ステップＳ７８；補間処理）、鮮鋭性強調が「強」で、粒状性補正が「弱」に設定されて、その設定値に基づいて注目画素に鮮鋭性強調及び粒状性補正がなされ（ステップＳ８０）、ステップＳ７４に移行される。傷ゴミ補間処理がなされた場所では、周辺画素を参照し、重み付け平均するステップを有する場合が多く、ノイズ成分が減少している。このため、粒状性補正は弱く施され、鮮鋭性強調は相対的に強く施される。
【０２１４】
以上述べた鮮鋭性強調及び粒状性補正の関係は、一例としての傷ゴミ補正処理及び傷ゴミ補間処理を仮定した場合の説明であり、各補正の強弱はこれに限定されるものではなく、傷ゴミ補正及び補間処理の手法を切り替えれば、適宜調整されるべきものである。
【０２１５】
次に、画像取得部１４における画像取得を透過原稿スキャナ１４２に代えて反射原稿スキャナ１４１で行う場合を説明する。本実施の形態のうちの図１０の傷ゴミ補間処理、図１４の傷ゴミ補正補間処理、図２５の第３の傷ゴミ補正処理は、画像補正あるいは補間に赤外画像情報を必要とせず、傷ゴミ候補領域を何らかの方法で求められれば良い。そこで、図２８を参照して、赤外画像情報を取得しない構成として、図１の画像取得部１４の反射原稿スキャナ１４１を説明する。図２８は、反射原稿スキャナ１４１の内部構成を示す図である。
【０２１６】
図２８に示すように、反射原稿スキャナ１４１は、写真原稿（光沢印画紙など）Ｂの光源４１，４２と、光源４１，４２から出射された光を透過及び反射させるハーフミラー４３，４４と、光を結像及び受光してアナログ信号を得るＣＣＤ４５〜４７と、ＣＣＤ４５〜４７から出力されたアナログ信号をそれぞれ増幅するアンプ４８〜５０と、アンプ４８〜５０から出力されたアナログ信号をデジタル信号にそれぞれ変換するＡ／Ｄコンバータ５１〜５３とを備える。
【０２１７】
また反射原稿スキャナ１４１は、光源４１，４２の光出射とＣＣＤ４５〜４７の光受光とのタイミングを制御するタイミング制御部５４と、Ａ／Ｄコンバータ５２から出力された画像情報のうち、光源４１，４２から出射されたそれぞれの光に対応する２つの画像情報を比較する画像比較部５５と、画像比較部５５から出力された比較結果から元画像情報を形成する元画像形成部５６と、Ａ／Ｄコンバータ５１，５３からそれぞれ出力された２つの画像情報から欠陥候補領域を決定する欠陥候補領域決定部５７と、欠陥候補領域決定部５７から出力される欠陥候補領域から欠陥領域を特定し、その欠陥領域を、元画像形成部５６から出力される元画像情報とともに出力する欠陥領域特定部５８とを備えて構成される。
【０２１８】
また、欠陥領域特定部５８から出力される欠陥領域及び元画像情報は、画像処理部１１内の補正補間処理部６１に送信される。写真原稿Ｂは、均一の光沢を示しているものであれば良い。
【０２１９】
次に、反射原稿スキャナ１４１の動作を簡単に説明する。先ず、写真原稿Ｂは、２つの光源４１，４２で照射される。光源４１，４２は、タイミング制御部５４によって交互に発光する。ＣＣＤ４６は両方の光源の画像を採取する。ＣＣＤ４７は、光源４１が点燈しているタイミングで、ハーフミラー４３を透過しハーフミラー４４で反射された光を採取する。ＣＣＤ４５は、光源４２が点燈しているタイミングで、ハーフミラー４４を透過しハーフミラー４３で反射された光を採取する。
【０２２０】
ＣＣＤ４６で採取された画像情報は、画像比較部５５及び元画像形成部５６において、光源間差が比較され、輝度差のある場合は暗いほうの画像情報が重視され、１枚の画像情報を元画像情報として形成される。このようにすることで、光沢印画紙（写真原稿Ｂ）の微細な凹凸による反射や、絹目印画紙（写真原稿Ｂ）の光沢反射を除去することができ、好ましいコピー結果が得られる。
【０２２１】
一方、ＣＣＤ４５，４７で採取された画像情報は、欠陥画像候補領域決定部５７において、所定の画像信号値の範囲に収まらない画像領域をＣＣＤ４５，４７が各々抽出し、両者の出力画像領域が一致した場合に傷、ゴミなどの欠陥候補領域とされる。欠陥候補領域は、欠陥領域特定部５８により、ノイズ処理が実行され、その後にオープニング処理によってサイズ拡大され、欠陥領域とされる。
【０２２２】
補正補間処理部６１は、欠陥領域特定部５８から元画像情報及び欠陥領域の情報を受信し、その元画像情報及び欠陥領域情報に基づいて、本実施の形態で説明したような傷ゴミ補正処理、傷ゴミ補間処理がなされる。
【０２２３】
以上、本実施の形態の傷ゴミ処理によれば、赤外画像情報を用いて傷ゴミ領域を求め、可視画像情報への傷ゴミ領域の影響を傷ゴミ補正し、その補正の良否を評価し、その評価結果に応じて、可視画像情報に傷ゴミ補間処理を行う。このため、例えばフィルムの損傷による画像欠損が発生し、赤外画像情報による信号強度の補正が逆補正となってしまう不具合にも有効に傷ゴミ補間処理して傷ゴミ領域を除去できる。また、赤外域と可視域との画像情報の若干のずれが、画面の一部分で発生している場合にも、そのずれが発生した画像領域の補正結果を評価することができるので、その画像領域に応じた傷ゴミ補間処理を行うことができる。また、本実施の形態の傷ゴミ補正補間処理によれば、補正済みと補正前との可視画像情報における、傷ゴミ領域とその周辺領域との代表値を比較して補正結果が評価されるので、補正結果の評価が簡単な手法で良好に実施できる。
【０２２４】
また、本実施の形態の傷ゴミ処理によれば、傷ゴミ補正処理の後に傷ゴミ補間処理を行い、補間処理に用いることのできる周辺画像情報としての正常画素を補正処理により最大限に増やすことができるので、より確実な補間処理を実行して補間結果を得ることができる。また、例えば、フィルムの損傷による不具合にも有効に対処でき、さらに、補正した正常画素も用いて補間を行うから、精度の高い画像処理結果が得られる。
【０２２５】
また、本実施の形態の拡大縮小処理によれば、正常画素領域と傷ゴミ領域と混在領域との分類情報に応じて、拡大縮小の補間処理を切り替えて実行するので、傷ゴミ処理の痕が残りにくい、より自然な画像処理結果が得られる。
【０２２６】
また、本実施の形態の鮮鋭性粒状性補正処理によれば、正常画素領域、周辺領域及び傷ゴミ領域の分類に加えて、傷ゴミ領域領域の補正対象領域と補間対象領域とに応じて、鮮鋭強調及び粒状性補正処理を実行するので、傷ゴミ処理の痕が残りにくい、さらに自然な画像処理結果が得られる。
【０２２７】
また、第２の傷ゴミ補正処理によれば、傷ゴミ領域の影響の大きさを、注目画素の領域サイズ情報に応じて評価し、その評価結果に基づき可視画像情報を傷ゴミ補正するので、通常想定される画像情報（正常画素数＞＞異常画素数）において、注目画素に与える傷ゴミ領域の影響の大きさを正しく評価して、その評価結果に応じた修正値に基づいて補正することができる。したがって、赤外画像情報に残存収差やフレア成分が残っていたとしても、これらが赤外画像情報に及ぼす影響を補償し、好ましい画像処理が実現できる。また、可視画像情報から仮の可視画像補正値を求め、赤外画像補正値の可視画像補正値による除値と領域サイズ情報との関係を示す代表曲線を取得する。このため、代表曲線を用いて領域サイズ情報により修正値を決定して、その修正値及び赤外画像補正値を用いて可視域の画像情報を傷ゴミ補正でき、未知の画像情報取得元（フィルムなど）に関しても安定した傷、ゴミ除去効果が得られる。
【０２２８】
なお、上述した本実施の形態における記述は、本発明に係る好適な画像処理システム１０の一例であり、これに限定されるものではない。
【０２２９】
【発明の効果】
請求項１、１１又は２１に記載の発明によれば、赤外域の画像情報を用いて画像欠陥領域を取得し、可視域の画像情報への画像欠陥領域の影響を補正し、その補正の良否を評価し、その評価結果に応じて、可視域の画像欠陥領域の処理を行う。このため、例えば画像情報取得元の画像記録層の欠損による画像欠損が発生し、赤外域の画像情報による信号強度の補正が逆補正となってしまう不具合や、赤外域と可視域との画像情報の若干のずれが、画面の一部分で発生している不具合にも、その不具合が発生した画像領域の補正結果を評価することができるので、その画像領域に応じた処理を選択して実行することにより、有効に対処できる。
【０２３０】
請求項２、１２又は２２に記載の発明によれば、補正済みと補正前との可視域の画像情報における、画像欠陥領域とその周辺領域との代表値を比較して補正結果が評価されるので、補正結果の評価が簡単な手法で良好に実施できる。
【０２３１】
請求項３、１３又は２３に記載の発明によれば、赤外域の画像情報の画像欠陥領域内のテクスチャ構造の特徴に応じて、可視域の画像情報に対して好ましい補正方法を選択して実行できる。また、例えば、画像情報取得元の画像記録層の欠損による画像欠損が発生し、赤外域の画像情報による信号強度の補正が逆補正となってしまう不具合にも、好ましい補正を選択して実行し、有効に対処できる。また、赤外域の画像情報の残存収差や、赤外域と可視域との画像情報の間に色ずれが存在してもこれに対応した良好な画像処理が可能になる。
【０２３２】
請求項４、１４又は２４に記載の発明によれば、補間処理に用いることのできる周辺画像情報としての正常画素を補正処理により最大限に増やすことができるので、より確実な補間処理を実行して補間結果を得ることができる。また、例えば、画像情報取得元の画像記録層の欠損による画像欠損が発生し、赤外域の画像情報による信号強度の補正が逆補正となってしまう不具合にも有効に対処でき、さらに、補正した正常画素も用いて補間を行うから、精度の高い画像処理結果が得られる。
【０２３３】
請求項５、１５又は２５に記載の発明によれば、補正及び／又は補間した第１の画素の属する領域と補正及び／又は補間していない第２の画素の属する領域と第３の画素の属する領域との分類情報に応じて、各種行われる後処理を切り替えて実行するので、欠陥画素の補正の痕が残りにくい、より自然な画像処理結果が得られる。
【０２３４】
請求項６、１６又は２６に記載の発明によれば、分類情報に加えて、第１の画素の画像領域の補正対象の第１のグループと補間対象の第２のグループとのグループ情報に応じて、各種行われる後処理を切り替えて第１の画素に実行するので、補正の痕が残りにくい、さらに自然な画像処理結果が得られる。
【０２３５】
請求項７、１７又は２７に記載の発明によれば、補間処理に用いることのできる周辺画像情報としての正常画素を補正処理により最大限に増やすことができるので、より確実な補間処理を実行して補間結果を得ることができる。また、例えば、画像情報取得元の画像記録層の欠損による画像欠損が発生し、赤外域の画像情報による信号強度の補正が逆補正となってしまう不具合にも有効に対処できる。さらに、補正した正常画素も用いて補間を行うから、精度の高い画像処理結果が得られる。
【０２３６】
請求項８、１８又は２８に記載の発明によれば、画像欠陥の領域に含まれない正常画素が有する信号値勾配情報に基づいて、画像欠陥の周辺領域の画像構造を評価し、その評価結果に基づいて補間処理するので、例えば、可能な範囲でエッジなどの画像構造に関する情報が復元でき、補間処理に用いるアルゴリズムがより効果的に機能し、さらにより確実な補間処理を実行して補間結果を得ることができる。
【０２３７】
請求項９、１９又は２９に記載の発明によれば、画像欠陥領域の影響の大きさを、注目画素を通る最短線分に応じて評価し、その評価結果に基づき可視域の画像情報を修正補正するので、通常想定される画像情報（正常画素数＞＞異常画素数）において、注目画素に与える画像欠陥領域の影響の大きさを正しく評価して、その評価結果に応じた修正値に基づいて補正することができる。したがって、赤外域の画像情報に残存収差やフレア成分が残っていたとしても、これらが赤外域の画像情報に及ぼす影響を補償し、好ましい画像処理が実現できる。
【０２３８】
請求項１０、２０又は３０に記載の発明によれば、可視域の画像情報から仮の可視画像補正値を求め、赤外域の補正値の可視画像補正値による除値と線分距離情報との関係を示す代表曲線を取得する。このため、代表曲線を用いて線分距離情報により修正値を決定し、その修正値と赤外域の補正値とを用いて可視域の画像情報を補正でき、未知の画像情報取得元に関しても安定した傷、ゴミ除去効果が得られる。
【図面の簡単な説明】
【図１】本発明に係る実施の形態の画像処理システム１０を示すブロック図である。
【図２】図１の透過原稿スキャナ１４２の構成を示す図である。
【図３】画像処理を示すフローチャートである。
【図４】傷ゴミ処理を示すフローチャートである。
【図５】各波長域の信号の吸収を示す図である。
【図６】傷ゴミ候補領域検出処理を示すフローチャートである。
【図７】傷ゴミ領域決定処理を示すフローチャートである。
【図８】第１の傷ゴミ補正処理を示すフローチャートである。
【図９】第２のゴミ補正処理を示すフローチャートである。
【図１０】傷ゴミ補間処理を示すフローチャートである。
【図１１】注目画素の補間処理候補選択を示す図である。
【図１２】注目画素の補間処理候補選択の３つの様態例を示す図である。
【図１３】注目画素の２つの方向特性例を示す図である。
【図１４】傷ゴミ補正補間処理を示すフローチャートである。
【図１５】傷ゴミ領域及びその周辺領域を示す図である。
【図１６】ウェーブレット関数を示す図である。
【図１７】１レベルのウェーブレット変換のフィルタ処理を示すシステムブロック図である。
【図１８】２次元信号における１レベルのウェーブレット変換のフィルタ処理を示すシステムブロック図である。
【図１９】入力信号Ｓ_０が、レベル１〜３の直交ウェーブレット変換又は双直交ウェーブレット変換により信号分解される過程を示す模式図である。
【図２０】レベル１のウェーブレット逆変換のフィルタ処理を示すシステムブロック図である。
【図２１】入力信号Ｓ_０の波形と、ウェーブレット変換により得られる各レベルの補正済高周波数帯域成分Ｗ・γの波形を示す図である。
【図２２】２次元信号における１レベルの二項ウェーブレット変換のフィルタ処理を示すシステムブロック図である。
【図２３】２次元信号における１レベルの二項ウェーブレット逆変換のフィルタ処理を示すシステムブロック図である。
【図２４】入力信号Ｓ_０に対する二項ウェーブレット変換から、画像劣化が補正された信号Ｓ_０’を得るまでの処理を示すシステムブロック図である。
【図２５】第３の傷ゴミ補正処理を示すフローチャートである。
【図２６】拡大縮小処理を示すフローチャートである。
【図２７】鮮鋭性粒状性補正処理を示すフローチャートである。
【図２８】反射原稿スキャナ１４１の内部構成を示す図である。
【符号の説明】
１０…画像処理システム
１１…画像処理部
１２…画像表示部
１３…指示入力部
１３１…接触センサ
１３２…マウス
１３３…キーボード
１４…画像取得部
１４１…反射原稿スキャナ
４１，４２…光源
４３，４４…ハーフミラー
４５〜４７…ＣＣＤ
４８〜５０…アンプ
５１〜５３…Ａ／Ｄコンバータ
５５…画像比較部
５６…元画像形成部
５７…欠陥候補領域決定部
５８…欠陥領域特定部
Ｂ…写真原稿
６１…補間補正処理部
１４２…透過原稿スキャナ
３１…光源部
３２…拡散部材
３３…フィルムキャリア
３４…ローラ
３５…光学レンズ
３６ＩＲ，３６Ｂ，３６Ｒ…ダイクロイックフィルタ
３７ＩＲ，３７Ｂ，３７Ｇ，３７Ｒ…ラインＣＣＤ
３８ＩＲ，３８Ｂ，３８Ｇ，３８Ｒ…アナログアンプ
３９ＩＲ，３９Ｂ，３９Ｇ，３９Ｒ…Ａ／Ｄコンバータ
４０…画像メモリ
Ａ…フィルム
１４３…メディアドライバ
１４４…情報通信Ｉ／Ｆ
１５…画像ストレージ部
１６…銀塩露光プリンタ
１７…ＩＪプリンタ
１８…各種画像記録メディア書込部
２１…印刷物
２２…現像済みフィルム
２３…画像メディア
２４…通信手段[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image processing method, an image processing device, and an image processing program.
[0002]
[Prior art]
2. Description of the Related Art In recent years, systems that use a silver halide photographic film for photographing and development processing, read it with an image reading device such as a film scanner, acquire it as image data, and variously use it have become widespread. Since the silver halide film has a very large amount of information, these image reading devices require a very high resolution to reliably read a small signal.
[0003]
On the other hand, a photographic film has an image recording layer mainly made of gelatin as a binder applied to a film base made of TAC (triacetylcellulose), PET (polyethylene terephthalate) or the like because of its easy handling. Are easily adhered, and the image recording layer and the film base both have low hardness, so that they are easily damaged.
[0004]
For this reason, photographic film must be handled carefully, which requires many man-hours. Further, as described above, fine scratches and dust are recorded in image information obtained using an image reading apparatus having a very high resolution. Was.
[0005]
In view of these circumstances, several solutions have been considered and proposed. The main one is a method using infrared rays by utilizing the characteristics of a photographic film in which image information is formed by a dye image, such as a color film, which is currently the mainstream of photographic films (see, for example, Patent Document 1). .
[0006]
The basic idea of these methods is based on the following idea. As described above, the image information is formed of a dye image. Due to the nature of these dyes, these dyes absorb visible electromagnetic waves having a specific wavelength, but hardly absorb infrared light having a long wavelength. On the other hand, scratches and dust are often colorless themselves, but they have the property of strongly scattering light, and when they enter the middle of the image forming system, scattered light is generated. The signal strength obtained as information will decrease. The effects of the scratches and dust appear almost uniformly irrespective of whether they are visible or infrared light. Therefore, observing an infrared image allows the positions and effects of the scratches and dust to be identified.
[0007]
Of course, as described in Patent Literature 1, the dyes that form image information, in particular, the cyan dye, also have some infrared absorption. Prior to observation of an infrared image, it is common to perform a process to reduce the influence of these dyes.
[0008]
In addition, there has been a method in which a scratch or dust is detected using an infrared image, the image is divided into normal pixels and abnormal pixels, and the abnormal pixels are interpolated from a group of the nearest normal pixels in the vicinity (for example, , Patent Document 2). According to this method, since the flaw area can be interpolated from the surrounding area to make up for it, flaws and dust can be erased from the screen. However, according to this method, an area determined to be an abnormal pixel is obtained by interpolation from pixel information of an area determined to be a normal pixel, and thus has a significant side effect that the original image information amount is greatly reduced. I was
[0009]
In order to solve this problem, the effect of scratches and dust on the infrared image is detected, and a considerable amount of correction is performed on the visible image. There has been a method of performing processing by an interpolation method (for example, see Patent Document 3). According to this method, a pixel having a flaw or dust is also subjected to a process for correcting the influence of the flaw or dust in a portion where the influence is relatively small. It is said that the decrease is small.
[0010]
[Patent Document 1]
Japanese Patent Publication No. 6-5914
[Patent Document 2]
JP-A-63-129469
[Patent Document 3]
Japanese Patent No. 2559970
[0011]
[Problems to be solved by the invention]
However, according to Patent Literature 3, the degree of the effect of scratches and dust is determined mainly using an infrared image, and thus the following problems occur.
[0012]
The first problem is that the method of Patent Document 3 limits the influence of scratches and dust to appear with a certain correlation between visible light and infrared light. In the case where a slight defect occurs in the image information, in the infrared image, the signal intensity is greatly reduced due to the scattering of light rays, and the signal decreases.However, in the visible image, the defect of the image recording layer that absorbs the visible light causes the defect. The signal strength may increase. In such a case, the technique disclosed in Patent Document 3 may perform the reverse correction, that is, increase the visible signal strength, and may increase the influence of scratches and dust.
[0013]
In addition, in the method of Patent Document 3, the influence of scratches and dust appears to have a certain correlation between visible light and infrared light. It has occurred. Furthermore, the coating process to reduce diffuse reflection at the lens interface and the scattering properties of the scratches on the film base itself also have wavelength dependence, so even if there is a scratch on the base surface, there is a correlation with the effects of the scratches, The correlation ratio is not always constant. Therefore, there is a concern that overcorrection or undercorrection may occur in the correction of scratches and dust.
[0014]
The second problem is that it is necessary to accurately detect the position of a flaw or dust, and more specifically, to detect the same at the same position as a visible image. A film scanner receives an image formed by an optical lens with a sensor such as a CCD to obtain image information.However, since the optical lens has chromatic aberration, an image is formed at exactly the same position from an infrared image to a visible image. It is very difficult. Further, even if the imaging can be realized, there is a possibility that the system may require a large number of adjustment steps and costs. Regarding the problem of this system, the method of Patent Document 2 is also a problem. If there is a slight shift between the infrared and visible imaging positions, interpolation may become impossible or a correction defect may occur. . Alternatively, if there is a margin in the region for performing the interpolation processing, the range in which the interpolation is necessary becomes large, and there is a possibility that a defect that the loss of the image is further increased may occur.
[0015]
Furthermore, the third problem is that, generally, after the processing for reducing the influence of scratches and dust is completed, various image processings are performed. There is a concern that unnaturalness may be emphasized, but in this regard, the above-described conventional method has not been particularly effective.
[0016]
It is an object of the present invention to satisfactorily remove the influence of a defective portion from image information. More specifically, the first object is to appropriately determine the influence of a flaw on visible image information and to obtain a good correction result for both the flaw of the image information acquisition source base surface and the flaw of the image recording layer. Is to get. The second object is that if sufficient aberration correction is performed on the visible image information, even if there is a slight residual aberration or a color shift from the visible image information, the visible image To effectively remove the effects of scratches and dust. Further, a third object is to provide a natural image without making the influence of the correction inconspicuous when the post-processing for edge enhancement, noise removal, enlargement, and reduction processing is performed on the image on which the correction of the scratches and dust is performed. It is to obtain the processing result.
[0017]
[Means for Solving the Problems]
In order to solve the above-mentioned problem, an invention according to claim 1 obtains image information on at least two wavelength bands of an infrared region and a visible region in an image document, performs image processing, and obtains output image information. In the processing method,
Using the image information in the infrared region, obtain an image defect area,
Estimating the effect of the image defect area on the image information in the visible region, to correct the effect, to correct the signal intensity of the image information in the visible region,
The quality of the correction is evaluated, and when the evaluation result is negative, the image defect area in the visible region is processed based on the value of a normal pixel near the defect area.
The image defect is, for example, a defective portion caused by a scratch (including damage) or dust. The image defect area described later is an area of the image defect.
[0018]
The invention according to claim 11 is an image processing apparatus that obtains output image information by performing image processing by acquiring image information on at least two wavelength bands of an infrared region and a visible region in an image document,
Using the image information in the infrared region, an image defect area acquisition unit that acquires an image defect area,
Estimating the effect of the image defect area on the image information in the visible region, correcting the influence, correcting the signal strength of the image information in the visible region, evaluating the quality of the correction, and evaluating the result. If the result is negative, there is provided an image processing executing means for processing the image defective area in the visible region based on the value of the normal pixel near the defective area.
[0019]
According to a twenty-first aspect of the present invention, there is provided a computer for realizing a function of acquiring image information on at least two wavelength bands of an infrared region and a visible region of an image document, performing image processing, and obtaining output image information. In the image processing program,
To the computer,
Using the image information in the infrared region, an image defect area acquisition function for acquiring an image defect area,
Estimating the effect of the image defect area on the image information in the visible region, correcting the influence, correcting the signal strength of the image information in the visible region, evaluating the quality of the correction, and evaluating the result. If the result is no, an image processing execution function of performing processing of the image defect area in the visible region based on the value of the normal pixel in the vicinity of the defect area,
Is realized.
[0020]
According to the invention as set forth in claim 1, 11 or 21, an image defect area is acquired using image information in the infrared region, and the influence of the image defect region on image information in the visible region is corrected. Is evaluated, and the image defect area in the visible region is processed according to the evaluation result. For this reason, for example, an image defect occurs due to a defect in the image recording layer from which the image information is acquired, and the correction of the signal intensity by the image information in the infrared region is reversely corrected. Even if a slight shift occurs in a part of the screen, it is possible to evaluate the correction result of the image area where the defect has occurred, so it is necessary to select and execute processing according to the image area With this, it is possible to deal effectively.
[0021]
According to a second aspect of the present invention, in the image processing method according to the first aspect,
In the evaluation, the representative values of the image defect area and its peripheral area in the corrected visible area image information and the uncorrected visible area image information are compared and evaluated. And
[0022]
According to a twelfth aspect of the present invention, in the image processing apparatus according to the eleventh aspect,
The image processing execution unit compares and evaluates representative values of the image defect area and its peripheral area in the corrected visible area image information and the uncorrected visible area image information. It is characterized by the following.
[0023]
According to a twenty-second aspect of the present invention, in the image processing program according to the twenty-first aspect,
The image processing execution function compares and evaluates respective representative values of the image defect area and its peripheral area in the corrected visible area image information and the uncorrected visible area image information. It is characterized by the following.
[0024]
According to the second, twelfth, or twenty-second aspect, the correction result is evaluated by comparing the representative values of the image defect area and the surrounding area in the image information of the visible area between the corrected and the uncorrected. Therefore, the evaluation of the correction result can be favorably performed by a simple method.
[0025]
According to a third aspect of the present invention, there is provided an image processing method for acquiring image information on at least two wavelength bands of an infrared region and a visible region of an image document, performing image processing, and obtaining output image information.
Using the image information in the infrared region, obtain an image defect area,
A texture structure in the image defect area of the image information in the infrared region is evaluated, and the image information in the visible region is corrected based on the evaluation result.
[0026]
According to a thirteenth aspect of the present invention, there is provided an image processing apparatus for acquiring image information on at least two wavelength bands of an infrared region and a visible region of an image document, performing image processing, and obtaining output image information.
Using the image information in the infrared region, an image defect area acquisition unit that acquires an image defect area,
Image processing execution means for evaluating a texture structure in the image defect area of the image information in the infrared region and correcting the image information in the visible region based on the evaluation result.
[0027]
According to a twenty-third aspect of the present invention, a computer has a function of acquiring image information on at least two wavelength bands of an infrared region and a visible region of an image document, performing image processing, and obtaining output image information. In the image processing program of
To the computer,
Using the image information in the infrared region, an image defect area acquisition function for acquiring an image defect area,
An image processing execution function for evaluating a texture structure in the image defect area of the image information in the infrared region and correcting the image information in the visible region based on the evaluation result;
Is realized.
[0028]
According to the third, thirteenth, or twenty-third aspect of the present invention, a preferable correction method is selected and executed for the visible-range image information according to the feature of the texture structure in the image defect area of the infrared-range image information. it can. Further, for example, an image defect occurs due to a defect in the image recording layer from which the image information is obtained, and the correction of the signal intensity by the image information in the infrared region is inversely corrected. Even if a slight deviation occurs in a part of the screen, a preferable correction can be selected and executed, and the problem can be effectively dealt with. Further, even if there is residual aberration in image information in the infrared region or color shift between image information in the infrared region and the visible region, it is possible to perform favorable image processing corresponding to this.
[0029]
According to a fourth aspect of the present invention, in the image processing method according to any one of the first to third aspects,
When performing a correction process and an interpolation process on the same image information in the visible region, the correction process is performed first, the corrected region is regarded as a normal pixel, and then the interpolation process is performed.
[0030]
According to a fourteenth aspect of the present invention, in the image processing apparatus according to any one of the eleventh to thirteenth aspects,
When performing a correction process and an interpolation process on the same image information in the visible region, the image processing execution unit first performs the correction process, sets a region after correction as a normal pixel, and then performs the interpolation process. It is characterized by performing.
[0031]
The invention according to claim 24 is the image processing program according to any one of claims 21 to 23, wherein:
The image processing execution function, when performing the correction process and the interpolation process on the same image information of the visible region, first performs the correction process, the corrected area as a normal pixel, then the interpolation process It is characterized by performing.
[0032]
According to the invention described in claim 4, 14, or 24, the number of normal pixels as peripheral image information that can be used in the interpolation processing can be increased to the maximum by the correction processing, so that more reliable interpolation processing can be performed. To obtain an interpolation result. Further, for example, it is possible to effectively cope with a problem in which an image loss occurs due to a loss in the image recording layer from which the image information is obtained, and the correction of the signal intensity by the image information in the infrared region is reversely corrected. Since interpolation is performed using normal pixels, a highly accurate image processing result can be obtained.
[0033]
According to a fifth aspect of the present invention, an image defect is detected from image information, and image processing for correcting and / or interpolating the image defect is performed on the image information. An image processing method for performing at least one post-process of adjustment of image quality, adjustment of graininess, and adjustment of image size,
The image information is located in the vicinity of the first pixel that has been subjected to the correction processing and / or the interpolation processing and the first pixel, and has not been subjected to the correction and / or the interpolation processing. Classifying into a second pixel and a third pixel that does not belong to any of the first pixel and the second pixel, and performing the post-processing based on classification information regarding the classification. .
[0034]
According to a fifteenth aspect of the present invention, an image defect is detected from the image information, and image processing for correcting and / or interpolating the image defect is performed on the image information. An image processing apparatus that performs at least one post-processing of the adjustment of the image quality, the adjustment of the graininess, and the adjustment of the image size.
The image information is located in the vicinity of the first pixel that has been subjected to the correction processing and / or the interpolation processing and the first pixel, and has not been subjected to the correction and / or the interpolation processing. An image processing execution unit that classifies the image into a second pixel and a third pixel that does not belong to any of the first pixel and the second pixel, and performs the post-processing based on the classification information on the classification. It is characterized by having.
[0035]
According to a twenty-fifth aspect of the present invention, a computer detects an image defect from image information, performs image processing for correcting and / or interpolating the image defect on the image information, and performs processing on the processed image information. In an image processing program for realizing a function of executing at least one post-process among adjustment of sharpness, adjustment of graininess, and adjustment of image size,
To the computer,
The image information is located in the vicinity of the first pixel that has been subjected to the correction processing and / or the interpolation processing and the first pixel, and has not been subjected to the correction and / or the interpolation processing. An image processing execution function of classifying the pixel into a second pixel and a third pixel that does not belong to any of the first pixel and the second pixel, and performing the post-processing based on the classification information on the classification; It is characterized by realizing.
[0036]
According to the invention described in claim 5, 15 or 25, the region to which the corrected and / or interpolated first pixel belongs, the region to which the uncorrected and / or interpolated second pixel belongs, and the third pixel Since various post-processes are switched and executed in accordance with the classification information of the area to which the image belongs, a more natural image processing result is obtained in which traces of correction of defective pixels hardly remain.
[0037]
According to a sixth aspect of the present invention, in the image processing method according to the fifth aspect,
In the classification, the first pixels are classified into a first group subjected to the correction processing and a second group subjected to the interpolation processing. The post-processing is performed on the first pixel based on information.
[0038]
According to a sixteenth aspect of the present invention, in the image processing apparatus according to the fifteenth aspect,
In the classification, the image processing execution unit may classify the first pixel into a first group that is a target of the correction processing and a second group that is a target of the interpolation processing. The post-processing is performed on the first pixel based on classification information and group information on the group.
[0039]
The invention according to claim 26 is the image processing program according to claim 25, wherein:
The image processing execution function is such that the first pixels are classified into a first group subjected to the correction processing and a second group subjected to the interpolation processing. The post-processing is performed on the first pixel based on group information about a group.
[0040]
According to the invention described in claim 6, 16 or 26, in addition to the classification information, according to the group information of the first group to be corrected and the second group to be interpolated in the image area of the first pixel. Since various post-processings are switched and executed for the first pixel, a more natural image processing result is obtained in which traces of correction hardly remain.
[0041]
According to a seventh aspect of the present invention, there is provided an image which detects an image defect from image information, performs image processing for removing the image defect by a correction process and an interpolation process according to characteristics of the image defect, and obtains output image information. In the processing method,
The correction processing is first performed on the same image information, and the interpolation processing is performed next, with the corrected area as a normal pixel.
[0042]
The invention according to claim 17 detects an image defect from image information, performs image processing for removing the image defect by using a correction process and an interpolation process according to the characteristics of the image defect, and outputs output image information. In the obtained image processing device,
The image processing apparatus further includes an image processing execution unit that first performs the correction process on the same image information, sets a region where the correction is completed as a normal pixel, and then performs the interpolation process.
[0043]
According to a twenty-seventh aspect of the present invention, a computer detects an image defect from image information, performs image processing for removing the image defect by using a correction process and an interpolation process according to characteristics of the image defect, and outputs the image. In an image processing program for realizing a function of obtaining image information,
To the computer,
It is characterized in that the correction processing is first performed on the same image information, and an area where the correction is completed is regarded as a normal pixel, and an image processing execution function of performing the interpolation processing is realized.
[0044]
According to the invention of claim 7, 17 or 27, the number of normal pixels as peripheral image information that can be used in the interpolation processing can be increased to the maximum by the correction processing, so that more reliable interpolation processing can be performed. To obtain an interpolation result. Further, for example, it is possible to effectively cope with a problem in which an image loss occurs due to a loss in the image recording layer from which the image information is acquired, and the correction of the signal intensity by the image information in the infrared region is the reverse correction. Furthermore, since interpolation is performed using the corrected normal pixels, a highly accurate image processing result can be obtained.
[0045]
According to an eighth aspect of the present invention, in the image processing method according to the seventh aspect,
In the image processing by the interpolation processing, an image structure of a peripheral area of the image defect is evaluated based on signal value gradient information of a normal pixel not included in the area of the image defect, and an interpolation processing is performed based on the evaluation result. Is performed.
[0046]
The invention according to claim 18 is the image processing device according to claim 17, wherein
The image processing execution unit evaluates an image structure of a peripheral region of the image defect based on signal value gradient information of a normal pixel not included in the image defect region, and performs an interpolation process based on the evaluation result. It is characterized by performing.
[0047]
The invention according to claim 28 is the image processing program according to claim 27, wherein:
The image processing execution function evaluates an image structure of a peripheral region of the image defect based on signal value gradient information of a normal pixel not included in the image defect region, and performs an interpolation process based on the evaluation result. It is characterized by performing.
[0048]
According to the invention described in claim 8, 18 or 28, the image structure of the peripheral area of the image defect is evaluated based on the signal value gradient information of the normal pixel not included in the area of the image defect, and the evaluation result Since the interpolation processing is performed based on, for example, information on the image structure such as an edge can be restored to the extent possible, the algorithm used for the interpolation processing functions more effectively, and the interpolation result is further improved by executing the interpolation processing more reliably. Can be obtained.
[0049]
According to a ninth aspect of the present invention, image information of at least two wavelength bands of an infrared region and a visible region of an image document is acquired, and an image defect area is obtained using the infrared region image information. A correction value for compensating for the influence of the image defect area is obtained, a correction value used when the correction value is applied to a visible signal is set, a correction correction value is obtained by a product of the correction value and the correction value, and the correction is performed. In an image processing method for performing image processing for correcting the image information in the visible region with a correction value, and obtaining output image information,
In the setting of the correction value, the length of the shortest line segment in the image defect area among the line segments passing through the pixel of interest in each of the target pixels present in the image defect area is set to The correction value is set based on the represented line segment distance information.
[0050]
The invention according to claim 19 obtains image information on at least two wavelength bands of an infrared ray and a visible ray of an image original, obtains an image defect area using the infrared ray image information, A correction value for compensating for the influence of the image defect area is obtained, a correction value used when the correction value is applied to a visible signal is set, a correction correction value is obtained by a product of the correction value and the correction value, and the correction is performed. In an image processing device that performs image processing for correcting the image information in the visible region with a correction value and obtains output image information,
For each pixel of interest present in the image defect area, based on line segment distance information indicating the length of the shortest line segment in the image defect area among the line segments passing through the pixel of interest in a plurality of directions. Correction value setting means for setting the correction value.
[0051]
According to a twenty-ninth aspect of the present invention, a computer acquires image information of at least two wavelength bands of an infrared region and a visible region of an image document, and uses the image information of the infrared region to determine an image defect area. Obtain, obtain a correction value for compensating for the effect of the image defect area, set a correction value used when applying the correction value to a visible signal, and obtain a correction correction value by multiplying the correction value and the correction value. An image processing program for performing image processing for correcting the image information in the visible region with the correction correction value and realizing a function of obtaining output image information,
To the computer,
For each pixel of interest present in the image defect area, based on line segment distance information indicating the length of the shortest line segment in the image defect area among the line segments passing through the pixel of interest in a plurality of directions. A correction value setting function of setting the correction value.
[0052]
According to the ninth, nineteenth, or twenty-ninth aspect, the magnitude of the influence of the image defect area is evaluated according to the shortest line segment passing through the pixel of interest, and the image information in the visible region is corrected based on the evaluation result. Since the correction is performed, in the normally assumed image information (the number of normal pixels >> the number of abnormal pixels), the magnitude of the influence of the image defect area on the target pixel is correctly evaluated, and based on a correction value according to the evaluation result. Can be corrected. Therefore, even if residual aberrations and flare components remain in image information in the infrared region, the effects of these components on image information in the infrared region are compensated for, and preferable image processing can be realized.
[0053]
According to a tenth aspect of the present invention, in the image processing method according to the ninth aspect,
Pixels existing around the image defect area are defined as a peripheral area,
In setting the correction value,
For each pixel of interest present in the image defect area, the line segment distance information and a pixel group in the peripheral region overlapping a line segment corresponding to the line segment distance information are extracted, and Obtain a temporary visible image correction value from the pixel group and the pixel of interest,
Obtain a divisor value obtained by dividing the correction value in the infrared region by the visible image correction value, plot the relationship between the divisor value and the line segment distance information, obtain a representative curve from the plot result, and obtain the representative curve. The correction value is set based on the correction value.
[0054]
The invention according to claim 20 is the image processing device according to claim 19, wherein:
Pixels existing around the image defect area are defined as a peripheral area,
The correction value setting unit extracts, for each pixel of interest present in the image defect area, the line segment distance information and a pixel group in the peripheral region overlapping a line segment corresponding to the line segment distance information. A temporary visible image correction value is obtained from the pixel group and the pixel of interest in the image information of the visible region, and a divisor obtained by dividing the correction value of the infrared region by the visible image correction value is obtained. The relation with the line segment distance information is plotted, a representative curve is obtained from the plot result, and the correction value is set based on the representative curve.
[0055]
The invention according to claim 30 is the image processing program according to claim 29, wherein:
Pixels existing around the image defect area are defined as a peripheral area,
The correction value setting function extracts, for each pixel of interest existing in the image defect area, the line segment distance information and a pixel group in the peripheral region overlapping with a line segment corresponding to the line segment distance information. A temporary visible image correction value is obtained from the pixel group and the pixel of interest in the image information of the visible region, and a divisor obtained by dividing the correction value of the infrared region by the visible image correction value is obtained. The relation with the line segment distance information is plotted, a representative curve is obtained from the plot result, and the correction value is set based on the representative curve.
[0056]
According to the invention as set forth in claim 10, 20, or 30, a temporary visible image correction value is obtained from image information in the visible region, and a difference between the correction value in the infrared region by the visible image correction value and the line segment distance information is calculated. Obtain a representative curve showing the relationship. For this reason, the correction value is determined by the line segment distance information using the representative curve, and the image information in the visible region can be corrected using the correction value and the correction value in the infrared region, and the unknown image information acquisition source is also stable. The effect of removing damaged scratches and dust is obtained.
[0057]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the scope of the invention is not limited to the illustrated example.
[0058]
First, the features of the apparatus according to the present embodiment will be described with reference to FIGS. FIG. 1 is a block diagram illustrating an image processing system 10 according to the present embodiment. FIG. 2 is a diagram showing a configuration of the transparent document scanner 142 of FIG.
[0059]
As shown in FIG. 1, an image processing system 10 such as a digital minilab system according to the present embodiment includes an image processing unit 11, an image display unit 12, an instruction input unit 13, an image acquisition unit 14, an image storage unit 15, a silver halide exposure printer 16, an IJ (inkjet) printer 17, and various image recording media writing units 18.
[0060]
The image processing unit 11 performs image processing on various image information acquired by the image acquisition unit 14 and / or image information stored in the image storage unit 15 based on various instruction information input by the instruction input unit 13. , An image storage unit 15, a silver halide exposure printer 16, an IJ printer 17, and various image recording media writing units 18. Further, the image processing unit 11 outputs image information to the image display unit 12.
[0061]
The image acquiring unit 14 scans an image recorded on a printed material 21 such as a photographic print, a printed material, and a document to acquire image information, and a recorded image recorded on a developed film 22 such as a negative film or a positive film. Document scanner 142 that scans a scanned image to acquire image information, and a medium that reads and acquires image information recorded on image media 23 such as a DSC (digital still camera), a CD-R, and an image memory card. A driver 143 and an information communication I / F 144 for receiving and acquiring image information from communication means 24 such as the Internet and a LAN, and for receiving and acquiring image information stored in the image storage unit 15. You. The input image information input from each of the acquisition units 141 to 144 is output to the image processing unit 11.
[0062]
The image display unit 12 is configured by an LCD (Liquid Crystal Display), a CRT (Cathode Ray Tube), a PDP (Plasma Display Panel), an EL (Electro Luminescent) display, or the like. Display data on the screen.
[0063]
The instruction input unit 13 is provided in the image display unit 12, and obtains a contact signal input by a contact by an operator and position information on the screen thereof, and outputs a signal to the image processing unit 11 as a touch sensor 131. A mouse 132 that acquires a position signal and a selection signal input by the user and outputs the signal to the image processing unit 11, a cursor key, numeric input keys, various function keys, and the like. And a keyboard 133 that acquires a signal and outputs the signal to the image processing unit 11.
[0064]
The image storage unit 15 is configured by an HDD (hard disk drive) or the like, and stores image information in a readable and writable manner. The silver halide exposure printer 16 prints image information as an image on a photographic paper or a film by a silver halide exposure method. The IJ printer 16 prints image information as an image on a recording paper or the like by an inkjet method. The various image recording medium writing unit 18 writes and records image information on an image recording medium such as a flexible disk, a CD-R, a DVD-R, and an image memory card.
[0065]
The image processing unit 11 performs flaw / dust processing for repairing an image on which a document is damaged or dust is attached, using various methods described later. In the image processing unit 11, the output image information in which the scratches and dust are repaired by the scratch and dust processing is output to various output destinations (the image storage unit 15, the silver halide exposure printer 16, the IJ printer 17, and the various image recording media writing units 18). ), And is transmitted to the output destination. The silver halide exposure printer 16 and the IJ printer 17 print the output image information received from the image processing unit 11. The various image recording media writing unit 18 writes the output image information received from the image processing unit 11 to various image recording media. The image storage unit 15 stores the output image information received from the image processing unit 11. The image information stored in the image storage unit 15 is stored and can be reused by reading out the image processing unit 11 as an image source.
[0066]
The configuration of the image processing system 10 has a feature in the image processing unit 11, and includes an image acquisition unit 14, an image display unit 12, an instruction input unit 13, a silver halide exposure printer 16, an IJ printer 17, and various types of image recording. The type and number of components of the media writing unit 18 are not limited to the example of FIG.
[0067]
Next, the configuration of the transparent document scanner 141 in the image acquisition unit 14 of the image processing system 10 will be described with reference to FIG. The transmissive document scanner 141 includes a light source unit 31 that emits light for scanning a transmitted image, a diffusion member 32 that uniformizes (equals) light emitted from the light source unit 31 and emits the light, and a film A. A film carrier 33 for transporting the film A in the direction of the arrow by rollers 34 for transporting the film A two-dimensionally, and an optical lens 35 for forming an image of the light beam passing through the film A. The light emitted from the light source unit 31 and the diffusion member 32 is linear light, and the light corresponding to the image on the film A is transmitted by the linear light and the conveyance of the film A.
[0068]
The transmission original scanner 141 includes dichroic filters 36IR, 36B, 36R, line CCDs (Charge Coupled Devices) 37IR, 37B, 37G, 37R, analog amplifiers 38IR, 38B, 38G, 38R, and A / A. It comprises D converters 39IR, 39B, 39G, 39R and an image memory 40.
The light beams emitted from the optical lens 35 are separated by the dichroic filters 36IR, 36B, 36R, and correspond to lines of blue (B), green (G), and red (R) in the infrared and visible regions, respectively. An image is formed on the CCDs 37IR, 37B, 37G, and 37R, received, and converted into electric signals. The electric signals are amplified by analog amplifiers 38IR, 38B, 38G, 38R, converted into digital signals by A / D converters 39IR, 39B, 39G, 39R, stored in an image memory 40, and used. The above-mentioned line CCDs are arranged perpendicular to the direction of transport of the film A, and the arrangement of the line CCDs is main scanning, and the above-described film transport is sub-scanning.
[0069]
Light in the green (G) color gamut transmitted without being reflected by the dichroic filters 36IR, 36B, 36R is imaged and received on the line CCD 37G. It is assumed that image information corresponding to an infrared region is an invisible image, and image information corresponding to a blue region, a green region, and a red region is visible image information. That is, the visible image information is image information in which signals of the blue, green, and red regions (R image information, G image information, and B image information) are combined.
[0070]
Next, the operation of the image processing system 10 will be described with reference to FIGS. FIG. 3 is a flowchart showing the image processing. First, the image processing executed in the image processing unit 11 of the image processing system 10 will be described with reference to FIG.
[0071]
For example, the image processing unit 11 is provided with a CPU, a RAM, and a storage unit (not shown). The image processing program stored in the storage unit is read and stored in the RAM, and the cooperation between the CPU and the image processing program in the RAM is performed. By the operation, the image processing is executed. The execution configuration of the process in the image processing unit 11 is the same as the execution configuration of each process described below. In this configuration, the image processing unit 11 has a function as an image defect area acquisition unit, an image processing execution unit, and a setting unit of a correction value setting unit.
[0072]
It is assumed that various image information is acquired in advance by the image acquiring unit 14 (hereinafter, referred to as acquired image information), and the acquired image information is input to the image processing unit 11. The image processing is a process of adding a predetermined process to the visible image information of the acquired image information and outputting the same to each of the output units 15 to 18 as output image information. Hereinafter, unless otherwise specified, it is assumed that the subject of the processing is the image processing unit 11.
[0073]
First, input color conversion is performed on the acquired image information, and flaw / dust processing described later is executed (step S1). The input color conversion is a color conversion in accordance with an input attribute of each of the obtaining units 141 to 144 of the image obtaining unit 14. For example, a sensor (CCD) receives a film transmitted light amount, and converts a signal value converted into a digital signal into a visual signal value. And a function of converting an image signal into a meaningful unit system such as an optical density, and a process of matching a color tone expressed according to each spectral characteristic to a standard color space. The flaw / dust processing is processing for removing the influence of flaws and dust on an image information acquisition source (such as a film) that causes defective pixels from visible image information of acquired image information. Further, an area on the acquired image information resulting from the scratch or dust is defined as a scratch or dust area.
[0074]
Then, an instruction to make the color and brightness of the visible image information subjected to the flaw / dust processing appropriate is input from the instruction input unit 13, and the color and brightness are adjusted (step S2). The visible image information whose color and brightness have been adjusted is displayed on the image display unit 12, the image information is visually checked by the operator, and the operator inputs an instruction to evaluate whether the color and brightness are appropriate or not. Input from the unit 13 (step S3).
[0075]
Then, it is determined whether or not the input evaluation is OK (step S4). When the evaluation is NG (step S4; NO), the process proceeds to step S2. The color and brightness are again adjusted and input by the operator. If the evaluation is OK (step S4; YES), the color and brightness-adjusted visible image information is desired as needed based on the flaw / dust processing information output in the flaw / dust processing in step S1. (Step S5). The scaled-down visible image information is subjected to noise removal processing for removing noise (step S6). The visible image information from which the noise has been removed is subjected to sharpness / granularity correction processing for appropriately correcting sharpness and granularity based on the flaw / dust processing information (step S7). In the image processing of each of the steps S5 to S7, as will be described later, information on the flaw / dust processing obtained from the step S1 can be used as sub-information.
[0076]
The visible image information that has been subjected to the sharpness / granularity processing is subjected to various processes such as a rotation process, an inset synthesis process of various images such as an image frame, and a character insertion process (step S8).
[0077]
Then, output color conversion is performed on the processed visible image information. The output color conversion includes a process of matching a color space corresponding to an output attribute of each of the output units 15 to 18. The output image-converted visible image information is output to at least one of the output units 15 to 18 (step S9), and the image processing ends. The image information subjected to the output color conversion is stored in the image storage unit 15, recorded on a photosensitive material (such as photographic paper) by the silver halide exposure printer 16, recorded on recording paper by the IJ printer 17, and written on various image recording media. The recording into various image recording media by the embedding unit 18 is performed. The details of the enlargement / reduction processing in step S5 and the sharpness / granularity correction processing in step S7 will be described later.
[0078]
Next, the flaw / dust processing in step S1 of the image processing in FIG. 3 will be described with reference to FIGS. FIG. 4 is a flowchart showing the flaw / dust processing. FIG. 5 is a diagram illustrating the absorption of signals in each wavelength range.
[0079]
First, visible image information indicating a visible image in acquired image information and infrared image information indicating an infrared image are acquired (step S11). Then, the infrared image information is corrected by the masking process (Step S12).
[0080]
FIG. 5 schematically illustrates the theory of extracting the flaw and dust information from the infrared image information. The dyes that make up the image have different light absorbencies, which are received as B, G, and R signals. However, it shows almost no absorption of infrared light, except for some absorption of the C dye. On the other hand, since scratches and dust have absorption or dispersion characteristics in both visible and infrared light, the position of the scratches and dust can be detected by observing infrared image information. "Damage" at the right end indicates a situation in which the image information carrier itself is affected even among the scratches, and some image information is lost. In the example of “damage” at the right end, the correlation (large or small change) between the influence of the flaw on the infrared signal and the influence of the visible signal is different from that of the “flaw or dust”. There is a danger that this flaw may be reversely corrected by using the method of Patent Document 3 described in the related art. However, as shown in FIG. 4 (described later), in this embodiment, the correction result is evaluated. It is possible to detect the occurrence of the correction and shift to the interpolation processing.
[0081]
As described above, the IR light is largely absorbed by the scratch, dust, or damaged portion, but is slightly absorbed by the C dye. Therefore, in order to extract only the scratches, dust, and damaged portions, in step S12, the absorption component of the C dye is removed by a masking process, and infrared reference data as infrared reference image information of the IR signal is subtracted. Then, infrared difference data indicating only the flaw, dust, and the damaged part is calculated, and the positions of the flaw, dust, and the damaged part are specified. Hereinafter, unless otherwise specified, “scratch, dust” refers to “scratch, dust and damage”.
[0082]
Then, a flaw / dust area in the infrared image information is detected based on the masked infrared image information (infrared difference data relating to the infrared image information), and a flaw / dust in the visible image information is detected based on the flaw / dust area in the infrared image information. A dust candidate area is detected (step S13). Next, a flaw / dust area in the visible image information is determined from the flaw / dust candidate area in the visible image information (step S14). Based on the flaw / dust area, flaw / dust correction processing is performed on the visible image information (step S15). The correction result of the flaw / dust correction processing is automatically evaluated (step S16).
[0083]
Then, it is determined whether or not the correction result of the flaw / dust processing is appropriate based on the evaluation result (step S17). If the evaluation is OK (step S17; YES), the flaw / dust processing is terminated. When the evaluation is NG (step S17; NO), the flaw / dust interpolation processing is performed on the image information on which the flaw / dust correction processing has been performed (step S18), and the flaw / dust processing ends. After the end of the flaw / dust processing, the process proceeds to the next step (step S2 in FIG. 3).
[0084]
Here, the difference between the flaw / dust correction processing in step S15 and the flaw / dust interpolation processing in step S18 will be described. Broadly speaking, two methods are generally known as methods for treating flaws and dust. One is that the signal is affected by attenuation or the like due to the influence of scratches or dust, and this effect is corrected. In the present embodiment, this is referred to as correction processing. The other is to attempt to restore a region where image information is lost due to the influence of scratches and dust using surrounding information. In the present embodiment, this is referred to as interpolation processing.
[0085]
Next, the flaw / dust candidate area detection processing corresponding to steps S11 to S13 during flaw / dust processing in FIG. 4 will be described with reference to FIG. FIG. 6 is a flowchart showing the flaw / dust candidate area detection processing. First, step S131 is the same as step S11. Then, the infrared image information is corrected (Step S132). Here, step S132 will be described in detail.
[0086]
An image correlation between each data (signal value) of the infrared image information and each red data in the visible image information is calculated (Step S133). Then, a correction constant for removing the red component corresponding to the C dye from the infrared image information is calculated (step S134). Then, the correction constant is applied to the red image information, and the product is subtracted from the infrared image information (step S135). In this way, the red component is removed from the infrared image information, and infrared image information in which only the influence of scratches and dust appears is obtained.
[0087]
Then, after step S132, infrared reference data is created as a signal value when there is no absorption of a dye, a scratch, or the like in the infrared image information (step S136). Then, the infrared reference data is subtracted from the infrared-corrected infrared image information obtained in step S132 (S135), and infrared difference data that is a signal value of only a flaw or dust component is created (step S132). S137).
[0088]
Then, an appropriate threshold value is determined, and the infrared difference data is binarized based on the threshold value (step S138). The binarized infrared difference data is subjected to noise processing (step S139). In the noise processing, for example, a signal value may be re-binarized after smoothing using a smoothing filter.
[0089]
Here, an example of a typical spatial frequency band filter used for the noise processing and the sharpness correction will be briefly described.
[0090]
First, the numbering of the image information is performed as shown in Table 1 below. Table 1 is a table showing a signal value of each pixel of the image information and a positional relationship of the pixel.
[Table 1]

[0091]
(Sharpness enhancement or smoothing filter)
In the case of sharpness enhancement or smoothing filter, the following calculation result is obtained using information of the central 5 * 5 pixels.

However, field [1] to field [7] are predetermined constants.
[0092]
Then, the following restrictions are set on FP1.

[0093]
Then, a new central pixel value, p55 ', is obtained by the following equation.
p55 '= p55 + FP1
[0094]
This processing example is a particularly preferable processing example in image processing from a silver halide film, and is an embodiment of an edge enhancement filter of 5 * 5 pixels. By increasing divdat, the effect of the sharpness enhancement filter is weakened, and by decreasing divdat, the effect of the sharpness enhancement filter is enhanced. If the upper limit value and the lower limit value are set to be small, the problem that the sesame salt (independent point) noise is extremely enhanced is reduced, smooth tone reproduction is obtained, and if the limit value is not increased or the limit is not provided, A natural edge enhancement effect can be obtained.
[0095]
Also, by changing the value of field, it also functions as a smoothing filter. When the setting of the threshold is not necessary, the following equation can be used, and the smoothing filter can be easily designed.
[0096]
Using the information of the central 5 * 5 pixels, the following calculation result is obtained to obtain a new central pixel value, p55 '.

However, field [1] to field [7] are predetermined constants.
These various parameters can be defined as correction amounts, and can be changed for each area according to the purpose.
[0097]
(Band cut filter)
In the case of a band cut filter, the following calculation result is obtained using an image of 9 * 9 pixels.

[0098]
Then, the following restrictions are imposed on FP2.

Then, a new central pixel value, p55 'is obtained by the following equation.
p55 '= p55 + FP2
[0099]
This processing example is an embodiment of a band cut filter of 9 * 9 pixels. Increasing the threshold 2 increases the signal removal effect in the target spatial frequency band, and suppresses the low-frequency fluctuations of the signal. These various parameters can be defined as correction amounts, the characteristics can be adjusted by changing the reference range of the peripheral data, and can be changed for each area or for each model.
[0100]
(Noise correction)
In the noise correction, numbering of images is performed as shown in Table 2 below. Table 2 is a table showing a signal value of a pixel of image information and a positional relationship of the pixel. Here, P: central pixel, X, Y: peripheral pixels.
[Table 2]

[0101]
Regarding the innermost combination in the X direction,
abs (X1 '+ X1-2 * P) <threshold
Is satisfied, X1 and X1 'are added to the data group to be averaged.
Then, it repeats up to the next one, the next,... Until the above determination formula is not satisfied or until the maximum radius (for example, 4 pixels) specified as an initial value in advance.
[0102]
The same is repeated for the Y direction.
Then, a simple average of the data added to the data group and the center pixel P is obtained, and is set as a new center pixel P '.
[0103]
When the threshold value is set to a large value in the above-described method, the noise removal effect is increased, while fine details may disappear. In addition, the "maximum radius specified as an initial value" is switched to change how much noise can be removed. In this example, there are the above two parameters, and the set value can be changed for each area.
In step S139 of the present embodiment, as an example of another noise filter, a contraction process (erosion, closing; hereinafter, referred to as closing), which is a form of morphological processing, is performed to remove isolated points. I use a technique to do it.
[0104]
The continuation of the flaw / dust candidate area detection processing will be described. The noise-differenced infrared difference data indicates a flaw / dust area on the infrared image, and a flaw / dust area expansion process of the infrared image information is performed to apply the area to a visible image (step S13A). The flaw / dust area on the infrared image information is expanded by a predetermined amount so as to surely include the flaw / dust area on the visible image information. In the flaw / dust area expansion processing, a necessary amount of expansion processing (dilation, opening; hereinafter, referred to as opening), which is known as one form of morphology processing, is performed. In the case where smoothing or another noise filter is used as the above-described noise filter, the same processing as the flaw / dust area expansion processing can be performed even when the threshold value is adjusted when the signal value is re-binarized. .
[0105]
Then, the pixels of the visible image information corresponding to the pixels not included in the extended flaw / dust area are determined as normal pixels (step S13B), and the flaw / dust candidate area detection processing is terminated. A pixel area that is not a normal pixel in the visible image information is set as a flaw / dust candidate area in the visible image information. After the end of the flaw / dust candidate area detection processing, the process moves to the next step (step S14 in FIG. 4).
[0106]
The method of determining the flaw / dust candidate area is not limited to one method. In addition to using the infrared image as in the present embodiment, a surface reflection image may be collected by a reflection document scanner, and a flaw / dust candidate area may be obtained from the surface discontinuity. The reflection original scanner that performs this method will be described later. In addition, in the case of a transmissive original, the light quality of the irradiation light source is switched, for example, transmitted light and reflected light, or a condensing light source that irradiates a substantially parallel light beam to the film and a diffusion box, and soft light is used. The diffused light source applied to the film may be alternately switched, an image may be collected, and a flaw / dust candidate area may be detected from the difference.
[0107]
Next, the flaw / dust area determination processing in step S14 during flaw / dust processing in FIG. 4 will be described with reference to FIG. FIG. 7 is a flowchart illustrating the flaw / dust area determination processing. The flaw / dust area determination processing is processing for determining a flaw / dust area by excluding normal pixels from flaw / dust candidate areas in visible image information.
[0108]
As shown in FIG. 7, first, visible image information is obtained in the same manner as in step S131 (step S141). One pixel is extracted as a pixel of interest from the obtained visible image information (step S142). Then, it is determined whether or not the target pixel is within the flaw / dust candidate area of the visible image information (step S143). If the target pixel is in the flaw / dust candidate region of the visible image information (step S143; YES), normal pixels in the vicinity of the target pixel are extracted (step S144). Then, the association between the extracted normal pixel and the target pixel is evaluated (step S145). Then, it is determined whether or not the relevance between the target pixel and the nearby normal pixel is high (step S146).
[0109]
Various evaluations of the association between the target pixel and the normal pixels in the vicinity can be considered. For example, a pixel whose absolute value of a signal value difference between a nearby normal pixel and a target pixel falls within a predetermined threshold is extracted, and if the number of pixels is equal to or more than a predetermined number, the pixel can be regarded as a normal pixel. The predetermined threshold value may be a fixed value. For example, when the noise filter is functioning after the image processing, the predetermined threshold value is determined according to the strength thereof (when the noise filter holds the threshold value, the threshold value). May be. In this way, the effect of minute scratches is eliminated by the noise filter at the subsequent stage, so there is no need to apply scratch dust processing here, and it is possible to minimize the area where scratch dust processing is required and minimize image processing capacity. Is improved.
[0110]
If the relevance is not high (step S146; NO), the target pixel is determined to be within the flaw / dust area (step S147). Then, it is determined whether or not all the pixels have been extracted in step S142 (step S148). If all the pixels have been extracted (step S148; YES), the flaw / dust area determination processing ends.
[0111]
If the target pixel is not in the flaw / dust candidate area of the visible image information (step S143; NO) or has a high relevance (step S146; YES), the target pixel is determined to be a normal pixel (step S149), and step S148 is performed. Will be migrated to. If all the pixels have not been extracted (step S148; NO), the process proceeds to step S142. After the end of the flaw / dust area determination processing, the process moves to the next step (step S15 in FIG. 4).
[0112]
Next, a first flaw / dust correction processing as an example of flaw / dust correction processing corresponding to steps S11 to S15 in the flaw / dust processing of FIG. 4 will be described with reference to FIG. FIG. 8 is a flowchart showing the first flaw / dust correction processing. First, steps S151 to S159 are the same as steps S131 to S139 of the flaw / dust candidate area detection processing in FIG. Then, a flaw or dust area on the infrared difference data subjected to the noise processing is set as a flaw / dust area (step S160). Here, the region expansion as in step S13A of FIG. 6 is not performed.
[0113]
Then, for the flaw / dust area obtained in step S160, a correction value for removing the influence of flaws and dust on the infrared image information is calculated using the infrared difference data (step S161). Then, the flaw / dust candidate area detection processing and flaw / dust area determination processing shown in FIGS. 6 and 7 are executed (step S162).
[0114]
Then, one pixel on the visible image information is extracted as a target pixel (step S163). Then, it is determined whether or not the pixel of interest is within the flaw / dust area of the visible image information (step S164). If the target pixel is not within the flaw / dust area of the visible image information (step S164; NO), the process proceeds to step S169. If the target pixel is within the flaw / dust area of the visible image information (step S164; YES), the infrared correction value within the flaw / dust area existing in the area near the target pixel (the infrared correction value of the infrared image information obtained at step S161) A plurality of (correction values) are extracted (step S165).
[0115]
Then, a representative value of the extracted plurality of infrared correction values is calculated (step S166). Specifically, abnormal data is calculated by a simple average of a plurality of infrared correction values, a weighted average based on the distance of each correction value from a pixel of interest, or statistical processing when there are many extracted infrared correction values. The data such as the average is removed from the remaining data.
[0116]
Then, the calculated representative value is set as a flaw / dust correction value of the target pixel (step S167). The flaw / dust correction of the visible image information is performed using the set flaw / dust correction value (step S168). Then, it is determined whether or not all the pixels have been extracted in step S168 (step S169). If all the pixels have been extracted (step S169; YES), the first flaw / dust correction processing ends. If all pixels have not been extracted (step S169; NO), the process moves to step S163, and an unextracted pixel is extracted as the next target pixel. After the end of the first flaw / dust correction processing, the process moves to the next step (step S16 in FIG. 4).
[0117]
According to the first flaw / dust correction processing, an infrared flaw / dust area and a visible flaw / dust area are separately determined, and their connection is performed based on a nearby signal. Even if there is a shift, and even if the shift is discontinuously different in each screen local portion, this method can perform correction with less risk of abnormal correction.
[0118]
Next, the second flaw / dust correction processing, which is an example of the flaw / dust correction processing corresponding to step S15 in the flaw / dust processing of FIG. 4, will be described with reference to FIG. FIG. 9 is a flowchart showing the second dust correction processing. In the flaw / dust correction processing and the first flaw / dust correction processing, a signal attenuation rate due to the influence of a flaw or dust is obtained from infrared image information, and the reciprocal thereof is multiplied by the corresponding visible image information. This concept is based on the assumption that the effect of infrared image information from scratches and dust and the effect of visible image information are in a 1: 1 relationship.
[0119]
However, such a situation does not exist in an actual image processing system, and the ratio of the relationship mainly changes depending on the following elements.
1: MTF (Modulation Transfer Function: contrast ratio of an output image to a sine wave input image) of the imaging system differs depending on the color light.
2: The flare characteristics of the imaging system differ depending on the color light.
3: The contrast characteristics of the signal processing system differ depending on the color light.
[0120]
Therefore, in consideration of these effects, it is necessary to determine a correction value for calculating the effect that the visible image information will have from the effect on the infrared image information, and correct the correction value. In short, the correction value may be set as a predetermined constant for each color light of BGR. However, it is obvious that the ratio of the above relationship varies depending on the form of the flaw / dust area due to the cause of the occurrence. Therefore, as described below, it is desirable to determine the correction value corresponding to the size of the flaw / dust area.
[0121]
First, one of B image information indicating a B (blue) image, G image information indicating a G (green) image, and R image information indicating an R (red) image is selected from the acquired image information (selected color). Image information), and one of the pixels in the selected color image information is extracted as a pixel of interest (step S251).
[0122]
Then, it is determined whether or not the target pixel is within the flaw / dust area of the visible image information (step S252). If the target pixel is not within the flaw / dust area of the visible image information (step S252; NO), the process proceeds to step S258. If the pixel is within the flaw / dust area (step S252; YES), area size information indicating the size of the flaw / dust area to which the pixel of interest belongs is acquired (step S253). For example, a line segment is taken in the vertical, horizontal, and oblique directions around the pixel of interest, and the shortest line segment is used as region size information. Taking the example of FIG. 11 described later as an example, the area size information is four pixels in the horizontal direction. Further, as shown in FIG. 11, the number of search directions may be increased according to the average size of the area.
[0123]
Then, an infrared correction value of the pixel of interest in the flaw / dust area is calculated (step S254). For example, the infrared image correction value is calculated from the infrared image information by the following equation with reference to the first flaw / dust correction processing in FIG.
(Infrared image correction value) = (infrared reference data) / (infrared pixel value or infrared pixel value subjected to noise processing)
However, the signal unit is the amount of transmitted energy.
[0124]
Then, a peripheral region of the pixel of interest is set, and a temporary visible image correction value is calculated from the average value of the signal of the selected color image information and the average value of the signal of the pixel of interest in the peripheral region (step S255). ). Here, the peripheral region is defined as shown in FIG. 15 to be described later, pixels in the peripheral region overlapping with the line segment in which the region size information obtained earlier is set are extracted, and the averaging of the pixel values is taken. From the averaging and the visible signal value of the pixel of interest, a visible image correction value is obtained by the following equation.
(Visible image correction value) = (Average value) / (Visible signal value of target pixel)
However, the signal unit is the amount of transmitted energy.
[0125]
Then, a temporary correction value is calculated from the area size information calculated in step S255 and the visible image information calculated in step S254. For example, a temporary correction value is obtained from the infrared image correction value and the visible image correction value by the following equation.
(Temporary correction value) = (Visible image correction value) / (Infrared image correction value)
[0126]
Then, the relationship between the area size information and the temporary correction value is plotted on a plot diagram (step S257). The plot diagram is a plot diagram on a program, which is virtually generated when a computer program is executed. Then, it is determined whether or not all the pixels have been extracted as the pixel of interest for the currently selected selected color image information (step S258). If all the pixels have not been extracted (step S258; NO), the process proceeds to step S251, and an unextracted pixel in the same selected color image information or an unextracted pixel in the unextracted color image information is set as a target pixel. Is extracted.
[0127]
If all pixels have been extracted (step S257; YES), a representative curve is generated based on the plot of the currently selected image (step S259). Although a specific method of generating the representative curve is not limited, an abnormal point can be removed by using various statistical methods, and a minimum error curve may be obtained. The above procedure is performed for each of the plurality of color lights (RGB) to obtain a representative curve. As in the present embodiment, if a representative curve is obtained for each of a plurality of (BGR) color lights, a correction value corresponding to the size information can be easily obtained.
[0128]
As described above, the representative curve is set as a related function between the area size information and the correction value (Step S260). Then, it is determined whether or not the representative curve has been generated for all the selected color image information (B, G, R) (step S261). If all the representative curves have not been generated (step S261; NO), the process proceeds to step S251, and the process is repeated until the generation of all the representative curves is completed.
[0129]
When the processes up to this point are completed and all the representative curves have been generated (step S261; YES), the process moves to the next block (flow). First, one pixel in the visible image information is extracted as a pixel of interest (step S262). Then, it is determined whether or not the target pixel is within the flaw / dust area of the visible image information (step S263). If the pixel of interest is not within the flaw / dust area (step S263; NO), the process proceeds to step S269.
[0130]
If the target pixel is within the flaw / dust area (step S263; YES), the area size information of the flaw / dust area where the target pixel is located is obtained in the same manner as step S253 (step S264). Then, any one of the B signal, the G signal, and the R signal of the target pixel is selected, and the infrared image correction value of the color signal (selected color signal) is calculated in the same manner as in step S254 (step S254). S265). Then, using the representative curve set in step SS260, a correction value corresponding to the area size information obtained in step S264 is obtained (step S266).
[0131]
Then, a product of the infrared image correction value calculated in step S265 and the correction value obtained in step S266 is generated (step S267). The color image information corresponding to the selected color signal is corrected using the product (step S268). The processing of steps S265 to S268 is repeatedly executed for all the color signals (BGR).
[0132]
Then, the target pixel is extracted from all the pixels in the visible image information, and it is determined whether or not the processing of steps S262 to S268 has been performed for all the pixels (step S269). If the processing has been performed for all pixels (step S269; YES), the second flaw / dust correction processing ends. If the processing has not been performed for all the pixels (step S269; NO), the process proceeds to step S262, and the processing of steps S262 to S269 is repeated. After the end of the second flaw / dust correction processing, the process moves to the next step (step S16 in FIG. 4).
[0133]
In the second flaw / dust correction processing, the representative curve is determined for each color image information. However, the representative curve may be determined in advance as data common to the system depending on the situation. Furthermore, a representative curve can be obtained independently for each flaw / dust area having a length of a certain degree or more (the area size is a narrow flaw / dust area because the area is the shortest width of the area).
[0134]
Next, a preferred example used in the flaw / dust interpolation processing in step S18 during the flaw / dust processing in FIG. 4 will be described with reference to FIGS. FIG. 10 is a flowchart showing the flaw / dust interpolation processing. FIG. 11 is a diagram illustrating interpolation process candidate selection of a target pixel. FIG. 12 is a diagram illustrating three examples of selecting an interpolation process candidate for a target pixel. FIG. 13 is a diagram illustrating two example directional characteristics of the pixel of interest.
[0135]
First, one pixel is extracted as a target pixel from the visible image information on which the flaw / dust correction has been performed (step S181). Then, it is determined whether or not the target pixel is within the flaw / dust area of the visible image information (step S182). If the target pixel is not in the flaw / dust area (step S182; NO), the process moves to step S189.
[0136]
If the target pixel is within the flaw / dust area (step S182; YES), one color image information of the BGR is selected, and in the color image information, a normal pixel pair facing the target pixel at the center is flaw / dust interpolation. (Step S183). For example, as shown in FIG. 11, normal pixels facing each other in four directions of vertical, horizontal, and oblique are extracted. In addition, as shown in FIGS. 12A to 12C, the number of directions to be extracted may be increased or decreased according to the size of the flaw / dust area (distance between opposing pixels). In FIG. 12, the number of directions increases as the distance between the opposing pixels increases.
[0137]
Then, the association between the extracted opposed pixels is evaluated (step S184). For example, the evaluation is made based on the difference data of the signal values of the opposite pixels, and the smaller the difference data, the higher the relevance. Then, the most relevant direction is extracted (step S185). Steps S183 to S185 are repeatedly executed for all color image information (RGB image information).
[0138]
Then, the relationship between the directions extracted for each of the BGR image information is evaluated (step S186). Based on the evaluation result, a direction in which the pixel to be interpolated is extracted is determined, and a pixel pair in that direction is extracted (step S187). For example, as shown in FIG. 13A, the directions obtained from the G and R image information are the same, a weight 6 is applied to the direction obtained from the G image information, and a weight 3 is applied to the direction obtained from the R image information. , Weighting 1 is applied to the direction obtained from the B image information perpendicular to the direction. In this case, the direction obtained from the BGR is determined in consideration of the weighting, and the direction in which the interpolation source pixel is obtained is the direction shown in the figure. This is because if the interpolation directions of BGR are not matched, an unnatural color may be added to the pixel after the interpolation. In addition, as shown in FIG. 13B, when the directivity is not found in the weighting in each direction, the interpolation may be performed without setting the direction of the pixel to be interpolated to a specific direction.
[0139]
The above-mentioned weighting is based on the luminosity factor, specifically, the weighting of the BGR image used in the color conversion of the NTSC system. For example, the weight of blue is set low because the sensitivity to the eyes of the observer is low, and the weight of green is set high because the sensitivity to the eyes of the observer is high. However, the present invention is not limited to this, and a configuration in which reliability information at the time of vector determination is separately obtained may be used. The reliability information may be obtained, for example, by statistically processing a difference from a selection index of another group that is not selected when the direction is selected from among a plurality of directions to obtain the reliability.
[0140]
Then, using the extracted pixel pair, the signal value of the target pixel after interpolation is estimated, and the signal value of the target pixel is replaced with the estimated pixel value to perform interpolation (step S188). Then, in step S181, it is determined whether or not all the pixels of the visible image information have been extracted as the pixel of interest (step S189). If all the pixels have been extracted (step S189; YES), the flaw / dust interpolation processing ends. If all the pixels have not been extracted (step S189; NO), the process moves to step S181, and an unextracted pixel is extracted as the next target pixel. After the end of the flaw / dust interpolation processing, the process proceeds to the next step (step S2 in FIG. 3).
[0141]
Next, the flaw / dust correction interpolation processing corresponding to steps S15 to S18 during flaw / dust processing in FIG. 4 will be described with reference to FIGS. FIG. 14 is a flowchart showing the flaw / dust correction interpolation processing. FIG. 15 is a diagram showing a flaw / dust area and its surrounding area.
[0142]
First, a flaw / dust area characteristic value before flaw / dust correction of the visible image information is calculated using the flaw / dust area of the visible image information which has already been obtained, and a region of a pixel near the flaw / dust area is set as a peripheral area ( Step S281). For example, as shown in FIG. 15, a predetermined range of neighboring pixels is one pixel, and a part of normal pixels in the vicinity of the flaw / dust area is a peripheral area. The characteristic value is a characteristic value calculated using pixel data in a flaw / dust area within a predetermined distance of a predetermined pixel. In step S281, the characteristic value is calculated for all pixels in the flaw / dust area. The flaw / dust area characteristic value is a characteristic value calculated using pixel data within a flaw / dust area within a predetermined distance around each pixel, and is, for example, a value of each pixel within the flaw / dust area within a predetermined distance. The average value of the pixel values is used. The calculation of the flaw / dust area characteristic value may be performed by a statistical method.
[0143]
Then, a flaw / dust correction process is performed on the visible image information (step S282). For example, a flaw / dust correction process using the infrared image correction values shown in FIG. Then, one pixel in the visible image information is extracted as the pixel of interest (step S283). Then, it is determined whether or not the extracted target pixel is in the flaw / dust area (step S284).
[0144]
If the pixel of interest is not in the flaw / dust area (step S284; NO), the process proceeds to step S289. If the target pixel is within the flaw / dust area (step S284; YES), a flaw / dust area characteristic value after flaw / dust correction and a peripheral area characteristic value corresponding to the target pixel are calculated (step S285). The flaw / dust area characteristic value is a property value calculated using pixel data in a flaw / dust area within a predetermined distance from the target pixel. The peripheral region characteristic value is a characteristic value calculated using pixel data of a peripheral region within a predetermined distance around the target pixel. As described above, each characteristic value is calculated by an average value of the pixel values of the pixels in each area within a predetermined distance or by a statistical method.
[0145]
Then, based on the value corresponding to the pixel of interest of the pre-correction flaw / dust area characteristic value calculated in step S281, the post-correction flaw / dust area characteristic value and the peripheral area characteristic value calculated in step S285, the pre-correction and the correction are performed. The difference between the characteristic values with respect to the characteristic value afterward (the characteristic value in the flaw / dust area−the characteristic value in the peripheral area) is calculated, and the relationship between the characteristic value difference before correction and the characteristic value difference after correction is evaluated (step S286). ). Then, it is determined whether or not the difference between the characteristic values after the correction is sufficiently reduced as compared with the difference between the characteristic values before the correction (step S287). For example, the determination is made based on whether or not the difference between the characteristic values after the correction is smaller than the difference between the characteristic values before the correction by a predetermined value or more. The evaluation criterion may be determined by actually performing evaluation using a plurality of sample images and preferably adjusting.
[0146]
When the difference between the characteristic values after the correction is sufficiently reduced (step S287; YES), the process proceeds to step S289. If the difference between the characteristic values after the correction has not been sufficiently reduced (step S287; NO), the flaw / dust interpolation processing is performed on the target pixel (step S288). For example, a flaw / dust interpolation process shown in FIG. Then, it is determined whether or not all the pixels in the visible image information have been extracted as the pixel of interest in step S283 (step S289). If all pixels have been extracted (step S289; YES), the flaw / dust correction interpolation processing ends. If all the pixels have not been extracted (step S289; NO), the process moves to step S283, and an unextracted pixel is extracted as the next extracted pixel. After the end of the flaw / dust correction interpolation processing, the process moves to the next step (step S2 in FIG. 3).
[0147]
The above-mentioned flaw / dust correction interpolation processing is automatically corrected and evaluated. In the flaw / dust correction interpolation processing, flaw / dust correction processing in flaw / dust processing is performed first, and then flaw / dust interpolation processing is performed. At this time, when performing the flaw / dust interpolation processing, it is desirable to treat the pixel corrected by the flaw / dust correction processing as a normal pixel. In other words, if there is an area that requires flaw / dust correction processing near the area where flaw / dust interpolation processing is necessary, first, the pixel corrected by the flaw / dust correction processing is set as a normal pixel, thereby making it necessary for flaw / dust interpolation processing. Effective peripheral information can be effectively increased. This technique is a very effective technique when, for example, a very large flaw / dust area exists.
[0148]
Here, three specific examples of implementation of the flaw / dust correction processing and flaw / dust interpolation processing will be described.
(First example)
First, a very wide flaw or dust area is recognized using infrared image information. Further, the texture structure in the region is evaluated using the infrared image information, and is divided into a homogeneous portion and a non-homogeneous portion. The homogeneous portion is where the effects of flaws and dust appear stably, and this portion is subjected to flaw / dust correction processing based on infrared image information. The non-homogeneous part performs the flaw / dust interpolation processing by regarding the corrected pixel as a normal pixel.
[0149]
In the combination of the flaw / dust correction processing and the flaw / dust interpolation processing, since the correction processing using the infrared image is performed in the stable region, substantially good correction results can be obtained over a wide area. As a secondary effect, even when the infrared image information does not accurately correspond to the position of the visible image information, the correction performance is not degraded because of the stable correction area, and the correction performance may be degraded. Since interpolation processing is performed on a certain non-homogeneous region, a processing result with less occurrence of side effects can be obtained.
[0150]
(Second example)
First, a very wide flaw / dust area is recognized using infrared image information. Further, the texture structure in the region is evaluated using the infrared image information, and is divided into a homogeneous portion and a non-homogeneous portion. The homogeneous portion is where the influence of scratches and dust appears stably, and is a region that can be easily corrected using only visible image information. The non-homogeneous portion is a portion that is relatively difficult to correct with visible image information, and performs a flaw / dust correction process after sufficient alignment between infrared image information and visible image information. After the non-homogeneous part is subjected to the flaw / dust correction processing, the interpolation processing or the correction processing of the homogeneous part using the visible image information is performed.
[0151]
In this combination of the flaw / dust correction processing and the flaw / dust interpolation processing, it is necessary to obtain sufficient positional alignment between the infrared image information and the visible image information. Dust correction can be performed effectively. In addition, the homogeneous region is a region where the visible correction processing is easy, and since the processing is performed based on the actual image information, a strong correction result can be obtained. An example of the flaw / dust correction processing using the visible image information will be described later.
[0152]
Further, when the flaw / dust interpolation processing is performed, it is also effective to perform a flaw / dust interpolation processing based on the result of globally investigating the surrounding area. In this case, the effect is exhibited even when the flaw / dust area for flaw / dust correction processing does not touch the flaw / dust area for flaw / dust interpolation processing.
[0153]
According to the first and second examples, it is possible to select and execute a preferable correction method for the visible image information according to the feature of the texture structure in the image defect area of the infrared image information. In addition, for example, a defect in which an image defect occurs due to a defect in the image recording layer from which the image information is obtained, and correction of the signal intensity based on the infrared image information becomes reverse correction, a preferable correction is selected and executed. Can deal effectively. Further, even if there is residual aberration of the infrared image information or color shift (position shift) between the infrared image information and the visible image information, good image processing corresponding to the color shift (position shift) can be performed.
[0154]
(Third example)
First, the flaw / dust area to be corrected is subjected to flaw / dust correction processing (reclassified as normal pixels). The image structure is analyzed only for the normal pixels. For example, a wavelet transform can be used as a resolution conversion technique for dividing an image into a plurality of spatial frequency bands and analyzing the image. Hereinafter, the wavelet transform and the binomial wavelet transform will be briefly described with reference to FIGS.
[0155]
2. Description of the Related Art As an efficient method of dividing a frequency band for each local part of an image and performing suppression / emphasis for each frequency band, a technique using a wavelet transform is known. For details of the wavelet transform, see, for example, “Wavelet and Filter Banks” by G.W. Strong & T. Nguyen, Wellesley-Cambridge Press (Japanese translation "Wavelet Analysis and Filter Bank", G. Strang and T. Nguyen co-author, Baifukan) and "A wavelet tour of signal processing 2ed." Although described in Mallat, Academic Press, an outline is described here. The documents listed here are referred to as first wavelet documents.
[0156]
The wavelet transform means a wavelet transform coefficient <f, に 対 す る for an input signal f (x) using a wavelet function (formula (1)) oscillating in a finite range as illustrated in FIG._{a, b}> Is obtained by the following equation (2), thereby decomposing it into a sum of wavelet functions represented by the following equation (3).
(Equation 1)

(Equation 2)

(Equation 3)

[0157]
In the above equations (1) to (3), a represents the scale of the wavelet function, and b represents the position of the wavelet function. As illustrated in FIG. 16, as the value of the scale a increases, the wavelet function ψ_{a, b}The frequency of (x) becomes smaller, and the wavelet function ψ_{a, b}The position where (x) vibrates moves. Therefore, the above equation (3) shows that the input signal f (x) is converted into a wavelet function ψ having various scales and positions._{a, b}(X) is decomposed into the sum.
[0158]
There are many known wavelet functions that enable the above-described transformation. In the field of image processing, orthogonal wavelet (orthogonal wavelet) transformation and biorthogonal wavelet (transformation), which are fast to calculate, are widely used. ing. The outline of the orthogonal wavelet / biorthogonal wavelet transform calculation will be described below.
[0159]
The wavelet functions of the orthogonal wavelet transform and the biorthogonal wavelet transform are defined as in the following equation (4).
(Equation 4)

Here, i is a natural number.
[0160]
Comparing Equation (4) and Equation (1), in the orthogonal wavelet transform and the biorthogonal wavelet transform, the value of the scale a is discretely defined by 2 to the power of i, and the minimum movement unit of the position b is 2ⁱIt can be seen that is defined discretely. This value of i is called a level.
[0161]
When the level i is limited to a finite upper limit N, the input signal f (x) is represented by the following equations (5) to (7).
(Equation 5)

(Equation 6)

(Equation 7)

[0162]
The second term of equation (5) is the wavelet function of level 1 ψ_{1, j}The low frequency band component of the residual that cannot be expressed by the sum of (x) is converted to a level 1 scaling function φ._{1, j}(X). An appropriate scaling function is used corresponding to the wavelet function (see the first wavelet document). By the wavelet transform of level 1 shown in the equation (5), the input signal f (x) = S₀Is the level 1 high frequency band component W₁And low frequency band component S₁That is, the signal is decomposed into
[0163]
Wavelet function ψ_{i, j}The minimum movement unit of (x) is 2ⁱTherefore, the input signal S₀High frequency band component W₁And low frequency band component S₁Are の each, and the high frequency band component W₁And low frequency band component S₁Is the sum of the input signals S₀Signal amount. Level 1 low frequency band component S₁Is the high frequency band component W of level 2 in equation (6).₂And low frequency band component S₂, And by repeating the conversion up to the level N in the same manner, the input signal S₀Is decomposed into the sum of the high frequency band components of levels 1 to N and the sum of the low frequency band components of level N, as shown in equation (7).
[0164]
Here, it is known that the one-level wavelet transform represented by Expression (6) can be calculated by a filter process as shown in FIG. In FIG. 17, LPF indicates a low-pass filter, and HPF indicates a high-pass filter. The filter coefficients of the low-pass filter LPF and the high-pass filter HPF are appropriately determined according to the wavelet function (see the first wavelet document). In FIG. 17, 2 ↓ indicates downsampling for thinning out every other signal.
[0165]
As shown in FIG._n-1Is processed by a low-pass filter LPF and a high-pass filter HPF, and the signal is decimated every other signal._{n- 1}To the high frequency band component W_nAnd low frequency band component S_nCan be decomposed into
[0166]
The one-level wavelet transform in a two-dimensional signal such as an image signal is calculated by a filter process as shown in FIG. In FIG. 18, LPFx, HPFx, 2 ↓ x indicates processing in the x direction, and LPFy, HPFy, 2 ↓ y indicates processing in the y direction. First, the input signal S_n-1Is filtered by a low-pass filter LPFx and a high-pass filter HPFx in the x direction, and down-sampled in the x direction. Thereby, the input signal S_n-1Is the low frequency band component SX_nAnd high frequency band component WX_nIs decomposed into Low frequency band component SX_nAnd high frequency band component WX_nAre subjected to filter processing by a low-pass filter LPFy and a high-pass filter HPFy in the y-direction, and down-sampled in the y-direction.
[0167]
By this one-level wavelet transform, the low frequency band component S_n-1Are the three high frequency band components Wv_n, Wh_n, Wd_nAnd one low frequency band component S_nIs decomposed into Wv generated by decomposition_n, Wh_n, Wd_n, S_nAre the signal amounts before decomposition._n-1縦 in both the vertical and horizontal directions, the sum of the signal amounts of the four components after decomposition is S_n-1Signal.
[0168]
Input signal S₀FIG. 19 schematically shows the process of signal decomposition by the three-level wavelet transform. As shown in FIG. 19, as the number of levels increases, the image signal is thinned out by downsampling, and the decomposed image becomes smaller.
[0169]
In addition, as shown in FIG._n, Wh_n, Wd_n, S_nIs subjected to the inverse wavelet transform calculated by the filter processing to obtain the signal S before decomposition._n-1It is known that can be completely reconstructed. In FIG. 20, LPF 'indicates a low-pass filter for inverse transform, and HPF' indicates a high-pass filter for inverse transform. In addition, 2 ア ッ プ indicates an upsampling process of inserting a zero into every other signal. LPF'x, HPF'x, 2 ↑ x indicate processing in the x direction, and LPF'y, HPF'y, 2 ↑ y indicate processing in the y direction.
[0170]
As shown in FIG._nIn the y-direction and a low-pass filter LPF '_yAnd the signal obtained by performing the filtering process by Wh_nIs up-sampled in the y-direction and a high-pass filter HPF '_ySX by adding the signal obtained by performing the_nGet. Similarly, Wv_nAnd Wd_nTo WX_nGenerate
[0171]
Furthermore, SX_nIn the x-direction and a low-pass filter LPF '_xSignal obtained by performing the filtering process by_nIn the x direction and a high-pass filter HPF '_xIs added to the signal obtained by performing the filtering process according to_n-1Can be reconstructed.
[0172]
As a filter used in the inverse wavelet transform, in the case of the orthogonal wavelet transform, a filter having the same coefficient as the coefficient used in the transform is used. In the case of the bi-orthogonal wavelet transform, a filter having a coefficient different from the coefficient used for the transform is used at the time of the inverse transform (see the first wavelet document).
[0173]
The following method can be used to correct the image degradation extracted by the wavelet transform. For example, if the block noise is to be corrected, for one or both of the luminance signal and the color difference signal, the interpolation processing of the image deteriorated pixel may be performed using the high frequency band component signal of the non-image deteriorated pixel adjacent to the image deteriorated pixel. .
[0174]
Also, when correcting mosquito noise and ringing, for one or both of the luminance signal and the color difference signal, the image deteriorated pixel is interpolated using the high frequency band component signal of the non-image deteriorated pixel adjacent to the image deteriorated pixel. Good. In particular, for the luminance signal, the first order differential value of the signal value of the image deteriorated pixel may be set to 0, or the first order differential value of the signal value of the image deteriorated pixel may be attenuated according to the steepness of the edge. .
[0175]
It should be noted that whether the processing relating to the high-frequency band component for the purpose of making the first-order differential value continuous is applied to one or both of the luminance signal and the chrominance signal depends on the image to be corrected. Preferably, it is selected according to the characteristics of the deterioration.
[0176]
Next, the binomial wavelet transform will be described. The Dyadic Wavelet transform is described in "Singularity detection and processing with wavelengths" by S.A. Mallat and W.M. L. Hwang, IEEE Trans. Inform. Theory 38 617 (1992) and "Characterization of signals from multiscale edges" by S.E. Mallat and S.M. Zhong, IEEE Trans. Pattern Anal. Machine Intel. 14 710 (1992) and "A wavelet tour of signal processing 2ed." Mallat, Academic Press has a detailed description, and the outline will be described below. The document mentioned here is a second wavelet document.
[0177]
The wavelet function used in the binomial wavelet transform is defined as the following equation (8).
(Equation 8)

Here, i is a natural number.
[0178]
As described above, the wavelet function of the orthogonal wavelet transform and the biorthogonal wavelet transform is such that the minimum movement unit of the position at the level i is 2ⁱIn the binomial wavelet transform, the minimum movement unit of the position is constant regardless of the level i. Due to this difference, the binomial wavelet transform has the following features.
[0179]
As a first feature, a high-frequency band component W generated by a one-level binomial wavelet transform represented by the following equation (9)_iAnd low frequency band component S_iIs the signal S before conversion._i-1Is the same as
(Equation 9)

[0180]
As a second feature of the binomial wavelet transform, a scaling function φ_{i, j}(X) and wavelet function ψ_{i, j}The following relational expression (10) is established during (x).
(Equation 10)

Therefore, the high frequency band component W generated by the binomial wavelet transform_iIs the low frequency band component S_iOf the first order (gradient) of
[0181]
As a third feature of the binomial wavelet transform, a coefficient γ determined according to the level i of the wavelet transform_iW (see reference for binomial wavelets above) multiplied by the high frequency band component_i・ Γ_i(Hereinafter, this is referred to as a corrected high frequency band component.) According to the singularity of the signal change of the input signal, the corrected high frequency band component W_i・ Γ_iThe relationship between the signal strength levels follows a certain law.
[0182]
FIG. 21 shows the input signal S₀And the waveforms of the corrected high frequency band components of each level obtained by the wavelet transform. In FIG. 21, (a) shows the input signal S₀(B) shows the corrected high frequency band component W obtained by the level-1 binomial wavelet transform.₁・ Γ₁(C) shows the corrected high frequency band component W obtained by the level 2 binomial wavelet transform.₂・ Γ₂(D) shows the corrected high frequency band component W obtained by the level 3 binomial wavelet transform.₃・ Γ₃(E) shows the corrected high frequency band component W · γ obtained by the level 4 binomial wavelet transform.₄Is shown.
[0183]
Looking at the change in the signal strength at each level, in (a), the corrected high frequency band component W corresponding to the gentle (differentiable) signal change indicated by “1” or “4”._i・ Γ_iAs shown in (b) → (e), as the number of levels i increases, the signal intensity increases.
[0184]
Input signal S₀, The corrected high frequency band component W corresponding to the step-like signal change indicated by “2”_i・ Γ_i, The signal strength is constant regardless of the number of levels i. Input signal S₀, The corrected high frequency band component W corresponding to the δ-function-like signal change indicated by “3”_i・ Γ_iAs shown in (b) → (e), as the number of levels i increases, the signal intensity decreases.
[0185]
As a fourth feature of the binomial wavelet transform, the one-level binomial wavelet transform method for a two-dimensional signal such as an image signal is different from the above-described orthogonal wavelet transform and biorthogonal wavelet transform in the method shown in FIG. Done.
[0186]
As shown in FIG. 22, the input signal S is obtained by one-level binomial wavelet transform._n-1Is processed by a low-pass filter in the x-direction and a low-pass filter in the y-direction to obtain a low-frequency band component S_nIs obtained. Also, the input signal S_n-1Is processed by a high-pass filter in the x direction to obtain a high frequency band component Wx_nIs obtained. Further, the input signal S_n-1Is processed by a high-pass filter in the y-direction to obtain another high-frequency band component Wy._nIs obtained.
[0187]
Thus, the input signal S is obtained by the one-level binomial wavelet transform._n-1Are two high frequency band components Wx_n, Wy_nAnd one low frequency band component S_nIs decomposed into Two high frequency band components Wx_n, Wy_nIs the low frequency band component S_nChange vector V in two dimensions_nX component and y component. Change vector V_nSize M_nAnd declination A_nIs given by the following equations (11) and (12).
(Equation 11)

(Equation 12)

[0188]
Also, two high frequency band components Wx obtained by the binomial wavelet transform_n, Wy_nAnd one low frequency band component S_nIs subjected to the inverse binomial wavelet transform shown in FIG._n-1Can be reconstructed. That is, S_nIs a low-pass filter LPF in the x direction_xAnd low-pass filter LPF in y direction_yAnd the signal obtained by processing_nIs a high-pass filter HPF in the x direction._xAnd low-pass filter LPF in y direction_yAnd the signal obtained by processing_nIs a low-pass filter LPF in the x direction_xAnd high-pass filter HPF_yIs added to the signal obtained by the above processing to obtain the signal S before the binomial wavelet transform._n-1Can be obtained.
[0189]
Next, based on the block diagram of FIG.₀From the binomial wavelet transform for₀'Will be described.
[0190]
Input signal S₀, The input signal S₀Are two high frequency band components Wx₁, Wy₁And low frequency band component S₁Is decomposed into The low-frequency band component S obtained by the level-1 binomial wavelet transform by the level-2 wavelet transform₁Are two more high frequency band components Wx₂, Wy₂And low frequency band component S₂Is decomposed into By repeating such a decomposition operation up to the level n, the input signal S₀Represents a plurality of high frequency band components Wx₁, Wx₂, ..., Wx_n, Wy₁, Wy₂, ..., Wy_nAnd one low frequency band component S_nAnd is decomposed into
[0191]
The high frequency band component Wx thus obtained₁, Wx₂, ..., Wx_n, Wy₁, Wy₂, ..., Wy_n, Low frequency band component S_n, The image deterioration is extracted, the correction processing for the image deterioration is performed, and the corrected high frequency band component Wx₁’, Wx₂’,…, Wx_n’, Wy₁’, Wy₂’,…, Wy_n′, Low frequency band component S_n'.
[0192]
Then, these high frequency band components Wx₁’, Wx₂’,…, Wx_n’, Wy₁’, Wy₂’,…, Wy_n′, Low frequency band component S_n'Is subjected to the inverse binomial wavelet transform. That is, the two high frequency band components Wx at the corrected level n_n’, Wy_n'And the low frequency band component S_n′, The corrected low frequency band component S of level n−1_n-1’Is configured. By repeating such an operation, the two high frequency band components Wx at the corrected level 2 are obtained.₂’, Wy₂'And the low frequency band component S₂′, The corrected level 1 low frequency band component S₁’Is configured. This low frequency band component S₁′ And two high frequency band components Wx at level 1₁’, Wy₁′ From the corrected image signal S₀’Is configured.
[0193]
Note that the filter coefficient of each filter used in FIG. 24 is appropriately determined according to the binomial wavelet transform (see the second wavelet document). In the binomial wavelet transform, the filter coefficient of the filter used differs for each level. The filter coefficients used at level n are 2 between each coefficient of the level 1 filter.^n-1The one in which one zero is inserted is used (see the second wavelet document).
[0194]
The following method can be used to correct the image deterioration extracted by the binomial wavelet transform. A high frequency band component (Wx) obtained by the binomial wavelet transform so that the first derivative of the luminance signal or the color difference signal is continuous._n, Wy_n) Calculated from the change vector V_nSize M_n(See equation 11) and declination A_nIt is preferable to correct one or both of (see Expression 12). More specifically, for one or both of the luminance signal and the color difference signal of the image-degraded pixel, the change vector V_nSize M_nAnd declination A_nMay be interpolated using the value of the change vector and the value of the argument of the non-image-deteriorated pixel adjacent to the image-deteriorated pixel. Note that a method known in the art is selected as a method of the interpolation processing, and is not limited to the above-described method.
[0195]
As described above, image conversion is performed using, for example, the binomial wavelet transform, and the magnitude of the change vector (signal value gradient information) shown in Expression (11) and the direction of the change vector shown in Expression (12) , A flaw / dust interpolation method can be determined. For example, flaw / dust correction processing and flaw / dust interpolation processing are sequentially performed on the image information, and the image structure of the peripheral area of the flaw / dust area is evaluated based on signal value gradient information of a normal pixel not included in the flaw / dust area. The configuration is such that interpolation processing is performed based on the evaluation result. In this case, information relating to the image structure such as edges can be restored to the extent possible, the algorithm used for the interpolation process functions more effectively, and a more reliable interpolation process can be executed to obtain an interpolation result.
[0196]
At this time, as the decomposition level of the multi-resolution conversion increases, the change vector indicates a global structure, and the global structure can be reflected in the interpolation result. Further, as described above, since the proportion of the normal pixel area is increased by the correction processing in advance, it is possible to evaluate a more accurate global structure.
[0197]
Although the above-mentioned wavelet transform requires a very large amount of processing, very useful information regarding an image can be obtained. This is a very useful technology for advanced processing such as subject pattern extraction and various masking processes such as dodging, so in advanced image processing systems that perform these processes, wavelets already used for these purposes are used. Since various types of conversion information can be used in the flaw / dust processing of the present embodiment, a large processing amount will not be a problem.
[0198]
Next, with reference to FIG. 25, a description will be given of a third flaw / dust correction process as one example of step S15 during flaw / dust processing in FIG. FIG. 25 is a flowchart showing the third flaw / dust correction processing. The third flaw / dust correction process is an example of a method of performing a correction process using visible image information.
[0199]
First, an area of a pixel in the vicinity of a flaw / dust area on the visible image information which has been obtained is set as a peripheral area (step S351). For example, as shown in FIG. 15, a predetermined range of neighboring pixels is one pixel. Then, one pixel in the visible image information is extracted as a target pixel (step S352). Then, it is determined whether or not the extracted target pixel is within the flaw / dust area (step S353).
[0200]
If the pixel of interest is not in the flaw / dust area (step S353; NO), the process proceeds to step S358. If the target pixel is within the flaw / dust area (step S353; YES), the flaw / dust area characteristic value of the target pixel is calculated (step S354). For example, the average value of the characteristic values of the pixels in the flaw / dust area in the predetermined area is calculated from the target pixel. Then, the peripheral region characteristic value of the target pixel is calculated (step S355). The peripheral region characteristic value is a characteristic value calculated using a pixel value in a peripheral region within a predetermined distance around the pixel of interest, for example, a pixel value of each pixel in the peripheral region within a predetermined distance. Average value. Further, the calculation of the flaw / dust area characteristic value and the peripheral area characteristic value may be performed by a statistical method, for example, by excluding abnormal data and thereafter obtaining a mode value.
[0201]
Then, a visible image correction value is calculated as a flaw / dust correction value based on the difference between the characteristic values, which is a difference between the calculated flaw / dust area characteristic value and the peripheral area property value (step S356). Using the calculated flaw / dust correction value (visible image correction value), flaw / dust correction is performed on the target pixel (step S357). Then, it is determined whether or not all the pixels in the visible image information have been extracted as the target pixel in step S352 (step S358). If all the pixels have been extracted (step S358; YES), the third flaw / dust correction processing ends. If all the pixels have not been extracted (step S358; NO), the process moves to step S352, and an unextracted pixel is extracted as the next extracted pixel. After the end of the third flaw / dust correction processing, the process moves to the next step (step S16 in FIG. 4).
[0202]
In general, the effects of scratches and dust are varied in the form of the region, but there is no significant difference in the degree of influence on adjacent pixels in one region. A visually favorable correction result can be obtained by the flaw / dust correction processing of No. 3. Further, according to the third flaw / dust correction processing, the flaw / dust area can be corrected using only an actually necessary image (here, visible image information). Even when there is a large difference between the MTF characteristics and the flare characteristics, it is possible to realize the correction performance without the overcorrection or the undercorrection. Furthermore, even when visible image information is lost (for example, when damage is present on the film), a good processing result is obtained in which there is little risk of reverse correction and the effect of defects is not noticeable.
[0203]
Next, with reference to FIGS. 26 and 27, the enlargement / reduction processing in step S5 and the sharpness / graininess correction processing in step S7 during the image processing in FIG. 3 will be described. FIG. 26 is a flowchart showing the enlargement / reduction processing. FIG. 27 is a flowchart showing the sharpness / granularity correction processing.
[0204]
First, the enlargement / reduction processing will be described with reference to FIG. The enlargement / reduction processing differs in processing result depending on the method, and in particular, there is a large difference in the smoothing action of image information. First, grid point coordinates required for enlargement / reduction in the visible image information are calculated (step S51). The grid point is, for example, an intermediate point for complementing between pixels to be enlarged. Then, four pixels around the calculated grid point are extracted (step S52). The surrounding four pixels are used for calculating the interpolation of the grid points. Then, it is determined whether the surrounding four pixels are only normal pixels or not (step S53).
[0205]
If the surrounding four pixels are not only normal pixels (step S53; NO), it is determined whether or not the surrounding four pixels exist only in the flaw / dust area (step S54). If the surrounding four pixels do not exist only in the flaw / dust area (step S54; NO), the surrounding four pixels are determined to be in the mixed area where the normal pixel and the flaw / dust area are mixed (step S55). Then, the pixels in the mixed region can be regarded as having a large signal difference, and enlargement / reduction is performed on the pixel of interest using an interpolation method having high smoothness (step S56). An image processing result in which a flaw / dust processing boundary is not noticeable can be obtained by the interpolation method having strong smoothness.
[0206]
When the surrounding four pixels are only normal pixels (step S53; YES), or when the surrounding four pixels exist only in the flaw / dust area (step S54; YES), the surrounding four pixels are in the normal pixel area or the flaw / dust area. Pixels included in any one of these areas and within the area can be regarded as having a small signal difference, and the pixel of interest is scaled up or down using an interpolation method with weak smoothness (step S57). .
[0207]
Then, it is determined whether or not all the grid points necessary for enlarging / reducing the entire visible image information have been extracted (calculated) in step S51 (step S58). If all the lattice points have been extracted (step S58; YES), the scaling process ends. If all the lattice points have not been extracted (step S58; NO), the process proceeds to step S51, and the unextracted lattice points are calculated as the next lattice points. After the end of the enlargement / reduction processing, the process moves to the next step (step S6 in FIG. 3).
[0208]
As an interpolation method in the scaling processing, for example, an interpolation method described in JP-A-2002-262094 can be used. Among them, a linear interpolation method using pixel data of nine points in the vicinity having a large smoothing effect is applied to enlargement / reduction of a slight scaling ratio in which moire or the like at the time of scaling is likely to occur. An interpolation method using pixel data of four neighboring points having a relatively small smoothing effect is applied. That is, in the present embodiment, a favorable result is obtained by applying the former to the mixed area of the flaw / dust correction area and the normal pixel area (corresponding to step S56), and applying the latter to other areas (corresponding to step S57). Is obtained.
[0209]
Next, the sharpness and graininess correction processing will be described with reference to FIG. First, one pixel in the visible image information is extracted as a target pixel (step S71). Then, it is determined whether or not the target pixel is a normal pixel (step S72). If the target pixel is a normal pixel (step S72; YES), the sharpness enhancement is set to “strong” and the graininess correction is set to “medium”, and the sharpness enhancement and the granularity are set to the target pixel based on the set values. Sex correction is performed (step S73).
[0210]
Then, it is determined whether or not all the pixels in the visible image information have been extracted as the target pixel in step S71 (step S74). When all the pixels have been extracted (step S74; YES), the sharpness / granularity correction processing ends. If all the pixels have not been extracted (step S74; NO), the process proceeds to step S71, and an unextracted pixel is extracted as the next extracted pixel. After the completion of the sharpness / granularity correction process, the process moves to the next step (step S8 in FIG. 3).
[0211]
If the target pixel is not a normal pixel (step S72; NO), it is determined whether or not the target pixel is in the peripheral area (step S75). The peripheral area is an area that exists at the boundary between the area in the visible image information that has undergone flaw / dust processing and the area that has not been processed. If the target pixel is in the peripheral area (step S75; YES), the sharpness enhancement is set to “medium” and the graininess correction is set to “strong”, and the sharpness enhancement and the sharpness enhancement are performed on the target pixel based on the set value. The graininess correction is performed (Step S76), and the routine goes to Step S74. If the sharpness enhancement of the peripheral area is strengthened, there is a risk that the boundary of the flaw / dust area becomes easy to see, so that a slight adjustment is made. At the same time, the graininess correction is slightly strengthened (graininess is suppressed).
[0212]
If the target pixel is not in the peripheral area (step S75; NO), it is determined that the target pixel is in the flaw / dust area (step S77). When the target pixel is in the flaw / dust area, by performing a correction different from that of the normal pixel, a processing result that is entirely homogenized and in which traces of flaw / dust processing are less noticeable can be obtained. It is preferable that the flaw / dust area is further classified according to the processing method. Accordingly, it is determined whether the target pixel has been subjected to the flaw / dust correction processing or the flaw / dust interpolation processing (step S78). If the flaw / dust correction processing has been performed (step S78; correction processing), the sharpness enhancement is set to "weak" and the graininess correction is set to "strong", and the sharpness enhancement is applied to the target pixel based on the set values. Then, the graininess correction is performed (Step S79), and the routine goes to Step S74. For example, when the flaw / dust correction processing is performed, the attenuated signal is enhanced and restored, so that the graininess correction of the region is made stronger than usual (graininess is strongly applied) to suppress noise. At the same time, the degree of sharpness enhancement is reduced.
[0213]
If the flaw / dust interpolation processing has been performed (step S78; interpolation processing), the sharpness enhancement is set to “strong” and the graininess correction is set to “weak”, and the sharpness enhancement is applied to the target pixel based on the set values. Then, the graininess correction is performed (step S80), and the process proceeds to step S74. At the place where the flaw / dust interpolation processing is performed, there is often a step of referring to neighboring pixels and performing weighted averaging, and the noise component is reduced. Therefore, the graininess correction is performed weakly, and the sharpness enhancement is performed relatively strongly.
[0214]
The relationship between the sharpness enhancement and the graininess correction described above is a description on the assumption of the flaw / dust correction processing and flaw / dust interpolation processing as examples, and the strength of each correction is not limited to this. If the methods of dust correction and interpolation processing are switched, they should be adjusted appropriately.
[0215]
Next, a case where the image acquisition in the image acquisition unit 14 is performed by the reflection original scanner 141 instead of the transparent original scanner 142 will be described. The flaw / dust interpolation processing of FIG. 10, the flaw / dust correction interpolation processing of FIG. 14, and the third flaw / dust correction processing of FIG. 25 do not require infrared image information for image correction or interpolation. It suffices if the flaw / dust candidate area can be obtained by any method. Therefore, with reference to FIG. 28, a description will be given of the reflection document scanner 141 of the image acquisition unit 14 in FIG. 1 as a configuration that does not acquire infrared image information. FIG. 28 is a diagram showing the internal configuration of the reflection document scanner 141.
[0216]
As shown in FIG. 28, the reflection document scanner 141 includes light sources 41 and 42 of a photograph document (such as glossy photographic paper) B, half mirrors 43 and 44 for transmitting and reflecting light emitted from the light sources 41 and 42, CCDs 45 to 47 that form an analog signal by imaging and receiving light, amplifiers 48 to 50 that amplify analog signals output from CCDs 45 to 47, respectively, and convert analog signals output from amplifiers 48 to 50 to digital signals. A / D converters 51 to 53 are provided for performing respective conversions.
[0217]
The reflection document scanner 141 includes a timing control unit 54 that controls the timing of the light emission of the light sources 41 and 42 and the light reception of the CCDs 45 to 47, and the light source 41 and 42 of the image information output from the A / D converter 52. An image comparison unit 55 that compares two pieces of image information corresponding to the respective lights emitted from the light source 42, an original image formation unit 56 that forms original image information from the comparison result output from the image comparison unit 55, A defect candidate area determining unit 57 that determines a defect candidate area from the two pieces of image information output from the D converters 51 and 53, and a defect area is identified from the defect candidate areas output from the defect candidate area determining unit 57. A defect area specifying unit 58 that outputs the defective area together with the original image information output from the original image forming unit 56 is provided.
[0218]
The defective area and the original image information output from the defective area specifying unit 58 are transmitted to the correction interpolation processing unit 61 in the image processing unit 11. The photo original B may be one that shows uniform gloss.
[0219]
Next, the operation of the reflection document scanner 141 will be briefly described. First, the photo original B is irradiated by two light sources 41 and 42. The light sources 41 and 42 emit light alternately by the timing control unit 54. The CCD 46 captures images of both light sources. The CCD 47 collects light transmitted through the half mirror 43 and reflected by the half mirror 44 at the timing when the light source 41 is turned on. The CCD 45 collects light transmitted through the half mirror 44 and reflected by the half mirror 43 at the timing when the light source 42 is turned on.
[0220]
The image information collected by the CCD 46 is compared in the image comparing unit 55 and the original image forming unit 56 with respect to the difference between the light sources. It is formed as image information. By doing so, reflection due to fine irregularities on the glossy photographic paper (photograph document B) and glossy reflection on the silk-screened photographic paper (photograph document B) can be removed, and a favorable copy result can be obtained.
[0221]
On the other hand, the image information collected by the CCDs 45 and 47 is extracted by the defective image candidate area determination unit 57 by the CCDs 45 and 47, respectively, to extract image areas that do not fall within the predetermined image signal value range. In this case, a defect candidate area such as a scratch or dust is determined. The defect candidate area is subjected to noise processing by the defect area specifying unit 58, and is thereafter enlarged in size by the opening processing to be a defective area.
[0222]
The correction interpolation processing unit 61 receives the original image information and the information of the defective area from the defective area specifying unit 58, and performs the flaw / dust correction processing as described in the present embodiment based on the original image information and the defective area information. Then, a flaw / dust interpolation process is performed.
[0223]
As described above, according to the flaw / dust processing of the present embodiment, a flaw / dust area is obtained using infrared image information, the influence of the flaw / dust area on visible image information is corrected for flaw / dust, and the quality of the correction is evaluated. Then, according to the evaluation result, a flaw / dust interpolation process is performed on the visible image information. For this reason, for example, even if a defect occurs in which an image is lost due to damage to the film and the signal intensity is inversely corrected by the infrared image information, the flaw / dust interpolation processing can be effectively performed to remove the flaw / dust area. Further, even when a slight shift in the image information between the infrared region and the visible region occurs in a part of the screen, the correction result of the image region in which the shift has occurred can be evaluated. Can be performed in accordance with the flaw / dust interpolation processing. Further, according to the flaw / dust correction interpolation processing of the present embodiment, the correction result is evaluated by comparing the representative values of the flaw / dust area and the surrounding area in the corrected and uncorrected visible image information. The evaluation of the correction result can be satisfactorily performed by a simple method.
[0224]
Further, according to the flaw / dust processing of the present embodiment, flaw / dust interpolation processing is performed after flaw / dust correction processing, and normal pixels as peripheral image information that can be used in the interpolation processing are maximally increased by the correction processing. Therefore, more reliable interpolation processing can be executed to obtain an interpolation result. In addition, for example, it is possible to effectively cope with a problem caused by damage to the film, and to perform interpolation using the corrected normal pixels, thereby obtaining a highly accurate image processing result.
[0225]
Further, according to the enlargement / reduction processing according to the present embodiment, the interpolation processing for enlargement / reduction is switched and executed in accordance with the classification information of the normal pixel area, the flaw / dust area, and the mixed area. A more natural image processing result that is difficult to remain is obtained.
[0226]
Further, according to the sharpness / granularity correction processing of the present embodiment, in addition to the classification of the normal pixel area, the peripheral area, and the flaw / dust area, in addition to the correction target area of the flaw / dust area and the interpolation target area, Since the sharpness enhancement and the graininess correction processing are executed, traces of flaw / dust processing are less likely to remain, and a more natural image processing result can be obtained.
[0227]
Further, according to the second flaw / dust correction processing, the magnitude of the influence of the flaw / dust area is evaluated according to the area size information of the target pixel, and the visible image information is flaw / dust corrected based on the evaluation result. In the normally assumed image information (the number of normal pixels >> the number of abnormal pixels), the magnitude of the influence of the flaw / dust area on the target pixel is correctly evaluated, and correction is performed based on a correction value according to the evaluation result. Can be. Therefore, even if residual aberrations and flare components remain in the infrared image information, the effects of these components on the infrared image information can be compensated, and preferable image processing can be realized. In addition, a provisional visible image correction value is obtained from the visible image information, and a representative curve indicating the relationship between the division value of the infrared image correction value by the visible image correction value and the area size information is obtained. For this reason, a correction value can be determined based on the area size information using the representative curve, and the image information in the visible region can be corrected for flaws and dust using the correction value and the infrared image correction value. Etc.), a stable scratch and dust removal effect can be obtained.
[0228]
The description in the present embodiment described above is an example of a suitable image processing system 10 according to the present invention, and the present invention is not limited to this.
[0229]
【The invention's effect】
According to the invention as set forth in claim 1, 11 or 21, an image defect area is acquired using image information in the infrared region, and the influence of the image defect region on image information in the visible region is corrected. Is evaluated, and the image defect area in the visible region is processed according to the evaluation result. For this reason, for example, an image defect occurs due to a defect in the image recording layer from which the image information is acquired, and the correction of the signal intensity by the image information in the infrared region is reversely corrected. Even if a slight shift occurs in a part of the screen, it is possible to evaluate the correction result of the image area where the defect has occurred, so it is necessary to select and execute processing according to the image area With this, it is possible to deal effectively.
[0230]
According to the second, twelfth, or twenty-second aspect, the correction result is evaluated by comparing the representative values of the image defect area and the surrounding area in the image information of the visible area between the corrected and the uncorrected. Therefore, the evaluation of the correction result can be favorably performed by a simple method.
[0231]
According to the third, thirteenth, or twenty-third aspect of the present invention, a preferable correction method is selected and executed for the visible-range image information according to the feature of the texture structure in the image defect area of the infrared-range image information. it can. Further, for example, even when a defect occurs in which an image loss occurs due to a loss in the image recording layer from which the image information is obtained, and the correction of the signal intensity by the image information in the infrared region is reversed, a preferable correction is selected and executed. , Can deal effectively. Further, even if there is residual aberration in image information in the infrared region or color shift between image information in the infrared region and the visible region, it is possible to perform favorable image processing corresponding to this.
[0232]
According to the invention described in claim 4, 14, or 24, the number of normal pixels as peripheral image information that can be used in the interpolation processing can be increased to the maximum by the correction processing, so that more reliable interpolation processing can be performed. To obtain an interpolation result. Further, for example, it is possible to effectively cope with a problem in which an image loss occurs due to a loss in the image recording layer from which the image information is obtained, and the correction of the signal intensity by the image information in the infrared region is reversely corrected. Since interpolation is performed using normal pixels, a highly accurate image processing result can be obtained.
[0233]
According to the invention described in claim 5, 15 or 25, the region to which the corrected and / or interpolated first pixel belongs, the region to which the uncorrected and / or interpolated second pixel belongs, and the third pixel Since various post-processes are switched and executed in accordance with the classification information of the area to which the image belongs, a more natural image processing result is obtained in which traces of correction of defective pixels hardly remain.
[0234]
According to the invention described in claim 6, 16 or 26, in addition to the classification information, according to the group information of the first group to be corrected and the second group to be interpolated in the image area of the first pixel. Since various post-processings are switched and executed for the first pixel, a more natural image processing result is obtained in which traces of correction hardly remain.
[0235]
According to the invention of claim 7, 17 or 27, the number of normal pixels as peripheral image information that can be used in the interpolation processing can be increased to the maximum by the correction processing, so that more reliable interpolation processing can be performed. To obtain an interpolation result. Further, for example, it is possible to effectively cope with a problem in which an image loss occurs due to a loss in the image recording layer from which the image information is acquired, and the correction of the signal intensity by the image information in the infrared region is the reverse correction. Furthermore, since interpolation is performed using the corrected normal pixels, a highly accurate image processing result can be obtained.
[0236]
According to the invention described in claim 8, 18 or 28, the image structure of the peripheral area of the image defect is evaluated based on the signal value gradient information of the normal pixel not included in the area of the image defect, and the evaluation result Since the interpolation processing is performed based on, for example, information on the image structure such as an edge can be restored to the extent possible, the algorithm used for the interpolation processing functions more effectively, and the interpolation result is further improved by executing the interpolation processing more reliably. Can be obtained.
[0237]
According to the ninth, nineteenth, or twenty-ninth aspect, the magnitude of the influence of the image defect area is evaluated according to the shortest line segment passing through the pixel of interest, and the image information in the visible region is corrected based on the evaluation result. Since the correction is performed, in the normally assumed image information (the number of normal pixels >> the number of abnormal pixels), the magnitude of the influence of the image defect area on the target pixel is correctly evaluated, and based on a correction value according to the evaluation result. Can be corrected. Therefore, even if residual aberrations and flare components remain in image information in the infrared region, the effects of these components on image information in the infrared region are compensated for, and preferable image processing can be realized.
[0238]
According to the invention as set forth in claim 10, 20, or 30, a temporary visible image correction value is obtained from image information in the visible region, and a difference between the correction value in the infrared region by the visible image correction value and the line segment distance information is calculated. Obtain a representative curve showing the relationship. For this reason, the correction value is determined by the line segment distance information using the representative curve, and the image information in the visible region can be corrected using the correction value and the correction value in the infrared region, and the unknown image information acquisition source is also stable. The effect of removing damaged scratches and dust is obtained.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an image processing system 10 according to an embodiment of the present invention.
FIG. 2 is a diagram showing a configuration of a transparent document scanner 142 of FIG.
FIG. 3 is a flowchart illustrating image processing.
FIG. 4 is a flowchart showing a flaw / dust processing.
FIG. 5 is a diagram showing absorption of signals in each wavelength range.
FIG. 6 is a flowchart illustrating a flaw / dust candidate area detection process.
FIG. 7 is a flowchart illustrating a flaw / dust area determination process.
FIG. 8 is a flowchart illustrating a first flaw / dust correction process.
FIG. 9 is a flowchart illustrating a second dust correction process.
FIG. 10 is a flowchart illustrating a flaw / dust interpolation process.
FIG. 11 is a diagram illustrating selection of an interpolation process candidate for a target pixel.
FIG. 12 is a diagram illustrating three examples of selecting an interpolation process candidate for a target pixel.
FIG. 13 is a diagram illustrating two examples of directional characteristics of a target pixel.
FIG. 14 is a flowchart illustrating a flaw / dust correction interpolation process.
FIG. 15 is a diagram showing a flaw / dust area and its surrounding area.
FIG. 16 is a diagram showing a wavelet function.
FIG. 17 is a system block diagram showing one-level wavelet transform filtering.
FIG. 18 is a system block diagram illustrating one-level wavelet transform filtering of a two-dimensional signal.
FIG. 19 shows an input signal S₀Is a schematic diagram showing a process in which signals are decomposed by level 1 to level 3 orthogonal wavelet transform or bi-orthogonal wavelet transform.
FIG. 20 is a system block diagram showing filter processing of the level 1 inverse wavelet transform.
FIG. 21 shows an input signal S;₀FIG. 7 is a diagram showing a waveform of a corrected high frequency band component W · γ of each level obtained by a wavelet transform.
FIG. 22 is a system block diagram showing one-level binomial wavelet transform filtering in a two-dimensional signal.
FIG. 23 is a system block diagram showing filter processing of one-level inverse binomial wavelet transform of a two-dimensional signal.
FIG. 24 shows an input signal S;₀From the binomial wavelet transform for₀'Is a system block diagram showing a process until a' is obtained.
FIG. 25 is a flowchart illustrating a third flaw / dust correction process.
FIG. 26 is a flowchart showing an enlargement / reduction process.
FIG. 27 is a flowchart showing a sharpness / graininess correction process.
FIG. 28 is a diagram showing an internal configuration of a reflection original scanner 141.
[Explanation of symbols]
10 ... Image processing system
11 Image processing unit
12 Image display unit
13 ... instruction input section
131 contact sensor
132 ... mouse
133 ... Keyboard
14. Image acquisition unit
141 ... Reflection original scanner
41, 42 ... light source
43,44 ... half mirror
45-47 ... CCD
48-50… Amplifier
51-53 ... A / D converter
55 ... Image comparison unit
56: Original image forming unit
57: Defect candidate area determination unit
58: Defect area specifying unit
B: Photo manuscript
61: interpolation correction processing unit
142 ... Transparent original scanner
31 Light source
32 ... Diffusion member
33 ... Film carrier
34 ... roller
35 ... Optical lens
36IR, 36B, 36R ... dichroic filter
37IR, 37B, 37G, 37R ... Line CCD
38IR, 38B, 38G, 38R ... analog amplifier
39IR, 39B, 39G, 39R ... A / D converter
40 ... Image memory
A: Film
143 ... Media driver
144 Information communication I / F
15 ... Image storage unit
16… Silver exposure printer
17 ... IJ printer
18 Various image recording media writing units
21 Printed matter
22 ... Developed film
23 ... Image media
24 ... Communication means

Claims (30)

An image processing method for acquiring image information on at least two wavelength bands of an infrared region and a visible region in an image document, performing image processing, and obtaining output image information,
Using the image information in the infrared region, obtain an image defect area,
Estimating the effect of the image defect area on the image information in the visible region, to correct the effect, to correct the signal intensity of the image information in the visible region,
Evaluating the quality of the correction and, if the evaluation result is negative, processing the image defect area in the visible region based on the value of a normal pixel in the vicinity of the defect area. . In the evaluation, the representative values of the image defect area and its peripheral area in the corrected visible area image information and the uncorrected visible area image information are compared and evaluated. The image processing method according to claim 1. An image processing method for acquiring image information on at least two wavelength bands of an infrared region and a visible region of an image document, performing image processing, and obtaining output image information,
Using the image information in the infrared region, obtain an image defect area,
An image processing method comprising: evaluating a texture structure in the image defect area of the image information in the infrared region; and correcting the image information in the visible region based on the evaluation result. When performing the correction process and the interpolation process on the same image information in the visible region, the correction process is performed first, the corrected region is regarded as a normal pixel, and then the interpolation process is performed. Item 4. The image processing method according to any one of Items 1 to 3. An image defect is detected from the image information, image processing for correcting and / or interpolating the image defect is performed on the image information, and the processed image information is adjusted for sharpness, graininess, and image quality. In the image processing method for performing at least one post-process of adjusting the size,
The image information is located in the vicinity of the first pixel subjected to the correction processing and / or the interpolation processing and the first pixel, and has not been subjected to the correction and / or the interpolation processing. Classifying into a second pixel and a third pixel that does not belong to any of the first pixel and the second pixel, and performing the post-processing based on classification information regarding the classification. Image processing method. In the classification, the first pixels are classified into a first group subjected to the correction processing and a second group subjected to the interpolation processing. The image processing method according to claim 5, wherein the post-processing is performed on the first pixel based on information. An image processing method for detecting an image defect from image information, performing image processing for removing the image defect by a correction process and an interpolation process according to the characteristics of the image defect, and obtaining output image information,
An image processing method comprising: first performing the correction processing on the same image information; setting a corrected area as a normal pixel; and then performing the interpolation processing. In the image processing by the interpolation processing, an image structure of a peripheral area of the image defect is evaluated based on signal value gradient information of a normal pixel not included in the area of the image defect, and an interpolation processing is performed based on the evaluation result. The image processing method according to claim 7, wherein: Correction for acquiring image information of at least two wavelength bands of an infrared region and a visible region of an image document, obtaining an image defect region using the image information of the infrared region, and compensating for the influence of the image defect region. A correction value used when applying the correction value to a visible signal, obtaining a correction correction value by a product of the correction value and the correction value, and using the corrected correction value to obtain image information of the visible region. In an image processing method of performing image processing for correcting
In the setting of the correction value, the length of the shortest line segment in the image defect area among the line segments passing through the pixel of interest in each of the target pixels present in the image defect area is defined as the length of the shortest line segment. An image processing method, wherein the correction value is set based on line segment distance information to be represented. Pixels existing around the image defect area are defined as a peripheral area,
In setting the correction value,
For each pixel of interest present in the image defect area, the line segment distance information and a pixel group in the peripheral region overlapping a line segment corresponding to the line segment distance information are extracted, and Obtain a temporary visible image correction value from the pixel group and the pixel of interest,
Obtain a divisor value obtained by dividing the correction value in the infrared region by the visible image correction value, plot the relationship between the divisor value and the line segment distance information, obtain a representative curve from the plot result, and obtain the representative curve. The image processing method according to claim 9, wherein the correction value is set based on the correction value. In an image processing apparatus for acquiring image information on at least two wavelength bands of an infrared region and a visible region in an image document and performing image processing to obtain output image information,
Using the image information in the infrared region, an image defect area acquisition unit that acquires an image defect area,
Estimating the effect of the image defect region on the image information in the visible region, correcting the influence, correcting the signal intensity of the image information in the visible region, evaluating the quality of the correction, and evaluating the result. If the result is negative, the image processing apparatus further comprises image processing execution means for processing the image defect area in the visible region based on the value of the normal pixel near the defect area. The image processing execution unit compares and evaluates representative values of the image defect area and its peripheral area in the corrected visible area image information and the uncorrected visible area image information. The image processing apparatus according to claim 11, wherein: In an image processing apparatus for acquiring image information on at least two wavelength bands of an infrared region and a visible region of an image document, performing image processing, and obtaining output image information,
Using the image information in the infrared region, an image defect area acquisition unit that acquires an image defect area,
Image processing apparatus for evaluating a texture structure in the image defect area of the image information in the infrared region, and performing image processing execution means for correcting the image information in the visible region based on the evaluation result. . When performing a correction process and an interpolation process on the same image information in the visible region, the image processing execution unit first performs the correction process, sets a region after correction as a normal pixel, and then performs the interpolation process. The image processing apparatus according to claim 11, wherein the image processing is performed. An image defect is detected from the image information, image processing for correcting and / or interpolating the image defect is performed on the image information, and the processed image information is adjusted for sharpness, graininess, and image quality. In an image processing apparatus that performs at least one post-process of adjusting a size,
The image information is located in the vicinity of the first pixel subjected to the correction processing and / or the interpolation processing and the first pixel, and has not been subjected to the correction and / or the interpolation processing. An image processing execution unit that classifies the image into a second pixel and a third pixel that does not belong to any of the first pixel and the second pixel, and performs the post-processing based on classification information on the classification. An image processing apparatus comprising: The image processing execution unit may be configured such that, in the classification, the first pixel is classified into a first group that is a target of the correction processing and a second group that is a target of the interpolation processing. The image processing apparatus according to claim 15, wherein the post-processing is performed on the first pixel based on classification information and group information on the group. An image processing device that detects an image defect from image information, performs an image process of removing the image defect by using a correction process and an interpolation process according to the characteristics of the image defect, and obtains output image information.
An image processing apparatus, comprising: an image processing execution unit that first performs the correction processing on the same image information, sets a corrected area as a normal pixel, and then performs the interpolation processing. The image processing execution unit evaluates an image structure of a peripheral region of the image defect based on signal value gradient information of a normal pixel not included in the image defect region, and performs an interpolation process based on the evaluation result. The image processing apparatus according to claim 17, wherein the image processing is performed. Correction for acquiring image information of at least two wavelength bands of an infrared region and a visible region of an image document, obtaining an image defect region using the image information of the infrared region, and compensating for the influence of the image defect region. A correction value used when applying the correction value to a visible signal, obtaining a correction correction value by a product of the correction value and the correction value, and using the corrected correction value to obtain image information of the visible region. In an image processing apparatus that performs image processing for correcting
For each pixel of interest present in the image defect area, based on line segment distance information representing the length of the shortest line segment in the image defect area among the line segments in multiple directions passing through the pixel of interest. An image processing apparatus comprising: a correction value setting unit configured to set the correction value by using the correction value setting unit. Pixels existing around the image defect area are defined as a peripheral area,
The correction value setting unit extracts, for each pixel of interest existing in the image defect area, the line segment distance information and a pixel group in the peripheral region overlapping a line segment corresponding to the line segment distance information. A temporary visible image correction value is obtained from the pixel group and the pixel of interest in the image information in the visible region, and a divisor obtained by dividing the correction value in the infrared region by the visible image correction value is obtained. 20. The image processing apparatus according to claim 19, wherein the relation with the line segment distance information is plotted, a representative curve is obtained from the plot result, and the correction value is set based on the representative curve. In an image processing program for realizing a function of acquiring image information on at least two wavelength bands of an infrared region and a visible region in an image document and performing image processing on the computer, and obtaining output image information,
To the computer,
Using the image information in the infrared region, an image defect area acquisition function for acquiring an image defect area,
Estimating the effect of the image defect area on the image information in the visible region, correcting the influence, correcting the signal strength of the image information in the visible region, evaluating the quality of the correction, and evaluating the result. If the result is negative, an image processing execution function of performing processing of the image defect area in the visible region based on the value of the normal pixel in the vicinity of the defect area,
Image processing program for realizing. The image processing execution function compares and evaluates respective representative values of the image defect area and its peripheral area in the corrected visible area image information and the uncorrected visible area image information. The image processing program according to claim 21, wherein: In an image processing program for realizing a function of acquiring image information on at least two wavelength bands of an infrared ray and a visible ray of an image original and performing image processing on the computer, and obtaining output image information,
To the computer,
Using the image information in the infrared region, an image defect area acquisition function for acquiring an image defect area,
An image processing execution function for evaluating a texture structure in the image defect area of the image information in the infrared region and correcting the image information in the visible region based on the evaluation result;
Image processing program for realizing. The image processing execution function, when performing the correction process and the interpolation process on the same image information of the visible region, first performs the correction process, the corrected area as a normal pixel, then the interpolation process The image processing program according to any one of claims 21 to 23, wherein the program is executed. A computer detects an image defect from the image information, performs image processing for correcting and / or interpolating the image defect on the image information, and adjusts sharpness and granularity to the processed image information. In an image processing program for realizing a function of executing at least one post-process out of adjustment and image size adjustment,
To the computer,
The image information is located in the vicinity of the first pixel subjected to the correction processing and / or the interpolation processing and the first pixel, and has not been subjected to the correction and / or the interpolation processing. An image processing execution function of classifying the pixel into a second pixel and a third pixel that does not belong to any of the first pixel and the second pixel, and performing the post-processing based on the classification information on the classification; Image processing program for realizing. The image processing execution function is such that the first pixels are classified into a first group subjected to the correction processing and a second group subjected to the interpolation processing. 26. The image processing program according to claim 25, wherein the post-processing is performed on the first pixel based on group information about a group. To realize a function of detecting an image defect from image information, performing image processing for removing the image defect by using a correction process and an interpolation process according to characteristics of the image defect, and obtaining output image information. In the image processing program of
To the computer,
An image processing program for realizing an image processing execution function of first performing the correction processing on the same image information, setting a corrected area as a normal pixel, and then performing the interpolation processing. The image processing execution function evaluates an image structure of a peripheral region of the image defect based on signal value gradient information of a normal pixel not included in the image defect region, and performs an interpolation process based on the evaluation result. The image processing program according to claim 27, wherein the program is executed. A computer obtains image information of at least two wavelength bands of an infrared region and a visible region of an image original, obtains an image defect region using the image information of the infrared region, and determines the influence of the image defect region. A correction value to be compensated is obtained, a correction value used when applying the correction value to a visible signal is set, a correction correction value is obtained by a product of the correction value and the correction value, and the visible correction region is obtained by the correction correction value. In the image processing program for performing the image processing for correcting the image information of, and realizing the function of obtaining the output image information,
To the computer,
For each pixel of interest present in the image defect area, based on line segment distance information representing the length of the shortest line segment in the image defect area among the line segments in multiple directions passing through the pixel of interest. An image processing program for realizing a correction value setting function for setting the correction value. Pixels existing around the image defect area are defined as a peripheral area,
The correction value setting function extracts, for each pixel of interest existing in the image defect area, the line segment distance information and a pixel group in the peripheral region overlapping with a line segment corresponding to the line segment distance information. A temporary visible image correction value is obtained from the pixel group and the pixel of interest in the image information in the visible region, and a divisor obtained by dividing the correction value in the infrared region by the visible image correction value is obtained. 30. The image processing program according to claim 29, wherein the relation with the line segment distance information is plotted, a representative curve is obtained from the plot result, and the correction value is set based on the representative curve.

Images