JP3693086B2 - Image processing apparatus, image processing method, and medium on which image processing program is recorded - Google Patents

Description

【０１１１】
これと同じ処理を緑成分と青成分について実行し、次に、各色成分間での特徴ベクトルの内積を求める。すなわち、赤成分と緑成分との間の特徴ベクトルの内積ｃｏｒｒ＿ｒｇは数２式に表すように求められ、緑成分と青成分との間の特徴ベクトルの内積ｃｏｒｒ＿ｇｂは数３式に表すように求められ、青成分と赤成分との間の特徴ベクトルの内積ｃｏｒｒ＿ｂｒは数４式に表すように求められる。
【０１１２】
【数２】

【０１１３】
【数３】

【０１１４】
【数４】

【０１１５】
ベクトルの内積は、両ベクトルの類似度を表すといえ、その値は「０」〜「１」となる。従って、しきい値ＣＯＲＲとして「０．７」を定めたとして、それぞれの内積ｃｏｒｒ＿ｒｇ，ｃｏｒｒ＿ｇｂ，ｃｏｒｒ＿ｂｒのうちいずれか一つでもこのしきい値ＣＯＲＲ以下のものがあれば類似具合が低いものと判断してステップＳ１０６の非処理を実行することにする。
【０１１６】
本実施形態においては、この特徴ベクトルの内積に基づく処理が類似具合判断手段を構成しており、特徴ベクトルに基づく内積の演算であれば手法が確立しており判断も容易である。しかしながら、むろんこの例に限るものではない。例えば、全階調範囲を四つの領域に分割しているが、これら以上とすることは自由であるし、度数分布の両端位置や、標準偏差、尖度などの統計的手法を用いてその近似度を求めることも可能である。
【０１１７】
ここで、このような統計的手法を用いて近似度を求める具体的手法について説明する。統計的手法の一例には、分布の代表値を利用するものがあげられる。いま、変数として平均値、中央値、標準偏差（分散）の差の絶対値を赤成分と緑成分、緑成分と青成分、青成分と赤成分との間で求めておく。そして、平均値と標準偏差について赤成分と緑成分の差の絶対値をＡｖｅ＿ｒｇ，Ｓｔｄ＿ｒｇとしたとき、赤成分と緑成分の間の評価関数として
ｈ（ｒｇ）＝（１−Ａｖｅ＿ｒｇ／２５５）×（１−Ｓｔｄ＿ｒｇ／２５５）と設定する。また、同様に緑成分と青成分の間の評価関数として、
ｈ（ｇｂ）＝（１−Ａｖｅ＿ｇｂ／２５５）×（１−Ｓｔｄ＿ｇｂ／２５５）と設定するとともに、青成分と赤成分との間の評価関数として、
ｈ（ｂｒ）＝（１−Ａｖｅ＿ｂｒ／２５５）×（１−Ｓｔｄ＿ｂｒ／２５５）と設定する。分布が似ている場合には平均値、中央値、標準偏差（分散）は殆ど同じになり、各変数の差も殆ど「０」になるから、評価関数ｈの値は「１」に近くなる。分布が異なる場合は差も大きくなり、評価関数ｈの値も小さくなる。従って、評価関数と実験で求められたしきい値とを比較することにより補正を行うか否かを決定することができるようになる。むろん、この場合に平均値の代わりに中央値を利用することも可能であるし、評価関数の具体例はこれに限られるものでもない。
【０１１８】
以上のような処理により各色成分毎に求めた度数分布についてある程度の類似具合が見つけられたら各度数分布から成分毎の特性を見出せるものと判断でき、かかる特性に基づいて各成分間の均一化を図る。なお、上述した各種の判断の結果、ステップＳ１０６にて非処理を実行してフラグをセットしている場合もある。このような場合には、ステップＳ２０２にて同フラグを参照しているので、以下の均一化の処理をすることなく本処理を終了する。
【０１１９】
特性の均一化の処理では、最初にオフセットの算出と修正を行う。いわゆる狭義の意味での色ずれの修正に該当し、特性の均一化を図るオフセットが本実施形態で実効値を構成する。本来であれば、図２０に示すように被写体の色成分とＲＧＢの各成分との間には正比例の関係がなければならないが、撮像素子の特性などによって各成分毎に変換特性がずれていることがある。従来は、このようなずれを通常の画像から判断することはできなかった。しかしながら、各色成分毎の度数分布を取ったときに概ね類似しているとすれば、これは本来的に一致すべきと判断できるので、各成分毎のずれであるオフセット量を検出できることになる。
【０１２０】
本実施形態においては、ステップＳ２０４にてこのオフセット量を求め、ステップＳ２０６ではかかるオフセット量を考慮して色ずれを修正するために利用するテーブルを作成している。
【０１２１】
ＲＧＢの階調データ（Ｒp,Ｇp,Ｂp ）を利用している場合、テレビジョンなどでは全体の輝度ｙｐを
ｙp＝０．３０Ｒp＋０．５９Ｇp＋０．１１Ｂp …（２）
として求めている。すなわち、緑成分が最も輝度に影響を及ぼしており、その意味で緑成分に対する他の色成分のずれを修正するのが全体の画像イメージを変化させないというメリットがある。
【０１２２】
一方、各色成分毎の度数分布のずれを求めるには、この度数分布の特徴部分を考慮することが望ましい。このため、本実施形態においては、上述した端部処理を施した度数分布の上端（Ｒｍａｘ，Ｇｍａｘ，Ｂｍａｘ）と度数分布上におけるメジアン（Ｒｍｅｄ，Ｇｍｅｄ，Ｂｍｅｄ）とを利用している。両端位置は分布を判断する意味で有効である。ただし、下端位置については元々ずれの影響が分かりにくい範囲であるため、敢えて省略している。これにより、ずれの影響が大きい範囲で得られるずれだけを重視した修正が可能となる。分布の中央となるメジアンは極端な画素があったとしても度数分布の山の位置を示すことができる。この場合、かかる山の部分は画像のイメージに大きく影響を与える部分でもあるので特性を把握する意味で効果的である。
【０１２３】
このようにして得られた、青緑赤についての上端（Ｒｍａｘ，Ｇｍａｘ，Ｂｍａｘ）とメジアン（Ｒｍｅｄ，Ｇｍｅｄ，Ｂｍｅｄ）から緑成分に対する他の色成分のずれｄＲｍａｘ，ｄＢｍａｘ，ｄＲｍｅｄ，ｄＢｍｅｄを次式に基づいて求める。
【０１２４】
ｄＲｍａｘ＝Ｇｍａｘ−Ｒｍａｘ …（３）
ｄＢｍａｘ＝Ｇｍａｘ−Ｂｍａｘ …（４）
ｄＲｍｅｄ＝Ｇｍｅｄ−Ｒｍｅｄ …（５）
ｄＢｍｅｄ＝Ｇｍｅｄ−Ｂｍｅｄ …（６）
そして、これらを参考として赤成分用オフセットｄＲと青成分用オフセットｄＢとを次式のようにして求める。
【０１２５】
ｄＲ＝（ｄＲｍａｘ＋ｄＲｍｅｄ）／２ …（７）
ｄＢ＝（ｄＢｍａｘ＋ｄＢｍｅｄ）／４ …（８）
ただし、−１２＜ｄＲ，ｄＢ＜１２とする。このように制限するのは度数分布だけで完全な色再現性を修正できる訳ではない場合に、この一例だけで大きく度数分布を修正してしまわないようにするためである。むろん、この範囲については実験的な経験より適当な値を定めればよい。また、分母の相違は（２）色に基づく影響の相違に対応しており、実験等に基づいて適宜変更可能である。
【０１２６】
これらはオフセット量に過ぎないから、実際の統計値を次式に基づいて修正して新たな上端Ｒｍａｘ２，Ｂｍａｘ２とメジアンＲｍｅｄ２，Ｂｍｅｄ２と下端Ｒｍｉｎ２，Ｂｍｉｎ２とをする。
【０１２７】
Ｒｍａｘ２＝Ｒｍａｘ＋ｄＲ …（９）
Ｒｍｅｄ２＝Ｒｍｅｄ＋ｄＲ …（１０）
Ｒｍｉｎ２＝Ｒｍｉｎ＋ｄＲ …（１１）
Ｂｍａｘ２＝Ｂｍａｘ＋ｄＢ …（１２）
Ｂｍｅｄ２＝Ｂｍｅｄ＋ｄＢ …（１３）
Ｂｍｉｎ２＝Ｂｍｉｎ＋ｄＢ …（１４）
なお、この対応関係は緑成分に対する赤成分と青成分のオフセット量に過ぎないことは明らかであり、階調値によって変化しているわけではない。従って、現実の各画素における画像データについては一律に当該オフセット量を加算すれば足りる。
【０１２８】
ただし、後述するように本実施形態においては他の要因に基づく画像データの修正も行なっており、個別に修正することは演算時間を要して不利である。従って、演算の効率化を図るべく、本実施形態においてはステップＳ２０６にて変換前のＲＧＢの階調データ（Ｒ1,Ｇ1,Ｂ1）に対する変換後のＲＧＢの階調データ（Ｒ2,Ｇ2,Ｂ2）という対応関係を表すテーブルを形成し、現実に画像データを修正するのは最後に一回だけとなるようにしている。
【０１２９】
一方、上述した実施例では、ベクトルの内積に基づいて類似度を求めるとともに同類似度をしきい値ＣＯＲＲ（「０．７」）と比較して特性の均一化を行うか否かを決めている。従って、ベクトルの内積としきい値とがほぼ一致するような場合には、同じ画像でも周囲に付加されるビットの影響を受けて異なる判断結果がなされることになりかねない。
【０１３０】
むろん、特性の均一化を行うと判断されると上述したようなオフセット量が加算され、特性の均一化を行なわないと判断されると上述したようなオフセット量が加算されないことになるので、しきい値を挟んで大きな差が生じる。
【０１３１】
このようにして結果が大きく変わってしまうのを防止する手法として連続的に変化する窓関数を利用することが効果的である。
【０１３２】
ｘ＝ｍｉｎ（ｃｏｒｒ＿ｒｇ，ｃｏｒｒ＿ｇｂ，ｃｏｒｒ＿ｂｒ） …（１４１）
とおくとともに窓関数ｆ（ｘ）は、
ｘ＜０．５ のとき ｆ（ｘ）＝０
０．５≦ｘ≦０．７ のとき ｆ（ｘ）＝５・ｘ−２．５
０．７＜ｘ のとき ｆ（ｘ）＝１
とし、上記しきい値ＣＯＲＲを「０．５」とする。窓関数ｆ（ｘ）の変化状況を図３０に示しており、ｘ＜０．５において一定の「０」となり、０．５≦ｘ≦０．７において「０」から「１．０」へと直線的に増加し、０．７＜ｘにおいて一定の「１．０」となる。そして、（７）式と（８）式において赤成分用オフセットｄＲと青成分用オフセットｄＢとを求めていたのをこのｆ（ｘ）との積に改める。すなわち、
ｄＲ＝ｆ（ｘ）・（ｄＲｍａｘ＋ｄＲｍｅｄ）／２ …（７）’
ｄＢ＝ｆ（ｘ）・（ｄＢｍａｘ＋ｄＢｍｅｄ）／４ …（８）’
とする。図５および図６に示すフローチャートにおいては類似度がしきい値を越えていなければ非処理としてしまうが、このようにオフセットに対して窓関数を乗算する場合は、図３１に示すフローチャートを実行する。すなわち、類似度に基づいて非処理とすることはやめ、ステップＳ２０５で窓関数を利用したオフセット量の算出を実行し、この算出したオフセット量を利用する。むろん、上述したようにｘ＜０．７において窓関数ｆ（ｘ）が急激に「０」へ近づくため、しきい値付近であった画像が処理によって大きく変化するということはなくなるし、類似度が低いものについてはオフセットも殆ど「０」となって悪影響は与えない。また、このようにして算出したオフセット量ｄＲ，ｄＢに基づいて（９）式〜（１４）式も算出される。
【０１３３】
窓関数ｆ（ｘ）は上述した関数に限られるものでないことは明らかである。上述した例では、相関係数の最小値に基づいて窓関数の値を変化させているが、最小値に限られる必要はない。例えば、
ｆ（ｃｏｒｒ＿ｒｇ，ｃｏｒｒ＿ｇｂ，ｃｏｒｒ＿ｂｒ） …（１４２）としてもよい。また、さらに一般的にして、
ｆ（統計量）
としても良い。この場合、統計量には最小値、最大値、メジアン、標準偏差などが該当する。
【０１３４】
一方、窓関数は窓を開け閉めするごとくある領域において演算値を有効化させるとともに別の領域において演算値を無効化させるように機能する。今、色かぶりをその原因別に分類すると、上述した画像入力装置１０のハードウェア性能などによる場合と、夕焼けなどによる意図的な色かぶりと、タングステンランプなどを利用した特殊な照明による色かぶりとに分類できる。そして、夕焼けなどによる意図的な色かぶりについては特性の統一化を図る必要がないことは上述したとおりであるが、タングステンランプなどを利用した特殊な照明による色かぶりの場合は特性の統一化を図ることも有意義である。
【０１３５】
夕焼けなどによる意図的な色かぶりとタングステンランプなどを利用した特殊な照明による色かぶりは、上述した相関係数から判断できる。すなわち、相関係数が極端に低い写真はこのような特殊な照明に起因するものであることが多く、窓関数を図３２に示すように変形した。
【０１３６】
すなわち、ｘ＜０．５の領域を変形させ、
ｘ＜０．１ のとき ｆ（ｘ）＝１．０
０．１≦ｘ＜０．３ のとき ｆ（ｘ）＝−５・ｘ＋１．５
０．３≦ｘ０．５ のとき ｆ（ｘ）＝０
とした。これにより、ｘが「０．３」以下となると窓関数ｆ’（ｘ）は再び直線的に上昇し、ｘが「０．１」以下の領域で同窓関数ｆ’（ｘ）は一定の「１．０」となる。むろん、窓関数が「１．０」となったときには各成分の分布状況に基づいて演算された大きなオフセット量がそのまま適用され、照明の影響を打ち消して特性の均一化が図られる。
【０１３７】
なお、窓関数は遷移領域で直線的に変化する必要はなく、単調増加もしくは単調減少するような曲線的なものであっても構わない。
【０１３８】
また、図５に示すようにステップＳ１１６にて類似度をチェックし、類似度が低い場合にステップＳ１０６にて非処理を実行してフラグをセットしたり、あるいは、図３１に示すようにステップＳ１１６を実行することなくステップＳ２０５にて窓関数を利用してオフセット量を算出することにより、実質的には類似度が低いときに特性の均一化を図らないようにしているので、これらのソフトウェア処理及びこれを実現するハードウェアなどによって修正制御手段を構成していると言える。
【０１３９】
一方、度数分布の広がり方に差がある場合にはこれを均一化させることが有効である。本実施形態においては、この度数分布の広がり方を均一化させつつ、度数分布を可能な範囲で拡大させて各成分毎にコントラストを強調させている。
【０１４０】
コントラストの強調は、階調範囲が「０」〜「２５５」としたときに、変換前の各色成分（Ｒ1 ，Ｇ1 ，Ｂ1 ）と各成分の最大値Ｒｍａｘ２，Ｇｍａｘ，Ｂｍａｘ２と最小値Ｒｍｉｎ２，Ｇｍｉｎ，Ｂｍｉｎ２から変換先の各色成分（Ｒ2 ，Ｇ2 ，Ｂ2 ）を次式に基づいて求める。
【０１４１】
Ｒ2＝far×Ｒ1＋fbr …（１５）
Ｇ2＝fag×Ｇ1＋fbg …（１６）
Ｂ2＝fab×Ｂ1＋fbb …（１７）
ただし
far＝２５５／（Ｒｍａｘ２−Ｒｍｉｎ２） …（１８）
fag＝２５５／（Ｇｍａｘ −Ｇｍｉｎ ） …（１９）
fav＝２５５／（Ｂｍａｘ２−Ｂｍｉｎ２） …（２０）
fbr＝−far×Ｒｍｉｎ２あるいは２５５−far×Ｒｍａｘ２ …（２１）
fbg＝−fag×Ｇｍｉｎ あるいは２５５−fag×Ｇｍａｘ２ …（２２）
fbb＝−fab×Ｂｍｉｎ２あるいは２５５−fab×Ｂｍａｘ２ …（２３）
また、上記変換式にてＲ2,Ｇ2,Ｂ2 ＜０ならばＲ2,Ｇ2,Ｂ2 ＝０とし、Ｒ2,Ｇ2,Ｂ2 ＞２５５ならばＲ2,Ｇ2,Ｂ2 ＝２５５とする。ここにおける、far,fag,fab は傾きであり、fbr,fbg,fbb はオフセットといえる。この変換式によれば、図２２に示すように、あるせまい幅を持った輝度分布を再現可能な範囲まで広げることができる。なお、基本的に輝度の分布範囲の拡大においては、画素数が変化するわけではないので、ヒストグラムの面積は一致する。ただし、このように再現可能な範囲を最大限に利用して輝度分布の拡大を図った場合、ハイライト部分が白く抜けてしまったり、ハイシャドウ部分が黒くつぶれてしまうことが起こる。これを防止するため本実施形態においては、拡大する階調範囲を制限している。すなわち、全階調範囲の上端と下端に拡大しない範囲として階調値で「５」だけ残している。この結果、変換式のパラメータは次式のようになる。
【０１４２】
far＝２４５／（Ｒｍａｘ２−Ｒｍｉｎ２） …（２４）
fag＝２４５／（Ｇｍａｘ −Ｇｍｉｎ ） …（２５）
fav＝２４５／（Ｂｍａｘ２−Ｂｍｉｎ２） …（２６）
fbr＝５−far×Ｒｍｉｎ２あるいは２５０−far×Ｒｍａｘ２ …（２７）
fbg＝５−fag×Ｇｍｉｎ あるいは２５０−fag×Ｇｍａｘ２ …（２８）
fbb＝５−fab×Ｂｍｉｎ２あるいは２５０−fab×Ｂｍａｘ２ …（２９）
そして、この場合には階調「５」未満と、階調「２５０」以上については変換を行わないようにする。
【０１４３】
なお、本実施形態においては、ハイライト部分とハイシャドウ部分とを保持するために一律に階調範囲の端部から階調値にして「５」の範囲を非拡大領域としているが、ハイライト部分やハイシャドウ部分を比較的再現しやすいような画像出力装置であればその範囲を狭くしても良いし、再現力がさらに弱い場合にはより範囲を大きくするようにしても良い。また、一律に拡大しないのではなく、ボーダー領域で徐々に拡大率を制限するようにしていっても良い。
【０１４４】
また、図２３（ａ）には画像の輝度分布が狭い場合を示しているが、これまで述べたようにして輝度分布の拡大率（far,fag,fabに対応）を適用してしまうと、再現可能な範囲に合わせて非常に大きな拡大率が得られる場合も生じてくる。すると、夕方のような薄暮の状態では最も明るい部分から暗い部分までのコントラストの幅が狭くて当然であるのに、この画像についてコントラストを大きく拡大しようとする結果、昼間の画像のように変換されてしまいかねない。このような変換は希望されないので、拡大率には制限を設けていおき、far,fag,fabが１．５（〜２）以上とはならないように制限する。これにより、薄暮は薄暮なりに表現されるようになる。
【０１４５】
拡大率に制限を設けない場合を図２３（ａ）の一点鎖線に示しており、変換後には再現可能な範囲で余分な部分は残っていない。しかしながら、拡大範囲を制限する場合には、同図（ｂ）の二点鎖線で示すように、変換後の分布をどこに持ってくるかの自由度が生じてしまい、場合によっては全体的に明るくなりすぎたり、暗くなり過ぎたりしかねない。従って、このような場合には、変換前における階調範囲内において上端側と下端側に残っている残余の領域の割合（ｍ１：ｍ２）が、変換後において上端側と下端側に残っている残余の領域の割合（ｎ１：ｎ２）と一致するように変換すればよい。
【０１４６】
以上のようにしてパラメータfar,fag,fab,fbr,fbg,fbbを得る処理をステップＳ２０８にて実行し、続くステップＳ２１０ではステップＳ２０６と同様に変換テーブルを作成する。このときの変換テーブルは図２１に示す変換後の成分値（Ｒ1，Ｇ1，Ｂ1）を入力として対応関係を演算し、変換テーブルにおける変換後の成分値（Ｒ1，Ｇ1，Ｂ1）と入れ換える。これにより、同変換テーブルを参照すればステップＳ２０４にて求めたオフセット量の加算とステップＳ２０８にて求めたコントラストの強調処理という二つの処理を同時に実施することになる。
【０１４７】
一方、度数分布の各色成分間のずれとして、もう一つ、全体的な明るさという要素が残る。従って、ステップＳ２１４にてこの明るさのずれを均一化させるγ補正をかけるため、ステップＳ２１２にてγを算出する。例えば、図２４にて実線で示す赤成分の度数分布の山が全体的に暗い側に寄っていたり、同図にて一点鎖線で示す青成分の度数分布の山が全体的に明るい側に寄っていたりした場合に、同図鎖線で示す緑成分の度数分布の山のように全体的に中央に寄るように修正移動させると良い。
【０１４８】
各種の実験を行った結果、本実施形態においては、度数分布におけるメジアンを基準に判断し、同メジアンが「８５」未満である場合に暗い画像と判断して以下のγ値に対応するγ補正で明るくする。
【０１４９】
γr＝Ｒｍｅｄ２／８５ …（３０）
γg＝Ｇｍｅｄ／８５ …（３１）
γb＝Ｂｍｅｄ２／８５ …（３２）
あるいは、
γr＝（Ｒｍｅｄ２／８５）**（１／２） …（３３）
γg＝（Ｇｍｅｄ／８５）**（１／２） …（３４）
γb＝（Ｂｍｅｄ２／８５）**（１／２） …（３５）
とする。
【０１５０】
この場合、γr，γg，γb＜０．７となっても、γr，γg，γb＝０．７とする。このような限界を設けておかないと夜の画像が昼間のようになってしまうからである。なお、明るくしすぎると全体的に白っぽい画像になってコントラストが弱い画像になりやすいため、彩度を合わせて強調するなどの処理が好適である。
【０１５１】
一方、メジアンが「１２８」より大きい場合に明るい画像と判断して以下のγ値に対応するγ補正で暗くする。
【０１５２】
γr＝Ｒｍｅｄ２／１２８ …（３６）
γg＝Ｇｍｅｄ／１２８ …（３７）
γb＝Ｂｍｅｄ２／１２８ …（３８）
あるいは、
γr＝（Ｒｍｅｄ２／１２８）**（１／２） …（３９）
γg＝（Ｇｍｅｄ／１２８）**（１／２） …（４０）
γb＝（Ｂｍｅｄ２／１２８）**（１／２） …（４１）
とする。この場合、γr，γg，γb＞１．３となっても、γr，γg，γb＝１．３として暗くなり過ぎないように限界を設けておく。なお、暗くしすぎると色が乗りすぎて濃い画像になるので、合わせて彩度強調を弱くするなどの処理が好適である。ただし、明るい背景の中の被写体に対してはこのような暗くする処理はかえって悪影響を及ぼす場合もある。例えば、空が画像の半分をしめるような風景画像や晴れた日の記念写真などでは、ただでさえ逆光で顔が暗くつぶれ気味であることが多いからである。これらの画像の場合は暗い部分と明るい部分とが混じっているので各色成分の標準偏差を求めると比較的高い値となっていることが多い。従って、そのような標準偏差が「７０」よりも大きいような場合には暗くするためのγ補正を行わないようにすることも可能である。γ補正をした場合における対応関係を図２５に示しており、γr，γg，γb＜１であれば上方に膨らむカーブとなり、γr，γg，γb＞１であれば下方に膨らむカーブとなる。なお、明るさの修正に関しては必ずしも度数分布に基づく必要はなく、他の要素から明るさを判断して修正するようにしても良い。
【０１５３】
以上のようにして決定したγr，γg，γbに対してγ補正するには次式のようにする。
変換前の階調値ｒ0,ｇ0,ｂ0に対して変換後の階調値Ｒ1,Ｇ1,Ｂ1は、
Ｒ1＝２５５＊（ｒ0／２５５）**γr …（４２）
Ｇ1＝２５５＊（ｇ0／２５５）**γg …（４３）
Ｂ1＝２５５＊（ｂ0／２５５）**γb …（４４）
なお、このγ補正も図２１に示す変換テーブルに対して実行する。すなわち、コントラスト強調の場合と同様に図２１に示す変換後の成分値（Ｒ1，Ｇ1，Ｂ1）を入力として対応関係を演算し、変換テーブルにおける変換後の成分値（Ｒ1，Ｇ1，Ｂ1）と入れ換える。これにより、同変換テーブルを参照すればオフセット量の加算とコントラストの強調処理に加えて明度の修正の処理を同時に実施することになる。
【０１５４】
そして、最後に、ステップＳ２１６にて画像データの変換を行い、全画素の画像データ（ｒｍ，ｇｍ，ｂｍ）について図２１に示す変換テーブルを参照し、変換後の画像データ（Ｒｍ，Ｇｍ，Ｂｍ）を得るという処理を繰り返すことになる。
【０１５５】
本実施形態においては、オフセット量による修正と、コントラストの強調と、明るさの修正とをこの順番に実施しているが、必ずしもこの全てを実施しなければならないわけではないし、また、個々の修正手法も適宜変更して実施することもできる。
【０１５６】
例えば、上述した実施形態においては、コントラストの強調処理においては、変換前の成分値に対して直線的な対応関係をなす変換式で修正を行うようにしているが、より滑らかな変換となるように、図２６に示すようないわゆるＳ字カーブの変換を行うようにしても良い。また、この場合、度数分布の広がりを両端位置で判断するのではなく、分布の散らばり具合を表す標準偏差の概念を利用することも可能である。以下、標準偏差を利用したＳ字カーブの対応関係でコントラストを強調する例を示す。なお、各成分毎の度数分布に基づいて変換は行うものの、演算手法は全てにおいて共通するため、輝度の表現で表し、変換前の輝度ｙより変換後の輝度Ｙを得る手順を示しておく。
【０１５７】
標準偏差については二つの考え方があるが本実施形態においては、次式に基づいて演算する。
【０１５８】
【数５】

【０１５９】
ｙp：各画素の変換前の輝度
ｙm：各画素の変換前の輝度の平均値
標準偏差は輝度分布の広がり量に対応するものであるが、広がり量を表す意味では分散を利用してもよい。また、本実施形態のように全体としての階調数が２５６階調となっているので、分布の尖度ｋから広がり量を求めることも可能である。
【０１６０】
【数６】

【０１６１】
なお、ここにいうｋ＝３の尖度が正規分布の広がり量に相当する。
【０１６２】
このようにして求めた輝度分布の広がり量である標準偏差σに基づいて、コントラストの強調は分布密度の大きい範囲に多くの階調数を与えつつ分布密度の小さい範囲に少ない階調数を割り当てることでもあり、図２６に示すように入出力の対応関係がいわゆるＳ字カーブとなると変換前に割り当てられている階調範囲rng0に対して変換後に割り当てられる階調範囲RNG1は大きくなり、割り当てられた階調数が多くなったことになる。一方、入力における低輝度側と高輝度側における階調範囲rng0を外れた範囲についていえば、変換後に割り当てられる階調範囲は少なくなったことになる。
【０１６３】
本実施形態においては、階調範囲の中心位置ｙmidを「１２８」として、この中心位置ｙmid以下ではγ１を与えるとともに、中心位置ｙmid より大きい範囲ではγ２を与えたγ補正を行うものとし、このγ１，γ２を標準偏差σに基づいて定める。すなわち、
ｙ≦１２８では、
γ１＝（σstd_limit／σ）**fc …（４７）
ｙ＞１２８では、
γ２＝（σ／σstd_limit）**fc …（４８）
とし、ステップＳ２０４にてこれらのパラメータ演算を実行する。ここにおいて、σstd_limitとfcは変換結果を考慮して実験的に求めて与えたパラメータであり、本実施形態においてはσstd_limitを「１２８」とするとともにfcを「０．１」としている。標準偏差σは概して「１２８」よりも小さな値となるからこれらの関係式では標準偏差σが大きいと、γ２とγ１はそれぞれ「１」に近づくことになり、Ｓ字カーブの傾斜は緩やかになる。これは、広がり量が大きいときに中心位置ｙmidを中心とする階調範囲rng0に対して変換先の階調範囲RNG0はさほど広くならないことを意味しており、より具体的には画像データの輝度が広く分布しているときには輝度範囲を拡大するような変換を行わないことを意味する。これに対して、標準偏差σが小さいと、γ２とγ１はそれぞれ「１」から離れることになり、Ｓ字カーブの傾斜は急になる。これは、広がり量が小さいときに中心位置ｙmidを中心とする階調範囲rng0に対して変換先の階調範囲RNG1が広く拡大されることを意味しており、より具体的には画像データの輝度が狭い範囲にしか分布していないときには輝度範囲を拡大させる変換を行なうことを意味する。
【０１６４】
本実施形態においては、Ｓ字カーブの対応関係をγ補正によって成立させているが、図２７には、階調「０」、下方側四分点ｙq1、中心位置ｙmid、上方側四分点ｙq3、階調「２５５」という五点を基準点としつつ、階調「０」と中心位置ｙmidと階調「２５５」に対してはＹ＝ｙとしつつ、下方側四分点ｙq1と上方側四分点ｙq3における変換点を標準偏差に基づいて決定する。そして、これらの五点を結ぶ対応関係をスプライン補間演算やニュートン補間で求める。むろん、中心位置ｙmidから下方側の三点や上方側の三点をそれぞれスプライン補間演算やニュートン補間で求めるようにしてもよい。
【０１６５】
すなわち、図２８に示すように、ステップＳ２３０にて各成分毎に標準偏差σr，σg，σb を求め、ステップＳ２３２にて各色成分毎のγ１，γ２を求め、ステップＳ２３４にてγ１，γ２を利用した対応関係に基づいて変換テーブルを作成する。これらのステップＳ２３０〜Ｓ２３４を図６に示すフローチャートにおけるステップＳ２０８〜Ｓ２１４の処理に代えて実行する。
【０１６６】
次に、上記構成からなる本実施形態の動作を順を追って説明する。
【０１６７】
スキャナ１１などで写真を撮像したとすると、同写真をＲＧＢの階調データで表した画像データがコンピュータ２１に取り込まれ、ＣＰＵは図５及び図６に示す画像処理のプログラムを実行して画像データの色再現性を修正する処理を実行する。
【０１６８】
まず、ステップＳ１０２では画像データを所定の誤差内となる範囲で間引き、選択した画素について各色成分毎に度数分布を求める。このままの度数分布を使用することはできず、まず、画像が白黒のような二値画像でないかステップＳ１０４にて判断するとともに、ステップＳ１０８では自然画か否かを判断する。二値画像である場合や自然画でない場合などを除き、ステップＳ１１０では画像データに枠部がないか判断し、枠部があれば除いた後、ステップＳ１１４にて度数分布の両端の不明瞭領域を取り除く。この状態で各色成分毎の度数分布について特徴ベクトルを求め、特徴ベクトル同士の内積から度数分布の類似度をチェックする。各色成分毎の度数分布があまり似ていないときには、元の画像データにおいて意図的にカラーバランスがずれていることの裏付けとなり、特性を均一化させる処理は行わない。しかしながら、所定のしきい値との比較においてある程度の類似性が見られる場合にはカラーバランスがずれてしまっているものと判断して、以下のような特性の均一化を図る。
【０１６９】
すなわち、ステップＳ２０４では度数分布の上端とメジアンを利用して緑成分に対する赤成分オフセットｄＲと青成分用オフセットｄＢとを求め、ステップＳ２０６では最後の画像データ変換のための変換テーブルを形成する。続いて、ステップＳ２０８では各色成分毎にコントラストの均一化を図りつつ同コントラストを強調するためのパラメータを演算し、ステップＳ２１０では同パラメータに基づいてコントラスト強調させつつ各色成分毎のコントラストのバランスを一致させるための変換テーブルを作成する。そして、ステップＳ２１２では各色成分毎の明るさを均一化させるためのγ補正のパラメータを演算し、かかるγ補正を施すことになる変換テーブルを作成する。
【０１７０】
最後に、ステップＳ２１６にて上述したようにして作成されている変換テーブルを参照して全画素についての画像データを変換する。
【０１７１】
この場合は類似性があるか否かに応じてステップＳ２０２で処理を分けているが、図３１に示すフローチャートを実行する場合には類似具合に応じてオフセット量という実効値を変化させ、実質的に特性の均一化を図ったり図らなかったりする。
【０１７２】
むろん、上述したように二値画像や自然画でない場合などにおいてはかかる画像処理は行われないが、本発明の画像処理が行われた場合には、写真の状態では色ずれなどの入力機器に起因するような色再現性の悪かった画像データであるにもかかわらず、各色成分毎に色ずれとコントラストと明るさとが均一化するとともにコントラストを強調されてメリハリのある良好な画像が極めて容易に得られるようになる。
【０１７３】
なお、上述した実施形態においては、いくつかのパラメータを一定としているが、コンピュータ２１上では所定のＧＵＩを介してユーザーが選択できるようにしても良い。
【０１７４】
このように、ステップＳ１０２で間引きするなどしながら画像データについて各色成分毎の度数分布を求め、ステップ１１６にて度数分布間に類似性があるか否かを判断し、類似性が低くなければ本来的に度数分布から見出される特性は一致するものと判断して、ステップＳ２０４〜Ｓ２１６にてオフセット修正やコントラスト強調や明るさ修正によってずれを修正することにより、色再現性の悪い画像データであってもメリハリのある良好な画像としつつ、かかる作業を自動化し、非熟練者でも容易にカラーバランスの修正を行うことができるようになる。
【図面の簡単な説明】
【図１】 本発明の一実施形態にかかる画像処理装置が適用される画像処理システムのブロック図である。
【図２】 同画像処理装置の具体的ハードウェア構成例を示すブロック図である。
【図３】 本発明の画像処理装置の他の適用例を示す概略ブロック図である。
【図４】 本発明の画像処理装置の他の適用例を示す概略ブロック図である。
【図５】 本発明の画像処理装置における度数分布検出手段と類似具合判断手段とに相当するフローチャートである。
【図６】 本発明の画像処理装置におけるオフセット量修正手段とコントラスト修正手段と明るさ修正手段とに相当するフローチャートである。
【図７】 変換元の画像における座標を示す図である。
【図８】 サンプリング周期を示す図である。
【図９】 サンプリング画素数を示す図である。
【図１０】 変換元の画像とサンプリングされる画素の関係を示す図である。
【図１１】 度数分布を検出するための配列変数を示す図である。
【図１２】 グレー画像抽出間引き処理のフローチャートである。
【図１３】 白黒の画像を示す図である。
【図１４】 白黒の画像の輝度分布を示す図である。
【図１５】 非自然画と自然画の場合の各色成分毎の度数分布の状態を示す図である。
【図１６】 枠部のある画像を示す図である。
【図１７】 枠部のある画像の度数分布を示す図である。
【図１８】 度数分布の端部処理と端部処理にて得られる端部を示す図である。
【図１９】 各色成分毎の特徴ベクトルとするための要素の抽出方法を示す図である。
【図２０】 色再現性の修正の必要のないリニアな関係を示す図である。
【図２１】 度数分布に基づいて画像データを変換する際の変換テーブルを示す図である。
【図２２】 度数分布の拡大と全階調の範囲を示す図である。
【図２３】 コントラストの拡大率に制限を与える場合を示す図である。
【図２４】 明るさの均一化を図る必要のある度数分布を示す図である。
【図２５】 γ補正で変更される変換関係を示す図である。
【図２６】 Ｓ字カーブの対応関係でコントラストを強調させる変換関係を示す図である。
【図２７】 特定した変換点を補間法で接続する場合の変換関係を示す図である。
【図２８】 Ｓ字カーブでコントラストと明るさを修正する場合の部分フローチャートである。
【図２９】 画像処理プログラムを記録する媒体から同プログラムをハードディスクに転送する状態を示す図である。
【図３０】 利用する窓関数の変化状況を示すグラフである。
【図３１】 窓関数を利用してオフセット量を調整する場合における画像処理プログラムのフローチャートである。
【図３２】 利用する他の窓関数の変化状況を示すグラフである。
【符号の説明】
１０…画像入力装置
１１…スキャナ
１１ｂ…スキャナ
１２…デジタルスチルカメラ
１２ａ…デジタルスチルカメラ
１２ｂ…デジタルスチルカメラ
１３ｂ…モデム
２０…画像処理装置
２１…コンピュータ
２２…ハードディスク
３０…画像出力装置
３１…プリンタ
３１ａ…プリンタ
３１ｂ…プリンタ
３２…ディスプレイ
３２ａ…ディスプレイ
４１…フレキシブルディスク
４２…ＣＤ−ＲＯＭ
４３…ハードディスク
４４…ＲＯＭ
４５…ＲＡＭ
４６ａ…通信回線
４６ｂ…モデム
４６ｃ…ファイルサーバ[0001]
BACKGROUND OF THE INVENTION
  The present invention relates to an image processing apparatus, an image processing method, and a medium on which an image processing program is recorded, and more particularly to an image processing apparatus, an image processing method for correcting color balance, and a medium on which an image processing program is recorded.
[0002]
[Prior art]
  Conventionally, correcting the color balance often refers to correcting a so-called color cast. That is, correction processing for color misregistration in an image input device or the like.
[0003]
  For example, in a digital still camera or the like, image data is output as RGB (red, green, blue) gradation color specification data. In this case, depending on the characteristics of the lens and the CCD element, a certain color, for example, a so-called color shift in which red is emphasized over the actual color can be seen.
[0004]
  Conventionally, when it is desired to make some corrections on image data with such color misregistration, it is read into image processing software, etc., and the operator reduces the emphasis color component value through trial and error after a predetermined operation. ing.
[0005]
[Problems to be solved by the invention]
  The conventional correction method described above has the following problems.
[0006]
  Since the operator makes corrections by trial and error, the accuracy is poor, and it may be difficult for an unskilled person.
[0007]
  In addition, since the predetermined component value is increased or decreased uniformly, it may not be said that the correction is correctly made over all gradations.
[0008]
  Furthermore, it cannot be said that efficient correction is possible when elements other than simple color misregistration are included.
[0009]
  In addition, it is fundamentally impossible to judge whether or not the gradation color specification data for each component representing the color of each pixel is correct unless it is compared with a standard color or the like. In view of the fact that it is natural that a certain component is emphasized by an external factor, it can be said that it is extremely difficult to automatically determine whether there is a color shift. Absent.
[0010]
  SUMMARY OF THE INVENTION The present invention has been made in view of the above problems. An image processing apparatus, an image processing method, and image processing capable of automating correction of color reproducibility such as so-called color misregistration and correcting color balance comprehensively. The purpose is to provide a medium on which the program is recorded.
[0011]
[Means for Solving the Problems]
  To achieve the above purposeInFor image data in which an image is represented as a set of pixels in the form of a dot matrix by gradation color data consisting of roughly equal color components, each component value of the image data is input, subjected to a predetermined conversion process, and output. Thus, the image processing apparatus is configured to perform conversion for correcting the color balance depending on the relationship between the input and the output, and obtains the distribution of gradation color data for each color component and shifts between the color components. Characteristic equalization means for equalizing the characteristics between the respective color components based on the obtained deviation.It is also good.
[0012]
  Configured as aboveIfWhen there is image data in which an image is represented as a set of pixels in a dot matrix by means of gradation color data consisting of roughly equal color components, each component value of the image data is input and a predetermined conversion process is performed. In this case, the characteristic equalizing means obtains the distribution of the gradation color data for each color component and converts each color component. The difference between the color components is obtained, and the characteristics among the color components are made uniform based on the obtained deviation.
[0013]
  Taking a digital still camera as an example, conventionally, correcting the color balance means whether or not the conversion characteristics of the color components for each pixel match, and whether or not the component values for each color are emphasized. It was adjusted by judging. On the other hand, in the present invention, the distribution of gradation color data is obtained for each color component, and the characteristics are found in a state where the color components are separated for each color component, so that the characteristics are made uniform. That is, regardless of the content of the image, it can be seen that when the distribution characteristics of the gradation color data match for each color component, a clear image without color misregistration can be obtained as the image. Uniformity is determined based on the distribution. Note that the homogenization here does not mean the homogenization in a strict sense, but it may be at least the degree of homogenization.
[0014]
  In this sense, the distribution need only be such that the distribution status of each component in the image data can be found, and various statistical methods including frequency distribution can also be employed. For example, it is also possible to use secondary processing data such as a standard deviation derived from a frequency distribution, a median, or a mode value. In this case, the frequency distribution has an advantage that it can be easily processed and simplified.
[0015]
  By the way, it is natural that the characteristics of the distribution do not match. For example, in the case of evening scenery, there is no unnaturalness even if only the red color is used. In such a case, it is unnatural to make the characteristics determined from the distribution for each color component uniform. For this reason, OverviewFor image data representing an image as a set of a plurality of pixels by gradation color data composed of substantially equal color components, by inputting each component value of the image data and performing a predetermined conversion process, An image processing apparatus that performs conversion that corrects the color balance according to the relationship between input and output, and obtains the degree of similarity of the distribution of gradation color data for each color component, and when this degree of similarity is small A configuration that prevents the color balance from being correctedMay be.
[0016]
  Configured as aboveIfFirst, the degree of similarity between the color components is determined. If it is an evening scene when it is composed of red (R), green (G), and blue (B) color components, a biased distribution that is distributed only in the red component will be obtained. The distribution is extremely reduced. In such a case, it is natural that the characteristics of the distribution are not uniform, and the degree of similarity between the color components is extremely small, and the characteristics are not uniformized. On the other hand, when each color component appears on average, it is judged that the degree of similarity of the distribution is large, and in such a case, by making the mutual characteristics uniform, an inherent bias in the image input device or the like is obtained. It can also be reduced.
  As a technique for avoiding equalization of characteristics when the degree of similarity is small, it is possible to compare the degree of similarity with a predetermined threshold value and branch the determination based on the comparison result. On the other hand, a method of controlling the effective value can also be adopted as a method of making the characteristics uniform when the degree of similarity is large and not making the characteristics uniform when the degree of similarity is small. As an example of this, the invention according to claim 1 inputs each component value of the image data of the image data in which the image is represented as a set of a plurality of pixels by the gradation color data composed of roughly equal color components. An image processing apparatus that performs conversion for correcting the color balance according to the relationship between input and output by performing predetermined conversion processing and outputting the gradation color specification data for each color component. In addition to obtaining the distribution similarity, the correction is performed by continuously changing the effective value for equalizing the characteristics of each color component according to the similarity, and using the effective value after the calculation. Therefore, the correction is made in accordance with the similarity.
  In the invention according to claim 1 configured as described above, an effective value for achieving uniform characteristics is obtained, and the effective value is changed or changed by substantially changing or changing the effective value. For such control, a window function or the like is effective. More specifically, a window function that takes “1” when it is desired to be activated and becomes “0” when it is desired to be invalidated is integrated into an effective value. do it. On the other hand, in this example, if it changes discontinuously in the region changing from “1” to “0” or from “0” to “1”, the degree of similarity may be determined depending on how the gradation color data is captured. It may change slightly. As a result, the determination result may be the opposite of the same image. Therefore, as a countermeasure in such a case, the effective value is continuously changed in accordance with the above-mentioned similarity. Of course, the window function can be modified in various ways, and depending on the factors, it is possible to make the effective value effective in a region with a small similarity, or vice versa, to handle it exceptionally. is there. Of course, the effective value here refers to a value that is widely involved in the correction, such as the correction amount, the correction amount, or the offset amount, and the effective value is also calculated for the window function described above. One variable used in the process may be used.
[0017]
  When judging the degree of similarity of distributions, it is not always necessary to individually compare the frequency distributions in all gradations. As an example suitable for such a case,The invention according to claim 2 is the above claim 1.In the image processing apparatus described in the item 1, the correction control unit divides the gradation range that the gradation colorimetric data can take into a plurality of regions, and determines the degree of similarity between the color components by comparing the distribution of each region. It is as.The means for correcting the color balance is a characteristic uniformizing means, and a correction control means is a means for judging the degree of similarity between the respective color components based on the distribution and preventing the correction when the degree of similarity is small. Shall be called as appropriate
[0018]
  Configured as aboveClaim 2According to the invention, in determining the similarity of distribution, the gradation range that the gradation color specification data can take is divided into a plurality of areas, and the distribution of each area is compared among the color components. This reduces the labor for comparing the distributions seen for each gradation.
[0019]
  On the other hand, various methods can be used to determine the similarity of distributions.The invention according to claim 3 is the above-mentioned claim 1 or claim 2.In the image processing apparatus according to any one of the above, the correction control unit is configured to determine the degree of similarity based on an inner product of vectors having the distribution of each color component as an element.
[0020]
  The inner product of vectors can be easily performed in order to see the correlation between each other. Therefore, in order to adopt this method, the distribution itself is considered in the same way as a vector element, and an inner product is obtained between each color. When the degree of similarity is the largest, it is “1”, and when the degree of similarity is small, it approaches “0”. In considering the distribution as a vector element, it is not always necessary to use the elements for the total number of tones,Claim 2In the case of the invention described in (4), the distribution in the region into which the entire gradation range is classified into several may be used as the vector element.
[0024]
  In general, a small degree of similarity is not a color shift caused by a device. However, the degree of similarity may change drastically due to special lighting. Therefore, even if the degree of similarity is small, it may be effective to make the characteristics uniform depending on the cause. Therefore, in general, characteristics may not be equalized when the degree of similarity is small, but characteristics may be equalized even when the degree of similarity is small depending on the cause. For example, when the degree of similarity becomes extremely small, the characteristics can be made uniform, and a window function corresponding to the degree of similarity can be used to realize this.
[0025]
  Note that the method of unifying the characteristics is not necessarily limited to the idea of unifying the characteristics based on the similarity between the color components.
[0026]
  Also,Claim 14In the invention of the image processing method described in the above, with respect to image data in which an image is represented as a set of a plurality of pixels by gradation color data composed of roughly equal color components, each component value of the image data is input and predetermined An image processing method for performing conversion to correct the color balance in relation to input and output by performing the conversion process ofWhile calculating the similarity of the distribution of gradation color specification data for each color component, the correction is performed by continuously changing the effective value for equalizing the characteristics of each color component according to the similarity. And correct according to the above similar condition by using the effective value after the same calculationAs a configuration,Claim 15In the present invention, for image data in which an image is represented as a set of a plurality of pixels by gradation color data composed of roughly equal color components, each component value of the image data is input, subjected to a predetermined conversion process, and output. A medium on which is recorded an image processing program that causes a computer to execute conversion for correcting color balance in relation to input and output,While calculating the similarity of the distribution of the gradation color specification data for each color component, the correction is performed by continuously changing the effective value for equalizing the characteristics of each color component according to the similarity. And make corrections according to the above similarities by using the effective value after the calculationAs a configuration.
[0027]
  Configured like thisClaims 14 and 15Also in the invention, the degree of similarity of the distribution of gradation color data is obtained for each color component, and the color balance is not corrected when the degree of similarity is small.
[0028]
  On the other hand, when obtaining the distribution as described above, it is not always necessary to treat all pixels equally. And as an example,The invention according to claim 4 provides the above-described claims 1 to 3.In the image processing apparatus according to any one of the above, the characteristic uniformizing unit is configured to target pixels whose gradation colorimetric data of each color component is approximated.
[0029]
  Originally, a person can judge a color misregistration by looking at an image when the achromatic color is colored. That is, when only a red object is viewed, it is impossible to determine whether or not the color is shifted unless the original color is known. Therefore, it can be said that it is effective to examine the variation in characteristics of only achromatic pixels in order to examine the balance between the colors. For this reason, the characteristic uniformizing means determines for each pixel whether or not the gradation color specification data of each color component is close to each other, and counts it as a distribution target only when it is approximated. The characteristics are made uniform based on the distribution obtained as described above.
[0030]
  Whether or not the gradation color data of the color component in each pixel is approximate may be determined by obtaining the maximum value and the minimum value and determining the difference between them. It is also possible that the extreme value data is not handled because it is highly possible that the data is saturated.
[0031]
  By the way, as an example to achieve uniform characteristics,The invention according to claim 5 is the above first to fourth aspects.In the image processing apparatus according to claim 1, the characteristic uniformizing unit determines the characteristic from a predetermined position in the distribution, obtains an offset amount corresponding to a deviation between the components so that the characteristic is uniformed, and calculates each component value. As a configuration to correct.
[0032]
  Configured as aboveClaim 5In the invention according to the present invention, several positions such as the upper end, the lower end, the average value, the median, and the mode value are obtained including the process of obtaining the distribution for each color component. Can be found. Since the difference between these components is considered to be a characteristic that can cause a specific color shift, an offset amount corresponding to the shift is obtained to correct each component value.
[0033]
  As an example of a specific position,The invention according to claim 6 is the above-mentioned claim 5.In the image processing apparatus described in the item 1, the characteristic uniformizing unit is configured to determine an end position of the distribution range as the characteristic of the distribution.
[0034]
  Configured as aboveClaim 6In the invention according to the above, the deviation between the components is obtained by regarding the upper end and the lower end positions, which are the end positions of the distribution range, as the characteristics of the distribution.
[0035]
  If individual distributions are seen, extremely diverse distributions are easily determined uniformly if they are at the upper end position or the lower end position, and characteristics can be easily acquired.
[0036]
  As another example,The invention according to claim 7 is the above-mentioned claim 5 or claim 6.In the image processing apparatus according to any one of the above, the characteristic uniformizing unit is configured to determine a substantially center position of the distribution as a characteristic of the distribution.
[0037]
  Configured as aboveClaim 7In the invention according to the present invention, the position between the average value, the median, and the mode value, which is the approximate center position of the distribution, is regarded as the characteristics of the distribution, and the deviation between the components is obtained.
[0038]
  Similarly, extremely diverse distributions are easily determined uniformly if they are positions such as an average value, a median, and a mode value that are substantially central positions, and characteristics can be easily acquired.
[0039]
  In general images, although there is a difference in the absolute value of each component, there is often no significant change in the spread of the distribution. Therefore, it can be said that it is natural that the spread amount is uniform in each component. For this reason,The invention according to claim 8 is the above first to seventh aspects.In the image processing apparatus according to any one of the above, the characteristic uniformizing unit is configured to substantially uniform the spread of the distribution for each color component.
[0040]
  Configured as aboveClaim 8In the invention according to the above, the spread of the distribution for each component value is obtained, and the conversion process is performed so as to be substantially uniform.
[0041]
  The idea of the spread of the distribution can be understood in various ways, and does not need to be limited to a specific one.The invention according to claim 9 is the above claim 8.In the image processing apparatus described in (1), the characteristic uniformizing unit is configured to expand both ends of the distribution spread within an effective gradation range.
[0042]
  Configured as aboveClaim 9In the invention according to the present invention, since the distribution has a mountain shape and the width of the distribution is known at both ends of the base, the width is converted so as to be expanded within an effective gradation range. For example, if the distribution width is only a part of the effective gradation width, it can be said that the image has low contrast, but in such a case, the distribution spreads over the entire width of the effective gradation range. Enlarge. If the distribution is expanded so as to spread over the entire width of the effective gradation range for each color component, the characteristics of the distribution width become uniform. Note that the distribution width in this case is not limited to the actual distribution width, but may be such that an error range or the like is omitted by cutting both end portions by a predetermined amount.
[0043]
  As another example,The invention according to claim 10 is the above-mentioned claim 8.In the image processing apparatus according to claim 1, the characteristic uniformizing unit assigns a small number of gradations to a range with a small distribution density while giving a large number of gradations to a range with a large distribution density based on the spread amount of the distribution. It is as.
[0044]
  Configured as aboveClaim 10In the invention according to the present invention, the characteristic uniformizing means obtains the spread amount of each component, and based on the obtained distribution, gives a large number of gradations to a range where the distribution density is large for each component, and the distribution density is small. Assign a smaller number of gradations to the range. The concept of the spread amount corresponds to how the distribution is dispersed, as with the width of the distribution. In mathematical expression, it can be standard deviation, dispersion, kurtosis, or the like. When the characteristics as the spread amount are brought close to match each component, if the portion having a large distribution density is converted to spread within an allowable range, the match of the spread amount shifted for each component is changed. In addition to being able to see, the distribution is distributed using the effective range as much as possible without concentrating on a part of the range, so that the contrast is enhanced as a whole. Of course, the spread amount may be other than the standard deviation, variance, or kurtosis, and in these cases, calculation in a strict sense is not required, and the calculation can be simplified.
[0045]
  further,The invention according to claim 11 is the above first to tenth aspects.In the image processing apparatus according to any one of the above, the characteristic uniformizing unit is configured to uniform the brightness based on the distribution of each component.
[0046]
  Configured as aboveClaim 11In the invention according to the present invention, since the brightness in each component distribution is non-uniform, it may look like a color shift. Let The brightness here represents the overall position of the distribution. For example, if the distribution is located on the entire bright side, it is considered bright, and if the distribution is located on the entire dark side, it is considered dark.
[0047]
  Of course, the method of determining the overall brightness can be changed as appropriate, and various correction methods can be used as appropriate to make it brighter or darker.The invention according to claim 12 is the above-mentioned claim 11.In the image processing apparatus described above, the characteristic uniformizing unit is configured to determine the brightness of the image in a comparison between a substantially central position of the distribution and a predetermined gradation.
[0048]
  Configured as aboveClaim 12In the invention according to the present invention, the approximate center position of the distribution is compared with a predetermined gradation in an effective gradation range, and the brightness of the image is determined based on the comparison result.
[0049]
  For example, the median obtained when obtaining the distribution has a condition as the central position of the distribution, but whether the overall brightness is bright if the median is larger or smaller than the median value in the total number of gradations. It is possible to judge whether it is dark.
[0050]
  Also,The invention according to claim 13 is the above-mentioned claim 11 or claim 12.In the image processing apparatus described above, the characteristic equalizing unit is configured to equalize the brightness of the image by γ correction for each component based on the determination result of the brightness of the image.
[0051]
  Configured as aboveClaim 13In the invention according to the present invention, after the brightness of the image is determined by various methods, the brightness of the image is equalized by γ correction for each component based on the determination result. For example, if the image is dark, the entire image is corrected brightly by γ correction with γ <1, and the median approaches the median value of the total number of gradations, and if the image is bright, γ> 1. As a result of the γ correction, the image is corrected to be dark overall, and the median approaches the median value of the total number of gradations.
[0052]
  As described above, the method for determining the color balance deviation based on the comparison of the distributions of the respective color components is not necessarily limited to a substantial apparatus.IsFor image data in which an image is represented as a set of pixels in the form of a dot matrix by gradation color data consisting of roughly equal color components, each component value of the image data is input, subjected to a predetermined conversion process, and output. Therefore, when performing conversion to correct the color balance based on the relationship between input and output, the distribution of gradation color data is obtained for each color component, and the deviation between the color components is obtained. Based on the obtained deviation As a configuration to equalize the characteristics of each color componentAlso good.
[0053]
  That is, it is not necessarily limited to a substantial apparatus, and there is no difference that the method is also effective.
[0054]
  By the way, such an image processing apparatus may exist independently, or may be used in a state of being incorporated in a certain device, and includes various aspects as an idea of the invention. Moreover, it can be changed as appropriate, such as software or hardware.
[0055]
  As an example, in a printer driver that converts image data corresponding to printing ink based on dot matrix image data composed of gradation color specification data for roughly equivalent color components and prints the image data on a predetermined color printer, It is possible to obtain a distribution of gradation color specification data for each color component, make the characteristics between the components uniform from the distribution, and perform printing based on the uniformed image data.
[0056]
  In this case, the printer driver converts the input image data corresponding to the printing ink. At this time, the distribution is obtained for each color component in the image data, and the characteristic found from the distribution is the color component. It is converted and printed so as to be uniform. That is, by comparing the distributions obtained for each color component and making corrections so that the distributions match, the overall color balance is adjusted, and the individual colors of each pixel also have good color development.
[0057]
  In the case of software for an image processing apparatus as an embodiment of the idea of the invention, it naturally exists on a recording medium on which such software is recorded and must be used.
[0058]
  As an exampleIsA medium in which a program for image processing by a computer is recorded, and each component value of the image data for image data in which an image is represented as a set of pixels in a dot matrix form by gradation color data consisting of roughly equal color components When performing conversion for correcting the color balance according to the relationship between input and output, the distribution of gradation color data is obtained for each color component and the color component data of each color component is output. As a configuration that obtains deviations between the color components and uniformizes the characteristics between the color components based on the obtained deviationsAlso good.
[0059]
  Of course, the recording medium may be a magnetic recording medium, a magneto-optical recording medium, or any recording medium that will be developed in the future. In addition, the duplication stages such as the primary duplication product and the secondary duplication product are equivalent without any question.
[0060]
  Further, even when a part is software and a part is realized by hardware, there is nothing completely different in the idea of the invention, and a part is stored on a recording medium, and it is appropriately changed as necessary. It may be in the form of being read. Furthermore, it goes without saying that the present invention can also be applied to image processing apparatuses such as color facsimile machines, color copiers, color scanners, digital still cameras, and digital video cameras.
[0061]
【The invention's effect】
  As described above, an image processing apparatus that can automate the determination of color misregistration that is fundamentally difficult to determine based on the concept of uniform distribution for each component and can adjust the color balance more easily. can do.
  According to the invention of claim 1,By continuously changing the effective value to achieve uniform characteristics, it is difficult to generate discontinuities in the image processing results.When the degree of similarity of distribution is small, do not try to make the characteristics uniform, and if the color balance is natural, the characteristics are made uniform to prevent an unnatural image. be able to.
[0062]
  According to the invention according to claim 2, since the distribution is obtained by dividing the gradation range into a plurality of regions, the comparison becomes easy. According to the invention according to claim 3, vectors having the distribution for each color component as elements are compared. Since the degree of similarity is judged based on the inner product ofThe
[0063]
  further,Claim 4According to the invention, since the pixel whose tone colorimetric data is originally approximated is picked up and used for the determination of the characteristic, it is effective for obtaining the characteristic variation.
[0064]
  further,Claim 5According to the invention, since the shift of the distribution position is used, the offset amount between the components can be determined relatively easily.
[0065]
  further,Claim 6According to the invention, the end of the distribution can be easily determined uniformly, and the determination can be easily performed.
[0066]
  further,Claim 7According to the invention, since the shift at the position where the distribution density is high is used, the error in correcting the overall shift is small.
[0067]
  further,Claim 8According to the present invention, in order to make the spread of the distribution uniform, it is possible to efficiently develop colors when the number of effective gradations is not effectively used for each color component, and it is possible to use sharpness. it can.
[0068]
  further,Claim 9According to the invention, since the distribution is expanded using both ends of the spread of the distribution that is easy to determine, the configuration for determining becomes easy.
[0069]
  further,Claim 10According to the invention, since a spread amount obtained by a mathematical method or the like is used, more accurate determination can be made.
[0070]
  further,Claim 11According to the invention, it is possible to eliminate color misregistration such that only one of the components protrudes by making the brightness uniform for each component.
[0071]
  further,Claim 12According to the invention, it is easy to determine the brightness by using the approximate median of the distribution.
[0072]
  further,Claim 13According to the invention, the brightness is made uniform by changing the brightness by the γ correction, and the configuration is easy because the γ correction is widely used.
[0073]
  further,eachWith the concept of uniform distribution for each component, it is possible to provide an image processing method capable of automating the determination of color misregistration that is fundamentally difficult to determine, and more easily adjusting the color balance.
[0074]
  furtherThisIt is possible to provide a medium on which a program for performing image processing as described above is recorded.
[0075]
  AndClaims 14 and 15In the invention according toBy continuously changing the effective value to achieve uniform characteristics, it is difficult to generate discontinuities in the image processing results.When the degree of similarity of distribution is small, do not try to make the characteristics uniform, and if the color balance is natural, the characteristics are made uniform to prevent an unnatural image. be able to.
[0076]
DETAILED DESCRIPTION OF THE INVENTION
  Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0077]
FIG. 1 is a block diagram illustrating an image processing system according to an embodiment of the present invention, and FIG. 2 is a block diagram illustrating a specific hardware configuration example.
[0078]
  In the figure, an image input device 10 captures an image and outputs image data to the image processing device 20, and the image processing device 20 performs image processing such as equalizing characteristics to the image output device 30. The image output device 30 displays an image with enhanced contrast.
[0079]
  Here, a specific example of the image input device 10 corresponds to the scanner 11, the digital still camera 12, or the video camera 14, and a specific example of the image processing device 20 corresponds to a computer system including the computer 21 and the hard disk 22. Specific examples of the output device 30 include a printer 31, a display 32, and the like. Of course, the present invention can be applied to color copiers, color facsimile machines and the like in addition to these.
[0080]
  In the present image processing system, an image having poor color reproducibility such as color misregistration is to be corrected. Therefore, the image processing system may be image data obtained by capturing a photograph with the scanner 11 serving as the image input device 10, or The image data captured by the digital still camera 12 or the moving image captured by the video camera 14 is a processing target and is input to a computer system as the image processing apparatus 20. Note that the calculation speed of the input image of the video camera 14 may not be in time. In such a case, it is possible to cope with this by setting the first condition that requires calculation time for each shooting scene and performing only the image conversion of each frame under the same condition setting during shooting.
[0081]
  The image processing apparatus 20 includes at least a frequency distribution detecting unit that obtains a frequency distribution for each color component, a similarity degree determining unit that determines the degree of similarity of the detected frequency distribution for each color component, and each color component from the frequency distribution. Offset amount correcting means for performing correction to determine and equalize deviations, contrast correcting means for performing correction to determine and equalize the difference in contrast of each color component from the frequency distribution, and each color component from the frequency distribution Brightness correction means for performing correction to determine and uniformize the brightness shift for each is configured. Of course, the image processing apparatus 20 may also be a color conversion unit that corrects a color difference for each model, or a resolution conversion unit that converts a resolution corresponding to each model. . In this example, the computer 21 executes each image processing program stored in the internal ROM or the hard disk 22 while using a RAM or the like. In recording these programs, as shown in FIG. 29, not only a portable medium such as a flexible disk 41 and a CD-ROM 42 but also a state installed in the hard disk 43, It is also possible to use an IC card having a ROM 44 or a RAM 45, or use the communication line 46a as a medium via a modem 46b or the like. In this case, a file server 46c is connected to the end of the communication line 46a, and the file server 46c provides predetermined software.
[0082]
  The execution result of this image processing program is obtained as sharp image data with corrected color reproducibility as will be described later, and is printed by the printer 31 which is the image output device 30 based on the obtained image data, or the same The image is displayed on the display 32 which is the image output device 30. More specifically, this image data is gradation data of “256” gradations for each of RGB (green, blue, red), and the image has a vertical direction (height) and a horizontal direction (width). ) In a matrix of dot matrix data.
[0083]
  In the present embodiment, a computer system is incorporated between image input / output devices to perform image processing. However, such a computer system is not necessarily required, and a digital still camera as shown in FIG. A system in which an image processing device for correcting color reproducibility and the like is incorporated in 12a and the converted image data is displayed on the display 32a or printed on the printer 31a may be used. Further, as shown in FIG. 4, in the printer 31b that inputs and prints image data without going through a computer system, the image data inputted through the scanner 11b, the digital still camera 12b, the modem 13b or the like is automatically selected. It is also possible to configure so as to correct the color reproducibility.
[0084]
  Of the image processing executed by the computer 21, the processing corresponding to the frequency distribution detecting means and the similarity determining means is shown in FIG. 5, and the offset amount correction executed when the similarity for each color component is not small. FIG. 6 shows processes corresponding to the means, the contrast correction means, and the brightness correction means. In addition, it can be said that the similarity determination means also constitutes a control means for controlling the effectiveness of the subsequent processing based on the similarity in a broad sense.
[0085]
  FIG. 5 mainly corresponds to processing for detecting the frequency distribution for each color component. First, the pixels to be distributed will be described.
[0086]
  Although the frequency distribution may be obtained for all pixels, it is not always necessary to obtain the frequency distribution for all pixels because the purpose is to determine the tendency of characteristics. Therefore, it is possible to perform thinning to such an extent that it is within a certain error range. In the present embodiment, as shown in step S102 in FIG. 5, a thinning process for thinning out target pixels is executed. According to the statistical error, the error with respect to the number of samples N can be expressed as 1 / (N ** (1/2)). However, ** represents a power. Therefore, in order to perform processing with an error of about 1%, N = 10000.
[0087]
  As shown in FIG. 7, a bitmap image is formed as a two-dimensional dot matrix composed of predetermined dots in the vertical direction and predetermined dots in the horizontal direction, and this bitmap screen has (width) × (height). Therefore, the sampling period ratio is
  ratio = min (width, height) / A + 1 (1)
And This min (width, height) is the smaller of width and height, and A is a constant. Further, the sampling period ratio here indicates how many pixels are sampled, and the pixels marked with ◯ in FIG. 8 indicate the case where the sampling period ratio = 2. That is, one pixel is sampled every two pixels in the vertical and horizontal directions, and every other pixel is sampled. The number of sampling pixels in one line when A = 200 is as shown in FIG.
[0088]
  As can be seen from the figure, except for the case where the sampling period ratio = 1 in which sampling is not performed, when there is a width of 200 pixels or more, the number of samples is at least 100 pixels. Therefore, in the case of 200 pixels or more in the vertical direction and the horizontal direction, (100 pixels) × (100 pixels) = (10000 pixels) is secured, and the error can be reduced to 1% or less.
[0089]
  Here, the reason for using min (width, height) as a reference is as follows. For example, as shown in FIG. 10 (a), when width >> height is satisfied and the sampling period ratio is determined by the longer width, as shown in FIG. 10 (b). In addition, in the vertical direction, only two lines of the upper end and the lower end may be extracted. However, if the sampling period ratio is determined based on the smaller one as min (width, height), thinning is performed so as to include the intermediate portion in the smaller vertical direction as shown in FIG. Will be able to.
[0090]
  Of course, the pixels sampled in this way are counted for each component value by array variables (CNT_R, CNT_G, CNT_B) prepared corresponding to all the gradations of RGB as shown in FIG. And obtain the frequency distribution.
[0091]
  In this example, the frequency distribution is obtained by thinning out the vertical and horizontal pixels at an accurate sampling period. This is suitable for processing while thinning out pixels that are sequentially input. However, when all the pixels have been input, the pixels may be selected by randomly specifying coordinates in the vertical direction or the horizontal direction. In this way, when the necessary minimum number of pixels, such as 10,000 pixels, is determined, the extraction process is repeated at random until the number of pixels reaches 10,000, and the extraction is stopped when the number of pixels reaches 10,000.
[0092]
  Further, it is also possible to perform decimation that specifies the target pixel for such simple decimation processing that performs decimation without specifying the target pixel. That is, if the difference between the RGB components of each pixel is small, it is close to gray, and if only gray pixels are extracted and the frequency distributions are compared, it can be said that it is easy to judge the color characteristics of the input device. is there. FIG. 12 shows a process of extracting and thinning out such gray pixels. In step S302, the maximum component among the components in the target pixel is specified, and in step S304, the minimum component is specified. In step S306, a maximum component difference that is the difference between the maximum value and the minimum value is calculated, and in step S308, it is determined whether or not the maximum component difference is within a range smaller than a predetermined threshold value. To do. As an example, since the pixel is close to gray, the maximum component difference is assumed to be within “52” gradations. In step S310, the frequency distribution is counted for pixels within such a gradation. On the other hand, if it exceeds such a gradation, it is not gray and is not used for counting. Thereafter, the target pixel is moved to the next pixel in step S312, and the above-described processing is repeated until it is determined as the final pixel in step S314.
[0093]
  Although the frequency distribution is obtained for the pixels selected by thinning out in this way, it may not always be appropriate to perform image correction using the frequency distribution. Therefore, the following three items are checked.
[0094]
  The first is when the image is a binary image such as a black and white image. The concept of correcting color reproducibility is inappropriate for binary images including black and white images. If there is a black-and-white image as shown in FIG. 13, the frequency distribution for each color component for this image is concentrated at both ends in the gradation number allocation range as shown in FIG. Basically, it concentrates on the gradation “0” and the gradation “255”.
[0095]
  Therefore, when performing the black and white check in step S104, it can be determined whether or not the sum of the number of pixels of gradation “0” and gradation “255” for each color component matches the number of pixels selected by thinning. . In the case of a black and white image, non-processing is executed in step S106 in order to interrupt the processing without executing the following processing. In the present embodiment, the process is roughly divided into the former process for obtaining the frequency distribution and the latter process for correcting the image data. Therefore, in this non-processing, a flag is set so that the subsequent luminance conversion process is not executed. Thus, the distribution extraction process is completed.
[0096]
  The binary data is not limited to black and white, and there may be binary data with color. In such a case as well, the process for correcting the color reproducibility is not necessary, and if the distribution state is examined and the distribution becomes two colors, the process may be interrupted as binary data. For example, in the case of two colors of an intermediate color and black, the distribution is concentrated on two gradations in the frequency distribution for each color component. On the other hand, in the case of two colors of blue and black, the distribution is concentrated on two gradations only in the frequency distribution of the blue component, and the distribution is concentrated only on the gradation “0” in the red component and the green component. That is, since the concentration of distribution is less than two gradations in the case of binary data, a binary image is obtained by scanning the array variable and counting how many gradations there are other than “0”. It can be determined whether or not.
[0097]
  Second, consider whether the image is like a business graph or a natural picture like a photograph. Needless to say, correction of color reproducibility is necessary for natural images, but it is natural that there is a bias in the colors originally used in business graphs and paintings. It is impossible to perform a process for making the characteristics of each color component uniform from the frequency distribution of the image. Accordingly, in step S108, it is checked whether or not the image is a natural image.
[0098]
  In natural paintings, the number of colors including shadows is extremely large, but in certain types of paintings such as business graphs and draws, the number of colors is often limited. Therefore, if the number of colors is small, it can be determined that the image is not a natural image. When each RGB component has 256 gradations, it can represent 16.7 million colors, and if it is attempted to accurately determine the number of colors, it is possible to determine how many of these 16.7 million colors are used. It is necessary to prepare as many array variables as possible, which is not realistic. On the other hand, since a frequency distribution has already been obtained, it can be determined whether or not the number of tones is effectively used for each color component by determining how many gradations are effectively used for each color component.
[0099]
  FIGS. 15A to 15C show an example of the frequency distribution in the case of a business graph, and FIGS. 15D to 15F show an example of the frequency distribution in the case of a natural image. As is clear from this example, the frequency distribution is a line spectrum in a non-natural image. In the processing in the computer 21, the number of gradations whose frequency is not “0” is counted for each color component and added together. If it is a natural image, it is considered that it is distributed almost uniformly over all gradations, and the count value is often “768 (= 256 × 3)”. Even if the “20” gradation is used, the count value is only about “60 (= 20 × 3)”. Therefore, if “200” is set as the threshold value for determining whether or not the image is a natural image, it may be determined that the image is a non-natural image if “200” or less. If it is determined that the image is a non-natural image, non-processing is executed in step S106. Of course, the threshold value “200” can be changed as appropriate.
[0100]
  It is also possible to determine whether or not the frequency distribution is a line spectrum based on the adjacent ratio of gradation values whose frequency is not “0”. That is, it is determined whether or not there is a distribution of adjacent gradation values that are gradation values whose frequency is not “0”. If at least one of the two adjacent gradation values is adjacent, nothing is performed, and if both are not adjacent, the count is performed. As a result, the number of gradation values other than “0” and the count value Judge by percentage. For example, if the number of gradation values other than “0” is “64” and the number of non-adjacent ones is “64”, it can be seen that the distribution is in a line spectrum.
[0101]
  Further, when the image processing program is executed via the operating system, it is possible to make a determination based on the extension of the image file. Of bitmap files, especially photographic images are compressed, and an implicit extension is often used to indicate the compression method. For example, an extension “JPG” indicates that the file is compressed in the JPEG format. Since the operating system manages the file name, if you send an inquiry to the operating system from the printer driver, the extension of the file will be answered. If so, the following processing may be executed. Also, an extension unique to a business graph such as “XLS” can be determined as a non-natural image, and non-processing may be selected as described above.
[0102]
  A third consideration is whether or not there is a frame around the image as shown in FIG. If such a frame is white or black, as shown in FIG. 17, the frequency distribution appears as a line spectrum at both ends in the gradation number allocation range, and both ends corresponding to the internal natural image. Appears as a smooth frequency distribution inside.
[0103]
  Of course, since it is appropriate not to include the frame part in the frequency distribution, the sum of the number of pixels of the gradation “0” and the gradation “255” is sufficiently large in the check of the frame part in step S108 and thinned out. It is determined whether the number of selected pixels does not match. If the result is affirmative, it is determined that there is a frame portion, and the frame portion processing is performed in step S112. In this frame processing, the number of pixels of gradation “0” and gradation “255” in the frequency distribution is set to “0” in order to ignore the frame. Thereby, in the following process, it can handle like the thing without a frame part.
[0104]
  In this example, a white or black frame portion is targeted, but there may be a case where there is a specific color frame. In such a case, a protruding line spectrum appears in the original smooth curve drawn by the frequency distribution. Accordingly, a line spectrum having a large difference between adjacent luminance values may be considered as a frame and not subjected to frequency distribution. In this case, since the color may be used other than the frame portion, the average of the frequencies of the gradation values on both sides may be calculated and assigned.
[0105]
  After undergoing the above consideration, if it is not non-processing, both ends of the frequency distribution are obtained in step S114. The frequency distribution in a natural image often appears in a mountain shape as shown in FIG. Of course, there are various positions and shapes. The base of such a frequency distribution changes as it approaches “0” as far as it can be seen statistically. Therefore, in order to specify a certain frequency distribution, it is important to specify both ends of the mountain, but any distribution characteristic can be obtained under the condition that the frequency is actually “0”. It can match.
[0106]
  For this reason, in the distribution range, portions that are inside by a certain distribution ratio from the side with the highest luminance and the side with the lowest luminance are defined as both ends of the distribution. In the present embodiment, as shown in FIG. 18, this distribution ratio is set to 0.5%. Of course, this ratio can be appropriately changed. In this way, by cutting the upper end and the lower end by a certain distribution ratio, it is possible to ignore white spots and black spots caused by noise or the like. On the other hand, if such a process is not performed, if there is even a single white point or black point, it will be at both ends of the frequency distribution. Therefore, in many cases, the lowest end is the gradation “0”, and the highest end Becomes the gradation “255”. However, such a problem can be eliminated by setting the portion that is inward from the both end portions by the number of pixels of 0.5%.
[0107]
  In actual processing, 0.5% of the number of pixels to be processed (the total number of pixels selected in the thinning process or the total number of pixels corresponding to the frame portion is deleted) is calculated, and the upper end side and lower end side in the frequency distribution Then, the respective frequencies are accumulated in order toward the inner side in order to obtain a gradation value of 0.5%. For each color component of RGB, the upper end side is called Rmax, Gmax, Bmax, and the lower end side is called Rmin, Gmin, Bmin.
[0108]
  As described above, it may be natural that the frequency distribution of each color component becomes non-uniform. In such a case, the color reproducibility should not be corrected. From the results, it is determined that the frequency distribution should be uniform in the situation where the frequency distribution of each color component is somewhat similar, and should not be uniform if the frequency distribution is not similar. it can.
[0109]
  Therefore, in this embodiment, the similarity of the frequency distribution for each color component is checked in step S116. Now, assuming that the frequency distribution for each color component appears as shown in FIG. 19, the entire gradation range is divided into four regions ([0-63], [64-127], [128-192], [193]. ˜255]), and consider a feature vector whose elements are frequencies belonging to each region. Taking the red component as an example, assuming that the frequency in each region is represented by r64, r128, r192, and r255, and the total number of effective pixels is r_pixel, the feature vector of the red component can be represented by Equation (1).
[0110]
[Expression 1]

[0111]
  The same processing is executed for the green component and the blue component, and then the inner product of the feature vectors between the color components is obtained. That is, the inner product corr_rg of the feature vector between the red component and the green component is obtained as expressed by Equation 2, and the inner product corr_gb of the feature vector between the green component and the blue component is obtained as expressed by Equation 3. The inner product corr_br of the feature vector between the blue component and the red component is obtained as expressed in Equation 4.
[0112]
[Expression 2]

[0113]
[Equation 3]

[0114]
[Expression 4]

[0115]
  It can be said that the inner product of the vectors represents the similarity between the two vectors, and the values are “0” to “1”. Therefore, assuming that “0.7” is set as the threshold CORR, if any one of the inner products corr_rg, corr_gb, corr_br is less than the threshold CORR, it is determined that the degree of similarity is low. Thus, the non-processing of step S106 is executed.
[0116]
  In the present embodiment, the processing based on the inner product of the feature vectors constitutes a similarity determination means, and a method is established and can be easily determined if the inner product is calculated based on the feature vectors. However, it is not limited to this example. For example, the whole gradation range is divided into four areas, but it is free to make it more than these, and it is approximated using statistical methods such as the position of both ends of the frequency distribution, standard deviation, kurtosis, etc. It is also possible to determine the degree.
[0117]
  Here, a specific method for obtaining the degree of approximation using such a statistical method will be described. An example of a statistical method is one that uses a representative value of a distribution. Now, the absolute values of the difference between the average value, the median value, and the standard deviation (variance) are obtained as variables between the red component and the green component, the green component and the blue component, and the blue component and the red component. Assuming that the absolute value of the difference between the red component and the green component with respect to the average value and the standard deviation is Ave_rg, Std_rg, as an evaluation function between the red component and the green component
Set h (rg) = (1-Ave_rg / 255) × (1-Std_rg / 255). Similarly, as an evaluation function between the green component and the blue component,
h (gb) = (1-Ave_gb / 255) × (1-Std_gb / 255) and, as an evaluation function between the blue component and the red component,
h (br) = (1-Ave_br / 255) × (1-Std_br / 255) is set. When the distributions are similar, the average value, median value, and standard deviation (variance) are almost the same, and the difference between the variables is almost “0”, so the value of the evaluation function h is close to “1”. . When the distributions are different, the difference becomes large and the value of the evaluation function h becomes small. Therefore, it is possible to determine whether or not to perform correction by comparing the evaluation function with a threshold value obtained by experiment. Of course, in this case, it is possible to use the median instead of the average value, and the specific example of the evaluation function is not limited to this.
[0118]
  If a certain degree of similarity is found for the frequency distribution obtained for each color component by the processing as described above, it can be determined that the characteristics of each component can be found from each frequency distribution, and uniformization between the components is performed based on such characteristics. Plan. As a result of the various determinations described above, there is a case where the flag is set by executing non-processing in step S106. In such a case, since the flag is referred to in step S202, the present process is terminated without performing the following equalization process.
[0119]
  In the process of equalizing the characteristics, first, the offset is calculated and corrected. This corresponds to correction of color misregistration in a so-called narrow sense, and an offset for achieving uniform characteristics constitutes an effective value in this embodiment. Originally, as shown in FIG. 20, there should be a direct proportional relationship between the color component of the subject and each of the RGB components, but the conversion characteristics are shifted for each component due to the characteristics of the image sensor. Sometimes. Conventionally, such a shift cannot be determined from a normal image. However, if the frequency distribution for each color component is substantially similar, it can be determined that they should be essentially matched, so that an offset amount that is a deviation for each component can be detected.
[0120]
  In this embodiment, the offset amount is obtained in step S204, and a table used to correct the color misregistration is created in step S206 in consideration of the offset amount.
[0121]
  When RGB gradation data (Rp, Gp, Bp) is used, the overall luminance yp is set for television or the like.
  yp = 0.30Rp + 0.59Gp + 0.11Bp (2)
Asking. That is, the green component has the most influence on the luminance, and in that sense, correcting the deviation of the other color components with respect to the green component has an advantage that the entire image is not changed.
[0122]
  On the other hand, in order to obtain the deviation of the frequency distribution for each color component, it is desirable to consider the characteristic portion of the frequency distribution. For this reason, in the present embodiment, the upper end (Rmax, Gmax, Bmax) of the frequency distribution subjected to the end processing described above and the median (Rmed, Gmed, Bmed) on the frequency distribution are used. Both end positions are effective in determining the distribution. However, the lower end position is omitted because it is in a range where it is difficult to understand the influence of the shift. As a result, it is possible to perform correction with emphasis only on the deviation obtained in a range where the influence of the deviation is large. The median at the center of the distribution can indicate the position of the mountain of the frequency distribution even if there are extreme pixels. In this case, since such a mountain portion is a portion that greatly affects the image, it is effective in grasping the characteristics.
[0123]
  The deviations dRmax, dBmax, dRmed, and dBmed of the other color components with respect to the green component from the upper end (Rmax, Gmax, Bmax) and the median (Rmed, Gmed, Bmed) for blue-green red obtained in this way are expressed by the following equations. Ask based on.
[0124]
  dRmax = Gmax−Rmax (3)
  dBmax = Gmax−Bmax (4)
  dRmed = Gmed−Rmed (5)
  dBmed = Gmed−Bmed (6)
  Then, with reference to these, the red component offset dR and the blue component offset dB are obtained as follows.
[0125]
  dR = (dRmax + dRmed) / 2 (7)
  dB = (dBmax + dBmed) / 4 (8)
  However, -12 <dR, dB <12. The reason for this limitation is to prevent the frequency distribution from being greatly corrected only by this example when the complete color reproducibility cannot be corrected only by the frequency distribution. Of course, an appropriate value may be determined for this range based on experimental experience. The difference in the denominator corresponds to (2) the difference in influence based on the color, and can be changed as appropriate based on experiments and the like.
[0126]
  Since these are only offset amounts, the actual statistical values are corrected based on the following equation to obtain new upper ends Rmax2, Bmax2, median Rmed2, Bmed2, and lower ends Rmin2, Bmin2.
[0127]
  Rmax2 = Rmax + dR (9)
  Rmed2 = Rmed + dR (10)
  Rmin2 = Rmin + dR (11)
  Bmax2 = Bmax + dB (12)
  Bmed2 = Bmed + dB (13)
  Bmin2 = Bmin + dB (14)
  It is obvious that this correspondence is only an offset amount of the red component and the blue component with respect to the green component, and does not change depending on the gradation value. Therefore, it is sufficient to uniformly add the offset amount for the actual image data in each pixel.
[0128]
  However, as will be described later, in the present embodiment, the image data is also corrected based on other factors, and it is disadvantageous to individually correct the image data because it requires a calculation time. Therefore, in order to increase the efficiency of the calculation, in this embodiment, the RGB gradation data (R2, G2, B2) after the conversion to the RGB gradation data (R1, G1, B1) before the conversion in step S206. The table representing the correspondence relationship is formed so that the image data is actually corrected only once.
[0129]
  On the other hand, in the above-described embodiment, the similarity is obtained based on the inner product of the vectors, and the similarity is compared with a threshold value CORR (“0.7”) to determine whether or not to equalize the characteristics. Yes. Therefore, when the inner product of the vectors and the threshold value are substantially the same, different judgment results may be made under the influence of the bits added to the periphery even in the same image.
[0130]
  Of course, the offset amount as described above is added when it is determined that the characteristics are uniformed, and the offset amount as described above is not added when it is determined that the characteristics are not uniformized. There is a big difference across the threshold.
[0131]
  It is effective to use a continuously changing window function as a method for preventing the result from changing greatly in this way.
[0132]
  x = min (corr_rg, corr_gb, corr_br) (141)
And the window function f (x) is
  When x <0.5, f (x) = 0
  When 0.5 ≦ x ≦ 0.7, f (x) = 5 · x−2.5
  When 0.7 <x, f (x) = 1
And the threshold CORR is set to “0.5”. The change state of the window function f (x) is shown in FIG. 30, and is constant “0” when x <0.5, and from “0” to “1.0” when 0.5 ≦ x ≦ 0.7. Increases linearly to a constant “1.0” when 0.7 <x. Then, the red component offset dR and the blue component offset dB obtained in the equations (7) and (8) are changed to the product of f (x). That is,
  dR = f (x) · (dRmax + dRmed) / 2 (7) ′
  dB = f (x) · (dBmax + dBmed) / 4 (8) ′
And In the flowcharts shown in FIGS. 5 and 6, no processing is performed unless the similarity exceeds the threshold value. When the offset is multiplied by the window function, the flowchart shown in FIG. 31 is executed. . That is, the non-processing is not performed based on the similarity, the offset amount is calculated using the window function in step S205, and the calculated offset amount is used. Of course, as described above, when x <0.7, the window function f (x) suddenly approaches “0”, so that the image near the threshold value is not greatly changed by the processing, and the similarity degree For those having a low value, the offset is almost “0”, and there is no adverse effect. Further, the equations (9) to (14) are also calculated based on the offset amounts dR and dB calculated in this way.
[0133]
  Obviously, the window function f (x) is not limited to the functions described above. In the example described above, the value of the window function is changed based on the minimum value of the correlation coefficient, but it is not necessary to be limited to the minimum value. For example,
  f (corr_rg, corr_gb, corr_br) (142) may be used. And more generally,
  f (statistic)
It is also good. In this case, the statistic includes a minimum value, a maximum value, a median, a standard deviation, and the like.
[0134]
  On the other hand, the window function functions to validate the calculated value in one area and invalidate the calculated value in another area as the window is opened and closed. Now, when color casts are classified according to their causes, there are cases based on the hardware performance of the image input device 10 described above, intentional color casts due to sunset, etc., and color casts caused by special illumination using a tungsten lamp or the like. Can be classified. As described above, it is not necessary to unify the characteristics of intentional color casts due to sunsets, etc. However, in the case of color casts by special lighting using tungsten lamps etc., the characteristics must be unified. It is also meaningful to plan.
[0135]
  Intentional color cast due to sunset or the like and color cast due to special illumination using a tungsten lamp or the like can be determined from the above-described correlation coefficient. That is, photographs with extremely low correlation coefficients are often caused by such special illumination, and the window function is modified as shown in FIG.
[0136]
  That is, the region of x <0.5 is deformed,
  When x <0.1, f (x) = 1.0
  When 0.1 ≦ x <0.3, f (x) = − 5 · x + 1.5
  When 0.3 ≦ x0.5, f (x) = 0
It was. As a result, when x becomes “0.3” or less, the window function f ′ (x) rises linearly again, and in the region where x is “0.1” or less, the window function f ′ (x) is constant “ 1.0 ". Of course, when the window function becomes “1.0”, the large offset amount calculated based on the distribution state of each component is applied as it is, and the influence of illumination is canceled and the characteristics are made uniform.
[0137]
  Note that the window function does not need to change linearly in the transition region, and may be a curve that monotonously increases or monotonously decreases.
[0138]
  Further, the similarity is checked in step S116 as shown in FIG. 5, and if the similarity is low, non-processing is executed in step S106 to set a flag, or step S116 is shown in FIG. Since the offset amount is calculated by using the window function in step S205 without executing the process, the characteristics are not made uniform when the similarity is substantially low. And it can be said that the correction control means is constituted by hardware and the like for realizing the above.
[0139]
  On the other hand, if there is a difference in how the frequency distribution spreads, it is effective to make it uniform. In the present embodiment, the frequency distribution is expanded as much as possible while making the frequency distribution spread, and the contrast is enhanced for each component.
[0140]
  In contrast enhancement, when the gradation range is “0” to “255”, each color component (R1, G1, B1) before conversion, the maximum value Rmax2, Gmax, Bmax2 and the minimum value Rmin2, Gmin of each component. , Bmin2 to obtain each color component (R2, G2, B2) of the conversion destination based on the following equation.
[0141]
  R2 = far x R1 + fbr (15)
  G2 = fag × G1 + fbg (16)
  B2 = fab × B1 + fbb (17)
However,
  far = 255 / (Rmax2-Rmin2) (18)
  fag = 255 / (Gmax−Gmin) (19)
  fav = 255 / (Bmax2-Bmin2) (20)
  fbr = −far × Rmin2 or 255−far × Rmax2 (21)
  fbg = −fag × Gmin or 255−fag × Gmax2 (22)
  fbb = −fab × Bmin2 or 255−fab × Bmax2 (23)
In the above conversion equation, if R2, G2, B2 <0, then R2, G2, B2 = 0, and if R2, G2, B2> 255, then R2, G2, B2 = 255. Here, far, fag, and fab are slopes, and fbr, fbg, and fbb are offsets. According to this conversion equation, as shown in FIG. 22, the luminance distribution having a certain narrow width can be expanded to a reproducible range. Basically, in expanding the luminance distribution range, the number of pixels does not change, so the areas of the histograms coincide. However, when the luminance distribution is expanded by making full use of the reproducible range in this way, the highlight portion may be whitened or the high shadow portion may be blackened. In order to prevent this, the gradation range to be enlarged is limited in this embodiment. That is, only “5” is left in the gradation value as a range that does not expand to the upper end and lower end of the entire gradation range. As a result, the parameters of the conversion equation are as follows:
[0142]
  far = 245 / (Rmax2-Rmin2) (24)
  fag = 245 / (Gmax−Gmin) (25)
  fav = 245 / (Bmax2-Bmin2) (26)
  fbr = 5-far × Rmin2 or 250-far × Rmax2 (27)
  fbg = 5-fag × Gmin or 250-fag × Gmax2 (28)
  fbb = 5-fab × Bmin2 or 250-fab × Bmax2 (29)
In this case, conversion is not performed for the gradation less than “5” and the gradation “250” or more.
[0143]
  In the present embodiment, in order to retain the highlight portion and the high shadow portion, the gradation value is uniformly set from the end of the gradation range and the range of “5” is set as the non-enlarged area. The range may be narrowed if it is an image output device that can relatively easily reproduce the portion and the high shadow portion, and may be enlarged if the reproducibility is weaker. Further, the enlargement rate may be limited gradually in the border area instead of being uniformly enlarged.
[0144]
  FIG. 23 (a) shows a case where the luminance distribution of the image is narrow. If the magnification ratio of the luminance distribution (corresponding to far, fag, fab) is applied as described above, In some cases, a very large enlargement ratio can be obtained in accordance with the reproducible range. Then, in the dusk state like the evening, the range of contrast from the brightest part to the dark part is naturally narrow, but as a result of trying to enlarge the contrast greatly for this image, it is converted like a daytime image. It can be. Since such conversion is not desired, the enlargement ratio is limited so that far, fag, and fab do not exceed 1.5 (˜2). As a result, twilight is expressed as twilight.
[0145]
  A case where there is no restriction on the enlargement ratio is shown by a one-dot chain line in FIG. 23 (a), and no extra portion remains in a reproducible range after conversion. However, when the expansion range is limited, as shown by the two-dot chain line in FIG. 5B, there is a degree of freedom where the distribution after the conversion is brought. It can be too dark or too dark. Therefore, in such a case, the ratio (m1: m2) of the remaining area remaining on the upper end side and the lower end side in the gradation range before conversion remains on the upper end side and the lower end side after conversion. What is necessary is just to convert so that it may correspond with the ratio (n1: n2) of a remaining area | region.
[0146]
  The process for obtaining the parameters far, fag, fab, fbr, fbg, and fbb as described above is executed in step S208, and in the subsequent step S210, a conversion table is created in the same manner as in step S206. The conversion table at this time is input with the converted component values (R 1, G 1, B 1) shown in FIG. 21 as input, and is replaced with the converted component values (R 1, G 1, B 1) in the conversion table. As a result, referring to the conversion table, the two processes of adding the offset amount obtained in step S204 and the contrast enhancement process obtained in step S208 are simultaneously performed.
[0147]
  On the other hand, another element of overall brightness remains as a shift between the color components of the frequency distribution. Therefore, in step S214, γ is calculated in step S212 in order to apply γ correction that makes this brightness shift uniform. For example, the peak of the red component frequency distribution indicated by the solid line in FIG. 24 is closer to the dark side, or the peak of the blue component frequency distribution indicated by the alternate long and short dash line is closer to the bright side. If it is, it may be corrected and moved so that it moves closer to the center as a mountain of the frequency distribution of the green component indicated by the chain line in FIG.
[0148]
  As a result of various experiments, in the present embodiment, the median in the frequency distribution is determined as a reference, and when the median is less than “85”, the image is determined to be a dark image and the γ correction corresponding to the following γ values Brighten with.
[0149]
  γr = Rmed2 / 85 (30)
  γg = Gmed / 85 (31)
  γb = Bmed2 / 85 (32)
Or
  γr = (Rmed2 / 85) ** (1/2) (33)
  γg = (Gmed / 85) ** (1/2) (34)
  γb = (Bmed2 / 85) ** (1/2) (35)
And
[0150]
  In this case, even if γr, γg, γb <0.7, γr, γg, γb = 0.7. This is because if such a limit is not set, the night image becomes like the daytime. Note that if the image is too bright, the overall image becomes whitish and the contrast tends to be low, so that processing such as enhancement of saturation is suitable.
[0151]
  On the other hand, if the median is larger than “128”, the image is judged to be bright and darkened by γ correction corresponding to the following γ values.
[0152]
  γr = Rmed2 / 128 (36)
  γg = Gmed / 128 (37)
  γb = Bmed2 / 128 (38)
Or
  γr = (Rmed2 / 128) ** (1/2) (39)
  γg = (Gmed / 128) ** (1/2) (40)
  γb = (Bmed2 / 128) ** (1/2) (41)
And In this case, even if γr, γg, γb> 1.3, a limit is set so that it is not too dark as γr, γg, γb = 1.3. Note that if the image is too dark, the color will be excessive and the image will become darker, so processing such as reducing the saturation enhancement is suitable. However, this darkening process may adversely affect a subject in a bright background. This is because, for example, a landscape image in which the sky is half of the image or a commemorative photo on a sunny day, the face is often dark and crushed due to backlight. In these images, a dark part and a bright part are mixed, so that the standard deviation of each color component is often a relatively high value. Accordingly, when such a standard deviation is larger than “70”, it is possible not to perform γ correction for darkening. FIG. 25 shows the correspondence when γ correction is performed. If γr, γg, γb <1, the curve swells upward, and if γr, γg, γb> 1, the curve swells downward. Note that the correction of the brightness is not necessarily based on the frequency distribution, and the brightness may be corrected by determining the brightness from other elements.
[0153]
  To correct for γr, γg, and γb determined as described above, the following equation is used.
The gradation values R1, G1, B1 after conversion with respect to the gradation values r0, g0, b0 before conversion are
  R1 = 255 * (r0 / 255) ** γr (42)
  G1 = 255 * (g0 / 255) ** γg (43)
  B1 = 255 * (b0 / 255) ** γb (44)
  This γ correction is also performed on the conversion table shown in FIG. That is, as in the case of contrast enhancement, the correspondence values are calculated by using the converted component values (R1, G1, B1) shown in FIG. 21 as input, and the converted component values (R1, G1, B1) in the conversion table Replace. Thus, referring to the conversion table, brightness correction processing is simultaneously performed in addition to offset amount addition and contrast enhancement processing.
[0154]
  Finally, the image data is converted in step S216, and the converted image data (Rm, Gm, Bm) is referred to the conversion table shown in FIG. 21 for the image data (rm, gm, bm) of all pixels. ) Is repeated.
[0155]
  In the present embodiment, correction based on the offset amount, contrast enhancement, and brightness correction are performed in this order. However, all of these are not necessarily performed, and each correction is performed. The method can also be changed as appropriate.
[0156]
  For example, in the above-described embodiment, in the contrast enhancement process, correction is performed using a conversion equation that forms a linear correspondence with the component value before conversion, but smoother conversion is achieved. In addition, so-called S-shaped curve conversion as shown in FIG. 26 may be performed. In this case, the spread of the frequency distribution is not determined at both end positions, but the concept of standard deviation representing the degree of dispersion of the distribution can be used. Hereinafter, an example in which the contrast is enhanced by the correspondence relationship of the S-shaped curve using the standard deviation will be shown. Note that although conversion is performed based on the frequency distribution for each component, the calculation method is common to all components, and therefore, a procedure for obtaining luminance Y after conversion from luminance y before conversion is shown.
[0157]
  There are two ways of thinking about the standard deviation, but in the present embodiment, calculation is performed based on the following equation.
[0158]
[Equation 5]

[0159]
  yp: luminance of each pixel before conversion
  ym: Average luminance before conversion for each pixel
  The standard deviation corresponds to the spread amount of the luminance distribution, but dispersion may be used to express the spread amount. In addition, since the number of gradations as a whole is 256 as in the present embodiment, the spread amount can be obtained from the kurtosis k of the distribution.
[0160]
[Formula 6]

[0161]
  The kurtosis of k = 3 here corresponds to the amount of spread of the normal distribution.
[0162]
  Based on the standard deviation σ, which is the amount of spread of the luminance distribution thus obtained, contrast enhancement assigns a small number of gradations to a range with a small distribution density while giving a large number of gradations to a range with a large distribution density. In other words, as shown in FIG. 26, when the input / output correspondence is a so-called S-shaped curve, the gradation range RNG1 assigned after conversion becomes larger and assigned to the gradation range rng0 assigned before conversion. The number of tones increased. On the other hand, regarding the range outside the gradation range rng0 on the low luminance side and the high luminance side in the input, the gradation range assigned after conversion is reduced.
[0163]
  In the present embodiment, the center position ymid of the gradation range is set to “128”, and γ1 is given below the center position ymid, and γ correction is performed by giving γ2 in the range larger than the center position ymid. , Γ2 are determined based on the standard deviation σ. That is,
  For y ≦ 128,
  γ1 = (σstd_limit / σ) ** fc (47)
  For y> 128,
  γ2 = (σ / σstd_limit) ** fc (48)
In step S204, these parameter calculations are executed. Here, σstd_limit and fc are parameters obtained experimentally in consideration of the conversion result. In the present embodiment, σstd_limit is set to “128” and fc is set to “0.1”. Since the standard deviation σ is generally smaller than “128”, in these relational expressions, when the standard deviation σ is large, γ2 and γ1 each approach “1”, and the slope of the S-shaped curve becomes gentle. . This means that the gradation range RNG0 of the conversion destination is not so wide with respect to the gradation range rng0 centered at the center position ymid when the spread amount is large. More specifically, the brightness of the image data Means that the conversion for expanding the luminance range is not performed. On the other hand, when the standard deviation σ is small, γ2 and γ1 are separated from “1”, and the slope of the S-curve becomes steep. This means that when the spread amount is small, the conversion target gradation range RNG1 is broadly expanded with respect to the gradation range rng0 centered on the center position ymid. When the luminance is distributed only in a narrow range, it means that conversion for expanding the luminance range is performed.
[0164]
  In the present embodiment, the correspondence relationship between the S-shaped curves is established by γ correction, but in FIG. 27, the gradation “0”, the lower quadrant yq1, the center position ymid, and the upper quadrant yq3 are shown. The lower quadrant yq1 and the upper quadrant are set to Y = y for the gradation “0”, the center position ymid, and the gradation “255”, with the five points of gradation “255” as reference points. A conversion point at the minute point yq3 is determined based on the standard deviation. And the correspondence which connects these five points is calculated | required by spline interpolation calculation or Newton interpolation. Of course, the lower three points and the upper three points from the center position ymid may be obtained by spline interpolation or Newton interpolation, respectively.
[0165]
  That is, as shown in FIG. 28, standard deviations σr, σg, and σb are obtained for each component in step S230, γ1 and γ2 are obtained for each color component in step S232, and γ1 and γ2 are used in step S234. A conversion table is created based on the corresponding relationship. These steps S230 to S234 are executed in place of the processing of steps S208 to S214 in the flowchart shown in FIG.
[0166]
  Next, the operation of the present embodiment having the above configuration will be described step by step.
[0167]
  If a photograph is taken with the scanner 11 or the like, image data representing the photograph as RGB gradation data is taken into the computer 21, and the CPU executes the image processing program shown in FIGS. 5 and 6 to execute the image data. Execute the process to correct the color reproducibility.
[0168]
  First, in step S102, the image data is thinned out within a predetermined error range, and a frequency distribution is obtained for each color component for the selected pixel. The frequency distribution as it is cannot be used. First, in step S104, it is determined whether the image is not a binary image such as black and white. In step S108, it is determined whether the image is a natural image. Except when the image is a binary image or not a natural image, in step S110, it is determined whether the image data has a frame portion. After the frame portion is removed, an unclear region at both ends of the frequency distribution is determined in step S114. Remove. In this state, a feature vector is obtained for the frequency distribution for each color component, and the similarity of the frequency distribution is checked from the inner product of the feature vectors. When the frequency distribution for each color component is not very similar, it is supported that the color balance is intentionally shifted in the original image data, and the process for equalizing the characteristics is not performed. However, if a certain degree of similarity is found in comparison with a predetermined threshold value, it is determined that the color balance has shifted, and the following characteristics are made uniform.
[0169]
  That is, in step S204, the red component offset dR and the blue component offset dB for the green component are obtained using the upper end of the frequency distribution and the median, and in step S206, a conversion table for the final image data conversion is formed. Subsequently, in step S208, a parameter for enhancing the contrast is calculated for each color component while making the contrast uniform. In step S210, the contrast balance for each color component is matched while enhancing the contrast based on the parameter. Create a conversion table for In step S212, a γ correction parameter for making the brightness of each color component uniform is calculated, and a conversion table for performing the γ correction is created.
[0170]
  Finally, in step S216, the image data for all the pixels is converted with reference to the conversion table created as described above.
[0171]
  In this case, the process is divided in step S202 depending on whether or not there is similarity. However, when the flowchart shown in FIG. 31 is executed, the effective value of the offset amount is changed depending on the degree of similarity. In some cases, the characteristics are made uniform or not.
[0172]
  Of course, as described above, such image processing is not performed when the image is not a binary image or a natural image. However, when the image processing according to the present invention is performed, an input device such as a color shift is used in the state of a photograph. Despite the poor image reproducibility of the image data, the color shift, contrast, and brightness are made uniform for each color component, and the contrast is enhanced to make a good image with sharpness extremely easy. It will be obtained.
[0173]
  In the above-described embodiment, some parameters are constant, but the user may be able to select on the computer 21 via a predetermined GUI.
[0174]
  In this manner, the frequency distribution for each color component is obtained for the image data while thinning out in step S102, and it is determined in step 116 whether there is similarity between the frequency distributions. Therefore, it is determined that the characteristics found from the frequency distribution coincide with each other, and image data with poor color reproducibility is obtained by correcting the shift by offset correction, contrast enhancement, or brightness correction in steps S204 to S216. However, this work can be automated while providing a sharp and good image, and even an unskilled person can easily correct the color balance.
[Brief description of the drawings]
FIG. 1 is a block diagram of an image processing system to which an image processing apparatus according to an embodiment of the present invention is applied.
FIG. 2 is a block diagram illustrating a specific hardware configuration example of the image processing apparatus.
FIG. 3 is a schematic block diagram illustrating another application example of the image processing apparatus of the present invention.
FIG. 4 is a schematic block diagram illustrating another application example of the image processing apparatus of the present invention.
FIG. 5 is a flowchart corresponding to frequency distribution detection means and similarity determination means in the image processing apparatus of the present invention.
FIG. 6 is a flowchart corresponding to offset amount correcting means, contrast correcting means, and brightness correcting means in the image processing apparatus of the present invention.
FIG. 7 is a diagram illustrating coordinates in a conversion source image.
FIG. 8 is a diagram showing a sampling period.
FIG. 9 is a diagram illustrating the number of sampling pixels.
FIG. 10 is a diagram illustrating a relationship between a conversion source image and pixels to be sampled.
FIG. 11 is a diagram showing array variables for detecting a frequency distribution.
FIG. 12 is a flowchart of a gray image extraction thinning process.
FIG. 13 is a diagram illustrating a black and white image.
FIG. 14 is a diagram illustrating a luminance distribution of a black and white image.
FIG. 15 is a diagram illustrating a state of a frequency distribution for each color component in the case of a non-natural image and a natural image.
FIG. 16 is a diagram showing an image with a frame portion;
FIG. 17 is a diagram illustrating a frequency distribution of an image having a frame portion.
FIG. 18 is a diagram showing an edge processing of the frequency distribution and an edge obtained by the edge processing.
FIG. 19 is a diagram illustrating an element extraction method for obtaining a feature vector for each color component;
FIG. 20 is a diagram illustrating a linear relationship that does not require correction of color reproducibility.
FIG. 21 is a diagram showing a conversion table when image data is converted based on a frequency distribution.
FIG. 22 is a diagram illustrating an expansion of a frequency distribution and a range of all gradations.
FIG. 23 is a diagram illustrating a case where a restriction is imposed on the contrast enlargement ratio.
FIG. 24 is a diagram showing a frequency distribution that requires uniform brightness.
FIG. 25 is a diagram showing a conversion relationship changed by γ correction.
FIG. 26 is a diagram showing a conversion relationship in which contrast is enhanced by a correspondence relationship of S-curves.
FIG. 27 is a diagram showing a conversion relationship when connecting specified conversion points by an interpolation method;
FIG. 28 is a partial flowchart in the case of correcting contrast and brightness with an S-curve.
FIG. 29 is a diagram illustrating a state in which the image processing program is transferred from the medium on which the image processing program is recorded to the hard disk.
FIG. 30 is a graph showing a change state of a window function to be used.
FIG. 31 is a flowchart of an image processing program when an offset amount is adjusted using a window function.
FIG. 32 is a graph showing changes in other window functions to be used.
[Explanation of symbols]
10. Image input device
11 ... Scanner
11b ... Scanner
12 ... Digital still camera
12a ... Digital still camera
12b ... Digital still camera
13b Modem
20 Image processing apparatus
21 ... Computer
22 ... Hard disk
30. Image output device
31 ... Printer
31a ... Printer
31b ... Printer
32 ... Display
32a ... Display
41 ... Flexible disk
42 ... CD-ROM
43 ... Hard disk
44 ... ROM
45 ... RAM
46a ... Communication line
46b Modem
46c: File server

Claims (15)

For image data in which an image is represented as a set of a plurality of pixels by gradation color specification data composed of roughly equal color components, by inputting each component value of the image data and performing a predetermined conversion process, An image processing apparatus that performs conversion to correct a color balance in relation to input and output,
Rutotomoni seek similar degree of distribution of the gradation table color data for each color component, when the above modification, calculating the effective value of the order to uniform the characteristics of the color components is continuously changed in accordance with the similarity degree An image processing apparatus characterized in that correction is performed in accordance with the similarity by using the effective value after the calculation . The image processing apparatus according to claim 1, wherein the gradation range that the gradation color specification data can take is divided into a plurality of regions, and the degree of similarity between the color components is determined by comparing the distribution of each region. A featured image processing apparatus. 3. The image processing apparatus according to claim 1, wherein the degree of similarity is determined based on an inner product of vectors having the distribution of each color component as an element. The image processing apparatus according to any one of claims 1 to 3 , wherein pixels whose gradation colorimetric data of each color component are approximated are targeted for the distribution. The image processing apparatus according to any one of claims 1 to 4 , wherein a characteristic is determined from a predetermined position in the distribution, and the effective value corresponding to a deviation between the components is set so that the characteristic is uniform. An image processing apparatus characterized in that each component value is corrected by obtaining an offset amount. 6. The image processing apparatus according to claim 5 , wherein an end position of the distribution range is determined as a characteristic of the distribution. 7. The image processing apparatus according to claim 5 , wherein an approximate center position of the distribution is determined as a characteristic of the distribution. 8. The image processing apparatus according to claim 1 , wherein the distribution spread for each color component is substantially uniform. 9. The image processing apparatus according to claim 8 , wherein both ends of the spread of the distribution are expanded within an effective gradation range. 9. The image processing apparatus according to claim 8 , wherein a small number of gradations is assigned to a range with a small distribution density while a large number of gradations is given to a range with a large distribution density based on a spread amount of the distribution. An image processing apparatus. 11. The image processing apparatus according to claim 1 , wherein the brightness based on the distribution of each component is made uniform. 12. The image processing apparatus according to claim 11 , wherein brightness and darkness of the image is determined by comparing the approximate center position of the distribution with a predetermined gradation. 13. The image processing apparatus according to claim 11 , wherein the brightness of the image is made uniform by γ correction for each component based on the determination result of the brightness of the image. . For image data in which an image is represented as a set of a plurality of pixels by gradation color specification data composed of roughly equal color components, by inputting each component value of the image data and performing a predetermined conversion process, an image processing method for converting to correct the color balance in relation to the input and the output, Rutotomoni seek similar degree of distribution of the gradation table color data for each color component, is when the modified characteristics of each color component An image is obtained by performing an operation for continuously changing an effective value for equalization of the image according to the similarity, and performing correction according to the similarity by using the effective value after the operation. Processing method. For image data in which an image is represented as a set of a plurality of pixels by gradation color specification data composed of roughly equal color components, by inputting each component value of the image data and performing a predetermined conversion process, a medium recording an image processing program for performing the conversion to correct the color balance to a computer in relation to the input and the output, Rutotomoni seek similar degree of distribution of the gradation table color data for each color component, the modified In performing the calculation, the effective value for uniformizing the characteristics of the respective color components is continuously changed according to the above-mentioned similarity, and the correction according to the above-mentioned similarity is performed by using the effective value after the calculation. medium recording an image processing program, characterized in that to perform.

Images