JP3702957B2 - Image processing apparatus, image processing method, and medium on which image processing control program is recorded - Google Patents

Description

なる演算式に基づいて積算する。この（１３）式において、「１３６」とは重み付け係数の合計値であり、サイズの異なる平滑化フィルタにおいては、それぞれ升目の合計値となる。すなわち、５×５画素であれば「４００」となるし、７×７画素であれば「９００」となる。また、Ｍｉｊはアンシャープマスクの升目に記載されている重み付け係数であり、Ｙ（ｘ，ｙ）は各画素における輝度成分である。さらに、ｉｊについてはアンシャープマスクに対して横列と縦列の座標値で示している。
【００８９】
Ｙunsharp （ｘ，ｙ）は色にじみ画素に対して周辺画素の重み付けを低くして加算したものであるから、この場合も同様にローパスフィルタをかけたものと同じ意味あいを持つ。従って、「Ｙ（ｘ，ｙ）−Ｙunsharp （ｘ，ｙ）」とは本来の全成分から低周波成分を引いたことになってハイパスフィルタをかけたものと同様の意味あいを持つ。そして、ハイパスフィルタを通過したこの高周波成分を「Ｙ（ｘ，ｙ）」に加えれば高周波成分を増したことになり、エッジが強調される結果となって画像のシャープさが向上する。
【００９０】
なお、これらのアンシャープマスクは図１８からも明らかなように、中央部にて最も重み付けが大きく、周縁に向かうにつれて徐々に重み付け係数が小さくなっている。従って、周縁側の重み付け係数が升目の合計値に与える影響はわずかであるといえる。一方、（１３）式等で示されるマトリクス演算は、処理対象画素の周囲の画素に対して、採用するアンシャープマスクの升目数だけ乗算演算と加算演算が必要になることから演算量が多大となることも観念される。そこで、本実施形態においては、予め周縁側の重み付け係数を省いて構成した３×３、５×５、７×７画素のアンシャープマスクを使用することにより演算量を削減して処理速度を高めるようにしている。
【００９１】
ところで、本実施形態においては、平滑化フィルタのサイズとアンシャープマスクのサイズを同一としているが、これは次のような理由による。例えば、図１９で示すビットマップ画像に図中斜線部分で示す領域に対して平滑化処理を行い、図中波線部分で示す領域に対してアンシャープマスク処理を行って輝度を強調したものとする。上述したように輝度を強調する意味は、平滑化処理されて失われたシャープさを補償するものであるから、平滑化領域の外側まで輝度を強調してしまうと、この外側の領域において不自然にシャープさが向上してしまうことが発生し得る。このため、本実施形態では平滑化フィルタとアンシャープマスクを同一サイズとしているが、むろん、前者に比べて後者のサイズが小さくなるようにしてもかまわない。
【００９２】
ステップＳ１６０では上記のようなエッジ強調処理を行うとともに、強調後の輝度Ｙ'と処理後の色差成分を用いつつ（６）〜（８）式に基づいてＲ'，Ｇ'，Ｂ'の階調データを得る。すなわち、ステップＳ１５２，Ｓ１５４にて色にじみを低減するとともに、ステップＳ１６０にて階調データを生成しており、これらを実行するハードウェア構成とソフトウェアとによって画像処理手段を構成することになる。
【００９３】
なお、ステップＳ１６０において、強調した輝度Ｙ'を使用することからＲ'，Ｇ'，Ｂ'の階調データが負の値となったり、「２５５」を越えるような値となることがある。しかし、階調幅としては、「０」〜「２５５」の範囲であるため、負の階調データは一律に「０」とし、「２５５」を越える階調データは一律に「２５５」とする。そして、次のステップＳ１７０にて処理対象画素を移動させ、ステップＳ１８０にて全画素について終了したと判断するまで同様の処理を繰り返す。
【００９４】
ところで、色にじみ画素を中心として平滑化フィルタを適用し、その後にエッジ強調処理を行えば、平滑化処理によって色にじみが低減され、エッジ強調処理によって平滑化処理で失われた画像のシャープさが補償される。従って、本来的には色にじみ画素がエッジ画素であるか否かを問わず、全ての色にじみ画素に対して一律にかかる処理を実行すればよく、このような構成としても効果を得ることができる。しかし、かかる構成においては、経験的な見地からエッジ部分がぼやけてしまうことがあったため、色にじみ画素がエッジ画素である場合にメジアンフィルタを適用したところ良好な結果を得ることができた。
【００９５】
ここで、本実施形態の平滑化処理およびエッジ強調処理の効果について、簡単な一次元のシミュレーションモデルを例示しつつ説明する。
図２０は、エッジ強調処理を行わない場合の一次元シミュレーションモデルにおいて各種のパラメータの値を示す表である。同図において、左端のＲ，Ｇ，ＢのマトリクスはＣＣＤの色フィルタ配列を示している。このマトリクスの右隣に示す「ＩＮ」は光の入射状況を示しており、「１（階調データは２５５）」ならば光があたっている、すなわち「白」を意味し、「０（階調データは０）」ならば「黒」を意味する。そして、ＣＣＤの各画素において、直接得ることができない色信号は隣接する画素の色信号から線形補間演算により得ているものとする。
【００９６】
ここで、本来二次元の配列で考察するべきではあるが、説明を簡略化するため、上記Ｒ，Ｇ，Ｂのマトリクスの中央縦列に着目する。この意味において、一次元のシミュレーションモデルという。すると、上述した原理から白・黒の境界付近にそれぞれＧ（Ｂ）＝０．５、Ｒ＝０．５というように偽の色信号が発生し、この偽の色信号が発生した画素が色にじみ画素として検出される。色にじみ画素が検出されると、（６）〜（８）式に従って色差成分Ｃ１およびＣ２が算出され、この色差成分Ｃ１，Ｃ２が平滑化処理されて色差成分Ｃ１'，Ｃ２'が得られる。なお、ここでは一次元の平滑化フィルタとして「1,1,1,1,1,1,1」を適用している。すなわち、処理対象画素の前後それぞれの三画素の画素値を合計して「７」で除算する。そして、得られた色差信号Ｃ１'，Ｃ２'を（６）〜（８）式に基づいて元の色信号に戻し、平滑処理後のＲ'，Ｇ'，Ｂ'が得られる。図からも明らかなように白・黒の境界付近の画素における（Ｒ'，Ｇ'，Ｂ'）はそれぞれ（０．７５，０．６１，０．６１）と、（０．２５，０．１１，０．１１）となり、本来の「ＩＮ」の値に近づいて色にじみが低減されたことがわかる。
【００９７】
また、図２１は、平滑化処理を行うとともに、平滑化フィルタと同一サイズのアンシャープマスクを用いてエッジ強調処理を行った場合の一次元シミュレーションモデルを同様に示している。この場合も色差成分Ｃ１，Ｃ２を求めて平滑化処理を行うことは上述したものと相違はないが、（６）〜（８）式を用いて元の色信号に戻す際に、「1,6,18,25,18,6,1」の重み付けを持たせた一次元のアンシャープマスクを利用して強調処理した輝度信号Ｙ'を使用する。すると、白・黒の境界付近の画素における（Ｒ'，Ｇ'，Ｂ'）はそれぞれ（０．８１，０．６７，０．６７）と、（０．１０，−０．０４，−０．０４）となる。むろん、ＲＧＢの信号値としてとり得る値は「０」〜「１」であるからから実際には後者のデータは（０．１０，０，０）となり、図２０に示すものに比べて色にじみがさらに解消されたことがわかる。
【００９８】
さらに、図２２は、平滑化フィルタよりも小さいサイズのアンシャープマスクを用いてエッジ強調処理を行った場合の一次元シミュレーションモデルを同様に示している。この場合、一次元の「1,11,25,11,1」という平滑化フィルタのサイズ（７桁）よりも小さい５桁のアンシャープマスクを使用している。そして、上述したものと同様にして演算を行うと、白・黒の境界付近の画素における（Ｒ'，Ｇ'，Ｂ'）はそれぞれ（０．７９，０．６５，０．６５）と、（０．１０，−０．０４，−０．０４）となる。このように、図２１に示すものに比べれば、色にじみの解消程度は劣るものの、図２０に示すものに比べれば色にじみがさらに解消されていることがわかる。
【００９９】
次に、上記構成からなる本実施形態の動作を説明する。
単板式のＣＣＤを有するデジタルスチルカメラ１２で撮像した実画像をコンピュータ２１に取り込んで、プリンタ３１にて印刷する場合を想定する。すると、まず、コンピュータ２１にてオペレーティングシステム２１ａが稼働しているもとで、画像処理アプリケーション２１ｄを起動させ、デジタルスチルカメラ１２から画像データを取り込む。画像データが同オペレーティングシステム２１ａを介して画像処理アプリケーション２１ｄに取り込まれたら、ステップＳ１１０にて（１）式に基づいて画像のサイズを判定するための指標値を取得するとともに、この指標値と所定のしきい値とを比較し、ステップＳ１２２〜ステップＳ１２６にて使用する平滑化フィルタおよびアンシャープマスクのサイズを決定してワークエリアに保存する。
【０１００】
そして、ステップＳ１３０にて処理対象画素を初期位置に設定し、（２）式に従って当該画素とその周辺画素におけるΔＲＢの値を算出する。その後、ステップＳ１４０にて（５）式に基づいて処理対象画素とその周辺画素との間で同ΔＲＢの値の変化度合いを調べ、所定のしきい値Ｔｈと比較して同変化度合いが大きい場合に色にじみ画素と判断する。
【０１０１】
色にじみ画素と判断されたら、ステップＳ１５０にて（６）〜（８）式に基づいて処理対象画素およびその周辺画素について色差成分Ｃ１，Ｃ２を算出し、（９）および（１０）式、あるいは（１１）式などの判定基準に従って当該色にじみ画素がエッジ画素であるか否かを判断する。ここで、エッジ画素ではないものと判断した場合には、ステップＳ１５２でステップＳ１２２〜ステップＳ１２６にてワークエリアに保存しておいたフィルタサイズに等しい平滑化フィルタを作用させる。すなわち、色差成分Ｃ１，Ｃ２のそれぞれにつき、平滑化フィルタの升目に対応する画素の色差成分を合計して升目数で除算することにより平滑化処理された色差成分Ｃ１'，Ｃ２'を得る。
【０１０２】
一方、ステップＳ１５０にて当該色にじみ画素がエッジ画素であると判断した場合には、ステップＳ１５４で同色にじみ画素を中心とした５×５画素のメジアンフィルタを作用させる。すなわち、色にじみ画素を中心とした全二十五画素の色差成分の値を昇順または降順にソートし、中央の値を選択して色にじみ画素の色差成分と置き換える。このとき、ステップＳ１５０にて（９）および（１０）式の判定基準を採用した場合には、色差成分Ｃ１，Ｃ２のそれぞれにて同式を充足する色差成分についてステップＳ１５４でメジアンフィルタを適用し、同式を充足しない色差成分についてはステップＳ１５２で上記の平滑化フィルタを適用する。むろん、ここにおいて双方の色差成分について同式を充足する場合にはステップＳ１５４で色差成分Ｃ１，Ｃ２の双方についてメジアンフィルタを適用する。他方、ステップＳ１５０にて（１１）式の判定基準を採用する場合には、ステップＳ１５４で色差成分Ｃ１，Ｃ２の双方に対してメジアンフィルタを適用することになる。
【０１０３】
その後、ステップＳ１６０にてステップＳ１２２〜Ｓ１２６で決定したサイズのアンシャープマスクを使用して処理対象画素およびその周辺画素について輝度成分Ｙの低周波成分Ｙunsharp を求めるとともに、元の輝度成分Ｙから同低周波成分Ｙunsharp を減算することにより高周波成分を求め、同元の輝度成分Ｙに当該高周波成分を加算して強調後の輝度成分Ｙ'を得る。具体的には、（１３）式などに基づいて演算を行うことになる。そして、この強調後の輝度成分Ｙ'と処理後の色差成分を用いつつ、（６）〜（８）式に基づいて平滑処理後のＲ'，Ｇ'，Ｂ'の階調データを得る。以上の処理をステップＳ１７０にて処理対象画素を移動させながらステップＳ１８０にて全画素について実行したと判断するまで繰り返す。
【０１０４】
全画素について実行し終えたら、画像処理された画像データをディスプレイドライバ２１ｃを介してディスプレイ３２に表示し、良好であればプリンタドライバ２１ｂを介してプリンタ３１にて印刷させる。すなわち、同プリンタドライバ２１ｂは色にじみが低減されたＲＧＢの階調データを入力し、所定の解像度変換を経てプリンタ３１の印字ヘッド領域に対応したラスタライズを行なうとともに、ラスタライズデータをＲＧＢからＣＭＹＫへ色変換し、その後でＣＭＹＫの階調データから二値データへ変換してプリンタ３１へ出力する。
【０１０５】
以上の処理により、デジタルスチルカメラ１２から取り込んだ実画像データは自動的に色にじみが発生している部分のみ色にじみを低減するように画像処理されてディスプレイ３２に表示された後、プリンタ３１にて印刷される。すなわち、色にじみが発生している部分のみ色にじみを低減するように画像処理するため、演算量を削減して高速な画像処理を実現することができる。
【０１０６】
このように、画像処理の中枢をなすコンピュータ２１はステップＳ１３０，Ｓ１４０にて低密度の要素色強度に基づいて処理対象画素が色にじみ画素か否かを判定するとともに、色にじみ画素と判定した場合にステップＳ１５０にてエッジ画素であるか否かを判断し、エッジ画素でない場合にはステップＳ１５２にて平滑化フィルタを作用させ、他方、エッジ画素である場合にはステップＳ１５４にてメジアンフィルタを作用させることにより、色にじみを低減すべく画像処理するようにしたため、演算量を削減して高速な画像処理を実行することができる。
【図面の簡単な説明】
【図１】 本発明の一実施形態にかかる画像処理装置を適用した画像処理システムのブロック図である。
【図２】 同画像処理装置の具体的ハードウェアのブロック図である。
【図３】 デジタルスチルカメラの簡単なハードウェア構成例を示すブロック図である。
【図４】 本発明の画像処理装置の他の適用例を示す概略ブロック図である。
【図５】 本発明の画像処理装置の他の適用例を示す概略ブロック図である。
【図６】 本発明の画像処理装置における色にじみ画素の判定と画像処理部分を示すフローチャートである。
【図７】 画像サイズと平滑化領域との関係を示す図である。
【図８】 ３×３、５×５および７×７画素の平滑化フィルタである。
【図９】 同一画素数の画像データにおいてｈｅｉｇｈｔとｗｉｄｔｈの関係を示す図である。
【図１０】 処理対象画素を移動させていく状態を示す図である。
【図１１】 要素色成分の変化度合いを斜め方向の隣接画素における差分値で求める場合の説明図である。
【図１２】 画像の変化度合いを斜め方向の隣接画素における差分値で求める場合の説明図である。
【図１３】 隣接する全画素間で画像の変化度合いを求める場合の説明図である。
【図１４】 隣接する画素間で画像の変化度合いを求める変形例の説明図である。
【図１５】 隣接する画素間で画像の変化度合いを求める別の変形例の説明図である。
【図１６】 色にじみ画素を含むエッジ部分の画像データにおいて各画素の色差成分値の一例を示す図である。
【図１７】 メジアンフィルタによる演算処理の一例を説明するための図である。
【図１８】 ３×３、５×５および７×７画素のアンシャープマスクである。
【図１９】 平滑化領域とエッジ強調領域との関係を示す図である。
【図２０】 エッジ強調処理を行わない場合の一次元シミュレーションモデルにおける各種パラメータの値を示す表である。
【図２１】 平滑化フィルタと同一サイズのアンシャープマスクを用いてエッジ強調処理を行った場合の一次元シミュレーションモデルにおける各種パラメータの値を示す表である。
【図２２】 平滑化フィルタよりも小さいサイズのアンシャープマスクを用いてエッジ強調処理を行った場合の一次元シミュレーションモデルにおける各種パラメータの値を示す表である。
【図２３】 色にじみの発生原理を説明するための図である。
【符号の説明】
１０…画像入力装置
２０…画像処理装置
２１…コンピュータ
２１ａ…オペレーティングシステム
２１ｂ…プリンタドライバ
２１ｃ…ディスプレイドライバ
２１ｄ…画像処理アプリケーション
２２…ハードディスク
２３…キーボード
２４…ＣＤ−ＲＯＭドライブ
２５…フロッピーディスクドライブ
２６…モデム
３０…画像出力装置[0001]
BACKGROUND OF THE INVENTION
  Image data captured using a single-plate solid-state image sensoretcThe present invention relates to an image processing apparatus, an image processing method, and a medium on which an image processing control program is recorded.
[0002]
[Prior art]
  In a digital still camera or the like using a solid-state image sensor, a single plate method is often adopted. In this single-plate method, as shown in FIG. 23A, the color filters of R, G, B (red, green, blue) corresponding to each pixel of the solid-state image sensor have a mosaic shape at a predetermined ratio. In particular, the G color filter is arranged in a staggered manner, and the component ratio is high. Therefore, in each pixel of the solid-state imaging device, only one color signal of R, G, B can be obtained, and a color signal of each color cannot be obtained. Therefore, a color signal that cannot be obtained directly in each pixel is interpolated from the color signals of adjacent pixels to obtain color signals of all R, G, and B colors, converted into gradation data, and output. It is displayed on the display based on the gradation data.
[0003]
  For example, in FIG. 23A, attention is focused on the middle color filter indicated by an arrow (→). Then, as shown in FIG. 5B, it is assumed that the left half of the center is exposed to light (white portion) and the right half of the center is not exposed to light (black portion). Here, assuming that the color signal level of each color in a state where light is shining is “1” and the same color signal level in a state where no light is shining is “0”, the color signal levels of R and G are essentially the same. The value should be as shown in FIG.
[0004]
  However, as described above, the G color signal cannot be obtained directly from the R color filter, and the R color signal cannot be obtained directly from the G color filter. The G color signal of the corresponding pixel is obtained by linear interpolation of the G color signal in the adjacent pixel. On the other hand, the R color signal of the pixel corresponding to the G color filter is obtained by linear interpolation of the R color signal of the adjacent pixel. Then, the G and R color signal levels in each pixel have values shown in FIGS. 23 (d) and 23 (e), respectively. As is apparent from the figure, a false color signal is generated in the pixel near the boundary between the area that is exposed to light and the area that is not exposed to light, and this false color signal causes color blur on the image.(Sometimes called false color)Will occur. In particular, it is known that this color blur appears prominently at the boundary between gray and white. Therefore, in order to reduce such color blur, a smoothing filter (low-pass filter) is applied to the color difference data of all the pixels constituting the image data to make the color blur inconspicuous.
[0005]
[Problems to be solved by the invention]
  The conventional techniques described above have the following problems.
  Applying a smoothing filter corresponds to distributing the color component of one pixel to surrounding pixels, and performs a matrix operation such as 5 × 5 pixels around the pixel. For example, when performing a matrix operation of 5 × 5 pixels, 5 × 5 = 25 operations are performed per pixel. Therefore, when all the pixels are targeted, the amount of calculation is 25 × the number of pixels, and the processing time may be long.
  On the other hand, when the above smoothing process is performed, the image tends to be blurred. In particular, when a color blur pixel is generated at an edge portion that is a boundary portion of an image, the image of the edge portion may be blurred.
[0006]
  The present invention has been made in view of the above problems,In reducing the color blur in the image data, the image of the edge portion is prevented from being blurred, andAn object of the present invention is to provide an image processing apparatus capable of shortening the processing time, an image processing method, and a medium recording an image processing control program.
[0007]
[Means for Solving the Problems]
  In order to achieve the above object, the invention according to claim 1Complementary element color gradation value data lacking in each pixel of the original image data obtained by imaging with a single-plate solid-state image sensor in which color filters of multiple element colors are arranged in a mosaic pattern An image processing apparatus that performs image processing on generated image data including a plurality of pixels,Color blur pixel detecting means for detecting a color blur pixel included in image data, means for determining whether or not the detected color blur pixel is an edge pixel, and changing the calculation method according to the determination result Image processing means for executing image processing for reducing color blur.
[0008]
  In the invention according to claim 1 configured as described above,When the color blur pixel detection means detects a color blur pixel included in the image data, it determines whether the detected color blur pixel is an edge pixel, and the image processing means changes the calculation method according to the determination result. Image processing to reduce color blurTo do.
[0009]
  As a more specific configuration, the invention according to claim 2 is the image processing apparatus according to claim 1, wherein the image processing unit is configured to display the same color bleeding pixel when the color bleeding pixel is an edge pixel. Replaces the median value of color difference components for pixels in a predetermined neighborhood range as a reference with the color difference components of the color blur pixels, and smoothes the color difference components for pixels in the neighborhood region if they are not edge pixels. The smoothed and smoothed chrominance component is replaced with the chrominance component of the same color blur pixelIt is as composition to do.
[0010]
  The sharpness lost by the smoothing process can be compensated by performing the above-described smoothing process on a predetermined range of pixels based on the color blur pixel and then performing the above-described edge enhancement process. However, from an empirical point of view, when a color blur pixel is generated at an edge portion that is a boundary portion of an image, the image of the edge portion may be blurred.
[0011]
  That is, when a color blur pixel is detected, it is determined whether or not the color blur pixel is an edge pixel. Here, when it is determined that the pixel is an edge pixel, the image processing unit detects a color difference component for a predetermined range of pixels with reference to the color blur pixel as described above, and among them, the center of the color difference component is detected. The color difference component having a value is replaced with the color difference component of the pixel that blurs the color, and then returned to the original element color component. On the other hand, if it is determined that the pixel is not an edge pixel, the color difference component of the pixels in a predetermined range based on the color blur pixel is smoothed as described above, and then returned to the original element color component.
[0012]
  For example, if a color bleed pixel occurs at the boundary between white and black, the color bleed pixel is eliminated because the chrominance component of the color bleed pixel is replaced with the color difference component of the surrounding pixels to become a pure white or black pixel. However, the boundary is not blurred. Further, in determining whether or not the pixel is an edge pixel, since the degree of change of the color difference component between adjacent pixels in the edge pixel increases, the degree of change of the color difference component is compared with a predetermined threshold value, What is necessary is just to judge that it is an edge pixel, when the change degree of the same color difference component is larger. Of course, the edge pixel may be regarded as a pixel having a large luminance gradient, and the edge pixel may be determined based on the degree of change in the luminance component between adjacent pixels.
[0013]
  Note that a specific median filter or the like may be applied as a specific configuration for selecting the central value in the color difference component for a predetermined range of pixels.
[0014]
  On the other hand, various methods can be adopted for detecting a color blur pixel. As an example, the invention according to claim 3 is the image processing apparatus according to claim 1 or 2, The color blur pixel detection means obtains difference data of gradation values between different element colors for each pixel, and when the degree of change of the difference data between pixels adjacent to each other is larger than a predetermined threshold value Color blur pixelsAs a configuration to detect.
0015]
  Claim 3 configured as described above.In the invention according toThe color blur pixel detecting means obtains difference data of gradation values between different element colors for each pixel, and the color when the change degree of the difference data between adjacent pixels is larger than a predetermined threshold value. A blur pixel is detected.
0016]
  For example, image data subject to image processing isConsists of dot-matrix pixels imaged with a single-plate solid-state image sensorCaseIn a single-chip solid-state imaging device, color filters of a plurality of element colors are arranged in a mosaic pattern with a non-uniform density, so that they are supplemented by calculation so as to obtain a uniform density. In this case, a false color component is generated and color bleeding occurs. The color bleed pixel detecting means detects a color bleed pixel in which such color bleed has occurred, and the image processing means performs color bleed for a predetermined range of pixels based on the detected color bleed pixel. Image processing is performed to reduce the image.
0017]
  When detecting the presence or absence of color blur for a certain pixel, it is not possible to detect the presence or absence of color blur from the element color components of that pixel in a uniform manner. become. In addition, the occurrence of color blur is caused by the false element color component as described above, and the false element color component is because the color filters of each element color are arranged at a non-uniform density in the solid-state imaging device. It can be said that the element color components for the low density color filter are likely to occur.
0018]
  there,Paying attention to the element color intensity (gradation value, etc.) for the low-density color filter between the pixel to be detected and the pixel adjacent to the pixel, the degree of change in this element color intensityBased on the above, a color blur pixel can be detected.
0019]
  Here, as an example of a specific method for detecting a color blur pixel,4The present invention relates to the above-mentioned claim.3In the image processing apparatus described in the above item, the color blur pixel detection unit includes:Configuration for obtaining difference data between low-density element colors and other element colors in the original image data when obtaining difference data of gradation values between different element colors for each pixelIt is as.
0020]
  Claims configured as above4In the present invention, the color blur pixel detecting means isFind difference data between the low-density element color and other element colors in the original image data,Degree of change of the difference dataOn the basis of theColor blur pixels are detected.
0021]
  For example, when it is known in advance that the R and G color filters are alternately arranged as shown in the middle line of FIG. 23A, the low-density color filter is R. G is adopted as a component, and the degree of change in the difference ΔRG = | RG | between the two is examined between adjacent pixels. Then, the value of ΔRG is as shown in FIG. 23 (f). When attention is paid to the degree of change in ΔRG between adjacent pixels, “0” is obtained in the pixels near the boundary between the area where light is applied and the area where light is not applied. .5 ", and such a pixel is detected as a color blur pixel.
0022]
  It is not necessary to be limited to the above-described method for detecting a color blur pixel based on the change degree of the difference in element color intensity.5The invention according to claim4In the image processing apparatus described in the item 1, the color bleed pixel detecting means has a low density.Element colorWhen there are multiple, low density between adjacent pixelsElement colorAgainstTone value dataThe color blur pixel is detected based on the degree of change in the difference.
0023]
  Claims configured as above5In the invention according to the present invention, there are a plurality of low density color filters.If you want toThe color blur pixel detecting means isFind the difference in gradation value data for each low density element color between adjacent pixels,The color blur pixel is detected based on the change degree of the difference.
0024]
  That is, as described above, since the false element color component appears conspicuously for the element color for the low density color filter, attention is paid to the difference in element color intensity for the low density color filter. For example, as shown in FIG. 23A, when the density of the G color filter is high and the density of the R and B color filters is low as a whole, false element color components appear prominently in R and B. It will be. Here, similarly, it is assumed that the middle line is focused, and the B color signal level in each pixel of the middle line is linearly interpolated from the B color filter adjacent to the upper line side. Then, the B color signal level in each pixel in the middle stage line is the same as that shown in FIG. 4D, and the difference between the R and B color signal levels is ΔRB = | RB− = ΔRG. Accordingly, since the value of ΔRB between adjacent pixels is also as shown in FIG. 5F, the degree of change of the same ΔRB is “0.5” in the pixels near the boundary between the area that is exposed to light and the area that is not exposed to light. Thus, such a pixel is detected as a color blur pixel.
0025]
  Regardless of whether the focus is on the degree of change in the difference between the reference element color intensity and the low-density element color intensity or on the degree of change in the difference between the low-density element color intensities, Various modes can be considered for checking the degree of change. For example, the degree of change may be examined between pixels adjacent to the eight directions of the horizontal direction, the vertical direction, and the diagonal direction, and the amount of calculation can be reduced by appropriately reducing the comparison direction. For the reasons described above, it can be said that there is a high possibility that a false element color component is generated in a pixel for a low-density color filter. Is preferred. Therefore, the claim6According to another aspect of the invention, in the image processing device according to any one of claims 3 to 5, the color blur pixel detection unit is adjacent to the image processing device.Among low-density pixels among pixelsThe color blur pixel is detected based on the degree of change.
0026]
  Claims configured as above6In the invention according to the above aspect, the color bleed pixel detecting means has the adjacent low density.Element colorCorresponding pixelBetweenA color blur pixel is detected by checking the degree of change.
0027]
  Further, since the sharpness of the image may be lost due to the smoothing process, the image processing means is configured to perform an edge enhancement process.May be.
0028]
  Configured as aboveIfThe image processing means performs edge enhancement processing on the image data to compensate for the sharpness lost in the smoothing processing.
0029]
  In this way, even if the edge enhancement process is performed, it is considered that if the edge enhancement is performed to the area where the smoothing process is not performed, the outside of the area subjected to the smoothing process is unnaturally edge enhanced. The image processing means may be configured to perform edge enhancement processing on pixels within a range where the smoothing processing is performed.
0030]
  Configured as aboveIn this case, the image processing meansEdge enhancement processing is performed on the pixels within the range where the above-described smoothing processing that tends to lose sharpness is performed.
0031]
  By the way, in performing the smoothing process, it is not a good idea to uniformly perform the smoothing process in the same range on all the image data. For example, it is assumed that smoothing processing in the same range is performed on both image data having a large image size and image data having a small image size. In this case, since the ratio of the smoothing process in the former to the image is different from the ratio in the latter, even if good results are obtained in the former, the whole may be too blurry in the latter. obtain. Therefore, the image processing means performs a smoothing process on a predetermined range of pixels based on the detected color blur pixels, and performs the smoothing process when the size of the image to be processed is large. And when the size of the same image is small,May be.
0032]
  When configured as described above, the image processing means detects the size of image data to be subjected to image processing in advance, and if the size of the image data is large, the range for smoothing processing is increased. If the size is small, the smoothing range is reduced. Specifically, a plurality of smoothing filters having different sizes are held, and the smoothing filter is changed according to the image size. This can be achieved by using different types of ruta.is there.
[0033]
  The method of changing the calculation method depending on whether or not the color blur pixel is an edge pixel is not necessarily limited to a substantial device, and as an example, the invention according to claim 7 includes:Complementary element color gradation value data lacking in each pixel of the original image data obtained by imaging with a single-plate solid-state image sensor in which color filters of multiple element colors are arranged in a mosaic pattern An image processing method for performing image processing on generated image data consisting of a plurality of pixels,Image processing is performed to detect a color blur pixel included in image data, determine whether the detected color blur pixel is an edge pixel, and change the calculation method according to the determination result to reduce color blur. It is configured to execute.
0034]
  That is, it is not necessarily limited to a substantial apparatus, and there is no difference that the method is also effective.
0035]
  By the way, as described above, there may be an image processing apparatus that detects a color blur pixel and reduces color blur for a predetermined range of pixels based on the color blur pixel, or is incorporated in a certain device. The idea of the invention includes various aspects such as being used in a state. Further, it can be changed as appropriate, for example, by hardware or by software.
[0036]
  In the case of software for controlling an image processing apparatus as an embodiment of the idea of the invention, it naturally exists on a recording medium on which such software is recorded, and it must be used. As an example, the invention according to claim 8 is:Complementary element color gradation value data lacking in each pixel of the original image data obtained by imaging with a single-plate solid-state image sensor in which color filters of multiple element colors are arranged in a mosaic pattern A medium on which an image processing control program for performing predetermined image processing on a computer with respect to generated image data consisting of a plurality of pixels is recorded,A function to detect color blur pixels included in image data, a function to determine whether the detected color blur pixels are edge pixels, and to reduce color blur by changing the calculation method according to the determination result And a function for executing the image processing to be performed.
0037]
  Of course, the recording medium may be a magnetic recording medium, a magneto-optical recording medium, or any recording medium that will be developed in the future. In addition, the duplication stages such as the primary duplication product and the secondary duplication product are equivalent without any question.
0038]
  Further, even when a part is software and a part is realized by hardware, there is nothing completely different in the idea of the invention, and a part is stored on a recording medium, and it is appropriately changed as necessary. It may be in the form of being read.
[0039]
【The invention's effect】
  As described above, according to the present invention, it is possible to prevent the edge from being blurred by changing the calculation method depending on whether or not the color blur pixel is an edge pixel.
[0040]
  As an example, according to the invention according to claim 2, when the color blur pixel is an edge pixel, the central value is used for the color blur pixel in the color difference component of the pixel in a predetermined range, and the color blur pixel is not an edge pixel. Since the smoothing process is performed, the edge portion of the image is not blurred.
[0041]
  And claimsAccording to the invention according to No. 3, since the color blur pixel is detected by a relatively simple calculation method and the color blur is reduced for a predetermined range of pixels based on the color blur pixel, the calculation amount is reduced. And providing an image processing apparatus capable of shortening the processing timebe able to.
[0042]
  And claimsAccording to the invention according to No. 4, since it is only necessary to focus on low-density element colors, it is possible to easily detect a color blur pixel.be able to.
[0043]
  In other words, it is only necessary to examine the degree of change in the difference between low-density element colors and other element colors, so that color blur pixels can be detected.Calculation can be performed easily.
[0044]
  And claimsAccording to the fifth aspect of the present invention, when there are a plurality of low-density element colors, it is only necessary to pay attention to the degree of change in the difference between the low-density element colors.I can.
[0045]
  And claimsAccording to the sixth aspect of the invention, since the change degree of the difference is examined with respect to low density element color pixels, detection with higher reliability can be performed.
[0046]
  further,Even if the color blur is reduced by the smoothing process, it is only necessary to smooth the color difference component and return it to the original element color component.It can be carried out.
[0047]
  further,If edge enhancement processing is performed, the reduction in image sharpness due to smoothing processing is compensated.be able to.
[0048]
  Furthermore, if the edge enhancement processing is performed on the pixels within the range to be smoothed, sharpness does not increase unnaturally for pixels outside the range of the smoothing processing.
  Further, if the range to be smoothed is changed according to the size of the image to be processed, the smoothing process in the optimum range can be performed.
  In addition, the central value is used for the color blur pixel in the color difference component of the pixel in the predetermined range.Thus, color bleeding can be reduced.
[0049]
  And claimsAccording to the seventh aspect of the invention, it is possible to provide an image processing method in which edges are not blurred in the same manner.According to the invention, a medium on which an image processing control program is recorded can be provided.
[0050]
DETAILED DESCRIPTION OF THE INVENTION
  Hereinafter, embodiments of the present invention will be described with reference to the drawings.
  FIG. 1 is a block diagram illustrating an image processing system to which an image processing apparatus according to an embodiment of the present invention is applied. FIG. 2 is a schematic block diagram illustrating a specific hardware configuration example.
  In FIG. 1, an image input device 10 outputs an actual image captured using a single-plate CCD to the image processing device 20 as image data represented by dot matrix pixels. Based on the principle described above, color blur occurs in the image data captured by the single-plate CCD, and the image processing device 20 detects the color blur pixel in the input image data, and within a predetermined range based on the color blur pixel. Image processing is performed so as to reduce color blur and the image is output to the image output device 30. The image output device 30 outputs image-processed image data as pixels in a dot matrix.
[0051]
  The image processing apparatus 20 includes color blur pixel detection means because it detects color blur pixels, and performs image processing so as to reduce color blur with reference to the detected color blur as image processing means. It can be said that it also has.
[0052]
  A specific example of the image input device 10 corresponds to the film scanner 11, the digital still camera 12, the video camera 14, or the like provided with a single-plate CCD in FIG. 2. For example, FIG. 3 is a schematic block diagram showing a simple hardware configuration example of the digital still camera 12. In the figure, incident light is received by a CCD 12b through an optical system 12a. In the CCD 12b, color filters of R, G, and B colors are arranged at a predetermined ratio corresponding to each pixel, and a color signal in each pixel is output by the drive circuit 12c. The output color signal is digitally converted and input to the interpolation calculation circuit 12d, and the color signal for the component color component that cannot be obtained directly in each pixel by the interpolation calculation circuit 12d is obtained by linear interpolation calculation from surrounding pixels. Obtained and stored in the image memory 12e as RGB gradation data.
[0053]
  On the other hand, a specific example of the image processing apparatus 20 corresponds to a computer system including a computer 21, a hard disk 22, a keyboard 23, a CD-ROM drive 24, a floppy disk drive 25, a modem 26, and the like, and a specific example of the image output apparatus 30 is a printer. 31 and the display 32 are applicable. The modem 26 is connected to a public communication line, connected to an external network via the public communication line, and can be installed by downloading software and data.
[0054]
  In this embodiment, the film scanner 11 or the digital still camera 12 as the image input device 10 outputs RGB gradation data as image data, and the printer 31 as the image output device 30 outputs CMY (cyan) as gradation data. , Magenta, yellow) or CMYK binary data with black added thereto is required as input, and the display 32 requires RGB gradation data as input.
[0055]
  On the other hand, an operating system 21a operates in the computer 21, and a printer driver 21b and a display driver 21c corresponding to the printer 31 and the display 32 are incorporated. The image processing application 21d is controlled by the operating system 21a to execute processing, and executes predetermined image processing in cooperation with the printer driver 21b and the display driver 21c as necessary. Therefore, the specific role of the computer 21 as the image processing apparatus 20 is to input RGB gradation data, and to detect RGB pixels and perform image processing so as to reduce color bleeding. Is displayed on the display 32 via the display driver 21c, and is converted into binary data of CMY (or CMYK) via the printer driver 21b and printed on the printer 31.
[0056]
  As described above, in this embodiment, a computer system is incorporated between image input / output devices to perform image processing. However, such a computer system is not necessarily required, and a single-plate CCD is used. As described above, the present invention can be applied to a system that performs image processing so as to reduce color blur for image data that has undergone interpolation calculation as described above and has color blur. For example, as shown in FIG. 4, an image processing device that performs image processing in the same manner is incorporated into a digital still camera 13a having a single-plate CCD, and image data with reduced color blur is displayed on the display 32a or the printer 31a. It is also possible to use a system that causes the printer to print. Further, as shown in FIG. 5, in a printer 31b that inputs and prints image data without going through a computer system, an image inputted through a film scanner 11b having a single-plate CCD, a digital still camera 13b, or the like. It is also possible to configure the image processing so that color blur is reduced for the data.
[0057]
  The above-described image processing for detecting a color blur pixel and reducing the color blur is specifically performed by the image processing program corresponding to the flowchart shown in FIG.
  The color blur reduction method employed in the present embodiment is the same as the conventional method in that a smoothing filter or the like is applied as described later, but the region in which the smoothing filter or the like is applied is the color blur pixel. It is different from the conventional one by being limited to the periphery. Here, regarding the image data to be processed, two types of image size bitmaps are considered as shown in FIGS. 7A and 7B, and smoothing of the same size indicated by the hatched portion in the figure is performed for both. Let the filter act. The image size is different. If each pixel of the image data and each pixel of the CCD have a one-to-one correspondence, the image size naturally varies depending on the number of pixels of the CCD used. Even when used, the image size may be changed by appropriately expanding and contracting, and such a case is also included.
[0058]
  As is apparent from the figure, as long as smoothing filters of the same size are applied, the proportion of the area to be smoothed with respect to the entire image varies depending on the image size. Accordingly, if a smoothing filter of the same size is uniformly applied to image data of all image sizes, an area of an appropriate size may be smoothed as shown in FIG. For example, as shown in FIG. 5B, the ratio of the area to be smoothed to the entire image may increase. Since the smoothing process is nothing but blurring of the image, most of the image may be blurred in the case shown in FIG.
[0059]
  Therefore, in the present embodiment, a plurality of smoothing filters having different sizes such as 3 × 3, 5 × 5, and 7 × 7 pixels as shown in FIG. 8 are held, and these smoothing filters are retained according to the image size. The appropriate filter is properly used. Here, in determining the image size of the bitmap image, (height) × (width) of the image is calculated to obtain the number of pixels, and the calculated number of pixels may be used as an index. Determines the image size of the processing target image according to A expressed by the following equation.
[0060]
  A = min (height, width) (1)
  Here, min (height, width) means the smaller of height and width. In step S110,
  If A <300,
    3 × 3 pixel smoothing filter
  If 300 ≦ A ≦ 600,
    5 × 5 pixel smoothing filter
  If A> 600,
    7 × 7 pixel smoothing filter
The type of smoothing filter used in steps S122 to S126 is stored in the work area. The work area may be the RAM in the computer 21 or the hard disk 22.
[0061]
  Here, the reason for using min (height, width) as a reference is as follows. In determining whether or not the size of the smoothing filter is appropriate, it may be determined based on the ratio of the number of pixels to be smoothed with respect to the total number of pixels. However, for example, even if the image data has the same number of pixels, there is a wide image like the bitmap image shown in FIG. 9A, but the height direction like the bitmap image shown in FIG. Some standard images have a slightly longer width in the width direction. If the smoothing filter to be used is determined according to the total number of pixels of the processing target image, the same smoothing filter is selected for both. Then, in the latter case, even if the size of the smoothing filter is appropriate, in the former case, the height direction may be smoothed over almost the entire length. In such a case, a visually blurred impression is received. . Therefore, if the smaller one of height and width is used as a reference, such an adverse effect can be avoided.
[0062]
  Referring to the flowchart of FIG. 6, as shown in FIG. 10, the pixel to be processed is moved by sub-scanning in the vertical direction while main-scanning the pixel to be processed in the image data including dot matrix pixels, and each pixel is blurred. It is determined whether or not it is a pixel.
[0063]
  When the image data is composed of dot-matrix pixels, each pixel is represented by RGB gradation data (“0” to “255”). After the type of the smoothing filter used in step S122 to step S126 is stored in the work area, in the next step S130, a difference ΔRB between the R and B gradation data in the target pixel and its surrounding pixels is calculated. This ΔRB is
  ΔRB = | R−B | (2)
It is expressed.
[0064]
  As shown in FIG. 23A, in the CCD according to the present embodiment, R, G, and B color filters are arranged in a mosaic pattern corresponding to each pixel, and only the G color filters are staggered. The density is high due to the arrangement, and the R and B color filters are low density. A color signal that cannot be obtained directly in each pixel is linearly interpolated from the color signals of adjacent pixels to obtain color signals of all the colors R, G, and B, and converted into gradation data. Therefore, from the viewpoint of the probability, the false element color component such as the original intensity cannot be obtained or the color component which is not originally required is generated for the low density R and B gradation data in a certain pixel as a result of the interpolation calculation. It can be said that color blur occurs.
[0065]
  In addition, it is known that color blurs appear prominently near the boundary between white and black. For example, when expressing white in the pixel corresponding to the R color filter, the original RGB gradation data is (R, G, B) = (255, 255, 255), and the result of the interpolation operation (R , G, B) = (255, 255, 127). Here, since the G color filter has a high density, the probability that a false element color component is generated by the interpolation operation is small for the G component, while the R component can be obtained directly from the color filter. The color component is never generated. Originally, if white is expressed in a certain pixel, ΔRB will be “0”, and even if black is expressed, ΔRB should be “0”, but in a color-bleeding pixel, ΔRB is compared to this. The value of tends to increase.
[0066]
  Of course, depending on the image data, there may naturally be pixels having high R and G components and low B components. However, in such a case, each element color component is held in the adjacent pixel, and the value of ΔRB often changes gently as the distance from the pixel increases. On the other hand, in principle, a color blur is generated in units of pixels, and any one of the adjacent pixels is a pixel such as white or black, and therefore, the degree of change in ΔRB between the adjacent pixels is large. It can be said that there is no mistake as a pixel that blurs the color. Therefore, in the present embodiment, as will be described later, the degree of change in the value of ΔRB between the processing target pixel and the peripheral pixels is examined, and a pixel having a large degree of change is detected as a color blur pixel. Of course, since the G color filter has a high density, it is unlikely that a false element color component will be generated by the interpolation operation.
  ΔGR = | G−R | (3)
  ΔGB = | GB | (4)
It is also possible to substitute for this formula.
[0067]
  Further, in examining such a change degree of ΔRB, it is preferable to compare between pixels corresponding to a low-density color filter. That is, since a false element color component is more likely to occur in a pixel corresponding to a low-density color filter than a pixel corresponding to a high-density color filter, ΔRB between pixels corresponding to a low-density color filter is larger. It is possible to perform detection with higher reliability by comparing the degree of change.
[0068]
  The actual color blur pixel is detected in step S140. In step S140, whether or not the pixel is a color blur pixel is determined based on the degree of change in ΔRB calculated as described above between the peripheral pixels. At this time, as shown in FIG. 11, a matrix with the processing target pixel as the center, x in the horizontal direction, and y in the vertical direction is considered.
  E = 4ΔRB (x, y) −2ΔRB (x−1, y−1)
                    -2ΔRB (x + 1, y-1) (5)
The magnitude relationship between the value of E expressed by the above and a predetermined threshold Th is compared. When E ≧ Th, it is determined that the pixel is a color blur pixel. Therefore, in step S130 described above, ΔRB (x, y) in the determination target pixel is obtained, and ΔRB (x−1, y−1) and ΔRB (x + 1, y−) are used as the values of ΔRB in the peripheral pixels. We will ask for 1).
[0069]
  The meaning of detecting a color-bleeding pixel according to the judgment criterion of E ≧ Th is as follows. If color blur occurs in the processing target pixel, the degree of change in the ΔRB value between the pixel and its surrounding pixels increases. Therefore, referring to equation (5), the value of E also increases, and a predetermined threshold Th If it exceeds, the pixel is determined to be a color blur pixel. Further, referring to the equation (5) and FIG. 11, it can be easily understood that the degree of change in ΔRB is examined for the determination target pixel and the pixel in the oblique direction. Here, referring to FIG. 23A, if the determination target pixel corresponds to R, the diagonal pixel corresponds to B, and if the determination target pixel corresponds to B, the diagonal pixel corresponds to R. If the target pixel corresponds to G, the diagonal pixel corresponds to G. That is, when the determination target pixel corresponds to a low-density element color, the degree of change in the ΔRB value between the pixels corresponding to the adjacent low-density element color is checked, and the determination target pixel has a high-density element color. In the case of corresponding to the color, the degree of change in ΔRB is examined between the pixels corresponding to the adjacent high-density element colors.
[0070]
  Therefore, in steps S130 and S140, a color blur pixel is detected based on the degree of change in the difference between the element color intensities for the low-density color filter between adjacent pixels, and the hardware configuration and software for executing these. Thus, a color blur pixel detecting means is constituted.
[0071]
  If it is determined in step S140 that the pixel is a color blur pixel, it is determined in step S150 whether the pixel is an edge pixel. In determining whether the pixel is an edge pixel, a technique based on the color difference component is effective. Therefore, in the present embodiment, the luminance component Y is subtracted from the R and B gradation data, and the color difference component C1 is obtained. , C2. The color difference components C1 and C2 are
  C1 = R−Y (6)
  C2 = BY (7)
It can be expressed as.
[0072]
  However, RGB gradation data does not have a luminance value directly, and it is possible to perform color conversion to the Luv color space in order to obtain the luminance, but this is not a good solution because of problems such as the amount of computation. For this reason, the following conversion formula for directly obtaining the luminance from RGB used in the case of a television or the like is used.
  Y = 0.30R + 0.59G + 0.11B (8)
[0073]
  Since the edge pixel is a boundary portion of the image, it can be said that the degree of change of the color difference components C1 and C2 is large between adjacent pixels. Therefore, when one of the following two criteria is satisfied, Judgment can be made.
  | C1 (x, y) -C1 (x-1, y-1) | ≧ Th1
                                                  ... (9)
  | C2 (x, y) -C2 (x-1, y-1) | ≧ Th2
                                                  (10)
  Here, x represents horizontal coordinates, and y represents vertical coordinates.
[0074]
  In other words, the expressions (9) and (10) mean that, as shown in FIG. 12, in image data composed of pixels in a dot matrix centered on the color-bleeding pixel, between adjacent pixels in the diagonal direction. This is nothing other than determining the degree of change of the color difference components C1 and C2 and determining whether or not they are equal to or greater than the threshold values Th1 and Th2, respectively. When one of the determination criteria is satisfied, it is determined that the pixel is an edge pixel. Of course, the pixels are essentially arranged in a grid pattern vertically and horizontally as shown in FIG. 13, and there are eight adjacent pixels when attention is paid to the center pixel. Accordingly, similarly, the degree of change of the color difference components C1 and C2 is obtained between the adjacent pixels, and the comparison calculation is performed between the threshold values Th1 and Th2, respectively. When the degree of change in C1 or C2 is greater than or equal to the threshold value Th1 or Th2, it may be determined that the pixel is a color blur pixel.
[0075]
  In determining whether or not a color blur pixel is an edge pixel in this way, although a method based on the degree of change of the color difference components C1 and C2 is effective, one color blur pixel is adjacent to each other. Since the comparison calculation for the color difference components C1 and C2 must be performed in each of the eight pixels, the amount of calculation becomes large, and the processing speed may decrease. Therefore, a comparison operation may be performed for four pixels in the vertical and horizontal directions as shown in FIG. 14, or a comparison operation may be performed for four pixels in the oblique direction as shown in FIG. Absent.
[0076]
  Further, whether or not the pixel is an edge pixel may be determined based on the magnitude of the luminance gradient. In this case, the above equations (9) and (10) may be replaced with the following equations.
  | Y (x, y) −Y (x−1, y−1) | ≧ Th3
                                                  ... (11)
If this equation (11) is used as a criterion for determining edge pixels, it can be easily understood that the amount of calculation is halved compared to the case where equations (9) and (10) are used.
[0077]
  If it is determined in step S150 that the color bleeding pixel is not an edge pixel, the smoothing filter determined in step S122 to step S126 is applied to the same color bleeding pixel in step S152 so that the same color bleeding pixel is detected. A smoothing process is performed on a predetermined range of pixels. Of course, since the smoothing of the color difference component is effective in performing the smoothing process, in this embodiment, the color difference components C1 and C2 calculated based on the above formulas (6) and (7) are smoothed. Apply processing.
[0078]
  Here, the smoothing process using the smoothing filter will be described. In each of the smoothing filters shown in FIG. 8, the numerical value at the center is used as the weighting of the color difference components C1 and C2 of the pixel to be processed in the matrix-like image data, and the numerical values corresponding to the squares of the smoothing filter for the peripheral pixels Used for weighting and summing. In this case, since “1” is set for all the cells, the color difference component C1 ′ after smoothing is obtained by summing the color difference components C1 of each cell and dividing by the total number of cells to obtain the color difference component C2 after smoothing. 'Is determined in the same way. Of course, each square may be appropriately weighted. However, if the smoothing filter is operated, the matrix operation as described above is performed. Therefore, if each square is weighted, the calculation amount increases. That is, in this embodiment, since “1” is set in each cell of the smoothing filter, the data of each pixel may be summed and divided by the number of cells, but when each cell is weighted Since a multiplication operation and an addition operation are required for the number of cells, the amount of calculation becomes large.
[0079]
  If such a matrix calculation is performed on all pixels of image data as in the conventional case, the calculation amount may be enormous and may take a lot of processing time. Therefore, if a color blur pixel is detected and smoothing processing is performed on the periphery of the color blur pixel as in this embodiment, a large reduction in the amount of calculation is expected, and high-speed image processing can be realized. Become.
[0080]
  On the other hand, if it is determined in step S150 that the color blur pixel is an edge pixel, a 5 × 5 pixel median filter is applied with the same color blur pixel as the center in step S154. Note that the size of the median filter to be actually applied is not necessarily 5 × 5 pixels, and can be appropriately changed by applying a median filter of 3 × 3 pixels. Therefore, for convenience of explanation, a case where a 3 × 3 pixel median filter is applied will be described.
[0081]
  For example, as shown in FIG. 16, a 3 × 3 pixel dot matrix centered on a color blur pixel is considered. In addition, the value of each square has shown the value of the color difference component C1, and the diagonal line in a figure corresponds to an edge direction. Here, applying the median filter of 3 × 3 pixels means that the values of the color difference components C1 of all nine pixels are sorted in ascending or descending order, and the central value is selected to replace the color difference component C1 of the color blur pixel. Means that. That is, in the figure, there are six pixels with a color difference component C1 value of “10”, one “40” pixel, and three “100” pixels. Therefore, if these values are sorted in ascending order, the result is as shown in FIG. Then, as is apparent from the figure, the central value is “10”, and the color difference component C1 of the color-bleeding pixel is “10”.
[0082]
  On the other hand, assuming that a smoothing filter of 3 × 3 pixels is applied here, “33”, which is a value obtained by dividing the total value “300” of the cells by the number of pixels “9”, is the color difference component after the smoothing process. C1 ′. Since the smoothed color difference component C1 ′ is obtained by averaging and adding the color difference component C1 of the peripheral pixels to the color blur pixel, the color difference component C1 ′ is smooth image data. The smoothed image has the same meaning as a so-called low-pass filter, and the image is smoothed so that the color blur is not noticeable. Absent. Therefore, it is clear from the calculation result that the edge portion is not blurred if the above-described median filter is applied to the edge pixel.
[0083]
  On the other hand, referring to FIG. 16, it can be seen that color blur occurs in a pixel having a color difference component C1 value of “40”. Therefore, when a median filter of 3 × 3 pixels is applied to this color blur pixel, the color difference component value of seven pixels among the adjacent eight pixels is “100”, and the value of the color difference component of only one pixel is “100”. Since it is “10”, the value of the color difference component of the color blur pixel is also replaced with “100”, and it can be seen that the color blur is reduced. As described above, the median filter has an effect of reducing the color blur while preventing the edge portion from being blurred. In step S150, it is determined whether or not the color blur pixel is an edge pixel. The meaning is due to such reasons.
[0084]
  The median filter exemplified here is 3 × 3 pixels, but of course, in the case of the 5 × 5 pixel median filter employed in this embodiment, the center value is selected from among all 25 pixels. It can be considered just like the case of 3 × 3 pixels. Further, although the color difference component C1 is exemplified, it goes without saying that the same applies to the color difference component C2.
[0085]
  By the way, comparing the calculation speeds of the smoothing filter and the median filter, the median filter is relatively slow. Therefore, it is preferable from the viewpoint of processing speed to suppress the scene where the median filter is applied as much as possible. When the expressions (9) and (10) are adopted as the judgment criterion of the edge pixel, the same expression may be satisfied in either one of the color difference components C1 and C2, or the same expression is satisfied in both the color difference components C1 and C2. It may be satisfied. Here, in the former case, it is not necessary to apply the median filter to both color difference components, and the processing speed is reduced. Therefore, the median filter is applied in step S154 only for the color difference components satisfying the equation, For another color difference component, the processing speed can be improved as a whole by applying a smoothing filter in step S152. Of course, when the equation (11) is adopted as the determination criterion for the edge pixel, it cannot be determined which color difference component has a large degree of change. In this case, the color difference components C1, C2 are determined in step S154. The median filter is applied to both of the above.
[0086]
  As described above, reducing the color blur is not only smoothing the image and making the color blur inconspicuous, but it can also be assumed that the image becomes blurred. Therefore, in this embodiment, edge enhancement processing is performed in the next step S160.
[0087]
  In this edge enhancement processing, the luminance Y ′ after enhancement is higher than the luminance Y of each pixel before enhancement.
  Y ′ = Y + (Y−Yunsharp) (12)
Is calculated as Here, Yunsharp is obtained by performing unsharp mask processing on image data of color-blurred pixels. Next, unsharp mask processing will be described. In this embodiment, as shown in FIG. 18, three types of unsharp masks each including 3 × 3, 5 × 5, and 7 × 7 pixels are provided, and the sizes of the smoothing filters determined in steps S122 to S126, respectively. It corresponds to. For example, if a 3 × 3 pixel smoothing filter is selected, a 3 × 3 pixel unsharp mask is selected. Similar to the smoothing filter described above, these unsharp masks use the central numerical value as the weight of Y (x, y) in each pixel, and the peripheral pixels are weighted corresponding to the numerical values in the squares of the mask. It is used to accumulate. For example, when using a 3 x 3 pixel unsharp mask,
[0088]
[Expression 1]

Is integrated based on the following equation. In this equation (13), “136” is the total value of the weighting coefficients, and in the smoothing filters of different sizes, it is the total value of the cells. That is, if it is 5 × 5 pixels, it will be “400”, and if it is 7 × 7 pixels, it will be “900”. Mij is a weighting coefficient written in the grid of the unsharp mask, and Y (x, y) is a luminance component in each pixel. Furthermore, ij is indicated by the coordinate values of the row and column with respect to the unsharp mask.
[0089]
  Yunsharp (x, y) is obtained by lowering the weight of the surrounding pixels and adding them to the color blur pixel, and in this case as well, the same meaning as that obtained by applying a low-pass filter. Therefore, “Y (x, y) −Yunsharp (x, y)” has the same meaning as that obtained by applying the high-pass filter by subtracting the low frequency component from the original all components. And high passIIf this high frequency component that has passed through the filter is added to “Y (x, y)”, the high frequency component is increased, and the sharpness of the image is improved as a result of enhancing the edges.
[0090]
  As is apparent from FIG. 18, these unsharp masks have the highest weighting at the center, and the weighting coefficient gradually decreases toward the periphery. Therefore, it can be said that the influence of the weighting coefficient on the peripheral side on the total value of the squares is small. On the other hand, the matrix calculation represented by the equation (13) requires a large amount of calculation because multiplication and addition operations are required for the pixels around the processing target pixel by the number of squares of the unsharp mask to be used. It is also envisioned. Therefore, in this embodiment, by using an unsharp mask of 3 × 3, 5 × 5, and 7 × 7 pixels that is configured by omitting the peripheral side weighting coefficient in advance, the amount of calculation is reduced and the processing speed is increased. I am doing so.
[0091]
  By the way, in this embodiment, the size of the smoothing filter and the size of the unsharp mask are the same. This is for the following reason. For example, assume that the bitmap image shown in FIG. 19 is subjected to smoothing processing on the area indicated by the hatched portion in the figure, and unsharp mask processing is applied to the area indicated by the wavy line in the drawing to enhance the luminance. . As described above, the meaning of emphasizing the brightness is to compensate for the sharpness lost by the smoothing process. Therefore, if the brightness is emphasized to the outside of the smoothing region, the outer region is unnatural. It may occur that the sharpness is improved. For this reason, in the present embodiment, the smoothing filter and the unsharp mask have the same size, but the latter size may be made smaller than the former.
[0092]
  In step S160, the edge enhancement processing as described above is performed, and the levels of R ′, G ′, and B ′ are calculated based on the expressions (6) to (8) using the luminance Y ′ after enhancement and the color difference component after the processing. Obtain key data. That is, color blur is reduced in steps S152 and S154, and gradation data is generated in step S160, and the image processing means is configured by a hardware configuration and software for executing these.
[0093]
  In step S160, since the emphasized luminance Y ′ is used, the gradation data of R ′, G ′, and B ′ may be negative values or values that exceed “255”. However, since the gradation width is in the range of “0” to “255”, negative gradation data is uniformly “0”, and gradation data exceeding “255” is uniformly “255”. Then, the processing target pixel is moved in the next step S170, and the same processing is repeated until it is determined in step S180 that all pixels have been completed.
[0094]
  By the way, if a smoothing filter is applied centering on the color blur pixel and then the edge enhancement process is performed, the color blur is reduced by the smoothing process, and the sharpness of the image lost in the smoothing process by the edge enhancement process is reduced. Compensated. Therefore, it is only necessary to perform processing uniformly for all color-bleeded pixels regardless of whether or not the color-bleeded pixel is an edge pixel. it can. However, in such a configuration, the edge portion may be blurred from an empirical point of view. Therefore, when the median filter is applied when the color blur pixel is an edge pixel, a satisfactory result can be obtained.
[0095]
  Here, the effects of the smoothing process and the edge enhancement process of the present embodiment will be described with reference to a simple one-dimensional simulation model.
  FIG. 20 is a table showing values of various parameters in the one-dimensional simulation model when the edge enhancement processing is not performed. In the figure, the leftmost R, G, B matrix shows the CCD color filter array. “IN” shown on the right side of the matrix indicates the incident state of light. If “1 (gradation data is 255)”, it means that the light is hit, that is, “white”. If the key data is 0) ", it means" black ". It is assumed that the color signal that cannot be obtained directly in each pixel of the CCD is obtained by linear interpolation from the color signal of the adjacent pixel.
[0096]
  Here, although it should originally be considered in a two-dimensional array, attention is paid to the central column of the R, G, B matrix in order to simplify the description. In this sense, it is called a one-dimensional simulation model. Then, from the above-described principle, false color signals are generated in the vicinity of the white / black boundary such that G (B) = 0.5 and R = 0.5, and the pixel where the false color signal is generated is colored. Detected as a blurred pixel. When a color blur pixel is detected, color difference components C1 and C2 are calculated according to equations (6) to (8), and the color difference components C1 and C2 are smoothed to obtain color difference components C1 ′ and C2 ′. Here, “1,1,1,1,1,1,1” is applied as a one-dimensional smoothing filter. That is, the pixel values of the three pixels before and after the processing target pixel are totaled and divided by “7”. Then, the obtained color difference signals C1 ′ and C2 ′ are returned to the original color signals based on the equations (6) to (8), and R ′, G ′, and B ′ after smoothing processing are obtained. As you can see from the figureWhite·(R ′, G ′, B ′) in the pixels near the black boundary are (0.75, 0.61, 0.61) and (0.25, 0.11, 0.11), respectively, It can be seen that the color blur is reduced by approaching the value of “IN”.
[0097]
  FIG. 21 also shows a one-dimensional simulation model in the case where the smoothing process is performed and the edge enhancement process is performed using an unsharp mask having the same size as the smoothing filter. Also in this case, obtaining the color difference components C1 and C2 and performing the smoothing process is not different from the above, but when returning to the original color signal using the equations (6) to (8), “1, The luminance signal Y ′ enhanced using a one-dimensional unsharp mask having a weight of “6, 18, 25, 18, 6, 1” is used. Then, (R ′, G ′, B ′) at pixels near the white / black boundary are (0.81, 0.67, 0.67) and (0.10, −0.04, −0), respectively. .04). Of course, since the possible values of RGB signal values are “0” to “1”, the latter data is actually (0.10, 0, 0), and the color blurs compared to the one shown in FIG. It can be seen that is further resolved.
[0098]
  Further, FIG. 22 similarly shows a one-dimensional simulation model when edge enhancement processing is performed using an unsharp mask having a size smaller than that of the smoothing filter. In this case, a 5-digit unsharp mask smaller than the one-dimensional smoothing filter size (1, 11, 25, 11, 1) (7 digits) is used. When the calculation is performed in the same manner as described above, (R ′, G ′, B ′) in the pixels near the white / black boundary are (0.79, 0.65, 0.65) and (0.10, -0.04, -0.04). Thus, it can be seen that although the degree of elimination of color blur is inferior to that shown in FIG. 21, the color blur is further eliminated compared to that shown in FIG.
[0099]
  Next, the operation of the present embodiment configured as described above will be described.
  Assume that a real image captured by a digital still camera 12 having a single-plate CCD is taken into a computer 21 and printed by a printer 31. Then, first, the image processing application 21d is activated and the image data is captured from the digital still camera 12 while the operating system 21a is running on the computer 21. When the image data is taken into the image processing application 21d via the operating system 21a, an index value for determining the image size is acquired based on the equation (1) in step S110, and this index value and a predetermined value are determined. And the sizes of the smoothing filter and unsharp mask used in steps S122 to S126 are determined and stored in the work area.
[0100]
  In step S130, the pixel to be processed is set to the initial position, and the value of ΔRB in the pixel and its surrounding pixels is calculated according to equation (2). After that, in step S140, the degree of change in the value of ΔRB between the processing target pixel and its surrounding pixels is checked based on the expression (5), and the degree of change is larger than the predetermined threshold Th. It is determined that the pixel is a color blur.
[0101]
  If it is determined that the pixel is a color blur pixel, the color difference components C1 and C2 are calculated for the processing target pixel and its peripheral pixels based on the equations (6) to (8) in step S150, and the equations (9) and (10), or It is determined whether or not the color-bleeding pixel is an edge pixel according to a criterion such as equation (11). If it is determined that the pixel is not an edge pixel, a smoothing filter equal to the filter size stored in the work area in steps S122 to S126 is applied in step S152. That is, for each of the color difference components C1 and C2, the color difference components C1 ′ and C2 ′ that have been smoothed are obtained by summing the color difference components of the pixels corresponding to the cells of the smoothing filter and dividing the sum by the number of cells.
[0102]
  On the other hand, if it is determined in step S150 that the color blur pixel is an edge pixel, a 5 × 5 pixel median filter centered on the same color blur pixel is applied in step S154. That is, the color difference component values of all 25 pixels centered on the color blur pixel are sorted in ascending or descending order, and the central value is selected and replaced with the color difference component of the color blur pixel. At this time, when the determination criteria of the equations (9) and (10) are adopted in step S150, the median filter is applied in step S154 to the color difference components satisfying the equation in each of the color difference components C1 and C2. In step S152, the smoothing filter is applied to color difference components that do not satisfy the above equation. Of course, when the same expression is satisfied for both color difference components, the median filter is applied to both color difference components C1 and C2 in step S154. On the other hand, when the criterion (11) is adopted in step S150, the median filter is applied to both color difference components C1 and C2 in step S154.
[0103]
  Thereafter, in step S160, the low-frequency component Yunsharp of the luminance component Y is obtained for the pixel to be processed and its surrounding pixels using the unsharp mask of the size determined in steps S122 to S126, and the same low value is obtained from the original luminance component Y. A high frequency component is obtained by subtracting the frequency component Yunsharp, and the enhanced luminance component Y ′ is obtained by adding the high frequency component to the same luminance component Y. Specifically, the calculation is performed based on the equation (13). Then, using the enhanced luminance component Y ′ and the processed color difference component, the gradation data of R ′, G ′, B ′ after smoothing processing is obtained based on the equations (6) to (8). The above process is repeated until it is determined in step S180 that all the pixels have been executed while moving the process target pixel in step S170.
[0104]
  When the processing is completed for all pixels, the image processed image data is displayed on the display 32 via the display driver 21c, and if it is satisfactory, the image is printed by the printer 31 via the printer driver 21b. That is, the printer driver 21b inputs RGB gradation data with reduced color blur, performs rasterization corresponding to the print head area of the printer 31 through predetermined resolution conversion, and changes the rasterized data from RGB to CMYK. After that, the CMYK gradation data is converted into binary data and output to the printer 31.
[0105]
  Through the above processing, the actual image data captured from the digital still camera 12 is image-processed so as to reduce the color blur only in the portion where the color blur is automatically generated and displayed on the display 32, and then is displayed on the printer 31. Printed. That is, since image processing is performed so as to reduce color blur only in a portion where color blur occurs, high-speed image processing can be realized with a reduced amount of computation.
[0106]
  As described above, when the computer 21 serving as the center of the image processing determines whether or not the processing target pixel is a color blur pixel based on the low-density element color intensity in steps S130 and S140, and determines that the pixel is a color blur pixel. In step S150, it is determined whether or not the pixel is an edge pixel. If the pixel is not an edge pixel, a smoothing filter is applied in step S152. If it is an edge pixel, a median filter is applied in step S154. By doing so, the image processing is performed so as to reduce the color blur, so that the amount of calculation can be reduced and high-speed image processing can be executed.
[Brief description of the drawings]
FIG. 1 is a block diagram of an image processing system to which an image processing apparatus according to an embodiment of the present invention is applied.
FIG. 2 is a block diagram of specific hardware of the image processing apparatus.
FIG. 3 is a block diagram illustrating a simple hardware configuration example of a digital still camera.
FIG. 4 is a schematic block diagram illustrating another application example of the image processing apparatus of the present invention.
FIG. 5 is a schematic block diagram illustrating another application example of the image processing apparatus of the present invention.
FIG. 6 is a flowchart showing a color blur pixel determination and image processing portion in the image processing apparatus of the present invention.
FIG. 7 is a diagram illustrating a relationship between an image size and a smoothed area.
FIG. 8 is a 3 × 3, 5 × 5 and 7 × 7 pixel smoothing filter.
FIG. 9 is a diagram illustrating a relationship between height and width in image data having the same number of pixels.
FIG. 10 is a diagram illustrating a state in which a processing target pixel is moved.
FIG. 11 is an explanatory diagram in a case where the change degree of the element color component is obtained by a difference value between adjacent pixels in an oblique direction.
FIG. 12 is an explanatory diagram in a case where an image change degree is obtained as a difference value between adjacent pixels in an oblique direction.
FIG. 13 is an explanatory diagram for obtaining the degree of change in image between all adjacent pixels.
FIG. 14 is an explanatory diagram of a modified example for obtaining a change degree of an image between adjacent pixels.
FIG. 15 is an explanatory diagram of another modified example for obtaining the degree of change in image between adjacent pixels.
FIG. 16 is a diagram illustrating an example of a color difference component value of each pixel in image data of an edge portion including a color blur pixel.
FIG. 17 is a diagram for explaining an example of arithmetic processing using a median filter;
FIG. 18 is an unsharp mask of 3 × 3, 5 × 5, and 7 × 7 pixels.
FIG. 19 is a diagram illustrating a relationship between a smoothing region and an edge enhancement region.
FIG. 20 is a table showing values of various parameters in a one-dimensional simulation model when edge enhancement processing is not performed.
FIG. 21 is a table showing values of various parameters in a one-dimensional simulation model when edge enhancement processing is performed using an unsharp mask having the same size as a smoothing filter.
FIG. 22 is a table showing values of various parameters in a one-dimensional simulation model when edge enhancement processing is performed using an unsharp mask having a size smaller than that of a smoothing filter.
FIG. 23 is a diagram for explaining the principle of occurrence of color blur.
[Explanation of symbols]
10. Image input device
20 Image processing apparatus
21 ... Computer
21a ... Operating system
21b ... Printer driver
21c ... Display driver
21d: Image processing application
22 ... Hard disk
23 ... Keyboard
24 ... CD-ROM drive
25. Floppy disk drive
26 Modem
30. Image output device

Claims (8)

Complementary element color gradation value data lacking in each pixel of the original image data obtained by imaging with a single-plate solid-state image sensor in which color filters of multiple element colors are arranged in a mosaic pattern An image processing apparatus that performs image processing on generated image data including a plurality of pixels,
Color blur pixel detecting means for detecting a color blur pixel included in image data;
Means for determining whether the detected color-bleeding pixel is an edge pixel;
An image processing apparatus comprising: an image processing unit that executes image processing for reducing color blur by changing a calculation method according to a determination result.   2. The image processing apparatus according to claim 1, wherein, when the color blur pixel is an edge pixel, the image processing means is a color difference component for a pixel in a predetermined neighborhood range based on the same color blur pixel. If it is not an edge pixel, the color difference component is smoothed for pixels in the same range, and the smoothed color difference component is replaced with the color difference of the same color blur pixel. An image processing apparatus characterized by replacing with a component.   3. The image processing apparatus according to claim 1, wherein the color blur pixel detecting unit obtains difference data of gradation values between different element colors for each pixel, and pixels adjacent to each other. An image processing apparatus that detects a color blur pixel when the change degree of the difference data is greater than a predetermined threshold value.   4. The image processing apparatus according to claim 3, wherein the color-bleeding pixel detection means obtains difference data of gradation values between different element colors for each pixel, and the low-density element color and the like in the original image data. An image processing apparatus for obtaining difference data between the element colors.   5. The image processing apparatus according to claim 4, wherein when there are a plurality of low density element colors, the color blur pixel detecting means changes a difference in gradation value data with respect to the low density element colors between adjacent pixels. An image processing apparatus for detecting a color blur pixel based on a degree.   The image processing apparatus according to any one of claims 3 to 5, wherein the color blur pixel detection unit is configured to detect the color blur pixel based on the degree of change between the adjacent pixels and a low density pixel. An image processing apparatus for detecting Complementary element color gradation value data lacking in each pixel of the original image data obtained by imaging with a single-plate solid-state image sensor in which color filters of multiple element colors are arranged in a mosaic pattern An image processing method for performing image processing on generated image data consisting of a plurality of pixels,
Detects color blur pixels in image data,
Determine whether this detected color blur pixel is an edge pixel,
An image processing method comprising: executing image processing for reducing color blur by changing a calculation method according to a determination result. Complementary element color gradation value data lacking in each pixel of the original image data obtained by imaging with a single-plate solid-state image sensor in which color filters of multiple element colors are arranged in a mosaic pattern A medium on which an image processing control program for performing predetermined image processing on a computer with respect to generated image data consisting of a plurality of pixels is recorded,
A function for detecting a color blur pixel included in image data;
A function of determining whether or not the detected color blur pixel is an edge pixel;
A medium having recorded thereon an image processing control program characterized by realizing a function of executing image processing for reducing color blurring by changing a calculation method according to a determination result.

Images