JP4019239B2 - Image sharpening method and image sharpening device - Google Patents

Description

なる演算式に基づいて積算する。同式において、「６３２」とは重み付け係数の合計値であり、むろんサイズの異なる三つのアンシャープマスク４１〜４３においては、それぞれ「３９６」、「６３２」「２５１６」というような値となる。また、Ｍｉｊはアンシャープマスクの升目に記載されている重み係数であり、Ｙ（ｘ，ｙ）は各画素の画像データである。なお、ｉｊについては異なる縦横サイズの三つのアンシャープマスク４１〜４３に対して横列と縦列の座標値で示している。
【００４９】
このような演算の意味するところは次のようになる。Ｙunsharp （ｘ，ｙ）は注目画素に対して周縁画素の重み付けを低くして加算したものであるから、いわゆる「なまった（アンシャープ）」画像データとしていることになる。このようにしてなまらせたものはいわゆるローパスフィルタをかけたものと同様の意味あいを持つ。従って、「Ｙ（ｘ，ｙ）−Ｙunsharp （ｘ，ｙ）」とは本来の全成分から低周波成分を引いたことになってハイパスフィルタをかけたものと同様の意味あいを持つ。そして、ハイパスフィルタを通過したこの高周波成分に対して強調係数Ｅenhance を乗算して「Ｙ（ｘ，ｙ）」に加えれば同強調係数Ｅenhance に比例して高周波成分を増したことになり、エッジが強調される結果となる。
【００５０】
また、エッジの強調度合いは、アンシャープマスクの大きさによっても変化する。縦横の升目数の異なる三つのアンシャープマスク４１〜４３であれば、大きなマスクほど注目画素の近隣の画素に対する重み付けが大きく、遠くの画素にいたるまでの距離の中で徐々に重み付けが減っていっている。これは言い換えればよりローパスフィルタとしての性格が強くなり、高周波成分を生成しやすくなるからである。
【００５１】
従って、強調係数Ｅenhance による強調度合いの調整に加えて、大きなサイズのアンシャープマスク４３を利用すれば強い強調処理を行うことになり、小さなサイズのアンシャープマスク４１を利用すれば弱い強調処理を行うことになる。むろん、中間的な強さの強調処理を行うのであれば中間サイズのアンシャープマスク４２を利用すればよくなる。
図９のフローチャートによれば、ステップ１３２にて非鮮鋭領域に強い強調処理を行うにはアンシャープマスク４３を使用しつつ完成アンシャープマスク１チャンネルを強調係数として使用して演算を行うし、ステップ１３４にて鮮鋭領域に弱い強調処理を行うにはアンシャープマスク４１を使用しつつ完成アンシャープマスク２チャンネルを強調係数として使用して演算を行うことになる。なお、アンシャープマスク４１〜４３のフィルタマスクは一例に過ぎず、適宜変更することも可能である。また、図１９を参照すると分かるように、最外周のパラメータは「０」または「１」であり、画素の画像データに乗算しても「６３２」で除算した場合の影響度を考えると殆ど無意味である。このため、最外周のパラメータを無視して５×５画素としたアンシャープマスク４４のフィルタマスクを使用すれば、除算の演算回数「４９（＝７×７）」回から「２５（＝５×５）」回へと半減し、演算処理時間を短縮化させることもできる。
【００５２】
以上のような処理を経ることにより、デジタルスチルカメラ１１ｂ２あるいはモデム１４ａ２等を介して取り込んだディジタルの画像データであっても、もともと鮮鋭なエッジ部分が過度にシャープになりすぎたり、妙に作り物の感じが強く表れてしまうということがなくなる。
このようにして得られた画像データ自体は背景レイヤに格納されており、この画像データをディスプレイドライバ１２ｂやプリンタドライバ１２ｃを介してディスプレイ１７ａやカラープリンタ１７ｂに出力すると、綺麗な画像となっている。
【００５３】
一般的にはこの強調処理で概ね自然な感じで画像の鮮鋭度が上がるが、人の肌が鮮鋭化してざらついた感じに見えることもあり得る。人自体が写真のオブジェクトであるため、特にその部分を注目してしまうためである。同様なことは、写真の中で広い面積を占める空の部分についても生じる。このような場合は、各画素が肌色や空色であるか否かを判断し、肌色や空色であったら強調処理を弱めるようにすればよい。ここで、各画素が肌色や空色であるか否かを判断する手法について説明する。
【００５４】
各画素の画像データが（Ｒ，Ｇ，Ｂ）で表されるとすると、色度は、
ｒ＝Ｒ／（Ｒ＋Ｇ＋Ｂ）
ｂ＝Ｂ／（Ｒ＋Ｇ＋Ｂ）
として表される。
図２１は人間の肌を表す画像データのサンプリング結果を示している。すなわち、左側の三つのデータは肌を構成する画素の（Ｒ，Ｇ，Ｂ）の値であり、その右方に（Ｒ＋Ｇ＋Ｂ）の合計（ｓｕｍ＿ｒｇｂ）を示し、その右方に上記計算に基づく色度ｒ，ｂと輝度Ｙとを示している。また、図２２は各画素についてｒｂ空間にプロットした場合のグラフを示している。同図に示すように、ＲＧＢデータとしては統一性を見出しにくいようでも、色度としてグラフにプロットしてみると規則性があることが見出される。すなわち、人の肌であれば暗く写っているときも明るく写っているときもあり得るが、それにもかかわらず、図１１に示すように直線状に分布しているのである。同図に示す直線状の分布は、
０．３３＜ｒ＜０．５１
｜０．７４ｒ＋ｂ−０．５７｜＜０．１
なる関係式が成立しているといえるから、各画素についてこの条件があてはめられれば肌色領域に属するものといえる。
【００５５】
また、図２３は同様にして青空を表す画像データのサンプリング結果を示しているとともに図２４は各画素についてｒｂ空間にプロットした場合のグラフを示しており、この場合は肌色の場合よりも変動幅が大きいことを考慮すると、
０．１７＜ｒ＜０．３０
｜１．１１ｒ＋ｂ−０．７０｜＜０．２
なる関係式が成立しているといえる。
図２５は、ステップ１３２の内部でこのような処理を実行するフローチャートを示しており、ステップ１３２ａにて色度を計算し、ステップ１３２ｂとステップ１３２ｃとで肌色や空色であるかを判断し、いずれにも引っかからなければステップ１３２ｄで強い強調処理を実行する。しかし、ステップ１３２ｂとステップ１３２ｃのいずれかで肌色や空色であると判断されると、ステップ１３２ｅで弱い強調処理を実行する。例えば、強調係数Ｅenhance として完成アンシャープマスク２チャンネルのものを利用しても良いし、小さいサイズのアンシャープマスク４１を使用しても良い。
【００５６】
ところで、以上の処理はコンピュータシステム１０を使用しつつ主にソフトウェア的な処理で実現している。しかしながら、本発明は必ずしもソフトウェア的な構成に限るものではなく、ハードウェアによるワイヤロジックで実現することもできる。
図２６は具体的なブロック回路を示しており、入力画像データは平滑化回路５１に入力されてぼかしの処理を経たものと経ていないものとを差分絶対値回路５２に入力し、両者の差分値の絶対値を演算し、トーンカーブ補正回路５３で図１６に示すようなトーンカーブ補正を実施する。次いで、縮小化回路５４では外縁部を狭める処理を実行し、反転回路５５で反転させた後、平滑化回路５６で最後の平滑化処理を実施する。この後、生成されたマスクデータを使用して第一の強調化回路５７で強い強調化を施し、反転回路５８を経たマスクデータを使用して第二の強調化回路５９で弱い強調化を施す。
【００５７】
各回路ではロジック回路でディジタル的に処理すればよいが、一部ではアナログ化して処理しても良い。
このように、デジタルスチルカメラ１１ｂ２等を介して取り込んだディジタルの画像データは、平滑化してぼかしたものと元のものとの差を演算し（ステップ１１２）て鮮鋭化領域を検出した後、トーンカーブ補正や縮小や反転や平滑化処理の各処理（ステップ１１８〜１２２）を経て非鮮鋭領域のマスクデータを完成アンシャープマスク１チャンネルに形成し、同マスクデータを使用して元の画像データに強い強調処理を施すようにした（ステップ１３２）ため、もともと鮮鋭なエッジ部分が過度にシャープになりすぎたり、妙に作り物の感じが強く表れてしまうということがなくなる。この場合、画像調整のための処理も含まれるし、鮮鋭な領域に対して弱い強調処理（ステップ１３４）を実施したりしている。
【図面の簡単な説明】
【図１】 本発明の一実施形態にかかる画像鮮鋭化装置のクレーム対応図である。
【図２】 本発明の一実施形態にかかる画像鮮鋭化装置が適用されるコンピュータシステ
ムのブロック図である。
【図３】 本発明の画像鮮鋭化装置の他の適用例を示す概略ブロック図である。
【図４】 本発明の画像鮮鋭化装置の他の適用例を示す概略ブロック図である。
【図５】 本発明の画像鮮鋭化装置の他の適用例を示す概略ブロック図である。
【図６】 本発明の画像鮮鋭化装置の他の適用例を示す概略ブロック図である。
【図７】 本発明の一実施形態にかかる画像鮮鋭化装置のフローチャートの一部である。
【図８】 同フローチャートの一部である。
【図９】 同フローチャートの一部である。
【図１０】 処理の全体を示す概略ブロック図である。
【図１１】 データの格納状態を示す概念図である。
【図１２】 トーンカーブの一例を示す図である。
【図１３】 平滑化処理に使用するフィルタマスクを示す図である。
【図１４】 画像データの変化過程を示す概念図である。
【図１５】 画像の変化過程を示す概念図である。
【図１６】 トーンカーブの一例を示す図である。
【図１７】 縮小化処理の過程を示す図である。
【図１８】 小サイズのアンシャープマスクを示す図である。
【図１９】 中サイズのアンシャープマスクを示す図である。
【図２０】 大サイズのアンシャープマスクを示す図である。
【図２１】 肌色の画素の画像データと色度と輝度を示す図である。
【図２２】 肌色の画素を色度のグラフで示す図である。
【図２３】 空色の画素の画像データと色度と輝度を示す図である。
【図２４】 空色の画素を色度のグラフで示す図である。
【図２５】 強調処理の変形例を示すフローチャートの一部である。
【図２６】 ハードウェアロジックで実現した画像鮮鋭化装置のブロック図である。
【符号の説明】
１０…コンピュータシステム
１１ａ…スキャナ
１１ａ２…スキャナ
１１ｂ…デジタルスチルカメラ
１１ｂ１…デジタルスチルカメラ
１１ｂ２…デジタルスチルカメラ
１１ｃ…ビデオカメラ
１２…コンピュータ本体
１２ａ…オペレーティングシステム
１２ｂ…ディスプレイドライバ
１２ｂ…ドライバ
１２ｃ…プリンタドライバ
１２ｄ…アプリケーション
１３ａ…フロッピーディスクドライブ
１３ｂ…ハードディスク
１３ｃ…ＣＤ−ＲＯＭドライブ
１４ａ…モデム
１４ａ２…モデム
１５ａ…キーボード
１５ｂ…マウス
１７ａ…ディスプレイ
１７ａ１…ディスプレイ
１７ｂ…カラープリンタ
１７ｂ１…カラープリンタ
１７ｂ２…カラープリンタ
１８ａ…カラーファクシミリ装置
１８ｂ…カラーコピー装置
４１〜４３…アンシャープマスク
５１…平滑化回路
５２…差分絶対値回路
５３…トーンカーブ補正回路
５４…縮小化回路
５５…反転回路
５６…平滑化回路
５７…第一の強調化回路
５８…反転回路
５９…第二の強調化回路[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image sharpening method and an image sharpening apparatus .
[0002]
[Prior art]
When an image is handled by a computer or the like, the image is expressed by a dot matrix pixel, and each pixel is expressed by a gradation value. For example, a photograph is often displayed on a computer screen with pixels of 640 dots in the horizontal direction and 480 dots in the vertical direction.
[0003]
Since each pixel has data representing color and brightness, image processing is performed by changing this data. At this time, a sharpening process for sharply displaying a blurred image is widely performed, and the entire image is uniformly sharpened.
[0004]
[Problems to be solved by the invention]
In the conventional image sharpening device described above, sharpening processing is uniformly applied to the entire image, but originally sharp edges and the like are excessively sharp, and the generated image is strangely shaped. There was a problem that the feeling of was strongly expressed.
[0005]
The present invention has been made in view of the above problems, and an object of the present invention is to provide an image sharpening method and an image sharpening apparatus capable of sharpening with a more natural feeling.
[0006]
[Means for Solving the Problems]
In order to achieve the above object, the invention according to claim 1 is a non-sharp region detection step for detecting a non-sharp region of an original image based on image data in which an image is expressed in multi-tone with dot matrix pixels. An image sharpening process for sharpening relatively strongly the detected portion of the non-sharp area in the image data, and an image data output process for outputting the sharpened image data, the non-sharp area detection process Then, an area having a large difference between the input original image data and the smoothed image data obtained by performing the smoothing process on the image data is detected as a sharp area, and the image sharpening process detects the detected area. The original image data is smoothed by using the unsharp area as a mask while reducing the sharp area and then reversing it into an unsharp gun area, and then the difference from the original image data. It is constituted to sharpen the image based.
[0007]
In the invention according to claim 1 configured as described above, with respect to image data in which an image is expressed in multi-tone with dot matrix pixels, first, in the unsharp area detection step, the input original image data and A tone curve correction is performed on the difference value between the smoothed image data and the smoothed image data, and the detection range of the unsharp area is adjusted, and the unsharp area of the original image is detected. In the sharpening step, the non-sharp area and the detected part in the image data are sharpened relatively strongly. Then, the sharpened image data is output in the image data output process.
[0008]
That is, an unsharp area is detected and the area is sharpened.
The image data may be obtained by acquiring image data already stored in a storage area, or obtained by sequentially inputting image data that is input like a camera. It can be modified as appropriate.
The same applies to the image data output process, where the output destination is a specific recording medium, a temporary storage area such as a memory, or an output destination via a communication line such as a network. good.
[0009]
Further, the image sharpening process itself may be one of a plurality of processes in the image process. In this case, the image data is substantially read / written from / to the image data acquired by the other process. Become.
That is, the input / output of the image data may be in any form as long as it can be processed in the following unsharpened area detecting step and image sharpening step. In addition, the image data may be any image in which the image is expressed in multi-tone with dot matrix pixels, and may be a monochrome image or a color image. In the case of a color image, the coordinate system in the color space is used. There is no particular limitation on the method of taking, the gradation range, and the like.
[0010]
The unsharp region detection step, detecting the unsharp regions of the original image, the specific method is varied, as an example, in the above SL unsharp region detection step, and manpower original image data, the image It is good also as a structure which detects an area | region with a small difference with the smoothed image data which applied the smoothing process with respect to data as an unsharp area | region.
[0011]
Oite on Symbol configuration, while leaving the input image data as original image data, to generate a smoothed image data by applying a smoothing processing on the image data, on the said unsharp regions In the detection step, an area where the difference between the original image data and the smoothed image data is small is detected as an unsharp area.
The smoothing process generates a pixel averaged with pixels around the target central pixel, which can be said to be an expected value of the central pixel when viewed from the peripheral pixels. If the difference from the specific value at the central pixel is to be obtained, the smaller the difference from the expected value, the smaller the change in the pixel, excluding so-called edge or high frequency pixels. Represents an unsharp area.
[0012]
However, even if an unsharp area is known as described above, the difference value itself is not always easy to use in subsequent processing. Therefore, the unsharp area detection step may be configured to adjust the detection range of the unsharp area by correcting the tone curve for the difference value between the original image data and the smoothed image data.
[0013]
With this configuration, tone curve correction is performed for the difference value between the original image data and the smoothed image data. When the difference value is used as it is, there may be a case where the semantic content is greatly different at a certain threshold. Even in such a case, the detection range of the non-sharp area can be adjusted by adding weight to the meaning of the difference value using the tone curve.
Since detection of a non-sharp area and detection of a sharp area are in a relation of front and back, it should be considered that the area to be detected is not necessarily limited to the non-sharp area itself. As an example, in the above SL unsharp region detection step detects a region having a large difference from the above original image data and the smoothed image data as sharpness area, in the image sharpening process, by reversing the same sharpness area Te while the non鮮銃area on the original image data of the unsharp regions as a mask and smoothed, the image may be configured to be sharpened on the basis of the difference between the original raw image data.
[0014]
Oite above Symbol arrangement detects a region having a large difference from the above original image data and the smoothed image data as sharpness area. Therefore, although it appears that the unsharp area is not detected on the surface, since the remaining area is actually the unsharp area, the non-sharp area detecting step detects the unsharp area. become. This inversion state is eliminated by the image sharpening process.
That is, in the image sharpening process, the original image data is smoothed using the non-sharp area as a mask while inverting the sharp area, thereby making the non-sharp area. Further, if the smoothed image is subtracted from the original image, the high-frequency component remains and is sharpened.
[0015]
Adjustment of the boundary between sharpening area and unsharp regions is not limited to the tone curve correction described above, as an example, may be configured to invert from shrinking the upper Symbol detected sharpness regions.
Oite on Symbol configuration, since the first reduction of the detected sharpness regions, it will detect a sharp areas sharp. After this, it reverses, but the sharpness acts to widen the outer edge of the unsharp area.
[0016]
Similarly, as another example for adjusting the boundary between the sharpening area and unsharp regions, above Symbol unsharp region detection process may be configured to apply a smoothing process to the edge of the unsharp regions.
Oite on Symbol configuration, for applying smoothing processing to the edge of the unsharp regions, edge by the exertion is more or less moved in and out, will be adjusted. In this case, it is not always necessary to apply the smoothing process only to the edge portion, and the same can be said to apply the smoothing process to the entire image.
[0017]
Depending on the value of the original image data, the judgment may change even with the same sharpness. For example, when the overall darkness and the overall brightness are considered to be proportional to the degree of change in brightness, it should be determined that the unsharp area is large in the darker side. However, such results are not reasonable made, as suitable examples of the measures, in the above SL unsharp region detection step, pre tone curve correction to the keep in unsharp regions with respect to the original image data It is good also as a structure utilized for the detection of this.
[0018]
Oite on Symbol configuration, since previously the tone curve correction to the original image data, so obtained a reasonable area when detecting said unsharp regions. Moreover, it is naturally possible to make an adjustment consciously without being limited to the purpose of adjusting the brightness.
Although failure of the prior art are overcome by sharpening unsharp regions, this does not necessarily exclude the sharpening sharpening area, as an example, in the above Symbol image sharpening process, also sharp region It is good also as a structure sharpened weakly rather than the said non-sharp area | region.
[0019]
Oite on Symbol configuration, the unsharp regions in the image sharpening process for sharpening, sharpening also sharp region. If the sharpening of the sharp area seems to be stronger than the sharpening of the non-sharp area, there is no difference from the prior art, so it is made weaker than the non-sharp area.
This is effective when the balance with the original sharp region is lost due to the sharpening of the non-sharp region. The degree of sharpening may be determined in advance at a fixed ratio between the non-sharp area and the sharp area, or the sharpness of the sharp area may be adjusted as appropriate according to the specific situation.
[0020]
On the other hand, depending on a specific subject, the observer's line of sight may concentrate and appear rough. In particular, the skin-colored region can be uncomfortable even though it is a good feeling as image data because it is sensitive to helping to be an original subject. Preferable examples in such a case, the upper Symbol image sharpening process may be configured to reduce the sharpening degree for the color region grainy in sharpening.
[0021]
Oite above Symbol arrangement, for reducing the sharpening degree for skin color and sky is the color area grainy in sharpening, it will reduce the inter-roughness. The color region may be determined by chromaticity, and if it is chromaticity, it is not affected by luminance, so that human skin can be detected regardless of whether it is bright or dark. The color region may be other than the skin color or sky blue, and can be adjusted as appropriate.
[0022]
As described above, it is easy to understand that the method for sharpening the non-sharp area is realized in a substantial apparatus, and in that sense, the present invention can also be applied as a substantial apparatus. For this reason, the invention according to claim 2 is a non-sharp area detecting unit that detects an unsharp area of an original image based on image data in which an image is expressed in multi-tone with dot matrix pixels, and the image data An image sharpening means for sharpening relatively strongly the detected portion and the non-sharp area; and an image data output means for outputting the sharpened image data. A region having a large difference between the original image data and the smoothed image data obtained by smoothing the image data is detected as a sharp region. In the image sharpening step, the detected sharp region is detected. The original image data is smoothed by using the unsharp area as a mask while reversing and reversing to make an unsharp gun area, and then the image is based on the difference from the original original image data. There is a configuration in which the sharpening.
That is, there is no difference in that it is effective as a substantial device. Such an image sharpening device may be implemented alone, or may be implemented together with other methods in a state of being incorporated in a certain device. Including embodiments. Therefore, it can be changed as appropriate, such as software or hardware.
[0023]
In the case of software that implements an image sharpening method as an embodiment of the idea of the invention, it naturally exists on a recording medium that records such software, and it must be used.
As an example, a medium recording an image sharpening processing program for sharpening an image by a computer based on the images in the image data multi-gradation pixels in dot matrix, input original image data And a non-sharp region detection step for detecting the non-sharp region while adjusting the detection range of the non-sharp region by correcting the tone curve for the difference value between the smoothed image data obtained by performing the smoothing process on the image data, A configuration is also possible in which the computer executes an image sharpening step for sharpening relatively strongly the detected portion and the non-sharp region in the image data, and an image data output step for outputting the sharpened image data. There is .
Of course, the recording medium may be a magnetic recording medium, a magneto-optical recording medium, or any recording medium that will be developed in the future. In addition, the duplication stages such as the primary duplication product and the secondary duplication product are equivalent without any question.
Further, even when a part is software and a part is realized by hardware, the idea of the invention is not completely different, and a part is stored on a recording medium and is appropriately changed as necessary. It may be in the form of being read. Of course, it goes without saying that the idea of the invention is reflected in the program itself.
[0024]
Of course, such an image sharpening method may be carried out independently or may be carried out together with other methods in a state of being incorporated in a certain device. Various embodiments are included and can be changed as appropriate.
[0025]
【The invention's effect】
As described above, the present invention can provide an image sharpening method capable of sharpening an image with a natural feeling and easily adjusting the detection range of an unsharp area .
[0026]
Furthermore , since the non- sharp area can be detected as a reflective effect for detecting the sharp area instead of directly detecting the non-sharp area, more flexible processing is possible.
Furthermore , the adjustment of the boundary between the non- sharp area and the sharp area can be flexibly performed.
[0027]
The present invention according to claim 2, cut with an image sharpening apparatus to obtain the same effect.
[0028]
DETAILED DESCRIPTION OF THE INVENTION
FIG. 1 shows an image sharpening apparatus according to an embodiment of the present invention by a claim correspondence diagram, and FIG. 2 shows a computer system 10 as an example of hardware for realizing the image sharpening apparatus by a block diagram. Yes. First, the computer system 10 will be described.
The computer system 10 includes a scanner 11 a, a digital still camera 11 b, and a video camera 11 c as image input devices for directly inputting image data, and is connected to a computer main body 12. Each input device can generate image data in which an image is expressed by a dot matrix pixel and output the image data to the computer main body 12. Here, the image data is displayed in 256 gradations for each of the three primary colors of RGB. About 16.7 million colors can be expressed.
[0029]
A floppy disk drive 13a, a hard disk 13b, and a CD-ROM drive 13c as external auxiliary storage devices are connected to the computer main body 12, and main programs related to the system are recorded on the hard disk 13b. Necessary programs and the like can be read from a CD-ROM or the like as appropriate.
In addition, a modem 14a is connected as a communication device for connecting the computer main body 12 to an external network or the like, and it can be connected to an external network via the public communication line, and can be installed by downloading software and data. It has become. In this example, the modem 14a accesses the outside via a telephone line, but it is also possible to adopt a configuration where the network is accessed via a LAN adapter. In addition, a keyboard 15a and a mouse 15b as a pointing device are also connected for operating the computer main body 12.
[0030]
Furthermore, a display 17a and a color printer 17b are provided as image output devices. The display 17a has a display area of 800 pixels in the horizontal direction and 600 pixels in the vertical direction, and the above-described display of 16.7 million colors can be performed for each pixel. Of course, this resolution is only an example, and can be changed as appropriate, such as 640 × 480 pixels or 1024 × 768 pixels.
[0031]
The color printer 17b is an ink jet printer, and can print an image by adding dots on a printing paper as a recording medium using four color inks of CMYK. The image density is capable of high-density printing such as 360 × 360 DPI or 720 × 720 DPI, but the gradation table limit is expressed in two gradations such as whether or not color ink is applied.
On the other hand, in order to display or output to the image output device while inputting an image using such an image input device, a predetermined program is executed in the computer main body 12. Among them, an operating system (OS) 12a is operating as a basic program, and the operating system 12a performs a print output to a display driver (DSP DRV) 12b for displaying on the display 17a and a color printer 17b. A printer driver (PRT DRV) 12c is installed. These types of drivers 12b and 12c depend on the models of the display 17a and the color printer 17b, and can be added to and changed from the operating system 12a according to the respective models. In addition, depending on the model, additional functions beyond standard processing can be realized. That is, various additional processes within an allowable range can be realized while maintaining a common processing system on the standard system called the operating system 12a.
[0032]
The application 12d is executed on the operating system 12a as the basic program. The processing contents of the application 12d are various. The operation of the keyboard 15a and the mouse 15b as operation devices is monitored, and when operated, various external devices are appropriately controlled to execute corresponding arithmetic processing, Furthermore, the processing result is displayed on the display 17a or output to the color printer 17b.
[0033]
By the way, an image photographed by the digital still camera 11b becomes image data, and can be displayed on the display 17a or output to the color printer 17b after various image processing is executed by the application 12d. As an example of such image processing, there is emphasis processing (the sharpening processing is hereinafter referred to as such), and in this embodiment, the application 12d executes the emphasis processing that can obtain an optimum result. As a thing, I will explain below.
[0034]
In the above, the process of acquiring the image data from the image input device or the like constitutes the image data acquisition means A1 shown in FIG. 1, and the hardware and software related thereto are actually applicable. Further, the non-sharp area detection means A2 corresponds to a process of detecting an area with relatively low sharpness in the image data acquired in this way, and is specifically implemented by the application 12d. Further, since the application 12d also performs the sharpening process of the unsharp area in the image data using the detection result, the image sharpening means A3 is also configured.
[0035]
In the present embodiment, the application 12d performs image processing. However, it is naturally possible to adopt a configuration in which sharpening processing is automatically realized when the display driver 12b or the printer driver 12c outputs an image. It is.
Of course, the application 12d, the display driver 12b, and the printer driver 12c for executing such processing are stored in the hard disk 13b, and are appropriately read and operated in the computer main body 12. At the time of introduction, it is recorded on a medium such as a CD-ROM or floppy disk and installed. Therefore, these media constitute a medium on which an image sharpening program is recorded.
[0036]
The application 12d leaves the processed image data in the file format to the subsequent processing process, and the process of outputting the image data in the file format in this way constitutes the image data output means A4. As described above, when the sharpening process is realized when the display driver 12b or the printer driver 12c outputs an image, it can be said that the output stage constitutes the image data output means A4. In the present embodiment, the image sharpening device is realized as the computer system 10, but such a computer system is not necessarily required, and any system that requires interpolation processing for similar image data may be used. . For example, as shown in FIG. 3, an image sharpening device for emphasizing processing is incorporated in the digital still camera 11b1, and the image data subjected to emphasis processing is displayed on the display 17a1 or printed on the color printer 17b1. Also good. As shown in FIG. 4, in the color printer 17b2 that inputs and prints image data without using a computer system, the image data input through the scanner 11a2, the digital still camera 11b2, the modem 14a2, or the like is automatically processed. It is also possible to configure such that the emphasis process is performed. In recent years, such a color printer 17b2 is often used as a video printer to make a hard copy of a scene by connecting to a home television or video, and acquiring image data from a removable recording medium. What is necessary is just to perform the optimal emphasis process in resolution conversion.
[0037]
In addition, the present invention can naturally be applied to various apparatuses that handle image data such as a color facsimile apparatus 18a as shown in FIG. 5 and a color copy apparatus 18b as shown in FIG.
Specifically, the above-described enhancement processing is performed in the computer main body 12 by an image processing program corresponding to the flowcharts shown in FIGS. FIG. 10 schematically shows changes in the processing target in the image processing program. FIG. 10 shows a work area in the image processing program, and each image data composed of RGB three-element color data is processed using a work area assuming individual planes called layers. Further, a work area called a channel is used to store calculation results and the like in order to control each image processing.
[0038]
In the flowcharts shown in FIGS. 7 to 9, in step 100, image data is input. This image data is fetched from the scanner 11a2, digital still camera 11b2, modem 14a2, etc. via the operating system 12a, and the fetched image data is stored in the original image layer in the above-described layer.
Next, the image data of the original image layer is copied to the background layer and the duplicate layer in order to leave the original image and proceed with processing (steps 102 and 104). In the present embodiment, final enhancement processing is added to the image data of the background layer, and image processing for generating a mask for the duplicate layer is performed.
[0039]
Tone curve correction is performed at the beginning of image processing (step 106), and the contrast of the image data stored in the duplicate layer is increased using the tone curve shown in FIG. Here, the significance of performing the process of increasing the contrast will be described later.
In step 108, the image data with increased contrast is copied to the hard copy layer to leave the original, and in step 110, the image data of the hard copy layer is smoothed. In the smoothing process, image data is averaged over a predetermined area centered on the pixel of interest, and is filtered using a filter mask shown in FIG. In this filter processing, all of the image data of the 8 pixels adjacent to the target pixel and the target pixel are added and divided by the number of pixels, so that it is nothing but to obtain an average value. Although the illustrated filter mask is 9 pixels of 3 × 3 pixels, filter masks having different sizes such as 5 × 5 pixels may be used, or smoothing may be performed so as to reduce the weighting of surrounding pixels. .
[0040]
The smoothed image data is stored in the hard copy layer, and in step 112, the absolute value of the difference between the hard copy layer image data and the copy layer image data is calculated, and the calculation result is stored in the alpha channel 1.
FIG. 14 is a reference diagram for explaining the meaning of this series of processing. Originally, two-dimensional image data is rearranged in an easy-to-understand one-dimensional manner. If the image data of the duplicate layer is as shown in FIG. 5A, the stepped portion becomes smooth as shown in FIG. Next, when the difference between the two is calculated, a difference value is generated in the pixel which has been smoothed and changed as shown in FIG. 6C, and the absolute value (shown by a one-dot chain line in the figure) becomes large. The image data of the layer is changing greatly. The portion where the absolute value is large is where the image is greatly changed, and this series of processing is nothing but detecting a sharp region of the image. Further, increasing the contrast of the image data of the duplicate layer in step 106 contributes to increasing the above-described difference value, making it easy to detect a sharp region. In this regard, if the sharp area can be detected first, the adjustment can be arbitrarily performed thereafter. However, if the sharp area is detected at the stage of detecting the sharp area, the room for adjustment is reduced. Therefore, the sharp region is detected in such a wide manner.
[0041]
FIG. 15 illustrates the meaning of the above-described processing with a specific image. Assuming that FIG. 5A is the original image, the smoothing process causes a change in the image data of the contour portion as shown in FIG. 5B, and the other portions remain in the original color. Become. Therefore, when the difference value from the original image is obtained, only the region centered on the contour of the original image remains as shown in FIG.
[0042]
Since alpha channel 1 is a general-purpose channel, it is stored in the unsharp mask original channel and the unsharp mask intensified channel in order to store the absolute value of the difference value (steps 114 and 116). Then, we will make adjustments to the unsharp mask contrast enhancement channel to improve the concrete calculation results.
First, in step 118, tone curve correction is performed using the tone curve shown in FIG. Even if the difference value corresponding to the sharpening of the image is obtained as shown in FIG. 14C, the data itself is not always easy to use for the calculation. In particular, since the absolute value of the difference value itself is only a small value, it is necessary to make it larger. The purpose shown in FIG. 15 is to increase a small absolute value proportionally, but adopt a steep S-curve that sets a certain threshold and does not change much below it. Is also possible.
[0043]
Next, in step 120, the unsharp mask contrast enhancement channel is reduced. The reduction processing is processing for narrowing the line width as an actual image, and narrows the region by one pixel (one pixel) as shown in FIG. As described above, in step 118, pixels that may be dropped depending on the setting of the threshold value are picked up. Therefore, in step 120, such reduction is performed in order to obtain an appropriate range. A specific image changes from FIG. 15C to FIG. Of course, this is the range of tuning and can be changed as appropriate.
[0044]
Although the sharp region has been adjusted as described above, since it is the non-sharp region that is mainly sought in the present invention, in step 122, the unsharp mask contrast enhancement channel is inverted. In the inversion process, the value of the image data is simply subtracted from 255. Therefore, “255” changes to “0”, “200” changes to “55”, “1” changes to “254”, and so on. The inverted state is as shown in FIG. 15 (e), and a mask having some value is formed in a region other than the contour portion in the initial drawing (a). Then, as the last of the mask processing, in step 124, the unsharp mask contrast enhancement channel is smoothed. Thereby, it can be expected that the boundary portion in the mask image shown in FIG. 15E becomes smooth, and the jump is not generated in the image in the enhancement processing to be performed next.
[0045]
Then, the completed mask is copied to the completed unsharp mask 1 channel and the completed unsharp mask 2 channel (step 126, step 128), and the inversion processing is performed for the completed unsharp mask 2 channel (step 130). By performing the inversion process in this way, a sharp area is detected again. In the present embodiment, strong enhancement processing is performed on the unsharp area, while weak enhancement processing is performed on the unsharp area, so that the sharp area is detected again as the completed unsharp mask 2 channel.
[0046]
In the enhancement process, an unsharp mask shown in FIGS. 18 to 19 is used. Here, the enhancement processing performed in steps 132 and 134 will be described using luminance as an example.
The luminance Y ′ after enhancement with respect to the luminance Y of each pixel before enhancement is
Y '= Y + Eenhance (Y-Yunsharp)
Is calculated as This Yunsharp is obtained by applying unsharp mask processing to the image data of each pixel, and the enhancement coefficient Eenhance is normalized by dividing the completed unsharp mask 1 channel and the completed unsharp mask 2 channel by “255”. It is the value.
[0047]
Here, the unsharp mask process will be described. 18 to 20 show three unsharp masks 41 to 43 having different sizes. The unsharp masks 41 to 43 use the value “100” in the center as the weighting of the pixel Y (x, y) to be processed in the matrix-like image data, and correspond to the numerical values in the squares of the mask for the peripheral pixels. It is used to add up the weights. Now, if the unsharp mask 42 shown in FIG. 19 is used,
[0048]
[Expression 1]

Is integrated based on the following equation. In the equation, “632” is a total value of the weighting coefficients, and of course the values of “396”, “632”, and “2516” are obtained for the three unsharp masks 41 to 43 having different sizes. Mij is a weighting factor written in the grid of the unsharp mask, and Y (x, y) is image data of each pixel. Note that ij is represented by horizontal and vertical coordinate values for three unsharp masks 41 to 43 having different vertical and horizontal sizes.
[0049]
The meaning of such an operation is as follows. Yunsharp (x, y) is obtained by lowering the weight of the peripheral pixel with respect to the target pixel and adding it, so that it is so-called “unsharp” image data. What is smoothed in this way has the same meaning as that obtained by applying a so-called low-pass filter. Therefore, “Y (x, y) −Yunsharp (x, y)” has the same meaning as that obtained by applying the high-pass filter by subtracting the low frequency component from the original all components. If the high frequency component that has passed through the high-pass filter is multiplied by the enhancement coefficient Eenhance and added to “Y (x, y)”, the high frequency component is increased in proportion to the enhancement coefficient Eenhance, and the edge is increased. The result is emphasized.
[0050]
Also, the degree of edge enhancement varies depending on the size of the unsharp mask. In the case of three unsharp masks 41 to 43 having different numbers of vertical and horizontal grids, the larger the mask, the greater the weighting of the neighboring pixels of the pixel of interest, and the weighting gradually decreases in the distance to the distant pixels. The In other words, this is because the characteristics as a low-pass filter become stronger and it becomes easier to generate a high-frequency component.
[0051]
Therefore, in addition to the adjustment of the degree of enhancement by the enhancement coefficient Eenhance, strong enhancement processing is performed if the large-size unsharp mask 43 is used, and weak enhancement processing is performed if the small-size unsharp mask 41 is used. It will be. Of course, if an intermediate strength enhancement process is performed, an intermediate-size unsharp mask 42 may be used.
According to the flowchart of FIG. 9, in order to perform strong enhancement processing in the unsharp area in step 132, calculation is performed using the unsharp mask 43 while using the completed unsharp mask 1 channel as an enhancement coefficient. In order to perform weak emphasis processing in the sharp region at 134, the calculation is performed using the unsharp mask 41 and the completed unsharp mask 2 channel as an emphasis coefficient. Note that the filter masks of the unsharp masks 41 to 43 are merely examples, and can be changed as appropriate. Further, as can be seen from FIG. 19, the outermost peripheral parameter is “0” or “1”, and even if the pixel image data is multiplied by “632”, there is almost no influence. Meaning. For this reason, if an unsharp mask 44 filter mask having 5 × 5 pixels ignoring the outermost peripheral parameters is used, the number of division operations “49 (= 7 × 7)” to “25 (= 5 × 5 ×) is used. 5) "can be halved and the processing time can be shortened.
[0052]
Through the above processing, even digital image data captured via the digital still camera 11b2 or the modem 14a2 or the like, the sharp edge portion is originally too sharp, There is no longer a strong feeling.
The image data itself obtained in this way is stored in the background layer, and when this image data is output to the display 17a or the color printer 17b via the display driver 12b or the printer driver 12c, a beautiful image is obtained. .
[0053]
In general, this enhancement process increases the sharpness of the image with a generally natural feeling, but the human skin may be sharpened and look rough. This is because the person himself / herself is a photographic object, and particularly pays attention to that portion. The same thing happens for empty areas that occupy a large area in a photograph. In such a case, it may be determined whether each pixel is a skin color or sky blue, and if it is a skin color or sky blue, the enhancement process may be weakened. Here, a method for determining whether or not each pixel is a skin color or sky blue will be described.
[0054]
If the image data of each pixel is represented by (R, G, B), the chromaticity is
r = R / (R + G + B)
b = B / (R + G + B)
Represented as:
FIG. 21 shows a sampling result of image data representing human skin. That is, the three data on the left are the values of (R, G, B) of the pixels constituting the skin, and the sum (sum_rgb) of (R + G + B) is shown on the right, and the color based on the above calculation is shown on the right. Degrees r, b and luminance Y are shown. FIG. 22 shows a graph when each pixel is plotted in the rb space. As shown in the figure, even if it is difficult to find unity as RGB data, it is found that there is regularity when plotted on a graph as chromaticity. In other words, human skin may appear dark or bright, but nevertheless, it is distributed linearly as shown in FIG. The linear distribution shown in the figure is
0.33 <r <0.51
| 0.74r + b−0.57 | <0.1
Therefore, if this condition is applied to each pixel, it can be said that it belongs to the skin color region.
[0055]
Similarly, FIG. 23 shows the sampling result of the image data representing the blue sky, and FIG. 24 shows a graph when each pixel is plotted in the rb space. In this case, the fluctuation range is larger than that in the case of skin color. Considering that is large,
0.17 <r <0.30
| 1.11r + b−0.70 | <0.2
It can be said that the following relational expression holds.
FIG. 25 shows a flowchart for executing such processing in step 132. In step 132a, the chromaticity is calculated. In steps 132b and 132c, it is determined whether the skin color or sky blue. If this is not the case, strong emphasis processing is executed in step 132d. However, if the skin color or sky blue color is determined in either step 132b or step 132c, weak enhancement processing is executed in step 132e. For example, a two-channel completed unsharp mask may be used as the enhancement coefficient Eenhance, or an unsharp mask 41 having a small size may be used.
[0056]
By the way, the above processing is realized mainly by software processing while using the computer system 10. However, the present invention is not necessarily limited to a software configuration, and can also be realized by hardware wire logic.
FIG. 26 shows a specific block circuit. The input image data is input to the smoothing circuit 51 and is subjected to the blurring process and is not input to the difference absolute value circuit 52. The tone curve correction circuit 53 performs tone curve correction as shown in FIG. Next, the reduction circuit 54 executes a process of narrowing the outer edge, and after the inversion circuit 55 inverts, the smoothing circuit 56 performs the final smoothing process. Thereafter, strong emphasis is given by the first emphasis circuit 57 using the generated mask data, and weak emphasis is given by the second emphasis circuit 59 using the mask data passed through the inversion circuit 58. .
[0057]
Each circuit may be digitally processed by a logic circuit, but some may be processed in an analog manner.
In this way, the digital image data captured via the digital still camera 11b2 or the like calculates the difference between the smoothed and blurred image and the original image (step 112), detects the sharpened area, After the curve correction, reduction, inversion, and smoothing processes (steps 118 to 122), the mask data of the unsharp area is formed in the completed unsharp mask 1 channel, and the mask data is used to restore the original image data. Since strong emphasis processing is performed (step 132), the originally sharp edge portion is not excessively sharpened, and the feeling of artifacts does not appear strongly. In this case, processing for image adjustment is included, and weak enhancement processing (step 134) is performed on a sharp region.
[Brief description of the drawings]
FIG. 1 is a view corresponding to claims of an image sharpening device according to an embodiment of the present invention.
FIG. 2 is a block diagram of a computer system to which an image sharpening apparatus according to an embodiment of the present invention is applied.
FIG. 3 is a schematic block diagram illustrating another application example of the image sharpening device of the present invention.
FIG. 4 is a schematic block diagram showing another application example of the image sharpening device of the present invention.
FIG. 5 is a schematic block diagram illustrating another application example of the image sharpening device of the present invention.
FIG. 6 is a schematic block diagram illustrating another application example of the image sharpening device of the present invention.
FIG. 7 is a part of a flowchart of an image sharpening apparatus according to an embodiment of the present invention.
FIG. 8 is a part of the flowchart.
FIG. 9 is a part of the flowchart.
FIG. 10 is a schematic block diagram showing the entire processing.
FIG. 11 is a conceptual diagram showing a data storage state.
FIG. 12 is a diagram illustrating an example of a tone curve.
FIG. 13 is a diagram showing a filter mask used for smoothing processing.
FIG. 14 is a conceptual diagram showing a change process of image data.
FIG. 15 is a conceptual diagram illustrating a process of changing an image.
FIG. 16 is a diagram illustrating an example of a tone curve.
FIG. 17 is a diagram illustrating a process of a reduction process.
FIG. 18 is a diagram showing a small-sized unsharp mask.
FIG. 19 is a diagram showing a medium-sized unsharp mask.
FIG. 20 is a view showing a large-sized unsharp mask.
FIG. 21 is a diagram illustrating image data, chromaticity, and luminance of a flesh color pixel.
FIG. 22 is a diagram illustrating skin-colored pixels in a chromaticity graph.
FIG. 23 is a diagram illustrating image data, chromaticity, and luminance of sky blue pixels.
FIG. 24 is a diagram illustrating a sky blue pixel by a graph of chromaticity.
FIG. 25 is a part of a flowchart showing a modification of the enhancement process.
FIG. 26 is a block diagram of an image sharpening device realized by hardware logic.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 10 ... Computer system 11a ... Scanner 11a2 ... Scanner 11b ... Digital still camera 11b1 ... Digital still camera 11b2 ... Digital still camera 11c ... Video camera 12 ... Computer main body 12a ... Operating system 12b ... Display driver 12b ... Driver 12c ... Printer driver 12d ... Application 13a ... Floppy disk drive 13b ... Hard disk 13c ... CD-ROM drive 14a ... Modem 14a2 ... Modem 15a ... Keyboard 15b ... Mouse 17a ... Display 17a1 ... Display 17b ... Color printer 17b1 ... Color printer 17b2 ... Color printer 18a ... Color facsimile machine 18b ... Color copy device 41-43 ... Anscher Mask 51 ... Smoothing circuit 52 ... Difference absolute value circuit 53 ... Tone curve correction circuit 54 ... Reduction circuit 55 ... Inversion circuit 56 ... Smoothing circuit 57 ... First enhancement circuit 58 ... Inversion circuit 59 ... Second enhancement Circuit

Claims (2)

A non-sharp area detection step for detecting a non-sharp area of the original image based on image data in which an image is expressed in multi-tone with a dot matrix pixel;
An image sharpening step for sharpening relatively strongly about the non-sharp area and the detected part in the image data;
An image data output step for outputting sharpened image data,
In the non-sharp area detection step, an area having a large difference between the input original image data and the smoothed image data obtained by smoothing the image data is detected as a sharp area,
In the image sharpening step, the detected sharp area is reduced and then inverted to form a non-sharp gun area, the original image data is smoothed using the non-sharp area as a mask, and then the original original image An image sharpening method comprising sharpening an image based on a difference from data . A non-sharp area detecting means for detecting a non-sharp area of the original image based on image data in which an image is expressed in multi-tone with a dot matrix pixel;
Image sharpening means for sharpening relatively strongly about the non-sharp region and the detected portion in the image data;
Image data output means for outputting sharpened image data,
The non-sharp area detection means detects an area having a large difference between the input original image data and the smoothed image data obtained by smoothing the image data as a sharp area,
The image sharpening means reduces the detected sharp area and then inverts it to make a non-sharp gun area, smooths the original image data using the non-sharp area as a mask, and then the original original image. An image sharpening apparatus characterized by sharpening an image based on a difference from data .

Images