JP3882427B2 - Image processing apparatus and image processing method - Google Patents

Description

【００３０】
【数２】

【００３１】
【数３】

【００３２】
差分信号ΔＲ，ΔＧ，ΔＢを生成すると、続いて、線分画素抽出部１４ａは、これらの差分信号ΔＲ，ΔＧ，ΔＢを用いて色検出信号「Ｄ」を生成する（Ｓ２０２）。色検出信号Ｄの生成は、以下の（４）式を用いて行う。
【００３３】
【数４】

【００３４】
ここで算出される色検出信号Ｄは、注目画素データ値Ｒ，Ｇ，Ｂと抽出対象色の色情報値Ｒｉ，Ｇｉ，Ｂｉとの間におけるＲＧＢ色空間上の距離を表すものである。すなわち、色検出信号Ｄは、ＲＧＢ色空間における各画素の有する色と抽出対象色との距離を示す信号に相当し、その値が大きくなるほど抽出対象色に近い色であることを表している。
【００３５】
なお、ここでは、注目画素データ値Ｒ，Ｇ，Ｂと抽出対象色の色情報値Ｒｉ，Ｇｉ，Ｂｉとの間のユークリッド距離、すなわち差分信号ΔＲ，ΔＧ，ΔＢの二乗和の平方根を色検出信号Ｄとして定義しているが、例えば差分信号ΔＲ，ΔＧ，ΔＢの平均値など、２点間の隔たりの尺度となる演算でこれを代替することも可能であることはいうまでもない。
【００３６】
色検出信号Ｄを求めると、次いで、線分画素抽出部１４ａは、求めた色検出信号Ｄを基に線分抽出を行う。線分抽出は、周知の手法を用いて行えばよい。例えば、注目画素周辺で階調レベルの変化が急峻であり、かつ、注目画素の階調レベルが十分に高い場合に、その注目画素を線分要素とある判断することが考えられる。
【００３７】
具体的には、線分画素抽出部１４ａは、先ず、求めた色検出信号Ｄを用いて、色検出信号Ｄのエッジ特徴量「Ｄｅ」およびレベル特徴量「Ｄｌ」を算出する（Ｓ２０３，Ｓ２０４）。エッジ特徴量Ｄｅおよびレベル特徴量Ｄｌは、それぞれ以下の（５）式および（６）式のように定義される。
【００３８】
【数５】

【００３９】
【数６】

【００４０】
ここで、エッジ特徴量Ｄｅは、図４の注目画素近傍における画素配置の説明図に示すように、注目画素を「Ｄｊ，ｋ」とすると、注目画素周辺の５×５（主走査×副走査）画素の平均値と注目画素周辺の３×３（主走査×副走査）画素の平均との差分の絶対値に相当する。また、レベル特徴量Ｄｌは、注目画素周辺の３×３（主走査×副走査）画素の平均値に相当する。
なお、（５）式および（６）式中における「Ｍ」および「Ｎ」は、調整用のオフセットパラメータであり、画像処理装置の特性に応じて予め実験的に定められている定数である。
【００４１】
エッジ特徴量Ｄｅおよびレベル特徴量Ｄｌを求めると、次いで、線分画素抽出部１４ａは、これらエッジ特徴量Ｄｅおよびレベル特徴量Ｄｌを、それぞれ予め設定されているしきい値ＴＨｅ，ＴＨｌと比較する。そして、エッジ特徴量Ｄｅがしきい値ＴＨｅ以上であり、かつ、レベル特徴量Ｄｌしきい値ＴＨｌ以上であるか否かを判定する（Ｓ２０５）。
【００４２】
この判定の結果、Ｄｅ≧ＴＨｅかつＤｌ≧ＴＨｌであれば、線分画素抽出部１４ａは、注目画素Ｄｊ，ｋ周辺で階調レベルの変化が急峻であり、かつ、注目画素の階調レベルが十分に高いので、注目画素Ｄｊ，ｋが線分を構成する画素であると判断する。そして、注目画素データ値Ｒ，Ｇ，Ｂに対する出力画素データ値ＯＵＴをＯＮとする（Ｓ２０６）。その一方で、Ｄｅ≧ＴＨｅかつＤｌ≧ＴＨｌでない場合には、線分画素抽出部１４ａは、出力画素データ値ＯＵＴをＯＦＦとする（Ｓ２０７）。
【００４３】
このように、線分画素抽出部１４ａでは、画像入力部１１が取得したデジタルカラー画像データを構成する各画素データに対して、抽出対象色の色情報値Ｒｉ，Ｇｉ，Ｂｉとの間の距離を表す色検出信号Ｄを基に二値化を行う。これにより、線分画素抽出部１４ａからは、抽出対象色の線分要素に該当する画素データ、すなわち抽出対象色の色成分に係る線分データが抽出され、不要画素除去部１４ｂに対して出力されることになる。
【００４４】
なお、ここでは、線分画素抽出部１４ａが比較的簡便な手法によって線分抽出を行う場合を例に挙げたが、本発明は特にこれに限定されるものではない。例えば、本願出願人が既にした特許出願（特願平１１−１６６６７６号）で提案しているように、所定の図形形状要素に相当する線分基本要素によりデジタルカラー画像データを走査することで、当該線分基本要素に対応する幅の線分を構成する画素データを抽出すれば、より高精度な線分抽出を行うことが可能となる。
【００４５】
次に、本実施形態の画像処理装置による処理動作のうち、不要画素除去部１４ｂが行う不要画素データの除去処理（図２におけるＳ１０６）について詳しく説明する。
図５および図６は、不要画素除去部１４ｂが行う処理動作の概要を示す説明図である。なお、図５中の矩形内は、線分画素抽出部１４ａから出力される二値化後の出力画素データＯＵＴをイメージ化した状態を表している。
【００４６】
不要画素除去部１４ｂが行う処理動作は、周知の文書構造認識／像域分離手法に基づくものである。先ず、不要画素除去部１４ｂは、図５（ａ）に示すように、二値化画像イメージに対し垂直方向および水平方向についての投影分布を作成し、頻度「０」の分布が最も長いポイントで領域を２分割する。さらに、不要画素除去部１４ｂは、図５（ｂ），（ｃ）に示すように、分割した２つの領域のそれぞれに対しても同様の領域分割処理を行う。そして、この処理を各領域が文書を構成する最小単位となるまで繰り返す。
【００４７】
その後、不要画素除去部１４ｂは、図６（ａ）に示すように、分割した各領域内で画素データ値がＯＮである部分について、その外接矩形を生成する。外接矩形を生成すると、不要画素除去部１４ｂは、各外接矩形について特徴量を算出し、その特徴量を基に、各外接矩形が文字要素に相当するものであるか、或いは網点領域や罫線など文字要素以外の文書構成要素や誤って抽出されたノイズ成分に相当するものであるかを判断する。ここで、特徴量としては、外接矩形の面積、縦横比、外接矩形内におけるデータ値がＯＮである部分の面積比、ＯＮ／ＯＦＦ反転回数等を用いることが考えられる。例えば、外接矩形の面積を用いた場合には、文字要素の大きさを表すポイント数を基準に、大きすぎるものや小さすぎるものを文字要素でないと判断する。これらの特徴量を用いた判断の結果、文字画素ではないと判断された外接矩形に属する画素データについては、不要画素除去部１４ｂが図６（ｂ）に示すようにその値をＯＮからＯＦＦに変換する。
【００４８】
そして、不要画素除去部１４ｂは、全ての外接矩形に対して処理を終了すると、その処理結果をワークメモリ１４ｄ内に保持記憶させる。このような不要画素データの除去処理を、不要画素除去部１４ｂがＮ個の抽出対象色すべてに対して実行することにより、ワークメモリ１４ｄ内には、Ｎ個の各抽出対象色について、当該抽出対象色に係る線分データのうち、文字要素を構成する線分データのみが保持記憶されることになる。
【００４９】
次に、本実施形態の画像処理装置による処理動作のうち、画像合成部１４ｃが行うＮ色分の抽出処理結果の合成処理（図２におけるＳ１１０）について詳しく説明する。
図７は、画像合成部１４ｃが行う処理動作の概要を示す説明図である。なお、図中では、簡単のため、Ｎ＝３で、「黒」、「白」、「赤」の３色が抽出対象色として設定されている場合の例を示す。
【００５０】
抽出対象色が「黒」、「白」、「赤」の３色である場合、線分画素抽出部１４ａによる線分抽出処理および不要画素除去部１４ｂによる不要画素データの除去処理が完了すると、ワークメモリ１４ｄ内には、図７中に▲１▼で示すような黒色成分に係る二値化線分データと、同じく▲２▼で示すような白色成分に係る二値化線分データと、同じく▲３▼で示すような赤色成分に係る二値化線分データとが、それぞれ個別に蓄積される。ここで、画像合成部１４ｃは、各色成分に係る二値化線分データ▲１▼，▲２▼，▲３▼をワークメモリ１４ｄ内から取り出し、これらに対する合成処理を行い、その合成処理結果を文字画像蓄積部１５に出力してそこに保存させる。
【００５１】
合成処理は、例えば、画像合成部１４ｃが各色成分に係る二値化線分データ▲１▼，▲２▼，▲３▼の論理和を取ることによって行い、これにより最終的な画像処理部１４による抽出処理結果、すなわち抽出対象色の文字要素を構成する二値画像データを得る（図７（ａ）参照）。
【００５２】
ただし、合成処理は、以下のように行ってもよい。
例えば、図７（ｂ）に示すように、各色成分に係る二値化線分データ▲１▼，▲２▼，▲３▼に、それぞれの色情報▲４▼を付加した状態で、これらを合成することが考えられる。このときの色情報▲４▼としては、例えば色情報記憶部１２に設定されている色No. 等が適当である。このような合成処理を行えば、最終的な画像処理部１４による抽出処理結果として、各文字要素の色情報が付加された多層二値画像データを得ることが可能である。
さらには、例えば図７（ｃ）に示すように、二値画像データのヘッダ情報として、色情報▲４▼を付加した２ビットのカラーマップを生成することも考えられる。このとき、色情報の上位１ビットは該当する画素データを表し、下位２４ビットはその色成分を表すものとする。このような合成処理を行えば、最終的な画像処理部１４による抽出処理結果として、各文字要素の色情報が付加された多値パレット画像データを得ることが可能である。
【００５３】
このように、色情報を付加した合成処理を行った場合には、その後に行う後処理に対して大きな付加価値を与えることができる。例えば、文字画像蓄積部１５に蓄積される文字画像データについて、後処理として文字認識処理を行う場合に、「黒い文字」や「赤い文字」といった原稿画像上で当該文字が有していた属性を利用することが可能となる。そのため、特にＨＴＭＬ（Hyper Text Markup Language）形式等へのデータ変換を行う場合などには、非常に有効なものとなる。
【００５４】
以上のような処理を行った場合の従来技術に対する優位点を図８〜図１１を用いて説明する。図８〜図１１は、いずれも、従来技術の項で説明した図１７と同様に、文書画像の構成要素を示した説明図である。
【００５５】
例えば、図８（ａ）に示すのような一様でない背景を有する文字については、適切に抽出対象色を設定して線分抽出処理を行うことによって、図８（ｂ）に示す二値画像を表す線分データと、図８（ｃ）に示す二値画像を表す線分データとに、分離して抽出することが可能となる。また、これらの線分データに対して不要画素データの除去処理を行うことによって、最終的には図８（ｂ）に示すような文字画像のみを表す線分データを抽出することが可能となる。
【００５６】
図９は、反転文字の抽出の様子を示している。通常の明度成分に基づく線分抽出処理を行う場合には、図９（ａ）のような周辺画素よりも高い明度値（周辺よりも明るい画素データ値）を持つ文字要素を、それ以外の色の文字と同一の処理論理で抽出することが困難である。しかしながら、本実施形態の画像処理装置および画像処理方法によれば、予め設定された抽出対象色からのユークリッド距離（隔たり）に着目して線分データの抽出処理を行うので、図９（ｂ）および（ｃ）に示したように反転した文字列でも良好に抽出することができる。また、図８に示した例と同様に、図９（ｃ）の枠線も文字以外の要素として除去されるため、最終的には図９（ｂ）に示すような文字画像のみを表す線分データを抽出することが可能となる。
【００５７】
図１０は、背景を有する文字の抽出の様子を示している。図１０（ａ）に示すように背景色を有する領域に文字が存在する場合については、その背景色が抽出対象色でなければ抽出されることがないので、図１０（ｂ）に示すように文字画像のみを表す線分データの抽出が可能となる。
【００５８】
図１１は、図８と場合と同様に、一様でない背景を有する文字の抽出の様子を示している。図１１（ａ）に示すように、背景がグラデーションで表現されるような場合には、明度の差異だけで文字抽出を行うのが非常に困難であるが、本実施形態の画像処理装置および画像処理方法では、抽出すべき文字の色に着目しているため、図１１（ｂ）に示すような良好な文字抽出が可能となる。また、図１１（ｃ）に示すように、抽出される文字以外の成分に関しては、不要画素データの除去処理を行うことによって消去することが可能である。
【００５９】
以上に説明したように、本実施形態の画像処理装置および画像処理方法は、請求項１または６に記載の発明の如く、抽出すべき文字の色に着目し、抽出対象色に係る線分データのみを抽出するようになっている。したがって、文字要素を抽出すべき文書画像中に多くの色成分が含まれていても、従来よりも良好かつ的確に、文字要素を構成する線分データの抽出を行うことができる。
【００６０】
また、本実施形態の画像処理装置および画像処理方法は、請求項２または７に記載の発明の如く、抽出した線分データの中から文字要素を構成しないと判断されるデータを除去するようになっている。したがって、文字要素を抽出すべき文書画像が多様化・複雑化しても、これに対応しつつ、従来よりも良好かつ的確に、文字要素を構成する線分データの抽出を行うことができる。
【００６１】
さらに、本実施形態の画像処理装置および画像処理方法は、請求項３または８に記載の発明の如く、複数の抽出対象色にも対応し得るようになっている。したがって、文書画像の多様化・複雑化への対応が容易となる。しかも、複数の抽出対象色について順次線分抽出処理を行うことによって、それぞれの色に対する抽出範囲を従来の場合よりもより狭く設定できるので、例えば網点を表す画素データやエッジ部のデータなどの誤抽出を大きく軽減することができるようになる。
【００６２】
〔第２の実施の形態〕
次に、本発明の第２の実施の形態について説明する。ただし、ここでは、上述した第１の実施の形態との相違点についてのみ説明するものとする。
【００６３】
本実施形態は、上述した第１の実施の形態の場合と、線分画素抽出部１４ａによる線分抽出処理のみが異なる。詳しくは、第１の実施の形態では、ＲＧＢ色空間における注目画素データ値Ｒ，Ｇ，Ｂと抽出対象色の色情報値Ｒｉ，Ｇｉ，Ｂｉとの間の三次元的距離を、抽出対象色と注目画素色との隔たりを表す尺度としたのに対し、本実施形態では、注目画素データ値Ｒ，Ｇ，Ｂとと抽出対象色それぞれの原色成分との間の一次元的距離を色検出用信号とし、これを用いて各原色成分毎の抽出結果の論理積を取ることで、抽出対象となる色線分を有した線分データの抽出を行う点で異なる。
【００６４】
以下、図１２を用いて、本実施形態における線分抽出処理を詳しく説明する。図１２は、線分画素抽出部１４ａが行う処理動作の他の例を示すフローチャートである。なお、第１の実施の形態の場合（図３参照）と同様に、図１２中において、「Ｒ，Ｇ，Ｂ」は画像入力部１１で取得したデジタルカラー画像データの注目画素データ値を、「Ｒｉ，Ｇｉ，Ｂｉ」は色情報記憶部１２からロードされる抽出対象色の色情報値を、「ＯＵＴ」は線分画素抽出部１４ａによる抽出結果である２値化された画素データを表している。
【００６５】
図例のように、本実施形態の場合、線分画素抽出部１４ａは、線分抽出処理を行うのにあたって、先ず、注目画素データの赤成分値Ｒと抽出対象色の赤成分値Ｒｉとの差分信号「ΔＲ」を生成する（Ｓ３０１）。差分信号ΔＲの生成は、第１の実施の形態の場合と同様に（１）式を用いて行う。
【００６６】
ここで、差分信号ΔＲは、注目画素データの赤成分値Ｒと抽出対象色の赤成分値Ｒｉとの間におけるＲ軸上の距離を表すものであり、値が大きくなるほど注目画素の赤成分が抽出対象色の赤成分に近い色であることを表している。
【００６７】
差分信号ΔＲを生成すると、続いて、線分画素抽出部１４ａは、その差分信号ΔＲを基に線分抽出を行う。線分抽出は、周知の手法を用いて行えばよい。例えば、第１の実施の形態で説明した場合と同様に、注目画素周辺で階調レベルの変化が急峻であり、かつ、注目画素の階調レベルが十分に高い場合に、その注目画素を線分要素とある判断することが考えられる。
具体的には、線分画素抽出部１４ａは、先ず、生成した差分信号ΔＲを用いて、そのエッジ特徴量「Ｒｅ」およびレベル特徴量「Ｒｌ」を算出する（Ｓ３０２，Ｓ３０３）。エッジ特徴量Ｒｅおよびレベル特徴量Ｒｌは、それぞれ以下の（７）式および（８）式のように定義される。
【００６８】
【数７】

【００６９】
【数８】

【００７０】
ここで、エッジ特徴量Ｒｅは、注目画素を「Ｒｊ，ｋ」とすると、注目画素周辺の５×５（主走査×副走査）画素の平均値と注目画素周辺の３×３（主走査×副走査）画素の平均との差分の絶対値に相当する。また、レベル特徴量Ｒｌは、注目画素周辺の３×３（主走査×副走査）画素の平均値に相当する（いずれも図４参照）。
なお、（７）式および（８）式中における「Ｍｒ」および「Ｎｒ」は、調整用のオフセットパラメータであり、画像処理装置の特性に応じて予め実験的に定められている定数である。
【００７１】
こうしてエッジ特徴量Ｒｅおよびレベル特徴量Ｒｌを求めると、、次いで、線分画素抽出部１４ａは、これらエッジ特徴量Ｒｅおよびレベル特徴量Ｒｌを、それぞれ予め設定されているしきい値ＴＨｅ，ＴＨｌと比較する。そして、エッジ特徴量Ｒｅがしきい値ＴＨｅ以上であり、かつ、レベル特徴量Ｒｌしきい値ＴＨｌ以上であるか否かを判定する（Ｓ３０４）。
【００７２】
この判定の結果、Ｒｅ≧ＴＨｅかつＲｌ≧ＴＨｌであれば、線分画素抽出部１４ａは、注目画素Ｒｊ，ｋ周辺で階調レベルの変化が急峻であり、かつ、注目画素の階調レベルが十分に高いので、注目画素Ｒｊ，ｋが線分を構成する画素であると判断する。そして、判定結果信号「Ｒｏ」をＯＮとする（Ｓ３０５）。その一方で、Ｒｅ≧ＴＨｅかつＲｌ≧ＴＨｌでない場合には、線分画素抽出部１４ａは、判定結果信号ＲｏをＯＦＦとする（Ｓ３０６）。
【００７３】
以上のような処理を、線分画素抽出部１４ａは、注目画素データの赤成分値Ｒについてのみならず、注目画素データの緑成分値Ｇおよび青成分値Ｂに対しても全く同様に実施し、それぞれの判定結果信号ＧｏおよびＢｏを得る（Ｓ３０７〜Ｓ３１８）。
【００７４】
こうして判定結果信号Ｒｏ，Ｇｏ，Ｂｏを得ると、線分画素抽出部１４ａは、それぞれの判定結果信号Ｒｏ，Ｇｏ，Ｂｏを参照して、これら全ての信号がＯＮであるか否かを判断し（Ｓ３１９）、全てがＯＮであれば出力画素データ値ＯＵＴをＯＮとし（Ｓ３２０）、そうでなければ出力画素データ値ＯＵＴをＯＦＦとする（Ｓ３２１）。
【００７５】
このように、線分画素抽出部１４ａでは、各原色成分毎に抽出対象色との隔たりを表す尺度を算出し、その判定結果に基づいて最終的な抽出結果を得ることも可能である。このような処理の長所としては、多次元での距離計算が必要な構成に比べ演算量が少なくて済むこと、画像入力部１１が画素順次ではなく面順次の形式で画像データを取得した場合にも対応できること、などが挙げられる。
【００７６】
なお、上述した第１および第２の実施の形態では、いずれの場合も、請求項４または９に記載の発明の如く、ＲＧＢ色空間上のユークリッド距離または各原色成分毎の隔たりの距離の検出結果を基に注目画素データを二値化することによって線分抽出を行う場合を例に挙げて説明した。これは、二値化処理を利用すれば、比較的簡単な処理にて良好かつ的確な線分抽出を実現することができるからである。ただし、線分画素抽出部１４ａは、これに限定されるものではなく、他の手法を用いて線分抽出を行うものであってもよい。
【００７７】
〔第３の実施の形態〕
次に、本発明の第３の実施の形態について説明する。ただし、ここでも、上述した第１の実施の形態との相違点についてのみ説明するものとする。
【００７８】
本実施形態は、上述した第１の実施の形態の場合と異なり、抽出対象色を操作部１３から指定入力するのではなく、画像入力部１１が取得したデジタルカラー画像データを参照して自動的に設定する機能を有する点に特徴がある。
図１３は、本実施形態における画像処理装置の概略構成を示すブロック図である。なお、図中において、上述した第１の実施の形態と同一の構成要素については、同一の符号を与えてその説明を省略するものとする。
【００７９】
図例のように、本実施形態における画像処理装置は、第１の実施の形態における場合に加えて、画像処理部１４が使用色検出部１４ｅを有している。
使用色検出部１４ｅは、画像入力部１１が取得したデジタルカラー画像データ中から、その中で使用されている文字色や線分色を抽出し、その抽出した色成分に関する色情報を色情報記憶部１２に保持記憶させるものである。なお、使用色検出部１４ｅは、所定プログラムを実行するマイクロコンピュータ等によりソフトウエア的に構成しても、あるいは所定機能を有した電子回路等によりハードウエア的に構成してもよい。
【００８０】
次に、以上のように構成された画像処理装置における処理動作の概要について説明する。図１４は、本実施形態の画像処理装置での処理動作例、すなわち本実施形態の画像処理装置によって行われる画像処理方法の概要を示すフローチャートである。
【００８１】
図例のように、本実施形態の画像処理装置では、線分抽出処理に際し、先ずはじめに、抽出対象色の数の設定を行う（Ｓ４０１）。ここでいう数は、後述するように画像入力部１１で取得した画像データから検出した色の数のみであっても良いし、例えば「黒」，「赤」等といった色情報記憶部１２内に予め保持記憶されている色に検出色を加えた数であっても良い。図中では、色情報記憶部１２に予め記憶されているＮ個の抽出対象色に、画像入力部１１で取得した画像データから検出したｎ個の色を加えた、合わせてＮ＋ｎ個を抽出対象色とする場合を例に挙げている。
【００８２】
抽出対象色の数が設定されると、次いで、画像入力部１１は、原稿画像に対する読み取りを行って、その原稿画像からＲ，Ｇ，Ｂの各原色成分で表現されるデジタルカラー画像データを取得する（Ｓ４０２）。ここで取得したカラー画像データは、画像処理部１４内のワークメモリ１４ｄに一旦保持される。
【００８３】
そして、ワークメモリ１４ｄがカラー画像データを保持すると、画像処理部１４の使用色検出部１４ｅは、ワークメモリ１４ｄ内のカラー画像データを参照して、そのカラー画像データ中に使用されている色成分を、ｎ個だけ検出する処理を行う（Ｓ４０３）。
【００８４】
ここで、使用色検出部１４ｅによるｎ個分の色検出処理について詳しく説明する。 図１５は、使用色検出部１４ｅが行う処理動作の概要を示すフローチャートである。
【００８５】
図例のように、使用色検出部１４ｅによる色検出処理は、画像入力部１１が取得したカラー画像データ中の色をサンプリング（画像内色サンプリング）するステップ（Ｓ５００）と、そのサンプリング結果に基づいてｎ個の色成分を抽出（色抽出）するステップ（Ｓ６００）と、から構成されている。
【００８６】
画像内色サンプリングは、取得したカラー画像データ中のエッジ成分を検出し、そのＲ，Ｇ，Ｂ信号の出現頻度を計測することにより行う。このときに検出すべき色成分は、文字要素や線分要素に用いられているものである。そのため、使用色検出部１４ｅは、エッジ部分に用いられる色成分のみをサンプリングし、例えば均一背景やグラデーション域などの色情報のサンプリングは行わない。
【００８７】
具体的には、先ず、使用色検出部１４ｅは、エッジ成分の検出のため、エッジ検出用の特徴量「Ｒｄｔ」，「Ｇｄｔ」，「Ｂｄｔ」を算出する（Ｓ５０１，Ｓ５０２，Ｓ５０３）。検出用特徴量Ｒｄｔ，Ｇｄｔ，Ｂｄｔは、それぞれ以下の（９）〜（１１）式のように定義される。
【００８８】
【数９】

【００８９】
【数１０】

【００９０】
【数１１】

【００９１】
ここで、検出用特徴量Ｒｄｔは、図１６の注目画素近傍における画素配置の説明図に示すように、注目画素を「Ｒｊ，ｋ」とすると、注目画素周辺の７×７（主走査×副走査）画素の平均値と注目画素周辺の３×３（主走査×副走査）画素の平均との差分の絶対値に相当する。検出用特徴量Ｇｄｔ，Ｂｄｔについても同様である。また、ここで、（９）〜（１１）式中における「Ｍｄｔ」は、調整用のオフセットパラメータであり、画像処理装置の特性に応じて予め実験的に定められている定数である。
【００９２】
検出用特徴量Ｒｄｔ，Ｇｄｔ，Ｂｄｔを求めると、次いで、使用色検出部１４ｅは、これら検出用特徴量Ｒｄｔ，Ｇｄｔ，Ｂｄｔを、それぞれ予め設定されているしきい値ＴＨｄｔと比較する。この比較の結果、「Ｒｄｔ≧ＴＨｄｔ」と「Ｇｄｔ≧ＴＨｄｔ」と「Ｂｄｔ≧ＴＨｄｔ」とのいずれかが満たされる場合、すなわちＲ，Ｇ，Ｂいずれかの原色成分にエッジが存在する場合に、使用色検出部１４ｅは、注目画素がエッジ部分に相当すると判断する（Ｓ５０４）。この判断の結果、注目画素がエッジ部分である場合に、使用色検出部１４ｅは、その注目画素データのＲ，Ｇ，Ｂの各信号値を検出し、その一方で注目画素がエッジ部分でなければ何の処理も行わない。そして、使用色検出部１４ｅは、以上のような処理を、取得したカラー画像データ中の全画素に対して行い、そのカラー画像データが有する色成分のサンプリング、すなわち出現頻度の計測を完了する（Ｓ５０５，Ｓ５０６）。
【００９３】
このようにして画像内色サンプリングを行うと、その後、使用色検出部１４ｅは、そのサンプリング結果によって得られた出現頻度データから、抽出対象となるｎ個の色成分について色抽出を行う。色抽出処理は、単純に出現頻度の高いｎ個の色成分を選択することで行われる。
【００９４】
具体的には、先ず、使用色検出部１４ｅは、画像データ取得のバラツキや各種ノイズ等を考慮して、画素あたり８ビットの中の上位４ビットを用いて３次元ヒストグラムを生成する（Ｓ６０１）。これにより、三次元の色空間は、４０９６個の空間に分割されたことになる。次いで、使用色検出部１４ｅは、４０９６個の局所領域の中で大きな頻度を有するもの上位ｎ個分を代表色領域として選択する（Ｓ６０２）。その後、使用色検出部１４ｅは、画像内色サンプリングで生成された出現頻度データを参照して、選択されたｎ個の代表色領域（局所領域）内の平均値を算出する（Ｓ６０３）。
【００９５】
そして、その算出結果から、使用色検出部１４ｅは、ｎ個の抽出対象色を得る。
ただし、色成分の頻度分布から代表色を選定するアルゴリズムは、この他にも各種提案されている。例えば、三次元色空間を分散の大きな軸から順次分割してｎ個の領域を生成する手法であるとか、ｎ個の塊を形成するまで領域の統合を繰り返していく手法等、が広く知られている。したがって、使用色検出部１４ｅは、他の手法を用いてｎ個の抽出対象色を得るようにしてもよい。
【００９６】
このような処理によって画像入力部１１が取得したカラー画像データ中のｎ個の使用色成分を検出すると、使用色検出部１４ｅでは、図１４に示すように、その検出した色成分に関する情報を色情報記憶部１２内に保持記憶させる（Ｓ４０４）。その後は、第１の実施の形態で説明したように、色情報記憶部１２内の色情報を基に特定色線分データの抽出を行い（Ｓ４０５〜Ｓ４０７）、不要画素データの除去を経て（Ｓ４０８）、Ｎ＋ｎ個の抽出対象色に関する抽出処理を文字画像蓄積部１５に記憶蓄積する（Ｓ４０９〜Ｓ４１２）。
【００９７】
以上に説明したように、本実施形態の画像処理装置および画像処理方法は、請求項５または１０に記載の発明の如く、処理対象となる文書画像中の文字や線分に使用されている色を検出し、その検出結果に基づいて特定色の文字画像を抽出するようになっている。したがって、文字要素を抽出すべき文書画像中に多くの色成分が含まれていても、ユーザ等による煩雑な操作を必要とすることなく、抽出対象色に係る線分データのみを従来よりも良好かつ的確に抽出することができるので、その操作性が大きく向上し、ユーザ等にとっては非常に便利なものとなる。
【００９８】
【発明の効果】
上述したように、本発明の画像処理装置および画像処理方法では、抽出すべき文字の色に着目し、抽出対象色に係る線分データのみを、取得した画像データから抽出するようになっている。そのため、例えば複雑で多様な構成を持つ文書画像のように、多くの色成分を含むものであっても、従来よりも良好かつ的確に、文字要素に使用される色成分に係る線分データの抽出を行うことができる。
【図面の簡単な説明】
【図１】 本発明の第１の実施の形態における画像処理装置の概略構成を示すブロック図である。
【図２】 第１の実施の形態における画像処理方法の概要を示すフローチャートである。
【図３】 第１の実施の形態における線分画素抽出部が行う処理動作の概要を示すフローチャートである。
【図４】 線分抽出処理を行う際の注目画素近傍における画素配置の一例を示す説明図である。
【図５】 第１の実施の形態における不要画素除去部が行う処理動作の概要を示す説明図（その１）であり、（ａ）〜（ｃ）はそれぞれ領域分割処理の様子を示す図である。
【図６】 第１の実施の形態における不要画素除去部が行う処理動作の概要を示す説明図（その２）であり、（ａ）は不要画素除去前の状態を示す図、（ｂ）は不要画素除去後の状態を示す図である。
【図７】 第１の実施の形態における画像合成部が行う処理動作の概要を示す説明図である。
【図８】 線分抽出処理結果の具体例を示す説明図（その１）であり、（ａ）は処理される原画像を示す図、（ｂ）は抽出される文字画像を示す図、（ｃ）は除去される線分画像を示す図である。
【図９】 線分抽出処理結果の具体例を示す説明図（その２）であり、（ａ）は処理される原画像を示す図、（ｂ）は抽出される文字画像を示す図、（ｃ）は除去される線分画像を示す図である。
【図１０】 線分抽出処理結果の具体例を示す説明図（その３）であり、（ａ）は処理される原画像を示す図、（ｂ）は抽出される文字画像を示す図である。
【図１１】 線分抽出処理結果の具体例を示す説明図（その４）であり、（ａ）は処理される原画像を示す図、（ｂ）は抽出される文字画像を示す図、（ｃ）は除去される線分画像を示す図である。
【図１２】 第２の実施の形態における線分画素抽出部が行う処理動作の概要を示すフローチャートである。
【図１３】 第３の実施の形態における画像処理装置の概略構成を示すブロック図である。
【図１４】 第３の実施の形態における画像処理方法の概要を示すフローチャートである。
【図１５】 第３の実施の形態における使用色検出部が行う処理動作の概要を示すフローチャートである。
【図１６】 使用色検出処理を行う際の注目画素近傍における画素配置の一例を示す説明図である。
【図１７】 線分抽出処理の対象となる原画像の一例を示す説明図である。
【符号の説明】
１１…画像入力部、１２…色情報記憶部、１４…画像処理部、１４ａ…線分画素抽出部、１４ｂ…不要画素除去部、１４ｃ…画像合成部、１４ｅ…使用色検出部[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus and an image processing method for extracting character elements, more specifically line segment data constituting the character elements, from color document image data including the character elements.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, as image processing apparatuses that extract line segments from document image data, those using a method described below are widely known. For example, a single threshold value for binarization processing is determined based on density information on the entire surface of document image data, and a high density pixel (character element) and a low density pixel are used using the single threshold value. Some perform line segment extraction by separating (non-character elements, background elements, etc.). Also, there are methods that use a method that generates binary images based on density information for each block by dividing the document image data into local blocks, and document image data that includes areas with character elements and pictures such as photographs. There is also a method using a method of generating a binary image by setting a threshold value for binarization processing for an area where character elements exist after being divided into areas to be processed. Further, for color document image data, a brightness signal representing brightness is generated from the primary color components of red (R), green (G), and blue (B), and the same method as in each of the above processes is generated. There is also a corresponding one.
[0003]
[Problems to be solved by the invention]
However, in recent years, document images to be processed have become extremely diversified and complicated as compared with the past due to changes and advances in the document creation environment such as computers, image input / output devices, and document processing software used in them. Therefore, the conventional image processing apparatus and the image processing method described above cannot cope with it, and as a result, line segment extraction may not be performed accurately.
[0004]
For example, FIG. 17 shows an example of a document image created in a recent document creation environment.
In the conventional image processing apparatus and image processing method, line segments constituting black characters on a white background as indicated by (1) in the figure are mainly extracted. For this reason, the line segment constituting the color character on the white background as shown in (2) in the figure and the line segment constituting the character having the background color as shown in (3) in the figure are on the white background. It is difficult to extract with the same accuracy as that of black characters. In response to this, parameter settings such as a threshold value are changed to make it easy to determine a line segment to be extracted when determining a character element and a non-character element even if the line segment is a color character. However, in such a case, there is a high possibility that an element that is not a character is erroneously extracted as a character. For example, in the case of a document having a picture element as shown in (4) in the figure, many false detections occur due to the edge component in the picture element and the picture is determined to be a line segment constituting a character. there is a possibility. In addition to this, a region where the character background is not uniform as shown in (5) in the figure, a region where color characters exist on the color background as shown in (6), or a character as shown in (7) In a region where the lightness of the background is reversed, line segment extraction becomes more difficult.
[0005]
Therefore, the present invention can cope with diversified and complicated document image data as described above, and accurately extracts line segment data constituting character elements from the document image data. An object of the present invention is to provide an image processing apparatus and an image treatment method that can be used.
[0006]
[Means for Solving the Problems]
The present invention provides an acquisition means for acquiring image data in an image processing apparatus devised to achieve the above object, Color extracting means for extracting a predetermined number of color components included in the image data based on the appearance frequency of the color component in the image data acquired by the acquiring means, and the predetermined number of colors extracted by the color extracting means Storage means for holding and storing color information relating to a color component; detection means for detecting a distance on a color space between the image data acquired by the acquisition means and the color information held and stored by the storage means; Line segment extraction means for extracting line segments from image data acquired by the acquisition means only for color components specified by the color information held and stored by the storage means based on the detection results of the detection means. It is characterized by.
[0007]
The present invention is an image processing method devised to achieve the above object, and is an image processing method for extracting line segment data constituting a character element from image data, A predetermined number of color components included in the image data are extracted based on the appearance frequency of the color components in the image data from which the line segment data is to be extracted. Color information relating to color components is stored and stored, and when image data from which line segment data is to be extracted is acquired, a distance in the color space between the image data and the color information is detected, and the detected color space Only line segment data related to the color component specified by the color information based on the distance is extracted from the acquired image data.
[0008]
Although it is a document image that has been diversified and complicated as described above, dozens to millions of colors use pattern elements such as background images and decorations in the document. In general, colors used for line segments such as information characters and ruled lines are about 3 to 4 colors. In fact, the colors used for the line segments are generally determined as “black”, “red”, “white (inverted)”, “dark blue”, and the like.
Therefore, in the image processing apparatus having the above configuration and the image processing method according to the above procedure, Pre-determined items extracted based on the appearance frequency of color components If the image data for which line segment data is to be extracted is obtained by storing and storing color information related to the color components, only the line segment data relating to the color component specified by the stored color information is obtained. Extracted from data. Therefore, if the color information used for the character elements in the document image is stored and stored, even if the document image includes many color components, the line segments related to the color components used for the character elements Only data can be extracted.
[0009]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an image processing apparatus and an image processing method according to the present invention will be described with reference to the drawings.
[0010]
[First Embodiment]
FIG. 1 is a block diagram showing a schematic configuration of an image processing apparatus according to the first embodiment of the present invention.
As shown in the figure, the image processing apparatus according to this embodiment includes an image input unit 11, a color information storage unit 12, an operation unit 13, an image processing unit 14, a character image storage unit 15, and these units. And a bus 16 connected to each other.
[0011]
The image input unit 11 acquires image data composed of a plurality of pixel data from outside the image processing apparatus. Specifically, as in a flatbed scanner, for example, an original image is optically read, and digital color image data expressed by R, G, B primary color components is acquired from the original image. However, the image input unit 11 may acquire image data from the outside through, for example, a network connected to the image processing apparatus.
It is assumed that the image data acquired by the image input unit 11 has a resolution of 400 pixels / inch with R, G, and B each having 8 bits / pixel gradation.
[0012]
The color information storage unit 12 holds and stores color information related to a predetermined color component, more specifically color information related to a color component that is a target color for character extraction. The color information storage unit 12 may be configured by a semiconductor memory such as a RAM (Random Access Memory) or an auxiliary storage device such as an HDD (Hard Disk Drive).
[0013]
The operation unit 13 performs an operation for controlling the operation of the entire image processing apparatus, and includes a touch panel, a switch, and the like operated by a user of the image processing apparatus. The operations performed by the operation unit 13 include setting and inputting color information to be held and stored in the color information storage unit 12.
[0014]
The image processing unit 14 performs character element extraction processing on the digital color image data acquired by the image input unit 11. For this purpose, the image processing unit 14 includes a line segment pixel extraction unit 14a, an unnecessary pixel removal unit 14b, an image synthesis unit 14c, and a work memory 14d.
[0015]
The line segment pixel extraction unit 14a receives line segment data relating to a predetermined color component, specifically pixel data constituting a line segment image having a predetermined color component, from the digital color image data acquired by the image input unit 11. To extract. The predetermined color component corresponds to one specified by the color information held and stored in the color information storage unit 12. If color information relating to a plurality of color components is held and stored in the color information storage unit 12, the line segment pixel extraction unit 14a extracts line segment data for each color component.
[0016]
The unnecessary pixel removing unit 14b removes data determined not to constitute a character element from the line segment data extracted by the line segment pixel extracting unit 14a, for example, edge pixel data or noise data that is erroneously extracted. To do. Note that the unnecessary pixel removing unit 14b also removes unnecessary data for each color component, similarly to the line segment pixel extracting unit 14a.
[0017]
The image synthesis unit 14c synthesizes the line segment data extraction result for each color component by the line segment pixel extraction unit 14a and the unnecessary data removal result for each color component by the unnecessary pixel removal unit 14b, The image data in the final output form, that is, the image data corresponding to the character elements included in the image data acquired by the image input unit 11 is generated.
[0018]
The work memory 14d is used as a work memory or a buffer memory when the line segment pixel extracting unit 14a, the unnecessary pixel removing unit 14b, and the image combining unit 14c perform processing operations.
The work memory 14d is constituted by a semiconductor memory such as a RAM. However, the line segment pixel extracting unit 14a, the unnecessary pixel removing unit 14b, and the image synthesizing unit 14c are configured by software such as a microcomputer that executes a predetermined program. Alternatively, it may be configured in hardware or an electronic circuit having a predetermined function.
[0019]
The character image accumulating unit 15 stores and accumulates image data corresponding to the character element that is the processing result of the image processing unit 14. For this purpose, the character image storage unit 15 includes a semiconductor memory such as a RAM or an auxiliary storage device such as an HDD.
[0020]
Here, an outline of the processing operation in the image processing apparatus configured as described above will be described. FIG. 2 is a flowchart showing an outline of an image processing method performed by the image processing apparatus of this embodiment, that is, an example of processing operation in the image processing apparatus of this embodiment.
[0021]
As shown in the figure, in the image processing apparatus according to the present embodiment, in the line segment extraction process, first, a color component to be extracted is set (step 101; step is hereinafter abbreviated as S). At this time, the extraction target color may be held and stored in advance in the color information storage unit 12 or may be newly specified and input from the operation unit 13. In the drawing, a case where N extraction target colors have already been designated and stored in the color information storage unit 12 is taken as an example.
[0022]
When the extraction target color is set, the image input unit 11 then reads the original image, and acquires digital color image data represented by R, G, and B primary color components from the original image ( S102). The color image data acquired here is temporarily held in the work memory 14d in the image processing unit 14.
[0023]
Then, the image processing unit 14 starts a line segment extraction process for the first extraction target color among the first to Nth extraction target colors (S103). That is, the image processing unit 14 reads the color image data in the work memory 14d and inputs it to the line segment pixel extraction unit 14a. At this time, the color information relating to the first extraction target color is loaded from the color information storage unit 12 into the line segment pixel extraction unit 14a (S104). Thus, the line segment pixel extraction unit 14a extracts line segment data having the color component of the first extraction target color from the input color image data (S105), and the line segment data is extracted from the unnecessary pixel removal unit 14b. Output to.
[0024]
The unnecessary pixel removing unit 14b receives the line segment data extracted by the line segment pixel extracting unit 14a, and removes pixel data regarding noise components, non-character components, and the like that are erroneously extracted from the line segment data. (S106). Then, the processed image data is output to the work memory 14d and temporarily stored in the work memory 14d (S107).
[0025]
When the line segment extraction process for the first extraction target color is completed in this way, the image processing unit 14 then performs the Nth extraction target color line for the second and subsequent extraction target colors (S108). Until the segment extraction process is completed (S109), the same segment extraction process is repeated (S105 to S109). As a result, the extraction results for all the N extraction target colors are held and stored in the work memory 14d.
[0026]
When the extraction process for all the extraction target colors is completed (S109), in the image processing unit 14, the image synthesis unit 14c performs a process of extracting extraction results for all colors in the work memory 14d and synthesizing them (S110). ). The composite image data obtained by this processing, that is, the extraction processing result by the image processing unit 14 is stored and accumulated in the character image accumulating unit 15, and then sent to a character recognition device (not shown). It will be used for recognition processing.
[0027]
Next, among the processing operations described above, the line segment extraction process (S105 in FIG. 2) for the specific color (1st to Nth extraction target colors) performed by the line segment pixel extraction unit 14a will be described in detail.
FIG. 3 is a flowchart showing an outline of the processing operation performed by the line segment pixel extraction unit 14a. In the figure, “R, G, B” is the pixel-of-interest data value of the digital color image data acquired by the image input unit 11, and “Ri, Gi, Bi” is an extraction loaded from the color information storage unit 12. For the color information value of the target color, “OUT” represents binarized pixel data that is an extraction result by the line segment pixel extraction unit 14a.
[0028]
As shown in the figure, in performing the line segment extraction process, the line segment pixel extraction unit 14a firstly calculates the difference between the target pixel data value R, G, B and the color information values Ri, Gi, Bi of the extraction target color. Signals “ΔR, ΔG, ΔB” are generated (S201). The difference signals ΔR, ΔG, ΔB are generated using the following equations (1) to (3).
[0029]
[Expression 1]

[0030]
[Expression 2]

[0031]
[Equation 3]

[0032]
When the difference signals ΔR, ΔG, ΔB are generated, the line segment pixel extraction unit 14a then generates a color detection signal “D” using the difference signals ΔR, ΔG, ΔB (S202). The generation of the color detection signal D is performed using the following equation (4).
[0033]
[Expression 4]

[0034]
The color detection signal D calculated here represents the distance in the RGB color space between the target pixel data value R, G, B and the color information value Ri, Gi, Bi of the color to be extracted. That is, the color detection signal D corresponds to a signal indicating the distance between the color of each pixel in the RGB color space and the extraction target color, and represents a color closer to the extraction target color as the value increases.
[0035]
Here, the color detection is performed for the Euclidean distance between the target pixel data values R, G, B and the color information values Ri, Gi, Bi of the extraction target color, that is, the square root of the square sum of the difference signals ΔR, ΔG, ΔB. Although it is defined as the signal D, it is needless to say that this can be replaced by an operation that is a measure of the distance between the two points, for example, the average value of the difference signals ΔR, ΔG, ΔB.
[0036]
When the color detection signal D is obtained, the line segment pixel extraction unit 14a then performs line segment extraction based on the obtained color detection signal D. The line segment extraction may be performed using a known method. For example, when the gradation level changes sharply around the pixel of interest and the gradation level of the pixel of interest is sufficiently high, it may be determined that the pixel of interest is a line segment element.
[0037]
Specifically, the line segment pixel extraction unit 14a first calculates the edge feature amount “De” and the level feature amount “Dl” of the color detection signal D using the obtained color detection signal D (S203, S204). ). The edge feature amount De and the level feature amount Dl are defined as in the following equations (5) and (6), respectively.
[0038]
[Equation 5]

[0039]
[Formula 6]

[0040]
Here, as shown in the explanatory diagram of the pixel arrangement in the vicinity of the target pixel in FIG. 4, when the target pixel is “Dj, k”, the edge feature amount De is 5 × 5 (main scan × sub scan) around the target pixel. This corresponds to the absolute value of the difference between the average value of the pixels and the average of 3 × 3 (main scanning × sub-scanning) pixels around the target pixel. The level feature amount Dl corresponds to an average value of 3 × 3 (main scanning × sub-scanning) pixels around the target pixel.
Note that “M” and “N” in the equations (5) and (6) are offset parameters for adjustment, and are constants experimentally determined in advance according to the characteristics of the image processing apparatus.
[0041]
When the edge feature amount De and the level feature amount Dl are obtained, the line segment pixel extraction unit 14a then compares the edge feature amount De and the level feature amount Dl with preset threshold values THe and THl, respectively. . Then, it is determined whether or not the edge feature amount De is equal to or greater than the threshold value THe and equal to or greater than the level feature amount Dl threshold value THl (S205).
[0042]
As a result of this determination, if De ≧ THe and Dl ≧ TH1, the line segment pixel extraction unit 14a has a sharp change in gradation level around the pixel of interest Dj, k, and the gradation level of the pixel of interest is Since it is sufficiently high, it is determined that the target pixel Dj, k is a pixel constituting the line segment. Then, the output pixel data value OUT for the target pixel data value R, G, B is turned ON (S206). On the other hand, if De ≧ THe and Dl ≧ THl are not satisfied, the line segment pixel extraction unit 14a turns off the output pixel data value OUT (S207).
[0043]
As described above, in the line segment pixel extraction unit 14a, the distance between the color information values Ri, Gi, Bi of the extraction target color with respect to each pixel data constituting the digital color image data acquired by the image input unit 11. Binarization is performed based on the color detection signal D representing Thus, pixel data corresponding to the line segment element of the extraction target color, that is, line segment data related to the color component of the extraction target color, is extracted from the line segment pixel extraction unit 14a and output to the unnecessary pixel removal unit 14b. Will be.
[0044]
In addition, although the case where the line segment pixel extraction unit 14a performs line segment extraction by a relatively simple technique is described here as an example, the present invention is not particularly limited thereto. For example, as proposed in the patent application already filed by the present applicant (Japanese Patent Application No. 11-166676), by scanning digital color image data with line segment basic elements corresponding to predetermined graphic shape elements, If pixel data constituting a line segment having a width corresponding to the line segment basic element is extracted, more accurate line segment extraction can be performed.
[0045]
Next, of the processing operations performed by the image processing apparatus according to the present embodiment, the unnecessary pixel data removal process (S106 in FIG. 2) performed by the unnecessary pixel removal unit 14b will be described in detail.
5 and 6 are explanatory diagrams showing an outline of the processing operation performed by the unnecessary pixel removing unit 14b. In addition, the inside of the rectangle in FIG. 5 represents the state which imaged the output pixel data OUT after binarization output from the line segment pixel extraction part 14a.
[0046]
The processing operation performed by the unnecessary pixel removing unit 14b is based on a known document structure recognition / image area separation method. First, as shown in FIG. 5A, the unnecessary pixel removing unit 14b creates a projection distribution in the vertical direction and the horizontal direction for the binarized image image, and the distribution with the frequency “0” is the longest point. The area is divided into two. Furthermore, as shown in FIGS. 5B and 5C, the unnecessary pixel removing unit 14b performs the same region dividing process on each of the two divided regions. This process is repeated until each area becomes the minimum unit constituting the document.
[0047]
Thereafter, as shown in FIG. 6A, the unnecessary pixel removing unit 14b generates a circumscribed rectangle for a portion where the pixel data value is ON in each divided area. When the circumscribed rectangle is generated, the unnecessary pixel removing unit 14b calculates a feature amount for each circumscribed rectangle, and based on the feature amount, each circumscribed rectangle corresponds to a character element, or a halftone dot region or a ruled line. It is determined whether it corresponds to a document constituent element other than a character element or a noise component extracted by mistake. Here, as the feature amount, it is conceivable to use the circumscribed rectangle area, the aspect ratio, the area ratio of the portion where the data value in the circumscribed rectangle is ON, the ON / OFF inversion number, and the like. For example, when the area of a circumscribed rectangle is used, it is determined that an object that is too large or too small is not a character element, based on the number of points that represent the size of the character element. As a result of the determination using these feature amounts, for the pixel data belonging to the circumscribed rectangle determined not to be a character pixel, the unnecessary pixel removing unit 14b changes the value from ON to OFF as shown in FIG. 6B. Convert.
[0048]
Then, when the unnecessary pixel removing unit 14b finishes processing for all circumscribed rectangles, the unnecessary pixel removing unit 14b holds and stores the processing result in the work memory 14d. The unnecessary pixel removal unit 14b performs such unnecessary pixel data removal processing on all the N extraction target colors, so that each of the N extraction target colors is extracted in the work memory 14d. Of the line segment data related to the target color, only the line segment data constituting the character element is held and stored.
[0049]
Next, among the processing operations performed by the image processing apparatus according to the present embodiment, the composition processing (S110 in FIG. 2) of N color extraction processing results performed by the image composition unit 14c will be described in detail.
FIG. 7 is an explanatory diagram showing an outline of the processing operation performed by the image composition unit 14c. In the figure, for simplicity, an example is shown in which N = 3 and three colors of “black”, “white”, and “red” are set as extraction target colors.
[0050]
When the extraction target colors are “black”, “white”, and “red”, when the line segment extraction process by the line segment pixel extraction unit 14a and the unnecessary pixel data removal process by the unnecessary pixel removal unit 14b are completed, In the work memory 14d, binary line segment data relating to the black component as indicated by (1) in FIG. 7, binary line segment data relating to the white component as indicated by (2), Similarly, binarized line segment data relating to the red component as indicated by (3) is accumulated individually. Here, the image synthesizing unit 14c takes out the binarized line segment data (1), (2), (3) relating to each color component from the work memory 14d, performs a synthesizing process on these, and obtains the result of the synthesizing process. The data is output to the character image storage unit 15 and stored there.
[0051]
For example, the composition processing is performed by the image composition unit 14c taking a logical sum of the binarized line segment data (1), (2), and (3) relating to each color component, and thereby the final image processing unit 14 is obtained. As a result, the binary image data constituting the character element of the color to be extracted is obtained (see FIG. 7A).
[0052]
However, the synthesis process may be performed as follows.
For example, as shown in FIG. 7B, the binarized line segment data (1), (2), (3) relating to each color component is added with the respective color information (4). It is possible to synthesize. As the color information (4) at this time, for example, a color number set in the color information storage unit 12 is appropriate. By performing such a synthesis process, it is possible to obtain multi-layer binary image data to which color information of each character element is added as a final extraction process result by the image processing unit 14.
Further, for example, as shown in FIG. 7C, it is conceivable to generate a 2-bit color map to which color information (4) is added as header information of binary image data. At this time, the upper 1 bit of the color information represents the corresponding pixel data, and the lower 24 bits represent the color component. By performing such a synthesis process, it is possible to obtain multi-value palette image data to which color information of each character element is added as a final extraction process result by the image processing unit 14.
[0053]
As described above, when the synthesis process with the color information added is performed, a great added value can be given to the post-processing performed thereafter. For example, when character recognition processing is performed as post-processing for character image data stored in the character image storage unit 15, attributes that the character has on the original image such as “black characters” and “red characters” are displayed. It can be used. Therefore, it is very effective particularly when data conversion to an HTML (Hyper Text Markup Language) format or the like is performed.
[0054]
Advantages of the conventional technique when the above processing is performed will be described with reference to FIGS. 8 to 11 are explanatory diagrams showing the components of the document image, as in FIG. 17 described in the section of the prior art.
[0055]
For example, for a character having a non-uniform background as shown in FIG. 8A, a binary image shown in FIG. 8B is obtained by appropriately setting an extraction target color and performing line segment extraction processing. Can be separated and extracted into line segment data representing the binary image shown in FIG. 8C. Further, by performing unnecessary pixel data removal processing on these line segment data, it is finally possible to extract line segment data representing only a character image as shown in FIG. .
[0056]
FIG. 9 shows how the inverted characters are extracted. When line segment extraction processing based on a normal lightness component is performed, a character element having a lightness value (pixel data value brighter than the surroundings) than the surrounding pixels as shown in FIG. It is difficult to extract with the same processing logic as that character. However, according to the image processing apparatus and the image processing method of the present embodiment, line segment data extraction processing is performed by paying attention to the Euclidean distance (distance) from a preset extraction target color. And, as shown in (c), even a reversed character string can be extracted well. Similarly to the example shown in FIG. 8, since the frame line in FIG. 9C is also removed as an element other than characters, a line that represents only the character image as shown in FIG. Minute data can be extracted.
[0057]
FIG. 10 shows how characters having a background are extracted. As shown in FIG. 10B, when a character exists in an area having a background color as shown in FIG. 10A, the background color is not extracted unless it is an extraction target color. Line segment data representing only a character image can be extracted.
[0058]
FIG. 11 shows the extraction of characters having a non-uniform background, as in FIG. As shown in FIG. 11A, when the background is expressed by gradation, it is very difficult to extract characters only by the difference in brightness. Since the processing method focuses on the color of the character to be extracted, good character extraction as shown in FIG. 11B is possible. Further, as shown in FIG. 11C, components other than the extracted character can be erased by performing unnecessary pixel data removal processing.
[0059]
As described above, the image processing apparatus and the image processing method according to the present embodiment pay attention to the color of the character to be extracted as in the first or sixth aspect of the invention, and the line segment data related to the extraction target color. Only to extract. Therefore, even if a document image from which a character element is to be extracted contains many color components, it is possible to extract line segment data constituting the character element better and more accurately than in the past.
[0060]
Further, the image processing apparatus and the image processing method according to the present embodiment removes data determined not to constitute a character element from the extracted line segment data, as in the invention described in claim 2 or 7. It has become. Therefore, even if the document image from which the character element is to be extracted is diversified and complicated, the line segment data constituting the character element can be extracted better and more accurately than the conventional case.
[0061]
Furthermore, the image processing apparatus and the image processing method according to the present embodiment can cope with a plurality of extraction target colors as in the invention described in claim 3 or 8. Therefore, it becomes easy to cope with diversification and complexity of document images. In addition, by sequentially performing line segment extraction processing for a plurality of extraction target colors, the extraction range for each color can be set narrower than in the conventional case. For example, pixel data representing halftone dots, edge portion data, etc. It becomes possible to greatly reduce erroneous extraction.
[0062]
[Second Embodiment]
Next, a second embodiment of the present invention will be described. However, only differences from the first embodiment described above will be described here.
[0063]
This embodiment is different from the above-described first embodiment only in the line segment extraction processing by the line segment pixel extraction unit 14a. Specifically, in the first embodiment, the three-dimensional distance between the target pixel data values R, G, B in the RGB color space and the color information values Ri, Gi, Bi of the extraction target color is calculated as the extraction target color. In this embodiment, the color detection is performed based on the one-dimensional distance between the target pixel data values R, G, and B and the primary color components of each extraction target color. This is different in that line segment data having a color line segment to be extracted is extracted by taking a logical product of extraction results for each primary color component using the signal as a signal for use.
[0064]
Hereinafter, the line segment extraction process in the present embodiment will be described in detail with reference to FIG. FIG. 12 is a flowchart illustrating another example of the processing operation performed by the line segment pixel extraction unit 14a. As in the case of the first embodiment (see FIG. 3), in FIG. 12, “R, G, B” represents the target pixel data value of the digital color image data acquired by the image input unit 11, “Ri, Gi, Bi” represents the color information value of the color to be extracted loaded from the color information storage unit 12, and “OUT” represents the binarized pixel data that is the extraction result by the line segment pixel extraction unit 14a. ing.
[0065]
As shown in the figure, in the case of the present embodiment, the line segment pixel extraction unit 14a first performs a line segment extraction process between the red component value R of the target pixel data and the red component value Ri of the extraction target color. A difference signal “ΔR” is generated (S301). The difference signal ΔR is generated using equation (1) as in the case of the first embodiment.
[0066]
Here, the difference signal ΔR represents the distance on the R-axis between the red component value R of the target pixel data and the red component value Ri of the color to be extracted, and the red component of the target pixel increases as the value increases. It represents that the color is close to the red component of the extraction target color.
[0067]
When the difference signal ΔR is generated, the line segment pixel extraction unit 14a performs line segment extraction based on the difference signal ΔR. The line segment extraction may be performed using a known method. For example, as in the case described in the first embodiment, when the gradation level changes sharply around the pixel of interest and the gradation level of the pixel of interest is sufficiently high, the pixel of interest is lined up. It can be considered that there is a division factor.
Specifically, the line segment pixel extraction unit 14a first calculates the edge feature amount “Re” and the level feature amount “Rl” using the generated difference signal ΔR (S302, S303). The edge feature amount Re and the level feature amount Rl are defined as in the following equations (7) and (8), respectively.
[0068]
[Expression 7]

[0069]
[Equation 8]

[0070]
Here, the edge feature amount Re is an average value of 5 × 5 (main scanning × sub-scanning) pixels around the pixel of interest and 3 × 3 (main scanning × X) around the pixel of interest when the pixel of interest is “Rj, k”. This corresponds to the absolute value of the difference from the average of the sub-scanning pixels. The level feature amount Rl corresponds to the average value of 3 × 3 (main scanning × sub-scanning) pixels around the target pixel (see FIG. 4 for both).
In the equations (7) and (8), “Mr” and “Nr” are offset parameters for adjustment, and are constants experimentally determined in advance according to the characteristics of the image processing apparatus.
[0071]
When the edge feature amount Re and the level feature amount Rl are obtained in this way, the line segment pixel extraction unit 14a then sets the edge feature amount Re and the level feature amount Rl to preset threshold values THE and TH1, respectively. Compare. Then, it is determined whether or not the edge feature amount Re is greater than or equal to the threshold value THe and greater than or equal to the level feature amount Rl threshold value THl (S304).
[0072]
If Re ≧ THe and Rl ≧ THl as a result of this determination, the line segment pixel extraction unit 14a has a sharp change in gradation level around the target pixel Rj, k, and the gradation level of the target pixel is Since it is sufficiently high, it is determined that the target pixel Rj, k is a pixel constituting the line segment. Then, the determination result signal “Ro” is turned ON (S305). On the other hand, if Re ≧ THe and Rl ≧ THl are not satisfied, the line segment pixel extraction unit 14a turns off the determination result signal Ro (S306).
[0073]
The line segment pixel extraction unit 14a performs the above processing not only for the red component value R of the target pixel data but also for the green component value G and the blue component value B of the target pixel data in exactly the same manner. The respective determination result signals Go and Bo are obtained (S307 to S318).
[0074]
When the determination result signals Ro, Go, and Bo are thus obtained, the line segment pixel extraction unit 14a refers to the respective determination result signals Ro, Go, and Bo to determine whether or not all these signals are ON. (S319) If all are ON, the output pixel data value OUT is turned ON (S320), and if not, the output pixel data value OUT is turned OFF (S321).
[0075]
In this way, the line segment pixel extraction unit 14a can calculate a scale representing a distance from the extraction target color for each primary color component, and obtain a final extraction result based on the determination result. The advantages of such processing are that the amount of calculation is smaller than that of a configuration that requires multi-dimensional distance calculation, and that the image input unit 11 acquires image data in a frame sequential format instead of pixel sequential. Can be supported.
[0076]
In any of the first and second embodiments described above, the detection of the Euclidean distance in the RGB color space or the separation distance for each primary color component as in the invention described in claim 4 or 9. The case where line segment extraction is performed by binarizing the target pixel data based on the result has been described as an example. This is because good and accurate line segment extraction can be realized by a relatively simple process if the binarization process is used. However, the line segment pixel extraction unit 14a is not limited to this, and may perform line segment extraction using other methods.
[0077]
[Third Embodiment]
Next, a third embodiment of the present invention will be described. Here, however, only differences from the first embodiment described above will be described.
[0078]
Unlike the first embodiment described above, the present embodiment does not specify and input an extraction target color from the operation unit 13, but automatically refers to the digital color image data acquired by the image input unit 11. It has a feature in that it has a function to set.
FIG. 13 is a block diagram illustrating a schematic configuration of the image processing apparatus according to the present embodiment. In the figure, the same components as those in the first embodiment described above are given the same reference numerals and the description thereof is omitted.
[0079]
As illustrated, in the image processing apparatus according to the present embodiment, in addition to the case of the first embodiment, the image processing unit 14 includes a use color detection unit 14e.
The used color detection unit 14e extracts the character color and line segment color used in the digital color image data acquired by the image input unit 11, and stores color information regarding the extracted color component in color information. This is held and stored in the unit 12. The used color detection unit 14e may be configured in software by a microcomputer or the like that executes a predetermined program, or may be configured in hardware by an electronic circuit having a predetermined function.
[0080]
Next, an outline of processing operations in the image processing apparatus configured as described above will be described. FIG. 14 is a flowchart illustrating an example of a processing operation in the image processing apparatus of the present embodiment, that is, an outline of an image processing method performed by the image processing apparatus of the present embodiment.
[0081]
As shown in the figure, in the image processing apparatus of the present embodiment, in the line segment extraction process, first, the number of extraction target colors is set (S401). The number here may be only the number of colors detected from the image data acquired by the image input unit 11 as will be described later. For example, the number may be stored in the color information storage unit 12 such as “black” or “red”. It may be the number obtained by adding the detected color to the color stored and stored in advance. In the figure, N extraction target colors previously stored in the color information storage unit 12 are added with n colors detected from the image data acquired by the image input unit 11, and a total of N + n is extracted. The case of color is given as an example.
[0082]
When the number of extraction target colors is set, the image input unit 11 then reads the original image and obtains digital color image data expressed by R, G, B primary color components from the original image. (S402). The color image data acquired here is temporarily held in the work memory 14d in the image processing unit 14.
[0083]
When the work memory 14d holds the color image data, the used color detection unit 14e of the image processing unit 14 refers to the color image data in the work memory 14d, and the color components used in the color image data. Is detected (n) (S403).
[0084]
Here, n color detection processes by the used color detection unit 14e will be described in detail. FIG. 15 is a flowchart showing an outline of the processing operation performed by the use color detection unit 14e.
[0085]
As shown in the figure, the color detection processing by the used color detection unit 14e is based on the step (S500) of sampling the color in the color image data acquired by the image input unit 11 (in-image color sampling) and the sampling result. Step (S600) for extracting (color extraction) n color components.
[0086]
In-image color sampling is performed by detecting edge components in the acquired color image data and measuring the appearance frequency of the R, G, B signals. The color components to be detected at this time are those used for character elements and line segment elements. Therefore, the used color detection unit 14e samples only the color components used for the edge portion, and does not sample color information such as a uniform background or gradation area.
[0087]
Specifically, first, the used color detection unit 14e calculates edge detection feature amounts “Rdt”, “Gdt”, and “Bdt” in order to detect edge components (S501, S502, and S503). The detection feature amounts Rdt, Gdt, and Bdt are defined as the following equations (9) to (11), respectively.
[0088]
[Equation 9]

[0089]
[Expression 10]

[0090]
[Expression 11]

[0091]
Here, as shown in the explanatory diagram of the pixel arrangement in the vicinity of the target pixel in FIG. 16, when the target pixel is “Rj, k”, the detection feature amount Rdt is 7 × 7 (main scanning × sub scan) around the target pixel. This corresponds to the absolute value of the difference between the average value of the scanning) pixel and the average of 3 × 3 (main scanning × sub-scanning) pixels around the target pixel. The same applies to the detection feature amounts Gdt and Bdt. Here, “Mdt” in equations (9) to (11) is an offset parameter for adjustment, and is a constant experimentally determined in advance according to the characteristics of the image processing apparatus.
[0092]
When the detection feature amounts Rdt, Gdt, and Bdt are obtained, the use color detection unit 14e then compares the detection feature amounts Rdt, Gdt, and Bdt with preset threshold values THdt. As a result of this comparison, when any of “Rdt ≧ THdt”, “Gdt ≧ THdt”, and “Bdt ≧ THdt” is satisfied, that is, when an edge exists in one of the primary color components of R, G, and B, The use color detection unit 14e determines that the target pixel corresponds to the edge portion (S504). As a result of this determination, when the target pixel is an edge portion, the used color detection unit 14e detects the R, G, and B signal values of the target pixel data, while the target pixel must be an edge portion. No processing is performed. Then, the used color detection unit 14e performs the above-described process on all the pixels in the acquired color image data, and completes sampling of color components included in the color image data, that is, measurement of appearance frequency ( S505, S506).
[0093]
When the in-image color sampling is performed in this manner, the used color detection unit 14e thereafter performs color extraction on n color components to be extracted from the appearance frequency data obtained from the sampling result. The color extraction process is performed by simply selecting n color components having a high appearance frequency.
[0094]
Specifically, first, the used color detection unit 14e generates a three-dimensional histogram using upper 4 bits of 8 bits per pixel in consideration of variations in image data acquisition and various noises (S601). . As a result, the three-dimensional color space is divided into 4096 spaces. Next, the used color detection unit 14e selects the top n most frequently occurring 4096 local regions as representative color regions (S602). Thereafter, the used color detection unit 14e refers to the appearance frequency data generated by the in-image color sampling and calculates an average value in the selected n representative color regions (local regions) (S603).
[0095]
Then, from the calculation result, the use color detection unit 14e obtains n extraction target colors.
However, various other algorithms for selecting a representative color from the frequency distribution of color components have been proposed. For example, a method of generating n regions by sequentially dividing a three-dimensional color space from a large axis of dispersion or a method of repeating region integration until n clusters are formed is widely known. ing. Therefore, the used color detection unit 14e may obtain n extraction target colors using another method.
[0096]
When n use color components are detected in the color image data acquired by the image input unit 11 by such processing, the use color detection unit 14e displays information on the detected color components as shown in FIG. The information is stored in the information storage unit 12 (S404). Thereafter, as described in the first embodiment, specific color line segment data is extracted based on the color information in the color information storage unit 12 (S405 to S407), and unnecessary pixel data is removed ( S408), extraction processing relating to N + n extraction target colors is stored and accumulated in the character image accumulation unit 15 (S409 to S412).
[0097]
As described above, the image processing apparatus and the image processing method according to the present embodiment use the colors used for characters and line segments in the document image to be processed as in the invention described in claim 5 or 10. And a character image of a specific color is extracted based on the detection result. Therefore, even if a large number of color components are included in the document image from which character elements are to be extracted, only line segment data related to the color to be extracted is better than before without requiring a complicated operation by the user or the like. Since it can be extracted accurately, its operability is greatly improved, which is very convenient for users and the like.
[0098]
【The invention's effect】
As described above, in the image processing apparatus and the image processing method of the present invention, focusing on the color of the character to be extracted, only the line segment data related to the extraction target color is extracted from the acquired image data. . For this reason, even if the document image includes a large number of color components, such as a document image having a complicated and diverse configuration, the line segment data relating to the color components used for the character elements is better and more accurate than the conventional one. Extraction can be performed.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a schematic configuration of an image processing apparatus according to a first embodiment of the present invention.
FIG. 2 is a flowchart showing an overview of an image processing method according to the first embodiment.
FIG. 3 is a flowchart showing an outline of a processing operation performed by a line segment pixel extraction unit in the first embodiment.
FIG. 4 is an explanatory diagram illustrating an example of a pixel arrangement in the vicinity of a target pixel when performing line segment extraction processing;
FIGS. 5A and 5B are explanatory diagrams (part 1) illustrating an outline of a processing operation performed by an unnecessary pixel removing unit according to the first embodiment, and FIGS. is there.
FIGS. 6A and 6B are explanatory diagrams (part 2) illustrating an outline of a processing operation performed by an unnecessary pixel removing unit according to the first embodiment, where FIG. 6A is a diagram illustrating a state before unnecessary pixel removal, and FIG. It is a figure which shows the state after unnecessary pixel removal.
FIG. 7 is an explanatory diagram illustrating an overview of processing operations performed by an image composition unit according to the first embodiment;
FIGS. 8A and 8B are explanatory diagrams (part 1) illustrating a specific example of the result of line segment extraction processing, where FIG. 8A is a diagram illustrating an original image to be processed, and FIG. 8B is a diagram illustrating a character image to be extracted; c) is a diagram showing a line segment image to be removed.
FIG. 9 is an explanatory diagram (part 2) showing a specific example of the result of line segment extraction processing, where (a) is a diagram showing an original image to be processed, (b) is a diagram showing a character image to be extracted, c) is a diagram showing a line segment image to be removed.
FIG. 10 is an explanatory diagram (part 3) illustrating a specific example of a line segment extraction processing result, where (a) is a diagram illustrating an original image to be processed, and (b) is a diagram illustrating a character image to be extracted; .
FIG. 11 is an explanatory diagram (part 4) of a specific example of a line segment extraction process result; (a) is a diagram illustrating an original image to be processed; (b) is a diagram illustrating a character image to be extracted; c) is a diagram showing a line segment image to be removed.
FIG. 12 is a flowchart illustrating an outline of a processing operation performed by a line segment pixel extraction unit according to the second embodiment.
FIG. 13 is a block diagram illustrating a schematic configuration of an image processing apparatus according to a third embodiment.
FIG. 14 is a flowchart illustrating an overview of an image processing method according to a third embodiment.
FIG. 15 is a flowchart illustrating an outline of a processing operation performed by a used color detection unit according to the third embodiment.
FIG. 16 is an explanatory diagram illustrating an example of a pixel arrangement in the vicinity of a target pixel when performing a use color detection process;
FIG. 17 is an explanatory diagram illustrating an example of an original image that is a target of line segment extraction processing;
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 11 ... Image input part, 12 ... Color information storage part, 14 ... Image processing part, 14a ... Line segment pixel extraction part, 14b ... Unnecessary pixel removal part, 14c ... Image composition part, 14e ... Used color detection part

Claims (8)

Acquisition means for acquiring image data;
Color extracting means for extracting a predetermined number of color components included in the image data based on the appearance frequency of the color components in the image data acquired by the acquiring means;
Storage means for holding and storing color information relating to a predetermined number of color components extracted by the color extraction means ;
Detection means for detecting a distance on a color space between the image data acquired by the acquisition means and the color information held and stored by the storage means;
Line segment extraction means for extracting line segments from the image data acquired by the acquisition means only for the color components specified by the color information held and stored by the storage means based on the detection result by the detection means. An image processing apparatus.   The image processing apparatus according to claim 1, further comprising: a removing unit that deletes an extraction result that does not constitute a character element from line segment extraction results obtained by the line segment extracting unit.   When the line information is extracted for the plurality of color components when the color information regarding the plurality of color components is held and stored in the storage means, the plurality of colors obtained by the line segment extraction The image processing apparatus according to claim 1, further comprising a synthesizing unit that synthesizes line segment extraction results for the components.   The line segment extraction unit performs line segment extraction by binarizing image data acquired by the acquisition unit based on a detection result by the detection unit. 3. The image processing apparatus according to 3. An image processing method for extracting line segment data constituting a character element from image data,
Extracting a predetermined number of color components included in the image data based on the appearance frequency of the color components in the image data from which line segment data is to be extracted,
Stores and stores color information related to the predetermined number of extracted color components,
When acquiring image data from which line segment data is to be extracted, a distance on the color space between the image data and the color information is detected,
An image processing method, wherein only line segment data related to a color component specified by the color information is extracted from acquired image data based on a detected distance in a color space. 6. The image processing method according to claim 5 , wherein after the line segment data is extracted, line segment data that does not constitute a character element is deleted from the line segment data. 7. The method according to claim 5 , wherein, when color information relating to a plurality of color components is held and stored, when the line segment data relating to the plurality of color components is extracted, the extracted line segment data are synthesized. Image processing method. When a distance on the color space between the image data and the color information is detected, the image data to be extracted is binarized on the basis of the detected distance on the color space, so that 8. The image processing method according to claim 5, wherein line segment data is extracted.

Images