JP4194020B2 - Character recognition method, program used for executing the method, and character recognition apparatus - Google Patents

Description

図４に示すように、文字認識結果の1行において、(a)，(b)2種類の文字間の距離を集計し、上記の規則性に照らして、当該行が定ピッチであるか或いはプロポーショナルピッチであるかを判定することができる。規則性に照らして一行のピッチを判定した後に、規則性に反する不正なピッチの箇所を検出し、そこに文字切り出し誤りが存在すると判断する。例えば、
・定ピッチと判定した場合：中心線距離が等間隔でない部分は文字切り出し誤り
・プロポーショナルピッチと判定した場合：文字矩形間距離が等しくない部分は文字切り出し誤り
の存在をそれぞれ判断する。
【００２１】
図５は、本実施形態の文字認識処理のフローの一例を示すチャートである。
図５を参照して、文字の配置ピッチを判定して、文字切り出し誤りを検出するステップを含む本例の文字認識処理を説明する。
まず、スキャナーなどの画像入力機器によって、原稿の文書画像を入力する（step 1）。
次に、入力された文書画像から文字認識の対象となる文字候補の切り出す一連のステップとして、行切り出しまでの処理（上記、参照）を行う。即ち、入力画像中の黒ランから文字要素と思われる矩形を統合して外接矩形を抽出し（step 2）、この外接矩形について近隣する矩形同士を所定の基準に従い統合して、行に成長させ、行として切り出す（step 3）。
ここで、切り出した行に対して文字切り出し処理を行う（step 4）。この処理は、前段の処理で切り出された行には長手方向（横書きの場合には横方向）に複数文字が配列されているので、黒画素射影、行高さ、などに基づいて行を１文字だと思われる範囲（文字単位を含むと見なせる文字候補領域）に分割する。文字種等によってサイズが一定ではないので、複数の候補が得られることがあるが、本例では複数の候補があっても構わないような処理方法を採用する。
次に、前段の文字切り出しで求められた文字候補に対して文字認識の処理を行う（step 5）。文字認識の処理は、対象となる文字候補の画像特徴と認識辞書に記載されている文字の画像特徴とを照合し、認識スコアを算出し、認識スコアが予め設定したしきい値以上の解を認識候補として残す。
この後、前段の文字認識処理で求められた認識候補の並びと、言語辞書及び文法と照合して、認識スコアを加味した上、妥当な解を選択する後処理（言語処理）を行う（step 6）。ここでは、文字切り出しで複数の候補が得られた場合や認識処理で複数の候補が得られた場合でも、妥当な解を一つに絞りそれを処理結果とする。
【００２２】
次に、ここまでの処理ステップで文字がほぼ確定したので、その処理結果を受けて、文字の配置ピッチの種別の判定及び判定されたピッチの種類の規則性に基づいて文字切り出しの誤りを検出する一連の処理を行う。
この処理の始めに、2種類の矩形距離を測定する（step 7）。この測定は、認識結果の各文字の外接矩形をもとに、隣接する文字矩形同士の矩形間距離及び中心線距離（図４参照）を測定するものであり、その測定結果は記憶される。
また、step 7の処理で記憶した文字矩形間距離および矩形中心線距離をもとに、2種類の矩形距離それぞれの平均値を算出する（step 8）。
次いで、文字の配置ピッチの種別を判定するための指数として用いるために、step 7で記憶した隣接する文字矩形同士の文字矩形間距離および矩形中心線距離とstep 8で算出した各々の平均値とを比較し、平均値としきい値以上相違している矩形距離の回数を2種類の矩形距離それぞれにカウントする（step 9）。
ここで、step 9の結果を受けて、2種類の距離値が一定か否かを判定する（step 10）。本実施形態では、例えば、相違している回数（step 9のカウント値）にしきい値処理をする。即ち、カウント値が、全距離数(= 文字の間隔数 = 全文字数 − 1)に応じて設定したしきい値（所定の割合）より多ければ、距離は一定でないと判定する、という方法により実現する。なお、別の実現方法としては、距離値の標準偏差を求め、それがしきい値以上であれば、一定ではないと判定する方法を採用しても良い。
【００２３】
次に、step 10で2種類の距離値が一定であるか否かを判定した結果を受けて、ピッチの判定を行う（step 11）。ここでは、文字矩形間距離が一定であり、かつ矩形中心線距離が一定でない場合に、文字の配置ピッチの種別をプロポーショナルピッチと判定し、又文字矩形間距離が一定でなく、かつ中心線距離が一定である場合に、定ピッチと判定する。
この後、step 10の判定結果を受けて、以降の処理を分岐するので、判定結果が定ピッチであるか否かをチェックする（step 12）。
step 12で定ピッチである場合に、文字の再切り出し処理等の後処理へ対応するために、対象とする文字行内の文字の配置ピッチを調べて、定ピッチにならない部分を検出する（step 13）。具体的には、対象とする文字の矩形中心線間の距離がstep 8で算出した平均値と大きく相違する部分を見つけて、そこを切り出し誤り箇所とみなして検出する。このstep 13の処理を終え、本例のフローを終了する。
他方、step 12でプロポーショナルピッチである（定ピッチではない）場合にも、文字の再切り出し処理等の後処理へ対応するために、対象とする文字行内の文字の配置ピッチを調べて、文字矩形間距離が一定にならない部分を検出する（step 14）。具体的には、対象とする文字の文字矩形間距離がstep 8で算出した平均値と大きく相違する部分を見つけて、そこを切り出し誤り箇所とみなして検出する。このstep 14の処理を終え、本例のフローを終了する。
【００２４】
「実施形態２」
本実施形態は、切り出された文字候補の文字列における文字配置ピッチを用いて文字切り出し誤りを求める方法に係わるもので、文字飾り範囲を考慮して行う方法である。
上記「実施形態１」に示した方法では、文字切り出しの対象とする行において部分的に文字飾り(即ち、半角・全角の混合、横倍角等の標準的な文字サイズを所定の規則に従って相違させる修飾の施された文字)が施され、部分的に文字配置ピッチが異なる場合、切り出し精度を低下させる要因となる。そこで、本実施形態では、文字飾りによりピッチが変化する部分を検出し、その部分を文字切り出し誤りを求めるための情報から除外して、除外した後に上記「実施形態１」と同様のピッチの検出ルールを適用して、精度低下の回避を図るものである。
文字列における文字配置ピッチに生じる文字飾りによる変化の検出原理は、文字飾りによるピッチの変化が1箇所のみのピッチの変化に現れるとは考えにくいので、連続して文字の配置ピッチが標準的な値（実際には平均値による）から大きく相違する部分を文字飾りによる変化として検出することによる。
つまり、文字切り出しの対象とする行の平均ピッチを求めた後、そのピッチに適合しない範囲を検査し、下記の半角、横倍角の例に示すような文字飾り特徴を満たしていれば、その文字飾り種類を判定する。
半角： 切り出し対象行の文字飾りされてない部分において算出した矩形中心線距離の約半分の矩形中心線距離が連続する箇所
横倍角： 切り出し対象行の文字飾りされてない部分において算出した矩形中心線距離の約２倍の矩形中心線距離が連続する箇所
【００２５】
図６は、本実施形態の文字認識処理のフローの一例を示すチャートである。
図６を参照して、文字切り出し誤りを求めるための文字の配置ピッチ情報から文字飾りにより生じるピッチ情報を除外して、文字の配置ピッチを判定して、文字切り出し誤りを検出するステップを含む本例の文字認識処理を説明する。
なお、本例では、上記「実施形態１」に文字飾り範囲を考慮した処理ステップを付加してその手順を実行する。従って、「実施形態１」におけるstep 1〜step 8（図５）までの処理ステップと同一の処理を行う本実施形態におけるstep 1〜step 8（図６）については、先の説明を参照することとし、ここでは記載を省略する。
本実施形態において、step 8で文字矩形間距離および矩形中心線距離の2種類の矩形距離それぞれの平均値を算出した後、文字飾り範囲を判定するために必要な情報として用いるために、step 7で記憶した隣接する文字矩形同士の文字矩形間距離および矩形中心線距離とstep 8で算出した各々の平均値とを比較し、平均値としきい値以上相違している値が連続している箇所を探索する（step 29）。この後、step 29の探索結果を受けて、以降の処理を分岐するので、探索結果により連続している箇所があるか否かをチェックする（step 30）。
step 30で連続している箇所がない場合には文字飾り範囲判定（step 31）をパスするが、連続している箇所がある場合には、文字飾り範囲判定を行う（step 31）。この判定は、
半角加工： 切り出し対象行の文字飾りされていないと思われる部分(矩形 中心線距離及び文字矩形間距離が平均値に近い値の部分)にお いて算出した矩形中心線距離の約半分の中心線距離値が連続す る箇所
横倍角加工： 切り出し対象行の文字飾りされてないと思われる部分(矩形中心線距離及び文字矩形間距離が平均値に近い値の部分)において算出した矩形中心線距離の約２倍の中心線距離値が連続する箇所
の各条件を吟味し、満足していれば、文字飾り範囲とみなし、文字飾り範囲を示す情報と文字飾りの種類を記憶する。
【００２６】
次に、step 31で行った文字飾り範囲の判定結果を受けて、文字飾り範囲と判定された部分の情報をstep 7で記憶した隣接する文字矩形同士の文字矩形間距離および矩形中心線距離の情報から除いて、2種類の矩形距離それぞれの平均値を算出し、算出した各々の平均値とstep 7で記憶した隣接する文字矩形同士の文字矩形間距離および矩形中心線距離とを比較し、平均値としきい値以上相違している矩形距離の回数を2種類の矩形距離それぞれにカウントする（step 32）。
ここで、step 32の結果を受けて、2種類の距離値が一定か否かを判定する（step 33）。ただし、このときに文字飾り範囲と判定された部分については判定をしない。それ以外については、「実施形態１」におけるstep 10（図５）の処理と変わりがない。
また、本実施形態では、次のステップであるピッチの判定（step 34）以降のフローを終了するまでの処理ステップについても上記「実施形態１」と同一の処理を行う。従って、「実施形態１」におけるstep 10〜step 14（図５）までの処理ステップと同一の処理を行う本実施形態におけるstep 33〜step 37（図６）については、先の説明を参照することとし、ここでは記載を省略する。
【００２７】
「実施形態３」
本実施形態は、切り出された文字候補に認識処理を行って得た文字に、その文字に固有の特徴量として本来持っているはずのアスペクト比（文字の外接矩形の縦横比）によるチェックをかけて、文字切り出し誤りを求める方法に係わる。
上述したように、文字によってそれぞれの形状が異なるために、文字の外接矩形も一定ではない。一般的に漢字はややサイズが大きく、正方形なものが多いし、英数記号は正方形でないものの割合も多い。
文字コードをその外接矩形の形状別に予め分類しておけば、切り出された文字候補に文字認識処理を行って文字コードが求められた後に、予め用意した文字コードの文字が本来持っている外接矩形の形状特徴（アスペクト比）と、実際の認識画像の外接矩形の形状とを比較することによって、両者に不整合があれば、文字切り出しが正しくない可能性があることを検出することができる。
【００２８】
図７は、本実施形態の文字認識処理のフローの一例を示すチャートである。
図７を参照して、切り出された文字候補に文字認識処理を行って得た結果に、本来文字に固有の特徴量であるアスペクト比（文字の外接矩形の縦横比）を用いて文字切り出し誤りを求める本例の文字認識処理を説明する。
なお、本例では、切り出された文字候補に文字認識処理を行うまでの処理ステップは、上記「実施形態１」において行ったと同一の手順を実行する。従って、「実施形態１」におけるstep 1〜step 6（図５）までの処理ステップと同一の処理を行う本実施形態におけるstep 1〜step 6（図６）については、先の説明を参照することとし、ここでは記載を省略する。
本実施形態において、切り出された文字候補にstep 7で文字認識処理を行って複数の文字の切り出し候補や認識文字候補があった場合でも、最も妥当と判断される文字がstep 6で特定され、その段階で一旦文字切り出し位置が確定するので、そのとき、確定した文字に対し、文字のアスペクト比を算出する（step 47）。
次に、算出した文字のアスペクト比に基づいて、次に示すような文字切り出し誤りの検出を行う（step 48）。
文字認識の結果として特定される文字コードの文字が固有の特徴量として本来持っているはずの外接矩形のアスペクト比と、実際に認識処理の対象となった文字画像の外接矩形のアスペクト比とを比較し、一致するか否かをチェックしその結果により、文字切り出し誤りが有るか否かを検出する。
【００２９】
例えば、「明細」という単語の場合、偏と旁が分離しても１文字として成立し得る文字で構成されているので、文字切り出し位置の決定が難しいが、文字コードと文字外接矩形の形状特徴との組み合わせによって、明らかに正しくない解は棄却できる。下記の〔第１表〕に「明細」を例にして、文字（文字コードで表現）と該文字から想定される文字外接矩形の形状特徴との対応表を例示する。

例えば、次のような認識結果に対して〔第１表〕を適用し一致性を判断した場合の結果を以下に示す。
明：正方形
糸：やや縦長 →× 〔第１表〕では正方形であり、合致しない
田：やや縦長 →× 〔第１表〕では正方形であり、合致しない
この一致性の判断結果により、「糸」「田」の部分が文字切り出し誤りである可能性が大であることが検出できる。
【００３０】
ところで、形状の特徴を示す「やや縦長」「正方形」などの表現は、文字の外接矩形のアスペクト比(縦横比)の一つの表現形式であり、数値によって定義することも可能である。
下記〔第２表〕に文字コードに対するアスペクト比の対応表を例示する。表中の文字SJISのアスペクト比(= 縦／横)は、標本の平均値Av. で示され、
外接矩形が縦長の文字ではAv. が１より大きい
外接矩形が正方形に近いものはAv. が１に近い
外接矩形が横長の文字ではAv. が１より小さい
を表現することができる。

【００３１】
認識結果に〔第２表〕を適用し一致性を判断することにより、認識対象とした文字の切り出し誤りを検出した実施形態における検出結果を下記の〔第３表〕に示す。
〔第３表〕中に示される、(a) 認識結果の外接矩形の実際のアスペクト比は、外接矩形座標から求められる。また、(b) 認識結果の文字コードから予想されるアスペクト比の平均値および(c) 標準偏差はそれぞれ〔第２表〕から求められる。
切り出し誤りを検出する手順は、実際の矩形のアスペクト比((a))と認識結果の文字コードから予想されるアスペクト比((b)、(c))とを比較し、大きく相違していれば、文字切り出し誤りだと判定する。
〔第３表〕の例では、正規分布の特徴に基づいて、
(b) - (c) * 3 < (a) < (b) + (c) * 3 ならば、相違無し(= 誤差範囲内)
(b) - (c) * 3 < (a) < (b) + (c) * 3 でなければ、相違有り
と判定して、◯×で表示している。
〔第３表〕に示すように、アスペクト比に注目することによって、
「理」→「王」「里」
のように誤って文字切り出ししてしまった箇所が検出できることがわかる。
ここで用いた判定条件：
(b) - (c) * 3 < (a) < (b) + (c) * 3
は、平均値よりどれだけ離れているかを標準偏差の定数倍を基準に判定しているが、この定数倍(例では3倍)を調整することによって、検出条件を厳しく／緩く、調整可能であることは言うまでもない。また、この判定条件はあくまで一例であり、標準的なアスペクト比と認識結果のアスペクト比を比較して、予め設定した範囲よりも相違していたら、文字切り出し誤りだとすることが実現できれば、実施例に示した条件に限定されるものではない。相違の範囲を限定する方法としては、例えば、アスペクト比の差や比にしきい値を設けることでも実現可能である。

【００３２】
「実施形態４」
本実施形態は、切り出された文字候補に認識処理を行って得た文字に、その文字に固有の特徴量として本来持っているはずのアスペクト比によるチェックをかけて、文字切り出し誤りを求める方法に係わるもので、文字飾り範囲を考慮して行う方法である。
上記「実施形態３」に示した方法では、文字切り出しの対象とする行において部分的に文字飾り(即ち、半角、横倍角等の標準的な文字形状を所定の規則に従って相違させる修飾の施された文字)が施されている場合に認識結果から予想される標準的なアスペクト比によりチェックすると、切り出し誤りが検出され、切り出し精度を低下させる要因となる。そこで、本実施形態では、文字飾りをかけた文字部分を検出し、その部分を文字切り出し誤りを求めるための情報から除外して、除外した後に上記「実施形態３」と同様のアスペクト比による検出ルールを適用して、精度低下の回避を図るものである。
文字列に生じる文字飾りによる変化の検出原理は、文字飾りによりアスペクト比が予想される値より大きく異なる場合が1文字のみに現れるとは考えにくいので、連続して文字のアスペクト比が予想される値より大きく相違する部分を文字飾りによる変化として検出することによる。
つまり、文字切り出しの対象とする行の文字認識後の文字に対し順次アスペクト比を算出し、アスペクト比が予想される値に適合しない範囲を検査し、下記の半角、横倍角の例に示すような文字飾り特徴を満たしていれば、その文字飾り種類を判定する。
半角： 切り出し対象文字について算出したアスペクト比が 0.4以上 0.6以下で連続していたら半角と判定し、その文字部分は、そのアスペクト比を2倍して予想アスペクト比と比較
横倍角： 切り出し対象文字について算出したアスペクト比が 1.7以上 2.2以下で連続していたら横倍角と判定し、その文字部分は、そのアスペクト比を1/2倍して予想アスペクト比と比較
本実施形態は、文字飾り範囲を考慮した上述の処理を上記「実施形態３」に示した文字認識処理フロー（図７）のアスペクト比に基づく文字切り出し誤りの検出ステップ（step 48）において実行する、という形態で実施し得る。
【００３３】
「実施形態５」
本実施形態は、切り出された文字候補に認識処理を行って得た文字に、その文字に固有の特徴量として本来持っているはずのアスペクト比によるチェックをかけて、文字切り出し誤りを求める方法に係わるもので、文字切り出し誤りの検出結果を提示する処理を伴う方法である。
上記「実施形態３」に示した方法では、文字切り出し誤りが検出された場合に、検出結果の利用の仕方が示されていない。本実施形態では、文字切り出し誤りの検出結果の利用方法を示すものである。
一般的な手法に従う上記した文字認識過程では、認識辞書との照合程度、言語的な尤度などの観点から認識スコアを算出するので、そのスコアにより認識結果がどの程度信頼できるかを示す認識信頼度が付与されている場合が多い。
この認識信頼度の利用法として、認識信頼度が低い場合に、認識結果を利用するユーザに知らせるようにすれば、もし認識に間違いがあっても、ユーザによるチェックや訂正作業を著しく効率化することか可能になる。このために、信頼度の低い文字はユーザに提示する際に、表示形態を変えるようにして、ユーザに注意を喚起し、その発見を容易にする。
ここでは、文字切り出し誤り検出によって、誤りだと判定された文字は少なくとも、表示形態を変えて提示する文字の対象とすることが適当である。
また本実施形態は、表示形態を変えて認識信頼度の低い文字を提示する上述の処理を上記「実施形態３」に示した文字認識処理フロー（図７）のアスペクト比に基づく文字切り出し誤りの検出ステップ（step 48）に付随して実行する、という形態で実施し得る。
【００３４】
「実施形態６」
本実施形態は、切り出された文字候補に認識処理を行って得た文字に、その文字に固有の特徴量として本来持っているはずのアスペクト比によるチェックをかけて、文字切り出し誤りを求める方法に係わるもので、文字切り出し誤りの検出結果に従い再度切り出し、認識処理を行う方法である。
上記「実施形態３」に示した方法では、文字切り出し誤りが検出された場合に、検出結果の利用の仕方が示されていない。本実施形態では、文字切り出し誤りの検出結果を用いて、誤り検出部分に新たな切り出し・認識を行い元の認識結果と競合させてより精度の高い認識処理を可能とするものである。
より詳細には、上記のようにアスペクト比によるチェックを最初にかけて、文字切り出し誤りが検出されても、一連の認識処理を経てきて選択された結果であり、実際には正解である可能性もあるので、文字切り出し誤りの判定により直ちに棄却してしまうのは危険である。そこで、元の解を棄却しないで保持しておき、文字切り出し誤りと判定された部分に、再度文字切り出し処理を行い、新たな文字切り出し位置候補を求め、ここに含まれる文字に対する認識処理を行い、新たに求められた文字候補(切り出し位置が異なる)を保持しておいた元の解と競合させて、後処理(言語処理)を行い、即ちこれまに対象とした全ての文字候補の中から尤度を表す認識スコアの最も高い文字を選択する。したがって、再度、元の解が選択される場合もあり得るが、構わない。
なお、本実施形態は、文字切り出し誤りが判定された部分に再度一連の認識処理を行う上述の処理を上記「実施形態３」に示した文字認識処理フロー（図７）のアスペクト比に基づく文字切り出し誤りの検出ステップ（step 48）に伴って実行する、という形態で実施し得る。
【００３５】
「実施形態７」
本実施形態は、切り出された文字候補に認識処理を行って得た文字に、その文字に固有の特徴量として本来持っているはずのアスペクト比によるチェックをかけて、文字切り出し誤りを求める方法に係わるもので、文字切り出し誤りの検出結果に従い再度切り出し、認識処理を行う方法である。
上記「実施形態６」に示した方法では、文字切り出し誤りが検出された場合の再切り出し・認識処理の対象部分として、誤り検出部分の隣接部分を考慮していなかったが、文字切り出し誤りは、検出された文字だけでなく、隣接する文字にも影響する可能性が高い。したがって、誤りとして検出された文字部分だけでなく、隣接する文字も含めた範囲において再度、切り出し候補を見つけることによって、誤り検出された文字だけでなく、検出から漏れたが実は誤認識していた隣接文字をも正解に転じることが可能になる。隣接する文字は、切り出された行が横行の場合左右に隣接する文字が、又縦行の場合上下に隣接する文字が対象になる。
なお、本実施形態は、「実施形態６」と同様に、文字切り出し誤りが判定された部分に隣接部分を加えて再度一連の認識処理を行う上述の処理を上記「実施形態３」に示した文字認識処理フロー（図７）のアスペクト比に基づく文字切り出し誤りの検出ステップ（step 48）に伴って実行する、という形態で実施し得る。
【００３６】
「実施形態８」
本実施形態は、切り出された文字候補に認識処理を行って得た文字に、その文字に固有の特徴量として本来持っているはずのアスペクト比によるチェックをかけて、文字切り出し誤りを求める方法に係わるもので、アスペクト比の変動の激しい文字を考慮して行う方法である。
上記「実施形態３」に示した方法では、文字切り出しの対象とする行において部分的にアスペクト比の変動の激しい文字、例えばドット系の文字(即ち、／・／．／，／“／”／などの文字)が施されている場合に認識結果から予想される標準的なアスペクト比によりチェックすると、切り出し誤りが検出され、切り出し精度を低下させる要因となる。そこで、本実施形態では、ドット系の文字等の部分を検出し、その部分を文字切り出し誤りを求めるための情報から除外して、除外した後に上記「実施形態３」と同様のアスペクト比による検出ルールを適用して、精度低下の回避を図るものである。
詳細には、例えばドット系の文字(／・／．／，／“／”／など)は、画像品質（解像度）に影響されて画素数が変動した場合、この文字の性質として縦横ともに画素数が少ないために、少々の変動でも、アスペクト比は大きく変動する。本来、縦長だったものでも横長になってしまうことも少くない。
そこで、ドット系の文字等のアスペクト比の変動の激しい文字の集合を予め設定しておき、この集合に含まれる文字はアスペクト比による文字切り出し誤り判定から除外する。これによって、本質的に正確な誤り位置検出を期待できない文字に対して、誤り検出処理を行う無駄を避けることができ、都合がよい。
なお、本実施形態は、アスペクト比の変動の激しい文字を考慮した上述の処理を上記「実施形態３」に示した文字認識処理フロー（図７）のアスペクト比に基づく文字切り出し誤りの検出ステップ（step 48）において実行する、という形態で実施し得る。
【００３７】
「実施形態９」
本実施形態は、切り出された文字候補に認識処理を行って得た文字に、その文字に固有の特徴量として本来持っているはずのアスペクト比によるチェックをかけて、文字切り出し誤りを求める方法に係わるもので、言語処理の結果を反映した処理を行う方法である。
上記「実施形態３」に示した方法では、単一文字として扱う場合に認識結果から予想される標準的なアスペクト比によりチェックすると、切り出し誤りが検出されるが、言語的な正しさからは切り出しに間違いがない場合があり、単一文字のアスペクト比によるチェックでは切り出し精度を低下させる要因となる。そこで、本実施形態では、言語的に正しいと判定された部分を文字切り出し誤りを求めるための情報から除外して、除外した後に上記「実施形態３」と同様のアスペクト比による検出ルールを適用して、精度低下の回避を図るものである。
詳細には、本来の正解文が言語的に正しいという前提では、文字切り出し誤り箇所は、その言語的な正しさが乱れてしまっている可能性が高い。逆にいえば、アスペクト比や矩形間距離に基づく文字切り出し誤り検出処理によって誤りだと判定されたとしても、言語的に正しければ正解であると考えた方が妥当である。よって、辞書単語に合致している範囲（文字列）は、文字切り出し誤りと判定されたとしても、誤りとはみなさない。
文字切り出し誤り検出対象である認識結果は、言語処理を経て求められたものであるから、当該文字が辞書単語内であるか否かを判定することは容易である。文字認識結果の内部データに照合した単語と、その何文字目なのかを記憶する項目を設ければよい。
なお、本実施形態は、言語的に正しいと判定された単語を考慮した上述の処理を上記「実施形態３」に示した文字認識処理フロー（図７）のアスペクト比に基づく文字切り出し誤りの検出ステップ（step 48）において実行する、という形態で実施し得る。
【００３８】
「実施形態１０」
本実施形態は、切り出された文字候補に認識処理を行って得た対象文字・文字列の文字配置ピッチ及び文字アスペクト比それぞれの算出結果の組み合わせ（文字の連続する並び）に、その文字・文字列の文字ピッチ及び文字アスペクト比の組み合わせに特有の性質（連続する並びの性質）によるチェックをかけて、文字切り出し誤りを求める方法に係わる。
上述の文字の連続する並びは、切り出された文字候補列に文字認識処理を行ったときに得た文字種・文字外接矩形の形状（アスペクト比）と2種の文字配置ピッチの条件（文字中心線距離、文字矩形間距離それぞれの行内標準値との違い）で表現する。一方、連続する並びの性質は、標準原稿の学習により連続する並びが違う場合（つまり文字種・文字外接矩形の形状（アスペクト比）と2種の文字配置ピッチの条件が違う場合）の出現確率をそれぞれ予め求めておく。
文字切り出し誤りの検出の際には、実際の対象文字列について連続する並びを得、得た並びの存在確率を標準原稿の学習により用意したそれぞれの連続する並びの出現確率を参照することにより求め、求めた出現確率の低い箇所を文字切り出しが正しくない可能性がある箇所として検出する。
【００３９】
図８は、本実施形態の文字認識処理のフローの一例を示すチャートである。
図８を参照して、本実施形態の文字認識処理を説明すると、本例では、切り出された文字候補に文字認識処理を行い、2種の矩形距離を求め、それらの平均値を求めるまでの処理ステップは、上記「実施形態１」において行ったと同一の手順を実行する。従って、「実施形態１」におけるstep 1〜step 8（図５）までの処理ステップと同一の処理を行う本実施形態におけるstep 1〜step 8（図８）については、先の説明を参照することとし、ここでは記載を省略する。
本実施形態において、切り出された文字候補にstep 7で文字認識処理を行って複数の文字の切り出し候補や認識文字候補があった場合でも、最も妥当と判断される文字がstep 6で特定され、その段階で一旦文字切り出し位置が確定するので、そのとき、確定した文字に対し、文字のアスペクト比を算出する（step 59）。
次に、算出した文字のアスペクト比及び2種の矩形距離とそれらの平均値に基づいて、次に示すような文字切り出し誤りの検出を行う（step 60）。
【００４０】
この文字切り出し誤り検出ステップでは、上述のように、対象とする文字の連続する並びを文字種・文字外接矩形のアスペクト比と2種の文字配置ピッチ条件で表現し、その並びの性質を調べる。
対象とする文字は、本来文字毎にアスペクト比が異なるが、実際には変動があるので精密にアスペクト比を数値で設定する必要はなく、代表的な有限個に分類しておけばよい。
本例では、例えば、アスペクト比を特徴的な下記の３種、

に分類し、文字コードの文字種を３種（漢字／ひらがな、カタカナ／英数記）に大別したものと組み合わせて、下記の９種(＝ ３種×３種)に分類する。
漢字：縦長
漢字：正方形
漢字：横長
ひらがな、カタカナ：縦長
ひらがな、カタカナ：正方形
ひらがな、カタカナ：横長
英数記：縦長
英数記：正方形
英数記：横長
一方、上記「実施形態１」で言及したように、文字の配置ピッチには代表的な２種があり、行内での標準的なピッチ幅（図８step8で算出した平均値）を基準にしてその長、短で表現すれば、各矩形間のピッチは下記の６種(＝ ３＋３)に分類される。
文字中心線距離（標準、長、短：３種）
文字矩形間距離（標準、長、短：３種）
また、上記「実施形態１」でも示したように、文字の外接矩形のアスペクト比と、文字の配置ピッチとの間には関連性がある。そこで、文字の外接矩形のアスペクト比と、配置ピッチとをまとめて分類して以下のように定義する。

ここに、分類の基準は、
a 文字矩形の分類（９種）
b 文字中心線距離（標準、長、短：３種）
c 文字矩形間距離（標準、長、短：３種）
であるから、種類の総数は、
a の種類 × b の種類 × c の種類 ＝ ９×３×３ = ８１種
と求められる。
【００４１】
このように、各基準（リファレンス）を有限個に分類することによって、文字の外接矩形のアスペクト比と、文字配置ピッチを併せて同時に有限個の種類で表現することが可能である。
この外接矩形のアスペクト比と配置ピッチとの連続する並びの性質をリファレンスとして記憶しておき、認識結果においても、それらを求め、記憶している性質と比較して、大きく相違していたら、文字切り出し誤りだと判定することができる。
連続する並びの性質を学習する手段としては n-gram が考えられる。これは、81種の例によると、81種の連続する並び分類を81種のシンボルでラベル付けしておき、学習時には、誤認識のほとんどない原稿によって、このシンボル列の n-gram の出現確率を学習する。
例えば、シンボルの３連続を定義する trigram を利用すれば、
P(xi | xi-2, xi-1)：xi-2, xi-1 が連続して出現した下に xi が出現する事後確率
を求めることが、連続する並びの性質を学習することに相当する。
ここで xi-2, xi-1, xi は、外接矩形のアスペクト比と配置ピッチを表現したシンボルにあたる。 P(xi | xi-2, xi-1) は、以下のように正しく文字切り出しされた学習データから容易に求めることができる。
P(xi | xi-2, xi-1) = C( xi-2, xi-1, xi) / C( xi-2, xi-1)
C( xi-2, xi-1) ： xi-2, xi-1 が連続して出現する数
C( xi-2, xi-1, xi) ： xi-2, xi-1, xi が連続して出現する数
trigram でモデル化すれば、81×81×81=531441種のtrigramに対する確率値をテーブルによって学習結果を記憶できる。出現しないか或いは出現しても少数のtrigramも多いことが予想され、その場合は低い一定値で代用しても問題はないので、記憶すべき trigramはこれより少なくなるであろう。
【００４２】
図９は、学習した trigram を用いて文字切り出し誤り検出を行う方法を説明するための図である。
切り出された文字候補に文字認識処理を行い、認識結果の外接矩形のアスペクト比、2種の文字配置ピッチを求め、各文字を trigram で用いるシンボルに変換する。この変換後に、学習した trigram の出現確率を順に適用していき、低い確率値を示す部分が、文字切り出し誤り箇所だと判定する。
図９の例において、「くな」が合併して1文字と誤って文字切り出しされ、「件」と認識されてしまったとする。その他は正しく認識されると仮定する。このときの各文字に trigram を適用した結果を下記の〔第４表〕に示す。

〔第４表〕の出現確率は図９の下部にプロットしている。図９に示すように、低確率しきい値を下回る部分を文字切り出し誤りであると判定する。
【００４３】
「実施形態１１」
本実施形態は、本発明に係わる文字認識装置の実施形態を示すものである。
上記した「実施形態１」〜「実施形態１０」に示した文字切り出し誤りの検出手順を含む文字認識方法に示した処理ステップを実行する手段として、汎用の処理装置（コンピュータ）を利用して構成される装置を例示するものである。
図１０は、本実施形態の文字認識装置の構成を例示する。図１０に示すように、本例は、汎用の処理装置（コンピュータ）により実施する例を示すものであり、構成要素としてCPU１、メモリ２、ハードディスクドライブ３、スキャナ、キーボード、マウス等の入力装置４、CD-ROMドライブ５、ディスプレイ６、フレキシブルディスクドライブ７、通信装置８などを用意し、これらをバス接続して構成する。
また、記憶手段としてのメモリ２、ハードディスクドライブ３、CD-ROMドライブ５、フレキシブルディスクドライブ７が用いる記憶媒体（図示せず）の一部には、本発明に係わる文字認識処理機能を実現し、上記「実施形態１」〜「実施形態１０」に示した文字切り出し誤りの検出手順を含む文字認識方法に示した各処理ステップを実行するためのプログラム（ソフトウェア）が記録されている。
処理対象の原稿文書画像は、スキャナー等の入力装置４により入力され、例えばハードディスク３などに格納されているものである。CPU１は、記憶手段が有する記録媒体から上記した処理機能・処理方法を実現するプログラムを読み出し、プログラムに従う処理を対象文書画像に実行し、その処理結果等をディスプレイ６に出力する。
なお、本発明に係わる文字認識装置を図１１に示すように、通信装置８によりインターネットなどの通信回線２０を介して、外部の装置１１〜１３と接続して、機能の一部をネットワーク上に持つような形態で実施してもよい。
また、本発明の文字認識方法の各処理ステップの実行プログラムを本実施形態のように汎用コンピュータに搭載することによって実施する以外に、各処理ステップの実行に必要な機能をファームウェアや電気回路の形態によって利用装置に組み込むことにより文字認識装置を構成することも可能である。
【００４５】
【発明の効果】
（１）請求項１、４の発明に対応する効果
文字やその配置が未知の段階で切り出された文字候補（複数存在）に文字認識処理を行い、認識結果がほぼ確定した段階で、この結果により定まった文字を対象に文字アスペクト比（形状特徴）を求め、予め辞書に格納した当該文字の標準化された文字アスペクト比と比較した結果、大きく相違する文字が存在する箇所を文字切り出し誤りとして検出するようにしたので、文字切り出し誤りを高精度に検出可能とし、また文字配置ピッチ以外のアプローチを可能にする。
さらに、文字切り出し誤りと判定された文字候補領域に再切り出しと再文字認識を行い、元の結果と併せて妥当性を評価し、尤もらしい解を選択するようにしたので、より高精度の文字認識をすることが可能になる。さらに、文字配列中の切り出し誤りと判定された文字候補領域の両側に隣接する文字を加えた範囲を対象に再切り出しを行うようにしたので、さらに高精度化を図ることが可能になる。
（２）請求項２の発明に対応する効果
文字切り出し誤りの検出結果から文字飾り判定し、判定した前記文字列の文字のアスペクト比を前記文字飾りが施されていない状態にして、標準化された文字アスペクト比との比較により再判定を行うことにより文字切り出し誤りの精度を低下させる要因に対応した処理をしたので、より精度を向上させることができる。
（３）請求項３の発明に対応する効果
切り出し誤りが判定された文字に対する認識結果への信頼度の低下を提示し、認識信頼度が低い場合に、その認識結果を利用するユーザに知らせるようにしたので、認識に間違いがあっても、ユーザによるチェックや訂正作業を著しく効率化することが可能になる。
（４）請求項５の発明に対応する効果
アスペクト比の変動の激しい文字（例えばドット系の文字）を通常の文字切り出し誤り判定の対象から除外するようにしたので、精度低下を避け、本質的に正確な誤り位置検出を期待できない文字に対して、誤り検出処理を行う無駄を避けることが可能になる。
（５）請求項６の発明に対応する効果
文字認識ステップで言語処理によって辞書単語と照合されている文字を通常の文字切り出し誤り判定の対象から除外し、言語の正しさを優先させるようにしたので、精度低下を回避することが可能になる。
（６）請求項７の発明に対応する効果
文字やその配置が未知の段階で切り出された文字候補（複数存在）に文字認識処理を行い、認識結果がほぼ確定した段階で、この結果により定まった文字を対象に文字矩形間距離、中心線距離及び文字アスペクト比（形状特徴）を求め、これらの情報を組み合わせ、この組み合わせの存在確率を予め標準原稿で学習し、格納したデータ値を参照して得ることにより、文字切り出し誤りの判定の基礎としたので、文字切り出し誤りを高精度に検出可能とし、安定した認識結果を得ることを可能にする。
（７）請求項８、９の発明に対応する効果
請求項１乃至７に記載された文字認識方法の各処理ステップを実行するためのプログラムを汎用のコンピュータに搭載することにより、上記（１）〜（６）の効果を容易に具現化し、また、該効果を奏する文字認識装置を提供することが可能になる。
【図面の簡単な説明】
【図１】 処理対象となる文書画像の一例を示す。
【図２】 文書画像の例（図１）における文字と見なせる黒ランの外接矩形を作成した結果を示す。
【図３】 統合処理の結果得られる文字行の矩形と文字外接矩形を示す図である。
【図４】 対象文字行における文字認識結果と2種類の文字矩形距離を説明する図である。
【図５】 「実施形態１」の文字認識処理のフローの一例を示すチャートである。
【図６】 「実施形態２」の文字認識処理のフローの一例を示すチャートである。
【図７】 「実施形態３」の文字認識処理のフローの一例を示すチャートである。
【図８】 「実施形態１０」の文字認識処理のフローの一例を示すチャートである。
【図９】 学習した trigram を用いて文字切り出し誤り検出を行う方法を説明するための図である。
【図１０】 本発明の実施形態に係わる文字認識装置の構成を示す。
【図１１】 本発明の実施形態に係わる文字認識装置の他の構成を示す。
【符号の説明】
１…CPU、 ２…メモリ、
３…ハードディスクドライブ、 ４…入力装置、
５…CD-ROMドライブ、 ６…ディスプレイ（表示装置）、
７…FDドライブ、 ８…通信装置。[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a character recognition process used in an OCR (optical character reading device) or the like, and more specifically, cuts out an area including one character (character unit) based on image data read from a document original. The present invention relates to a character recognition method including processing steps for performing "" with high accuracy, a program used for executing the method, and a character recognition device.
[0002]
[Prior art]
Conventionally, as performed in an OCR (optical character reader) or the like, processing for recognizing characters written on a document based on image data read from the document document by a scanner is performed. In performing this process, it is indispensable to accurately perform “character segmentation” for extracting character unit data from the read document image data in order to obtain high recognition accuracy.
As a character cut-out method, a method for deriving a cut-out condition from the regularity appearing in the print pitch (assuming that the original is printed according to a set print condition) or various attributes of the character. Has been proposed.
In Japanese Patent No. 3148106 (character segmentation method), which can be shown as an example of the former, when character segmentation is performed from a character string whose print pitch width is unknown, first, a plurality of projection (projection) distributions are used. Character pitch is assumed, and the most appropriate one is determined to be an estimated character pitch, which is applied to character string segmentation. However, this method is suitable for characters having a constant pitch (the distance between the center lines of the characters is constant), and may be applied to the case where characters are arranged according to other rules (for example, proportional pitch described later). The pitch width cannot be obtained accurately, and the accuracy is not sufficiently guaranteed.
[0003]
In addition, Japanese Patent Laid-Open No. 5-197840 (character cutout control method), which can be shown as the latter example, refers to feedback control for recutout, and uses graphic information to determine the likelihood of the cutout portion. Character recognition is performed on cutout candidates that are evaluated according to generalized criteria and determined in descending order, and feedback is given to indicate a portion that needs to be cut out again based on the recognition result. However, in this method, the information to be fed back instructs the re-extraction of the part that failed to be recognized, that is, only instructs the processing for the next candidate, and is determined by the above generalized evaluation criteria one after another until the recognition succeeds. The process for the candidate is repeated, and the number of processes may increase.
Further, as the latter example, Japanese Patent Laid-Open No. 8-212301 (character cutting device and its cutting method) determines the character type of the input character (determined based on the number or shape of character data) The purpose is to change the cut-out parameters and realize high-precision character cut-out. However, the character type determination method assumes input by online writing and is based on the number of strokes or shape of character data, and is not suitable for off-line character recognition. In addition, since the outer shape of the character image is greatly different even within the same character type, there is a limit in changing the extraction parameter based only on the character type information, and it cannot be said that it is sufficient for realizing high-precision extraction processing.
[0004]
[Problems to be solved by the invention]
The present invention has been made in view of the above-described problems of the prior art in character extraction when character data is extracted from read image data of a document original and character recognition is performed on the extracted portion. The object is to provide a character recognition method including a processing step (or processing means) that enables proper character segmentation even when applied to a character arrangement other than characters arranged at a fixed pitch, a program used to execute the method, and The object is to provide a character recognition device.
In addition, the present invention has been made in view of the above-described problems of the prior art in character extraction when character-unit data is extracted from read image data of a document original and character recognition is performed on the extracted portion. The purpose is to enable feedback of appropriate cutout information from the character recognition result for the character cutout candidate, and to repeat the process to the candidate determined based on the generalized evaluation criteria, which increases the number of processes. It is possible to avoid the prior art processing, and further, a processing step (or suitable for off-line character recognition for a document printed in accordance with set printing conditions and enabling more accurate character segmentation (or A character recognition method provided with processing means), a program used to execute the method, and a character recognition device A.
[0007]
[Means for Solving the Problems]
  According to the first aspect of the present invention, a character extraction step of extracting a character candidate area that can be regarded as including a character unit from a document image having a character array input as a processing target, and collation with a character image feature in which the extracted character is stored in a dictionary A character recognition method including a character recognition step for recognizing characters based on a collation result, and further calculating a character aspect ratio for calculating an aspect ratio of a circumscribed rectangle of the character for the character recognized in the character recognition step A character cut-out error determination step for comparing a character aspect ratio calculation result with a standard character aspect ratio of the character stored in the dictionary in advance to determine a character cut-out errorA character re-cutting step for re-cutting the character candidate area determined to be a cut-off error by the character cut-off error determining step, character recognition of the re-cut-out character, and evaluating the validity together with the original result Re-character recognition step to select plausible solutionsIs a character recognition method characterized by including.
  The invention according to claim 2 is the character recognition method according to claim 1, wherein the character cut-out error determination step sequentially calculates an aspect ratio with respect to the characters after character recognition in the line to be cut out, Inspect the range where the aspect ratio does not match the expected value, and if the non-conforming continuous character string satisfies the character decoration characteristics, determine the character decoration type, and determine the character aspect ratio of the character string In a state where the character decoration is not applied, and the determination is performed again by comparison with a standardized character aspect ratio.
  According to a third aspect of the present invention, in the character recognition method according to the first or second aspect, the step of presenting a decrease in the reliability of the recognition result for the character for which the cut error has been determined by the character cut error determining step. It is the character recognition method characterized by including.
  According to a fourth aspect of the present invention, in the character recognition method according to any one of the first to third aspects, the character re-cutout step includes a character adjacent to both sides of the character candidate area determined to be a cut-out error in the character arrangement. This is a character recognition method characterized by performing re-cutting for a range to which the character is added.
  According to a fifth aspect of the present invention, in the character recognition method according to any one of the first to fourth aspects, the character cut-out error determination step excludes a character whose aspect ratio fluctuates from a normal character cut-out error determination target. This is a character recognition method characterized by the above.
  According to a sixth aspect of the present invention, in the character recognition method according to any one of the first to fifth aspects, the character cut-out error determination step normally includes characters that are collated with dictionary words by language processing in the character recognition step. The character recognition method is characterized in that it is excluded from the target of character cutout error determination.
  According to a seventh aspect of the present invention, there is provided a character extracting step of extracting a character candidate area that can be regarded as including a character unit from a document image having a character arrangement input as a processing target, and collation with a character image feature in which the extracted character is stored in a dictionary A character recognition method including a character recognition step for recognizing characters based on a collation result, further targeting a character array recognized in the character recognition step and a distance between adjacent rectangles and adjacent rectangles in a circumscribed rectangle of the character A character pitch calculating step for calculating a rectangle center line distance of the character, a character aspect ratio calculating step for calculating an aspect ratio of a circumscribed rectangle of the character for the character recognized in the character recognition step, and a target character / character. Presence probability of combinations of calculation results for character pitch and character aspect ratio of columns is prepared in advance as a standard value A character recognition method comprising: a character cutout error determination step for determining a character cutout error based on a reference value obtained by referring to the existence probability of the combination of the character pitch and the character aspect ratio. is there.
  The invention according to claim 8 is a program for causing a computer to execute each processing step of the character recognition method according to any one of claims 1 to 7.
  A ninth aspect of the present invention is a character recognition device comprising a computer on which the program according to the eighth aspect is mounted, and processing data of a target image by the computer.
[0017]
DETAILED DESCRIPTION OF THE INVENTION
The present invention will be described based on the following embodiments shown with the accompanying drawings.
The present invention provides a method for realizing high-accuracy character recognition. Character candidates that can be regarded as including character units are cut out from a character string, character recognition is performed, and recognition is performed when the recognition result is almost finalized. Character cutout error positions in the cutout character candidates are accurately detected using information such as the character arrangement pitch in the target character / character string and the aspect ratio which is a characteristic amount unique to the character. Furthermore, after detecting a character cutout error, the detected portion is cut out again, and character recognition is performed again for the added character candidate, thereby realizing the object.
In the following “Embodiment 1” and “Embodiment 2”, character extraction is performed using the distance between adjacent rectangles in the circumscribed rectangle of the character and the rectangle centerline distance information of the adjacent rectangle for the character arrangement recognized in the primary processing. An example for obtaining an error is shown. “Embodiment 3” to “Embodiment 8” show examples for obtaining a character segmentation error using information such as an aspect ratio that is a characteristic amount unique to a character, and “Embodiment 10” An example of obtaining a character segmentation error by trigram using information such as distance between adjacent rectangles in a circumscribed rectangle of a character, rectangle centerline distance information of the adjacent rectangle, and information such as an aspect ratio that is a character-specific feature amount will be described.
[0018]
FIG. 1 shows an example of a document image to be processed. In each of the following embodiments, a horizontal text manuscript written in Japanese will be described as an example. However, unless otherwise specified, the present invention is not limited to an example, and the arrangement of character strings included in a character image is not limited. Determining the pitch, determining the character placement pitch portion of the character cut out in accordance with a predetermined rule as a character cut-out error, and preparing a standard value of a circumscribed rectangle characteristic specific to each character in advance, By comparing with the circumscribing rectangle of the character to be recognized, if there is a difference exceeding the allowable range, it is determined that the character cutout error is high, and it is guaranteed that the character cutout process is performed with high accuracy. Language, character image type (handwritten / printed characters, etc.) and format (vertical writing / horizontal writing).
In addition, as a process common to each of the following embodiments, a process of cutting out a character candidate that can be regarded as including a character unit from a target document image is performed. Here, since a character line is used as a unit, character candidate extraction is performed. In the previous stage, the line cut-out process is first performed.
In this line cut-out process, a cut-out method by extracting a circumscribed rectangle of a black run in the target image is applied. In this clipping method, since characters are usually drawn in black, a black run that is a cluster of continuous pixels indicating a black value is obtained, and rectangles that are considered to be character elements are integrated from the obtained black run to extract a circumscribed rectangle. FIG. 2 shows the result of extracting this circumscribed rectangle. Next, with respect to the circumscribed rectangle of the black run that seems to be the extracted character element, the neighbors are connected according to a predetermined standard, grown into a line, and a process of cutting out this as a target line is performed. FIG. 3 shows the result obtained by the row cutout process.
Thereafter, a character candidate to be recognized is further cut out from the line.
[0019]
“Embodiment 1”
In the present embodiment, the character arrangement pitch in the character string of the extracted character candidate, that is, the distance between adjacent rectangles of the circumscribed rectangles of the character to be recognized and the rectangle centerline distance information of the adjacent rectangles are used to perform character extraction errors. It relates to the method to ask for.
A plurality of black pixel circumscribed rectangles exist in the line obtained by the line cut-out process (see FIG. 3), and the final combination is performed in consideration of the recognition score and the linguistic likelihood among the variety obtained by combining these rectangles. For example, a recognition result as shown in FIG. 4 is obtained.
As shown in FIG. 4, the distance between characters is also a feature used in the character segmentation process, and is considered in the character segmentation process. Generally, at the time of the character segmentation process, the character recognition result and the arrangement pitch are unknown. In many cases, only extremely invalid candidates are excluded, the cut-out position cannot be determined uniquely, and a plurality of possibilities are required. From the candidates, the maximum likelihood solution is selected based on the recognition dictionary matching score and the linguistic likelihood. Therefore, when such a recognition result is almost finalized, if the character arrangement pitch is checked again and a character cutout error is determined based on the presence of an illegal pitch portion, the character cutout error can be accurately detected. .
Character placement pitches are roughly classified into two types, constant pitches and proportional pitches, and the distance between the characters changes depending on the shape characteristics of the circumscribed rectangle of the characters. The distance between characters is
(a) Distance between character rectangles
(b) Rectangle center line distance
There are two possible types (see FIG. 4).
[0020]
When characters are arranged on a proportional pitch, it is clear that (a) the distance between character rectangles is almost constant. However, the shape of the circumscribed rectangle of the character is not constant depending on the character, such as square / vertical / horizontal ... Therefore, (b) the rectangular centerline distance cannot be constant.
On the other hand, when the characters are arranged at a constant pitch, the character is placed in a fixed range, so that the center line distance of the circumscribed rectangle is constant regardless of the character shape. However, as described above, since the shape of the character circumscribed rectangle differs depending on the character, the distance between the circumscribed rectangles is not constant when the center positions are arranged constant.
These are summarized as follows.

As shown in FIG. 4, in one line of the character recognition result, (a) and (b) the distances between the two kinds of characters are totaled, and in light of the above regularity, the line has a constant pitch or Whether the pitch is proportional or not can be determined. After determining the pitch of one line in light of regularity, an illegal pitch portion that is contrary to regularity is detected, and it is determined that there is a character segmentation error there. For example,
・ If the pitch is determined to be constant: Character segmentation error occurs when the center line distance is not equal
・ If the pitch is determined to be proportional: Character cut-out error in parts where the distance between character rectangles is not equal
The existence of each is judged.
[0021]
FIG. 5 is a chart showing an example of the flow of character recognition processing of the present embodiment.
With reference to FIG. 5, the character recognition process of this example including the step of determining the character arrangement pitch and detecting the character cutout error will be described.
First, an original document image is input by an image input device such as a scanner (step 1).
Next, as a series of steps for cutting out character candidates to be subjected to character recognition from the input document image, processing up to line cutting (see above) is performed. In other words, rectangles that are considered to be character elements are extracted from black runs in the input image to extract circumscribing rectangles (step 2), and neighboring rectangles of these circumscribing rectangles are integrated according to a predetermined criterion to grow into a line. , And cut out as a line (step 3).
Here, a character cut-out process is performed on the cut-out line (step 4). In this process, since a plurality of characters are arranged in the longitudinal direction (horizontal direction in the case of horizontal writing) in the line cut out in the preceding process, the line is set to 1 based on the black pixel projection, the line height, and the like. Divide it into a range that seems to be characters (character candidate areas that can be considered to include character units). Since the size is not constant depending on the character type or the like, a plurality of candidates may be obtained, but in this example, a processing method that may have a plurality of candidates is adopted.
Next, a character recognition process is performed on the character candidate obtained by the preceding character cutout (step 5). The character recognition process is performed by comparing the image feature of the target character candidate with the image feature of the character described in the recognition dictionary, calculating a recognition score, and finding a solution whose recognition score is equal to or greater than a preset threshold value. Leave as a recognition candidate.
After that, a list of recognition candidates obtained in the preceding character recognition process is collated with a language dictionary and grammar, and after a recognition score is added, a post-process (language process) for selecting an appropriate solution is performed (step) 6). Here, even when a plurality of candidates are obtained by character segmentation or when a plurality of candidates are obtained by the recognition process, an appropriate solution is narrowed down to one and set as a processing result.
[0022]
Next, since the character is almost confirmed in the processing steps so far, the character extraction pitch type is determined based on the determination result of the character arrangement pitch type and the detected pitch type error is detected. A series of processing is performed.
At the beginning of this process, two types of rectangular distances are measured (step 7). In this measurement, the distance between rectangles and the center line distance (see FIG. 4) between adjacent character rectangles are measured based on the circumscribed rectangle of each character of the recognition result, and the measurement results are stored.
Further, the average value of each of the two types of rectangular distances is calculated based on the distance between the character rectangles and the distance between the rectangle center lines stored in step 7 (step 8).
Next, for use as an index for determining the type of character arrangement pitch, the distance between the character rectangles and the rectangle centerline distance between adjacent character rectangles stored in step 7 and the average values calculated in step 8 Are compared, and the number of rectangular distances that differ from the average value by more than a threshold value is counted for each of the two types of rectangular distances (step 9).
Here, based on the result of step 9, it is determined whether or not the two types of distance values are constant (step 10). In the present embodiment, for example, threshold processing is performed on the number of times of difference (count value of step 9). In other words, if the count value is greater than the threshold (predetermined percentage) set according to the total distance number (= number of character intervals = total number of characters-1), the distance is determined not to be constant. To do. As another realization method, a method may be employed in which the standard deviation of the distance value is obtained, and if it is equal to or greater than the threshold value, it is determined that the distance is not constant.
[0023]
Next, on the basis of the result of determining whether or not the two types of distance values are constant in step 10, the pitch is determined (step 11). Here, when the distance between the character rectangles is constant and the rectangle center line distance is not constant, the character arrangement pitch type is determined to be proportional pitch, the distance between the character rectangles is not constant, and the center line distance Is determined to be a constant pitch.
Thereafter, the subsequent processing is branched upon receiving the determination result of step 10, so it is checked whether or not the determination result is a constant pitch (step 12).
If the pitch is constant at step 12, in order to deal with post-processing such as character re-cutout processing, the arrangement pitch of characters in the target character line is examined to detect a portion that does not become the constant pitch (step 13). ). Specifically, a portion where the distance between the rectangle center lines of the target character is greatly different from the average value calculated in step 8 is found, and the portion is detected as a cut-out error portion. The process of step 13 is finished, and the flow of this example is finished.
On the other hand, even if the pitch is proportional (not constant) in step 12, in order to deal with post-processing such as character re-cutout processing, the character arrangement pitch in the target character line is checked, and the character rectangle is checked. A portion where the distance is not constant is detected (step 14). Specifically, a portion where the distance between the character rectangles of the target character is significantly different from the average value calculated in step 8 is found, and the portion is detected as an extraction error portion. The process of step 14 is finished, and the flow of this example is finished.
[0024]
“Embodiment 2”
The present embodiment relates to a method for obtaining a character cutout error by using a character arrangement pitch in a character string of a cutout character candidate, and is a method performed in consideration of a character decoration range.
In the method described in the above-mentioned “Embodiment 1”, character decorations (that is, standard character sizes such as a mixture of half-width and full-width characters, double-width characters, etc.) are partially made different according to a predetermined rule in a line to be cut out. If a character with a modification) is applied and the character arrangement pitch is partially different, it causes a reduction in cutout accuracy. Therefore, in the present embodiment, a portion where the pitch changes due to the character decoration is detected, the portion is excluded from the information for obtaining the character segmentation error, and after the exclusion, the same pitch detection as in the above-mentioned “Embodiment 1” is performed. A rule is applied to avoid a decrease in accuracy.
The detection principle of the change due to the character decoration that occurs in the character arrangement pitch in the character string is unlikely to appear that the change in pitch due to the character decoration appears in the change in the pitch of only one place, so the character arrangement pitch is standard This is because a portion that is greatly different from the value (actually due to the average value) is detected as a change by character decoration.
In other words, after obtaining the average pitch of the line that is the target of character segmentation, the range that does not fit the pitch is inspected, and if the character decoration features as shown in the following half-width and double-width examples are satisfied, the character is Determine the decoration type.
Half-width: Location where the rectangle center line distance is about half the rectangle center line distance calculated in the undecorated portion of the line to be cut out.
Horizontal double angle: Location where the rectangle center line distance is approximately twice as long as the rectangle center line distance calculated in the undecorated portion of the line to be cut out.
[0025]
FIG. 6 is a chart showing an example of the flow of character recognition processing according to this embodiment.
Referring to FIG. 6, a book including a step of detecting a character cutout error by determining a character arrangement pitch by excluding pitch information generated by character decoration from character arrangement pitch information for obtaining a character cutout error. An example character recognition process will be described.
In this example, a process step that considers the character decoration range is added to the “embodiment 1” and the procedure is executed. Therefore, for step 1 to step 8 (FIG. 6) in the present embodiment in which the same processing steps as step 1 to step 8 (FIG. 5) in “Embodiment 1” are performed, refer to the above description. The description is omitted here.
In the present embodiment, after calculating the average value of each of the two types of rectangle distances of the distance between the character rectangles and the rectangle center line distance in step 8, in order to use as information necessary for determining the character decoration range, step 7 Compare the distance between adjacent character rectangles and the distance between the rectangle centerlines stored in step 1 and the average values calculated in step 8, and the difference between the average value and the threshold value is continuous. Is searched (step 29). Thereafter, the subsequent processing is branched upon receiving the search result of step 29, so it is checked whether there is a continuous portion based on the search result (step 30).
If there is no continuous portion in step 30, the character decoration range determination (step 31) is passed, but if there is a continuous portion, the character decoration range determination is performed (step 31). This decision is
Half-width processing: Center line that is about half of the center line distance calculated for the part that is not decorated with characters on the line to be cut out (the part that has the center line distance and the distance between the rectangles close to the average value) Locations where distance values are continuous
Horizontal double-angle processing: A centerline that is approximately twice the centerline distance of the rectangle that is calculated in the part that is not decorated with characters (the part where the centerline distance between the rectangles and the distance between the rectangles is close to the average value) Locations where distance values are continuous
Each condition is examined, and if satisfied, it is regarded as a character decoration range, and information indicating the character decoration range and the type of character decoration are stored.
[0026]
Next, in response to the determination result of the character decoration range performed in step 31, the information of the portion determined as the character decoration range is stored in step 7, and the distance between adjacent character rectangles and the distance between the rectangle center lines are stored. Excluding from the information, calculate the average value of each of the two types of rectangular distance, compare the calculated average value and the distance between the character rectangles of adjacent character rectangles stored in step 7 and the rectangle centerline distance, The number of rectangular distances that differ from the average value by more than a threshold is counted for each of the two types of rectangular distances (step 32).
Here, based on the result of step 32, it is determined whether or not the two types of distance values are constant (step 33). However, the portion determined to be the character decoration range at this time is not determined. Other than that, there is no difference from the processing of step 10 (FIG. 5) in “Embodiment 1”.
Further, in the present embodiment, the same processing as that of the above-described “Embodiment 1” is performed for the processing steps until the flow after the pitch determination (step 34), which is the next step, is completed. Therefore, for step 33 to step 37 (FIG. 6) in the present embodiment in which the same processing steps as step 10 to step 14 (FIG. 5) in “Embodiment 1” are performed, refer to the above description. The description is omitted here.
[0027]
“Embodiment 3”
In this embodiment, a character obtained by performing recognition processing on the extracted character candidate is checked based on an aspect ratio (aspect ratio of the circumscribed rectangle of the character) that should originally be included as a characteristic amount unique to the character. Thus, it relates to a method for obtaining a character segmentation error.
As described above, since the shape of each character differs depending on the character, the circumscribed rectangle of the character is not constant. In general, Kanji characters are slightly larger in size and are often square, and alphanumeric symbols are often not square.
If character codes are classified in advance according to the shape of their circumscribed rectangles, after the character recognition process is performed on the extracted character candidates and the character codes are obtained, the circumscribed rectangles originally possessed by the characters of the prepared character codes By comparing the shape feature (aspect ratio) of the image and the shape of the circumscribed rectangle of the actual recognition image, it is possible to detect that there is a possibility that character extraction is not correct if there is a mismatch between the two.
[0028]
FIG. 7 is a chart showing an example of the flow of character recognition processing according to this embodiment.
Referring to FIG. 7, character segmentation error using the aspect ratio (the aspect ratio of the circumscribed rectangle of the character) that is the characteristic amount inherent to the character, as a result obtained by performing the character recognition processing on the extracted character candidate The character recognition processing of this example for obtaining the above will be described.
In this example, the processing steps until the character recognition process is performed on the extracted character candidates are the same as those performed in the “embodiment 1”. Therefore, for step 1 to step 6 (FIG. 6) in the present embodiment in which the same processing steps as step 1 to step 6 (FIG. 5) in “Embodiment 1” are performed, refer to the above description. The description is omitted here.
In this embodiment, even if there are a plurality of character extraction candidates and recognition character candidates after performing character recognition processing on the extracted character candidates in step 7, the character that is determined to be the most appropriate is identified in step 6, At that stage, the character cutout position is once determined. At that time, the character aspect ratio is calculated for the confirmed character (step 47).
Next, based on the calculated character aspect ratio, the following character extraction error is detected (step 48).
The aspect ratio of the circumscribed rectangle that the character code specified as a result of character recognition should originally have as a characteristic feature and the aspect ratio of the circumscribed rectangle of the character image that was actually subject to recognition processing A comparison is made to check whether or not they match, and based on the result, it is detected whether or not there is a character segmentation error.
[0029]
For example, in the case of the word “detail”, since it is composed of characters that can be formed as one character even if the bias and the heel are separated, it is difficult to determine the character cutout position, but the character code and the character feature of the circumscribed rectangle By combining with, an obviously incorrect solution can be rejected. [Table 1] below shows an example of a correspondence table between characters (represented by character codes) and character features of the character circumscribed rectangle assumed from the characters by taking “details” as an example.

For example, the results when [Table 1] is applied to the following recognition results to determine the coincidence are shown below.
Ming: Square
Thread: Slightly long → × [Table 1] is square and does not match
Ta: Slightly long → × In [Table 1], it is square and does not match
Based on the result of the coincidence determination, it is possible to detect that there is a high possibility that the portions of “thread” and “field” are character extraction errors.
[0030]
By the way, expressions such as “slightly long” and “square” indicating the feature of the shape are one expression format of the aspect ratio (aspect ratio) of the circumscribed rectangle of the character, and can be defined by a numerical value.
The following [Table 2] illustrates a correspondence table of aspect ratios with respect to character codes. The aspect ratio (= vertical / horizontal) of the letter SJIS in the table is indicated by the average value Av.
Av. Is greater than 1 if the circumscribed rectangle is vertically long
Av. Is close to 1 when the circumscribed rectangle is close to a square
Av. Is less than 1 when the circumscribed rectangle is horizontally long
Can be expressed.

[0031]
[Table 3] below shows the detection results in the embodiment in which the recognition error is detected by applying [Table 2] to the recognition result and judging the matching.
The actual aspect ratio of the circumscribed rectangle of (a) recognition result shown in [Table 3] is obtained from the circumscribed rectangle coordinates. Further, (b) the average aspect ratio expected from the character code of the recognition result and (c) the standard deviation are obtained from [Table 2], respectively.
The procedure for detecting cropping errors may differ greatly by comparing the actual rectangular aspect ratio ((a)) with the expected aspect ratio ((b), (c)). For example, it is determined that the character is cut out incorrectly.
In the example of [Table 3], based on the characteristics of the normal distribution,
If (b)-(c) * 3 <(a) <(b) + (c) * 3, no difference (= within error range)
If (b)-(c) * 3 <(a) <(b) + (c) * 3
It is judged and displayed as ◯ ×.
As shown in [Table 3], by paying attention to the aspect ratio,
`` Ri '' → `` King '' `` Sato ''
It can be seen that the portion where the character is cut out by mistake can be detected.
Judgment conditions used here:
(b)-(c) * 3 <(a) <(b) + (c) * 3
Is determined based on a constant multiple of the standard deviation based on a constant multiple of the standard deviation. By adjusting this constant multiple (3 times in the example), the detection conditions can be adjusted to be strict / loose. Needless to say. This judgment condition is only an example. If the standard aspect ratio and the aspect ratio of the recognition result are compared, and if they are different from the preset range, if it can be determined that it is a character segmentation error, The conditions are not limited to those shown in the examples. As a method for limiting the range of difference, for example, a threshold value can be provided for the difference or ratio of aspect ratios.

[0032]
“Embodiment 4”
The present embodiment is a method for obtaining a character segmentation error by checking a character obtained by performing recognition processing on a segmented character candidate according to an aspect ratio that should originally have as a characteristic amount inherent to the character. This is a method that takes into account the character decoration range.
In the method described in the above “Embodiment 3”, a modification is applied in which a character decoration (that is, a standard character shape such as half-width, double-width, etc.) is partially different according to a predetermined rule in a line to be cut out. If the standard aspect ratio expected from the recognition result is checked, a cutout error is detected, which causes a reduction in cutout accuracy. Therefore, in the present embodiment, a character portion with a character decoration is detected, the portion is excluded from information for obtaining a character cut-out error, and the detection is performed with the same aspect ratio as in the above-described “third embodiment” after the exclusion. A rule is applied to avoid a decrease in accuracy.
The detection principle of character decoration changes that occur in character strings is unlikely to appear in only one character when the aspect ratio differs greatly from the expected value depending on the character decoration, so the aspect ratio of characters is expected continuously This is because a portion that differs greatly from the value is detected as a change due to character decoration.
In other words, the aspect ratio is calculated sequentially for characters after character recognition in the line to be cut out, and the range where the aspect ratio does not match the expected value is inspected, as shown in the half-width and double-width examples below. If the character decoration feature is satisfied, the character decoration type is determined.
Half-width: If the aspect ratio calculated for the character to be cut out is 0.4 or more and 0.6 or less, it is determined as half-width, and the character portion is doubled that aspect ratio and compared with the expected aspect ratio.
Horizontal double-angle: If the calculated aspect ratio of the character to be cut out is 1.7 or more and 2.2 or less, it is determined as horizontal double-angle, and the character part is compared with the expected aspect ratio by halving the aspect ratio.
In the present embodiment, the above-described processing in consideration of the character decoration range is executed in the character segmentation error detection step (step 48) based on the aspect ratio in the character recognition processing flow (FIG. 7) shown in the above-mentioned “third embodiment”. , Can be implemented.
[0033]
“Embodiment 5”
The present embodiment is a method for obtaining a character segmentation error by checking a character obtained by performing recognition processing on a segmented character candidate according to an aspect ratio that should originally have as a characteristic amount inherent to the character. This is a method involving a process of presenting a detection result of a character segmentation error.
In the method shown in the above “Embodiment 3,” how to use the detection result is not shown when a character segmentation error is detected. In the present embodiment, a method of using the detection result of the character segmentation error is shown.
In the above character recognition process according to a general method, the recognition score is calculated from the viewpoint of the degree of matching with the recognition dictionary, the linguistic likelihood, etc., so that the recognition confidence indicating how reliable the recognition result is based on the score. Degrees are often given.
As a method of using this recognition reliability, if the recognition reliability is low, the user who uses the recognition result is notified, so that even if there is an error in the recognition, the check and correction work by the user is remarkably made efficient. It becomes possible. For this reason, when a character with low reliability is presented to the user, the display form is changed to alert the user and facilitate its discovery.
Here, it is appropriate that at least a character determined to be an error by detecting a character cut-out error is a target of a character to be presented by changing the display form.
Further, in the present embodiment, the above-described process of presenting a character with low recognition reliability by changing the display form is used for character segmentation error based on the aspect ratio of the character recognition process flow (FIG. 7) shown in the above-mentioned “third embodiment”. It can be implemented in the form of being executed in association with the detection step (step 48).
[0034]
“Embodiment 6”
The present embodiment is a method for obtaining a character segmentation error by checking a character obtained by performing recognition processing on a segmented character candidate according to an aspect ratio that should originally have as a characteristic amount inherent to the character. This is a method for re-cutting and recognizing according to the detection result of the character cut-off error.
In the method shown in the above “Embodiment 3,” how to use the detection result is not shown when a character segmentation error is detected. In the present embodiment, using the detection result of the character cut-out error, a new cut-out / recognition is performed on the error detection portion and is made to compete with the original recognition result, thereby enabling more accurate recognition processing.
More specifically, even if a character segmentation error is detected by first performing the check by aspect ratio as described above, it is the result of selection through a series of recognition processes, and there is a possibility that it is actually correct. Therefore, it is dangerous to immediately reject the character by determining the character segmentation error. Therefore, the original solution is retained without being rejected, the character cutout process is performed again on the portion determined to be a character cutout error, a new character cutout position candidate is obtained, and the recognition process for the characters included therein is performed. , And perform post-processing (language processing) by competing with the original solution that holds the newly obtained character candidates (with different cutout positions), that is, among all the character candidates that have been targeted so far The character with the highest recognition score representing the likelihood is selected. Therefore, the original solution may be selected again, but it does not matter.
In the present embodiment, the above-described processing in which a series of recognition processing is performed again on the portion where the character cut-off error has been determined is the character based on the aspect ratio of the character recognition processing flow (FIG. 7) shown in the above-mentioned “third embodiment”. It can be implemented in the form of being executed in conjunction with the step of detecting a clipping error (step 48).
[0035]
“Embodiment 7”
The present embodiment is a method for obtaining a character segmentation error by checking a character obtained by performing recognition processing on a segmented character candidate according to an aspect ratio that should originally have as a characteristic amount inherent to the character. This is a method for re-cutting and recognizing according to the detection result of the character cut-off error.
In the method described in the above “Embodiment 6”, the adjacent part of the error detection part is not considered as the target part of the re-cutout / recognition process when the character cutout error is detected. There is a high possibility of affecting not only the detected characters but also adjacent characters. Therefore, by not only detecting the character part detected as an error but also finding the extraction candidates again in the range including the adjacent character, not only the error detected character but also the error was detected, but it was actually misrecognized. Neighboring characters can be turned into correct answers. The adjacent characters are the characters adjacent to the left and right when the cut out line is horizontal, and the characters adjacent to the upper and lower when the extracted line is vertical.
Note that in the present embodiment, as in “Embodiment 6”, the above-described processing for performing a series of recognition processing again by adding an adjacent portion to a portion where a character segmentation error has been determined is shown in “Embodiment 3”. It can be implemented in the form of executing in accordance with the character extraction error detection step (step 48) based on the aspect ratio of the character recognition processing flow (FIG. 7).
[0036]
“Eighth embodiment”
The present embodiment is a method for obtaining a character segmentation error by checking a character obtained by performing recognition processing on a segmented character candidate according to an aspect ratio that should originally have as a characteristic amount inherent to the character. This is a method that takes into account characters with a large variation in aspect ratio.
In the method described in the above “Embodiment 3,” a character whose aspect ratio fluctuates partially in a line to be cut out, such as a dot-type character (ie, //././,/“/”/ If the standard aspect ratio expected from the recognition result is checked, a cutout error is detected, which causes a drop in cutout accuracy. Therefore, in the present embodiment, a portion such as a dot-type character is detected, the portion is excluded from the information for obtaining the character cut-out error, and the detection is performed with the same aspect ratio as in the above-described “third embodiment” after the exclusion. A rule is applied to avoid a decrease in accuracy.
Specifically, for example, when the number of pixels of a dot-type character (///./, / “/” /, etc.) is affected by the image quality (resolution), the number of pixels in both the vertical and horizontal directions is the property of this character. Therefore, the aspect ratio greatly fluctuates even with slight fluctuations. It is not uncommon for things that were originally vertically long to become horizontally long.
Therefore, a set of characters such as dot-based characters whose aspect ratio varies greatly is set in advance, and characters included in this set are excluded from character cutout error determination based on the aspect ratio. As a result, it is possible to avoid the waste of performing error detection processing on characters that cannot be expected to be detected with an accurate error position, which is convenient.
In the present embodiment, the character cut error detection step based on the aspect ratio in the character recognition processing flow (FIG. 7) shown in the above-described “third embodiment” is performed in the above-described processing in consideration of characters whose aspect ratio varies greatly ( It can be implemented in the form of executing in step 48).
[0037]
“Embodiment 9”
The present embodiment is a method for obtaining a character segmentation error by checking a character obtained by performing recognition processing on a segmented character candidate according to an aspect ratio that should originally have as a characteristic amount inherent to the character. This is a method of performing processing reflecting the result of language processing.
In the method described in the above “Embodiment 3”, a cut error is detected when the standard aspect ratio expected from the recognition result is checked when handling as a single character, but cut out from the linguistic correctness. There are cases where there is no mistake, and a check based on the aspect ratio of a single character causes a reduction in the extraction accuracy. Therefore, in the present embodiment, the portion determined to be linguistically correct is excluded from the information for obtaining the character segmentation error, and after the exclusion, the detection rule based on the aspect ratio similar to that of the above “Embodiment 3” is applied. Thus, avoiding a decrease in accuracy.
Specifically, on the premise that the original correct sentence is linguistically correct, there is a high possibility that the linguistic correctness of the character segmentation error portion is disturbed. Conversely, even if it is determined that there is an error in the character segmentation error detection process based on the aspect ratio or the distance between rectangles, it is more appropriate to consider that the answer is correct if it is linguistically correct. Therefore, even if it is determined that the range (character string) matching the dictionary word is a character segmentation error, it is not regarded as an error.
Since the recognition result, which is a character segmentation error detection target, is obtained through language processing, it is easy to determine whether or not the character is in a dictionary word. What is necessary is just to provide the item which memorize | stores the word collated with the internal data of a character recognition result, and its character number.
In this embodiment, the character cut-out error detection based on the aspect ratio of the character recognition processing flow (FIG. 7) shown in the above-mentioned “embodiment 3” is applied to the above-described processing in consideration of a word determined to be linguistically correct. It can be implemented in the form of executing in step (step 48).
[0038]
“Embodiment 10”
In the present embodiment, the combination of the calculation results of the character arrangement pitch and the character aspect ratio of the target character / character string obtained by performing the recognition process on the extracted character candidate (a continuous arrangement of characters) The present invention relates to a method for obtaining a character segmentation error by performing a check based on a property (characteristic of continuous arrangement) peculiar to a combination of character pitch and character aspect ratio of a column.
The above-mentioned continuous arrangement of characters indicates the character type / character circumscribing rectangle shape (aspect ratio) obtained when character recognition processing is performed on the extracted character candidate string and the two character arrangement pitch conditions (character center line) This is expressed in terms of the distance and the distance between character rectangles and the inline standard value). On the other hand, the nature of the continuous sequence is the appearance probability when the continuous sequence is different due to learning of the standard manuscript (that is, when the character type / character circumscribed rectangle shape (aspect ratio) and the two types of character arrangement pitch conditions are different). Each is obtained in advance.
When detecting a character segmentation error, obtain a continuous sequence for the actual target character string, and obtain the existence probability of the obtained sequence by referring to the appearance probability of each continuous sequence prepared by learning the standard manuscript. Then, the portion having a low appearance probability is detected as a portion where character cutout may be incorrect.
[0039]
FIG. 8 is a chart showing an example of the flow of character recognition processing of the present embodiment.
Referring to FIG. 8, the character recognition process of this embodiment will be described. In this example, the character recognition process is performed on the extracted character candidates, two types of rectangular distances are obtained, and the average value thereof is obtained. The processing step executes the same procedure as that performed in the above “Embodiment 1”. Therefore, for step 1 to step 8 (FIG. 8) in the present embodiment in which the same processing steps as step 1 to step 8 (FIG. 5) in “Embodiment 1” are performed, refer to the above description. The description is omitted here.
In this embodiment, even if there are a plurality of character extraction candidates and recognition character candidates after performing character recognition processing on the extracted character candidates in step 7, the character that is determined to be the most appropriate is identified in step 6, At that stage, the character cut-out position is once determined. At that time, the character aspect ratio is calculated for the confirmed character (step 59).
Next, based on the calculated character aspect ratio, the two types of rectangular distances, and their average values, the following character extraction errors are detected (step 60).
[0040]
In this character segmentation error detection step, as described above, a continuous sequence of the target characters is expressed by the character type / character circumscribed rectangle aspect ratio and two types of character arrangement pitch conditions, and the characteristics of the sequence are examined.
The aspect ratio of the target character is originally different for each character, but since it actually varies, it is not necessary to precisely set the aspect ratio numerically, and it may be classified into a representative finite number.
In this example, for example, the following three types of aspect ratios are characteristic:

The character codes are classified into the following nine types (= 3 types × 3 types) in combination with those roughly divided into 3 types (Kanji / Hiragana, Katakana / Alphabetic).
Kanji: Vertical
Kanji: Square
Kanji: Horizontal
Hiragana, Katakana: Longitudinal
Hiragana, Katakana: Square
Hiragana, Katakana: Horizontal
Alphanumeric: Vertical
Alphanumeric: Square
Alphanumeric: Horizontal
On the other hand, as mentioned in the above “Embodiment 1,” there are two typical character arrangement pitches, which are based on the standard pitch width in the line (the average value calculated in step 8 in FIG. 8). If expressed in terms of long and short, the pitch between the rectangles is classified into the following six types (= 3 + 3).
Character center line distance (standard, long, short: 3 types)
Distance between character rectangles (standard, long, short: 3 types)
Further, as shown in the above “Embodiment 1”, there is a relationship between the aspect ratio of the circumscribed rectangle of the character and the arrangement pitch of the character. Therefore, the aspect ratio of the circumscribed rectangle of the character and the arrangement pitch are classified and defined as follows.

Here, the classification criteria are
a Character rectangle classification (9 types)
b Character center line distance (standard, long, short: 3 types)
c Distance between character rectangles (standard, long, short: 3 types)
Therefore, the total number of types is
a type x b type x c type = 9 x 3 x 3 = 81 types
Is required.
[0041]
In this way, by classifying each reference (reference) into a finite number, it is possible to simultaneously represent the aspect ratio of the circumscribed rectangle of the character and the character arrangement pitch by a finite number of types.
The characteristics of the continuous arrangement of the aspect ratio of the circumscribed rectangle and the arrangement pitch are stored as a reference, and in the recognition result, the characters are obtained and compared with the stored characteristics. It can be determined that it is a clipping error.
N-gram can be considered as a means of learning the nature of a continuous sequence. This is because, according to the 81 examples, 81 consecutive array classifications are labeled with 81 symbols, and at the time of learning, the occurrence probability of the n-gram of this symbol sequence is determined by a manuscript with little misrecognition. To learn.
For example, if you use a trigram that defines three consecutive symbols,
P (xi | xi-2, xi-1) ： A posteriori probability that xi appears under the consecutive occurrence of xi-2, xi-1
Is equivalent to learning the nature of a continuous sequence.
Here, xi-2, xi-1, and xi are symbols representing the aspect ratio and arrangement pitch of the circumscribed rectangle. P (xi | xi-2, xi-1) can be easily obtained from the learning data correctly cut out as follows.
P (xi | xi-2, xi-1) = C (xi-2, xi-1, xi) / C (xi-2, xi-1)
C (xi-2, xi-1): Number of consecutive occurrences of xi-2, xi-1
C (xi-2, xi-1, xi): Number of consecutive occurrences of xi-2, xi-1, xi
If modeling with trigrams, the learning results can be stored in a table with probability values for 81 × 81 × 81 = 531441 types of trigrams. It is expected that there will be a small number of trigrams that do not appear or even if they appear, in which case there will be no problem even if they are substituted with a low constant value, so there will be fewer trigrams to remember.
[0042]
FIG. 9 is a diagram for explaining a method of performing character segmentation error detection using a learned trigram.
Character recognition processing is performed on the extracted character candidates to determine the aspect ratio of the circumscribed rectangle and the two character arrangement pitches of the recognition result, and each character is converted into a symbol used in the trigram. After this conversion, the appearance probability of the learned trigram is applied in order, and the portion showing a low probability value is determined to be a character segmentation error location.
In the example of FIG. 9, it is assumed that “Kuna” merges and is mistakenly cut out as one character, and is recognized as “case”. Others are assumed to be correctly recognized. The result of applying trigram to each character at this time is shown in [Table 4] below.

The appearance probabilities in [Table 4] are plotted at the bottom of FIG. As shown in FIG. 9, it is determined that the portion below the low probability threshold is a character cut-out error.
[0043]
“Embodiment 11”
This embodiment shows the embodiment of the character recognition apparatus concerning this invention.
A general-purpose processing device (computer) is used as means for executing the processing steps shown in the character recognition method including the character extraction error detection procedure shown in the above-mentioned “Embodiment 1” to “Embodiment 10”. This is an example of a device to be used.
FIG. 10 illustrates the configuration of the character recognition device of this embodiment. As shown in FIG. 10, this example shows an example implemented by a general-purpose processing device (computer). As components, an input device 4 such as a CPU 1, a memory 2, a hard disk drive 3, a scanner, a keyboard, and a mouse. A CD-ROM drive 5, a display 6, a flexible disk drive 7, a communication device 8, etc. are prepared and connected by a bus.
A part of a storage medium (not shown) used by the memory 2, the hard disk drive 3, the CD-ROM drive 5, and the flexible disk drive 7 as a storage means realizes the character recognition processing function according to the present invention. A program (software) for executing each processing step shown in the character recognition method including the character extraction error detection procedure shown in the above “Embodiment 1” to “Embodiment 10” is recorded.
An original document image to be processed is input by an input device 4 such as a scanner and is stored in, for example, the hard disk 3. The CPU 1 reads a program for realizing the processing functions and processing methods described above from a recording medium included in the storage unit, executes processing according to the program on the target document image, and outputs the processing result and the like to the display 6.
As shown in FIG. 11, the character recognition apparatus according to the present invention is connected to external apparatuses 11 to 13 via a communication line 20 such as the Internet by a communication apparatus 8 so that a part of the functions is placed on the network. You may carry out with the form which has.
In addition to implementing the processing program of each processing step of the character recognition method of the present invention by installing it in a general-purpose computer as in this embodiment, functions necessary for execution of each processing step are in the form of firmware or electric circuit. It is also possible to configure a character recognition device by incorporating it into the utilization device.
[0045]
【The invention's effect】
(1) Claim 1,4Effects corresponding to the invention
  Character recognition processing is performed on the character candidates (multiple existences) cut out at a stage where the characters and their arrangement are unknown, and the character aspect ratio (shape feature) is targeted for the characters determined based on this result when the recognition result is almost finalized. As a result of comparison with the standardized character aspect ratio of the character stored in the dictionary in advance, the location where a greatly different character exists is detected as a character cutout error, so the character cutout error is detected with high accuracy. It enables and approaches other than the character arrangement pitch.
  In addition, re-cutting and re-recognizing the character candidate area determined to be a character cut-off error, evaluating the validity together with the original result, and selecting a plausible solution, a more accurate character It becomes possible to recognize. Furthermore, since re-cutting is performed for a range including characters adjacent to both sides of a character candidate area determined to be a cut-off error in the character arrangement, higher accuracy can be achieved.
(2) Effects corresponding to the invention of claim 2
  Character decoration is judged from the detection result of character cutout error,Re-determination is performed by comparing the standard character aspect ratio with the character aspect ratio of the character string thus determined not applied to the character decoration.Thus, since the processing corresponding to the factor that reduces the accuracy of character segmentation errors is performed, the accuracy can be further improved.
(3) Effects corresponding to the invention of claim 3
  Presenting a decrease in the reliability of the recognition result for characters for which a clipping error has been determined, and informing the user using the recognition result when the recognition reliability is low, so even if there is an error in recognition, It becomes possible to remarkably improve the checking and correction work by the user.
(4) Effect corresponding to the invention of claim 5
  Characters with a large variation in aspect ratio (for example, dot-type characters) are excluded from the target of normal character segmentation error determination. Thus, it is possible to avoid waste of performing error detection processing.
(5) Effect corresponding to invention of claim 6
  Characters that are collated with dictionary words by language processing in the character recognition step are excluded from normal character segmentation error determination targets, and priority is given to language correctness, so it is possible to avoid accuracy degradation. .
(6) Effects corresponding to the invention of claim 7
  Character recognition processing is performed on the character candidates (multiple existence) that were cut out at the stage where the characters and their arrangement are unknown, and when the recognition result is almost confirmed, the distance between the character rectangles and the center line for the character determined by this result By determining the distance and character aspect ratio (shape feature), combining these information, learning the existence probability of this combination in advance from a standard manuscript, and referring to the stored data value, the basis for character cutout error determination Therefore, it is possible to detect a character segmentation error with high accuracy and obtain a stable recognition result.
(7) Effects corresponding to the inventions of claims 8 and 9
  By mounting a program for executing each processing step of the character recognition method according to claim 1 on a general-purpose computer, the effects (1) to (6) are easily realized, It is possible to provide a character recognition device that exhibits the effect.
[Brief description of the drawings]
FIG. 1 shows an example of a document image to be processed.
FIG. 2 shows a result of creating a circumscribed rectangle of a black run that can be regarded as a character in the example of the document image (FIG. 1).
FIG. 3 is a diagram illustrating a character line rectangle and a character circumscribing rectangle obtained as a result of integration processing;
FIG. 4 is a diagram for explaining a character recognition result and two types of character rectangle distances in a target character line.
FIG. 5 is a chart showing an exemplary flow of a character recognition process of “Embodiment 1”;
6 is a chart showing an example of a flow of character recognition processing of “Embodiment 2”. FIG.
FIG. 7 is a chart showing an exemplary flow of a character recognition process of “Embodiment 3”.
FIG. 8 is a chart showing an exemplary flow of a character recognition process of “Embodiment 10”.
FIG. 9 is a diagram for explaining a method of performing character segmentation error detection using a learned trigram.
FIG. 10 shows a configuration of a character recognition apparatus according to an embodiment of the present invention.
FIG. 11 shows another configuration of the character recognition apparatus according to the embodiment of the present invention.
[Explanation of symbols]
1 ... CPU, 2 ... memory,
3 ... Hard disk drive 4 ... Input device,
5 ... CD-ROM drive, 6 ... Display (display device),
7 ... FD drive, 8 ... Communication device.

Claims (9)

A character extraction step for extracting a character candidate area that can be regarded as including a character unit from a document image having a character arrangement input as a processing target, and the extracted character is compared with a character image feature stored in a dictionary. A character recognition method including a character recognition step of recognizing,
Further, a character aspect ratio calculating step for calculating an aspect ratio of a circumscribed rectangle of the character for the character recognized in the character recognition step;
A character cut-out error determination step for comparing a calculation result of the character aspect ratio with a standardized character aspect ratio of the character stored in the dictionary in advance and determining a character cut-out error ;
A character re-cutting step for re-cutting the character candidate area determined to be a cut-off error by the character cut-off error determination step, character recognition of the re-cut-out character, and evaluating the validity together with the original result, A character recognition method comprising a re-character recognition step of selecting a plausible solution . 2. The character recognition method according to claim 1, wherein the character cut-out error determination step sequentially calculates an aspect ratio for characters after character recognition in a line that is a target of character cut-out, and a value for which an aspect ratio is expected. If the non-conforming continuous character string satisfies the character decoration characteristics, the character decoration type is determined, and the character aspect ratio of the determined character string is applied to the character decoration. A character recognition method characterized by performing determination again by comparing with a standardized character aspect ratio .   3. The character recognition method according to claim 1, further comprising a step of presenting a decrease in reliability of the recognition result for the character whose cut-off error is determined by the character cut-off error determining step. Recognition method. 4. The character recognition method according to claim 1 , wherein the character re-cutout step targets a range in which adjacent characters are added on both sides of a character candidate area determined to be a cut-out error in the character arrangement. 5. A character recognition method characterized by performing re-cutting . The character recognition method according to any of claims 1 to 4, characterized in that said character segmentation error determination step, which is to exclude the violent character of variation of aspect ratio from the normal character segmentation error determination of the target Character recognition method. The character recognition method according to any one of claims 1 to 5, wherein the character segmentation error determination step includes subjecting a character collated with a dictionary word by language processing in the character recognition step to a normal character segmentation error determination target. Character recognition method characterized by being excluded from the above. A character extraction step for extracting a character candidate area that can be regarded as including a character unit from a document image having a character arrangement input as a processing target, and the extracted character is compared with a character image feature stored in a dictionary. A character recognition method including a character recognition step for recognizing, further targeting a character array recognized in the character recognition step, a distance between adjacent rectangles in a circumscribed rectangle of a character and a rectangle center line distance of the adjacent rectangle, respectively A character pitch calculation step to calculate; a character aspect ratio calculation step to calculate an aspect ratio of a circumscribed rectangle of the character for the character recognized in the character recognition step; and a character pitch and a character aspect of the target character / character string The existence probability of the combination of the calculation results of each ratio is the character pitch prepared as a standard value in advance. And determined by referring to the existence probability of the combination of letters aspect ratio, character recognition method characterized by including the character segmentation error determination step of determining the character extracting error based on the reference value determined. A program for causing a computer to execute each processing step of the character recognition method according to any one of claims 1 to 7. A character recognition apparatus comprising: a computer on which the program according to claim 8 is mounted, wherein the computer processes target image data .

Images