JP3958003B2 - Character recognition method, character recognition program, computer-readable recording medium recording character recognition program, and character recognition apparatus - Google Patents

Description

【００６０】
実験用データであるオンライン文字パターンデータは、筆記者数５名であって、データ筆記時には、この５名の筆記者に例文を与えた。筆記者には、この例文を横書き、縦書き、及び筆記方向自由（例えば、斜め方向）の３種類のパターンで手書き入力させ、これらのデータを収集した。また、このオンライン文字パターンデータは、総文字数３８４０文字であって、１人当たりの筆記文字数は、７８６文字である。また、このオンライン文字パターンデータは、１セット当たり１０ページ（２５６文字）からなり、１ページ平均約２５文字の文章となっているため、筆記者は、それぞれ３セット分（３０ページ、７８６文字）の文字の手書き入力を行った。
【００６１】
また、この実験では、ページごとに文字列が区切れるため、ページ単位での正認識率を算出し、１セットごとにその平均を求めることにする。
【００６２】
（実験の結果）
図１４は、実験用オンライン枠なし文字パターンデータを示す図である。
ここでのオンライン文字パターンデータは、例えば、筆記者の自由筆記（斜め方向）による「短期的に成功しても、」と、横書きによる「長期的には失敗する。」とを含む。
【００６３】
図１５は、正認識の例を示す図（１）である。
ここでは、オンライン文字パターンデータに含まれる自由筆記による「短期的に成功しても、」と、横書きによる「長期的には失敗する。」とは、それぞれ各文字要素ごと外接矩形に囲まれ、正しい文字認識が実行されている。認識結果は、"短期的に成長しても、長期的には失敗する。"である。
【００６４】
図１６は、正認識の例を示す図（２）である。
ここでのオンライン文字パターンデータは、例えば、筆記者の縦書きによる「入園料は小学生」「以上５９０円、」と、横書きによる「小学生未満無料。」とを含む。また、ここでは、オンライン文字パターンデータに含まれる縦書きによる「入園料は小学生」「以上５９０円、」と、横書きによる「小学生未満無料。」とは、それぞれ各文字要素ごと外接矩形に囲まれ、正しい文字認識が実行されている。認識結果は、"入園料は小学生以上５９０円、小学生未満無料。"である。
表２に筆記方向別認識率（％）を示す。
【００６５】
【表２】

【００６６】
ここでは、各筆記者Ａ〜Ｅに対し、データセットごとのページ内の筆記方向別認識率（％）の平均を表している。なお、認識時間については、ページ内平均認識時間は、約２秒、１文字当たりの平均認識時間は、約０．０９秒であった。
【００６７】
（実験結果の考察）
上述の実験結果についての考察としては、横書きの文字列に対しては、文字間隔が狭いため、切出し精度の悪さが認識率に影響した。縦書きの場合、横書きに比べ文字間隔が広くなる傾向にあり、切出し精度および認識率が高かった。自由筆記文字列では横書き・縦書きの両方の特徴を併せ持つため、両者の間の結果となった。
【００６８】
これにより、仮分割時のしきい値を横書きと縦書きで変化させることによって、さらに高精度な切出しが実現できる。また、上述の細かな筆記方向変化のある文字列では、筆記方向変化点を抽出する代わりに、仮分割時に推定文字サイズで分割を打ち切ることで、切出しを行うことができる。
【００６９】
また、本発明では、筆記方向に依存しない文字切出し手法を用いた自由筆記文字列認識を、既存のシステムにおけるパフォーマンスをあまり落とすことなく、自由筆記文字列に対応した文字切出し、及び認識を行うことができる。さらに、分割時のしきい値を動的に設定することによって、切出し精度を向上させること、及びより細かな筆記方向変化への対応を行うことが期待できる。なお、上述の認識実験では、適宜の大規模なオンライン枠なし文字パターンデータベースを収集することができ、本発明による文字認識システムのさらに細かな性能評価を行うこともできる。
【００７０】
（４）第３の実施の形態
つぎに、文字列の筆記方向だけでなく文字方向をも考慮した、自由度の一層高い枠なし文字認識方法について説明する。なお、上述の第１及び２の実施の形態と重複する部分は、説明が冗長とならない程度に省略すると共に、新たに定義する概念については、詳細に説明する。また、文字認識システムの概略構成は、第１の実施の形態と同様である。
【００７１】
図１８は、筆点座標とストロークの説明図である。
ストロークとは、例えば、ユーザ２が適宜の筆記面に対して、接触から非接触に至る間に入力出力部４を利用してサンプリングされる接触座標（筆点座標）の系列をいう。ここでは、図示のように、「あ」に含まれる１つのストロークを用いて説明した。
【００７２】
図１９は、文字方向と筆記方向の説明図である。
文字方向とは、例えば、文字の底辺から上に向けた方向を指し、一方、筆記方向とは、文字列が筆記される方向をいう。また、上述のように、改行、筆記方向変化点、一定間隔以上の空白などで分離される文字列を、それぞれ文字列構成要素とする。文字列構成要素は、それぞれが筆記方向を持つ。ここでは、文字列構成要素である「今日の献立」「親子丼」「うな丼」「天丼」「カツ丼」の文字方向及び筆記方向を、それぞれ点線及び実線の矢印としてそれぞれ示している。
【００７３】
図２０は、本実施の形態に関する文字認識方法のフローチャートである。なお、説明の便宜上、ここでのフローチャートの説明は、概略的なものとし、具体的な説明は後述する。この文字認識方法は、例えば、ユーザ２による枠なしで自由な方向で入力された手書きストローク列に対し、文字方向および筆記方向に依存しない文字認識を行うものである。
入力文字列パターン入力部５により、ユーザ２による枠なしで自由な方向で入力されたストローク列が認識コントロール部１０に入力された後、入力された各ストローク列は、文字サイズ推定処理部１１により、文字サイズを推定される（文字サイズ推定処理：Ｓ２０１）。なお、ステップＳ２０１で推定された文字サイズは、例えば、改行検出に用いられると共に、仮分割処理や文字認識処理における評価ための要素にもなる。
【００７４】
文字サイズ推定処理部１１は、ステップＳ２０１での文字サイズの推定後、推定文字サイズおよびストローク間隔（例えば重心間距離）を基に、改行、行と行を分ける一定以上の間隔、筆記方向の変化点のいずれかを検出して、文字列構成要素へ分割する（文字列構成要素分割処理：Ｓ２０３）。なお、これら文字サイズの推定、改行の検出及び文字列構成要素の分割は、上述のステップＳ１０３、１０５での処理と同様である。
【００７５】
つぎに、文字サイズ推定処理部１１は、ステップＳ２０３により得られた文字列構成要素ごとで文字サイズが変動することを考慮して、文字列構成要素ごとに適当な（平均的な）文字サイズを詳細に推定し直す（文字サイズ詳細推定処理：Ｓ２０５）。なお、ここでの文字サイズ詳細推定処理は、ステップＳ２０１での文字サイズ推定処理と同じであるが、それを文字列構成要素ごとに適用することにより、文字サイズ推定の精度を向上させることができる。
【００７６】
次に、文字切出し処理部１２は、例えば、文字列構成要素ごとに、ストローク間の変動を除いたストローク内の筆点座標の変動情報から文字方向を推定する（文字方向推定処理：Ｓ２０７）。また、文字切出し処理部１２は、文字列構成要素の筆記方向から良く現れる文字方向を仮定する（文字方向仮定処理：Ｓ２０９）。なお、ステップＳ２０９により仮定される文字方向としては、例えば、筆記方向の正負方向、この正負方向に垂直な２方向、入力装置（又は筆記面）の上下方向、左右方向のうち予め定められた方向のいずれか又は複数を仮定する。ここで、文字方向を求める処理は、ステップＳ２０７又はＳ２０９のいずれでもよいし、両方でもよい。なお、これらステップＳ２０７及びＳ２０９を両方行って、後述のステップＳ２２１の最適文字方向選択処理で最も高得点のものを選択するほうが認識精度は高くなる。
【００７７】
このように推定及び／又は仮定されたひとつ又は複数の文字方向の候補によって、各文字方向補正文字列構成要素に対して、それぞれ以下ステップＳ２１１〜Ｓ２１９の処理を行う。なお、各文字方向補正文字列構成要素について、各ステップで求められた文字方向の候補及び中間結果は、文字切出し処理部１２により適宜のメモリに対して書き込み又は読み出しが行われる。
また、文字切出し処理部１２は、ステップＳ２０７により推定された、および／または、ステップＳ２０９により仮定されたひとつ又は複数の文字方向の候補ごとに、その文字方向が上になるように内部データとして元の文字列構成要素を回転する（認識用文字方向補正回転処理：Ｓ２１１）。なお、文字列構成要素は、上述のように、筆点座標の系列で表現されているため、任意角度の回転処理は容易かつ高速に行うことができる。ここで、それぞれの回転後のパターン（文字列構成要素）を、文字方向補正文字列構成要素と呼ぶ。
【００７８】
つぎに、文字切出し処理部１２は、ステップＳ２１１で得られた文字方向補正文字列構成要素ごとの筆記方向を、少なくとも４方向（上下左右）に量子化する（筆記方向量子化処理：Ｓ２１３）。なお、８方向であれば、斜め方向も含むことができる。ここで、ステップＳ２１３以降の処理のうち、文字切出し処理部１２で行われる仮分割処理（Ｓ２１５）及び仮分割修正処理（Ｓ２１７）と、最尤文字列探索処理部１３で行われる文脈処理を伴った認識処理（Ｓ２１９）とは、上述のステップＳ１０７での仮分割処理及びステップＳ１０９での仮分割修正処理と、ステップＳ１１１での認識結果を出力する処理とそれぞれ同様である。
【００７９】
つぎに、最尤文字列探索処理部１３は、ステップＳ２０７とステップＳ２０９によって推定あるいは仮定された文字方向ごとに、ステップＳ２１１からステップＳ２１９までの処理で得られた、各文字方向による複数の文字方向補正文字列構成要素に対する認識結果のなかで最も高い認識の確からしさに関する得点の文字方向とその文字方向における文字方向補正文字列構成要素を選択する（最適文字方向選択処理：Ｓ２２１）。ここで、最尤文字列探索処理部１３は、各文字方向において文字方向補正文字列構成要素の筆記方向に対して、必要に応じて文脈を仮定して文字認識を行い、また、文字認識の確からしさ、文字分割の確からしさ、文字サイズの確からしさ、文脈の確からしさのうちいずれか又は複数を適宜加味して認識の確からしさを示す点数を求める。最尤文字列探索処理部１３は、この点数の高い文字方向を最適文字方向として選択する。なお、ステップＳ２０７及びＳ２０９でひとつの文字方向のみ推定・仮定されている場合は、ステップＳ２２１は省略される。また、最尤文字列探索処理部１３は、ステップＳ２２１により選択された文字方向補正文字列構成要素の回転以前の情報、メモリに記憶された回転角・回転中心等の情報から、元に筆記されたストローク列の筆記方向、文字方向、文字サイズ、文字ピッチなどを復元すると共に、この復元結果を認識結果として、認識結果出力部６を介して、アプリケーション３の画面上の表示に反映する（認識結果の整形表示処理：Ｓ２２３）。
【００８０】
以下、上述のフローチャートの各ステップの処理について詳細に説明する。なお、上述の第１及び第２の実施の形態と重複する処理については、冗長とならない程度に省略する。
（ステップＳ２０１：文字サイズ推定処理について）
文字サイズ推定処理部１１は、例えば、入力文字列パターン入力部５により画面上に記入されたストローク列から、個々の文字要素の文字サイズを推定する。ここで、推定された文字サイズは、改行や文字間隔の検出、仮分割処理および認識処理に用いられる。この文字サイズ推定処理では、画面上の全てのストロークに対して、そのストロークサイズから文字サイズを推定する。なお、文字サイズ推定処理は、上述のステップＳ１０３での文字サイズ推定処理と同様である（図３及びその説明箇所参照）。
【００８１】
（ステップＳ２０３：文字列構成要素分割処理について）
図２１は、文字列構成要素分割処理についての説明図である。
文字サイズ推定処理部１１は、ステップＳ２０３の文字サイズ推定処理で求められた推定文字サイズをもとに、改行検出および文字列間隔の検出をクラスタリングを用いて行なう（改行・文字列間検出：Ｓ２０３１）。なお、ここでの処理は、上述のステップＳ１０５での筆記方向推定（筆記推定）と同様である（図４及びその説明箇所参照）。また、文字サイズ推定処理部１１は、ステップＳ２０３１によって検出された改行・文字列間隔を元に図１９に示すように画面上のストローク列を文字列構成要素に分割する（Ｓ２０３２）。なお、ここでの処理は、上述のステップＳ１０５での筆記方向推定（筆記推定）に含まれる改行検出の処理と同様である（図５及びその説明箇所参照）。
【００８２】
つぎに、文字サイズ推定処理部１１は、筆記方向変化点を検出する（Ｓ２０３３）。ここでは、ステップＳ２０３２での行分割において、図１９に示すように、箇条書きなどにおいて、横書きで書いた見出しの後に縦書きの行を続けて書く場合や、適宜の図などに阻まれた画面上で筆記方向を変えて続けて書くといった場合には、改行や文字列間隔を検出するだけでは行として分割を行うことができず、筆記方向の推定が困難となる場合がある。そこで、ステップＳ２０３３によって、筆記方向が途中で急激に変化している箇所（上述の筆記方向変化点）を特定することで、文字列の分割や筆記方向の推定を正確に行なうことができる。
【００８３】
この筆記方向変化点の検出は、上述のように、対象となる文字列の始点と終点を結ぶ直線からの最遠点を求めることで実現できる。また、この手法を再帰的に用いることで、複数の筆記方向変化に対応できる。これらの筆記方向変化点により、文字列構成要素はさらに短い文字列構成要素に分割される。なお、ここでの処理は、上述のステップＳ１０５での筆記方向推定（筆記推定）に含まれる処理と同様である（図６、７、８及びその説明箇所参照）。つぎに、文字サイズ推定処理部１１は、文字列構成要素ごとに、最初のストロークの始点座標と最終ストロークの終点座標から筆記方向を求める（Ｓ２０３４）。
【００８４】
（ステップＳ２０５：文字サイズ詳細推定処理について）
図２２は、文字サイズ詳細推定処理の説明図である。
文字サイズ推定処理部１１は、図２２に示すように、文字列構成要素ごとで文字サイズが変動することを考慮して、文字列構成要素ごとに文字サイズを詳細に推定し直す。ここでの処理は、上述の文字サイズ推定処理と同様であるが、それを文字列構成要素ごとに適用することで、文字サイズ推定の精度を向上させる。例えば、文字列構成要素「今日の献立」の文字サイズは、それ以外の文字列構成要素の文字サイズよりも大きくなっている。具体的には、ステップＳ２０３で求められた文字要素ごとの文字サイズの平均、最大値・最小値を除いた平均等、適宜の手法で求めることができる。
【００８５】
この文字サイズ詳細推定処理を、ステップＳ２０３の後に挿入することにより、同一画面上で文字サイズの異なった文字列構成要素が存在している場合にも、文字サイズの推定がより正確に行なえるようになる。その結果、文字切出し処理部１２における、文字列認識の前処理である文字切出しでは、推定文字サイズを元に高精度な切出しを行うことができる。ただし、このステップＳ２０５の処理は、画面上の文字サイズが一定であると仮定できる場合、又は、すでに推定されている場合には、省略してもよい。
【００８６】
（ステップＳ２０７：文字方向推定処理について）
図２３は、文字方向と方向別筆点間距離累積ヒストグラムの説明図である。
ここで、文字方向について説明すると、文字を記入する際、ストローク間の変動を除いた筆点座標の変動は、文字底辺を下にとれば右方向か下方向への変動が顕著に見られる。これは、日本語（特に、漢字）が、右方向あるいは下方向に筆記するストロークを多く含むためである。このため、方向別に変動をヒストグラムに取ると、例えば、横書きの文字列であれば９０°程度離れたところに二つのピークが現れることになる。ここでは、それぞれ文字方向の異なる３つの文字列「農工大」の筆点座標の変動としては、概ね９０°離れた部分にピークが現れている。
【００８７】
したがって、文字列全体で見た場合、これらの筆点座標の変動を検出することで、文字方向を推定することができる。また、文字方向が推定できれば、文字方向を正方向（つまり、上向き）になるように文字列全体を回転させることによって、通常どおりの認識処理を行なうことができる。
図２４は、方向別筆点間距離累積ヒストグラムにおけるピークの検出についての説明図である。
文字切出し処理部１２では、文字方向の推定のためには、ストローク内の連続する二つの筆点について、その方向別にその筆点間の距離を累積するヒストグラムを取る。ここでのヒストグラムは、例えば、１６方向分割で累積されており、１６方向のなかで約９０°離れた２つの方向にピークを見出すことができる。この２つの方向でのピークが下方向と右方向からどれだけ回転しているかを求めることで、文字方向を特定することができる。なお、分割の程度は、８方向、４方向、３２方向等適宜の値とすることができる。
【００８８】
ここでは、入力パターンである文字列構成要素「あ」の文字方向を推定する場合を示しており、まず、文字切出し処理部１２は、各文字列構成要素ごとに筆点座標変動のヒストグラムをとり（ステップ１）、このヒストグラムのピークを検出する（ステップ２）。つぎに、文字切出し処理部１２は、検出したヒストグラムの２点以上のピークについてピーク間の角度を検出し（ステップ３）、ここで、あるピークを基準とし、反時計回りに約９０°のところに他のピークが現れたとき、基準としたピークの反対方向を文字方向（図中、ステップ４の点線の矢印）とする。
【００８９】
図２５は、ピークの検出を示すフローチャートである。
まず、文字切出し処理部１２は、筆点変動の方向別のヒストグラムを求め（Ｓ２０７１）、一つの基準となるピークを検出する（Ｓ２０７２）。文字切出し処理部１２は、ステップＳ２０７２で検出された基準となるピークから反時計回りにピークを走査し（Ｓ２０７３）、基準となるピークから９０°付近にもう一つのピークを検出できるか否かを判定する（Ｓ２０７４）。文字切出し処理部１２は、ステップＳ２０７４でもう一つのピークが検出できた場合、基準となるピークの逆方向を文字方向とし（Ｓ２０７６）、一方、もう一つのピークが検出できない場合、基準となる別のピークを検出して（Ｓ２０７５）、再び、ステップＳ２０７３に戻ってピークを走査する。
【００９０】
また、ピークの検出を用いて文字方向を特定する際の他の方法としては、例えば、角度θ方向へのヒストグラムの強度をｆ（θ）として、ｆ（θ）＊ｆ（θ＋π／２）のピークを探し、角度θの反対方向を文字方向とするようにしてもよい。より具体的には、さらに、いわゆるガウス関数ｇ（δ）＝ｅｘｐ（−δ^２／σ^２）を、上述のｆ（θ）＊ｆ（θ＋π／２）に畳み込んで、右上がりの文字のように文字の下方向と右方向の角度差がπ／２から多少ずれている場合でも、二つのピークを安定に抽出することもできる。
【００９１】
図２６は、文字方向が上下しかない場合でのピークの検出についての説明図である。
ここで、文字方向が下向きか上向きしかないと分かっている場合について説明すると、文字切出し処理部１２では、例えば、Ｙ座標（つまり、縦方向）の筆点座標変動から文字方向の推定を行なうことになる。ここでは、上下方向に記入方向を限定しているので、図２６に示すように、記入方向が画面上方向か画面下方向かどちらかに判定できればよい。この場合、横書き・縦書きに関わらず筆点間のＹ方向の差を加えていくことで文字方向を推定できる。すなわち、文字切出し処理部１２は、画面の座標系を、Ｘ軸を左右方向、Ｙ軸を上下方向とすることにより、文字列構成要素（ここでは、「今日の献立」）の筆点列のＹ座標変動の和が「負」の値であれば、文字方向が上方向、一方、「正」の値であれば、下方向とそれぞれ判断できる。
【００９２】
（ステップＳ２０９：文字方向仮定処理について）
図２７は、筆記方向から複数の文字方向を仮定する場合についての説明図である。なお、文字方向推定処理では、文字列構成要素のなかの文字数が少ないと正しく文字方向を推定できない場合が想定される。それを防ぐために、良く現れる文字方向を仮定して、認識候補に加えるのが文字方向仮定処理の目的のひとつである。
【００９３】
ここでの処理では、例えば、図示のように、文字列構成要素（ここでは、「今日の献立」）の筆記方向（図中、実線の矢印）から良く現れる文字方向（図中、点線の矢印）を仮定する。この文字方向としては、筆記方向の正負方向、それに垂直な２方向、及び、図２７のように、入力装置（又は筆記面）の上下方向、左右方向を仮定しているが、これらに限定するわけではない（ここでは、文字方向としては、筆記面の上下左右の方向を仮定している）。なお、文字方向の推定によって絞り込める場合は、これらの仮定の候補数を少なくしたり省略したりすることができる。なお、文字方向仮定処理は、図２６のように、文字方向が表示画面に対し上下方向に限られる場合には省略することができる。
【００９４】
（ステップＳ２１１：認識用文字方向補正回転処理について）
図２８は、認識用文字方向補正回転処理の説明図である。
文字切出し処理部１２は、例えば、ステップＳ２０７での文字方向推定処理によって推定された文字方向、および、ステップＳ２０９での文字方法仮定処理によって仮定された文字方向ごとに、文字列構成要素を内部データとして適宜の内部メモリにコピーし、その文字方向が上向きになるように回転する。ここでは、図中、表示画面上に表示された、文字方向がほぼ右向きの文字列構成要素「今日の献立」を回転することにより、文字方向が上向きである文字方向補正文字列構成要素を得ている。なお、文字切出し処理部１２は、回転後の文字列構成要素である文字方向補正文字列構成要素は、ステップＳ２２３での認識結果の整形表示処理で利用するため、文字方向補正文字列構成要素ごとに回転中心及び回転角を適宜の内部メモリに保持する。また、文字方向補正文字列構成要素ごとの回転角は、文字方向補正文字列構成要素に含まれる文字ごとにも適用される。
【００９５】
（ステップＳ２１３：筆記方向量子化処理について）
図２９は、筆記方向量子化処理を示すフローチャートである。
文字切出し処理部１２は、ここでは、ステップＳ２１１により得られた文字方向補正文字列構成要素ごとに、最初のストロークの始点座標と最終ストロークの終点座標から筆記方向を求め、それを一定の量子化区分に分割する。ここでの処理は、図９に示した筆記方向の推定をより精密化した処理であって、少なくとも４方向（上下左右）に量子化するものである。なお、水平右向きをＸの正方向、垂直下向きをＹの正方向とする。
【００９６】
まず、文字切出し処理部１２は、文字方向補正文字列構成要素の外接矩形の縦Ｙと横Ｘとの縦横比がＸ＞Ｙであるか否かを判定し（Ｓ２１３１）、Ｘ＞Ｙでなければ、文字方向補正文字列構成要素中の始点と終点とのＹ座標の差（始点Ｙ座標−終点Ｙ座標）が０より大きいか否かを判定する（Ｓ２１３２）。つぎに、文字切出し処理部１２は、ステップＳ２１３２でＹ座標の差が０より大きい場合、筆記方向を上向きと判定し、一方、Ｙ座標の差が０以下である場合、筆記方向を下向きと判定し、それぞれの判定結果である筆記方向をストローク内部情報として、文字方向補正文字列構成要素ごとに付加する（Ｓ２１３３）。
【００９７】
一方、文字切出し処理部１２は、ステップＳ２１３１において、Ｘ＞Ｙであれば、文字方向補正文字列構成要素中の始点と終点とのＸ座標の差が０より大きいか否かを判定する（Ｓ２１３４）。つぎに、文字切出し処理部１２は、ステップＳ２１３４でＸ座標の差（始点Ｘ座標−終点Ｘ座標）が０より大きい場合、筆記方向を左向きと判定し、一方、Ｘ座標の差が０以下である場合、筆記方向を右向きと判定し、それぞれの判定結果である筆記方向をストローク内部情報として、文字方向補正文字列構成要素ごとに付加する（Ｓ２１３３）。
【００９８】
このように、筆記方向量子化を少なくとも４方向で行うことは、不自然な過分割を防ぐためである。具体的には、例えば、横書き（さらに、右書き）の場合、そのなかでの左向きの運筆は文字の区切りでないことが多いために、それ以前のストロークをまとめる（結合する）という処理を行うことができ、その結果、細かな過分割が少なくなり認識率を向上することができる。
【００９９】
４方向以上にさらに細かく量子化すれば、斜めに書かれた文字列に対して、より安定に過分割を防止できる。しかし、現実的には４方向でほぼ十分である。今仮に、筆記方向が右下斜め方向の場合を考える。この場合、左上斜め方向への運筆によって文字をまとめることができる。筆記方向を４方向に量子化することにより、右下斜めへの筆記方向が右方向と判定される場合（ア）と、下方向と判定される場合（イ）を考える。（ア）の右方向と判定される場合、左上斜め方向への運筆が左方向と判定されれば、文字をまとめる情報として利用される。一方、この運筆が上方向と判定されると、文字をまとめる情報としては利用されないが、幾分過分割が増えるだけで、仮分割修正処理と文脈処理を伴った認識処理で、文字列はほぼ正しく分割および認識がなされる。（イ）の筆記方向が下方向と判定される場合、左上斜め方向への運筆が上方向と判定されれば、文字をまとめる情報として利用される。一方、この運筆が左方向と判定されると、文字をまとめる情報としては利用されないが、幾分過分割が増えるだけで、仮分割修正処理と文脈処理を伴った認識処理で、文字列はほぼ正しく分割および認識がなされる。今、右下斜め方向への筆記方向を取り上げたが、左下斜め方向、右上斜め方向、左上斜め方向も同様である。
【０１００】
（ステップＳ２１５：仮分割処理について）
ここでは、文字切出し処理部１２は、例えば、改行や文字列間隔などで区切られる文字列構成要素ごとに、文字サイズ、ストローク間の重心間距離（重心間射影距離）、ストロークの重なり具合などを考慮し、文字としての分割候補点の決定やストローク同士の結合などの判定を行なう。この仮分割処理は、上述のステップＳ１０７での仮分割処理と同様である（図１０、１１及びその説明箇所参照）。
【０１０１】
（ステップＳ２１７：仮分割修正処理について）
ここでの仮分割修正処理も、ステップＳ２１５での仮分割処理と同様に、上述のステップＳ１０９での仮分割修正処理と同様である（図１２及びその説明箇所参照）。すなわち、仮分割修正処理は、ステップＳ２１５での仮分割処理によって生成された文字候補同士を、文字サイズ、文字要素間の重心間距離（重心間射影距離）、文字要素同士の重なり具合などを考慮し、仮分割によって生成された文字要素をさらに評価し、要素同士の結合を行なっていく処理である。
【０１０２】
（ステップＳ２１９：文脈処理を伴った認識処理について）
ここでの文脈処理を伴った認識処理は、上述のステップＳ１１１での処理と同様であって、最尤文字列探索部１５により、文字サイズや文脈、認識結果などを統合して動的計画法に基づいて行われ、最終的な認識結果を求める処理である。なお、ステップＳ２１１での認識用文字方向補正回転処理からステップＳ２１９での文脈処理を伴った認識処理までは、上述のように、複数の文字方向補正文字列構成要素に対して適用されることになる。
【０１０３】
（ステップＳ２２１：最適文字方向選択処理について）
つぎに、最尤文字列探索処理部１３は、ステップＳ２０７とステップＳ２０９によって推定あるいは仮定された文字方向ごとに、ステップＳ２１１からステップＳ２１９までの処理で得られた複数の文字方向補正文字列構成要素のそれぞれに対する認識処理の結果としての認識スコアを求める。そして、最尤文字列探索処理部１３は、求められた認識スコアのなかで最も高い得点の文字方向と、その文字方向における文字方向補正文字列構成要素を選択する。
【０１０４】
（ステップＳ２２３：認識結果の整形表示処理について）
図３０は、認識結果の整形表示処理の前段階の認識結果を示す図（１）である。ここで、図中、上述のステップＳ２０１〜Ｓ２２１までの認識結果の一例を示す。
まず、認識前の文字列構成要素（２つの「農工大」）は、上述の各処理によって、それぞれの文字列構成要素の重心を中心にして文字方向が上向きになるように回転補正して認識されている。この回転補正された文字列構成要素は、文字方向補正文字列構成要素として、図３０の認識後の上側のウインドウ（筆記面）に表示される。また、文字方向補正文字列構成要素（２つの「農工大」）の認識結果は、図３０の認識後の筆記面の右下のウインドウ内に表示される。
【０１０５】
図３１は、認識結果の整形表示処理の前段階の認識結果を示す図（２）である。
まず、認識前の文字列構成要素（４つの「農工大」）は、上述の各処理によって、それぞれの文字列構成要素の重心を中心にして文字方向が上向きになるように回転補正して認識されている。この回転補正された文字列構成要素は、文字方向補正文字列構成要素として、図３１の認識後の上側のウインドウ（筆記面）に表示される。また、文字方向補正文字列構成要素（４つの「農工大」）の認識結果は、図３１の認識後の筆記面の右下のウインドウ内に表示される。なお、図３１の認識前の筆記面において上から逆向きに書かれた文字列構成要素「農工大」の認識結果の表示が、認識後の筆記面で傾いているのは、文字方向の推定がこの程度の誤差を伴うことを示している。しかしながら、最尤文字列探索処理部１３の認識エンジン部１４は、一般にこの程度の文字方向変動には対処できるので認識結果のコード列は、図３１の認識後の筆記面の右下のウインドウ内に表示されているように、正しい結果を得ることができる。
【０１０６】
つぎに、上述した認識結果の整形表示処理の前段階の処理を経た、認識結果の整形表示処理について説明する。なお、認識結果の整形表示処理としては、例えば、文字方向補正文字列構成要素毎に回転する場合と、文字方向補正文字列構成要素に含まれる各文字毎に回転する場合とがある。
図３２は、認識結果の整形表示処理についてのフローチャート（１）である。なお、ここでの処理は、例えば、上述した認識結果を利用して、最初の手書き文字列に近いレイアウト又は状況で認識結果表示を行う処理であって、筆記方向を推定した文字列構成要素ごとに、その筆記方向、文字方向、平均文字サイズ、平均文字間隔に従って、認識結果である文字フォント列を配置して表示する。
【０１０７】
最尤文字列探索処理部１３は、文字方向補正文字列構成要素ごとに手書き文字の平均サイズを求め（Ｓ２２３１）、この平均サイズに近い出力文字ポイント数（フォントの大きさ）を決めると共に（Ｓ２２３２）、手書き文字間の縦横それぞれの平均間隔を求める（Ｓ２２３３）。つぎに、最尤文字列探索処理部１３は、文字コード列に従い、ステップＳ２２３２での出力ポイント数、かつ、ステップＳ２２３３での平均間隔で、筆記方向に従って文字フォント列を内部バッファに配置する（Ｓ２２３４）。また、最尤文字列探索処理部１３は、ステップＳ２２３４により、内部バッファに配置された文字フォント列を、回転中心において回転角だけ逆回転して表示する（Ｓ２２３５）。すなわち、ここでの認識結果の整形表示処理は、文字列構成要素のフォントパターン列を内部バッファに配置し、それを回転することにより整形表示を実現している。
【０１０８】
なお、ステップＳ２２３５での文字フォントの表示方法としては、元の手書き文字の近くに配置したり、筆記文字に重ね書きすることもできる。なお、ステップＳ２２３１での文字サイズ、ステップＳ２２３３での平均間隔に伴う文字位置及びステップＳ２２３４で用いられる文字コードは、上述の図３０、３１での認識後の筆記面に示すように、認識結果の整形表示処理の前段階の処理又はそれ以前の処理により決定されている。
【０１０９】
図３３は、認識結果の整形表示処理において各文字毎に回転する場合の説明図である。図３４は、認識結果の整形表示処理についてのフローチャート（２）である。なお、ここでの処理は、文認識結果の整形表示面において、文字方向補正文字列構成要素に含まれる文字（ここでは、文字フォント）毎のフォント表示の基点を算出し、この基点に文字フォントを回転して配置する。
【０１１０】
最尤文字列探索処理部１３は、文字方向補正文字列構成要素ごとに手書き文字の平均サイズを求め（Ｓ２２３１）、この平均サイズに近い出力文字ポイント数を決めると共に（Ｓ２２３２）、手書き文字間の縦横それぞれの平均間隔を求める（Ｓ２２３３）。つぎに、最尤文字列探索処理部１３は、文字コード列に従い、ステップＳ２２３２での出力ポイント数、かつ、ステップＳ２２３３での平均間隔で、筆記方向に従って、ｉ番目の文字フォントを配置する場合の基点（Ｘ_ｉ、Ｙ_ｉ）を、回転中心を原点として計算する（Ｓ２２３６）。また、最尤文字列探索処理部１３は、認識結果の整形表示面において、次の数式１に従って、フォント表示の基点を求める（Ｓ２２３７）。
【０１１１】
【数１】

【０１１２】
ただし、θは、この文字方向補正文字列構成要素に記録され、これを文字毎に適用するための回転角である。また、（Ｘ_Ｃ、Ｙ_Ｃ）は、表示画面における回転中心である。
また、最尤文字列探索処理部１３は、ｉ番目のフォントについて、フォントを−θだけ回転して、ステップＳ２２３７により得られた（Ｘ_ｉ´、Ｙ_ｉ´）を基点に配置する（Ｓ２２３８）。ここでは、図３３に示すように、文字方向補正文字列構成要素「農工大」に含まれる１番目のフォント「農」、２番目のフォント「工」、３番目のフォント「大」を、−θだけ回転してそれぞれ配置することにより、最初の手書き文字列に近い認識結果の表示を行っている。すなわち、ここでの認識結果の整形表示処理は、整形表示後の文字位置を文字ごとに計算して、それに合うようにフォントを回転して配置することにより整形表示を実現している。
【０１１３】
以下に、図３２、３４に示す認識結果の整形表示処理を行うことで示される認識結果の整形表示を示す。
図３５は、認識結果の整形表示を示す図（１）である。ここでの文字フォントは、手書き文字に置き換えて表示する例を示している。なお、図中左側が手書き入力、図中右側が手書き入力に対する整形表示後の出力を示している。
ここで、文字サイズ、文字位置、筆記方向及び文字方向の異なる、手書き入力された各文字列構成要素「今日の献立」「カツ丼」「天丼」「うな丼」「親子丼」は、整形表示後、それぞれ最初の手書き文字列に近い認識結果の表示が行われる。
【０１１４】
図３６は、認識結果の整形表示を示す図（２）である。なお、ここでは、図３６での手書き入力を示す図に対応するのは、図３１での認識前を示す図である。
ここで、文字サイズ、文字位置、筆記方向及び文字方向の異なる、手書き入力された各文字列構成要素（ここでは、４つの「農工大」）は、整形表示後、それぞれ最初の手書き文字列に近い認識結果の表示が行われる。なお、図３１に示す認識前の筆記面で上から逆向きに書かれた「農工大」の認識結果表示が、認識後の筆記面で傾いていたことにより、文字認識結果は正しいが、文字方向の推定に誤差が発生したことになり、その文字方向を復元した図３６での認識結果の整形表示では、文字フォントが多少傾いて表示されることになる。
【０１１５】
このように、本実施の形態によれば、キーボードに変わる（あるいは共存する）より自然な文字入力を可能とする文字入力手段や、マウスやキーボードが苦手な人のための文字入力手法及びペンインタフェースを用いることによる、思考を妨げない自然な文字入力手法を提供することができると共に、文字の筆記方向や文字の向いている文字方向に依存しない認識を行うことで、これまで困難であった、文字の傾きやあらゆる方向に書かれた文字列を認識することができる。
【０１１６】
なお、本発明の文字認識方法は、文字認識プログラム、文字認識プログラムを記録したコンピュータ読み取り可能な記録媒体、文字認識プログラムを含みコンピュータの内部メモリにロード可能なプログラム製品、文字認識プログラムを含みコンピュータが使用可能な記録媒体にストアされたプログラム製品等により提供されることができる。
【０１１７】
【発明の効果】
本発明によると、以上説明した通り、枠なしで手書き入力された文字に対するより自由度の高い、筆記方向に依存しない枠なし文字認識（自由筆記文字列認識）を行うことができる。
【０１１８】
また、本発明は、改行によって区切られた文字列ごとに、この文字列の幾何的特徴に基づいて筆記方向を推定すると共に、文字切出し時に筆記方向に応じた仮分割を行うことができる。また、本発明は、例えば、既存の認識システムに適用することにより、筆記方向に依存しない枠なし文字認識、自由筆記文字列認識を行うことができる。
【０１１９】
また、本発明は、例えば、文字切出しの確からしさ、文字認識の確からしさ、文字サイズの確からしさ、及び文字連接の確からしさを、それぞれ総合的に判断することができる。また、本発明は、例えば、筆記方向、文字方向、行ごとの文字の大きさ変動に依存しない枠なし文字認識を行うことができる。
【図面の簡単な説明】
【図１】本発明に関する筆記方向に依存しない枠なし文字認識装置を適用したオンライン枠なし文字認識システムの概略構成図。
【図２】本発明に関する文字認識方法のフローチャート。
【図３】文字サイズ推定処理についての説明図。
【図４】文字要素間遷移分布を示す説明図。
【図５】横書きと縦書きが混在する文字列を示す図。
【図６】筆記方向変化点の検出を示す説明図。
【図７】最遠点を再帰的に求めた場合についての説明図。
【図８】筆記方向変化が微小である場合について説明図。
【図９】筆記方向推定についての説明図。
【図１０】筆記方向別の仮分割を示す説明図。
【図１１】分割時の評価についての説明図。
【図１２】仮分割修正についての説明図。
【図１３】重なりの評価を示す説明図。
【図１４】実験用オンライン枠なし文字パターンデータを示す図。
【図１５】正認識の例を示す図（１）。
【図１６】正認識の例を示す図（２）。
【図１７】従来の枠なし文字列認識及び単文字認識を開始した際の表示画面についての説明図。
【図１８】筆点座標とストロークの説明図。
【図１９】文字方向と筆記方向の説明図。
【図２０】本実施の形態に関する文字認識方法のフローチャート。
【図２１】文字列構成要素分割処理についての説明図。
【図２２】文字サイズ詳細推定処理の説明図。
【図２３】文字方向と方向別筆点間距離累積ヒストグラムの説明図。
【図２４】方向別筆点間距離累積ヒストグラムにおけるピークの検出についての説明図。
【図２５】ピークの検出を示すフローチャート。
【図２６】文字方向が上下しかない場合でのピークの検出についての説明図。
【図２７】筆記方向から複数の文字方向を仮定する場合についての説明図。
【図２８】認識用文字方向補正回転処理の説明図。
【図２９】筆記方向量子化処理を示すフローチャート。
【図３０】認識結果の整形表示処理の前段階の認識結果を示す図（１）。
【図３１】認識結果の整形表示処理の前段階の認識結果を示す図（２）。
【図３２】認識結果の整形表示処理についてのフローチャート（１）。
【図３３】認識結果の整形表示処理において各文字毎に回転する場合の説明図。
【図３４】認識結果の整形表示処理についてのフローチャート（２）。
【図３５】認識結果の整形表示を示す図（１）。
【図３６】認識結果の整形表示を示す図（２）。
【符号の説明】
１ 文字列認識システム
２ ユーザ
３ アプリケーション
４ 入力出力部
５ 入力文字列パターン入力部
６ 認識結果出力部
１０ 認識コントロール部
１１ 文字サイズ推定処理部
１２ 文字切出し処理部
１３ 最尤文字列探索処理部
１４ 認識エンジン部
１５ 最尤文字列探索部[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a character recognition method, a character recognition program, a computer-readable recording medium on which the character recognition program is recorded, and a character recognition device, and in particular, without using a keyboard and without providing an entry frame (writing frame), Character recognition method capable of recognizing characters (character strings) input by handwriting in a free direction (for example, vertical, horizontal, and diagonal directions), a character recognition program, a computer-readable recording medium recording the character recognition program, and Character recognition device
About.
[0002]
[Prior art]
In recent years, when inputting characters into a computer equipped with a display device, handwriting input is performed within a set frame using an appropriate desktop digitizer (coordinate reading device) such as a mouse or tablet without using a keyboard. And a method for recognizing the handwritten character is known. Here, the handwriting input is also referred to as stroke input (character element input), and refers to input of characters and lines that can be expressed, such as character input, a single stroke of drawing, and a drawn line between two points. According to this method, even people who are unfamiliar with the keyboard operation can input desired characters on a display device such as a computer and create various sentences.
[0003]
Further, in the conventional method, a frame to be entered for each character may be set in advance depending on the computer application (application software). Such recognition with a single character within a frame (hereinafter referred to as single character recognition) was originally put to practical use as a method of character input in a small terminal such as a PDA (portable information terminal). Is sufficiently durable for practical use and has been established as a character input interface for existing PDAs and notebook PCs.
[0004]
However, in this single character recognition, when a writer (user) performs handwriting input, the entry position is limited within the frame, making it difficult to write, or handwriting only one character within one frame. Since input is not possible, it is assumed that the user needs time and effort when creating continuous sentences. As a result, in this single character recognition, characters entered one by one in the character entry box are sequentially recognized, so that the writer inputs the characters as individual characters rather than as a plurality of character strings. For example, this may be a psychological burden, and smooth character input may not be expected.
[0005]
In order to prevent the above-described matters, what is necessary is to propose a so-called “no frame” character string recognition in which no frame is set in advance. This frameless character string recognition has been accompanied by the introduction of interactive electronic blackboards (usually white boards) in school education accompanying the recent “informatization of education” and the increase in the size of liquid crystal screens mounted on PDAs and pen PCs. Due to the expansion of the entry and drawing space, etc., for example, attention has been attracted in response to demands regarding the degree of freedom when inputting characters using a pen interface (inputting characters in a desired writing direction). In the flow of the above, from the recognition with a single character in the frame (hereinafter referred to as single character recognition), the above-described concept of the character entry frame is removed, and a desired character string (phrase) intended by the writer is input without psychological burden. The technique is shifting to a frameless character string recognition that can collectively recognize character strings input to the entry / drawing space.
[0006]
FIG. 17 is an explanatory diagram of a display screen when conventional frameless character string recognition and single character recognition are started.
For example, an input / output display screen (drawing area) 101 without a frame and a function input unit 102 are displayed on the frameless character string recognition display screen 100. When the writer enters a character string (here, “Department of Electronic Information Engineering, Faculty of Engineering, Tokyo University of Agriculture and Technology”), a circumscribed rectangle 103 indicating the delimiter position of each character is displayed, and the character strings are collectively recognized. The recognition result of each character can also be displayed in the rectangle 103. The function input unit 102 corresponds to a plurality of functions, and for example, if an appropriate instruction is executed by clicking on a handwritten input character that is a recognition target, the function input unit 102 performs an optimum operation on the handwritten input character. Has been placed. The function input unit 102 includes, for example, a recognition button, an erase button, and the like.
[0007]
In addition, on the single character recognition display screen 200, for example, an input / output display screen 201 in which a frame 203 is set in advance and a function input unit 202 are displayed. Further, when the writer enters, for example, one character at a time in the frame 203 using the electronic pen 204 (here, “Study”, “Engineer”, “Large”), the entered character is recognized as a single character. The
[0008]
[Problems to be solved by the invention]
However, most of the existing research on frameless character recognition focused on horizontal character strings. For example, in online recognition of frameless written strings by candidate character lattice (Hiroshi Murase, Toru Wakahara, Michio Umeda: IEICE Transactions, vol.J68, no. 4, pp.765-772 (1985)) If there is no frame, the character may be separated between the bias and the heel, so the one with the higher probability of character recognition when separated and when recognized together is adopted The process is done. However, this method improves recognition rate, such as using language processing only for verification, not evaluating the accuracy of character extraction, and empirically setting evaluation function parameters. There are factors that prevent it.
[0009]
In addition, a frameless online string recognition method that integrates clipping, recognition, and language certainty (Shu Shuda, Masahiko Hatanaka, Keigo Yamada: IEICE Technical Report, PRMU, vol.98-138, no.489, pp.17 -24 (1998.12).) Does not consider the accuracy of the character size of the characters entered by the writer. In Japanese Patent Application No. 8-250558, a character string in which horizontal writing and vertical writing are mixed is not subject to recognition, and the writer provides a baseline in order to teach the writing direction of the character string to the system. It is necessary to fill in.
[0010]
In the above-described method, processes other than Japanese Patent Application No. 8-250558 are shown assuming that a horizontally written character string is assumed. In these, for example, it is possible to perform processing corresponding to vertical writing by reversing the x-axis and y-axis which are the coordinate axes of the character string. In such frameless character recognition, not only horizontal writing but also the writing direction is limited to facilitate the estimation of the character size of each character element included in the character string, and the character extraction direction is limited to the horizontal or vertical direction. Thus, overdivision can be prevented and character type candidates can be narrowed down.
[0011]
However, if the writer mixes vertical writing and horizontal writing, moves from horizontal writing to vertical writing, and vice versa, the conventional method cannot handle it, and sufficient character recognition is possible. It was difficult to do. Furthermore, in the existing system, a loss of the degree of freedom of input by the writer (referring to the degree of freedom during writing such as the writing direction) is assumed.
[0012]
That is, when the writing direction is limited, there are some advantages on the system side, but this advantage does not satisfy the user's request. In frameless character recognition, an easy-to-use recognition system for a user is considered to be a system that can recognize a sentence that comes to mind in a state where it is naturally written. Furthermore, the loss of freedom during writing can be a cause of disturbing the writer's thinking.
Furthermore, in the above-described frameless character string recognition, for example, it is assumed that one or a plurality of writers write vertically, horizontally, or diagonally from the side of the front or from the opposite side of the writing surface. It is necessary to further relax the writing restriction, which is a loss of freedom during writing. In addition, due to various situations such as the introduction of interactive electronic whiteboards in school education and the increase in the size of liquid crystal screens associated with the recent “informatization of education”, there is a need for relaxed restrictions when entering characters using a pen interface. It can be expected to increase further.
As a specific situation, for example, in a counter service such as a bank, when a receptionist and a customer need to recognize a character string written from different directions (for example, facing each other), the electronic whiteboard is slanted. When written, and in development scenes using CAD, etc., for example, when a plurality of developers enclose a drawing displayed on a flat screen and enter characters, etc. This situation is assumed.
Therefore, in frameless character string recognition, for example, as character input, free character input using a pen tablet or the like, interactive or cooperative character input, and characters from a plurality of writers on the same tablet or the like It is desirable to be able to perform writing on the electronic whiteboard, displaying and formatting, storing, and reusing it while considering the input.
[0013]
In view of the above, it is an object of the present invention to perform frameless character recognition (free handwritten character string recognition) independent of the writing direction with a higher degree of freedom for a character input by handwriting without a frame.
Another object of the present invention is to estimate the writing direction for each character string delimited by a line feed based on the geometric characteristics of the character string and to perform provisional division according to the writing direction at the time of character extraction. . Another object of the present invention is to perform frameless character recognition and free handwritten character string recognition independent of the writing direction, for example, by applying to an existing recognition system.
[0014]
Another object of the present invention is to comprehensively determine, for example, the accuracy of character extraction, the accuracy of character recognition, the accuracy of character size, and the accuracy of character concatenation. Another object of the present invention is to perform frameless character recognition that does not depend on, for example, the writing direction, the character direction, and the variation in character size for each line.
[0015]
According to the first solution of the present invention,
A character string input step for inputting information on each character element of a character string input by handwriting without specifying a writing direction without a frame or a line;
A character size estimating step for estimating a character size of the character string input by the character string input step;
The distance between each character element in the string, Estimated Corresponding to text size First Based on the threshold and detect the line break status of the string For character string components that are divided into character string components and separated by line breaks The farthest away from the line connecting the start and end points Dot By asking The farthest point Change in writing direction As a point Detect Each character string component separated by a line feed is further divided at the change point in the writing direction, and for each character string component based on the vertical length and horizontal length of the circumscribed rectangle of the character string component A writing direction estimation step for estimating a writing direction;
A character string along the writing direction estimated by the writing direction estimation step The Provisional division To find character components A temporary split step;
By the provisional division step For each character component that has been temporarily divided, if the overlapping area of the circumscribed rectangle of each character component is greater than or equal to the second threshold value, the character components are integrated Temporary split correction step and
A character recognition method, a character recognition program, and a computer-readable recording medium on which the character recognition program is recorded.
[0016]
According to the second solution of the present invention,
A character string input step for inputting information on each character element of a character string input by handwriting without specifying a writing direction without a frame or a line;
A character size estimating step for estimating a character size of the character string input by the character string input step;
A writing estimation step for identifying a character string based on a displacement of a writing point coordinate equal to or greater than a threshold corresponding to the character size;
A temporary dividing step of performing a temporary division of the written estimated character string obtained by the writing estimating step;
A character recognition method and a character recognition program including a provisional division correction step of dividing or integrating each character element of the estimated handwritten character string determined as a provisional division candidate by the provisional division step according to the overlapping state of the character elements. And a computer-readable recording medium on which a character recognition program is recorded.
[0017]
According to the third solution of the present invention,
An input character string pattern input unit for inputting information on each character element of a character string input by handwriting without specifying a writing direction without a frame or a line;
A character size estimation processing unit that estimates a character size of a character string input by the input character string pattern input unit;
A character cutout processing unit for cutting out the character string;
With
The character cutout processing unit
The distance between each character element in the string, Estimated Corresponding to text size First Based on the threshold and detect the line break status of the string For character string components that are divided into character string components and separated by line breaks The farthest away from the line connecting the start and end points Dot By asking The farthest point Change in writing direction As a point Detect Each character string component separated by a line feed is further divided at the change point in the writing direction, and for each character string component based on the vertical length and horizontal length of the circumscribed rectangle of the character string component A writing direction estimating means for estimating a writing direction;
A character string along the writing direction estimated by the writing direction estimating means The Provisional division To find character components Temporary dividing means;
By the temporary dividing means For each character component that has been temporarily divided, if the overlapping area of the circumscribed rectangle of each character component is greater than or equal to the second threshold value, the character components are integrated Temporary division correction means
A character recognition device comprising:
According to the fourth solution of the present invention,
A character string input step for inputting information on each character element of a character string input by handwriting without specifying a character size, a writing direction, and a character direction without a frame or a line;
A character size estimating step for estimating a character size of the character element input by the character string input step;
Based on the estimated character size estimated in the character size estimation step and the interval between adjacent strokes constituting the character, a line feed, a certain interval separating lines and lines, or a change point in writing direction is detected. A dividing step of dividing the first character string component,
A character direction estimating step for estimating the character direction from the variation information of the handwriting coordinate in the stroke excluding the variation between strokes for each first character string component divided by the dividing step;
A rotation center and a rotation angle for each first character string component are obtained and stored in a memory so that the character direction estimated by the character direction estimation step is upward, and the first character string component is rotated. A rotation step for obtaining a second character string component;
A recognition step for performing character recognition on each character element of the second character string component;
A character recognition method, a character recognition program, and a computer-readable recording medium on which the character recognition program is recorded.
According to the fifth solution of the present invention,
A character string input step for inputting information on each character element of a character string input by handwriting without specifying a character size, a writing direction, and a character direction without a frame or a line;
A character size estimating step for estimating a character size of the character element input by the character string input step;
Based on the estimated character size estimated in the character size estimation step and the interval between adjacent strokes constituting the character, a line feed, a certain interval separating lines and lines, or a change point in writing direction is detected. A dividing step of dividing the first character string component,
A character direction assumption step in which one or more of the positive and negative directions of the writing direction, the two directions perpendicular to the writing direction, and the upper, lower, left, and right directions of the desired writing surface are assumed to be the character direction;
A rotation center and a rotation angle for each of the first character string components are obtained and stored in a memory so that the character direction is upward with respect to the plurality of character directions assumed in the character direction assumption step. A rotation step of rotating a first character string component to obtain a plurality of second character string components;
A recognition step for performing character recognition on each character element of the second character string component for each character direction;
A selection step of selecting a character direction and a second character string component having a high probability of recognition among the second character string components corresponding to the respective character directions recognized by the recognition step;
A character recognition method, a character recognition program, and a computer-readable recording medium on which the character recognition program is recorded.
[0018]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
(1) First embodiment
FIG. 1 is a schematic configuration diagram of an on-line frameless character recognition system to which a frameless character recognition device independent of the writing direction according to the present invention is applied.
The character recognition system 1 is a base for recognizing a free handwritten character string. For example, an input output unit 4 for inputting a character string and outputting a recognition result, and a recognition control unit for a frameless handwritten character (character string). 10, a handwritten character string is input from the user 2, and a recognition result is output to the application 3. The input output unit 4 includes, for example, an input character string pattern input unit 5 and a recognition result output unit 6.
[0019]
The input character string pattern input unit 5 uses, for example, an appropriate desktop type digitizer such as an interactive electronic white board, a display-integrated tablet, and other tablets, a pointing device such as a mouse, a trackball, and an electronic pen ( The character string pattern input by the writer 2 in any direction (for example, vertical, horizontal, and diagonal directions) is input to the recognition control unit 10. The recognition result output unit 6 outputs the recognition result of character string recognition output from the recognition control unit 10 to the application 3.
[0020]
The recognition control unit 10 includes, for example, a character size estimation processing unit 11, a character cutout processing unit 12, and a maximum likelihood character string search processing unit 13. The character size estimation processing unit 11 estimates the size of each character included in the character string input by the writer. The character cutout processing unit 12 estimates the writing direction based on the line feed situation, the change in the writing direction, the temporary division based on the geometric feature evaluation between strokes (character elements), and the character break caused by the temporary division correction. In addition to detection (details will be described later), character candidates are generated (character cutout). The maximum likelihood character string search processing unit 13 performs a probable character string search and character recognition for each character element included in the character string after character extraction. Further, the maximum likelihood character string search processing unit 13 performs, for example, an existing recognition engine unit 14 that performs character recognition of a character string without a frame, and a character that seems to be certain based on the character shape, context, character division, and character size. And a maximum likelihood character string search unit 15 for searching for a string.
[0021]
FIG. 2 is a flowchart of a character recognition method according to the present invention. For convenience of explanation, the explanation of the flowchart here is schematic, and the specific explanation will be described later.
First, a description will be given of a character cutting method based on writing direction estimation, in which a writing direction is estimated in advance and cutting is performed for each writing direction. The character cut-out method based on the writing direction estimation is based on the characteristics of strokes (character elements) constituting the character string, and estimates the character size and writing (handwriting) direction, as well as provisional division and provisional division correction for each writing direction. Is to do.
[0022]
The input character string pattern input unit 5 inputs a character string pattern input in any direction without a frame by the user 2 (S101). The character size of each character included in the input character string pattern is estimated by the character size estimation processing unit 11 (S103). Next, the character cutout processing unit 12 estimates the writing direction (horizontal, vertical, diagonal, etc.) of the character string based on, for example, a line feed and a writing direction change point (S105). Furthermore, the character cutout processing unit 12 performs provisional division and provisional division correction based on the estimated writing direction (S107, S109).
[0023]
Next, the maximum likelihood character string search processing unit 13 performs character recognition on the character string that has been subjected to character extraction by the above-described recognition engine unit 14 and maximum likelihood character string search unit 15, and performs the character recognition. The result is output to the recognition result output unit 6. The recognition result output unit 6 outputs the recognition result for display on a display screen (not shown) of the application 3 (S111). Note that in order to make an existing system compatible with free-writing character string recognition, generation of character candidates (character extraction) is required for any character string. Therefore, the recognition engine unit 14 that performs character recognition and the context The maximum likelihood character string search unit 15 that performs processing or the like is used as it is, and character extraction processing corresponding to steps S105, 107, and 109 in the character extraction processing unit 12 is performed without depending on the writing direction.
Hereinafter, the process of each step of the above-described flowchart will be described in detail.
[0024]
(About Step S101)
A handwritten character (character string) is input by the user 2 using the input / output unit 4, and information on each character element is stored. The information regarding each character element includes, for example, a start point, an end point, a brush point coordinate position therebetween, a writing direction, an order, and the like.
[0025]
(About Step S103)
FIG. 3 is an explanatory diagram of character size estimation processing.
First, when a character string handwritten by a writer includes character strings such as horizontal writing, vertical writing, diagonal writing, etc., a method of using projection with a fixed writing direction to estimate the character size It is not appropriate to use. For this reason, it is conceivable to estimate the character size by obtaining an average value of height and width for each character element of the character string.
[0026]
On the other hand, for example, when the character string includes kanji, which is a set of fine (short) character elements, in a large number of strokes, the estimated character size may be considerably smaller than the true character size. Therefore, when the character string includes kanji characters, etc., the fine character elements are ignored, and a relatively large (long) character element is regarded as a character element representing a character (for example, kanji). What is necessary is just to obtain the average of long character elements. According to this method, it is possible to expect faster processing than the above-described method using projection.
[0027]
The specific procedure is shown below.
Here, as an example, a case will be described in which the size of a kanji character “song” having a total number of strokes of 6 is estimated.
First, each stroke size of the total number of strokes 6 is measured. Each stroke size is represented by, for example, 33, 40, 52, 35, 47, and 20 using appropriate units such as the number of dots and length, respectively. Are stored in blocks in the figure according to the writing order. The stroke size is the larger of the width and height of the character element. At this time, the average value of the total stroke size is 38, and this value becomes the estimated character size (estimated character size). However, this estimated character size is significantly smaller than the true character size.
[0028]
On the other hand, the above stroke sizes are sorted in descending order and stored in blocks (here, 52, 47, 40, 35, 33, 20). Furthermore, the average value of the upper rank (for example, upper half, 52, 47, 40) among the stroke sizes stored in the block is obtained. At this time, the average value of the top three stroke sizes is 46, and this value becomes the estimated character size (estimated character size). This estimated character size is somewhat close to the true character size. As an example, the average value of the upper order (for example, the upper half) × r (here, r = 1) = character size estimated value may be defined.
[0029]
(About Step S105)
Here, a writing direction estimation process for determining a writing direction based on an aspect ratio of a circumscribed rectangle of the character string by specifying a character string for estimating the writing direction by detecting a line feed and a writing direction change point. explain.
[0030]
First, detection of a line break will be described.
FIG. 4 is an explanatory diagram showing a transition distribution between character elements.
The character element interval can be classified into two types, for example, a small transition in a character string (transition distribution within a character string) and a large transition between character strings due to a line feed (distribution indicating a line feed). In addition, the several point which shows the transition distribution in a character string and the distribution which shows a new line in the figure has shown distribution of the distance between each character element. The distance between each character element is, for example, using appropriate values such as the start point, end point, and center of gravity of each character element, between the start points, between the end points, between the start point of one character element and the end point of another character element, It can be determined from the distance between the centers of gravity.
[0031]
If it is possible to determine in advance whether or not the character string handwritten by the writer has a line break, the line break can be detected by clustering the above two types of transitions. Here, application of clustering using a so-called LBG algorithm can be considered. However, with this method, it is assumed that there is a slight difference in the recognition time, but clustering is performed only in the initialization part of the LBG (decision of centroid, setting of sphere of influence). .
[0032]
First, a line feed threshold is determined based on the character size estimated in step S103, and a strict line feed is detected when a distance between character elements equal to or greater than the threshold is detected. . In the detection of line breaks, the minimum and maximum values of transitions between character elements are set as initial values of centroids, and the clusters obtained by the centroids are respectively clusters indicating transitions within character strings (transitions between character elements). And a cluster indicating a line break (based on the maximum value of transition between character elements). At this time, the loop part serving as the core of the LBG is omitted. However, since clustering is performed when there is clearly a line break, the solution obtained by this centroid is close to the quasi-optimal solution. Even in this method, it is judged that there is no problem in line feed detection accuracy. Since this process is simply viewed by the distance between the character elements, it is possible to detect a line break accompanying a transition in the Y direction or diagonal direction.
[0033]
Next, detection of the writing direction change point will be described.
FIG. 5 is a diagram showing a character string in which horizontal writing and vertical writing are mixed.
A character string that is freely handwritten by a writer can be assumed to be bent at a right angle from the middle of the character string. Here, the text string by the writer is the horizontal “Today's menu”, the vertical “Today ’s menu”, and the vertical “Katsudon” that is not separated only by line breaks. Including “Tengu”, “Una” and “Parent and child” separated by line feeds.
[0034]
For this reason, even when a character string is specified by the above-described line feed detection, it is assumed that it is difficult to estimate the writing direction if there are abrupt changes in writing direction or character strings with different writing directions are mixed. Is done. In other words, a character string that is delimited as a character string but not separated only by a line feed appears (here, it corresponds to “Today's menu cutlet”). Therefore, in such a case, it is necessary to detect a point where the writing direction is changed and to divide the character string in the same manner as a line feed.
[0035]
FIG. 6 is an explanatory diagram showing detection of a writing direction change point.
The writing direction change point is detected by obtaining the farthest point from the straight line connecting the start point and the end point of the character string. However, if the farthest point of the character string is simply obtained, it is expected that a minute change will be detected even in a character string with no change in the writing direction. Thus, only a relatively large change in writing direction can be detected. In addition, instead of the farthest point, a relatively far point may be obtained, and an appropriate writing direction change point may be obtained.
[0036]
Specifically, here, the text string by the writer is the horizontal "Today's menu" and the vertical "curry and rice" that is bent at a right angle from this horizontal "Today's menu" and not separated only by line breaks. Including. First, a straight line A is created by connecting the start point of the character string (here, the start point of the “first” first stroke) and the end point of the character string (here, the end point of the second stroke of “su”). The farthest point among the points where the distance d from the straight line A is larger than the estimated character size is obtained, and this farthest point is set as the writing direction change point of the character string (here, the second stroke of “standing”). End point).
[0037]
FIG. 7 is an explanatory diagram of a case where the farthest point is obtained recursively.
In the above-described method, only one writing direction change point can be detected. Here, a case where a plurality of writing direction change points are included in the character string will be described. In this case, the process for finding the farthest point is used recursively (K. Ishigaki, T. Morishita: "A Top-down Online Handwritten Character Recognition Method via the Denotation of Variation," Proc. Int'l Conf.on Computer Processing. on Chinese and Oriental Languages, Toronto, pp141-145 (1988.8).), it is possible to handle character strings whose writing direction changes multiple times. Specifically, a character string including three writing direction change points (here, a “curvy character string”, which is simplified by a dotted line in the figure), a start point S (here, “the number of the“ music ”). The start point of one stroke) and the end point E (here, the end point of the sixth stroke of the “row”) are connected by a straight line, and the farthest point P1 from the line segment SE (here, the first stroke of “sentence”) by the method described above. The starting point).
[0038]
Next, the starting point S of the character string (here, the starting point of the first stroke of “Song”) and P1 (here, the starting point of the first stroke of “Sentence”) are connected by a straight line, and from the line segment SP1 by the method described above. The farthest point P2 (here, the farthest point from the line segment SP1 in the second stroke of “RI”) is obtained. The character string P1 (here, the start point of the first stroke of “sentence”) and the farthest point P2 (here, the farthest point from the line segment SP1 in the second stroke of “ri”) are connected by a straight line. The farthest point P3 from the line segment P1P2 (here, the starting point of the second stroke of “Ne”) is obtained by the method described above.
Thus, by recursively obtaining the farthest point, the writing direction change point of various character strings can be detected.
[0039]
FIG. 8 is an explanatory diagram of a case where the writing direction change is minute.
In addition, when the writing direction change is small, it is assumed that the character size (threshold value), which is a condition for obtaining the farthest point, is deviated. Specifically, when a character string including one writing direction change point that cannot be detected by the writer (here, “I hope it will clear tomorrow”) is input by handwriting, the starting point S of the character string “M” Even if the end point E “na” is connected with a straight line and the farthest point from the line segment SE is obtained by the above-described method, for example, the distance from the line segment SE (for example, “re” separated by the distance d1 and the distance d2). If both are separated from the estimated character size th (for example, d2 <d1 <th), the farthest point cannot be obtained. The estimated character size th can be appropriately determined such as the average of each character size, the average of each character size excluding one or a plurality from the minimum, the minimum, or a predetermined value.
[0040]
Therefore, even when the writing direction change is very small, the process proceeds as it is, and when performing the temporary division process described later, the character size after division (here, “L” is provisionally divided as one character). If it is clearly larger than the estimated character size th, a process of dividing the character element under evaluation is performed (here, “le” is divided into “re” and “ru”, respectively). Further, for each character string delimited by a line feed, the character string is divided into character string components by delimiting each character string at a change point in the writing direction. Thereby, a writing direction can be estimated per character string component.
[0041]
Next, estimation of the writing direction will be described.
FIG. 9 is an explanatory diagram of writing direction estimation.
As described above, by detecting the line feed and the writing direction change point, it is possible to specify the character string (character string component) for estimating the writing direction. Specifically, each character string component is divided by a line feed, and includes, for example, a character string component 1 “Today is rainy” and a character string component 2 “I hope it will clear tomorrow”. . Further, the aspect ratio of the circumscribed rectangle of these character string components 1 and 2 (for example, the horizontal direction X and the vertical direction Y) is examined. If X> Y, the writing direction is estimated as the horizontal direction, If X <Y, the writing direction is estimated as the vertical direction. Further, after estimating the writing direction for each character string, writing direction information may be added to each character element. In the provisional division described later, provisional division is performed in the vertical or horizontal direction using the writing direction information. In general, when a character string component is written by moving a brush in a direction opposite to the writing direction, there is almost no character separation between the two strokes. If the writing direction is estimated, this can be used to detect line breaks and writing direction change points. Furthermore, the distance between the two strokes can be determined by the division and integration processing described later.
[0042]
(About Step S107)
Next, provisional division will be described.
FIG. 10 is an explanatory diagram showing provisional division for each writing direction.
In provisional division, as described above, provisional division is performed in the X direction for horizontal writing and in the Y direction for vertical writing based on the writing direction information. Specifically, if the writing direction is horizontal writing, it is provisionally divided in the X direction (here, “agriculture” “engineering” in horizontal writing). If the writing direction is vertical writing, it is provisionally divided in the Y direction (here, “farming” and “engineering” in vertical writing). In the case of diagonal writing, division is performed assuming that the inclination is closer to either vertical writing or horizontal writing. In the provisional division by the above-described method, the number of provisional divisions can be reduced to, for example, half compared to a method of simple division in a divisible direction described later.
[0043]
FIG. 11 is an explanatory diagram of evaluation at the time of division.
In the temporary division process, various evaluations are given to each character element based on the geometric features between the character elements. Specifically, in the character string “Mojito”, each character (here, six) is divided (divided), and each character element constituting the character “M” is combined (combine). , Each character element constituting the character “ji” is evaluated as a division candidate (vague).
[0044]
This division determination (division, combination, division candidate) is based on the writing direction information added to the character elements. For example, as described above, the relationship between the character elements in the X direction for horizontal writing and the Y direction for vertical writing as described above. To evaluate. Also, this evaluation item is the projection distance between the centers of gravity of the character elements and the degree of overlap of the circumscribed rectangles of the character elements. Here, in order to determine that each character element of “MO” is combined, for example, the overlap of circumscribed rectangles of each character element is evaluated by a threshold, and if the overlap is equal to or greater than a predetermined threshold, It can be determined as a bond.
[0045]
(About Step S109)
Next, provisional division correction will be described.
FIG. 12 is an explanatory diagram of provisional division correction.
In the provisional division correction, as in the case of the provisional division described above, character components are combined in the Y direction for vertical writing and in the X direction for horizontal writing while looking at the writing direction. In addition, after the provisional division processing and before executing the frameless character recognition, by performing some determinations independently of the previous processing, a portion determined as a division candidate (vague) The determination is made as much as possible for division or combination, and the search space for estimating the maximum likelihood character string in the character recognition stage is reduced.
[0046]
Further, here, the overlap of circumscribed rectangles between the character constituent elements is evaluated by a threshold value, and if the overlap is equal to or greater than the threshold value, a process for determining the elements to be integrated is performed. Specifically, when the character “Michi” hen and 旁 are respectively separate character components, it is determined that the overlapping area is equal to or greater than the threshold (th) (here, X * Y). > Th). That is, here, the determination is based on the area of the overlap in the horizontal direction and the overlap in the vertical direction.
[0047]
(About Step S111)
After the temporary division correction in step S109, the maximum likelihood character string search processing unit 13 seems to be certain based on the character shape, context, character division, and character size for each character element by the maximum likelihood character string search unit 15. The search for the character string is performed based on the dynamic programming, and the character recognition of the character string is performed by the recognition engine unit 14. The character recognition result is output to the recognition result output unit 6. The recognition result output unit 6 outputs the recognition result for display on a display screen (not shown) of the application 3.
[0048]
As a result, the character cutting method based on the writing direction estimation described above includes the process of estimating the writing direction based on the detection of the line feed and the writing direction change point in step S105. However, if the writing direction can be correctly estimated, an accurate provisional division is performed. It can be carried out. This method is a method of estimating the writing direction of a character string in advance and dividing the character string in the writing direction using a threshold value, and can minimize the number of temporary divisions. By minimizing the number of provisional divisions, the number of recognized character strings can be reduced, and the search for a character string that seems to be likely based on the above-mentioned character shape, context, character division, and character size can be performed. It is possible to minimize the trouble of finding the optimum route in the maximum likelihood character string search unit 15 based on the genetic programming method.
[0049]
Therefore, the character cutting method based on the estimation of the writing direction can perform frameless character recognition (free writing character string recognition) independent of the writing direction with a higher degree of freedom with respect to characters input by handwriting without a frame.
In addition, in the character extraction method based on the writing direction estimation, for example, for each character string delimited by line breaks, the writing direction is estimated based on the geometric characteristics of the character string, and provisional division according to the writing direction at the time of character extraction is performed. It can be performed. Further, by applying the character cutout method based on the writing direction estimation to, for example, an existing recognition system, it is possible to perform frameless character recognition and free writing character string recognition independent of the writing direction. Further, in the character extraction method based on the writing direction estimation, for example, the accuracy of character extraction, the accuracy of character recognition, the accuracy of character size, and the accuracy of character connection can be comprehensively determined.
[0050]
(2) Second embodiment
Next, in the flowchart of the character recognition method shown in FIG. 2, a description will be given of a character cutout method that does not consider the writing direction of the character string by simplifying the process of step S105 described above. In addition, the part which overlaps with the character extraction method by the above-mentioned writing direction estimation is abbreviate | omitted to such an extent that description becomes redundant.
[0051]
This method is a method of dividing a character string in a direction in which the character string can be divided based on a threshold value obtained from the character size. This character cutting method that does not consider the writing direction is a method of performing temporary division in a direction that can be divided regardless of the writing direction, and performs heuristic cutting regardless of the writing direction.
[0052]
First, the character size estimation processing unit 11 described above estimates the character size and determines a threshold value used for cutout and line feed determination (similar to S103 described above). Next, the character cutout processing unit 12 performs an evaluation between two character elements of the character string input by the writer, and when there is a displacement of the handwriting coordinate exceeding the threshold value in the X direction and the Y direction. A character string is obtained as a division point (writing estimation step corresponding to simplified step S105).
[0053]
Temporary division is performed in a direction in which this character string can be divided (similar to S107 described above). Further, the character cutout processing unit 12 evaluates the overlap in the X direction and the Y direction with respect to the character components temporarily divided in step S107, and combines them if the degree of overlap is equal to or greater than a threshold value. If the result is larger than the estimated character size, provisional division correction is performed to cancel the combination (similar to S109 described above).
[0054]
FIG. 13 is an explanatory diagram showing evaluation of overlap.
Two methods (overlapping evaluation 1 and 2) can be considered for this overlapping evaluation.
In the overlap evaluation 1, if either the X direction or the Y direction overlaps, it is determined as an overlap (X> th1, Y> th1). That is, when a horizontally written character string is considered as an example, it is determined that the character strings do not overlap in the horizontal direction but overlap in the vertical direction.
[0055]
In overlap evaluation 2, when the overlap area is equal to or greater than the threshold value (th2), it is determined that the overlap is present (X * Y> th2, where the threshold is th1 ≠ th2). That is, here, the determination is based on the area of the overlap in the horizontal direction and the overlap in the vertical direction.
[0056]
Thereby, in the above-described character cutout method that does not consider the writing direction, simple and high-speed processing can be performed depending on the state of the character string by performing the writing estimation step corresponding to the simplified step S105.
[0057]
(3) Discussion of experiments and experimental results
Here, a recognition experiment using online character pattern data was performed using a system to which the character cutting method based on the writing direction estimation according to the present invention was applied. The outline of the experiment, the experimental results, and the discussion of the experiment are described below.
[0058]
(Summary of experiment)
In this experiment, the performance of recognizing the free handwritten character string for the data in each writing direction in the system using the character cutting method independent of the writing direction according to the present invention was examined. Since there is no standard database currently written without a frame, online character pattern data written by five writers was used as experimental data.
Table 1 shows the details of the online character pattern data.
[0059]
[Table 1]

[0060]
The on-line character pattern data, which is experimental data, has five writers. When writing data, example sentences were given to the five writers. The writer wrote this example sentence by handwriting in three types of patterns, horizontal writing, vertical writing, and free writing direction (for example, diagonal direction), and collected these data. The online character pattern data has a total number of 3840 characters, and the number of written characters per person is 786 characters. This online character pattern data consists of 10 pages (256 characters) per set, and is an average of about 25 characters per page. Therefore, the writer has 3 sets (30 pages, 786 characters) each. Handwritten input of the characters.
[0061]
In this experiment, since the character string is divided for each page, the correct recognition rate for each page is calculated, and the average is obtained for each set.
[0062]
(results of the experiment)
FIG. 14 is a diagram showing experimental online frameless character pattern data.
The online character pattern data here includes, for example, “even if it succeeds in the short term” by free writing (oblique direction) by the writer and “fails in the long term” by horizontal writing.
[0063]
FIG. 15 is a diagram (1) illustrating an example of correct recognition.
Here, “even if it succeeds in the short term” by free writing included in the online character pattern data and “fails in the long term” by horizontal writing are surrounded by a circumscribed rectangle for each character element, Correct character recognition is being performed. The recognition result is "Even if it grows in the short term, it fails in the long term."
[0064]
FIG. 16 is a diagram (2) illustrating an example of correct recognition.
The on-line character pattern data here includes, for example, “entrance fee is elementary school student” “more than 590 yen” written vertically by the writer, and “less than elementary school students free” written horizontally. In addition, “entrance fee for elementary school students” “more than 590 yen” in vertical writing included in the online character pattern data and “less than elementary school students free” in horizontal writing are enclosed in a circumscribed rectangle for each character element. Correct character recognition is being performed. The recognition result is “Admission fee is 590 yen for elementary school students and above, free for elementary school students and below”.
Table 2 shows the recognition rate (%) by writing direction.
[0065]
[Table 2]

[0066]
Here, the average of the recognition rate (%) according to the writing direction in the page for each data set is shown for each writer A to E. Regarding the recognition time, the in-page average recognition time was about 2 seconds, and the average recognition time per character was about 0.09 seconds.
[0067]
(Consideration of experimental results)
As a consideration on the experimental results described above, since the character spacing is narrow for horizontally written character strings, the poor cutting accuracy has affected the recognition rate. In vertical writing, the character spacing tends to be wider than in horizontal writing, and the cutting accuracy and recognition rate are high. Since the free-written character string has both horizontal writing and vertical writing characteristics, the result is between them.
[0068]
Thus, by changing the threshold value at the time of provisional division between horizontal writing and vertical writing, it is possible to realize cutting with higher accuracy. Moreover, in the character string with the fine writing direction change mentioned above, instead of extracting a writing direction change point, it can cut out by cutting off with the estimated character size at the time of temporary division.
[0069]
Further, in the present invention, free handwritten character string recognition using a character cutting method that does not depend on the writing direction is performed in accordance with the free handwritten character string without recognizing the performance in the existing system. Can do. Furthermore, it can be expected that the threshold value at the time of division is dynamically set to improve the cutout accuracy and cope with a finer change in writing direction. In the above recognition experiment, an appropriate large-scale online frameless character pattern database can be collected, and further performance evaluation of the character recognition system according to the present invention can be performed.
[0070]
(4) Third embodiment
Next, a frameless character recognition method with a higher degree of freedom that considers not only the writing direction of the character string but also the character direction will be described. Note that portions overlapping with the first and second embodiments described above are omitted to the extent that the description is not redundant, and the newly defined concept will be described in detail. The schematic configuration of the character recognition system is the same as that of the first embodiment.
[0071]
FIG. 18 is an explanatory diagram of handwriting point coordinates and strokes.
The stroke refers to, for example, a series of contact coordinates (writing point coordinates) sampled using the input / output unit 4 while the user 2 is in contact with a non-contact with respect to an appropriate writing surface. Here, as shown in the figure, the description has been given using one stroke included in “A”.
[0072]
FIG. 19 is an explanatory diagram of the character direction and the writing direction.
The character direction refers to, for example, the direction from the bottom of the character toward the top, while the writing direction refers to the direction in which the character string is written. Further, as described above, each character string separated by a line feed, a writing direction change point, a space of a certain interval or more is defined as a character string component. Each character string component has a writing direction. Here, the character direction and the writing direction of “today's menu”, “parent and child bowl”, “una bowl”, “tengu”, and “cutlet bowl”, which are character string components, are shown as dotted and solid arrows, respectively.
[0073]
FIG. 20 is a flowchart of the character recognition method according to this embodiment. For convenience of explanation, the explanation of the flowchart here is schematic, and the specific explanation will be described later. In this character recognition method, for example, a character recognition independent of the character direction and the writing direction is performed on a handwritten stroke sequence input in a free direction without a frame by the user 2.
After the input character string pattern input unit 5 inputs a stroke sequence input in any direction without a frame by the user 2 to the recognition control unit 10, each input stroke sequence is input by the character size estimation processing unit 11. The character size is estimated (character size estimation process: S201). Note that the character size estimated in step S201 is used, for example, for line feed detection, and also serves as an element for evaluation in provisional division processing and character recognition processing.
[0074]
After estimating the character size in step S201, the character size estimation processing unit 11 changes the writing direction, a line break, a predetermined interval separating lines from each other, or a change in writing direction based on the estimated character size and stroke interval (for example, the distance between the centers of gravity) One of the points is detected and divided into character string components (character string component dividing process: S203). Note that the estimation of the character size, the detection of line breaks, and the division of the character string constituent elements are the same as the processes in the above-described steps S103 and S105.
[0075]
Next, the character size estimation processing unit 11 considers that the character size fluctuates for each character string component obtained in step S203, and sets an appropriate (average) character size for each character string component. Re-estimate in detail (character size detail estimation processing: S205). The detailed character size estimation process here is the same as the character size estimation process in step S201, but by applying it to each character string component, the accuracy of the character size estimation can be improved. .
[0076]
Next, for example, for each character string component, the character cutout processing unit 12 estimates the character direction from the variation information of the pen point coordinates in the stroke excluding the variation between strokes (character direction estimation processing: S207). In addition, the character cutout processing unit 12 assumes a character direction that often appears from the writing direction of the character string component (character direction assumption processing: S209). As the character direction assumed in step S209, for example, a positive / negative direction of the writing direction, two directions perpendicular to the positive / negative direction, a vertical direction of the input device (or writing surface), and a predetermined direction among the horizontal direction Any one or more of these are assumed. Here, the process for obtaining the character direction may be either step S207 or S209, or both. It should be noted that the recognition accuracy is higher when both of these steps S207 and S209 are performed and the one with the highest score is selected in the optimum character direction selection processing in step S221 described later.
[0077]
The processes in steps S211 to S219 are performed for each character direction corrected character string component, based on one or a plurality of character direction candidates estimated and / or assumed as described above. For each character direction corrected character string component, the character direction candidates and intermediate results obtained in each step are written to or read from an appropriate memory by the character cutout processing unit 12.
In addition, the character cutout processing unit 12 generates original data as internal data so that the character direction is upward for each of one or more candidate character directions estimated in step S207 and / or assumed in step S209. Are rotated (recognition character direction correction rotation processing: S211). In addition, since the character string component is represented by a series of handwriting coordinates as described above, the rotation processing at an arbitrary angle can be performed easily and at high speed. Here, each rotated pattern (character string component) is referred to as a character direction corrected character string component.
[0078]
Next, the character cutout processing unit 12 quantizes the writing direction for each character direction corrected character string component obtained in step S211 in at least four directions (up, down, left, and right) (writing direction quantization process: S213). In addition, if it is 8 directions, the diagonal direction can also be included. Here, of the processes after step S213, the provisional division process (S215) and provisional division correction process (S217) performed by the character cutout processing unit 12 and the context process performed by the maximum likelihood character string search processing unit 13 are included. The recognition processing (S219) is the same as the above-described provisional division processing in step S107, provisional division correction processing in step S109, and processing for outputting the recognition result in step S111.
[0079]
Next, the maximum likelihood character string search processing unit 13 obtains a plurality of character directions for each character direction obtained by the processing from step S211 to step S219 for each character direction estimated or assumed in steps S207 and S209. Among the recognition results for the corrected character string component, the character direction of the score related to the highest probability of recognition and the character direction corrected character string component in that character direction are selected (optimum character direction selection processing: S221). Here, the maximum likelihood character string search processing unit 13 performs character recognition assuming the context as necessary for the writing direction of the character direction corrected character string component in each character direction. A score indicating the probability of recognition is obtained by appropriately adding any one or more of the certainty, the character division probability, the character size probability, and the context probability. The maximum likelihood character string search processing unit 13 selects a character direction having a high score as the optimum character direction. If only one character direction is estimated / assumed in steps S207 and S209, step S221 is omitted. Further, the maximum likelihood character string search processing unit 13 is written based on the information before rotation of the character direction corrected character string component selected in step S221 and information such as the rotation angle and the rotation center stored in the memory. The writing direction, character direction, character size, character pitch, etc. of the stroke sequence are restored, and the restoration result is reflected as a recognition result on the display on the screen of the application 3 via the recognition result output unit 6 (recognition). Result shaping display processing: S223).
[0080]
Hereinafter, the process of each step of the above-described flowchart will be described in detail. It should be noted that the processing that overlaps the first and second embodiments described above is omitted to the extent that it does not become redundant.
(Step S201: Character size estimation process)
For example, the character size estimation processing unit 11 estimates the character size of each character element from the stroke sequence entered on the screen by the input character string pattern input unit 5. Here, the estimated character size is used for line feed and character spacing detection, provisional division processing, and recognition processing. In this character size estimation process, the character size is estimated from the stroke size for all the strokes on the screen. The character size estimation process is the same as the character size estimation process in step S103 described above (see FIG. 3 and the description thereof).
[0081]
(Step S203: Character string component dividing process)
FIG. 21 is an explanatory diagram of the character string component dividing process.
The character size estimation processing unit 11 performs line feed detection and character string interval detection using clustering based on the estimated character size obtained in the character size estimation process of step S203 (line feed / character string detection: S2031). ). Note that the processing here is the same as the writing direction estimation (writing estimation) in step S105 described above (see FIG. 4 and the description thereof). Further, the character size estimation processing unit 11 divides the stroke string on the screen into character string components as shown in FIG. 19 based on the line feed / character string interval detected in step S2031 (S2032). Note that the processing here is the same as the line feed detection processing included in the writing direction estimation (writing estimation) in step S105 described above (see FIG. 5 and the description thereof).
[0082]
Next, the character size estimation processing unit 11 detects a writing direction change point (S2033). Here, in the line division in step S2032, as shown in FIG. 19, in the itemized list or the like, when the horizontal writing is followed by the vertical writing line, or a screen obstructed by an appropriate figure or the like If the writing direction is changed and the writing is continued, it is difficult to estimate the writing direction because the line cannot be divided only by detecting the line break or the character string interval. Therefore, by identifying a location where the writing direction is changing abruptly in the middle (the above-described writing direction change point) in step S2033, it is possible to accurately divide the character string and estimate the writing direction.
[0083]
The detection of the writing direction change point can be realized by obtaining the farthest point from the straight line connecting the start point and the end point of the target character string as described above. Moreover, by using this method recursively, a plurality of writing direction changes can be handled. With these writing direction change points, the character string component is further divided into shorter character string components. The processing here is the same as the processing included in the writing direction estimation (writing estimation) in step S105 described above (see FIGS. 6, 7, and 8 and the description thereof). Next, the character size estimation processing unit 11 obtains the writing direction from the start point coordinates of the first stroke and the end point coordinates of the final stroke for each character string component (S2034).
[0084]
(Step S205: Character size detail estimation process)
FIG. 22 is an explanatory diagram of the character size detail estimation process.
As shown in FIG. 22, the character size estimation processing unit 11 re-estimates the character size in detail for each character string component in consideration of the variation in character size for each character string component. The processing here is the same as the character size estimation processing described above, but the accuracy of character size estimation is improved by applying it to each character string component. For example, the character size of the character string component “Today's menu” is larger than the character size of the other character string components. Specifically, it can be obtained by an appropriate method such as the average character size for each character element obtained in step S203, the average excluding the maximum value / minimum value, or the like.
[0085]
By inserting this character size detail estimation process after step S203, the character size can be estimated more accurately even when there are character string components having different character sizes on the same screen. become. As a result, in the character cutout, which is the preprocessing for character string recognition in the character cutout processing unit 12, high-precision cutout can be performed based on the estimated character size. However, the processing in step S205 may be omitted if it can be assumed that the character size on the screen is constant or if it has already been estimated.
[0086]
(Step S207: Character direction estimation process)
FIG. 23 is an explanatory diagram of a cumulative histogram between inter-stroke points according to character direction and direction.
Here, the character direction will be described. When the character is entered, the variation of the handwriting coordinates excluding the variation between strokes is noticeable in the rightward or downward direction when the bottom of the character is taken down. This is because Japanese (especially kanji) contains many strokes written in the right direction or downward direction. For this reason, when fluctuations are taken in the histogram for each direction, for example, in the case of a horizontally written character string, two peaks appear at a position about 90 ° apart. Here, as the fluctuation of the handwriting coordinates of three character strings “Agricultural and Technological University” each having a different character direction, a peak appears at a portion approximately 90 ° apart.
[0087]
Therefore, when viewed from the entire character string, the direction of the character can be estimated by detecting the variation of the handwriting coordinates. If the character direction can be estimated, the normal recognition process can be performed by rotating the entire character string so that the character direction is the positive direction (that is, upward).
FIG. 24 is an explanatory diagram of peak detection in the direction-to-brush-point distance cumulative histogram.
In order to estimate the character direction, the character cutout processing unit 12 takes a histogram for accumulating the distance between the two strokes for each successive stroke in the stroke. The histogram here is accumulated, for example, in 16 direction divisions, and peaks can be found in two directions that are about 90 ° apart from each other in the 16 directions. The character direction can be specified by determining how much the peak in these two directions is rotating from the downward direction and the right direction. The degree of division can be set to an appropriate value such as 8 directions, 4 directions, or 32 directions.
[0088]
Here, a case is shown in which the character direction of the character string component “a” that is the input pattern is estimated. First, the character cutout processing unit 12 takes a histogram of handwriting coordinate fluctuations for each character string component. (Step 1), the peak of this histogram is detected (Step 2). Next, the character cutout processing unit 12 detects the angle between the peaks of two or more peaks in the detected histogram (step 3), where a certain peak is used as a reference and the position is about 90 ° counterclockwise. When another peak appears, the direction opposite to the reference peak is the character direction (dotted arrow in step 4 in the figure).
[0089]
FIG. 25 is a flowchart showing peak detection.
First, the character cutout processing unit 12 obtains a histogram for each direction of handwriting variation (S2071), and detects a peak serving as one reference (S2072). The character extraction processing unit 12 scans a peak counterclockwise from the reference peak detected in step S2072 (S2073), and determines whether another peak can be detected around 90 ° from the reference peak. Determination is made (S2074). If another peak can be detected in step S2074, the character cutout processing unit 12 sets the reverse direction of the reference peak as the character direction (S2076). On the other hand, if the other peak cannot be detected, the character extraction processing unit 12 Is detected (S2075), and the process returns to step S2073 to scan the peak again.
[0090]
Further, as another method for specifying the character direction by using peak detection, for example, the intensity of the histogram in the direction of the angle θ is f (θ), and f (θ) * f (θ + π / 2) The peak may be searched and the direction opposite to the angle θ may be the character direction. More specifically, the so-called Gaussian function g (δ) = exp (−δ ² / Σ ² ) Is convolved with the above-mentioned f (θ) * f (θ + π / 2), and even when the angle difference between the downward direction and the right direction of the character is slightly deviated from π / 2, such as a character that rises to the right, Two peaks can be extracted stably.
[0091]
FIG. 26 is an explanatory diagram of peak detection when the character direction is only up and down.
Here, a description will be given of a case where it is known that the character direction is only downward or upward. In the character cutout processing unit 12, for example, the character direction is estimated from fluctuations in the Y-coordinate (that is, the vertical direction) pen point coordinates. become. Here, since the writing direction is limited to the vertical direction, it is only necessary to determine whether the writing direction is the screen upward direction or the screen downward direction as shown in FIG. In this case, the character direction can be estimated by adding the difference in the Y direction between the writing points regardless of horizontal writing or vertical writing. That is, the character cutout processing unit 12 sets the coordinate system of the screen to the left-right direction on the X axis and the vertical direction on the Y axis, so that the character string component (here, “Today's menu”) If the sum of the Y-coordinate fluctuations is a “negative” value, the character direction can be determined as an upward direction, while if it is a “positive” value, it can be determined as a downward direction.
[0092]
(Step S209: Character direction assumption process)
FIG. 27 is an explanatory diagram of a case where a plurality of character directions are assumed from the writing direction. In the character direction estimation process, it is assumed that the character direction cannot be estimated correctly if the number of characters in the character string component is small. In order to prevent this, one of the purposes of the character direction assumption process is to assume a character direction that appears frequently and add it to a recognition candidate.
[0093]
In the processing here, for example, as shown in the figure, the character direction (dotted arrow in the figure) that often appears from the writing direction (solid arrow in the figure) of the character string component (here, “Today's menu”) ) Is assumed. As the character direction, the positive and negative directions of the writing direction, two directions perpendicular to the writing direction, and the vertical direction and the horizontal direction of the input device (or writing surface) as shown in FIG. This is not the case (here, the direction of writing is assumed to be the top / bottom / left / right direction of the writing surface). In addition, when narrowing down by estimating the character direction, the number of candidates for these assumptions can be reduced or omitted. The character direction assumption process can be omitted when the character direction is limited to the vertical direction with respect to the display screen as shown in FIG.
[0094]
(Step S211: Recognizing character direction correction rotation process)
FIG. 28 is an explanatory diagram of the character direction correction rotation process for recognition.
For example, the character cutout processing unit 12 stores the character string components in the internal data for each character direction estimated by the character direction estimation process in step S207 and for each character direction assumed by the character method assumption process in step S209. Are copied to an appropriate internal memory and rotated so that the character direction is upward. Here, by rotating the character string component “Today's menu” displayed on the display screen in the figure with the character direction substantially rightward, the character direction corrected character string component with the character direction upward is obtained. ing. Note that the character cutout processing unit 12 uses the character direction corrected character string component, which is the rotated character string component, in the shaping display process of the recognition result in step S223. The rotation center and rotation angle are held in an appropriate internal memory. The rotation angle for each character direction corrected character string component is also applied to each character included in the character direction corrected character string component.
[0095]
(Step S213: Writing direction quantization process)
FIG. 29 is a flowchart showing the writing direction quantization process.
Here, the character cutout processing unit 12 obtains the writing direction from the start point coordinates of the first stroke and the end point coordinates of the final stroke for each character direction corrected character string component obtained in step S211, and calculates it to a fixed quantization. Divide into sections. The process here is a process in which the estimation of the writing direction shown in FIG. 9 is further refined and is quantized in at least four directions (up, down, left and right). The horizontal right direction is the positive direction of X, and the vertical downward direction is the positive direction of Y.
[0096]
First, the character cutout processing unit 12 determines whether or not the aspect ratio between the vertical Y and the horizontal X of the circumscribed rectangle of the character direction correction character string component is X> Y (S2131), and X> Y must be satisfied. For example, it is determined whether or not the difference between the Y coordinate (start point Y coordinate−end point Y coordinate) between the start point and the end point in the character direction corrected character string component is greater than 0 (S2132). Next, in step S2132, the character cutout processing unit 12 determines that the writing direction is upward when the Y coordinate difference is greater than 0, and determines that the writing direction is downward when the Y coordinate difference is 0 or less. Then, the writing direction as the respective determination results is added as stroke internal information for each character direction corrected character string component (S2133).
[0097]
On the other hand, if X> Y in step S2131, the character cutout processing unit 12 determines whether the difference between the X coordinates of the start point and the end point in the character direction corrected character string component is larger than 0 (S2134). ). Next, when the X coordinate difference (start point X coordinate-end point X coordinate) is greater than 0 in step S2134, the character cutout processing unit 12 determines that the writing direction is leftward, while the X coordinate difference is 0 or less. If there is, the writing direction is determined to be rightward, and the writing direction as the determination result is added as stroke internal information for each character direction corrected character string component (S2133).
[0098]
Thus, performing writing direction quantization in at least four directions is to prevent unnatural overdivision. Specifically, for example, in the case of horizontal writing (and right writing), the left stroke in that is often not a character break, so the previous strokes are combined (combined). As a result, fine over-division is reduced and the recognition rate can be improved.
[0099]
If quantization is performed more finely in four directions or more, overdivision can be more stably prevented with respect to a character string written obliquely. However, in reality, four directions are almost sufficient. Let us consider a case where the writing direction is a diagonally lower right direction. In this case, the characters can be collected by brushing in the upper left diagonal direction. Consider the case where the writing direction to the lower right diagonal is determined to be the right direction (a) and the case where the writing direction is determined to be the downward direction (b) by quantizing the writing direction into four directions. When it is determined that the right direction is (a), if the stroke in the upper left diagonal direction is determined to be the left direction, it is used as information for grouping characters. On the other hand, if this stroke is determined to be in the upward direction, it is not used as information for grouping the characters, but the character string is almost the same as the recognition process with provisional division correction processing and context processing. Dividing and recognizing correctly. When it is determined that the writing direction of (a) is a downward direction, if it is determined that a stroke in the upper left diagonal direction is an upward direction, it is used as information for grouping characters. On the other hand, if this stroke is determined to be in the left direction, it is not used as information for grouping the characters, but the character string is almost the same as the recognition process with provisional division correction processing and context processing. Dividing and recognizing correctly. The writing direction in the lower right diagonal direction has been taken up, but the same applies to the lower left diagonal direction, the upper right diagonal direction, and the upper left diagonal direction.
[0100]
(Step S215: Temporary division processing)
Here, for example, for each character string constituent element delimited by a line feed or a character string interval, the character cutout processing unit 12 determines the character size, the distance between the centers of gravity between the strokes (projection distance between the centers of gravity), the degree of overlapping of the strokes, and the like. Considering this, determination of division candidate points as characters and determination of the combination of strokes are performed. This provisional division process is the same as the provisional division process in step S107 described above (see FIGS. 10 and 11 and the description thereof).
[0101]
(Step S217: Temporary division correction process)
The temporary division correction process here is also the same as the temporary division correction process in step S109 described above (see FIG. 12 and the description thereof), similarly to the temporary division process in step S215. In other words, the provisional division correction process considers character candidates generated by the provisional division process in step S215 in terms of character size, distance between centroids between character elements (projection distance between centroids), overlap between character elements, and the like. In this process, the character elements generated by the temporary division are further evaluated and the elements are combined.
[0102]
(Step S219: Recognition processing with context processing)
The recognition processing with context processing here is the same as the processing in step S111 described above, and the maximum likelihood character string search unit 15 integrates the character size, context, recognition result, and the like into dynamic programming. Is a process for obtaining a final recognition result. Note that the process from the recognition character direction correction rotation process in step S211 to the recognition process with context process in step S219 is applied to a plurality of character direction correction character string components as described above. Become.
[0103]
(Step S221: Optimal Character Direction Selection Process)
Next, the maximum likelihood character string search processing unit 13 includes a plurality of character direction corrected character string components obtained by the processing from step S211 to step S219 for each character direction estimated or assumed in steps S207 and S209. A recognition score is obtained as a result of the recognition process for each of. Then, the maximum likelihood character string search processing unit 13 selects the character direction with the highest score among the obtained recognition scores and the character direction corrected character string component in the character direction.
[0104]
(Step S223: Recognition display processing of recognition result)
FIG. 30 is a diagram (1) illustrating a recognition result in a previous stage of the recognition result shaping display process. Here, an example of the recognition result from the above-mentioned steps S201 to S221 is shown in the figure.
First, the character string components before recognition (two “Agricultural and Technical University”) are recognized by rotating the rotation so that the character direction is centered on the center of gravity of each character string component by the above-described processes. Has been. This rotation-corrected character string component is displayed as a character direction corrected character string component in the upper window (writing surface) after recognition in FIG. The recognition result of the character direction corrected character string component (two “Agricultural and Technological University”) is displayed in the lower right window of the writing surface after recognition in FIG.
[0105]
FIG. 31 is a diagram (2) illustrating a recognition result in a previous stage of the recognition result shaping display process.
First, the character string components before recognition (four “Agricultural and Technological University”) are recognized by rotation correction so that the character direction is upward with the center of gravity of each character string component as the center by the above-described processes. Has been. This rotation-corrected character string component is displayed as a character direction corrected character string component in the upper window (writing surface) after recognition in FIG. Further, the recognition result of the character direction corrected character string component (four “Agricultural and Technological University”) is displayed in the lower right window of the writing surface after the recognition in FIG. In addition, the display of the recognition result of the character string component “Agricultural and Technological University” written in the reverse direction from the top on the writing surface before recognition in FIG. 31 is inclined on the writing surface after recognition. Indicates that there is such an error. However, since the recognition engine unit 14 of the maximum likelihood character string search processing unit 13 can generally cope with this degree of character direction fluctuation, the code string of the recognition result is in the lower right window of the writing surface after recognition in FIG. As you can see, you can get the correct results.
[0106]
Next, the recognition result shaping display process that has undergone the previous processing of the recognition result shaping display process will be described. As the recognition result shaping display process, for example, there are a case of rotating for each character direction corrected character string component and a case of rotating for each character included in the character direction corrected character string component.
FIG. 32 is a flowchart (1) of the recognition result shaping display process. The processing here is, for example, processing for displaying the recognition result in a layout or situation close to the first handwritten character string using the recognition result described above, and for each character string component whose writing direction is estimated. The character font string as the recognition result is arranged and displayed according to the writing direction, character direction, average character size, and average character spacing.
[0107]
The maximum likelihood character string search processing unit 13 obtains the average size of handwritten characters for each character direction corrected character string component (S2231), and determines the number of output character points (font size) close to this average size (S2232). ), Average vertical and horizontal intervals between handwritten characters are obtained (S2233). Next, the maximum likelihood character string search processing unit 13 arranges the character font string in the internal buffer according to the writing direction with the number of output points in step S2232 and the average interval in step S2233 according to the character code string (S2234). ). Further, in step S2234, the maximum likelihood character string search processing unit 13 displays the character font string arranged in the internal buffer by rotating it reversely by the rotation angle at the rotation center (S2235). That is, the recognition display processing of the recognition result here realizes the display by arranging the font pattern string of the character string component in the internal buffer and rotating it.
[0108]
In addition, as a display method of the character font in step S2235, it can arrange | position near the original handwritten character, and can also be overwritten on a written character. Note that the character size in step S2231, the character position associated with the average interval in step S2233, and the character code used in step S2234 are recognized as shown in the writing surface after recognition in FIGS. It is determined by the process before the shaping display process or the process before that.
[0109]
FIG. 33 is an explanatory diagram of a case where the characters are rotated for each character in the recognition result shaping display processing. FIG. 34 is a flowchart (2) of the recognition result shaping display process. The processing here calculates the base point of the font display for each character (here, the character font) included in the character direction correction character string component on the formatted display surface of the sentence recognition result, and the character font is used as the base point. Rotate to position.
[0110]
The maximum likelihood character string search processing unit 13 obtains an average size of handwritten characters for each character direction corrected character string component (S2231), determines the number of output character points close to this average size (S2232), and between handwritten characters. Average intervals in the vertical and horizontal directions are obtained (S2233). Next, the maximum likelihood character string search processing unit 13 arranges the i-th character font according to the writing direction with the number of output points in step S2232 and the average interval in step S2233 according to the character code string. Base point (X _i , Y _i ) Is calculated using the rotation center as the origin (S2236). Further, the maximum likelihood character string search processing unit 13 obtains a base point for font display on the shaped display surface of the recognition result according to the following formula 1 (S2237).
[0111]
[Expression 1]

[0112]
However, (theta) is a rotation angle recorded on this character direction correction | amendment character string component, and applying this for every character. Also, (X _C , Y _C ) Is the center of rotation on the display screen.
The maximum likelihood character string search processing unit 13 rotates the font by −θ for the i-th font, and is obtained in step S2237 (X _i ', Y _i ') Is placed at the base point (S2238). Here, as shown in FIG. 33, the first font “Agriculture”, the second font “Engineering”, and the third font “Large” included in the character direction correction character string component “Agricultural Engineering University” are The recognition result close to the first handwritten character string is displayed by rotating and arranging each by θ. In other words, the recognition display processing of the recognition result here realizes the display by calculating the character position after the formatting display for each character and rotating the font so as to match it.
[0113]
In the following, the recognition result shaping display shown by performing the recognition result shaping display processing shown in FIGS.
FIG. 35 is a diagram (1) illustrating the shaped display of the recognition result. In this example, the character font is replaced with handwritten characters and displayed. In addition, the left side in the figure shows the handwritten input, and the right side in the figure shows the output after the shaping display for the handwritten input.
Here, each character string component “today's menu”, “cutlet bowl”, “tengu”, “una bowl”, “parent and child bowl” entered with handwriting and different in character size, character position, writing direction and character direction is displayed after formatting. The recognition result close to the first handwritten character string is displayed.
[0114]
FIG. 36 is a diagram (2) illustrating the shaped display of the recognition result. Here, the figure corresponding to the figure showing the handwritten input in FIG. 36 is the figure before the recognition in FIG.
Here, each character string component (in this case, four “Agricultural and Technological University”) input by handwriting, which has different character size, character position, writing direction, and character direction, is displayed in the first handwritten character string after being shaped and displayed. The near recognition result is displayed. Note that the recognition result display of “Agricultural and Technological University” written in the reverse direction from the top in the writing surface before recognition shown in FIG. 31 is tilted in the writing surface after recognition. An error has occurred in the direction estimation, and the character font is displayed with a slight inclination in the recognition display of the recognition result in FIG. 36 in which the character direction is restored.
[0115]
As described above, according to the present embodiment, a character input means that enables more natural character input that replaces (or coexists with) a keyboard, a character input method and a pen interface for a person who is not good at a mouse or a keyboard. By using, it is possible to provide a natural character input method that does not interfere with thinking, and it has been difficult until now by performing recognition independent of the writing direction of the character and the character direction the character is facing, Character strings written in any direction or in any direction can be recognized.
[0116]
The character recognition method of the present invention includes a character recognition program, a computer-readable recording medium that records the character recognition program, a program product that includes the character recognition program and can be loaded into the internal memory of the computer, and a computer that includes the character recognition program. It can be provided by a program product or the like stored in a usable recording medium.
[0117]
【The invention's effect】
According to the present invention, as described above, frameless character recognition (free handwritten character string recognition) independent of the writing direction can be performed with a higher degree of freedom for characters input by handwriting without a frame.
[0118]
Further, according to the present invention, for each character string delimited by line breaks, the writing direction can be estimated based on the geometric characteristics of the character string, and provisional division according to the writing direction can be performed at the time of character extraction. Further, the present invention can be applied to an existing recognition system, for example, to perform frameless character recognition and free written character string recognition independent of the writing direction.
[0119]
Further, the present invention can comprehensively determine, for example, the accuracy of character extraction, the accuracy of character recognition, the accuracy of character size, and the accuracy of character concatenation. Further, the present invention can perform frameless character recognition that does not depend on, for example, the writing direction, the character direction, and the character size variation for each line.
[Brief description of the drawings]
FIG. 1 is a schematic configuration diagram of an on-line frameless character recognition system to which a frameless character recognition device independent of the writing direction according to the present invention is applied.
FIG. 2 is a flowchart of a character recognition method according to the present invention.
FIG. 3 is an explanatory diagram of character size estimation processing.
FIG. 4 is an explanatory diagram showing a transition distribution between character elements.
FIG. 5 is a diagram showing a character string in which horizontal writing and vertical writing are mixed.
FIG. 6 is an explanatory diagram showing detection of a writing direction change point.
FIG. 7 is an explanatory diagram of a case where the farthest point is obtained recursively.
FIG. 8 is an explanatory diagram of a case where the writing direction change is minute.
FIG. 9 is an explanatory diagram of writing direction estimation.
FIG. 10 is an explanatory diagram showing provisional division for each writing direction.
FIG. 11 is an explanatory diagram of evaluation at the time of division.
FIG. 12 is an explanatory diagram of provisional division correction.
FIG. 13 is an explanatory diagram showing evaluation of overlap.
FIG. 14 is a diagram showing experimental online frameless character pattern data.
FIG. 15 shows an example of correct recognition (1).
FIG. 16 is a diagram (2) illustrating an example of correct recognition.
FIG. 17 is an explanatory diagram of a display screen when conventional frameless character string recognition and single character recognition are started.
FIG. 18 is an explanatory diagram of handwriting point coordinates and strokes.
FIG. 19 is an explanatory diagram of a character direction and a writing direction.
FIG. 20 is a flowchart of a character recognition method according to the present embodiment.
FIG. 21 is an explanatory diagram of character string component dividing processing.
FIG. 22 is an explanatory diagram of a character size detail estimation process.
FIG. 23 is an explanatory diagram of a cumulative histogram of inter-stroke distances according to character direction and direction.
FIG. 24 is an explanatory diagram regarding detection of a peak in a cumulative inter-stroke point distance histogram.
FIG. 25 is a flowchart showing peak detection.
FIG. 26 is an explanatory diagram of peak detection when the character direction is only up and down.
FIG. 27 is an explanatory diagram of a case where a plurality of character directions are assumed from the writing direction.
FIG. 28 is an explanatory diagram of character direction correction rotation processing for recognition.
FIG. 29 is a flowchart showing a writing direction quantization process;
FIG. 30 is a diagram (1) showing a recognition result in a previous stage of recognition result shaping display processing;
FIG. 31 is a diagram (2) showing a recognition result in a previous stage of a recognition result shaping display process;
FIG. 32 is a flowchart (1) of a recognition result shaping display process;
FIG. 33 is an explanatory diagram in the case of rotating for each character in the recognition result shaping display processing;
FIG. 34 is a flowchart (2) of the recognition result shaping display process;
FIG. 35 is a diagram (1) showing a shaped display of recognition results.
FIG. 36 is a diagram (2) showing a shaped display of recognition results.
[Explanation of symbols]
1 Character string recognition system
2 users
3 Application
4 Input / output section
5 Input string pattern input part
6 Recognition result output part
10 Recognition control section
11 Character size estimation processing section
12 character extraction processing part
13 Maximum likelihood character string search processing unit
14 Recognition Engine
15 Maximum likelihood character string search unit

Claims (11)

A character string input step for inputting information on each character element of a character string input by handwriting without specifying a writing direction without a frame or a line;
A character size estimating step for estimating a character size of the character string input by the character string input step;
Based on the distance between the character elements included in the character string and the first threshold value corresponding to the estimated character size, the line break situation of the character string is detected and the character string is divided into character string components. , And detecting the farthest point as a change point in the writing direction by obtaining the farthest point separated from the straight line connecting the start point and the end point of the character string by a character size or more with respect to the character string components separated by the line feed, Is further divided at the change point of the writing direction, and the writing direction for each character string component based on the vertical length and horizontal length of the circumscribed rectangle of the character string component Writing direction estimation step for estimating
A provisional division step that provisionally divides a character string to obtain a character component along the writing direction estimated by the writing direction estimation step;
For each character component temporarily divided by the temporary division step , if the overlapping area of the circumscribed rectangle of each character component is equal to or greater than a second threshold value, a temporary division correction step for integrating the character components Including character recognition method. The relative corrected string by the temporary division modification step, performs character recognition, the character recognition method according to claim 1, further comprising a recognition result output step of outputting a recognition result of the character recognition. In the character size estimation step,
The character recognition method according to claim 1 or 2 , wherein a larger value of the width or height of each character element of the character is set as a stroke size, and the character size is estimated from the stroke size. In the writing direction estimation step,
The character recognition method according to any one of claims 1 to 3 , wherein a change point in the writing direction is detected recursively so as to correspond to a character string whose writing direction changes a plurality of times. In the writing direction estimation step,
When estimating the writing direction, if the horizontal length of the circumscribed rectangle of the specified character string component is larger than the vertical length, the writing direction is estimated as the horizontal direction, and the horizontal length must be smaller than the vertical length. character recognition method according to any one of claims 1 to 4 which is adapted to estimate the field writing direction and the vertical direction. In the provisional division step,
Writing direction change of a string is smaller than a predetermined value, and the larger the character size after division than the estimated character size, the character elements in the evaluation to any one of claims 1 to 5 so as to provisionally separated The character recognition method described. In the previous Kikari division step,
Based on the writing direction information of each string component of the character string, between the centers of gravity projection distance character elements each other horizontally or vertically, and / or overlap in the lateral direction or the longitudinal direction of the circumscribed rectangle of the character elements evaluate the tool case, the character recognition method according to any one of claims 1 to 6 to perform the provisional division. A character string input step for inputting information on each character element of a character string input by handwriting without specifying a writing direction without a frame or a line;
A character size estimating step for estimating a character size of the character string input by the character string input step;
Based on the distance between the character elements included in the character string and the first threshold value corresponding to the estimated character size, the line break situation of the character string is detected and the character string is divided into character string components. , And detecting the farthest point as a change point in the writing direction by obtaining the farthest point separated from the straight line connecting the start point and the end point of the character string by a character size or more with respect to the character string components separated by the line feed, Is further divided at the change point of the writing direction, and the writing direction for each character string component based on the vertical length and horizontal length of the circumscribed rectangle of the character string component Writing direction estimation step for estimating
A provisional division step that provisionally divides a character string to obtain a character component along the writing direction estimated by the writing direction estimation step;
For each character component temporarily divided by the temporary division step , if the overlapping area of the circumscribed rectangle of each character component is equal to or greater than a second threshold value, a temporary division correction step for integrating the character components A computer-readable recording medium on which a character recognition program to be executed by a computer is recorded. An input character string pattern input unit for inputting information on each character element of a character string input by handwriting without specifying a writing direction without a frame or a line;
A character size estimation processing unit that estimates a character size of a character string input by the input character string pattern input unit;
A character cutout processing unit for performing the cutout processing of the character string,
The character cutout processing unit
Based on the distance between the character elements included in the character string and the first threshold value corresponding to the estimated character size, the line break situation of the character string is detected and the character string is divided into character string components. , And detecting the farthest point as a change point in the writing direction by obtaining the farthest point separated from the straight line connecting the start point and end point of the character string by a character size or more with respect to the character string components separated by the line feed, Is further divided at the change point of the writing direction, and the writing direction for each character string component based on the vertical length and horizontal length of the circumscribed rectangle of the character string component Writing direction estimation means for estimating
Along said writing direction estimated by the writing direction estimation means, the temporary dividing means for obtaining a character component by provisionally separated strings,
Temporary division correcting means for integrating the character components if the overlapping area of the circumscribed rectangle of each character component is equal to or greater than a second threshold value for each character component temporarily divided by the temporary division means A character recognition device. A recognition engine unit that performs character recognition on the character string corrected by the provisional division correction unit ;
The character recognition device according to claim 9 , further comprising an application for displaying a recognition result by the recognition engine unit. A character string input step for inputting information on each character element of a character string input by handwriting without specifying a writing direction without a frame or a line;
A character size estimating step for estimating a character size of the character string input by the character string input step;
Based on the distance between the character elements included in the character string and the first threshold value corresponding to the estimated character size, the line break situation of the character string is detected and the character string is divided into character string components. and the farthest point is detected as a change point of the writing direction by calculating the farthest point away from the straight line or character size connecting the start and end of the string for separated string elements with a new line, line feed separated further divided by the change point of the writing direction each string components were, and the length of the vertical of the circumscribed rectangles of the character string configuration elements and horizontal writing for every string components based on the length in A writing direction estimation step for estimating a direction;
A provisional division step that provisionally divides a character string to obtain a character component along the writing direction estimated by the writing direction estimation step;
For each character component temporarily divided by the temporary division step , if the overlapping area of the circumscribed rectangle of each character component is equal to or greater than a second threshold value, a temporary division correction step for integrating the character components A character recognition program to be executed by a computer.

Images