JP2004178002A

JP2004178002A - Retouch extraction device, retouch extraction method, program therefor and recording medium

Info

Publication number: JP2004178002A
Application number: JP2002339955A
Authority: JP
Inventors: Hirofumi Horikawa; 裕文堀川; Sadao Takahashi; 禎郎高橋
Original assignee: Ricoh Co Ltd
Current assignee: Ricoh Co Ltd
Priority date: 2002-11-22
Filing date: 2002-11-22
Publication date: 2004-06-24

Abstract

<P>PROBLEM TO BE SOLVED: To provide a retouch extraction device, a retouch extraction method, a program therefor and a recording medium which divide an area to align, and take the difference of widths of character lines of the both images into consideration to perform difference processing, when generating a retouch image from an electronic image and a retouch superposition image, thereby performing retouch extraction with higher precision. <P>SOLUTION: The retouch extraction device has: an area division means which breaks down a manuscript image of an electronic document with handwritten retouch superimposed after the electronic document is outputted on a paper and an image which has been converted into electronic form again with an image reader; an image correlation means which performs correlation calculation for aligning among the images with respect to each area divided with the area division means; and an image difference means which performs image difference processing among the images which have been aligned with the image correlation means, and at the same time performs difference processing among the images which have been displaced. <P>COPYRIGHT: (C)2004,JPO

Description

【０００１】
【発明の属する技術分野】
本発明は、手書き文字入力システムに対する加筆抽出方法に関するものであり加筆抽出および加筆画像作成の抽出精度を向上させる加筆抽出装置、加筆抽出方法、加筆抽出プログラム及び記録媒体に関する。
【０００２】
【従来の技術】
従来、文書画像ファイリング装置に登録された画像データの内容を更新する際には、装置内に取り込まれた画像データに対して直接書き込み処理を行うか、または別に文字データを用意して重ね合わせるなどの方法がある。また、紙ベースで加筆修正した更新画像を読み取り、この更新画像を記録する方法もある。
【０００３】
従来の技術例としては、文書処理装置において、手書きで記入された情報を画像情報として検出する手段の１つとして用いることにより、入力された加筆重畳画像をブロック分割した後、その小領域毎に差分をとり、スキューが発生しているか否かを確認しながら最適な位置合わせを行う文書処理装置がある（例えば、特許文献１参照）。また、画像の中の、外枠の隅などの特徴的な四点を検出し、それぞれの点が電子原稿画像中の対応する点に移動する様に補完処理を行う方法や、一点を対応点へ平行移動しその点を中心に回転させ伸縮などを行うアフィン変換式により補正処理を行う方法などで、位置合わせを行った後に差分処理を行う文書画像ファイリング装置及び文書画像更新方法がある（例えば、特許文献２参照）。また、マーク未記入の原稿画像とマークが記入された加筆重畳画像との差分画像を生成し、差分画像における各チェック領域内の黒画素数および黒画素割合を算出することでマークの有無を判定するマーク認識装置がある（例えば、特許文献３参照）。
【０００４】
【特許文献１】
特開平５−２４４３８７号公報
【特許文献２】
特開２００１−２８３２３１号公報
【特許文献３】
特開平１１−２６５４２２号公報
【０００５】
【発明が解決しようとする課題】
しかしながら、上述の特許文献１から３記載の従来例は、いずれも電子画像と、印刷とスキャンのプロセスを通過する加筆重畳画像とでは印刷文字の太さが異なることを考慮に入れていない。さらに、特許文献１記載の文書処理装置および特許文献３記載のマーク認識装置は、差分処理後のノイズ除去等に関して何ら対策がなされておらず、差分処理後の画像には細かいノイズ成分が残る場合がある。また、特許文献２記載の文書画像ファイリング装置および文書画像更新方法および特許文献３記載のマーク認識装置は、プリント時の紙送りムラ、ドラム回転ムラ、スキャン時における光学系の速度ムラなどによる加筆した画像の局所的な倍率の変動に対応できない。
【０００６】
本発明は上記事情に鑑みてなされたものであり、電子画像と加筆重畳画像から加筆画像を作成する際に、領域を分割して位置合わせをしつつ、両画像の文字線幅の違いを考慮に入れて差分処理を行う事により、より精度の高い加筆抽出を行う加筆抽出装置、加筆抽出方法、加筆抽出プログラム及び記録媒体を提供することを目的としている。
【０００７】
【課題を解決するための手段】
かかる目的を達成するために、請求項１記載の加筆抽出装置は、電子文書の原稿画像と、電子文書を紙へ出力した後に手書きの加筆が重畳した状態で画像読取装置により再び電子化された画像とを細分化する領域分割手段と、領域分割手段によって分割されたそれぞれの領域に対して画像間の位置合わせを行うために相関計算を行う画像相関手段と、画像相関手段によって前記位置合わせが行われた画像間での画像差分処理と同時に、位置からずらした画像間での差分処理を行う画像差分手段とを有することを特徴としている。
【０００８】
請求項２記載の加筆抽出装置は、請求項１記載の加筆抽出装置において、画像相関手段は、計算の対象となる２枚の画像の画素値の総和や画像の面積に依存しない相関値を算出することを特徴としている。
【０００９】
請求項３記載の加筆抽出装置は、請求項１記載の加筆抽出装置において、画像差分手段は、画像相関手段によって位置合わせが行われた画像間の相関値が所定の閾値Ｔｈ_１以上の場合にのみ差分処理を行うことを特徴としている。
【００１０】
請求項４記載の加筆抽出装置は、請求項１記載の加筆抽出装置において、画像差分手段は、画像相関手段によって位置合わせが行われ、その位置からわずかにずらした画像間の相関値が所定の閾値Ｔｈ_２以上の場合にのみ差分処理を行うことを特徴としている。
【００１１】
請求項５記載の加筆抽出方法は、電子文書の原稿画像と、電子文書を紙へ出力した後に手書きの加筆が重畳した状態で画像読取装置により再び電子化された画像とを細分化する領域分割ステップと、領域分割ステップによって分割されたそれぞれの領域に対して画像間の位置合わせを行うために相関計算を行う画像相関ステップと、画像相関ステップによって位置合わせが行われた画像間での画像差分処理と同時に、位置合わせの位置からわずかにずらした画像間での差分処理を行う画像差分ステップとを有することを特徴としている。
【００１２】
請求項６記載の加筆抽出方法は、請求項５記載の加筆抽出方法において、画像相関ステップは、計算の対象となる２枚の画像の画素値の総和や画像の面積に依存しない相関値を算出することを特徴としている。
【００１３】
請求項７記載の加筆抽出方法は、請求項５記載の加筆抽出方法において、画像差分ステップは、前記画像相関ステップによって位置合わせが行われた画像間の相関値が所定の閾値Ｔｈ_１以上の場合にのみ差分処理を行うことを特徴としている。
【００１４】
請求項８記載の加筆抽出方法は、請求項５記載の加筆抽出方法において、画像差分ステップは、画像相関ステップによって位置合わせが行われ、その位置からわずかにずらした画像間の相関値が所定の閾値Ｔｈ_２以上の場合にのみ差分処理をすることを特徴としている。
【００１５】
請求項９記載のプログラムは、電子文書を紙へ出力した後に手書きの加筆が重畳した状態で前記電子文書の原稿画像と、画像読取装置で再び電子化された画像とを細分化する領域分割処理と、領域分割処理によって分割されたそれぞれの領域に対して画像間の位置合わせを行うために相関計算を行う画像相関処理と、画像相関処理によって位置合わせが行われた画像間での画像差分処理と同時に、位置からずらした画像間での差分処理を行う画像差分処理と、をコンピュータに実行させることを特徴としている。
【００１６】
請求項１０記載の記録媒体は、電子文書の原稿画像と、電子文書を紙へ出力した後に手書きの加筆が重畳した状態で画像読取装置により再び電子化された画像とを細分化する領域分割処理と、領域分割処理によって分割されたそれぞれの領域に対して画像間の位置合わせを行うために相関計算を行う画像相関処理と、画像相関処理によって位置合わせが行われた画像間での画像差分処理と同時に、位置合わせの位置からずらした画像間での差分処理を行う画像差分処理と、をコンピュータに実行させるためのプログラムを記録していることを特徴としている。
【００１７】
【発明の実施の形態】
以下、本発明の実施の形態について図面を参照しながら詳細に説明する。
【００１８】
図２に本発明での加筆抽出の処理手順を示す。Ｓ１０１では加筆抽出を行う領域の個数Ｎ、大きさなどを設定し、電子原稿画像とその画像を紙に出力した後に加筆されスキャナなどの画像読取装置で取り込まれた加筆重畳画像を入力する。Ｓ１０２では領域数変数ｎを１に設定する。Ｓ１０３では加筆抽出の対象となる領域を加筆重畳画像から切り出し、同時に電子原稿画像からは、先に加筆重畳画像から抽出した領域に対応した領域を切り出す。Ｓ１０４ではＳ１０３で切り出された電子原稿画像をＭヶに小領域に細分化する。
【００１９】
図３は細分化ステップＳ１０４の詳細であり図３（ａ）はＳ１０３で切り出された電子原稿画像の抽出領域Ｏ（ｘ，ｙ）を格子状に分割しておりその分割された小領域それぞれをｏ_ｉｊ（ｘ，ｙ）とする。Ｓ１０５では小領域数変数ｍを１に設定する。Ｓ１０６ではＳ１０４で細分化され作成された電子原稿画像の小領域ｏ_ｉｊ（ｘ，ｙ）が加筆重畳画像上でどの位置に対応するかを決める為に、各々の小領域と加筆重畳画像Ａ（ｘ，ｙ）との相関値の分布を計算する。この時に電子原稿画像と同様に加筆重畳画像の方もＳ１０４で細分化しそれをａ_ｉｊ（ｘ，ｙ）とすると、空間的に対応する両画像の小領域同士、ｏ_ｉｊ（ｘ，ｙ）とａ_ｉｊ（ｘ，ｙ）で相関値計算を行う方が計算時間の観点からも効率的である。
【００２０】
それを示したのが図３（ｂ）でありこの図では加筆重畳画像側の抽出領域Ａ（ｘ，ｙ）をＯ（ｘ，ｙ）と同様に細分化し対応する位置を探すためにａ_ｉｊ（ｘ，ｙ）をｏ_ｉｊ（ｘ，ｙ）よりも大きな領域として抽出している。この相関値計算の結果、相関値が最も高い値を取る１つの画像の相対位置を「最も確からしい位置」とする。
【００２１】
ここでＳ１０６において計算される相関値の分布は、以降の工程で設定する閾値を領域によらずに一意に設定するためには式（１）のように電子原稿画像と加筆彫像画像、それぞれ小領域内の画素値の総和や小領域の面積に依存しないものにしておくとよい。式（１）の場合はｏ_ｉｊ（ｘ，ｙ）とａ_ｉｊ（ｘ，ｙ）がどのような大きさ画像であっても必ず−１．０から＋１．０の間の値を取り、特に画素値が０または１しか取らない２値画像に関しては０から＋１．０までの値を取る。
【００２２】
【数１】

【００２３】
Ｓ１０７での２枚の画像間で３×３画素ずらし差分処理を示したのが図３である。Ｓ４０１にはＳ１０６で算出された相関値の分布Ｃ（ｘ，ｙ）と『最も確からしい位置』（ｘ_ｍ，ｙ_ｍ）が入力される。Ｓ４０１では（ｘ_ｍ，ｙ_ｍ）での相関値Ｃ（ｘ_ｍ，ｙ_ｍ）がある閾値Ｔｈ_１よりも大きいかどうかの判定を行う。大きければＳ４０２に進みそうでなければＳ１０７自体を抜け、図２のＳ１０８に進む。以下のＳ４０２、Ｓ４０３およびＳ４０６〜Ｓ４０９は（ｘ_ｍ，ｙ_ｍ）とその周囲の３×３の領域での繰り返し処理を示したものである。この領域の大きさを変える場合はＳ４０２、Ｓ４０３での設定値、Ｓ４０６、Ｓ４０８での加算値、Ｓ４０７、Ｓ４０９での条件値を変更する。Ｓ４０４ではＳ４０２とＳ４０３において変更された位置（ｘ_ｍ＋ｘ，ｙ_ｍ＋ｙ）での相関値Ｃ（ｘ_ｍ＋ｘ，ｙ_ｍ＋ｙ）が閾値Ｔｈ_２（＜Ｔｈ_１）を超えているかどうかの判定を行う。越えていなければＳ４０８まで進み画素ずらし量を変化させる。超えている場合はＳ４０５に進みここで加筆重畳画像と電子原稿画像、２枚の小領域画像の位置関係を（ｘ_ｍ＋ｘ，ｙ_ｍ＋ｙ）に対応する位置とし画像差分処理を行う。
【００２４】
このような「画素ずらし」での差分処理を行う理由は、電子原稿画像と加筆重畳画像との間の線幅の違いにある。図５は電子原稿画像と加筆重畳画像の拡大図であるが、図５左のように電子原稿画像上で太さ１ドットの線をいったん紙に出力しスキャナなどのデバイスで再び画像として取りこむと、図５右のように線幅は太化されその太さも局所的にバラツキのある線となる。これが差分処理に影響し加筆重畳画像の原稿成分の引き残しが発生する為、上記のＳ１０７ような画素ずらしでの差分処理が単純な差分処理よりも有効となる。
【００２５】
ここまでの処理を終えた結果例が図６である。図６左が加筆重畳画像の小領域、図５中が電子原稿画像の小領域、図６右が２つの小領域画像に対し蒸気の処理を行い精製した加筆画像である。Ｓ１０８においてｍを１だけ増加させた後Ｓ１０９において小領域変数ｍが上限Ｍを超えているか否かを判定を行い、超えていなければＳ１０６から処理を繰り返す。
【００２６】
図７及び図８はこうして作成された加筆画像の一部である。図左が加筆重畳画像、図右が加筆画像の一部となる。すべての小領域についての処理を終える、つまり、ｍがＭを超えるとＳ１１０では差分処理後の画像を白紙の画像のＳ１０３で抽出した位置に挿入する事で加筆のみの画像を作成する。Ｓ１１１に領域数変数ｎを１だけ増加させた後Ｓ１１２においてｎが上限Ｎを超えているかどうかの判定を行い、超えていなければＳ１０３から処理を繰り返す。ｎがＮを超えるとＳ１１３においてＳ１１０において張り合わされた加筆画像を出力する。
【００２７】
図１に本発明での加筆抽出装置の一実施例を示す。８０１は領域抽出部であり図２のＳ１０１で設定される抽出領域設定情報とが加筆重畳画像と電子原稿画像の両画像入力され、その領域情報に基づいてＳ１０３での所望の領域切り出しを両画像から行う。８０２は領域細分部でありここでは領域抽出部５０１で切り出された電子原稿画像の領域に対してＳ１０４の領域細分化を行う。８０３は画像相関部であり領域細分部８０２において細分化された小領域画像に対しＳ１０６の画像相関計算を実行し２つの画像間のズレ量を算出する。８０４は差分処理部で領域細分部８０２から入力される小領域画像と画像相関部８０３で算出されたズレ量に基いて位置合わせを行い電子原稿画像と加筆重畳画像との間でＳ１０７の画素ずらし差分処理を行う。８０５は画像合成部で領域抽出部８０１から入力される抽出領域情報に基づいて差分処理部８０４から入力される差分処理後の画像、つまり加筆画像の張り合わせ処理Ｓ１１０を実行しＳ１１３の加筆画像の出力を行う。
【００２８】
【発明の効果】
請求項１記載の加筆抽出装置によれば、電子文書の原稿画像と前記電子文書を紙へ出力した後に手書きの加筆が重畳した状態でスキャナ等の画像読取装置で再び電子化された画像とを細分化する領域分割手段と、領域分割手段によって分割されたそれぞれの領域に対して画像間の位置合わせを行うために相関計算を行う画像相関手段と、画像相関手段によって位置合わせが行われた画像間での画像差分処理と同時にその位置からわずかにずらした画像間での差分処理を行う画像差分手段とを有しているので、加筆の抽出精度が向上する。
【００２９】
請求項２記載の加筆抽出装置によれば、請求項１記載の加筆抽出装置において、画像相関手段は、計算の対象となる２枚の画像の画素値の総和や画像の面積に依存しない相関値を算出することを特徴とするので、閾値を一意に設定する事が可能になり計算効率が向上する。
【００３０】
請求項３記載の加筆抽出装置によれば、請求項１記載の加筆抽出装置において、画像差分手段は、画像相関手段によって位置合わせが行われた画像間の相関値が所定の閾値Ｔｈ_１以上の場合にのみ差分処理をすることを特徴とするので、加筆の抽出精度が向上する。
【００３１】
請求項４記載の加筆抽出装置によれば、請求項１記載の加筆抽出装置において、画像差分手段は、画像相関手段によって位置合わせが行われその位置からわずかにずらした画像間の相関値が所定の閾値Ｔｈ_２以上の場合にのみ差分処理をすることを特徴とするので、加筆の抽出精度が向上する。
【００３２】
請求項５記載の加筆抽出方法によれば、電子文書の原稿画像と電子文書を紙へ出力した後に手書きの加筆が重畳した状態でスキャナ等の画像読取装置で再び電子化された画像とを細分化する領域分割ステップと、領域分割ステップによって分割されたそれぞれの領域に対して画像間の位置合わせを行うために相関計算を行う画像相関ステップと、画像相関ステップによって位置合わせが行われた画像間での画像差分処理と同時にその位置からわずかにずらした画像間での差分処理を行う画像差分ステップとを有しているので、加筆の抽出精度が向上する。
【００３３】
請求項６記載の加筆抽出方法によれば、請求項５記載の加筆抽出方法において、画像相関ステップは、計算の対象となる２枚の画像の画素値の総和や画像の面積に依存しない相関値を算出することを特徴とするので、閾値を一意に設定する事が可能になり、計算効率が向上する。
【００３４】
請求項７記載の加筆抽出方法によれば、請求項５記載の加筆抽出方法において、画像差分ステップは、画像相関ステップによって位置合わせが行われた画像間の相関値が所定の閾値Ｔｈ_１以上の場合にのみ差分処理をすることを特徴とするので、加筆の抽出精度が向上する。
【００３５】
請求項８記載の加筆抽出方法によれば、請求項５記載の加筆抽出方法において、画像差分ステップは、画像相関ステップによって位置合わせが行われ、その位置からわずかにずらした画像間の相関値が所定の閾値Ｔｈ_２以上の場合にのみ差分処理をすることを特徴とするので、加筆の抽出精度が向上する。
【００３６】
請求項９記載のプログラムによれば、電子文書を紙へ出力した後に手書きの加筆が重畳した状態で電子文書の原稿画像と、画像読取装置で再び電子化された画像とを細分化する領域分割処理と、領域分割処理によって分割されたそれぞれの領域に対して前記画像間の位置合わせを行うために相関計算を行う画像相関処理と、画像相関処理によって位置合わせが行われた画像間での画像差分処理と同時に、その位置からずらした画像間での差分処理を行う画像差分処理とを有することを特徴としているので、加筆の抽出精度が向上する。
【００３７】
請求項１０記載の記録媒体によれば、電子文書の原稿画像と電子文書を紙へ出力した後に手書きの加筆が重畳した状態でスキャナ等の画像読取装置で再び電子化された画像とを細分化する領域分割処理と、領域分割処理によって分割されたそれぞれの領域に対して画像間の位置合わせを行うために相関計算を行う画像相関処理と、画像相関処理によって位置合わせが行われた画像間での画像差分処理と同時に、その位置からずらした画像間での差分処理を行う画像差分処理と、をコンピュータに実行させるためのプログラムを記録していることを特徴としているので、加筆の抽出精度が向上する。
【図面の簡単な説明】
【図１】本発明の実施の形態である加筆抽出装置の構成を示す構成図である。
【図２】本発明の実施の形態に係る加筆抽出の処理手順を示すフローチャートである。
【図３】本発明の実施の形態に係る細分化ステップの例を示す図である。
【図４】本発明の実施の形態に係る画素ずらし差分処理の処理手順を示すフローチャートである。
【図５】本発明の実施の形態に係る電子原稿画像と加筆重畳画像の例を示す拡大図である。
【図６】本発明の実施の形態に係る画素ずらし差分処理後の結果例を示す図である。
【図７】本発明の実施の形態に係る画素ずらし差分処理によって作成された加筆画像の第１の例を示す図である。
【図８】本発明の実施の形態に係る画素ずらし差分処理によって作成された加筆画像の第２の例を示す図である。
【符号の説明】
８００加筆抽出装置
８０１領域抽出部
８０２領域細分部
８０３画像相関部
８０４差分処理部
８０５画像合成部[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a handwriting extraction method for a handwritten character input system, and more particularly to a handwriting extraction device, a handwriting extraction method, a handwriting extraction program, and a recording medium that improve the extraction accuracy of handwriting extraction and handwriting image creation.
[0002]
[Prior art]
Conventionally, when updating the content of image data registered in the document image filing device, write processing is directly performed on the image data imported into the device, or character data is separately prepared and superimposed. There is a method. There is also a method of reading a retouched updated image on a paper basis and recording the updated image.
[0003]
As a conventional technology example, in a document processing apparatus, by using information written by hand as one of means for detecting as image information, an input retouched superimposed image is divided into blocks, and each of the small areas is divided into blocks. There is a document processing apparatus that takes a difference and performs optimal alignment while checking whether or not skew has occurred (for example, see Patent Document 1). Also, a method of detecting characteristic four points such as corners of an outer frame in an image and performing a complementing process such that each point moves to a corresponding point in the electronic document image, or a method of detecting one point as a corresponding point For example, there is a document image filing apparatus and a document image updating method for performing a difference process after performing a position alignment by a method of performing a correction process using an affine transformation formula for performing a parallel movement, rotating around the point, and expanding / contracting the point (for example, , Patent Document 2). In addition, a difference image is generated between a document image with no mark and a retouched superimposed image with a mark, and the presence or absence of a mark is determined by calculating the number of black pixels and the ratio of black pixels in each check area in the difference image. There is a mark recognizing device that performs the operation (for example, see Patent Document 3).
[0004]
[Patent Document 1]
Japanese Patent Application Laid-Open No. H5-244387 [Patent Document 2]
JP 2001-283231 A [Patent Document 3]
JP-A-11-265422
[Problems to be solved by the invention]
However, none of the conventional examples described in Patent Documents 1 to 3 described above takes into account the fact that the thickness of a printed character is different between an electronic image and a retouched superimposed image passing through a printing and scanning process. Further, the document processing device described in Patent Document 1 and the mark recognition device described in Patent Document 3 do not take any measures regarding noise removal after differential processing, and fine noise components remain in the image after differential processing. There is. Further, the document image filing apparatus and the document image updating method described in Patent Document 2 and the mark recognition apparatus described in Patent Document 3 perform retouching due to uneven paper feeding during printing, uneven rotation of the drum, uneven speed of the optical system during scanning, and the like. Inability to cope with local fluctuations in magnification of the image.
[0006]
The present invention has been made in view of the above circumstances, and when creating a retouched image from an electronic image and a retouched superimposed image, taking into account the difference in character line width between the two images while dividing and aligning the area. It is an object of the present invention to provide a retouching extraction device, a retouching extraction method, a retouching extraction program, and a recording medium for performing retouching with higher accuracy by performing difference processing in a retouching process.
[0007]
[Means for Solving the Problems]
In order to achieve this object, the retouch extraction device according to claim 1 is digitized again by the image reading device in a state where the original image of the electronic document and the handwritten retouch are superimposed after outputting the electronic document to paper. An area dividing unit that subdivides the image, an image correlating unit that performs a correlation calculation to perform an alignment between the images with respect to each area divided by the area dividing unit, and the image correlating unit performs the alignment. Image difference means for performing difference processing between images shifted from the position simultaneously with the performed image difference processing between images.
[0008]
According to a second aspect of the present invention, in the retouching and extracting apparatus according to the first aspect, the image correlation unit calculates a sum of pixel values of two images to be calculated and a correlation value independent of an area of the image. It is characterized by doing.
[0009]
Categories extraction apparatus according to claim 3, wherein, in the retouch extraction apparatus according to claim 1, the image difference means, when the correlation value between images alignment has been performed by the image correlation means of _one or more predetermined threshold Th Only the difference processing is performed.
[0010]
According to a fourth aspect of the present invention, in the retouching and extracting apparatus according to the first aspect, the image difference unit is positioned by the image correlating unit, and a correlation value between the images slightly shifted from the position is determined by a predetermined value. It is characterized by performing the differential processing only when the threshold value Th ₂ or more.
[0011]
The method for extracting and adding a portion according to claim 5, wherein the original image of the electronic document is divided into an image digitized again by the image reading device in a state where the handwriting is superimposed after outputting the electronic document to paper. Step, an image correlation step of performing a correlation calculation to perform alignment between images for each area divided by the area division step, and an image difference between the images aligned by the image correlation step. Simultaneously with the processing, an image difference step of performing a difference process between images slightly shifted from the alignment position.
[0012]
According to a sixth aspect of the present invention, in the retouching and extracting method according to the fifth aspect, the image correlation step calculates a sum of pixel values of two images to be calculated and a correlation value independent of an area of the image. It is characterized by doing.
[0013]
Categories extraction method according to claim 7, wherein, in the retouch extraction method of claim 5, wherein the image difference step, the case image correlation value between image alignment is performed by the correlation step is a predetermined threshold value Th ₁ or more It is characterized in that the difference processing is performed only for.
[0014]
According to a further aspect of the present invention, in the retouching extraction method according to the fifth aspect, the image difference step is performed by an image correlation step, and a correlation value between images slightly shifted from the position is determined by a predetermined value. is characterized in that the differential processing only when the threshold value Th ₂ or more.
[0015]
10. The program according to claim 9, wherein after the electronic document is output to paper, a handwritten retouch is superimposed, and the original image of the electronic document is subdivided into an image digitized again by the image reading device. And image correlation processing for performing a correlation calculation in order to perform alignment between images for each area divided by the area division processing, and image difference processing between images that have been aligned by the image correlation processing At the same time, the present invention is characterized in that a computer executes image difference processing for performing difference processing between images shifted from the position.
[0016]
11. The recording medium according to claim 10, wherein the original image of the electronic document is divided into an image digitized again by the image reading apparatus in a state in which handwriting is superimposed after outputting the electronic document on paper. And image correlation processing for performing a correlation calculation in order to perform alignment between images for each area divided by the area division processing, and image difference processing between images that have been aligned by the image correlation processing At the same time, it is characterized in that a program for causing a computer to execute an image difference process for performing a difference process between images shifted from the alignment position is recorded.
[0017]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
[0018]
FIG. 2 shows a processing procedure of retouching extraction in the present invention. In step S101, the number N, size, and the like of areas to be retouched are set, and an electronic original image and a retouched superimposed image that has been retouched after being output on paper and captured by an image reading device such as a scanner are input. In S102, the number-of-regions variable n is set to one. In S103, a region to be retouched is extracted from the retouched superimposed image, and at the same time, an area corresponding to the region previously extracted from the retouched superimposed image is cut out from the electronic document image. In S104, the electronic document image cut out in S103 is subdivided into M small areas.
[0019]
FIG. 3 shows the details of the subdividing step S104. FIG. 3A shows the extraction area O (x, y) of the electronic document image cut out in S103 divided in a grid pattern, and the divided small areas are each divided. Let o _ij (x, y). In S105, the small area number variable m is set to 1. In S106, in order to determine the position of the small area o _ij (x, y) of the electronic document image subdivided and created in S104 on the retouched superimposed image, each of the small areas and the retouched superimposed image A ( x, y) is calculated. At this time, similarly to the electronic original image, the retouched superimposed image is also subdivided in S104 and is defined as a _ij (x, y), and the small regions of both spatially corresponding images, o _ij (x, y) and Performing the correlation value calculation with a _ij (x, y) is more efficient in terms of calculation time.
[0020]
This is shown in FIG. 3B. In this figure, a _ij is used to subdivide the extraction area A (x, y) on the side of the retouched superimposed image in the same manner as O (x, y) and search for a corresponding position. (X, y) is extracted as a region larger than o _ij (x, y). As a result of the correlation value calculation, the relative position of one image having the highest correlation value is defined as the “most likely position”.
[0021]
Here, the distribution of the correlation values calculated in S106 is based on the electronic original image and the hand-drawn engraved image as shown in Expression (1) in order to uniquely set the threshold value set in the subsequent steps regardless of the area. It is preferable that the value does not depend on the sum of the pixel values in the area or the area of the small area. In the case of equation (1), o _ij (x, y) and a _ij (x, y) always take values between −1.0 and +1.0 regardless of the size of the image. A binary image having a pixel value of only 0 or 1 takes a value from 0 to +1.0.
[0022]
(Equation 1)

[0023]
FIG. 3 shows the 3 × 3 pixel shift difference processing between the two images in S107. Distribution C (x, y) of the correlation value calculated in S106 to S401 as "most probable position" _(x m, _{y m)} are inputted. In S401 _(x _{m, y} m) correlation value _{C (x} _{m, y} m) in the determination of whether or larger than a threshold Th ₁ there is performed. If it is larger, the process proceeds to S402. If not, the process exits S107 itself and proceeds to S108 in FIG. The following S402, S403, and S406 to S409 show the repetition processing in (x _m , y _m ) and the surrounding 3 × 3 area. When the size of this area is changed, the set values in S402 and S403, the added values in S406 and S408, and the condition values in S407 and S409 are changed. Changed position in S402 and S403 in _{_{S404 (x m + x, y}} m + y) correlation value _{_{C (x m + x, y}} m + y) in the determination of whether exceeds the threshold _Th 2 _(<Th ₁₎ Do. If not, the process advances to step S408 to change the pixel shift amount. Beyond proceeds to S405 if have revision superimposed image and the electronic document image here, the positional relationship between the two small-area image _{(x m + x, y m} + y) an image difference processing and the position corresponding to perform.
[0024]
The reason for performing such a difference process by “pixel shift” is a difference in line width between the electronic document image and the retouched superimposed image. FIG. 5 is an enlarged view of the electronic original image and the retouched superimposed image. As shown on the left side of FIG. 5, when a line having a thickness of 1 dot is once output on paper and re-captured as an image by a device such as a scanner. As shown in the right side of FIG. 5, the line width is thickened, and the thickness is also a line having local variation. This affects the difference processing, and the document component of the added superimposed image is left behind, so that the difference processing by pixel shift as described in S107 is more effective than the simple difference processing.
[0025]
FIG. 6 shows an example of the result obtained after the above processing. 6 shows a small area of the retouched superimposed image, FIG. 5 shows a small area of the electronic original image, and FIG. 6 right shows a retouched image obtained by performing steam processing on the two small area images and refining them. After m is increased by 1 in S108, it is determined in S109 whether or not the small area variable m exceeds the upper limit M. If not, the process is repeated from S106.
[0026]
FIGS. 7 and 8 show a part of the retouched image thus created. The left part of the drawing is a superimposed image, and the right part of the drawing is a part of the modified image. When the processing for all the small areas is completed, that is, when m exceeds M, in S110, the image after the difference processing is inserted at the position extracted in S103 of the blank image to create an image with only the retouching. After increasing the number-of-regions variable n by 1 in S111, it is determined in S112 whether n exceeds the upper limit N. If not, the process is repeated from S103. If n exceeds N, the retouched image attached in S110 is output in S113.
[0027]
FIG. 1 shows an embodiment of a retouching and extracting apparatus according to the present invention. Reference numeral 801 denotes an area extracting unit which receives the extracted area setting information set in step S101 in FIG. 2 as both the superimposed superimposed image and the electronic document image, and cuts out a desired area in step S103 based on the area information. Do from. Reference numeral 802 denotes an area subdivision unit, which performs area subdivision in step S104 on the area of the electronic document image cut out by the area extraction unit 501. Reference numeral 803 denotes an image correlation unit that executes the image correlation calculation in S106 on the small area image subdivided in the area subdivision unit 802 to calculate the amount of deviation between the two images. Reference numeral 804 denotes a difference processing unit that performs positioning based on the small area image input from the area subdivision unit 802 and the amount of deviation calculated by the image correlation unit 803, and shifts the pixel in step S107 between the electronic original image and the retouched superimposed image. Perform difference processing. Reference numeral 805 denotes an image synthesizing unit that executes the difference-processed image input from the difference processing unit 804 based on the extraction area information input from the area extraction unit 801, that is, the addition processing of the retouched image S110 and the output of the retouched image in S113 I do.
[0028]
【The invention's effect】
According to the retouch extraction device of the first aspect, the original image of the electronic document and the image digitized again by an image reading device such as a scanner in a state where the handwriting is superimposed after the electronic document is output to paper are superimposed. An area dividing unit that subdivides the image, an image correlating unit that performs a correlation calculation to perform an alignment between images for each of the regions divided by the area dividing unit, and an image that is aligned by the image correlating unit Image difference means for performing difference processing between images slightly shifted from the position at the same time as image difference processing between images, so that the accuracy of addition of a pen can be improved.
[0029]
According to the second aspect of the present invention, in the retouching and extracting apparatus according to the first aspect, the image correlating means includes a correlation value independent of a sum of pixel values of two images to be calculated and an area of the image. , The threshold value can be uniquely set, and the calculation efficiency is improved.
[0030]
According to Categories extraction apparatus according to claim 3, wherein, in the retouch extraction apparatus according to claim 1, the image difference means, the correlation value between images alignment has been performed by the image correlation means is a predetermined threshold value Th ₁ or more Since the difference processing is performed only in the case, the extraction accuracy of the correction is improved.
[0031]
According to the retouching and extracting apparatus of the fourth aspect, in the retouching and extracting apparatus of the first aspect, the image difference unit is positioned by the image correlating unit, and a correlation value between the images slightly shifted from the position is determined. since, characterized in that the only difference processing in the case of the threshold value Th ₂ or more, thereby improving the retouch extraction accuracy.
[0032]
According to the retouch extraction method according to the fifth aspect, the original image of the electronic document and the image digitized again by the image reading device such as a scanner with the handwritten retouch superimposed after the electronic document is output to paper are subdivided. An image dividing step, an image correlation step of performing a correlation calculation to perform an alignment between images for each of the regions divided by the area dividing step, and an image alignment performed by the image correlation step. And an image difference step for performing a difference process between images slightly shifted from the position at the same time as the image difference process in step (1).
[0033]
According to the retouching and extracting method according to the sixth aspect, in the retouching and extracting method according to the fifth aspect, the image correlation step is a correlation value that does not depend on a sum of pixel values of two images to be calculated or an area of the image. , The threshold value can be uniquely set, and the calculation efficiency is improved.
[0034]
According to Categories extraction method according to claim 7, wherein, in the retouch extraction method of claim 5, wherein the image difference step, the correlation value between images alignment has been performed by the image correlation step is a predetermined threshold value Th ₁ or more Since the difference processing is performed only in the case, the extraction accuracy of the correction is improved.
[0035]
According to the retouching and extracting method according to the eighth aspect, in the retouching and extracting method according to the fifth aspect, in the image difference step, alignment is performed by an image correlation step, and a correlation value between images slightly shifted from the position is calculated. since, characterized in that the differential processing only when the predetermined threshold value Th ₂ or more, thereby improving the retouch extraction accuracy.
[0036]
According to the program of the ninth aspect, after the electronic document is output to paper, the area division for subdividing the original image of the electronic document and the image digitized again by the image reading device in a state in which the handwriting is superimposed. Processing, image correlation processing for performing a correlation calculation for performing alignment between the images with respect to each area divided by the area division processing, and images between the images aligned by the image correlation processing. Since the difference processing is performed simultaneously with the difference processing and the difference processing between the images shifted from the position is performed, the extraction accuracy of the correction is improved.
[0037]
According to the recording medium of the tenth aspect, the document image of the electronic document and the image digitized again by the image reading device such as the scanner in a state where the handwriting is superimposed after the electronic document is output to paper are superimposed. Area division processing, image correlation processing for performing correlation calculation to perform image alignment for each area divided by the area division processing, and image alignment performed by the image correlation processing. And a program for causing a computer to execute image difference processing for performing difference processing between images shifted from the position at the same time as the image difference processing of the above. improves.
[Brief description of the drawings]
FIG. 1 is a configuration diagram showing a configuration of a retouching and extracting apparatus according to an embodiment of the present invention.
FIG. 2 is a flowchart showing a processing procedure of retouching extraction according to the embodiment of the present invention.
FIG. 3 is a diagram illustrating an example of a subdivision step according to the embodiment of the present invention.
FIG. 4 is a flowchart illustrating a processing procedure of a pixel shift difference process according to the embodiment of the present invention.
FIG. 5 is an enlarged view showing an example of an electronic document image and a retouched superimposed image according to the embodiment of the present invention.
FIG. 6 is a diagram illustrating an example of a result after a pixel shift difference process according to the embodiment of the present invention.
FIG. 7 is a diagram showing a first example of a retouched image created by the pixel shift difference process according to the embodiment of the present invention.
FIG. 8 is a diagram showing a second example of a retouched image created by the pixel shift difference process according to the embodiment of the present invention.
[Explanation of symbols]
800 Retouch extraction device 801 Area extraction unit 802 Area subdivision unit 803 Image correlation unit 804 Difference processing unit 805 Image synthesis unit

Claims

Area dividing means for subdividing a document image of an electronic document and an image digitized again by the image reading apparatus in a state where handwriting is superimposed after outputting the electronic document on paper;
Image correlation means for performing a correlation calculation to perform alignment between the images for each area divided by the area division means,
Simultaneously with the image difference processing between the images that have been aligned by the image correlation means, image difference means that performs difference processing between images shifted from the position,
A retouching and extracting apparatus characterized by having:

2. The retouching and extracting apparatus according to claim 1, wherein the image correlating means calculates a sum of pixel values of two images to be calculated and a correlation value independent of an area of the image.

2. The retouching and extracting apparatus according to claim 1, wherein the image difference unit performs the difference process only when a correlation value between the images subjected to the alignment by the image correlation unit is equal to or greater than a predetermined threshold. .

The image difference unit is characterized in that the alignment is performed by the image correlation unit, and the difference process is performed only when a correlation value between the images slightly shifted from the position is equal to or greater than a predetermined threshold value. The rewriting extraction device according to claim 1.

An area dividing step of subdividing the original image of the electronic document and the image digitized again by the image reading device in a state where the handwritten retouch is superimposed after outputting the electronic document to paper,
An image correlation step of performing a correlation calculation to perform alignment between the images for each area divided by the area division step;
Simultaneously with the image difference processing between the images subjected to the alignment by the image correlation step, an image difference step of performing a difference processing between images slightly shifted from the position,
A retouch extraction method characterized by having:

6. The retouching and extracting method according to claim 5, wherein the image correlation step calculates a sum value of pixel values of two images to be calculated and a correlation value independent of an area of the image.

6. The retouching extraction according to claim 5, wherein in the image difference step, the difference processing is performed only when a correlation value between the images subjected to the alignment in the image correlation step is equal to or larger than a predetermined threshold value. Method.

The image difference step is characterized in that the alignment is performed by the image correlation step, and the difference processing is performed only when a correlation value between the images slightly shifted from the position is equal to or greater than a predetermined threshold value. The retouch extraction method according to claim 5.

An area dividing process for subdividing the original image of the electronic document in a state where the handwritten retouch is superimposed after outputting the electronic document on paper, and the image digitized again by the image reading device,
Image correlation processing for performing a correlation calculation to perform alignment between the images for each area divided by the area division processing,
Simultaneously with the image difference processing between the images subjected to the alignment by the image correlation processing, image difference processing to perform a difference processing between the images shifted from the position,
Program to make a computer execute.

A region division process of subdividing an original image of an electronic document and an image digitized again by an image reading device such as a scanner in a state where handwriting is superimposed after outputting the electronic document on paper,
Image correlation processing for performing a correlation calculation to perform alignment between the images for each area divided by the area division processing,
Simultaneously with the image difference processing between the images subjected to the alignment by the image correlation processing, image difference processing to perform a difference processing between the images shifted from the position,
Recording medium for recording a retouching extraction program for causing a computer to execute the program.