JP2004129189A - Image distortion correction unit, image reading unit, image forming unit, program, and storage medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image distortion correction unit, an image reading unit, an image forming unit, program, and a storage medium, where a good image distortion correction result is obtained even if the positions of top (or bottom) character of character lines in columnar writing is widely fluctuated in vertical directions for each line. <P>SOLUTION: The character lines in columnar writing used for calculating a polynomial approximated curve are extracted limited to the ones having a constant positional relationship based on the positional information of the top of character line (or the bottom of character line) of the character lines in columnar writing (shown by a shaded rectangle). Accordingly, the character lines in columnar writing can be extracted limited to the ones having a constant positional relationship based on the positional information of the top of character line (or the bottom of character line) of the character lines in columnar writing, to obtain the good image distortion correction result even if the positions of the top (or bottom) characters of character lines in columnar writing are fluctuated widely in vertical directions by each line. <P>COPYRIGHT: (C)2004,JPO

Description

【０１１４】
算出された比率ｋが、予め定められた閾値よりも大きい場合に、スキャン画像にページ外形が存在すると判断する。
【０１１５】
なお、スキャン画像の上下にページ外形が存在する場合には、ヒストグラムの左右両端に高い縦棒が現れることになるので、このような場合には、ヒストグラムの左右両端の高い縦棒に基づいてスキャン画像にページ外形が存在するか否かの判断がそれぞれ実行される。
【０１１６】
ここに、ページ外形判別手段の機能が実行される。
【０１１７】
以上の処理により、スキャン画像にページ外形が存在すると判断された場合には、左右ページの上下辺のいずれにページ外形が存在しているのかという情報とともにページ外形を抽出し、ＲＡＭ３３に一時的に記憶する。
【０１１８】
なお、このスキャン画像にページ外形が存在するか否かの判断処理は、スキャン画像の綴じ部境界線を境にした左右ページ毎に実行される。
［スキャン画像からの罫線の抽出］
続くステップＳ４２においては、スキャン画像からの罫線の抽出処理を実行する。ステップＳ４２におけるスキャン画像からの罫線の抽出処理について説明する。
【０１１９】
［罫線候補の検出］
ここで、図１４は長い罫線が存在するスキャン画像の一例を示す説明図、図１５は図１４に示したスキャン画像の綴じ部境界線左側の黒画素ヒストグラムである。図１５に示すヒストグラムのｘ軸はスキャン画像の主走査方向（図１４の上下方向）を示すものであり、スキャン画像の上端はヒストグラムの左端に対応付けられている。図１４に示すようにスキャン画像に罫線が存在する場合には、図１５に示すヒストグラムに幅の狭いピークが現れることになる。本実施の形態では、このような特性を利用して、スキャン画像に罫線が存在するか否かの判断を行う。
【０１２０】
より具体的には、まず、図１５に示すヒストグラムに現れた幅の狭いピークの高さＨを求めるとともに、求められた各ピークの中央位置（高さが半分の位置）における幅Ｗを求める。そして、ピークの高さＨが予め定められた閾値ｔｈＨよりも高く、かつ、ピークの中央位置の幅Ｗが予め定められた閾値ｔｈＷより小さなピークが存在する場合、そのピークを罫線の候補とする。
【０１２１】
ここに、罫線候補抽出手段の機能が実行される。
【０１２２】
続いて、罫線の候補とされたピークについて、罫線の連続性を利用して、更に罫線か否かの判断をする。図１６に示すように、候補罫線上の適当な位置（例えば、ページの中心線の位置）を開始点とし、この開始点から候補罫線を左右方向へ探索し、切断点（罫線がかすれて途切れている部分）の数を累積する。切断点の数が予め定められた閾値より少なければ、この候補を罫線と判断する。このように罫線連続性に基づいて罫線か否かの判断をすることにより、罫線として誤って検出された小さな文字で構成された横書き文字行や点線等を排除することが可能になる。
【０１２３】
ここに、罫線判別手段の機能が実行される。
【０１２４】
［罫線の座標検出］
以上のようにして罫線を判別した後、各罫線の座標を検出する。罫線座標の検出は、図１７に示すように、罫線の主走査方向（図１７のｙ軸方向）の座標値を罫線部の黒画素ランの中点座標とした場合、図１７に示す罫線の左端のｘ１における主走査方向座標値はｙ１となる。
【０１２５】
［最適罫線の選択］
次に、候補罫線の中から歪み補正に最適な罫線を選択する。図１８に示すように複数の罫線が検出される場合、どの罫線を用いて歪み補正するかを選択する必要がある。最適な罫線の選択基準の一例としては、罫線の長さが予め定められた閾値より長く、かつ、綴じ部境界線を挟んだ左右の一定幅領域内（図１８の網掛け領域）に罫線の一部がかかっていることを条件とし、その中で上下何れかのページ外形に最も近い罫線を選択するようにする。図１８においては、左右ページから各１本ずつの罫線を選択する場合を示している。ここでは、罫線▲１▼と罫線▲２▼とが選択されている。
【０１２６】
また、最適な罫線の選択基準の別の例としては、罫線の長さが予め定められた閾値より長く、かつ、綴じ部境界線を挟んだ左右の一定幅領域内（図１９の網掛け領域）に罫線の一部がかかっていることを条件とし、各ページの上部では上端のページ外形に、各ページの下部では下端のページ外形に、それぞれ最も近い罫線を選択するようにする。図１９においては、左右ページをさらに上下部分に分け、その各４ブロックにおいて１本ずつの罫線を選択する場合を示している。ここでは、左上のブロックでは罫線▲１▼、右下のブロックでは罫線▲２▼、左下のブロックでは罫線▲３▼が選択されている。なお、図１９中の右上のブロックには上記２条件（罫線の長さが予め定められた閾値より長く、かつ、綴じ部境界線を挟んだ左右の一定幅領域内に罫線の一部がかかっている）を満足する罫線が存在しないので、選択された罫線はない。
【０１２７】
なお、上記２条件（罫線の長さが予め定められた閾値より長く、かつ、綴じ部境界線を挟んだ左右の一定幅領域内に罫線の一部がかかっている）については、その両方ではなく何れか一方のみを満足するものであっても良い。また、選択基準として上例では「ページ外形に最も近い」を用いているが、これに限るものではなく、「罫線の湾曲が最も大きい」を用いても良い。ここで、「罫線の湾曲」は罫線の左右両端点の主走査方向の座標値の差で表すものとする。
【０１２８】
［最適罫線の座標値の決定］
最適な罫線が選択された場合には、罫線の（主走査方向の）座標値を決定する。罫線の（主走査方向の）座標値は、選択された罫線を左右ページのそれぞれ両端に達するまで近似して延長することにより決定される。図２０において、罫線が存在しているＢＣ部については、前述した罫線座標検出処理により既に座標値は決まっていることから、それ以外の延長部分について罫線の（主走査方向の）座標値を決定することになる。より詳細には、図２０に示すＡＢ部は直線近似で（主走査方向の）座標値を推定し、ＣＤ部は多項式近似曲線で（主走査方向の）座標値を推定する。
【０１２９】
［不適切な罫線の排除］
最後に不適切な罫線を排除する。これは、前述したように多項式近似により座標値を推定する際に、多項式近似による推定曲線の形状が不適切である場合には補正の際にかえって歪みが増大する恐れがあるので、このような罫線を排除するものである。不適切な近似曲線形状の例としては、図２１に示すように、曲線が書籍の外側へ向かうような曲線▲１▼や、中心線を超えて大きく内側へ食い込むような曲線▲２▼である。
【０１３０】
なお、推定曲線の形状が不適切であるとして罫線を排除した場合には、再び最適な罫線を選択し、上記の処理を繰り返すことになる。
【０１３１】
以上の処理により、スキャン画像に罫線が存在すると判断された場合には、左右各ページのいずれの位置に罫線が存在しているのかという情報とともに罫線を抽出し、ＲＡＭ３３に一時的に記憶する。
［スキャン画像からの文字行の抽出］
続くステップＳ４３においては、スキャン画像からの文字行の抽出処理を実行する。ステップＳ４３におけるスキャン画像からの文字行の抽出処理について説明する。本実施の形態においては、まず、スキャン画像中の文字行が縦書き文字行なのか、横書き文字行なのかの判別を行う。
【０１３２】
［文字行の判別］
スキャン画像中の文字行が縦書き文字行なのか、横書き文字行なのかの判別手法について説明する。ここで、図２２は図８に示した画像の副走査方向の黒白反転数ヒストグラムである。図２２中の横軸は、副走査方向（左右方向）の黒画素（スキャン画像を黒白反転させた画素の中でその濃度値が予め定めた濃度値よりも濃い画素）の主走査方向上での位置を示し、図２２中の縦軸は、その位置毎の黒画素数を示すものである。また、図２３は図８に示した画像の主走査方向の黒白反転数ヒストグラムである。図２３中の横軸は、主走査方向（上下方向）の黒画素（スキャン画像を黒白反転させた画素の中でその濃度値が予め定めた濃度値よりも濃い画素）の副走査方向上での位置を示し、図２３中の縦軸は、その位置毎の黒画素数を示すものである。画像中の文字が横書きの図８に示したようなスキャン画像の場合、図２２に示すような副走査方向のヒストグラムは激しく変化するが、図２３に示すような主走査方向のヒストグラムの変化は少ない。また、特に図示しないが、スキャン画像中の文字行が縦書き文字行である場合には、主走査方向のヒストグラムは激しく変化するが、副走査方向のヒストグラムの変化は少ない。
【０１３３】
上述したような判別手法は、具体的には下記に示す各式により実現される。まず、下記に示す式（２）により、
【０１３４】
【数２】

【０１３５】
主走査方向ｙの位置でのヒストグラム値Ｐｎｔ（ｙ）の平均値ｍｅａｎ_Ｈが算出される。ここで、ｈｅｉｇｈｔは画像の高さである。
そして、下記に示す式（３）により、
【０１３６】
【数３】

【０１３７】
副走査方向のヒストグラムの主走査方向に関する分散σ_Ｈが得られる。
同様に、下記に示す式（４）により、
【０１３８】
【数４】

【０１３９】
副走査方向ｘの位置でのヒストグラム値Ｐｎｔ（ｘ）の平均値ｍｅａｎ_Ｖが算出される。ここで、ｗｉｄｔｈは画像の幅である。
そして、下記に示す式（５）により、
【０１４０】
【数５】

【０１４１】
主走査方向のヒストグラムの副走査方向に関する分散σ_ｖが得られる。
【０１４２】
上述したようにスキャン画像中の文字行が横書き文字行である場合には、副走査方向のヒストグラムの主走査方向に関する分散σ_Ｈが、主走査方向のヒストグラムの副走査方向に関する分散σ_ｖより大きい。逆に、スキャン画像中の文字行が縦書き文字行である場合には、主走査方向のヒストグラムの副走査方向に関する分散σ_ｖが、副走査方向のヒストグラムの主走査方向に関する分散σ_Ｈより大きい。つまり、分散σ_Ｈと分散σ_ｖとの比較により、スキャン画像中の文字行が縦書き文字行なのか、横書き文字行なのかの判別が可能になっている。
【０１４３】
なお、スキャン画像中の文字行が縦書き文字行なのか、横書き文字行なのかの判別に、黒白反転数ヒストグラムを用いたのは、文字行と写真部分との混同を避けるためである。一般に、黒画素ヒストグラムの値が同程度の場合、文字領域のほうが写真領域よりも黒白反転数ヒストグラムの値が大きくなるからである。
【０１４４】
ここに、文書判別手段の機能が実行される。
【０１４５】
［横書き文字行の座標検出］
以上のようにして文字行を判別した後、まず、各横書き文字行の座標を検出する。横書き文字行の座標の検出にあたっては、文字単位の外接矩形抽出処理を行うとともに、横書き文字行の抽出処理を行う。なお、文字認識処理については周知の技術であるので、その説明は省略する。ここで、スキャン画像の文字外接矩形抽出処理および文字行抽出処理の結果の一例を図２４に示す。そして、各文字の外接矩形の中心点の座標をその文字の座標とみなし、横書き文字行の座標を検出する。
【０１４６】
［最適横書き文字行の選択］
次に、抽出した横書き文字行の中から歪み補正に最適な横書き文字行を選択する。複数の横書き文字行が検出される場合、どの横書き文字行を用いて歪み補正するかを選択する必要がある。最適な横書き文字行の選択基準の一例としては、前述した最適な罫線の選択基準と基本的に同様であって、図２５に示すように横書き文字行の長さＢＣが予め定められた閾値より長く、かつ、綴じ部境界線を挟んだ左右の一定幅領域内（図２５の網掛け領域）に横書き文字行の一部Ｃがかかっていることを条件とし、その中で上下何れかのページ外形に最も近い横書き文字行を選択するようにする。ここで、Ｂは文字行の一番左の矩形の中心であり、Ｃは一番右の矩形の中心である。なお、最適な横書き文字行の選択は、左右ページから各１本ずつのページ外形に最も近い横書き文字行を選択するものであっても良いし、左右ページをさらに上下部分に分け、その各４ブロックにおいて１本ずつのページ外形に最も近い横書き文字行を選択するものであっても良い。
【０１４７】
なお、上記２条件（横書き文字行の長さが予め定められた閾値より長く、かつ、綴じ部境界線を挟んだ左右の一定幅領域内に横書き文字行の一部がかかっている）については、その両方ではなく何れか一方のみを満足するものであっても良い。また、選択基準として上例では「ページ外形に最も近い」を用いているが、これに限るものではなく、「横書き文字行の湾曲が最も大きい」を用いても良い。ここで、「横書き文字行の湾曲」は横書き文字行の両端の文字外接矩形の中心座標の主走査方向の座標値の差で表すものとする。
【０１４８】
［最適横書き文字行の座標値の決定］
最適な横書き文字行が選択された場合には、横書き文字行の（主走査方向の）座標値を決定する。横書き文字行の（主走査方向の）座標値は、横書き文字行内の各文字外接矩形の中心点を連結し、直線部分と曲線部分とを近似して抽出することにより横書き文字行の（主走査方向の）座標値を決定することになる。より詳細には、図２５に示すＤは綴じ部境界線であり、ＢＤの間は多項式近似曲線で（主走査方向の）座標値を推定し、一番左端のＡとＢとの間は近似直線の値で（主走査方向の）座標値を推定する。
【０１４９】
［不適切な横書き文字行の排除］
最後に不適切な横書き文字行を排除する。これは、前述したように多項式近似により座標値を推定する際に、多項式近似による推定曲線の形状が不適切である場合には補正の際にかえって歪みが増大する恐れがあるので、このような横書き文字行を排除するものである。不適切な近似曲線形状の例としては、前述した罫線の場合と同様であって、特に図示しないが、曲線が書籍の外側へ向かうような場合や、中心線を超えて大きく内側へ食い込むような場合である。
【０１５０】
なお、推定曲線の形状が不適切であるとして横書き文字行を排除した場合には、再び最適な横書き文字行を選択し、上記の処理を繰り返すことになる。
【０１５１】
以上の処理により、スキャン画像に横書き文字行が存在すると判断された場合には、左右各ページのいずれの位置に横書き文字行が存在しているのかという情報とともに横書き文字行を抽出し、ＲＡＭ３３に一時的に記憶する。
【０１５２】
［縦書き文字行に基づく横書き文字行の抽出］
次に、各縦書き文字行から横書き文字行を抽出する。
【０１５３】
図２６は、各縦書き文字行からの横書き文字行の抽出処理の流れを概略的に示すフローチャートである。図２６に示すように、まず、縦書き文字行の行切り出し矩形を抽出する（ステップＳ２０１）。なお、縦書き文字行の行切り出し矩形の抽出処理は、ＯＣＲ等で一般に用いられている周知の技術をそのまま利用することができるので、その説明は省略する。図２７は、抽出した行切り出し矩形を例示的に示す説明図である。
【０１５４】
次いで、縦書き文字行の先頭（もしくは末尾）のｙ座標が最大（もしくは最小）の縦書き文字行を抽出し、さらに、そこから予め定めた距離範囲内に先頭（もしくは末尾）が存在する縦書き文字行を抽出する（ステップＳ２０２：第一文字行抽出手段）。より具体的には、図２７に示した例においては、図２８に示すように、縦書き文字行の先頭文字のｙ座標が最大の縦書き文字行はＡで示した縦書き文字行である。そして、その先頭位置から予め定めた距離範囲ｈ内に存在する行先頭文字は、図２８中、黒丸“●”で示した文字である。すなわち、黒丸“●”で示す文字を含む縦書き文字行のみを抽出し、それ以外の縦書き文字行Ｂ，Ｃは除外する。なお、ｈはスキャン画像の解像度によって定められる定数である。
【０１５５】
次に、抽出した縦書き文字行の先頭（もしくは末尾）のｙ座標に関してヒストグラムを構成する（ステップＳ２０３：ヒストグラム構成手段）。図２９では、ページの左端に近い縦書き文字行Ｄを基準行とし、その先頭のｙ座標（ｙ_Ｄ）を基準座標としている。以後、ｙ_Ｄに対して一定幅ｄ（例えば抽出した縦書き文字行の平均幅の１／２）の範囲内に先頭が存在する縦書き文字行の数を、ｙ_Ｄに関するヒストグラムの値とする。図２９では、ｙ_Ｄを示す直線を上下に挟む点線の範囲内に先頭が存在する縦書き文字行がその対象となる。したがって、ページの左端に近い縦書き文字行Ｄの右隣の縦書き文字行Ｅは、その範囲外である。このように、既存の基準座標の対象範囲に先頭が含まれない縦書き文字行が出現した場合は、その縦書き文字行を新たな基準行とし、その先頭座標を新たな基準座標（ここでは、ｙ_Ｅ）とする。また、縦書き文字行Ｅの右隣の縦書き文字行Ｆの行先頭座標はｙ_Ｄの対象範囲に含まれるので、新たな基準座標を設けることなく、ｙ_Ｄに関するヒストグラムの値を１だけカウントアップする。
【０１５６】
以下、同様の処理を綴じ部境界線に向かって続けて行く。その結果、図２９に示す例では、ｙ_Ｄの対象範囲に含まれる縦書き文字行は斜線を施した矩形で囲まれた７つで、ｙ_Ｅの対象範囲に含まれる縦書き文字行は網掛けを施した矩形で囲まれた４つとなる（これら以外の矩形で囲まれた縦書き文字行に関しても、基準行、基準座標と対象範囲がそれぞれ定められるが、図２９では省略している）。なお、ｙ_Ｄの対象範囲には本来無関係であるべき縦書き文字行Ｇも含まれているが、次のステップＳ２０４にてこれは除外される。
【０１５７】
続いて、ステップＳ２０３にて構成したヒストグラムの中で、最大の値に対応する基準行の対象範囲に含まれる縦書き文字行の中で、最もページの左端（もしくは右端）にある縦書き文字行（基準行）を開始行として、綴じ部境界線へ向かって、先頭（もしくは末尾）のｙ座標が近接した縦書き文字行を抽出する（ステップＳ２０４：第二文字行抽出手段）。図２９では、基準座標ｙ_Ｄの対象範囲に含まれる文字行が７つと最大であったので、その中の左端の縦書き文字行Ｄを開始行とし、開始行（縦書き文字行Ｄ）から綴じ部境界線へ向かって先頭のｙ座標が近接した縦書き文字行を抽出していく。
【０１５８】
ところで、開始行（縦書き文字行Ｄ）から綴じ部境界線へ向かって先頭のｙ座標が近接した縦書き文字行を抽出する際には、画像の歪みを生じていない部分と歪みを生じている部分とで処理内容を切り替える。
【０１５９】
まず、画像の歪みを生じていない部分における処理について図３０を参照しつつ説明する。画像の歪みを生じていない部分では、着目行Ｈを基準として、次の２条件を満足する縦書き文字行を抽出する。
１．ｙ座標の正方向（図３０中、上方へ向かう方向）に関して、着目行Ｈの先頭位置から一定範囲内ｂ１（例えば平均文字行幅の１／２）に抽出する縦書き文字行の先頭が存在すること
２．ｙ座標の負方向（図３０中、下方へ向かう方向）に関して、着目行Ｈの先頭位置から見てｘ座標の正方向（綴じ部境界線へ向かう方向）に対して予め定めた一定角度（ここでは、角度を直線の傾き（ｂ２／ａ１）で表している）の範囲内に抽出する縦書き文字行の先頭が存在すること
すなわち、着目行Ｈの次の縦書き文字行Ｉの先頭は上記の範囲外なので除外することになるが、さらに次の縦書き文字行Ｊの先頭は範囲内に存在するので抽出することになる。以下、縦書き文字行Ｊを新たな着目行として同様の処理を続ける。
【０１６０】
次に、画像の歪みを生じている部分における処理について図３１を参照しつつ説明する。画像の歪みを生じている部分では、着目行Ｌを基準として、次の２条件を満足する縦書き文字行を抽出する。
１．ｙ座標の負方向（図３１中、下方へ向かう方向）に関して、着目行Ｌの先頭位置から見てｘ座標の正方向（綴じ部境界線へ向かう方向）に対して予め定めた一定角度（ここでは、角度を直線の傾き（ｂ３／ａ２）で表しているが、歪みを生じている部分では基本的にページの内側へ文字行の先頭が食い込んでいくのを考慮して、ｂ２／ａ１＜ｂ３／ａ２とする）の範囲内に抽出する縦書き文字行の先頭が存在すること
２．着目行Ｌの先頭位置と抽出する縦書き文字行の先頭位置を結ぶ直線の傾き（ｂ４／ａ２）が、着目行Ｌの先頭位置と直前の抽出行Ｋの先頭位置を結ぶ直線の傾き（ｂ５／ａ３）から一定値αを引いた値よりも大きい。すなわち、“ｂ４／ａ２＞ｂ５／ａ３−α”を満足すること（基本的には、“ｂ４／ａ２＞ｂ５／ａ３”で良いが、誤差を考慮して一定値αを導入する。一定値αは予め定めた値である）
すなわち、着目行Ｌの次の縦書き文字行Ｍの先頭はこの条件外なので除外することになるが、さらに次の縦書き文字行Ｎの先頭は条件を満足するので抽出することになる。以下、縦書き文字行Ｎを新たな着目行として同様の処理を続ける。
【０１６１】
さて、ここで問題となるのは、歪みを生じていない部分と歪みを生じている部分をいかに識別するかであるが、これは次のように行っている。すなわち、着目行と次の抽出行の先頭のｙ座標をそれぞれｙ_Ｃ，ｙ_Ｎとすると、“ｙ_Ｎ−ｙ_Ｃ”が一定値（例えば、平均文字行幅の１／４）以上となれば、それ以降を歪みを生じている部分とする。
【０１６２】
以上の方法により図２９から抽出した縦書き文字行を、図３２において斜線を施した矩形で囲んで示す。
【０１６３】
最後に、抽出した縦書き文字行の先頭（もしくは末尾）の位置座標に関する近似曲線多項式を算出する（ステップＳ２０５）。抽出した各縦書き文字行の行切り出し矩形の先頭を連結して外形を形成する場合には、図３３に示すように、連結する各縦書き文字行の行切り出し矩形の上辺中心点に基づき、抽出した縦書き文字行の先頭の位置座標に関する近似曲線多項式を算出する。また、抽出した各縦書き文字行の行切り出し矩形の末尾を連結して外形を形成する場合には、図３３に示すように、連結する各縦書き文字行の行切り出し矩形の下辺中心点に基づき、抽出した縦書き文字行の末尾の位置座標に関する近似曲線多項式を算出する。
【０１６４】
なお、最後に不適切な縦書き文字行の外形を排除する。これは、前述したように多項式近似により座標値を推定する際に、多項式近似による推定曲線の形状が不適切である場合には補正の際にかえって歪みが増大する恐れがあるので、このような縦書き文字行の外形を排除するものである。不適切な近似曲線形状の例としては、前述した罫線や横書き文字行の場合と同様であって、特に図示しないが、曲線が書籍の外側へ向かうような場合や、中心線を超えて大きく内側へ食い込むような場合である。
【０１６５】
なお、推定曲線の形状が不適切であるとして縦書き文字行の外形を排除した場合には、歪み補正用の縦書き文字行の外形は無いということになる。
【０１６６】
以上の処理により、スキャン画像に縦書き文字行の外形が存在すると判断された場合には、左右各ページのいずれの位置に縦書き文字行の外形が存在しているのかという情報とともに縦書き文字行の外形を抽出し、ＲＡＭ３３に一時的に記憶する。
【０１６７】
なお、以下においては、横書き文字行及び縦書き文字行の外形を文字行として扱うものとする。
【０１６８】
以上、ステップＳ４１〜Ｓ４３の処理により、ページ外形／罫線／文字行の抽出処理（ステップＳ４）が終了する。
【０１６９】
続くステップＳ５（図６参照）においては、画像歪み補正処理を実行する。画像歪み補正処理は、図３４に示すように、概略的には、歪み補正（伸長）に際しての基準となる線（基準線）としてスキャン画像の上辺（もしくは下辺）の近傍に位置するページ外形／罫線／文字行の何れかを選択する処理（ステップＳ５１：基準線選択処理）、基準線に対応するものであって補正率（伸長率）の算出用の参照線としてスキャン画像の上辺（もしくは下辺）の近傍に位置するページ外形／罫線／文字行の何れかを選択する処理（ステップＳ５２：参照線選択処理）、基準線が罫線や文字行の場合に、基準線より下部の画像情報の欠落を最小限にするための仮想的なページ外形を算出する処理（ステップＳ５３：仮想ページ外形算出処理）、仮想的なページ外形に基づいてスキャン画像に伸長処理を施して主走査方向の歪みを補正する処理（ステップＳ５４：主走査方向歪み補正処理）、補正画像の文字外接矩形に基づいてスキャン画像に伸長処理を施して副走査方向の歪みを補正する処理（ステップＳ５５：副走査方向歪み補正処理）により構成されている。
【０１７０】
ここで、基準線選択処理（ステップＳ５１）及び参照線選択処理（ステップＳ５２）においては、基準線または参照線として、スキャン画像の上辺（もしくは下辺）の近傍に位置するページ外形／罫線／文字行の何れかを選択することになるが、本実施の形態におけるページ外形、罫線、文字行の選択の優先順位は、
ページ外形＞罫線＞文字行
とされている。このような選択優先順位にしたのは、文字行はページ外形や罫線に比べて抽出精度が低く、また、画像の外側にあるページ外形を利用する方が精度の高い歪み補正率を得ることができるためである。ただし、本実施の形態においては、ページ外形は一般に左右ページにまたがるが、罫線や文字行は両ページにあるとは限らないので、スキャン画像の上辺または下辺からの基準線選択の優先順位は、下記に示すようになる。
【０１７１】
１．「ページ外形」（ページ外形は、常に左右両ページに存在する）
２．左右両ページともに「罫線」
３．一方のページが「罫線」、他方のページが「文字行」
４．左右両ページともに「文字行」
５．一方のページのみに「罫線」、他方のページには「手がかりなし」
６．一方のページのみに「文字行」、他方のページには「手がかりなし」
ここで、「手がかりなし」とは、ページ外形、罫線、文字行のいずれも抽出できなかった場合を言う。
【０１７２】
次に、仮想ページ外形算出処理（ステップＳ５３）及び主走査方向歪み補正処理（ステップＳ５４）の例について、上記優先順位に基づいて順に説明する。
【０１７３】
１．スキャン画像の上辺及び下辺のいずれにも「ページ外形」が存在する場合ここでは、スキャン画像の上辺及び下辺のいずれにも「ページ外形」が存在する場合について説明する。図３５に示すように、スキャン画像の上辺及び下辺のいずれにも「ページ外形」が存在する場合は、いずれを基準線、参照線にしても構わない。なお、本実施の形態においては、基準線として選択した「ページ外形」、「罫線」、「文字行」は、下辺に位置させるものとする。この場合においては、基準線と仮想ページ外形とは一致していることから、仮想ページ外形算出処理（ステップＳ５３）においては特に処理を実行しなくても良い。
【０１７４】
次いで、主走査方向歪み補正処理（ステップＳ５４）について説明する。ここで、ｘ０は「ページ外形」の直線部分と曲線部分との境界点である。主走査方向歪み補正処理としては、まず、この境界点ｘ０における基準線と参照線間の距離ｈ０（図３６参照）に基づいて歪み補正率を算出する。歪み補正率は、主走査方向に画像を伸長して補正する際に、基準線と参照線との間の距離を副走査方向の全ての位置において等しくさせるためのものである。すなわち、位置ｘにおける基準線と参照線間の距離をｈ（図３６参照）とした場合には、位置ｘにおける歪み補正率は、
ｈ０／ｈ
として表すことができる。即ち、歪み補正率とは、主走査方向に画像を伸長して補正する際に、基準線と参照線との間の距離を副走査方向の全ての位置ｘにおいて等しくなるように、各ｘに関して算出した値である。
【０１７５】
次に、実際に補正をする際は、まず、図３６に示すように仮想ページ外形（ここでは基準線）が直線（最下辺）になるように、主走査方向に各画素をシフトする。その後、副走査方向の全ての位置ｘにおいて、画像を主走査方向の上辺側にｈ０／ｈで伸長すれば、図３６の上辺の「ページ外形」ＥがＲＥとなるように画像が補正される。
【０１７６】
ここに、ページ外形画像歪み補正手段の機能が実行される。
【０１７７】
２．スキャン画像の上辺及び下辺の何れか一方に「ページ外形」が存在し、他方の辺には「罫線」と「文字行」とが存在する場合
ここでは、スキャン画像の上辺及び下辺の何れか一方に「ページ外形」が存在し、他方の辺ではページ外形が途中で切れているが「罫線」と「文字行」とが存在する場合について説明する。図３７に示すように、スキャン画像の上辺及び下辺の何れか一方に「ページ外形」が存在し、他方の辺には「罫線」と「文字行」とが存在する場合は、「ページ外形」を基準線として下辺に位置させ、「文字行」と「罫線」とを参照線とする（図３７に示す例では、左ページが「文字行」、右ページが「罫線」である）。この場合においても、基準線と仮想ページ外形とは一致していることから、仮想ページ外形算出処理（ステップＳ５３）においては特に処理を実行しなくても良い。
【０１７８】
次いで、主走査方向歪み補正処理（ステップＳ５４）について説明する。ここで、ｘ０は「ページ外形」の直線部分と曲線部分との境界点である。主走査方向歪み補正処理としては、まず、この境界点ｘ０における基準線と参照線間の距離ｈ０（図３８参照）に基づいて歪み補正率を算出する。位置ｘにおける基準線と参照線間の距離をｈ（図３８参照）とした場合には、位置ｘにおける歪み補正率は、
ｈ０／ｈ
として表すことができる。
【０１７９】
次に、実際に補正をする際は、まず、図３８に示すように仮想ページ外形（ここでは基準線）が直線（最下辺）になるように、主走査方向に各画素をシフトする。その後、副走査方向の全ての位置ｘにおいて、画像を主走査方向の上辺側にｈ０／ｈで伸長すれば、図３８の右ページでは、上辺の「罫線」ＬＵがＲＬＵとなるように画像が補正される。左ページ上辺の「文字行」についても同様に行う。
【０１８０】
ここに、ページ外形画像歪み補正手段の機能が実行される。
【０１８１】
３．スキャン画像の左右両ページの上辺及び下辺の何れか一方に「罫線」が存在し、他方の辺には「罫線」と「文字行」とが存在する場合
ここでは、スキャン画像の左右両ページの上辺及び下辺の何れか一方に「罫線」が存在し、他方の辺には「罫線」と「文字行」とが存在する場合について説明する。スキャン画像の左右両ページの上辺及び下辺の何れか一方に「罫線」が存在し、他方の辺には「罫線」と「文字行」とが存在する場合は、図３９に示すように、左右両ページに「罫線」が存在する側を下辺に位置させ、その２本の「罫線」を基準線とし、他方の辺に位置する「文字行」と「罫線」とを参照線とする（図３９に示す例では、左ページが「文字行」、右ページが「罫線」である）。ところで、このように、基準線が「ページ外形」ではない場合は、基準線をそのまま仮想ページ外形と見なしてはならない。なぜなら、両者を一致させた場合、後の主走査方向歪み補正処理において仮想ページ外形（基準線）を最下辺に画素シフトする際、基準線より下方の画像情報が全て欠落してしまうからである。
【０１８２】
そこで、基準線が「ページ外形」でない場合は、図３９において点線で示すような仮想ページ外形ＶＥを求める仮想ページ外形算出処理（ステップＳ５３）を実行する。ここで、ｘ０は下辺の「罫線」ＬＤの直線部分と曲線部分との境界点である。仮想ページ外形算出処理は、境界点ｘ０における「罫線」ＬＤからスキャン画像の最下辺までの距離ａ０と、境界点ｘ０における「罫線」ＬＤから上辺の「罫線」ＬＵまでの距離ｂ０とに基づいて仮想ページ外形ＶＥを算出するものである。すなわち、「罫線」ＬＤ上の位置ｘから上辺の「罫線」ＬＵまでの距離ｂが解かれば、
ａ／ｂ＝ａ０／ｂ０
であることから、「罫線」ＬＤ上の位置ｘから仮想ページ外形ＶＥまでの距離ａを算出することができる。したがって、「罫線」ＬＤ上の位置から仮想ページ外形ＶＥまでの距離を副走査方向の全ての位置ｘにおいて算出することにより、仮想ページ外形ＶＥを求めることができる。このような処理は、左右ページについてそれぞれ独立に行われる。
【０１８３】
次いで、主走査方向歪み補正処理（ステップＳ５４）について説明する。主走査方向歪み補正処理としては、まず、前述した境界点ｘ０における「罫線」ＬＤからスキャン画像の最下辺までの距離ａ０と、境界点ｘ０における「罫線」ＬＤから上辺の「罫線」ＬＵまでの距離ｂ０とを加算した距離ｈ０（ｈ０＝ａ０＋ｂ０）に基づいて歪み補正率を算出する。位置ｘにおける上辺の「罫線」ＬＵからスキャン画像の最下辺までの距離をｈ（ｈ＝ａ＋ｂ）とした場合には、位置ｘにおける歪み補正率は、
ｈ０／ｈ
として表すことができる。
【０１８４】
次に、実際に補正をする際は、まず、図４０に示すように仮想ページ外形ＶＥが直線（最下辺）になるように、主走査方向に各画素をシフトする。その後、副走査方向の全ての位置ｘにおいて、画像を主走査方向の上辺側にｈ０／ｈで伸長すれば、図３４の右ページでは、上辺の「罫線」ＬＵがＲＬＵ（厳密には、ｘの位置は「罫線」ＬＵの直線部と曲線部との境界には一致しないが、ここでは「罫線」ＬＤの境界をそのまま用いる）となるように画像が補正される。左ページ上辺の「文字行」についても同様に行う。
【０１８５】
このように、仮想ページ外形ＶＥを利用することで、シフトによる画像の欠落を最小限に抑えることができる。
【０１８６】
ここに、罫線／文字行画像歪み補正手段の機能が実行される。
【０１８７】
４．スキャン画像の左右両ページの上辺及び下辺の何れか一方に「罫線」と「文字行」とが存在し、他方の辺の一方のページのみに「罫線」が存在する場合
ここでは、スキャン画像の左右両ページの上辺及び下辺の何れか一方に「罫線」と「文字行」とが存在し、他方の辺の一方のページのみに「罫線」が存在する場合について説明する。スキャン画像の左右両ページの上辺及び下辺の何れか一方に「罫線」と「文字行」とが存在し、他方の辺の一方のページのみに「罫線」が存在する場合（他方のページは「手がかりなし」）は、図４１に示すように、「罫線」と「文字行」とが存在する側を下辺に位置させ、それらの「罫線」と「文字行」とを基準線とし、他方の辺に位置する「罫線」とスキャン画像を副走査方向に貫く中心線Ｃとを参照線とする（図４１に示す例では、左ページが「罫線」、右ページが「手がかりなし」である）。なお、中心線Ｃは、スキャン画像の主走査方向の中心を副走査方向に横切る線であり、ブック原稿４０の中心線ではない。
【０１８８】
このように基準線が「ページ外形」でない場合は、前述したように、図４１において点線で示すような仮想ページ外形ＶＥを求める仮想ページ外形算出処理（ステップＳ５３）を実行する。ここで、ｘ０は下辺の「罫線」ＬＤの直線部分と曲線部分との境界点である。仮想ページ外形算出処理は、境界点ｘ０における「罫線」ＬＤからスキャン画像の最下辺までの距離ａ０と、境界点ｘ０における「罫線」ＬＤから中心線Ｃまでの距離ｂ０とに基づいて仮想ページ外形ＶＥを算出するものである。すなわち、「罫線」ＬＤ上の位置ｘから中心線Ｃまでの距離ｂが解かれば、
ａ／ｂ＝ａ０／ｂ０
であることから、「罫線」ＬＤ上の位置ｘから仮想ページ外形ＶＥまでの距離ａを算出することができる。したがって、「罫線」ＬＤ上の位置から仮想ページ外形ＶＥまでの距離を副走査方向の全ての位置ｘにおいて算出することにより、仮想ページ外形ＶＥを求めることができる。このような処理は、左右ページについてそれぞれ独立に行われる。なお、右ページのように上辺に「罫線」が存在する場合については、中心線Ｃに代えて、前述したように上辺の「罫線」を用いる。
【０１８９】
次いで、主走査方向歪み補正処理（ステップＳ５４）について説明する。主走査方向歪み補正処理としては、まず、前述した境界点ｘ０における「罫線」ＬＤからスキャン画像の最下辺までの距離ａ０と、境界点ｘ０における「罫線」ＬＤから中心線Ｃまでの距離ｂ０とを加算した距離ｈ０（ｈ０＝ａ０＋ｂ０）に基づいて歪み補正率を算出する。位置ｘにおける中心線Ｃからスキャン画像の最下辺までの距離をｈ（ｈ＝ａ＋ｂ）とした場合には、位置ｘにおける歪み補正率は、
ｈ０／ｈ
として表すことができる。
【０１９０】
次に、実際に補正をする際は、まず、図４２に示すように仮想ページ外形ＶＥが直線（最下辺）になるように、主走査方向に各画素をシフトする。その後、副走査方向の全ての位置ｘにおいて、画像を主走査方向の上辺側にｈ０／ｈで伸長して補正する。左ページ上辺の「罫線」についても同様に行う。なお、右ページについては、補正によってシフトにより一旦歪んだ中心線Ｃが元の直線ＲＣに戻るが、Ｃがスキャナ部１の光軸と一致する場合以外は、画像全体の補正は不完全となる。
【０１９１】
ここに、罫線／文字行画像歪み補正手段の機能が実行される。
【０１９２】
５．スキャン画像の左右何れかのページの上辺及び下辺の何れか一方に「罫線」が存在し、他方の辺の他方のページのみに「文字行」が存在する場合
ここでは、スキャン画像の左右何れかのページの上辺及び下辺の何れか一方に「罫線」が存在し、他方の辺の他方のページのみに「文字行」が存在する場合について説明する。スキャン画像の左右何れかのページの上辺及び下辺の何れか一方に「罫線」が存在し（他方のページは「手がかりなし」）、他方の辺の他方のページのみに「文字行」が存在する場合（一方のページは「手がかりなし」）は、図４３に示すように、「罫線」が存在する側を下辺に位置させ、その「罫線」を基準線とし、「罫線」が存在しない方のページ（「文字行」が存在する方のページ）については、文字行Ｌを中心線Ｃを挟んで線対称の位置に移動させた曲線ＳＬを基準線とする。参照線については、「罫線」が存在するページは中心線Ｃを、「文字行」が存在するページはその「文字行」を、それぞれ参照線とする。
【０１９３】
なお、歪み補正率の算出や画素シフト、補正（伸長）処理についてはこれまでの例と同様であるため、それらの説明は省略する。
【０１９４】
ここに、罫線／文字行画像歪み補正手段の機能が実行される。
【０１９５】
最後に、副走査方向歪み補正処理（ステップＳ５５）について説明する。ここで、図４４は副走査方向歪み補正処理の流れを概略的に示すフローチャートである。図４４に示すように、ステップＳ１０１においては、主走査方向歪み補正されたスキャン画像に基づいて文字の外接矩形Ａ（図４５参照）を抽出する。ここで、文字認識処理については周知の技術であるので、その説明は省略する。このように文字の外接矩形Ａを抽出するのは、この文字の外接矩形Ａの形状の変化を基に、副走査方向の歪みを補正するためである。ここでは、図４５に示すように、文字外接矩形Ａの横辺の長さｗ、縦辺の長さｈ、および、文字の中心Ｂを定義する。ここで、文字の中心Ｂは、外接矩形Ａの対角線の交点である。
【０１９６】
続いて、図４６に示すように、スキャン画像をブック原稿４０のページ綴じ部４１に平行な方向の複数の短冊状の領域Ｃに分割した後（ステップＳ１０２）、各短冊領域Ｃについて、そこに含まれる文字外接矩形Ａに関する特徴量を求める（ステップＳ１０３）。ここで、ある短冊領域Ｃに含まれる文字外接矩形Ａとは、その中心が当該短冊領域Ｃに含まれるような外接矩形Ａのことである。例えば、図４６の短冊領域Ｃ１に含まれる外接矩形Ａは、図中の網掛けを施した矩形である。
【０１９７】
さて、上記の文字外接矩形Ａに関する特徴量は、
（文字の横辺の長さ）／（文字の縦辺の長さ）＝ｗ／ｈ
を基に求められる。すなわち、各短冊領域Ｃについて、そこに含まれる全ての文字外接矩形Ａのｗ／ｈの値の平均値をその短冊領域Ｃの特徴量とするのである。
【０１９８】
しかしながら、単に、ｗ／ｈの平均値を算出すると不適切な場合がある。文字の中には、句読点や数式中の記号のようにそのサイズが元々小さく、ｗ／ｈの値が不安定なものがある。また、矩形抽出の際に隣接する文字同士がくっついて抽出されてしまい、ｗが極端に大きい文字外接矩形Ａが生じる場合もある。特徴量を求める場合は、このような特殊な文字や極端にｗが大きいものを予め排除しておく必要がある。そこで、続くステップＳ１０４においては、予め閾値を定めておいて、ｈの値がその閾値より小さな文字外接矩形Ａを予め排除するとともに、ｗ／ｈの比率に関する閾値を予め定めておき、ｗ／ｈの値が其の閾値よりも大きい文字外接矩形Ａも予め排除する。例えば、図４７中に示した網掛けを施した文字外接矩形Ａが予め排除されることになる。
【０１９９】
続くステップＳ１０５においては、前述したように極端な文字外接矩形Ａを排除した後に、各短冊領域Ｃ内の文字外接矩形Ａのｗ／ｈの平均値を求める。図４８に各短冊領域Ｃ内の外接矩形Ａのｗ／ｈの平均値の一例を示す。なお、図４８中の短冊領域Ｃ２は、ブック原稿４０のページ綴じ部４１を含む短冊領域である。
【０２００】
続いて、ブック原稿４０のページ綴じ部４１を含む短冊領域Ｃ２に文字外接矩形Ａが存在するか否かを判断する（ステップＳ１０６）。これは、図４７に示すように、一般にはブック原稿４０のページ綴じ部４１付近には文字外接矩形Ａが存在しない場合が多いからである。ブック原稿４０のページ綴じ部４１を含む短冊領域Ｃ２に文字外接矩形Ａが存在する場合は（ステップＳ１０６のＹ）、その文字外接矩形Ａを利用して特徴量は算出されているのでそのままステップＳ１０８に進む。
【０２０１】
一方、ブック原稿４０のページ綴じ部４１を含む短冊領域Ｃ２に文字外接矩形Ａが存在しない場合は（ステップＳ１０６のＮ）、ステップＳ１０７に進み、ブック原稿４０のページ綴じ部４１を含む短冊領域Ｃ２の特徴量を求める。なお、ブック原稿４０のページ綴じ部４１を含む短冊領域Ｃ２の識別は、例えば、スキャン画像（例えば、モノクロ多値画像）の地肌濃度変化を各短冊領域Ｃごとに求め、短冊領域Ｃ内の最も濃度の薄い濃度値を求めることにより実現される。図４９は地肌濃度変化を求めた一例を示したものであり、地肌濃度が最も濃い短冊領域が、ブック原稿４０のページ綴じ部４１を含む短冊領域Ｃ２であるとみなされる。
【０２０２】
なお、スキャン画像がカラー多値画像の場合におけるブック原稿４０のページ綴じ部４１を含む短冊領域Ｃ２の識別は、例えばＲＧＢ成分の何れか一つの成分（例えばＧ成分）に着目し、そのＧ成分の地肌濃度を使用して識別するようにすれば良い。また、ＲＧＢを色変換して輝度成分と色差成分とに分け、輝度成分を使用してブック原稿４０のページ綴じ部４１を含む短冊領域Ｃ２を識別するようにしても良い。
【０２０３】
ブック原稿４０のページ綴じ部４１を含む短冊領域Ｃ２の特徴量は、次のようにして定められる。ここでは、統計的特徴量の算出対象となり得る文字外接矩形Ａが存在し、かつ、ページ綴じ部４１を含む短冊領域Ｃ２の最近傍である短冊領域Ｃの特徴量に対して予め定めた定数値を乗じることにより算出された値が、ブック原稿４０のページ綴じ部４１を含む短冊領域Ｃ２における特徴量とみなされるものである。つまり、図４８に示した例では、ブック原稿４０のページ綴じ部４１を含む短冊領域Ｃ２の左右何れの短冊領域Ｃ３，Ｃ４にも文字外接矩形Ａが存在するので、どちらか適当な方の特徴量を選択し（ここでは右側の○印の方）、それに予め定めた定数値（ここでは０．５）を乗じて、これをブック原稿４０のページ綴じ部４１を含む短冊領域Ｃ２の特徴量としている。
【０２０４】
続くステップＳ１０８においては、各短冊領域Ｃの特徴量に対する適切なフィルタリング処理、例えば、短冊領域Ｃの位置の変化方向（即ち副走査方向）に関する移動平均を求める処理を施すなどして、短冊領域Ｃの位置の変化に対する（副走査方向の）特徴量の変化がなだらかになるようにする。ただし、ここでもブック原稿４０のページ綴じ部４１付近は特別な処理が必要となる。なぜなら、副走査方向に関して長さが全て等しいウィンドウを用いてフィルタリングを行うと、ブック原稿４０のページ綴じ部４１付近の特徴量の変化の鋭さが失われてしまうからである。
【０２０５】
ここで、長さが全て５であるウィンドウを用いて図４８に示した各短冊領域Ｃの特徴量に対してフィルタリング処理を施した結果を図５０に示す。図５０に示すように、長さが全て５であるウィンドウを用いてフィルタリング処理を施した場合には、ブック原稿４０のページ綴じ部４１付近の特徴量（ｗ／ｈ）の変化がなだらかになり過ぎてしまう。このような場合には、ブック原稿４０のページ綴じ部４１付近での適切な画像補正が不可能になってしまう。
【０２０６】
そこで、本実施の形態においては、フィルタリング処理の際には、フィルタのウィンドウがブック原稿４０のページ綴じ部４１を含む短冊領域Ｃ２の両側の短冊領域Ｃ３，Ｃ４に跨がらないように、ページ綴じ部４１付近でウィンドウ長を調整する。ここで、図５１はページ綴じ部４１付近でウィンドウ長を調整してフィルタリング処理を施した結果を示すグラフである。図５１に示すように、ページ綴じ部４１付近でウィンドウ長を調整した場合には、ページ綴じ部４１付近の特徴量（ｗ／ｈ）の変化を適切に表現できるので、良好な画像補正が実現できる。
【０２０７】
続くステップＳ１０９においては、各短冊領域Ｃの推定歪み量を算出する。各短冊領域Ｃの推定歪み量の算出手法は、以下に示す通りである。
【０２０８】
まず、短冊領域の歪み量を算出するための基準となる短冊領域（基準短冊領域）を定める。ここでは、歪みが最も小さいと考えられる短冊領域Ｃ、例えば、特徴量（ｗ／ｈ）が最大である短冊領域Ｃを基準短冊領域とする。この処理は左右ページで共通に行っても良いが、左右独立に基準短冊領域を定めても構わない。図５１においては、左右独立に基準短冊領域を定めた例を示しており、○印を施した短冊領域Ｃが基準短冊領域であり、左側の基準特徴量を“Ｌｗ０／Ｌｈ０”、右側の基準特徴量を“Ｒｗ０／Ｒｈ０”、でそれぞれ示している。
【０２０９】
次に、基準短冊領域の特徴量ｗ０／ｈ０をスキャン画像全体の基準特徴量とし、
（各短冊領域の特徴量）／（基準特徴量）＝（ｗ／ｈ）／（ｗ０／ｈ０）
の値を、各短冊領域の推定歪み量として算出する。
【０２１０】
なお、ブック原稿４０のページ綴じ部４１から外れたページ外側付近の短冊領域Ｃを基準短冊領域とすると、ブック原稿４０のページ綴じ部４１付近とはフォントや活字のサイズの違いが大きくて、適切な推定歪み量が算出できない場合も考えられる。そのような画像を対象とする場合は、基準短冊領域の探索範囲を予めブック原稿４０のページ綴じ部４１付近に限定しておくのが有効である。これを実現するためには、地肌濃度が予め定めた濃度よりも濃い短冊領域Ｃのみから基準短冊領域を求めるようにすれば良い。
【０２１１】
最後に、スキャン画像に対して、短冊領域Ｃの短辺方向（副走査方向）の拡大処理を行い、ブック原稿４０のページ綴じ部４１付近の歪みを補正する（ステップＳ１１０）。その場合の拡大率は、ステップＳ１０９において算出した推定歪み量の逆数、すなわち、
（基準特徴量）／（各短冊領域の特徴量）＝（ｗ０／ｈ０）／（ｗ／ｈ）
とする。ここで、上記の基準短冊領域を左右共通に定めた場合には、この拡大率も左右共通の基準特徴量によって算出し、独立に定めた場合には、左右それぞれの基準特徴量で独立に算出するようにする。図５２は、図５１に示した特徴量に基づいて算出した補正拡大率を示したものである。
【０２１２】
なお、ここでも、ブック原稿４０のページ綴じ部４１付近から離れた短冊領域Ｃはもともと画像の歪みが無い領域である可能性が高いので、拡大処理の対象としない方が良い場合がある。拡大処理を行ったために、かえって不自然な歪みが生じてしまう可能性があるからである。これを防ぐために、地肌濃度が予め定めた濃度よりも薄い短冊領域Ｃについては、推定歪み量を“１”とする。
【０２１３】
また、短冊領域Ｃ内において共通の補正拡大率を適用した場合、隣接する短冊領域Ｃの境界部での補正拡大率が不連続となるため、補正画像が不自然となる。そこで、隣接する短冊領域Ｃの境界部での補正拡大率が連続的に変化するように、補正拡大率を予め補正しておく。これは、例えば図５２に示す短冊領域Ｃの中央部分の補正拡大率を推定歪み量の逆数を示す点としてプロットし、これらの点を線分で結んで直線補完することで、他の部分の補正拡大率とすることで実現できる。以上の処理により、スキャン画像の副走査方向の補正拡大率が確定する。
【０２１４】
なお、画像の拡大処理は、例えばコピーの変倍機能として良く使われる３次関数のコンボリューション法等を用いて実行される。
【０２１５】
以上、ステップＳ１０１〜Ｓ１１０の処理により副走査方向歪み補正処理（ステップＳ５５）が終了し、図６に示すスキャン画像の歪み補正処理が終了する。ここで、図５３は歪みを補正した画像を示す平面図である。以上の処理によれば、図８に示したようなブック原稿４０のページ綴じ部４１の近傍において生じていたスキャン画像の歪みが、図５３に示すように補正されることになる。
【０２１６】
ここに、多項式近似曲線の算出に用いる縦書き文字行が、縦書き文字行の文字行先頭（もしくは文字行末尾）の位置情報に基づいて一定の位置関係を有するものに限定して抽出される。これにより、縦書き文字行の文字行先頭（もしくは文字行末尾）の位置情報に基づいて一定の位置関係を有するものに限定して縦書き文字行を抽出することが可能になるので、縦書き文字行の一番上（もしくは一番下）の文字の位置が行ごとに上下方向に大きく変動しているような場合にも良好な補正結果を得ることが可能になる。
【０２１７】
なお、本実施の形態においては、ブック原稿の見開きページの左上の部分に関して説明したが、それ以外の左下、右上、右下の部分についても基本的に同様の処理を行う。ただし、その場所に応じて、行の先頭文字を末尾の文字と読み換えたり、ページの左端を右端に、ｙ座標が最大の行を最小の行に、ｙ座標の正（負）方向を負（正）方向に、それぞれ読み換える必要がある。
【０２１８】
次に、本発明の第二の実施の形態について説明する。なお、第一の実施の形態において説明した部分と同一部分については同一符号を用い、説明も省略する。本実施の形態は、図２６で説明したステップＳ２０２の処理内容が異なるものである。
【０２１９】
本実施の形態は、ステップＳ２０２で抽出した縦書き文字行の数が予め定めた値より少ない場合には、ステップＳ２０５で抽出行の先頭位置に関する安定した近似曲線多項式を算出できないので、抽出した縦書き文字行を除外した残りの行に関して、再び同様の縦書き文字行の抽出処理を行うようにしたものである。
【０２２０】
ここで、図５４は本実施の形態の縦書き文字行の抽出処理を示す説明図である。ここでは、抽出すべき最小文字行数を５としておく。図５４（ａ）中、行の先頭文字のｙ座標が最大の縦書き文字行はＡで示した縦書き文字行である。そして、その先頭位置から予め定めた距離範囲ｈ内に存在する行先頭文字は、図５４（ａ）に黒丸“●”で示した文字であるが、その数が３個と少ない。そこで、本実施の形態においては、抽出した３個の先頭文字を含んだ縦書き文字行を除外して、もう一度同様の処理を行う。すなわち、図５４（ｂ）に示すように、残りの縦書き文字行の中で先頭文字のｙ座標が最大の縦書き文字行Ｂを求め、縦書き文字行Ｂの先頭位置から予め定めた距離範囲ｈ内に存在する行先頭文字（黒丸“●”で示す）を改めて抽出する。図５４（ｂ）に示すように、距離範囲ｈ内に存在する行先頭文字（黒丸“●”で示す）として、１２個が抽出される。
【０２２１】
したがって、本実施の形態においては、この１２個の行先頭文字を含む１２行の縦書き文字行に従って、ステップＳ２０３以降の処理を実行することになる。
【０２２２】
ここに、抽出した縦書き文字行の数が少ない場合には、安定した近似曲線多項式を算出することが難しいため、再度縦書き文字行の抽出処理を行うことにより、安定した近似曲線多項式の算出を図ることが可能になる。
【０２２３】
なお、ステップＳ２０４で抽出した縦書き文字行の数が予め定めた値より少ない場合にも、抽出した縦書き文字行を除外した残りの行に関して、再び同様の縦書き文字行の抽出処理を行うようにしても良い。
【０２２４】
次に、本発明の第三の実施の形態について説明する。なお、第一の実施の形態において説明した部分と同一部分については同一符号を用い、説明も省略する。本実施の形態は、図２６で説明したステップＳ２０４の処理内容が異なるものである。
【０２２５】
本実施の形態は、ステップＳ２０４において、綴じ部境界線から一定範囲内に抽出した縦書き文字行が存在しない場合は、綴じ部境界線付近において安定した近似曲線多項式を算出できないので、抽出した縦書き文字行を除外した残りの行に関して、再び同様の縦書き文字行の抽出処理を行うようにしたものである。
【０２２６】
ここで、図５５は本実施の形態の縦書き文字行の抽出処理を示す説明図である。ここでは、綴じ部境界線からの一定範囲Ｗを画像全体の横幅の１／８としておく。図５５（ａ）はステップＳ２０３において構成したヒストグラム情報に対応する。図５５（ａ）に示すように、縦書き文字行Ａの先頭座標ｙ_Ａの対象範囲に含まれる縦書き文字行が７行と最も多く、次に多いのが縦書き文字行Ｂの先頭座標ｙ_Ｂの対象範囲に含まれる縦書き文字行であって４行である。そこで、縦書き文字行Ａを開始行として先頭のｙ座標が近接した縦書き文字行を抽出していく（第一の実施の形態と同様）。その結果、抽出された縦書き文字行を、図５５（ｂ）において斜線を施した矩形で囲んで示す。図５５（ｂ）に示すように、この場合、綴じ部境界線から一定範囲Ｗ内には抽出された縦書き文字行が存在しない。そこで、本実施の形態においては、一定範囲Ｗ内に抽出された縦書き文字行が存在しない場合には、抽出した縦書き文字行を除外して、もう一度同様の処理を行う。すなわち、図５５（ａ）において２番目にヒストグラムの値が大きい対象範囲（先頭座標ｙ_Ｂ）に対応する縦書き文字行Ｂを開始行として、行抽出作業を再度行う。その結果、抽出された縦書き文字行を、図５５（ｃ）において斜線を施した矩形で囲んで示す。図５５（ｃ）に示すように、この場合には、綴じ部境界線から一定範囲Ｗ内に抽出された縦書き文字行が存在する。
【０２２７】
したがって、本実施の形態においては、図５５（ｃ）に示す８行の縦書き文字行に従って、ステップＳ２０５以降の処理を実行することになる。
【０２２８】
ここに、抽出した縦書き文字行がブック原稿の綴じ部から一定範囲内に存在しない場合には、安定した近似曲線多項式を算出することが難しいため、再度縦書き文字行の抽出処理を行うことにより、安定した近似曲線多項式の算出を図ることが可能になる。
【０２２９】
なお、ステップＳ２０２においても、綴じ部境界線から一定範囲内に抽出した縦書き文字行が存在しない場合に、抽出した縦書き文字行を除外した残りの行に関して、再び同様の縦書き文字行の抽出処理を行うようにしても良い。
【０２３０】
次に、本発明の第四の実施の形態について説明する。なお、第一の実施の形態において説明した部分と同一部分については同一符号を用い、説明も省略する。本実施の形態は、図２６で説明したステップＳ２０２の処理内容が異なるものである。
【０２３１】
極端に細い縦書き文字行や短い縦書き文字行は主要な本文以外のものまたはノイズである可能性があるので、安定した近似曲線多項式を算出する阻害要因となり得る。そこで、本実施の形態は、予め定めた幅以下の縦書き文字行や行の長さが幅に対して予め定めた比率以下の縦書き文字行を除外して縦書き文字行の抽出処理を行うようにしたものである。
【０２３２】
ここで、図５６は本実施の形態の縦書き文字行の抽出処理を示す説明図である。本実施の形態においては、縦書き文字行の行幅が１ｗ以下の縦書き文字行や、行の長さが行幅に対して１．５倍以下の縦書き文字行を除外するようにしている。例えば、図５６（ａ）に示すように、縦書き文字行Ａ，Ｂは長さが行幅に対して１．５倍以下であるので除外し、縦書き文字行Ｃは行幅が１ｗ以下であるので除外する。図５６（ｂ）は、所定の縦書き文字行を除外後に、行の先頭文字のｙ座標が最大の縦書き文字行Ｄから一定範囲ｈ内に先頭文字が存在する縦書き文字行を抽出した結果を示すものである。縦書き文字行Ｄの先頭位置から予め定めた距離範囲ｈ内に存在する行先頭文字は、図５６（ｂ）中、黒丸“●”で示した文字である。
【０２３３】
ここに、極端に細い縦書き文字行は、主要な本文以外のものまたはノイズである可能性があることから、これを排除することで、安定した近似曲線多項式を算出することが可能になる。
【０２３４】
また、極端に短い縦書き文字行は、主要な本文以外のものまたはノイズである可能性があることから、これを排除することで、安定した近似曲線多項式を算出することが可能になる。
【０２３５】
次に、本発明の第五の実施の形態について説明する。なお、第一の実施の形態において説明した部分と同一部分については同一符号を用い、説明も省略する。本実施の形態は、図３４で説明したステップＳ５３及びステップＳ５４の処理内容が異なるものである。本実施の形態では、仮想ページ外形算出処理（ステップＳ５３）においては何ら処理を実行することなく、主走査方向歪み補正処理（ステップＳ５４）を実行する。
【０２３６】
本実施の形態の主走査方向歪み補正処理（ステップＳ５４）について説明する。本実施の形態の主走査方向歪み補正処理（ステップＳ５４）は、ページ外形、罫線、文字行の何れかに基づき、基準線選択処理（ステップＳ５１）において選択された基準線と参照線選択処理（ステップＳ５２）において選択された参照線とが、スキャン画像の左右両ページに存在することを前提とする。
【０２３７】
図５７は、スキャン画像の左右両ページのいずれにも基準線又は参照線が存在する場合の主走査方向歪み補正処理を示す説明図である。図５７においては、主走査方向歪み補正前の基準線と参照線とをそれぞれ実線で示し、主走査方向歪み補正後の基準線と参照線とをそれぞれ点線で示している。主走査方向歪み補正後の点線は、主走査方向歪み補正前の基準線または参照線の直線部分（平坦部）をそのまま延長したものである。
【０２３８】
図５７に示すように、副走査方向の位置ｘにおいて、基準線上の点ＰがＰ’に、参照線上の点ＱがＱ’に、それぞれ補正されることになる。さて、位置ｘにおける主走査方向上の任意の点ＹがＹ’に補正されるとして、次の関係式
ＹＰ／ＹＱ＝Ｙ’Ｐ’／Ｙ’Ｑ’
が成り立つ。よって、各点の主走査方向の位置を、点ＰならＰ（ｘ）というように表すと、
（Ｙ（ｙ）−Ｐ（ｙ））／（Ｙ（ｙ）−Ｑ（ｙ））＝（Ｙ’（ｙ）−Ｐ’（ｙ））／（Ｙ’（ｙ）−Ｑ’（ｙ））
となるので、これを変形して、
Ｙ（ｙ）＝（（Ｐ（ｙ）−Ｑ（ｙ））／（Ｐ’（ｙ）−Ｑ’（ｙ）））Ｙ’（ｙ）＋（Ｐ’（ｙ）Ｑ（ｙ）−Ｐ（ｙ）Ｑ’（ｙ））／（Ｐ’（ｙ）−Ｑ’（ｙ））
となる。この式を利用して、主走査方向歪み補正後にＹ’（ｙ）の位置に来るべき点の主走査方向歪み補正前の位置Ｙ（ｙ）を求めることができる。
【０２３９】
つまり、本実施の形態においては、上述した式を用いて、主走査方向歪み補正前のＹ（ｙ）の画素値を補正後のＹ’（ｙ）に移すことにより、主走査方向歪み補正処理を実行するものである。
【０２４０】
ただし、Ｙ（ｙ）の算出値は一般に小数となるので、補正画像の画質を落としてしまう場合がある。そこで、本実施の形態においてはこれを解消するため、その前後の整数位置に対応する画素値の線形補間結果を用いるべく、画素の線形補間処理も併せて行う。
【０２４１】
ここで、図５８は画素値の線形補間処理による求め方を示す説明図である。図５８に示すように、Ｙ（ｙ）の前後の整数をそれぞれＮ，Ｎ＋１、これに対応する画素値をＤ（Ｎ），Ｄ（Ｎ＋１）とすると、Ｙ（ｙ）における画素値Ｄ（Ｙ（ｙ））は図５８に示したような線形関係を満足するように算出する。
【０２４２】
なお、各実施の形態においては、画像読取装置としてデジタル複写機のスキャナ部１を適用したが、これに限るものではなく、例えば自動ページ捲り機能を搭載したスキャナ等に適用するようにしても良い。
【０２４３】
また、各実施の形態においては、ブック原稿４０のページ綴じ部４１とスキャナ部１の画像読み取りの主走査方向とが平行になるように位置させてブック原稿４０をコンタクトガラス２上に下向きに載置した場合について説明したが、これに限るものではない。例えば、図５９に示すように、上向きのブック原稿４０をコンタクトガラス２の下方からコンタクトガラス２に対して押し付けるように接触させるものであっても良い。
【０２４４】
さらに、各実施の形態においては、画像歪み補正装置を画像形成装置であるデジタル複写機１６に備え、デジタル複写機１６のスキャナ部１で読み取ったスキャン画像に対して画像の歪み補正処理を施すようにしたが、これに限るものではない。例えば、原稿画像を読み取る画像読取手段を備えたイメージスキャナをパーソナルコンピュータに接続するとともに、このパーソナルコンピュータのＨＤＤに記憶媒体であるＣＤ−ＲＯＭ３７に格納されたプログラムをインストールすることによって画像歪み補正装置を構成しても、前述したような各種の作用効果と同様の作用効果を得ることができる。また、記憶媒体であるＣＤ−ＲＯＭ３７に格納されたプログラムをパーソナルコンピュータのＨＤＤにインストールすることによって画像歪み補正装置を構成し、予め画像読取手段により読み取られたスキャン画像に対して歪み補正処理を施すようにしても良い。
【０２４５】
【発明の効果】
請求項１，２２記載の発明によれば、多項式近似曲線の算出に用いる縦書き文字行を、縦書き文字行の文字行先頭（もしくは文字行末尾）の位置情報に基づいて一定の位置関係を有するものに限定して抽出することにより、縦書き文字行の文字行先頭（もしくは文字行末尾）の位置情報に基づいて一定の位置関係を有するものに限定して縦書き文字行を抽出することができるので、縦書き文字行の一番上（もしくは一番下）の文字の位置が行ごとに上下方向に大きく変動しているような場合にも良好な補正結果を得ることができる。
【０２４６】
請求項２，２３記載の発明によれば、縦書き文字行の中から先頭（もしくは末尾）の綴じ部方向の座標が最大（もしくは最小）の縦書き文字行を抽出し、抽出された縦書き文字行の先頭（もしくは末尾）から予め定めた距離範囲内に先頭（もしくは末尾）が存在する縦書き文字行を抽出することにより、各縦書き文字行の文字行先頭（もしくは文字行末尾）の位置を所定範囲に限定することができる。
【０２４７】
請求項３，２４記載の発明によれば、極端に細い縦書き文字行は、主要な本文以外のものまたはノイズである可能性があることから、これを排除することで、安定した近似曲線多項式を算出することができる。
【０２４８】
請求項４，２５記載の発明によれば、極端に短い縦書き文字行は、主要な本文以外のものまたはノイズである可能性があることから、これを排除することで、安定した近似曲線多項式を算出することができる。
【０２４９】
請求項５，２６記載の発明によれば、抽出した縦書き文字行の数が少ない場合には、安定した近似曲線多項式を算出することが難しいため、再度縦書き文字行の抽出処理を行うことにより、安定した近似曲線多項式の算出を図ることができる。
【０２５０】
請求項６，２７記載の発明によれば、抽出した縦書き文字行がブック原稿の綴じ部から一定範囲内に存在しない場合には、安定した近似曲線多項式を算出することが難しいため、再度縦書き文字行の抽出処理を行うことにより、安定した近似曲線多項式の算出を図ることができる。
【０２５１】
請求項７，２８記載の発明によれば、ブック原稿の綴じ部からの一定範囲は、画像全体の幅に応じて定められることにより、安定した近似曲線多項式の算出を図ることができる。
【０２５２】
請求項８，２９記載の発明によれば、縦書き文字行の先頭（もしくは末尾）の綴じ部方向の座標に関するヒストグラムの中で、値が最大である基準行を開始行とし、綴じ部方向に向かって先頭（もしくは末尾）の座標が近接した縦書き文字行を抽出することにより、各縦書き文字行の文字行先頭（もしくは文字行末尾）の位置を更に狭小な所定範囲に限定することができる。
【０２５３】
請求項９，３０記載の発明によれば、基準行を定め、この基準行の先頭（もしくは末尾）の綴じ部方向の座標に対して、一定幅の範囲内に行の先頭（もしくは末尾）が存在する縦書き文字行の数を、基準行に関する前記ヒストグラムの値とすることにより、縦書き文字行の先頭（もしくは末尾）の綴じ部方向の座標に関するヒストグラムを確実に構成することができる。
【０２５４】
請求項１０，３１記載の発明によれば、抽出した縦書き文字行の平均行幅に応じ、基準行の先頭（もしくは末尾）の綴じ部方向の座標に対する一定幅を定めることにより、安定した近似曲線多項式の算出を図ることができる。
【０２５５】
請求項１１，３２記載の発明によれば、抽出した縦書き文字行をページの外側から綴じ部方向に向かって順次探索し、着目している行の先頭（もしくは末尾）の綴じ部方向の座標が既に定めた基準行のうちの何れかの基準行の先頭（もしくは末尾）の綴じ部方向の座標に対して一定幅の範囲内に存在する場合には、その基準行に関するヒストグラム値を１だけ増加し、一定幅の範囲内に存在しない場合には、着目行を新たな基準行とみなすことにより、安定した近似曲線多項式の算出を図ることができる。
【０２５６】
請求項１２，３３記載の発明によれば、画像の歪みにより本来無関係であるべき縦書き文字行が抽出されるが、画像の歪みを生じていない部分と歪みを生じている部分とで綴じ部方向の座標が近接した縦書き文字行を抽出する基準を切り替えることにより、本来無関係であるべき縦書き文字行を除外することができる。
【０２５７】
請求項１３，３４記載の発明によれば、抽出処理の開始時は歪みを生じていない部分とし、連続して抽出した２つの縦書き文字行の先頭（もしくは末尾）の綴じ部方向の座標の差が一定値以上となれば、それ以降を歪みを生じている部分とすることにより、安定した近似曲線多項式の算出を図ることができる。
【０２５８】
請求項１４，３５記載の発明によれば、抽出した縦書き文字行の平均行幅に応じ、綴じ部方向の座標の差に関する一定値を定めることにより、安定した近似曲線多項式の算出を図ることができる。
【０２５９】
請求項１５，３６記載の発明によれば、歪みを生じていない部分は、着目行を基準として、綴じ部方向の座標のブック原稿のページ外側へ向かう方向に関しては着目行の先頭（もしくは末尾）位置から一定範囲内に抽出する前記縦書き文字行の先頭（もしくは末尾）が存在し、かつ、綴じ部方向の座標のブック原稿のページの内側へ向かう方向に関しては着目行の先頭（もしくは末尾）位置から見て綴じ部へ向かう方向に対して予め定めた一定角度の範囲内に抽出する行の先頭（もしくは末尾）が存在するように、次の前記縦書き文字行の抽出を行うことにより、画像の歪みを生じていない部分では、縦書き文字行を確実に抽出することができる。
【０２６０】
請求項１６，３７記載の発明によれば、歪みを生じている部分は、着目行を基準として、綴じ部方向の座標のブック原稿のページの内側へ向かう方向に関しては着目行の先頭（もしくは末尾）位置から見て綴じ部へ向かう方向に対して予め定めた一定角度の範囲内に抽出する行の先頭（もしくは末尾）が存在し、かつ、着目行の先頭（もしくは末尾）位置と抽出する行の先頭（もしくは末尾）位置を結ぶ直線の傾きが着目行の先頭（もしくは末尾）位置と着目行の直前の抽出行の先頭（もしくは末尾）位置を結ぶ直線の傾きから予め定めた値を引いた値よりも大きくなるように、次の前記縦書き文字行の抽出を行うことにより、画像の歪みを生じている部分では、本来無関係であるべき縦書き文字行を確実に除外することができる。
【０２６１】
請求項１７，３８記載の発明によれば、抽出した縦書き文字行の数が少ない場合には、安定した近似曲線多項式を算出することが難しいため、再度縦書き文字行の抽出処理を行うことにより、安定した近似曲線多項式の算出を図ることができる。
【０２６２】
請求項１８，３９記載の発明によれば、抽出した縦書き文字行がブック原稿の綴じ部から一定範囲内に存在しない場合には、安定した近似曲線多項式を算出することが難しいため、再度縦書き文字行の抽出処理を行うことにより、安定した近似曲線多項式の算出を図ることができる。
【０２６３】
請求項１９，４０記載の発明によれば、ブック原稿の綴じ部からの一定範囲は、画像全体の幅に応じて定められることにより、安定した近似曲線多項式の算出を図ることができる。
【０２６４】
請求項２０記載の発明の画像読取装置によれば、請求項１ないし１９の何れか一記載の発明と同様の作用効果を奏する画像読取装置を得ることができる。
【０２６５】
請求項２１記載の発明の画像形成装置によれば、請求項１ないし１９の何れか一記載の発明と同様の作用効果を奏する画像形成装置を得ることができる。
【０２６６】
請求項４１記載の発明のコンピュータに読取り可能な記憶媒体によれば、請求項２２ないし４０の何れか一記載のプログラムを記憶していることにより、この記憶媒体に記憶されたプログラムをコンピュータに読み取らせることで、請求項２２ないし４１の何れか一記載の発明と同様の作用効果を得ることができる。
【図面の簡単な説明】
【図１】本発明の第一の実施の形態のスキャナ部の構成を示す縦断正面図である。
【図２】スキャナ部を搭載したデジタル複写機の上部部分を示す斜視図である。
【図３】スキャナ部の制御系の電気的な接続を示すブロック図である。
【図４】画像処理部の基本的な内部構成を示すブロック図である。
【図５】メイン制御部の電気的な接続を示すブロック図である。
【図６】スキャン画像の歪み補正処理の流れを概略的に示すフローチャートである。
【図７】スキャナ部のコンタクトガラス上にブック原稿を載置した状態を示す斜視図である。
【図８】入力した画像の一例を示す平面図である。
【図９】スキャン画像のページ綴じ部の近傍の歪みを示す説明図である。
【図１０】図８に示した画像の黒画素ヒストグラムである。
【図１１】ページ外形／罫線／文字行の抽出処理の流れを概略的に示すフローチャートである。
【図１２】上端にページ外形が存在するスキャン画像の一例を示す説明図である。
【図１３】図１２に示したスキャン画像の綴じ部境界線左側の黒画素ヒストグラムである。
【図１４】長い罫線が存在するスキャン画像の一例を示す説明図である。
【図１５】図１４に示したスキャン画像の綴じ部境界線左側の黒画素ヒストグラムである。
【図１６】罫線の連続性の判別を示す説明図である。
【図１７】罫線の座標検出を示す説明図である。
【図１８】左右ページから各１本ずつの罫線を選択する場合を示す説明図である。
【図１９】左右ページをさらに上下部分に分け、その各４ブロックにおいて１本ずつの罫線を選択する場合を示す説明図である。
【図２０】罫線の座標抽出を示す説明図である。
【図２１】排除される不適切な罫線の例を示す説明図である。
【図２２】図８に示した画像の副走査方向の黒白反転数ヒストグラムである。
【図２３】図８に示した画像の主走査方向の黒白反転数ヒストグラムである。
【図２４】スキャン画像の文字外接矩形抽出処理および文字行抽出処理の結果の一例を示す説明図である。
【図２５】最適な横書き文字行の選択を示す説明図である。
【図２６】各縦書き文字行からの横書き文字行の抽出処理の流れを概略的に示すフローチャートである。
【図２７】抽出した行切り出し矩形を例示的に示す説明図である。
【図２８】予め定めた距離範囲内に先頭が存在する縦書き文字行を例示的に示す説明図である。
【図２９】抽出した縦書き文字行の先頭のｙ座標に関してヒストグラムを構成する状態を示す説明図である。
【図３０】画像の歪みを生じていない部分における処理を示す説明図である。
【図３１】画像の歪みを生じている部分における処理を示す説明図である。
【図３２】抽出した縦書き文字行を示す説明図である。
【図３３】縦書き文字行の行切り出し矩形を示す説明図である。
【図３４】画像歪み補正処理の流れを概略的に示すフローチャートである。
【図３５】スキャン画像の上辺及び下辺のいずれにも「ページ外形」が存在する場合を示す説明図である。
【図３６】図３５を画素シフトした状態を示す説明図である。
【図３７】スキャン画像の上辺及び下辺の何れか一方に「ページ外形」が存在し、他方の辺には「罫線」と「文字行」とが存在する場合を示す説明図である。
【図３８】図３７を画素シフトした状態を示す説明図である。
【図３９】スキャン画像の左右両ページの上辺及び下辺の何れか一方に「罫線」が存在し、他方の辺には「罫線」と「文字行」とが存在する場合を示す説明図である。
【図４０】図３９を画素シフトした状態を示す説明図である。
【図４１】スキャン画像の左右両ページの上辺及び下辺の何れか一方に「罫線」と「文字行」とが存在し、他方の辺の一方のページのみに「罫線」が存在する場合を示す説明図である。
【図４２】図４１を画素シフトした状態を示す説明図である。
【図４３】スキャン画像の左右何れかのページの上辺及び下辺の何れか一方に「罫線」が存在し、他方の辺の他方のページのみに「文字行」が存在する場合を示す説明図である。
【図４４】副走査方向歪み補正処理の流れを概略的に示すフローチャートである。
【図４５】抽出された文字外接矩形を示す説明図である。
【図４６】二値化画像をブック原稿のページ綴じ部に平行な方向の複数の短冊状の領域に分割した状態を示す説明図である。
【図４７】予め排除される文字外接矩形を示す説明図である。
【図４８】各短冊領域内の外接矩形の特徴量の平均値の一例を示すグラフである。
【図４９】各短冊領域内の地肌濃度変化の一例を示すグラフである。
【図５０】図４８に示した各短冊領域の特徴量に対して長さが全て５であるウィンドウを用いてフィルタリング処理を施した結果を示すグラフである。
【図５１】図４８に示した各短冊領域の特徴量に対してページ綴じ部付近でウィンドウ長を調整してフィルタリング処理を施した結果を示すグラフである。
【図５２】図５１に示した特徴量に基づいて算出した補正拡大率を示すグラフである。
【図５３】歪みを補正した画像を示す平面図である。
【図５４】本発明の第二の実施の形態の縦書き文字行の抽出処理を示す説明図である。
【図５５】本発明の第三の実施の形態の縦書き文字行の抽出処理を示す説明図である。
【図５６】本発明の第四の実施の形態の縦書き文字行の抽出処理を示す説明図である。
【図５７】本発明の第五の実施の形態の主走査方向歪み補正処理を示す説明図である。
【図５８】画素値の線形補間処理による求め方を示す説明図である。
【図５９】コンタクトガラスにブック原稿を接触させた状態を示す正面図である。
【図６０】コンタクトガラス上にブック原稿を載置した状態を示す正面図である。
【符号の説明】
１　　　　画像読取装置
１６　　　画像形成装置
２９　　　画像歪み補正装置[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image distortion correction device, an image reading device, an image forming device, a program, and a storage medium.
[0002]
[Prior art]
Most originals read using a flatbed scanner are sheet-like originals.An openable and closable pressure plate is provided on the contact glass, and after placing the original on the contact glass, the pressure plate is closed and the original is scanned. I have. However, the manuscript is not limited to a sheet, and book manuscripts (books, booklets, etc.) may be treated as manuscripts. In such a case, the book manuscript is placed on the contact glass and the manuscript is scanned. Will do.
[0003]
However, when a book original is used as the original, the page binding unit 101 of the book original 100 rises from the contact glass 102 as shown in FIG. When the page binding section 101 of the book document 100 rises from the contact glass 102 in this manner, the page binding section 101 moves away from the focal plane, and thus the scanned image of the raised section includes image distortion and shadow. Image degradation such as blurred characters occurs. The page binding unit 101 of the deteriorated image is difficult to read, and the recognition rate when performing character recognition processing by OCR is significantly reduced. In particular, the ratio is high in thick bookbinding, and if a pressing operation is performed so that the page binding portion 101 of the book document 100 does not separate from the focal plane, the book document 100 itself may be damaged.
[0004]
In order to solve such a problem, there has been proposed a method of correcting image distortion using a method of estimating a three-dimensional shape of an object from image density information. As a method of estimating the three-dimensional shape of the object from the density information of the image,
T. Wada, H .; Uchida and T.S. Matsuyama, "Shape from Shading with Interreflections under a Proximal Light Source": Distortion-Free Copies of a Union of Foreign Resources.
Is a representative example of a method called Shape from Shadowing described in US Pat.
[0005]
Japanese Patent Application Laid-Open No. Hei 5-161002 proposes a method of measuring the shape of a book by a triangulation method and correcting distortion.
[0006]
Further, Japanese Patent Application Laid-Open No. H11-41455 proposes a method of estimating the three-dimensional shape of the book surface using the shape of the page outline of the read scan image.
[0007]
However, according to the above-mentioned method called Shape from Shadowing, the amount of calculation is large and the calculation time of the distortion correction processing is long, so that practical use is difficult.
[0008]
Further, the method described in Japanese Patent Application Laid-Open No. 5-161002 is not suitable because a special shape measuring device for measuring the shape of a book by a triangulation method is required.
[0009]
Further, according to the method described in Japanese Patent Application Laid-Open No. H11-41455, distortion can be corrected with a small amount of calculation. However, when the page outline is not completely contained in the image and is cut off halfway, Cannot be effectively corrected.
[0010]
Therefore, in recent years, there has been proposed an image distortion correcting apparatus capable of effectively correcting the distortion of a read scan image in which the page outline is cut off in the middle with a small amount of calculation.
[0011]
[Problems to be solved by the invention]
However, according to the image distortion correction apparatus that can effectively correct the distortion with a small amount of calculation even for a read scan image in which the page outline is cut off in the middle as described above, the image is not a page outline. Although the image distortion is corrected using the character line information and the ruled line information, there is still a problem that is not complete and must be solved.
[0012]
For example, if a ruled line is cut in the middle due to blurring or the like, inappropriate ruled line information is extracted, and there is a problem that the correction accuracy is reduced. In addition, there is also a problem that improper recognition of a character line and a photograph portion results in extraction of inappropriate character line information, thereby lowering correction accuracy.
[0013]
When correcting the image distortion using the outer shape obtained by connecting the characters at the bottom of each vertically written character line, the center of the lower side of the circumscribed rectangle of each character to be connected is connected, and the straight line portion and the curve By extracting and approximating the portion, the coordinate value (in the main scanning direction) of the outer shape of the vertically written character line is determined. However, if the position of the top character of the vertical writing character line fluctuates greatly in the vertical direction for each line, a stable polynomial approximation curve cannot be obtained when these are connected, and the result correction There is a problem that unnatural distortion occurs in the image. The same applies to the case where the position of the bottom character of the character line greatly changes.
[0014]
An object of the present invention is to obtain a good image distortion correction result even when the position of the top (or bottom) character of a vertically written character line fluctuates greatly in the vertical direction for each line. It is an object of the present invention to provide an image distortion correcting device, an image reading device, an image forming device, a program, and a storage medium that can perform the above.
[0015]
[Means for Solving the Problems]
The image distortion correction apparatus according to claim 1, wherein the distortion of the scanned image obtained by reading the book original contacting above or below the scanning surface with the image reading means is converted into a vertical character line existing in the scanned image. In an image distortion correction apparatus for correcting based on a polynomial approximate curve related to a position of a head (or a character line end), the vertical character line used for calculating the polynomial approximate curve is replaced with a character line head of the vertical character line (or A character line extracting means is provided for extracting only those having a certain positional relationship based on the positional information at the end of the character line.
[0016]
Therefore, the vertical writing character line used for calculating the polynomial approximation curve is extracted by limiting the vertical writing character line to those having a certain positional relationship based on the position information of the character line head (or the character line end). This makes it possible to extract vertical writing character lines limited to those having a certain positional relationship based on the position information of the character line head (or character line end) of the vertical writing character line. A good correction result can be obtained even when the position of the character at the top (or the bottom) of the character line fluctuates greatly in the vertical direction for each line.
[0017]
According to a second aspect of the present invention, in the image distortion correcting apparatus according to the first aspect, the character line extracting means has a maximum (or a minimum) coordinate in a binding portion direction at the head (or the end) of the vertical writing character line. ) To extract the vertical writing character line having a head (or end) within a predetermined distance range from the head (or end) of the extracted vertical writing character line. One character line extracting means is provided.
[0018]
Therefore, a vertical character line having the maximum (or minimum) coordinate in the binding part direction at the beginning (or end) of the vertical character line is extracted from the vertical (character) line. Vertically written character lines having a head (or tail) within the determined distance range are extracted. This makes it possible to limit the position of the character line head (or character line end) of each vertically written character line to a predetermined range.
[0019]
According to a third aspect of the present invention, in the image distortion correcting apparatus according to the second aspect, the vertical writing character line whose line width is smaller than a predetermined value is excluded from the extraction target.
[0020]
Accordingly, since an extremely thin vertical writing character line may be noise other than the main text or noise, by eliminating this, a stable approximate curve polynomial can be calculated.
[0021]
According to a fourth aspect of the present invention, in the image distortion correcting apparatus according to the second aspect, the vertically written character line in which a ratio of a line length to a line width is smaller than a predetermined value is excluded from extraction targets.
[0022]
Therefore, since an extremely short vertical character line may be noise other than the main text or noise, by eliminating this, a stable approximate curve polynomial can be calculated.
[0023]
According to a fifth aspect of the present invention, in the image distortion correction apparatus according to any one of the second to fourth aspects, when the number of the vertically written character lines extracted by the first character line extracting means is equal to or less than a predetermined value. Executes the first character line extracting means again on the remaining vertical character lines excluding the extracted vertical character lines.
[0024]
Therefore, when the number of extracted vertical writing character lines is small, it is difficult to calculate a stable approximate curve polynomial. By performing vertical character line extraction processing again, a stable approximation curve polynomial can be calculated. It becomes possible to plan.
[0025]
According to a sixth aspect of the present invention, in the image distortion correction apparatus according to any one of the second to fourth aspects, the vertical writing character line extracted by the first character line extracting means is within a certain range from a binding portion of a book document. If not, the first character line extraction unit is executed again for the remaining vertical character lines excluding the extracted vertical character lines.
[0026]
Therefore, if the extracted vertical writing character line does not exist within a certain range from the binding portion of the book document, it is difficult to calculate a stable approximate curve polynomial. Thus, it is possible to calculate a stable approximate curve polynomial.
[0027]
According to a seventh aspect of the present invention, in the image distortion correcting apparatus of the sixth aspect, the predetermined range from the binding portion of the book document is determined according to the width of the entire image.
[0028]
Therefore, a stable approximation curve polynomial can be calculated.
[0029]
According to an eighth aspect of the present invention, in the image distortion correcting apparatus according to the second aspect, the character line extracting unit is configured to bind the head (or the end) of the vertical writing character line extracted by the first character line extracting unit to a binding portion direction. And a histogram forming means for forming a histogram with respect to the coordinates, and a reference row having a maximum value as a start row in the histogram formed by the histogram forming means, and a leading (or trailing) end in a binding portion direction. Second character line extracting means for extracting the vertically written character line whose coordinates are close to each other.
[0030]
Therefore, in the histogram relating to the coordinates of the head (or end) of the vertical writing character line in the binding portion direction, the reference line having the largest value is set as the start line, and the coordinates of the head (or end) toward the binding portion are An adjacent vertical writing character line is extracted. This makes it possible to limit the position of the character line head (or character line end) of each vertically written character line to a narrower predetermined range.
[0031]
According to a ninth aspect of the present invention, in the image distortion correction apparatus according to the eighth aspect, the histogram structuring means determines a reference line, and sets a constant with respect to a coordinate of a head (or end) of the reference line in a binding portion direction. The number of the vertically written character lines in which the head (or the end) of the line exists within the width range is defined as the value of the histogram for the reference line.
[0032]
Therefore, it is possible to reliably form a histogram relating to the coordinates of the head (or the end) of the vertical writing character line in the binding portion direction.
[0033]
According to a tenth aspect of the present invention, in the image distortion correcting apparatus according to the ninth aspect, the binding of the head (or the end) of the reference line is performed according to the average line width of the vertically written character line extracted by the first character line extracting unit. A fixed width for the coordinates in the part direction is determined.
[0034]
Therefore, a stable approximation curve polynomial can be calculated.
[0035]
According to an eleventh aspect of the present invention, in the image distortion correction apparatus according to the ninth or tenth aspect, the vertically-written character lines extracted by the first character line extracting means are sequentially searched from the outside of the page toward the binding portion. A range in which the coordinates of the head (or end) of the line of interest in the direction of the binding portion are a certain width with respect to the coordinates of the head (or end) of any of the reference lines that are already determined. If it exists within the range, the histogram value for the reference row is increased by 1, and if it does not exist within the range of a certain width, the row of interest is regarded as a new reference row.
[0036]
Therefore, a stable approximation curve polynomial can be calculated.
[0037]
According to a twelfth aspect of the present invention, in the image distortion correcting apparatus according to the eighth aspect, the second character line extracting means includes a coordinate in a binding portion direction between a portion where the image is not distorted and a portion where the distortion is generated. Switches the criterion for extracting the vertically written character line in the vicinity.
[0038]
Therefore, a vertically written character line that is originally irrelevant due to image distortion is extracted. However, a vertically written character line in which the coordinates in the binding portion direction are close to each other between the undistorted portion and the distorted portion of the image. Is switched, it is possible to exclude vertical writing character lines that should be irrelevant.
[0039]
According to a thirteenth aspect of the present invention, in the image distortion correcting apparatus according to the twelfth aspect, at the start of the extraction process by the second character line extracting means, a portion where no distortion is generated, and the two vertically extracted two vertical portions are extracted. If the difference between the coordinates of the head (or the end) of the writing character line in the binding portion direction is equal to or more than a certain value, the portion after that is regarded as a portion where distortion occurs.
[0040]
Therefore, a stable approximation curve polynomial can be calculated.
[0041]
According to a fourteenth aspect of the present invention, in the image distortion correction apparatus according to the thirteenth aspect, a constant value relating to a difference in coordinates in a binding portion direction according to an average line width of the vertical writing character line extracted by the first character line extracting means. Is determined.
[0042]
Therefore, a stable approximation curve polynomial can be calculated.
[0043]
According to a fifteenth aspect of the present invention, in the image distortion correction apparatus according to any one of the twelfth to fourteenth aspects, a portion in which no distortion is generated is located outside a page of a book document in coordinates in a binding portion direction with respect to a line of interest. As for the direction toward, there is the beginning (or end) of the vertical writing character line to be extracted within a certain range from the beginning (or end) position of the line of interest, and the inside of the page of the book document at the coordinates in the binding part direction. As for the direction toward the heading (or the end) of the line of interest, the next (or the end) of the line to be extracted exists within a predetermined angle range with respect to the direction toward the binding portion when viewed from the position of the head (or the end) of the line of interest. The above-mentioned vertical writing character line is extracted.
[0044]
Therefore, it is possible to reliably extract a vertically written character line in a portion where the image is not distorted.
[0045]
According to a sixteenth aspect of the present invention, in the image distortion correction device according to any one of the twelfth to fourteenth aspects, the portion having the distortion is a page of a book document having coordinates in the binding portion direction with respect to the line of interest. As for the direction toward the inside, the head (or the end) of the line to be extracted exists within a predetermined angle range with respect to the direction toward the binding portion when viewed from the head (or the end) position of the line of interest, and The slope of the straight line connecting the head (or end) position of the line of interest and the head (or end) position of the line to be extracted is the head (or end) position of the line of interest and the head (or end) position of the extracted line immediately before the line of interest Then, the next vertical writing character line is extracted so as to be larger than a value obtained by subtracting a predetermined value from the inclination of the straight line connecting.
[0046]
Therefore, it is possible to reliably exclude a vertically written character line that should be irrelevant in a portion where the image is distorted.
[0047]
According to a seventeenth aspect of the present invention, in the image distortion correcting apparatus according to the eighth aspect, when the number of the vertically written character lines extracted by the second character line extracting means is equal to or less than a predetermined value, the extracted character line is extracted. The second character line extracting means is executed again for the remaining vertical character lines excluding the vertical character lines.
[0048]
Therefore, when the number of extracted vertical writing character lines is small, it is difficult to calculate a stable approximated curve polynomial. By performing vertical character line extraction processing again, a stable approximated curve polynomial can be calculated. It becomes possible to plan.
[0049]
The invention according to claim 18 is the image distortion correction apparatus according to claim 8, wherein the vertically-written character line extracted by the second character line extracting means does not exist within a certain range from the binding portion of the book document. The second character line extracting means is executed again for the remaining vertical character lines excluding the extracted vertical character lines.
[0050]
Therefore, if the extracted vertical writing character line does not exist within a certain range from the binding portion of the book document, it is difficult to calculate a stable approximate curve polynomial. Thus, it is possible to calculate a stable approximate curve polynomial.
[0051]
According to a nineteenth aspect of the present invention, in the image distortion correcting apparatus of the eighteenth aspect, the fixed range of the book document from the binding portion is determined according to the width of the entire image.
[0052]
Therefore, a stable approximation curve polynomial can be calculated.
[0053]
20. An image reading apparatus according to claim 20, wherein the image reading means reads an original image, and corrects the scanned image read by the image reading means. And.
[0054]
Therefore, an image reading apparatus having the same effect as the invention according to any one of claims 1 to 19 is obtained.
[0055]
20. An image forming apparatus according to claim 21, wherein said image reading means reads an original image, and corrects the scanned image read by said image reading means. And an image printing device that prints an image on paper based on image data output from the image distortion correction device.
[0056]
Therefore, an image forming apparatus having the same operation as the invention according to any one of the first to nineteenth aspects is obtained.
[0057]
The program according to claim 22, wherein a distortion of a scan image obtained by reading a book document contacting above or below a scan surface by an image reading means is corrected by changing a vertical character line present in the scan image to a character line head (or A program for causing a computer to execute an image distortion correction process for correcting based on a polynomial approximate curve relating to a position of a character line end), wherein the computer writes the vertically written character line used for calculating the polynomial approximate curve to the vertical position. Based on the position information of the head of the writing character line (or the end of the character line), a character line extraction function is executed to extract only those having a certain positional relationship.
[0058]
Therefore, the vertical writing character line used for calculating the polynomial approximation curve is extracted by limiting the vertical writing character line to those having a certain positional relationship based on the position information of the character line head (or the character line end). This makes it possible to extract vertical writing character lines limited to those having a certain positional relationship based on the position information of the character line head (or character line end) of the vertical writing character line. A good correction result can be obtained even when the position of the character at the top (or the bottom) of the character line fluctuates greatly in the vertical direction for each line.
[0059]
According to a twenty-third aspect of the present invention, in the program according to the twenty-second aspect, the character line extracting function is arranged such that coordinates of a leading (or end) binding portion direction in the vertical writing character line are maximum (or minimum). First character line extraction for extracting a vertically written character line and extracting the vertically written character line having a head (or end) within a predetermined distance range from the head (or end) of the extracted vertical character line Causing the computer to execute a function.
[0060]
Therefore, a vertical writing character line having the maximum (or minimum) coordinate in the binding part direction at the beginning (or end) of the vertical writing character line is extracted, and the leading (or end) of the extracted vertical writing character line is determined in advance. Vertically written character lines having a head (or tail) within the determined distance range are extracted. This makes it possible to limit the position of the character line head (or character line end) of each vertically written character line to a predetermined range.
[0061]
According to a twenty-fourth aspect of the present invention, in the program according to the twenty-third aspect, the vertical writing character line whose line width is smaller than a predetermined value is excluded from the target of extraction.
[0062]
Accordingly, since an extremely thin vertical writing character line may be noise other than the main text or noise, by eliminating this, a stable approximate curve polynomial can be calculated.
[0063]
According to a twenty-fifth aspect of the present invention, in the program according to the twenty-third aspect, the vertical writing character line in which the ratio of the line length to the line width is smaller than a predetermined value is excluded from the extraction target.
[0064]
Therefore, since an extremely short vertical character line may be noise other than the main text or noise, by eliminating this, a stable approximate curve polynomial can be calculated.
[0065]
According to a twenty-sixth aspect of the present invention, in the program according to any one of the twenty-third to twenty-fifth aspects, when the number of the vertically written character lines extracted by the first character line extraction function is equal to or less than a predetermined value, the extraction is performed. The computer causes the computer to execute the first character line extraction function again for the remaining vertical character lines excluding the vertical character lines.
[0066]
Therefore, when the number of extracted vertical writing character lines is small, it is difficult to calculate a stable approximate curve polynomial. By performing vertical character line extraction processing again, a stable approximation curve polynomial can be calculated. It becomes possible to plan.
[0067]
The invention according to claim 27 is the program according to any one of claims 23 to 25, wherein the vertical writing character line extracted by the first character line extraction function does not exist within a certain range from a binding portion of a book document. And causing the computer to execute the first character line extraction function again on the remaining vertical character lines excluding the extracted vertical character lines.
[0068]
Therefore, if the extracted vertical writing character line does not exist within a certain range from the binding portion of the book document, it is difficult to calculate a stable approximate curve polynomial. Thus, it is possible to calculate a stable approximate curve polynomial.
[0069]
According to a twenty-eighth aspect of the present invention, in the program according to the twenty-seventh aspect, a certain range from the binding portion of the book document is determined according to the width of the entire image.
[0070]
Therefore, a stable approximation curve polynomial can be calculated.
[0071]
According to a twenty-ninth aspect of the present invention, in the program according to the twenty-third aspect, the character line extracting function relates to a coordinate (in a binding part direction) of a head (or end) of the vertical writing character line extracted by the first character line extracting function. A histogram construction function for constructing a histogram, and among the histograms constructed by the histogram construction function, a reference row having a maximum value is set as a start row, and the leading (or end) coordinates are close to the binding portion direction. And a second character line extracting function for extracting the vertical character line.
[0072]
Therefore, in the histogram relating to the coordinates of the head (or end) of the vertical writing character line in the binding portion direction, the reference line having the largest value is set as the start line, and the coordinates of the head (or end) toward the binding portion are An adjacent vertical writing character line is extracted. This makes it possible to limit the position of the character line head (or character line end) of each vertically written character line to a narrower predetermined range.
[0073]
According to a thirtieth aspect of the present invention, in the program according to the twenty-ninth aspect, the histogram construction function defines a reference line, and a range of a fixed width with respect to a coordinate of a head (or end) of the reference line in a binding portion direction. The number of the vertically-written character lines in which the beginning (or the end) of the line exists is defined as the value of the histogram for the reference line.
[0074]
Therefore, it is possible to reliably form a histogram relating to the coordinates of the head (or the end) of the vertical writing character line in the binding portion direction.
[0075]
According to a thirty-first aspect of the present invention, in the program according to the thirty-seventh aspect, a head (or end) of a reference line in a binding part direction is determined according to an average line width of the vertical writing character line extracted by the first character line extracting function. Define a fixed width for coordinates.
[0076]
Therefore, a stable approximation curve polynomial can be calculated.
[0077]
According to a thirty-second aspect of the present invention, in the program according to the thirty-third or thirty-first aspect, the vertically-written character lines extracted by the first character line extraction function are sequentially searched from the outside of the page toward the binding portion. The coordinates of the head (or end) of the binding line in the direction of the binding part are within a certain width with respect to the coordinates of the binding part direction of the head (or end) of any of the previously determined reference lines. In such a case, the histogram value for the reference row is increased by one, and if the histogram value does not exist within the fixed width range, the target row is regarded as a new reference row.
[0078]
Therefore, a stable approximation curve polynomial can be calculated.
[0079]
According to a thirty-third aspect of the present invention, in the program according to the thirty-ninth aspect, the second character line extraction function is such that coordinates in the binding portion direction are close to each other in a portion where the image is not distorted and a portion where the image is distorted. The reference for extracting the vertically written character line is switched.
[0080]
Therefore, a vertically written character line that is originally irrelevant due to image distortion is extracted. However, a vertically written character line in which the coordinates in the binding portion direction are close to each other between the undistorted portion and the distorted portion of the image. Is switched, it is possible to exclude vertical writing character lines that should be irrelevant.
[0081]
According to a thirty-fourth aspect of the present invention, in the program according to the thirty-third aspect, when the extraction process by the second character line extraction function is started, a portion where no distortion occurs, and the two vertically-written character lines successively extracted. If the difference between the coordinates of the head (or the end) of the binding portion in the direction of the binding portion is equal to or greater than a certain value, the portion after that is regarded as a portion where distortion occurs.
[0082]
Therefore, a stable approximation curve polynomial can be calculated.
[0083]
According to a thirty-fifth aspect of the present invention, in the program according to the thirty-fourth aspect, a constant value relating to a coordinate difference in a binding portion direction is determined according to an average line width of the vertically-written character lines extracted by the first character line extracting function.
[0084]
Therefore, a stable approximation curve polynomial can be calculated.
[0085]
According to a thirty-sixth aspect of the present invention, in the program according to any one of the thirty-third to thirty-fifth aspects, the portion having no distortion is directed to the outside of the page of the book document at the coordinates in the binding portion direction with respect to the line of interest. As for the direction, there is the beginning (or end) of the vertical writing character line to be extracted within a certain range from the beginning (or end) position of the line of interest, and the direction toward the inside of the page of the book document at the coordinates in the binding part direction. The following vertical direction is selected so that the head (or end) of the line to be extracted exists within a predetermined angle range with respect to the direction toward the binding portion when viewed from the head (or end) position of the line of interest. Extract the writing character line.
[0086]
Therefore, it is possible to reliably extract a vertically written character line in a portion where the image is not distorted.
[0087]
According to a thirty-seventh aspect of the present invention, in the program according to any one of the thirty-third to thirty-fifth aspects, the portion having the distortion is directed toward the inside of the page of the book document having the coordinates in the binding portion direction with respect to the line of interest. Regarding the direction, the head (or end) of the line to be extracted exists within a predetermined angle range with respect to the direction toward the binding portion when viewed from the head (or end) position of the line of interest. A straight line connecting the head (or end) position and the head (or end) position of the line to be extracted has a slope connecting the head (or end) position of the line of interest and the head (or end) position of the extracted line immediately before the line of interest. The next vertical writing character line is extracted so as to be larger than a value obtained by subtracting a predetermined value from the inclination of the character string.
[0088]
Therefore, it is possible to reliably exclude a vertically written character line that should be irrelevant in a portion where the image is distorted.
[0089]
The invention according to claim 38 is the program according to claim 29, wherein the number of the vertically written character lines extracted by the second character line extraction function is equal to or less than a predetermined value. The computer causes the computer to execute the second character line extraction function again for the remaining vertical character lines excluding the lines.
[0090]
Therefore, when the number of extracted vertical writing character lines is small, it is difficult to calculate a stable approximate curve polynomial. By performing vertical character line extraction processing again, a stable approximation curve polynomial can be calculated. It becomes possible to plan.
[0091]
According to a thirty-ninth aspect of the present invention, in the program according to the thirty-ninth aspect, the vertical character line extracted by the second character line extracting function is extracted when the vertical character line does not exist within a predetermined range from a binding portion of a book document. The second character line extraction function is executed again by the computer for the remaining vertical character lines excluding the vertical character lines.
[0092]
Therefore, if the extracted vertical writing character line does not exist within a certain range from the binding portion of the book document, it is difficult to calculate a stable approximate curve polynomial. Thus, it is possible to calculate a stable approximate curve polynomial.
[0093]
The invention according to claim 40 is the program according to claim 39, wherein the predetermined range from the binding portion of the book document is determined according to the width of the entire image.
[0094]
Therefore, a stable approximation curve polynomial can be calculated.
[0095]
A computer readable storage medium according to the present invention stores the program according to any one of claims 22 to 40.
[0096]
Therefore, by causing a computer to read the program stored in the storage medium, it is possible to obtain the same operation as the invention according to any one of claims 22 to 41.
[0097]
BEST MODE FOR CARRYING OUT THE INVENTION
A first embodiment of the present invention will be described with reference to FIGS. The image distortion correcting apparatus according to the present embodiment is provided in a digital copying machine as an image forming apparatus, and a scanner unit of the digital copying machine is applied as an image reading apparatus.
[0098]
Here, FIG. 1 is a longitudinal sectional front view showing the configuration of the scanner unit 1. As shown in FIG. 1, the scanner unit 1 includes a contact glass 2 on which a document is placed, a first traveling body 5 including an exposure lamp 3 for exposing the document and a first reflection mirror 4, and a second reflection mirror 6. And a second traveling body 8 including a third reflection mirror 7, a CCD (Charge Coupled Device) 9 as an image sensor for reading an image of the document, a lens unit 10 for forming an image on the CCD 9, and a document placed thereon. The document scale 11 includes a document scale 11 that serves as a reference for preventing displacement and detachment of the contact glass 2, a white reference plate 12 for shading correction installed below the document scale 11, and a frame 14. The CCD 9 is formed on a sensor board 13.
[0099]
During scanning of the document, the first traveling body 5 and the second traveling body 8 are moved in the sub-scanning direction by the stepping motor 24 (see FIG. 3). That is, the first traveling body 5 and the second traveling body 8 travel under the contact glass 2 to expose and scan the original with the exposure lamp 3, and reflect the reflected light to the first reflecting mirror 4, the second reflecting mirror 6, and The light is reflected by the third reflection mirror 7 and forms an image on the CCD 9 through the lens unit 10. Here, an image reading unit is realized.
[0100]
Such a scanner unit 1 is, for example, a printer unit (not shown) which is an image printing apparatus that forms an image on paper in an electrophotographic manner in accordance with image data based on an image of a document read by the scanner unit 1. ) Is mounted on the digital copying machine 16. FIG. 2 is a perspective view showing an upper portion of the digital copying machine 16 on which the scanner unit 1 is mounted. As shown in FIG. 2, the scanner unit 1 is provided with a pressure plate 17 that can be opened and closed with respect to the contact glass 2 and an open / close sensor 18 that detects whether the pressure plate 17 is opened or closed. As a printer provided in the digital copying machine 16, various printing methods such as an ink jet method, a sublimation type thermal transfer method, a silver halide photographic method, a direct thermal recording method, and a fusion type thermal transfer method are applied in addition to the electrophotographic method. be able to. Since the specific configuration is well known, detailed description is omitted.
[0101]
FIG. 3 is a block diagram showing an electrical connection of a control system of the scanner unit 1. As shown in FIG. 3, the control system includes a main control unit 19 that controls the entire scanner unit 1, an image processing unit 20 that is a circuit that performs various types of image processing on image data read by the CCD 9, A traveling body control unit 21 which is a circuit for controlling the traveling body 5 and the second traveling body 8, an operation panel 22 for receiving various operations on the digital copying machine 16 and displaying various messages, and an image read by the CCD 9 A memory 23 for storing data, predetermined data, and the like is connected. The operation panel 22 is provided with a copy start key and the like for declaring the start of copying. In the traveling body control unit 21, the exposure lamp 3, the stepping motor 24 for driving the first traveling body 5 and the second traveling body 8, and the first traveling body 5 and the second traveling body 8 are at home positions. A scanner home position sensor (HP sensor) 25 for detecting whether or not this is the case and the open / close sensor 18 are connected.
[0102]
Here, FIG. 4 is a block diagram showing a basic internal configuration of the image processing unit 20. As shown in FIG. 4, the image processing unit 20 includes an analog video processing unit 26 that performs an amplification process and a digital conversion process of an analog image signal obtained by reading a document by the CCD 9, a shading correction processing unit 27 that performs a shading correction process, and a shading. An image data processing unit 28 that performs various image data processing such as MTF correction, scaling processing, and γ correction on the digital image signal after the correction processing to generate a scan image, and a scan image which is a characteristic function of the present embodiment. The image distortion correcting section 29 that realizes the distortion correcting function of FIG. The digital image signal after the image processing as described above is transmitted to the printer unit via the main control unit 19 and used for image formation.
[0103]
As shown in FIG. 5, the main control unit 19 includes a CPU (Central Processing Unit) 31 that centrally controls each unit. The CPU 31 has a read-only memory (ROM) that stores a BIOS and the like. An only memory (32) and a random access memory (RAM) 33 that rewritably stores various data and functions as a work area of the CPU 31 are connected by a bus 34 to constitute a microcomputer. Further, the bus 34 is connected to an HDD 35 storing a control program, a CD-ROM drive 36 for reading a CD (Compact Disc) -ROM 37, and an interface (I / F) 38 for communicating with a printer unit and the like. ing.
[0104]
The CD-ROM 37 shown in FIG. 5 embodies the storage medium of the present invention, and stores a predetermined control program. The CPU 31 reads the control program stored in the CD-ROM 37 with the CD-ROM drive 36 and installs the control program in the HDD 35. As a result, the main control unit 19 is in a state where it can perform various processes as described below.
[0105]
As the storage medium, not only the CD-ROM 37 but also various types of media such as various types of optical disks such as DVDs, various types of magneto-optical disks, various types of magnetic disks such as flexible disks, and semiconductor memories can be used. Alternatively, the program may be downloaded from a network such as the Internet and installed in the HDD 35. In this case, the storage device storing the program in the server on the transmission side is also a storage medium of the present invention. Note that the program may operate on a predetermined OS (Operating System), and in that case, the OS may replace a part of execution of various processes described below with the OS. It may be included as a part of a group of program files constituting predetermined application software such as software or OS.
[0106]
Next, the contents of various processes executed by the CPU 31 provided in the main control unit 19 based on the control program will be described. Here, of the processing executed by the CPU 31, the scan image in the image distortion correction unit 29, which is an image distortion correction device that realizes a scan image distortion correction function that is a characteristic function of the scanner unit 1 according to the present embodiment. Only the distortion correction processing will be described.
[0107]
FIG. 6 is a flowchart schematically showing the flow of the scan image distortion correction process. Here, as shown in FIG. 7, the book document 40 is placed on the contact glass 2 so that the page binding section 41 and the main scanning direction of the image reading by the scanner section 1 are parallel to each other. Will be described.
[0108]
First, in step S1, a scan image of the book document 40 placed on the contact glass 2 output from the image data processing unit 28 is input. Here, FIG. 8 shows an example of the input image. Then, as shown in FIG. 9, the input scan image of the book document 40 has distortion near the page binding unit 41.
[0109]
Next, optimal binarization processing of a scanned image (for example, a monochrome multi-valued image) of the book document 40 is executed (step S2), and black pixels in the sub-scanning direction (the density values of the pixels of the scanned image are predetermined). The histogram of the number of pixels (darker than the density value) is obtained (step S3). FIG. 10 is a black pixel histogram on the left side of the binding portion boundary line of the image shown in FIG. The horizontal axis in FIG. 10 indicates the position of the black pixel in the main scanning direction (the pixel whose density value is higher than a predetermined density value in the pixels of the scanned image), and the vertical axis in FIG. It shows the number of black pixels for each position. Note that, as a binding portion boundary line which is a scan image of the page binding portion 41, a position in the sub-scanning direction at which a pixel whose density value is originally dark among pixels in the scan image is selected.
[0110]
Note that the binarization processing in the case where the scanned image is a color multi-valued image focuses on, for example, any one of the RGB components (for example, the G component), and designates a component larger than a predetermined density threshold of the G component as a black pixel. What is smaller than the predetermined density threshold of the G component may be a white pixel. Alternatively, RGB may be color-converted into a luminance component and a color difference component, and threshold processing may be performed on the luminance component.
[0111]
In the following step S4, a page outline / ruled line / character line extraction process is executed. Here, FIG. 11 is a flowchart schematically showing the flow of the page outline / ruled line / character line extraction process.
[Extraction of page outline from scanned image]
First, the process of extracting the page outline from the scanned image in step S41 will be described. Here, FIG. 12 is an explanatory diagram showing an example of a scan image having a page outline at the upper end, and FIG. 13 is a black pixel histogram on the left side of the bounding portion boundary line of the scan image shown in FIG. The x-axis of the histogram shown in FIG. 13 indicates the main scanning direction (vertical direction in FIG. 12) of the scanned image, and the upper end of the scanned image is associated with the left end of the histogram. Note that, in the case of a scan image in which the page outline exists at the lower end, the lower end of the scan image is associated with the right end of the histogram. Therefore, when a page outline exists at the upper end of the scanned image as shown in FIG. 12, a black band appears at the upper part of the scanned image, and a high vertical bar appears at the left end of the histogram shown in FIG. In the present embodiment, it is determined whether or not a page outline exists in a scanned image by using such characteristics.
[0112]
More specifically, as shown in FIG. 13, the distance AO from the bounding portion boundary line to the left end of the scanned image (the left end in FIG. 12), the height BO of the histogram vertical bar, and the ratio are expressed by the following equation ( Calculated by 1),
[0113]
(Equation 1)

[0114]
If the calculated ratio k is larger than a predetermined threshold value, it is determined that the scanned image has a page outline.
[0115]
Note that if there are page outlines above and below the scanned image, high vertical bars will appear at the left and right ends of the histogram. In such a case, the scanning is performed based on the high vertical bars at the left and right ends of the histogram. A determination is made as to whether a page outline exists in the image.
[0116]
Here, the function of the page outline determining means is executed.
[0117]
According to the above processing, when it is determined that the page outline exists in the scanned image, the page outline is extracted together with information on which of the upper and lower sides of the left and right pages exists, and temporarily stored in the RAM 33. Remember.
[0118]
Note that the process of determining whether or not a page outline exists in the scanned image is executed for each of the left and right pages on the boundary of the bound portion of the scanned image.
[Extraction of ruled lines from scanned image]
In a succeeding step S42, a process of extracting a ruled line from the scanned image is executed. The process of extracting ruled lines from a scanned image in step S42 will be described.
[0119]
[Detection of ruled line candidates]
Here, FIG. 14 is an explanatory diagram showing an example of a scanned image having a long ruled line, and FIG. 15 is a black pixel histogram on the left side of the bounding portion boundary line of the scanned image shown in FIG. The x-axis of the histogram shown in FIG. 15 indicates the main scanning direction (the vertical direction in FIG. 14) of the scanned image, and the upper end of the scanned image is associated with the left end of the histogram. When a ruled line exists in the scanned image as shown in FIG. 14, a narrow peak appears in the histogram shown in FIG. In the present embodiment, it is determined whether or not a ruled line exists in a scanned image by using such characteristics.
[0120]
More specifically, first, the height H of the narrow peaks appearing in the histogram shown in FIG. 15 is obtained, and the width W at the center position (half height position) of each obtained peak is obtained. Then, when there is a peak whose height H is higher than a predetermined threshold thH and whose width W at the center of the peak is smaller than the predetermined threshold thW, the peak is set as a ruled line candidate. .
[0121]
Here, the function of the ruled line candidate extracting means is executed.
[0122]
Subsequently, it is further determined whether or not the peak which is a candidate for a ruled line is a ruled line using the continuity of the ruled line. As shown in FIG. 16, an appropriate position on the candidate ruled line (for example, the position of the center line of the page) is set as a start point, and a candidate ruled line is searched for from the start point in the left and right direction. Part). If the number of cutting points is smaller than a predetermined threshold, this candidate is determined as a ruled line. By judging whether or not a line is a ruled line based on the ruled line continuity in this manner, it is possible to eliminate a horizontally written character line or a dotted line composed of small characters erroneously detected as a ruled line.
[0123]
Here, the function of the ruled line determining means is executed.
[0124]
[Detection of ruled line coordinates]
After the ruled lines are determined as described above, the coordinates of each ruled line are detected. As shown in FIG. 17, when the coordinate value of the ruled line in the main scanning direction (the y-axis direction in FIG. 17) is set to the middle point coordinate of the black pixel run of the ruled line as shown in FIG. The main scanning direction coordinate value at x1 at the left end is y1.
[0125]
[Selection of optimal ruled line]
Next, a ruled line optimal for distortion correction is selected from the candidate ruled lines. When a plurality of ruled lines are detected as shown in FIG. 18, it is necessary to select which ruled line should be used to correct the distortion. As an example of an optimum ruled line selection criterion, the ruled line length is longer than a predetermined threshold value, and the ruled line is set within a fixed width region on the left and right sides (a hatched region in FIG. 18) of the bounding portion boundary line. Under the condition that a part of the page is covered, the ruled line closest to the upper or lower page outline is selected. FIG. 18 shows a case where one ruled line is selected from each of the left and right pages. Here, the ruled line (1) and the ruled line (2) are selected.
[0126]
Further, as another example of the selection criterion of the optimal ruled line, the ruled line length is longer than a predetermined threshold value and is within a fixed width region on the left and right sides of the bounding portion boundary line (the hatched region in FIG. 19). ), A ruled line that is closest to the top page outline at the top of each page and the bottom page outline at the bottom of each page is selected on the condition that a part of the ruled line is applied. FIG. 19 shows a case where the left and right pages are further divided into upper and lower portions, and one ruled line is selected in each of the four blocks. Here, the ruled line (1) is selected in the upper left block, the ruled line (2) is selected in the lower right block, and the ruled line (3) is selected in the lower left block. In the upper right block in FIG. 19, the above two conditions (the length of the ruled line is longer than a predetermined threshold value, and a part of the ruled line is applied to the left and right fixed width regions sandwiching the bounding portion boundary line) There is no ruled line that satisfies (1), so there is no selected ruled line.
[0127]
Regarding the above two conditions (the length of the ruled line is longer than a predetermined threshold value, and a part of the ruled line is applied to the left and right fixed width regions sandwiching the bounding portion boundary line), However, only one of them may be satisfied. In the above example, “closest to the page outline” is used as the selection criterion. However, the present invention is not limited to this, and “the curve of the ruled line is the largest” may be used. Here, the “curve of the ruled line” is represented by the difference between the coordinate values of the left and right end points of the ruled line in the main scanning direction.
[0128]
[Determination of coordinate value of optimal ruled line]
When the optimum ruled line is selected, the coordinate value (in the main scanning direction) of the ruled line is determined. The coordinate value (in the main scanning direction) of the ruled line is determined by approximating and extending the selected ruled line until it reaches both ends of the left and right pages. In FIG. 20, since the coordinate value of the BC portion where the ruled line exists has already been determined by the above-described ruled line coordinate detection processing, the coordinate value (in the main scanning direction) of the ruled line is determined for the other extended portions. Will do. More specifically, the AB section shown in FIG. 20 estimates the coordinate values (in the main scanning direction) by linear approximation, and the CD section estimates the coordinate values (in the main scanning direction) using a polynomial approximation curve.
[0129]
[Elimination of inappropriate ruled lines]
Finally, eliminate inappropriate ruled lines. This is because, as described above, when estimating the coordinate value by the polynomial approximation, if the shape of the estimated curve by the polynomial approximation is inappropriate, the distortion may increase instead of the correction. Ruled lines are eliminated. As an example of an inappropriate approximate curve shape, as shown in FIG. 21, a curve (1) in which the curve goes to the outside of the book, or a curve (2) in which the curve goes deeper inward beyond the center line. .
[0130]
If the ruled line is excluded because the shape of the estimated curve is inappropriate, the optimum ruled line is selected again and the above processing is repeated.
[0131]
If it is determined by the above processing that a ruled line exists in the scanned image, the ruled line is extracted together with information on where the ruled line exists on each of the left and right pages, and is temporarily stored in the RAM 33.
[Extraction of character line from scanned image]
In a succeeding step S43, a process of extracting a character line from the scanned image is executed. The process of extracting a character line from a scanned image in step S43 will be described. In the present embodiment, first, it is determined whether the character line in the scanned image is a vertical writing character line or a horizontal writing character line.
[0132]
[Character line determination]
A method for determining whether a character line in a scanned image is a vertical writing character line or a horizontal writing character line will be described. Here, FIG. 22 is a black-and-white inversion number histogram in the sub-scanning direction of the image shown in FIG. The horizontal axis in FIG. 22 indicates the black pixels in the sub-scanning direction (horizontal direction) (pixels in which the density value is higher than a predetermined density value among the pixels obtained by inverting the black and white of the scanned image) in the main scanning direction. The vertical axis in FIG. 22 indicates the number of black pixels at each position. FIG. 23 is a histogram of the number of black and white reversals in the main scanning direction of the image shown in FIG. The horizontal axis in FIG. 23 represents the black pixel in the main scanning direction (vertical direction) (the pixel whose density value is higher than a predetermined density value among the pixels obtained by inverting the black and white of the scanned image) in the sub-scanning direction. The vertical axis in FIG. 23 indicates the number of black pixels at each position. When the characters in the image are horizontally written scan images as shown in FIG. 8, the histogram in the sub-scanning direction as shown in FIG. 22 changes drastically, but the change in the histogram in the main scanning direction as shown in FIG. Few. Although not shown, when the character line in the scanned image is a vertical writing character line, the histogram in the main scanning direction changes sharply, but the change in the histogram in the sub-scanning direction is small.
[0133]
The above-described determination method is specifically realized by the following equations. First, according to the following equation (2),
[0134]
(Equation 2)

[0135]
Mean value mean of histogram value Pnt (y) at position in main scanning direction y _H Is calculated. Here, height is the height of the image.
Then, according to the following equation (3),
[0136]
[Equation 3]

[0137]
Variance σ of main scanning direction of histogram in sub scanning direction _H Is obtained.
Similarly, according to the following equation (4),
[0138]
(Equation 4)

[0139]
Average value mean of histogram values Pnt (x) at positions in sub-scanning direction x _V Is calculated. Here, width is the width of the image.
Then, according to the following equation (5),
[0140]
(Equation 5)

[0141]
Variance σ of histogram in main scanning direction with respect to sub scanning direction _v Is obtained.
[0142]
As described above, when the character line in the scanned image is a horizontally written character line, the variance σ of the histogram in the sub-scanning direction in the main scanning direction is used. _H Is the variance σ of the histogram in the main scanning direction with respect to the sub-scanning direction. _v Greater than. Conversely, if the character line in the scanned image is a vertically written character line, the variance σ of the histogram in the main scanning direction in the sub-scanning direction _v Is the variance σ of the histogram in the main scanning direction in the sub-scanning direction. _H Greater than. That is, the variance σ _H And variance σ _v By comparison with the above, it is possible to determine whether the character line in the scanned image is a vertical writing character line or a horizontal writing character line.
[0143]
The reason why the black and white reversal number histogram is used to determine whether a character line in a scanned image is a vertical character line or a horizontal character line is to avoid confusion between a character line and a photograph portion. In general, when the values of the black pixel histograms are almost the same, the value of the black-and-white inversion number histogram is larger in the character area than in the photographic area.
[0144]
Here, the function of the document determination means is executed.
[0145]
[Coordinate detection of horizontal text line]
After the character line is determined as described above, first, the coordinates of each horizontal writing character line are detected. In detecting the coordinates of the horizontal writing character line, a circumscribing rectangle extraction process for each character is performed, and a horizontal writing character line extraction process is performed. Since the character recognition processing is a well-known technique, the description thereof is omitted. Here, an example of the result of the character circumscribed rectangle extraction process and the character line extraction process of the scanned image is shown in FIG. Then, the coordinates of the center point of the circumscribed rectangle of each character are regarded as the coordinates of the character, and the coordinates of the horizontal writing character line are detected.
[0146]
[Select optimal horizontal writing line]
Next, a horizontal writing character line optimal for distortion correction is selected from the extracted horizontal writing character lines. When a plurality of horizontal writing character lines are detected, it is necessary to select which horizontal writing character line is to be used to correct the distortion. An example of the selection criterion of the optimal horizontal writing character line is basically the same as the above-described selection rule of the optimal ruled line, and as shown in FIG. 25, the length BC of the horizontal writing character line is smaller than a predetermined threshold value. It is conditional on that a part C of a horizontal writing character line is placed in a long and constant width area (shaded area in FIG. 25) on the left and right sides of the bounding portion boundary line. Select the horizontal writing line closest to the outline. Here, B is the center of the leftmost rectangle of the character line, and C is the center of the rightmost rectangle. Note that the selection of the optimum horizontal writing character line may be to select the horizontal writing character line closest to the page outline of each one from the left and right pages, or to further divide the left and right pages into upper and lower portions, The horizontal writing character line closest to the page outline of each block in the block may be selected.
[0147]
Note that the above two conditions (the length of a horizontally written character line is longer than a predetermined threshold value and a part of the horizontally written character line covers a certain width area on the left and right sides of the binding portion boundary line) , But not both, may be satisfied. Further, in the above example, “closest to the page outline” is used as the selection criterion, but the present invention is not limited to this, and “horizontal curvature of a horizontally written character line” may be used. Here, the "curvature of a horizontally written character line" is represented by a difference between coordinate values in the main scanning direction of center coordinates of character circumscribed rectangles at both ends of the horizontally written character line.
[0148]
[Determining the coordinate value of the optimal horizontal writing character line]
When the optimal horizontal writing character line is selected, the coordinate value (in the main scanning direction) of the horizontal writing character line is determined. The coordinate value (in the main scanning direction) of the horizontal writing character line is obtained by connecting the center points of the circumscribed rectangles of the characters in the horizontal writing character line and approximating and extracting the straight line portion and the curved line portion. (Direction) will be determined. More specifically, D shown in FIG. 25 is a bounding portion boundary line, a coordinate value (in the main scanning direction) is estimated by a polynomial approximation curve during BD, and an approximation is made between A and B at the leftmost end. The coordinate value (in the main scanning direction) is estimated from the value of the straight line.
[0149]
[Elimination of inappropriate horizontal character lines]
Finally, eliminate inappropriate horizontal character lines. This is because, as described above, when estimating the coordinate value by the polynomial approximation, if the shape of the estimated curve by the polynomial approximation is inappropriate, the distortion may increase instead of the correction. This excludes horizontal writing character lines. An example of an inappropriate approximate curve shape is the same as the case of the ruled line described above. Although not particularly shown, a case where the curve goes to the outside of the book or a case where the curve goes deeply beyond the center line is obtained. Is the case.
[0150]
If the horizontal character line is excluded because the shape of the estimated curve is inappropriate, the optimum horizontal character line is selected again, and the above processing is repeated.
[0151]
By the above processing, when it is determined that the horizontal writing character line exists in the scanned image, the horizontal writing character line is extracted together with information on which position of the left and right pages the horizontal writing character line exists. Remember temporarily.
[0152]
[Extraction of horizontal writing character line based on vertical writing character line]
Next, horizontal writing character lines are extracted from each vertical writing character line.
[0153]
FIG. 26 is a flowchart schematically showing a flow of a process of extracting a horizontally written character line from each vertically written character line. As shown in FIG. 26, first, a line cutout rectangle of a vertically written character line is extracted (step S201). In addition, since the well-known technique generally used in the OCR or the like can be used as it is for the extraction processing of the line cutout rectangle of the vertically written character line, the description thereof is omitted. FIG. 27 is an explanatory diagram exemplifying the extracted line cutout rectangle.
[0154]
Next, a vertical character line having the maximum (or minimum) y-coordinate at the beginning (or the end) of the vertical character line is extracted. A written character line is extracted (step S202: first character line extracting means). More specifically, in the example shown in FIG. 27, as shown in FIG. 28, the vertical character line having the maximum y coordinate of the first character of the vertical character line is the vertical character line indicated by A. . Then, a line head character existing within a predetermined distance range h from the head position is a character indicated by a black circle "●" in FIG. That is, only vertically written character lines including the character indicated by the black circle “●” are extracted, and the other vertically written character lines B and C are excluded. Here, h is a constant determined by the resolution of the scanned image.
[0155]
Next, a histogram is constructed for the y-coordinate of the head (or the end) of the extracted vertically written character line (step S203: histogram construction means). In FIG. 29, a vertically written character line D near the left end of the page is set as a reference line, and the y coordinate (y _D ) Are used as reference coordinates. Hereafter, y _D The number of vertical writing character lines having a head within a range of a fixed width d (for example, の of the average width of the extracted vertical writing character lines) is represented by y _D And the value of the histogram for In FIG. 29, y _D The vertical writing character line whose head exists within the range of the dotted line sandwiching the straight line indicating the above and below is the target. Therefore, the vertical writing character line E to the right of the vertical writing character line D near the left end of the page is out of the range. As described above, when a vertical writing character line that does not include the head appears in the target range of the existing reference coordinates, the vertical writing character line is set as a new reference line, and the starting coordinates are set to the new reference coordinates (here, , Y _E ). Also, the line start coordinate of the vertical writing character line F on the right of the vertical writing character line E is y _D , And without providing new reference coordinates, y _D The value of the histogram related to is incremented by one.
[0156]
Hereinafter, the same processing is continued toward the binding part boundary line. As a result, in the example shown in FIG. _D The vertical writing character lines included in the target range are seven enclosed by a hatched rectangle. _E The vertical writing character lines included in the target range are four rectangles surrounded by hatched rectangles. Although it is determined, it is omitted in FIG. 29). Note that y _D The target range includes a vertically written character line G which should be irrelevant, but this is excluded in the next step S204.
[0157]
Subsequently, among the vertical character lines included in the target range of the reference line corresponding to the maximum value in the histogram constructed in step S203, the vertical character line at the left end (or right end) of the page With the (reference line) as the start line, a vertically written character line whose y coordinate at the beginning (or the end) is close to the binding portion boundary line is extracted (step S204: second character line extraction means). In FIG. 29, the reference coordinate y _D Since the maximum number of character lines included in the target range is seven, the leftmost vertical writing character line D among them is set as a starting line, and the leading line is started from the starting line (vertical writing character line D) toward the binding part boundary line. Vertically written character lines whose y coordinates are close to each other are extracted.
[0158]
By the way, when extracting a vertical writing character line in which the leading y coordinate is close to the binding line boundary line from the start line (vertical writing character line D), a portion where the image is not distorted and a distortion are generated The processing contents are switched depending on the part that is present.
[0159]
First, processing in a portion where no image distortion occurs will be described with reference to FIG. In a portion where the image is not distorted, a vertical writing character line satisfying the following two conditions is extracted based on the line of interest H.
1. In the positive direction of the y-coordinate (in the upward direction in FIG. 30), the top of a vertically written character line to be extracted within a certain range b1 (eg, の of the average character line width) from the top position of the line of interest H To do
2. Regarding the negative direction of the y-coordinate (the downward direction in FIG. 30), a predetermined angle (here, the direction toward the binding portion boundary line) with respect to the positive direction of the x-coordinate (the direction toward the binding portion boundary line) as viewed from the head position of the line of interest H. In this case, the angle of the line is represented by the slope (b2 / a1) of the straight line.
That is, the beginning of the vertically written character line I next to the line of interest H is outside the above range, and thus is excluded. However, since the beginning of the next vertically written character line J is within the range, it is extracted. . Hereinafter, similar processing is continued with the vertically written character line J as a new line of interest.
[0160]
Next, processing in a portion where image distortion occurs will be described with reference to FIG. In the portion where the image is distorted, a vertical writing character line that satisfies the following two conditions is extracted based on the line of interest L.
1. With respect to the negative direction of the y-coordinate (the downward direction in FIG. 31), a predetermined angle (here, the direction toward the binding portion boundary line) with respect to the positive direction of the x-coordinate (the direction toward the binding portion boundary line) as viewed from the head position of the line of interest L. In this example, the angle is represented by the inclination of the straight line (b3 / a2). However, in the portion where the distortion occurs, the b2 / a1 < b3 / a2) must be within the range of the vertical writing character line to be extracted.
2. The slope (b4 / a2) of the straight line connecting the head position of the line of interest L and the head position of the vertically written character line to be extracted is the slope (b5) of the line connecting the head position of the line of interest L and the head position of the immediately preceding extracted line K. / A3) is larger than a value obtained by subtracting the constant value α from / a3). That is, “b4 / a2> b5 / a3-α” is satisfied (basically, “b4 / a2> b5 / a3” may be satisfied, but a constant value α is introduced in consideration of an error. α is a predetermined value)
That is, the beginning of the vertically written character line M following the line of interest L is outside the above condition, and is therefore excluded. However, the beginning of the next vertically written character line N satisfies the condition and is extracted. Hereinafter, similar processing is continued with the vertically written character line N as a new line of interest.
[0161]
The problem here is how to distinguish a portion where distortion is not generated from a portion where distortion is generated. This is performed as follows. That is, the y-coordinates at the head of the line of interest and the next line are _C , Y _N Then, "y _N -Y _C Is greater than or equal to a certain value (for example, 1/4 of the average character line width), the portion after that is regarded as a portion where distortion occurs.
[0162]
The vertically written character lines extracted from FIG. 29 by the above method are shown in FIG. 32 by enclosing them with hatched rectangles.
[0163]
Finally, an approximate curve polynomial for the position coordinates of the head (or end) of the extracted vertical writing character line is calculated (step S205). When the outer shape is formed by connecting the tops of the extracted line cutout rectangles of each vertical writing character line, as shown in FIG. 33, based on the center point of the upper side of the line extraction rectangle of each connected vertical writing character line, Calculate an approximate curve polynomial relating to the position coordinates of the head of the extracted vertical writing character line. In addition, when the outer shape is formed by connecting the ends of the line cutout rectangles of the extracted vertical writing character lines, as shown in FIG. Then, an approximate curve polynomial for the position coordinates at the end of the extracted vertical writing character line is calculated.
[0164]
Finally, the outline of the inappropriate vertical writing character line is excluded. This is because, as described above, when estimating the coordinate value by the polynomial approximation, if the shape of the estimated curve by the polynomial approximation is inappropriate, the distortion may increase instead of the correction. This eliminates the outer shape of the vertical writing character line. Examples of inappropriate approximate curve shapes are the same as in the case of ruled lines and horizontal writing lines described above. Although not specifically shown, the case where the curve goes to the outside of the book, or the curve goes far beyond the center line This is the case when you dig into it.
[0165]
If the shape of the vertical writing character line is excluded because the shape of the estimated curve is inappropriate, it means that there is no external shape of the vertical writing character line for distortion correction.
[0166]
If it is determined by the above processing that the outer shape of the vertically-written character line exists in the scanned image, the vertical-written character line is displayed together with information on where the outer shape of the vertically-written character line exists on each of the left and right pages. The outline of the row is extracted and stored in the RAM 33 temporarily.
[0167]
In the following, the outlines of the horizontal writing character line and the vertical writing character line are treated as character lines.
[0168]
As described above, the process of extracting the page outline / ruled line / character line (step S4) is completed by the processes of steps S41 to S43.
[0169]
In the following step S5 (see FIG. 6), an image distortion correction process is performed. As shown in FIG. 34, the image distortion correction processing is performed by roughly setting a line (reference line) serving as a reference for distortion correction (expansion) near the upper side (or lower side) of the scanned image. A process of selecting either a ruled line or a character line (step S51: reference line selection process), which corresponds to the reference line and serves as a reference line for calculating a correction rate (extension rate), ie, the upper side (or lower side) of the scanned image )) (Step S52: reference line selection processing) for selecting any one of the page outline / ruled line / character line located in the vicinity of ()). If the reference line is a ruled line or a character line, the image information below the reference line is missing. (Step S53: virtual page outline calculation process) for calculating a virtual page outline for minimizing the distortion, and applying a decompression process to the scanned image based on the virtual page outline to obtain distortion in the main scanning direction. Correction processing (step S54: main scanning direction distortion correction processing), processing of performing expansion processing on the scanned image based on the character circumscribed rectangle of the corrected image to correct distortion in the sub scanning direction (step S55: sub scanning direction distortion correction) Processing).
[0170]
Here, in the reference line selection processing (step S51) and the reference line selection processing (step S52), a page outline / ruled line / character line located near the upper side (or lower side) of the scanned image is used as the reference line or reference line. In this embodiment, the selection order of the page outline, ruled line, and character line is
Page outline> ruled line> character line
It has been. The selection priority is such that a character line has a lower extraction accuracy than a page outline or a ruled line, and a more accurate distortion correction rate can be obtained by using a page outline outside an image. This is because we can do it. However, in the present embodiment, the page outline generally extends to the left and right pages, but ruled lines and character lines are not always present on both pages. Therefore, the priority of the reference line selection from the upper side or the lower side of the scanned image is: It is as shown below.
[0171]
1. "Page outline" (Page outline always exists on both left and right pages)
2. "Rules" for both left and right pages
3. One page is "ruled line", the other page is "character line"
4. "Character line" for both left and right pages
5. "Borders" on one page only, "No clues" on the other page
6. "Character line" on one page only, "No clue" on the other page
Here, "no clue" means that none of the page outline, ruled lines, and character lines could be extracted.
[0172]
Next, examples of the virtual page outer shape calculation process (step S53) and the main scanning direction distortion correction process (step S54) will be described in order based on the above-mentioned priorities.
[0173]
1. When “Page Outline” Exists in Both Upper Side and Lower Side of Scanned Image Here, a case where “page outline” exists in both the upper side and lower side of the scanned image will be described. As shown in FIG. 35, when the “page outline” exists on both the upper side and the lower side of the scanned image, any of the reference line and the reference line may be used. In this embodiment, the “page outline”, the “ruled line”, and the “character line” selected as the reference lines are positioned on the lower side. In this case, since the reference line matches the virtual page outline, the virtual page outline calculation processing (step S53) does not need to perform any particular processing.
[0174]
Next, the main scanning direction distortion correction processing (step S54) will be described. Here, x0 is a boundary point between a straight line portion and a curved portion of the “page outline”. In the main scanning direction distortion correction processing, first, a distortion correction rate is calculated based on a distance h0 (see FIG. 36) between the reference line and the reference line at the boundary point x0. The distortion correction rate is used to make the distance between the reference line and the reference line equal at all positions in the sub-scanning direction when the image is expanded and corrected in the main scanning direction. That is, when the distance between the reference line and the reference line at the position x is h (see FIG. 36), the distortion correction rate at the position x is
h0 / h
Can be expressed as In other words, the distortion correction rate is defined for each x so that the distance between the reference line and the reference line is equal at all positions x in the sub-scanning direction when the image is expanded and corrected in the main scanning direction. This is a calculated value.
[0175]
Next, when actually performing the correction, first, each pixel is shifted in the main scanning direction so that the virtual page outer shape (here, the reference line) becomes a straight line (the lowest side) as shown in FIG. Thereafter, if the image is extended at h0 / h toward the upper side in the main scanning direction at all positions x in the sub-scanning direction, the image is corrected so that the “page outline” E on the upper side in FIG. 36 becomes RE. .
[0176]
Here, the function of the page outline image distortion correction means is executed.
[0177]
2. "Page outline" exists on one of the upper side and lower side of the scanned image, and "ruled line" and "character line" exist on the other side
Here, a case will be described in which a “page outline” exists on one of the upper side and the lower side of the scanned image, and the “page outline” is cut off on the other side but “ruled lines” and “character lines” exist. I do. As shown in FIG. 37, when the “page outline” exists on one of the upper side and the lower side of the scanned image and the “ruled line” and “character line” exist on the other side, the “page outline” Is located on the lower side as a reference line, and “character line” and “ruled line” are used as reference lines (in the example shown in FIG. 37, the left page is “character line” and the right page is “ruled line”). Also in this case, since the reference line and the virtual page outline match, the virtual page outline calculation processing (step S53) does not need to perform any particular processing.
[0178]
Next, the main scanning direction distortion correction processing (step S54) will be described. Here, x0 is a boundary point between a straight line portion and a curved portion of the “page outline”. In the main scanning direction distortion correction processing, first, a distortion correction rate is calculated based on the distance h0 (see FIG. 38) between the reference line and the reference line at the boundary point x0. When the distance between the reference line and the reference line at the position x is h (see FIG. 38), the distortion correction rate at the position x is
h0 / h
Can be expressed as
[0179]
Next, when actually performing the correction, first, each pixel is shifted in the main scanning direction so that the virtual page outer shape (here, the reference line) becomes a straight line (the lowest side) as shown in FIG. Thereafter, at all positions x in the sub-scanning direction, if the image is extended by h0 / h to the upper side in the main scanning direction, the image is expanded so that the “ruled line” LU on the upper side becomes RLU on the right page in FIG. Will be corrected. The same applies to the “character line” on the upper side of the left page.
[0180]
Here, the function of the page outline image distortion correction means is executed.
[0181]
3. When a "ruled line" exists on one of the upper and lower sides of the left and right pages of the scanned image, and a "ruled line" and "character line" exist on the other side
Here, a case where a “ruled line” exists on one of the upper side and the lower side of both left and right pages of the scanned image, and a “ruled line” and “character line” exist on the other side will be described. When a “ruled line” exists on one of the upper side and the lower side of the left and right pages of the scanned image and a “ruled line” and “character line” exist on the other side, as shown in FIG. The side where the “ruled line” exists on both pages is positioned on the lower side, the two “ruled lines” are used as reference lines, and the “character line” and “ruled line” located on the other side are used as reference lines (see FIG. In the example shown in FIG. 39, the left page is “character line” and the right page is “ruled line”). By the way, when the reference line is not the “page outline”, the reference line should not be regarded as the virtual page outline as it is. This is because, when the two are matched, when the virtual page outline (reference line) is shifted to the lowermost side in the subsequent main scanning direction distortion correction processing, all the image information below the reference line is lost. .
[0182]
Therefore, if the reference line is not the “page outline”, a virtual page outline calculation process (step S53) for obtaining a virtual page outline VE as shown by a dotted line in FIG. 39 is executed. Here, x0 is a boundary point between the straight line portion and the curved portion of the "ruled line" LD on the lower side. The virtual page outline calculation process is based on the distance a0 from the “ruled line” LD at the boundary point x0 to the bottom side of the scanned image and the distance b0 from the “ruled line” LD at the boundary point x0 to the “ruled line” LU on the upper side. This is for calculating the virtual page outline VE. That is, if the distance b from the position x on the “ruled line” LD to the “ruled line” LU on the upper side is known,
a / b = a0 / b0
Therefore, the distance a from the position x on the “ruled line” LD to the virtual page outline VE can be calculated. Therefore, the virtual page outline VE can be obtained by calculating the distance from the position on the "ruled line" LD to the virtual page outline VE at all positions x in the sub-scanning direction. Such processing is performed independently for the left and right pages.
[0183]
Next, the main scanning direction distortion correction processing (step S54) will be described. In the main scanning direction distortion correction processing, first, the distance a0 from the “ruled line” LD at the boundary point x0 to the lowermost side of the scanned image and the distance from the “ruled line” LD at the boundary point x0 to the “ruled line” LU on the upper side are described. The distortion correction rate is calculated based on a distance h0 (h0 = a0 + b0) obtained by adding the distance b0. If the distance from the “ruled line” LU on the upper side at the position x to the lowermost side of the scanned image is h (h = a + b), the distortion correction rate at the position x is
h0 / h
Can be expressed as
[0184]
Next, when actually performing the correction, first, each pixel is shifted in the main scanning direction so that the virtual page outline VE becomes a straight line (lowest side) as shown in FIG. Thereafter, at all positions x in the sub-scanning direction, if the image is extended by h0 / h toward the upper side in the main scanning direction, the “ruled line” LU on the upper side is RLU (strictly speaking, x Is not coincident with the boundary between the straight part and the curved part of the “ruled line” LU, but the image is corrected so as to be the boundary of the “ruled line” LD as it is. The same applies to the “character line” on the upper side of the left page.
[0185]
In this way, by using the virtual page outline VE, it is possible to minimize the loss of the image due to the shift.
[0186]
Here, the function of the ruled line / character line image distortion correcting means is executed.
[0187]
4. When a "ruled line" and a "character line" exist on one of the upper and lower sides of both left and right pages of a scanned image, and a "ruled line" exists only on one page of the other side
Here, a case will be described in which a “ruled line” and a “character line” exist on one of the upper side and the lower side of both left and right pages of the scanned image, and a “ruled line” exists on only one page on the other side. . When a “ruled line” and a “character line” exist on one of the upper side and the lower side of the left and right pages of the scanned image, and a “ruled line” exists only on one page of the other side (the other page is “ As shown in FIG. 41, the side where the “ruled line” and the “character line” exist is located on the lower side, and the “ruled line” and the “character line” are set as the reference line, and The "ruled line" located on the side and the center line C passing through the scanned image in the sub-scanning direction are used as reference lines (in the example shown in FIG. 41, the left page is "ruled line" and the right page is "no clue"). . The center line C is a line that crosses the center of the scanned image in the main scanning direction in the sub scanning direction, and is not the center line of the book document 40.
[0188]
When the reference line is not the “page outline”, as described above, the virtual page outline calculation processing (step S53) for obtaining the virtual page outline VE as shown by the dotted line in FIG. 41 is executed. Here, x0 is a boundary point between the straight line portion and the curved portion of the "ruled line" LD on the lower side. The virtual page outline calculation processing is performed based on the distance a0 from the “ruled line” LD at the boundary point x0 to the lowermost side of the scanned image and the distance b0 from the “ruled line” LD at the boundary point x0 to the center line C. VE is calculated. That is, if the distance b from the position x on the “ruled line” LD to the center line C is known,
a / b = a0 / b0
Therefore, the distance a from the position x on the “ruled line” LD to the virtual page outline VE can be calculated. Therefore, the virtual page outline VE can be obtained by calculating the distance from the position on the "ruled line" LD to the virtual page outline VE at all positions x in the sub-scanning direction. Such processing is performed independently for the left and right pages. When a "ruled line" exists on the upper side as in the right page, the "ruled line" on the upper side is used instead of the center line C as described above.
[0189]
Next, the main scanning direction distortion correction processing (step S54) will be described. In the main scanning direction distortion correction processing, first, the distance a0 from the “ruled line” LD at the boundary point x0 to the lowermost side of the scanned image and the distance b0 from the “ruled line” LD at the boundary point x0 to the center line C are calculated. The distortion correction rate is calculated based on the distance h0 (h0 = a0 + b0) to which is added. If the distance from the center line C at the position x to the lowermost side of the scanned image is h (h = a + b), the distortion correction rate at the position x is
h0 / h
Can be expressed as
[0190]
Next, when actually performing the correction, first, each pixel is shifted in the main scanning direction so that the virtual page outline VE becomes a straight line (the lowest side) as shown in FIG. Thereafter, at all positions x in the sub-scanning direction, the image is extended and corrected at h0 / h to the upper side in the main scanning direction. The same applies to the “ruled line” on the upper side of the left page. For the right page, the center line C once distorted by the shift due to the correction returns to the original straight line RC, but the correction of the entire image is incomplete unless C coincides with the optical axis of the scanner unit 1. .
[0191]
Here, the function of the ruled line / character line image distortion correcting means is executed.
[0192]
5. When a "ruled line" exists on one of the upper and lower sides of the left or right page of the scanned image, and a "character line" exists only on the other page of the other side
Here, a case will be described in which a “ruled line” exists on one of the upper side and the lower side of one of the left and right pages of the scanned image, and a “character line” exists only on the other page on the other side. A “ruled line” exists on one of the upper side and the lower side of one of the left and right pages of the scanned image (the other page has “no clue”), and a “character line” exists only on the other page on the other side. In the case (one page has “no clue”), as shown in FIG. 43, the side where the “ruled line” exists is positioned on the lower side, and the “ruled line” is used as the reference line, and the “ruled line” is not used. For the page (the page on which the “character line” exists), a curve SL obtained by moving the character line L to a position symmetrical with respect to the center line C is used as a reference line. Regarding the reference line, the page having the “ruled line” is the center line C, and the page having the “character line” is the “character line” as the reference line.
[0193]
Note that the calculation of the distortion correction rate, the pixel shift, and the correction (expansion) processing are the same as those in the previous examples, and thus description thereof will be omitted.
[0194]
Here, the function of the ruled line / character line image distortion correcting means is executed.
[0195]
Finally, the sub-scanning direction distortion correction processing (step S55) will be described. Here, FIG. 44 is a flowchart schematically showing the flow of the sub-scanning direction distortion correction processing. As shown in FIG. 44, in step S101, a circumscribed rectangle A (see FIG. 45) of a character is extracted based on a scan image in which distortion in the main scanning direction has been corrected. Here, the character recognition processing is a known technique, and a description thereof will be omitted. The reason why the circumscribed rectangle A of the character is extracted in this way is to correct distortion in the sub-scanning direction based on a change in the shape of the circumscribed rectangle A of the character. Here, as shown in FIG. 45, the length w of the horizontal side, the length h of the vertical side, and the center B of the character of the character circumscribed rectangle A are defined. Here, the center B of the character is the intersection of the diagonal lines of the circumscribed rectangle A.
[0196]
Subsequently, as shown in FIG. 46, after the scan image is divided into a plurality of strip-shaped areas C in a direction parallel to the page binding section 41 of the book document 40 (step S102), each strip area C A feature amount relating to the included character circumscribed rectangle A is obtained (step S103). Here, the character circumscribed rectangle A included in a certain strip area C is a circumscribed rectangle A whose center is included in the strip area C. For example, the circumscribed rectangle A included in the strip area C1 in FIG. 46 is a shaded rectangle in the figure.
[0197]
By the way, the feature quantity relating to the character circumscribed rectangle A is
(Length of horizontal side of character) / (length of vertical side of character) = w / h
Required based on. That is, with respect to each strip area C, the average value of the w / h values of all the character circumscribed rectangles A included therein is used as the feature amount of the strip area C.
[0198]
However, simply calculating the average value of w / h may be inappropriate. Some characters, such as punctuation marks and symbols in mathematical formulas, are originally small in size and have unstable w / h values. In addition, in extracting a rectangle, adjacent characters may be stuck together and extracted, resulting in a character circumscribed rectangle A where w is extremely large. When a feature amount is obtained, it is necessary to eliminate such special characters and those having an extremely large w in advance. Therefore, in the subsequent step S104, a threshold value is set in advance, the character circumscribed rectangle A having a value of h smaller than the threshold value is excluded in advance, and a threshold value for the w / h ratio is set in advance. Is also excluded in advance. For example, the shaded character circumscribed rectangle A shown in FIG. 47 is eliminated in advance.
[0199]
In the following step S105, after excluding the extreme character circumscribed rectangle A as described above, the average value of w / h of the character circumscribed rectangle A in each strip region C is obtained. FIG. 48 shows an example of the average value of w / h of the circumscribed rectangle A in each strip region C. The strip area C2 in FIG. 48 is a strip area including the page binding section 41 of the book document 40.
[0200]
Subsequently, it is determined whether or not the character circumscribed rectangle A exists in the strip area C2 including the page binding section 41 of the book document 40 (step S106). This is because, as shown in FIG. 47, in general, the character circumscribed rectangle A does not generally exist near the page binding section 41 of the book document 40 in many cases. When the character circumscribed rectangle A exists in the strip area C2 including the page binding section 41 of the book document 40 (Y in step S106), the feature amount is calculated using the character circumscribed rectangle A, so that step S108 is performed as it is. Proceed to.
[0201]
On the other hand, if the character circumscribed rectangle A does not exist in the strip area C2 including the page binding section 41 of the book document 40 (N in step S106), the process proceeds to step S107, and the strip area C2 including the page binding section 41 of the book document 40 is determined. Is obtained. The strip area C2 including the page binding section 41 of the book document 40 is identified, for example, by calculating the background density change of a scanned image (for example, a monochrome multivalued image) for each strip area C, This is realized by obtaining a low density value. FIG. 49 shows an example in which the background density change is obtained. The strip region having the highest background density is regarded as the strip region C2 including the page binding portion 41 of the book document 40.
[0202]
In the case where the scanned image is a color multi-valued image, the identification of the strip region C2 including the page binding portion 41 of the book document 40 focuses on, for example, one of the RGB components (for example, the G component), and focuses on the G component. What is necessary is just to make it distinguish using the background density of. Alternatively, the RGB area may be converted into a luminance component and a color difference component by color conversion, and the strip region C2 including the page binding portion 41 of the book document 40 may be identified using the luminance component.
[0203]
The feature amount of the strip area C2 including the page binding section 41 of the book document 40 is determined as follows. Here, there is a character circumscribed rectangle A that can be a statistical feature amount calculation target, and a predetermined constant value for the feature amount of the strip region C that is the closest to the strip region C2 including the page binding unit 41. Is calculated as the characteristic amount in the strip region C2 including the page binding portion 41 of the book document 40. That is, in the example shown in FIG. 48, since the character circumscribed rectangle A exists in any of the left and right strip areas C3 and C4 of the strip area C2 including the page binding section 41 of the book document 40, whichever is more appropriate. An amount is selected (here, the circle on the right side), multiplied by a predetermined constant value (here, 0.5), and is multiplied by the characteristic amount of the strip area C2 including the page binding section 41 of the book document 40. And
[0204]
In the subsequent step S108, appropriate filtering processing for the feature amount of each strip area C, for example, processing for obtaining a moving average in the direction of change in the position of the strip area C (ie, the sub-scanning direction) is performed. The change of the feature amount (in the sub-scanning direction) with respect to the change of the position is made gentle. However, special processing is required also in the vicinity of the page binding section 41 of the book document 40 here. This is because if filtering is performed using windows having the same length in the sub-scanning direction, the sharpness of the change in the feature amount near the page binding unit 41 of the book document 40 is lost.
[0205]
Here, FIG. 50 shows a result obtained by performing a filtering process on the feature amount of each strip region C shown in FIG. 48 using a window whose length is all five. As shown in FIG. 50, when the filtering process is performed using the windows whose lengths are all 5, the change in the feature amount (w / h) near the page binding unit 41 of the book document 40 becomes gentle. It will pass. In such a case, appropriate image correction near the page binding section 41 of the book document 40 becomes impossible.
[0206]
Therefore, in the present embodiment, during the filtering process, the page binding of the filter is performed so that the window of the filter does not straddle the strip areas C3 and C4 on both sides of the strip area C2 including the page binding section 41 of the book document 40. The window length is adjusted near the section 41. Here, FIG. 51 is a graph showing the result of performing the filtering process by adjusting the window length near the page binding unit 41. As shown in FIG. 51, when the window length is adjusted in the vicinity of the page binding unit 41, a change in the characteristic amount (w / h) in the vicinity of the page binding unit 41 can be appropriately expressed, so that good image correction is realized. it can.
[0207]
In the following step S109, the estimated distortion amount of each strip region C is calculated. The calculation method of the estimated distortion amount of each strip region C is as follows.
[0208]
First, a strip area (reference strip area) serving as a reference for calculating the distortion amount of the strip area is determined. Here, the strip area C with the smallest distortion, for example, the strip area C with the largest characteristic amount (w / h) is set as the reference strip area. This processing may be performed in common for the left and right pages, but the reference strip area may be independently defined for the left and right pages. FIG. 51 shows an example in which the reference strip regions are defined independently for the left and right sides. A strip region C marked with a circle is the reference strip region, and the reference feature amount on the left is “Lw0 / Lh0” and the reference feature on the right is The feature amount is indicated by “Rw0 / Rh0”, respectively.
[0209]
Next, the feature amount w0 / h0 of the reference strip region is set as the reference feature amount of the entire scanned image,
(Feature value of each strip region) / (Reference feature value) = (w / h) / (w0 / h0)
Is calculated as the estimated distortion amount of each strip region.
[0210]
If the strip area C near the outside of the page outside the page binding section 41 of the book document 40 is set as the reference strip area, the difference in the size of fonts and prints from the vicinity of the page binding section 41 of the book document 40 is large. It is also conceivable that a large estimated distortion amount cannot be calculated. When such an image is targeted, it is effective to limit the search range of the reference strip region to the vicinity of the page binding section 41 of the book document 40 in advance. In order to realize this, the reference strip region may be obtained only from the strip region C whose background density is higher than a predetermined density.
[0211]
Finally, enlargement processing is performed on the scanned image in the short side direction (sub-scanning direction) of the strip area C to correct distortion near the page binding section 41 of the book document 40 (step S110). The enlargement ratio in that case is the reciprocal of the estimated distortion amount calculated in step S109, that is,
(Reference feature amount) / (Feature amount of each strip region) = (w0 / h0) / (w / h)
And Here, when the above-mentioned reference strip region is defined in common for the left and right, this enlargement ratio is also calculated using the reference feature amounts common to the left and right, and when independently defined, it is independently calculated for each of the right and left reference feature amounts. To do. FIG. 52 illustrates the correction magnification calculated based on the feature amounts illustrated in FIG. 51.
[0212]
In this case as well, the strip region C of the book document 40 distant from the vicinity of the page binding portion 41 is likely to be a region in which the image is not originally distorted. This is because there is a possibility that unnatural distortion may be caused by performing the enlargement processing. In order to prevent this, the estimated distortion amount is set to “1” for the strip region C whose background density is lower than a predetermined density.
[0213]
Further, when a common correction magnification rate is applied in the strip area C, the correction magnification rate at the boundary between the adjacent strip areas C becomes discontinuous, and the corrected image becomes unnatural. Therefore, the correction magnification is corrected in advance so that the correction magnification at the boundary between the adjacent strip regions C changes continuously. This is achieved by plotting the corrected enlargement ratio of the central portion of the strip region C shown in FIG. 52 as points indicating the reciprocal of the estimated distortion amount, connecting these points with a line segment, and complementing the straight line, thereby obtaining other portions. This can be realized by setting the correction magnification. With the above processing, the correction magnification rate in the sub-scanning direction of the scanned image is determined.
[0214]
The image enlargement process is performed using, for example, a convolution method of a cubic function often used as a copy scaling function.
[0215]
As described above, the sub-scanning direction distortion correction processing (step S55) is completed by the processing of steps S101 to S110, and the distortion correction processing of the scanned image illustrated in FIG. 6 is completed. Here, FIG. 53 is a plan view showing an image in which distortion has been corrected. According to the above processing, the distortion of the scanned image generated near the page binding section 41 of the book document 40 as shown in FIG. 8 is corrected as shown in FIG.
[0216]
Here, the vertical writing character line used for the calculation of the polynomial approximation curve is extracted by limiting the vertical writing character line to those having a certain positional relationship based on the position information of the character line head (or the character line end). . This makes it possible to extract vertical writing character lines limited to those having a certain positional relationship based on the position information of the character line head (or character line end) of the vertical writing character line. A good correction result can be obtained even when the position of the character at the top (or the bottom) of the character line fluctuates greatly in the vertical direction for each line.
[0217]
Although the present embodiment has been described with reference to the upper left portion of a two-page spread of a book document, basically the same processing is performed for the other lower left, upper right, and lower right portions. However, depending on the location, the first character of the line is replaced with the last character, the left end of the page is the right end, the line with the largest y coordinate is the smallest line, and the positive (negative) direction of the y coordinate is negative. It is necessary to read each of them in the (positive) direction.
[0218]
Next, a second embodiment of the present invention will be described. The same parts as those described in the first embodiment are denoted by the same reference numerals, and description thereof is omitted. This embodiment is different from the embodiment in the processing content of step S202 described with reference to FIG.
[0219]
In the present embodiment, if the number of vertically written character lines extracted in step S202 is smaller than a predetermined value, a stable approximate curve polynomial for the leading position of the extracted line cannot be calculated in step S205. With respect to the remaining lines excluding the writing character line, similar vertical character line extraction processing is performed again.
[0220]
Here, FIG. 54 is an explanatory diagram showing the process of extracting vertically written character lines according to the present embodiment. Here, the minimum number of character lines to be extracted is set to 5. In FIG. 54A, the vertical writing character line in which the y coordinate of the first character of the line is the maximum is the vertical writing character line indicated by A. Then, the line head characters existing within a predetermined distance range h from the head position are the characters indicated by black circles “●” in FIG. 54A, but the number is as small as three. Therefore, in the present embodiment, the same processing is performed once again except for the vertically written character line including the extracted three leading characters. That is, as shown in FIG. 54 (b), a vertical character line B having the maximum y coordinate of the first character among the remaining vertical character lines is obtained, and a predetermined distance from the head position of the vertical character line B is determined. A line head character (indicated by a black circle “●”) existing in the range h is extracted again. As shown in FIG. 54B, twelve line head characters (indicated by black circles “丸”) within the distance range h are extracted.
[0221]
Therefore, in the present embodiment, the processing after step S203 is executed according to the twelve vertical character lines including the twelve line head characters.
[0222]
Here, when the number of extracted vertical writing character lines is small, it is difficult to calculate a stable approximated curve polynomial. Therefore, by performing vertical character line extraction processing again, a stable approximated curve polynomial is calculated. Can be achieved.
[0223]
Even when the number of vertical writing character lines extracted in step S204 is smaller than a predetermined value, similar vertical writing character line extraction processing is performed again on the remaining lines excluding the extracted vertical writing character lines. You may do it.
[0224]
Next, a third embodiment of the present invention will be described. The same parts as those described in the first embodiment are denoted by the same reference numerals, and description thereof is omitted. This embodiment is different from the embodiment in the processing content of step S204 described with reference to FIG.
[0225]
In this embodiment, in step S204, if there is no vertically written character line extracted within a certain range from the binding portion boundary line, a stable approximate curve polynomial near the binding portion boundary line cannot be calculated. With respect to the remaining lines excluding the writing character line, similar vertical character line extraction processing is performed again.
[0226]
Here, FIG. 55 is an explanatory diagram showing the process of extracting a vertically written character line according to the present embodiment. Here, the fixed range W from the binding portion boundary line is set to 1/8 of the width of the entire image. FIG. 55A corresponds to the histogram information configured in step S203. As shown in FIG. 55 (a), the start coordinate y of the vertical writing character line A _A The vertical coordinate character line included in the target range is the largest with seven lines, and the next largest is the top coordinate y of the vertical character line B _B Are four vertical writing character lines included in the target range. Therefore, with the vertical writing character line A as the starting line, vertical writing character lines with the closest y-coordinate are extracted (similar to the first embodiment). As a result, the extracted vertically-written character lines are shown enclosed by hatched rectangles in FIG. As shown in FIG. 55 (b), in this case, there is no extracted vertical writing character line within a certain range W from the binding part boundary line. Therefore, in the present embodiment, when there is no vertically written character line extracted within the fixed range W, the extracted vertical written character line is excluded, and the same processing is performed again. That is, in FIG. 55A, the target range (the head coordinate y _B The line extraction work is performed again with the vertical writing character line B corresponding to ()) as a starting line. As a result, the extracted vertical writing character line is shown by being surrounded by a hatched rectangle in FIG. As shown in FIG. 55 (c), in this case, there is a vertically written character line extracted within a certain range W from the binding part boundary line.
[0227]
Therefore, in the present embodiment, the processing after step S205 is executed according to the eight vertical character lines shown in FIG.
[0228]
Here, if the extracted vertical character line does not exist within a certain range from the binding portion of the book document, it is difficult to calculate a stable approximate curve polynomial, so the vertical character line extraction process must be performed again. Thereby, it is possible to stably calculate the approximate curve polynomial.
[0229]
Also in step S202, if there is no vertical writing character line extracted within a certain range from the binding portion boundary line, the remaining vertical lines excluding the extracted vertical writing character line are returned to the same vertical writing character line again. An extraction process may be performed.
[0230]
Next, a fourth embodiment of the present invention will be described. The same parts as those described in the first embodiment are denoted by the same reference numerals, and description thereof is omitted. This embodiment is different from the embodiment in the processing content of step S202 described with reference to FIG.
[0231]
An extremely thin vertical writing character line or a short vertical writing character line may be noise other than the main text or noise, and may be a hindrance factor for calculating a stable approximate curve polynomial. Therefore, in the present embodiment, the vertical character line extraction processing is performed by excluding a vertical character line having a predetermined width or less and a vertical character line having a line length equal to or less than a predetermined ratio with respect to the width. It is something to do.
[0232]
Here, FIG. 56 is an explanatory diagram showing the process of extracting vertically written character lines according to the present embodiment. In the present embodiment, a vertical writing character line whose vertical line width is 1 w or less and a vertical writing line whose line length is 1.5 times or less the line width are excluded. I have. For example, as shown in FIG. 56A, vertical character lines A and B are excluded because their length is 1.5 times or less the line width, and vertical character lines C have a line width of 1 w or less. Therefore, it is excluded. FIG. 56B illustrates a vertical character line in which the first character exists within a certain range h from a vertical character line D having the maximum y coordinate of the first character of the line after a predetermined vertical character line is excluded. It shows the results. The line head character existing within a predetermined distance range h from the head position of the vertical writing character line D is a character indicated by a black circle “●” in FIG. 56B.
[0233]
Here, the extremely thin vertical writing character line may be noise other than the main text or noise. By eliminating this, it is possible to calculate a stable approximate curve polynomial.
[0234]
Further, since an extremely short vertical character line may be noise other than the main text or noise, by eliminating this, a stable approximate curve polynomial can be calculated.
[0235]
Next, a fifth embodiment of the present invention will be described. The same parts as those described in the first embodiment are denoted by the same reference numerals, and description thereof is omitted. This embodiment is different from the first embodiment in the processing contents of steps S53 and S54 described with reference to FIG. In the present embodiment, the main scanning direction distortion correction processing (step S54) is performed without performing any processing in the virtual page outer shape calculation processing (step S53).
[0236]
The main scanning direction distortion correction processing (step S54) of the present embodiment will be described. In the main scanning direction distortion correction processing (step S54) of the present embodiment, the reference line selected in the reference line selection processing (step S51) and the reference line selection processing (step S51) are based on any of the page outline, ruled lines, and character lines. It is assumed that the reference line selected in step S52) exists on both the left and right pages of the scanned image.
[0237]
FIG. 57 is an explanatory diagram showing the main scanning direction distortion correction processing when a reference line or a reference line exists in both the left and right pages of the scanned image. In FIG. 57, the reference line and the reference line before distortion correction in the main scanning direction are indicated by solid lines, and the reference line and reference line after distortion correction in the main scanning direction are indicated by dotted lines. The dotted line after the main scanning direction distortion correction is obtained by extending the straight line portion (flat portion) of the reference line or the reference line before the main scanning direction distortion correction.
[0238]
As shown in FIG. 57, at the position x in the sub-scanning direction, the point P on the reference line is corrected to P ′ and the point Q on the reference line is corrected to Q ′. Now, assuming that an arbitrary point Y in the main scanning direction at the position x is corrected to Y ′, the following relational expression is obtained.
YP / YQ = Y'P '/ Y'Q'
Holds. Therefore, if the position of each point in the main scanning direction is expressed as P (x) for a point P,
(Y (y) -P (y)) / (Y (y) -Q (y)) = (Y '(y) -P' (y)) / (Y '(y) -Q' (y) )
So, transform this,
Y (y) = ((P (y) -Q (y)) / (P '(y) -Q' (y))) Y '(y) + (P' (y) Q (y) -P (Y) Q '(y)) / (P' (y) -Q '(y))
It becomes. Using this equation, the position Y (y) of the point that should come to the position of Y ′ (y) after the distortion correction in the main scanning direction before the distortion correction in the main scanning direction can be obtained.
[0239]
That is, in the present embodiment, the main scanning direction distortion correction processing is performed by using the above-described formula to shift the pixel value of Y (y) before the main scanning direction distortion correction to Y ′ (y) after the correction. Is to execute.
[0240]
However, since the calculated value of Y (y) is generally a decimal number, the image quality of the corrected image may be reduced. Therefore, in the present embodiment, in order to solve this, linear interpolation processing of pixels is also performed in order to use a linear interpolation result of pixel values corresponding to integer positions before and after the pixel position.
[0241]
Here, FIG. 58 is an explanatory diagram showing how to obtain pixel values by linear interpolation processing. As shown in FIG. 58, assuming that integers before and after Y (y) are N and N + 1, and corresponding pixel values are D (N) and D (N + 1), a pixel value D (Y) in Y (y) is obtained. (Y)) is calculated so as to satisfy the linear relationship shown in FIG.
[0242]
In each of the embodiments, the scanner unit 1 of the digital copying machine is applied as an image reading apparatus. However, the present invention is not limited to this, and may be applied to, for example, a scanner having an automatic page turning function. .
[0243]
In each embodiment, the book document 40 is placed on the contact glass 2 with the page binding section 41 of the book document 40 and the main scanning direction of image reading by the scanner section 1 positioned parallel to each other. Although the description has been given of the case where the display is placed, the present invention is not limited to this. For example, as shown in FIG. 59, an upward book document 40 may be brought into contact with the contact glass 2 by pressing the book document 40 from below.
[0244]
Further, in each of the embodiments, the image distortion correcting apparatus is provided in the digital copying machine 16 as the image forming apparatus, and the image distortion correction processing is performed on the scanned image read by the scanner unit 1 of the digital copying machine 16. However, the present invention is not limited to this. For example, an image scanner provided with an image reading means for reading a document image is connected to a personal computer, and a program stored in a CD-ROM 37 which is a storage medium is installed in an HDD of the personal computer, thereby implementing an image distortion correcting device. Even with such a configuration, it is possible to obtain the same functions and effects as those described above. Also, an image distortion correction device is configured by installing a program stored in a CD-ROM 37 as a storage medium into an HDD of a personal computer, and performs distortion correction processing on a scanned image read in advance by an image reading unit. You may do it.
[0245]
【The invention's effect】
According to the first and second aspects of the invention, the vertical writing character line used for calculating the polynomial approximation curve is determined to have a fixed positional relationship based on the position information of the character line head (or the character line end) of the vertical writing character line. By extracting only those having the vertical writing character line, it is possible to extract the vertical writing character line only to those having a certain positional relationship based on the position information of the beginning (or the end of the character line) of the vertical writing character line Therefore, a good correction result can be obtained even when the position of the character at the top (or the bottom) of the vertical writing character line fluctuates greatly in the vertical direction for each line.
[0246]
According to the second and 23rd aspects of the present invention, a vertical writing character line having the maximum (or minimum) coordinate in the binding portion direction at the head (or the end) is extracted from the vertical writing character line, and the extracted vertical writing character line is extracted. By extracting vertical writing character lines that have a head (or end) within a predetermined distance range from the head (or end) of the character line, the character line head (or character line end) of each vertical writing character line is extracted. The position can be limited to a predetermined range.
[0247]
According to the invention as set forth in claims 3 and 24, an extremely thin vertical writing character line may be noise other than the main text or noise. Can be calculated.
[0248]
According to the fourth and 25th aspects of the present invention, an extremely short vertical writing character line may be noise other than the main text or noise. Can be calculated.
[0249]
According to the inventions described in claims 5 and 26, when the number of extracted vertical writing character lines is small, it is difficult to calculate a stable approximate curve polynomial. Thus, stable calculation of the approximate curve polynomial can be achieved.
[0250]
According to the inventions described in claims 6 and 27, when the extracted vertically written character line does not exist within a certain range from the binding portion of the book document, it is difficult to calculate a stable approximate curve polynomial, so By performing the writing character line extraction processing, it is possible to calculate a stable approximate curve polynomial.
[0251]
According to the seventh and 28th aspects of the present invention, the fixed range from the binding portion of the book document is determined according to the width of the entire image, so that a stable approximate curve polynomial can be calculated.
[0252]
According to the eighth and 29th aspects of the present invention, the reference line having the maximum value in the histogram relating to the coordinates of the head (or end) of the vertical writing character line in the binding portion direction is set as the start line, and By extracting vertical writing character lines whose leading (or end) coordinates are close to each other, it is possible to limit the position of the character line head (or character line end) of each vertical writing character line to a narrower predetermined range. it can.
[0253]
According to the ninth and thirtieth aspects of the present invention, the reference line is determined, and the start (or end) of the line is within a fixed width range with respect to the coordinates of the start (or end) of the reference line in the binding portion direction. By setting the number of existing vertically-written character lines as the value of the histogram for the reference line, it is possible to reliably form a histogram for the coordinates of the head (or end) of the vertically-written character line in the binding portion direction.
[0254]
According to the tenth and thirty-first aspects of the present invention, a fixed width is determined with respect to the coordinates of the head (or the end) of the reference line in the binding portion direction according to the average line width of the extracted vertical writing character lines, thereby providing a stable approximation. It is possible to calculate a curve polynomial.
[0255]
According to the eleventh and thirty-second aspects of the invention, the extracted vertically written character lines are sequentially searched from the outside of the page toward the binding portion, and the coordinates of the head (or end) of the line of interest in the binding portion direction. Exists within a fixed width range with respect to the coordinates in the binding portion direction at the head (or end) of any of the reference rows that have already been determined, the histogram value for that reference row is set to one. If it increases and does not exist within the range of the fixed width, the approximate line polynomial can be stably calculated by regarding the line of interest as a new reference line.
[0256]
According to the twelfth and thirty-third aspects of the present invention, a vertically written character line which is originally irrelevant due to image distortion is extracted. By switching the criterion for extracting a vertically written character line whose direction coordinates are close to each other, a vertically written character line that is originally irrelevant can be excluded.
[0257]
According to the thirteenth and thirty-fourth aspects of the present invention, at the start of the extraction process, a portion having no distortion is generated, and the coordinates of the coordinates (in the binding part direction) of the head (or end) of two vertically written character lines that are successively extracted. If the difference is equal to or more than a certain value, the subsequent portion is regarded as a portion where distortion occurs, whereby a stable approximation curve polynomial can be calculated.
[0258]
According to the fourteenth and thirty-fifth aspects of the present invention, a stable approximation curve polynomial is calculated by determining a constant value relating to a difference in coordinates in the binding portion direction according to the average line width of the extracted vertical writing character lines. Can be.
[0259]
According to the invention as set forth in claims 15 and 36, the portion having no distortion is the head (or end) of the line of interest with respect to the direction toward the outside of the page of the book document at the coordinates of the binding portion direction with respect to the line of interest. The head (or end) of the vertical writing character line to be extracted within a certain range from the position exists, and the heading (or end) of the line of interest is directed to the inside of the page of the book document at the coordinates of the binding part direction. By extracting the next vertical writing character line so that the head (or the end) of the line to be extracted exists within a range of a predetermined fixed angle with respect to the direction toward the binding portion when viewed from the position, Vertically written character lines can be reliably extracted from portions where no image distortion occurs.
[0260]
According to the sixteenth and thirty-seventh aspects of the present invention, the portion where the distortion is generated is defined as the head (or the end) of the line of interest with respect to the direction toward the inside of the page of the book document at the coordinates of the binding portion direction with respect to the line of interest. ) The beginning (or end) of the line to be extracted exists within a range of a predetermined angle with respect to the direction toward the binding portion when viewed from the position, and the beginning (or end) position of the line of interest and the line to be extracted Is determined by subtracting a predetermined value from the slope of the line connecting the start (or end) position of the line of interest and the start (or end) position of the extracted line immediately before the line of interest. By extracting the next vertical writing character line so as to be larger than the value, it is possible to surely exclude a vertical writing character line that is originally irrelevant in a portion where the image is distorted.
[0261]
According to the invention as set forth in claims 17 and 38, when the number of extracted vertical writing character lines is small, it is difficult to calculate a stable approximated curve polynomial, so the vertical writing character line extraction process is performed again. Thus, stable calculation of the approximate curve polynomial can be achieved.
[0262]
According to the invention as set forth in claims 18 and 39, if the extracted vertical writing character line does not exist within a certain range from the binding portion of the book document, it is difficult to calculate a stable approximate curve polynomial. By performing the writing character line extraction processing, it is possible to calculate a stable approximate curve polynomial.
[0263]
According to the nineteenth and forty-ninth aspects, the fixed range from the binding portion of the book document is determined according to the width of the entire image, so that a stable approximate curve polynomial can be calculated.
[0264]
According to the image reading apparatus of the twentieth aspect, it is possible to obtain an image reading apparatus having the same operation and effect as the invention of any one of the first to nineteenth aspects.
[0265]
According to the image forming apparatus of the twenty-first aspect, it is possible to obtain an image forming apparatus having the same operation and effect as the invention of any one of the first to nineteenth aspects.
[0266]
According to the computer-readable storage medium of the invention described in claim 41, since the program according to any one of claims 22 to 40 is stored, the program stored in the storage medium can be read by the computer. By doing so, the same function and effect as the invention according to any one of claims 22 to 41 can be obtained.
[Brief description of the drawings]
FIG. 1 is a vertical sectional front view showing a configuration of a scanner unit according to a first embodiment of the present invention.
FIG. 2 is a perspective view showing an upper portion of a digital copier equipped with a scanner unit.
FIG. 3 is a block diagram illustrating electrical connection of a control system of the scanner unit.
FIG. 4 is a block diagram illustrating a basic internal configuration of an image processing unit.
FIG. 5 is a block diagram showing an electrical connection of a main control unit.
FIG. 6 is a flowchart schematically illustrating a flow of a scan image distortion correction process.
FIG. 7 is a perspective view showing a state in which a book document is placed on a contact glass of a scanner unit.
FIG. 8 is a plan view showing an example of an input image.
FIG. 9 is an explanatory diagram showing distortion near a page binding portion of a scanned image.
FIG. 10 is a black pixel histogram of the image shown in FIG. 8;
FIG. 11 is a flowchart schematically illustrating a flow of a page outline / ruled line / character line extraction process.
FIG. 12 is an explanatory diagram illustrating an example of a scanned image having a page outline at the upper end.
FIG. 13 is a black pixel histogram on the left side of the bounding portion boundary line of the scan image shown in FIG. 12;
FIG. 14 is an explanatory diagram illustrating an example of a scanned image including a long ruled line.
FIG. 15 is a black pixel histogram on the left side of the bounding portion boundary line of the scan image shown in FIG. 14;
FIG. 16 is an explanatory diagram illustrating determination of continuity of a ruled line.
FIG. 17 is an explanatory diagram showing the detection of the coordinates of a ruled line.
FIG. 18 is an explanatory diagram illustrating a case where one ruled line is selected from each of left and right pages.
FIG. 19 is an explanatory diagram showing a case where the left and right pages are further divided into upper and lower portions, and one ruled line is selected in each of the four blocks.
FIG. 20 is an explanatory diagram showing coordinate extraction of a ruled line.
FIG. 21 is an explanatory diagram illustrating an example of an inappropriate ruled line to be excluded;
FIG. 22 is a black-and-white inversion number histogram of the image shown in FIG. 8 in the sub-scanning direction.
23 is a black-and-white inversion number histogram of the image shown in FIG. 8 in the main scanning direction.
FIG. 24 is an explanatory diagram illustrating an example of a result of a character circumscribed rectangle extraction process and a character line extraction process of a scan image.
FIG. 25 is an explanatory diagram showing selection of an optimal horizontal writing character line.
FIG. 26 is a flowchart schematically showing a flow of a process of extracting a horizontally written character line from each vertically written character line.
FIG. 27 is an explanatory view exemplifying an extracted line cutout rectangle;
FIG. 28 is an explanatory diagram exemplarily illustrating a vertically written character line having a head within a predetermined distance range.
FIG. 29 is an explanatory diagram showing a state in which a histogram is formed with respect to the y-coordinate at the head of an extracted vertically written character line.
FIG. 30 is an explanatory diagram showing processing in a portion where no image distortion occurs.
FIG. 31 is an explanatory diagram showing processing in a portion where image distortion occurs.
FIG. 32 is an explanatory diagram showing extracted vertical writing character lines.
FIG. 33 is an explanatory diagram showing a line cutout rectangle of a vertically written character line.
FIG. 34 is a flowchart schematically showing a flow of an image distortion correction process.
FIG. 35 is an explanatory diagram showing a case where a “page outline” exists on both the upper side and the lower side of a scanned image.
FIG. 36 is an explanatory diagram showing a state where FIG. 35 is pixel-shifted.
FIG. 37 is an explanatory diagram showing a case where a “page outline” exists on one of an upper side and a lower side of a scanned image, and a “ruled line” and a “character line” exist on the other side.
FIG. 38 is an explanatory diagram showing a state where FIG. 37 is pixel-shifted.
FIG. 39 is an explanatory diagram showing a case where a “ruled line” exists on one of the upper side and the lower side of both left and right pages of a scanned image, and a “ruled line” and a “character line” exist on the other side. .
40 is an explanatory diagram showing a state where FIG. 39 is pixel-shifted.
FIG. 41 illustrates a case where a “ruled line” and a “character line” exist on one of the upper side and the lower side of both left and right pages of a scanned image, and a “ruled line” exists on only one page on the other side. FIG.
FIG. 42 is an explanatory diagram showing a state where FIG. 41 is shifted by a pixel;
FIG. 43 is an explanatory diagram showing a case where a “ruled line” exists on one of the upper side and the lower side of any one of the left and right pages of a scanned image, and a “character line” exists only on the other page on the other side; is there.
FIG. 44 is a flowchart schematically showing a flow of a sub-scanning direction distortion correction process.
FIG. 45 is an explanatory diagram showing extracted character circumscribed rectangles.
FIG. 46 is an explanatory diagram showing a state in which a binarized image is divided into a plurality of strip-shaped regions in a direction parallel to a page binding portion of a book document.
FIG. 47 is an explanatory diagram showing a character circumscribed rectangle that is excluded in advance.
FIG. 48 is a graph showing an example of the average value of the feature amounts of the circumscribed rectangles in each strip region.
FIG. 49 is a graph showing an example of background density change in each strip region.
50 is a graph showing a result of performing a filtering process on the feature amount of each strip region shown in FIG. 48 using a window whose length is all 5;
FIG. 51 is a graph showing a result of performing a filtering process by adjusting a window length near a page binding portion with respect to the feature amount of each strip region shown in FIG. 48;
FIG. 52 is a graph showing a corrected magnification calculated based on the feature amounts shown in FIG. 51.
FIG. 53 is a plan view showing an image in which distortion has been corrected.
FIG. 54 is an explanatory diagram showing a vertical character line extraction process according to the second embodiment of this invention;
FIG. 55 is an explanatory diagram showing an extraction process of a vertically written character line according to the third embodiment of this invention.
FIG. 56 is an explanatory diagram showing vertical character line extraction processing according to the fourth embodiment of this invention;
FIG. 57 is an explanatory diagram showing main scanning direction distortion correction processing according to the fifth embodiment of the present invention.
FIG. 58 is an explanatory diagram showing how to obtain a pixel value by a linear interpolation process.
FIG. 59 is a front view showing a state in which a book document is brought into contact with the contact glass.
FIG. 60 is a front view showing a state in which a book document is placed on the contact glass.
[Explanation of symbols]
1 Image reading device
16 Image forming apparatus
29 Image distortion correction device

Claims (41)

The distortion of the scan image obtained by reading the book document contacting above or below the scan surface with the image reading means is converted into a polynomial approximation curve relating to the position of the character line head (or character line end) of the vertical character line present in the scan image. In an image distortion correction device that corrects based on
Characters for extracting the vertical writing character line used for calculating the polynomial approximation curve by limiting the vertical writing character line to those having a certain positional relationship based on the position information of the character line head (or character line end) of the vertical character line An image distortion correction device comprising a line extracting unit. The character line extracting means,
From among the vertical writing character lines, the vertical writing character line having the maximum (or minimum) coordinate in the binding part direction at the beginning (or the end) is extracted, and the vertical writing character line is extracted from the head (or the end). 2. The image distortion correction device according to claim 1, further comprising a first character line extracting unit that extracts the vertical character line having a head (or end) within a predetermined distance range. 3. The image distortion correction apparatus according to claim 2, wherein the vertically written character line having a line width smaller than a predetermined value is excluded from the target of extraction. 3. The image distortion correction device according to claim 2, wherein the vertically written character line in which the ratio of the line length to the line width is smaller than a predetermined value is excluded from the extraction target. If the number of the vertical character lines extracted by the first character line extracting means is equal to or less than a predetermined value, the first character line 5. The image distortion correction device according to claim 2, wherein the extraction unit is executed again. If the vertical character line extracted by the first character line extraction means does not exist within a certain range from the binding part of the book document, the remaining vertical character line excluding the extracted vertical character line, The image distortion correction device according to claim 2, wherein the first character line extraction unit is executed again. 7. The image distortion correction apparatus according to claim 6, wherein a predetermined range from a binding portion of the book document is determined according to a width of the entire image. The character line extracting means,
Histogram forming means for forming a histogram with respect to the coordinates of the head (or end) of the vertical writing character line extracted by the first character line extracting means in the binding portion direction;
In the histogram constituted by the histogram constructing means, the reference line having the largest value is set as the start line, and the vertical writing character line whose leading (or end) coordinates are close to the binding portion direction is extracted. Second character line extracting means;
The image distortion correction apparatus according to claim 2, further comprising: The histogram structuring means determines a reference line, and the vertical writing character in which the head (or end) of the line exists within a fixed width range with respect to the coordinates of the head (or end) of the reference line in the binding portion direction. 9. The image distortion correction apparatus according to claim 8, wherein the number of rows is a value of the histogram for a reference row. 10. The fixed width with respect to the coordinates of the head (or the end) of the reference line in the binding portion direction is determined according to the average line width of the vertically written character lines extracted by the first character line extraction unit. Image distortion correction device. The vertically-written character lines extracted by the first character line extracting means are sequentially searched from the outside of the page toward the binding portion, and the coordinates of the head (or end) of the line of interest in the binding portion direction are already determined. If any of the reference rows is within a fixed width range with respect to the coordinates of the leading (or end) binding portion direction of any of the reference rows, the histogram value for the reference row is increased by 1 and The image distortion correction apparatus according to claim 9, wherein, if the target line does not exist within the width range, the line of interest is regarded as a new reference line. The second character line extracting means switches a reference for extracting the vertically written character line in which coordinates in a binding portion direction are close to each other between a portion where the image is not distorted and a portion where the image is distorted. An image distortion correction apparatus according to claim 8. At the start of the extraction process by the second character line extracting means, a portion where no distortion is generated, and the difference between the coordinates of the head (or the end) of the two vertically written character lines successively extracted in the binding portion direction is constant. 13. The image distortion correction device according to claim 12, wherein if the value is equal to or more than the value, the subsequent portion is regarded as a portion where distortion has occurred. 14. The image distortion correction device according to claim 13, wherein a constant value relating to a difference in coordinates in a binding portion direction is determined according to an average line width of the vertically written character lines extracted by the first character line extraction unit. The portion in which no distortion is generated is the vertical writing character which is extracted within a certain range from the head (or end) position of the line of interest in the direction toward the outside of the page of the book document with the coordinates in the binding portion direction based on the line of interest. The head of the line (or the end) is present, and the direction toward the inside of the page of the book document at the coordinates of the binding part direction is determined with respect to the direction toward the binding part when viewed from the head (or the end) position of the line of interest. The method according to any one of claims 12 to 14, wherein the next vertical writing character line is extracted so that the head (or the end) of the line to be extracted exists within a predetermined angle range. Image distortion correction device. With respect to the direction toward the inside of the page of the book manuscript in the direction of the binding portion with respect to the line of interest, the portion where the distortion is generated is the same as the direction toward the binding portion when viewed from the head (or end) position of the line of interest. The head (or end) of the line to be extracted exists within a predetermined angle range, and the inclination of a straight line connecting the head (or end) position of the line of interest and the head (or end) position of the line to be extracted Is larger than a value obtained by subtracting a predetermined value from the inclination of a straight line connecting the head (or end) position of the line of interest and the head (or end) position of the extracted line immediately before the line of interest. 15. The image distortion correction apparatus according to claim 12, wherein a written character line is extracted. When the number of the vertical writing character lines extracted by the second character line extraction unit is equal to or less than a predetermined value, the remaining vertical writing character lines excluding the extracted vertical writing character lines are 9. The image distortion correction apparatus according to claim 8, wherein the character line extracting means is executed again. If the vertical character line extracted by the second character line extraction unit does not exist within a certain range from the binding portion of the book document, the vertical character line extracted from the binding portion of the book document is excluded. 9. The apparatus according to claim 8, wherein the second character line extracting means is executed again. 19. The image distortion correction apparatus according to claim 18, wherein a certain range from a binding portion of the book document is determined according to a width of the entire image. Image reading means for reading a document image,
20. The image distortion correction device according to claim 1, wherein the image correction unit corrects the scanned image read by the image reading unit.
An image reading apparatus comprising: Image reading means for reading a document image,
20. The image distortion correction device according to claim 1, wherein the image correction unit corrects the scanned image read by the image reading unit.
An image printing device that prints an image on paper based on image data output from the image distortion correction device,
An image forming apparatus comprising: The distortion of the scan image obtained by reading the book document contacting above or below the scan surface by the image reading means is converted into a polynomial approximation curve relating to the position of the character line head (or character line end) of the vertical writing character line existing in the scan image. A program that causes a computer to execute image distortion correction processing for correcting based on
Characters to be extracted by limiting the vertical writing character line used for calculating the polynomial approximation curve to those having a certain positional relationship based on the position information of the character line head (or the character line end) of the vertical writing character line A program for executing a line extracting function. The character line extraction function includes:
From among the vertical writing character lines, the vertical writing character line having the maximum (or minimum) coordinate in the binding portion direction at the beginning (or the end) is extracted, and 23. The non-transitory computer-readable storage medium according to claim 22, wherein the program causes the computer to execute a first character line extraction function of extracting the vertically written character line having a head (or end) within a predetermined distance range. 24. The program according to claim 23, wherein the vertical writing character line whose line width is smaller than a predetermined value is excluded from the extraction target. 24. The program according to claim 23, wherein the vertically written character line in which a ratio of a line length to a line width is smaller than a predetermined value is excluded from an extraction target. When the number of the vertical character lines extracted by the first character line extraction function is equal to or less than a predetermined value, the first character line is used for the remaining vertical character lines excluding the extracted vertical character lines. 26. The program according to claim 23, wherein the program causes the computer to execute the extraction function again. If the vertical character line extracted by the first character line extraction function does not exist within a certain range from the binding portion of the book document, the remaining vertical character line excluding the extracted vertical character line, 26. The program according to claim 23, wherein the program causes the computer to execute the first character line extraction function again. 28. The program according to claim 27, wherein a predetermined range from the binding portion of the book document is determined according to the width of the entire image. The character line extraction function includes:
A histogram construction function for constructing a histogram with respect to the coordinates in the binding portion direction of the head (or end) of the vertically written character line extracted by the first character line extraction function;
In the histogram configured by the histogram configuration function, the reference line having the largest value is set as a start line, and the vertical writing character line whose leading (or end) coordinates are close to the binding portion direction is extracted. Second character line extraction function,
The program according to claim 23, wherein the program is executed by the computer. The histogram configuration function defines a reference line, and the vertical writing character in which the head (or end) of the line exists within a fixed width range with respect to the coordinates of the head (or end) of the reference line in the binding portion direction. 30. The program according to claim 29, wherein the number of rows is a value of the histogram for a reference row. 31. The fixed width relative to the coordinates of the head (or the end) of the reference line in the binding portion direction according to the average line width of the vertically written character lines extracted by the first character line extraction function. program. The vertically written character lines extracted by the first character line extraction function are sequentially searched from the outside of the page toward the binding portion direction, and the coordinates of the head (or end) of the line of interest in the binding portion direction have already been determined. If any of the reference rows is within a fixed width range with respect to the coordinates of the leading (or end) binding portion direction of any of the reference rows, the histogram value for the reference row is increased by 1 and 32. The program according to claim 30, wherein the line of interest is regarded as a new reference line when the line of interest does not exist within the range of the width. The second character line extraction function switches a reference for extracting the vertical writing character line in which coordinates in the binding portion direction are close to each other between a portion where the image is not distorted and a portion where the image is distorted. A program according to claim 29. At the start of the extraction process by the second character line extraction function, a portion where no distortion is generated, and the difference between the coordinates of the head (or the end) of the two vertically written character lines successively extracted in the binding portion direction is constant. 34. The program according to claim 33, wherein when the value is equal to or more than the value, a portion after that is regarded as a portion where distortion occurs. 35. The non-transitory computer-readable storage medium according to claim 34, wherein a predetermined value related to a difference in coordinates in a binding portion direction is determined according to an average line width of the vertically written character lines extracted by the first character line extraction function. The portion in which no distortion is generated is the vertical writing character which is extracted within a certain range from the head (or end) position of the line of interest in the direction toward the outside of the page of the book document with the coordinates in the binding portion direction based on the line of interest. The head of the line (or the end) is present, and the direction toward the inside of the page of the book document at the coordinates of the binding part direction is determined with respect to the direction toward the binding part when viewed from the head (or the end) position of the line of interest. 36. The next vertical writing character line is extracted so that the head (or the end) of the line to be extracted exists within a range of a predetermined fixed angle. Program. With respect to the direction toward the inside of the page of the book manuscript in the direction of the binding portion with respect to the line of interest, the portion where the distortion is generated is the same as the direction toward the binding portion when viewed from the head (or end) position of the line of interest. The head (or end) of the line to be extracted exists within a predetermined angle range, and the inclination of a straight line connecting the head (or end) position of the line of interest and the head (or end) position of the line to be extracted Is larger than a value obtained by subtracting a predetermined value from the inclination of a straight line connecting the head (or end) position of the line of interest and the head (or end) position of the extracted line immediately before the line of interest. The program according to any one of claims 33 to 35, wherein a written character line is extracted. If the number of the vertical writing character lines extracted by the second character line extraction function is equal to or less than a predetermined value, the remaining vertical writing character lines excluding the extracted vertical writing character lines are 30. The program according to claim 29, wherein the program causes the computer to execute a character line extraction function again. If the vertical character line extracted by the second character line extraction function does not exist within a certain range from the binding portion of the book document, the vertical character line extracted from the binding portion of the book document is excluded. 30. The program according to claim 29, wherein the program causes the computer to execute the second character line extraction function again. 40. The program according to claim 39, wherein a predetermined range from the binding portion of the book document is determined according to the width of the entire image. A computer-readable storage medium storing the program according to any one of claims 22 to 40.

Images