JP2004328705A - Picture distortion correction device, image reader, image forming device, picture distortion correction method, picture distortion correction program, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To make possible picture distortion correction with high correction accuracy even when the appropriate shape of a page, character line information or ruled line information can not be extracted as a reference line. <P>SOLUTION: A reference line complement means determines a page binding part and distortion areas on left and right pages from a ground density of a scanned image, symmetrically moves the reference line with the page binding part as a center, and extends/contracts the symmetrically moved reference line in a sub-scanning direction to complement the reference line in accordance with width of the distortion areas on the left and right pages. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

算出された比率ｋが、予め定められた閾値よりも大きい場合に、スキャン画像にページ外形が存在すると判断する。
なお、スキャン画像の上下にページ外形が存在する場合には、ヒストグラムの左右両端に高い縦棒が現れることになるので、このような場合には、ヒストグラムの左右両端の高い縦棒に基づいてスキャン画像にページ外形が存在するか否かの判断がそれぞれ実行される。
ここに、ページ外形判別手段の機能が実行される。
以上の処理により、スキャン画像にページ外形が存在すると判断された場合には、左右ページの上下辺のいずれにページ外形が存在しているのかという情報とともにページ外形を抽出し、ＲＡＭ３３に一時的に記憶する。
なお、このスキャン画像にページ外形が存在するか否かの判断処理は、スキャン画像の綴じ部境界線を境にした左右ページ毎に実行される。
［スキャン画像からの罫線の抽出］
続くステップＳ４２においては、スキャン画像からの罫線の抽出処理を実行する。
【００１４】
［罫線候補の検出］
ここで、図１４は長い罫線が存在するスキャン画像の一例を示す説明図、図１５は図１４に示したスキャン画像の綴じ部境界線左側の黒画素ヒストグラムである。図１５に示すヒストグラムのｘ軸はスキャン画像の主走査方向（図１４の上下方向）を示すものであり、スキャン画像の上端はヒストグラムの左端に対応付けられている。図１４に示すようにスキャン画像に罫線が存在する場合には、図１５に示すヒストグラムに幅の狭いピークが現れることになる。本実施の形態では、このような特性を利用して、スキャン画像に罫線が存在するか否かの判断を行う。
より具体的には、まず、図１５に示すヒストグラムに現れた幅の狭いピークの高さＨを求めるとともに、求められた各ピークの中央位置（高さが半分の位置）における幅Ｗを求める。そして、ピークの高さＨが予め定められた閾値ｔｈＨよりも高く、かつ、ピークの中央位置の幅Ｗが予め定められた閾値ｔｈＷより小さなピークが存在する場合、そのピークを罫線の候補とする。
ここに、罫線候補抽出手段の機能が実行される。
続いて、罫線の候補とされたピークについて、罫線の連続性を利用して、更に罫線か否かの判断をする。図１６に示すように、候補罫線上の適当な位置（例えば、ページの中心線の位置）を開始点とし、この開始点から候補罫線を左右方向へ探索し、切断点（罫線がかすれて途切れている部分）の数を累積する。切断点の数が予め定められた閾値より少なければ、この候補を罫線と判断する。このように罫線連続性に基づいて罫線か否かの判断をすることにより、罫線として誤って検出された小さな文字で構成された横書き文字行や点線等を排除することが可能になる。
ここに、罫線判別手段の機能が実行される。
［罫線の座標検出］
以上のようにして罫線を判別した後、各罫線の座標を検出する。罫線座標の検出は、図１７に示すように、罫線の主走査方向（図１７のｙ軸方向）の座標値を罫線部の黒画素ランの中点座標とした場合、図１７に示す罫線の左端のｘ１における主走査方向座標値はｙ１となる。
【００１５】
［最適罫線の選択］
次に、候補罫線の中から歪み補正に最適な罫線を選択する。図１８に示すように複数の罫線が検出される場合、どの罫線を用いて歪み補正するかを選択する必要がある。最適な罫線の選択基準の一例としては、罫線の長さが予め定められた閾値より長く、かつ、綴じ部境界線を挟んだ左右の一定幅領域内（図１８の網掛け領域）に罫線の一部がかかっていることを条件とし、その中で上下いずれかのページ外形に最も近い罫線を選択するようにする。図１８においては、左右ページから各１本ずつの罫線を選択する場合を示している。ここでは、罫線▲１▼と罫線▲２▼とが選択されている。
また、最適な罫線の選択基準の別の例としては、罫線の長さが予め定められた閾値より長く、かつ、綴じ部境界線を挟んだ左右の一定幅領域内（図１９の網掛け領域）に罫線の一部がかかっていることを条件とし、各ページの上部では上端のページ外形に、各ページの下部では下端のページ外形に、それぞれ最も近い罫線を選択するようにする。図１９においては、左右ページをさらに上下部分に分け、その各４ブロックにおいて１本ずつの罫線を選択する場合を示している。ここでは、左上のブロックでは罫線▲１▼、右下のブロックでは罫線▲２▼、左下のブロックでは罫線▲３▼が選択されている。なお、図１９中の右上のブロックには上記２条件（罫線の長さが予め定められた閾値より長く、かつ、綴じ部境界線を挟んだ左右の一定幅領域内に罫線の一部がかかっている）を満足する罫線が存在しないので、選択された罫線はない。
なお、上記２条件（罫線の長さが予め定められた閾値より長く、かつ、綴じ部境界線を挟んだ左右の一定幅領域内に罫線の一部がかかっている）については、その両方ではなくいずれか一方のみを満足するものであっても良い。また、選択基準として上例では「ページ外形に最も近い」を用いているが、これに限るものではなく、「罫線の湾曲が最も大きい」を用いても良い。ここで、「罫線の湾曲」は罫線の左右両端点の主走査方向の座標値の差で表すものとする。
【００１６】
［最適罫線の座標値の決定］
最適な罫線が選択された場合には、罫線の（主走査方向の）座標値を決定する。罫線の（主走査方向の）座標値は、選択された罫線を左右ページのそれぞれ両端に達するまで近似して延長することにより決定される。図２０において、罫線が存在しているＢＣ部については、前述した罫線座標検出処理により既に座標値は決まっていることから、それ以外の延長部分について罫線の（主走査方向の）座標値を決定することになる。より詳細には、図２０に示すＡＢ部は直線近似で（主走査方向の）座標値を推定し、ＣＤ部は多項式近似曲線で（主走査方向の）座標値を推定する。
［不適切な罫線の排除］
最後に不適切な罫線を排除する。これは、前述したように多項式近似により座標値を推定する際に、多項式近似による推定曲線の形状が不適切である場合には補正の際にかえって歪みが増大する恐れがあるので、このような罫線を排除するものである。不適切な近似曲線形状の例としては、図２１に示すように、曲線が書籍の外側へ向かうような曲線▲１▼や、中心線を超えて大きく内側へ食い込むような曲線▲２▼である。
なお、推定曲線の形状が不適切であるとして罫線を排除した場合には、再び最適な罫線を選択し、上記の処理を繰り返すことになる。
以上の処理により、スキャン画像に罫線が存在すると判断された場合には、左右各ページのいずれの位置に罫線が存在しているのかという情報とともに罫線を抽出し、ＲＡＭ３３に一時的に記憶する。
［スキャン画像からの文字行の抽出］
続くステップＳ４３においては、スキャン画像からの文字行の抽出処理を実行する。ステップＳ４３におけるスキャン画像からの文字行の抽出処理について説明する。本実施の形態においては、まず、スキャン画像中の文字行が縦書き文字行なのか、横書き文字行なのかの判別を行う。
【００１７】
［文字行の判別］
スキャン画像中の文字行が縦書き文字行なのか、横書き文字行なのかの判別手法について説明する。ここで、図２２は図８に示した画像の副走査方向の黒白反転数ヒストグラムである。図２２中の横軸は、副走査方向（左右方向）の黒画素（スキャン画像を黒白反転させた画素の中でその濃度値が予め定めた濃度値よりも濃い画素）の主走査方向上での位置を示し、図２２中の縦軸は、その位置毎の黒画素数を示すものである。また、図２３は図８に示した画像の主走査方向の黒白反転数ヒストグラムである。図２３中の横軸は、主走査方向（上下方向）の黒画素（スキャン画像を黒白反転させた画素の中でその濃度値が予め定めた濃度値よりも濃い画素）の副走査方向上での位置を示し、図２３中の縦軸は、その位置毎の黒画素数を示すものである。画像中の文字が横書きの図８に示したようなスキャン画像の場合、図２２に示すような副走査方向のヒストグラムは激しく変化するが、図２３に示すような主走査方向のヒストグラムの変化は少ない。また、特に図示しないが、スキャン画像中の文字行が縦書き文字行である場合には、主走査方向のヒストグラムは激しく変化するが、副走査方向のヒストグラムの変化は少ない。
【００１８】
上述したような判別手法は、具体的には下記に示す各式により実現される。まず、下記に示す式（２）により、
【数２】

主走査方向ｙの位置でのヒストグラム値Ｐｎｔ（ｙ）の平均値ｍｅａｎ_Ｈが算出される。ここで、ｈｅｉｇｈｔは画像の高さである。そして、下記に示す式（３）により、
【数３】

副走査方向のヒストグラムの主走査方向に関する分散σ_Ｈが得られる。
同様に、下記に示す式（４）により、
【数４】

副走査方向ｘの位置でのヒストグラム値Ｐｎｔ（ｘ）の平均値ｍｅａｎ_Ｖが算出される。ここで、ｗｉｄｔｈは画像の幅である。そして、下記に示す式（５）により、
【数５】

主走査方向のヒストグラムの副走査方向に関する分散σ_Ｖが得られる。
上述したようにスキャン画像中の文字行が横書き文字行である場合には、副走査方向のヒストグラムの主走査方向に関する分散σ_Ｈが、主走査方向のヒストグラムの副走査方向に関する分散σ_Ｖより大きい。逆に、スキャン画像中の文字行が縦書き文字行である場合には、主走査方向のヒストグラムの副走査方向に関する分散σ_Ｖが、副走査方向のヒストグラムの主走査方向に関する分散σ_Ｈより大きい。つまり、分散σ_Ｈと分散σ_Ｖとの比較により、スキャン画像中の文字行が縦書き文字行なのか、横書き文字行なのかの判別が可能になっている。
なお、スキャン画像中の文字行が縦書き文字行なのか、横書き文字行なのかの判別に、黒白反転数ヒストグラムを用いたのは、文字行と写真部分との混同を避けるためである。一般に、黒画素ヒストグラムの値が同程度の場合、文字領域のほうが写真領域よりも黒白反転数ヒストグラムの値が大きくなるからである。
ここに、文書判別手段の機能が実行される。
【００１９】
［横書き文字行の座標検出］
以上のようにして文字行を判別した後、まず、各横書き文字行の座標を検出する。横書き文字行の座標の検出にあたっては、文字認識処理に用いられる文字単位の外接矩形抽出処理を行うとともに、横書き文字行の抽出処理を行う。なお、文字認識処理については周知の技術であるので、その説明は省略する。ここで、スキャン画像の文字外接矩形抽出処理および文字行抽出処理の結果の一例を図２４に示す。そして、各文字の外接矩形の中心点の座標をその文字の座標とみなし、横書き文字行の座標を検出する。
［最適横書き文字行の選択］
次に、抽出した横書き文字行の中から歪み補正に最適な横書き文字行を選択する。
複数の横書き文字行が検出される場合、どの横書き文字行を用いて歪み補正するかを選択する必要がある。最適な横書き文字行の選択基準の一例としては、前述した最適な罫線の選択基準と基本的に同様であって、図２５に示すように横書き文字行の長さＢＣが予め定められた閾値より長く、かつ、綴じ部境界線を挟んだ左右の一定幅領域内（図２５の網掛け領域）に横書き文字行の一部Ｃがかかっていることを条件とし、その中で上下いずれかのページ外形に最も近い横書き文字行を選択するようにする。ここで、Ｂは文字行の一番左の矩形の中心であり、Ｃは一番右の矩形の中心である。なお、最適な横書き文字行の選択は、左右ページから各１本ずつのページ外形に最も近い横書き文字行を選択するものであっても良いし、左右ページをさらに上下部分に分け、その各４ブロックにおいて１本ずつのページ外形に最も近い横書き文字行を選択するものであっても良い。
なお、上記２条件（横書き文字行の長さが予め定められた閾値より長く、かつ、綴じ部境界線を挟んだ左右の一定幅領域内に横書き文字行の一部がかかっている）については、その両方ではなくいずれか一方のみを満足するものであっても良い。また、選択基準として上例では「ページ外形に最も近い」を用いているが、これに限るものではなく、「横書き文字行の湾曲が最も大きい」を用いても良い。ここで、「横書き文字行の湾曲」は横書き文字行の両端の文字外接矩形の中心座標の主走査方向の座標値の差で表すものとする。
【００２０】
［最適横書き文字行の座標値の決定］
最適な横書き文字行が選択された場合には、横書き文字行の（主走査方向の）座標値を決定する。横書き文字行の（主走査方向の）座標値は、横書き文字行内の各文字外接矩形の中心点を連結し、直線部分と曲線部分とを近似して抽出することにより横書き文字行の（主走査方向の）座標値を決定することになる。より詳細には、図２５に示すＤは綴じ部境界線であり、ＢＤの間は多項式近似曲線で（主走査方向の）座標値を推定し、一番左端のＡとＢとの間は近似直線の値で（主走査方向の）座標値を推定する。
［不適切な横書き文字行の排除］
最後に不適切な横書き文字行を排除する。これは、前述したように多項式近似により座標値を推定する際に、多項式近似による推定曲線の形状が不適切である場合には補正の際にかえって歪みが増大する恐れがあるので、このような横書き文字行を排除するものである。不適切な近似曲線形状の例としては、前述した罫線の場合と同様であって、特に図示しないが、曲線が書籍の外側へ向かうような場合や、中心線を超えて大きく内側へ食い込むような場合である。
なお、推定曲線の形状が不適切であるとして横書き文字行を排除した場合には、再び最適な横書き文字行を選択し、上記の処理を繰り返すことになる。
以上の処理により、スキャン画像に横書き文字行が存在すると判断された場合には、左右各ページのいずれの位置に横書き文字行が存在しているのかという情報とともに横書き文字行を抽出し、ＲＡＭ３３に一時的に記憶する。
［縦書き文字行の各文字の座標検出］
続いて、各縦書き文字行の座標を検出する。縦書き文字行の座標の検出にあたっては、文字認識処理に用いられる文字単位の外接矩形抽出処理を行う。なお、文字認識処理については周知の技術であるので、その説明は省略する。そして、各文字の外接矩形の中心点の座標をその文字の座標とみなす。
【００２１】
［横書き文字行の抽出］
次に、各縦書き文字行から横書き文字行を抽出する。横書き文字行の抽出は、各縦書き文字行の一番上の文字を連結した外形、または、各縦書き文字行の一番下の文字を連結した外形を横書き文字行とみなすものである。より詳細には、左右ページから各１本ずつの文字行を選択する場合は、各縦書き文字行の一番上もしくは一番下の各一文字を連結した外形を一つの行とみなして抽出する。また、左右ページをさらに上下部分に分け、その４ブロックにおいて１本ずつの文字行を選択する場合は、左上部、右上部においては各縦書き文字行の一番上の各一文字を連結した外形を一つの行とみなし、左下部、右下部においては各縦書き文字行の一番下の各一文字を連結した外形を一つの行とみなして抽出する。
その際、図２６に示すように抽出された外形の長さＢＣが予め定められた閾値より長く、かつ、綴じ部境界線を挟んだ左右の一定幅領域内（図２６の網掛け領域）に抽出された外形の一部Ｃがかかっていることを条件とし、外形を抽出する。なお、上記２条件（抽出された外形の長さが予め定められた閾値より長く、かつ、綴じ部境界線を挟んだ左右の一定幅領域内に抽出された外形の一部がかかっている）については、その両方ではなくいずれか一方のみを満足するものであっても良い。したがって、上記条件を満足しない場合は、歪み補正用の縦書き文字行の外形は無いということになる。
［抽出された外形の座標値の決定］
縦書き文字行に応じて外形が抽出された場合には、なお、縦書き文字行の外形の（主走査方向の）座標値を決定する。縦書き文字行の外形の座標値は、図２７に示すように、各縦書き文字行の一番上の各一文字を連結した外形である場合には、連結する各文字の外接矩形の上辺中心点を連結し、直線部分と曲線部分とを近似して抽出することにより縦書き文字行の外形の（主走査方向の）座標値を決定することになる。また、図２７に示すように、各縦書き文字行の一番下の各一文字を連結した外形である場合には、連結する各文字の外接矩形の下辺中心点を連結し、直線部分と曲線部分とを近似して抽出することにより縦書き文字行の外形の（主走査方向の）座標値を決定することになる。より詳細には、図２６に示すＤは綴じ部境界線であり、ＢＤの間は多項式近似曲線で（主走査方向の）座標値を推定し、一番左端のＡとＢとの間は近似直線の値で（主走査方向の）座標値を推定する。
【００２２】
［不適切な縦書き文字行の外形の排除］
最後に不適切な縦書き文字行の外形を排除する。これは、前述したように多項式近似により座標値を推定する際に、多項式近似による推定曲線の形状が不適切である場合には補正の際にかえって歪みが増大する恐れがあるので、このような縦書き文字行の外形を排除するものである。不適切な近似曲線形状の例としては、前述した罫線や横書き文字行の場合と同様であって、特に図示しないが、曲線が書籍の外側へ向かうような場合や、中心線を超えて大きく内側へ食い込むような場合である。
なお、推定曲線の形状が不適切であるとして縦書き文字行の外形を排除した場合には、歪み補正用の縦書き文字行の外形は無いということになる。
以上の処理により、スキャン画像に縦書き文字行の外形が存在すると判断された場合には、左右各ページのいずれの位置に縦書き文字行の外形が存在しているのかという情報とともに縦書き文字行の外形を抽出し、ＲＡＭ３３に一時的に記憶する。以上、ステップＳ４１〜Ｓ４３の処理により、ページ外形／罫線／文字行の抽出処理（ステップＳ４）が終了する。
しかしながら、このページ外形／罫線／文字行の抽出処理において、適切な対象が見つからない場合もある。図６のステップＳ５、画像歪み補正処理の説明の前に、これらの補充処理の方法を説明する。
【００２３】
図２８は参照線の検出および補充の処理を示すフローチャートである。なお、この補充処理の説明の中にはページ外形／罫線／文字行の抽出処理が参照線の検出という表現で含まれているが同一の処理であり、図２８を一連の処理として実現しても、あるいは参照線の検出という部分を前に説明したページ外形／罫線／文字行の抽出処理として先に別途実行しても良い。また、以降このページ外形／罫線／文字行を、補正のための位置を参照する線という意味で参照線と呼ぶこととする。
まず、入力スキャン画像の地肌濃度を検出する（ステップＳ４５）。基本的に、各主走査方向ラインにおいて、最大画素値（最も明るい画素値）をそのラインの地肌濃度とみなす。図２９は綴じ部を含む左右見開きページのスキャン画像で、綴じ部付近に地肌濃度の濃い部分があり、一般的には原稿の置き方にむらができ、左右対称にはならない。このスキャン画像の各主走査方向ラインの地肌濃度を副走査方向にプロットしたものが図３０に示す画素値の変化グラフである。ここで、地肌濃度が最も濃い位置をスキャン画像の綴じ部位置とみなす。
次に、左右各ページの歪み領域比率を算出する（図２８のステップＳ４６）。綴じ部から、左右各ページで地肌濃度が等しい位置までの距離の比を、歪み領域の比率とみなす。図３０の例では、地肌濃度Ｉを予め与えておくと、左ページにおいて綴じ部から地肌濃度がＩとなる位置までの距離はｄ１、右ページではｄ２なので、距離比はｄ１：ｄ２となる。これは、図２９の網がけで表した部分に対応する。あるいは、図３１に示すように、地肌濃度Ｉを予め与える代わりに、左右ページに定点ｘ１、ｘ２を予め定めておき、ｘ１における地肌濃度とｘ２における地肌濃度のより濃い（画素値が小さい＝暗い）方をＩとみなしても良い。
次に、参照線の検出を行う（図２８のステップＳ４７）。この処理はページ外形／罫線／文字行の抽出処理と全く同じ処理である。図３２は図２９のスキャン画像から参照線を検出した状態を示す図である。この例では、左ページからは１対（２本）の参照線を検出できたが、右ページからは１本の参照線しか検出できていない。便宜的に、主走査方向の下方にある参照線を参照線１、上方にあるものを参照線２としておく。
参照線の補充処理（図２８のステップＳ４８）は、片方のページから１本しか参照線が検出できなかった場合は、もう一方のページの参照線を利用して、欠落した参照線を補充するものである。基本的には、もう一方のページの参照線を綴じ部を中心に対称移動させ、ステップＳ４６にて算出した左右ページの歪み領域比率に応じてこれを伸縮し、当該ページの欠落した参照線とみなす処理である。
【００２４】
図３２の例では右ページ上方からの参照線が検出できていない。そこで、左ページの参照線２を右ページに対称移動して伸縮する。図３３は図３２で検出した参照線を補充する処理を示す説明図である。左右ページの歪み領域比率はｄ１：ｄ２かつｄ１＞ｄ２なので、左ページの参照線２上の任意の点Ｐと綴じ部との距離ｌ１（エル１）と、右ページでの対応点Ｐ’と綴じ部との距離ｌ２（エル２）の関係が、ｌ１：ｌ２＝ｄ１：ｄ２となるように、参照線２を対称移動して短縮すれば良い。右ページの点Ｑ’は左ページの左端の点Ｑの対応点である。点Ｑ’よりさらに右方の部分は、点Ｑ’と主走査方向（ｙ方向）の座標が等しい位置に直線を延長する。
図３４および図３５は図３２および図３３と類似の例であるが、ここでは、左右ページの歪み領域比率がｄ３：ｄ４で、かつｄ３＜ｄ４であり、右ページのほうが広くなっている点が異なっている。この場合は、左ページの参照線２を右ページに対称移動して伸張することになる。左ページの点Ｒと綴じ部の距離ｌ３（エル３）および右ページの対応点Ｒ’と綴じ部の距離ｌ４（エル４）の関係はｌ３：ｌ４＝ｄ３：ｄ４である。右ページの右端の点Ｓ’の対応元の点は左ページの点Ｓであり、これより左方の部分は、右ページの参照線上には対応点を持たないことになる。
図３６は、左ページに参照線１が、そして右ページに参照線２が、それぞれ検出できなかった例である。左右ページの歪み領域比率はｄ５：ｄ６（ｄ５＞ｄ６）である。この例では、左ページの参照線２を右ページに対称移動して短縮し、かつ、右ページの参照線１を左ページに対象移動して伸張することにより、参照線をそれぞれ補充する（図３７）。
【００２５】
図３８は、左ページは上辺の参照線のみが検出できて、右ページ上下辺とも参照線が検出できなかった例である（歪み領域比率はｄ７：ｄ８（ｄ７ ＞ ｄ８））。本方式では左ページの上辺の参照線を右ページに対象移動して短縮する。これにより右ページにも上辺の参照線が補充される（図３９）。下辺の参照線のほうは両ページとも検出できていないので、スキャナレンズの光軸の主走査方向の位置が既知の場合はこの位置を通る副走査方向に平行な直線を、未知の場合はスキャン画像の主走査方向の中心位置を通る副走査方向に平行な直線を、下辺の参照線とみなす（図の中央を横切る一点鎖線）。これにより、右ページの補正も可能となる。
なお、図３８の例とは逆に上辺の参照線が左右両ページとも検出できなかった場合も、スキャナレンズの光軸の主走査方向の位置が既知の場合はこの位置を通る副走査方向に平行な直線を、未知の場合はスキャン画像の主走査方向の中心位置を通る副走査方向に平行な直線を、参照線とみなす（図４０， ４１）。
以上が参照線の補充処理であり、続くステップＳ５（図６参照）においては、画像歪み補正処理を実行する。なお、上で補充した参照線の位置や形状を示す情報はＲＡＭ３３に記憶し、次のステップの画像歪み補正処理に使用されることになる。
画像歪み補正処理は、図４２に示すように、概略的には、歪み補正（伸張）に際しての基準となる線（参照線１）としてスキャン画像の下辺の近傍に位置するページ外形／罫線／文字行のいずれかを選択する処理（ステップＳ５１：参照線１選択処理）、上記参照線１に対応するものであって補正率（伸張率）の算出用の参照線２としてスキャン画像の上辺の近傍に位置するページ外形／罫線／文字行のいずれかを選択する処理（ステップＳ５２：参照線２選択処理）、参照線１と参照線２に基づいてスキャン画像に伸張処理を施して主走査方向の歪みを補正する処理と、参照線１もしくは参照線２から算出されたページ面のスキャン面からの距離（浮きの高さ）もしくは補正画像の文字外接矩形に基づいてスキャン画像に伸張処理を施して副走査方向の歪みを補正する処理（ステップＳ５３：主走査方向歪み補正処理と副走査方向歪み補正処理）とにより構成されている。
【００２６】
ここで、参照線１選択処理（ステップＳ５１）及び参照線２選択処理（ステップＳ５２）においては、これまで説明してきた参照線の抽出および補充によって生成された参照線情報から、参照線１または参照線２として、スキャン画像の上辺（もしくは下辺）の近傍に位置するページ外形／罫線／文字行のいずれかを、左右各ページについて選択することになる。本実施の形態におけるページ外形、罫線、文字行の選択の優先順位は、
ページ外形＞罫線＞文字行
としている。このような選択優先順位にしたのは、文字行はページ外形や罫線に比べて抽出精度が低く、また、画像の外側にあるページ外形を利用する方が精度の高い歪み補正率を得ることができるためである。
次に、主走査方向歪み補正処理と副走査方向歪み補正処理（ステップＳ５３）について説明する。この処理はスキャナパラメータが既知の場合と未知の場合とで処理が異なる。即ち、スキャナの機種が既知の場合は当該スキャナのスキャナレンズの光軸の主走査方向の位置（図４３のＡｋ）と、スキャナレンズの中心とスキャン面間の距離（図４３の焦点面距離）をスキャナパラメータとして、予め記憶しておく。
まず、スキャナパラメータが既知の場合を説明する。この場合、基本的に、主走査方向の１ラインを補正したら、次にそのラインに関して副走査方向の補正を行う、という処理を主走査方向の全ライン、もしくは、参照線１または参照線２が曲線である部分の主走査方向ラインについて行う。
主走査方向の補正は次のように行う。図４４には、補正前の参照線１、参照線２をそれぞれ実線で、補正後をそれぞれ点線で表している。補正後の点線は、補正前の参照線１、参照線２の直線部分（平坦部）をそのまま延長したものである。ここで、副走査方向の位置Ｘにおいて、参照線上の点ＰがＰ’に、参照線上の点ＱがＱ’に、それぞれ補正される。さて、位置Ｘにおける主走査方向上の任意の点ＹがＹ’に補正されるとして、次の関係式が成り立つ、
ＹＰ／ＹＱ＝Ｙ’Ｐ’／Ｙ’Ｑ’
よって、各点の主走査方向の位置を、点ＰならＰ（ｙ）というように表すと、

となるので、これを変形して、

となる。
上式を利用して、補正後にＹ’（ｙ）の位置に来るべき点の補正前の位置Ｙ（ｙ）を求めることができる。つまり、補正前のＹ（ｙ）の画素値を補正後のＹ’（ｙ）に移せばよいわけである。ただし、Ｙ（ｙ）の算出値は一般に小数となるので、その前後の整数位置に対応する画素値の線形補間結果を用いる。即ち、図４５において、Ｙ（ｙ）の前後の整数をそれぞれＮ、Ｎ＋１、これに対応する画素値をＤ（Ｎ）、Ｄ（Ｎ＋１）とすると、Ｙ（ｙ）における画素値Ｄ（Ｙ（ｙ））は図に示したような線形関係を満足するように算出する。
【００２７】
以上は、全ての参照線１と参照線２がページ外形／罫線／文字行のいずれかである場合であるが、適切なページ外形や罫線、文字行が抽出されなかった場合は次のようにする。図４６は、右ページ上方にページ外形／罫線／文字行が存在しない場合の例である。この場合は、レンズ光軸の軌跡（図４６のＬ）を参照線２とみなして、上記と同様の処理を行う。ただしこの場合は図４４におけるＱとＱ’は一致する。なお、ページ下方にページ外形／罫線／文字行が存在しない場合は、Ｌを参照線１とみなして同様に行う（ＰとＰ’が一致）。ページの上方／下方ともにページ外形／罫線／文字行が存在しない場合は、手がかり無しとして、そのページの補正は行わない。
以上の処理をスキャン画像の左右ページ毎に独立して実行する。また、ある見開き左右ページにおける参照線１および参照線２についての情報を保持しておき、他の見開き左右ページの補正にこれを利用してもよい。例えば、連続する見開きページをスキャンする際、見開き左右ページを１セットと考えて、参照線１、参照線２の抽出は５セットに１回のみ行うものとし、あるセットで参照線１、参照線２を抽出したら、その後に続く４セットではこれらの情報をそのまま流用するということを行っても良い。これにより、参照線１、参照線２を抽出するための時間が節約できるので、全体の処理速度が向上する。
【００２８】
副走査方向の補正は次のようにする。即ち、ページ面のスキャン面からの距離の変化量を利用して、書籍原稿の断面曲線が直線となるように展開して画像を伸長する。具体的には、ページ面のスキャン面からの浮きの高さの変化量により画像の伸長を行う。図４７に示すように、ブック原稿ページ面の形状を読み取ってライン毎の微少な三角形とし、ページの読み取り１ラインにおける画像長さＬｎを次の式
Ｌｎ＝平方根｛１＋（Ｔｎ−Ｔｎ−１）＊＊２｝
により算出し、画像長さＬｎの累積をページの伸長長さとする。その結果、近似した三角形の斜辺は湾曲したページの形状とほぼ等しくなるので（折れ線近似）、その累積をページの画像長さとすることにより、正確なページ長さを得る。特に、１ライン毎の最小ピッチによる形状近似により、その長さの補正精度は高い。
隣接する主走査方向のライン上の注目画素の画素間隔を基準“１”として演算することにより、読み取りライン間隔が変化、すなわち、平面状としたときの原稿に対してサンプリング画素間隔が逐次替わっていくのに対し、画像伸長処理を適応する。また、幾何学的に主走査方向の画像投影倍長さとページ面のスキャン面からの距離は比例関係にあり、図４３に示すように、参照線１もしくは参照線２の主走査方向のアドレスＡ３と、参照線１もしくは参照線２の直線部分（平坦部）のアドレスＫａよりページ面のスキャン面からの距離Ｔを以下の式
Ｔ＝焦点面距離＊（Ａ３−Ａｋ）／｛（Ａｋ−Ｋａ）−（Ａ３−Ａｋ）｝
により、求めることができる。このようにして算出される隣接する主走査ラインの深さの差により、１画素毎の微少ピッチの直線で近似してページの補正位置を算出することができる。なお、参照線１／参照線２の求め方や、適切な参照線１／参照線２が存在しない場合の対応は主走査方向の補正の場合と基本的に同様である。
このようにして算出されたページ面のスキャン面からの距離Ｔは、図４７に示すように、１ライン毎の復元すべき副走査方向の画像長さＬｎの算出に用いられることになる。すなわち、前述した次の式
Ｌｎ＝平方根｛１＋（Ｔｎ−Ｔｎ−１）＊＊２｝
により、１ライン毎の補正すべき副走査方向の画像長さＬｎが算出される。したがって、画像長さＬｎの累積が副走査方向のページ長さになる。
通常、ページ面のスキャン面からの距離が高くなるに連れてその高さの変化量が増す。そこで、本実施の形態においては、その位置のページ面のスキャン面からの距離に応じて画像伸長に制限をかける。例えば、隣り合う境界アドレスの差をその位置の（高さ［ｍｍ］／５）［画素］に制限して境界の誤検出による副走査方向補正のエラーを抑制する。
【００２９】
副走査方向の補正では画像長さＬｎは位置によって異なり、図４８に示すように元画像と補正画像の対応する点の位置は一般に異なる。Ｎに存在した点がＮ’に、Ｎ＋１の点がＮ＋１’にそれぞれ補正される場合、求めるべきは整数Ｘ’の位置（これも一般にはＮ＋１’と一致するとは限らない）の画素値である。Ｎ’とＸ’間の距離をａ、Ｘ’とＮ＋１’間の距離をｂとそれぞれおくと、元画像におけるＸ’の対応点Ｘは、
（Ｘ−Ｎ）／（Ｎ＋１−Ｘ）＝ａ／ｂ
の関係にある。
上式によりＸの位置が分かったら、補正前のＸの画素値を補正後のＸ’に移せばよいわけである。ただし、Ｘの算出値は一般に小数となるので、その前後の整数位置に対応する画素値の線形補間結果を用いる。即ち、図４９において、Ｘの前後のＮ、Ｎ＋１に対応する画素値をＤ（Ｎ）、Ｄ（Ｎ＋１）とすると、Ｘにおける画素値Ｄ（Ｘ）は図に示したような線形関係を満足するように算出する。以上の処理を、副走査方向の全ての位置、もしくは、参照線１または参照線２が曲線である部分の副走査方向の位置に対して行う。
以上の処理をスキャン画像の左右ページ毎に独立して実行する。また、ある見開き左右ページにおける書籍原稿のスキャン面からの距離と、参照線１または参照線２についての情報を保持しておき、他の見開き左右ページの補正にこれを利用してもよい。例えば、連続する見開きページをスキャンする際、見開き左右ページを１セットと考えて、書籍原稿のスキャン面からの距離の算出や参照線１、参照線２の抽出は５セットに１回のみ行うものとし、あるセットで距離の算出や参照線１、参照線２の抽出を行ったら、その後に続く４セットではこれらの情報をそのまま流用するということを行っても良い。これにより、書籍原稿のスキャン面からの距離の算出や参照線１、参照線２を抽出するための時間が節約できるので、全体の処理速度が向上する。
ここまでが、スキャナパラメータが既知の場合の、主走査方向歪み補正処理と副走査方向歪み補正処理（ステップＳ５３）の説明である。
【００３０】
次に、スキャナパラメータが未知の場合を説明する。
主走査方向の補正はスキャナパラメータが既知の場合と基本的に同様であるが、適切なページ外形や罫線、文字行が抽出されなかった場合の処理が若干異なる。図４６において、スキャナパラメータが既知の場合はレンズ光軸の軌跡（図４６のＬ）を参照線２とみなしたが、未知の場合（図５０）は副走査方向に平行な中心線（図５０のＣ）を参照線２とみなす。この場合も図４４におけるＱとＱ’は一致する。ページ下方にページ外形／罫線／文字行が存在しない場合は、Ｃを参照線１とみなして同様に行う（ＰとＰ’が一致）。ページの上方／下方ともにページ外形／罫線／文字行が存在しない場合は、手がかり無しとして、そのページの補正は行わない。
次に、副走査方向歪み補正処理について説明する。ここで、図５１は副走査方向歪み補正処理の流れを概略的に示すフローチャートである。図５１に示すように、ステップＳ１０１においては、主走査方向歪み補正されたスキャン画像に基づいて文字認識処理に用いられる方法で文字の外接矩形Ａ（図５２参照）を抽出する。ここで、文字認識処理については周知の技術であるので、その説明は省略する。このように文字の外接矩形Ａを抽出するのは、この文字の外接矩形Ａの形状の変化を基に、副走査方向の歪みを補正するためである。ここでは、図５２に示すように、文字外接矩形Ａの横辺の長さｗ、縦辺の長さｈ、および、文字の中心Ｂを定義する。ここで、文字の中心Ｂは、外接矩形Ａの対角線の交点である。続いて、図５３に示すように、スキャン画像をブック原稿４０のページ綴じ部４１に平行な方向の複数の短冊状の領域Ｃに分割した後（ステップＳ１０２）、各短冊領域Ｃについて、そこに含まれる文字外接矩形Ａに関する特徴量を求める（ステップＳ１０３）。ここで、ある短冊領域Ｃに含まれる文字外接矩形Ａとは、その中心が当該短冊領域Ｃに含まれるような外接矩形Ａのことである。例えば、図５３の短冊領域Ｃ１に含まれる外接矩形Ａは、図中の網掛けを施した矩形である。
【００３１】
さて、上記の文字外接矩形Ａに関する特徴量は、
（文字の横辺の長さ）／（文字の縦辺の長さ）＝ｗ／ｈ
を基に求められる。すなわち、各短冊領域Ｃについて、そこに含まれる全ての文字外接矩形Ａのｗ／ｈの値の平均値をその短冊領域Ｃの特徴量とするのである。
しかしながら、単に、ｗ／ｈの平均値を算出すると不適切な場合がある。文字の中には、句読点や数式中の記号のようにそのサイズが元々小さく、ｗ／ｈの値が不安定なものがある。また、矩形抽出の際に隣接する文字同士がくっついて抽出されてしまい、ｗが極端に大きい文字外接矩形Ａが生じる場合もある。特徴量を求める場合は、このような特殊な文字や極端にｗが大きいものを予め排除しておく必要がある。そこで、続くステップＳ１０４においては、予め閾値を定めておいて、ｈの値がその閾値より小さな文字外接矩形Ａを予め排除するとともに、ｗ／ｈの比率に関する閾値を予め定めておき、ｗ／ｈの値が其の閾値よりも大きい文字外接矩形Ａも予め排除する。例えば、図５４中に示した網掛けを施した文字外接矩形Ａが予め排除されることになる。
続くステップＳ１０５においては、前述したように極端な文字外接矩形Ａを排除した後に、各短冊領域Ｃ内の文字外接矩形Ａのｗ／ｈの平均値を求める。図５５に各短冊領域Ｃ内の外接矩形Ａのｗ／ｈの平均値の一例を示す。なお、図５５中の短冊領域Ｃ２は、ページ綴じ部を含む短冊領域である。
続いて、ページ綴じ部を含む短冊領域Ｃ２に文字外接矩形Ａが存在するか否かを判断する（ステップＳ１０６）。これは、図５４に示すように、一般にはページ綴じ部付近には文字外接矩形Ａが存在しない場合が多いからである。ページ綴じ部を含む短冊領域Ｃ２に文字外接矩形Ａが存在する場合は（ステップＳ１０６のＹ）、その文字外接矩形Ａを利用して特徴量は算出されているのでそのままステップＳ１０８に進む。
一方、ページ綴じ部を含む短冊領域Ｃ２に文字外接矩形Ａが存在しない場合は（ステップＳ１０６のＮ）、ステップＳ１０７に進み、ページ綴じ部を含む短冊領域Ｃ２の特徴量を求める。なお、ページ綴じ部を含む短冊領域Ｃ２の識別は、例えば、スキャン画像（例えば、モノクロ多値画像）の地肌濃度変化を各短冊領域Ｃごとに求め、短冊領域Ｃ内の最も濃度の薄い濃度値を求めることにより実現される。
【００３２】
図５６は地肌濃度変化を求めた一例を示したものであり、地肌濃度が最も濃い短冊領域が、ページ綴じ部を含む短冊領域Ｃ２であるとみなされる。
なお、スキャン画像がカラー多値画像の場合におけるページ綴じ部を含む短冊領域Ｃ２の識別は、例えばＲＧＢ成分の何れか一つの成分（例えばＧ成分）に着目し、そのＧ成分の地肌濃度を使用して識別するようにすれば良い。また、ＲＧＢを色変換して輝度成分と色差成分とに分け、輝度成分を使用してページ綴じ部を含む短冊領域Ｃ２を識別するようにしても良い。
ページ綴じ部を含む短冊領域Ｃ２の特徴量は、次のようにして定められる。ここでは、統計的特徴量の算出対象となり得る文字外接矩形Ａが存在し、かつ、ページ綴じ部を含む短冊領域Ｃ２の最近傍である短冊領域Ｃの特徴量に対して予め定めた定数値を乗じることにより算出された値が、ページ綴じ部を含む短冊領域Ｃ２における特徴量とみなされるものである。つまり、図５５に示した例では、ページ綴じ部を含む短冊領域Ｃ２の左右何れの短冊領域Ｃ３、Ｃ４にも文字外接矩形Ａが存在するので、どちらか適当な方の特徴量を選択し（ここでは右側の○印の方）、それに予め定めた定数値（ここでは０．５）を乗じて、これをページ綴じ部を含む短冊領域Ｃ２の特徴量としている。
続くステップＳ１０８においては、各短冊領域Ｃの特徴量に対する適切なフィルタリング処理、例えば、短冊領域Ｃの位置の変化方向（即ち副走査方向）に関する移動平均を求める処理を施すなどして、短冊領域Ｃの位置の変化に対する（副走査方向の）特徴量の変化がなだらかになるようにする。ただし、ここでもページ綴じ部付近は特別な処理が必要となる。なぜなら、副走査方向に関して長さが全て等しいウィンドウを用いてフィルタリングを行うと、ページ綴じ部付近の特徴量の変化の鋭さが失われてしまうからである。
【００３３】
ここで、長さが全て５であるウィンドウを用いて図５５に示した各短冊領域Ｃの特徴量に対してフィルタリング処理を施した結果を図５７に示す。図５７に示すように、長さが全て５であるウィンドウを用いてフィルタリング処理を施した場合には、ページ綴じ部付近の特徴量（ｗ／ｈ）の変化がなだらかになり過ぎてしまう。このような場合には、ページ綴じ部付近での適切な画像補正が不可能になってしまう。
そこで、本実施の形態においては、フィルタリング処理の際には、フィルタのウィンドウがページ綴じ部を含む短冊領域Ｃ２の両側の短冊領域Ｃ３、Ｃ４に跨がらないように、ページ綴じ部付近でウィンドウ長を調整する。ここで、図５８はページ綴じ部付近でウィンドウ長を調整してフィルタリング処理を施した結果を示すグラフである。図５８に示すように、ページ綴じ部付近でウィンドウ長を調整した場合には、ページ綴じ部付近の特徴量（ｗ／ｈ）の変化を適切に表現できるので、良好な画像補正が実現できる。
続くステップＳ１０９においては、各短冊領域Ｃの推定歪み量を算出する。各短冊領域Ｃの推定歪み量の算出手法は、以下に示す通りである。
まず、短冊領域の歪み量を算出するための基準となる短冊領域（基準短冊領域）を定める。ここでは、歪みが最も小さいと考えられる短冊領域Ｃ、例えば、特徴量（ｗ／ｈ）が最大である短冊領域Ｃを基準短冊領域とする。この処理は左右ページで共通に行っても良いが、左右独立に基準短冊領域を定めても構わない。図５８においては、左右独立に基準短冊領域を定めた例を示しており、○印を施した短冊領域Ｃが基準短冊領域であり、左側の基準特徴量を“Ｌｗ０／Ｌｈ０”、右側の基準特徴量を“Ｒｗ０／Ｒｈ０”、でそれぞれ示している。
次に、基準短冊領域の特徴量ｗ０／ｈ０をスキャン画像全体の基準特徴量とし、
（各短冊領域の特徴量）／（基準特徴量）＝（ｗ／ｈ）／（ｗ０／ｈ０）
の値を、各短冊領域の推定歪み量として算出する。
なお、ページ綴じ部から外れたページ外側付近の短冊領域Ｃを基準短冊領域とすると、ページ綴じ部付近とはフォントや活字のサイズの違いが大きくて、適切な推定歪み量が算出できない場合も考えられる。そのような画像を対象とする場合は、基準短冊領域の探索範囲を予めページ綴じ部付近に限定しておくのが有効である。これを実現するためには、参照線１または参照線２が曲線である部分の副走査方向の位置に対してのみ行うようにすれば良い。
【００３４】
最後に、スキャン画像に対して、短冊領域Ｃの短辺方向（副走査方向）の拡大処理を行い、ページ綴じ部付近の歪みを補正する（ステップＳ１１０）。その場合の拡大率（副走査方向の歪み補正率（伸張率））は、ステップＳ１０９において算出した推定歪み量の逆数、すなわち、
（基準特徴量）／（各短冊領域の特徴量）＝（ｗ０／ｈ０）／（ｗ／ｈ）
とする。ここで、上記の基準短冊領域を左右共通に定めた場合には、この拡大率も左右共通の基準特徴量によって算出し、独立に定めた場合には、左右それぞれの基準特徴量で独立に算出するようにする。図５９は、図５８に示した特徴量に基づいて算出した補正拡大率を示したものである。
ここに、副走査方向歪み補正率算出手段の機能および副走査方向歪み補正手段の機能が実行される。
なお、ここでも、ページ綴じ部付近から離れた短冊領域Ｃはもともと画像の歪みが無い領域である可能性が高いので、拡大処理の対象としない方が良い場合がある。拡大処理を行ったために、かえって不自然な歪みが生じてしまう可能性があるからである。これを防ぐために、参照線１または参照線２が曲線である部分の副走査方向の位置に対してのみ拡大すれば良い。即ち、参照線１または参照線２が直線の部部では、推定歪み量を“１”とする。
また、短冊領域Ｃ内において共通の補正拡大率を適用した場合、隣接する短冊領域Ｃの境界部での補正拡大率が不連続となるため、補正画像が不自然となる。そこで、隣接する短冊領域Ｃの境界部での補正拡大率が連続的に変化するように、補正拡大率を予め補正しておく。これは、例えば図５９に示す短冊領域Ｃの中央部分の補正拡大率を推定歪み量の逆数を示す点としてプロットし、これらの点を線分で結んで直線補完することで、他の部分の補正拡大率とすることで実現できる。以上の処理により、スキャン画像の副走査方向の補正拡大率が確定する。
以上の処理をスキャン画像の左右ページ毎に独立して実行する。また、ある見開き左右ページにおける補正拡大率についての情報を保持しておき、他の見開き左右ページの補正にこれを利用してもよい。例えば、連続する見開きページをスキャンする際、見開き左右ページを１セットと考えて、補正拡大率の算出は５セットに１回のみ行うものとし、あるセットで補正拡大率を算出したら、その後に続く４セットではその情報をそのまま流用するということを行っても良い。これにより、補正拡大率を算出するための時間が節約できるので、全体の処理速度が向上する。
以上、ステップＳ１０１〜Ｓ１１０の処理により副走査方向歪み補正処理（ステップＳ５５）が終了し、図６に示すスキャン画像の歪み補正処理が終了する。ここで、図６０は歪みを補正した画像を示す平面図である。以上の処理によれば、図８に示したようなページ綴じ部の近傍において生じていたスキャン画像の歪みが、図６０に示すように補正されることになる。
【００３５】
【発明の効果】
以上説明したように本発明によれば、スキャナパラメータが既知の場合と未知の場合において、それぞれ適切な補正方法を選択して補正でき、また、片方のページにて参照線が検出できなかった場合でも、もう一方のページで検出できた場合は、これを綴じ部に関して対象移動し、左右ページの歪み領域比率に応じて伸縮して補充するので、補正精度が従来方式より向上する。特に、片ページにて参照線が１本も検出できなかった場合であっても、もう一方のページで参照線が検出できれば両ページとも補正が可能となる。
【図面の簡単な説明】
【図１】本発明のスキャナ部の構成を示す縦断正面図である。
【図２】スキャナ部を搭載したデジタル複写機の上部部分を示す斜視図である。
【図３】スキャナ部の制御系の電気的な接続を示すブロック図である。
【図４】本発明の画像処理部の基本的な内部構成を示すブロック図である。
【図５】メイン制御部の電気的な接続を示すブロック図である。
【図６】スキャン画像の歪み補正処理の流れを概略的に示すフローチャートである。
【図７】ブック原稿を読み取る際の状態を示す斜視図である。
【図８】入力した画像の一例を示す平面図である。
【図９】スキャン画像で歪みが生じる部分を説明する図である。
【図１０】図８に示した画像の綴じ部境界線左側の黒画素ヒストグラムを示す図である。
【図１１】ページ外形／罫線／文字行の抽出処理の流れを概略的に示すフローチャートである。
【図１２】上端にページ外形が存在するスキャン画像の一例を示す説明図である。
【図１３】図１２に示したスキャン画像の綴じ部境界線左側の黒画素ヒストグラムを示す図である。
【図１４】長い罫線が存在するスキャン画像の一例を示す説明図である。
【図１５】図１４に示したスキャン画像の綴じ部境界線左側の黒画素ヒストグラムを示す図である。
【図１６】罫線連続性に基づく罫線か否かの判断を説明する図である。
【図１７】罫線の座標検出を説明する図である。
【図１８】左右ページから各１本ずつの罫線を選択する場合を示す説明図である。
【図１９】左右ページをさらに上下部分に分け、その各４ブロックにおいて１本ずつの罫線を選択する場合を示す説明図である。
【図２０】延長した最適罫線の座標値の決定を示す説明図である。
【図２１】不適切な近似罫線の例を示す説明図である。
【図２２】図８に示した画像の副走査方向の黒白反転数ヒストグラムを示す図である。
【図２３】図８に示した画像の主走査方向の黒白反転数ヒストグラムを示す図である。
【図２４】スキャン画像の文字外接矩形抽出処理および文字行抽出処理の結果の一例を示す図である。
【図２５】最適横書き文字行の選択を示す説明図である。
【図２６】縦書き文字行からの横書き文字行の抽出を示す説明図である。
【図２７】縦書き文字行の外形の座標値の決定を示す説明図である。
【図２８】参照線の検出および補充の処理を示すフローチャートである。
【図２９】綴じ部を含む左右見開きページのスキャン画像の例を示す図である。
【図３０】歪み領域比率の求め方を示す説明図である。
【図３１】歪み領域比率の求め方を示す説明図である。
【図３２】図２９のスキャン画像から参照線を検出した状態を示す図である。
【図３３】図３２で検出した参照線を補充する処理を示す説明図である。
【図３４】他の例を示すスキャン画像から参照線を検出した状態の図である。
【図３５】図３４で検出した参照線を補充する処理を示す説明図である。
【図３６】他の例を示すスキャン画像から参照線を検出した状態の図である。
【図３７】図３６で検出した参照線を補充する処理を示す説明図である。
【図３８】左ページは上辺の参照線のみが検出できて、右ページ上下辺とも参照線が検出できなかった状態の図である。
【図３９】図３８で検出した参照線を補充する処理を示す説明図である。
【図４０】上辺の参照線が左右両ページとも検出できなかった状態の図である。
【図４１】図４０で検出した参照線を補充する処理を示す説明図である。
【図４２】画像歪み補正処理の流れを概略的に示すフローチャートである。
【図４３】スキャナパラメータを説明する図である。
【図４４】主走査方向の補正前後の参照線を示す説明図である。
【図４５】整数位置に対応する画素値の線形補間を示す説明図である。
【図４６】右ページ上方にページ外形／罫線／文字行が存在しない場合を示す説明図である。
【図４７】湾曲したページの形状の近似を示す説明図である。
【図４８】元画像と補正画像の対応する点の位置を示す図である。
【図４９】整数位置に対応する画素値の線形補間を示す説明図である。
【図５０】スキャナパラメータが既知の場合での主走査方向の補正前後の参照線を示す説明図である。
【図５１】副走査方向歪み補正処理の流れを概略的に示すフローチャートである。
【図５２】スキャン画像に基づく文字の外接矩形を示す説明図である。
【図５３】スキャン画像をブック原稿のページ綴じ部に平行な方向の複数の短冊状の領域に分割した状態を示す図である。
【図５４】排除する文字外接矩形を示す説明図である。
【図５５】各短冊領域内の外接矩形の特徴量の平均値の一例を示すグラフ図である。
【図５６】各短冊領域ごとの地肌濃度変化の一例を示すグラフ図である。
【図５７】図５５に示した各短冊領域の特徴量に対してフィルタリング処理を施した結果を示すグラフ図である。
【図５８】ページ綴じ部付近でウィンドウ長を調整してフィルタリング処理を施した結果を示すグラフ図である。
【図５９】図５８に示した特徴量に基づいて算出した補正拡大率を示すグラフ図である。
【図６０】歪みを補正した画像を示す平面図である。
【図６１】ブック原稿のページ綴じ部がコンタクトガラスから浮き上がった状態を示す正面図である。
【符号の説明】
１ スキャナ部、２ コンタクトガラス、１６ 画像形成装置、２０ 画像処理部、２９ 画像歪み補正部、３１ ＣＰＵ、３３ ＲＡＭ、４０ ブック原稿、４１ ページ綴じ部[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image distortion correction device, an image reading device, an image forming device, and a program.
[0002]
[Prior art]
Most originals read using a flatbed scanner are sheet-like originals.An openable and closable pressure plate is provided on the contact glass, and after placing the original on the contact glass, the pressure plate is closed and the original is scanned. I have. However, the manuscript is not limited to a sheet, and book manuscripts (books, booklets, etc.) may be treated as manuscripts. In such a case, the book manuscript is placed on the contact glass and the manuscript is scanned. Will do.
However, when a book original is used as the original, the page binding section 101 of the book original 100 rises from the contact glass 102 as shown in FIG. When the page binding section 101 of the book document 100 rises from the contact glass 102 in this manner, the page binding section 101 moves away from the focal plane, and thus the scanned image of the raised section includes image distortion and shadow. Image degradation such as blurred characters occurs. The image of the deteriorated page binding unit 101 is difficult to read, and the recognition rate when performing character recognition processing by OCR is significantly reduced. In particular, the ratio is high in thick bookbinding, and if a pressing operation is performed so that the page binding portion 101 of the book document 100 does not separate from the focal plane, the book document 100 itself may be damaged.
[0003]
In order to solve such a problem, there has been proposed a method of correcting image distortion using a method of estimating a three-dimensional shape of an object from image density information. As a method of estimating the three-dimensional shape of an object from such density information of an image, a method called Shape from Shadowing described in Non-Patent Document 1 is a typical example.
Patent Document 1 proposes a method of measuring the shape of a book by a triangulation method and correcting distortion.
Further, Patent Document 2 proposes a method of estimating a three-dimensional shape of a book surface using a shape of a page outline of a read scan image.
However, according to the above-mentioned method called Shape from Shadowing, the amount of calculation is large and the calculation time of the distortion correction processing is long, so that practical use is difficult.
Further, according to the method described in Patent Document 1, a special shape measuring device for measuring the shape of a book by a triangulation method is required, which is not appropriate for reducing the size of the device.
Further, according to the method described in Patent Document 2, distortion correction can be performed with a small amount of calculation. However, when the page outline is not completely contained in the image and is cut off halfway, it is effective correction. Can not.
Accordingly, the present applicant has disclosed an image distortion correction apparatus capable of effectively correcting the distortion of a read scan image in which the outer shape of a page is cut off in the middle with a small amount of calculation (Japanese Patent Application No. 2002-247643). Reference 3). Here, the image distortion is corrected using not only the page outline but also character line information or ruled line information, and further, the intrinsic parameters of the scanner (image reading means) (focal length of the lens, position of scan optical axis (address)) ), The output image of any scanner can be corrected.
[Patent Document 1] JP-A-5-161002
[Patent Document 2] JP-A-11-41455
[Patent Document 3] Japanese Patent Application No. 2002-247643
[Non-Patent Document 1] Wada, H .; Uchida and T.S. Matsuyama, "Shape from Shading with Interreflections under a Proximal Light Source": Distortion-Free Copies of a Union of Foreign Resources.
[0004]
[Problems to be solved by the invention]
However, there are cases where the intrinsic parameters of the scanner are known, and in that case, the use of the parameters generally improves the correction accuracy. Further, when appropriate page outline, character line information, or ruled line information cannot be extracted, correction may not be performed.
The present invention has been made in view of the above-described circumstances, and provides an appropriate correction method for each of a case where a scanner parameter is known and an unknown case, and further provides an appropriate page outline, character line information, or ruled line. It is an object of the present invention to provide an image distortion correction device, an image reading device, an image forming device, and a program that can improve correction accuracy even when information cannot be extracted.
[0005]
[Means for Solving the Problems]
In order to solve the above-mentioned problem, a first aspect of the present invention is a method for reading a page document image placed on a scanning surface such that a page binding portion is substantially parallel to a main scanning direction. In an image distortion correcting apparatus for performing distortion correction by expanding a scan image including a binding portion and images for two left and right pages in a main scanning direction and a sub-scanning direction, a curve extending in the sub-scanning direction included in the document from the scan image is provided. Reference line extracting means for extracting a portion to be formed as a reference line, reference line supplementing means for generating a reference line that could not be extracted on one page based on a reference line on another page, and the extracted or generated reference line And a main scanning direction distortion correcting means for correcting distortion in the main scanning direction of the scan image based on the shape of the scan image.
Further, according to a second aspect of the present invention, in the image distortion correction apparatus according to the first aspect, the reference line extracting means includes a page outline, a ruled line located near an upper side and a lower side in the main scanning direction in the scan image. , A line connecting the positions of each character in a character line perpendicular to the main scanning direction (usually horizontal lines) or the position of the first or last character in a character line parallel to the main scanning direction (usually vertical lines) Are extracted as a pair of reference lines, and the main scanning direction distortion correcting means assumes a straight line such that a curved portion of the reference line coincides with an extension of the straight line portion. The distortion in the main scanning direction is corrected by obtaining corresponding points of the image before and after the correction based on the straight line.
According to a third aspect of the present invention, in the image distortion correction device according to the first or second aspect, the reference line supplementing unit determines a page binding portion and a distortion region of the left and right pages from a background density of the scanned image, The reference line is symmetrically moved about the page binding portion, and the symmetrically moved reference line is expanded and contracted in the sub-scanning direction in accordance with the width of the distortion region of the left and right pages.
According to a fourth aspect of the present invention, in the image distortion correcting apparatus according to the third aspect, the reference line supplementing means sets a brightest pixel value on each line in the main scanning direction as a background density of the line, The position of the darkest line in the sub-scanning direction is regarded as a page binding portion, and a region having a background density of a predetermined brightness or less near the page binding portion is determined as a distortion region.
[0006]
According to a fifth aspect of the present invention, in the image distortion correction apparatus according to the third aspect, the reference line supplementing means sets a brightest pixel value on each line in the main scanning direction as a background density of the line, and The position of the darkest line in the sub-scanning direction is defined as the position of the page binding portion, and the position of the density value of the background density at two predetermined positions in the sub-scanning direction and the other page equal to this density value The position of the background density in the sub-scanning direction is detected and determined as a distortion region.
According to a sixth aspect of the present invention, in the image distortion correcting apparatus according to any one of the first to fifth aspects, the main scanning direction distortion correcting means is configured such that a reference line on an upper side or a lower side in the main scanning direction is left and right. If there is no page and the position of the optical axis of the scanner lens in the main scanning direction is known, a straight line passing through the position and parallel to the sub-scanning direction is regarded as a reference line on the upper side or lower side in the main scanning direction, and the main scanning is performed. It is characterized in that directional distortion is corrected.
According to a seventh aspect of the present invention, in the image distortion correcting apparatus according to any one of the first to fifth aspects, the main scanning direction distortion correcting means is configured such that a reference line on an upper side or a lower side in the main scanning direction is left and right. If neither the page exists nor the position of the optical axis of the scanner lens in the main scanning direction is unknown, a straight line parallel to the sub-scanning direction passing through the center position of the scanned image in the main scanning direction is set on the upper side or the lower side in the main scanning direction. The distortion in the main scanning direction is corrected by regarding the reference line.
According to an eighth aspect of the present invention, in the image distortion correction device according to any one of the first to seventh aspects, the main scanning direction distortion correction unit is configured such that the reference line extraction unit extracts or extracts the reference line. The information on the reference line generated by the means is retained, and distortion correction in the main scanning direction is performed using the information on the reference line for correcting another scan image to be subsequently corrected.
According to a ninth aspect of the present invention, in the image distortion correcting apparatus according to any one of the first to eighth aspects, the main scanning direction distortion correcting means includes a main scanning direction distortion correcting unit that adjusts a vertical scanning direction corresponding to a curved portion of a reference line. The distortion correction in the main scanning direction is performed only at the position.
[0007]
According to another aspect of the present invention, there is provided an image reading device for reading a book document image placed on a scanning surface such that a page binding portion is substantially parallel to a main scanning direction; The image distortion correction device according to any one of claims 1 to 9, wherein the image distortion correction device corrects a scanned image including a page binding portion read by the means and two left and right pages.
Further, according to the image forming apparatus of the present invention, there is provided an image reading means for reading a book document image placed on a scanning surface such that a page binding portion is substantially parallel to a main scanning direction; The image distortion correction device according to any one of claims 1 to 9, which corrects a scanned image including a page binding portion read by the means and images of two left and right pages, and an image corrected by the image distortion correction device. An image printing apparatus for printing an image based on data on paper.
A twelfth aspect of the present invention relates to a page binding part obtained by reading a book document image placed on a scanning surface such that the page binding part is substantially parallel to the main scanning direction. In the image distortion correcting method for performing distortion correction by extending a scanned image including the image in the main scanning direction, a reference line for extracting a portion forming a curve extending in the sub-scanning direction included in the document from the scanned image as a reference line An extracting step, a reference line supplementing step of generating a reference line that could not be extracted on one page based on a reference line of another page, and a main scan of the scan image based on the shape of the extracted or generated reference line. And a main scanning direction distortion correcting step for correcting the direction distortion.
Further, according to a thirteenth aspect of the present invention, the reference line extracting step includes the steps of: defining each of a page outer shape, a ruled line, and a character line perpendicular to the main scanning direction in the scan image in the vicinity of an upper side and a lower side in the main scanning direction. A pair of lines connecting character positions or a line connecting character positions at the beginning or end of a character line parallel to the main scanning direction is extracted as a pair of reference lines, and the main scanning direction distortion correcting step includes: A straight line whose curved portion coincides with an extension of the straight line portion is assumed, and corresponding points of the image before and after correction are obtained based on the assumed straight line to correct distortion in the main scanning direction.
[0008]
According to a fourteenth aspect of the present invention, in the reference line supplementing step, a page binding portion and a distortion region of right and left pages are determined from a background density of the scanned image, and a reference line is symmetrically moved about the page binding portion. The symmetrically moved reference line is expanded and contracted in the sub-scanning direction in accordance with the width of the distorted area of the left and right pages.
According to a fifteenth aspect of the present invention, in the reference line supplementing step, the brightest pixel value on each line in the main scanning direction is set as the background density of the line, and the position of the line with the highest background density in the sub-scanning direction is set. Is regarded as a page binding portion, and a region in which the background density is equal to or lower than a predetermined brightness in the vicinity of the page binding portion is determined as a distortion region.
According to a sixteenth aspect of the present invention, in the reference line supplementing step, the brightest pixel value on each line in the main scanning direction is set as the background density of the line, and the position of the line with the highest background density in the sub-scanning direction is set. Is the position of the page binding portion, the position of the density value of the background density at the two predetermined positions in the sub-scanning direction, and the position of the background density in the other page equal to this density value in the sub-scanning direction Are respectively detected and determined as a distortion region.
According to a seventeenth aspect of the present invention, in the main scanning direction distortion correcting step, the reference line on the upper side or the lower side in the main scanning direction does not exist on the left and right pages, and the position of the optical axis of the scanner lens in the main scanning direction is known. In the case of, the straight line parallel to the sub-scanning direction passing through this position is regarded as a reference line on the lower side in the main scanning direction, and distortion in the main scanning direction is corrected.
Further, in the image forming apparatus according to the present invention, in the main scanning direction distortion correcting step, the reference line on the upper side or the lower side in the main scanning direction does not exist on the left and right pages, and the position of the optical axis of the scanner lens in the main scanning direction is unknown. In the case of, distortion in the main scanning direction is corrected by regarding a straight line parallel to the sub-scanning direction passing through the center position of the scanned image in the main scanning direction as a reference line on the upper side or the lower side in the main scanning direction.
According to a nineteenth aspect of the present invention, in the main scanning direction distortion correcting step, information on a reference line extracted by the reference line extracting step or generated by the reference line replenishing step is held and subsequently corrected. The distortion correction in the main scanning direction is performed using the information on the reference line in the correction of the scan image.
According to a twentieth aspect of the present invention, in the main scanning direction distortion correction step, the main scanning direction distortion is corrected only at a position in the sub scanning direction corresponding to a curved portion of the reference line.
A twenty-first aspect of the present invention is characterized in that the image distortion correction method according to any one of the twelfth to twentieth aspects is programmed to be controllable by a computer.
A twenty-second aspect of the present invention is characterized in that the image distortion correction program according to the twenty-first aspect is recorded in a computer-readable format.
[0009]
BEST MODE FOR CARRYING OUT THE INVENTION
One embodiment of the present invention will be described with reference to FIGS. The image distortion correcting apparatus according to the present embodiment is provided in a digital copying machine as an image forming apparatus, and a scanner unit of the digital copying machine is applied as an image reading apparatus.
Here, FIG. 1 is a longitudinal sectional front view showing the configuration of the scanner unit 1. As shown in FIG. 1, the scanner unit 1 includes a contact glass 2 on which a document is placed, a first traveling body 5 including an exposure lamp 3 for exposing the document and a first reflection mirror 4, and a second reflection mirror 6. And a second traveling body 8 including a third reflection mirror 7, a CCD (Charge Coupled Device) 9 as an image sensor for reading an image of the document, a lens unit 10 for forming an image on the CCD 9, and a document placed thereon. The document scale 11 includes a document scale 11 that serves as a reference for preventing displacement and detachment of the contact glass 2, a white reference plate 12 for shading correction installed below the document scale 11, and a frame 14. The CCD 9 is formed on a sensor board 13.
During scanning of the document, the first traveling body 5 and the second traveling body 8 are moved in the sub-scanning direction by the stepping motor 24 (see FIG. 3). That is, the first traveling body 5 and the second traveling body 8 travel under the contact glass 2 to expose and scan the original with the exposure lamp 3, and reflect the reflected light to the first reflecting mirror 4, the second reflecting mirror 6, and The light is reflected by the third reflection mirror 7 and forms an image on the CCD 9 through the lens unit 10. Here, an image reading unit is realized.
Such a scanner unit 1 is, for example, a printer unit (not shown) which is an image printing apparatus that forms an image on paper in an electrophotographic manner in accordance with image data based on an image of a document read by the scanner unit 1. ) Is mounted on the digital copying machine 16.
FIG. 2 is a perspective view showing an upper portion of the digital copying machine 16 on which the scanner unit 1 is mounted. As shown in FIG. 2, the scanner unit 1 is provided with a pressure plate 17 that can be opened and closed with respect to the contact glass 2 and an open / close sensor 18 that detects whether the pressure plate 17 is opened or closed.
As a printer provided in the digital copying machine 16, various printing methods such as an ink jet method, a sublimation type thermal transfer method, a silver halide photographic method, a direct thermal recording method, and a fusion type thermal transfer method are applied in addition to the electrophotographic method. be able to. Since the specific configuration is well known, detailed description is omitted.
[0010]
FIG. 3 is a block diagram showing an electrical connection of a control system of the scanner unit 1. As shown in FIG. 3, the control system includes a main control unit 19 that controls the entire scanner unit 1, an image processing unit 20 that is a circuit that performs various types of image processing on image data read by the CCD 9, A traveling body control unit 21 which is a circuit for controlling the traveling body 5 and the second traveling body 8, an operation panel 22 for receiving various operations on the digital copying machine 16 and displaying various messages, and an image read by the CCD 9 A memory 23 for storing data, predetermined data, and the like is connected. The operation panel 22 is provided with a copy start key and the like for declaring the start of copying. In the traveling body control unit 21, the exposure lamp 3, the stepping motor 24 for driving the first traveling body 5 and the second traveling body 8, and the first traveling body 5 and the second traveling body 8 are at home positions. A scanner home position sensor (HP sensor) 25 for detecting whether or not this is the case and the open / close sensor 18 are connected.
Here, FIG. 4 is a block diagram showing a basic internal configuration of the image processing unit 20. As shown in FIG. 4, the image processing unit 20 includes an analog video processing unit 26 that performs an amplification process and a digital conversion process of an analog image signal obtained by reading a document by the CCD 9, a shading correction processing unit 27 that performs a shading correction process, and a shading. An image data processing unit 28 that performs various image data processing such as MTF correction, scaling processing, and γ correction on the digital image signal after the correction processing to generate a scan image, and a scan image which is a characteristic function of the present embodiment. The image distortion correcting section 29 that realizes the distortion correcting function of FIG. The digital image signal after the image processing as described above is transmitted to the printer unit via the main control unit 19 and used for image formation.
As shown in FIG. 5, the main control unit 19 includes a CPU (Central Processing Unit) 31 that centrally controls each unit. The CPU 31 has a read-only memory (ROM) that stores a BIOS and the like. An only memory (32) and a random access memory (RAM) 33, which stores various data in a rewritable manner and functions as a work area of the CPU 31, are connected by a bus 34 to constitute a microcomputer. Further, an HDD 35 storing a control program, a CD-ROM drive 36 for reading a CD (Compact Disc) -ROM 37, and an interface (I / F) 38 for communicating with a printer unit and the like are connected to the bus 34. ing.
[0011]
The CD-ROM 37 shown in FIG. 5 embodies the storage medium of the present invention, and stores a predetermined control program. The CPU 31 reads the control program stored in the CD-ROM 37 with the CD-ROM drive 36 and installs the control program in the HDD 35. As a result, the main control unit 19 is in a state where it can perform various processes as described below.
As the storage medium, not only the CD-ROM 37 but also various types of media such as various types of optical disks such as DVDs, various types of magneto-optical disks, various types of magnetic disks such as flexible disks, and semiconductor memories can be used. Alternatively, the program may be downloaded from a network such as the Internet and installed in the HDD 35. In this case, a storage device storing a program in the server on the transmission side is also a storage medium of the present invention. Note that the program may operate on a predetermined OS (Operating System), and in that case, the OS may replace a part of execution of various processes described below with the OS. It may be included as a part of a group of program files constituting predetermined application software such as software or OS.
Next, the contents of various processes executed by the CPU 31 provided in the main control unit 19 based on the control program will be described. Here, of the processing executed by the CPU 31, the scan image in the image distortion correction unit 29, which is an image distortion correction device that realizes a scan image distortion correction function that is a characteristic function of the scanner unit 1 according to the present embodiment. Only the distortion correction processing will be described.
[0012]
FIG. 6 is a flowchart schematically showing the flow of the scan image distortion correction process. Here, as shown in FIG. 7, the book document 40 is placed on the contact glass 2 so that the page binding section 41 and the main scanning direction of the image reading by the scanner section 1 are parallel to each other. Will be described.
First, in step S1, a scan image of the book document 40 placed on the contact glass 2 output from the image data processing unit 28 is input. Here, FIG. 8 shows an example of the input image. Then, as shown in FIG. 9, the input scan image of the book document 40 has distortion near the page binding unit 41.
Next, optimal binarization processing of a scanned image (for example, a monochrome multi-valued image) of the book document 40 is executed (step S2), and black pixels in the sub-scanning direction (the density values of the pixels of the scanned image are predetermined). The histogram of the number of pixels (darker than the density value) is obtained (step S3). FIG. 10 is a black pixel histogram on the left side of the binding portion boundary line of the image shown in FIG. The horizontal axis in FIG. 10 indicates the position of the black pixel in the main scanning direction (the pixel whose density value is higher than a predetermined density value in the pixels of the scanned image), and the vertical axis in FIG. It shows the number of black pixels for each position. Note that a position in the sub-scanning direction where the pixel having the highest density value among the pixels in the scanned image is selected as the binding portion boundary line.
Note that the binarization processing in the case where the scanned image is a color multi-valued image focuses on, for example, any one of the RGB components (for example, the G component), and designates a component larger than a predetermined density threshold of the G component as a black pixel What is smaller than the predetermined density threshold of the G component may be a white pixel. Alternatively, RGB may be color-converted into a luminance component and a color difference component, and threshold processing may be performed on the luminance component.
In the following step S4, a process of extracting an object such as a page outline / ruled line / character line, which can be regarded as a straight line in the sub-scanning direction in the document, is executed. Here, FIG. 11 is a flowchart schematically showing the flow of the page outline / ruled line / character line extraction process.
[0013]
[Extraction of page outline from scanned image]
First, the process of extracting the page outline from the scanned image in step S41 will be described. Here, FIG. 12 is an explanatory diagram showing an example of a scan image having a page outline at the upper end, and FIG. 13 is a black pixel histogram on the left side of the bounding portion boundary line of the scan image shown in FIG. The x-axis of the histogram shown in FIG. 13 indicates the main scanning direction (vertical direction in FIG. 12) of the scanned image, and the upper end of the scanned image is associated with the left end of the histogram. Note that, in the case of a scanned image in which the page outline exists at the lower end, the lower end of the scanned image is associated with the right end of the histogram. Therefore, when a page outline exists at the upper end of the scanned image as shown in FIG. 12, a black band appears at the upper part of the scanned image, and a high vertical bar appears at the left end of the histogram shown in FIG. In the present embodiment, it is determined whether or not a scanned image has a page outline by using such characteristics.
More specifically, as shown in FIG. 13, the distance AO from the bounding portion boundary line to the left end of the scanned image (the left end in FIG. 12), the height BO of the histogram vertical bar, and the ratio are expressed by the following equation ( Calculated by 1),
(Equation 1)

If the calculated ratio k is larger than a predetermined threshold value, it is determined that the scanned image has a page outline.
Note that if there are page outlines above and below the scanned image, high vertical bars will appear at the left and right ends of the histogram. In such a case, the scanning is performed based on the high vertical bars at the left and right ends of the histogram. A determination is made as to whether a page outline exists in the image.
Here, the function of the page outline determining means is executed.
According to the above processing, when it is determined that the page outline exists in the scanned image, the page outline is extracted together with information on which of the upper and lower sides of the left and right pages exists, and temporarily stored in the RAM 33. Remember.
Note that the process of determining whether or not a page outline exists in the scanned image is executed for each of the left and right pages on the boundary of the bound portion of the scanned image.
[Extraction of ruled lines from scanned image]
In a succeeding step S42, a process of extracting a ruled line from the scanned image is executed.
[0014]
[Detection of ruled line candidates]
Here, FIG. 14 is an explanatory diagram showing an example of a scanned image having a long ruled line, and FIG. 15 is a black pixel histogram on the left side of the bounding portion boundary line of the scanned image shown in FIG. The x-axis of the histogram shown in FIG. 15 indicates the main scanning direction (the vertical direction in FIG. 14) of the scanned image, and the upper end of the scanned image is associated with the left end of the histogram. When a ruled line exists in the scanned image as shown in FIG. 14, a narrow peak appears in the histogram shown in FIG. In the present embodiment, it is determined whether or not a ruled line exists in a scanned image by using such characteristics.
More specifically, first, the height H of the narrow peaks appearing in the histogram shown in FIG. 15 is obtained, and the width W at the center position (half height position) of each obtained peak is obtained. Then, when there is a peak whose height H is higher than a predetermined threshold thH and whose width W at the center of the peak is smaller than the predetermined threshold thW, the peak is set as a ruled line candidate. .
Here, the function of the ruled line candidate extracting means is executed.
Subsequently, it is further determined whether or not the peak which is a candidate for a ruled line is a ruled line using the continuity of the ruled line. As shown in FIG. 16, an appropriate position on the candidate ruled line (for example, the position of the center line of the page) is set as a start point, and a candidate ruled line is searched for from the start point in the left and right direction. Part). If the number of cutting points is smaller than a predetermined threshold, this candidate is determined as a ruled line. By judging whether or not a line is a ruled line based on the ruled line continuity in this manner, it is possible to eliminate a horizontally written character line or a dotted line composed of small characters erroneously detected as a ruled line.
Here, the function of the ruled line determining means is executed.
[Detection of ruled line coordinates]
After the ruled lines are determined as described above, the coordinates of each ruled line are detected. As shown in FIG. 17, when the coordinate value of the ruled line in the main scanning direction (the y-axis direction in FIG. 17) is set to the middle point coordinate of the black pixel run of the ruled line as shown in FIG. The main scanning direction coordinate value at x1 at the left end is y1.
[0015]
[Selection of optimal ruled line]
Next, a ruled line optimal for distortion correction is selected from the candidate ruled lines. When a plurality of ruled lines are detected as shown in FIG. 18, it is necessary to select which ruled line should be used to correct the distortion. As an example of an optimum ruled line selection criterion, a ruled line length is longer than a predetermined threshold value, and a ruled line is set within a fixed width region on the left and right across a binding portion boundary line (a hatched region in FIG. 18). A ruled line closest to the outer shape of one of the upper and lower pages is selected on the condition that the ruled line is partially covered. FIG. 18 shows a case where one ruled line is selected from each of the left and right pages. Here, the ruled line (1) and the ruled line (2) are selected.
Further, as another example of the selection criterion of the optimal ruled line, the ruled line length is longer than a predetermined threshold value and is within a fixed width region on the left and right sides of the bounding portion boundary line (the hatched region in FIG. 19). ), A ruled line that is closest to the top page outline at the top of each page and the bottom page outline at the bottom of each page is selected on the condition that a part of the ruled line is applied. FIG. 19 shows a case where the left and right pages are further divided into upper and lower portions, and one ruled line is selected in each of the four blocks. Here, the ruled line (1) is selected in the upper left block, the ruled line (2) is selected in the lower right block, and the ruled line (3) is selected in the lower left block. In the upper right block in FIG. 19, the above two conditions (the length of the ruled line is longer than a predetermined threshold value, and a part of the ruled line is applied to the left and right fixed width regions sandwiching the bounding portion boundary line) There is no ruled line that satisfies (1), so there is no selected ruled line.
Regarding the above two conditions (the length of the ruled line is longer than a predetermined threshold value and a part of the ruled line is applied to the left and right fixed width regions sandwiching the bounding portion boundary line), However, only one of them may be satisfied. In the above example, “closest to the page outline” is used as the selection criterion. However, the present invention is not limited to this, and “the curve of the ruled line is the largest” may be used. Here, the “curve of the ruled line” is represented by the difference between the coordinate values of the left and right end points of the ruled line in the main scanning direction.
[0016]
[Determination of coordinate value of optimal ruled line]
When the optimum ruled line is selected, the coordinate value (in the main scanning direction) of the ruled line is determined. The coordinate value (in the main scanning direction) of the ruled line is determined by approximating and extending the selected ruled line until it reaches both ends of the left and right pages. In FIG. 20, since the coordinate value of the BC portion where the ruled line exists has already been determined by the above-described ruled line coordinate detection processing, the coordinate value (in the main scanning direction) of the ruled line is determined for the other extended portions. Will do. More specifically, the AB section shown in FIG. 20 estimates the coordinate values (in the main scanning direction) by linear approximation, and the CD section estimates the coordinate values (in the main scanning direction) using a polynomial approximation curve.
[Elimination of inappropriate ruled lines]
Finally, eliminate inappropriate ruled lines. This is because, as described above, when estimating the coordinate value by the polynomial approximation, if the shape of the estimated curve by the polynomial approximation is inappropriate, the distortion may increase instead of the correction. Ruled lines are eliminated. As an example of an inappropriate approximate curve shape, as shown in FIG. 21, a curve (1) in which the curve goes to the outside of the book, or a curve (2) in which the curve goes deeper inward beyond the center line. .
If the ruled line is excluded because the shape of the estimated curve is inappropriate, the optimum ruled line is selected again and the above processing is repeated.
If it is determined by the above processing that a ruled line exists in the scanned image, the ruled line is extracted together with information on where the ruled line exists on each of the left and right pages, and is temporarily stored in the RAM 33.
[Extraction of character line from scanned image]
In a succeeding step S43, a process of extracting a character line from the scanned image is executed. The process of extracting a character line from a scanned image in step S43 will be described. In the present embodiment, first, it is determined whether the character line in the scanned image is a vertical writing character line or a horizontal writing character line.
[0017]
[Character line determination]
A method for determining whether a character line in a scanned image is a vertical writing character line or a horizontal writing character line will be described. Here, FIG. 22 is a black-and-white inversion number histogram in the sub-scanning direction of the image shown in FIG. The horizontal axis in FIG. 22 indicates the black pixels in the sub-scanning direction (horizontal direction) (pixels in which the density value is higher than a predetermined density value among the pixels obtained by inverting the black and white of the scanned image) in the main scanning direction. The vertical axis in FIG. 22 indicates the number of black pixels at each position. FIG. 23 is a histogram of the number of black and white reversals in the main scanning direction of the image shown in FIG. The horizontal axis in FIG. 23 represents the black pixel in the main scanning direction (vertical direction) (the pixel whose density value is higher than a predetermined density value among the pixels obtained by inverting the black and white of the scanned image) in the sub-scanning direction. The vertical axis in FIG. 23 indicates the number of black pixels at each position. When the characters in the image are horizontally written scan images as shown in FIG. 8, the histogram in the sub-scanning direction as shown in FIG. 22 changes drastically, but the change in the histogram in the main scanning direction as shown in FIG. Few. Although not shown, when the character line in the scanned image is a vertical writing character line, the histogram in the main scanning direction changes sharply, but the change in the histogram in the sub-scanning direction is small.
[0018]
The above-described determination method is specifically realized by the following equations. First, according to the following equation (2),
(Equation 2)

Average value mean of histogram values Pnt (y) at a position in the main scanning direction y _H Is calculated. Here, height is the height of the image. Then, according to the following equation (3),
[Equation 3]

Variance σ of the histogram in the sub-scanning direction in the main scanning direction _H Is obtained.
Similarly, according to the following equation (4),
(Equation 4)

Mean value mean of histogram value Pnt (x) at position in sub-scanning direction x _V Is calculated. Here, width is the width of the image. Then, according to the following equation (5),
(Equation 5)

Variance σ of histogram in main scanning direction with respect to sub scanning direction _V Is obtained.
As described above, when the character line in the scanned image is a horizontally written character line, the variance σ of the histogram in the sub-scanning direction in the main scanning direction is used. _H Is the variance σ of the histogram in the main scanning direction with respect to the sub-scanning direction. _V Greater than. Conversely, if the character line in the scanned image is a vertically written character line, the variance σ of the histogram in the main scanning direction in the sub-scanning direction _V Is the variance σ of the histogram in the main scanning direction in the sub-scanning direction. _H Greater than. That is, the variance σ _H And variance σ _V By comparison with the above, it is possible to determine whether the character line in the scanned image is a vertical writing character line or a horizontal writing character line.
The reason why the black and white reversal number histogram is used to determine whether a character line in a scanned image is a vertical character line or a horizontal character line is to avoid confusion between a character line and a photograph portion. In general, when the values of the black pixel histograms are almost the same, the value of the black-and-white inversion number histogram is larger in the character area than in the photographic area.
Here, the function of the document determination means is executed.
[0019]
[Coordinate detection of horizontal text line]
After the character line is determined as described above, first, the coordinates of each horizontal writing character line are detected. In detecting the coordinates of the horizontal writing character line, a circumscribing rectangle extraction process for each character used in the character recognition process is performed, and a horizontal writing character line extraction process is performed. Since the character recognition processing is a well-known technique, the description thereof is omitted. Here, an example of the result of the character circumscribed rectangle extraction process and the character line extraction process of the scanned image is shown in FIG. Then, the coordinates of the center point of the circumscribed rectangle of each character are regarded as the coordinates of the character, and the coordinates of the horizontal writing character line are detected.
[Select optimal horizontal writing line]
Next, a horizontal writing character line optimal for distortion correction is selected from the extracted horizontal writing character lines.
When a plurality of horizontal writing character lines are detected, it is necessary to select which horizontal writing character line is to be used to correct the distortion. An example of the selection criterion of the optimal horizontal writing character line is basically the same as the above-described selection rule of the optimal ruled line, and as shown in FIG. 25, the length BC of the horizontal writing character line is smaller than a predetermined threshold value. It is conditional on that a part of a horizontal writing character line is placed in a long and fixed width area (shaded area in FIG. 25) on the left and right sides of the bounding portion boundary line. Select the horizontal writing line closest to the outline. Here, B is the center of the leftmost rectangle of the character line, and C is the center of the rightmost rectangle. Note that the selection of the optimum horizontal writing character line may be to select the horizontal writing character line closest to the page outline of each one from the left and right pages, or to further divide the left and right pages into upper and lower portions, The horizontal writing character line closest to the page outline of each block in the block may be selected.
The above two conditions (the length of a horizontally written character line is longer than a predetermined threshold value and a part of the horizontally written character line extends over a fixed width region on the left and right across the binding portion boundary line) , But not both, may be satisfied. Further, in the above example, “closest to the page outline” is used as the selection criterion, but the present invention is not limited to this, and “horizontal curvature of a horizontally written character line” may be used. Here, the "curvature of a horizontally written character line" is represented by a difference between coordinate values in the main scanning direction of center coordinates of character circumscribed rectangles at both ends of the horizontally written character line.
[0020]
[Determining the coordinate value of the optimal horizontal writing character line]
When the optimal horizontal writing character line is selected, the coordinate value (in the main scanning direction) of the horizontal writing character line is determined. The coordinate value (in the main scanning direction) of the horizontal writing character line is obtained by connecting the center points of the circumscribed rectangles of the characters in the horizontal writing character line and approximating and extracting the straight line portion and the curved line portion. (Direction) will be determined. More specifically, D shown in FIG. 25 is a bounding portion boundary line, a coordinate value (in the main scanning direction) is estimated by a polynomial approximation curve during BD, and an approximation is made between A and B at the leftmost end. The coordinate value (in the main scanning direction) is estimated from the value of the straight line.
[Elimination of inappropriate horizontal character lines]
Finally, eliminate inappropriate horizontal character lines. This is because, as described above, when estimating the coordinate value by the polynomial approximation, if the shape of the estimated curve by the polynomial approximation is inappropriate, the distortion may increase instead of the correction. This excludes horizontal writing character lines. An example of an inappropriate approximate curve shape is the same as the case of the ruled line described above. Although not particularly shown, a case where the curve goes to the outside of the book or a case where the curve goes deeply beyond the center line is obtained. Is the case.
If the horizontal character line is excluded because the shape of the estimated curve is inappropriate, the optimum horizontal character line is selected again, and the above processing is repeated.
By the above processing, when it is determined that the horizontal writing character line exists in the scanned image, the horizontal writing character line is extracted together with information on which position of the left and right pages the horizontal writing character line exists. Remember temporarily.
[Detection of coordinates of each character in vertical writing line]
Subsequently, the coordinates of each vertical writing character line are detected. In detecting the coordinates of a vertically written character line, a circumscribed rectangle extraction process for each character used in the character recognition process is performed. Since the character recognition processing is a well-known technique, the description thereof is omitted. Then, the coordinates of the center point of the circumscribed rectangle of each character are regarded as the coordinates of the character.
[0021]
[Extraction of horizontal writing line]
Next, horizontal writing character lines are extracted from each vertical writing character line. In the extraction of the horizontal writing character line, an outer shape obtained by connecting the uppermost characters of each vertical writing character line or an outer shape obtained by connecting the lowermost characters of each vertical writing character line is regarded as a horizontal writing character line. More specifically, when one character line is selected from each of the left and right pages, an outer shape obtained by connecting each character at the top or bottom of each vertical writing character line is extracted as one line. . When the left and right pages are further divided into upper and lower parts, and one character line is selected in each of the four blocks, the upper left and upper right parts are connected by connecting the top one character of each vertically written character line. Is regarded as one line, and in the lower left and lower right portions, the outer shape obtained by connecting the lowermost one character of each vertically written character line is regarded as one line and extracted.
At this time, as shown in FIG. 26, the extracted outer shape length BC is longer than a predetermined threshold value, and is within a fixed width region on the left and right across the binding portion boundary line (the hatched region in FIG. 26). Under the condition that a part C of the extracted outline is covered, the outline is extracted. Note that the above two conditions (the length of the extracted outer shape is longer than a predetermined threshold value and a part of the extracted outer shape is applied to the left and right fixed width regions sandwiching the binding portion boundary line) May satisfy either one or not both. Therefore, if the above condition is not satisfied, it means that there is no outer shape of the vertical writing character line for distortion correction.
[Determination of coordinate values of extracted outline]
When the outline is extracted according to the vertically written character line, the coordinate value (in the main scanning direction) of the outline of the vertically written character line is determined. As shown in FIG. 27, when the outer shape of a vertical writing character line is a shape obtained by connecting the top one character of each vertical writing character line, as shown in FIG. The coordinates (in the main scanning direction) of the outline of the vertically written character line are determined by connecting the points and approximating and extracting the straight line portion and the curved portion. In addition, as shown in FIG. 27, when the outer shape is formed by connecting the lowermost one character of each vertical writing character line, the center of the lower side of the circumscribed rectangle of each connected character is connected, and the straight line portion and the curved line By extracting and approximating the portion, the coordinate value (in the main scanning direction) of the outer shape of the vertically written character line is determined. More specifically, D shown in FIG. 26 is a binding portion boundary line, a coordinate value (in the main scanning direction) is estimated by a polynomial approximation curve during BD, and an approximation is made between A and B at the leftmost end. The coordinate value (in the main scanning direction) is estimated from the value of the straight line.
[0022]
[Removal of inappropriate vertical character lines]
Finally, eliminate the appearance of inappropriate vertical text lines. This is because, as described above, when estimating the coordinate value by the polynomial approximation, if the shape of the estimated curve by the polynomial approximation is inappropriate, the distortion may increase instead of the correction. This eliminates the outer shape of the vertical writing character line. Examples of inappropriate approximate curve shapes are the same as in the case of ruled lines and horizontal writing lines described above. Although not specifically shown, the case where the curve goes to the outside of the book, or the curve goes far beyond the center line This is the case when you dig into it.
If the shape of the vertical writing character line is excluded because the shape of the estimated curve is inappropriate, it means that there is no external shape of the vertical writing character line for distortion correction.
If it is determined by the above processing that the outer shape of the vertically-written character line exists in the scanned image, the vertical-written character line is displayed together with information on where the outer shape of the vertically-written character line exists on each of the left and right pages. The outline of the row is extracted and stored in the RAM 33 temporarily. As described above, the process of extracting the page outline / ruled line / character line (step S4) is completed by the processes of steps S41 to S43.
However, in the process of extracting the page outline / ruled line / character line, an appropriate target may not be found. Prior to the description of the image distortion correction processing in step S5 of FIG. 6, a method of these supplementary processings will be described.
[0023]
FIG. 28 is a flowchart showing a process of detecting and supplementing a reference line. Although the description of the replenishment process includes the process of extracting the page outline / ruled line / character line in the expression of reference line detection, it is the same process, and FIG. 28 is realized as a series of processes. Alternatively, the process of detecting the reference line may be separately executed as the page outline / ruled line / character line extraction process described above. Hereinafter, this page outline / ruled line / character line is referred to as a reference line in the sense of a line that refers to a position for correction.
First, the background density of the input scan image is detected (step S45). Basically, in each line in the main scanning direction, the maximum pixel value (the brightest pixel value) is regarded as the background density of the line. FIG. 29 shows a scanned image of a right-and-left spread page including a binding portion. There is a portion with a high background density near the binding portion. FIG. 30 is a graph showing the change in pixel value in which the background density of each line in the main scanning direction of the scanned image is plotted in the sub-scanning direction. Here, the position where the background density is highest is regarded as the binding portion position of the scanned image.
Next, the distortion area ratio of each of the left and right pages is calculated (step S46 in FIG. 28). The ratio of the distance from the binding portion to the position where the background density is equal on each of the left and right pages is regarded as the ratio of the distortion region. In the example of FIG. 30, if the background density I is given in advance, the distance from the binding portion on the left page to the position where the background density is I is d1, and the right page is d2, so the distance ratio is d1: d2. This corresponds to the shaded portion in FIG. Alternatively, as shown in FIG. 31, instead of giving the background density I in advance, fixed points x1 and x2 are predetermined on the left and right pages, and the background density at x1 and the background density at x2 are higher (the pixel value is smaller = dark). ) May be regarded as I.
Next, a reference line is detected (step S47 in FIG. 28). This processing is exactly the same as the extraction processing of the page outline / ruled line / character line. FIG. 32 is a diagram showing a state where a reference line is detected from the scan image of FIG. In this example, a pair of (two) reference lines could be detected from the left page, but only one reference line could be detected from the right page. For convenience, a reference line below the main scanning direction is referred to as a reference line 1, and a reference line above the reference line is referred to as a reference line 2.
In the reference line supplementing process (step S48 in FIG. 28), if only one reference line is detected from one page, the missing reference line is supplemented by using the reference line of the other page. Things. Basically, the reference line of the other page is symmetrically moved around the binding portion, and is expanded or contracted in accordance with the distortion area ratio of the left and right pages calculated in step S46. This is the processing to be regarded.
[0024]
In the example of FIG. 32, a reference line from above the right page has not been detected. Therefore, the reference line 2 on the left page is symmetrically moved to the right page to expand and contract. FIG. 33 is an explanatory diagram showing a process of supplementing the reference line detected in FIG. Since the distortion area ratio of the left and right pages is d1: d2 and d1> d2, the distance l1 (ell 1) between the arbitrary point P on the reference line 2 on the left page and the binding portion, and the corresponding point P ′ on the right page What is necessary is just to symmetrically move the reference line 2 and shorten it so that the relationship of the distance l2 (ell 2) with the binding portion becomes l1: l2 = d1: d2. The point Q 'on the right page is a corresponding point of the point Q on the left end of the left page. The portion further to the right of the point Q 'extends a straight line to a position where the coordinates in the main scanning direction (y direction) are equal to the point Q'.
FIGS. 34 and 35 are examples similar to FIGS. 32 and 33, except that the distortion area ratio of the left and right pages is d3: d4, d3 <d4, and the right page is wider. Are different. In this case, the reference line 2 on the left page is symmetrically moved to the right page and expanded. The relationship between the distance R3 between the point R on the left page and the binding portion and the distance l4 between the corresponding point R 'on the right page and the binding portion is l3: 14 = d3: d4. The original point of the right end point S 'of the right page is the point S of the left page, and the portion to the left of this point has no corresponding point on the reference line of the right page.
FIG. 36 shows an example in which the reference line 1 cannot be detected on the left page and the reference line 2 cannot be detected on the right page. The distortion area ratio of the left and right pages is d5: d6 (d5> d6). In this example, the reference line 2 on the left page is symmetrically moved to the right page to shorten it, and the reference line 1 on the right page is symmetrically moved to the left page and expanded, thereby supplementing the reference lines respectively (FIG. 37).
[0025]
FIG. 38 shows an example in which only the reference line on the upper side of the left page can be detected, and no reference line can be detected on the upper and lower sides of the right page (the distortion area ratio is d7: d8 (d7> d8)). In this method, the reference line on the upper side of the left page is moved to the right page and shortened. As a result, the reference line on the upper side is also added to the right page (FIG. 39). The reference line on the lower side is not detected on both pages, so if the position of the optical axis of the scanner lens in the main scanning direction is known, a straight line parallel to the sub-scanning direction passing through this position is scanned. A straight line parallel to the sub-scanning direction passing through the center position of the image in the main scanning direction is regarded as a lower-side reference line (a dashed line crossing the center of the figure). As a result, the right page can be corrected.
Note that, contrary to the example of FIG. 38, when the reference line on the upper side cannot be detected on both the left and right pages, if the position of the optical axis of the scanner lens in the main scanning direction is known, A parallel straight line and, if unknown, a straight line parallel to the sub-scanning direction passing through the center position of the scanned image in the main scanning direction are regarded as reference lines (FIGS. 40 and 41).
The above is the reference line supplement processing. In the subsequent step S5 (see FIG. 6), the image distortion correction processing is executed. The information indicating the position and the shape of the reference line supplemented above is stored in the RAM 33 and used for the image distortion correction processing in the next step.
As shown in FIG. 42, the image distortion correction process is performed as a line (reference line 1) serving as a reference for distortion correction (expansion), which is located near the lower side of the scanned image. A process of selecting one of the rows (step S51: reference line 1 selection process), which corresponds to the above-described reference line 1 and is located near the upper side of the scanned image as a reference line 2 for calculating a correction rate (expansion rate) (Step S52: reference line 2 selection process), and expands the scanned image based on reference line 1 and reference line 2 in the main scanning direction by selecting one of the page outline / ruled line / character line. A process for correcting distortion and a process for expanding a scanned image based on a distance (height of floating) of the page surface from the scan surface calculated from the reference line 1 or the reference line 2 or a character circumscribed rectangle of the corrected image. Processing for correcting distortion of the scanning direction: it is constituted by (step S53 main scanning direction distortion correction process and the sub-scanning direction distortion correction processing) and.
[0026]
Here, in the reference line 1 selection processing (step S51) and the reference line 2 selection processing (step S52), the reference line 1 or the reference line is generated from the reference line information generated by the extraction and supplement of the reference line described above. As the line 2, any one of the page outline / ruled line / character line located near the upper side (or lower side) of the scanned image is selected for each of the left and right pages. The priority of selection of the page outline, ruled line, and character line in the present embodiment is as follows.
Page outline> ruled line> character line
And The selection priority is such that a character line has a lower extraction accuracy than a page outline or a ruled line, and a more accurate distortion correction rate can be obtained by using a page outline outside an image. This is because we can do it.
Next, the main scanning direction distortion correction processing and the sub-scanning direction distortion correction processing (step S53) will be described. This processing differs depending on whether the scanner parameter is known or unknown. That is, if the model of the scanner is known, the position of the optical axis of the scanner lens of the scanner in the main scanning direction (Ak in FIG. 43) and the distance between the center of the scanner lens and the scanning plane (focal plane distance in FIG. 43) Are stored in advance as scanner parameters.
First, a case where the scanner parameters are known will be described. In this case, basically, after correcting one line in the main scanning direction, the process of correcting the line in the sub-scanning direction is performed on all the lines in the main scanning direction or the reference line 1 or the reference line 2. This is performed for the main scanning direction line of the curved portion.
The correction in the main scanning direction is performed as follows. In FIG. 44, the reference line 1 and the reference line 2 before correction are indicated by solid lines, and the lines after correction are indicated by dotted lines. The dotted line after the correction is obtained by directly extending the straight line portion (flat portion) of the reference lines 1 and 2 before the correction. Here, at the position X in the sub-scanning direction, the point P on the reference line is corrected to P 'and the point Q on the reference line is corrected to Q'. Now, assuming that an arbitrary point Y in the main scanning direction at the position X is corrected to Y ′, the following relational expression holds.
YP / YQ = Y'P '/ Y'Q'
Therefore, if the position of each point in the main scanning direction is expressed as P (y) for point P,

So, transform this,

It becomes.
Using the above equation, the position Y (y) before correction of the point to come to the position of Y ′ (y) after correction can be obtained. That is, the pixel value of Y (y) before correction may be moved to Y ′ (y) after correction. However, since the calculated value of Y (y) is generally a decimal number, a linear interpolation result of pixel values corresponding to integer positions before and after the decimal point is used. That is, in FIG. 45, assuming that the integers before and after Y (y) are N and N + 1, and the corresponding pixel values are D (N) and D (N + 1), the pixel value D (Y (Y ( y)) is calculated so as to satisfy the linear relationship shown in the figure.
[0027]
The above is a case where all the reference lines 1 and 2 are any of the page outline / ruled line / character line. If an appropriate page outline, ruled line, or character line is not extracted, the following is performed. I do. FIG. 46 shows an example in which the page outline / ruled line / character line does not exist above the right page. In this case, the locus of the lens optical axis (L in FIG. 46) is regarded as the reference line 2, and the same processing as described above is performed. However, in this case, Q and Q ′ in FIG. 44 match. If there is no page outline / ruled line / character line below the page, the process is performed in the same manner assuming that L is the reference line 1 (P and P 'coincide). If there is no page outline / ruled line / character line above / below the page, there is no clue, and the page is not corrected.
The above processing is executed independently for each of the left and right pages of the scanned image. In addition, information on the reference line 1 and the reference line 2 in a certain spread left and right page may be held, and this may be used for correcting another spread left and right page. For example, when scanning a continuous spread page, it is assumed that the left and right spread pages are considered as one set, and the reference line 1 and the reference line 2 are extracted only once in five sets. After extracting 2, the information may be used as it is in the subsequent four sets. This saves time for extracting the reference lines 1 and 2, thereby improving the overall processing speed.
[0028]
The correction in the sub-scanning direction is performed as follows. That is, using the amount of change in the distance of the page surface from the scan surface, the image is expanded by expanding the book document so that the cross-sectional curve of the book document becomes a straight line. Specifically, the image is expanded based on the amount of change in the height of the page surface from the scan surface. As shown in FIG. 47, the shape of the book document page surface is read to make a minute triangle for each line, and the image length Ln in one line for reading the page is expressed by the following equation.
Ln = square root {1+ (Tn−Tn−1) ** 2}
, And the accumulation of the image length Ln is defined as the extension length of the page. As a result, the hypotenuse of the approximated triangle becomes almost equal to the shape of the curved page (polygonal approximation), and the accumulation is used as the image length of the page to obtain an accurate page length. In particular, due to the shape approximation using the minimum pitch for each line, the correction accuracy of the length is high.
The calculation is performed with the pixel interval of the pixel of interest on the adjacent line in the main scanning direction as the reference “1”, so that the reading line interval changes, that is, the sampling pixel interval is sequentially changed with respect to the original when the document is flat. In contrast, the image decompression process is applied. In addition, geometrically the image projection double length in the main scanning direction and the distance of the page surface from the scanning surface are proportional to each other, and as shown in FIG. 43, the address A3 of the reference line 1 or the reference line 2 in the main scanning direction. And the distance T from the scan surface of the page surface from the address Ka of the straight line portion (flat portion) of the reference line 1 or 2 to the following equation.
T = focal plane distance * (A3-Ak) / {(Ak-Ka)-(A3-Ak)}
Can be obtained by Based on the difference between the depths of the adjacent main scanning lines calculated in this way, it is possible to calculate the correction position of the page by approximating with a fine pitch straight line for each pixel. The method of obtaining the reference line 1 / reference line 2 and the case where there is no appropriate reference line 1 / reference line 2 are basically the same as the case of correction in the main scanning direction.
The distance T of the page surface from the scan surface calculated in this way is used for calculating the image length Ln in the sub-scanning direction to be restored for each line as shown in FIG. That is, the following equation
Ln = square root {1+ (Tn−Tn−1) ** 2}
Thus, the image length Ln in the sub-scanning direction to be corrected for each line is calculated. Therefore, the accumulation of the image length Ln becomes the page length in the sub-scanning direction.
Usually, as the distance of the page surface from the scan surface increases, the amount of change in the height increases. Thus, in the present embodiment, image expansion is restricted in accordance with the distance of the position from the scan surface of the page surface. For example, the difference between adjacent boundary addresses is limited to (height [mm] / 5) [pixels] at that position to suppress errors in sub-scanning direction correction due to erroneous boundary detection.
[0029]
In the correction in the sub-scanning direction, the image length Ln differs depending on the position, and the positions of the corresponding points of the original image and the corrected image generally differ as shown in FIG. When the points existing in N are corrected to N ′ and the points in N + 1 are corrected to N + 1 ′, the pixel value at the position of the integer X ′ (which also does not always coincide with N + 1 ′) is to be obtained. . If the distance between N ′ and X ′ is a and the distance between X ′ and N + 1 ′ is b, the corresponding point X of X ′ in the original image is
(X−N) / (N + 1−X) = a / b
In a relationship.
When the position of X is known from the above equation, the pixel value of X before correction may be moved to X 'after correction. However, since the calculated value of X is generally a decimal number, a linear interpolation result of pixel values corresponding to integer positions before and after the decimal point is used. That is, in FIG. 49, assuming that pixel values corresponding to N and N + 1 before and after X are D (N) and D (N + 1), the pixel value D (X) at X satisfies the linear relationship as shown in FIG. Is calculated. The above processing is performed for all positions in the sub-scanning direction, or positions in the sub-scanning direction where the reference line 1 or the reference line 2 is a curve.
The above processing is executed independently for each of the left and right pages of the scanned image. Alternatively, information on the distance from the scan surface of the book document and the reference line 1 or reference line 2 in a certain spread left and right page may be stored and used for correcting other spread left and right pages. For example, when scanning a continuous double-page spread, the spread left and right pages are considered as one set, and the calculation of the distance from the scan surface of the book document and the extraction of the reference lines 1 and 2 are performed only once in five sets. After calculating the distance and extracting the reference line 1 and the reference line 2 in a certain set, the information may be used as it is in the subsequent four sets. This saves time for calculating the distance of the book manuscript from the scan surface and extracting the reference lines 1 and 2, thereby improving the overall processing speed.
Up to here, the main scanning direction distortion correction processing and the sub-scanning direction distortion correction processing (step S53) when the scanner parameters are known are described.
[0030]
Next, a case where the scanner parameters are unknown will be described.
The correction in the main scanning direction is basically the same as that in the case where the scanner parameters are known, but the processing when an appropriate page outline, ruled line, or character line is not extracted is slightly different. In FIG. 46, when the scanner parameters are known, the locus of the lens optical axis (L in FIG. 46) is regarded as the reference line 2, but when unknown (FIG. 50), the center line parallel to the sub-scanning direction (FIG. 50) is used. C) is regarded as reference line 2. Also in this case, Q and Q ′ in FIG. 44 match. If there is no page outline / ruled line / character line at the bottom of the page, the same operation is performed by regarding C as the reference line 1 (P and P 'coincide). If there is no page outline / ruled line / character line above / below the page, there is no clue, and the page is not corrected.
Next, the sub-scanning direction distortion correction processing will be described. Here, FIG. 51 is a flowchart schematically showing the flow of the sub-scanning direction distortion correction processing. As shown in FIG. 51, in step S101, a circumscribed rectangle A (see FIG. 52) of a character is extracted by a method used for character recognition processing based on a scan image in which distortion in the main scanning direction has been corrected. Here, the character recognition processing is a known technique, and a description thereof will be omitted. The reason why the circumscribed rectangle A of the character is extracted in this way is to correct distortion in the sub-scanning direction based on a change in the shape of the circumscribed rectangle A of the character. Here, as shown in FIG. 52, the length w of the horizontal side, the length h of the vertical side, and the center B of the character of the character circumscribed rectangle A are defined. Here, the center B of the character is the intersection of the diagonal lines of the circumscribed rectangle A. Subsequently, as shown in FIG. 53, the scan image is divided into a plurality of strip-shaped areas C in a direction parallel to the page binding section 41 of the book document 40 (step S102). A feature amount relating to the included character circumscribed rectangle A is obtained (step S103). Here, the character circumscribed rectangle A included in a certain strip area C is a circumscribed rectangle A whose center is included in the strip area C. For example, the circumscribed rectangle A included in the strip area C1 in FIG. 53 is a shaded rectangle in the figure.
[0031]
By the way, the feature quantity relating to the character circumscribed rectangle A is
(Length of horizontal side of character) / (length of vertical side of character) = w / h
Required based on. That is, with respect to each strip area C, the average value of the w / h values of all the character circumscribed rectangles A included therein is used as the feature amount of the strip area C.
However, simply calculating the average value of w / h may be inappropriate. Some characters, such as punctuation marks and symbols in mathematical formulas, are originally small in size and have unstable w / h values. In addition, in extracting a rectangle, adjacent characters may be stuck together and extracted, resulting in a character circumscribed rectangle A where w is extremely large. When a feature amount is obtained, it is necessary to eliminate such special characters and those having an extremely large w in advance. Therefore, in the subsequent step S104, a threshold value is set in advance, the character circumscribed rectangle A having a value of h smaller than the threshold value is excluded in advance, and a threshold value for the w / h ratio is set in advance. Is also excluded in advance. For example, the shaded character circumscribed rectangle A shown in FIG. 54 is removed in advance.
In the following step S105, after excluding the extreme character circumscribed rectangle A as described above, the average value of w / h of the character circumscribed rectangle A in each strip region C is obtained. FIG. 55 shows an example of the average value of w / h of the circumscribed rectangle A in each strip region C. The strip area C2 in FIG. 55 is a strip area including a page binding portion.
Subsequently, it is determined whether or not the character circumscribed rectangle A exists in the strip region C2 including the page binding portion (step S106). This is because, as shown in FIG. 54, in general, the character circumscribed rectangle A generally does not exist near the page binding portion. If the character circumscribed rectangle A exists in the strip area C2 including the page binding portion (Y in step S106), the feature amount has been calculated using the character circumscribed rectangle A, and thus the process directly proceeds to step S108.
On the other hand, when the character circumscribed rectangle A does not exist in the strip region C2 including the page binding portion (N in step S106), the process proceeds to step S107, and the feature amount of the strip region C2 including the page binding portion is obtained. The strip region C2 including the page binding portion is identified, for example, by calculating the background density change of a scanned image (for example, a monochrome multi-valued image) for each strip region C, and determining the density value of the lightest density in the strip region C. Is realized by seeking
[0032]
FIG. 56 shows an example of calculating the background density change, and the strip region having the highest background density is considered to be the strip region C2 including the page binding portion.
In the case where the scanned image is a color multivalued image, the strip region C2 including the page binding portion is identified by, for example, focusing on one of the RGB components (for example, the G component) and using the background density of the G component. And identify them. Alternatively, RGB may be color-converted into a luminance component and a color difference component, and the strip region C2 including the page binding portion may be identified using the luminance component.
The feature amount of the strip region C2 including the page binding portion is determined as follows. Here, there is a character circumscribed rectangle A that can be a target of calculation of the statistical feature amount, and a predetermined constant value is set for the feature amount of the strip region C which is the closest to the strip region C2 including the page binding portion. The value calculated by the multiplication is regarded as the feature amount in the strip region C2 including the page binding portion. That is, in the example shown in FIG. 55, since the character circumscribed rectangle A exists in both the left and right strip regions C3 and C4 of the strip region C2 including the page binding portion, either one of the appropriate feature amounts is selected ( Here, the right side of the mark (○) is multiplied by a predetermined constant value (here, 0.5), and this is used as the feature amount of the strip region C2 including the page binding portion.
In the subsequent step S108, appropriate filtering processing for the feature amount of each strip area C, for example, processing for obtaining a moving average in the direction of change in the position of the strip area C (ie, the sub-scanning direction) is performed. The change of the feature amount (in the sub-scanning direction) with respect to the change of the position is made gentle. However, special processing is required in the vicinity of the page binding section also here. This is because if filtering is performed using windows having the same length in the sub-scanning direction, the sharpness of the change in the feature amount near the page binding portion is lost.
[0033]
Here, FIG. 57 shows a result obtained by performing a filtering process on the feature amount of each strip region C shown in FIG. 55 using a window whose length is all five. As shown in FIG. 57, when the filtering process is performed using a window whose length is all 5, the change in the feature amount (w / h) near the page binding portion becomes too gentle. In such a case, it becomes impossible to perform appropriate image correction near the page binding portion.
Therefore, in the present embodiment, at the time of the filtering process, the window length near the page binding portion is set so that the window of the filter does not straddle the strip regions C3 and C4 on both sides of the strip region C2 including the page binding portion. To adjust. Here, FIG. 58 is a graph showing the result of performing the filtering process by adjusting the window length near the page binding portion. As shown in FIG. 58, when the window length is adjusted in the vicinity of the page binding section, a change in the characteristic amount (w / h) in the vicinity of the page binding section can be appropriately expressed, so that good image correction can be realized.
In the following step S109, the estimated distortion amount of each strip region C is calculated. The calculation method of the estimated distortion amount of each strip region C is as follows.
First, a strip area (reference strip area) serving as a reference for calculating the distortion amount of the strip area is determined. Here, the strip area C with the smallest distortion, for example, the strip area C with the largest characteristic amount (w / h) is set as the reference strip area. This processing may be performed in common for the left and right pages, but the reference strip area may be independently defined for the left and right pages. FIG. 58 shows an example in which the reference strip areas are defined independently for the left and right sides. The strip area C marked with a circle is the reference strip area, and the left reference feature amount is “Lw0 / Lh0”, and the right reference The feature amount is indicated by “Rw0 / Rh0”, respectively.
Next, the feature amount w0 / h0 of the reference strip region is set as the reference feature amount of the entire scanned image,
(Feature value of each strip region) / (Reference feature value) = (w / h) / (w0 / h0)
Is calculated as the estimated distortion amount of each strip region.
If the strip area C near the outside of the page outside the page binding section is set as the reference strip area, there may be a case where it is not possible to calculate an appropriate estimated distortion amount due to a large difference in font and print size from the vicinity of the page binding section. Can be When targeting such an image, it is effective to limit the search range of the reference strip region to the vicinity of the page binding portion in advance. In order to realize this, it suffices to carry out only for the position in the sub-scanning direction of the portion where the reference line 1 or the reference line 2 is a curve.
[0034]
Finally, enlargement processing is performed on the scanned image in the short side direction (sub-scanning direction) of the strip region C to correct distortion near the page binding portion (step S110). In this case, the enlargement ratio (distortion correction ratio (expansion ratio) in the sub-scanning direction) is the reciprocal of the estimated distortion amount calculated in step S109, that is,
(Reference feature amount) / (Feature amount of each strip region) = (w0 / h0) / (w / h)
And Here, when the above-mentioned reference strip region is defined in common for the left and right, this enlargement ratio is also calculated using the reference feature amounts common to the left and right, and when independently defined, it is independently calculated for each of the right and left reference feature amounts. To do. FIG. 59 shows the corrected magnification calculated based on the feature amounts shown in FIG.
Here, the function of the sub-scanning direction distortion correction ratio calculating means and the function of the sub-scanning direction distortion correcting means are executed.
In this case as well, the strip region C distant from the vicinity of the page binding portion is likely to be a region having no original image distortion, and thus it may be better not to be subjected to enlargement processing. This is because there is a possibility that unnatural distortion may be caused by performing the enlargement processing. In order to prevent this, it is sufficient to enlarge only the position in the sub-scanning direction of the portion where the reference line 1 or the reference line 2 is a curve. That is, when the reference line 1 or the reference line 2 is a straight line, the estimated distortion amount is “1”.
Further, when a common correction magnification rate is applied in the strip area C, the correction magnification rate at the boundary between the adjacent strip areas C becomes discontinuous, and the corrected image becomes unnatural. Therefore, the correction magnification is corrected in advance so that the correction magnification at the boundary between the adjacent strip regions C changes continuously. This is done, for example, by plotting the corrected enlargement ratio of the central portion of the strip region C shown in FIG. 59 as points indicating the reciprocal of the estimated distortion amount, connecting these points with a line segment, and complementing the straight line, thereby obtaining other portions. This can be realized by setting the correction magnification. With the above processing, the correction magnification rate in the sub-scanning direction of the scanned image is determined.
The above processing is executed independently for each of the left and right pages of the scanned image. Alternatively, information on the correction enlargement ratio for a certain spread left and right page may be stored and used for correction of another spread left and right page. For example, when scanning a continuous double-page spread, it is assumed that the spread right and left pages are considered as one set, and the correction magnification is calculated only once in five sets. In the four sets, the information may be diverted as it is. This saves time for calculating the correction magnification rate, thereby improving the overall processing speed.
As described above, the sub-scanning direction distortion correction processing (step S55) is completed by the processing of steps S101 to S110, and the distortion correction processing of the scanned image illustrated in FIG. 6 is completed. Here, FIG. 60 is a plan view showing an image in which distortion has been corrected. According to the above processing, the distortion of the scanned image generated near the page binding portion as shown in FIG. 8 is corrected as shown in FIG.
[0035]
【The invention's effect】
As described above, according to the present invention, an appropriate correction method can be selected and corrected when a scanner parameter is known or unknown, and a reference line cannot be detected on one page. However, if it can be detected on the other page, the target is moved with respect to the binding portion and expanded and contracted in accordance with the distortion area ratio of the left and right pages, so that the correction accuracy is improved as compared with the conventional method. In particular, even when no reference line is detected on one page, correction can be performed on both pages if the reference line can be detected on the other page.
[Brief description of the drawings]
FIG. 1 is a vertical sectional front view showing a configuration of a scanner unit of the present invention.
FIG. 2 is a perspective view showing an upper portion of a digital copier equipped with a scanner unit.
FIG. 3 is a block diagram illustrating electrical connection of a control system of the scanner unit.
FIG. 4 is a block diagram illustrating a basic internal configuration of an image processing unit according to the present invention.
FIG. 5 is a block diagram showing an electrical connection of a main control unit.
FIG. 6 is a flowchart schematically illustrating a flow of a scan image distortion correction process.
FIG. 7 is a perspective view showing a state when reading a book document.
FIG. 8 is a plan view showing an example of an input image.
FIG. 9 is a diagram illustrating a portion where distortion occurs in a scanned image.
FIG. 10 is a diagram illustrating a black pixel histogram on the left side of a binding portion boundary line of the image illustrated in FIG. 8;
FIG. 11 is a flowchart schematically illustrating a flow of a page outline / ruled line / character line extraction process.
FIG. 12 is an explanatory diagram illustrating an example of a scanned image having a page outline at the upper end.
13 is a diagram illustrating a black pixel histogram on the left side of a bounding portion boundary line of the scan image illustrated in FIG. 12;
FIG. 14 is an explanatory diagram illustrating an example of a scanned image including a long ruled line.
FIG. 15 is a diagram illustrating a black pixel histogram on the left side of a bounding portion boundary line of the scan image illustrated in FIG. 14;
FIG. 16 is a diagram for explaining determination of whether or not a ruled line is based on ruled line continuity;
FIG. 17 is a diagram for describing coordinate detection of a ruled line;
FIG. 18 is an explanatory diagram illustrating a case where one ruled line is selected from each of left and right pages.
FIG. 19 is an explanatory diagram showing a case where the left and right pages are further divided into upper and lower portions, and one ruled line is selected in each of the four blocks.
FIG. 20 is an explanatory diagram illustrating determination of coordinate values of an extended optimum ruled line.
FIG. 21 is an explanatory diagram illustrating an example of an inappropriate approximate ruled line.
FIG. 22 is a diagram showing a black-and-white inversion number histogram in the sub-scanning direction of the image shown in FIG. 8;
FIG. 23 is a diagram showing a black-and-white inversion number histogram in the main scanning direction of the image shown in FIG. 8;
FIG. 24 is a diagram illustrating an example of a result of a character circumscribed rectangle extraction process and a character line extraction process of a scan image.
FIG. 25 is an explanatory diagram showing selection of an optimal horizontal writing character line.
FIG. 26 is an explanatory diagram showing extraction of a horizontally written character line from a vertically written character line.
FIG. 27 is an explanatory diagram illustrating determination of coordinate values of the outer shape of a vertically written character line.
FIG. 28 is a flowchart showing a process of detecting and supplementing a reference line.
FIG. 29 is a diagram illustrating an example of a scanned image of a left-right spread page including a binding unit.
FIG. 30 is an explanatory diagram showing how to calculate a distortion region ratio.
FIG. 31 is an explanatory diagram showing a method of obtaining a distortion region ratio.
FIG. 32 is a diagram illustrating a state where a reference line is detected from the scan image of FIG. 29;
FIG. 33 is an explanatory diagram showing a process for supplementing the reference line detected in FIG. 32;
FIG. 34 is a diagram showing a state where a reference line is detected from a scan image showing another example.
FIG. 35 is an explanatory diagram showing a process of supplementing the reference line detected in FIG.
FIG. 36 is a diagram showing a state where a reference line is detected from a scan image showing another example.
FIG. 37 is an explanatory diagram showing a process of supplementing the reference line detected in FIG. 36;
FIG. 38 is a diagram showing a state where only the reference line on the upper side of the left page can be detected, and no reference line can be detected on the upper and lower sides of the right page.
FIG. 39 is an explanatory diagram showing a process of supplementing the reference line detected in FIG. 38;
FIG. 40 is a diagram showing a state in which a reference line on the upper side cannot be detected on both left and right pages.
FIG. 41 is an explanatory diagram showing a process of supplementing the reference line detected in FIG. 40;
FIG. 42 is a flowchart schematically showing a flow of an image distortion correction process.
FIG. 43 is a diagram illustrating scanner parameters.
FIG. 44 is an explanatory diagram showing reference lines before and after correction in the main scanning direction.
FIG. 45 is an explanatory diagram showing linear interpolation of pixel values corresponding to integer positions.
FIG. 46 is an explanatory diagram showing a case where a page outline / ruled line / character line does not exist above the right page.
FIG. 47 is an explanatory diagram showing an approximation of the shape of a curved page.
FIG. 48 is a diagram illustrating the positions of points corresponding to the original image and the corrected image.
FIG. 49 is an explanatory diagram showing linear interpolation of pixel values corresponding to integer positions.
FIG. 50 is an explanatory diagram showing reference lines before and after correction in the main scanning direction when scanner parameters are known.
FIG. 51 is a flowchart schematically showing the flow of sub-scanning direction distortion correction processing.
FIG. 52 is an explanatory diagram showing a circumscribed rectangle of a character based on a scanned image.
FIG. 53 is a diagram showing a state in which a scanned image is divided into a plurality of strip-shaped areas in a direction parallel to a page binding portion of a book document.
FIG. 54 is an explanatory diagram showing a character circumscribed rectangle to be excluded;
FIG. 55 is a graph showing an example of the average value of the feature values of the circumscribed rectangles in each strip region.
FIG. 56 is a graph showing an example of a background density change for each strip region.
FIG. 57 is a graph illustrating a result of performing a filtering process on the feature amount of each strip region illustrated in FIG. 55;
FIG. 58 is a graph showing a result of performing a filtering process by adjusting a window length near a page binding portion;
FIG. 59 is a graph showing a correction magnification calculated based on the feature amounts shown in FIG. 58;
FIG. 60 is a plan view showing an image in which distortion has been corrected.
FIG. 61 is a front view showing a state in which a page binding portion of a book document has risen from a contact glass.
[Explanation of symbols]
1 scanner section, 2 contact glass, 16 image forming apparatus, 20 image processing section, 29 image distortion correcting section, 31 CPU, 33 RAM, 40 book original, 41 page binding section

Claims (22)

A scanned image including a page binding portion obtained by reading a book document image placed on a scanning surface and two left and right images is read in the main scanning direction so that the page binding portion is substantially parallel to the main scanning direction. And an image distortion correction device that performs distortion correction by extending in the sub-scanning direction, wherein a reference line extracting unit that extracts, as a reference line, a portion forming a curve extending in the sub-scanning direction included in the document from the scanned image, Reference line supplementing means for generating a reference line that could not be extracted on a page based on a reference line of another page, and correcting distortion in the main scanning direction of the scan image based on the shape of the extracted or generated reference line An image distortion correction device comprising: a main scanning direction distortion correction unit. The reference line extracting means may be a line or a main scanning line connecting the positions of characters of a page outline, a ruled line, and a character line perpendicular to the main scanning direction, which are located near an upper side and a lower side in the main scanning direction in the scanned image. A pair of lines connecting the positions of the leading or trailing characters of the character line parallel to the direction is extracted as a pair of reference lines, and the main scanning direction distortion correction unit determines that the curved portion of the reference line matches the extension of the straight line portion 2. The image distortion correcting apparatus according to claim 1, wherein a straight line is assumed, and corresponding points of the image before and after the correction are determined based on the assumed straight line to correct the distortion in the main scanning direction. The reference line supplementing means determines a page binding portion and a distortion region of the left and right pages from the background density of the scanned image, and symmetrically moves the reference line around the page binding portion to obtain a width of the distortion region of the left and right pages. The image distortion correction apparatus according to claim 1, wherein the symmetrically moved reference line is expanded and contracted in the sub-scanning direction in accordance with the following. The reference line replenishing means sets the brightest pixel value on each line in the main scanning direction as the background density of the line, and regards the position of the line with the highest background density in the sub-scanning direction as a page binding section, and 4. The image distortion correction device according to claim 3, wherein an area having a background density equal to or lower than a predetermined brightness is determined as a distortion area in the vicinity of the part. The reference line replenishing means sets the brightest pixel value on each line in the main scanning direction as the background density of the line, sets the position of the line with the highest background density in the sub-scanning direction as the position of the page binding section, The position of the density value with the higher background density at two predetermined positions in the direction and the position of the background density in the sub-scanning direction on the other page, which is equal to the density value, are detected and determined as a distortion area. 4. The image distortion correction device according to claim 3, wherein: The main scanning direction distortion correcting means may include a sub-scanning line passing through the main lens in the main scanning direction if the reference line on the upper side or the lower side does not exist on the left and right pages and the position of the optical axis of the scanner lens in the main scanning direction is known. The image distortion correction device according to claim 1, wherein a straight line parallel to the direction is regarded as a reference line on an upper side or a lower side in the main scanning direction, and distortion in the main scanning direction is corrected. The main scanning direction distortion correction unit is configured such that when the reference line on the upper side or the lower side of the main scanning direction does not exist on both the left and right pages and the position of the optical axis of the scanner lens in the main scanning direction is unknown, the main scanning direction of the scanned image is 6. The distortion in the main scanning direction is corrected by regarding a straight line parallel to the sub-scanning direction passing through the center position as a reference line on the upper side or the lower side in the main scanning direction. An image distortion correction apparatus according to claim 1. The main scanning direction distortion correction unit holds information on the reference line extracted by the reference line extraction unit or generated by the reference line supplementation unit, and then corrects the reference line for correction of another scan image to be subsequently corrected. The image distortion correction apparatus according to claim 1, wherein distortion correction in the main scanning direction is performed using the information of (1). 9. The apparatus according to claim 1, wherein the main-scanning-direction distortion correction unit performs distortion correction in the main-scanning direction only at a position in the sub-scanning direction corresponding to a curved portion of the reference line. 10. Image distortion correction device. Image reading means for reading a book document image placed on the scanning surface such that the page binding section is substantially parallel to the main scanning direction; and an image for the left and right pages of the page binding section read by the image reading means. An image reading device comprising: the image distortion correction device according to claim 1, wherein the image reading device corrects a scanned image including: Image reading means for reading a book document image placed on a scanning surface such that a page binding portion is substantially parallel to the main scanning direction; and a page binding portion read by the image reading portion and two left and right images. 10. The image distortion correction device according to claim 1, wherein the image distortion correction device corrects a scanned image, and an image printing device that prints an image on paper based on image data corrected by the image distortion correction device. An image forming apparatus comprising: A scanned image including a page binding portion obtained by reading a book document image placed on a scanning surface and two left and right images is read in the main scanning direction so that the page binding portion is substantially parallel to the main scanning direction. A reference line extracting step of extracting a portion forming a curve extending in the sub-scanning direction included in the original document from the scanned image as a reference line, and extracting the reference image from one of the pages. A reference line supplementing step of generating the missing reference line based on a reference line of another page; and a main scanning direction distortion for correcting distortion in the main scanning direction of the scan image based on the shape of the extracted or generated reference line. And a correction step. The reference line extracting step is a line or main scan in which the position of each character of a page outline, a ruled line, a character line perpendicular to the main scanning direction located near the upper side and the lower side in the main scanning direction in the scanned image is connected. A pair of lines connecting the positions of the leading or trailing characters of the character line parallel to the direction is extracted as a pair of reference lines, and the main scanning direction distortion correction step is such that the curved portion of the reference line coincides with the extension of the straight line portion. 13. The image distortion correcting method according to claim 12, wherein a straight line is assumed, and a corresponding point in the image before and after the correction is determined based on the assumed straight line to correct the distortion in the main scanning direction. The reference line refilling step determines a page binding portion and a left and right page distortion region from the background density of the scanned image, and symmetrically moves a reference line around the page binding portion to obtain a width of the left and right page distortion region. 14. The image distortion correction method according to claim 12, wherein the symmetrically moved reference line is expanded and contracted in the sub-scanning direction in accordance with the following. In the reference line supplementing step, the brightest pixel value on each line in the main scanning direction is regarded as the background density of the line, and the position of the line having the highest background density in the sub-scanning direction is regarded as a page binding section. 15. The image distortion correction method according to claim 14, wherein an area where the background density is equal to or lower than a predetermined brightness is determined as a distortion area in the vicinity. In the reference line supplementing step, the brightest pixel value on each line in the main scanning direction is set as the background density of the line, the position of the line with the highest background density in the sub-scanning direction is set as the position of the page binding section, The position of the density value of the background density at the two predetermined positions in the direction and the position of the background density in the sub-scanning direction on the other page which is equal to this density value are detected and determined as a distortion area. The image distortion correction method according to claim 14, wherein: In the main scanning direction distortion correction step, if the reference line on the upper side or the lower side of the main scanning direction does not exist on both the left and right pages, and the position of the optical axis of the scanner lens in the main scanning direction is known, the sub-scanning passing through this position 17. The image distortion correction method according to claim 12, wherein a straight line parallel to the direction is regarded as a reference line on a lower side of the main scanning direction, and the distortion in the main scanning direction is corrected. In the main scanning direction distortion correction step, if the reference line on the upper side or lower side of the main scanning direction does not exist on both the left and right pages, and the position of the optical axis of the scanner lens in the main scanning direction is unknown, the main scanning direction of the scanned image is 17. The distortion in the main scanning direction is corrected by regarding a straight line parallel to the sub-scanning direction passing through the center position as a reference line on the upper side or the lower side in the main scanning direction. The described image distortion correction method. The main scanning direction distortion correction step holds information on the reference line extracted by the reference line extraction step or generated by the reference line supplementation step, and subsequently corrects the reference line in another scan image to be corrected. 19. The image distortion correcting method according to claim 12, wherein the distortion correction in the main scanning direction is performed using the information of (1). 20. The distortion according to claim 12, wherein the distortion correction in the main scanning direction performs the distortion correction in the main scanning direction only at a position in the sub-scanning direction corresponding to a curved portion of the reference line. Image distortion correction method. 21. An image distortion correction program, wherein the image distortion correction method according to claim 12 is programmed so as to be controllable by a computer. A recording medium, wherein the image distortion correction program according to claim 21 is recorded in a computer-readable format.

Images