JP2004199542A - Image inspection device, program, storage medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve the inspection precision in an image inspection device for inspecting the quality of a taken image by comparing the taken image with a reference image. <P>SOLUTION: A document area specification part 116 specifies the apex coordinates of a trapezoidal document part corresponding to a rectangular document in the taken image acquired by a camera head 102. An area cutout part 118 cuts out the trapezoidal document part in the taken image as the document image of an object of inspection in reference to the apex coordinates specified by the specification part 116. A geometric transformation part 212 perspectively transforms the shape of the reference image so as to have the same trapezoidal shape of the document part in the taken image on the basis of the apex coordinates of the document specified by the specification part 16. A blurring processing part 216 performs blurring correction processing according to a pixel position, on the basis of the apex coordinates of the document specified by the part 116, so that the blurring degree of the reference image is equal to that of the document part cut out by the cutout part 118. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

【００３２】
ここで、“ｎ”は、補正後のビット分解能で、８ビット分解能ならｎ＝８となり、“０〜２５５”階調の値をとる。ここでは、全画素について白基準データを保持する方法を述べたが、たとえば簡易的には、撮影された画像全体のピーク値を白基準ＤＷとする方法や、ラインごとのピーク値ＤＷ［ｊ］を白基準とする方法などが適用可能である。
【００３３】
光学歪補正部１１４は、撮影レンズ１０４に起因する収差を以下のようにして補正する。たとえば撮影レンズ１０４への入射角θでの収差をｄ、撮影レンズ１０４からＣＣＤ撮像素子１０６の結像面までの距離をｃ、結像面における結像位置の光軸からの距離をｒとすると、収差ｄは次式（２）で表される。そこで、光学歪補正部１１４は、この特性に基づいて歪曲収差を補正する。なお収差ｄは、一般にｒの３乗に比例するのでレンズ特性に基づいて比例定数を求めて歪曲収差を補正することもできる。
【数２】

【００３４】
原稿領域特定部１１６は、カメラヘッド１０２によって画像を撮像する都度、毎回撮像画像から原稿領域を抽出する。領域切出部１１８は、撮影の都度抽出された原稿領域情報を参照して、撮像画像中における原稿Ｐの部分を基準画像との間での比較対象の撮像画像範囲として切り出す。
【００３５】
原稿Ｐが排紙トレイ５８４に排出される場所や向きはある程度一定していると考え、切り出し部分を固定することも考えられるが、実際には、排紙トレイ５８４上の上下左右の位置がずれたり、傾いて排紙されたりする。この場合、切り出し部分を固定していると、原稿部分を正確に切り出すことができなくなる。原稿でない部分を比較対象の撮像画像範囲として切り出すと、その部分を欠陥として誤検出してしまう虞れがある。したがって、本実施形態のように、実際の原稿Ｐの部分を撮影の都度特定し、この特定した部分を切り出すようにすることで、原稿部分を精度よく切り出すことができるようになり、惹いては、精度のよい欠陥検出ができるようになる。
【００３６】
なお、カメラヘッド１０２により読み取られた撮像画像と検査用の基準画像との誤差をより小さくするための誤差補正を施す仕組みとして、たとえば本願出願人が特願２００１−３３４１６３号にて提案したような補正係数算出部および画像補正処理部を設けてもよい。なお、補正係数算出部や画像補正処理部の詳細は、特願２００１−３３４１６３号を参照するとよい。
【００３７】
たとえば、画像補正処理部は、カメラヘッド１０２により取得された撮像画像に対して、画像処理で一般的に用いられるエッジ強調フィルタなどを適用して強調処理（シャープネス処理）を施したり、撮像画像が有するノイズ成分を低減させるノイズ低減部の一例であって、平滑化フィルタなどを適用してモアレを除去したり、中間調データを平滑化することで撮像画像が有するノイズ成分を低減させるノイズ除去処理（平滑化処理）を施す。
【００３８】
補正係数算出部は、カメラヘッドにより取得された撮像画像のチルトスキャンに起因した暈け特性の場所依存性を補正するための暈け特性補正係数を算出する。たとえば、台形の原稿領域から強調処理やノイズ除去処理のライン単位の補正係数を算出し、この算出した補正係数を画像補正処理部にセットする。
【００３９】
画像補正処理部は、強調処理や平滑化処理を施す際には、それぞれのフィルタのパラメータを補正係数算出部により算出された補正係数で調整する。またこの際には、さらに好ましくは、ライン単位でシャープネスとノイズ除去の度合いを両方ともに変化させる、つまり、強調処理とノイズ除去処理をライン単位で同時に施すのが好ましい。
【００４０】
画像検査処理部２００は、本願出願人が特願２００１−３３４１６３号にて提案した構成と同様に、データ格納部３００に格納してある画像検査の基準となる基準画像（元画像データ）をカメラヘッド１０２にて取得された撮像画像と同じように解像度変換しかつ暈かし処理（平滑化処理）をすることで両画像を比較する際の誤差を少なくする基準画像処理部２１０と、基準画像処理部２１０から出力された基準画像に基づいて撮像画像の良否を検査する画像検査部２３０とを備える。
【００４１】
基準画像処理部２１０は、原稿領域特定部１１６により特定された原稿領域情報を参照して、データ格納部３００から読み出した基準画像を原稿領域と同一形状に変換（透視変換）する幾何変換部（透視変換部ともいう）２１２と、透視変換された画像に対して線形補間法などを用いて解像度変換処理を施す解像度変換部２１４と、解像度変換処理が施された基準画像に対して高周波成分を低減させる暈かし処理を施す暈かし処理部２１６とを有する。
【００４２】
本実施形態では、カメラヘッド１０２は、原稿Ｐを斜め上方から撮影するので、矩形である原稿Ｐは撮影された画像データ上では撮像画像エリアに対して台形状に変形している。幾何変換部２１２は、撮像画像の台形形状と一致するように矩形である基準画像を変換する。
【００４３】
暈かし処理部２１６は、解像度変換部２１４により解像度変換が施された基準画像に対して、撮像画像の暈け具合と同じになるように、ガウス関数の分散σに基づいて（つまりガウスフィルタを用いて）検査用の基準画像の鮮鋭度を調整する（一般的には低下させる）。要するに、撮像画像データのカメラヘッド１０２の光学系に起因する暈けを反映させる。この際には、原稿領域特定部１１６により特定された原稿領域の台形形状を参照してガウスフィルタ処理の補正係数を算出し、これを元に透視変換後の台形状の基準画像に暈かし処理（ガウスフィルタ処理）を施す。なお、分散σを適用したガウスフィルタ処理は、たとえば、特開平５−２２５３８８号などに示されているように、公知の技術であるので、その詳細な説明を割愛する。
【００４４】
図３は、撮像画像と基準画像との関係を示す図であって、カメラヘッド１０２により読み取られる画像の撮影状態を示す図（Ａ）、撮像画像を示す図（Ｂ）、および撮像画像を矩形状に変換した場合の画像を示す図（Ｃ）である。
【００４５】
本実施形態では、カメラヘッド１０２は、多数の矩形状のテストパッチが印刷された矩形状の原稿を任意のチルト角で撮影するので、カメラヘッド１０２により撮影された撮像画像は、図３（Ｂ）に示すように、台形画像に変形されて取得される。この台形画像を表す画像データは、一旦データ格納部３００に蓄えられる。
【００４６】
また、カメラヘッド１０２にて、図３（Ａ）に示すように、原稿Ｐの手前の縁（図中太線）に焦点があった状態で、チルトスキャンにより原稿Ｐを撮影する。この場合、撮影画像データのうち台形の原稿領域（図３（Ｂ）参照）では、原稿Ｐの下底（下辺）に焦点が合っているため、上底（上辺）に向かって焦点がしだいにずれていることになる。つまり画素位置に応じて画像の暈け量が異なる。
【００４７】
この影響を補正するため、幾何変換部２１２は、基準画像の形状を、撮像画像の台形形状に合致させる。このためには、たとえば、図３（Ｂ）に示すような撮像画像の台形形状を、図３（Ｃ）に示すような矩形状に戻す際に使用される変換式と逆特性の変換式を基準画像に適用して透視変換処理を施したり、撮像画像中の原稿領域を示す情報を参照して補正係数を求め、この補正係数を適用して透視変換処理を施したりしてもよい。
【００４８】
たとえば、撮像画像の台形形状を矩形状の戻す透視変換処理として、原稿Ｐからカメラまでの距離、カメラ高さｈ、パン角度、チルト角度に基づいて変換する方法が知られている。これは同次座標表現を用いた行列演算を解いて変換式を導出したものであり、変換式は以下の式（３），（４）で示される。
【数３】

【数４】

【００４９】
ここで、ｕ，ｖは変換後の座標、ｘ，ｙはソース画像の座標、ｈは原稿面からのカメラ高さ、α（＝tan^-1(d/h)）はカメラのチルト角、βはカメラのパン角である。予め測定したカメラ高さｈ，チルト角α，パン角βから透視変換の式に撮影画像の各画素を代入することで、台形の原稿領域を矩形に透視変換処理することができる。そして、この式（３），（４）の逆変換式（式の記載は割愛する）を求めて、基準画像の各画素を代入することで、矩形状の基準画像を、撮像画像の台形状に合致させることができる。
【００５０】
あるいは、カメラヘッド１０２により取得された撮像画像の原稿Ｐを示す領域の形状（原稿領域形状）に基づいて、斜め上方から撮影すること（チルトスキャン）に起因する撮像画像の幾何学的な歪み（幾何学歪み）を補正するための歪補正係数を求め、この求めた歪補正係数に基づいて、撮像画像の幾何学的な歪みを補正することもできる。たとえば、矩形である原稿Ｐは、図３（Ｂ）に示したように、チルトスキャンにより撮像された画像データ上では台形に変形している。原稿領域特定部１１６は、エッジ検出フィルタなどを用いて原稿画像のエッジ（辺縁）の全体を検出することで撮影画像から台形の原稿領域形状を示す座標点Ａ（左下頂点），Ｂ（右下頂点），Ｃ（右上頂点），Ｄ（右上頂点）を抽出する。幾何変換部２１２は、抽出された４つの頂点Ａ，Ｂ，Ｃ，Ｄの座標から透視変換処理に必要な補正係数（変換パラメータ）を算出して、この補正係数を適用して基準画像に対して透視変換処理を施すことで、基準画像を台形に透視変換する。
【００５１】
図４は、抽出された撮像画像の４つの頂点Ａ，Ｂ，Ｃ，Ｄの座標から基準画像に対する透視変換処理に必要な補正係数を算出する手法を説明する図である。なお、撮像画像の頂点座標に基づいて基準画像に対して透視変換処理を施す処理の詳細は、たとえば、“監修；高木幹雄・下田陽久、画像解析ハンドブック、p584〜p585、東京大学出版”にも示されている。
【００５２】
図４において、変換前の座標を（ｘ，ｙ）、変換後の座標を（Ｘ，Ｙ）、透視変換の補正係数（変換パラメータ）をａ，ｂ，ｃ，ｄ，ｅ，ｆ，ｐ，ｑとすると、変換後の各座標値Ｘ，Ｙは、式（５），（６）で求めることができる。
【数５】

【数６】

【００５３】
式（５），（６）を用いて元画像の座標（０，０），（ｘｍａｘ，０），（ｘｍａｘ，ｙｍａｘ），（０，ｙｍａｘ）をそれぞれ（ｘ，ｙ）に、撮像画像から抽出した座標Ａ（Ｘ０，Ｙ０），Ｂ（Ｘ１，Ｙ１），Ｃ（Ｘ２，Ｙ２），Ｄ（Ｘ３，Ｙ３）をそれぞれ（Ｘ，Ｙ）に代入し、変換パラメータａ，ｂ，ｃ，ｄ，ｅ，ｆ，ｐ，ｑをそれぞれ求めると、式（７）のようになる。
【数７】

【００５４】
基準画像に対する透視変換処理は、上述した何れの手法を用いる場合であっても、少ない画素点からより多くの画素点の画素値を求めるという補間処理を必要としない。場合によっては、単純な画素間引きとしてもよい。したがって、台形状の撮像画像を矩形状に戻す透視変換処理の場合に生じる、補間後のデータの信頼性が低くなるという問題は生じない。
【００５５】
図５は、基準画像に対する暈かし処理（暈け補正処理）の際の補正係数の算出処理を説明する図である。暈かし処理部２１６は、撮影画像データから座標データとして、座標値ｘ１，ｘ２と画像高さｃｈおよび画像幅ｄｗを抽出する。ここで、座標値ｘ２は、台形の原稿領域形状を示す座標点Ａの座標値、座標値ｘ１は、画像高さｃｈにおける座標点Ｂに対応する座標値である。
【００５６】
前述のように、カメラヘッド１０２によりチルトスキャンで撮影された画像を領域切出部１１８により切り出すと、画素位置に応じて下側から上側に向かって暈け量が大きくなる。ここで仮に暈け量の変化が奥行き方向に向かって１次関数で表わされるとすると、補正係数ｇｘを示す１次関数式ｆ１は、次式（８）で表わされる。この１次関数ｆ１を図で示すと図５（Ａ）のようになる。
【数８】

【００５７】
暈かし処理部２１６は、このようにして求められる補正係数ｇｘを適用して、領域切出部１１８により切り出された矩形原稿領域の画像に対し、ガウスフィルタを適用して暈かし処理を基準画像に施す。すなわち、暈かし処理部２１６は、原稿領域特定部１１６により特定された原稿の頂点座標に基づいて、基準画像の暈け具合が領域切出部１１８により切り出された原稿部分の暈け具合と同じになるように、画素位置に応じた暈け補正処理を施す。
【００５８】
上記から分かるように、幾何変換部２１２や暈かし処理部２１６における計算は非常に簡単である。なお、上記説明では、撮影するときに原稿Ｐの手前の縁で焦点を合わすよう設定したが、たとえば奥の縁や用紙の中央に焦点を合わせてもよい。この際には、上記と同じ手法で暈かし処理に使用する補正係数ｇｘを算出するが、補正係数ｇｘの関数は、たとえば奥に焦点を合わせた場合は、図５（Ｂ）のような関係となり、中央に合わせた場合は図５（Ｃ）のように算出すればよい。さらに上記説明では、補正係数ｇｘの関数を１次関数として説明したが、その他の次数の関数で示してもよい。たとえば図５で示す関数をもとに他の次数に変換することで、さまざまな関数を表現でき、より現実に即した処理が可能となる。
【００５９】
図６は、上記構成の画像検査装置５における処理手順を示すフローチャートである。上記実施形態の画像検査装置５においては、プリンタ部５００が待機状態にあるとき、カメラヘッド１０２は制御部３２０の指示により、排紙トレイ５８４上を定期的に撮影し、その画像を出力する。
【００６０】
制御部３２０は、基準画像の一例であるデータ格納部３００に格納してある排紙トレイ５８４の画像とカメラヘッド１０２により取得された撮像画像とを比較し、原稿が排紙トレイ５８４上にあるか否かを判断し、原稿が排紙トレイ５８４上にある場合には画像検査可能のフラグをオフにしてそのまま待機状態を続ける一方、排紙トレイ５８４上に原稿がない場合には画像検査可能のフラグをオンにして待機状態を続ける（Ｓ１００）。
【００６１】
そして、制御部３２０は、待機状態においてプリントジョブを受け取ると（Ｓ１０２）、画像検査可能のフラグを判定する（Ｓ１０４）。そして、画像検査可能のフラグがオフの場合（Ｓ１０４−ＮＯ）、制御部３２０は、プリントエンジン５７０に通常のプリント処理をさせ（Ｓ１０６）、待機状態へ戻る（Ｓ１８０）。一方、画像検査可能のフラグがオンの場合（Ｓ１０４−ＹＥＳ）、制御部３２０は、プリントジョブのＮ枚目（最初は当然に１枚目）について、画像検査を以下のように実行する。
【００６２】
すなわち、制御部３２０は、先ずＮ枚目（最初は当然に１枚目）のプリント処理をプリントエンジン５７０に開始させ（Ｓ１０８）、展開後の画像データを、プリントエンジン５７０のプリント出力処理部５７２へ送出するとともに、データ格納部３００に格納する。そして、最後の画像データのレンダリング処理が終了すると、プリント出力処理部５７２からの１枚目処理完了の信号を待ち（Ｓ１３０）、完了信号がくると規定の時間後にカメラヘッド１０２へ撮影開始トリガ信号を送る（Ｓ１３２）。
【００６３】
カメラヘッド１０２は、トリガ信号に同期してシャッタを切る（撮影する）が、このとき排紙トレイ５８４上に原稿Ｐ（プリント出力紙）が出力された直後となっている。カメラヘッド１０２は、矩形状の原稿を撮影して得た台形状の撮像画像をデータ格納部３００に格納する。
【００６４】
プリントジョブが複数枚（Ｎ枚）の場合、制御部３２０は、その複数枚（Ｎ枚）に対する処理が全て完了するまで後述する処理（Ｓ１３０〜Ｓ１６２）と並行して残りのプリント処理を進行させる（Ｓ１１０）。これらの処理は図では並列処理のように示しているが、時分割の逐次処理として実行することもできる。複数枚（Ｎ枚）が全て完了すると、制御部３２０は、待機状態へ戻る（Ｓ１１２）。このようにして、カメラヘッド１０２により撮像された全ての撮像画像は、デジタル信号に変換されてデータ格納部３００へ格納される。
【００６５】
撮像画像処理部１１０は、カメラヘッド１０２により撮影された撮像画像に対して前処理を施す。すなわち、シェーディング補正部１１２は、カメラヘッド１０２により撮影された撮像画像をデータ格納部３００から読み出し、照度むらやセンサ画素特性バラ付きを補正するシェーディング補正を行なう（Ｓ１３４）。また、光学歪補正部１１４は、このシェーディング補正が施された画像に対して、撮影レンズ１０４の歪曲収差などを補正する光学歪み補正を施し、処理済の画像データをデータ格納部３００に格納する（Ｓ１３６）。
【００６６】
また、原稿領域特定部１１６は、シェーディング補正や光学歪み補正が施された撮像画像について、背景の排紙トレイ１８４の色と原稿Ｐの縁の白色との差やエッジ検出などを利用して台形の原稿領域を特定する座標点Ａ，Ｂ，Ｃ，Ｄを抽出し、抽出した４点の座標値の情報を領域切出部１１８に通知するとともにデータ格納部３００に格納する（Ｓ１３８）。
【００６７】
次に、制御部３２０は、撮像画像処理部１１０によりシェーディング補正や光学歪み補正が施された撮像画像に対して透視変換（透視変換）処理を行なうかどうかを決定する（Ｓ１４０）。本願発明においては、撮像画像に対して透視変換処理を施さないで、基準画像に対して透視変換処理を施すので、撮像画像に対する透視変換処理（Ｓ１４２）をパスする（Ｓ１４０−ＮＯ）。
【００６８】
なお、撮像画像に対して透視変換処理を行なう場合（Ｓ１４０−ＹＥＳ）、制御部３２０は、図２では示さない撮像画像処理部１１０側に設けられた幾何変換部に対して透視変換処理を指示する。この指示を受けた幾何変換部は、シェーディング補正された撮像画像をデータ格納部３００から読み出し、この撮影画像からエッジ検出などによって台形の原稿領域を抽出し、台形の斜辺から透視変換処理前の補正係数を算出し、実際に台形の原稿領域を矩形に透視変換処理する（Ｓ１４２）。
【００６９】
領域切出部１１８は、カメラヘッド１０２にて撮像された撮像画像のうちの原稿領域を検査対象の撮像画像範囲として切り出す（Ｓ１４４）。本実施形態においては、領域切出部１１８は、原稿領域特定部１１６にて抽出された４頂点の座標値で囲まれる台形状の領域を検査対象の撮像画像範囲として切り出し、この切り出した台形状の原稿領域の画像を示す画像データをデータ格納部３００に格納する。
【００７０】
なお、撮像画像に対して透視変換処理を施す場合には、原稿領域特定部１１６は、この透視変換処理された矩形状に戻された原稿領域の座標点Ａ，Ｂ，Ｃ，Ｄの座標値の情報を領域切出部１１８に通知するとともにデータ格納部３００に格納する（Ｓ１３８）。領域切出部１１８は、カメラヘッド１０２にて撮像され透視変換処理が施された撮像画像のうち、原稿領域特定部１１６にて抽出された４頂点の座標値で囲まれる領域を検査対象の撮像画像範囲として切り出す（Ｓ１４４）。
【００７１】
次に、基準画像処理部２１０の幾何変換部２１２は、基準画像であるプリント処理された元画像を示す画像データと原稿領域特定部１１６により抽出された撮像画像の４頂点の座標値を示す情報とをデータ格納部３００から読み出す。そして、先ず、撮像画像の４頂点の座標値に基づいて透視変換処理に必要な補正係数を算出し、この算出した補正係数を適用して基準画像に対して撮像画像の台形形状に合致するように透視変換処理を施す（Ｓ１５０）。
【００７２】
次に、解像度変換部２１４は、台形状の基準画像のサイズと検査対象である台形状の撮像画像のサイズとが等しくなるように、基準画像の解像度を変換する（Ｓ１５２）。
【００７３】
次に、暈かし処理部２１６は、透視変換された基準画像に対して、撮像画像データにおけるカメラヘッド１０２の光学系に起因する暈けを反映させるために、原稿領域特定部１１６により抽出された台形を示す頂点座標Ａ，Ｂ，Ｃ，Ｄを参照して、ガウスフィルタ処理において使用する画像の台形形状に応じた補正係数の値ｇｘ（たとえば式（５）で示した１次関数ｆ１で示される値）を算出し、これを元に透視変換後の台形状の基準画像に暈かし処理（ガウスフィルタ処理）を施し、処理済みの画像をデータ格納部３００に格納する（Ｓ１５４）。
【００７４】
ここで、基準画像に暈かし処理を施すのは、基準画像は理想的な濃淡特性を持っているために、撮像画像の暈け特性と同等の暈かし処理を施すことで、撮影画像の暈け分布に対応して暈かし処理を施すことができ、次に説明する差分処理の際の誤差を最小にできるためである。
【００７５】
次に、位置合わせ処理部２３２は、基準画像処理部２１０の暈かし処理部２１６にて暈かし処理が施された基準画像と画像読取部１００にて前処理が施された撮像画像の位置合わせ処理を開始する（Ｓ１５６）。たとえば、位置合わせ処理部２３２は、基準画像をテンプレートとして撮像画像内を走査しながら、正規化相互相関係数を逐次計算し、最も相関係数の高い場所が位置の合った部分と見なす、いわゆるテンプレートマッチングで位置合わせを行なう。
【００７６】
次に、欠陥異常検出処理部２３４は、撮像画像処理部１１０にて前処理が施されデータ格納部３００に格納された検査対象の撮像画像（台形形状のもの）と、基準画像処理部２１０にて透視変換処理や暈かし処理がほど越された基準画像（（台形形状のもの）との差を求める（両者を比較する）差分処理を行なう（Ｓ１６０）。そして、欠陥異常検出処理部２３４は、差分処理により得られた差分結果に基づいて、撮像画像の欠陥や異常の有無を検出する（Ｓ１６２）。位置合わせ処理部２３２および欠陥異常検出処理部２３４は、以上の処理を基準画像の全面について行ない、完了後には再び待機状態へ戻る。
【００７７】
たとえば、欠陥異常検出処理部２３４は、位置が合った部分での正規化相互相関係数が判定指標値を下回る場合、その部分に画像欠陥が存在すると見なして欠陥異常検出処理を行なう。これにより、たとえば、プリント出力すなわち撮像画像に黒点などの欠陥パターンがあると、その欠陥を含む場所で正規化相互相関係数が明らかに低下するので、差分結果に黒点が残る。
【００７８】
よって、たとえば、適当な濃度の閾値を設定して差分結果を２値化することで黒点を特定することができ、画像欠陥の存在の有無を容易に判定することができる。なお、黒点などの局所的な画像欠陥の検出に限らず、撮像画像の全体的な濃度のむらや濃度シフトなどの画質不良の検出にも適用できる。
【００７９】
欠陥異常検出処理部２３４が画像欠陥や画質不良を検出した場合には、画像検査装置５は、所定の表示デバイスあるいは音声信号発生器（たとえばスピーカ）などの図示しないユーザインタフェース上で警告を発したり、プリント管理ソフトウェアなどを利用して、通信網を通してプリントしようとしているクライアント端末（たとえばパソコン）へ警告を発したり、さらにはリモート保守システムなどを介して通信網を通してプリンタの保守業者へ通知する。
【００８０】
以上説明したように、上記実施形態の画像検査装置５では、撮像画像を正面から見た画像に変換して（つまり矩形状の画像に戻して）元の画像である基準画像と比較するのではなく、撮像状態に起因する撮像画像の形状（一般的には台形状）に一致するように基準画像に対して透視変換処理を施して、ともに矩形状ではない（前例では台形状）の両画像を比較するようにした。先にも述べたように、基準画像に対する透視変換処理は、少ない画素点からより多くの画素点の画素値を求めるという補間処理を必要としないので、透視変換後のデータの信頼性が低くなるという問題は生じない。よって、より精度よく画像欠陥の有無を検査できるようになる。
【００８１】
なお、この思想に従えば、光学歪に対する処理も撮像画像を補正するのではなく、基準画像を歪ませて撮像画像の形状に一致させるべきであるが、透視変換による形状変化分に比べれば光学歪み補正による画像情報の変化量ははるかに少なく、欠陥検出精度への影響は極めて小さい。加えて、画像欠陥検査時において、光学歪みのある画像同士を比較するよりも、歪みのない画像同士を比較する方がはるかに簡便な処理になる。よって、上記実施形態で示したように、撮像画像に光学歪み補正を施した方が、画像欠陥検査システム全体の性能向上に繋がる。
【００８２】
また、撮像ごとに原稿領域の頂点座標を抽出して原稿領域を特定し、この特定した原稿領域の情報に基づき基準画像に対する透視変換処理の補正係数を算出して、この算出した補正係数を適用して基準画像に対して透視変換処理を施すようにすることで、実際の撮像状態に即した透視変換処理を基準画像に施すことができるので、原稿の排紙状態に関わらず、撮像画像と基準画像との比較・検査の際の精度が向上し、より検査精度の良好な画像欠陥や画質異常の検査ができるようになる。
【００８３】
加えて、特定した原稿領域の情報に基づき基準画像に対して暈かし処理を施すことで、チルトスキャンの際に生じる暈け度合いの場所による分布に応じた暈かし処理を実際の撮像状態に即して基準画像に対して施すことができ、撮像画像に近い暈け特性を持つ比較画像を生成することができる。
【００８４】
このように、撮像画像に対しては殆ど画像処理を加えることなく、基準画像側に透視変換処理や暈け補正処理を施すようにすれば、欠陥異常検出処理部２３４における差分処理の際の誤差が非常に少なくなり、一層正確な検査ができるようになる。
【００８５】
図７は、ＣＰＵやメモリを利用して、ソフトウェア的に画像検査装置５を構成する、すなわち電子計算機（コンピュータ）を用いて構成する場合のハードウェア構成の一例を示した図である。
【００８６】
画像検査装置５を構成するコンピュータシステム９００は、ＣＰＵ９０２、ＲＯＭ（Read Only Memory）９０４、ＲＡＭ９０６、および通信Ｉ／Ｆ（インターフェース）９０８を備える。ＲＡＭ９０６は、撮像画像データを格納する領域を含んでいる。
【００８７】
また、たとえばメモリ読出部９０７、ハードディスク装置９１４、フレキシブルディスク（ＦＤ）ドライブ９１６、あるいはＣＤ−ＲＯＭ（Compact Disk ROM）ドライブ９１８などの、記憶媒体からデータを読み出したり記録したりするための記録・読取装置を備えてもよい。画像データは、データバスを通じて各ハードウェア間をやり取りされる。
【００８８】
ハードディスク装置９１４、ＦＤドライブ９１６、あるいはＣＤ−ＲＯＭドライブ９１８は、たとえば、ＣＰＵ９０２にソフトウェア処理をさせるためのプログラムデータを登録するなどのために利用される。また、ハードディスク装置９１４は、処理対象画像データを格納する領域を含んでいる。
【００８９】
通信Ｉ／Ｆ９０８は、インターネットなどの通信網との間の通信データの受け渡しを仲介する。またコンピュータシステム９００は、カメラヘッド１０２との間のインターフェース（ＩＯＴコントローラ）の機能をなすカメラＩ／Ｆ部９３０と、プリントエンジン５７０との間のインターフェースの機能をなすＩＯＴコントローラ９３２とを備える。カメラＩ／Ｆ部９３０は、カメラヘッド１０２からの出力データをアナログデジタル変換してＲＡＭ９０６もしくはハードディスク装置９１４へ格納する。
【００９０】
なお、上記実施形態で示した撮像画像処理部１１０や画像画像検査処理部２００の各機能部分の全ての処理をソフトウェアで行なうのではなく、これら機能部分の一部をハードウェアにて処理回路９４０として設けてもよい。
【００９１】
このような構成のコンピュータシステム９００は、上記実施形態に示した画像検査装置５の基本的な構成および動作と同様とすることができる。また、上述した処理をコンピュータに実行させるプログラムは、ＣＤ−ＲＯＭ９２２などの記録媒体を通じて配布される。あるいは、プログラムは、ＣＤ−ＲＯＭ９２２ではなくＦＤ９２０に格納されてもよい。また、ＭＯドライブを設け、ＭＯに前記プログラムを格納してもよく、またフラッシュメモリなどの不揮発性の半導体メモリカード９２４など、その他の記録媒体に前記プログラムを格納してもよい。
【００９２】
さらに、他のサーバなどからインターネットなどの通信網を経由して前記プログラムをダウンロードして取得したり、あるいは更新したりしてもよい。なお、記録媒体としては、ＦＤ９２０やＣＤ−ＲＯＭ９２２などの他にも、ＤＶＤなどの光学記録媒体、ＭＤなどの磁気記録媒体、ＰＤなどの光磁気記録媒体、テープ媒体、磁気記録媒体、ＩＣカードやミニチュアーカードなどの半導体メモリを用いることができる。
【００９３】
記録媒体の一例としてのＦＤ９２０やＣＤ−ＲＯＭ９２２などには、上記実施形態で説明した画像検査装置５における処理の一部または全ての機能を格納することができる。したがって、以下のプログラムや当該プログラムを格納した記憶媒体を提供することができる。たとえば、基準画像処理部２１０や画像検査部２３０用のプログラム、すなわちＲＡＭ９０６などにインストールされるソフトウェアは、上記実施形態に示された基準画像処理部２１０や画像検査部２３０と同様に、幾何変換部２１２、解像度変換部２１４、暈かし処理部２１６、位置合わせ処理部２３２、あるいは欠陥異常検出処理部２３４などの各機能部をソフトウェアとして備える。
【００９４】
このようなソフトウェアは、画像処理ソフトウェア、あるいは画像検査用アプリケーションソフトなどとして、ＣＤ−ＲＯＭやＦＤなどの可搬型の記憶媒体に格納され、あるいはネットワークを介して配布されるとよい。
【００９５】
そして、画像検査装置５をコンピュータにより構成する場合、ＣＤ−ＲＯＭドライブ９１８は、ＣＤ−ＲＯＭ９２２からデータまたはプログラムを読み取ってＣＰＵ９０２に渡す。そしてソフトウェアはＣＤ−ＲＯＭ９２２からハードディスク装置９１４にインストールされる。ハードディスク装置９１４は、ＦＤドライブ９１６またはＣＤ−ＲＯＭドライブ９１８によって読み出されたデータまたはプログラムや、ＣＰＵ９０２がプログラムを実行することにより作成されたデータを記憶するとともに、記憶したデータまたはプログラムを読み取ってＣＰＵ９０２に渡す。
【００９６】
ハードディスク装置９１４に格納されたソフトウェアは、ＲＡＭ９０６に読み出された後にＣＰＵ９０２により実行される。たとえばＣＰＵ９０２は、記録媒体の一例であるＲＯＭ９０４およびＲＡＭ９０６に格納されたプログラムに基づいて上記の処理を実行することにより、撮像画像における原稿の部分を原稿領域特定部により特定し、特定された原稿の領域情報に基づいて、基準画像の形状が撮像画像における原稿部分の形状と略同一となるように透視変換したり、あるいは、原稿領域特定部により特定された原稿の部分を示す領域情報に基づいて暈かし処理のための補正係数を算出し、この算出した補正係数を適用して暈かし処理を施したりすることで、高精度な画像検査機能をソフトウェア的に実現することができる。これにより、撮像画像の欠陥検出精度を向上させることができる。
【００９７】
以上、本発明を実施形態を用いて説明したが、本発明の技術的範囲は上記実施形態に記載の範囲には限定されない。発明の要旨を逸脱しない範囲で上記実施形態に多様な変更または改良を加えることができ、そのような変更または改良を加えた形態も本発明の技術的範囲に含まれる。
【００９８】
また、上記の実施形態は、クレーム（請求項）にかかる発明を限定するものではなく、また実施形態の中で説明されている特徴の組合せの全てが発明の解決手段に必須であるとは限らない。前述した実施形態には種々の段階の発明が含まれており、開示される複数の構成要件における適宜の組合せにより種々の発明を抽出できる。実施形態に示される全構成要件から幾つかの構成要件が削除されても、効果が得られる限りにおいて、この幾つかの構成要件が削除された構成が発明として抽出され得る。
【００９９】
たとえば、上記実施形態では、矩形状の原稿を対象としたので、撮像画像における原稿部分を示す４つの頂点座標を領域情報として特定していたが、検査対象の原稿の形状は必ずしも矩形状のものに限定されない。たとえば、矩形以外の多角形のものであってもよいし、多角形に限らず、円形や楕円形などの曲線部分を有するものであってもかまわない。多角形の原稿の場合には、矩形の場合と同様に、頂点座標を領域情報として抽出するとよい。また、曲線部分を有する原稿の場合には、その曲線部分を確定する座標情報、たとえば曲率や円の中心座標と半径（楕円の場合には２つの中心座標と長径および短径）などを領域情報として抽出するとよい。
【０１００】
そして、原稿形状が何れの場合であっても、撮像画像における原稿の部分を特定する領域情報を抽出し、この抽出した領域情報に基づいて、基準画像に対して透視変換したり、暈け補正処理を施したりすればよい。
【０１０１】
また、上記実施形態では、基準画像に対して幾何変換処理を施した後に暈け補正処理を施していたが、逆、基準画像に対して暈け補正処理を施した後に幾何変換処理を施すようにしてもよい。
【０１０２】
【発明の効果】
以上のように、本発明によれば、撮像画像に対して透視変換処理や暈け補正処理を施すのではなく、先ず、撮像部により取得された撮像画像における原稿の部分を特定し、原稿に対応する基準画像に対して、特定された原稿の領域情報に基づいて、基準画像の形状が撮像画像における原稿部分の形状と略同一となるように透視変換したり、あるいは暈け補正処理を施したりするようにした。
【０１０３】
このため、被検査画像に含まれる情報を損なうことなく、より精度の高い画像検査を行なうことができるようになった。
【図面の簡単な説明】
【図１】本発明に係る画像検査装置の一実施形態を示す概略図である。
【図２】図１に示した画像検査装置の詳細（第１実施形態）を示すブロック図である。
【図３】撮像画像と基準画像との関係を示す図である。
【図４】抽出された撮像画像の頂点座標から基準画像に対する透視変換処理に必要な補正係数を算出する手法を説明する図である。
【図５】基準画像に対する暈かし処理における補正係数の算出処理を説明する図である。
【図６】画像検査装置における処理手順を示すフローチャートである。
【図７】電子計算機を用いて画像検査装置を構成する場合のハードウェア構成の一例を示した図である。
【符号の説明】
５…画像検査装置、１００…画像読取部、１０２…カメラヘッド、１０４…撮影レンズ、１０６…ＣＣＤ撮像素子、１１０…撮像画像処理部、１１２…シェーディング補正部、１１４…光学歪補正部、１１６…原稿領域特定部、１１８…領域切出部、２００…画像検査処理部、２１０…基準画像処理部、２１２…幾何変換部、２１４…解像度変換部、２１６…暈かし処理部、２３０…画像検査部、２３２…位置合わせ処理部、２３４…欠陥異常検出処理部、３００…データ格納部、３２０…制御部、５００…プリンタ部、５７０…プリントエンジン[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a device that reads information such as characters, figures, or images written on a medium such as paper, and compares the read captured image with a reference image to perform an image inspection, and an electronic computer (computer). The present invention relates to a program for performing an image inspection using a computer and a computer-readable storage medium storing the program.
[0002]
[Prior art]
[Patent Document 1]
JP-A-2000-22869
[0003]
2. Description of the Related Art An image reading apparatus for reading an image of a document, a so-called scanner apparatus, is used in various fields. For example, it may be used alone, or may be incorporated in a copying apparatus or the like. Alternatively, it may be used in an image inspection apparatus that inspects an image defect or the like by comparing a read image with a reference image for inspection.
[0004]
As an image reading apparatus, for example, a document is placed on a platen glass serving as a document placing table, and a line sensor scans the document under the document, and a user scans the image sensor by hand to scan the document. There is a handy image scanner for reading, or an indirect image scanner (also referred to as a non-contact image scanner) that indirectly reads a document with a certain distance between the document and an imaging device (sensor). The indirect image scanner has the advantage that the size and flatness of the object to be read are less restricted.
[0005]
As an example of the indirect image scanner, for example, Patent Document 1 proposes a method in which a print image is read from an obliquely upper side by an imaging device such as an area sensor and converted into an image viewed from the front. In the method of photographing obliquely from above, there is a problem that the image is distorted into a substantially trapezoidal shape (referred to as geometric distortion of the image), or a difference in the degree of blur of the image due to a shift in focus between the front and back of the document, that is, a blur. There is a problem that the characteristics depend on the place and the image quality is deteriorated.
[0006]
For this reason, in the invention described in Patent Document 1, in order to correct the location dependency of the geometric distortion and the blur characteristic in the captured image, the distance from the target pixel to be corrected to the photographing camera is measured, and the measurement result is obtained. , And a high-quality captured image is obtained by controlling a correction coefficient for correcting geometric distortion or blur according to the distance value.
[0007]
[Problems to be solved by the invention]
However, in the method described in Patent Literature 1, a predetermined calculation is performed to correct the location dependency of geometric distortion and blur characteristics. Lower.
[0008]
For example, when performing perspective conversion of an image read by an area sensor into an image viewed from the front, the number of pixels that need interpolation when converting to an image viewed from the front increases in a portion having a larger distortion, and the reliability of the data after interpolation is increased. Becomes lower. In addition, due to the interpolation, data which is not originally generated may be generated and may appear as a false signal.
[0009]
In addition, blur correction is performed on a captured image using a sharpening filter for a portion with a lot of blur due to a lens. However, this blur correction process allows only a component to be sharpened (emphasized). Unnecessary components are also emphasized and may become noise.
[0010]
Due to these problems, the method described in Patent Literature 1 may reduce the accuracy of image inspection.
[0011]
The present invention has been made in view of the above circumstances, and has as its object to provide an image inspection apparatus capable of inspecting a captured image with higher accuracy. Another object of the present invention is to provide a program suitable for realizing the image inspection apparatus as software using an electronic computer and a computer-readable storage medium storing the program.
[0012]
[Means for Solving the Problems]
That is, a first image inspection apparatus according to the present invention is an image inspection apparatus that inspects a captured image by comparing a captured image obtained by optically reading a document with a reference image for inspection, An image capturing unit that captures a captured image from obliquely above, a document area specifying unit that specifies a portion of the document in the captured image obtained by the image capturing unit, and a reference image that is an original image indicating the document. Based on the area information indicating the part of the document specified by the document area specifying unit, a geometric shape is set such that the shape of the reference image is substantially the same as the shape of the document part in the captured image obtained by the imaging unit. A geometric conversion unit that performs a conversion process (perspective conversion process); a reference image converted by the geometric conversion unit; and an image of a document portion specified by the document region specifying unit in the captured image acquired by the imaging unit. Comprising an image comparison section for comparing, the image inspection unit for inspecting an image of a document part based on the comparison result by the image comparison section.
[0013]
A second image inspection apparatus according to the present invention is a processing unit that performs a blurring process on a reference image in addition to the first configuration, and indicates a portion of a document specified by a document region specifying unit. The image processing apparatus includes a blur processing unit that calculates a correction coefficient for the blur processing based on the area information, and performs the blur processing by applying the calculated correction coefficient. Then, the image comparison unit converts the reference image converted by the geometric conversion unit and subjected to the blurring processing by the blurring processing unit, and the document specified by the document area specifying unit in the captured image acquired by the imaging unit. The comparison was made with the partial image.
[0014]
The dependent claims define further advantageous specific examples of the image inspection apparatus according to the present invention. Further, the program according to the present invention is suitable for realizing the image inspection apparatus according to the present invention by software using an electronic computer. The program may be provided by being stored in a computer-readable storage medium, or may be distributed via a wired or wireless communication unit.
[0015]
[Action]
In the first image inspection apparatus having the above-described configuration, instead of performing perspective transformation on the captured image acquired by the imaging unit to return the geometric shape of the captured image to the original document shape, first, The document area in the captured image obtained by the section is specified by the document area specifying section, and the geometric conversion section performs the reference image corresponding to the document on the basis of the specified document area information based on the specified area information of the document. The perspective transformation is performed so that the shape is substantially the same as the shape of the document portion in the captured image.
[0016]
Further, in the second image inspection apparatus having the above configuration, the blur correction processing is performed not on the captured image but on the reference image. At this time, the blur correction processing unit calculates a correction coefficient for the blur processing based on the area information indicating the portion of the document specified by the document area specifying unit, and applies the calculated correction coefficient. Blurring processing is applied.
[0017]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
[0018]
FIG. 1 is a schematic diagram showing an image inspection apparatus provided with an embodiment of the image processing apparatus according to the present invention. Here, FIG. 1A is a diagram illustrating an installation state of a camera head, and FIG. 1B is a diagram illustrating an example of a print engine.
[0019]
As shown in FIG. 1A, an image inspection device 5 includes an image reading unit 100, which is an example of an image reading device, and an image inspection processing unit that inspects an image read by the image reading unit 100 for a defect. 200, a data storage unit 300 that stores an image read by the image reading unit 100, a reference image used for inspection by the image inspection processing unit 200, or intermediate data during data processing, and an image read by the image reading unit 100. A printer unit 500 for outputting to a predetermined recording medium is provided.
[0020]
The printer section 500 has a discharge port 582 and a paper discharge tray 584, and a print engine is arranged in a housing. The paper discharge tray 584 having a function as a document mounting table is colored so that the document (printed paper) P and the background (here, the color of the paper discharge tray) can be distinguished.
[0021]
The image reading unit 100 has a camera head 102 that is an example of an imaging unit that reads an image of a document P. The camera head 102 includes a photographing lens 104, a CCD image sensor 106, and a read signal processing unit 108. The camera head 102 is supported by a support member (not shown) such that the direction of the photographing lens 104 with respect to P discharged onto the paper discharge tray 584 is adjustable. The camera is installed so as to be able to capture images from diagonally above to obtain captured images.
[0022]
The printer section 500 is operable with various functions of the image reading section 100, and includes a raster output scan (ROS) -based print engine 570 for operating the image inspection apparatus 5 as a digital printing system.
[0023]
The print engine 570 records an image on a predetermined recording medium using an electrophotographic process. The print engine 570 functions as a printing device (printer) that forms a visible image on a predetermined recording medium based on print data input from a client terminal.
[0024]
That is, as shown in FIG. 1B, the print engine 570 has the function of an interface (IOT controller) with the image reading unit 100, and also prints image data D 10 output from the image reading unit 100. A print output processing unit 572 that performs a predetermined process for print output, a laser light source 574 that emits a light beam, a laser drive unit 576 that controls or modulates the laser light source 574 according to data output from the print output processing unit 572, A polygon mirror (rotating polygon mirror) 578 for reflecting the light beam emitted from the laser light source 574 toward the photosensitive member 579.
[0025]
The print output processing unit 572 generates and renders (expands into raster data) the image data D10 according to a known technique, generating data representing a plurality of, preferably at least three, separation colors. For example, mapping is performed to at least three (preferably four) from the YCrCb color system represented by the data D10, for example, to the CMY color system or the CMYK color system, and color-separated raster data for print output is generated.
[0026]
The print output processing unit 572 also performs under color removal (UCR) for reducing the CMY components of the color image in such raster data conversion processing, or a gray component for partially replacing the reduced CMY components with K components. Exchange (GCR). Further, the print output processing unit 572 may perform linearization of color separation or similar processing in order to adjust a toner image of an output image created in response to output data (such as CMYK).
[0027]
With this configuration, the print engine 570 reflects the light beam generated by the laser light source 574 on a plurality of surfaces on the polygon mirror 578 to expose the photosensitive member 579, and forms a latent image on the photosensitive member 579 by scan scanning. Form. When the latent image is formed, the image is developed according to any of a number of methods known in the art, and the color image read by image reading unit 100 is output as a visible image. When a color image is output as a visible image, data representing the image includes at least three (preferably four) color separation data (for example, C, M, Y, K, etc.), and each color is separated. Or in luminance-chrominance format.
[0028]
FIG. 2 is a block diagram showing details of the image inspection apparatus 5 shown in FIG. The image inspection device 5 according to the present embodiment includes an image reading unit 100 that captures an image and acquires a captured image, an image inspection processing unit 200, a data storage unit 300, and a control unit 320 that controls each unit of the image inspection device 5. Is provided.
[0029]
The data storage unit 300 stores a captured image acquired by the camera head 102, a processed image processed by the captured image processing unit 110, various intermediate calculation results used for processing in image inspection, and the like.
[0030]
The image reading unit 100 includes a captured image processing unit 110 that performs a predetermined process on a captured image acquired by the camera head 102. The captured image processing unit 110 corrects shading by the shading correction unit 112 and the shading correction unit 112 that corrects uneven illuminance (light amount distribution characteristics of the optical system) and variation in sensor pixel characteristics of the captured image read by the camera head 102. And an optical distortion correction unit 114 for correcting distortion (curvature) aberration mainly caused by the photographing lens 104 for the captured image. Also, the captured image processing unit 110 refers to the document area specifying unit 116 that specifies the area of the document P in the captured image and the information of the document area specified by the document area specifying unit 116, and performs distortion by the optical distortion correction unit 114. An area cutout section 118 for cutting out a necessary portion (area of the document P) from the captured image in which the aberration has been corrected.
[0031]
The shading correction performed by the shading correction unit 112 is a process for correcting unevenness in illuminance of a photographing area, variation in characteristics of each pixel of the CCD image sensor 106, and the like. For example, the camera head 102 captures a blank sheet in advance and stores it in the data storage unit 300 as a white reference DW [i, j]. i and j represent pixel positions. Next, the shading correction unit 112 corrects the captured image D [i, j] based on Expression (1).
(Equation 1)

[0032]
Here, “n” is the bit resolution after correction. If the resolution is 8 bits, n = 8, and the value of the gradation is “0 to 255”. Here, the method of holding the white reference data for all pixels has been described. For example, in a simplified manner, the peak value of the entire captured image is set to the white reference DW, or the peak value DW [j] of each line is set. May be applied as a white reference.
[0033]
The optical distortion correction unit 114 corrects the aberration caused by the photographing lens 104 as follows. For example, assuming that the aberration at the incident angle θ to the photographing lens 104 is d, the distance from the photographing lens 104 to the image plane of the CCD image sensor 106 is c, and the distance from the optical axis of the image forming position on the image plane is r. , And aberration d are expressed by the following equation (2). Therefore, the optical distortion correction unit 114 corrects distortion based on this characteristic. Since the aberration d is generally proportional to the cube of r, the distortion can be corrected by obtaining a proportional constant based on the lens characteristics.
(Equation 2)

[0034]
The document area specifying unit 116 extracts a document area from a captured image every time an image is captured by the camera head 102. The region cutout unit 118 cuts out the portion of the document P in the captured image as a captured image range to be compared with the reference image with reference to the document region information extracted each time shooting is performed.
[0035]
It is considered that the place and the direction in which the document P is discharged to the discharge tray 584 are constant to some extent, and the cut-out portion may be fixed. However, in actuality, the upper, lower, left, and right positions on the discharge tray 584 are shifted. Or it is ejected at an angle. In this case, if the cutout portion is fixed, the original portion cannot be cut out accurately. If a non-document portion is cut out as a captured image range to be compared, the portion may be erroneously detected as a defect. Therefore, as in the present embodiment, by specifying an actual portion of the document P each time shooting is performed, and by cutting out the specified portion, the document portion can be cut out with high accuracy. Thus, highly accurate defect detection can be performed.
[0036]
As a mechanism for performing error correction for reducing an error between a captured image read by the camera head 102 and a reference image for inspection, for example, as proposed by the present applicant in Japanese Patent Application No. 2001-334163. A correction coefficient calculation unit and an image correction processing unit may be provided. For details of the correction coefficient calculation unit and the image correction processing unit, refer to Japanese Patent Application No. 2001-334163.
[0037]
For example, the image correction processing unit performs an enhancement process (sharpness process) on a captured image acquired by the camera head 102 by applying an edge enhancement filter or the like generally used in image processing, This is an example of a noise reduction unit that reduces a noise component included in the captured image by applying a smoothing filter or the like to remove moire or smoothing halftone data to reduce a noise component included in a captured image. (Smoothing process).
[0038]
The correction coefficient calculation unit calculates a blur characteristic correction coefficient for correcting the location dependency of the blur characteristic due to the tilt scan of the captured image acquired by the camera head. For example, a correction coefficient for each line of the emphasis processing or the noise removal processing is calculated from the trapezoidal document area, and the calculated correction coefficient is set in the image correction processing unit.
[0039]
The image correction processing unit adjusts the parameters of each filter with the correction coefficient calculated by the correction coefficient calculation unit when performing the enhancement processing and the smoothing processing. In this case, it is more preferable that both the sharpness and the degree of noise removal are changed in line units, that is, it is preferable that the enhancement process and the noise removal process are simultaneously performed in line units.
[0040]
The image inspection processing unit 200 converts the reference image (original image data) stored in the data storage unit 300, which is a reference for image inspection, into a camera similarly to the configuration proposed by the present applicant in Japanese Patent Application No. 2001-334163. A reference image processing unit 210 that converts the resolution in the same manner as the captured image acquired by the head 102 and performs a blurring process (smoothing process) to reduce errors when comparing the two images; An image inspection unit 230 that inspects the quality of the captured image based on the reference image output from the processing unit 210 is provided.
[0041]
The reference image processing unit 210 refers to the document area information specified by the document area specifying unit 116, and converts the reference image read from the data storage unit 300 into the same shape as the document area (perspective transformation). A resolution conversion unit 214 for performing a resolution conversion process on the perspective-transformed image using a linear interpolation method or the like, and a high-frequency component for a reference image subjected to the resolution conversion process. And a blur processing unit 216 for performing blur processing for reduction.
[0042]
In the present embodiment, since the camera head 102 captures the document P from obliquely above, the rectangular document P has a trapezoidal shape with respect to the captured image area on the captured image data. The geometric conversion unit 212 converts the rectangular reference image so as to match the trapezoidal shape of the captured image.
[0043]
The blurring processing unit 216 is based on the variance σ of the Gaussian function (that is, the Gaussian filter) so that the reference image subjected to the resolution conversion by the resolution conversion unit 214 has the same blurring degree as the captured image. (Using) to adjust (generally reduce) the sharpness of the reference image for inspection. In short, the blur caused by the optical system of the camera head 102 in the captured image data is reflected. In this case, the correction coefficient of the Gaussian filter process is calculated with reference to the trapezoidal shape of the document area specified by the document area specifying unit 116, and based on this, the trapezoidal reference image after perspective transformation is blurred. Processing (Gaussian filter processing) is performed. Note that the Gaussian filter processing to which the variance σ is applied is a known technique as shown in, for example, Japanese Patent Application Laid-Open No. 5-225388, and therefore a detailed description thereof is omitted.
[0044]
3A and 3B are diagrams showing the relationship between the captured image and the reference image, wherein FIG. 3A is a diagram illustrating a shooting state of an image read by the camera head 102, FIG. 3B is a diagram illustrating the captured image, and FIG. FIG. 7C is a diagram illustrating an image when converted into a shape.
[0045]
In the present embodiment, since the camera head 102 captures a rectangular document on which a large number of rectangular test patches are printed at an arbitrary tilt angle, the captured image captured by the camera head 102 is shown in FIG. As shown in ()), the image is acquired after being transformed into a trapezoidal image. The image data representing the trapezoidal image is temporarily stored in the data storage unit 300.
[0046]
Also, as shown in FIG. 3A, the document P is photographed by tilt scan with the camera head 102 focused on the front edge (the thick line in the figure) of the document P. In this case, in the trapezoid original area (see FIG. 3B) of the captured image data, the lower bottom (lower side) of the original P is focused, so that the focus gradually moves toward the upper bottom (upper side). It will be shifted. That is, the blur amount of the image differs depending on the pixel position.
[0047]
To correct this effect, the geometric conversion unit 212 matches the shape of the reference image with the trapezoidal shape of the captured image. For this purpose, for example, a conversion formula having an inverse characteristic to a conversion formula used when a trapezoidal shape of a captured image as shown in FIG. 3B is returned to a rectangular shape as shown in FIG. Perspective transformation processing may be applied to the reference image, or a correction coefficient may be obtained by referring to information indicating the document area in the captured image, and perspective transformation processing may be performed by applying this correction coefficient.
[0048]
For example, as a perspective conversion process for returning a trapezoidal shape of a captured image to a rectangular shape, a method of performing conversion based on a distance from a document P to a camera, a camera height h, a pan angle, and a tilt angle is known. This is obtained by solving a matrix operation using a homogeneous coordinate expression to derive a conversion formula. The conversion formula is represented by the following formulas (3) and (4).
[Equation 3]

(Equation 4)

[0049]
Here, u and v are the coordinates after conversion, x and y are the coordinates of the source image, h is the camera height from the document surface, and α (= tan ^-1 (d / h)) is the tilt angle of the camera, and β is the pan angle of the camera. By substituting each pixel of the captured image into a perspective transformation equation from the camera height h, tilt angle α, and pan angle β measured in advance, the trapezoid document area can be perspective-transformed into a rectangle. Then, a rectangular reference image is converted into a trapezoidal shape of the captured image by obtaining an inverse conversion expression of Expressions (3) and (4) (the description of the expression is omitted) and substituting each pixel of the reference image. Can be matched.
[0050]
Alternatively, based on the shape of the region indicating the document P of the captured image acquired by the camera head 102 (document region shape), geometric distortion of the captured image (tilt scan) caused by shooting from an obliquely upper side (tilt scan) It is also possible to obtain a distortion correction coefficient for correcting the (geometric distortion), and to correct the geometric distortion of the captured image based on the obtained distortion correction coefficient. For example, as shown in FIG. 3B, a rectangular original P is transformed into a trapezoid on image data captured by tilt scanning. The document area specifying unit 116 detects the entire edge (edge) of the document image using an edge detection filter or the like, and thereby detects the coordinate points A (lower left vertex) and B (right Lower vertex), C (upper right vertex), and D (upper right vertex) are extracted. The geometric transformation unit 212 calculates a correction coefficient (conversion parameter) necessary for the perspective transformation processing from the coordinates of the extracted four vertices A, B, C, and D, and applies the correction coefficient to the reference image. The reference image is perspective-transformed into a trapezoid by performing perspective transformation processing.
[0051]
FIG. 4 is a diagram illustrating a method of calculating a correction coefficient required for a perspective transformation process on a reference image from the coordinates of four vertices A, B, C, and D of the extracted captured image. The details of the process of performing the perspective transformation process on the reference image based on the vertex coordinates of the captured image are described in, for example, “Supervision: Mikio Takagi and Hirohisa Shimoda, Image Analysis Handbook, p584-p585, University of Tokyo Press”. It is shown.
[0052]
In FIG. 4, the coordinates before the transformation are (x, y), the coordinates after the transformation are (X, Y), and the correction coefficients (transformation parameters) of the perspective transformation are a, b, c, d, e, f, p, Assuming that q, the coordinate values X and Y after the conversion can be obtained by Expressions (5) and (6).
(Equation 5)

(Equation 6)

[0053]
Using the equations (5) and (6), the coordinates (0, 0), (xmax, 0), (xmax, ymax), (0, ymax) of the original image are converted to (x, y), respectively, from the captured image. The extracted coordinates A (X0, Y0), B (X1, Y1), C (X2, Y2), and D (X3, Y3) are substituted into (X, Y), respectively, and conversion parameters a, b, c, d , E, f, p, and q, respectively, are given by equation (7).
(Equation 7)

[0054]
The perspective transformation process on the reference image does not require the interpolation process of obtaining the pixel values of more pixel points from fewer pixel points, regardless of the method described above. In some cases, simple pixel thinning may be used. Therefore, there is no problem that the reliability of the interpolated data is lowered, which occurs in the case of the perspective transformation processing for returning the trapezoidal captured image to the rectangular shape.
[0055]
FIG. 5 is a diagram illustrating a process of calculating a correction coefficient in the blurring process (blurring correction process) for the reference image. The blurring processing unit 216 extracts coordinate values x1, x2, an image height ch, and an image width dw as coordinate data from the captured image data. Here, the coordinate value x2 is the coordinate value of the coordinate point A indicating the trapezoidal document area shape, and the coordinate value x1 is the coordinate value corresponding to the coordinate point B at the image height ch.
[0056]
As described above, when the image captured by the tilt scan by the camera head 102 is cut out by the area cutout unit 118, the blur amount increases from the lower side to the upper side according to the pixel position. Here, assuming that the change in the blur amount is represented by a linear function in the depth direction, a linear function expression f1 indicating the correction coefficient gx is represented by the following expression (8). FIG. 5A shows this linear function f1.
(Equation 8)

[0057]
The blurring processing unit 216 applies the Gaussian filter to the image of the rectangular document area cut out by the area cutout unit 118 by applying the correction coefficient gx obtained in this way, and performs the blurring processing. Apply to reference image. That is, based on the vertex coordinates of the document specified by the document area specifying unit 116, the blur processing unit 216 determines the degree of blur of the reference image based on the vertex coordinates of the document portion cut out by the area cutout unit 118. A blur correction process corresponding to the pixel position is performed so as to be the same.
[0058]
As can be seen from the above, the calculations in the geometric transformation unit 212 and the blurring processing unit 216 are very simple. In the above description, the focus is set on the front edge of the document P when the image is taken. However, the focus may be on the back edge or the center of the sheet. At this time, the correction coefficient gx used for the blurring processing is calculated by the same method as described above, and the function of the correction coefficient gx is, for example, as shown in FIG. The relationship is established, and when it is set at the center, it may be calculated as shown in FIG. Furthermore, in the above description, the function of the correction coefficient gx is described as a linear function, but may be represented by a function of another order. For example, by converting the function shown in FIG. 5 to another order, various functions can be expressed, and processing that is more realistic can be performed.
[0059]
FIG. 6 is a flowchart showing a processing procedure in the image inspection apparatus 5 having the above configuration. In the image inspection apparatus 5 of the above embodiment, when the printer unit 500 is in the standby state, the camera head 102 periodically captures an image on the discharge tray 584 and outputs the image according to an instruction from the control unit 320.
[0060]
The control unit 320 compares the image of the paper output tray 584 stored in the data storage unit 300, which is an example of the reference image, with the captured image obtained by the camera head 102, and the document is on the paper output tray 584. It is determined whether or not the document is on the discharge tray 584, the image inspection possible flag is turned off, and the standby state is continued. On the other hand, if no document is on the discharge tray 584, the image inspection is possible. The flag is turned on and the standby state is continued (S100).
[0061]
Then, when receiving a print job in the standby state (S102), the control unit 320 determines a flag indicating that image inspection is possible (S104). Then, when the image inspection possible flag is OFF (S104-NO), the control unit 320 causes the print engine 570 to perform normal print processing (S106), and returns to the standby state (S180). On the other hand, if the image inspection possible flag is ON (S104-YES), the control unit 320 executes the image inspection on the Nth print job (first, naturally, the first print job) as follows.
[0062]
That is, the control unit 320 first causes the print engine 570 to start print processing of the N-th sheet (first, naturally, the first sheet) (S108), and outputs the developed image data to the print output processing unit 572 of the print engine 570. And stores it in the data storage unit 300. When the rendering processing of the last image data is completed, a signal of completion of processing of the first sheet from the print output processing unit 572 is waited for (S130). Is sent (S132).
[0063]
The camera head 102 releases the shutter (photographs) in synchronization with the trigger signal, but at this time immediately after the document P (print output paper) is output onto the paper discharge tray 584. The camera head 102 stores a trapezoidal captured image obtained by photographing a rectangular document in the data storage unit 300.
[0064]
When the print job is a plurality of sheets (N sheets), the control unit 320 proceeds with the remaining print processing in parallel with the processing (S130 to S162) described later until the processing for all the plurality of sheets (N sheets) is completed. (S110). Although these processes are shown as parallel processes in the figure, they can be executed as time-division sequential processes. When the plurality of sheets (N sheets) are all completed, the control unit 320 returns to the standby state (S112). In this manner, all captured images captured by the camera head 102 are converted into digital signals and stored in the data storage unit 300.
[0065]
The captured image processing unit 110 performs pre-processing on the captured image captured by the camera head 102. That is, the shading correction unit 112 reads the captured image captured by the camera head 102 from the data storage unit 300, and performs shading correction for correcting uneven illuminance and variation in sensor pixel characteristics (S134). Further, the optical distortion correction unit 114 performs an optical distortion correction on the image subjected to the shading correction to correct the distortion of the photographing lens 104 and the like, and stores the processed image data in the data storage unit 300. (S136).
[0066]
In addition, the document area specifying unit 116 uses the difference between the color of the background discharge tray 184 and the white color of the edge of the document P, the edge detection, and the like for the captured image on which shading correction and optical distortion correction have been performed. The coordinate points A, B, C, and D specifying the original document area are extracted, and information on the extracted four coordinate values is notified to the area extracting unit 118 and stored in the data storage unit 300 (S138).
[0067]
Next, the control unit 320 determines whether or not to perform perspective transformation (perspective transformation) on the captured image on which shading correction and optical distortion correction have been performed by the captured image processing unit 110 (S140). In the present invention, since the perspective transformation process is performed on the reference image without performing the perspective transformation process on the captured image, the perspective transformation process (S142) on the captured image is passed (S140-NO).
[0068]
When performing the perspective transformation process on the captured image (S140-YES), the control unit 320 instructs the geometric transformation unit provided on the captured image processing unit 110 side not shown in FIG. I do. Upon receiving this instruction, the geometric conversion unit reads the captured image subjected to shading correction from the data storage unit 300, extracts a trapezoidal document area from the captured image by edge detection or the like, and corrects the trapezoidal hypotenuse from the oblique side before the perspective transformation process The coefficient is calculated, and the trapezoidal document area is actually perspective-transformed into a rectangle (S142).
[0069]
The area cutout unit 118 cuts out a document area in the captured image captured by the camera head 102 as a captured image range to be inspected (S144). In the present embodiment, the area cutout unit 118 cuts out a trapezoidal area surrounded by the coordinate values of the four vertices extracted by the document area specifying unit 116 as a captured image range to be inspected, and Is stored in the data storage unit 300.
[0070]
When performing the perspective transformation process on the captured image, the document area specifying unit 116 determines the coordinate values of the coordinate points A, B, C, and D of the document area that has been returned to the rectangular shape after the perspective transformation processing. Is notified to the area extracting unit 118 and stored in the data storage unit 300 (S138). The area cutout unit 118 captures an area surrounded by the coordinate values of the four vertices extracted by the document area identification unit 116 in the image captured by the camera head 102 and subjected to the perspective transformation processing. It is cut out as an image range (S144).
[0071]
Next, the geometric transformation unit 212 of the reference image processing unit 210 outputs image data indicating the original image subjected to the print processing, which is the reference image, and information indicating the coordinate values of the four vertexes of the captured image extracted by the document area identification unit 116. From the data storage unit 300. First, a correction coefficient required for the perspective transformation processing is calculated based on the coordinate values of the four vertices of the captured image, and the calculated correction coefficient is applied so that the reference image matches the trapezoidal shape of the captured image. Is subjected to perspective transformation processing (S150).
[0072]
Next, the resolution conversion unit 214 converts the resolution of the reference image so that the size of the trapezoidal reference image is equal to the size of the trapezoidal captured image to be inspected (S152).
[0073]
Next, the blur processing unit 216 is extracted by the document area specifying unit 116 in order to reflect the blur caused by the optical system of the camera head 102 in the captured image data on the perspective-transformed reference image. With reference to the vertex coordinates A, B, C, and D indicating the trapezoidal shape, a value gx of the correction coefficient corresponding to the trapezoidal shape of the image used in the Gaussian filter processing (for example, by the linear function f1 shown in Expression (5)) The calculated value is calculated, and the trapezoidal reference image after the perspective transformation is subjected to a blurring process (Gaussian filter process) based on the calculated value, and the processed image is stored in the data storage unit 300 (S154).
[0074]
Here, the blurring process is performed on the reference image because the reference image has an ideal density characteristic, and the blurring process equivalent to the blurring characteristic of the captured image is performed. This is because a blurring process can be performed in accordance with the blur distribution, and an error in the difference processing described below can be minimized.
[0075]
Next, the alignment processing unit 232 performs processing between the reference image subjected to the shading processing by the shading processing unit 216 of the reference image processing unit 210 and the captured image subjected to the preprocessing by the image reading unit 100. The positioning process is started (S156). For example, the alignment processing unit 232 sequentially calculates the normalized cross-correlation coefficient while scanning the captured image using the reference image as a template, and regards a position having the highest correlation coefficient as a portion where the position is matched, so-called Positioning is performed by template matching.
[0076]
Next, the defect abnormality detection processing unit 234 transmits the inspection image (trapezoidal shape) that has been pre-processed by the captured image processing unit 110 and stored in the data storage unit 300 to the reference image processing unit 210. In step S160, a difference process is performed to obtain a difference between the reference image (the trapezoidal shape) and the reference image (the one having a trapezoidal shape). Detects the presence or absence of a defect or abnormality in the captured image based on the difference result obtained by the difference processing (S162) .The alignment processing unit 232 and the defect / abnormality detection processing unit 234 perform the above processing on the reference image. The entire process is performed, and after completion, the process returns to the standby state again.
[0077]
For example, when the normalized cross-correlation coefficient at a position where the position is matched is lower than the determination index value, the defect abnormality detection processing unit 234 performs a defect abnormality detection process on the assumption that an image defect exists at that portion. Thus, for example, if there is a defect pattern such as a black spot in a print output, that is, a captured image, the normalized cross-correlation coefficient clearly decreases at a location including the defect, and a black spot remains in the difference result.
[0078]
Therefore, for example, by setting an appropriate density threshold and binarizing the difference result, a black point can be specified, and the presence or absence of an image defect can be easily determined. The present invention is not limited to the detection of local image defects such as black spots, and can be applied to the detection of image quality defects such as overall density unevenness and density shift of a captured image.
[0079]
When the defect abnormality detection processing unit 234 detects an image defect or a poor image quality, the image inspection device 5 issues a warning on a user interface (not shown) such as a predetermined display device or an audio signal generator (for example, a speaker). Using a print management software or the like, a warning is issued to a client terminal (for example, a personal computer) trying to print through a communication network, and further a notification is sent to a printer maintenance company through a communication network via a remote maintenance system or the like.
[0080]
As described above, in the image inspection apparatus 5 of the above embodiment, the captured image is converted into an image viewed from the front (that is, returned to a rectangular image) and compared with the reference image which is the original image. In addition, a perspective transformation process is performed on the reference image so as to match the shape (generally trapezoidal) of the captured image resulting from the imaging state, and both images are not rectangular (trapezoidal in the previous example). Was compared. As described above, since the perspective transformation process for the reference image does not require the interpolation process of finding the pixel values of more pixel points from fewer pixel points, the reliability of the data after the perspective transformation is reduced. The problem does not arise. Therefore, the presence or absence of an image defect can be inspected more accurately.
[0081]
According to this idea, the process for optical distortion should not correct the captured image, but should distort the reference image to match the shape of the captured image. The amount of change in image information due to distortion correction is much smaller, and the effect on defect detection accuracy is extremely small. In addition, at the time of image defect inspection, comparing images without distortion is a much simpler process than comparing images with optical distortion. Therefore, as described in the above embodiment, performing the optical distortion correction on the captured image leads to an improvement in the performance of the entire image defect inspection system.
[0082]
Also, the vertex coordinates of the document area are extracted for each image capture, the document area is specified, the correction coefficient of the perspective transformation process for the reference image is calculated based on the information of the specified document area, and the calculated correction coefficient is applied. By performing the perspective transformation process on the reference image, the perspective transformation process according to the actual imaging state can be performed on the reference image. Accuracy at the time of comparison / inspection with the reference image is improved, and inspection of image defects and abnormal image quality with higher inspection accuracy can be performed.
[0083]
In addition, by performing the blurring process on the reference image based on the information of the specified document area, the blurring process according to the distribution of the blurring degree occurring at the time of the tilt scan according to the actual imaging state is performed. Thus, a comparison image having a blur characteristic close to that of a captured image can be generated.
[0084]
As described above, if the perspective transformation processing and the blur correction processing are performed on the reference image side with almost no image processing performed on the captured image, the error in the difference processing in the defect abnormality detection processing unit 234 can be improved. Is very small, and a more accurate inspection can be performed.
[0085]
FIG. 7 is a diagram illustrating an example of a hardware configuration when the image inspection apparatus 5 is configured as software using a CPU and a memory, that is, when configured using an electronic computer (computer).
[0086]
The computer system 900 constituting the image inspection apparatus 5 includes a CPU 902, a ROM (Read Only Memory) 904, a RAM 906, and a communication I / F (interface) 908. The RAM 906 includes an area for storing captured image data.
[0087]
In addition, recording / reading for reading / recording data from / to a storage medium such as a memory reading unit 907, a hard disk device 914, a flexible disk (FD) drive 916, or a CD-ROM (Compact Disk ROM) drive 918. An apparatus may be provided. Image data is exchanged between the hardware via a data bus.
[0088]
The hard disk device 914, the FD drive 916, or the CD-ROM drive 918 is used, for example, for registering program data for causing the CPU 902 to perform software processing. Further, the hard disk device 914 includes an area for storing image data to be processed.
[0089]
The communication I / F 908 mediates the exchange of communication data with a communication network such as the Internet. Further, the computer system 900 includes a camera I / F unit 930 that functions as an interface (IOT controller) with the camera head 102, and an IOT controller 932 that functions as an interface with the print engine 570. The camera I / F unit 930 converts the output data from the camera head 102 from analog to digital and stores the data in the RAM 906 or the hard disk device 914.
[0090]
Note that not all processing of each functional part of the captured image processing unit 110 and the image image inspection processing unit 200 described in the above embodiment is performed by software, but a part of these functional parts is processed by a processing circuit 940 by hardware. May be provided.
[0091]
The computer system 900 having such a configuration can be the same as the basic configuration and operation of the image inspection device 5 described in the above embodiment. Further, a program for causing a computer to execute the above-described processing is distributed through a recording medium such as a CD-ROM 922. Alternatively, the program may be stored in FD 920 instead of CD-ROM 922. Further, an MO drive may be provided, and the program may be stored in the MO, or the program may be stored in another recording medium such as a nonvolatile semiconductor memory card 924 such as a flash memory.
[0092]
Further, the program may be downloaded from another server or the like via a communication network such as the Internet, acquired, or updated. As a recording medium, in addition to the FD 920 and the CD-ROM 922, an optical recording medium such as a DVD, a magnetic recording medium such as an MD, a magneto-optical recording medium such as a PD, a tape medium, a magnetic recording medium, an IC card, A semiconductor memory such as a miniature card can be used.
[0093]
The FD 920, the CD-ROM 922, or the like as an example of a recording medium can store some or all of the functions of the processing in the image inspection apparatus 5 described in the above embodiment. Therefore, it is possible to provide the following program and a storage medium storing the program. For example, the program for the reference image processing unit 210 and the image inspection unit 230, that is, the software installed in the RAM 906 and the like is the same as the reference image processing unit 210 and the image inspection unit 230 shown in the above-described embodiment. Each functional unit such as 212, resolution conversion unit 214, blurring processing unit 216, alignment processing unit 232, or defect abnormality detection processing unit 234 is provided as software.
[0094]
Such software may be stored in a portable storage medium such as a CD-ROM or FD as image processing software or application software for image inspection, or distributed via a network.
[0095]
When the image inspection apparatus 5 is configured by a computer, the CD-ROM drive 918 reads data or a program from the CD-ROM 922 and transfers the data or the program to the CPU 902. Then, the software is installed from the CD-ROM 922 to the hard disk device 914. The hard disk device 914 stores the data or the program read by the FD drive 916 or the CD-ROM drive 918, the data created by the CPU 902 executing the program, and reads the stored data or the program to read the CPU 902. Pass to.
[0096]
The software stored in the hard disk device 914 is read by the RAM 906 and executed by the CPU 902. For example, the CPU 902 performs the above-described processing based on programs stored in the ROM 904 and the RAM 906, which are examples of a recording medium, to specify a document portion in the captured image by the document region specifying unit, and Based on the area information, a perspective transformation is performed so that the shape of the reference image is substantially the same as the shape of the original part in the captured image, or based on the area information indicating the original part specified by the original area specifying unit. By calculating a correction coefficient for blurring processing and performing blurring processing by applying the calculated correction coefficient, a highly accurate image inspection function can be realized by software. Thus, the accuracy of detecting a defect in a captured image can be improved.
[0097]
As described above, the present invention has been described using the embodiment. However, the technical scope of the present invention is not limited to the scope described in the embodiment. Various changes or improvements can be made to the above-described embodiment without departing from the spirit of the invention, and embodiments with such changes or improvements are also included in the technical scope of the present invention.
[0098]
Further, the above embodiments do not limit the invention according to the claims (claims), and all combinations of the features described in the embodiments are not necessarily essential to the means for solving the invention. Absent. The embodiments described above include inventions at various stages, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent features. Even if some components are deleted from all the components shown in the embodiment, as long as the effect is obtained, a configuration from which some components are deleted can be extracted as an invention.
[0099]
For example, in the above-described embodiment, a rectangular document is targeted, and the coordinates of four vertices indicating the document portion in the captured image are specified as the area information. However, the shape of the document to be inspected is not necessarily rectangular. It is not limited to. For example, it may be a polygon other than a rectangle, or not limited to a polygon, but may have a curved portion such as a circle or an ellipse. In the case of a polygonal original, the vertex coordinates may be extracted as region information, as in the case of a rectangular original. In the case of a document having a curved portion, coordinate information for defining the curved portion, such as curvature and center coordinates and a radius of a circle (two center coordinates and a major axis and a minor axis in the case of an ellipse), are used as area information. Should be extracted as
[0100]
Regardless of the shape of the document, area information for specifying the portion of the document in the captured image is extracted, and based on the extracted area information, perspective conversion or blur correction is performed on the reference image. Processing may be performed.
[0101]
Further, in the above embodiment, the blur correction process is performed after the geometric conversion process is performed on the reference image, but the geometric conversion process is performed after the blur correction process is performed on the reference image. It may be.
[0102]
【The invention's effect】
As described above, according to the present invention, instead of performing perspective transformation processing or blur correction processing on a captured image, first, a document portion in the captured image acquired by the imaging unit is specified, and Based on the specified document area information, the corresponding reference image is subjected to perspective transformation or blur correction processing so that the shape of the reference image is substantially the same as the shape of the document portion in the captured image. I did it.
[0103]
For this reason, a more accurate image inspection can be performed without impairing the information included in the image to be inspected.
[Brief description of the drawings]
FIG. 1 is a schematic diagram showing an embodiment of an image inspection apparatus according to the present invention.
FIG. 2 is a block diagram showing details (first embodiment) of the image inspection apparatus shown in FIG. 1;
FIG. 3 is a diagram illustrating a relationship between a captured image and a reference image.
FIG. 4 is a diagram illustrating a method of calculating a correction coefficient required for a perspective transformation process on a reference image from vertex coordinates of an extracted captured image.
FIG. 5 is a diagram illustrating a process of calculating a correction coefficient in a blurring process for a reference image.
FIG. 6 is a flowchart illustrating a processing procedure in the image inspection apparatus.
FIG. 7 is a diagram illustrating an example of a hardware configuration when an image inspection apparatus is configured using an electronic computer.
[Explanation of symbols]
5: Image inspection apparatus, 100: Image reading unit, 102: Camera head, 104: Shooting lens, 106: CCD image sensor, 110: Captured image processing unit, 112: Shading correction unit, 114: Optical distortion correction unit, 116 ... Document region specifying unit, 118: region extraction unit, 200: image inspection processing unit, 210: reference image processing unit, 212: geometric conversion unit, 214: resolution conversion unit, 216: blurring processing unit, 230: image inspection Unit, 232: alignment processing unit, 234: defect / abnormality detection processing unit, 300: data storage unit, 320: control unit, 500: printer unit, 570: print engine

Claims (8)

An image inspection apparatus that inspects the captured image by comparing a captured image obtained by optically reading a document with a reference image for inspection,
An imaging unit which captures the original from obliquely above to obtain a captured image,
A document area specifying unit that specifies a portion of the document in the captured image acquired by the imaging unit;
For a reference image that is an original image indicating the document, the shape of the reference image is acquired by the imaging unit based on region information indicating a portion of the document specified by the document region specifying unit. A geometric conversion unit that performs a geometric conversion process so as to be substantially the same as the shape of the original portion in the captured image;
The reference image converted by the geometric conversion unit, and an image comparison unit that compares the image of the document portion specified by the document region specifying unit in the captured image obtained by the imaging unit,
An image inspection apparatus, comprising: an image inspection unit that inspects an image of the document portion based on a comparison result by the image comparison unit. An image inspection apparatus that inspects the captured image by comparing a captured image obtained by optically reading a document with a reference image for inspection,
An imaging unit which captures the original from obliquely above to obtain a captured image,
A document area specifying unit that specifies a portion of the document in the captured image acquired by the imaging unit;
For a reference image that is an original image indicating the document, the shape of the reference image is acquired by the imaging unit based on region information indicating a portion of the document specified by the document region specifying unit. A geometric conversion unit that performs a geometric conversion process so as to be substantially the same as the shape of the original portion in the captured image;
A processing unit for performing blurring processing on the reference image, wherein a correction coefficient for the blurring processing is calculated based on area information indicating a portion of the document specified by the document area specifying unit. And a blur processing unit that performs the blur processing by applying the calculated correction coefficient;
The reference image converted by the geometric conversion unit and subjected to the shading process by the shading processing unit, and the document specified by the document area specifying unit in the captured image acquired by the imaging unit An image comparison unit that compares the partial image with the image;
An image inspection apparatus, comprising: an image inspection unit that inspects an image of the document portion based on a comparison result by the image comparison unit. 3. The document area specifying unit according to claim 1, wherein the document area specifying unit specifies a portion of the document in the captured image acquired by the image capturing unit each time the image capturing unit captures the document. Image inspection equipment. The document area specifying unit extracts the coordinates of the vertices of the polygon as the area information indicating the polygonal part of the document,
The geometric conversion unit calculates a correction coefficient for the geometric conversion processing based on the coordinates of the vertices extracted by the document area specifying unit, and applies the calculated correction coefficient to the geometric conversion. The image inspection apparatus according to any one of claims 1 to 3, wherein a general conversion process is performed. A program for inspecting the captured image by comparing a captured image obtained by optically reading a document with a reference image for inspection,
Computer
A document area specifying unit that specifies a portion of the document in a captured image obtained by capturing an image;
For a reference image that is an original image indicating the document, the shape of the reference image is based on the area information indicating the portion of the document specified by the document area specifying unit. A geometric conversion unit that performs a geometric conversion process so as to be substantially the same as the shape;
An image comparing unit that compares the reference image converted by the geometric conversion unit and the image of the document portion specified by the document region specifying unit in the captured image;
A program that functions as an image inspection unit that inspects an image of the document portion based on a comparison result by the image comparison unit. A program for inspecting the captured image by comparing a captured image obtained by optically reading a document with a reference image for inspection,
Computer
A document area specifying unit that specifies a portion of the document in the captured image acquired by the imaging unit;
For a reference image that is an original image indicating the document, the shape of the reference image is acquired by the imaging unit based on region information indicating a portion of the document specified by the document region specifying unit. A geometric conversion unit that performs a geometric conversion process so as to be substantially the same as the shape of the original portion in the captured image;
A processing unit for performing blurring processing on the reference image, wherein a correction coefficient for the blurring processing is calculated based on area information indicating a portion of the document specified by the document area specifying unit. And a blur processing unit that performs the blur processing by applying the calculated correction coefficient;
The reference image converted by the geometric conversion unit and subjected to the shading process by the shading processing unit, and the document specified by the document area specifying unit in the captured image acquired by the imaging unit An image comparison unit that compares the partial image with the image;
A program that functions as an image inspection unit that inspects an image of the document portion based on a comparison result by the image comparison unit. A computer-readable storage medium storing a program for inspecting the captured image by comparing a captured image obtained by optically reading a document and a reference image for inspection,
The program causes the computer to:
A document area specifying unit that specifies a portion of the document in a captured image obtained by capturing an image;
For a reference image that is an original image indicating the document, the shape of the reference image is based on the area information indicating the portion of the document specified by the document area specifying unit. A geometric conversion unit that performs a geometric conversion process so as to be substantially the same as the shape;
An image comparing unit that compares the reference image converted by the geometric conversion unit and the image of the document portion specified by the document region specifying unit in the captured image;
A storage medium that functions as an image inspection unit that inspects an image of the document portion based on a comparison result by the image comparison unit. A computer-readable storage medium storing a program for inspecting the captured image by comparing a captured image obtained by optically reading a document and a reference image for inspection,
The program causes the computer to:
A document area specifying unit that specifies a portion of the document in a captured image obtained by capturing an image;
For a reference image that is an original image indicating the document, the shape of the reference image is based on the area information indicating the portion of the document specified by the document area specifying unit. A geometric conversion unit that performs a geometric conversion process so as to be substantially the same as the shape;
A processing unit for performing blurring processing on the reference image, wherein a correction coefficient for the blurring processing is calculated based on area information indicating a portion of the document specified by the document area specifying unit. And a blur processing unit that performs the blur processing by applying the calculated correction coefficient;
The reference image converted by the geometric conversion unit and subjected to the shading process by the shading processing unit, and the document specified by the document area specifying unit in the captured image acquired by the imaging unit An image comparison unit that compares the partial image with the image;
A storage medium that functions as an image inspection unit that inspects an image of the document portion based on a comparison result by the image comparison unit.

Images