JP4849757B2 - Self-calibrating multi-camera machine vision measurement system - Google Patents

Description

【００６２】
ＵＡ０、ＵＡ１およびＵＡ２は、ＣＳＡの単位ベクトル、すなわち、そのｘ軸、ｙ軸およびｚ軸である。ＣＳＷに対して、ＵＡ０の成分は以下のとおりである。
【００６３】
【数２】

【００６４】
ＵＡ１、ＵＡ２、ＵＢ０、ＵＢ１およびＵＢ２の成分は、同様に計算することができる。
【００６５】
ＰＡは、ＣＳＡの原点からＰへのベクトルである。ＣＳＡに対して、ＰＡの成分は以下のとおりである。
【００６６】
【数３】

【００６７】
ＰＢは、ＣＳＢの原点からＰへのベクトルである。ＣＳＢに対して、ＰＢの成分は以下のとおりである。
【００６８】
【数４】

【００６９】
したがって、一実施例においては、コンピュータ記憶装置を用いて、ＭＷＡの４×４行列の値を使用して、ＣＳＷに対するＣＳＡを完全に記述することができる。ＭＷＡの最初の３列の４ベクトルは、ＣＳＷに対するＣＳＡの単位３ベクトルであり、その第４の成分はゼロの値を有する。ＭＷＡの最後の４ベクトルは、ＣＳＡに対する、ＣＳＷの原点（中心）からＣＳＡの原点への３ベクトルであり、その第４の成分は１の値を有する。これらの４ベクトルおよび４×４行列を「同次座標（"homogeneous coordinates"）」と呼ぶ。
【００７０】
左上方の３×３行列は単にＣＳＡをＣＳＷに対応付ける回転行列（rotation matrix）であり、最右行は平行移動または並進（translation）である。
【００７１】
ＣＳＡに対する任意の点（すなわち、ＣＳＡにおけるその点の座標、これは、ＣＳＡの単位ベクトルに対する、ＣＳＡの原点からその点へのベクトルの成分である）が与えられると、行列ＭＷＡは、ＣＳＷにおける当該点の座標をどのように計算すべきかを示す。すなわち、ＰＡ（ＣＳＡに対する座標ベクトル）をＭＷＡで乗算することにより、ＰＷ（ＣＳＷに対する座標ベクトル）が得られる。
【００７２】
それぞれの値が行列の形で得られたので、行列数学（matrix mathematics）を使用することができる。特定的に、もしＰＷ＝ＭＷＡ*ＰＡであれば、ＰＡ＝ＭＷＡ^-1*ＰＷである。
【００７３】
上の定義により、４×４行列ＭＷＡ^-1は、ＣＳＡに対するＣＳＷを完全に特徴付けるかまたは記述する。上記はまた、ＰＡがＰＷで置換されかつＭＷＡがＭＷＡ^-1で置換された場合にも当てはまる。ＭＷＡ^-1を得るのに、以下のプロセスが用いられる。
【００７４】
１．左上方の３×３行列を転置（transpose）する。
２．最右列のベクトル（ＣＡｘ、ＣＡｙ、ＣＡｚ、１）、すなわち、ＣＳＷに対する、ＣＳＷの原点からＣＳＡへのベクトルを、（−ＣＡ０、−ＣＡ１、−ＣＡ２、１）、すなわち、ＣＳＡに対する、ＣＳＡの原点からＣＳＷの原点へのベクトル（後者は、ＣＳＷの原点からＣＳＡの原点へのベクトルＣＡとは反対方向である）に置き換える。
【００７５】
【数５】

【００７６】
上の３×３行列は、４×４行列ＭＷＡにおける左上方の３×３行列の転置であり、ＭＷＡ^-1の左上方の３×３行列位置に入るものである。したがって、以下のとおりとなる。
【００７７】
【数６】

【００７８】
表記を一致させる目的で、４×４行列ＭＷＡがＣＳＷに対するＣＳＡを完全に特徴付けるかまたは記述し、かつＭＷＡ^-1がＣＳＡに対するＣＳＷを完全に特徴付けるかまたは記述する場合には、ＭＷＡ^-1＝ＭＡＷである。
【００７９】
【数７】

【００８０】
４×４行列ＭＡＢは、ＣＳＡに対するＣＳＢを完全に特徴付けるかまたは記述する。したがって、ＭＡＢはＲＣＰまたはＲＴＰ行列である。
【００８１】
ある例示的なソフトウェア実現においては、VECTOR（ベクトル）構造は３つの数字のアレイとして定義され、MATRIX（行列）構造は３つのVECTORのアレイとして定義され、PLANE（平面）構造は１つのMATRIXおよび１つのVECTORである。MATRIXは３×３の回転行列であり、そのVECTORは座標系の３つの単位ベクトルであり、各VECTORはその座標系の原点へのベクトルである。これらすべてのVECTORの成分は、ベース座標系に対して表わされる。
【００８２】
ある例示的な関数においては、ＷＣＳに対して定義される平面１はＭＷＡであり、ＷＣＳに対して定義される平面２はＭＷＢであり、平面１に対して定義される平面２はＭＡＢであり、平面２に対して定義される平面１はＭＢＡである。そこで、ＡＰＩを有する関数が以下のように定義され得る。
【００８３】
【数８】

【００８４】
積４×４行列ＭＷＢの左上方の３×３は、ＭＷＡとＭＡＢの左上方における３×３行列値の積である。これは、すべての４×４行列の最下行における０値の結果である。積４×４行列ＭＷＢの最右列は、ＭＷＡの左上方３×３およびＭＡＢの最右列ベクトルの積と、ＭＷＡの最右列ベクトルとの和であり、これもまた、すべての４×４行列の最下行における０値および１値の結果である。
【００８５】
計算時間を短縮するために、一実施例においては、４×４行列の最下行について０または１による乗算は行なわれない。したがって、３×３行列および３ベクトルの乗算、加算および転置演算が行なわれる。
【００８６】
他の変換（transformation）についても、同様の関数が以下のように定義され得る。
【００８７】
【数９】

【００８８】
上記の説明から、ＭＷＡの４×４行列の値を使用して、ＣＳＷに対するＣＳＡを完全に記述できることがわかる。さらに、ＭＷＡの逆行列である行列ＭＷＡ^-1が、ＣＳＡに対するＣＳＷを完全に記述する。したがって、以下が成り立つ。
【００８９】
【数１０】

【００９０】
機械視覚分析技術を使用して、上述のシステムは、目標のカメラ画像、たとえば図３Ｂに示す画像９０を得て、そのカメラに対するその目標の座標系を計算することができる。
【００９１】
相対目標位置（ＲＴＰ）の計算について、図３Ａを参照して以下に説明する。ＲＴＰを計算する目的で、
ＣＳＬを左カメラ１０Ｌの座標系とする。
【００９２】
ＣＳＡを目標７２の座標系とする。
ＭＬＡはＣＳＬに対するＣＳＡを表わす。
【００９３】
ＣＳＢを目標７４の座標システムとする。
ＭＬＢはＣＳＬに対するＣＳＢを表わす。
【００９４】
左カメラ画像９０が目標７２および７４の画像９２を含むとすると、機械視覚分析技術により、行列ＭＬＡおよびＭＬＢが作成され記憶される。したがって、（目標７２と目標７４との間の）ＲＴＰ値は以下によって得られる。
【００９５】
【数１１】

【００９６】
これに基づいて、該システムは、ＭＬＢからＭＬＡをまたその反対を、以下によって計算することができる。
【００９７】
【数１２】

【００９８】
ＲＴＰの値が作成され記憶されると、目標アセンブリ７０を、左カメラ１０Ｌが目標７２を見、校正カメラ２０が目標７４を見るように動かすことができる。ＣＳＣが校正カメラ２０の座標系であるとする。左カメラ画像９０が目標７０の画像９４（図３Ｄ）を含むものとし、また、校正カメラ画像９６が目標７４の画像９８（図３Ｄ）を含むものとして、機械視覚分析技術により、ＣＳＬに対するＣＳＡを記述する行列ＭＬＡ、および、ＣＳＣに対するＣＳＢを記述するＭＣＢが作成されかつ記憶される。
【００９９】
これらの行列に基づいて、校正カメラ２０に対する左カメラ１０Ｌの相対カメラ位置ＲＣＰの値を、以下のように計算することができる。
【０１００】
【数１３】

【０１０１】
また、左カメラ１０Ｌに対する校正カメラ２０の相対カメラ位置ＲＣＰを、以下のように計算することができる。
【０１０２】
【数１４】

【０１０３】
次に、右カメラ１０Ｒに関する値の計算を提示する。図４Ａを参照して、
ＣＳＳがセットアップカメラ１００の座標系であるとする。
【０１０４】
ＣＳＲが右カメラ１０Ｒの座標系であるとする。
ＣＳＱが校正目標１６の座標系であるとする。
【０１０５】
ＣＳＤがデータ目標１０４の座標系であるとする。
図４Ｂのセットアップカメラ画像１０６が校正目標１６およびデータ目標１０４の画像１６′および１０４′を含むものとすると、機械視覚分析技術により、ＣＳＳに対するＣＳＱを記述する行列ＭＳＱと、ＣＳＳに対するＣＳＤを記述するＭＳＤとが作成され記憶される。さらに、右カメラ画像１０８がデータ目標１０４の画像１０４″を含むものとすると、機械視覚分析技術により、ＣＳＲに対するＣＳＤを記述する行列ＭＲＤが作成され記憶される。ＭＡＱは、校正目標１６とデータ目標１０４との間のＲＴＰ（ＣＳＱに対するＣＳＤ）を記述する。
【０１０６】
その後、右カメラに対する校正目標１６の座標系、すなわちＣＳＲに対するＣＳＱが、以下によって得られる。
【０１０７】
【数１５】

【０１０８】
したがって、左カメラに対する校正カメラを記述するＭＬＣの値、および、右カメラに対する校正目標を記述するＭＲＱの値を計算することができる。
【０１０９】
通常の動作において、システムは、図５Ａ、図５Ｂ、図５Ｃに示される種類の画像を生成する。左カメラ１０Ｌは、２つの左ホイール目標の画像１１０（図５Ａ）を生成する。機械視覚分析技術により、左カメラの座標系におけるホイール目標の座標系に関する値が作成され記憶される。左カメラの座標系がワールド座標系として規定される場合には、左ホイール目標がワールド座標系へと転置される。
【０１１０】
校正カメラ２０は、校正目標の画像１１２（図５Ｂ）を生成する。機械視覚分析技術により、校正カメラの座標系、ＭＣＱにおける、校正目標の座標系に関する値が作成され記憶される。
【０１１１】
右カメラ１０Ｌは、２つの右ホイールの目標について図５Ｃに示す画像１１４を生成する。機械視覚分析技術により、右カメラの座標系、ＭＲＷにおける、それらホイール目標の座標系に関する値が作成され記憶される。ここでの「Ｗ」は「ワールド」ではなく「ホイール」を意味する。通常、左カメラの座標系がワールド座標系として機能する。
【０１１２】
右ホイール目標の値は、ＭＬＷ、すなわち左（ワールド）座標系における右ホイール目標を計算することによって、左ホイール目標と同じワールド座標系へと転置することができる。校正プロセスから、ＭＬＣおよびＭＲＱの値はわかっている。該システムは、図５Ｂの画像に基づいてＭＣＱを測定し、図５Ｃの画像に基づいてＭＲＷを測定する。したがって、以下が成り立つ。
【０１１３】
【数１６】

【０１１４】
−−ハードウェアの概要
図７は、本発明の実施例を実現することが可能なコンピュータシステム７００を示すブロック図である。コンピュータシステム７００は、デバイス１００の一部もしくはすべてのための配列として、または、デバイス１００と通信する外部コンピュータもしくはワークステーションの配列のために、使用され得る。
【０１１５】
コンピュータシステム７００は、情報を通信するためのバス７０２または他の通信機構と、バス７０２に結合されて情報を処理するためのプロセッサ７０４とを含む。コンピュータシステム７００はまた、バス７０２に結合されてプロセッサ７０４によって実行されるべき命令および情報を記憶するための、ランダムアクセスメモリ（ＲＡＭ）または他の動的記憶デバイス等のメインメモリ７０６を含む。メインメモリ７０６は、プロセッサ７０４によって実行されるべき命令の実行中に、一時的な変数または他の中間情報を記憶するのにも使用され得る。コンピュータシステム７００はさらに、バス７０２に結合されてプロセッサ７０４のための命令および静的な情報を記憶するための、読出専用メモリ（ＲＯＭ）７０８または他の静的な記憶デバイスを含む。磁気ディスクまたは光ディスク等の記憶デバイス７１０が、情報および命令を記憶するために備えられ、バス７０２に結合される。
【０１１６】
コンピュータシステム７００は、バス７０２を介して、コンピュータユーザに対して情報を表示するための陰極線管（ＣＲＴ）等のディスプレイ７１２に結合され得る。英数その他のキーを含む入力デバイス７１４がバス７０２に結合されて、プロセッサ７０４に対して情報およびコマンド選択を通信する。ユーザ入力デバイスの別の種類として、マウス、トラックボールまたはカーソル指示キー等のカーソルコントロール７１６があるが、これは、プロセッサ７０４に対して指示情報およびコマンド選択を通信し、また、ディスプレイ７１２上のカーソルの動きを制御する。この入力デバイスは典型的に２つの軸、すなわち第１の軸（たとえばｘ）および第２の軸（たとえばｙ）で２度の自由度を有し、これにより該デバイスは、平面内の位置を特定することができる。
【０１１７】
本発明の複数の実施例は、アライナの自動校正のためのコンピュータシステム７００の使用に関連する。本発明の一実施例に従えば、アライナの自動校正は、プロセッサ７０４がメインメモリ７０６に含まれる１または複数の命令の１または複数のシーケンスを実行することに応答して、コンピュータシステム７００によって提供される。それらの命令は、記憶デバイス７１０等の別のコンピュータ可読媒体からメインメモリ７０６内に読込まれ得る。メインメモリ７０６に含まれる命令のシーケンスの実行により、プロセッサ７０４はここに記載されたプロセス工程を行なう。代替的な実施例においては、この発明を実現するソフトウェア命令に代えて、またはそれらのソフトウェア命令と組合せて、ハードワイヤード回路が使用され得る。したがって、本発明の実施例は、ハードウェア回路およびソフトウェアのどのような特定的な組合せにも限定されることはない。
【０１１８】
ここで使用される「コンピュータ可読媒体」という語は、命令を実行のためにプロセッサ７０４に提供することに関わるどのような媒体をも意味する。そのような媒体は、限定するものではないが、不揮発性媒体、揮発性媒体および伝送媒体を含む、多数の形をとることができる。不揮発性媒体は、たとえば、記憶デバイス７１０のような光または磁気ディスクを含む。揮発性媒体は、メインメモリ７０６のような動的メモリを含む。伝送媒体は、バス７０２を構成するワイヤを含む、同軸ケーブル、銅製ワイヤおよび光ファイバを含む。伝送媒体はまた、電波および赤外線データ通信中に生成されるような、音波または光波の形をとってもよい。
【０１１９】
コンピュータ可読媒体の一般的な形として、たとえば、フロッピー（Ｒ）ディスク、フレキシブルディスク、ハードディスク、磁気テープ、もしくは他の磁気媒体、ＣＤ−ＲＯＭ、その他の光学媒体、パンチカード、紙テープ、その他の穴のパターンを有する物理的媒体、ＲＡＭ、ＰＲＯＭおよびＥＰＲＯＭ、ＦＬＡＳＨ−ＥＰＲＯＭ、その他のメモリチップもしくはカートリッジ、以下に記載するような搬送波、および、コンピュータで読取り可能なその他の媒体、が挙げられる。
【０１２０】
コンピュータ可読媒体の種々の形は、プロセッサ７０４で実行されるように１または複数の命令の１または複数のシーケンスを搬送することに関連し得る。たとえば、それらの命令はまず、遠隔コンピュータの磁気ディスク上で搬送され得る。遠隔コンピュータは、それらの命令をその動的メモリへとロードし、モデムを使用して電話線を介して送信することができる。コンピュータシステム７００に対してローカルなモデムは、その電話線上のデータを受信して、赤外線トランスミッタを使用してそのデータを赤外線信号に変換することができる。赤外線検出器は、赤外線信号で搬送されたデータを受信することができ、適切な回路がそのデータをバス７０２上に置くことができる。バス７０２はそのデータをメインメモリ７０６へと搬送し、プロセッサ７０４はそこから命令を取出して実行する。メインメモリ７０６によって受取られた命令は、プロセッサ７０４によって実行される前にまたはその後に、記憶デバイス７１０に適宜記憶され得る。
【０１２１】
コンピュータシステム７００はまた、バス７０２に結合された通信インターフェイス７１８を含む。通信インターフェイス７１８は、ローカルネットワーク７２２に接続されたネットワークリンク７２０に結合して双方向データ通信を提供する。たとえば、通信インターフェイス７１８は、対応する種類の電話線へのデータ通信接続を提供する、総合デジタル通信網（ＩＳＤＮ）カードまたはモデムであってもよい。別の例として、通信インターフェイス７１８は、互換性のあるＬＡＮへのデータ通信接続を提供する、ローカルエリアネットワーク（ＬＡＮ）カードであってもよい。無線リンクが実現されてもよい。どのような実現においても、通信インターフェイス７１８は、種々の情報を表わすデジタルデータストリームを搬送する電気信号、電磁信号または光学信号を送受信する。
【０１２２】
ネットワークリンク７２０は典型的に、１または複数のネットワークを介した他のデータデバイスとのデータ通信を提供する。たとえば、ネットワークリンク７２０は、ローカルネットワーク７２２を介したホストコンピュータ７２４への、またはインターネットサービスプロバイダ（ＩＳＰ）７２６によって動作するデータ機器への接続を提供し得る。そしてＩＳＰ７２６は、今では一般に「インターネット」７２８と称されるワールドワイドパケットデータ通信ネットワークを介して、データ通信サービスを提供する。ローカルネットワーク７２２およびインターネット７２８はいずれも、デジタルデータストリームを搬送する電気信号、電磁信号または光学信号を使用する。種々のネットワークを通る信号および、デジタルデータをコンピュータシステム７００との間で搬送する通信インターフェイス７１８を通るネットワークリンク７２０上の信号は、情報を運搬する搬送波の例示的な形である。
【０１２３】
コンピュータシステム７００は、１または複数のネットワーク、ネットワークリンク７２０および通信インターフェイス７１８を介して、メッセージを送信しかつプログラムコードを含むデータを受信することができる。インターネットの例においては、サーバ７３０がインターネット７２８、ＩＳＰ７２６、ローカルネットワーク７２２および通信インターフェイス７１８を介して、アプリケーションプログラムに必要とされるコードを伝送し得る。本発明の実施例に従えば、そのようなダウンロードされたあるアプリケーションが、ここに記載したアライナの自動校正を可能にする。
【０１２４】
受取られたコードは、そのままの形で、プロセッサ７０４によって実行され、および／または、後に実行できるように記憶デバイス７１０もしくは他の不揮発性記憶装置に記憶され得る。このようにして、コンピュータシステム７００は、搬送波の形でアプリケーションコードを得ることができる。
【０１２５】
−−利点およびさらなる変形例
本書に開示された実施例は、他のコンテキストにも適用可能である。特に、これらの実施例は、２つ以上のカメラを有する機械視覚測定システムを校正するのに有益である。さらに、これらの実施例は、リクレーション用車両（ＲＶ）のアライメントに関連して使用することができる。ＲＶのためのアライナは、通常、標準的なアライナよりも幅の広いブームを必要とするが、本書に開示された実施例を使用すれば、ＲＶ用のアライナは、新たなハードウェアを必要とせずに、単にその直立材同士の幅をわずかに広げてボルト固定することによって構築することができる。
【０１２６】
上述の装置は自己校正するので、セットアップ後の校正は不要であり、したがって、携帯用アライナに組込むことが可能である。携帯用アライメント作業は、駐車場、ガレージまたは同様の環境において、時間のかかる校正を必要とせずに、三脚上の２つのカメラを使用して、行なうことができる。したがって、該装置を使用して、全く新しいサービス、遠隔アライメントまたは現場アライメントを容易にすることが可能である。
【０１２７】
さらに、上述の、右カメラに対する左カメラの相対位置を測定（または校正）する技術は、複数のデバイスを有するシステムにおいて用いることができる。これらの技術は、複数のデバイスのうち、１つのデバイスの別のデバイスに対する相対位置を測定するのに使用される。このような条件で、複数のデバイスのうち、第１のデバイスおよび第２のデバイスを含むいずれかのデバイスの対を、上述の技術における左カメラと右カメラの対として取扱うことが可能である。この場合、校正デバイスは第１のデバイスの近くに取付けられ、第１のデバイスに対する校正デバイスの相対位置は予め定められている。同様に、校正目標は第２のデバイスの近くに取付けられ、第２のデバイスに対する校正目標の相対位置は予め定められている。その後、校正目標に対する校正デバイスの相対位置が測定される。最後に、第２のデバイスに対する第１のデバイスの相対位置が、１）第１のデバイスに対する校正デバイスの相対位置と、２）第２のデバイスに対する校正目標の相対位置と、３）校正目標に対する校正デバイスの相対位置とに基づいて計算される。一実施例において、校正デバイスは、校正目標に対する校正デバイスの相対位置を測定するように構成される。
【０１２８】
上記明細書に本発明を特定的な実施例を参照して説明したが、本発明のより広範な精神および範囲から離れることなく種々の修正および変更をそれらに加えることができることは明らかであろう。したがって、本明細書および図面は、限定的な意味ではなく例示的な意味で捉えられるべきである。
【図面の簡単な説明】
【図１】 ３Ｄ自動車アライメントシステムの概略平面図である。
【図２】 アライメントシステムの直立要素の図である。
【図３Ａ】 アライメントカメラと校正カメラとの相対位置を測定し、校正するための方法における相対目標位置を測定するステップに用いられてもよい装置の図である。
【図３Ｂ】 カメラが見るビューの図である。
【図３Ｃ】 アライメントカメラと校正カメラとの相対位置を測定し、校正するための方法における相対カメラ位置を測定するステップに用いられてもよい装置の図である。
【図３Ｄ】 アライメントカメラと校正カメラとが見るビューの図である。
【図４Ａ】 アライメントカメラと校正目標との相対位置を測定し、校正するための方法において用いられてもよい装置の図である。
【図４Ｂ】 図４Ａの装置のセットアップカメラが見るビューの図である。
【図４Ｃ】 図４Ａの装置のアライメントカメラが見るビューの図である。
【図５Ａ】 図１の装置の第１のアライメントカメラが見るような、２つのホイール目標のビューの図である。
【図５Ｂ】 校正中に図１の装置の校正カメラが見るビューの図である。
【図５Ｃ】 図１の装置の第２のアライメントカメラが見るような、２つのホイール目標のビューの図である。
【図６Ａ】 ２つのカメラを有するカメラモジュールを校正するプロセスを示すフロー図である。
【図６Ｂ】 カメラと校正目標とを有するカメラモジュールを校正するプロセスを示すフロー図である。
【図６Ｃ】 アライメント中にカメラ校正を実行することを含むアライメントプロセスのフロー図である。
【図７】 一実施例が実現されるかもしれないコンピュータシステムのブロック図である。
【図８】 上述のシステムに用いられる数値のコンピュータベース数学計算の基礎を提供する、カメラおよび座標系の幾何学的関係の簡易図である。[0001]
FIELD OF THE INVENTION
The present invention relates generally to the calibration of machine vision measurement systems having two or more cameras, and more particularly to an apparatus and method for providing automatic self-calibration of a computer-aided three-dimensional aligner for automotive wheels.
[0002]
BACKGROUND OF THE INVENTION
Machine vision measurement systems with two or more cameras are used in many applications. For example, an automobile wheel may perform alignment on an alignment rack using a computer-aided three-dimensional (3D) machine vision alignment device and associated alignment methods. Examples of methods and apparatus useful for 3D alignment of vehicles are described in US Pat. No. 5,724,743 “Method and apparatus for determining the alignment of motor vehicle wheels”, And U.S. Pat. No. 5,535,522, “Method and apparatus for determining the alignment of motor vehicle wheels”. The devices described in these documents may be referred to as “3D aligners” or “aligners”.
[0003]
To determine the alignment of an automobile wheel, such a 3D aligner uses a camera that looks at a target attached to the wheel. These aligners generally require a calibration process to be performed after the aligner is first installed at the work site. In order to accurately determine the position of the wheel on one side of the vehicle and the wheel on the other side of the vehicle, the aligner must know where one camera is positioned relative to the other camera. According to one calibration method, a large target is positioned within the field of view of the camera, typically along the centerline of the alignment rack and away from the camera. The information obtained from each camera is then used to determine the relative position and orientation of the camera. Each camera shows where the target is relative to itself, and each sees the same target, so the system can calculate where each camera is located and oriented relative to the other . This is called relative camera position (RCP) calibration.
[0004]
Such calibration allows the results obtained from one side of the vehicle to be compared with others. Thus, by mounting the two cameras fixedly relative to each other and then performing an RCP calibration, the system can subsequently be used to position the wheel on one side of the vehicle relative to the other side of the vehicle. The RCP transfer function is used to transform the coordinate system of one camera to the coordinate system of another camera, so that a target viewed by one camera can be directly related to a target viewed by another camera. One approach to performing RCP is described in US Pat. No. 5,809,658 issued to Jackson et al. On September 22, 1998, “Method for Calibrating Cameras Used for Automobile Wheel Alignment. And Apparatus (Method and Apparatus for Calibrating Cameras Used in the Alignment of Motor Vehicle Wheels).
[0005]
RCP calibration is accurate but requires special fixtures and trained operators to perform. For this reason, there is a need for a simpler and simpler calibration process for aligners.
[0006]
In addition, the aligner may lose calibration over time, even after calibration has been performed. The aligner disclosed in the aforementioned document has a camera mounted on a boom designed to minimize calibration losses. However, the aligner loses calibration if the camera is shaken or removed, or if the boom itself bends. The aligner cannot detect the loss of its own calibration. Calibration loss is usually not detected unless a technician performs a calibration check or a full calibration. It may take a long time for the technician to notice that the alignment of the aligner is misaligned.
[0007]
Also, the boom is large and expensive and presents obstacles to the vehicle that enters and exits the alignment rack. A “drive-through” alignment approach may be used in which the vehicle advances into the maintenance facility, undergoes alignment, and then advances to exit the maintenance facility. This allows other cars to form a row behind the vehicle being maintained, improving the speed and efficiency of alignment maintenance. In an approach with drive-through alignment with a fixed boom, the camera boom needs to be lifted from the path as each vehicle passes. This is time consuming, expensive and can be awkward.
[0008]
Based on the foregoing, there is a clear need in the art for an apparatus and method that provides automatic self-calibration of a machine vision measurement system having two or more cameras.
[0009]
There is also a need for an aligner that may be installed at an alignment facility without calibration at the installation site, thereby eliminating the need for extra hardware and trained operators.
[0010]
There is also a need for an aligner that can automatically recalibrate itself if the camera is shaken or removed, or if the boom is bent.
[0011]
There is also a need for an aligner that can be quickly recalibrated if the technician determines that the aligner was measuring incorrectly, or if the technician suspects that the relative position of the aligner's camera has changed. .
[0012]
It would also be advantageous to have a 3D aligner that does not require a fixed mounting boom for operation and thus allows drive-through alignment without having to raise the beam and camera.
[0013]
SUMMARY OF THE INVENTION
The foregoing needs and objectives, as well as other needs that will become apparent from the following description, are realized by embodiments of the present invention, which in one aspect includes an apparatus for calibrating a mechanical measurement system. In one embodiment, a machine measurement system having a first camera and a second camera includes a first calibration target attached in a predetermined relationship to the first camera of the machine vision measurement system, and a machine measurement. And a third camera attached to the second camera of the system in a predetermined relationship. The calibration target is seen from the third camera. The data processor is configured to calculate a relative camera position value of the mechanical measurement system based on the relative position of the first calibration target with respect to the third camera, the relative camera position value being the second camera position value with respect to the second camera. 1 represents the relative position of one camera. This calibration can often be performed each time the first and second cameras measure an object such as a wheel target.
[0014]
The present invention is illustrated in the figures of the accompanying drawings for purposes of illustration and not limitation. The same reference numbers refer to similar elements.
[0015]
Detailed Description of the Preferred Embodiment
A method and apparatus for automatic calibration of a machine vision measurement system having two or more cameras is described. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the present invention.
[0016]
-Structural overview
FIG. 1 is a schematic plan view of certain elements of a computer-aided 3D automotive wheel alignment system (aligner) that typically includes a left camera module 2 and a right camera module 4 that are used to align the vehicle wheel. Such an aligner is an example of a machine vision measurement system having two or more cameras, but the invention is not limited to the situation of an automotive aligner, which can be applied to any machine vision measurement system having two or more cameras. Or equally applicable to any mechanical measurement system having two or more measurement devices. Also, the terms “left” and “right” are used for convenience and are not intended to require that a particular element be placed in a particular location or in a particular relationship with respect to another element. Any element described as being a “left” element may be placed in a “right” location, and vice versa.
[0017]
Arrow 30 schematically represents the automobile undergoing alignment. The vehicle includes a left front wheel 22L and a right front wheel 22R, and a left rear wheel 24L and a right rear wheel 24R. The alignment targets 80a, 80b, 80c, and 80d are fixed to the wheels 22L, 22R, 24L, and 24R, respectively. Each alignment target generally includes a plate 82 having target information imprinted thereon and a clamping mechanism 88 for securing the target to the wheel.
[0018]
The left camera module 2 includes a left alignment camera 10L and a calibration camera 20. The left alignment camera 10L faces the vehicle and sees the left targets 80a, 80b along the axis 42. The left alignment camera 10L is disclosed in US Pat. No. 5,724,743 “Method and apparatus for determining the alignment of motor vehicle wheels” and US Pat. No. 5,535. , No. 522, “Method and apparatus for determining the alignment of motor vehicle wheels”, may serve as one of the alignment cameras in the aligner. The camera 10L is fixedly attached to the left rigid mounting base 12.
[0019]
The calibration camera 20 faces the right camera module 4 and sees the calibration target 16 along the axis 46. The calibration camera 20 is also fixedly attached to the mount 12. In one embodiment, shaft 42 and shaft 46 are at an angle of about 90 °. However, this particular angular relationship is not required or required.
[0020]
In this exemplary embodiment, the calibration camera 20 is illustrated as forming part of the left camera module 2. However, the calibration camera 20 may also be configured as part of the right camera module 4, in which case the view is directed to the left to the left camera module 2.
[0021]
The right camera module 4 includes a right camera 10R that faces the vehicle and functions as a second alignment camera in the 3D alignment system. The right camera 10R is attached to a rigid camera mount 14. The calibration target 16 is fixedly attached to the camera mount 14 at a position that can be seen from the calibration camera 20 along the axis 46.
[0022]
The calibration camera 20 and the left camera 10L are fixed at known positions determined in advance. Similarly, the right camera 10R and the calibration target 16 are fixed at predetermined positions. For this reason, the relative position of the calibration camera with respect to the left camera 10L is known, and the relative position of the right camera 10R with respect to the calibration target 16 is also known. The relative position of the two cameras included in the left camera module can be obtained by using precision camera mounting hardware. Another approach is to factory calibrate the positions of the two cameras and save them for later use.
[0023]
The attachment of the left camera 10L and the calibration camera 20 to the left mounting base 12 needs to be robust so as not to cause a calibration error that may occur when the camera moves relative to the mounting base. Similarly, the attachment of the right camera 10R and the calibration target 16 to the mounting base 14 also needs to be robust.
[0024]
In order to illuminate the calibration target 16 and the wheel targets 80a-80d, the left camera module 2 and the right camera module 4 may further include light sources 62, 64, 66. In one embodiment, the first light source 62 is aligned perpendicular to the axis 46 and directs light along that axis to illuminate the calibration target 16. The second light source 64 is aligned perpendicular to the axis 42 and directs light along that axis to illuminate the left wheel targets 80a, 80b. The third light source 66 is aligned perpendicular to the axis 44 and directs light along that axis to illuminate the right wheel targets 80c, 80d. In one embodiment, each of the light sources 62, 64, 66 includes a circuit board or other substrate having a plurality of light emitting diodes (LEDs) mounted thereon and facing the illumination direction. However, any other light source may be used.
[0025]
FIG. 2 is a diagram of an alternative embodiment in which the alignment system includes a left upright 52 and a right upright 54. Each upright 52, 54 may include a rigid post attached to an alignment rack or maintenance facility floor. The left alignment camera 10L and the calibration camera 20 are mounted in a left upright 52 that functions as a protective housing and a rigid mounting base. The camera may view the vehicle being aligned and the alignment target 16 through a suitable hole or window in the upright 52. The right alignment camera 10R may be mounted and enclosed within the right upright 54, and the camera 10R may view the vehicle through a suitable hole or window in the right upright 54.
[0026]
The calibration target 16 may be attached to the external surface of the upright 54 at a position visible from the calibration camera 20. Alternatively, the calibration target 16 may be mounted within the upright 54 and viewed by the calibration camera 20 through a suitable hole or window in the upright 54.
[0027]
The light sources 62, 64, 66 may be attached to the outer surface of the uprights 52, 54.
[0028]
-Outline of calibration of the first camera module (first and third cameras)
Its components for each of the camera modules or pods (one pod with the first and third cameras and the second pod with the second camera and calibration target) before the aligner is available The relative position of must be determined.
[0029]
If the rigid mount 12 is manufactured with high tolerances (eg, 0.01 ″ and 0.01 °), the relative position of the left camera 10L and the calibration camera 20 is known and the relative of the two cameras There is no need to calibrate the positions, their relative positions will be known and the same will be true for all assemblies, but as a way to reduce costs, the relative position for each of the cameras or targets in the pod is It may be calibrated or measured.
[0030]
FIG. 6A is a flow diagram illustrating the process of calibrating the first camera module of a machine vision measurement system having three or more cameras.
[0031]
In general, in one embodiment, for calibration of the left camera module 10L, two targets fixedly attached to each other are placed in the field of view of one of the cameras. The camera may be any one of the three cameras, or may be another camera for the purpose of facilitating setup during manufacturing. The computer calculates the relative position (RTP) of the two targets. The targets are then moved so that the first camera sees one target and the third camera sees the second target. A measurement of the target position is calculated. Based on the RTP and the measured target position, the positions of the first camera and the third camera are calculated.
[0032]
A sub-process for calibrating the relative positions of the alignment camera and the calibration camera will be described below with reference to FIGS. 3A, 3B, 3C, 3D, and 6A.
[0033]
At block 602, a left camera module having a first camera and a calibration camera is set up. At block 604, multiple targets are set up in the left camera view. For example, FIG. 3A is a diagram illustrating an apparatus that may be used in a method for measuring and calibrating the relative positions of the left alignment camera 10L and the calibration camera 20. The target assembly 70 includes two targets 72, 74 that are fixed to the frame 76. The target assembly is positioned along the axis 42 in the field of view of the camera 10L so that both targets 72, 74 are visible from the left alignment camera 10L. With this arrangement, the camera 10L sees the targets 72 and 74 in the configuration shown in FIG. 3B. An image 90 is generated by the camera 10L, which includes target images 92 for targets 72,74.
[0034]
At block 606, a relative target position value is calculated. For example, using known machine vision techniques, a data processor programmed according to appropriate software can receive image 90 and measure the location of each target 72, 74 based on target image 92. . The data processor can then calculate a relative target position (RTP) value for each of the targets 72,74.
[0035]
At block 608, multiple targets are set up such that one target is visible in the field of view of the first camera and the other target is visible from the calibration camera. For example, referring to FIG. 3C, the target assembly is moved so that targets 72 and 74 are visible by left alignment camera 10L and calibration camera 20, respectively. The target frame may be manually moved by a calibration technician or may be moved by a motorized device. At this position, the left alignment camera 10L forms an image similar to the image 90 shown in FIG. 3D. Image 90 includes a target image 94 representing the view of target 72 viewed by camera 10L. As shown in block 610, the current target position value is calculated. For example, the position value of each target 72, 74 for each camera is calculated using machine vision image analysis techniques.
[0036]
Thereafter, as shown in block 612, the position of the first camera relative to the calibration camera is calculated. For example, a value representing the relative camera position of the left alignment camera 10L with respect to the calibration camera 20 (“RCP left module value”) is calculated based on the RTP value and the target position value.
[0037]
--Outline of calibration of the second camera module (second camera and calibration target)
A process for calibrating a pod or module that includes a camera and a target, such as the right camera module 10R, is described below with reference to the flowchart of FIG. 6B. As indicated by block 613, the right camera module 10R is first set up. The right camera module 10R may be manufactured with tight tolerances, but a calibration approach involving relative position measurements may be used to reduce costs. Generally, as shown in FIGS. 4A, 4B, and 4C, the data target is placed in the field of view of the second camera. Another camera ("setup camera") is placed in a position to see both the calibration target and the data target. The setup camera works with the computer to measure two target RTPs. The second camera measures the position of the data target, and the computer determines the position of the calibration target relative to the second camera based on the RTP value.
[0038]
FIG. 4A shows an apparatus that can be used in a method for measuring and thus calibrating the position of the right camera 10R relative to the calibration target 16. FIG. This device can be used when the relative position of the right camera and the calibration target is not known in advance. In one embodiment, the apparatus of FIG. 4A is made as part of an aligner factory calibration process.
[0039]
As shown in block 614, a setup camera and additional goals (“data goals”) may be placed in place. The data target 104 is positioned in front of the right camera module 4 and is made visible by the right alignment camera 10R. An additional camera, the setup camera 100, is positioned to the side of the right camera module 4 so that both the data target 104 and the calibration target 16 can be viewed.
[0040]
As shown in block 616, a relative target position value is calculated based on the position of the data target and the calibration target using the view of the setup camera. For example, FIG. 4B is a diagram illustrating the view 106 viewed by the setup camera 100 in the above configuration. View 106 includes a first image 16 ′ of a calibration target and a second image 104 ′ of data target 104. Using this view as an input to the machine vision processing system, the positions of the data target 104 and the calibration target 16 are measured using the setup camera 100. From these measured values, a value for the relative target position between the data target 104 and the calibration target 16 (“RTP setup value”) is obtained.
[0041]
At block 617, the relative position of the data target relative to the second camera is obtained. As shown in block 618, a relative camera target position value is calculated based on the relative target position setup value and the relative position of the data target with respect to the second camera.
[0042]
For example, FIG. 4C is a diagram illustrating the view 108 viewed by the right alignment camera 10R in the configuration described above. View 108 includes a second image 104 "of data target 104. Using view 108 as an input to the machine vision processing system, the position of data target 104 relative to right alignment camera 10R is measured. Right alignment camera Using the value representing the relative position of the data target 104 relative to 10R and the RTP setup value, a relative camera target position value (RCTP) is calculated, which is the relationship of the right alignment camera 10R to the right calibration target 16. Represent.
[0043]
At this point, the relative position of the left alignment camera 10L and the calibration camera 20 is now known in the form of an RCP left module value. Furthermore, the relative position of the right alignment camera 10R with respect to the calibration target 16 is also known in the form of an RCTP value. Since the left alignment camera 10L is fixedly attached to the calibration camera 20, and the right alignment camera 10R is fixedly attached to the calibration target 16, their relative positions change. Absent. In one embodiment, the above process is typically performed at the manufacturer's site where the aligner system is manufactured. The aligner system is therefore calibrated at the manufacturer's site, as shown in block 620.
[0044]
-Use of a system calibrated at the manufacturer's site
Alignment can be performed with a system that has been calibrated at the manufacturer's site. As shown in FIG. 1, camera modules 2 and 4 are located in front of the vehicle to be aligned. The left camera module 2 is oriented so that the left alignment camera 10L can see the left side of the vehicle and the calibration camera 20 can see the calibration target 10 of the right camera module 4. The right camera module 4 is positioned such that the right alignment camera 10R looks at the right side of the vehicle and the calibration target 16 is visible from the calibration camera 20. This is as shown in FIG.
[0045]
FIG. 5A is a diagram showing the view 110 seen from the left alignment camera 10L in this configuration while the alignment operation is being performed. View 110 includes images of alignment targets 80a, 80b on the left wheel of the vehicle being aligned.
[0046]
FIG. 5B shows a view 112 viewed by the calibration camera 20 in this configuration. View 112 includes an image 16 ″ of calibration target 16.
[0047]
FIG. 5C is a diagram illustrating the view 114 viewed by the right alignment camera 10R. View 114 includes images of alignment targets 80c, 80d on the right wheel of the vehicle being aligned.
[0048]
FIG. 6C is a flow diagram illustrating a process for performing camera calibration during an automobile alignment operation. This is done in the workplace in one embodiment. At block 629, an aligner having a first camera, a second camera, a calibration camera and a calibration target is set up as described above. At block 630, an automobile wheel alignment operation or process is initiated. Block 630 may include moving the vehicle to an alignment rack, attaching a wheel target to the vehicle wheel, initializing the aligner, and capturing the wheel target in view with the aligner's camera.
[0049]
At block 632, the calibration camera measures the position and orientation of the calibration target relative to the calibration camera. For example, the calibration camera 20 may measure the position and orientation of the calibration target 16 relative to the calibration camera when the aligner is installed and periodically during use or vehicle alignment.
[0050]
At block 634, RCP and RCTP values are typically obtained from memory. In one embodiment, these values are calculated and stored in memory as described above. As shown in block 636, based on these values (RCP left module value, RCTP right module value, and calibration target position), a value representing the relative position between the left alignment camera 10L and the right alignment camera 10R is calculated. These values are referred to as the aligner's relative camera position (RCP). The aligner can then view the vehicle forward and proceed with vehicle alignment measurements, as shown at block 638.
[0051]
The calibration process can be performed in a “background” mode or background processor while the computer performs other normal functions in alignment.
[0052]
The calculation of the RCP value may be performed at any time and may be performed before, during or after the vehicle alignment measurement. For example, the calculation of the RCP value may be performed as appropriate several times per second, once a day, at the beginning or end of a work day, to provide accurate alignment.
[0053]
-Modification
In an alternative embodiment, the relative positions of the calibration camera 20 and the left alignment camera 10L of the left camera module 2 and the camera-to-target position of the right camera module 4 are determined before the aligner is used in the field environment or service shop. The above-described apparatus and process can be used without measuring at. In this alternative, standard field calibration is performed using the RCP procedure or equivalent process described in the above patent document for calculating the RCP of the first and second cameras. Thereafter, the calibration camera 20 measures the position of the calibration target 16, and the calibration camera 20 periodically looks at the calibration target 16 and measures its relative position. If the measured value indicates a change in the relative position between the calibration camera 20 and the target 16, the left camera module 2 has moved relative to the right camera module 4. The change value can be used to recalculate and update the aligner's RCP value.
[0054]
In another alternative embodiment, to further simplify the process, the relative position of the calibration camera 20 and the calibration target 16 is measured after the aligner's RCP value has been calculated. This measured value is periodically compared with the initial measured value of the relative position of the calibration camera 20 with respect to the calibration target 16 performed at the time of aligner setting. If these two measurements differ by more than a predetermined tolerance, the aligner informs the operator that the aligner is no longer calibrated. In response, the operator or service technician can recalibrate using, for example, the RCP method.
[0055]
In another alternative embodiment, the aligner is provided with more than two alignment camera modules. The apparatus includes an additional calibration camera and calibration target for each additional alignment camera module. Each additional alignment camera module is calibrated with additional processing steps to calibrate the additional module according to the process described above. If each camera in each additional module is fixedly attached to its associated calibration target, the entire device can be automatically calibrated.
[0056]
In another embodiment, the calibration camera and the calibration target are mounted on different measurement modules. This configuration can be used with a non-contact aligner that uses one or more laser systems to determine if the wheels are aligned.
[0057]
Further, the process described herein may be used in embodiments that use elements other than cameras that perform the functions of the calibration camera 20. In some embodiments, a video camera is not required, but may be used, and any suitable image capture device or any conventional measurement device may be used. For example, a gravity gauge or string gauge may be arranged to detect movement of one or more alignment cameras 10L, 10R relative to each other or relative to a fixed point. Alternatively, the LED light source may be attached to one camera module so that the light beam is applied to a detector attached to the opposite camera module. The detector determines the point with the maximum light intensity on the detector surface, but if the point moves over time, it is determined that the camera has moved and the RCP value is updated, or , A flag is set indicating that the system is not calibrated.
[0058]
--- Computer-based mathematical calculations
FIG. 8 is a simplified diagram illustrating the geometric relationship between a camera and a coordinate system that provides the basis for numerical computer-based mathematical calculations used in the system described above.
[0059]
In FIG. 8, CSA (coordinate system A) specifies the first three-dimensional coordinate system related to the first alignment camera. The CSB specifies a second coordinate system associated with the second alignment camera. The CSW specifies the left wheel coordinate system used for reference purposes. CA is a vector from the origin of CSW to the origin of CSA. CB is a vector from the origin of CSW to the origin of CSB. P is a point in space.
[0060]
PW is a vector from the origin of CSW to P. The components of PW with respect to CSW are as follows. Here, · indicates the calculation of the dot product.
[0061]
[Expression 1]

[0062]
UA0, UA1, and UA2 are CSA unit vectors, that is, their x-axis, y-axis, and z-axis. The components of UA0 with respect to CSW are as follows.
[0063]
[Expression 2]

[0064]
The components of UA1, UA2, UB0, UB1 and UB2 can be calculated similarly.
[0065]
PA is a vector from the origin of CSA to P. For CSA, the components of PA are as follows:
[0066]
[Equation 3]

[0067]
PB is a vector from the origin of CSB to P. The components of PB with respect to CSB are as follows.
[0068]
[Expression 4]

[0069]
Thus, in one embodiment, a computer storage device can be used to completely describe a CSA for a CSW using the values of a 4 × 4 matrix of MWA. The first 3 columns of 4 vectors of the MWA are the CSA unit 3 vectors for the CSW, whose fourth component has a value of zero. The last four vectors of the MWA are the three vectors for the CSA from the CSW origin (center) to the CSA origin, whose fourth component has a value of one. These four vectors and a 4 × 4 matrix are called “homogeneous coordinates”.
[0070]
The upper left 3 × 3 matrix is simply a rotation matrix that maps CSA to CSW, and the rightmost row is translation or translation.
[0071]
Given an arbitrary point relative to the CSA (ie, the coordinates of that point in the CSA, which is the component of the vector from the CSA origin to that point relative to the unit vector of the CSA), the matrix MWA is Indicates how the point coordinates should be calculated. That is, PW (coordinate vector for CSW) is obtained by multiplying PA (coordinate vector for CSA) by MWA.
[0072]
Since each value was obtained in the form of a matrix, matrix mathematics can be used. Specifically, if PW = MWA * PA, PA = MWA ^-1 * PW.
[0073]
4 × 4 matrix MWA as defined above ^-1 Fully characterizes or describes the CSW for the CSA. The above also shows that PA is replaced with PW and MWA is MWA ^-1 This also applies if it is replaced with. MWA ^-1 The following process is used to obtain:
[0074]
1. Transpose the upper left 3x3 matrix.
2. The rightmost column vector (CAx, CAy, CAz, 1), that is, the CSW origin to CSA vector for CSW, (-CA0, -CA1, -CA2, 1), ie, for CSA, the CSA's Replace with the vector from the origin to the origin of the CSW (the latter is in the opposite direction to the vector CA from the origin of the CSW to the origin of the CSA).
[0075]
[Equation 5]

[0076]
The upper 3 × 3 matrix is the transpose of the upper left 3 × 3 matrix in the 4 × 4 matrix MWA, and the MWA ^-1 Is located in the 3 × 3 matrix position at the upper left. Therefore, it becomes as follows.
[0077]
[Formula 6]

[0078]
For purposes of matching notation, a 4 × 4 matrix MWA fully characterizes or describes the CSA for the CSW, and the MWA ^-1 MWA if it fully characterizes or describes the CSW for the CSA ^-1 = MAW.
[0079]
[Expression 7]

[0080]
The 4 × 4 matrix MAB fully characterizes or describes the CSB for the CSA. Thus, MAB is an RCP or RTP matrix.
[0081]
In one exemplary software implementation, a VECTOR structure is defined as an array of three numbers, a MATRIX structure is defined as an array of three VECTORs, and a PLANE structure is one MATRIX and one There are two VECTOR. MATRIX is a 3 × 3 rotation matrix, its VECTOR is three unit vectors of the coordinate system, and each VECTOR is a vector to the origin of the coordinate system. All these VECTOR components are expressed relative to the base coordinate system.
[0082]
In one exemplary function, plane 1 defined for WCS is MWA, plane 2 defined for WCS is MWB, and plane 2 defined for plane 1 is MAB. , Plane 1 defined relative to plane 2 is an MBA. Thus, a function with an API can be defined as follows:
[0083]
[Equation 8]

[0084]
The upper left 3 × 3 of the product 4 × 4 matrix MWB is the product of the 3 × 3 matrix values in the upper left of MWA and MAB. This is a zero value result in the bottom row of all 4 × 4 matrices. The rightmost column of the product 4 × 4 matrix MWB is the product of the upper left 3 × 3 of MWA and the rightmost column vector of MAB and the rightmost column vector of MWA, which is also all 4 × It is the result of 0 value and 1 value in the bottom row of 4 matrices.
[0085]
To reduce computation time, in one embodiment, multiplication by 0 or 1 is not performed on the bottom row of the 4 × 4 matrix. Therefore, multiplication, addition and transposition of 3 × 3 matrix and 3 vectors are performed.
[0086]
Similar functions can be defined for other transformations as follows.
[0087]
[Equation 9]

[0088]
From the above description, it can be seen that the MSA 4 × 4 matrix values can be used to fully describe the CSA for the CSW. Further, a matrix MWA that is an inverse matrix of MWA ^-1 Completely describes the CSW for the CSA. Therefore, the following holds.
[0089]
[Expression 10]

[0090]
Using machine vision analysis techniques, the system described above can obtain a target camera image, such as the image 90 shown in FIG. 3B, and calculate the target coordinate system for that camera.
[0091]
The calculation of the relative target position (RTP) will be described below with reference to FIG. 3A. For the purpose of calculating RTP,
Let CSL be the coordinate system of the left camera 10L.
[0092]
Let CSA be the coordinate system of the target 72.
MLA represents CSA for CSL.
[0093]
Let CSB be the coordinate system of target 74.
MLB represents the CSB for the CSL.
[0094]
Assuming that the left camera image 90 includes images 92 of targets 72 and 74, matrices MLA and MLB are created and stored by machine vision analysis techniques. Thus, the RTP value (between target 72 and target 74) is obtained by:
[0095]
[Expression 11]

[0096]
Based on this, the system can calculate MLB to MLA and vice versa by:
[0097]
[Expression 12]

[0098]
Once the RTP value is created and stored, the target assembly 70 can be moved so that the left camera 10L sees the target 72 and the calibration camera 20 sees the target 74. Assume that CSC is the coordinate system of the calibration camera 20. A CSA for CSL is described by machine vision analysis techniques, assuming that the left camera image 90 includes the image 94 (FIG. 3D) of the target 70 and the calibration camera image 96 includes the image 98 (FIG. 3D) of the target 74. An MCB describing the matrix MLA and the CSB for the CSC is created and stored.
[0099]
Based on these matrices, the value of the relative camera position RCP of the left camera 10L with respect to the calibration camera 20 can be calculated as follows.
[0100]
[Formula 13]

[0101]
Further, the relative camera position RCP of the calibration camera 20 with respect to the left camera 10L can be calculated as follows.
[0102]
[Expression 14]

[0103]
Next, the calculation of the value for the right camera 10R is presented. With reference to FIG.
Assume that CSS is the coordinate system of the setup camera 100.
[0104]
Assume that CSR is the coordinate system of the right camera 10R.
It is assumed that CSQ is the coordinate system of the calibration target 16.
[0105]
Let CSD be the coordinate system of the data target 104.
Assuming that the setup camera image 106 of FIG. 4B includes images 16 ′ and 104 ′ of the calibration target 16 and data target 104, the machine vision analysis technique uses a matrix MSQ describing the CSQ for the CSS and an MSD describing the CSD for the CSS. Are created and stored. Further, assuming that the right camera image 108 includes an image 104 ″ of the data target 104, a machine vision analysis technique creates and stores a matrix MRD that describes the CSD for the CSR. The MAQ is the calibration target 16 and the data target 104. RTP (CSD for CSQ) between
[0106]
Thereafter, the coordinate system of the calibration target 16 for the right camera, ie the CSQ for the CSR, is obtained by:
[0107]
[Expression 15]

[0108]
Thus, an MLC value describing the calibration camera for the left camera and an MRQ value describing the calibration target for the right camera can be calculated.
[0109]
In normal operation, the system generates an image of the type shown in FIGS. 5A, 5B, and 5C. The left camera 10L generates two left wheel target images 110 (FIG. 5A). Machine vision analysis technology creates and stores values for the coordinate system of the wheel target in the coordinate system of the left camera. If the left camera coordinate system is defined as the world coordinate system, the left wheel target is transposed to the world coordinate system.
[0110]
The calibration camera 20 generates a calibration target image 112 (FIG. 5B). The machine vision analysis technique creates and stores values related to the calibration target coordinate system in the calibration camera coordinate system, MCQ.
[0111]
The right camera 10L generates the image 114 shown in FIG. 5C for the two right wheel targets. The machine vision analysis technique creates and stores values relating to the coordinate system of the wheel target in the coordinate system of the right camera, MRW. “W” here means “wheel”, not “world”. Normally, the coordinate system of the left camera functions as the world coordinate system.
[0112]
The value of the right wheel target can be transposed to the same world coordinate system as the left wheel target by calculating the MLW, the right wheel target in the left (world) coordinate system. From the calibration process, the values of MLC and MRQ are known. The system measures MCQ based on the image of FIG. 5B and measures MRW based on the image of FIG. 5C. Therefore, the following holds.
[0113]
[Expression 16]

[0114]
--- Hardware overview
FIG. 7 is a block diagram that illustrates a computer system 700 upon which an embodiment of the invention may be implemented. Computer system 700 may be used as an arrangement for some or all of device 100 or for an arrangement of external computers or workstations that communicate with device 100.
[0115]
Computer system 700 includes a bus 702 or other communication mechanism for communicating information, and a processor 704 coupled with bus 702 for processing information. Computer system 700 also includes a main memory 706, such as random access memory (RAM) or other dynamic storage device, that is coupled to bus 702 for storing instructions and information to be executed by processor 704. Main memory 706 may also be used to store temporary variables or other intermediate information during execution of instructions to be executed by processor 704. Computer system 700 further includes a read only memory (ROM) 708 or other static storage device coupled to bus 702 for storing instructions and static information for processor 704. A storage device 710 such as a magnetic disk or optical disk is provided for storing information and instructions and is coupled to the bus 702.
[0116]
Computer system 700 may be coupled via bus 702 to a display 712 such as a cathode ray tube (CRT) for displaying information to a computer user. An input device 714 that includes alphanumeric and other keys is coupled to the bus 702 to communicate information and command selections to the processor 704. Another type of user input device is a cursor control 716, such as a mouse, trackball or cursor instruction key, which communicates instruction information and command selections to the processor 704 and also displays a cursor on the display 712. To control the movement. The input device typically has two degrees of freedom in two axes, a first axis (eg, x) and a second axis (eg, y) so that the device can be positioned in a plane. Can be identified.
[0117]
Embodiments of the invention relate to the use of computer system 700 for automatic calibration of aligners. In accordance with one embodiment of the present invention, automatic calibration of the aligner is provided by computer system 700 in response to processor 704 executing one or more sequences of one or more instructions contained in main memory 706. Is done. Those instructions may be read into main memory 706 from another computer-readable medium, such as storage device 710. Execution of the sequence of instructions contained in main memory 706 causes processor 704 to perform the process steps described herein. In alternative embodiments, hardwired circuitry may be used in place of or in combination with software instructions that implement the present invention. Thus, embodiments of the invention are not limited to any specific combination of hardware circuitry and software.
[0118]
The term “computer-readable medium” as used herein refers to any medium that participates in providing instructions to processor 704 for execution. Such a medium may take many forms, including but not limited to, non-volatile media, volatile media, and transmission media. Non-volatile media includes, for example, optical or magnetic disks, such as storage device 710. Volatile media includes dynamic memory, such as main memory 706. Transmission media includes coaxial cables, copper wires, and optical fibers, including the wires that make up bus 702. Transmission media may also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.
[0119]
Common forms of computer readable media include, for example, floppy (R) disks, flexible disks, hard disks, magnetic tapes or other magnetic media, CD-ROMs, other optical media, punch cards, paper tapes, other holes Physical media with patterns, RAM, PROM and EPROM, FLASH-EPROM, other memory chips or cartridges, carrier waves as described below, and other computer readable media.
[0120]
Various forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to be executed by processor 704. For example, the instructions may first be carried on a remote computer magnetic disk. The remote computer can load the instructions into its dynamic memory and send it over a telephone line using a modem. A modem local to computer system 700 can receive the data on the telephone line and use an infra-red transmitter to convert the data to an infra-red signal. The infrared detector can receive the data carried in the infrared signal and appropriate circuitry can place the data on the bus 702. Bus 702 carries the data to main memory 706, from which processor 704 retrieves and executes instructions. The instructions received by main memory 706 may be suitably stored on storage device 710 either before or after being executed by processor 704.
[0121]
Computer system 700 also includes a communication interface 718 coupled to bus 702. Communication interface 718 couples to network link 720 connected to local network 722 to provide bi-directional data communication. For example, communication interface 718 may be an integrated digital network (ISDN) card or modem that provides a data communication connection to a corresponding type of telephone line. As another example, communication interface 718 may be a local area network (LAN) card that provides a data communication connection to a compatible LAN. A wireless link may be implemented. In any implementation, communication interface 718 sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.
[0122]
Network link 720 typically provides data communication with other data devices over one or more networks. For example, the network link 720 may provide a connection to a host computer 724 via a local network 722 or to a data device operated by an Internet service provider (ISP) 726. ISP 726 then provides data communication services through the world wide packet data communication network now commonly referred to as the “Internet” 728. Local network 722 and Internet 728 both use electrical, electromagnetic or optical signals that carry digital data streams. Signals through various networks and signals on network link 720 through communication interface 718 that carries digital data to and from computer system 700 are exemplary forms of carriers that carry information.
[0123]
Computer system 700 can send messages and receive data, including program code, via one or more networks, network link 720 and communication interface 718. In the Internet example, server 730 may transmit codes required for application programs via Internet 728, ISP 726, local network 722 and communication interface 718. In accordance with an embodiment of the present invention, one such downloaded application enables automatic calibration of the aligner described herein.
[0124]
The received code may be executed as is by processor 704 and / or stored in storage device 710 or other non-volatile storage for later execution. In this manner, computer system 700 can obtain application code in the form of a carrier wave.
[0125]
-Advantages and further variations
The embodiments disclosed herein are applicable to other contexts. In particular, these embodiments are useful for calibrating machine vision measurement systems having two or more cameras. In addition, these embodiments can be used in connection with recreational vehicle (RV) alignment. Aligners for RV typically require a wider boom than standard aligners, but using the embodiments disclosed herein, aligners for RV may require new hardware. Instead, it can be constructed by simply widening the uprights slightly and fixing them with bolts.
[0126]
Since the above-described device is self-calibrating, no post-setup calibration is required and can therefore be incorporated into a portable aligner. Portable alignment tasks can be performed using two cameras on a tripod in a parking lot, garage or similar environment without the need for time consuming calibration. Thus, the device can be used to facilitate a completely new service, remote alignment or field alignment.
[0127]
Furthermore, the technique described above for measuring (or calibrating) the relative position of the left camera relative to the right camera can be used in a system having multiple devices. These techniques are used to measure the relative position of one device to another device among a plurality of devices. Under such conditions, any pair of devices including the first device and the second device among the plurality of devices can be handled as the pair of the left camera and the right camera in the above-described technology. In this case, the calibration device is mounted near the first device, and the relative position of the calibration device with respect to the first device is predetermined. Similarly, the calibration target is mounted near the second device, and the relative position of the calibration target with respect to the second device is predetermined. Thereafter, the relative position of the calibration device with respect to the calibration target is measured. Finally, the relative position of the first device relative to the second device is 1) the relative position of the calibration device relative to the first device, 2) the relative position of the calibration target relative to the second device, and 3) relative to the calibration target. Calculated based on the relative position of the calibration device. In one embodiment, the calibration device is configured to measure the relative position of the calibration device relative to the calibration target.
[0128]
Although the invention has been described in the foregoing specification with reference to specific embodiments, it will be apparent that various modifications and changes can be made thereto without departing from the broader spirit and scope of the invention. . The specification and drawings are, accordingly, to be regarded in an illustrative sense rather than a restrictive sense.
[Brief description of the drawings]
FIG. 1 is a schematic plan view of a 3D automobile alignment system.
FIG. 2 is a diagram of an upright element of an alignment system.
FIG. 3A is a diagram of an apparatus that may be used for measuring a relative target position in a method for measuring and calibrating a relative position between an alignment camera and a calibration camera.
FIG. 3B is a view of the view as seen by the camera.
FIG. 3C is a diagram of an apparatus that may be used in the step of measuring relative camera position in a method for measuring and calibrating the relative position of an alignment camera and a calibration camera.
FIG. 3D is a view of the view seen by the alignment camera and the calibration camera.
FIG. 4A is a diagram of an apparatus that may be used in a method for measuring and calibrating the relative position of an alignment camera and a calibration target.
4B is a view of the view seen by the setup camera of the apparatus of FIG. 4A.
4C is a view of the view seen by the alignment camera of the apparatus of FIG. 4A.
5A is a view of a view of two wheel targets as seen by the first alignment camera of the apparatus of FIG.
FIG. 5B is a view of the view seen by the calibration camera of the apparatus of FIG. 1 during calibration.
5C is a view of a view of two wheel targets as seen by the second alignment camera of the apparatus of FIG.
FIG. 6A is a flow diagram illustrating a process for calibrating a camera module having two cameras.
FIG. 6B is a flow diagram illustrating a process for calibrating a camera module having a camera and a calibration target.
FIG. 6C is a flow diagram of an alignment process that includes performing camera calibration during alignment.
FIG. 7 is a block diagram of a computer system in which one embodiment may be implemented.
FIG. 8 is a simplified diagram of camera and coordinate system geometric relationships that provide the basis for numerical computer-based mathematical computations used in the system described above.

Claims (22)

A mechanical measurement system having a first measurement device and a second measurement device, wherein the first measurement device and the second measurement device capture an image of at least one target attached to the vehicle. A configured image capture device, the system comprising:
A first calibration target attached in a predetermined relationship to a first measurement device of the machine measurement system;
A calibration device attached in a predetermined relationship to a second measurement device of the mechanical measurement system;
A memory for storing a calibration value representing the position of the first calibration target with respect to the calibration device;
A data processor configured to calculate a position of the first measurement device relative to the second measurement device based on the relative position of the first calibration target relative to the calibration device, the calibration device comprising: In operation, configured to periodically measure a new value representing a new position of the first calibration target relative to the calibration device;
During operation, the difference between the calibration value and the new value is used to update the relative position between the first measuring device and the second measuring device, and the calibration value exceeds the allowable amount and differs from the new value. A system that alerts and alerts.   The relative position value between the first measuring device and the second measuring device is recalculated upon recognition that, in operation, the calibration value exceeds the allowable amount and differs from the new value. System. Data processor
A relative measurement device position value representing the position of the second measurement device relative to the calibration device;
Further configured to calculate a relative position between the first measurement device and the second measurement device based on a relative measurement device position value representative of the position of the first measurement device relative to the first calibration target. The system according to claim 1 or 2.   In operation, a relative measurement device target position value representative of the position of the first measurement device relative to the first calibration target is calculated based on the position of the first calibration target relative to the second calibration target. The described system.   In operation, the position of the first calibration target relative to the second calibration target is obtained by using a third measuring device that provides information for calculating the position of the first calibration target relative to the second calibration target. The system of claim 4, wherein: in action,
The position of the first calibration target relative to the second calibration target is obtained by an image capture device;
Images of the first calibration target and the second calibration target are provided by a first calibration target and a second calibration target positioned within the view of the image capture device;
The system of claim 4, wherein the images of the first calibration target and the second calibration target are input to a data processor and a relative position of the first calibration target with respect to the second calibration target is calculated.   The data processor is configured to provide a relative relationship between the first measurement device and the second measurement device while the first measurement device and the second measurement device of the machine measurement system are measuring the target of the object being measured. The system according to claim 1, further configured to calculate a position. Data processor
Calculating a relative position between the first measuring device and the second measuring device while the first measuring device and the second measuring device of the mechanical measuring system are measuring the target of the object being measured;
The apparatus is further configured to modify a measurement generated by measuring a target of an object under measurement based on a relative position between the first measurement device and the second measurement device. The system according to any one of 1 to 7.   The data processor is configured such that the modified relative position between the first measurement device and the second measurement device exceeds a predetermined value and the relative position between the first measurement device and the second measurement device. To correct the measurement generated by measuring the target of the object under measurement based on the corrected relative position between the first measurement device and the second measurement device only when The system of claim 8, further configured.   The system according to claim 1, wherein the calibration device is a system that is an image capture device that measures an object by capturing an image.   The data processor receives information from the calibration device based on predetermined information identifying the position of the second measurement device relative to the calibration device and indicating a change in the position of the first measurement device relative to the second measurement device. And a data processor configured to measure the position of the first measurement device relative to the second measurement device.   12. A system according to any preceding claim, wherein the light source is mounted in a fixed relationship to the second measuring device to illuminate the first calibration target. A method for calibrating a mechanical measurement system having a first measurement device and a second measurement device, comprising:
Providing an image capture device configured to capture an image of at least one target attached to the vehicle relative to the first measurement device and the second measurement device;
Attaching a first calibration target to a first measurement device of a mechanical measurement system in a predetermined relationship;
Attaching a calibration device to a second measurement device of the mechanical measurement system in a predetermined relationship;
Storing a value representing the position of the first calibration target relative to the calibration device as a calibration value;
Calculating the position of the first measurement device relative to the second measurement device based on the position of the first calibration target relative to the calibration device;
Periodically measuring a new value representative of the new position of the first calibration target relative to the calibration device during operation of the machine measurement system;
Applying the difference between the calibration value and the new value to update the relative position between the first measuring device and the second measuring device;
Issuing a warning alert if the calibration value exceeds an acceptable amount and differs from the new value. A first relative measurement device position value representative of the position of the second measurement device relative to the calibration device;
Calculating a relative position between the first measurement device and the second measurement device based on a relative measurement device target position value representative of the position of the first measurement device relative to the first calibration target; The method according to claim 13.   The method of claim 14, wherein the second relative measurement device target position value is calculated based on the position of the first calibration target relative to the second calibration target.   The position of the first calibration target relative to the second calibration target is obtained by using a third measuring device that provides information for calculating the position of the first calibration target relative to the second calibration target. Item 16. The method according to Item 15. The position of the first calibration target relative to the second calibration target is obtained by using an image capture device;
Images of the first calibration target and the second calibration target are provided by placing the first calibration target and the second calibration target in a view of the image capture device;
16. The method of claim 15, wherein images of the first calibration target and the second calibration target are applied to calculate a relative position of the first calibration target with respect to the second calibration target.   Calculating a relative position between the first measuring device and the second measuring device while the first measuring device and the second measuring device of the mechanical measuring system are measuring the target of the object being measured; The method according to claim 13, further comprising a step. While the first measuring device and the second measuring device of the mechanical measuring system are measuring the target of the object being measured, the corrected relative position between the first measuring device and the second measuring device is A calculating step;
Modifying the measurement generated by measuring the target of the object being measured based on the modified relative position between the first measurement device and the second measurement device. Item 19. The method according to any one of Items 13 to 18.   Based on the modified relative position between the first measuring device and the second measuring device, modifying the measurement generated by measuring the target of the object being measured comprises the first measurement. Only when the modified relative position between the device and the second measuring device exceeds a predetermined value and differs from the relative position between the first measuring device and the second measuring device. Item 19. The method according to any one of Items 13 to 18.   The measurement of the position of the first measurement device relative to the second measurement device is based on predetermined information identifying the position of the first measurement device relative to the calibration device and the first measurement device relative to the second measurement device. 21. A method according to any of claims 13 to 20, based on information received from a calibration device indicating a change in the position of the device.   The method of claim 21, comprising using a light source attached in a fixed relationship to the second measuring device to illuminate the calibration target.

Images