JP3616949B2 - Object position measuring method and apparatus, and recording medium storing object position measuring method program - Google Patents

Description

を成分とするＴ行Ｎ列の行列Ｕ’を作成する座標行列作成部と、Ｔ行１列の行列Θ”と１行Ｎ列の行列Ａ’について、行列Θ”と行列Ａ’との積が、行列Ｕ’と見なせるように行列Θ”と行列Ａ’を決定する行列分解部と、あらかじめ測定された実測値に対応するようにある値ｑ及び一定値θ_０ を算出し、行列Θ”にｑを乗じてθ_０ を加えて、Ｔ枚の画像を撮影したときのカメラ部の向きを成分とする行列Θを算出し、また行列Ａ’をｑで除して行列Ａを算出する正規化部と、正規化部が算出した行列Θの成分をθ_ｔ とし、また行列Ａの各成分をα_ｉ として、回転中心から特徴点までの距離ｒ_ｉ をｒ_ｉ ＝Ｌ／（１−１／α_ｉ ）で算出し、また、前記回転中心から特徴点までの方向φ_ｉ を
【数９】

で算出する座標算出部とを備えている。
【００１０】
また、本発明に係る物体の位置測定装置は、特徴点追跡部が検出した全ての特徴点が、複数枚の画像の全てに含まれていなかった場合、あらかじめ測定された実測値に基づいて、画像を撮影したときのカメラの向きの値並びに中心から各特徴点までの距離及び方向が算出可能な特徴点及び画像の組合せを基本グループとして作成し、座標行列作成部、行列分解部、正規化部及び座標算出部に画像を撮影したときのカメラの向きの値並びに中心から各特徴点までの距離及び方向を算出させる基本グループ作成部と、算出された撮影したときのカメラの向きの値並びに中心から各特徴点までの距離及び方向を算出結果として記憶する位置記憶部と、位置記憶部に記憶された算出結果に基づいて、画像を撮影したときのカメラの向きの値並びに中心から各特徴点までの距離及び方向が算出可能な特徴点及び画像の組合せを連鎖グループとして作成し、座標行列作成部、行列分解部、正規化部及び座標算出部に画像を撮影したときのカメラの向きの値並びに中心から各特徴点までの距離及び方向を算出させて算出結果として位置記憶部に記憶させ、全ての画像に対して撮影したときのカメラの向きの値が算出され、全ての特徴点に対して中心から各特徴点までの距離及び方向が算出されるまで連鎖グループを作成する連鎖グループ作成部とを備えている。
【００１１】
また、本発明に係る物体の位置測定方法のプログラムを記録した記録媒体は、回転中心から一定距離にある円周上を移動して、円周外側の複数枚の画像をカメラに撮影させ、複数枚の画像の全てに含まれる１又は複数の特徴点を検出し、各画像における各特徴点の移動方向の座標値を検出させ、各画像における各特徴点の移動方向の座標値を、撮影したカメラに基づいた焦点距離で正規化させ、また、複数の画像における正規化した座標値の平均を特徴点ごとに算出させて各正規化した座標値との差を算出させ、算出した差の値を成分とする行列を作成させ、ある２つの行列の積が、作成した行列に最も近い行列となるように、ある２つの行列を決定させ、あらかじめ測定された実測値に対応するようにある値及び一定値を算出させ、ある２つの行列の一方の行列にある値を乗じて一定値を加えさせ、またある２つの行列の他方の行列をある値で除させて、それらの行列に基づいて、中心から各特徴点までの距離及び方向を算出させることをコンピュータに行わせるプログラムを記録したものである。
【００１２】
また、本発明に係る物体の位置測定方法のプログラムを記録した記録媒体は、検出した全ての特徴点が、複数枚の画像の全てに含まれていなかった場合に、あらかじめ測定された実測値に基づいて、画像を撮影したときのカメラの向きの値並びに中心から各特徴点までの距離及び方向が算出可能な特徴点及び画像の組合せを基本グループとして作成させ、基本グループに含まれる特徴点及び画像の組合せについて、画像を撮影したときのカメラの向きの値並びに中心から各特徴点までの距離及び方向を算出させて、その算出結果を記憶させ、算出結果に基づいて、画像を撮影したときのカメラの向きの値並びに中心から各特徴点までの距離及び方向が算出可能な特徴点及び画像の組合せを連鎖グループとして作成させ、連鎖グループに含まれる特徴点及び画像の組合せについて、画像を撮影したときのカメラの向きの値並びに中心から各特徴点までの距離及び方向を算出させ、その算出結果を記憶させ、全ての特徴点に対して中心から各特徴点までの距離及び方向が算出されるまで連鎖グループを作成させて、画像を撮影したときのカメラの向きの値並びに中心から各特徴点までの距離及び方向を算出させ、その算出結果を記憶させることをコンピュータに行わせるプログラムを記録したものである。
【００１３】
【発明の実施の形態】
実施形態１．
図１は本発明の第１の実施の形態に係る位置測定方法を実現するための装置のブロック図である。図１において、１００はカメラ部、２００は特徴点追跡部、３００は座標行列作成部、４００は行列分解部、５００は正規化部、６００は座標算出部である。具体的な構成及び動作説明をする前に、本実施の形態による位置測定の原理について説明する。本実施の形態は、撮影された画像のうち、少なくとも２枚の画像についてカメラの向きの値が実測されている場合に、
回転中心から特徴点（例えば、対象物体の形状を表すために特徴とされている点等）までの距離及び方向を算出するものである。
【００１４】
図２はカメラ部１００の設置状態例を表す図である。本実施の形態では、図２のように、撮影部分となるカメラが、一定半径Ｌの円周上を移動し、その円周の外側を撮影するように設置する。ここで、本実施の形態は、撮影される全ての画像に特徴点が撮られているものとする。
【００１５】
図３は本実施の形態で用いる近似について説明するための図である。ここで、特徴点はカメラ部１００の回転中心から半径ｒ_ｉ 上の円周上にあるものとする。特徴点がカメラの正面に十分近い位置にある場合、特徴点は、回転中心とカメラ正面とを結ぶ線分の延長線上にある距離ｒ_ｉ の地点から延長線に垂直にｌ_ｉ だけ移動した点と近似しても、大きな誤差はない。したがって、本実施の形態ではこの近似を用いることにする。本発明はカメラの設置条件と近似の工夫により、カメラが異なった向きとなる複数の画像と少ない実測値から、回転中心から特徴点までの距離及び方向を得るものである。
【００１６】
図４はカメラ部１００上部から見たカメラ部１００と特徴点との関係を表す図である。図４ではカメラ部と特徴点との各関係をパラメータで表している。撮影された画像上における特徴点の横方向の位置を、カメラより特有な値を有する焦点距離で正規化すると、正規化された値は次式（１）で表される。
ｕ_ｔｉ＝ｘ_ｔｉ／ｚ_ｔｉ＝α_ｉθ_ｔ＋β_ｉ …（１）
ここで、α_ｉ＝１／（１−Ｌ／ｒ_ｉ）、β_ｉ＝φ_ｉ／（１−Ｌ／ｒ_ｉ）
【００１７】
実際に撮影された画像から得られるのは（１）式のｕ_ｔｉであり、このｕ_ｔｉに基づいてｒ_ｉ、φ_ｉを得ることが目的となる。ここで、ｉは各特徴点につけられた番号である。また、ｔは撮影された各画像につけられた番号である。
【００１８】
（１）式のｕ_ｔｉについて、各特徴点（ｉ＝１，２，…Ｎ：Ｎは自然数）の全画像（ｔ＝１，２，…Ｔ：Ｔは自然数）での平均値は次式（２）で表される。（２）式では全ての値を平等に扱い、単なる平均値で算出するようにしたが、場合によっては重み付けを行って算出するようにしてもよい。
【００１９】
【数１０】

【００２０】
（２）式で算出した平均値から（１）式の値を引くと、次式（３）が得られる。
【００２１】
【数１１】

【００２２】
これを全てのｕ_ｔｉについて行い、行列で表せば、次式（４）のようにＵ’＝Θ’Ａで表される。ここでＵ’は実際に撮影された画像から得られる特徴点の位置のみで算出されるので、この式の左辺は画像を何らかの画像処理を行うことで得ることができる。そこで、画像処理により得られたＵ’に基づいて、式（４）が成り立つように、一定の条件の下でΘ’及びＡを算出する。そして、Θ’及びＡに基づいて、特徴点までの距離ｒ_ｉ と各画像が撮影された時のカメラの向きとを得ることができる。また、式（２）からβ_ｉ を算出することができるので、そのβ_ｉ を式（１）に代入し、φ_ｉ を得ることができる。
【００２３】
【数１２】

【００２４】
次に図１の装置の具体的な構成を説明する。カメラ部１００は、図２に示したように半径Ｌの円周上で外方向に撮影するように設置されており、円周上の任意の地点の画像を撮影及び出力する。特徴点追跡部２００は、カメラ１００部が撮影した各画像において、その距離を得ようとする１又は複数の特徴点について、カメラの回転方向の座標を探索する。この特徴点追跡部２００では、各特徴点を各画像上での対応付けができればよく、対象物体に付けられた特定のマークを、パターン照合（パターンマッチング）等のような一般に知られた手法で探索してもよい。また、マークが付いていない場合でもオプティカルフロー等のように一般に知られた画像処理の手法を用いて探索してもよい。
【００２５】
座標行列作成部３００は、特徴点追跡部２００で探索した画像ｔにおける特徴点ｉについて、画像中心を原点とし、横方向座標（カメラ回転方向座標）を焦点距離で正規化したｕ_ｔｉを算出する。そして、（３）式に基づいてｕ_ｔｉ’を算出し、行列Ｕ’を算出する。したがって、各画像上でＮ個の特徴点を探索し、Ｔ枚の画像が撮影されたとすると、行列Ｕ’はＴ行Ｎ列の行列となる。
【００２６】
行列分解部４００は、Ｔ行１列の行列Θ”と１行Ｎ列の行列Ａ’との積が行列Ｕ’に近くなるように、行列Θ”と行列Ａ’とを算出する。この行列Θ”及び行列Ａ’の算出は、例えば次のようにして実現する。前述した近似が十分成り立つ場合には、Ｕ’の特異値は、最大の特異値ｋを除いて０と近似できる。したがって、Ｕ’＝Θ”ｋＡ”と特異値分解することができる。ここでＡ’＝ｋＡ”として行列Ａ’を算出し、行列Θ”と行列Ａ’の算出を実現する。
【００２７】
正規化部５００では、撮影された画像のうち、少なくとも２枚の画像についてカメラの向きの値が実測されている場合に、カメラの向きの差があらかじめ測定された値に最も近くなるように、行列Θ”にｑをかけ、また行列Ａ’に１／ｑをかけて、それぞれ行列Θ’及び行列Ａを得る。ｑは、行列Θ”にかけたｑと行列Ａ’にかけた１／ｑとが互いに相殺されるので、行列Ｕ’では現れない。また、行列Θ’について、撮影されたときのカメラの向きの値があらかじめ測定された値になるように、Θ’の成分である各θ_ｔ’ に一定値θ_０ を加えてθ_ｔ （行列Θの成分）を得る。ここで、ｑ及びθ_０ は、実測値とそれに対応する算出された行列の成分との差を少なくするように調整するための調整値である。座標算出部６００では、Ａの各成分であるα_ｉ から次式（５）に基づいて各特徴点のカメラ回転中心からの距離ｒ_ｉ を算出する。また、次式（６）に基づいて、各特徴点の位置φ_ｉ を算出する。
【００２８】
【数１３】

【００２９】
図５は対象物体の特徴点と撮影状況を表す図である。次に具体的な数値に基づいて本実施の形態における装置の動作を説明する。カメラの回転中心からカメラまでの距離Ｌは０．６ｍとして説明する。カメラ部１００のカメラ部分をカメラ回転中心を軸として回転移動させる。このとき、各特徴点の距離と方向の実測値を表１で示す（この値にできるだけ近づくような値を算出するのが本実施の形態の目的である）。
【００３０】
【表１】

【００３１】
特徴点追跡部２００では、パターンマッチング、オプティカルフロー等の一般的な手法で各画像での各特徴点の位置を検出する。この場合、各画像について各特徴点のカメラの回転方向（図２のような設置では、カメラは水平（横方向）に回転する）の位置を、カメラの焦点距離で正規化した値ｕ_ｔｉを、次の表２に示す。
【００３２】
【表２】

【００３３】
座標行列作成部３００では、各特徴点毎に、各画像から導かれた座標値（距離）の平均を算出し、それと各座標との差を算出した行列を導き出す。例えば、特徴点１の場合、ｔ＝１〜１１までの各座標の平均値は−０．１６６となる。これを各座標から減算し、ｔ＝１、ｉ＝１では−０．３０９−（−０．１６６）＝−０．１４３であり、これが作成される行列の第１行第１列目の成分となる。このようにして算出した行列を次式（７）に示す。
【００３４】
【数１４】

【００３５】
行列分解部４００では（７）式の行列を行列Θ”と行列Ａ’に分解する。具体的には、以下に例示する手順で、最大の特異値以外は０であると近似した特異値分解を行い、分解を実現する。まず、（７）式の行列とその転置行列との積を算出する。その行列を次式（８）に示す。
【００３６】
【数１５】

【００３７】
次にこの行列の最大の固有値とそれに対する固有ベクトルを求める。これを次式（９）に示す。固有値、固有ベクトルの算出方法については、累乗法等の多数の方法が一般に知られている。
【００３８】
【数１６】

【００３９】
（７）式の行列とこの固有ベクトルとの積を、固有値の平方根で除算したものを行列Θ”とする。また、この固有値の平方根と固有ベクトルとの積を行列Ａ’とする。行列Θ”及び行列Ａ’を次式（１０）及び（１１）に示す。
【００４０】
【数１７】

【００４１】
ここで、例えばｔ＝１とｔ＝１１の画像を撮影したときのカメラの向きが、それぞれ−０．８７２７ｒａｄ及び０．８７２７ｒａｄと測定されていたとする。正規化部５００では次式（１２）に基づいてｑを算出し、Θ”にｑをかけて行列Θ’を算出する。また行列Ａ’に１／ｑをかけて行列Ａを算出する。算出した行列Θ’の成分及び行列Ａの成分を表３に示す。
ｑ＝｛０．０８７２７−（−０．０８７２７）｝／（θ_１１”−θ_１”）…（１２）
【００４２】
【表３】

【００４３】
さらに、通常、θ_１’ が実測値−０．８７２７ｒａｄとなるように調整し、その調整に伴って、全θ_ｔ’ に一定値を加えるが、この例では、θ_１’ はθ_１ と既に一致しているので、θ_ｉ’ がそのままθ_ｉ となる。
【００４４】
座標算出部６００では、例えば特徴点１について、（１）式に基づいて次式（１３）のようにして、カメラ焦点から特徴点１までの距離ｒ_１ を算出する。また、同様に、（１）式に基づいて、β_１ ＝−０．１６６−１．６４８３２５×０＝−０．１６６、φ_１ ＝−０．１６６と算出できる。このようにして算出した他の特徴点の算出結果を表４に示す。これらの値と表１の実測値とを比較すると、算出した値が、実測値に近い値となっていることがわかる。
ｒ_１ ＝０．６／（１−１／１．６４８３２５）＝１．５２５４６２…（１３）
【００４５】
【表４】

【００４６】
以上のように、第１の実施の形態によれば、中心から一定距離にある円周上を回転移動して、円周外側画像を撮影するように条件を工夫したカメラ部１００が撮影したＴ枚の画像について、特徴点追跡部２００が全ての画像に含まれる特徴点をＮ個検出して、各画像における各特徴点の回転方向の座標値を検出し、座標行列作成部３００が、それらの座標値を撮影したカメラ部１００に特有な焦点距離で正規化してｕ_ｔｉを算出し、また各特徴点について、Ｔ枚の画像のｕ_ｔｉの平均を算出してｕ_ｔｉとの差を算出して、算出した差の値を成分とする行列Ｕ’を作成し、また行列分解部４００が、Ｔ行１列の行列Θ”と１行Ｎ列の行列Ａ’との２つの行列の積が、行列Ｕ’に最も近い行列となるように、行列Θ”と行列Ａ’を決定し、少なくとも２枚の画像を撮影したときのカメラの向きの値が実測されている場合に、正規化部５００は、その実測値と対応するように、行列Θ”にある値ｑをかけて、一定値θ_０ を加えて行列Θを算出する。この行列Θが画像を撮影したときのカメラの向きの値を表す。また、行列Ａ’をある値ｑで割って、行列Ａを算出し、座標算出部が、行列Ａに基づいて、中心から各特徴点までの距離及び方向を算出するようにしたので、少なくとも２枚の画像について撮影されたときのカメラの向きの値を実測するだけで、少ない実測値から精度よく回転中心から特徴点までの距離及び方向並びに各画像が撮影されたときのカメラの向きの値を算出することができる。また、行列分解部４００は、座標行列作成部３００が作成した行列Ｕ’における最大の特異値以外の特異値を０と近似し、特異値分解を行い、Ｔ行１列の行列Θ”と１行Ｎ列の行列Ａ’との２つの行列を決定するようにしたので、図３で示した近似が十分に成り立つ場合には、はやくて有効な算出ができる。
【００４７】
実施形態２．
第１の実施の形態では、少なくとも２つの画像を撮影したときのカメラの向きの値が実測されている必要があった。本実施の形態では、実測値が、少なくとも２つのカメラの向きの値ではなく、少なくとも１つの特徴点について、回転中心からの距離及び方向の値が実測されている場合の位置測定方法について説明する。
【００４８】
本実施の形態の位置測定方法を実現する装置は、第１の実施の形態で説明した図１に示したものとほぼ同じである。ただ、正規化部５００の代わりに正規化部５０１を用いて実施する。正規化部５００は、少なくとも２つの画像を撮影したときに測定されたカメラの向きに基づいてｑを算出したが、正規化部５０１は、少なくとも１つの特徴点について測定された回転中心からの距離及び方向に基づいてｑを算出する点で正規化部５００とは異なる。
【００４９】
具体的な動作説明をする前に、本実施の形態の正規化部５０１の原理について説明する。ここで、少なくとも１つの特徴点の回転中心からの距離及び方向が実測されているものとする。ここでは、実測されている距離から算出される値とそれに対応する行列Ａの成分とが最も近くなるように第１の実施の形態で述べたｑの値を決定する。例えば、１つの特徴点について位置（回転中心からの距離及び方向）が測定されている場合、その距離に対するＡ’の該当する成分の比をｑとする。そして第１の実施の形態と同じように、行列Θ”にｑをかけて行列Θ’を算出する。また、行列Ａ’に１／ｑをかけて行列Ａを算出する。
【００５０】
また、実測された方向と（６）式により算出された方向とが最もよく一致するような次式（１４）が得られるように、Θ’の成分である各θ_ｔ’ に一定値θ_０ を加え、θ_ｔ を得る。例えば、測定された特徴点が１つであれば、次式（１５）のようにθ_０ を算出する。
【００５１】
【数１８】

【００５２】
次に第１の実施の形態と動作が異なる正規化部５０１の動作について、具体的な数値に基づいて説明する。ここで、表１に基づいて特徴点４の距離ｒ_４ が２、方向φ_４ が０であると実測されているものとする。この場合、（１）式に基づいて、α_４ ＝１／（１−０．６／２）＝１．４２８５７１と算出される。行列分解部４００で算出した行列Ａ’における４列目の成分は０．２６２２０２であるから、ｑの値は、ｑ＝０．２６２２０２／１．４２８５７１＝０．１８３５４１となる。このｑの値に基づいて、Θ”にｑをかけて行列Θ’を算出する。また行列Ａ’に１／ｑをかけて行列Ａを算出する。算出した行列Θ’の成分及び行列Ａの成分を表５に示す。
【００５３】
【表５】

【００５４】
次にφ_４ ＝０であるので、（９）式の右辺の第１項と第２項では、０＋（−０．１２５＋（−０．１）＋（−０．０７５）＋（−０．０５）＋（−０．０２５）＋０＋０．０２５＋０．０５＋０．０７５＋０．１＋０．１２５）＝０となる。また、第３項についてはΘ’の各成分の平均値は０となる。したがって、θ_０ ＝０−０＝０となり、行列Θ’の各成分であるθ_ｔ’が行列Θの各成分θ_ｔ となる。本実施の形態で算出した各特徴点の位置の算出結果を表６に示す。
【００５５】
【表６】

【００５６】
以上のように、第２の実施の形態では、正規化部５０１により、カメラの設置条件を工夫することにより、カメラで撮影された特徴点のうち、少なくとも１つの特徴点について、回転中心からの距離及び方向が実測されていれば、全ての特徴点について回転中心からの距離及び方向を算出することができるので、少ない実測値でも精度の高い位置測定を行える。る。
【００５７】
実施形態３．
第１の実施の形態では、少なくとも２つの画像を撮影したときのカメラの向きが実測されている必要があった。本実施の形態では、実測された値が、少なくとも２つのカメラの向きの値ではなく、少なくとも２つの特徴点について回転中心からの方向が値として実測されている場合の位置測定方法について説明する。
【００５８】
本実施の形態の位置測定方法を実現する装置は、第１の実施の形態で説明した図１に示したものとほぼ同じである。ただ、正規化部５００の代わりに正規化部５０２を用いて実施する。正規化部５００は、少なくとも２つの画像を撮影したときに測定されたカメラの向きの値に基づいてｑを算出したが、正規化部５０２は、少なくとも２つの特徴点について測定された回転中心からの方向の値に基づいてｑを算出する点で正規化部５００とは異なる。
【００５９】
具体的な動作説明をする前に、本実施の形態の正規化部５０２の原理について説明する。ここで、少なくとも２つの特徴点の回転中心からの方向が実測されているものとする。（６）式に基づいて算出された特徴点の回転中心からの方向と、実測された特徴点の方向との差の二乗和が最小となるように、ｑ及びθ_０ を決定する。例えば、ｉ，ｊの２つの特徴点について、回転中心からの方向が実測されているとすると、次式（１６）によりｑを算出し、行列Θ’及び行列Ａを算出する。ここでα_ｉ’ 及びα_ｊ’ は行列分解部４００で算出されている。（１６）式により得られた行列Ａのｉ列目の成分α_１ に基づいて、前述した（１５）式によりθ_０ を決定する。
【００６０】
【数１９】

【００６１】
次に第１の実施の形態と動作が異なる正規化部５０２の動作について、具体的な数値に基づいて説明する。ここで、表１に基づいて特徴点１の方向φ_１ ＝０．０９９６７、特徴点７の方向φ_７ ＝０．０９９６７と実測されているものとする。（１６）式に基づいてｑを算出すると、次式（１７）のようになる。
【００６２】
【数２０】

【００６３】
このｑの値に基づいて、Θ”にｑをかけて行列Θ’を算出する。また行列Ａ’に１／ｑをかけて行列Ａを算出する。算出した行列Θ’の成分及び行列Ａの成分を表７に示す。
【００６４】
【表７】

【００６５】
ここで、行列Ａの第１番目の成分は１．６６５５０２であるので、これに基づいてθ_０ を算出すると、θ_０ ＝０．０９９６７＋（−１．８２６）／（１１×１．６６５５０２）＝０であるので、行列Θ’の各成分であるθ_ｔ’ が行列Θの各成分θ_ｔ となる。本実施の形態で算出した各特徴点の位置の算出結果を表８に示す。
【００６６】
【表８】

【００６７】
以上のように第３の実施の形態では、正規化部５０２により、カメラの設置条件を工夫することにより、カメラで撮影された特徴点のうち、少なくとも２つの特徴点について、回転中心からの方向が実測されていれば、全ての特徴点について回転中心からの距離及び方向を算出することができるので、少ない実測値でも精度の高い位置測定を行える。
【００６８】
実施形態４．
図６は特徴点が広範囲にわたって配置されている場合を示す図である。上述した第１〜３の実施の形態では、測定対象となる全ての特徴点が全ての画像に含まれている場合について説明した。
【００６９】
図７は、各画像に含まれる特徴点の関係例を表す図である。ここで、必ずしも全ての画像に全ての特徴点が含まれていない場合について考える。例えば、広範囲にわたって特徴点が配置されている場合、図１７のように、「○」で示された箇所しか特徴点が観測されない（含まれていない）場合がある。本実施の形態では、このような場合でも全ての特徴点について位置測定ができるように処理する方法について説明する。
【００７０】
図８は本発明の第４の実施の形態に係る位置測定方法を実現するための装置のブロック図である。図において、カメラ部１００及び特徴点追跡部２００は第１の実施の形態で説明したことと同様の動作を行うので、説明を省略する。また、座標行列作成部１０３００及び２０３００、行列分解部１０４００及び２０４００、座標算出部１０６００及び２０６００も、それぞれ入力されるデータに基づいて座標行列作成部３００、行列分解部４００、座標算出部６００と同様の動作を行うので説明を省略する。また、正規化部１０５００及び２０５００についても、第１〜第３の実施の形態で説明した正規化部５００、正規化部５０１又は正規化部５０２と同様の動作を行う。ただ、ここで正規化部２０５００については、実際の実測値ではなく、座標算出部１０６００又は２０６００により算出され、位置記憶部９００に記憶されたカメラの向き並びにカメラの回転中心からカメラまでの距離及び方向に基づいて行列Θ及び行列Ａを算出する。
【００７１】
図において、基本グループ作成部７００は、特徴点追跡部２００において各画像の特徴点の中からまず第１、第２又は第３の実施の形態（以下、第１〜第３の実施の形態という）で説明したような処理が行えるような画像及び特徴点の組合せ（以下、基本グループという）を１つ選択し、作成する。この基本グループについて、座標行列作成部１０３００、行列分解部１０４００、正規化部１０５００及び座標算出部１０６００は、第１〜第３の実施の形態で説明したような処理を行い、基本グループに含まれる特徴点の回転中心からの距離及び方向並びに各画像のカメラの向きを算出する。算出された特徴点の回転中心からの距離及び方向並びに各画像のカメラの向き（算出結果）は、位置記憶部９００に記憶される。
【００７２】
この基本グループの選択方法とは、少なくとも２枚の画像についてカメラの向きが実測されている場合、それらの画像のうち、少なくともある２枚の画像に共通して含まれている特徴点から１又は複数の特徴点を選択する。その選択した特徴点と選択した特徴点を全て含む画像とに関するデータの組合せを基本グループとする方法である（この基本グループでは、第１の実施の形態で説明した方法で処理ができる）。
【００７３】
また、少なくとも１つの特徴点について、カメラ回転中心からの距離及び方向が実測されている場合、それらの特徴点のうち、少なくともある１つの特徴点を共通して含む画像を複数選択する。その選択した画像と選択した画像全てに共通して含まれている特徴点に関するデータの組合せを基本グループとする方法もある（この基本グループでは、第２の実施の形態で説明した方法で処理ができる）。
【００７４】
さらに、少なくとも２つの特徴点について、カメラ回転中心からの方向が実測されている場合、それらの特徴点のうち、少なくともある２つの特徴点を共通して含む画像を複数選択する。その選択した画像と選択した画像全てに共通して含まれている特徴点に関するデータの組合せを基本グループとする方法もある（この基本グループでは、第３の実施の形態で説明した方法で処理ができる）。なお、この選択方法においては、処理に用いるデータ数が多くできるということから、例えば画像数と特徴点数の積が最大（座標行列作成部１０３００で作成される行列の成分数が最大）になるように基本グループを選択するのが一般的であると思われるが、特にこれに限定されるわけではない。
【００７５】
連鎖グループ作成部８００は、既に位置記憶部９００に算出結果が記憶された特徴点及び画像のうちの、少なくとも１つの特徴点又は少なくとも２つの画像を含み、まだ位置記憶部９００に算出結果が記憶されていない特徴点又は画像を含む組合せを、第１〜第３の実施の形態で説明したような処理が行えるように選択し、連鎖グループとして作成する。この連鎖グループについても、前述した基本グループの選択方法と同じ方法で選択し、作成する。連鎖グループの選択方法においても、処理に用いるデータ数が多くできるということから、例えば画像数と特徴点数の積が最大（座標行列作成部２０３００で作成される行列の成分数が最大）になるように連鎖グループを選択するのが一般的であると思われるが、特にこれに限定されるわけではない。
【００７６】
この連鎖グループについて、座標行列作成部２０３００、行列分解部２０４００、正規化部２０５００及び座標算出部２０６００は、第１〜第３の実施の形態で説明したような処理を行い、連鎖グループに含まれる特徴点の回転中心からの距離及び方向並びに各画像のカメラの向きを算出する。算出された特徴点の回転中心からの距離及び方向並びに各画像のカメラの向き（算出結果）は、位置記憶部９００に記憶される。この、連鎖グループ作成部８００の連鎖グループ作成から座標算出部２０６００の算出結果を位置記憶部９００に記憶するまでの処理は、全ての特徴点及び画像が何れかのグループに含まれる（全ての特徴点及び画像に関して算出結果が算出される）まで繰り返す。
【００７７】
例えば、既に位置記憶部９００に算出結果が記憶されている特徴点のうち、最も左端にある特徴点を、基本グループ又は既に計算結果が算出されている連鎖グループと共有している特徴点とする。この特徴点及びこれより左側にある、１又は複数の特徴点を選択し、選択した特徴点を全て含む複数の画像と選択した特徴点に関するデータの組合せを新たな連鎖グループとして処理を行う。これを、既に位置記憶部９００に算出結果が記憶されている特徴点のうち、最も右端にある特徴点についても行う。
【００７８】
次に具体的数値に基づいて、本実施の形態の装置の動作について説明する。ここでは、図１７において「○」で示された部分について、特徴点追跡部２００が各画像において各特徴点を観測したものとする。また、画像１と画像４を撮影したときのカメラの向きの値が実測されているものとする。
【００７９】
カメラ部１００のカメラが撮影した画像に基づいて、特徴点追跡部２００が各画像において各特徴点のカメラ回転方向の座標を探索する。基本グループ作成部７００は、まず、画像１と画像４でカメラの向きの値が実測されていることから、画像１と画像４に共通して含まれる特徴点を検出する。ここでは特徴点１、特徴点２、特徴点３及び特徴点４（以下、このような場合には特徴点１〜４のように記述する）が検出される。また検出した特徴点１〜４を全て含んでいる画像として、画像１及び画像４だけでなく、画像２及び画像３を検出する。このときの画像数と特徴点数との積は４×４＝１６である。ここで、特徴点３及び特徴点４を含んでいる画像として、画像１〜６を検出する等、様々な組合せが選択可能である。しかし、この場合でも画像数と特徴点数との積は２×６＝１２となり、特徴点は１〜４を選択した場合が、最も画素数と特徴点数との積が大きくなるので、ここでは特徴点１〜４、画像１〜４の組合せをを基本グループとして選択する。
【００８０】
この基本グループとして選択された特徴点１〜４、画像１〜４に関するデータに基づいて、座標行列作成部１０３００、行列分解部１０４００、正規化部１０５００及び座標算出部１０６００は、第１の実施の形態で説明した処理を行う。この結果、特徴点１〜４の回転中心からの距離及び方向、画像１〜４のカメラの向きが算出結果として位置記憶部９００に記憶される。この時点では、位置記憶部９００には、特徴点１〜４、画像１〜４に関するデータが記憶されていることになる。
【００８１】
連鎖グループ作成部８００は、まず、位置記憶部９００に記憶されている特徴点のうち、最も端にある特徴点１及び特徴点４を得る。ここで、特徴点１からはこれ以上の結果を出すことができないので、特徴点４だけを対象とする。特徴点４を含む特徴点５以降の中から、例えば特徴点４〜６を検出し、それらを全て含む画像として画像４〜７を検出する。このときの特徴点数と画像数との積は３×４＝１２となる。特徴点４を含み、それ以後の特徴点からの組合せの中で、特徴点数と画像数との積はこれが最も大きいので、これを連鎖グループ１とする。ここで、第３の実施の形態に基づいて処理を行うために、特徴点３及び特徴点４の回転中心からの方向により、特徴点３〜６、画像４〜６の選択する方法も考えられる。また新たな特徴点の算出結果は多くないが、第１の実施の形態に基づいて処理を行うために、画像３及び画像４のカメラ向きの値により、特徴点３〜５、画像３〜７の選択する方法も考えられる。
【００８２】
この連鎖グループとして選択した特徴点４〜６、画像４〜７に関するデータに基づいて、座標行列作成部２０３００、行列分解部２０４００、正規化部２０５００及び座標算出部２０６００は、第２の実施の形態で説明した処理を行う。正規化部２０５００では位置記憶部９００に記憶された特徴点４の中心からの距離及び方向を実測値として用いた処理を行う。この結果、特徴点５〜６の回転中心からの距離及び方向、画像５〜７のカメラの向きが新たな算出結果として位置記憶部９００に記憶される。この時点では、位置記憶部９００には、特徴点１〜６、画像１〜７に関するデータが記憶されていることになる。
【００８３】
ここで、特徴点７〜９（画像８及び画像９）に関しては、まだ算出結果が記憶されていないので、連鎖グループ作成部８００は再び連鎖グループを作成する。位置記憶部９００に記憶されている特徴点のうち、最も端にある特徴点６を得る。特徴点６を含む特徴点７以降の中から、例えば特徴点６〜９を検出し、それらを全て含む画像として画像７〜９を検出する。このときの特徴点数と画像数との積は４×３＝１２となる。特徴点６を含み、それ以後の特徴点からの組合せの中で、特徴点数と画像数との積はこれが最も大きいので、これを連鎖グループ２とする。この連鎖グループとして選択された特徴点６〜９、画像７〜９に関するデータに基づいて、座標行列作成部２０３００、行列分解部２０４００、正規化部２０５００及び座標算出部２０６００は、第２の実施の形態で説明した処理を行い、算出結果を位置記憶部９００に記憶する。以上で全ての特徴点についてカメラ回転中心からの距離及び方向が得られる。また全画像について、カメラの向きを得ることができる。
【００８４】
以上のように第４の実施の形態では、特徴点追跡部２００が検出した全ての特徴点が、全ての画像に含まれていなかった場合、基本グループ作成部７００が、あらかじめ測定された実測値に基づいて、第１〜第３の実施の形態により、特徴点の位置が測定できるような特徴点及び画像の組合せを基本グループとして作成し、座標行列作成部１０３００、行列分解部１０４００、正規化部１０５００及び座標算出部１０６００が画像を撮影したときのカメラの向きの値並びに中心から各特徴点までの距離及び方向を第１〜第３の実施の形態で説明したように算出して、位置記憶部９００に算出結果として記憶し、連鎖グループ作成部８００が、算出結果に基づいて、第１〜第３の実施の形態により、特徴点の位置が測定できるような特徴点及び画像の組合せを連鎖グループとして作成し、座標行列作成部２０３００、行列分解部２０４００、正規化部２０５００及び座標算出部２０６００が画像を撮影したときのカメラの向きの値並びに中心から各特徴点までの距離及び方向を第１〜第３の実施の形態で説明したように算出して、位置記憶部９００に算出結果として記憶し、これを全ての特徴点に対して回転中心からの距離及び方向が算出されるまで繰り返すようにしたので、全画像に、全特徴点が含まれていなくても、全ての特徴点に対して回転中心からの距離及び方向を算出することができ、全ての画像に対して、その画像を撮影したときのカメラの向きの値を算出することができる。
【００８５】
また第４の実施の形態によれば、実測値が、２枚以上の画像を撮影したときのカメラの向きの値であった場合に、基本グループ作成部７００は、カメラの向きの値が実測されている２枚の画像の両方に含まれる複数の特徴点を選択し、選択した複数の特徴点と選択した複数の特徴点を全て含む画像との組合せを基本グループとして作成するようにしたので、第１の実施の形態で説明した処理に基づいて、基本グループ内の全ての特徴点に対して回転中心からの距離及び方向を算出することができ、全ての画像に対して、その画像を撮影したときのカメラの向きの値を算出することができる。
【００８６】
また第４の実施の形態によれば、実測値が、少なくとも１つの特徴点の回転中心からの距離及び方向の値であった場合に、基本グループ作成部７００は、実測されている特徴点を含む複数の画像を選択し、選択した前記複数の画像の全てに含まれている特徴点と選択した複数の画像との組合せを基本グループとして作成するようにしたので、第２の実施の形態で説明した処理に基づいて、基本グループ内の全ての特徴点に対して回転中心からの距離及び方向を算出することができ、全ての画像に対して、その画像を撮影したときのカメラの向きの値を算出することができる。
【００８７】
また第４の実施の形態によれば、連鎖グループ作成部８００は、位置記憶部９００に記憶された特徴点のうち、位置的に最も端にある特徴点を検出し、その特徴点と位置記憶部９００に記憶されていない特徴点を全て含む画像を選択し、それらの特徴点と選択した画像との組合せを連鎖グループとして作成するようにしたので、第２の実施の形態で説明した処理に基づいて、連鎖グループ内の全ての特徴点に対して回転中心からの距離及び方向を算出することができ、全ての画像に対して、その画像を撮影したときのカメラの向きの値を算出することができる。
【００８８】
また第４の実施の形態によれば、選択した特徴点と画像との組合せのうち、画像数とその画像に含まれる特徴点の数との積が最も大きいものを基本グループ及び連鎖グループとして作成するようにしたので、より多くのデータ数に基づいて、基本グループ内又は連鎖グループ内の特徴点及び画像について算出結果を算出することができる。
【００８９】
実施形態５．
例えば、次に示す表９のように各画像での各特徴点のカメラ回転方向の座標が探索されているものとする。ここで、特徴点１及び特徴点７について、カメラ回転中心からの方向が実測されている場合、特徴点１と特徴点７とを同時に含む画像がないため、基本グループ及び連鎖グループを作成できず、第４の実施の形態に基づく位置測定は行えない。そこで、本実施の形態では、仮の実測値を設定した上で、第４の実施の形態で説明した処理を行い、実際の実測値と最もよく合う計算結果を最終的に算出結果とし、位置測定を行う方法について説明する。
【００９０】
【表９】

【００９１】
図９は本発明の第５の実施の形態に係る位置測定方法を実現するための装置のブロック図である。図において、カメラ部１００、特徴点追跡部２００、基本グループ作成部７００、連鎖グループ作成部８００、位置記憶部９００、座標行列作成部１０３００及び２０３００、行列分解部１０４００及び２０４００、座標算出部１０６００及び２０６００並びに正規化部２０５００は、第４の実施の形態で説明したことと同様の動作を行うので説明を省略する。正規化部１０５０１についても正規化部１０５００とほぼ同様の動作を行うが、実測値ではなく、仮カメラ向き提供部１０００から提供された２つの画像に割り当てられた１又は複数の仮のカメラ向きの値に基づいて、行列Θ及び行列Ａを算出する点で異なる。
【００９２】
画像選択部１３００は、仮のカメラ向きを設定するために２枚の画像を選択する。画像選択部１３００における画像の選択方法は、第１の実施の形態を行うことができるような２枚の画像が選択できればどのような方法を用いて選択してもよい。例えば、最初に選択した画像（最初の画像）を２枚のうちの１枚として選択する。その画像に含まれる特徴点から複数の特徴点を選択し、選択した特徴点を全て含む画像の中で最初の画像と最も離れた位置を撮影した画像を、もう１枚の画像として選択する。ここで複数の特徴点の選択は、選択した特徴点数とそれらの特徴点を全て含む画像との積が最大になるように選択する。
【００９３】
仮カメラ向き提供部１０００は、画像選択部１３００で選択された画像に対する１又は複数の仮のカメラ向きの値を算出し、正規化部１０５０１に送信する。仮カメラ向き提供部１０００は、実測されている特徴点の回転中心からの方向に基づいて適当な方法で仮のカメラ向きを予測し、算出する。例えば、実測されている２つの特徴点について、それぞれの特徴点が最も中心近くに撮影された画像のカメラの向きが、仮にその特徴点の実測された回転中心からの方向に一致するものとして、外挿又は内挿により、各特徴点が等距離間隔で分布しているものとして計算し、予測する。仮のカメラ向きを複数送信した場合は、それぞれについて位置測定処理を行い、それぞれの算出結果を出力することになる。
【００９４】
実測値比較部１１００は、２つの特徴点の回転中心からの方向の実測値と計算値とを比較する。その結果、必要な精度以下の誤差となる計算結果が算出されたか、又は仮カメラ向き修正部１２００で修正を行っても誤差が小さくならないと判断すると、その計算結果を最終的な算出結果とする。仮カメラ向き修正部１２００では、実測値比較部１１００での実測値と計算値との比較結果に基づいて、よりよい仮のカメラ向きを仮カメラ向き提供部１０００に設定し、再度計算を行わせる。例えば、２つの特徴点の回転中心からの方向が実測されている場合、それらの値をそれぞれφ_ｉ φ_ｊ とし、また算出された値をφ_ｃｉ、φ_ｃｊとする。その値が得られた仮カメラ向きに対し、φ_ｉ −φ_ｊ ／（φ_ｃｉ−φ_ｃｊ）をかけ、 （φ_ｊφ_ｃｉ−φ_ｉφ_ｃｊ） ／（φ_ｉ −φ_ｊ ）を加えたものを新たに修正された仮カメラ向きとして、仮カメラ向き提供部１０００に送信する。φ_ｉ −φ_ｊ ／（φ_ｃｉ−φ_ｃｊ）は実測値における２つのカメラ向きの角度差と計算結果におけるカメラ向きの角度差の比を表す。また、（φ_ｊφ_ｃｉ−φ_ｉφ_ｃｊ）／（φ_ｉ −φ_ｊ ）は、実測値と計算結果のカメラ向きの回転角の基準のずれを表す。
【００９５】
次に具体的数値に基づいて、本実施の形態の装置の動作について説明する。ここでは図５の対象物体について、カメラの視野角が第１の実施の形態１で使用したものより狭かったために、特徴点の追跡の結果が表１３のようになったものとして説明する。また、回転中心からの方向が実測されている特徴点は特徴点１と特徴点７とし、その実測値はそれぞれ０．００９９６７ｒａｄ及び−０．００９９６７ｒａｄであるとする。さらに、実測値比較部１１００は０．００１ｒａｄ以下の精度を目標とする。
【００９６】
カメラ部１００のカメラが撮影した画像に基づいて、特徴点追跡部２００が各画像において各特徴点のカメラ回転方向の座標を探索する。画像選択部１３００は、画像１を仮カメラ向きを設定する２枚の画像うちの１枚として選択する。画像１に含まれる特徴点４〜７を全て含んだ画像は画像１〜６の６枚で、特徴点数と画像数との積が４×６＝２４で最大であるため、特徴点４〜７を全て含み、画像１から最も離れた画像６を２枚目の画像として選択する。
【００９７】
仮カメラ向き提供部１０００は、仮のカメラ向きの値を算出する。特徴点１が最も中心近くで撮影されているのは画像１１である。そこで、仮に画像１１のカメラ向きを０．００９９６７とする。また、特徴点７が最も中心近くで撮影されているのは画像１である。そこで、仮に画像１のカメラ向きを−０．００９９６７とする。ここで、画像選択部１３００で選択された画像１は仮のカメラ向きをそのまま−０．００９９６７とする。画像６は、画像１と画像１１とに基づいて内挿して算出する。したがって、｛０．００９９６７−（−０．００９９６７）｝×（６−１）／（１１−１）＋（−０．００９９６７）＝０ｒａｄとし、これを画像６の仮のカメラ向きの値とする。仮カメラ向き提供部１０００は、正規化部１０５０１にこれらの値を送信する。
【００９８】
基本グループ作成部７００、連鎖グループ作成部８００、位置記憶部９００、座標行列作成部１０３００及び２０３００、行列分解部１０４００及び２０４００、正規化部１０５０１及び２０５００並びに座標算出部１０６００及び２０６００は、第４の実施の形態で説明したことと同様の動作を行い、全ての特徴点及び画像に関して算出結果を算出する。このとき、基本グループと連鎖グループは表９に波線で示したようにグループ分けする。これらの値に基づいて算出した計算結果を示したのが表１０の「１回目」の行である。計算によって特徴点１及び特徴点７についても回転中心からの方向の計算値が得られたが、その誤差はそれぞれ０．０１４８１５、０．０１４６１であり、実測値比較部１１００が目標値とする０．００１ｒａｄ以下の精度には達していない。
【００９９】
【表１０】

【０１００】
仮カメラ向き修正部１２００は、画像１と画像６との仮のカメラ向き−０．００９９６７、０に、｛０．００９９６７−（−０．００９９６７）｝／｛０．１１４４８４８−（−０．１１４２８０５）＝０．８７１３７をかける。また、｛０．１１４４８４８×（−０．００９９６７）−（−０．１１４２８０５）×０．００９９６７｝／｛０．１１４４８４８−（−０．１１４２８０５）｝＝−０．００００８９を加える。その結果、修正された画像１と画像６との仮のカメラ向きは、それぞれ−０．０８６９３８、−０．００００８９となる。この値を新たな仮のカメラ向きとして仮カメラ向き提供部１０００に送信し、再度位置測定処理を行う。この計算結果を示したのが表１０の「修正後」の行である。実測比較部１１００は計算結果と実測値とを比較する。特徴点１及び特徴点７共にその誤差は０．００１ｒａｄ以下なので、ここで処理を終了し、このときの計算結果を最終的な算出結果として出力する。本実施の形態で算出した各特徴点の位置の算出結果を表１１に示す。
【０１０１】
【表１１】

【０１０２】
以上のように第５の実施の形態によれば、画像選択部１３００が２枚の画像を選択して、それらの画像に基づいて仮カメラ向き提供部１０００が算出した仮のカメラ向きに基づいて、正規化部１０５０１に処理させて算出した算出結果に基づいて、実測値比較部１１００が、実測値に基づいて仮のカメラ向きを再度設定して再度計算を行うかどうかの判断を行い、再度計算を行うものと判断すると、カメラ向き修正部１２００が仮のカメラ向きを再度設定する計算を行い、再度位置測定処理を行って、最も精度のいい計算結果を算出結果として出力するようにしたので、たとえ、回転中心からの方向の値が実測された少なくとも２つの特徴点を同時に含む画像が存在しない場合でも、全ての特徴点について位置の測定を行うことができる。また、このため、実測する特徴点は、同一画面に含まれてなくてもよくなり、特徴点の選択に関しての制限が緩和される。また、カメラ向き修正部１２００において、仮のカメラ向きに対して（φ_ｉ −φ_ｊ ）／（φ_ｃｉ−φ_ｃｊ）を乗じ、 （φ_ｊφ_ｃｉ−φ_ｉφ_ｃｊ） ／（φ_ｉ −φ_ｊ ）を加えた値を、修正した仮のカメラ向きとするようにしたので、より精度の高い仮のカメラ向きを正規化部１０５０１に提供でき、精度の高い位置測定を行うことができる。
【０１０３】
実施形態６．
本実施の形態では、カメラ部１００のカメラを約１回転させ、約２πｒａｄ（３６０度）にわたって画像を撮影する場合を考える。最初の画像上で撮影されたある特徴点とカメラを約１回転させた時に撮影された別の画像上の同一の特徴点とを、２π離れた異なる特徴点として２つの特徴点の回転中心からの方向の実測値とする。そして、第５の実施の形態の実測値比較部１１００において、計算結果と比較するための実測値として用い、第５の実施の形態の位置測定処理を行うものである。
【０１０４】
図１０は本発明の第６の実施の形態に係る位置測定方法を実現するための装置のブロック図である。図において、同一点検出部１４００と実測値保持部１５００以外は、図９で同じ番号を付したものと同様の動作を行い、第５の実施の形態で既に説明したので省略する。同一点検出部１４００は、特徴点追跡部２００で探索された座標が特徴点のうち、例えば、最初の数枚の画像中に含まれる特徴点と、カメラを約１回転した後の数枚の画像中に含まれる特徴点とから同一の特徴点を検出する。この検出は、単純なパターンマッチングによりで同一の特徴点を探してもよいし、オペレータが同一の特徴点を指定してもよい。実測値保持部１５００は、同一点検出部１４００で同一の特徴点として検出された一組の特徴点のうち、最初の方の画像に含まれる特徴点の回転中心からの方向の実測値を０ｒａｄとする。また、カメラを約１回転した後の画像に含まれる特徴点の回転中心からの方向の実測値を２πｒａｄとして別の特徴点として扱い、実測値比較部１１００の比較対象とするための２つの実測値とする。
【０１０５】
図１１は第６の実施の形態における装置の動作を説明するための図である。図１１に基づいて、具体的数値を用いて本実施の形態の装置の動作について説明する。カメラ部１００のカメラを約１回転させ、約２πｒａｄ（３６０度）にわたって画像を撮影したものとする。ここで、図２４に示したようにカメラ位置１での画像を画像１とし、カメラ位置Ｔでの画像を画像Ｔとする。また、特徴点１は画像１では特徴点Ａ、画像Ｔでは特徴点Ｂとして検出されているものとする。
【０１０６】
例えば、オペレータが画像１及び画像Ｔに基づいて、特徴点Ａと特徴点Ｂが同一の特徴点であると同一点検出部１４００に指定すると、同一点検出部１４００は特徴点Ａと特徴点Ｂが同一の特徴点であるとのデータを実測値保持部１５００に送信する。実測値保持部１５０は、特徴点Ａ（画像１における特徴点１）の回転中心からの方向の実測値を０ｒａｄとする。また特徴点Ｂ（画像Ｔにおける特徴点１）の回転中心からの方向の実測値を２πｒａｄとし、特徴点Ｂを新たに特徴点（Ｎ＋１）として実測値比較部１１００に出力する。２つの特徴点の回転中心からの方向の実測値を得た実測値比較部１１００は、その実測値を用いて比較を行える。ここでは、特に精密な実測をしているわけではないが、第５の実施の形態で説明したように、仮のカメラ向きに基づいて位置測定処理を行うことができ、算出結果を得ることができる。
【０１０７】
以上のように第６の実施の形態によれば、カメラ部１００が約１周して撮影する場合に、同一点検出部が、ある特徴点について０ｒａｄを撮影した画像と２πｒａｄを撮影した画像とを検出し、実測値保持部１５００において、一方の特徴点の回転中心からの方向の実測値を０ｒａｄとし、もう一方の特徴点の回転中心からの方向の実測値を２πｒａｄとして別の特徴点として扱い、実測値比較部１１００において、比較対象の実測値とするようにしたので、特徴点を選択するだけで、０ｒａｄ、２πｒａｄの２つの特徴点の回転中心からの方向を得られるので、特に精密な実測を必要としなくても、全ての特徴点について回転中心からの距離及び方向を算出することができる。
【０１０８】
実施形態７．
上述の第１の実施の形態において、正規化部５００では、最小二乗和を用いてｑ及びθ_０ を算出することができる。例えば、第１の実施の形態では、３枚以上の画像でカメラの向きが実測されている場合、例えば最小二乗法等の一般的な方法で、θ_１’ とθ_１ とが最もよく一致するように最小二乗和によりｑを決定することができる。
【０１０９】
実施形態８．
また、上述の第２の実施の形態において、正規化部５０１では、最小二乗和を用いてｑを算出することができる。例えば２つ以上の特徴点で回転中心からの距離ｒ_ｉ 及び方向φ_ｉ が実測されている場合、実測されたそれぞれの距離ｒ_ｉ に対して、１／（１−Ｌ／ｒ_ｉ ）を算出して最小二乗和を算出し、これをｑとする。また、θ_０ は（１５）式で求める。
【０１１０】
実施形態９．
また、上述の第３の実施の形態において、正規化部５０２では、最小二乗和を用いてｑ及びθ_０ を算出することができる。例えば２つ以上の特徴点で回転中心からの方向φ_ｉ が実測されている場合、行列Θ”の成分をθ_ｔ” とし、行列Ａ’の成分をα_ｉ’ として、各特徴点について次式（１８）を算出し、この二乗和が最小になるようにｑ及びθ_０ を決定する。
【０１１１】
【数２１】

【０１１２】
実施形態１０．
上述の実施の形態は、各部を設けてそれぞれの算出を行ったが、カメラ部１００以外の各部については、実際にはコンピュータ、コンピュータに上述した実施の形態の動作を行わせるプログラム及びそのプログラムを記録する記録媒体によって実現するものである。
【０１１３】
【発明の効果】
以上のように本発明によれば、中心から一定距離にある円周上を回転移動して、円周外側画像を撮影するように条件を工夫して撮影した複数枚の画像について、全ての画像に含まれる特徴点を１又は複数検出して、各画像における各特徴点の回転方向の座標値を検出し、撮影したカメラの焦点距離で正規化して、また各特徴点について、正規化した値の平均を算出して差を算出して行列を作成し、ある２つの行列の積が、作成した行列に最も近くなるように、ある２つの行列を決定し、実測値に対応するようにある値及び一定値を算出し、一方の行列にある値をかけて一定値を加え、また他方の行列をある値で割り、それらの行列に基づいて、中心から各特徴点までの距離及び方向を算出するようにしたので、少ない実測値（例えば、１つの特徴点の回転中心からの距離と方向、２つの特徴点の回転中心からの方向、ある２つの画像を撮影したときのカメラの向き）でも精度よく、回転中心から全ての特徴点までの距離及び方向並びに各画像が撮影されたときのカメラの向きの値を算出することができる。
【０１１４】
また、本発明によれば、検出した全ての特徴点が、全ての画像に含まれていなかった場合でも、あらかじめ測定された実測値に基づいて、特徴点の位置が測定できるような特徴点及び画像の組合せを基本グループとして作成して、基本グループについてカメラの向きの値並びに中心から各特徴点までの距離及び方向を算出して記憶し、また基本グループで算出された算出結果を一部に含んだ特徴点又は画像の組合せを連鎖グループとして作成して、連鎖グループについてカメラの向きの値並びに中心から各特徴点までの距離及び方向を算出して記憶し、これを全ての特徴点について回転中心からの距離及び方向が算出されるまで行われるようにしたので、全画像に、全特徴点が含まれていなくても、全ての特徴点に対して回転中心からの距離及び方向を算出することができ、全ての画像に対して、その画像を撮影したときのカメラの向きの値を算出することができる。
【図面の簡単な説明】
【図１】本発明の第１の実施の形態に係る位置測定方法を実現するための装置のブロック図である。
【図２】カメラ部１００の設置状態例を表す図である。
【図３】本実施の形態で用いる近似について説明するための図である。
【図４】カメラ部１００上部から見たカメラ部１００と特徴点との関係を表す図である。
【図５】対象物体の特徴点と撮影状況を表す図である。
【図６】特徴点が広範囲にわたって配置されている場合を示す図である。
【図７】各画像に含まれる特徴点の関係例を表す図である。
【図８】本発明の第４の実施の形態に係る位置測定方法を実現するための装置のブロック図である。
【図９】本発明の第５の実施の形態に係る位置測定方法を実現するための装置のブロック図である。
【図１０】本発明の第６の実施の形態に係る位置測定方法を実現するための装置のブロック図である。
【図１１】第６の実施の形態における装置の動作を説明するための図である。
【符号の説明】
１００ カメラ部
２００ 特徴点追跡部
３００、１０３００、２０３００ 座標行列作成部
４００、１０４００、２０４００ 行列分解部
５００、５０１、５０２、１０５００、１０５０１、２０５００ 正規化部
６００、１０６００、２０６００ 座標算出部
７００ 基本グループ作成部
８００ 連鎖グループ作成部
９００ 位置記憶部
１０００ 仮カメラ向き提供部
１１００ 実測値比較部
１２００ 仮カメラ向き修正部
１３００ 画像選択部
１４００ 同一点検出部
１５００ 実測値保持部[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a method for calculating a distance and a direction to a target object using an image photographed by a camera.
[0002]
[Prior art]
Conventionally, as a method for extracting three-dimensional information from an image (two-dimensional) photographed by a camera, for example, the method shown in Fumiaki Tomita, “Stereo Vision for Intelligent Robots”, 22nd Image Engineering Conference, 1991 (Hereinafter referred to as a conventional example).
[0003]
In the conventional example, images are taken by a camera from two or more positions, and for the target objects reflected in the images taken at the respective positions, a position where a vector from the camera toward the target object intersects is calculated. The distance and direction are obtained.
[0004]
In the case of multi-eye measurement, the position of an object is measured from the position and orientation of each camera and the direction of the object viewed from each camera. Here, when there are three or more eyes, two of them are main, and after three eyes, it is mainly used for verifying the correspondence of the feature points rather than measuring the object position. In the case of moving vision, the camera is moved at a constant speed in parallel at a known speed, and the distance is obtained from the trajectory of the feature points.
[0005]
[Problems to be solved by the invention]
However, if the position is to be measured by a method as shown in the conventional example, it is necessary to accurately grasp the orientation and positional relationship of each camera when measuring with multiple eyes. Further, even if there are three or more cameras, the three or more cameras are not directly used for measuring the position of the object. In addition, in the case of moving vision, it is necessary to move the camera in parallel. For example, in order to measure the positions of objects distributed in a width of 5 m, a mechanism for moving the camera at a constant speed by a distance of 5 m is required. Become. Therefore, many measurement values with high accuracy are required for measurement by both methods.
[0006]
Therefore, it has been desired to realize an object position measurement method that can obtain the distance and direction from the image captured by the camera to the target object even if the measurement value is small.
[0007]
[Means for Solving the Problems]
The object position measuring method according to the present invention moves on a circumference at a certain distance from the center of rotation, and Multiple sheets The process of taking a picture of Multiple sheets A step of detecting one or a plurality of feature points included in all of the images, detecting a coordinate value of the movement direction of each feature point in each image, and photographing a coordinate value of the movement direction of each feature point in each image Normalized by the focal length based on the camera, and the average of the normalized coordinate values in multiple images is calculated for each feature point, and the difference from each normalized coordinate value is calculated, and the calculated difference value Corresponding to the step of creating a matrix having a component, the step of determining two matrices so that the product of the two matrices is the closest to the created matrix, and the measured values measured in advance In this way, a certain value and a constant value are calculated, one matrix of two matrices is multiplied by a certain value, a certain value is added, and the other matrix of the two matrices is divided by a certain value, Based on the distance and direction from the center to each feature point And a step of calculating.
[0008]
Further, in the object position measurement method according to the present invention, when all of the detected feature points are not included in all of the plurality of images, an image is taken based on the actually measured values. A step of creating a combination of feature points and images capable of calculating a camera orientation value and a distance and direction from the center to each feature point as a basic group, and a combination of feature points and images included in the basic group, The camera orientation value when the image is taken and the distance and direction from the center to each feature point are calculated, and the calculation result is stored. Based on the stored calculation result, the camera when the image is taken Creating a combination of feature points and images capable of calculating the orientation value and the distance and direction from the center to each feature point as a chain group, and the feature points included in the chain group For the combination of images, the camera orientation value when the image was taken and the distance and direction from the center to each feature point are calculated, the calculation result is stored, and each feature from the center for all feature points Creating a chained group until the distance and direction to the point are calculated, calculating the camera orientation value and the distance and direction from the center to each feature point when the image was taken, and storing the calculation results; have.
[0009]
The object position measuring apparatus according to the present invention includes a camera unit that moves on a circumference at a certain distance L from the center of rotation and captures T images outside the circumference; and A feature point tracking unit that detects N feature points included in all and detects a coordinate value in the moving direction of each feature point i in each image t, and a coordinate value in the moving direction of each feature point i in each image t Is normalized by the focal length based on the camera part that was shot _ti (T = 1, 2,..., T, i = 1, 2,..., N), and
[Equation 8]

A coordinate matrix creation unit that creates a matrix U ′ of T rows and N columns, and a matrix Θ ″ and a matrix A ′ of 1 row N columns and the product of the matrix Θ ″ and the matrix A ′ A matrix decomposition unit for determining the matrix Θ ″ and the matrix A ′ so that the matrix U ′ can be regarded as a matrix U ′, a certain value q and a constant value θ so as to correspond to the actually measured values measured in advance. ₀ And multiply the matrix Θ "by q ₀ And a normalization unit that calculates a matrix Θ whose component is the orientation of the camera unit when T images are captured, and calculates the matrix A by dividing the matrix A ′ by q, and a normalization unit The component of the matrix Θ calculated by _t And each component of matrix A is α _i The distance r from the center of rotation to the feature point _i R _i = L / (1-1 / α _i ) And the direction φ from the center of rotation to the feature point _i The
[Equation 9]

And a coordinate calculation unit for calculating in (1).
[0010]
Further, in the object position measuring apparatus according to the present invention, when all the feature points detected by the feature point tracking unit are not included in all of the plurality of images, based on actually measured values, A combination of feature points and images that can calculate the camera orientation value and the distance and direction from the center to each feature point when the image was taken is created as a basic group, coordinate matrix creation unit, matrix decomposition unit, normalization A basic group creation unit that calculates a camera orientation value when an image is captured and a distance and direction from the center to each feature point, and a calculated camera orientation value and A position storage unit that stores the distance and direction from the center to each feature point as a calculation result, and the camera orientation value and the center when the image is captured based on the calculation result stored in the position storage unit A combination of feature points and images for which distances and directions to each feature point can be calculated as a chain group, and a camera when images are taken in the coordinate matrix creation unit, matrix decomposition unit, normalization unit, and coordinate calculation unit The orientation value and the distance and direction from the center to each feature point are calculated and stored in the position storage unit as calculation results, and the camera orientation values when all images are taken are calculated. A chain group creating unit that creates a chain group until the distance and direction from the center to each feature point with respect to the feature point are calculated.
[0011]
Further, the recording medium on which the program of the object position measuring method according to the present invention is recorded moves on the circumference at a certain distance from the center of rotation, and moves outside the circumference. Multiple sheets Let the camera take a picture of Multiple sheets A camera that detects one or a plurality of feature points included in all of the images, detects the coordinate value of each feature point in each image in the moving direction, and captures the coordinate value of each feature point in each image in the moving direction Normalized by the focal length based on the image, and the average of the normalized coordinate values in a plurality of images is calculated for each feature point, and the difference from each normalized coordinate value is calculated. A matrix is generated as a component, and two matrixes are determined so that the product of the two matrices is the closest to the created matrix, and a certain value corresponding to an actual measurement value measured in advance Calculate a constant value, multiply one matrix of one of the two matrices by a certain value, add the constant value, and divide the other matrix of the two matrices by a certain value, and based on those matrices, Distance and direction from the center to each feature point It is obtained by recording a program for causing a computer to calculate.
[0012]
Further, the recording medium on which the program of the object position measuring method according to the present invention is recorded, when not all the detected feature points are included in all of the plurality of images, the measured value measured in advance. Based on the orientation value of the camera when the image was taken and the combination of the feature point and the image capable of calculating the distance and direction from the center to each feature point as a basic group, and the feature points included in the basic group and For the combination of images, when the camera orientation value and the distance and direction from the center to each feature point are calculated, the calculation result is stored, and the image is shot based on the calculation result A combination of feature points and images that can calculate the camera orientation value and the distance and direction from the center to each feature point is created as a chain group and included in the chain group. For the combination of feature points and images, the camera orientation value when the image was taken and the distance and direction from the center to each feature point are calculated, and the calculation results are stored. Create a chain group until the distance and direction to each feature point are calculated, calculate the value of the camera orientation when shooting the image and the distance and direction from the center to each feature point, and calculate the results. This is a recording of a program that causes a computer to perform storage.
[0013]
DETAILED DESCRIPTION OF THE INVENTION
Embodiment 1. FIG.
FIG. 1 is a block diagram of an apparatus for realizing a position measuring method according to a first embodiment of the present invention. In FIG. 1, 100 is a camera unit, 200 is a feature point tracking unit, 300 is a coordinate matrix creation unit, 400 is a matrix decomposition unit, 500 is a normalization unit, and 600 is a coordinate calculation unit. Before describing the specific configuration and operation, the principle of position measurement according to this embodiment will be described. In the present embodiment, when the value of the camera orientation is actually measured for at least two of the captured images,
The distance and direction from the center of rotation to a feature point (for example, a point that is characteristic to represent the shape of the target object) are calculated.
[0014]
FIG. 2 is a diagram illustrating an installation state example of the camera unit 100. In the present embodiment, as shown in FIG. 2, the camera that is the photographing part is installed so as to move on the circumference of a certain radius L and photograph the outside of the circumference. Here, in the present embodiment, it is assumed that feature points are taken in all the images to be taken.
[0015]
FIG. 3 is a diagram for explaining the approximation used in the present embodiment. Here, the feature point is a radius r from the rotation center of the camera unit 100. _i It is on the upper circle. When the feature point is sufficiently close to the front of the camera, the feature point is the distance r on the extension line of the line connecting the rotation center and the camera front. _i Perpendicular to the extension line from the point of _i Even if it is approximated to a point that has just moved, there is no significant error. Therefore, this approximation is used in the present embodiment. The present invention obtains the distance and direction from the center of rotation to the feature point from a plurality of images in which the camera has different orientations and a small number of actual measurement values by means of camera installation conditions and approximation.
[0016]
FIG. 4 is a diagram illustrating the relationship between the camera unit 100 and the feature points as viewed from above the camera unit 100. In FIG. 4, each relationship between the camera unit and the feature points is represented by parameters. When the lateral position of the feature point on the photographed image is normalized by the focal length having a characteristic value from the camera, the normalized value is expressed by the following equation (1).
u _ti = X _ti / Z _ti = Α _i θ _t + Β _i ... (1)
Where α _i = 1 / (1-L / r _i ), Β _i = Φ _i / (1-L / r _i )
[0017]
What is obtained from the actually captured image is u in the equation (1). _ti And this u _ti Based on r _i , Φ _i Is the purpose. Here, i is a number assigned to each feature point. T is a number assigned to each photographed image.
[0018]
U in equation (1) _ti The average value of all feature points (i = 1, 2,... N: N is a natural number) in all images (t = 1, 2,... T: T is a natural number) is expressed by the following equation (2). . In equation (2), all values are treated equally and calculated as a simple average value. However, in some cases, the values may be calculated by weighting.
[0019]
[Expression 10]

[0020]
When the value of the formula (1) is subtracted from the average value calculated by the formula (2), the following formula (3) is obtained.
[0021]
[Expression 11]

[0022]
This all u _ti When expressed in a matrix, it is expressed by U ′ = Θ′A as in the following equation (4). Here, U ′ is calculated only from the position of the feature point obtained from the actually captured image, so the left side of this equation can be obtained by performing some image processing on the image. Therefore, based on U ′ obtained by image processing, Θ ′ and A are calculated under certain conditions so that Expression (4) is established. Then, based on Θ ′ and A, the distance r to the feature point _i And the orientation of the camera when each image is taken. From equation (2), β _i So that β _i Is substituted into equation (1) and φ _i Can be obtained.
[0023]
[Expression 12]

[0024]
Next, a specific configuration of the apparatus shown in FIG. 1 will be described. As shown in FIG. 2, the camera unit 100 is installed so as to photograph outward on the circumference of the radius L, and captures and outputs an image of an arbitrary point on the circumference. The feature point tracking unit 200 searches the coordinates of the rotation direction of the camera for one or more feature points for which the distance is to be obtained in each image captured by the camera 100 unit. The feature point tracking unit 200 only needs to be able to associate each feature point on each image, and a specific mark attached to the target object can be identified by a generally known method such as pattern matching (pattern matching). You may search. Further, even when no mark is attached, a search may be performed using a generally known image processing method such as an optical flow.
[0025]
The coordinate matrix creation unit 300 normalizes the horizontal coordinates (camera rotation direction coordinates) with the focal length of the feature point i in the image t searched by the feature point tracking unit 200 with the image center as the origin. _ti Is calculated. And u based on equation (3) _ti 'Is calculated and matrix U' is calculated. Therefore, if N feature points are searched on each image and T images are taken, the matrix U ′ is a matrix of T rows and N columns.
[0026]
The matrix decomposition unit 400 calculates the matrix Θ ″ and the matrix A ′ so that the product of the matrix Θ ″ of T rows and 1 column and the matrix A ′ of 1 row and N columns is close to the matrix U ′. The calculation of the matrix Θ ″ and the matrix A ′ is realized, for example, as follows. If the above approximation is sufficiently established, the singular value of U ′ can be approximated to 0 except for the maximum singular value k. Therefore, the singular value decomposition can be performed with U ′ = Θ “kA”. Here, the matrix A ′ is calculated as A ′ = kA ″, and the calculation of the matrix Θ ″ and the matrix A ′ is realized.
[0027]
In the normalization unit 500, when the camera orientation value is measured for at least two of the captured images, the difference in camera orientation is closest to the value measured in advance. The matrix Θ ″ is multiplied by q, and the matrix A ′ is multiplied by 1 / q to obtain the matrix Θ ′ and the matrix A, respectively. The q is obtained by multiplying the matrix Θ ″ by q and the matrix A ′ by 1 / q. Since they cancel each other, they do not appear in the matrix U ′. In addition, for the matrix Θ ′, each θ, which is a component of Θ ′, is set so that the value of the camera orientation at the time of shooting becomes a value measured in advance. _t 'Is a constant value θ ₀ Plus θ _t (Component of matrix Θ) is obtained. Where q and θ ₀ Is an adjustment value for adjusting so as to reduce the difference between the actually measured value and the calculated matrix component corresponding thereto. In the coordinate calculation unit 600, each component of A is α _i To the distance r from the camera rotation center of each feature point based on the following equation (5) _i Is calculated. Further, based on the following equation (6), the position φ of each feature point _i Is calculated.
[0028]
[Formula 13]

[0029]
FIG. 5 is a diagram illustrating feature points of the target object and shooting conditions. Next, the operation of the apparatus in the present embodiment will be described based on specific numerical values. The distance L from the camera rotation center to the camera will be described as 0.6 m. The camera part of the camera unit 100 is rotated around the camera rotation center. At this time, the measured values of the distance and direction of each feature point are shown in Table 1 (the purpose of this embodiment is to calculate a value that is as close as possible to this value).
[0030]
[Table 1]

[0031]
The feature point tracking unit 200 detects the position of each feature point in each image by a general method such as pattern matching or optical flow. In this case, for each image, a value u obtained by normalizing the position of each feature point in the rotation direction of the camera (in the installation shown in FIG. 2, the camera rotates horizontally (lateral direction)) with the focal length of the camera. _ti Is shown in Table 2 below.
[0032]
[Table 2]

[0033]
The coordinate matrix creation unit 300 calculates the average of coordinate values (distances) derived from each image for each feature point, and derives a matrix that calculates the difference between each coordinate value and each coordinate. For example, in the case of feature point 1, the average value of each coordinate from t = 1 to 11 is −0.166. This is subtracted from each coordinate. When t = 1 and i = 1, −0.309 − (− 0.166) = − 0.143, and the first row and first column components of the matrix in which this is created It becomes. The matrix calculated in this way is shown in the following equation (7).
[0034]
[Expression 14]

[0035]
The matrix decomposition unit 400 decomposes the matrix of equation (7) into a matrix Θ ″ and a matrix A ′. Specifically, the singular value decomposition approximated to 0 except for the maximum singular value in the procedure illustrated below. First, the product of the matrix of equation (7) and its transposed matrix is calculated, which is shown in the following equation (8).
[0036]
[Expression 15]

[0037]
Next, the maximum eigenvalue of this matrix and its eigenvector are obtained. This is shown in the following equation (9). As a method for calculating eigenvalues and eigenvectors, many methods such as a power method are generally known.
[0038]
[Expression 16]

[0039]
The product of the matrix of equation (7) and this eigenvector divided by the square root of the eigenvalue is defined as a matrix Θ ″. The product of the square root of this eigenvalue and the eigenvector is defined as a matrix A ′. The matrix A ′ is shown in the following equations (10) and (11).
[0040]
[Expression 17]

[0041]
Here, for example, it is assumed that the camera orientations when images of t = 1 and t = 11 are measured are −0.8727 rad and 0.8727 rad, respectively. The normalizing unit 500 calculates q based on the following equation (12), and calculates q by Θ ″ to calculate the matrix Θ ′. Also, the matrix A ′ is calculated by multiplying 1 / q by the matrix A ′. Table 3 shows the components of the matrix Θ ′ and the matrix A.
q = {0.08727 − (− 0.08727)} / (θ ₁₁ "-Θ ₁ ") ... (12)
[0042]
[Table 3]

[0043]
Furthermore, usually θ ₁ 'Is adjusted to the actual measurement value -0.8727 rad, and all the θ _t Add a constant value to ', but in this example, θ ₁ 'Is θ ₁ Is already in agreement with θ _i 'Is the same as θ _i It becomes.
[0044]
In the coordinate calculation unit 600, for example, for the feature point 1, the distance r from the camera focus to the feature point 1 as shown in the following equation (13) based on the equation (1). ₁ Is calculated. Similarly, based on the equation (1), β ₁ = −0.166−1.648325 × 0 = −0.166, φ ₁ = −0.166 can be calculated. Table 4 shows the calculation results of other feature points calculated in this way. Comparing these values with the actually measured values in Table 1, it can be seen that the calculated values are close to the actually measured values.
r ₁ = 0.6 / (1-1 / 1.648325) = 1.525462 ... (13)
[0045]
[Table 4]

[0046]
As described above, according to the first embodiment, the T taken by the camera unit 100 devised to rotate and move on the circumference at a certain distance from the center and take an image outside the circumference. For each image, the feature point tracking unit 200 detects N feature points included in all the images, detects the coordinate values in the rotation direction of each feature point in each image, and the coordinate matrix creation unit 300 The coordinate value is normalized with the focal length peculiar to the camera unit 100 that captured the image, and u _ti And for each feature point, u of the T images _ti Calculate the average of u _ti And a matrix U ′ having the calculated difference value as a component is created, and the matrix decomposing unit 400 generates a matrix Θ ″ of T rows and 1 column and a matrix A ′ of 1 row and N columns. The matrix Θ ″ and the matrix A ′ are determined so that the product of the two matrices becomes a matrix closest to the matrix U ′, and the value of the camera orientation when at least two images are captured is actually measured. In this case, the normalization unit 500 multiplies the value q in the matrix Θ ″ so as to correspond to the actual measurement value to obtain a constant value θ ₀ Is added to calculate the matrix Θ. This matrix Θ represents the value of the direction of the camera when an image is taken. Further, the matrix A ′ is divided by a certain value q to calculate the matrix A, and the coordinate calculation unit calculates the distance and direction from the center to each feature point based on the matrix A, so at least 2 Just measure the camera orientation when the images were taken, and accurately measure the distance and direction from the center of rotation to the feature point from the few measured values and the camera orientation when each image was taken. Can be calculated. Further, the matrix decomposition unit 400 approximates a singular value other than the maximum singular value in the matrix U ′ generated by the coordinate matrix generation unit 300 to 0, performs singular value decomposition, and performs a matrix Θ ″ and 1 of T rows and 1 column. Since two matrices, the matrix A ′ of row N columns, are determined, when the approximation shown in FIG. 3 is sufficiently established, the calculation can be performed quickly and effectively.
[0047]
Embodiment 2. FIG.
In the first embodiment, it is necessary to actually measure the orientation value of the camera when at least two images are taken. In the present embodiment, a position measurement method in the case where the measured value is not the value of the orientation of at least two cameras but the value of the distance from the rotation center and the value of the direction is measured for at least one feature point will be described. .
[0048]
The apparatus for realizing the position measurement method of the present embodiment is almost the same as that shown in FIG. 1 described in the first embodiment. However, the normalization unit 501 is used instead of the normalization unit 500. The normalization unit 500 calculates q based on the orientation of the camera measured when at least two images are taken, but the normalization unit 501 determines the distance from the rotation center measured for at least one feature point. And it differs from the normalization part 500 by the point which calculates q based on a direction.
[0049]
Before describing the specific operation, the principle of the normalization unit 501 of this embodiment will be described. Here, it is assumed that the distance and direction from the rotation center of at least one feature point are actually measured. Here, the value of q described in the first embodiment is determined so that the value calculated from the actually measured distance is closest to the component of the matrix A corresponding thereto. For example, when the position (distance and direction from the rotation center) is measured for one feature point, the ratio of the corresponding component of A ′ to the distance is q. Then, in the same manner as in the first embodiment, the matrix Θ ″ is multiplied by q to calculate the matrix Θ ′. Further, the matrix A ′ is multiplied by 1 / q to calculate the matrix A.
[0050]
Further, each θ which is a component of Θ ′ is obtained so as to obtain the following equation (14) in which the actually measured direction and the direction calculated by the equation (6) are best matched. _t 'Is a constant value θ ₀ And θ _t Get. For example, if there is one measured feature point, θ ₀ Is calculated.
[0051]
[Expression 18]

[0052]
Next, the operation of the normalization unit 501 that is different in operation from the first embodiment will be described based on specific numerical values. Here, the distance r of the feature point 4 based on Table 1 ₄ Is 2, direction φ ₄ Is actually measured to be 0. In this case, based on the equation (1), α ₄ = 1 / (1-0.6 / 2) = 1.428571. Since the fourth column component in the matrix A ′ calculated by the matrix decomposing unit 400 is 0.262202, the value of q is q = 0.262202 / 1.428571 = 0.183541. Based on this q value, Θ ″ is multiplied by q to calculate matrix Θ ′. Also, matrix A ′ is multiplied by 1 / q to calculate matrix A. The calculated components of matrix Θ ′ and matrix A The ingredients are shown in Table 5.
[0053]
[Table 5]

[0054]
Then φ ₄ Since 0, 0 + (− 0.125 + (− 0.1) + (− 0.075) + (− 0.05) + (−) in the first term and the second term on the right side of equation (9). −0.025) + 0 + 0.025 + 0.05 + 0.075 + 0.1 + 0.125) = 0. For the third term, the average value of each component of Θ ′ is zero. Therefore, θ ₀ = 0−0 = 0, and each component of the matrix Θ ′ is θ _t 'Is each component θ of matrix Θ _t It becomes. Table 6 shows the calculation result of the position of each feature point calculated in the present embodiment.
[0055]
[Table 6]

[0056]
As described above, in the second embodiment, by devising the camera installation conditions by the normalizing unit 501, at least one of the feature points photographed by the camera is detected from the rotation center. If the distance and direction are actually measured, the distance and direction from the center of rotation can be calculated for all the feature points, so that highly accurate position measurement can be performed even with a small number of actually measured values. The
[0057]
Embodiment 3. FIG.
In the first embodiment, it is necessary to actually measure the orientation of the camera when at least two images are taken. In the present embodiment, a position measurement method in the case where the actually measured values are not the values of the orientation of at least two cameras but the directions from the center of rotation are measured as values for at least two feature points will be described.
[0058]
The apparatus for realizing the position measurement method of the present embodiment is almost the same as that shown in FIG. 1 described in the first embodiment. However, the normalization unit 502 is used instead of the normalization unit 500. The normalization unit 500 calculates q based on the value of the camera orientation measured when at least two images are taken, but the normalization unit 502 uses the rotation center measured for at least two feature points. This is different from the normalization unit 500 in that q is calculated based on the value of the direction.
[0059]
Before describing the specific operation, the principle of the normalization unit 502 of the present embodiment will be described. Here, it is assumed that the directions from the rotation center of at least two feature points are actually measured. Q and θ so that the sum of squares of the difference between the direction from the rotation center of the feature point calculated based on the equation (6) and the direction of the measured feature point is minimized. ₀ To decide. For example, assuming that the direction from the rotation center is actually measured for two feature points i and j, q is calculated by the following equation (16), and the matrix Θ ′ and the matrix A are calculated. Where α _i 'And α _j 'Is calculated by the matrix decomposition unit 400. The component α of the i-th column of the matrix A obtained by the equation (16) ₁ On the basis of the above equation (15) ₀ To decide.
[0060]
[Equation 19]

[0061]
Next, the operation of the normalization unit 502 whose operation is different from that of the first embodiment will be described based on specific numerical values. Here, the direction φ of the feature point 1 based on Table 1 ₁ = 0.09967, direction φ of feature point 7 ₇ Suppose that it is actually measured as = 0.09967. When q is calculated based on the equation (16), the following equation (17) is obtained.
[0062]
[Expression 20]

[0063]
Based on this q value, Θ ″ is multiplied by q to calculate matrix Θ ′. Also, matrix A ′ is multiplied by 1 / q to calculate matrix A. The calculated components of matrix Θ ′ and matrix A The ingredients are shown in Table 7.
[0064]
[Table 7]

[0065]
Here, since the first component of the matrix A is 1.665502, based on this, θ ₀ Is calculated as θ ₀ = 0.09967 + (− 1.826) / (11 × 1.665502) = 0, so each component of the matrix Θ ′ is θ _t 'Is each component θ of matrix Θ _t It becomes. Table 8 shows the calculation result of the position of each feature point calculated in the present embodiment.
[0066]
[Table 8]

[0067]
As described above, in the third embodiment, the normalization unit 502 devises the installation conditions of the camera, so that at least two of the feature points photographed by the camera are directions from the rotation center. Since the distance and direction from the center of rotation can be calculated for all the feature points, highly accurate position measurement can be performed even with a small number of actually measured values.
[0068]
Embodiment 4 FIG.
FIG. 6 is a diagram illustrating a case where feature points are arranged over a wide range. In the first to third embodiments described above, a case has been described in which all feature points to be measured are included in all images.
[0069]
FIG. 7 is a diagram illustrating a relationship example of feature points included in each image. Here, a case is considered in which not all feature points are necessarily included in all images. For example, when feature points are arranged over a wide range, the feature points may be observed (not included) only at the locations indicated by “◯” as shown in FIG. In the present embodiment, a description will be given of a method of processing so that position measurement can be performed for all feature points even in such a case.
[0070]
FIG. 8 is a block diagram of an apparatus for realizing a position measuring method according to the fourth embodiment of the present invention. In the figure, the camera unit 100 and the feature point tracking unit 200 perform the same operations as those described in the first embodiment, and thus description thereof is omitted. The coordinate matrix creation units 10300 and 20300, the matrix decomposition units 10400 and 20400, and the coordinate calculation units 10600 and 20600 are also similar to the coordinate matrix creation unit 300, the matrix decomposition unit 400, and the coordinate calculation unit 600 based on the input data, respectively. The description of the operation will be omitted. Further, the normalization units 10500 and 20500 also perform the same operation as the normalization unit 500, normalization unit 501, or normalization unit 502 described in the first to third embodiments. However, the normalization unit 20500 is not an actual actual measurement value, but is calculated by the coordinate calculation unit 10600 or 20600 and stored in the position storage unit 900, and the distance from the rotation center of the camera to the camera. Matrix Θ and matrix A are calculated based on the direction.
[0071]
In the figure, a basic group creation unit 700 first uses the first, second or third embodiment (hereinafter referred to as first to third embodiments) from the feature points of each image in the feature point tracking unit 200. A combination of images and feature points (hereinafter referred to as a basic group) that can perform the processing described in (1) is selected and created. For this basic group, the coordinate matrix creation unit 10300, the matrix decomposition unit 10400, the normalization unit 10500, and the coordinate calculation unit 10600 perform processing as described in the first to third embodiments and are included in the basic group. The distance and direction from the rotation center of the feature point and the camera direction of each image are calculated. The calculated distance and direction from the rotation center of the feature point and the camera orientation (calculation result) of each image are stored in the position storage unit 900.
[0072]
This method of selecting a basic group means that when the camera orientation is actually measured for at least two images, one or more of the feature points that are commonly included in at least two of the images is selected. Select multiple feature points. In this method, a combination of data relating to the selected feature point and an image including all of the selected feature points is used as a basic group (this basic group can be processed by the method described in the first embodiment).
[0073]
Further, when at least one feature point is actually measured for the distance and direction from the camera rotation center, a plurality of images that include at least one feature point in common among those feature points are selected. There is also a method in which a combination of data relating to the selected image and feature points included in all selected images is used as a basic group (in this basic group, processing is performed by the method described in the second embodiment). it can).
[0074]
Furthermore, when the direction from the camera rotation center is actually measured for at least two feature points, a plurality of images that include at least two feature points in common among those feature points are selected. There is also a method in which a combination of data relating to the selected image and feature points commonly included in all the selected images is used as a basic group (in this basic group, processing is performed by the method described in the third embodiment). it can). In this selection method, since the number of data used for processing can be increased, for example, the product of the number of images and the number of feature points is maximized (the number of matrix components created by the coordinate matrix creation unit 10300 is maximized). It is common to select a basic group, but this is not a limitation.
[0075]
The chain group creation unit 800 includes at least one feature point or at least two images among the feature points and images whose calculation results are already stored in the position storage unit 900, and the calculation results are still stored in the position storage unit 900. A combination including a feature point or an image that has not been performed is selected so that the processing described in the first to third embodiments can be performed, and is created as a chain group. This chain group is selected and created in the same manner as the basic group selection method described above. Also in the chain group selection method, since the number of data used for processing can be increased, for example, the product of the number of images and the number of feature points is maximized (the number of matrix components created by the coordinate matrix creation unit 20300 is maximized). Although it is common to select a linkage group, it is not limited to this.
[0076]
For this chain group, the coordinate matrix creation unit 20300, the matrix decomposition unit 20400, the normalization unit 20500, and the coordinate calculation unit 20600 perform the processes described in the first to third embodiments and are included in the chain group. The distance and direction from the rotation center of the feature point and the camera direction of each image are calculated. The calculated distance and direction from the rotation center of the feature point and the camera orientation (calculation result) of each image are stored in the position storage unit 900. The processing from the chain group creation by the chain group creation unit 800 to the calculation result of the coordinate calculation unit 20600 stored in the position storage unit 900 includes all feature points and images (all features). Repeat until the calculation result is calculated for points and images).
[0077]
For example, among the feature points whose calculation results are already stored in the position storage unit 900, the leftmost feature point is set as a feature point shared with the basic group or the chain group whose calculation result has already been calculated. . This feature point and one or more feature points on the left side thereof are selected, and a combination of a plurality of images including all the selected feature points and data relating to the selected feature point is processed as a new chain group. This is performed for the feature point at the right end among the feature points whose calculation results are already stored in the position storage unit 900.
[0078]
Next, the operation of the apparatus of the present embodiment will be described based on specific numerical values. Here, it is assumed that the feature point tracking unit 200 observes each feature point in each image for the part indicated by “◯” in FIG. Further, it is assumed that the value of the camera orientation when the images 1 and 4 are taken is actually measured.
[0079]
Based on the image captured by the camera of the camera unit 100, the feature point tracking unit 200 searches the coordinates of each feature point in the camera rotation direction in each image. First, since the camera orientation values are actually measured in images 1 and 4, basic group creating section 700 detects feature points that are included in images 1 and 4 in common. Here, feature point 1, feature point 2, feature point 3, and feature point 4 (hereinafter referred to as feature points 1 to 4 in such a case) are detected. Further, not only the image 1 and the image 4 but also the image 2 and the image 3 are detected as images including all the detected feature points 1 to 4. The product of the number of images and the number of feature points at this time is 4 × 4 = 16. Here, as the image including the feature point 3 and the feature point 4, various combinations such as detecting the images 1 to 6 can be selected. However, even in this case, the product of the number of images and the number of feature points is 2 × 6 = 12, and when the feature points of 1 to 4 are selected, the product of the number of pixels and the number of feature points is the largest. A combination of points 1 to 4 and images 1 to 4 is selected as a basic group.
[0080]
Based on the data relating to the feature points 1 to 4 and the images 1 to 4 selected as the basic group, the coordinate matrix creation unit 10300, the matrix decomposition unit 10400, the normalization unit 10500, and the coordinate calculation unit 10600 perform the first embodiment. The processing described in the form is performed. As a result, the distance and direction from the rotation center of the feature points 1 to 4 and the camera direction of the images 1 to 4 are stored in the position storage unit 900 as calculation results. At this time, the position storage unit 900 stores data relating to the feature points 1 to 4 and the images 1 to 4.
[0081]
First, the chain group creation unit 800 obtains the feature points 1 and 4 at the extreme ends among the feature points stored in the position storage unit 900. Here, since no more results can be obtained from the feature point 1, only the feature point 4 is targeted. For example, the feature points 4 to 6 are detected from the feature points 5 and after including the feature point 4, and the images 4 to 7 are detected as images including all of them. The product of the number of feature points and the number of images at this time is 3 × 4 = 12. Among the combinations from the feature points including the feature point 4 and the subsequent feature points, the product of the number of feature points and the number of images is the largest. Here, in order to perform processing based on the third embodiment, a method of selecting the feature points 3 to 6 and the images 4 to 6 depending on the directions of the feature points 3 and 4 from the rotation center is also conceivable. . Although there are not many calculation results of new feature points, in order to perform processing based on the first embodiment, the feature points 3 to 5 and the images 3 to 7 are determined depending on the camera orientation values of the images 3 and 4. The method of selecting is also conceivable.
[0082]
Based on the data regarding the feature points 4 to 6 and the images 4 to 7 selected as the chain group, the coordinate matrix creation unit 20300, the matrix decomposition unit 20400, the normalization unit 20500, and the coordinate calculation unit 20600 are the second embodiment. Perform the process described in. The normalization unit 20500 performs processing using the distance and direction from the center of the feature point 4 stored in the position storage unit 900 as the actual measurement values. As a result, the distance and direction from the rotation center of the feature points 5 to 6 and the camera orientation of the images 5 to 7 are stored in the position storage unit 900 as new calculation results. At this time, the position storage unit 900 stores data relating to the feature points 1 to 6 and the images 1 to 7.
[0083]
Here, regarding the feature points 7 to 9 (the image 8 and the image 9), since the calculation result is not yet stored, the chain group creation unit 800 creates the chain group again. Among the feature points stored in the position storage unit 900, the feature point 6 at the end is obtained. For example, the feature points 6 to 9 are detected from the feature points 7 and the following including the feature point 6, and the images 7 to 9 are detected as images including all of them. At this time, the product of the number of feature points and the number of images is 4 × 3 = 12. Among the combinations from the feature points including the feature point 6 and the subsequent feature points, the product of the number of feature points and the number of images is the largest, so this is set as a linkage group 2. Based on the data relating to the feature points 6 to 9 and the images 7 to 9 selected as the chain group, the coordinate matrix creation unit 20300, the matrix decomposition unit 20400, the normalization unit 20500, and the coordinate calculation unit 20600 perform the second embodiment. The processing described in the embodiment is performed, and the calculation result is stored in the position storage unit 900. Thus, the distance and direction from the camera rotation center can be obtained for all feature points. Also, the camera orientation can be obtained for all images.
[0084]
As described above, in the fourth embodiment, when all feature points detected by the feature point tracking unit 200 are not included in all images, the basic group creation unit 700 performs measurement values measured in advance. Based on the above, according to the first to third embodiments, a combination of feature points and images that can measure the position of the feature points is created as a basic group, a coordinate matrix creation unit 10300, a matrix decomposition unit 10400, and a normalization As described in the first to third embodiments, the position value of the camera when the unit 10500 and the coordinate calculation unit 10600 take an image and the distance and direction from the center to each feature point are calculated as described above. Stored in the storage unit 900 as a calculation result, and the chain group creation unit 800, based on the calculation result, according to the first to third embodiments, a feature point and a feature point that can be measured A combination of images is created as a chain group, and the coordinate matrix creation unit 20300, the matrix decomposition unit 20400, the normalization unit 20500, and the coordinate calculation unit 20600 capture the image orientation value and the center to each feature point. The distance and direction are calculated as described in the first to third embodiments, stored as calculation results in the position storage unit 900, and the distances and directions from the rotation center are stored for all feature points. Since the calculation is repeated until it is calculated, the distance and direction from the center of rotation can be calculated for all the feature points even if all the feature points are not included in all the images. On the other hand, the value of the direction of the camera when the image is taken can be calculated.
[0085]
According to the fourth embodiment, when the actual measurement value is the value of the camera orientation when two or more images are captured, the basic group creation unit 700 determines that the camera orientation value is the actual measurement value. Since a plurality of feature points included in both of the two images being selected are selected, a combination of the selected plurality of feature points and an image including all of the selected plurality of feature points is created as a basic group. Based on the processing described in the first embodiment, the distance and direction from the center of rotation can be calculated for all the feature points in the basic group. It is possible to calculate the value of the orientation of the camera when the image is taken.
[0086]
Further, according to the fourth embodiment, when the actual measurement value is a value of the distance and direction from the rotation center of at least one feature point, the basic group creation unit 700 displays the actually measured feature point. Since a plurality of images to be included are selected and a combination of feature points included in all of the selected plurality of images and the plurality of selected images is created as a basic group, in the second embodiment Based on the processing described, the distance and direction from the center of rotation can be calculated for all feature points in the basic group, and the orientation of the camera when the image was taken for all images A value can be calculated.
[0087]
Further, according to the fourth embodiment, the chain group creation unit 800 detects a feature point that is located at the extreme end among the feature points stored in the position storage unit 900, and stores the feature point and the position storage. Since an image including all the feature points not stored in the unit 900 is selected and a combination of the feature points and the selected image is created as a chain group, the processing described in the second embodiment is performed. Based on this, it is possible to calculate the distance and direction from the center of rotation for all feature points in the chain group, and for all images, calculate the value of the orientation of the camera when the image is taken. be able to.
[0088]
According to the fourth embodiment, a combination of selected feature points and images having the largest product of the number of images and the number of feature points included in the image is created as a basic group and a chain group. Thus, based on a larger number of data, calculation results can be calculated for feature points and images in the basic group or in the chain group.
[0089]
Embodiment 5. FIG.
For example, it is assumed that the coordinates in the camera rotation direction of each feature point in each image are searched as shown in Table 9 below. Here, when the direction from the camera rotation center is actually measured for the feature point 1 and the feature point 7, there is no image including the feature point 1 and the feature point 7 at the same time, so the basic group and the chain group cannot be created. The position measurement based on the fourth embodiment cannot be performed. Therefore, in the present embodiment, after setting the temporary actual measurement value, the processing described in the fourth embodiment is performed, and the calculation result that best matches the actual actual measurement value is finally set as the calculation result. A method for performing the measurement will be described.
[0090]
[Table 9]

[0091]
FIG. 9 is a block diagram of an apparatus for realizing a position measuring method according to the fifth embodiment of the present invention. In the figure, a camera unit 100, a feature point tracking unit 200, a basic group creation unit 700, a chain group creation unit 800, a position storage unit 900, coordinate matrix creation units 10300 and 20300, matrix decomposition units 10400 and 20400, a coordinate calculation unit 10600 and Since 20600 and the normalization unit 20500 perform the same operations as those described in the fourth embodiment, description thereof will be omitted. The normalization unit 10501 performs substantially the same operation as that of the normalization unit 10500. However, the normalization unit 10501 is not an actual measurement value, and is directed to one or a plurality of temporary cameras assigned to two images provided from the temporary camera direction providing unit 1000. The difference is that the matrix Θ and the matrix A are calculated based on the values.
[0092]
The image selection unit 1300 selects two images in order to set a temporary camera orientation. The image selection unit 1300 may select an image using any method as long as two images that can perform the first embodiment can be selected. For example, the first selected image (first image) is selected as one of the two images. A plurality of feature points are selected from the feature points included in the image, and an image obtained by photographing a position farthest from the first image among images including all the selected feature points is selected as another image. Here, the plurality of feature points are selected so that the product of the number of selected feature points and an image including all the feature points is maximized.
[0093]
The provisional camera direction providing unit 1000 calculates one or a plurality of provisional camera direction values for the image selected by the image selection unit 1300 and transmits the calculated values to the normalization unit 10501. The temporary camera direction providing unit 1000 predicts and calculates the temporary camera direction by an appropriate method based on the measured direction of the feature point from the rotation center. For example, for two feature points that have been measured, assuming that the orientation of the camera of the image in which each feature point is photographed closest to the center coincides with the direction from the measured rotation center of the feature point, The feature points are calculated and predicted as being distributed at equidistant intervals by extrapolation or interpolation. When a plurality of provisional camera orientations are transmitted, the position measurement process is performed for each, and the respective calculation results are output.
[0094]
The actual measurement value comparison unit 1100 compares the actual measurement value in the direction from the rotation center of the two feature points with the calculated value. As a result, if a calculation result with an error less than the required accuracy is calculated, or if it is determined that the error is not reduced even if correction is performed by the temporary camera direction correction unit 1200, the calculation result is set as a final calculation result. . The temporary camera direction correcting unit 1200 sets a better temporary camera direction in the temporary camera direction providing unit 1000 based on the comparison result between the actual measurement value and the calculated value in the actual value comparison unit 1100, and performs the calculation again. . For example, if the directions from the center of rotation of two feature points are actually measured, their values are _i φ _j And the calculated value is φ _ci , Φ _cj And For the orientation of the temporary camera from which the value was obtained, φ _i −φ _j / (Φ _ci −φ _cj ) And (φ _j φ _ci −φ _i φ _cj ) / (Φ _i −φ _j ) Is added to the temporary camera direction providing unit 1000 as the newly corrected temporary camera direction. φ _i −φ _j / (Φ _ci −φ _cj ) Represents the ratio of the angle difference between the two camera orientations in the measured value and the angle difference between the camera orientations in the calculation result. Also, (φ _j φ _ci −φ _i φ _cj ) / (Φ _i −φ _j ) Represents the difference between the measured value and the calculated calculation result for the camera-oriented rotation angle.
[0095]
Next, the operation of the apparatus of the present embodiment will be described based on specific numerical values. Here, the target object in FIG. 5 will be described on the assumption that the tracking result of the feature points is as shown in Table 13 because the viewing angle of the camera is narrower than that used in the first embodiment. Further, feature points whose directions from the center of rotation are actually measured are feature points 1 and 7, and the actually measured values are 0.009967 rad and -0.009967 rad, respectively. Furthermore, the actual value comparison unit 1100 targets accuracy of 0.001 rad or less.
[0096]
Based on the image captured by the camera of the camera unit 100, the feature point tracking unit 200 searches the coordinates of each feature point in the camera rotation direction in each image. The image selection unit 1300 selects the image 1 as one of the two images for setting the temporary camera orientation. Since the image 1 includes all the images 6 to 6 including the feature points 4 to 7 and the product of the number of feature points and the number of images is 4 × 6 = 24, the feature points 4 to 7 are the maximum. And image 6 that is farthest from image 1 is selected as the second image.
[0097]
The temporary camera orientation providing unit 1000 calculates a temporary camera orientation value. It is an image 11 in which the feature point 1 is photographed closest to the center. Therefore, the camera orientation of the image 11 is assumed to be 0.009967. In addition, the image in which the feature point 7 is captured closest to the center is the image 1. Therefore, the camera orientation of image 1 is assumed to be −0.009967. Here, for the image 1 selected by the image selection unit 1300, the temporary camera orientation is set to −0.009967 as it is. The image 6 is calculated by interpolation based on the image 1 and the image 11. Therefore, {0.009967 − (− 0.009967)} × (6-1) / (11−1) + (− 0.009967) = 0 rad, which is the value of the image 6 for the temporary camera. . The provisional camera orientation providing unit 1000 transmits these values to the normalizing unit 10501.
[0098]
The basic group creation unit 700, the chain group creation unit 800, the position storage unit 900, the coordinate matrix creation units 10300 and 20300, the matrix decomposition units 10400 and 20400, the normalization units 10501 and 20500, and the coordinate calculation units 10600 and 20600 The same operation as described in the embodiment is performed, and calculation results are calculated for all feature points and images. At this time, the basic group and the chained group are grouped as shown by the wavy lines in Table 9. The calculation result calculated based on these values is shown in the “first” row of Table 10. The calculated values in the direction from the rotation center were also obtained for the feature point 1 and the feature point 7 by the calculation, but the errors were 0.014815 and 0.01461, respectively, and the actually measured value comparison unit 1100 sets the target value to 0. The accuracy of less than 0.001 rad has not been reached.
[0099]
[Table 10]

[0100]
The temporary camera direction correcting unit 1200 sets {0.009967 − (− 0.009967)} / {0.1144848 − (− 0.1142805) to the temporary camera direction −0.009967, 0 between the image 1 and the image 6. ) = 0.87137. Also, {0.1144848 × (−0.009967) − (− 0.1142805) × 0.009967} / {0.1144848 − (− 0.1142805)} = − 0.000089 is added. As a result, the temporary camera orientations of the corrected image 1 and image 6 are −0.0869938 and −0.000089, respectively. This value is transmitted to the temporary camera direction providing unit 1000 as a new temporary camera direction, and the position measurement process is performed again. This calculation result is shown in the “after correction” row of Table 10. The actual measurement comparison unit 1100 compares the calculation result with the actual measurement value. Since both the feature point 1 and the feature point 7 have an error of 0.001 rad or less, the process ends here, and the calculation result at this time is output as the final calculation result. Table 11 shows the calculation result of the position of each feature point calculated in the present embodiment.
[0101]
[Table 11]

[0102]
As described above, according to the fifth embodiment, the image selection unit 1300 selects two images, and based on the temporary camera orientation calculated by the temporary camera orientation providing unit 1000 based on these images. Based on the calculation result calculated by the normalization unit 10501, the actual measurement value comparison unit 1100 determines whether to perform the calculation again by setting the temporary camera orientation again based on the actual measurement value. When it is determined that the calculation is to be performed, the camera direction correction unit 1200 performs the calculation for setting the temporary camera direction again, performs the position measurement process again, and outputs the calculation result with the highest accuracy as the calculation result. Even if there is no image including at least two feature points whose measured values in the direction from the rotation center are measured at the same time, the positions of all the feature points can be measured. For this reason, the feature points to be actually measured do not have to be included in the same screen, and restrictions on the selection of feature points are relaxed. Further, in the camera orientation correcting unit 1200, (φ _i −φ _j ) / (Φ _ci −φ _cj ), (Φ _j φ _ci −φ _i φ _cj ) / (Φ _i −φ _j ) Is used as the corrected temporary camera orientation, so that a more accurate temporary camera orientation can be provided to the normalization unit 10501, and highly accurate position measurement can be performed.
[0103]
Embodiment 6. FIG.
In the present embodiment, a case is considered in which the camera of the camera unit 100 is rotated about once and an image is captured over about 2π rad (360 degrees). A feature point photographed on the first image and the same feature point on another image photographed when the camera is rotated about one turn are used as different feature points separated by 2π from the rotation center of the two feature points. The measured value in the direction of. Then, in the actual value comparison unit 1100 of the fifth embodiment, the position measurement process of the fifth embodiment is performed using the actual value for comparison with the calculation result.
[0104]
FIG. 10 is a block diagram of an apparatus for realizing a position measuring method according to the sixth embodiment of the present invention. In the figure, operations other than the identical point detection unit 1400 and the actual measurement value holding unit 1500 are the same as those given the same numbers in FIG. 9, and are already described in the fifth embodiment, and thus will be omitted. The same point detection unit 1400 includes, for example, the feature points included in the first few images among the feature points searched for by the feature point tracking unit 200, and several sheets after the camera is rotated about one time. The same feature point is detected from the feature points included in the image. In this detection, the same feature point may be searched by simple pattern matching, or the operator may designate the same feature point. The actual measurement value holding unit 1500 sets the actual measurement value in the direction from the rotation center of the feature point included in the first image among the set of feature points detected as the same feature point by the same point detection unit 1400 to 0 rad. And In addition, the actual measurement value of the direction from the rotation center of the feature point included in the image after about one rotation of the camera is treated as another feature point as 2π rad, and two actual measurement values to be compared with the actual measurement value comparison unit 1100 Value.
[0105]
FIG. 11 is a diagram for explaining the operation of the apparatus according to the sixth embodiment. Based on FIG. 11, the operation of the apparatus of the present embodiment will be described using specific numerical values. It is assumed that the camera of the camera unit 100 is rotated about once and an image is captured over about 2π rad (360 degrees). Here, as shown in FIG. 24, an image at the camera position 1 is an image 1 and an image at the camera position T is an image T. Further, it is assumed that feature point 1 is detected as feature point A in image 1 and feature point B in image T.
[0106]
For example, when the operator designates that the feature point A and the feature point B are the same feature point to the same point detection unit 1400 based on the image 1 and the image T, the same point detection unit 1400 will detect the feature point A and the feature point B. Are transmitted to the actual measurement value holding unit 1500. The actual measurement value holding unit 150 sets the actual measurement value in the direction from the rotation center of the feature point A (feature point 1 in the image 1) to 0 rad. Further, the actual measurement value in the direction from the rotation center of the feature point B (feature point 1 in the image T) is set to 2π rad, and the feature point B is newly output to the actual measurement value comparison unit 1100 as the feature point (N + 1). The actual value comparison unit 1100 that has obtained the actual measurement values in the direction from the rotation center of the two feature points can perform comparison using the actual measurement values. Here, although the measurement is not particularly precise, as described in the fifth embodiment, the position measurement process can be performed based on the temporary camera orientation, and the calculation result can be obtained. it can.
[0107]
As described above, according to the sixth embodiment, when the camera unit 100 shoots about one turn, the same point detection unit captures an image obtained by photographing 0 rad and an image obtained by photographing 2π rad with respect to a certain feature point. In the actual measurement value holding unit 1500, the actual measurement value in the direction from the rotation center of one feature point is set to 0 rad, and the actual measurement value in the direction from the rotation center of the other feature point is set to 2π rad as another feature point. In the actual measurement value comparison unit 1100, the actual measurement value of the comparison target is used. Therefore, the direction from the rotation center of the two feature points of 0 rad and 2π rad can be obtained only by selecting the feature point. Even if no actual measurement is required, the distance and direction from the center of rotation can be calculated for all feature points.
[0108]
Embodiment 7. FIG.
In the above-described first embodiment, the normalization unit 500 uses the least square sum to q and θ. ₀ Can be calculated. For example, in the first embodiment, when the orientation of the camera is actually measured with three or more images, for example, by a general method such as the least square method, θ ₁ 'And θ ₁ Q can be determined by the least-squares sum so that.
[0109]
Embodiment 8. FIG.
In the second embodiment described above, the normalization unit 501 can calculate q using the least square sum. For example, the distance r from the center of rotation at two or more feature points _i And direction φ _i Is measured, each measured distance r _i 1 / (1-L / r _i ) To calculate the least squares sum, and let this be q. And θ ₀ Is obtained by equation (15).
[0110]
Embodiment 9. FIG.
In the third embodiment described above, the normalization unit 502 uses the least-squares sum to calculate q and θ. ₀ Can be calculated. For example, the direction φ from the center of rotation at two or more feature points _i Is measured, the component of the matrix Θ ”is θ _t ”And the component of matrix A ′ is α _i ', The following equation (18) is calculated for each feature point, and q and θ ₀ To decide.
[0111]
[Expression 21]

[0112]
Embodiment 10 FIG.
In the above-described embodiment, each unit is provided and the respective calculations are performed. However, for each unit other than the camera unit 100, in practice, a computer, a program that causes the computer to perform the operation of the above-described embodiment, and the program are provided. This is realized by a recording medium for recording.
[0113]
【The invention's effect】
As described above, according to the present invention, a photograph was taken by devising conditions so as to photograph an image outside the circumference by rotating on the circumference at a certain distance from the center. Multiple sheets One or a plurality of feature points included in all the images are detected, the coordinate value in the rotation direction of each feature point in each image is detected, normalized by the focal length of the photographed camera, and each feature For a point, calculate the average of the normalized values and calculate the difference to create a matrix. Determine the two matrices so that the product of the two matrices is closest to the created matrix. A certain value and a constant value are calculated so as to correspond to the value, a certain value is applied to one matrix, a certain value is added, and the other matrix is divided by a certain value. Since the distance and direction to the point are calculated, a few actual values (for example, the distance and direction from the center of rotation of one feature point, the direction from the center of rotation of two feature points, two images taken) Camera direction when) Distance and direction, as well as each image until all of the feature points can be calculated value of orientation of the camera when it is taken from the center.
[0114]
Further, according to the present invention, even when not all the detected feature points are included in all the images, the feature points that can measure the position of the feature points based on the actually measured values measured in advance and Create a combination of images as a basic group, and calculate and store the camera orientation value and the distance and direction from the center to each feature point for the basic group. Creates a combination of included feature points or images as a chain group, calculates and stores the camera orientation value and distance and direction from the center to each feature point for the chain group, and rotates this for all feature points Since it is performed until the distance and direction from the center are calculated, even if not all feature points are included in all images, the distance from the rotation center for all feature points Can be calculated fine direction, with respect to all images, it is possible to calculate the value of the orientation of the camera when capturing the image.
[Brief description of the drawings]
FIG. 1 is a block diagram of an apparatus for realizing a position measuring method according to a first embodiment of the present invention.
FIG. 2 is a diagram illustrating an installation state example of a camera unit.
FIG. 3 is a diagram for explaining approximation used in the present embodiment.
4 is a diagram illustrating a relationship between a camera unit 100 and feature points as viewed from above the camera unit 100. FIG.
FIG. 5 is a diagram illustrating a feature point of a target object and a shooting situation.
FIG. 6 is a diagram illustrating a case where feature points are arranged over a wide range.
FIG. 7 is a diagram illustrating a relationship example of feature points included in each image.
FIG. 8 is a block diagram of an apparatus for realizing a position measuring method according to a fourth embodiment of the present invention.
FIG. 9 is a block diagram of an apparatus for realizing a position measuring method according to a fifth embodiment of the present invention.
FIG. 10 is a block diagram of an apparatus for realizing a position measuring method according to a sixth embodiment of the present invention.
FIG. 11 is a diagram for explaining the operation of an apparatus according to a sixth embodiment.
[Explanation of symbols]
100 camera section
200 Feature point tracking unit
300, 10300, 20300 Coordinate matrix generator
400, 10400, 20400 Matrix decomposition unit
500, 501, 502, 10500, 10501, 20500 Normalizer
600, 10600, 20600 Coordinate calculation unit
700 Basic Group Creation Department
800 Chain Group Creation Department
900 Position storage unit
1000 Provisional part for temporary cameras
1100 Measured value comparison unit
1200 Temporary camera orientation correction unit
1300 Image selection unit
1400 Same point detector
1500 Actual value holding part

Claims (21)

Moving on a circumference at a certain distance from the center of rotation, and taking a plurality of images outside the circumference with a camera;
Detecting one or a plurality of feature points included in all of the plurality of images, and detecting a coordinate value of the movement direction of each feature point in each image;
The coordinate value in the moving direction of each feature point in each image is normalized by the focal length based on the photographed camera, and the average of the normalized coordinate values in the plurality of images is calculated for each feature point. Calculating a difference between each normalized coordinate value and creating a matrix having the calculated difference value as a component;
Determining the two matrices so that the product of the two matrices is the closest matrix to the created matrix;
A certain value and a constant value are calculated so as to correspond to an actually measured value measured in advance, a certain value is multiplied by one of the two matrices, and a certain value is added. The other matrix of the two matrices Dividing the value by the certain value, and calculating the distance and direction from the center to the feature points based on the matrix thereof. When all the detected feature points are not included in all of the plurality of images,
Based on the actually measured values measured in advance, a combination of a feature point and an image that can calculate a camera orientation value and a distance and direction from the center to each feature point when the image is taken as a basic group. Process to create,
For the combination of feature points and images included in the basic group, calculating the value of the camera orientation when taking an image and the distance and direction from the center to the feature points, and storing the calculation results;
Based on the stored calculation results, a combination of feature points and images capable of calculating the value of the camera orientation when the image is taken and the distance and direction from the center to each feature point is created as a chain group. Process,
For a combination of feature points and images included in the chain group, calculating a camera orientation value when capturing an image and a distance and direction from the center to each feature point, and storing the calculation results;
Create chain groups for all feature points until the distance and direction from the center to each feature point are calculated, and the direction value of the camera when the image was taken and the center to each feature point The object position measuring apparatus according to claim 1, further comprising a step of calculating a distance and a direction and storing the calculation result. A camera unit that moves on a circumference at a certain distance L from the center of rotation and shoots T ( T is a natural number of 2 or more ) images outside the circumference;
A feature point tracking unit that detects N (N is a natural number) feature points included in all of the T images and detects a coordinate value in the moving direction of each feature point i in each image t;
The coordinate values in the moving direction of each feature point i in each image t are normalized by the focal length based on the photographed camera unit u _ti (t = 1, 2,..., T, i = 1, 2, ..., N), and

A coordinate matrix creation unit that creates a matrix U ′ of T rows and N columns with the components as components,
The matrix Θ ″ and the matrix A ′ are set so that the product of the matrix Θ ″ and the matrix A ′ can be regarded as the matrix U ′ with respect to the T × 1 matrix Θ ″ and the 1 × N matrix A ′. A matrix decomposition unit for determining
When a certain value q and a constant value θ ₀ are calculated so as to correspond to the actually measured values measured in advance, the matrix Θ ″ is multiplied by q and the θ ₀ is added, and the T images are taken. A normalization unit that calculates a matrix Θ having the orientation of the camera unit as a component, and calculates the matrix A by dividing the matrix A ′ by the q;
The component of the matrix Θ calculated by the normalization unit is θ _t and each component of the matrix A is α _i , and the distance r _i from the rotation center to the feature point is set as r _i = L / (1-1 / α _i ) and the direction φ _i from the center of rotation to the feature point

An object position measuring apparatus, comprising: a coordinate calculating unit that calculates the position of the object. The matrix decomposition unit performs singular value decomposition by approximating a singular value other than the maximum singular value in the matrix created by the coordinate matrix creation unit to 0, and determines the matrix Θ ″ and the matrix A ′. The object position measuring apparatus according to claim 3. For the previously measured actual value, which is a value of the orientation of each camera when two or more images are captured, the normalization unit is configured to use the actual value and the component of the matrix Θ corresponding to the actual value. 4. The object position measuring apparatus according to claim 3, wherein the q and the θ ₀ are calculated so that the sum of squares of the difference between the two and the square is minimized. For the previously measured actual value, which is the value of the orientations of the two cameras when two images are taken, the normalization unit calculates the difference between the actual values of the two camera orientations and the 2 The ratio of the difference between the components of the matrix Θ ″ corresponding to the measured values of the orientations of two cameras is q, and any one of the measured values of the orientations of the two cameras and the measured values corresponding to the measured values 4. The object position measuring apparatus according to claim 3, wherein a difference from a value obtained by multiplying a component of the matrix Θ ″ by q is the θ ₀ . For the previously measured actual value, which is the value of the distance r _i from the rotation center of at least one feature point and the direction Φ _i , the normalization unit has a value of 1 / (1-L / r _i ). The q is calculated such that the sum of squares of the difference between the value and the corresponding component α _{i of the} matrix A is minimized, and the calculated q _i is calculated for each φ _i .

4. The object position measuring apparatus according to claim 3, wherein the difference between the average value of the values and the average value of each component when the matrix Θ ″ is multiplied by q is the θ ₀ . With respect to the measured value measured in advance, which is the value of the distance r _i from the rotation center of one feature point and the direction Φ _i , the normalization unit sets a value of 1 / (1−L / r _i ). And the ratio between the value and the corresponding component α _{i of the} matrix A is calculated as q, and is calculated for each measured φ _i .

4. The object position measuring apparatus according to claim 3, wherein the difference between the average value of the values and the average value of each component when the matrix Θ ″ is multiplied by q is the θ ₀ . The corresponding component of the matrix Θ ″ is θ _i ″ and the corresponding component of the matrix A ′ is the corresponding value of the direction Φ _i from the rotation center of at least two feature points. α _i '

4. The object position measuring apparatus according to claim 3, wherein the q and the θ ₀ are calculated so that a sum of squares of all of the actually measured φ _i is minimized. For the measured values measured in advance, which are the values of the directions Φ _i and Φ _j from the rotation center of the two feature points, the normalization unit α _i ′, which is a corresponding component of the matrix A ′, From α _j '

Based on the component θ _t ′ of the matrix Θ ′ created by multiplying the matrix Θ ″ by the q and the component α _i of the matrix A created by dividing the matrix A ′ by the q And

The object position measuring apparatus according to claim 3, wherein the θ ₀ satisfying the following condition is calculated. When all feature points detected by the feature point tracking unit are not included in all of the plurality of images,
Based on the actually measured values measured in advance, a combination of a feature point and an image that can calculate a camera orientation value and a distance and direction from the center to each feature point when the image is taken as a basic group. Create and calculate the orientation value of the camera and the distance and direction from the center to each feature point when the coordinate matrix creation unit, the matrix decomposition unit, the normalization unit, and the coordinate calculation unit capture an image. The basic group creation section
A position storage unit that stores the calculated orientation value of the camera at the time of shooting and the distance and direction from the center to each feature point as a calculation result;
Based on a calculation result stored in the position storage unit, a combination of a feature point and an image capable of calculating a value of a camera orientation when the image is captured and a distance and direction from the center to each feature point. Created as a chain group, the coordinate matrix creation unit, the matrix decomposition unit, the normalization unit and the coordinate calculation unit when taking an image of the direction of the camera and the distance from the center to each feature point and The direction is calculated and stored in the position storage unit as a calculation result, and the value of the orientation of the camera when shooting for all images is calculated. From all the feature points to the respective feature points The object position measuring apparatus according to claim 3, further comprising: a chain group creating unit that creates a chain group until the distance and the direction of the object are calculated. With respect to the previously measured actual value, which is the value of the camera orientation when two or more images are captured, the basic group creation unit has two images in which the camera orientation value is actually measured. A plurality of feature points included in both of the selected feature points, and a combination of the selected feature points and an image including all of the selected feature points is selected and created as a basic group. Item 12. The object position measuring apparatus according to Item 11. The basic group creation unit has actually measured the distance and direction values from the rotation center with respect to the measured values measured in advance, which are the distance and direction values from the rotation center of at least one feature point . Selecting a plurality of images including one feature point, selecting a combination of the feature points included in all of the selected plurality of images and the plurality of selected images, and creating a basic group The object position measuring apparatus according to claim 11. 13. The basic group is created as a basic group having a largest product of the number of images and the number of feature points included in the image among combinations of selected feature points and images. Or the position measuring apparatus of the object of 13. The chain group creation unit detects a feature point located at the extreme end among the feature points stored in the position storage unit, and the detected feature point and the distance and direction from the center are calculated. Selecting an image including all of one or more feature points that are not present, and selecting a combination of the detected feature points and one or more feature points whose distance and direction from the center are not calculated and the selected image. 12. The object position measuring apparatus according to claim 11, wherein the object position measuring apparatus is created as a chain group. 16. The chain group is created as a chain group having a largest product of the number of images and the number of feature points included in the image among combinations of selected feature points and images. The object position measuring apparatus described. A value of a direction from the rotation center of at least two feature points, and there is no image including the feature points at the same time.
An image selection unit for selecting two images among the images of the basic group created by the basic group creation unit;
A temporary camera orientation providing unit that sets one or a plurality of temporary camera orientations to each of the two images selected by the image selection unit, and causes the normalization unit to process the measured values as the measured values;
When the coordinate matrix creation unit, the matrix decomposition unit, the normalization unit, and the coordinate calculation unit calculate the direction from the rotation center for at least two feature points whose directions from the rotation center are actually measured, the calculated value And the actual measurement value measured in advance, and based on the comparison, again set the temporary camera direction, to the coordinate matrix creation unit, the matrix decomposition unit, the normalization unit and the coordinate calculation unit An actual value comparison unit for determining whether to calculate a calculation result;
If it is determined that the actual value comparison unit again sets the temporary camera direction and calculates the calculation result, the fixed camera direction is provided with a certain calculation result based on the calculated value and the previously measured actual value. 12. The object position measuring apparatus according to claim 11, further comprising: a temporary camera direction correcting unit that sends the temporary camera direction to the temporary camera direction setting unit. When the measured values are φ _i and φ _j and the directions of the calculated two feature points from the rotation center are φ _ci and φ _cj , respectively, (φ _i −φ _j ) / A value obtained by multiplying (φ _ci −φ _cj ) and adding (φ _j φ _ci −φ _i φ _cj ) / (φ _i −φ _j ) is set as the new temporary camera direction. Item 17. The object position measuring apparatus according to Item 17. In the camera unit, when shooting over at least 2π rad,
The same point detection unit for detecting an image obtained by photographing 0 rad and an image obtained by photographing 2π rad with respect to a certain feature point based on at least two images;
For the same detected feature point, the measured value of the direction from the rotation center of one feature point is set to 0 rad, the measured value of the direction from the rotation center of the other feature point is set to 2π rad, and another feature point is set. 18. The object position measuring apparatus according to claim 17, wherein the actual value comparison unit includes an actual value as an actual value to be compared. Move on the circumference at a certain distance from the center of rotation, let the camera take a plurality of images outside the circumference,
Detecting one or a plurality of feature points included in all of the plurality of images, and detecting a coordinate value of the moving direction of each feature point in each image;
The coordinate value in the moving direction of each feature point in each image is normalized by the focal length based on the photographed camera, and the average of the normalized coordinate values in the plurality of images is normalized for each feature point. Calculate the difference between each normalized coordinate value and create a matrix with the calculated difference value as a component;
The two matrices are determined so that a product of the two matrices is a matrix closest to the matrix created by the coordinate matrix creation unit;
A certain value and a constant value are calculated so as to correspond to an actual measurement value measured in advance, a certain value is multiplied by one of the two matrices, and a certain value is added, and the other of the two matrices is calculated. A recording medium on which a program for an object position measurement method is recorded that causes a computer to calculate a distance and a direction from the center to the feature points based on the matrix by dividing the matrix by the certain value . When all the detected feature points are not included in all of the plurality of images,
Based on the actually measured values measured in advance, a combination of feature points and images capable of calculating the value of the orientation of the camera when the image is taken and the distance and direction from the center to each feature point is defined as a basic group. Let them create
For the combination of feature points and images included in the basic group, the camera orientation value when the image is taken and the distance and direction from the center to the feature points are calculated, and the calculation results are stored.
Based on the calculation result, the value of the orientation of the camera when the image was taken and the combination of feature points and images that can calculate the distance and direction from the center to the feature points are created as a chain group,
For the combination of feature points and images included in the chain group, the value of the camera direction when the image is taken and the distance and direction from the center to each feature point are calculated, and the calculation result is stored.
A chain group is created for all feature points until the distance and direction from the center to each feature point is calculated, and the orientation value of the camera when the image is taken and each feature point from the center. 21. A recording medium on which a program for an object position measuring method according to claim 20 is recorded which causes a computer to calculate a distance and a direction up to and to store the calculation result.

Images