JP2004325072A - Photogrammetry and photogrammetric program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To compute the coordinates of survey points using image data of photographs, with high accuracy. <P>SOLUTION: This photogrammetry is provided with a first step, a second step, and a third step. In the first step, photographing is performed by a camera from different observation points, in such a way as to include both an object to be surveyed and reference angles to be used, when survey computations on the object to be surveyed are performed to acquire at least two different images. In the second step, both angle reference points for specifying the attitude of the camera at survey points and the observation points and reference angle specifying points for specifying the reference angles, which are common to both images, are set on the object to be surveyed to compute two-dimensional coordinates of the angle reference points and the reference angle specifying points in the images. In the third step, a three-dimensional coordinate system including the object to be surveyed is set. On the basis of the two-dimensional coordinates of the angle reference points and of the reference angle specifying points and the reference angles, three-dimensional coordinates of the survey points are computed. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

なお、ｘｄ（ｎ），ｙｄ（ｎ），ｘｆ（ｎ），ｙｆ（ｎ）は、図１においてはそれぞれＸｐ，Ｙｐ，Ｘｓ，Ｙｓに相当している。
【００５０】
上記式（１２）における係数ａ１およびａ２は、例えば図６の像点ｔ（ｋ）に対して垂直方向の上側に位置する像点ｔ（ｋ＋１）の特性データ要素｛ＸＴ（ｋ＋１），ＹＴ（ｋ＋１），ｘｔ（ｋ＋１），ｙｔ（ｋ＋１）｝および水平方向の左側に位置する像点ｔ（ｋ＋２）の特性データ要素｛ＸＴ（ｋ＋２），ＹＴ（ｋ＋２），ｘｔ（ｋ＋２），ｙｔ（ｋ＋２）｝に基づいて算出される。
【００５１】
詳述すると、係数ａ１およびａ２に関して次の連立方程式（１４）、（１５）が成り立つ。
ΔＸ１＝ａ１・Δｘ１＋ａ２・Δｙ１ （１４）
ΔＸ２＝ａ１・Δｘ２＋ａ２・Δｙ２ （１５）
ただし、
ΔＸ１＝ＸＴ（ｋ＋１）−ＸＴ（ｋ） ΔＹ１＝ＹＴ（ｋ＋１）−ＹＴ（ｋ）
ΔＸ２＝ＸＴ（ｋ＋２）−ＸＴ（ｋ） ΔＹ２＝ＹＴ（ｋ＋２）−ＹＴ（ｋ）
Δｘ１＝ｘｔ（ｋ＋１）−ｘｔ（ｋ） Δｙ１＝ｙｔ（ｋ＋１）−ｙｔ（ｋ）
Δｘ２＝ｘｔ（ｋ＋２）−ｘｔ（ｋ） Δｙ２＝ｙｔ（ｋ＋２）−ｙｔ（ｋ）
である。
【００５２】
従って、係数ａ１およびａ２は、以下の式（１６）および式（１７）により求められる。また、式（１３）における係数ａ３およびａ４も、上記係数ａ１およびａ２と同様にして、以下の式（１８）および（１９）により求められる。
ａ１＝（ΔＸ１・Δｙ２−ΔＸ２・Δｙ１）／Ｍ （１６）
ａ２＝（Δｘ１・ΔＸ２−Δｘ２・ΔＸ１）／Ｍ （１７）
ａ３＝（ΔＹ１・Δｙ２−ΔＹ２・Δｙ１）／Ｍ （１８）
ａ４＝（Δｘ１・ΔＹ２−Δｘ２・ΔＹ１）／Ｍ （１９）
ただし、Ｍ＝Δｘ１・Δｙ２−Δｘ２・Δｙ１である。
【００５３】
なお、ステップＳ２０３において、中心像Ｃｄのピクセル座標（ｘｃ，ｙｃ）を中心Ｃｆの相対２次元座標（Ｘｃ，Ｙｃ）に座標交換する場合には、式（１２）の右辺から項（−Ｘｃ）を除いた式および式（１３）の右辺から項（−Ｙｃ）を除いた式により算出する。
【００５４】
次のステップ２０４ではアングル定数を求める。アングル参照点Ｐ１〜Ｐ５のデータを使用して連立方程式（１１）を解き、カメラのアングル定数αａ０、αｂ０、σａ０、σｂ０、Δβを求める。この連立方程式の解法の一つとして逐次近似解法がある。この逐次近似解法については、後に具体例を詳述する。以上のようにアングル定数が求まったら得られたアングル定数に基づき三角測量計算をする。
【００５５】
次のステップＳ２０５では、３次元の三角測量計算に必要なデータの準備をする。すなわち、観測点ａ，ｂから見たカメラ基軸７に対する測量点までの角度αａｎ，αｂｎ，観測点ａ，ｂに共通の基準方向から見た測量点までの回転角βを求める（後述する図８を参照）。三角測量計算は測量点Ｐｎの他、寸法参照点Ｓ１，Ｓ２、参照角規定点Ｓ３〜Ｓ６およびアングル参照点Ｐ１〜Ｐ５についても行われるが、代表して測量点Ｐｎで説明する。計算に際してはステップＳ２０３で示したとおり測量点Ｐｎについてカメラ特性ファイルを用いて撮像面Ｈｄの画素位置（Ｘｐ，Ｙｐ）を仮想撮像面Ｈｆ上の位置（Ｘｓ，Ｙｓ）に変換しておく。
【００５６】
観測点ａからみた測量点Ｐ１までの角度αａ（ｎ）は、図１（Ａ）、（Ｂ）から明らかなように、次式（２０）で算出することができる。
αａ（ｎ）＝ｔａｎ^{− １}（ｈ／Ｚｕ） （２０）
ここでｈ／Ｚｕは、式（８）、（９）から求められる。観測点ｂから見た測量点Ｐｎまでの角度αｂ（ｎ）についても同様に求める。
【００５７】
回転角βに関しては、式（１０）から角βａ（ｎ），βｂ（ｎ）を得られるが、これらはそれぞれカメラ基軸７回りの回転角である。三角測量に必要なのは観測点ａ，ｂに共通な基準方向からの回転角であるから、両方のカメラに写し込まれた点を任意に基準にして共通の回転角を求める。例えば、図８に示すように、測量点Ｐ１を基準とした回転角β’ａ（ｎ）＝βａ（ｎ）−βａ１を求め、これを使用する。なお、回転角β’ｂ（ｎ）＝βｂ（ｎ）−βｂ１も得られるが、β’ａ（ｎ）＝β’ｂ（ｎ）であるから、誤差がなければ片方だけを求めればよい。
【００５８】
ステップＳ２０６では、ステップＳ２０５のデータに基づいて、先ず、モデルを算出する。三角測量のためには観測点ａ，ｂ間の距離が必要である。しかし、ここでは不明とし、仮にこの距離が１であるとして相似形の３次元座標（モデルと呼ぶ）を求める。図８に示すように、観測点ａを原点とした測量点Ｐｎの直交座標（ｘ，ｙ，ｚ）は、三角測量の原理から次の式（２１）〜（２４）によって求められる。ただし、測量点は図８に示すような関係にある。図８は測量点Ｐ１（アングル参照点でもある）とＰｎとを（Ａ）ではＹ−Ｚ投影図、（Ｂ）ではＸ−Ｙ投影図で示している。回転角βの基準となる測量点Ｐ１が図８（Ｂ）の紙面上にあるものとし、カメラ基軸７をＸ軸方向としている。図８（Ｂ）の紙面をＸ−Ｙ平面、それに直交する軸をＺ軸方向、観測点ａ，ｂ間距離をＥ，Ｘ軸から鉛直に測量点Ｐｎまでの距離をＨとする。
【００５９】
【数１】

このとき、観測点ａ，ｂ間の距離Ｅを仮に１とする。以上をＰ１〜Ｐ５、Ｓ１〜Ｓ６についても求めてモデルとする。
【００６０】
ステップＳ２０７において、仮の値とした距離Ｅを実際の値とした場合と同等になるように修正する。それにはまず、ステップＳ２０１で間隔を実測した図２で示した寸法参照点２点（Ｓ１，Ｓ２）について、求められたモデル即ち相似形の座標値からその間隔ｇを次の式により求める。なお、式（２５）中添字Ｓ１，Ｓ２は寸法参照点Ｓ１，Ｓ２のモデルの座標であることを示す。
【数２】

【００６１】
次いで、予め実測した参照寸法Ｓ（Ｓ１，Ｓ２の実際の間隔）と間隔ｇとの比を求める。この比を求めたモデルの各座標値に掛けて数値を算出する。以上のステップにより、観測点ａを原点とした、測量点Ｐｎの他参照点等の３次元座標における実際の値を算出する。
【００６２】
しかし、算出された座標の原点が観測点ａであったり、Ｐ１を含む面がＸ−Ｙ面、Ｘ軸がカメラ基軸７に固定されるのでは不便なこともある。必要に応じて、その原点、基準方向を調整する。例えば、測量点の内のある点を原点としたり、ＸＹの基準方向を指定して変更する。そのために、例えば座標基準指示ファイルを予めＨＤＤ３７に記憶しておく。すなわち、原点とする点、Ｘ軸を定める点、Ｙ軸方向を定める点等を参照点等の中から選択してキーボードなどから入力したものを記録したデータファイルであり、希望する原点および基準方向となるように同ファイルを参照して、求められた３次元座標を平行移動または回転させ座標変換を行えるように設定しておくことが望ましい。
【００６３】
ステップＳ２０８では、以上の測量計算結果が、表示装置２９に表示され、またプリンタ３１等に出力され利用者に読取られる。また、用途に応じてＣＡＤ用等ファイルにして出力し、ＣＡＤ、ＣＧの骨格データ等として使用することができる。
【００６４】
前後するが、ここで前述したステップＳ２０４で式（１１）からアングル定数αａ０、αｂ０、σａ０、σｂ０、Δβを求めるための逐次近似解法について説明する。式（１）〜式（１０）を変形しβａｎ（βｂｎにも共通）を求めると、
【数３】

となる。なお、以降の計算のため式（２６）の右辺の括弧内を
【数４】

とする。
【００６５】
上式（２６）に観測点を示す添字ａ，ｂおよびアングル参照点の個別を示す添字１〜５を適切に添えて式（１１）に代入すれば、変数を詳しく示した式になるがここでの表示は省略する。
（１１）式についてアングル定数のみで示した式を関数ｆで表すと、
Δβ＝βａｎ−βｂｎ＝ｆ（αａ０，σａ０，αｂ０，σｂ０） （２８）
となる。
【００６６】
連立方程式を解けるよう、関数ｆを一次近似式にするためαａ０，σａ０の偏導関数を誘導すると
【数５】

となる。
なお、αｂ０，σｂ０の偏導関数についても（２９）（３０）式の添字ａ、ｂを入れかえれば同様に表せる。
【００６７】
上記（２９）式および（３０）式を組み合わせ、（１１）式について一次近似式にすると
【数６】

となる。
【００６８】
上式（３１）中ｆｎはアングル参照点Ｐｎの現在のアングル定数値におけるβａｎ−βｂｎ、各偏導関数式はＰｎの現在のアングル定数における偏微分係数、Δαａ０，Δσａ０，Δαｂ０，Δσｂ０は現在のアングル定数からの各増分である。また、Ｒは本近似式で一次を超える成分を省いたことよる誤差を示す。このＲは無視する。なお、この前後の文中で「現在の」と表記するのは逐次近似の１回毎の初期値を意味するものとする。
【００６９】
各偏微分係数を行列要素Ａｎｋ、ｆｎを行列要素Ｂｎと表して、上記式（１１）の全式に当てはめて整理すると
【数７】

となる。
ただし、ｎはアングル参照点の数、Ａｎ５はΔβの符号である−１であり常に一定である。
【００７０】
上記（３２）式を、求めたいアングル定数の増分Δαａ０，Δσａ０，Δαｂ０，Δσｂ０，Δβを行列の要素ｘｎとした行列ｘで表すと
Ａｘ＝Ｂ （３３）
ただし、ＡはＡｎｋを要素とする行列、ＢはＢｎを要素とする行列である。
【００７１】
式（１１）ではアングル参照点を５つとして式を５つ連立させているが、６点以上とした場合はその数だけ連立させ、次の式により（最小二乗法）により５行に変換する。
【数８】

ただし、ＡバーはＡの転置行列、Ａ’，Ｂ’は５行に変換した行列である。
【００７２】
アングル定数の増分、即ちｘの要素を求めるには次のようにクラーメルの公式による。
【数９】

ただし、｜Ａ‘｜はＡ’を行列式にしたもの、｜Ａ‘_Ｂ’｜は｜Ａ‘｜のｘｎに関する列をＢ’で置き換えた行列式である。
【００７３】
以上で得られたｘｎ、即ちアングル定数の増分Δαａ０，Δσａ０，Δαｂ０，Δσｂ０を現在のアングル定数αａ０，σａ０，αｂ０，σｂ０に加算して、次の現在のアングル定数αａ０，σａ０，αｂ０，σｂ０として以上を繰り返す。繰り返しは、Δβの最大偏差が解とみなせる規定値を下回るまで、もしくは特定回数行う。
【００７４】
ここで、前述した追加する拘束条件式について詳細を示す。図９は図２に示した参照角θを形成する２つの線分Ａ１，Ａ２に対応するモデルとして線分ａ１、ａ２との関係を示した図である。図９（Ａ）において、ａ１’は２本の線分が離れていた場合、２本の線が成す角度θを正確に示すためこの２本の線が最も接近する位置でａ１をａ２に平行移動した線である。図９（Ｂ）はθを計算するため（Ａ）における線分ａ２をｓ３とｓ５が一致するよう平行移動して線分ａ２’とした図である。Ｌ１〜Ｌ３は各線分が作る三角形の各辺の長さを示している。
【００７５】
前述した式（１）〜（１１）および式（２０）〜（２４）により測量点等の参照点モデルにおける座標が求められる。同様に参照角を成す線分Ａ１、Ａ２に対応するモデルにおける線分ａ１、ａ２の末端Ｓ３〜Ｓ６も求められる。そのｘｙｚ座標をｘ、ｙ、ｚに添字Ｓ３〜Ｓ６を付して示す。図２で示した線分Ａ１、Ａ２に対応するモデルにおける線分ａ１、ａ２の成す参照角θは次のように求められる。
【数１０】

【００７６】
算出したアングル定数が正しければ、図２で示した線分Ａ１，Ａ２により形成される参照角を実測した角度Θとの関係は、
Θ＝θ （４１）
である。
あるいはアングル定数に誤差Δθがある場合は
Θ＝θ＋Δθ （４２）
となる。
【００７７】
上式（４２）について誤差を左辺にし（３２）式に合わせた変数で近似式にすると
【数１１】

となる。なお、Ｒは近似残差で逐次計算上は無視する。本方程式ではΔβは無関係なので係数は０とする。
【００７８】
上式（４３）の左辺の各項の係数をＡ行列の追加行の要素とし（３３）式に含めて解く。なお、各項の係数はこの場合偏微分導関数としたが、次のように増分を求めることによってもよい。（４３）式の左辺第一項を例に示す。
【００７９】
θをｆ（ｘ）と表すと
【数１２】

となる。
この場合、各Δα０、Δσ０は例えばπ／１００００程度にすればよい。
【００８０】
以上のようにして参照角θとして画像内に設定した部分の角度が、実際の角度Θと同じ角度となるように拘束することで、他の部分の算出値も実際の値に近い値とすることができるので、本実施例によると精度よく写真測量を行うことができるようになる。
【００８１】
前述したように式（１１）の連立方程式はそれだけでも解を持ちカメラのアングル定数αａ０、αｂ０、σａ０、σｂ０、Δβを求めることもできる。しかし、この式（１１）に基づいた単なる測量計算ではカメラアングル定数の内の角σａ０およびσｂ０に同じ誤差があるとその誤差が大きくても正解と区別がつき難い場合がある。本発明は計算結果であるモデル座標上の角度、寸法を実際の角度（参照角θ）で点検、調整（拘束）することで、σａ０およびσｂ０を正しく拘束できるのでモデル座標の計算のみの場合と比較して測量計算の精度を向上させることができる。
【００８２】
なお、本実施例では参照角θを１つ設定する場合を説明したが、この参照角θを２つ設定するようにしてもよい。このように２つの参照角を設定するときには、例えば、カメラ基軸を含む面またはそれに近い面上に見出される三角形または四角形に含まれる角２つを特定または設定すればよい。ここでカメラ基軸を含む面またはそれに近い面とは、観測点ａ，ｂを水平に配置しほぼ水平方向に向けて撮影した場合に、地面、床面、屋上面などの水平面が該当する。例えば既知の参照角としてビルなどの方形状である建物の角部に着目し、屋上の四隅に含まれる２つの直角を参照角として利用することができる。また、タイル、フロア材など方形状である床材やこれに描かれた方形状の模様等を利用するようにしてもよい。
【００８３】
以上本発明の好ましい実施例について詳述したが、本発明は係る特定の実施形態に限定されるものではなく、特許請求の範囲に記載された本発明の要旨の範囲内において、種々の変形・変更が可能である。
【００８４】
【発明の効果】
以上詳述したところから明らかなように、発明によれば、参照角を利用することにより、測量対象物を高精度に測量することができる。
【図面の簡単な説明】
【図１】本発明の一実施例に係る写真測量方法を説明するために示した図である。
【図２】図１に示す写真測量方法においての前記参照角の設定例、及び各観測点から撮影される画像例を示した図である。
【図３】実施例の写真測量方法を実施するためのハードウェア構成の一例を示したブロック図である。
【図４】カメラ特性データを作成する処理工程を示したフローチャートである。
【図５】カメラ特性データを作成する様子を説明するために示した図である。
【図６】カメラ特性データを作成する際の画像の一部を示した図である。
【図７】測量作業での処理を示したフローチャートである。
【図８】測量点Ｐ１とＰｎとの関係及び三角測量計算を説明するために示した図であり、（Ａ）はＹ−Ｚ投影図、（Ｂ）はＸ−Ｙ投影図である。
【図９】図２に示した参照角θを形成する２つの線分Ａ１，Ａ２に対応するモデルとして線分ａ１、ａ２との関係を示した図である。
【符号の説明】
１ カメラ
３ 測量対象物（被写体）
５ａ、５ｂ カメラ画像
７ カメラ基軸
９ａ，９ｂ カメラ視線
ａ、ｂ 観測点
Ｐ１〜Ｐ５ アングル参照点
Ｐｎ 測量点
Ｓ１，Ｓ２ 寸法参照点
Ｓ３〜Ｓ６ 参照角規定点
θ 参照角[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to photogrammetry in which a survey object (subject) is photographed by a camera to perform surveying. In particular, the present invention relates to a photogrammetry method for accurately calculating coordinates of a survey target using image data from different directions obtained by free photographing of a camera.
[0002]
[Prior art]
As a conventional photogrammetry, for example, a photogrammetry using a stereo camera is known. In this surveying method, the same surveying object is photographed by two cameras fixed and separated from each other, and a surveyed point (hereinafter, referred to as “hereinafter”) on the surveying object is obtained from the obtained two images by using the principle of triangulation. (Referred to simply as survey points). The attitude and three-dimensional position of the camera at the time of photographing are measured manually or detected by a dedicated sensor, and are used for calculating the three-dimensional coordinates of the survey points.
[0003]
[Problems to be solved by the invention]
However, according to the conventional method, there are problems that the device becomes large-scale and the photographing operation becomes complicated. Therefore, in the present invention, the coordinates of the survey points are calculated with high accuracy using image data obtained by photographing an arbitrary posture and position without worrying about the posture and three-dimensional position of the camera at the time of photographing. The aim is to provide a method that can.
[0004]
[Means for Solving the Problems]
A first step of obtaining at least two different images by photographing with a camera from different observation points so as to include a survey object and a reference angle used when performing a survey calculation process of the survey object,
In common between the images, an angle reference point for specifying a survey point and a camera attitude at the observation point on the survey target is set, and a reference angle defining point that defines the reference angle is set. A second step of calculating two-dimensional coordinates in each of the images for the survey points, the angle reference points, and the reference angle defining points;
A three-dimensional coordinate system including the survey object is set, and based on the two-dimensional coordinates of the survey point, the angle reference point, and the reference angle defining point calculated in the second step, and the three-dimensional coordinate of the survey point, And a third step of calculating coordinates.
[0005]
According to the present invention, it is possible to calculate three-dimensional coordinates of a survey point by using an angle reference point even when the camera posture at the time of shooting is free, and to further improve accuracy by referring to a reference angle. Then, the coordinates can be calculated. That is, in the data processing of photogrammetry, the present invention improves the surveying accuracy by providing a corner for increasing the accuracy of the calculation result. More specifically, the present invention relates to a process of surveying calculation of an angle (reference angle) having a known angle or a known angle by actual measurement commonly projected in an image taken at a plurality of observation points for photogrammetry. In addition to the above, the calculation result is derived to be more accurate.
[0006]
The third step may be configured to calculate the three-dimensional coordinates of the survey point by adjusting the angle of the reference angle calculated by the survey calculation process to match the actual angle. desirable. In addition, the reference angle may be specified by setting the reference angle defining point on an object whose angle, which is commonly printed between the images, is known, and the first angle may be specified. Providing an actual measurement step between the step and the second step, wherein the reference angle is arbitrarily set to the reference angle defining point on an object commonly photographed between the images, and May be specified by actually measuring the angle of the angle defined by Further, in the second step, it is preferable that the two-dimensional coordinates are calculated using camera characteristic data that is converted into a value obtained by correcting optical distortion of the camera.
[0007]
Also, the present invention provides a first method of reading at least two different images by photographing from different observation points with a camera so as to include a survey object and a reference angle used for performing a survey calculation process on the survey object. Processing,
In common between the images, an angle reference point for specifying a survey point and a camera attitude at the observation point on the survey target is set, and a reference angle defining point that defines the reference angle is set. A second process of calculating two-dimensional coordinates in each of the images for the survey point, the angle reference point, and the reference angle defining point;
A three-dimensional coordinate system including the surveying object is set, and based on the two-dimensional coordinates of the survey point, the angle reference point, and the reference angle defining point calculated in the second process and the reference angle, the three-dimensional coordinates of the survey point are calculated. A photogrammetry program that causes a computer to execute the third process of calculating coordinates is included.
[0008]
The third process may be configured to calculate the three-dimensional coordinates of the survey point by adjusting the angle of the reference angle calculated by the survey calculation process to match the actual angle. desirable. In the second processing, it is preferable that the two-dimensional coordinates are calculated using camera characteristic data that is converted into a value obtained by correcting optical distortion of the camera.
[0009]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, an embodiment of the present invention will be described with reference to the drawings. According to the present invention, a reference angle (sharp) for confirming an angle (°) is set in image data obtained by camera photographing, and it is checked whether the value obtained by the survey calculation is accurate. This is a photogrammetry method that is used as a reference angle for obtaining a highly accurate result. The corners used in this way are specifically referred to herein as “reference angles”.
[0010]
More specifically, the present invention provides a surveying method in which a survey result can be more accurately guided by using a reference angle commonly printed in images taken from different observation points in a survey calculation process. It is. The angle (°) of the reference angle can be confirmed as a known value when the reference angle is set using a part of the object whose angle is known. When the surveyor arbitrarily sets the reference angle, the reference angle is actually measured and the value is confirmed.
[0011]
The reference angle may be arbitrarily set by the surveyor using a building or a natural object commonly printed in images taken from different observation points. For example, when a building is commonly photographed in a captured image, a reference angle (the angle in this case is generally 90 °) can be set using the corner of the building. Further, the surveyor may arbitrarily set the reference angle by using a stone, a tree, or the like commonly printed in the image. In this case, the reference angle is measured and the angle value (°) is known. As described above, the reference angle is an angle at which the angle can be confirmed by a known or actual measurement using a building or the like commonly printed in images captured at different observation points. It should be noted that a building, a natural object, or the like for which a reference angle is set is commonly included in images taken from different observation points regardless of whether or not the object is included in a survey target (hereinafter, referred to as a subject). It should just be. Hereinafter, a surveying method according to an embodiment of the present invention will be described step by step.
[0012]
FIG. 1 is a view for explaining a photogrammetry method according to an embodiment of the present invention. FIG. 2 is a diagram showing an example of setting the reference angle in the photogrammetry method shown in FIG. 1 and an example of an image taken from each observation point. In this photogrammetry method, as shown in FIGS. 1 and 2, an electronic still camera 1 having an image sensor such as a CCD is installed at different observation points a and b, and images having different directions are taken for each observation point. Suppose you do. On the subject 3, angle reference points P1 to P5 are set as clues for measuring the subject. The angle reference points P1 to P5 are used to calculate a constant (hereinafter, referred to as an angle constant) for indicating the positional relationship and attitude of the cameras 1 installed at different observation points a and b. The angle constant will be described later in detail.
[0013]
As shown in the figure, the angle reference points are set by selecting at least five points from the constituent parts of the subject 3 and the like. As shown in the lower part of FIG. 2, image data is created such that all of the five angle reference points P1 to P5 are imprinted on the camera images 5a and 5b taken at different observation points a and b. . Further, as shown in FIG. 2, other points are similarly photographed so as to be included in the left and right camera images 5a and 5b. Such points include a survey point Pn set on the subject 3, dimension reference points S1 and S2 for dimension reference, and a reference for obtaining two line segments set to obtain the above-described reference angle. The corner defined points S3 to S6 are included. Note that at least five angle reference points P1 to P5 set on the subject 3 for calculating the attitude of the camera 1 are required. A survey point for surveying the subject 3 is naturally set on the subject 3. The angle reference point can be treated as a survey point in the survey calculation process. Therefore, these points set on the subject 3 may be collectively referred to as a survey point Pn, or may be displayed as an angle reference point Pn for explanation.
[0014]
In this example, a reference angle is set using the four stones ST1 to ST4 rolling beside the subject 3, and this is used for surveying. In this example, the angle θ formed by the line segment A1 defined by the two stones ST1 and ST2 and the line segment A2 defined by the two stones ST3 and ST4 is the reference angle. That is, the positions of the stones ST1 to ST4 are the reference angle defining points S3 to S6, respectively, as shown in FIG.
[0015]
The line segments A1 and A2 are desirably set so that one is in a different direction, such as one in the perspective direction of imaging, and the other is in a direction parallel to the observation points a and b. It is desirable to specify or set two extremely obtuse and non-acute angles (for example, 90 ° and 45 °) on a plane including or near the camera base axis as the reference angle. These two angles are angles formed at positions at a certain distance from each other, and desirably exist on the same plane or a plane close to the same plane. The reference dimension S and the reference angle θ are actually measured at the time of photographing and recorded as predetermined data. However, when the reference angle θ is set using the corners of the building or the like as described above, for example, since this angle is known as 90 °, there is no need to actually measure the angle.
[0016]
With reference to FIG. 1, the photogrammetry of the present embodiment will be described step by step. FIG. 1 is a composite view of different diagrams (A) to (E) for explaining the surveying method. FIG. 1A is a conceptual plan view showing the positional relationship between the camera 1 and the subject 3 set at the observation points a and b. Here, a line connecting the observation points a and b is referred to as a camera base axis 7. The observation points a and b are given subscripts a and b. In addition, numbers 1 to 5 are attached as subscripts to the angle reference points. Further, directions representative of the direction of the camera 1 are referred to as camera lines of sight 9a and 9b, respectively. The items related to the camera line of sight are shown with a suffix 0 (zero).
[0017]
In addition, angles and lengths existing in the paper surface of FIG. 1A are indicated by αa0 or the like without parentheses () when an angle or length includes a three-dimensional component, that is, a component orthogonal to the paper surface. (Αa1), (αb1), etc. In FIG. 1A, angles formed by the camera lines of sight 9a and 9b with respect to the camera base axis 7 are denoted by αa0 and αb0, respectively. A plane including the camera base axis 7 and the camera line of sight 9a or 9b is called a camera line of sight. In FIG. 1A, the paper surface is a camera line of sight including the camera line of sight 9a (see the projection lines 13 in (B) and (C)).
[0018]
The angles of the angle reference points P1 to P5 with respect to the camera line of sight when rotated around the camera base axis 7 and measured at point a are rotation angles βa1,. Only shown). Similarly, those measured at point b are rotation angles βb1,..., Βb5 (collectively referred to as βbn; omitted in the figure). FIGS. 1B to 1E are shown in relation to FIG.
[0019]
FIG. 1B shows a view projected on a cross section orthogonal to the camera base axis 7. In FIG. 7B, the directions of the angle reference point P1 viewed from the observation points a and b and the rotation angles βa1 and βb1 around the camera base axis 7 are shown. Next, FIG. 1C is a projection view of a cross section orthogonal to the camera line of sight 9a. FIG. 1D is a diagram for explaining a state in which the inclination of the camera is considered in FIG. (D) is a plane on which coordinates of a virtual imaging plane Hf described later, that is, two-dimensional coordinates on the image are set. FIG. 1E is a diagram showing a state in which the image of the angle reference point P1 is projected on an imaging surface Hd of a camera such as a CCD sensor.
[0020]
Although the present invention is a method for performing photogrammetry with high accuracy using a reference angle as described above, a case where the attitude of a camera is obtained without using a reference angle will be described first for easy understanding. Then, a case in which photogrammetry is performed with high accuracy by using a characteristic reference angle in the present invention will be described. In this photogrammetry method, the above-described angle constant is used to obtain the camera attitude. As the angle constant, the angles αa0 and αb0 of the camera lines of sight 9a and 9b with respect to the camera base axis 7 described above with reference to FIG. 1, and the angle σa0 based on the camera line of sight with the inclination of the camera around the camera lines of sight 9a and 9b. (See FIG. 1 (D)), σb0, and the difference Δβ in the direction component of the rotation angle β that rotates vertically about the camera base axis 7 among the direction differences between the camera lines of sight 9a and 9b.
[0021]
By the way, when the camera 1 performs free shooting at different observation points a and b, the camera base axis 7 (a line connecting the observation points a and b) is not generally horizontal because the height positions of the cameras are different. However, here, the following description will be made on the assumption that the camera base axis 7 is horizontal for easy understanding. When the angle of rotation in the vertical direction about the camera base axis 7 is indicated by the rotation angle β, the rotation angles β in the direction of the angle reference points P1 to P5 with respect to the horizontal are the observation points a and The same is true regardless of which of b is measured. The angle constant is determined using this as a clue.
[0022]
As described above, βan and βbn are rotation angles based on the camera line-of-sight plane, and can be considered as follows when the clues based on the horizontal are represented by βan and βbn. At observation points a and b, it is assumed that there is a difference in rotation angle between the camera lines of sight 9a and 9b about the camera base axis 7 by Δβ (see FIG. 1B). Then, the difference between the rotation angle βa1 and the rotation angle βb1 for the same angle reference point P1 viewed from each of the observation points a and b is also Δβ. Similarly, the difference Δβn between the rotation angle βan and the rotation angle βbn for the other angle reference point Pn is equal to Δβ. Therefore, a number (five) of simultaneous equations corresponding to the number (five in this example) in which the angle reference points are set is set as an equation expressing Δβn by an angle constant, and the simultaneous equations are set on condition that all Δβn are equal. The angle constant can be obtained by solving.
[0023]
With reference to FIG. 1, a method of setting an equation to obtain an angle constant will be described. The image picked up by the camera 1 is formed on the CCD sensor on the side opposite to the subject 3 with the camera lens interposed therebetween, and the image is generally distorted due to lens characteristics. On the other hand, as shown in FIG. 1A, an imaging Hfa (or Hfb) that is established and photographed in front of the camera 1 (subject side) is imagined, and the optical characteristics of the imaging Hfa (or Hfb) are different from those of a pinhole camera. Make it not exist. The virtual imaging plane Hfa is a plane that vertically crosses the camera line of sight 9a, and the distance from the camera 1 is L.
[0024]
The angle reference point image Pf1 is formed at a position where a straight line connecting the center of the camera lens (which is regarded as a point like a hole of a pinhole camera) and the reference point P1 on the subject 3 intersects the virtual imaging plane Hf. It shall be. FIG. 1 (C) is a view obtained by projecting the virtual imaging surface Hf on which Pf1 forms an image from the observation point a direction, and the camera line of sight, that is, the paper surface of FIG. I have. Here, reference numeral Cf is a projection point of the camera line of sight 9a, and if this point Cf is set as the center of the origin (hereinafter simply referred to as center Cf), P1 is represented by a two-dimensional coordinate position (Xt, Yt). . Note that Pcc is a projection point of the camera base axis 7 and is shown for reference, but does not exist on an actual image. Also, the position of Pcc may be outside the screen.
[0025]
In the case of free shooting, the camera 1 may be inclined around the camera line of sight 9a or 9b. FIG. 1 (D) shows that the horizontal axis 15 of the screen of the camera 1 is inclined by an angle σa0 at the observation point a with respect to the projection line 13 of the camera line of sight in consideration of the above. Is the coordinate position (Xs, Ys) when the horizontal axis 15 of the screen of the camera 1 is set to the X axis on the XY plane. FIG. 1E illustrates an angle reference point Pd1 captured on the CCD imaging surface Hd of the camera 1, and the position of Pd1 indicates a coordinate position (Xp, Yp) in pixel units.
[0026]
In FIG. 1 (D), the angle between the line connecting the center Cf and the angle reference point image Pf1 and the horizontal axis 15 and the angle between the projection line 13 on the camera viewing plane are σs and σt, respectively, and connect Cf and Pf1. Assuming that the length of the line is d1 and the distance from the camera base axis 7 to the angle reference point image Pf1 on the virtual imaging plane Hf in FIG. 1B is h, the following relationship is established from FIG. .
[0027]
σs = tan^-1(Ys / Xs) (1)
σt = σs-σa0 (2)
d1 = √ (Ys²+ Xs²(3)
Xt = d1 · cosσt (4)
Yt = d1 · sinσt (5)
Xt = Xs · cosσa0 + Ys · sinσa0 (4) ′
Yt = −Xs · sinσa0 + Ys · cosσa0 (5) ′
Xu = L · sinαa0−Xt · cosαa0 (6)
Yu = Yt (7)
Zu = L · cosαa0 + Xt · sinαa0 (8)
h = √ (Xu²+ Yu²) (9)
βa1 = tan^-1(Yu / Xu) (10)
Note that the above equations (4) and (5) may be replaced with the above equations (4) 'and (5)'. The same relational expression holds for the observation point b. In that case, the suffix a may be changed to the suffix b in the above equation (example: σa0 → σb0, αa0 → αb0).
[0028]
The difference between the rotation angles βan and βbn obtained as described above is obtained for each angle reference point, and an equation is set up to make the differences equal. Since the number of unknowns to be obtained is five, for example, five simultaneous equations can be set as follows.
Δβ = βa1-βb1
Δβ = βa2-βb2
Δβ = βa3-βb3
Δβ = βa4-βb4
Δβ = βa5-βb5 (11)
[0029]
By solving the above simultaneous equations, angle constants αa0, αb0, σa0, σb0, Δβ representing the attitude of the camera can be obtained. Based on these, the coordinates of each survey point can be calculated from the principle of triangulation. When the rotation angles βan and βbn are viewed as relative values based on one of the rotation angles, the relative positions are equal between the two observation points a and b. For example, assuming that the angle reference point P1 is the reference direction, the following relationship is established between the rotation angles βn of the respective points n viewed from the observation points a and b. Even if (11) ′ is set, the angle constant can be similarly obtained.
βan-βa1 = βbn-βb1 (11) ′
Through the above processing, the angle constants αa0, αb0, σa0, σb0, and Δβ are obtained. Since these are solutions having a certain degree of accuracy, the coordinates of the surveying points can be obtained temporarily based on these solutions.
[0030]
However, in the present embodiment, a new method of simultaneously adding and adding equations in consideration of the reference angle is adopted in order to more accurately obtain the coordinates of the survey points. This method calculates model coordinates (similar coordinates) of two end points (reference angle defining points) of a line segment forming a reference angle by using equations (20) to (25) described later, and calculates the coordinates of the two-line model. Obtain an angle corresponding to the reference angle. Then, an equation under the condition that the angle (°) of the reference angle て い る at which the angle (°) is actually measured and confirmed and the reference angle θ in the model is the same is added to the equation (11). As described above, the simultaneous equation of the equation (11) has a solution by itself, but the additional equation is called a constraint equation in the sense that the equation is forcibly restricted under another condition. That is, the reference angle used in the present embodiment is used as a kind of reference angle, and the calculation result is set such that the angle (°) of the reference angle calculated by the survey calculation process matches the actual angle (°). It is guided so that the survey result becomes a more correct value by restricting. The additional constraint condition expression will be described in detail in the description of the successive approximation solution described later.
[0031]
Here, hardware for implementing the photogrammetry method according to the present invention using image data obtained by photographing with a camera will be briefly described. FIG. 3 is a block diagram illustrating an example of a hardware configuration for implementing the photogrammetry method according to the embodiment. As a device having such a configuration, an information processing device such as a personal computer 30 can be used. As shown in FIG. 3, a central processing unit (CPU) 21 for controlling the entire system includes a keyboard 25, a mouse 27, a display (CRT or LCD) 29, a printer 31, an IC card reader via an interface unit 23. 33 are connected. For example, an image captured by a digital camera is input via the IC card 34. The image can be confirmed on the display 29. A photograph taken by a normal camera is converted into a digital image by a film scanner 35 and input. A hard disk 37 is connected to the CPU 21. The hard disk 37 has a computer program for performing photogrammetry, characteristic data described later, data input from the keyboard 25, the mouse 27, the IC card reader 33, and the CPU 21. And the like are stored.
[0032]
Further, here, camera characteristic data (hereinafter, simply referred to as characteristic data) employed in the present embodiment will be described. Referring to FIG. 1, if the camera 1 is a pinhole camera, an image point on an imaging plane corresponding to a straight line passing through the angle reference points P1 to P5, the dimension reference points S1 and S2, or the survey point Pn and the pinhole. Are located respectively. However, the camera 1 is a camera having a taking lens. Therefore, the image point may be projected to a position deviating from the straight line due to distortion of the photographing lens. In addition, distortion may occur due to misalignment of the pixel arrangement on the image receiving element CCD. Note that the same applies to the reference angle defining points S3 to S6, but illustration is omitted here.
[0033]
For this reason, in this example, the virtual imaging surface Hf without lens distortion is set as a two-dimensional coordinate on the image at a position separated from the observation point a (or b) by the distance L toward the subject (FIGS. 1A and 1B). B)). The points at which each reference point is projected on this virtual imaging plane Hf are defined as virtual image points Pf1 to Pf5, Sf1, Sf2, and Pf (n), respectively, and the image of the reference point on the light receiving element CCD (or on the image). Characteristic data indicating the corresponding positional relationship between the points Pd1 to Pd5, Sd1, Sd2, and Pd (n) and the virtual image point is prepared in advance (see FIG. 1D). FIG. 1D exemplarily shows only the virtual image point Pf1.
[0034]
Then, using the characteristic data, the two-dimensional coordinates of the image points Pd1 to Pd5, Sd1, Sd2, and Pd (n) are converted to the virtual image points Pf1 to Pf5, Sf1, Sf2, and Pf (n) on the virtual imaging surface Hf. Convert to dimensional coordinates. That is, the two-dimensional coordinates of the virtual image point are obtained by correcting the distortion of the photographing lens with respect to the two-dimensional coordinates of the image point. These virtual image points are used for surveying calculation assuming that they are located on a straight line passing through the reference point and the like and the pinhole.
[0035]
FIG. 4 is a flowchart showing the processing steps for creating the camera characteristic data. The process of creating characteristic data will be described with reference to FIG. In step S101, the graph paper 50 (which is the source of the coordinates of the virtual imaging surface Hf) is photographed using the camera 1 as shown in FIG. The camera 1 is placed substantially at the center of the graph paper 50 so that the camera line of sight 9 is orthogonal to the graph paper 50. Then, the distance L from the observation point C, which is the position where the lens 51 is equivalent to the pinhole of the pinhole camera, to the graph paper 50 is measured.
[0036]
More specifically, the graph paper 50 is placed horizontally, and the camera 1 is placed vertically above substantially the center with a tripod (not shown) with the camera 1 facing the graph paper 50. At this time, the horizontal and vertical directions of the visual field of the camera 1 are made to substantially match the horizontal and vertical directions of the graph paper 50. After the grid paper 50 and the camera 1 are installed, a first photographing is performed. Subsequently, a plane mirror 52 (shown by a broken line in the figure) is further provided between the graph paper 50 and the camera 1 in a state where the relative positional relationship between the graph paper 50 and the camera 1 does not change. Specifically, the plane mirror 52 is placed so that the mirror surface faces the camera 1 and is parallel to the graph paper 50. Then, it is checked that the mirror image of the observation point C is visually recognized from the viewfinder or the monitor of the camera 1 so that the camera line of sight 9 is orthogonal to the graph paper 50. The second shooting is performed. An image obtained by the first shooting (hereinafter, referred to as a first reference image) and an image obtained by the second shooting (hereinafter, referred to as a second reference image) are attached to the second reference image by a camera. The images are the same except that one mirror image is shown. Each image data is stored in the IC card 34 of FIG.
[0037]
Steps S102 to S104 are processing using the personal computer 30 shown in FIG. In step S102, a photogrammetry program is started in the personal computer 30, a characteristic data generation process is selected from a menu prepared in the program, and the first and second reference images obtained in step S101 are stored in the IC card shown in FIG. The data is read into the personal computer 30 by the reading device 33 or the like and displayed on the screen of the display device 29. FIG. 6 shows a first reference screen displayed on the screen, that is, a partial image 60 of the image of the grid paper 50.
[0038]
In step S102, for each of the K grid intersections (hereinafter referred to as grid points) on the graph paper 50, the position on the screen is designated by the mouse 27, and the respective pixel coordinates are read and designated by the mouse 27. The keyboard 25 inputs the relative two-dimensional coordinates in which the grid points of the grid paper 50 corresponding to the set grid points are regarded as coordinates. In FIG. 6, grid points designated by the mouse are indicated by circles.
[0039]
In FIG. 5, an arbitrary grid point is defined as T (k) (the subscript k is a variable indicating an arbitrary grid point and takes an integer value of 1 or more and the number of grid points K or less), and its relative two-dimensional coordinates are ( XT (k), YT (k)). XT (k) is an actual measured distance in the horizontal direction from an arbitrary lattice point T0 on the graph paper 50 to the lattice point T (k), and YT (k) is a lattice point T (k) from the origin T0. ) Is the measured distance in the vertical direction up to. The actual measurement may be measured on a scale of graph paper. The grid point t (k) projected on the grid paper image 60 on the screen is regarded as an image point t (k) obtained by projecting the grid point T (k) onto the imaging surface Hd, and the pixel of this image point t (k) The coordinates can be represented by (xt (k), yt (k)). xt (k) is the number of horizontal pixels from an arbitrary pixel on the screen, for example, the pixel tp at the upper right corner to the image point t (k) from the origin, and yt (k) is the image point t ( k) is the number of pixels in the vertical direction up to k).
[0040]
In step S103, the position of the mirror image of the observation point C on the screen is designated by the mouse 27 as the center image Cd of the imaging surface Hd with reference to the second reference image, and the pixel coordinates (xc, yc) are read. . The center image Cd is a point at which a point at which the camera line of sight 9 passing through the observation point C is orthogonal to the mirror surface of the plane mirror 52 is projected on the imaging surface Hd. Since the mirror surface of the plane mirror 52 and the graph paper 50 are parallel to each other, the center image Cd is obtained by projecting a point where the camera line of sight 9 is orthogonal to the graph paper 50 (hereinafter referred to as the center Cf) on the imaging surface Hd. Become. Note that the term “center” here means that the position through which the camera line of sight passes is considered as the center of the image pickup, and does not mean the center of the screen. Further, since the imaging conditions are the same in the first and second reference images, the imaging center Cd (xc, yc) specified in the second reference image is the center image of the first reference image (indicated by the same reference sign Cd). Can be considered
[0041]
The relative two-dimensional coordinates of the center Cf on the graph paper 50 are represented by a horizontal distance Xc and a vertical distance Yc from the origin T0 to the center Cf. In step S104, the measured shooting distance L is input from the keyboard 25. Subsequently, in step S105, the relative two-dimensional coordinates (XT (k), YT (k)) on the graph paper 50 and the image point t on the imaging plane Hd are obtained for all grid points T (k) (k = 1 to K). A series of coordinate data elements {XT (k), YT (k), xt (k), xT (k) for the grid point T (k) combined with the pixel coordinates (xt (k), yt (k)) of (k) yt (k)}, Cf coordinate data elements (Xc, Yc, xc, yc), and distance L are stored as characteristic data in an external storage medium such as a hard disk 37 or a flexible disk. Note that Xc and Yc may be difficult to read from the scale of the graph paper, and thus may be obtained by a calculation described later without being input here.
[0042]
By using the characteristic data, for example, a point Pf (n) located at a position xf (n) in the horizontal direction and yf (n) in the vertical direction from the center Cf on the graph paper 50 is located on the imaging surface Hd. If it is defined that the image is projected to an image point Pd (n) separated by xd (n) pixels in the horizontal direction and yd (n) pixels in the vertical direction from the origin tp, the image is projected on the screen in an image obtained by survey photography. When the image of the survey point Pn on the subject coincides with the image point Pd (n), the virtual image point corresponding to the survey point Pn is considered to coincide with the point Pf (n). In other words, the point Pf (n) where the observation point C matches the virtual image point and the survey point Pn are considered to be on the same straight line. The above-described characteristic data is obtained by converting the pixel coordinates (xd (n), yd (n)) of the image point Pd (n) into the two-dimensional coordinates (hereinafter, virtual image point) of the virtual image point Pf (n) corrected for optical distortion and the like. (Referred to as point coordinates) (xf (n), yf (n)).
[0043]
It is desirable that the characteristic data be prepared for all the pixels of the CCD. However, in the present embodiment, in order to save the data amount and the calculation time, the characteristic data is obtained only for the lattice points at predetermined intervals as described above. The image points are generated, and the image points located therebetween are calculated by an interpolation method. In this embodiment, a mirror image of the lens is separately photographed. However, by using grid paper which is transparent except for the grid, such as forming a grid on transparent paper, this is placed on a mirror and the grid and the lens are formed in one photograph. May be obtained. In the present embodiment, the lens data is corrected by the characteristic data to correct the coordinate value error caused by the lens assembly error with respect to the CCD or the misalignment of the CCD pixels. It can be easily corrected without any calculation. In addition, since the characteristic data is obtained by photographing graph paper and specifying the grid points on the image plane, even a general user can observe the state of distortion and be interested, and without requiring advanced technology, can obtain characteristic data. Can be easily created. Since the corresponding positional relationship between the image point Pd (n) and the virtual image point Pf (n) changes according to the photographing conditions such as the focal length and magnification of the camera 1, the photographing conditions to be set when photographing the subject 3 It is desirable to prepare the characteristic data corresponding to.
[0044]
When the characteristic data is prepared as described above, the surveying operation is started. FIG. 7 is a flowchart showing a process in the surveying operation. In step S201, image data 5a and 5b of the subject 3 photographed from two directions are created by using the camera 1 as shown in FIG. 2 and stored in the IC card 34 of FIG. In addition, a reference dimension between the dimension reference points S1 and S2 is actually measured. In addition, the fact that the above-mentioned reference angle θ is used in the survey calculation processing will be described later, but it is actually measured at this time. When the reference angle θ is set using windows of an existing building or the like, the angle (°) is known, so that it is not necessary to actually measure it.
[0045]
Subsequent steps S202 to S208 are processing using the personal computer 30. In step S202, a photogrammetry program is started in the personal computer 30, surveying calculation processing is selected from a menu prepared in the program, and image data is read by the personal computer 30 by the IC card reader 33 or the like in FIG. It is displayed on the screen of the display device 29.
[0046]
Pixel positions on the screen of the angle reference points P1 to P5, the survey point Pn, the reference angle defining points S3 to S6, and the dimension reference points S1 and S2 common to each image data are designated and input by the mouse 27. The input is performed for each image of the observation points a and b. For example, by pointing and clicking a point image to be measured with a pointer using the mouse 27 or the like, the position data (represented by a pixel unit pixel on the CCD image plane) is recorded in the HDD 37. Instructions and reference angles for which points should be the angle reference points P1 to P5, which points should be the dimension reference points S1 and S2, or the reference angle defining points S3 to S6 are input in advance through the keyboard 25 and input to the HDD 37, for example. It is stored as a reference file, and read out from this file in steps that require them.
[0047]
In step S203, the pixel coordinates of the angle reference points P1 to P5, the survey point Pn, the dimension reference points S1 and S2, and the reference angle defining points S3 to S6 at the observation points a and b are converted into the two-dimensional coordinates of the virtual image point coordinates by the characteristic data. Is converted. Here, the coordinate conversion based on the characteristic data will be described in detail. For example, the image point t (k) corresponding to the image point Pd1 input in the previous step S202, that is, the image point t (k) having the same pixel coordinates as the pixel coordinates (xd1, yd1) in the coordinate data element of the characteristic data ), The relative two-dimensional coordinates (XT (k), YT (k)) of the corresponding grid point T (k) of the same element are given as virtual image point coordinates Pf1 (xf1, yf1). However, since Pf1 needs to be a value with the center Cf as the origin, Xc and Yc are actually subtracted. On the other hand, when there is no image point t (k) that matches the image point Pd1, for example, when the image point Pd (n) is located between the lattices as shown in FIG. .
[0048]
When the image point t (k) (xt (k), yt (k)) of the coordinate data element closest to the image point Pd (n) (xd (n), yd (n)) is selected, two image points Pd Since (n), t (k) and the corresponding virtual image points Pf (n), T (k) are close to each other, it can be considered that a linear equation is established, and the virtual image point coordinates Pf (n) (xf (n ), Yf (n)) are obtained by the following equations (12) and (13).
[0049]

Note that xd (n), yd (n), xf (n) and yf (n) correspond to Xp, Yp, Xs and Ys in FIG. 1, respectively.
[0050]
The coefficients a1 and a2 in the above equation (12) are, for example, the characteristic data elements ｛XT (k + 1), YT () of the image point t (k + 1) located vertically above the image point t (k) in FIG. k + 1), xt (k + 1), yt (k + 1)} and characteristic data elements {XT (k + 2), YT (k + 2), xt (k + 2), yt (k + 2) of the image point t (k + 2) located on the left side in the horizontal direction. ) Calculated based on｝.
[0051]
More specifically, the following simultaneous equations (14) and (15) hold for the coefficients a1 and a2.
ΔX1 = a1Δx1 + a2Δy1 (14)
ΔX2 = a1Δx2 + a2Δy2 (15)
However,
ΔX1 = XT (k + 1) −XT (k) ΔY1 = YT (k + 1) −YT (k)
ΔX2 = XT (k + 2) −XT (k) ΔY2 = YT (k + 2) −YT (k)
Δx1 = xt (k + 1) −xt (k) Δy1 = yt (k + 1) −yt (k)
Δx2 = xt (k + 2) −xt (k) Δy2 = yt (k + 2) −yt (k)
It is.
[0052]
Therefore, the coefficients a1 and a2 are obtained by the following equations (16) and (17). The coefficients a3 and a4 in the equation (13) are also obtained by the following equations (18) and (19), similarly to the coefficients a1 and a2.
a1 = (ΔX1Δy2−ΔX2Δy1) / M (16)
a2 = (Δx1ΔX2-Δx2ΔX1) / M (17)
a3 = (ΔY1 · Δy2-ΔY2 · Δy1) / M (18)
a4 = (Δx1ΔY2−Δx2ΔY1) / M (19)
Here, M = Δx1 · Δy2-Δx2 · Δy1.
[0053]
In step S203, when the pixel coordinates (xc, yc) of the center image Cd are exchanged with the relative two-dimensional coordinates (Xc, Yc) of the center Cf, the term (-Xc) is calculated from the right side of the equation (12). And the expression excluding the term (−Yc) from the right side of the expression (13).
[0054]
In the next step 204, an angle constant is determined. The simultaneous equations (11) are solved using the data of the angle reference points P1 to P5, and the camera angle constants αa0, αb0, σa0, σb0, Δβ are obtained. One of the solutions of this simultaneous equation is a successive approximation solution. A specific example of this successive approximation solution will be described later in detail. When the angle constant is obtained as described above, triangulation is calculated based on the obtained angle constant.
[0055]
In the next step S205, data necessary for three-dimensional triangulation calculation is prepared. That is, angles αan and αbn from the observation points a and b to the survey point with respect to the camera base axis 7 and a rotation angle β from the reference direction common to the observation points a and b to the survey point are determined (FIG. 8 described later). See). The triangulation calculation is performed for the dimension reference points S1 and S2, the reference angle defining points S3 to S6, and the angle reference points P1 to P5 in addition to the survey point Pn. In the calculation, as shown in step S203, the pixel position (Xp, Yp) of the imaging plane Hd is converted to the position (Xs, Ys) on the virtual imaging plane Hf using the camera characteristic file for the survey point Pn.
[0056]
The angle αa (n) from the observation point a to the survey point P1 can be calculated by the following equation (20) as is clear from FIGS. 1 (A) and 1 (B).
αa (n) = tan^{− 1}(H / Zu) (20)
Here, h / Zu is obtained from Expressions (8) and (9). The angle αb (n) from the observation point b to the survey point Pn is obtained in the same manner.
[0057]
Regarding the rotation angle β, angles βa (n) and βb (n) can be obtained from Expression (10), which are rotation angles around the camera base axis 7 respectively. Since what is needed for triangulation is a rotation angle from the reference direction common to the observation points a and b, a common rotation angle is obtained by arbitrarily using the points projected on both cameras. For example, as shown in FIG. 8, a rotation angle β′a (n) = βa (n) −βa1 with reference to the survey point P1 is obtained and used. Although the rotation angle β′b (n) = βb (n) −βb1 can be obtained, β′a (n) = β′b (n), so that if there is no error, only one of them needs to be obtained.
[0058]
In step S206, first, a model is calculated based on the data in step S205. For triangulation, the distance between observation points a and b is required. However, here, it is assumed that the distance is 1 and the similar three-dimensional coordinates (called a model) are obtained. As shown in FIG. 8, the orthogonal coordinates (x, y, z) of the survey point Pn with the observation point a as the origin can be obtained from the following equations (21) to (24) based on the principle of triangulation. However, the survey points have a relationship as shown in FIG. FIG. 8 shows the survey points P1 (which are also angle reference points) and Pn in the YZ projection view in (A) and the XY projection view in (B). It is assumed that the survey point P1 serving as a reference for the rotation angle β is on the paper surface of FIG. 8B, and the camera base axis 7 is in the X-axis direction. In FIG. 8B, the paper surface is an XY plane, the axis perpendicular to the XY plane is the Z-axis direction, the distance between the observation points a and b is E, and the distance from the X axis to the survey point Pn is H.
[0059]
(Equation 1)

At this time, the distance E between the observation points a and b is temporarily set to 1. The above is also obtained for P1 to P5 and S1 to S6 to form a model.
[0060]
In step S207, the distance E, which is the provisional value, is corrected so as to be equivalent to the case where the distance E is the actual value. First, for the two dimension reference points (S1, S2) shown in FIG. 2 where the distance is actually measured in step S201, the distance g is obtained from the obtained model, that is, the coordinate value of the similar shape, by the following equation. The subscripts S1 and S2 in the equation (25) indicate the coordinates of the model of the dimension reference points S1 and S2.
(Equation 2)

[0061]
Next, the ratio between the previously measured reference dimension S (the actual interval between S1 and S2) and the interval g is determined. The ratio is multiplied by each coordinate value of the model to calculate a numerical value. Through the above steps, the actual value in the three-dimensional coordinates of the survey point Pn and other reference points with the observation point a as the origin is calculated.
[0062]
However, it may be inconvenient if the origin of the calculated coordinates is the observation point a, the plane including P1 is fixed on the XY plane, and the X axis is fixed on the camera base axis 7. If necessary, adjust the origin and reference direction. For example, a certain point among the survey points is set as the origin, or the reference direction of XY is designated and changed. For this purpose, for example, a coordinate reference instruction file is stored in the HDD 37 in advance. That is, the data file is a data file in which a point set as the origin, a point defining the X-axis, a point defining the Y-axis direction, or the like is selected from among reference points and the like and is input from a keyboard or the like. It is desirable to refer to the file so that the three-dimensional coordinates obtained are translated or rotated to perform coordinate conversion.
[0063]
In step S208, the result of the above-described survey calculation is displayed on the display device 29, output to the printer 31 or the like, and read by the user. In addition, it can be output as a file for CAD or the like according to the application, and can be used as skeleton data of CAD or CG.
[0064]
Before and after, a successive approximation solution for obtaining the angle constants αa0, αb0, σa0, σb0, and Δβ from equation (11) in step S204 described above will be described. When Equations (1) to (10) are modified to obtain βan (also common to βbn),
(Equation 3)

Becomes For the following calculations, the parentheses on the right side of equation (26)
(Equation 4)

And
[0065]
By substituting the equations (11) into the above equation (26) by appropriately adding the suffixes a and b indicating the observation point and the suffixes 1 to 5 indicating the individual angle reference points, the variable can be expressed in detail. Is omitted.
Expression (11), which is expressed only by an angle constant, is represented by a function f.
Δβ = βan−βbn = f (αa0, σa0, αb0, σb0) (28)
Becomes
[0066]
Deriving the partial derivatives of αa0 and σa0 to make the function f a linear approximation so that the simultaneous equations can be solved,
(Equation 5)

Becomes
The partial derivatives of αb0 and σb0 can be similarly expressed by replacing the subscripts a and b in the equations (29) and (30).
[0067]
Combining the above equations (29) and (30) to form a first-order approximation for equation (11)
(Equation 6)

Becomes
[0068]
In the above equation (31), fn is βan−βbn at the current angle constant value of the angle reference point Pn, each partial derivative equation is a partial differential coefficient at the current angle constant of Pn, and Δαa0, Δσa0, Δαb0, and Δσb0 are the current Each increment from the angle constant. R represents an error due to omitting components exceeding the first order in this approximate expression. This R is ignored. Note that the expression “present” in the text before and after this means an initial value for each successive approximation.
[0069]
Expressing each partial differential coefficient as a matrix element Ank and fn as a matrix element Bn, and applying them to all the expressions of the above expression (11) to rearrange them
(Equation 7)

Becomes
Here, n is the number of angle reference points, and An5 is −1, which is the sign of Δβ, and is always constant.
[0070]
The above equation (32) is expressed by a matrix x in which the increments Δαa0, Δσa0, Δαb0, Δσb0, Δβ of the angle constants to be obtained are elements xn of the matrix.
Ax = B (33)
Here, A is a matrix having Ank as an element, and B is a matrix having Bn as an element.
[0071]
In equation (11), five angle reference points are used and five equations are simultaneously used. However, when six or more points are used, the number is simultaneously calculated and converted into five rows by the following equation (least square method). .
(Equation 8)

Here, A bar is a transposed matrix of A, and A 'and B' are matrices converted into 5 rows.
[0072]
To determine the increment of the angle constant, that is, the element of x, is based on the Kramel's formula as follows.
(Equation 9)

Where | A ‘| is the determinant of A ′, | A‘_{B '}| Is a determinant in which the column relating to xn of | A ‘| is replaced with B ′.
[0073]
The obtained xn, that is, the increments of the angle constants Δαa0, Δσa0, Δαb0, Δσb0 are added to the current angle constants αa0, σa0, αb0, σb0 to obtain the next current angle constants αa0, σa0, αb0, σb0. Repeat the above. The repetition is performed until the maximum deviation of Δβ falls below a specified value that can be regarded as a solution or a specific number of times.
[0074]
Here, the above-described additional constraint condition expression will be described in detail. FIG. 9 is a diagram showing a relationship between line segments a1 and a2 as models corresponding to two line segments A1 and A2 forming the reference angle θ shown in FIG. In FIG. 9A, a1 'is parallel to a2 at a position where the two lines are closest in order to accurately indicate the angle θ formed by the two lines when the two line segments are separated from each other. This is the line that has moved. FIG. 9B is a diagram in which the line segment a2 in FIG. 9A is translated to a line segment a2 'so that s3 and s5 coincide to calculate θ. L1 to L3 indicate the length of each side of the triangle formed by each line segment.
[0075]
The coordinates in the reference point model such as the survey point are obtained by the above-described equations (1) to (11) and equations (20) to (24). Similarly, the ends S3 to S6 of the line segments a1 and a2 in the model corresponding to the line segments A1 and A2 forming the reference angle are obtained. The xyz coordinates are shown by adding subscripts S3 to S6 to x, y, and z. The reference angle θ formed by the line segments a1 and a2 in the model corresponding to the line segments A1 and A2 shown in FIG. 2 is obtained as follows.
(Equation 10)

[0076]
If the calculated angle constant is correct, the relationship with the actually measured reference angle Θ formed by the line segments A1 and A2 shown in FIG.
Θ = θ (41)
It is.
Or, if the angle constant has an error Δθ,
Θ = θ + Δθ (42)
Becomes
[0077]
In the above equation (42), the error is set to the left side, and the approximation equation is obtained by using the variable adjusted to the equation (32)
(Equation 11)

Becomes Note that R is an approximate residual and is ignored in successive calculations. In this equation, the coefficient is set to 0 because Δβ is irrelevant.
[0078]
The coefficient of each term on the left side of the above equation (43) is set as an element of an additional row of the A matrix, and is solved by being included in the equation (33). In this case, the coefficient of each term is a partial differential derivative. However, the coefficient may be obtained by calculating an increment as follows. The first term on the left side of equation (43) is shown as an example.
[0079]
When θ is expressed as f (x),
(Equation 12)

Becomes
In this case, each of Δα0 and Δσ0 may be set to, for example, about π / 10000.
[0080]
By constraining the angle of the portion set in the image as the reference angle θ as described above to be the same angle as the actual angle Θ, the calculated values of the other portions are also close to the actual values. According to this embodiment, photogrammetry can be performed with high accuracy.
[0081]
As described above, the system of equations (11) alone has a solution, and the camera angle constants αa0, αb0, σa0, σb0, and Δβ can be obtained. However, in the mere survey calculation based on the equation (11), if the angles σa0 and σb0 of the camera angle constant have the same error, it may be difficult to distinguish the correct answer even if the error is large. According to the present invention, σa0 and σb0 can be correctly constrained by checking and adjusting (constraining) the angles and dimensions on the model coordinates, which are the calculation results, at the actual angles (reference angles θ). In comparison, the accuracy of the survey calculation can be improved.
[0082]
In the present embodiment, a case where one reference angle θ is set has been described, but two reference angles θ may be set. When two reference angles are set in this way, for example, two angles included in a triangle or a quadrangle found on a plane including the camera base axis or a plane close to the plane may be specified or set. Here, the plane including the camera base axis or a plane close thereto corresponds to a horizontal plane such as the ground, a floor, or a roof when the observation points a and b are arranged horizontally and photographed in a substantially horizontal direction. For example, focusing on a corner of a rectangular building such as a building as a known reference angle, two right angles included in the four corners of the roof can be used as reference angles. Alternatively, a square floor material such as a tile or a floor material, a square pattern drawn on the floor material, or the like may be used.
[0083]
Although the preferred embodiment of the present invention has been described in detail above, the present invention is not limited to the specific embodiment, and various modifications and changes may be made within the scope of the present invention described in the appended claims. Changes are possible.
[0084]
【The invention's effect】
As is apparent from the details described above, according to the present invention, the survey target can be measured with high accuracy by using the reference angle.
[Brief description of the drawings]
FIG. 1 is a view for explaining a photogrammetry method according to an embodiment of the present invention.
FIG. 2 is a diagram showing an example of setting the reference angles in the photogrammetry method shown in FIG. 1, and an example of an image taken from each observation point.
FIG. 3 is a block diagram showing an example of a hardware configuration for implementing the photogrammetry method of the embodiment.
FIG. 4 is a flowchart showing processing steps for creating camera characteristic data.
FIG. 5 is a diagram illustrating a state in which camera characteristic data is created.
FIG. 6 is a diagram showing a part of an image when camera characteristic data is created.
FIG. 7 is a flowchart showing a process in a surveying operation.
8A and 8B are diagrams illustrating a relationship between the survey points P1 and Pn and triangulation calculation, wherein FIG. 8A is a YZ projection diagram and FIG. 8B is an XY projection diagram.
9 is a diagram showing a relationship between line segments a1 and a2 as models corresponding to two line segments A1 and A2 forming the reference angle θ shown in FIG. 2;
[Explanation of symbols]
1 camera
3 Survey object (subject)
5a, 5b camera image
7 Camera base
9a, 9b Camera look
a, b observation points
P1 to P5 Angle reference point
Pn survey point
S1, S2 Dimension reference point
S3 to S6 Reference angle specified point
θ reference angle

Claims (8)

A first step of obtaining at least two different images by photographing with a camera from different observation points so as to include a survey object and a reference angle used when performing a survey calculation process of the survey object;
In common between the images, an angle reference point for specifying a survey point and a camera attitude at the observation point on the survey target is set, and a reference angle defining point that defines the reference angle is set. A second step of calculating two-dimensional coordinates in each of the images for the survey point, the angle reference point, and the reference angle defining point;
A three-dimensional coordinate system including the survey object is set, and based on the two-dimensional coordinates of the survey point, the angle reference point, and the reference angle defining point calculated in the second step, and the three-dimensional coordinate of the survey point, A third step of calculating coordinates. The method according to claim 1, wherein in the third step, three-dimensional coordinates of the survey point are calculated by adjusting the angle of the reference angle calculated by the survey calculation process so as to match the actual angle. The photogrammetric method described. 2. The method according to claim 1, wherein the reference angle is specified by setting the reference angle defining point on an object having a known angle that is commonly captured between the images. 3. Photogrammetry method. An actual measurement step is provided between the first step and the second step, and the reference angle is arbitrarily set on the object commonly photographed between the images, and the reference angle specified point is set. The photogrammetry method according to claim 1, wherein the angle is specified by actually measuring an angle defined by the angle defining point. The photograph according to any one of claims 1 to 4, wherein, in the second step, the two-dimensional coordinates are calculated using camera characteristic data that is converted into a value obtained by correcting optical distortion of the camera. Survey method. A first process of reading at least two different images taken by a camera from different observation points, so as to include a survey target and a reference angle used when performing a survey calculation process of the survey target;
In common between the images, an angle reference point for specifying a survey point and a camera attitude at the observation point on the survey target is set, and a reference angle defining point that defines the reference angle is set. A second process of calculating two-dimensional coordinates in each of the images for the survey point, the angle reference point, and the reference angle defining point;
A three-dimensional coordinate system including the surveying object is set, and based on the two-dimensional coordinates of the survey point, the angle reference point, and the reference angle defining point calculated in the second process and the reference angle, the three-dimensional coordinates of the survey point are calculated. A photogrammetry program causing a computer to execute a third process of calculating coordinates. 7. The method according to claim 6, wherein in the third process, three-dimensional coordinates of the survey point are calculated by adjusting the angle of the reference angle calculated by the survey calculation process so as to match the actual angle. Photogrammetry program as described. 8. The photogrammetry program according to claim 6, wherein in the second processing, the two-dimensional coordinates are calculated using camera characteristic data that is converted into a value obtained by correcting optical distortion of the camera. 9.

Images