JP4034003B2 - Flying object image processing apparatus and flying object image processing method - Google Patents

Description

【００３１】
幾何特性算出装置１３は、前処理装置１２が標的部分を構成する画素と背景部分を構成する画素を識別すると、標的部分の幾何特性量、即ち、標的の面積Ｓ，図心（ｘ_c ，ｙ_c ），断面二次モーメントＪ_xx，Ｊ_xy，Ｊ_yyを下記に示すように計算する（ステップＳＴ３）。
【００３２】
【数２】

【００３３】
慣性主軸推定装置１４は、幾何特性算出装置１３が幾何特性量を計算すると、標的の断面二次モーメントＪ_xx，Ｊ_xy，Ｊ_yyに基づいて、標的画像における慣性主軸方向を推定する（ステップＳＴ４）。
一般に慣性主軸は、図４の３３，３４に示すように、標的画像内において２個存在し(最大，最小慣性主軸に対応)、その各々は直交関係を有しているが、そのいずれかと画像面水平方向３６とのなす角度をαとすれば、下記の関係を満足する。なお、図４において、３５は図心、３７は画像面の垂直方向である。
ｔａｎ２α＝２Ｊ_xy／（Ｊ_yy−Ｊ_xx）
０≦α≦π
【００３４】
また、特徴点検出装置１５は、幾何特性算出装置１３が幾何特性量を計算すると、標的の特徴点である標的先端部を検出する（ステップＳＴ５）。
図５に示すように、飛翔体に対する標的の姿勢によって、標的先端部３９が可視な場合(右)と、可視でない場合(左)が存在する。なお、図５において、３８は標的機軸、４０は最小慣性主軸である。
そこで、標的先端部３９が可視な場合には、例えば文献（徐、辻、"３次元ビジョン"、共立出版、ｐｐ．３６−３７、１９９８）等で紹介されているＳＵＳＡＮオペレータ等のコーナ検出器を用いて、標的先端部３９の位置（ｘ_v ，ｙ_v ）を検出する。
【００３５】
ＳＵＳＡＮオペレータでは、図６に示すように、例えば、円形のマスクａ〜ｄを画像に対してそれぞれ重ね合わせると、そのマスクａ〜ｄ内の画素の中で、マスク中心の画素の輝度値と同じ輝度値を持つ画素の個数を数え上げる処理を実行する。
例えば、背景４２内にある画素Ｐ₁ を中心とするマスクａ、並びに、標的４１内にある画素Ｐ₂ を中心とするマスクｂでは、その個数がマスク面積と等しくなる。一方、直線輪郭上の画素Ｐ₃ を中心とするマスクｃでは、その個数がマスク面積の半分となり、また、コーナ画素Ｐ₄ を中心とするマスクｄでは、その個数がマスク面積の１／４となる。
【００３６】
標的画像における各画素に対して、上記のような処理を実施し、その個数が最小となる画素、即ち、画素Ｐ₄ を標的先端部３９として検出する。
なお、個数に対する閾値を設けることで、標的先端部３９が可視か否かも判別することができるので、図５における左右どちらの場合かを判別することができる。
【００３７】
次に、相対状態量推定装置１７は、飛翔体に対する標的の相対位置及び相対姿勢を推定する。
具体的には、まず、機軸選択装置１８が、慣性主軸推定装置１４により推定された２個の慣性主軸のうち、どちらが標的の機軸に対応するのかを判別する。
図５に示すように、飛翔体に対する標的の姿勢によって二種類の画像が存在するが、右図のような場合には、特徴点検出装置１５により標的先端部３９が検出されるため、その標的先端部３９と２個の慣性主軸との間の距離を計算し、小さい方を標的機軸３８として選択する（ステップＳＴ６）。
一方、左図のような場合には、特徴点検出装置１５により標的先端部３９が検出されないため、慣性主軸回りの慣性モーメントが大きい方（最大慣性主軸）を標的機軸３８として選択する（ステップＳＴ６）。
【００３８】
そして、全幅算出装置１９は、機軸選択装置１８が標的機軸３８を選択すると、図７に示すように、標的画像における２個の慣性主軸方向の全幅値を算出する（ステップＳＴ７）。機軸方向の全幅値をｈ、機軸方向と直交する方向の全幅値をｄとする。
なお、全幅値の算出方法としては、例えば、慣性主軸を平行移動しながら、慣性主軸上に存在する画素の画素値（前処理装置１２による前処理後の画素値）をスキャンし、“１”の画素値が最も連続する線分の長さを標的の全幅値とする。
【００３９】
そして、照合処理装置２０は、全幅算出装置１９が標的の全幅値ｈ，ｄを算出すると、標的データベース１６には、図８に示すように、標的であるミサイル弾頭の形状に関するデータ（高さＨ、底面直径Ｄ）が格納されているので、これらのデータと全幅値ｈ，ｄを照合して、標的の相対位置及び相対姿勢を推定する（ステップＳＴ８）。
【００４０】
例えば、図５の左図に示すような標的画像の場合、標的の相対姿勢は、カメラ座標系から標的固定座標系に対する３−２−１オイラー角を用いて表すと下記のように与えられる。
θ_x ＝±ｃｏｓ^-1（ｈ／ｄ）
θ_y ＝π
θ_z ＝α_min −π
また、標的の相対位置は下記のように与えられる。
ｘ_C （ｒ_B ）＝Ｄ［ｘ_c −Ｉ_W ／２］／ｄ
ｙ_C （ｒ_B ）＝Ｄ［ｙ_c −Ｉ_H ／２］／ｄ
ｚ_C （ｒ_B ）＝Ｄ・ｆ／ｄ
ここで、α_min は画像面水平方向からの最小慣性主軸４０の傾き角、ｆはカメラ焦点距離である。
【００４１】
一方、図５の右図に示すような標的画像の場合、標的の相対姿勢は、下記のように与えられる。

また、標的の相対位置は下記のように与えられる。
ｘ_C （ｒ_B ）＝Ｄ［ｘ_v −Ｉ_W ／２］／ｄ−Ｈ・ｓｉｎθ_x ・ｓｉｎθ_z
ｙ_C （ｒ_B ）＝Ｄ［ｙ_v −Ｉ_H ／２］／ｄ＋Ｈ・ｓｉｎθ_x ・ｃｏｓθ_z
ｚ_C （ｒ_B ）＝Ｄ・ｆ／ｄ
ここで、α_sym は画像面水平方向からの標的機軸３８の傾き角である。
【００４２】
最後に、照準点算定装置２１は、相対状態量推定装置１７が標的の相対位置及び相対姿勢を推定すると、下記に示すように、標的データベース１６に格納されている照準点４３の位置情報を参照して、その相対位置及び相対姿勢から標的の照準点を算定する（ステップＳＴ９）。
【００４３】
【数３】

【００４４】
以上で明らかなように、この実施の形態１によれば、標的画像から標的の幾何特性量を演算し、その幾何特性量から標的の慣性主軸方向を特定すると、その慣性主軸方向に基づいて飛翔体に対する標的の相対位置及び相対姿勢を推定し、その相対位置及び相対姿勢から標的の照準点を算定するように構成したので、飛翔体に対する標的の相対位置及び相対姿勢が変化しても、標的の照準点を正確に算定することができる効果を奏する。
【００４５】
実施の形態２．
図９はこの発明の実施の形態２による飛翔体の画像処理装置を示す構成図であり、図において、図１と同一符号は同一または相当部分を示すので説明を省略する。
２２は相対状態量推定装置１７により推定された相対位置及び相対姿勢から標的の推定画像（二値化画像）を生成する推定画像生成装置（更新手段）、２３は推定画像生成装置２２により生成された推定画像と前処理装置１２により前処理された標的画像の誤差である推定誤差を計算する推定誤差算出装置（更新手段）、２４は推定誤差算出装置２３により計算された推定誤差の勾配を計算する勾配計算装置（更新手段）、２５は勾配計算装置２４により計算された勾配に基づいて推定誤差が減少する方向に標的の全幅値ｄ，ｈを更新する高次全幅更新装置（更新手段）である。
【００４６】
次に動作について説明する。
上記実施の形態１では、相対状態量推定装置１７が標的の相対位置と相対姿勢を推定すると、その相対位置と相対姿勢から直ちに標的の照準点を算定するものについて示したが、その相対位置と相対姿勢から推定画像を生成して、その推定画像と標的画像の誤差を求め、その誤差が最小化するように標的の全幅値ｄ，ｈを更新してから、標的の照準点を算定するようにしてもよい。
【００４７】
上記実施の形態１の場合、慣性主軸方向の全幅値ｄ，ｈをピクセル精度で算出して、標的の相対位置及び相対姿勢を推定するので、飛翔体に対する標的の相対位置が比較的近距離にあり、標的画像に対する画像センサ分解能が十分に確保されている場合には問題ないが、飛翔体に対する標的の相対位置が遠距離にあり、標的画像に対する画像センサ分解能が十分に確保されない場合には、相対位置及び相対姿勢の推定精度が劣化する。
【００４８】
そこで、この実施の形態２では、標的画像に対する画像センサ分解能が十分に確保されない場合でも、相対位置及び相対姿勢の推定精度を高めるため以下に示す処理を実行する。
【００４９】
例えば、標的の相対位置が比較的遠距離にあり、図１０に示すような画像が取得されたものとする。この場合には、標的の慣性主軸方向の最大幅ｈ’，ｄ’は、図中×印が付された４個の画素の中心を基準として測定される。
この最大幅ｈ’，ｄ’は、実際に測定されるべきｄ，ｈに対し、常に画像センサ１１における量子化分の誤差が含まれる。
この量子化誤差が、照合処理装置２０における標的の相対位置及び相対姿勢の推定値に対する推定誤差として現れ、その結果、照準点の算定精度を低下させることになる。
【００５０】
同様な状況を図１１及び図１２を用いて説明する。両図には仮想的に標的の輪郭が実線で描かれている。このような２つの画像においては、どちらの場合においても、図中斜線で示された４個の画素の中心を基準として全幅値が算出されるため、照合処理装置２０において推定される相対位置及び相対姿勢の値には、両者の間で差が生じない。
【００５１】
しかしながら、それぞれの画像を領域全体で眺めれば、図中黒く塗りつぶされた画素４４，４５が輝度値を持つか否かの差が存在する。
即ち、図１１の場合には、画素４４，４５が輝度値を持ち、図１２の場合には、画素４４，４５が輝度値を持たない。
したがって、慣性主軸方向の最大幅といった一次元直線上の特徴量ではなく、画像全体に渡る更に高次の特徴量を利用すれば、両者の差異を推定量に反映することが可能であり、画像センサ１１における量子化誤差の影響を低減することができる。
【００５２】
推定画像生成装置２２は、相対状態量推定装置１７が相対位置及び相対姿勢を推定すると、その相対位置及び相対姿勢から標的の推定画像（二値化画像）を生成する。
そして、推定誤差算出装置２３は、推定画像生成装置２２が標的の推定画像を生成すると、その推定画像と前処理装置１２により前処理された標的画像（二値化画像）の誤差である推定誤差ｅを計算する。
即ち、推定画像と標的画像における画素間の輝度の二乗誤差、あるいは、等価的に輝度差の絶対値の総和を次のように定義して、推定誤差ｅを計算する。なお、二値化画像のため、両画像間における画素値の二乗誤差の総和と絶対値の総和は同じ値となる。
【００５３】
【数４】

【００５４】
勾配計算装置２４は、推定誤差算出装置２３が推定誤差ｅを計算すると、その推定誤差ｅを全幅値ｄ，ｈで遍微分して、その推定誤差ｅの勾配∂ｅ／∂ｄ，∂ｅ／∂ｈを計算する。
【００５５】
そして、高次全幅更新装置２５は、勾配計算装置２４が推定誤差ｅの勾配∂ｅ／∂ｄ，∂ｅ／∂ｈを計算すると、その勾配∂ｅ／∂ｄ，∂ｅ／∂ｈを用いて、その推定誤差ｅが減少する方向に慣性主軸方向の全幅値ｄ，ｈを更新する。
即ち、前回の全幅値（ここでは説明の便宜上、前回の全幅値をｄ_old ，ｈ_old で表す）から（∂ｅ／∂ｄ）・Δｄ，（∂ｅ／∂ｈ）・Δｈを減算することにより、慣性主軸方向の全幅値ｄ，ｈを更新する（ここでは説明の便宜上、更新後の全幅値をｄ_new ，ｈ_new で表す）。
ｄ_new ＝ｄ_old −（∂ｅ／∂ｄ）・Δｄ
ｈ_new ＝ｈ_old −（∂ｅ／∂ｈ）・Δｈ
ただし、Δｄ，Δｈは推定誤差ｅを減少させるように決定される刻み幅である。
【００５６】
照合処理装置２０は、高次全幅更新装置２５が慣性主軸方向の全幅値ｄ，ｈを更新すると、上記実施の形態１と同様にして、更新後の全幅値に基づいて相対位置及び相対姿勢の候補値を算出する。
このようにして、照合処理装置２０が相対位置と相対姿勢の候補値を算出すると、今度は、その相対位置と相対姿勢の候補値から標的の推定画像を生成して、上記の処理を繰り返し、更に相対位置及び相対姿勢の候補値を算出する。
【００５７】
そして、推定誤差算出装置２３は、上記の処理が複数回繰り返された後、推定誤差ｅが停留値を取るところで、標的の相対位置及び相対姿勢の候補値を確定値として、照準点算定装置２１に出力する。
照準点算定装置２１は、推定誤差算出装置２３が標的の相対位置及び相対姿勢を出力すると、上記実施の形態１と同様にして、飛翔体に対する標的の照準点を算出する。
【００５８】
以上で明らかなように、この実施の形態２によれば、標的の相対位置及び相対姿勢から標的の推定画像を生成して、その推定画像と標的画像の誤差を計算し、その誤差が減少する方向に標的の全幅値を更新するように構成したので、全幅値をサブピクセル精度で算出可能となり、飛翔体に対する標的の相対距離が比較的遠距離の場合においても、画像センサ１１の量子化誤差の影響が抑えられるようになり、その結果、標的の相対位置及び相対姿勢を精度よく推定することができる効果を奏する。
【００５９】
また、二つの慣性主軸方向の全幅値ｄ，ｈをパラメータとする２変数に対する勾配計算を実施して全幅値ｄ，ｈを更新し、その全幅値ｄ，ｈを用いて照合処理装置２０から間接的に標的の相対位置と相対姿勢を推定するように構成したので、標的の相対位置及び相対姿勢の６変数をパラメータとして直接的に勾配計算を実施するよりも、効率的に相対位置と相対姿勢を推定することができる効果を奏する。
【００６０】
実施の形態３．
図１３はこの発明の実施の形態３による飛翔体の画像処理装置を示す構成図であり、図において、図９と同一符号は同一または相当部分を示すので説明を省略する。
２６は標的画像及び推定画像の面積と断面二次モーメントを算出する高次特徴量算出装置（更新手段）、２７は高次特徴量算出装置２６により算出された面積と断面二次モーメントを用いて標的の全幅値を更新する高次全幅更新装置（更新手段）である。
【００６１】
次に動作について説明する。
上記実施の形態２では、推定誤差ｅの勾配を計算し、その勾配を用いて慣性主軸方向の全幅値ｄ，ｈを更新するものについて示したが、標的画像及び推定画像の面積と断面二次モーメントを算出し、その面積と断面二次モーメントを用いて標的の全幅値ｄ，ｈを更新するようにしてもよい。
【００６２】
具体的には、高次特徴量算出装置２６は、推定画像生成装置２２より生成された標的の推定画像の面積と、機軸に垂直な慣性主軸回りの断面二次モーメントを算出する。
ここで、断面二次モーメントに関しては、標的画像が図１４の左図のような場合には、図心３５を通る標的機軸３８に垂直な慣性主軸４０回りの値を計算する。
一方、図１４の右図のような場合には、標的先端部３９を通る標的機軸３８に垂直な軸４６回りの値を計算する。
【００６３】
高次全幅更新装置２７は、高次特徴量算出装置２６が高次特徴量（面積及び断面二次モーメント）を算出すると、その高次特徴量と、幾何特性算出装置１３により算出された幾何特性量（面積及び断面二次モーメント）を用いて、両者間の差を評価関数とする。
即ち、標的面積の差を第１の評価関数とし、断面二次モーメントの差を第２の評価関数とする。
【００６４】
そして、高次全幅更新装置２７は、第１の評価関数が減少する方向に、標的機軸３８に垂直な慣性主軸４０方向の全幅値を更新し、第２の評価関数が減少する方向に、機軸方向の全幅値を更新する。
即ち、図１４の左図のような画像の場合には、原画像である標的画像と推定画像に対して算出された標的面積を用いて、標的機軸３８に垂直な慣性主軸４０の方向の全幅値ｄを次のように更新する。
ｄ_new ＝ｄ_old ×（Ｓ_org ／Ｓ_est ）^1/2
ここで、Ｓ_org は標的画像における標的面積であり、Ｓ_est は推定画像における標的面積である。
【００６５】
標的機軸３８の方向の全幅値ｈは、標的機軸３８に垂直な慣性主軸回りの断面二次モーメントに基づいて次のように更新する。
ｈ_new ＝ｈ_old ×（Ｊ_ξξ ^org ／Ｊ_ξξ ^est ）^1/3
Ｊ_ξξ＝Ｊ_yy・ｓｉｎ² α＋Ｊ_xx・ｃｏｓ² α−Ｊ_xy・ｓｉｎ２α
ここで、Ｊ_ξξ ^org は標的画像における標的機軸３８に垂直な慣性主軸４０回りの断面二次モーメント、Ｊ_ξξ ^est は推定画像における標的機軸３８に垂直な慣性主軸４０回りの断面二次モーメントである。
【００６６】
一方、図１４の右図のような画像の場合には、原画像である標的画像と推定画像に対して算出された標的面積を用いて、標的機軸３８に垂直な慣性主軸４０の方向の全幅値ｄを、図１４の左図の場合と同様に更新する。
標的機軸３８の方向の全幅値ｈは、標的機軸３８に垂直な慣性主軸回りの断面二次モーメントに基づいて次のように更新する。
ｈ_new ＝ｈ_old ×（Ｊ_ξ’ξ’ ^org ／Ｊ_ξ’ξ’ ^est ）^1/3
ここで、Ｊ_ξ’ξ’ ^org は標的画像における標的先端部３９を通る標的機軸３８に垂直な慣性主軸４６回りの断面二次モーメント、Ｊ_ξ’ξ’ ^est は推定画像における標的先端部３９を通る標的機軸３８に垂直な慣性主軸４６回りの断面二次モーメントである。
【００６７】
その他は上記実施の形態２と同様であるため説明を省略する。
以上で明らかなように、この実施の形態３によれば、標的画像及び推定画像の面積と断面二次モーメントを計算し、その面積と断面二次モーメントを用いて標的の全幅値を更新するように構成したので、全幅値をサブピクセル精度で算出可能となり、飛翔体に対する標的の相対距離が比較的遠距離の場合においても、画像センサ１１の量子化誤差の影響が抑えられるようになり、その結果、標的の相対位置及び相対姿勢を精度よく推定することができる効果を奏する。
【００６８】
また、標的の相対位置と相対姿勢を推定する際、比較的簡単に算出することが可能な標的画像の面積や断面二次モーメントに基づいて推定することができるので、勾配計算等の複雑な演算処理が不要になり、その結果、上記実施の形態２よりも、飛翔体に対する標的の相対位置や相対姿勢を効率的に推定することができる効果を奏する。
【００６９】
実施の形態４．
図１５はこの発明の実施の形態４による飛翔体の画像処理装置を示す構成図であり、図において、図１と同一符号は同一または相当部分を示すので説明を省略する。
６１は多値化処理を実施して標的部分と背景部分を識別する前処理装置、６２は前処理装置６１により識別された標的部分（以下、標的画像という）から標的の幾何特性量を算出する幾何特性算出装置（特性量演算手段）、６３は標的の特徴点における標的先端部を検出する特徴点検出装置（推定手段）、６４は標的画像の特徴点における開口率を算出する開口率算出装置（推定手段）である。
【００７０】
６５は慣性主軸方向において標的の最大幅を与える線分の両端点に位置する画素の輝度値と、標的画像の特徴点における開口率とから標的の全幅値を計算し、その全幅値と標的の形状に関するモデルデータから標的の相対位置及び相対姿勢を推定する相対状態量推定装置（推定手段）、６６は慣性主軸方向において標的の最大幅を与える線分の両端点に位置する画素の輝度値と、標的画像の特徴点における開口率とから標的の全幅値を計算する高次全幅算出装置である。
【００７１】
次に動作について説明する。
前処理装置６１は、画像センサ１１が標的を含む画像を取得すると、その画像を構成する画素のうち、標的部分を構成する画素と背景部分を構成する画素を識別し、各画素を多値化処理する。
即ち、背景部分を構成する画素の画素値を零とする一方、標的領域では、その領域における最大輝度値で、標的部分を構成する画素の画素値を正規化し、標的領域において階調を持った画像を生成する。
【００７２】
幾何特性算出装置６２は、前処理装置６１が各画素の多値化処理を実施すると、標的部分の幾何特性量、即ち、標的の面積Ｓ，図心（ｘ_c ，ｙ_c ），断面二次モーメントＪ_xx，Ｊ_xy，Ｊ_yyを計算する。
幾何特性算出装置６２の計算は、計算対象となる画像が多値化画像になるだけで、基本的には上記実施の形態１における幾何特性算出装置１３と同様の計算である（計算においては、標的部分を構成する画素の輝度値を全て１に置き換えればよい)。
【００７３】
慣性主軸推定装置１４は、幾何特性算出装置１３が幾何特性量を計算すると、上記実施の形態１と同様に、標的の断面二次モーメントＪ_xx，Ｊ_xy，Ｊ_yyに基づいて、標的画像における慣性主軸方向を推定する。
また、特徴点検出装置６３は、幾何特性算出装置６２が幾何特性量を計算すると、上記実施の形態１と同様に、標的の特徴点である標的先端部を検出する。
【００７４】
なお、ＳＵＳＡＮオペレータ等のコーナ検出器を用いて標的先端部を検出する場合、多値化画像に対しても二値化画像の場合と同様に円形のマスクを作用させ、そのマスク内の画素の中で、マスク中心の画素の輝度値と近い輝度値を持つ画素数を数え上げる。標的画像における各画素に対してこの処理を実施し、その個数が最小となる画素を標的先端部として検出する。
【００７５】
そして、開口率算出装置６４は、特徴点検出装置６３により標的の特徴点である標的先端部が検出された場合には、その標的先端部における開口率を算出する。
即ち、標的先端部の近傍領域における画素値に基づいて標的先端部の開口率を算出する。
【００７６】
図１６に示すように、特徴点検出装置６３により検出された標的先端部３９を中心とする円形マスク４７を標的画像に重ね合わせて、マスク内領域において、その中心の画素の輝度と近い輝度値を有する画素の個数Ｎ（ｘ_v ，ｙ_v ）を数える。
そして、画素の個数Ｎ（ｘ_v ，ｙ_v ）を数えると、マスク面積Ｍで除算して、開口率φを算出する。
φ＝Ｎ（ｘ_v ，ｙ_v ）／Ｍ
なお、Ｎ（ｘ_v ，ｙ_v ）に関しては、仮に、特徴点検出装置６３において、ＳＵＳＡＮオペレータを用いるのであれば、これは既に計算された値である。従って、開口率の算出処理は、ＳＵＳＡＮオペレータに対する付加的な処理として済ませることができる。
【００７７】
次に、相対状態量推定装置６５は、飛翔体に対する標的の相対位置及び相対姿勢を推定する。
具体的には、まず、機軸選択装置１８が上記実施の形態１と同様に、慣性主軸推定装置１４により推定された２個の慣性主軸のうち、どちらが標的の機軸に対応するのかを判別する。
【００７８】
そして、高次全幅算出装置６６は、開口率算出装置６４により算出された標的先端部における開口率、並びに、二つの慣性主軸方向それぞれにおいて標的の最大幅を与える線分の両端点に対応する画素の輝度値に基づいて、慣性主軸方向それぞれにおける標的の全幅値をサブピクセル精度で算出する。
【００７９】
一例として、図１７のような標的画像について考える。
図１７において、慣主軸方向の最大幅を与える線分４８，４９の両端点に対応する画素は、図中斜線で示されるＰ₁ 〜Ｐ₄ である。
標的先端部を示す画素Ｐ₁ を拡大したのが図１８である。
【００８０】
標的領域における各画素の輝度値は、近似的にその画素を占める標的領域の面積に比例するものと考えられる。
即ち、標的領域内部では画素値が一様に１であり、標的輪郭部における画素値は各画素に占める標的領域の割合（面積比率）で与えられるものと、近似的に見なすことができる。
【００８１】
そこで、図１８に示すように画素Ｐ₁ を、開口率算出装置６４で求めた開口率φを用いて、二つの領域（図中斜線で示された領域５０とそれ以外の領域５１)に分割する。
即ち、図１８に示すように、標的画像の図心３５を通る機軸３８上の１点を頂点Ｖ₁ とし、頂点Ｖ₁ から標的機軸３８に対して、対称に角度ψ＝２πφをなす直線５２，５３を描いて、画素を二つの領域５０，５１に分割する。
そして、画素Ｐ₁ の面積に対する図中斜線で示された領域５０の面積比率が、画素Ｐ₁ の持つ輝度値と等しくなるように画素Ｐ₁ 内における頂点Ｖ₁ の位置を決定する。
【００８２】
また、図１７における特徴点以外の画素Ｐ₂ 〜Ｐ₄ においては、例えば、Ｐ₂ において、図１９に示すように画素Ｐ₂ を通る標的機軸３８に垂直な慣性主軸５４ａ上の１点をＶ₂ とし、頂点Ｖ₂ を通り慣性主軸５４ａに垂直な直線５４ｂで画素を二つの領域（図中斜線で示された領域５５とそれ以外の領域５６）に分割する。
そして、画素Ｐ₂ の面積に対する図中斜線で示された領域５５の面積比率が、画素Ｐ₂ の持つ輝度値と等しくなるように点Ｖ₂ の位置を決定する。画素Ｐ₃ ，Ｐ₄ においても同様な手法で点Ｖ₃ ，Ｖ₄ を決定する。
【００８３】
こうして決定された点Ｖ₁ 〜Ｖ₄ の位置から、二つの慣性主軸方向それぞれにおける標的最大幅をサブピクセル精度で算出し、機軸方向の全幅をｈ、それと直交する方向の全幅をｄとする。
【００８４】
そして、照合処理装置２０は、全幅算出装置１９が標的の全幅値ｄ，ｈを算出すると、上記実施の形態１と同様に、標的の相対位置及び相対姿勢を推定する。
最後に、照準点算定装置２１は、相対状態量推定装置６５が標的の相対位置及び相対姿勢を推定すると、上記実施の形態１と同様に、標的の照準点を算定する。
【００８５】
以上で明らかなように、この実施の形態４によれば、標的の慣性主軸方向において標的の最大幅を与える線分の両端点に位置する画素の輝度値と、標的画像の特徴点における開口率とから標的の全幅値をサブピクセル精度で計算し、その全幅値と標的の形状に関するモデルデータから標的の相対位置及び相対姿勢を推定するように構成したので、飛翔体に対する標的の相対距離が比較的遠距離の場合においても、画像センサ１１の量子化誤差の影響が抑えられるようになり、その結果、標的の相対位置及び相対姿勢を精度よく推定することができる効果を奏する。
【００８６】
また、標的の相対位置と相対姿勢を推定する際、勾配計算等の複雑な繰り返し演算処理を必要としないため、飛翔体に対する標的の相対位置及び相対姿勢に関して、効率的な推定を行なうことができる効果を奏する。
【００８７】
実施の形態５．
図２０はこの発明の実施の形態５による飛翔体の画像処理装置を示す構成図であり、図において、図１と同一符号は同一または相当部分を示すので説明を省略する。
６７は前方三次元世界を画像センサ６８，６９に出力するハーフミラー、６８は画像センサ１１と同様の画像センサ、６９は画像センサ６８が撮影する標的画像に対して画像面内で水平方向及び垂直方向にそれぞれ０．５画素変位する位置に配置された画像センサ、７０は画像センサ６８側の標的画像に係る慣性主軸方向と画像センサ６９側の標的画像に係る慣性主軸方向とに基づいて飛翔体に対する標的の相対位置及び相対姿勢を推定する相対状態量推定装置（推定手段）、７１は二つの慣性主軸方向における標的の最大幅を与える線分の両端点に対応する画素を選択する主軸両端点選択装置、７２は主軸両端点選択装置７１により選択された各画素の位置関係から標的の全幅値をサブピクセル精度で算出する高次全幅算出装置である。
【００８８】
次に動作について説明する。
まず、ハーフミラー６７は、飛翔体前方の標的を含む三次元世界を画像センサ６８，６９に出力する。
画像センサ６８，６９は、一方の画像センサで取得される画像が他方の画像センサで取得される画像に対し、画像面内水平方向で０．５画素、画像面内垂直方向に０．５画素並進変位したものとなるように配置されている。
このような配置は図２１に示すように、一方の画像センサ、例えば、画像センサ６８を光軸ａに対して固定した状態で、画像センサ６９を光軸ａ，ｂが形成する平面内の光軸ｂに垂直な矢印ｃ、並びに、上記平面に垂直な方向の矢印ｄ方向に並進変位させることによって実現することができる。
【００８９】
画像センサ６８で取得された飛翔体前方の標的を含む第一画像、並びに、画像センサ６９で取得された飛翔体前方の標的を含む第二画像に対してそれぞれ、上記実施の形態１と同様に、前処理装置１２，幾何特性算出装置１３，慣性主軸推定装置１４及び特徴点検出装置１５が動作する。
【００９０】
次に、相対状態量推定装置７０は、飛翔体に対する標的の相対位置及び相対姿勢を推定する。
まず、前処理後の第一画像、並びに、第二画像のそれぞれに対して、機軸選択装置１８が、慣性主軸推定装置１４により推定された二つの慣性主軸のうち、どちらが標的の機軸に対応するのかを判別する。
【００９１】
次に、第一画像、並びに、第二画像のそれぞれに対して、主軸両端点選択装置７１，７１が、二つの慣性主軸方向それぞれにおける標的の最大幅を与える線分４８，４９の両端点に対応する画素を選択して、その位置を出力する。
例えば、図２２においては、第一画像に対する場合を示したものであり、図中斜線で示された画素Ｐ₁ 〜Ｐ₄ を選択するとともに、第一画像におけるそれぞれの位置を出力する。
第二画像に対しても同様な処理を実施し、慣性主軸方向の最大幅を与える線分の両端点に対応する画素Ｑ₁ 〜Ｑ₄ を選択し、第二画像におけるそれぞれの位置を出力する。
【００９２】
高次全幅算出装置７２は、主軸両端点選択装置７１によりそれぞれ選択された第一画像に対する画素Ｐ₁ 〜Ｐ₄ 、並びに、第二画像に対する画素Ｑ₁ 〜Ｑ₄ の相対的な位置関係に基づいて、標的画像における主軸方向の最大幅をサブピクセル精度で算出する。
例えば、画像センサ６９では、画像センサ６８で得られた画像を画像面内水平方向に＋０．５画素、垂直方向に＋０．５画素だけ平行移動したものが得られるものとする。
【００９３】
この時、両画像において標的先端部３９に対応する特徴点を画素Ｐ₁ ，Ｑ₁ とすれば、図２３の左図に示すように、実際の標的先端部３９が画素Ｐ₁ において、画素内を水平、垂直方向に４分割した時の第３象限８４にある場合、第二画像における実際の標的先端部３９は右図に示すように同じ画素内の第１象限８５にあるため、両画像において検出される画素Ｐ₁ とＱ₁ の位置に差異は生じない。
【００９４】
しかるに、例えば、実際の標的先端部３９が、図２４の左図に示すように第１象限８６にある場合、第二画像における実際の標的先端部３９は右図に示すように画像面内水平方向に＋１画素、垂直方向に＋１画素だけ並進変位した画素Ｑ₁ における第３象限８７にあるため、両画像において検出される画素Ｐ₁ と画素Ｑ₁ の間の位置関係は、画素Ｑ₁ は画素Ｐ₁ に対して画像面内水平方向に＋１画素、垂直方向に＋１画素だけ並進変位している。
【００９５】
同様に、実際の標的先端部３９が第二象限にある場合には、画素Ｑ₁ は画素Ｐ₁ に対して画像面内垂直方向にのみ＋１画素並進変位したものとなり、実際の標的先端部３９が第４象限にある場合には、画素Ｑ₁ は画素Ｐ₁ に対して画像面内水平方向にのみ＋１画素並進変位したものとなる。
【００９６】
即ち、各画素Ｐ_i とＱ_i （ｉ＝１，２，３，４）の画像面における相対的な位置関係を調べることによって、実際の標的主軸方向両端点が第一画像における画素Ｐ_i 内のどの象限にいるのかを判別することができる。
標的の二つの慣性主軸方向における最大幅は、この象限の中心点、例えば図２３及び図２４では×印で示した点Ｃ₁ を基準として、サブピクセル精度で、その値を算出する。
【００９７】
そして、照合処理装置２０は、高次全幅算出装置６６が標的の全幅値ｈ，ｄを算出すると、上記実施の形態１と同様に、標的の相対位置及び相対姿勢を推定する。
最後に、照準点算定装置２１は、相対状態量推定装置１７が標的の相対位置及び相対姿勢を推定すると、上記実施の形態１と同様に、標的の照準点を算定する。
【００９８】
以上で明らかなように、この実施の形態５によれば、画像センサ６８側の標的画像に係る慣性主軸方向と画像センサ６９側の標的画像に係る慣性主軸方向とに基づいて飛翔体に対する標的の相対位置及び相対姿勢を推定するように構成したので、見かけ上、画像センサの分解能を水平方向、垂直方向ともに２倍にすることができるようになり、その結果、飛翔体に対する標的の相対距離が比較的遠距離の場合においても、画像センサの量子化誤差の影響が抑えられるため、標的の相対位置及び相対姿勢を精度よく推定し、それを用いて標的の照準点を精度よく算定することができる効果を奏する。
【００９９】
また、標的の相対位置と相対姿勢を推定する際、勾配計算等の複雑な繰り返し演算処理を必要としないため、飛翔体に対する標的の相対位置及び相対姿勢に関して、効率的な推定を行なうことができる効果を奏する。
【０１００】
【発明の効果】
以上のように、この発明によれば、方向特定手段により特定された慣性主軸方向に基づいて飛翔体に対する標的の相対位置及び相対姿勢を推定する推定手段を設けるように構成したので、飛翔体に対する標的の相対位置及び相対姿勢が変化しても、標的の照準点を正確に算定することができる効果がある。
【０１０１】
この発明によれば、方向特定手段により特定された慣性主軸方向から標的の全幅値を計算し、その全幅値と標的の形状に関するデータから標的の相対位置及び相対姿勢を推定する推定手段を設けるように構成したので、簡単に標的の相対位置と相対姿勢を推定することができる効果がある。
【０１０２】
この発明によれば、推定手段により推定された相対位置及び相対姿勢から標的の推定画像を生成して、その推定画像と標的画像の誤差を計算し、その誤差が減少する方向に標的の全幅値を更新する更新手段を設けるように構成したので、飛翔体に対する標的の相対距離が比較的遠距離の場合においても、画像センサの量子化誤差の影響が抑えられるようになり、その結果、標的の相対位置及び相対姿勢を精度よく推定することができる効果がある。
【０１０３】
この発明によれば、誤差の勾配を計算し、その勾配を用いて標的の全幅値を更新する更新手段を設けるように構成したので、飛翔体に対する標的の相対位置や相対姿勢を精度よく推定することができる効果がある。
【０１０４】
この発明によれば、標的画像及び推定画像の面積と断面二次モーメントを計算し、その面積と断面二次モーメントを用いて標的の全幅値を更新する更新手段を設けるように構成したので、飛翔体に対する標的の相対位置や相対姿勢を効率的に推定することができる効果がある。
【０１０５】
この発明によれば、方向特定手段により特定された慣性主軸方向において標的の最大幅を与える線分の両端点に位置する画素の輝度値と、標的画像の特徴点における開口率とから標的の全幅値を計算し、その全幅値と標的の形状に関するデータから標的の相対位置及び相対姿勢を推定する推定手段を設けるように構成したので、飛翔体に対する標的の相対距離が比較的遠距離の場合においても、画像センサの量子化誤差の影響が抑えられるようになり、その結果、標的の相対位置及び相対姿勢を精度よく推定することができる効果がある。
【０１０６】
この発明によれば、第１の方向特定手段により特定された慣性主軸方向と第２の方向特定手段により特定された慣性主軸方向に基づいて飛翔体に対する標的の相対位置及び相対姿勢を推定する推定手段を設けるように構成したので、飛翔体に対する標的の相対距離が比較的遠距離の場合においても、画像センサの量子化誤差の影響が抑えられるようになり、その結果、標的の相対位置及び相対姿勢を精度よく推定し、それを用いて標的の照準点を精度よく算定することができる効果がある。
【０１０７】
この発明によれば、第１の方向特定手段により特定された慣性主軸方向に基づいて標的の最大幅を与える線分の両端点に位置する画素を選択するとともに、第２の方向特定手段により特定された慣性主軸方向に基づいて標的の最大幅を与える線分の両端点に位置する画素を選択し、それらの画素の位置関係から標的の相対位置及び相対姿勢を推定する推定手段を設けるように構成したので、飛翔体に対する標的の相対位置や相対姿勢を精度よく推定することができる効果がある。
【０１０８】
この発明によれば、標的画像から標的の幾何特性量を演算し、その幾何特性量から標的の慣性主軸方向を特定すると、その慣性主軸方向に基づいて飛翔体に対する標的の相対位置及び相対姿勢を推定し、その相対位置及び相対姿勢から標的の照準点を算定するように構成したので、飛翔体に対する標的の相対位置及び相対姿勢が変化しても、標的の照準点を正確に算定することができる効果がある。
【０１０９】
この発明によれば、標的の慣性主軸方向から標的の全幅値を計算し、その全幅値と標的の形状に関するデータから標的の相対位置及び相対姿勢を推定するように構成したので、簡単に標的の相対位置と相対姿勢を推定することができる効果がある。
【０１１０】
この発明によれば、標的の相対位置及び相対姿勢から標的の推定画像を生成して、その推定画像と標的画像の誤差を計算し、その誤差が減少する方向に標的の全幅値を更新するように構成したので、飛翔体に対する標的の相対距離が比較的遠距離の場合においても、画像センサの量子化誤差の影響が抑えられるようになり、その結果、標的の相対位置及び相対姿勢を精度よく推定することができる効果がある。
【０１１１】
この発明によれば、誤差の勾配を計算し、その勾配を用いて標的の全幅値を更新するように構成したので、飛翔体に対する標的の相対位置や相対姿勢を精度よく推定することができる効果がある。
【０１１２】
この発明によれば、標的画像及び推定画像の面積と断面二次モーメントを計算し、その面積と断面二次モーメントを用いて標的の全幅値を更新するように構成したので、飛翔体に対する標的の相対位置や相対姿勢を効率的に推定することができる効果がある。
【０１１３】
この発明によれば、標的の慣性主軸方向において標的の最大幅を与える線分の両端点に位置する画素の輝度値と、標的画像の特徴点における開口率とから標的の全幅値を計算し、その全幅値と標的の形状に関するデータから標的の相対位置及び相対姿勢を推定するように構成したので、飛翔体に対する標的の相対距離が比較的遠距離の場合においても、画像センサの量子化誤差の影響が抑えられるようになり、その結果、標的の相対位置及び相対姿勢を精度よく推定することができる効果がある。
【０１１４】
この発明によれば、一方の慣性主軸方向と他方の慣性主軸方向に基づいて飛翔体に対する標的の相対位置及び相対姿勢を推定し、その相対位置及び相対姿勢から標的の照準点を算定するように構成したので、飛翔体に対する標的の相対距離が比較的遠距離の場合においても、画像センサの量子化誤差の影響が抑えられるようになり、その結果、標的の相対位置及び相対姿勢を精度よく推定し、それを用いて標的の照準点を精度よく算定することができる効果がある。
【０１１５】
この発明によれば、一方の慣性主軸方向に基づいて標的の最大幅を与える線分の両端点に位置する画素を選択するとともに、他方の慣性主軸方向に基づいて標的の最大幅を与える線分の両端点に位置する画素を選択し、それらの画素の位置関係から標的の相対位置及び相対姿勢を推定するように構成したので、飛翔体に対する標的の相対位置や相対姿勢を精度よく算定することができる効果がある。
【図面の簡単な説明】
【図１】 この発明の実施の形態１による飛翔体の画像処理装置を示す構成図である。
【図２】 この発明の実施の形態１による飛翔体の画像処理方法を示すフローチャートである。
【図３】 カメラ座標系及び標的固定座標系を示す説明図である。
【図４】 標的画像の慣性主軸方向を示す説明図である。
【図５】 飛翔体に対する標的の姿勢によって画像（標的形状）が異なることを示す説明図である。
【図６】 コーナ検出器の一例を示すＳＵＳＡＮオペレータの原理を説明する説明図である。
【図７】 二つの慣性主軸方向における最大幅を示す説明図である。
【図８】 標的データベースに蓄えられたデータを示す説明図である。
【図９】 この発明の実施の形態２による飛翔体の画像処理装置を示す構成図である。
【図１０】 画像センサにおける量子化誤差の影響を説明する説明図である。
【図１１】 画像センサにおける量子化誤差の影響を説明する説明図である。
【図１２】 画像センサにおける量子化誤差の影響を説明する説明図である。
【図１３】 この発明の実施の形態３による飛翔体の画像処理装置を示す構成図である。
【図１４】 慣性主軸方向における全幅値の更新を説明する説明図である。
【図１５】 この発明の実施の形態４による飛翔体の画像処理装置を示す構成図である。
【図１６】 標的先端部における画像開口率の算出を説明する説明図である。
【図１７】 慣性主軸方向の最大幅を与える線分と、その両端点に対応する画素を示す説明図である。
【図１８】 画素Ｐ₁ における端点Ｖ₁ の決定を説明する説明図である。
【図１９】 画素Ｐ₂ における端点Ｖ₂ の決定を説明する説明図である。
【図２０】 この発明の実施の形態５による飛翔体の画像処理装置を示す構成図である。
【図２１】 画像センサ６８，６９の配置を説明する説明図である。
【図２２】 慣性主軸方向の最大幅を与える線分の両端点に対応する画素を示す説明図である。
【図２３】 標的先端部の第一画像と第二画像の相対的な位置関係（相対位置誤差なし）を示す説明図である。
【図２４】 標的先端部の第一画像と第二画像の相対的な位置関係（相対位置誤差あり）を示す説明図である。
【図２５】 飛翔体の前方に搭載されたカメラにより撮影された画像を示す画像図である。
【図２６】 従来の飛翔体の画像処理装置を説明する説明図である。
【図２７】 従来の飛翔体の画像処理装置を説明する説明図である。
【図２８】 従来の飛翔体の画像処理装置を説明する説明図である。
【符号の説明】
１３，６２ 幾何特性算出装置（特性量演算手段）、１４ 慣性主軸推定装置（方向特定手段）、１５，６３ 特徴点検出装置（推定手段）、１６ 標的データベース（推定手段）、１７，６５，７０ 相対状態量推定装置（推定手段）、２１ 照準点算定装置（照準点算定手段）、２２ 推定画像生成装置（更新手段）、２３ 推定誤差算出装置（更新手段）、２４ 勾配計算装置（更新手段）、２５，２７ 高次全幅更新装置（更新手段）、２６ 高次特徴量算出装置（更新手段）、６４ 開口率算出装置（推定手段）。[0001]
BACKGROUND OF THE INVENTION
The present invention is directed to an image processing apparatus for a flying object and an image of the flying object that directs the flying object to an aiming point of the target when the flying object is directly hit by a flying target (for example, a missile warhead) and the target is destroyed. It relates to a processing method.
[0002]
[Prior art]
FIG. 25 is an image diagram showing an image photographed by a camera mounted in front of the flying object. FIGS. 26 to 28 are, for example, a conventional flying object image processing apparatus disclosed in Japanese Patent Laid-Open No. 6-323788. It is explanatory drawing explaining these. In the figure, reference numeral 1 is a target which is an intercept target, 2 is an exhaust image data segment indicating an exhaust image of the target 1 among the near infrared images acquired by the near infrared wavelength sensor, and 3 is a medium acquired by the mid infrared wavelength sensor. Of the infrared image, a main body image data segment indicating a main body image of the target 1, 4 is an exhaust image data segment indicating an exhaust image of the target 1 among the mid-infrared images acquired by the mid-infrared wavelength sensor.
[0003]
In the conventional image processing apparatus, when the exhaust image data segments 2 and 4 and the main body image data segment 3 are acquired using the near-infrared wavelength sensor and the mid-infrared wavelength sensor, as shown in FIG. For the exhaust image data segment 4, the centroid O, the inclination angle α of the inertial main axis ξ, the length L in the main axis direction₁, Length L in orthogonal axis direction₂Ask for.
Similarly, the same various quantities are obtained for the exhaust image data segment 2 of the near-infrared wavelength sensor and the main body image data segment 3 of the mid-infrared wavelength sensor.
[0004]
When various amounts for the image data segments 2, 3, and 4 are obtained, the image data segments 2, 3, and 4 are obtained for the image acquired by the near infrared wavelength sensor and the image acquired by the mid infrared wavelength sensor. By taking the position correlation, the exhaust of the target 1 and the others are identified.
That is, among the image data segments in the mid-infrared image, those whose positions include the image data segment in the near-infrared image and the inclination angle α of the principal axis of inertia ξ is close are recognized as the exhaust image data segment 4 of the target 1. To do.
[0005]
Next, the axis of the target 1 is detected from the inclination angle α of the principal axis of inertia ξ of the exhaust image data segment 4 of the mid-infrared wavelength sensor.
In general, the centroid O of the exhaust image data segment 4 of the mid-infrared wavelength sensor is utilized by utilizing the fact that the hot exhaust image data segment 2 is closer to the body of the target 1 than the cooler exhaust image data segment 4. To the centroid O of the exhaust image data segment 2 of the near infrared wavelength sensor is detected as the nose direction.
[0006]
The midpoint between the centroid O of the main body image data segment 3 of the mid-infrared wavelength sensor, which is the tip of the target 1, and the centroid O of the exhaust image data segment 4 of the mid-infrared wavelength sensor is set as the optimum induction point. The flying object is guided to the target 1 by outputting the guidance signal.
As described above, since the induction point is calculated based on both the main body image information and the exhaust image information of the target 1, the erroneous detection of the induction point due to the positional deviation between the centroid of the main body image and the centroid of the exhaust image. Can be prevented, and the guidance accuracy can be improved.
[0007]
[Problems to be solved by the invention]
Since the conventional flying object image processing apparatus is configured as described above, the centroid O of the main body image data segment 3 of the mid-infrared wavelength sensor and the centroid O of the exhaust image data segment 4 of the mid-infrared wavelength sensor. The guide point is the midpoint, but in general, the two-dimensional centroid for the target image does not necessarily correspond to the optical projection point for the image plane of the three-dimensional center of gravity of the target 1. (Depending on the posture of the target 1, the centroid and the center of gravity may or may not match.) The aim point of the target 1 cannot always be accurately obtained from the centroid of the target image, and the target 1 destruction rate is reduced. There were issues such as.
Here, the aiming point of the target 1 is a pinpoint that is most likely to be able to destroy the target 1 when the flying object hits the point, and is, for example, the center of gravity of the target.
[0008]
The present invention has been made in order to solve the above-described problems, and an object thereof is to obtain a flying object image processing apparatus and a flying object image processing method capable of accurately calculating a target aiming point. .
[0009]
[Means for Solving the Problems]
The flying object image processing apparatus according to the present invention is provided with estimating means for estimating the relative position and relative posture of the target with respect to the flying object based on the inertial principal axis direction specified by the direction specifying means.
[0010]
The flying object image processing device according to the present invention calculates a target full width value from the inertial principal axis direction specified by the direction specifying means, and estimates a target relative position and relative posture from the total width value and data related to the target shape. Estimating means is provided.
[0011]
The flying object image processing apparatus according to the present invention generates a target estimated image from the relative position and relative posture estimated by the estimating means, calculates an error between the estimated image and the target image, and reduces the error. Update means for updating the full width value of the target in the direction is provided.
[0012]
The flying object image processing apparatus according to the present invention is provided with update means for calculating the gradient of the error and updating the full width value of the target using the gradient.
[0013]
The flying object image processing apparatus according to the present invention includes an updating unit that calculates an area and a cross-sectional secondary moment of the target image and the estimated image, and updates the full width value of the target using the area and the cross-sectional secondary moment. Is.
[0014]
The flying object image processing apparatus according to the present invention includes a luminance value of pixels located at both end points of a line segment that gives the maximum width of the target in the inertial principal axis direction specified by the direction specifying means, and an opening at a feature point of the target image. An estimation means is provided for calculating the full width value of the target from the rate and estimating the relative position and relative posture of the target from the data relating to the full width value and the shape of the target.
[0015]
The flying object image processing apparatus according to the present invention includes a relative position of the target with respect to the flying object based on the inertia main axis direction specified by the first direction specifying means and the inertia main axis direction specified by the second direction specifying means, and An estimation means for estimating the relative posture is provided.
[0016]
The flying object image processing apparatus according to the present invention selects pixels located at both end points of a line segment that gives the maximum target width based on the inertial principal axis direction specified by the first direction specifying means, and the second Based on the inertial principal axis direction specified by the direction specifying means, pixels located at both end points of the line segment that gives the maximum width of the target are selected, and the relative position and relative posture of the target are estimated from the positional relationship of these pixels. An estimation means is provided.
[0017]
The flying object image processing method according to the present invention calculates a geometric characteristic amount of a target from a target image, and specifies the inertia principal axis direction of the target from the geometric characteristic amount. Based on the inertia principal axis direction, the target object relative to the flying object is calculated. The relative position and relative posture are estimated, and the aiming point of the target is calculated from the relative position and relative posture.
[0018]
The flying object image processing method according to the present invention calculates the full width value of the target from the inertial principal axis direction of the target, and estimates the relative position and relative posture of the target from the data related to the full width value and the shape of the target. It is.
[0019]
According to the flying object image processing method of the present invention, an estimated image of a target is generated from the relative position and relative posture of the target, an error between the estimated image and the target image is calculated, and the error of the target is reduced. The full width value is updated.
[0020]
The flying object image processing method according to the present invention calculates an error gradient and updates the full width value of the target using the gradient.
[0021]
The flying object image processing method according to the present invention calculates the area and the cross-sectional secondary moment of the target image and the estimated image, and updates the full width value of the target using the area and the cross-sectional secondary moment. is there.
[0022]
The flying object image processing method according to the present invention is based on the luminance values of pixels located at both end points of a line segment that gives the maximum width of the target in the direction of the principal axis of the target and the aperture ratio at the feature point of the target image. The full width value is calculated, and the relative position and the relative posture of the target are estimated from the data about the full width value and the shape of the target.
[0023]
The flying object image processing method according to the present invention estimates the relative position and relative attitude of the target with respect to the flying object based on one inertial principal axis direction and the other inertial principal axis direction, and targets the target from the relative position and relative attitude. The point is calculated.
[0024]
The flying object image processing method according to the present invention selects pixels located at both end points of a line segment that gives the maximum width of the target based on one inertial principal axis direction, and selects the target based on the other inertial principal axis direction. Pixels located at both end points of the line segment giving the maximum width are selected, and the relative position and relative posture of the target are estimated from the positional relationship of these pixels.
[0025]
DETAILED DESCRIPTION OF THE INVENTION
An embodiment of the present invention will be described below.
Embodiment 1 FIG.
FIG. 1 is a block diagram showing an image processing apparatus for a flying object according to Embodiment 1 of the present invention. In the figure, 11 is an image sensor mounted in front of the flying object and images a target, and 12 is an image sensor 11. Among the captured images, a preprocessing device for identifying a target portion and a background portion, and 13 is a geometric characteristic calculation for calculating a target geometric characteristic amount from a target portion identified by the preprocessing device 12 (hereinafter referred to as a target image). Device (characteristic amount calculation means), 14 is an inertia main axis estimation apparatus (direction specifying means) for estimating two target inertia main axis directions from the geometric characteristic amount calculated by the geometric characteristic calculation device 13, and 15 is a target feature point. It is the feature point detection apparatus (estimating means) which detects a target front-end | tip part.
[0026]
Reference numeral 16 denotes a target database (estimating means) for storing model data relating to the target shape and aiming point, and 17 denotes the relative position and relative of the target with respect to the projectile based on the two inertial principal axis directions estimated by the inertial principal axis estimation device 14. A relative state quantity estimating device (estimating means) 18 for estimating a posture, an axis selecting device 19 for selecting an axis from two inertial spindle directions with reference to a target tip detected by the feature point detecting device 15, 19 Is a full width calculation device that calculates the full width value of the target from two inertial main axis directions, and 20 is a comparison between the full width value calculated by the full width calculation device 19 and model data relating to the shape of the target stored in the target database 16. The collation processing device 21 for estimating the relative position and relative posture of the target, and 21 indicates the relative position and relative posture estimated by the relative state quantity estimating device 17 and the target illumination. A sighting point calculating device to calculate the aiming point of target from the model data about the point (aiming point calculation means).
FIG. 2 is a flowchart showing the flying object image processing method according to the first embodiment of the present invention.
[0027]
Next, the operation will be described.
In the first embodiment, for example, a missile warhead is assumed as a target, and its shape is approximated by a cone 31 as shown in FIG.
The following two coordinate systems are defined.
[0028]
・ Camera coordinate system (X_C , Y_C , Z_C )
The camera coordinate system has an origin O at the camera optical center in the image sensor 11 mounted in front of the flying object._C With Z_C The axis is determined in the direction of the optical axis. X_C The axis is set in the horizontal direction of the image plane 32, and Y_C X axis_C Axis and Z_C Determine in the direction perpendicular to the axis.
・ Target fixed coordinate system (X_B , Y_B , Z_B )
The target fixed coordinate system has an origin O at the center of the bottom surface of the cone 31._B With X in the bottom_B , Y_B Z in the direction of the axis of symmetry_B Define the axis.
[0029]
At this time, the image sensor 11 mounted on the flying object acquires an image including a target located in front of the flying object (step ST1).
When the image sensor 11 acquires an image including a target, the preprocessing device 12 identifies a pixel constituting the target portion and a pixel constituting the background portion from among the pixels constituting the image, and binarizes each pixel. Process (step ST2).
Specifically, among the pixels constituting the image acquired by the image sensor 11, the pixel value of the pixel constituting the target portion is larger than the pixel value of the pixel constituting the background portion. The pixel value of each pixel constituting the image is compared with a threshold value, a pixel having a pixel value larger than the threshold value is recognized as a pixel constituting the target portion, and a pixel having a pixel value smaller than the threshold value is regarded as the background portion. Recognized as a constituent pixel.
Here, the image size of the image sensor 11 is set to I_W × I_H When the (i, j) -th pixel value in the pre-processed image is described as I (i, j), it can be expressed as follows.
[0030]
[Expression 1]

[0031]
When the preprocessing device 12 identifies the pixels constituting the target portion and the pixels constituting the background portion, the geometric characteristic calculation device 13 identifies the geometric characteristic amount of the target portion, that is, the target area S, the centroid (x_c , Y_c ), Sectional moment J_xx, J_xy, J_yyIs calculated as shown below (step ST3).
[0032]
[Expression 2]

[0033]
When the geometric characteristic calculation device 13 calculates the geometric characteristic amount, the inertial spindle estimation device 14 calculates the target cross-sectional secondary moment J_xx, J_xy, J_yyBased on the above, the inertial principal axis direction in the target image is estimated (step ST4).
In general, there are two main inertia axes in the target image (corresponding to the maximum and minimum inertia main axes), as shown by 33 and 34 in FIG. 4, each of which has an orthogonal relationship. If the angle formed with the surface horizontal direction 36 is α, the following relationship is satisfied. In FIG. 4, 35 is the centroid, and 37 is the vertical direction of the image plane.
tan2α = 2J_xy/ (J_yy-J_xx)
0 ≦ α ≦ π
[0034]
Further, when the geometric characteristic calculation device 13 calculates the geometric characteristic amount, the feature point detection device 15 detects the target tip that is the target feature point (step ST5).
As shown in FIG. 5, there are a case where the target tip 39 is visible (right) and a case where it is not visible (left) depending on the posture of the target with respect to the flying object. In FIG. 5, reference numeral 38 denotes a target axis, and reference numeral 40 denotes a minimum inertia spindle.
Therefore, when the target tip 39 is visible, for example, a corner detector such as a SUSAN operator introduced in the literature (Xu, Hajime, “3D Vision”, Kyoritsu Shuppan, pp. 36-37, 1998), etc. To position the target tip 39 (x_v , Y_v ) Is detected.
[0035]
In the SUSAN operator, as shown in FIG. 6, for example, when circular masks a to d are respectively superimposed on an image, the luminance value of the pixel at the center of the mask among the pixels in the masks a to d is the same. A process of counting the number of pixels having luminance values is executed.
For example, the pixel P in the background 42₁And the pixel P in the target 41₂The number of masks b centered at is equal to the mask area. On the other hand, the pixel P on the straight contour_Three, The number of masks is half of the mask area, and the corner pixel P_FourThe number of masks d centered at is 1/4 of the mask area.
[0036]
The above processing is performed on each pixel in the target image, and the pixel having the smallest number, that is, the pixel P_FourIs detected as the target tip 39.
It is possible to determine whether the target tip 39 is visible or not by providing a threshold value for the number, so that it is possible to determine whether the target tip 39 is left or right in FIG.
[0037]
Next, the relative state quantity estimation device 17 estimates the relative position and relative posture of the target with respect to the flying object.
Specifically, first, the axle selection device 18 determines which of the two inertial spindles estimated by the inertial spindle estimation device 14 corresponds to the target axle.
As shown in FIG. 5, there are two types of images depending on the posture of the target with respect to the flying object. However, in the case shown in the right figure, the target tip 39 is detected by the feature point detection device 15. The distance between the tip 39 and the two inertial main axes is calculated, and the smaller one is selected as the target machine axis 38 (step ST6).
On the other hand, in the case shown in the left figure, since the target tip 39 is not detected by the feature point detection device 15, the one having the larger inertia moment around the inertia main axis (maximum inertia main axis) is selected as the target aircraft axis 38 (step ST6). ).
[0038]
Then, when the axis selection device 18 selects the target axis 38, the full width calculation device 19 calculates the full width value in the direction of two inertial main axes in the target image as shown in FIG. 7 (step ST7). The full width value in the direction of the axis is h, and the full width value in the direction orthogonal to the direction of the axis is d.
As a method for calculating the full width value, for example, the pixel value of the pixel existing on the inertial main axis (the pixel value after preprocessing by the preprocessing device 12) is scanned while moving the inertial main axis in parallel. The length of the line segment with the most continuous pixel value is defined as the full width value of the target.
[0039]
Then, when the full width calculation device 19 calculates the full width values h and d of the target, the collation processing device 20 stores in the target database 16 data related to the shape of the target missile warhead (height H) as shown in FIG. Since the bottom surface diameter D) is stored, the relative position and the relative posture of the target are estimated by comparing these data with the full width values h and d (step ST8).
[0040]
For example, in the case of a target image as shown in the left diagram of FIG. 5, the relative posture of the target is given as follows using the 3-2-1 Euler angle from the camera coordinate system to the target fixed coordinate system.
θ_x = ± cos^-1(H / d)
θ_y= Π
θ_z= Α_min−π
The relative position of the target is given as follows.
x_C (R_B) = D [x_c -I_W / 2] / d
y_C (R_B) = D [y_c -I_H / 2] / d
z_C (R_B) = D · f / d
Where α_min Is the tilt angle of the minimum principal axis of inertia 40 from the horizontal direction of the image plane, and f is the camera focal length.
[0041]
On the other hand, in the case of the target image as shown in the right diagram of FIG. 5, the relative posture of the target is given as follows.

The relative position of the target is given as follows.
x_C (R_B) = D [x_v -I_W / 2] / dH · sin θ_x ・ Sinθ_z
y_C (R_B) = D [y_v -I_H / 2] / d + H · sin θ_x ・ Cos θ_z
z_C (R_B) = D · f / d
Where α_sym Is the inclination angle of the target machine axis 38 from the horizontal direction of the image plane.
[0042]
Finally, when the relative state quantity estimating device 17 estimates the relative position and relative posture of the target, the aiming point calculation device 21 refers to the position information of the aiming point 43 stored in the target database 16 as shown below. Then, the aiming point of the target is calculated from the relative position and the relative posture (step ST9).
[0043]
[Equation 3]

[0044]
As apparent from the above, according to the first embodiment, when the geometric characteristic amount of the target is calculated from the target image and the inertia principal axis direction of the target is specified from the geometric characteristic amount, the flight is performed based on the inertia principal axis direction. Since the relative position and relative posture of the target with respect to the body are estimated and the aiming point of the target is calculated from the relative position and relative posture, even if the relative position and relative posture of the target with respect to the flying object change, the target There is an effect that the aiming point can be accurately calculated.
[0045]
Embodiment 2. FIG.
FIG. 9 is a block diagram showing a flying object image processing apparatus according to Embodiment 2 of the present invention. In the figure, the same reference numerals as those in FIG.
22 is an estimated image generating device (updating unit) that generates an estimated image (binarized image) of the target from the relative position and relative posture estimated by the relative state quantity estimating device 17, and 23 is generated by the estimated image generating device 22. An estimation error calculation device (update means) that calculates an estimation error that is an error between the estimated image and the target image preprocessed by the preprocessing device 12, and 24 calculates the gradient of the estimation error calculated by the estimation error calculation device 23. A gradient calculating device (updating unit) 25 is a high-order full-width updating unit (updating unit) that updates the target full width values d and h in a direction in which the estimation error decreases based on the gradient calculated by the gradient calculating unit 24. is there.
[0046]
Next, the operation will be described.
In the first embodiment, when the relative state quantity estimation device 17 estimates the relative position and relative posture of the target, the target aiming point is immediately calculated from the relative position and relative posture. An estimated image is generated from the relative posture, an error between the estimated image and the target image is obtained, and the target full width values d and h are updated so that the error is minimized, and then the target aiming point is calculated. It may be.
[0047]
In the case of the first embodiment, since the full width values d and h in the inertial principal axis direction are calculated with pixel accuracy and the target relative position and posture are estimated, the target relative position with respect to the flying object is relatively short. Yes, there is no problem when the image sensor resolution for the target image is sufficiently secured, but the relative position of the target with respect to the flying object is at a long distance, and when the image sensor resolution for the target image is not sufficiently secured, The estimation accuracy of the relative position and the relative posture deteriorates.
[0048]
Therefore, in the second embodiment, even when the image sensor resolution with respect to the target image is not sufficiently ensured, the following processing is executed in order to improve the estimation accuracy of the relative position and the relative posture.
[0049]
For example, it is assumed that the relative position of the target is relatively far and an image as shown in FIG. 10 is acquired. In this case, the maximum widths h ′ and d ′ in the direction of the principal axis of inertia of the target are measured with reference to the centers of the four pixels marked with “X” in the drawing.
The maximum widths h ′ and d ′ always include an error of quantization in the image sensor 11 with respect to d and h to be actually measured.
This quantization error appears as an estimation error with respect to the estimated value of the relative position and relative posture of the target in the verification processing device 20, and as a result, the calculation accuracy of the aiming point is lowered.
[0050]
A similar situation will be described with reference to FIGS. In both figures, the outline of the target is virtually drawn with a solid line. In these two images, in both cases, the full width value is calculated with reference to the center of the four pixels indicated by the hatched lines in the figure, so the relative position estimated by the collation processing device 20 and There is no difference between the values of the relative postures.
[0051]
However, if each image is viewed as a whole area, there is a difference whether or not the pixels 44 and 45 painted black in the figure have luminance values.
That is, in the case of FIG. 11, the pixels 44 and 45 have a luminance value, and in the case of FIG. 12, the pixels 44 and 45 do not have a luminance value.
Therefore, if a higher-order feature amount over the entire image is used instead of a feature amount on the one-dimensional straight line such as the maximum width in the inertial principal axis direction, the difference between the two can be reflected in the estimated amount. The influence of the quantization error in the sensor 11 can be reduced.
[0052]
When the relative state quantity estimating device 17 estimates the relative position and relative posture, the estimated image generating device 22 generates a target estimated image (binarized image) from the relative position and relative posture.
Then, when the estimated image generating device 22 generates an estimated image of the target, the estimated error calculating device 23 estimates an error that is an error between the estimated image and the target image (binarized image) preprocessed by the preprocessing device 12. e is calculated.
That is, the estimation error e is calculated by defining the square error of luminance between pixels in the estimated image and the target image, or equivalently, the sum of absolute values of luminance differences as follows. In addition, since it is a binarized image, the sum of square errors of pixel values and the sum of absolute values between both images are the same value.
[0053]
[Expression 4]

[0054]
When the estimation error calculation device 23 calculates the estimation error e, the gradient calculation device 24 performs a finite differentiation on the estimation error e by the full width values d and h, and the gradients そ の e / ∂d and ∂e / of the estimation error e. Calculate ∂h.
[0055]
Then, when the gradient calculation device 24 calculates the gradients ∂e / ∂d and ∂e / ∂h of the estimation error e, the high-order full width update device 25 uses the gradients ∂e / ∂d and ∂e / ∂h. Thus, the full width values d and h in the inertial spindle direction are updated in the direction in which the estimation error e decreases.
That is, the previous full width value (here, for convenience of explanation, the previous full width value is set to d_old , H_old (∂e / ∂d) · Δd and (∂e / ∂h) · Δh are subtracted from each other to update the full width values d and h in the inertial spindle direction (here, for the sake of convenience of explanation, updated) The full width value of d_new , H_new ).
d_new = D_old -(∂e / ∂d) · Δd
h_new = H_old -(∂e / ∂h) ・ Δh
However, Δd and Δh are step sizes determined so as to reduce the estimation error e.
[0056]
When the high-order full width update device 25 updates the full width values d and h in the inertial spindle direction, the collation processing device 20 changes the relative position and the relative posture based on the full width value after the update, as in the first embodiment. Candidate values are calculated.
When the matching processing device 20 calculates the relative position and relative orientation candidate values in this manner, this time, the target estimated image is generated from the relative position and relative orientation candidate values, and the above processing is repeated. Further, candidate values for the relative position and the relative posture are calculated.
[0057]
Then, after the above process is repeated a plurality of times, the estimation error calculation device 23 uses the target relative position and the relative orientation candidate values as definite values, and the aim point calculation device 21 where the estimation error e takes a stop value. Output to.
When the estimation error calculation device 23 outputs the relative position and relative posture of the target, the aiming point calculation device 21 calculates the target aiming point for the flying object in the same manner as in the first embodiment.
[0058]
As apparent from the above, according to the second embodiment, an estimated image of the target is generated from the relative position and relative posture of the target, the error between the estimated image and the target image is calculated, and the error is reduced. Since the full width value of the target is updated in the direction, the full width value can be calculated with sub-pixel accuracy, and the quantization error of the image sensor 11 even when the relative distance of the target to the flying object is relatively long. As a result, the relative position and the relative posture of the target can be accurately estimated.
[0059]
Further, gradient calculation is performed on two variables having the full width values d and h in the two inertial spindle directions as parameters, the full width values d and h are updated, and the full width values d and h are used to indirectly from the verification processing device 20. Since the relative position and relative posture of the target are estimated, the relative position and relative posture are more efficient than performing the gradient calculation directly using the six variables of the target relative position and relative posture as parameters. There is an effect that can be estimated.
[0060]
Embodiment 3 FIG.
FIG. 13 is a block diagram showing a flying object image processing apparatus according to Embodiment 3 of the present invention. In the figure, the same reference numerals as those in FIG.
Reference numeral 26 denotes a high-order feature quantity calculation device (update means) that calculates the area and cross-sectional secondary moment of the target image and the estimated image, and 27 denotes the area and cross-section secondary moment calculated by the high-order feature quantity calculation device 26. A high-order full width update device (update means) that updates the full width value of a target.
[0061]
Next, the operation will be described.
In the second embodiment, the gradient of the estimation error e is calculated, and the full width values d and h in the inertial principal axis direction are updated using the gradient. The moment may be calculated, and the full width values d and h of the target may be updated using the area and the moment of inertia of the cross section.
[0062]
Specifically, the high-order feature quantity calculating device 26 calculates the area of the target estimated image generated by the estimated image generating device 22 and the cross-sectional secondary moment around the inertial main axis perpendicular to the axis.
Here, regarding the cross-sectional secondary moment, when the target image is as shown in the left diagram of FIG. 14, a value around the inertia main axis 40 perpendicular to the target axis 38 passing through the centroid 35 is calculated.
On the other hand, in the case of the right diagram in FIG. 14, the value around the axis 46 perpendicular to the target machine axis 38 passing through the target tip 39 is calculated.
[0063]
When the high-order feature quantity calculation device 26 calculates the high-order feature quantity (area and cross-section secondary moment), the high-order full width update device 27 calculates the high-order feature quantity and the geometric characteristic calculated by the geometric characteristic calculation apparatus 13. Using the quantity (area and cross-sectional second moment), the difference between the two is used as the evaluation function.
That is, the difference in target area is defined as a first evaluation function, and the difference in cross-sectional second moment is defined as a second evaluation function.
[0064]
Then, the high-order full width update device 27 updates the full width value in the direction of the inertia spindle 40 perpendicular to the target axle 38 in the direction in which the first evaluation function decreases, and in the direction in which the second evaluation function decreases. Update the full width value of the direction.
That is, in the case of the image as shown in the left diagram of FIG. 14, the full width in the direction of the inertial main axis 40 perpendicular to the target aircraft axis 38 using the target area calculated for the target image that is the original image and the estimated image. Update the value d as follows:
d_new = D_old × (S_org / S_est )^1/2
Where S_org Is the target area in the target image and S_est Is the target area in the estimated image.
[0065]
The full width value h in the direction of the target aircraft axis 38 is updated as follows based on the sectional moment of inertia about the inertial main axis perpendicular to the target aircraft axis 38.
h_new = H_old × (J_ξξ ^org / J_ξξ ^est )^1/3
J_ξξ= J_yy・ Sin² α + J_xx・ Cos² α-J_xy・ Sin2α
Where J_ξξ ^org Is the moment of inertia of the section about the principal axis 40 perpendicular to the target axis 38 in the target image, J_ξξ ^est Is the moment of inertia of the cross section around the principal inertia axis 40 perpendicular to the target axis 38 in the estimated image.
[0066]
On the other hand, in the case of the image as shown in the right diagram of FIG. 14, the full width in the direction of the inertial main axis 40 perpendicular to the target aircraft axis 38 using the target area calculated for the target image that is the original image and the estimated image. The value d is updated as in the case of the left diagram of FIG.
The full width value h in the direction of the target aircraft axis 38 is updated as follows based on the sectional moment of inertia about the inertial main axis perpendicular to the target aircraft axis 38.
h_new = H_old × (J_{ξ'ξ '} ^org / J_{ξ'ξ '} ^est )^1/3
Where J_{ξ'ξ '} ^org Is the moment of inertia of the section around the principal axis 46 perpendicular to the target axis 38 through the target tip 39 in the target image, J_{ξ'ξ '} ^est Is a moment of inertia of the section around the principal axis of inertia 46 perpendicular to the target axis 38 passing through the target tip 39 in the estimated image.
[0067]
Others are the same as those in the second embodiment, and the description thereof is omitted.
As apparent from the above, according to the third embodiment, the target image and estimated image area and cross-sectional secondary moment are calculated, and the full width value of the target is updated using the area and cross-sectional secondary moment. Since the full width value can be calculated with sub-pixel accuracy, the influence of the quantization error of the image sensor 11 can be suppressed even when the relative distance of the target to the flying object is relatively long. As a result, there is an effect that the relative position and relative posture of the target can be accurately estimated.
[0068]
Also, when estimating the relative position and orientation of the target, it can be estimated based on the target image area and cross-section second moment, which can be calculated relatively easily, so that complex calculations such as gradient calculation As a result, there is an effect that the relative position and the relative posture of the target with respect to the flying object can be estimated more efficiently than the second embodiment.
[0069]
Embodiment 4 FIG.
FIG. 15 is a block diagram showing a flying object image processing apparatus according to Embodiment 4 of the present invention. In the figure, the same reference numerals as those in FIG.
61 is a pre-processing device that performs multi-value processing to discriminate between a target portion and a background portion, and 62 calculates a geometric characteristic amount of the target from the target portion identified by the pre-processing device 61 (hereinafter referred to as a target image). Geometric characteristic calculation device (characteristic amount calculation means), 63 is a feature point detection device (estimation means) for detecting the target tip at the target feature point, and 64 is an aperture ratio calculation device for calculating the aperture ratio at the feature point of the target image. (Estimating means).
[0070]
65 calculates the full width value of the target from the luminance values of the pixels located at the two end points of the line segment that gives the maximum width of the target in the inertial principal axis direction, and the aperture ratio at the feature point of the target image. Relative state quantity estimating device (estimating means) for estimating the relative position and relative posture of the target from the model data relating to the shape, 66 represents the luminance values of pixels located at both end points of the line segment that gives the maximum width of the target in the inertial principal axis direction This is a high-order full width calculation device for calculating the full width value of the target from the aperture ratio at the feature point of the target image.
[0071]
Next, the operation will be described.
When the image sensor 11 acquires an image including a target, the preprocessing device 61 identifies a pixel constituting the target portion and a pixel constituting the background portion from among the pixels constituting the image, and multi-values each pixel. Process.
That is, while the pixel value of the pixel constituting the background portion is set to zero, the pixel value of the pixel constituting the target portion is normalized with the maximum luminance value in the target region, and the target region has gradation. Generate an image.
[0072]
When the pre-processing device 61 performs the multi-value processing for each pixel, the geometric characteristic calculation device 62 performs the geometric characteristic amount of the target portion, that is, the target area S, the centroid (x_c , Y_c ), Sectional moment J_xx, J_xy, J_yyCalculate
The calculation of the geometric characteristic calculation device 62 is basically the same calculation as the geometric characteristic calculation device 13 in the first embodiment, except that the image to be calculated is a multi-valued image (in the calculation, All the luminance values of the pixels constituting the target portion may be replaced with 1).
[0073]
When the geometric characteristic calculation device 13 calculates the geometric characteristic amount, the inertial principal axis estimation device 14 is similar to the first embodiment described above in that the target cross-sectional secondary moment J_xx, J_xy, J_yyBased on the above, the inertial principal axis direction in the target image is estimated.
In addition, when the geometric characteristic calculation device 62 calculates the geometric characteristic amount, the feature point detection device 63 detects the target tip that is the target feature point, as in the first embodiment.
[0074]
In addition, when detecting a target front-end | tip part using a corner detector, such as a SUSAN operator, a circular mask is made to act also on a multi-valued image similarly to the case of a binarized image, and the pixel in the mask is detected. Among them, the number of pixels having a luminance value close to the luminance value of the pixel at the center of the mask is counted. This process is performed on each pixel in the target image, and the pixel with the smallest number is detected as the target tip.
[0075]
When the feature point detection device 63 detects the target tip that is the target feature point, the aperture ratio calculation device 64 calculates the aperture ratio at the target tip.
That is, the aperture ratio of the target tip is calculated based on the pixel value in the region near the target tip.
[0076]
As shown in FIG. 16, a circular mask 47 centered on the target tip 39 detected by the feature point detection device 63 is superimposed on the target image, and in the mask area, the luminance value close to the luminance of the center pixel. The number of pixels having N (x_v , Y_v ).
The number of pixels N (x_v , Y_v ) Is divided by the mask area M to calculate the aperture ratio φ.
φ = N (x_v , Y_v ) / M
N (x_v , Y_v For the feature point detection device 63, if the SUSAN operator is used, this is a value already calculated. Therefore, the aperture ratio calculation process can be completed as an additional process for the SUSAN operator.
[0077]
Next, the relative state quantity estimation device 65 estimates the relative position and relative posture of the target with respect to the flying object.
Specifically, first, the axle selection device 18 determines which of the two inertial spindles estimated by the inertial spindle estimation device 14 corresponds to the target axle, as in the first embodiment.
[0078]
Then, the high-order full width calculation device 66 has pixels corresponding to the opening ratio at the target tip calculated by the opening ratio calculation device 64 and both end points of the line segment that gives the maximum width of the target in each of the two principal axes of inertia. Based on the luminance value, the full width value of the target in each of the inertial principal axis directions is calculated with subpixel accuracy.
[0079]
As an example, consider a target image as shown in FIG.
In FIG. 17, the pixels corresponding to the end points of the line segments 48 and 49 that give the maximum width in the direction of the principal axis are indicated by diagonal lines in the figure.₁~ P_FourIt is.
Pixel P indicating target tip₁FIG. 18 is an enlarged view of FIG.
[0080]
The luminance value of each pixel in the target area is considered to be approximately proportional to the area of the target area that occupies the pixel.
That is, the pixel value is uniformly 1 inside the target area, and the pixel value in the target contour portion can be approximately regarded as being given by the ratio (area ratio) of the target area to each pixel.
[0081]
Therefore, as shown in FIG.₁Is divided into two regions (a region 50 indicated by hatching in the figure and a region 51 other than that) using the aperture ratio φ obtained by the aperture ratio calculation device 64.
That is, as shown in FIG. 18, one point on the axis 38 passing through the centroid 35 of the target image is set to the vertex V.₁And apex V₁The pixels are divided into two regions 50 and 51 by drawing straight lines 52 and 53 symmetrically with respect to the target machine axis 38 at an angle ψ = 2πφ.
And pixel P₁The area ratio of the region 50 indicated by oblique lines in the figure to the area of₁Pixel P to be equal to the luminance value of₁Vertex V₁Determine the position.
[0082]
Further, the pixel P other than the feature points in FIG.₂~ P_FourIn, for example, P₂, The pixel P as shown in FIG.₂A point on the inertial main axis 54a perpendicular to the target axis 38 passing through₂And apex V₂The pixel is divided into two regions (a region 55 indicated by hatching in the drawing and a region 56 other than that) along a straight line 54b that passes through and is perpendicular to the inertial main axis 54a.
And pixel P₂The area ratio of the region 55 indicated by oblique lines in the figure to the area of₂Point V so that it is equal to the brightness value of₂Determine the position. Pixel P_Three, P_FourIn the same way, point V_Three, V_FourTo decide.
[0083]
The point V thus determined₁~ V_FourFrom the position, the target maximum width in each of the two principal axes of inertia is calculated with subpixel accuracy, and the total width in the machine axis direction is h, and the total width in the direction orthogonal thereto is d.
[0084]
Then, when the full width calculation device 19 calculates the full width values d and h of the target, the collation processing device 20 estimates the relative position and relative posture of the target as in the first embodiment.
Finally, when the relative state quantity estimating device 65 estimates the relative position and relative posture of the target, the aiming point calculating device 21 calculates the target aiming point as in the first embodiment.
[0085]
As is apparent from the above, according to the fourth embodiment, the luminance values of the pixels located at both end points of the line segment that gives the maximum width of the target in the direction of the target principal axis and the aperture ratio at the feature point of the target image The target's full width value is calculated with subpixel accuracy, and the target's relative position and posture are estimated from the full width value and model data related to the target's shape. Even in the case of a long distance, the influence of the quantization error of the image sensor 11 can be suppressed, and as a result, the relative position and the relative posture of the target can be accurately estimated.
[0086]
In addition, when estimating the relative position and relative posture of the target, it is not necessary to perform complicated repeated calculation processing such as gradient calculation, so that it is possible to efficiently estimate the relative position and relative posture of the target with respect to the flying object. There is an effect.
[0087]
Embodiment 5. FIG.
FIG. 20 is a block diagram showing a flying object image processing apparatus according to Embodiment 5 of the present invention. In the figure, the same reference numerals as those in FIG.
67 is a half mirror that outputs the front three-dimensional world to the image sensors 68 and 69, 68 is an image sensor similar to the image sensor 11, and 69 is horizontal and vertical in the image plane with respect to the target image captured by the image sensor 68. An image sensor arranged at a position displaced by 0.5 pixel in the direction, 70 is a flying object based on the inertial principal axis direction related to the target image on the image sensor 68 side and the inertial principal axis direction related to the target image on the image sensor 69 side Relative state quantity estimating device (estimating means) for estimating the relative position and relative position of the target with respect to the main axis both end points 71 for selecting pixels corresponding to the end points of the line segment that gives the maximum width of the target in the two inertial main axis directions A selection device 72 is a high-order full width calculation device that calculates the full width value of the target with sub-pixel accuracy from the positional relationship of each pixel selected by the spindle end point selection device 71.
[0088]
Next, the operation will be described.
First, the half mirror 67 outputs the three-dimensional world including the target in front of the flying object to the image sensors 68 and 69.
The image sensors 68 and 69 are configured such that an image acquired by one image sensor is 0.5 pixels in the horizontal direction in the image plane and 0.5 pixels in the vertical direction in the image plane with respect to an image acquired by the other image sensor. They are arranged so that they are translated.
As shown in FIG. 21, such an arrangement is such that one image sensor, for example, the image sensor 68 is fixed with respect to the optical axis a, and the image sensor 69 is light in a plane formed by the optical axes a and b. This can be realized by translational displacement in the direction of the arrow c perpendicular to the axis b and the direction of the arrow d perpendicular to the plane.
[0089]
The first image including the target in front of the flying object acquired by the image sensor 68 and the second image including the target in front of the flying object acquired by the image sensor 69 are respectively the same as in the first embodiment. The preprocessing device 12, the geometric characteristic calculation device 13, the inertial spindle estimation device 14, and the feature point detection device 15 operate.
[0090]
Next, the relative state quantity estimation device 70 estimates the relative position and relative posture of the target with respect to the flying object.
First, for each of the first image after the pre-processing and the second image, the axis selection device 18 corresponds to the target axis of the two inertia main axes estimated by the inertia main axis estimation device 14. Is determined.
[0091]
Next, for each of the first image and the second image, the principal axis end point selectors 71 and 71 are arranged at the end points of the line segments 48 and 49 that give the maximum width of the target in each of the two inertial principal axis directions. The corresponding pixel is selected and its position is output.
For example, FIG. 22 shows the case for the first image, and the pixel P indicated by the oblique lines in the figure.₁~ P_FourAre selected and the respective positions in the first image are output.
A similar process is performed on the second image, and the pixels Q corresponding to the end points of the line segment that gives the maximum width in the inertial principal axis direction.₁~ Q_FourAre selected and the respective positions in the second image are output.
[0092]
The high-order full-width calculating device 72 has pixels P corresponding to the first images selected by the principal-axis end point selecting device 71, respectively.₁~ P_Four, And pixel Q for the second image₁~ Q_FourThe maximum width in the main axis direction of the target image is calculated with sub-pixel accuracy based on the relative positional relationship of.
For example, it is assumed that the image sensor 69 is obtained by translating the image obtained by the image sensor 68 by +0.5 pixel in the horizontal direction in the image plane and +0.5 pixel in the vertical direction.
[0093]
At this time, a feature point corresponding to the target tip 39 in both images is represented by a pixel P.₁, Q₁Then, as shown in the left diagram of FIG.₁, The actual target tip 39 in the second image is in the first quadrant 85 in the same pixel as shown in the right figure. Therefore, the pixel P detected in both images₁And Q₁There is no difference in position.
[0094]
Thus, for example, when the actual target tip 39 is in the first quadrant 86 as shown in the left diagram of FIG. 24, the actual target tip 39 in the second image is horizontal in the image plane as shown in the right diagram. Pixel Q displaced in translation by +1 pixel in the direction and +1 pixel in the vertical direction₁In the third quadrant 87, the pixel P detected in both images₁And pixel Q₁The positional relationship between is the pixel Q₁Is the pixel P₁On the other hand, the translational displacement is +1 pixel in the horizontal direction in the image plane and +1 pixel in the vertical direction.
[0095]
Similarly, if the actual target tip 39 is in the second quadrant, the pixel Q₁Is the pixel P₁When the actual target tip 39 is in the fourth quadrant only in the vertical direction in the image plane, the pixel Q₁Is the pixel P₁On the other hand, the translational displacement is +1 pixel only in the horizontal direction in the image plane.
[0096]
That is, each pixel P_iAnd Q_iBy examining the relative positional relationship in the image plane (i = 1, 2, 3, 4), the actual target principal axis direction both ends are the pixels P in the first image._iYou can determine which quadrant you are in.
The maximum width of the target in the direction of the two principal axes of inertia is the center point of this quadrant, for example, the point C shown by x in FIGS.₁As a reference, the value is calculated with sub-pixel accuracy.
[0097]
Then, when the high-order full-width calculating device 66 calculates the target full-width values h and d, the collation processing device 20 estimates the target relative position and relative posture as in the first embodiment.
Finally, when the relative state quantity estimating device 17 estimates the relative position and relative posture of the target, the aiming point calculating device 21 calculates the target aiming point as in the first embodiment.
[0098]
As is apparent from the above, according to the fifth embodiment, the target object relative to the flying object is based on the inertial principal axis direction related to the target image on the image sensor 68 side and the inertial principal axis direction related to the target image on the image sensor 69 side. Since the relative position and the relative posture are estimated, the resolution of the image sensor can be apparently doubled in both the horizontal direction and the vertical direction. As a result, the relative distance of the target with respect to the flying object is increased. Since the influence of the quantization error of the image sensor can be suppressed even at relatively long distances, it is possible to accurately estimate the relative position and relative posture of the target and to accurately calculate the target aiming point using the target. There is an effect that can be done.
[0099]
In addition, when estimating the relative position and relative posture of the target, it is not necessary to perform complicated repeated calculation processing such as gradient calculation, so that it is possible to efficiently estimate the relative position and relative posture of the target with respect to the flying object. There is an effect.
[0100]
【The invention's effect】
As described above, according to the present invention, since the estimation means for estimating the relative position and relative posture of the target with respect to the flying object based on the inertial principal axis direction specified by the direction specifying means is provided, Even if the relative position and relative posture of the target change, there is an effect that the aiming point of the target can be accurately calculated.
[0101]
According to the present invention, there is provided estimation means for calculating the full width value of the target from the inertia main axis direction specified by the direction specifying means, and estimating the relative position and relative posture of the target from the data related to the full width value and the shape of the target. Thus, it is possible to easily estimate the relative position and relative posture of the target.
[0102]
According to this invention, an estimated image of the target is generated from the relative position and relative posture estimated by the estimating means, an error between the estimated image and the target image is calculated, and the full width value of the target in a direction in which the error decreases. As a result, the influence of the quantization error of the image sensor can be suppressed even when the relative distance of the target to the flying object is relatively long. There is an effect that the relative position and the relative posture can be estimated with high accuracy.
[0103]
According to the present invention, since the update means for calculating the gradient of the error and updating the full width value of the target using the gradient is provided, the relative position and the relative posture of the target with respect to the flying object are accurately estimated. There is an effect that can.
[0104]
According to the present invention, since the area and the cross-sectional secondary moment of the target image and the estimated image are calculated and the update means for updating the full width value of the target using the area and the cross-sectional secondary moment is provided, There is an effect that the relative position and relative posture of the target with respect to the body can be efficiently estimated.
[0105]
According to the present invention, the total width of the target is determined from the luminance values of the pixels located at both end points of the line segment that gives the maximum width of the target in the inertial principal axis direction specified by the direction specifying means, and the aperture ratio at the feature point of the target image. In the case where the relative distance of the target with respect to the projectile is relatively long, the estimation means for calculating the value and estimating the relative position and relative posture of the target from the data on the full width value and the shape of the target is provided. However, the influence of the quantization error of the image sensor is suppressed, and as a result, there is an effect that the relative position and the relative posture of the target can be accurately estimated.
[0106]
According to this invention, the estimation for estimating the relative position and the relative posture of the target with respect to the flying object based on the inertia main axis direction specified by the first direction specifying means and the inertia main axis direction specified by the second direction specifying means. Since the means is provided, the influence of the quantization error of the image sensor can be suppressed even when the relative distance of the target to the flying object is relatively long. There is an effect that the posture can be accurately estimated and the aiming point of the target can be accurately calculated using the posture.
[0107]
According to the present invention, the pixels located at both end points of the line segment that gives the maximum width of the target are selected based on the inertia principal axis direction specified by the first direction specifying means, and specified by the second direction specifying means. Based on the inertial principal axis direction, a pixel located at both end points of the line segment that gives the maximum width of the target is selected, and an estimation means for estimating the relative position and relative posture of the target from the positional relationship of these pixels is provided. Since it comprised, there exists an effect which can estimate the relative position and relative attitude | position of a target with respect to a flying body accurately.
[0108]
According to the present invention, when the geometric characteristic amount of the target is calculated from the target image and the inertia principal axis direction of the target is specified from the geometric characteristic amount, the relative position and the relative posture of the target with respect to the flying object are calculated based on the inertia principal axis direction. Since the target sighting point is calculated from the relative position and relative posture, the target sighting point can be accurately calculated even if the target relative position and relative posture with respect to the flying object change. There is an effect that can be done.
[0109]
According to the present invention, the total width value of the target is calculated from the inertial principal axis direction of the target, and the relative position and the relative posture of the target are estimated from the data regarding the total width value and the target shape. There is an effect that the relative position and the relative posture can be estimated.
[0110]
According to the present invention, an estimated image of a target is generated from the relative position and orientation of the target, an error between the estimated image and the target image is calculated, and the full width value of the target is updated in a direction in which the error decreases. Therefore, even when the relative distance of the target to the flying object is relatively long, the influence of the quantization error of the image sensor can be suppressed, and as a result, the relative position and relative posture of the target can be accurately determined. There is an effect that can be estimated.
[0111]
According to the present invention, since the gradient of the error is calculated and the full width value of the target is updated using the gradient, it is possible to accurately estimate the relative position and relative posture of the target with respect to the flying object. There is.
[0112]
According to the present invention, the area and the cross-sectional secondary moment of the target image and the estimated image are calculated, and the total width value of the target is updated using the area and the cross-sectional secondary moment. There is an effect that the relative position and the relative posture can be efficiently estimated.
[0113]
According to the present invention, the total width value of the target is calculated from the luminance values of the pixels located at both end points of the line segment that gives the maximum width of the target in the direction of the principal axis of the target and the aperture ratio at the feature point of the target image, Since the relative position and orientation of the target are estimated from the data on the full width value and the target shape, the quantization error of the image sensor can be reduced even when the relative distance of the target to the projectile is relatively long. As a result, there is an effect that the relative position and relative posture of the target can be accurately estimated.
[0114]
According to the present invention, the relative position and relative posture of the target with respect to the flying object are estimated based on one inertia main axis direction and the other inertia main axis direction, and the target aiming point is calculated from the relative position and relative posture. As a result, even when the relative distance of the target to the flying object is relatively long, the influence of the quantization error of the image sensor can be suppressed, and as a result, the relative position and orientation of the target can be estimated accurately. In addition, it is possible to accurately calculate the target aiming point using the target.
[0115]
According to the present invention, a pixel located at both end points of a line segment that gives the maximum target width based on one inertia main axis direction and a line segment that gives the maximum target width based on the other inertia main axis direction are selected. Since the target position and relative posture of the flying object are accurately calculated, the relative position and relative posture of the target are estimated by selecting the pixels located at both end points of the target. There is an effect that can.
[Brief description of the drawings]
FIG. 1 is a configuration diagram showing a flying object image processing apparatus according to Embodiment 1 of the present invention;
FIG. 2 is a flowchart showing a flying object image processing method according to Embodiment 1 of the present invention;
FIG. 3 is an explanatory diagram showing a camera coordinate system and a target fixed coordinate system.
FIG. 4 is an explanatory diagram showing the inertial principal axis direction of a target image.
FIG. 5 is an explanatory diagram showing that an image (target shape) varies depending on a posture of a target with respect to a flying object.
FIG. 6 is an explanatory diagram for explaining the principle of a SUSAN operator showing an example of a corner detector.
FIG. 7 is an explanatory diagram showing maximum widths in two inertial spindle directions.
FIG. 8 is an explanatory diagram showing data stored in a target database.
FIG. 9 is a block diagram showing a flying object image processing apparatus according to Embodiment 2 of the present invention;
FIG. 10 is an explanatory diagram for explaining the influence of a quantization error in an image sensor.
FIG. 11 is an explanatory diagram for explaining an influence of a quantization error in the image sensor.
FIG. 12 is an explanatory diagram for explaining the influence of a quantization error in the image sensor.
FIG. 13 is a block diagram showing a flying object image processing apparatus according to Embodiment 3 of the present invention;
FIG. 14 is an explanatory diagram for explaining the update of the full width value in the inertial spindle direction.
FIG. 15 is a block diagram showing a flying object image processing apparatus according to Embodiment 4 of the present invention;
FIG. 16 is an explanatory diagram illustrating calculation of an image aperture ratio at a target tip.
FIG. 17 is an explanatory diagram showing a line segment giving the maximum width in the inertial principal axis direction and pixels corresponding to both end points thereof.
FIG. 18 Pixel P₁End point V₁It is explanatory drawing explaining the determination of.
FIG. 19 Pixel P₂End point V₂It is explanatory drawing explaining the determination of.
FIG. 20 is a block diagram showing a flying object image processing apparatus according to Embodiment 5 of the present invention;
FIG. 21 is an explanatory diagram for explaining the arrangement of the image sensors 68 and 69;
FIG. 22 is an explanatory diagram showing pixels corresponding to both end points of a line segment that gives the maximum width in the inertial principal axis direction;
FIG. 23 is an explanatory diagram showing a relative positional relationship (no relative position error) between the first image and the second image of the target tip.
FIG. 24 is an explanatory diagram showing a relative positional relationship (with a relative position error) between the first image and the second image of the target tip.
FIG. 25 is an image view showing an image taken by a camera mounted in front of a flying object.
FIG. 26 is an explanatory diagram for explaining a conventional flying object image processing apparatus.
FIG. 27 is an explanatory diagram for explaining a conventional flying object image processing apparatus.
FIG. 28 is an explanatory diagram for explaining a conventional flying object image processing apparatus.
[Explanation of symbols]
13, 62 Geometric characteristic calculation device (characteristic amount calculation means), 14 Inertial spindle estimation device (direction specifying means), 15, 63 Feature point detection device (estimation means), 16 Target database (estimation means), 17, 65, 70 Relative state quantity estimation device (estimation means), 21 Aim point calculation device (aim point calculation means), 22 Estimated image generation device (update means), 23 Estimation error calculation device (update means), 24 Gradient calculation device (update means) 25, 27 Higher order full width updating device (updating means), 26 Higher order feature amount calculating device (updating means), 64 Aperture ratio calculating device (estimating means).

Claims (16)

A characteristic amount calculating means for calculating a geometric characteristic amount of the target from the target image, a direction specifying means for specifying the inertial principal axis direction of the target from the geometric characteristic amount calculated by the characteristic amount calculating means, and the direction specifying means Estimating means for estimating the relative position and relative posture of the target with respect to the flying object based on the inertial principal axis direction, and aiming point calculating means for calculating the aiming point of the target from the relative position and relative posture estimated by the estimating means. A flying object image processing apparatus provided. The estimation means calculates the full width value of the target from the direction of the principal axis of inertia specified by the direction specifying means, and estimates the relative position and relative posture of the target from data relating to the full width value and the shape of the target. The flying object image processing apparatus according to 1. Update means for generating an estimated image of the target from the relative position and relative posture estimated by the estimating means, calculating an error between the estimated image and the target image, and updating the full width value of the target in a direction in which the error decreases. The flying object image processing apparatus according to claim 2, wherein the flying object image processing apparatus is provided. 4. The flying object image processing apparatus according to claim 3, wherein the updating means calculates an error gradient and updates the full width value of the target using the gradient. The update means calculates the area and the cross-sectional secondary moment of the target image and the estimated image, and updates the full width value of the target using the area and the cross-sectional secondary moment. Image processing device. The estimation means calculates the full width value of the target from the luminance values of the pixels located at both end points of the line segment that gives the maximum width of the target in the inertial principal axis direction specified by the direction specifying means, and the aperture ratio at the feature point of the target image. 2. The flying object image processing apparatus according to claim 1, wherein the target image is calculated and the relative position and the relative posture of the target are estimated from data relating to the full width value and the shape of the target. A first characteristic amount calculating means for calculating a target geometric characteristic amount from the target image, and a target geometry from the target image displaced by 0.5 pixels in the horizontal direction and the vertical direction in the image plane with respect to the target image; Second characteristic amount calculating means for calculating a characteristic amount; first direction specifying means for specifying the inertial principal axis direction of the target from the geometric characteristic amount calculated by the first characteristic amount calculating means; and the second Second direction specifying means for specifying the target inertial principal axis direction from the geometric characteristic amount calculated by the characteristic amount calculating means, inertial main axis direction specified by the first direction specifying means, and second direction specifying means Estimating means for estimating the relative position and relative posture of the target relative to the flying object based on the inertial principal axis direction specified by, and the aiming point for calculating the target aiming point from the relative position and relative posture estimated by the estimating means The image processing apparatus of the projectile with a constant section. The estimating means selects pixels located at both end points of the line segment that gives the maximum width of the target based on the inertial principal axis direction specified by the first direction specifying means, and is specified by the second direction specifying means 8. A pixel located at both end points of a line segment that gives the maximum width of the target based on the inertial principal axis direction is selected, and the relative position and relative posture of the target are estimated from the positional relationship of these pixels. The flying object image processing apparatus described. When calculating the geometric characteristic amount of the target from the target image and specifying the inertia principal axis direction of the target from the geometric characteristic amount, the relative position and relative posture of the target with respect to the flying object are estimated based on the inertia principal axis direction, and the relative position And a flying object image processing method for calculating a target aiming point from a relative posture. 10. The flying object image processing method according to claim 9, further comprising: calculating a full width value of the target from the inertial principal axis direction of the target, and estimating a relative position and a relative posture of the target from data relating to the full width value and the shape of the target. . An estimated image of the target is generated from the relative position and relative posture of the target, an error between the estimated image and the target image is calculated, and the full width value of the target is updated in a direction in which the error decreases. The flying object image processing method according to 10. 12. The flying object image processing method according to claim 11, wherein a gradient of the error is calculated, and the full width value of the target is updated using the gradient. 12. The flying object image processing method according to claim 11, wherein the area and the cross-sectional secondary moment of the target image and the estimated image are calculated, and the full width value of the target is updated using the area and the cross-sectional secondary moment. The total width value of the target is calculated from the luminance values of the pixels located at both end points of the line segment that gives the maximum width of the target in the direction of the target principal axis and the aperture ratio at the feature point of the target image. 10. The flying object image processing method according to claim 9, wherein the relative position and relative posture of the target are estimated from data relating to the shape. While calculating the geometric characteristic amount of the target from the target image and specifying the inertial principal axis direction of the target from the geometric characteristic amount, the target image is displaced by 0.5 pixel in the horizontal direction and the vertical direction in the image plane. The target geometric characteristic amount is calculated from the target image, and the target principal axis direction of the target is determined from the geometric characteristic amount. A flying object image processing method for calculating a target aiming point from the relative position and relative posture. Select pixels located at both end points of the line segment that gives the maximum width of the target based on one inertial principal axis direction, and locate at both end points of the line segment that gives the maximum target width based on the other inertial principal axis direction 16. The flying object image processing method according to claim 15, wherein pixels are selected, and a relative position and a relative posture of the target are estimated from a positional relationship between the pixels.

Images