JP4031306B2 - 3D information detection system - Google Patents

Description

ここで、Ｔ_Ｌはカメラレンズ７などのレンズ光学系の透過率、ρは被写体１の表面の反射特性係数、Ｆ_０は光の最大照射強度、Δｔは撮像時間幅であり、光変調周期に対して十分小さい値である。また、ｃは光速、２ｄ／ｃは高速撮像素子９から被写体１までの距離ｄを光が往復する時間、ｌはカメラレンズ７から被写体１までの距離であり、式の分母は光の拡散による減衰を考慮した項である。
【００３７】
次に、図８（Ｂ）に示すように、時間と共に係数ｓで光強度が減少する強度変調光２６を照射し、被写体１からの反射光２７を時刻ｔｓにパルス状撮像ゲイン２５で短時間撮像した場合、高速撮像素子９で検出される信号量Ｅ₋（ｄ，ｔｓ）は、（３）式で表される。
【００３８】
【数２】

ここで、Ｔは光強度の変調周期である。なお、１回の撮像では感度が不十分である場合は、１フィールド内に、撮像ゲインの変調周波数と同等の繰り返しパルス光を照射し、蓄積型の撮像素子で蓄積し十分な感度を確保する。なお、図８（Ｂ）の強度変調光は図８（Ａ）の強度変調光と連続して送出しても良い。
【００３９】
（２）式と（３）式より、光強度の異なる２枚の画像間での強度比Ｒ＝Ｅ_＋／Ｅ₋をとり、距離ｄを求めると、（４）式となる。
【００４０】
【数３】

（４）式で示されるように、光強度増加時と光強度減少時に撮像した２つの画像間の比Ｒを計算するだけで、被写体１の反射率や光の拡散による光の減衰効果等の影響をキャンセルし、高速に距離を求めることができる。
【００４１】
このように、時間と共に強度変化する光源は、数十ＭＨｚの低い周波数でも実現できるため、レーザ光源以外に、ＬＥＤ（発光ダイオード）などインコヒーレントな照明も使用でき、高輝度で広い範囲を照明でき、人物の距離検出なども可能となる。なお、この方法による距離の検出範囲は、撮像時刻ｔｓと変調周波数ｆで決まり、（５）式で表される。
【００４２】
【数４】

実際の運用時には、撮像時刻ｔｓを調整し、被写体の位置に検出範囲を合わせる。また、測定レンジｄ_ｒは、変調周波数ｆで決まり（６）式で表される。
【００４３】
ｄ_ｒ＝ｃ／４ｆ …（６）
測定レンジｄ_ｒが７．５ｍのとき変調周波数ｆは１０ＭＨｚとなり、測定レンジｄ_ｒが１．５ｍのとき変調周波数ｆは５０ＭＨｚとなる。一方、撮像時の撮像時間幅Δｔは光源２の変調周期１／ｆより短く、数ｎｓｅｃのシャッタ機能が必要である。
【００４４】
次に、強度変調光として短時間のパルス光を使用し、高速撮像素子９の撮像ゲインを時間と共に高速に増加させながら撮像した画像と、撮像ゲインを時間と共に高速に減少させながら撮像した画像間で演算を行い、距離を検出する方法を説明する。
【００４５】
距離ｄの位置にある被写体１に、図９（Ａ）に示すように、光強度Ｆ_０のパルス光２８を時刻ｔｐに照射し、被写体１からの反射光２９を時間と共に係数ｇで増加する撮像ゲイン３０を持つ高速撮像素子９で撮像した場合のカメラで検出される信号量Ｅ_＋（ｄ，ｔｐ）は、（７）式で表される。
【００４６】
【数５】

ここで、Ｔ_Ｌはカメラレンズ７などのレンズ光学系の透過率、ρは被写体１の表面の反射特性係数、τはパルス幅、ｃは光速、２ｄ／ｃは高速撮像素子９から被写体１までの距離ｄを光が往復する時間、ｌはカメラレンズ７から被写体１までの距離であり、式の分母は光の拡散による減衰を考慮した項である。
【００４７】
次に、図９（Ｂ）に示すように、時間と共に係数ｇで減少する撮像ゲイン３１を持つ高速撮像素子９で撮像した場合のカメラで検出される信号量Ｅ₋（ｄ，ｔｐ）は、（８）式で表される。
【００４８】
【数６】

ここで、Ｔは撮像ゲインの変調周期である。なお、１回の撮像では感度が不十分である場合は、１フィールド内に、撮像ゲインの変調周波数と同等の繰り返しパルス光を照射し、蓄積型の撮像素子で蓄積し十分な感度を確保する。なお、図９（Ｂ）の撮像ゲインの変調は図９（Ａ）の撮像ゲインの変調に連続するように設定しても良い。
【００４９】
（７）式と（８）式より、撮像ゲインの異なる２枚の画像間での強度比Ｒ＝Ｅ_＋／Ｅ₋をとり、距離ｄを求めると、（９）式となる。
【００５０】
【数７】

上記（９）式で示されるように、撮像ゲイン増加時と撮像ゲイン減少時に撮像した２つの画像間の強度比Ｒを計算するだけで、被写体１の反射率や光の拡散による光の減衰効果等の影響をキャンセルし、高速に距離を求めることができる。
【００５１】
このように、パルス状の光と、時間と共に変化する撮像ゲインの組合せでは、パルスレーザが使用でき、Ｑスイッチパルスレーザ光、モード同期パルス光など、パルス幅がピコ秒からフェムト秒のパルスレーザ光が使用できるため、分解能が高い距離検出が可能となる。
【００５２】
この場合、測定できる範囲（測定レンジ）Ｄは、（１０）式で表される。
【００５３】
【数８】

次に、矩形波状に変調した強度変調光と、矩形波状に撮像ゲインが変化する高速撮像素子９による距離検出方法を説明する。距離ｄに位置する被写体１に、図１０（Ａ）に破線で示すような矩形の強度変調光３２（光強度Ｆ_０、変調周期Ｔ）を照射し、その反射光３３を強度変調光３２と同一周期で同位相の矩形状の撮像ゲイン３４（ゲインｇ、周期Ｔ）で撮像する場合、カメラで検出される信号量Ｅ_＋（ｄ，ｔｐ）は、（１１）式で表される。
【００５４】
【数９】

ここで、Ｔ_Ｌはカメラレンズ７などのレンズ光学系の透過率、ρは被写体１表面の反射特性係数、ｃは光速、２ｄ／ｃは高速撮像素子９から被写体１までのまでの距離ｄを光が往復する時間、ｌはカメラレンズ７から被写体１までの距離であり、式の分母は光の拡散による減衰を考慮した項である。
【００５５】
次に、図１０（Ｂ）に示すように、周期Ｔだけシフトした強度変調光３２と逆位相の矩形状の撮像ゲイン３５（ゲインｇ、周期Ｔ）で撮像する場合、カメラで検出される信号量Ｅ₋（ｄ，ｔｐ）は、（１２）式で表される。
【００５６】
【数１０】

なお、１回の撮像では感度が不十分である場合は、１フィールド内に、撮像ゲインの変調周波数と同等の繰り返しパルス光を照射し、蓄積型の撮像素子で蓄積し十分な感度を確保する。
【００５７】
（１１）式と（１２）式より、２つの画像間の強度比Ｒ＝Ｅ_＋／Ｅ₋をとり、距離ｄを求めると、（１３）式となる。
【００５８】
【数１１】

（１３）式で示されるように、２つの画像間の比Ｒを計算するだけで、被写体１の反射率や光の拡散による光の減衰効果等の影響をキャンセルし、高速に距離を求めることができる。
【００５９】
このように強度変調照射光と、撮像ゲインを同じ周期の幅を持つ矩形波状とすることで、撮像素子で検出される検出信号量が多くなり、信号対ノイズ比が高い撮像が可能であり、高分解能な距離検出が可能となる。
【００６０】
図１１に、色分離結像光学系８の一実施例を示す。同図中、カメラレンズ７のあとにカラー画像撮影に使用する第２光源６からの可視光成分を含む光３７を透過し、距離検出に使用する第１光源２からの可視領域外の光３８を反射させるダイクロイックプリズム３６を有する。このダイクロイックプリズム３６のサイズと材質は、カメラレンズ７のバックフォーカス長を考慮し、高速撮像素子９にカメラレンズ７よりダイクロイックプリズム３６を通して結像できる光路長となるように選択する。
【００６１】
距離検出用の強度変調光源である第１光源２の出力光を近赤外光とした場合、ダイクロイックプリズム３６による色分離特性は、可視光を透過し、近赤外光を反射する特性とする。第２光源６の波長を７８０ｎｍから９００ｎｍの範囲を選択すれば、可視光の透過率を高く、しかも近赤外光の反射率を高く設計することができる。
【００６２】
また、ダイクロイックプリズム３６で反射した光成分に含まれる第１光源２の波長成分より長波長の光を除去する赤外透過の光学フィルタ３９を配置することで、外光の影響を低減し、信号光である近赤外光のみを高速撮像素子９に入力できる特性が得られる。
【００６３】
ダイクロイックプリズム３６を通過した可視光はリレーレンズ４０を通してカラーカメラ１２に入力される。このリレーレンズ４０の倍率はほぼ等倍であり、レンズにはフォーカス調整機能を備える。このとき、リレーレンズ４０の構成をテレセントリック光学系とすることでフォーカス調整が容易となる。
【００６４】
リレーレンズ４０は、可視領域において、透過率や収差などの結像光学特性の最適化を行い、ＭＴＦ（Ｍｏｄｕｌａｔｉｏｎ Ｔｒａｎｓｆｅｒ Ｆｕｎｃｔｉｏｎ）の低下を押さえ、カメラレンズによる結像光学画像を高精細、かつ、画像の色変化や輝度の低下なくカラーカメラ１２に伝達入力する。また、リレーレンズ４０の開口数をカメラレンズ７と同等とすることで、明るさの低減を抑える。また、リレーレンズ４０には絞り調整機構をもたせることで、距離検出側への近赤外光量に影響なく、カラー画像の絞り調整が可能となる。
【００６５】
図１２に、高速撮像素子９として、イメージインテンシファイア４１とＣＣＤカメラ４２を用いた実施例を示す。同図中、ダイクロイックプリズム３６で反射した第１光源２の光成分をイメージインテンシファイア４１に結像入力する。イメージインテンシファイア４１は、光電変換面４３、マイクロチャンネルプレート（ＭＣＰ）４４、蛍光面４５から構成されるものが使用できる。
【００６６】
イメージインテンシファイア４１の光電変換面４３とＭＣＰ４４間の印加電圧の値を制御することで、時間と共に撮像感度を高速に変化できる。また、印加電圧をパルス形状とすることで、１ｎｓｅｃ以下の短時間のゲート撮像も可能となる。
【００６７】
光電変換面４３の材料は、距離検出に使用する強度変調光の波長感度特性により選択する。例えば、波長８００ｎｍの場合、マルチアルカリ材料（Ｓｂ−Ｋ−Ｎａ−Ｃｓ）が使用でき、更に長波長の８５０ｎｍを使用する場合は、ガリウム砒素（ＧａＡｓ）材料を使用することで高い量子効率が得られるため、Ｓ／Ｎのよい撮像画像が得られる。
【００６８】
撮像ゲインを高速に変化させ、または短時間のゲート動作をさせて画像を撮像するため、１回の撮像で得られる信号量は非常に微弱であるため、強度変調光４の光源２の出力タイミングと撮像ゲイン変調タイミングを同期させ、繰り返し撮像し、イメージインテンシファイア４１の出力蛍光面４５の画像を画像伝達光学系４６でＣＣＤカメラ４２に入力し、ＣＣＤカメラ４２の蓄積効果で十分な感度を確保することができる。
【００６９】
画像伝達光学系４６はファイバプレートもしくはリレーレンズ光学系で実現できる。画像伝達光学系４６をファイバプレートとし、イメージインテンシファイアの蛍光面４５とＣＣＤカメラ４２のＣＣＤ面を連結するごとで、高い効率で蛍光面出力光をＣＣＤカメラ４２に入力できる。
【００７０】
一方、リレーレンズの場合、フォーカス調整や画像位置の微調整機構を持たせることができる。リレーレンズの透過率や画像の結像収差特性は、イメージインテンシファイアの蛍光面４５の蛍光波長で最適化する。リレーレンズを用いた場合、ファイバプレートを用いた場合より光の利用効率が下がるが、ＣＣＤカメラ４２が交換でき、ＣＣＤカメラ４２の選択の幅が広がるとともに、結像倍率の制御が可能であると共に、ファイバプレート使用時に発生する固定パターンノイズの影響がなく、良好な画像が得られる特徴がある。画像の結像倍率は、カラー画像の画角とＣＣＤカメラ４２で撮像する画角が同じになる倍率とする。
【００７１】
イメージインテンシファイア４１の撮像の解像度は、イメージインテンシファイア４１を構成する光電変換面４３とＭＣＰ４４の間隔、ＭＣＰ４４と蛍光面４５間の間隔、ＭＣＰ４４を構成する導電性のガラスキャピラリの直径に大きく依存する。そのため、光電変換面４３とＭＣＰ４４の間隔、ＭＣＰ４４と蛍光面４５間の間隔を近接させると共に、ＭＣＰを構成する導電性のガラスキャピラリの直径を細かいものを使用すると高解像度に有利である。
【００７２】
また、イメーイジンテシファイア４１ヘの結像入力画像を拡大入力し、画像伝達光学系４６で縮小しＣＤＤカメラ４２に入力することで解像度を高く保つことができる。ＣＣＤカメラ４２としては、標準ＴＶ用カメラやハイビジョンカメラが使用できるとともに、その他産業用の高精細ＣＣＤカメラも使用できる。
【００７３】
図１３は、信号処理部１０の一実施例のブロック図を示す。同図中、高速撮像素子９の出力信号は画角変換部４７に供給され、所望の出力画像のアスペクト比に変換される。例えば、１３００（Ｈ）×１０３０（Ｖ）のＣＣＤを高速撮像素子９に使用した場合、ハイビジョン映像の距離画像を出力するときには、１画面の１６：９の領域（例えば１２８０（Ｈ）×８５３（Ｖ））の信号を画角変換部４７で抽出する。
【００７４】
次に、光強度及び撮像パターンの異なるタイミングで撮像した信号を、外部同期４８に同期したスイッチ４９で切り換えて、それぞれメモリ５０Ａとメモリ５０Ｂに記憶する。メモリ５０Ａ，５０Ｂの記憶タイミングは外部同期４８に同期して行われる。これにより、例えば光強度が増加する強度変調光の照射時に短時間撮像した被写体画像がメモリ５０Ａに記憶され、光強度が減少する強度変調光の照射時に短時間撮像した被写体画像がメモリ５０Ａに記憶される。
【００７５】
メモリ５０Ａ，５０Ｂから読み出された信号は信号調整回路５１Ａ，５１Ｂにて画像の輝度調整や、画像全体にある一定の輝度レベルを付加するためのセットアップ調整を行われたのち、距離算出演算部５３で（４）式、または（９）式、または（１３）式の演算を行って距離ｄを算出し、距離を明暗で表す距離映像信号を得る。その後、距離映像信号は信号変換部５４により所望のテレビジョン信号規格に変換されて出力される。
【００７６】
図１４は、本発明の３次元情報検出装置の第２実施例の構成図を示す。同図中、カメラレンズ７の周囲に配置されたＬＥＤ（発光ダイオード）アレイ５５より近赤外波長（中心波長８５０ｎｍ）の強度変調光を被写体１に照射する。また、蛍光灯５６より可視光を被写体１に照射する。
【００７７】
被写体１からの反射光の可視光成分は、カメラレンズ７の後に配置されたダイクロイックプリズム３６を透過し、リレーレンズ４０を通してカラーカメラ１２に結像される。
【００７８】
一方、ＬＥＤアレイ５５からの近赤外光は、ダイクロイックプリズム３６で反射され、高速撮像素子９側に入射される。高速撮像素子９では、高速シャッタ機能をもつイメージインテンシファイア４１により短時間シャッタ撮影を行い、イメージインテンシファイア４１出力の光画像をリレーレンズ４６でＣＣＤカメラ４２へ入力する。
【００７９】
ＬＥＤアレイ５５に供給する変調信号とイメージインテンシファイア４１のシャッタトリガ信号は信号発生器５７で同期して発生する。これにより、増加変調光照射時と減少変調光照射時の画像を交互に撮影する。ＣＣＤカメラ４２の出力信号を信号処理部１０に供給して距離算出を行い、距離を明暗で表す距離映像信号１１をハイビジョン信号出力する。上記ＬＥＤアレイ５５，イメージインテンシファイア４１，ＣＣＤカメラ４２，カラーカメラ１２，カメラレンズ７それぞれの仕様を図１５に示す。
【００８０】
また、図１６に、カラーカメラ１２側への光透過率と、距離検出側（高速撮像素子９側）への光透過率、及び蛍光灯５６出力の可視光とＬＥＤアレイ５５出力の近赤外光の出力スペクトルを示す。ダイクロイックプリズム３６により、可視光成分と距離検出用の近赤外光成分が、高効率で分割されている。距離検出側への近赤外光の透過率が高いため、距離検出に十分な感度があり、Ｓ／Ｎのよい良好な距離検出が可能となっている。
【００８１】
白紙を被写体１とし、カメラレンズ７からの距離を変化させ、距離映像信号１１のレベルを測定した。このときの強度変調光の変調周波数は１５ＭＨｚ、撮像時間幅は５ｎｓｅｃ（半値幅）、測定中心位置を６ｍに設定した。その結果、図１７に示すように、カメラレンズ７から被写体１までの距離とともに、距離映像信号レベルが変化し、距離に応じた明暗映像が得られる。また、測定中心の６ｍ付近では、比較的リニアな特性を有しているが、測定範囲の周辺部で非線型性が生じている。これはＬＥＤの光変調特性の非線型成分の影響であり、出力信号のγ補正によりリニアな特性に補正できる。
【００８２】
本実施例における距離検出範囲は１５ＭＨｚで約４ｍであった。被写体１の奥行きがより小さい場合、変調周波数を４５ＭＨｚと高くすることで奥行き検出範囲は約１．５ｍとなり、相対的に奥行き検出分解能を高めることができる。
【００８３】
また、距離映像信号のノイズ成分量より、距離検出分解能を評価した。測定は強度変調光４の周波数を４５ＭＨｚ、撮像時間幅を２ｎｓとして、カメラレンズ７から被写体１までの距離を１ｍ〜１０ｍまで変化させ、距離映像信号のノイズ成分の実効値σを測定し、３×σに相当する距離の値を距離検出分解能とした場合の距離と距離検出分解能の関係を図１８に示す。被写体１がカメラレンズ７より２ｍの位置にあるとき、距離映像信号のＳ／Ｎは４４．４ｄＢであり、距離検出分解能は１．７ｃｍであった。
【００８４】
本発明では、カラー画像とともに、高速かつ高精細な距離画像を取得できる３次元情報検出方法及び装置を示した。撮影光軸と光照射光軸を近接もしくは一致させることで、光照射の影を低減し撮影画角全体の距離を検出することが可能となる。更に、強度変調光または撮像ゲインの変調特性の組合せにより、簡易な演算処理で高速に距離を算出でき、動画像に対しても距離を算出できることが可能となった。
【００８５】
また、カメラレンズのあとに色分離結像光学系を導入し、カラー画像と距離画像を同じ画角で、標準のＴＶ画像やハイビジョンクラスの高精細な画像の距離画像をビデオレートで撮影できるとともに、カメラレンズのズーム機能にも対応した３次元情報の検出が可能となる。
【００８６】
本発明装置は、被写体のカラー画像と共に奥行き距離情報が取得できる３次元カメラとして各分野に応用できる。その一例として、テレビジョン放送用カメラとしては、距離情報を基にした画像の抽出や合成など、また、立体テレビジョン用のカメラとしても適用できる。更に、３次元モデリング、医療診断、自動車や航空、宇宙、船舶分野におけるエンジン等の機器内の観察や、ガス管、排気口、水道管、ボイラー、タービン内の検査、古墳や遺跡内の調査や動物生態の観察などの分野に適用できる。
【００８７】
なお、第１光源２，光照射光学系３が請求項記載の第１光照射手段に対応し、第２光源６が第２光照射手段に対応し、色分離結像光学系８が光分離手段に対応し、高速撮像素子９がゲイン可変撮像手段に対応し、カラーカメラ１２が撮影手段に対応し、信号処理部１０が信号処理手段に対応し、ハーフミラー１７，偏光ビームスプリッタ１９が光軸一致手段に対応する。
【００８８】
【発明の効果】
上述の如く、本発明によれば、可視領域外の光照射で発生する影を小さくして、被写体において距離検出ができない部分を縮小し、高精細な画像の距離映像をビデオレートで取得でき、カメラレンズのズーム機能に対応したカラー映像と距離映像を取得することができる。
【図面の簡単な説明】
【図１】本発明の３次元情報検出装置の第１実施例の構成図である。
【図２】第１光源及び光照射光学系の配置の一実施例を示す図である。
【図３】カメラレンズの両脇に光源を配置した実施例を示す図である。
【図４】カメラレンズの両脇に光源を配置した実施例を示す図である。
【図５】撮影光軸と照射光軸を一致させる方法を示す図である。
【図６】撮影光軸と照射光軸を一致させる方法を示す図である。
【図７】光照射光学系の一実施例を示す図である。
【図８】パルス状の撮像ゲインと光強度が増加および減少する強度変調光による距離検出方法を説明するための図である。
【図９】パルス状の強度変調光と光強度が増加および減少する撮像ゲインによる距離検出方法を説明するための図である。
【図１０】パルス状の強度変調光とパルス状の撮像ゲインによる距離検出方法を説明するための図である。
【図１１】色分離結像光学系の一実施例を示す図である。
【図１２】高速撮像素子としてイメージインテンシファイアとＣＣＤカメラを用いた実施例を示す図である。
【図１３】信号処理部の一実施例のブロック図である。
【図１４】本発明の３次元情報検出装置の第２実施例の構成図である。
【図１５】ＬＥＤアレイ，イメージインテンシファイア，ＣＣＤカメラ，カラーカメラ，カメラレンズの仕様を示す図である。
【図１６】透過率と、光源光出力スペクトルを示す図である。
【図１７】距離と距離映像信号レベルの関係を示す図である。
【図１８】距離と距離検出分解能の関係を示す図である。
【符号の説明】
１ 被写体
２ 第１光源
３ 光照射光学系
４ 強度変調光
５ 可視光
６ 第２光源
７ カメラレンズ
８ 色分離結像光学系
９ 高速撮像素子
１０ 信号処理部
１１ 距離映像信号
１２ カラーカメラ
１３ カラー映像信号
１４ 撮影光軸
１５ 光照射光軸
１６ 影
１７ ハーフミラー
１８ 遮光体
１９ 偏光ビームスプリッタ
２０ λ／４板
２１ レンズ
２２ 拡散板
２３，２６ 強度変調光
２４，２７，２９，３３ 反射光
２５ パルス状撮像ゲイン
２８ パルス光
３０，３１，３４，３５ 撮像ゲイン
３２ 強度変調光
３６ ダイクロイックプリズム
３７ 可視光成分を含む光
３８ 可視領域外の光
３９ 光学フィルタ
４０ リレーレンズ
４１ イメージインテンシファイア
４２ ＣＣＤカメラ
４３ 光電変換面
４４ マイクロチャンネルプレート
４５ 蛍光面
４６ 画像伝達光学系
４７ 画角変換部
４８ 外部同期
４９ スイッチ
５０Ａ，５０Ｂ メモリ
５１Ａ，５１Ｂ 信号調整回路
５３ 距離算出演算部
５４ 信号変換部
５５ ＬＥＤアレイ
５６ 蛍光灯
５７ 信号発生器[0001]
BACKGROUND OF THE INVENTION
The present invention provides a three-dimensional information detection system. System In particular, a three-dimensional information detection system for detecting the depth shape of a subject and acquiring three-dimensional information. System To do.
[0002]
[Prior art]
As a method for detecting the three-dimensional position or depth distance of an object, there is a stereo method for acquiring three-dimensional information from a multi-viewpoint image obtained using a plurality of cameras. As a method of irradiating light and detecting a distance, there is a method of calculating the distance by driving the irradiation light in a sine wave and detecting the phase on the reflection side.
[0003]
Japanese Patent Application No. 10-293817 describes that a distance can be calculated at a higher speed than two images taken by selecting two types of combinations of temporal changes in irradiation light and gain modulation on the imaging side. ing.
[0004]
[Problems to be solved by the invention]
In the method of acquiring three-dimensional information by the stereo method, since there is no need to irradiate light, there is a merit that it is not affected by external light, but the arrangement of a plurality of cameras and the calibration of camera lenses are complicated. Moreover, the apparatus is large and lacks versatility. Furthermore, it is not possible to freely change the shooting angle of view by zooming the camera lens. Furthermore, in a high-definition image that is higher than that of a standard television image, calculating the distance for each pixel requires time for image matching processing for searching for corresponding points between a plurality of images, so the video rate (frame rate 60 Hz) There is a problem that it is difficult to detect multiple pixels of about 400,000 or more pixels at a high speed.
[0005]
In addition, in the method of irradiating a subject with sinusoidal light and detecting the phase on the reflection side, it is necessary to calculate the phase of the light for each shooting pixel, and the subject is generally used with a range finder at a practical level. In the case of detecting the entire distance, a two-dimensional scanning mechanism for the light beam is required, so that there is a problem that the high speed is impaired and the configuration is complicated.
[0006]
Japanese Patent Application No. 10-293817 discloses a method for acquiring a distance image (an image in which distance information is represented by luminance of an image signal) at a high speed. However, a distance image having the same angle of view as a color image is shown. Taking into account the specific method of shooting color images and distance video in correspondence with the zoom function of the camera lens, enabling high-quality video shooting of high-definition video class without simultaneous occlusion for distance detection. There was a problem that was not.
[0007]
The present invention has been made in view of the above points, and can obtain a distance image of a high-definition image at a video rate, and can acquire a color image and a distance image corresponding to a zoom function of a camera lens. Detection system System The purpose is to provide.
[0008]
[Means for Solving the Problems]
The invention as set forth in claim 1 is a With central wavelength Irradiate the subject with modulated light intensity Outside the visible range A light irradiating means, a camera lens for condensing the reflected light from the subject, a light separating means for wavelength-separating the light collected by the camera lens into light outside the visible region and light in the visible region, and imaging A gain variable imaging unit that modulates the gain and captures an optical image outside the visible region wavelength-separated by the light separating unit, and an optical image in the visible region separated by the light separating unit is captured and a video signal is output. A three-dimensional information detection apparatus having a photographing means and a signal processing means for calculating a distance from an intensity ratio between a plurality of images picked up by the variable gain image pickup means and outputting a distance video signal expressing the distance in light and dark;
Irradiate the subject with visible light Visible A three-dimensional information detection system comprising a light irradiation device,
The three-dimensional information detection apparatus includes:
Above Outside the visible range The light irradiation means increases and decreases the light intensity of the output light with time, the gain variable imaging means sets the imaging gain to a constant value in a short time in a pulsed manner, and the signal processing means performs imaging while increasing the light intensity with time. Calculating the distance from the intensity ratio between the captured image and the image captured while decreasing the light intensity with time,
Or the above Outside the visible range The light irradiation means outputs pulse light, the variable gain imaging means increases and decreases the imaging gain with time, and the signal processing means decreases the captured image and the imaging gain with time while increasing the imaging gain with time. While calculating the distance from the intensity ratio between the captured image and
Or the above Outside the visible range The light irradiation means modulates and outputs the light intensity of the output light in a rectangular wave shape, the variable gain imaging means modulates the imaging gain in a rectangular wave shape with the same period as the light intensity, and the signal processing means While calculating the distance from the intensity ratio between the image captured when the imaging gain is in phase and the light intensity and the image captured when the imaging gain is in reverse phase,
Above Outside the visible range The light irradiating means is a three-dimensional information detecting device in which a plurality of light irradiation means are arranged around the camera lens so that the optical axis of the light outside the visible region is close to the optical axis of the camera lens.
Color images corresponding to the zoom function of the camera lens can be obtained by reducing the shadow that occurs due to light irradiation outside the visible region, reducing the area where distance detection is not possible in the subject, and obtaining distance video of high-definition images at the video rate. And distance video can be acquired.
[0011]
In the three-dimensional information detection apparatus,
Above Outside the visible range Instead of arranging a plurality of light irradiation means around the camera lens, Outside the visible range An optical axis matching means for matching the optical axis of light outside the visible region irradiated by the light irradiation means with the optical axis of the camera lens can be provided.
[0012]
DETAILED DESCRIPTION OF THE INVENTION
FIG. 1 shows a configuration diagram of a first embodiment of the three-dimensional information detection apparatus of the present invention. In the drawing, the output light of the first light source 2 is irradiated through the light irradiation optical system 3 to the entire shooting range including the subject 1 to be shot. At this time, the irradiation light is intensity-modulated light 4 whose light intensity changes in the time axis direction. As the first light source 2, one or a plurality of laser diodes or light emitting diodes can be used.
[0013]
Further, the wavelength of the output light of the first light source 2 can be a wavelength other than the wavelength in the visible region necessary for color image photography, for example, the wavelength in the ultraviolet region or the infrared region can be used. In particular, near-infrared light of 750 nm to 900 nm has a wavelength close to the visible region and optical characteristics are not significantly different from the visible region. Therefore, a normal camera lens for photographing a color image can be used. Also in the optical system 8, wavelength separation from the visible region can be performed with high efficiency.
It is easy to produce an optical system that ensures an optically high color separation imaging optical system ratio.
[0014]
As a means for generating the intensity modulated light 4, the drive current of the semiconductor laser or the light emitting diode may be directly modulated, and the light source output light may be modulated through an external modulator such as an acoustooptic device. As the light irradiation optical system 3, for example, a minute lens is used on the output end face, and a lens shape corresponding to the range of the subject to be irradiated is employed. In order to ensure the uniformity of the illumination intensity, the optical fiber bundle is used, a microlens is placed on each optical fiber, and the individual illumination patterns are superimposed on the illumination surface to increase the uniformity of the illumination light intensity. be able to.
[0015]
Further, a diffusion plate can be used as the light irradiation optical system 3. As the diffusion plate, an element that exhibits a ground glass-like scattering effect, or an element that forms fine irregularities on a diffraction grating on the surface and controls light distribution by the diffraction effect can be used. In particular, with respect to a diffusion plate using surface diffraction, the light distribution shape and intensity distribution can be controlled by the surface irregularity formation pattern, so that uniform illumination can be realized efficiently according to the photographing aspect ratio. In addition, the use of a diffuser plate provides the same conditions as a surface light source, and reduces the burden on the eyes when photographing a person compared to a point light source, thereby ensuring safety.
[0016]
The subject 1 is illuminated with a second light source 6 that emits visible light 5. The second light source 6 does not need to be in the vicinity of the camera lens 7. When near-infrared light is used for the first light source 2, fluorescent lamp illumination containing almost no near-infrared component can be used as the second light source 6.
[0017]
The reflected light from the subject 1 illuminated by the first light source 2 and the second light source 6 is collected by the camera lens 7. As a characteristic of the camera lens, a camera lens with a zoom function for photographing a normal color image can be used. In addition, by applying a surface coating according to the wavelength of the first light source, the light from the first light source 2 can be taken in efficiently, and the occurrence of ghost in the lens can be suppressed. It is advantageous for acquisition.
[0018]
A color separation imaging optical system 8 that separates colors of the light wavelength component from the first light source 2 and the light wavelength component from the second light source and forms an image respectively is disposed behind the camera lens 7. The formed image of the light component from the first light source 2 is incident on the high-speed imaging device 9 that can change the imaging gain at high speed.
[0019]
The high-speed imaging device 9 shoots a plurality of images by modulating the imaging gain under the condition that the time change of the intensity modulated light of the first light source 2 is different. The signal processing unit 10 performs signal processing for calculating the distance from the camera to the subject 1 based on the obtained signals of the plurality of images, and outputs a distance video signal 11 that represents the depth distance of the subject 1 by the density of the image. To do.
[0020]
On the other hand, the formed image of the light component from the second light source 6 separated by the color separation imaging optical system 8 is incident on the color camera 12, and the color camera 12 outputs a color video signal 13.
[0021]
FIG. 2 shows an example of the arrangement of the first light source 2 and the light irradiation optical system 3. The first light source 2 is installed in the vicinity of the camera lens 7, and the shadow of the illumination light is reduced by arranging the photographing optical axis 14 of the camera lens 7 and the light irradiation optical axis 15 of the light source 2 close to each other. As shown in FIG. 2, when the subject 1 located on the optical axis of the camera lens 7 at a distance d from the camera lens 7 has a step of depth Δd, the distance 1 from the photographing optical axis 14 of the camera lens 7 is as follows. The light from the light irradiation optical system 3 at a distant position causes a shadow 16 on a part of the subject, and its width Δl is expressed by the equation (1).
[0022]
Δl = l · Δd / d (1)
As is clear from the equation (1), the shadow region can be reduced as the distance l between the photographing optical axis 14 of the camera lens 7 and the light irradiation optical axis 15 of the light source 2 is reduced.
[0023]
FIG. 3 shows an embodiment in which light sources are arranged on both sides of the camera lens 7. By arranging the first light sources 2 and 2 ′ and the light irradiation optical systems 3 and 3 ′ on both sides of the camera lens 7, the shadow 16 portion of the illumination by the first light source 2 on one side is replaced with the other light source 2 ′. Can be irradiated. In this way, by placing the optical axis 14 of the camera lens 7 and the irradiation optical axes 15 and 15 ′ close to each other and arranging the camera lens 7 so as to be sandwiched between the left and right sides and the upper and lower sides thereof, an illumination shadow 16 that becomes an occlusion for distance detection. Can be reduced.
[0024]
In addition, in FIG. 3, the portion that becomes the shadow 16 of illumination by one of the light sources 2 is illuminated by the other light source 2 ′, and the light intensity is about half that of the portion illuminated by both light sources. Unevenness occurs.
[0025]
Further, as shown in FIG. 4, there are projections 1a and 1b at positions sandwiching the optical axis 14 of the camera lens 7 of the subject 1, and the projections 1a and 1b prevent illumination by the light sources 2 and 2 ′. An illumination shadow 16 'is produced. In these cases, by making the optical axis 14 of the camera lens 7 coincide with the irradiation optical axes 15 and 15 'of the illumination light, it is possible to eliminate the occurrence of these unevenness in intensity and the shadow that becomes an occlusion for distance detection.
[0026]
FIG. 5 shows a method of matching the photographing optical axis with the irradiation optical axis. In the figure, a half mirror 17 is disposed at an angle of 45 degrees with respect to the photographing optical axis 14 of the camera lens 7 on the front surface of the camera lens 7, and the first light source 2 and the light irradiation optical system 3 from a direction orthogonal to the photographing optical axis 14. Illuminates via the half mirror 17. The intensity-modulated light 4 output from the light irradiation optical system 3 is reflected 90 degrees by the half mirror 17 and illuminates the subject 1 with an optical axis 15 that coincides with the photographing optical axis 14 of the camera lens 7.
[0027]
As a result, it is possible to eliminate the illumination shadows 16 and 16 'and the light intensity unevenness. Of the intensity-modulated light 4, the light that is transmitted without being reflected by the half mirror 17 is absorbed by the light shielding body 18 having a characteristic of absorbing light of this wavelength.
[0028]
Further, instead of the half mirror 17, a polarization beam splitter 19 can be used as shown in FIG. In this case, the polarization direction of the intensity-modulated light 4 is aligned with the direction reflected by the polarization beam splitter 19. In the case of non-polarized light, a polarization conversion element for aligning the polarization direction may be arranged immediately after the light irradiation optical system 3 to align the polarization direction. Most of the intensity-modulated light 4 is reflected by 90 degrees by the polarization beam splitter 19.
[0029]
The output light passes through the λ / 4 plate 20 and becomes circularly polarized light, and is irradiated onto the subject 1. The reflected component close to the circularly polarized light of the reflected light component is incident on the λ / 4 plate 20 again to become linearly polarized light, passes through the polarizing beam splitter 19 and enters the camera lens 7.
[0030]
On the other hand, since a part of the output light from the first light source 2 passes through the polarization beam splitter 19, a light shielding plate 18 having a large absorption with respect to the light of this wavelength component is installed to suppress reflection toward the camera lens. Further, instead of the light shielding plate 18, a mirror for preserving polarization can be disposed and reflected again to the light irradiation optical system 3 side. As described above, the use efficiency of the light source output light can be increased by using the polarization beam splitter 19.
[0031]
By the way, the irradiation angle of the irradiation light is controlled by the light irradiation optical system 3 attached in front of the first light source 2. FIG. 7 shows an embodiment of the light irradiation optical system 3. In the figure, the first light source 2 is composed of a plurality of light sources, and one or a plurality of lenses 21 are attached to the output ends of the individual light sources, and the light is spread to the maximum angle of view taken by the camera lens 7 to be used. Irradiate.
[0032]
At this time, the light irradiation optical system 3 is configured by vertically and horizontally placing lenses having different divergence angles in the longitudinal and lateral directions, for example, bifocal lenses and cylindrical lenses having different focal lengths, and the angle of view of the camera image. The light utilization efficiency can be increased by irradiating light having a shape according to the above.
[0033]
For example, when using a standard television camera, the aspect ratio is 4: 3, and for high-definition shooting, it is 16: 9. Further, a diffusion plate 22 can be used for the light irradiation optical system 3. As the diffusion plate, an element that exhibits a ground glass-like scattering effect, or an element that forms fine irregularities on a diffraction grating on the surface and controls light distribution by the diffraction effect can be used.
[0034]
In particular, with respect to a diffusion plate using surface diffraction, the light distribution shape and intensity distribution can be controlled by the surface irregularity formation pattern, so that efficient and uniform illumination can be realized according to the shooting angle of view.
[0035]
Here, a description will be given of a distance detection method using an imaging device having light intensity that increases and decreases with time as intensity-modulated light and a short-time shutter function. As shown in FIG. 8A, the subject placed at the distance d is irradiated with intensity-modulated light 23 whose light intensity increases with a factor s as time passes, and the reflected light 24 from the subject is imaged in a pulsed manner at time ts. When the image is captured for a short time with a gain of 25, the signal amount E detected by the high speed image sensor 9 ₊ (D, ts) is expressed by equation (2).
[0036]
[Expression 1]

Where T _L Is the transmittance of the lens optical system such as the camera lens 7, ρ is the reflection characteristic coefficient of the surface of the subject 1, and F ₀ Is the maximum light irradiation intensity, and Δt is the imaging time width, which is a sufficiently small value with respect to the light modulation period. Also, c is the speed of light, 2d / c is the time for light to reciprocate the distance d from the high-speed image sensor 9 to the subject 1, l is the distance from the camera lens 7 to the subject 1, and the denominator of the equation is due to light diffusion This is a term considering attenuation.
[0037]
Next, as shown in FIG. 8B, the intensity-modulated light 26 whose light intensity decreases with the coefficient s with time is irradiated, and the reflected light 27 from the subject 1 is briefly applied with the pulsed imaging gain 25 at time ts. When imaged, the signal amount E detected by the high-speed image sensor 9 ₋ (D, ts) is expressed by equation (3).
[0038]
[Expression 2]

Here, T is a modulation cycle of light intensity. If the sensitivity is insufficient for one imaging, a pulsed light equivalent to the modulation frequency of the imaging gain is irradiated in one field and accumulated by a storage type imaging device to ensure sufficient sensitivity. . Note that the intensity-modulated light in FIG. 8B may be transmitted continuously with the intensity-modulated light in FIG.
[0039]
From the equations (2) and (3), the intensity ratio R = E between two images having different light intensities. ₊ / E ₋ And the distance d is obtained as equation (4).
[0040]
[Equation 3]

As shown by the equation (4), only by calculating the ratio R between the two images taken when the light intensity is increased and when the light intensity is decreased, the reflectance of the subject 1, the light attenuation effect due to light diffusion, etc. The influence can be canceled and the distance can be obtained at high speed.
[0041]
In this way, a light source whose intensity changes with time can be realized even at a low frequency of several tens of MHz. Therefore, in addition to a laser light source, incoherent illumination such as an LED (light emitting diode) can be used, and a wide range can be illuminated with high brightness. It also becomes possible to detect the distance of a person. Note that the distance detection range by this method is determined by the imaging time ts and the modulation frequency f, and is expressed by equation (5).
[0042]
[Expression 4]

In actual operation, the imaging time ts is adjusted to adjust the detection range to the position of the subject. Measurement range d _r Is determined by the modulation frequency f and is expressed by equation (6).
[0043]
d _r = C / 4f (6)
Measurement range d _r Is 7.5 m, the modulation frequency f is 10 MHz, and the measurement range d _r Is 1.5 m, the modulation frequency f is 50 MHz. On the other hand, the imaging time width Δt at the time of imaging is shorter than the modulation cycle 1 / f of the light source 2, and a shutter function of several nsec is required.
[0044]
Next, a short-time pulse light is used as intensity-modulated light, and an image captured while increasing the imaging gain of the high-speed image sensor 9 with time and an image captured while decreasing the imaging gain with time are increased. A method for calculating the distance and detecting the distance will be described.
[0045]
As shown in FIG. 9A, the light intensity F is applied to the subject 1 at the position of the distance d. ₀ The amount of signal E detected by the camera when the pulsed light 28 is irradiated at time tp and the reflected light 29 from the subject 1 is imaged by the high-speed imaging device 9 having the imaging gain 30 that increases with time by the coefficient g. ₊ (D, tp) is expressed by equation (7).
[0046]
[Equation 5]

Where T _L Is the transmittance of the lens optical system such as the camera lens 7, ρ is the reflection characteristic coefficient of the surface of the subject 1, τ is the pulse width, c is the speed of light, 2d / c is the distance d from the high-speed image sensor 9 to the subject 1 Is the distance from the camera lens 7 to the subject 1, and the denominator of the equation is a term considering attenuation due to light diffusion.
[0047]
Next, as shown in FIG. 9B, the signal amount E detected by the camera when imaged by the high-speed image sensor 9 having the imaging gain 31 that decreases with the coefficient g with time. ₋ (D, tp) is expressed by equation (8).
[0048]
[Formula 6]

Here, T is an imaging gain modulation period. If the sensitivity is insufficient for one imaging, a pulsed light equivalent to the modulation frequency of the imaging gain is irradiated in one field and accumulated by a storage type imaging device to ensure sufficient sensitivity. . Note that the imaging gain modulation in FIG. 9B may be set to be continuous with the imaging gain modulation in FIG.
[0049]
From the equations (7) and (8), the intensity ratio R = E between two images having different imaging gains. ₊ / E ₋ And the distance d is obtained as equation (9).
[0050]
[Expression 7]

As shown in the above equation (9), only by calculating the intensity ratio R between two images taken when the imaging gain is increased and when the imaging gain is reduced, the light attenuation effect due to the reflectance of the subject 1 and the diffusion of light. It is possible to cancel the influence of the above and obtain the distance at high speed.
[0051]
In this way, a pulse laser can be used with a combination of pulsed light and imaging gain that changes with time, and pulse laser light having a pulse width of picoseconds to femtoseconds, such as Q-switched pulsed laser light and mode-locked pulsed light. Therefore, distance detection with high resolution becomes possible.
[0052]
In this case, the measurable range (measurement range) D is expressed by equation (10).
[0053]
[Equation 8]

Next, a distance detection method using the intensity-modulated light modulated in a rectangular wave shape and the high-speed imaging element 9 in which the imaging gain changes in a rectangular wave shape will be described. A rectangular intensity modulated light 32 (light intensity F as shown by a broken line in FIG. 10A) is applied to the subject 1 located at a distance d. ₀ , Modulation period T), and when the reflected light 33 is imaged with a rectangular imaging gain 34 (gain g, period T) having the same period and the same phase as the intensity-modulated light 32, the signal amount detected by the camera E ₊ (D, tp) is expressed by equation (11).
[0054]
[Equation 9]

Where T _L Is the transmittance of the lens optical system such as the camera lens 7, ρ is the reflection characteristic coefficient of the surface of the subject 1, c is the speed of light, 2 d / c is the time for light to reciprocate the distance d from the high-speed image sensor 9 to the subject 1 , L is the distance from the camera lens 7 to the subject 1, and the denominator of the equation is a term that takes into account attenuation due to light diffusion.
[0055]
Next, as shown in FIG. 10B, in the case of imaging with a rectangular imaging gain 35 (gain g, period T) having an opposite phase to the intensity-modulated light 32 shifted by the period T, a signal detected by the camera Amount E ₋ (D, tp) is expressed by equation (12).
[0056]
[Expression 10]

If the sensitivity is insufficient for one imaging, a pulsed light equivalent to the modulation frequency of the imaging gain is irradiated in one field and accumulated by a storage type imaging device to ensure sufficient sensitivity. .
[0057]
From the equations (11) and (12), the intensity ratio R = E between the two images. ₊ / E ₋ And the distance d is obtained as equation (13).
[0058]
## EQU11 ##

As shown in the equation (13), by simply calculating the ratio R between two images, the influence of the reflectance of the subject 1 and the light attenuation effect due to the diffusion of light can be canceled and the distance can be obtained at high speed. Can do.
[0059]
In this way, by making the intensity-modulated irradiation light and the imaging gain into a rectangular wave shape having the same period width, the amount of detection signal detected by the imaging device increases, and imaging with a high signal-to-noise ratio is possible. High-resolution distance detection is possible.
[0060]
FIG. 11 shows an embodiment of the color separation imaging optical system 8. In the figure, light 37 including a visible light component from the second light source 6 used for color image photography after the camera lens 7 is transmitted, and light 38 outside the visible region from the first light source 2 used for distance detection. The dichroic prism 36 is reflected. The size and material of the dichroic prism 36 are selected in consideration of the back focus length of the camera lens 7 so as to have an optical path length capable of forming an image on the high-speed imaging device 9 from the camera lens 7 through the dichroic prism 36.
[0061]
When the output light of the first light source 2 that is an intensity-modulated light source for distance detection is near-infrared light, the color separation characteristic by the dichroic prism 36 is a characteristic that transmits visible light and reflects near-infrared light. . If the wavelength of the second light source 6 is selected in the range of 780 nm to 900 nm, the visible light transmittance can be increased and the near infrared light reflectance can be increased.
[0062]
Further, by arranging an infrared transmission optical filter 39 that removes light having a longer wavelength than the wavelength component of the first light source 2 included in the light component reflected by the dichroic prism 36, the influence of external light is reduced, and the signal The characteristic that only the near-infrared light which is light can be input to the high-speed image sensor 9 is obtained.
[0063]
The visible light that has passed through the dichroic prism 36 is input to the color camera 12 through the relay lens 40. The magnification of the relay lens 40 is approximately equal, and the lens has a focus adjustment function. At this time, focus adjustment is facilitated by using a telecentric optical system as the configuration of the relay lens 40.
[0064]
The relay lens 40 optimizes image formation optical characteristics such as transmittance and aberration in the visible region, suppresses a decrease in MTF (Modulation Transfer Function), and forms a high-definition image of the image formed by the camera lens. Is transmitted to the color camera 12 without any color change or luminance reduction. Moreover, the reduction in brightness is suppressed by making the numerical aperture of the relay lens 40 equal to that of the camera lens 7. Further, by providing the relay lens 40 with an aperture adjustment mechanism, it is possible to adjust the aperture of a color image without affecting the near-infrared light quantity toward the distance detection side.
[0065]
FIG. 12 shows an embodiment in which an image intensifier 41 and a CCD camera 42 are used as the high-speed image sensor 9. In the figure, the light component of the first light source 2 reflected by the dichroic prism 36 is imaged and input to the image intensifier 41. As the image intensifier 41, an image intensifier 41 including a photoelectric conversion surface 43, a microchannel plate (MCP) 44, and a phosphor screen 45 can be used.
[0066]
By controlling the value of the applied voltage between the photoelectric conversion surface 43 of the image intensifier 41 and the MCP 44, the imaging sensitivity can be changed at high speed with time. In addition, by applying a pulse shape to the applied voltage, it is possible to perform gate imaging for a short time of 1 nsec or less.
[0067]
The material of the photoelectric conversion surface 43 is selected according to the wavelength sensitivity characteristic of intensity modulated light used for distance detection. For example, when the wavelength is 800 nm, a multi-alkali material (Sb—K—Na—Cs) can be used, and when a long wavelength of 850 nm is used, a high quantum efficiency can be obtained by using a gallium arsenide (GaAs) material. Therefore, a captured image with good S / N can be obtained.
[0068]
Since an image is picked up by changing the image pickup gain at high speed or by performing a short-time gate operation, the signal amount obtained by one image pickup is very weak. Therefore, the output timing of the light source 2 of the intensity modulated light 4 And the imaging gain modulation timing are synchronized, and repeated imaging is performed, and the image on the output fluorescent screen 45 of the image intensifier 41 is input to the CCD camera 42 by the image transmission optical system 46, and sufficient sensitivity is obtained by the accumulation effect of the CCD camera 42. Can be secured.
[0069]
The image transmission optical system 46 can be realized by a fiber plate or a relay lens optical system. Each time the image transmission optical system 46 is a fiber plate and the phosphor screen 45 of the image intensifier and the CCD surface of the CCD camera 42 are connected, the phosphor screen output light can be input to the CCD camera 42 with high efficiency.
[0070]
On the other hand, in the case of a relay lens, a focus adjustment and a fine adjustment mechanism for an image position can be provided. The transmittance of the relay lens and the imaging aberration characteristic of the image are optimized by the fluorescence wavelength of the phosphor screen 45 of the image intensifier. When a relay lens is used, the light use efficiency is lower than when a fiber plate is used. However, the CCD camera 42 can be replaced, the selection range of the CCD camera 42 is widened, and the imaging magnification can be controlled. There is a feature that a good image can be obtained without the influence of the fixed pattern noise generated when the fiber plate is used. The image forming magnification is a magnification at which the angle of view of the color image is the same as the angle of view captured by the CCD camera 42.
[0071]
The imaging resolution of the image intensifier 41 is greatly increased in the distance between the photoelectric conversion surface 43 and the MCP 44 constituting the image intensifier 41, the distance between the MCP 44 and the fluorescent screen 45, and the diameter of the conductive glass capillary constituting the MCP 44. Dependent. For this reason, it is advantageous for high resolution if the distance between the photoelectric conversion surface 43 and the MCP 44 and the distance between the MCP 44 and the fluorescent surface 45 are close to each other, and a conductive glass capillary having a small diameter is used.
[0072]
Further, the resolution can be kept high by enlarging and inputting the image formation input image to the image stabilizer 41, reducing it by the image transmission optical system 46, and inputting it to the CDD camera 42. As the CCD camera 42, a standard TV camera or a high-definition camera can be used, and other industrial high-definition CCD cameras can be used.
[0073]
FIG. 13 shows a block diagram of an embodiment of the signal processing unit 10. In the figure, the output signal of the high-speed image sensor 9 is supplied to the angle-of-view conversion unit 47 and converted into the desired aspect ratio of the output image. For example, when a CCD of 1300 (H) × 1030 (V) is used for the high-speed image sensor 9, a 16: 9 area (for example, 1280 (H) × 853 (for example) of one screen is output when a distance image of a high-definition video is output. The signal of V)) is extracted by the angle-of-view conversion unit 47.
[0074]
Next, signals captured at different timings of the light intensity and the imaging pattern are switched by the switch 49 synchronized with the external synchronization 48 and stored in the memory 50A and the memory 50B, respectively. The storage timing of the memories 50A and 50B is performed in synchronization with the external synchronization 48. Thus, for example, a subject image captured for a short time when irradiated with intensity-modulated light whose light intensity increases is stored in the memory 50A, and a subject image captured for a short time when irradiated with intensity-modulated light whose light intensity decreases is stored in the memory 50A. Is done.
[0075]
The signals read from the memories 50A and 50B are subjected to image brightness adjustment by the signal adjustment circuits 51A and 51B, and setup adjustment for adding a certain brightness level to the entire image, and then a distance calculation calculation unit. 53, the distance d is calculated by calculating the expression (4), the expression (9), or the expression (13) to obtain a distance video signal expressing the distance in light and dark. Thereafter, the distance video signal is converted into a desired television signal standard by the signal converter 54 and output.
[0076]
FIG. 14 shows a configuration diagram of a second embodiment of the three-dimensional information detection apparatus of the present invention. In the figure, the subject 1 is irradiated with intensity-modulated light having a near infrared wavelength (center wavelength 850 nm) from an LED (light emitting diode) array 55 arranged around the camera lens 7. Further, the subject 1 is irradiated with visible light from the fluorescent lamp 56.
[0077]
The visible light component of the reflected light from the subject 1 passes through the dichroic prism 36 disposed after the camera lens 7 and is imaged on the color camera 12 through the relay lens 40.
[0078]
On the other hand, near-infrared light from the LED array 55 is reflected by the dichroic prism 36 and is incident on the high-speed imaging element 9 side. In the high-speed imaging device 9, the image intensifier 41 having a high-speed shutter function performs short-time shutter photography, and the light image output from the image intensifier 41 is input to the CCD camera 42 by the relay lens 46.
[0079]
The modulation signal supplied to the LED array 55 and the shutter trigger signal of the image intensifier 41 are generated in synchronization by the signal generator 57. As a result, images at the time of increasing modulated light irradiation and decreasing modulated light irradiation are taken alternately. An output signal from the CCD camera 42 is supplied to the signal processing unit 10 to calculate the distance, and a distance video signal 11 representing the distance in light and dark is output as a high-definition signal. The specifications of the LED array 55, the image intensifier 41, the CCD camera 42, the color camera 12, and the camera lens 7 are shown in FIG.
[0080]
FIG. 16 shows the light transmittance to the color camera 12 side, the light transmittance to the distance detection side (the high-speed imaging device 9 side), the visible light of the fluorescent lamp 56 output, and the near infrared of the LED array 55 output. The output spectrum of light is shown. The dichroic prism 36 divides the visible light component and the near-infrared light component for distance detection with high efficiency. Since the transmittance of near infrared light to the distance detection side is high, there is sufficient sensitivity for distance detection, and good distance detection with good S / N is possible.
[0081]
A blank paper was the subject 1, the distance from the camera lens 7 was changed, and the level of the distance video signal 11 was measured. At this time, the modulation frequency of the intensity-modulated light was set to 15 MHz, the imaging time width was set to 5 nsec (half-value width), and the measurement center position was set to 6 m. As a result, as shown in FIG. 17, the distance video signal level changes with the distance from the camera lens 7 to the subject 1, and a bright and dark video corresponding to the distance is obtained. Further, in the vicinity of 6 m of the measurement center, it has a relatively linear characteristic, but nonlinearity occurs in the peripheral part of the measurement range. This is an influence of a non-linear component of the light modulation characteristic of the LED, and can be corrected to a linear characteristic by γ correction of the output signal.
[0082]
The distance detection range in this example was about 4 m at 15 MHz. When the depth of the subject 1 is smaller, by increasing the modulation frequency to 45 MHz, the depth detection range becomes about 1.5 m, and the depth detection resolution can be relatively increased.
[0083]
The distance detection resolution was evaluated from the amount of noise components in the distance video signal. In the measurement, the frequency of the intensity modulated light 4 is 45 MHz, the imaging time width is 2 ns, the distance from the camera lens 7 to the subject 1 is changed from 1 m to 10 m, and the effective value σ of the noise component of the distance video signal is measured. FIG. 18 shows the relationship between the distance and the distance detection resolution when the distance value corresponding to xσ is the distance detection resolution. When the subject 1 is at a position 2 m from the camera lens 7, the S / N of the distance video signal is 44.4 dB, and the distance detection resolution is 1.7 cm.
[0084]
In the present invention, a three-dimensional information detection method and apparatus capable of acquiring a high-speed and high-definition distance image together with a color image have been shown. By making the photographing optical axis and the light irradiation optical axis close or coincide with each other, it becomes possible to reduce the shadow of light irradiation and detect the distance of the entire photographing field angle. Furthermore, the combination of intensity-modulated light or modulation characteristics of imaging gain enables the distance to be calculated at high speed with a simple calculation process, and the distance can also be calculated for a moving image.
[0085]
In addition, a color separation imaging optical system is introduced after the camera lens, and color images and distance images can be taken at the same angle of view, and standard TV images and high-definition-class distance images can be taken at video rates. It becomes possible to detect three-dimensional information corresponding to the zoom function of the camera lens.
[0086]
The apparatus of the present invention can be applied to various fields as a three-dimensional camera that can acquire depth distance information together with a color image of a subject. As an example, a television broadcast camera can be applied to extraction and synthesis of images based on distance information, and a stereoscopic television camera. In addition, three-dimensional modeling, medical diagnosis, observation in automobiles, aviation, space, marine engines and other equipment, inspection of gas pipes, exhaust outlets, water pipes, boilers, turbines, investigation of ancient tombs and ruins, Applicable to fields such as observation of animal ecology.
[0087]
The first light source 2 and the light irradiation optical system 3 correspond to the first light irradiation means, the second light source 6 corresponds to the second light irradiation means, and the color separation and imaging optical system 8 performs the light separation. The high-speed imaging device 9 corresponds to the gain variable imaging unit, the color camera 12 corresponds to the imaging unit, the signal processing unit 10 corresponds to the signal processing unit, and the half mirror 17 and the polarization beam splitter 19 are light. Corresponds to the axis matching means.
[0088]
【The invention's effect】
As mentioned above, According to the present invention, the shadow generated by light irradiation outside the visible region is reduced, and the portion where the distance cannot be detected in the subject is reduced. A high-definition distance image can be acquired at a video rate, and a color image and a distance image corresponding to the zoom function of the camera lens can be acquired.
[Brief description of the drawings]
FIG. 1 is a configuration diagram of a first embodiment of a three-dimensional information detection apparatus of the present invention.
FIG. 2 is a diagram illustrating an example of an arrangement of a first light source and a light irradiation optical system.
FIG. 3 is a diagram showing an embodiment in which light sources are arranged on both sides of a camera lens.
FIG. 4 is a diagram showing an embodiment in which light sources are arranged on both sides of a camera lens.
FIG. 5 is a diagram illustrating a method for matching a photographing optical axis with an irradiation optical axis.
FIG. 6 is a diagram illustrating a method for matching a photographing optical axis with an irradiation optical axis.
FIG. 7 is a diagram illustrating an example of a light irradiation optical system.
FIG. 8 is a diagram for explaining a distance detection method using intensity-modulated light in which a pulsed imaging gain and light intensity increase and decrease;
FIG. 9 is a diagram for explaining a distance detection method using pulsed intensity-modulated light and an imaging gain in which the light intensity increases and decreases.
FIG. 10 is a diagram for explaining a distance detection method using pulsed intensity-modulated light and pulsed imaging gain.
FIG. 11 is a diagram illustrating an example of a color separation imaging optical system.
FIG. 12 is a diagram showing an embodiment in which an image intensifier and a CCD camera are used as a high-speed image sensor.
FIG. 13 is a block diagram of an example of a signal processing unit.
FIG. 14 is a configuration diagram of a second embodiment of the three-dimensional information detection apparatus of the present invention.
FIG. 15 is a diagram illustrating specifications of an LED array, an image intensifier, a CCD camera, a color camera, and a camera lens.
FIG. 16 is a diagram showing a transmittance and a light source light output spectrum.
FIG. 17 is a diagram illustrating a relationship between distance and a distance video signal level.
FIG. 18 is a diagram showing the relationship between distance and distance detection resolution;
[Explanation of symbols]
1 Subject
2 First light source
3 Light irradiation optical system
4 Intensity modulated light
5 Visible light
6 Second light source
7 Camera lens
8 color separation imaging optical system
9 High-speed image sensor
10 Signal processor
11 Distance video signal
12 Color camera
13 Color video signal
14 Shooting optical axis
15 Light irradiation optical axis
16 Shadow
17 half mirror
18 Shading body
19 Polarizing beam splitter
20 λ / 4 plate
21 Lens
22 Diffuser
23,26 Intensity modulated light
24, 27, 29, 33 Reflected light
25 Pulsed imaging gain
28 Pulsed light
30, 31, 34, 35 Imaging gain
32 Intensity modulated light
36 Dichroic Prism
37 Light containing visible light components
38 Light outside the visible range
39 Optical filters
40 Relay lens
41 Image Intensifier
42 CCD camera
43 Photoelectric conversion surface
44 microchannel plate
45 phosphor screen
46 Image Transmission Optical System
47 Angle conversion unit
48 External synchronization
49 switch
50A, 50B memory
51A, 51B Signal adjustment circuit
53 Distance calculation unit
54 Signal converter
55 LED array
56 Fluorescent Light
57 Signal generator

Claims (3)

The light having a central wavelength outside the visible region is intensity-modulated to irradiate the subject with the light outside the visible region , the camera lens that collects the reflected light from the subject, and the light collected by the camera lens. Light separating means for wavelength-separating light into light outside the visible region and light in the visible region, and variable gain imaging means for modulating an imaging gain and picking up an optical image outside the visible region wavelength-separated by the light separating means A distance between the image capturing means for capturing an optical image of the visible region separated by the light separating means and outputting a video signal; and an intensity ratio between the plurality of images captured by the variable gain image capturing means; A three-dimensional information detection apparatus having signal processing means for outputting a distance video signal expressed in light and darkness;
A three-dimensional information detection system configured with a visible light irradiation device that irradiates the subject with light in a visible region,
The three-dimensional information detection apparatus includes:
The visible light irradiating means increases and decreases the light intensity of the output light with time, the gain variable imaging means sets the imaging gain to a constant value in a short time in a pulsed manner, and the signal processing means increases the light intensity with time. Calculate the distance from the intensity ratio between the image captured while increasing and the image captured while decreasing the light intensity with time,
Alternatively, the outside visible light irradiation unit outputs pulsed light, the variable gain imaging unit increases and decreases the imaging gain with time, and the signal processing unit captures images and images while increasing the imaging gain with time. Calculate the distance from the intensity ratio between the captured image while decreasing the gain over time,
Alternatively, the light irradiation means outside visible region modulates and outputs the light intensity of the output light in a rectangular wave shape, and the gain variable imaging means modulates the imaging gain in a rectangular wave shape with the same period as the light intensity, and the signal processing The means calculates a distance from an intensity ratio between an image captured when the light intensity and the imaging gain are in phase and an image captured when the light intensity and the imaging gain are in antiphase, and
The visible region outside light irradiating means, so that the optical axis of the irradiated visible outside of the light is close to the optical axis of the camera lens, a three-dimensional-information detecting apparatus in which a plurality disposed around the camera lens,
A three-dimensional information detection system characterized by that. The three-dimensional information detection system according to claim 1,
Instead of arranging a plurality of the outside-visible-light irradiating means around the camera lens, an optical axis for matching the optical axis of the light outside the visible-area irradiated by the outside-visible-light irradiating means with the optical axis of the camera lens Matching means
Three-dimensional-information detecting system characterized in that it has. The three-dimensional information detection system according to claim 2,
The three-dimensional information detection system, wherein the optical axis matching means is a half mirror or a polarization beam splitter .

Images