JP3570635B2 - Vehicle periphery monitoring device - Google Patents

Description

で表わされ、物体位置算出部１６は式（１）によって物体位置ｘ，ｙおよびｚを算出する。
【００２９】
つぎに、図３を参照して、第１の実施例の動作を説明する。
処理Ｓ１では、処理部２２は、フィルタ挿入除去部２１に指令して、帯域フィルタ２０をカメラ１１の前面の撮像経路内に挿入する。
処理Ｓ２では、カメラ１１は車両周辺の監視領域を撮像し、処理Ｓ３で撮像された画像データがフレームメモリ１２に記録される。
【００３０】
処理Ｓ４では、輝点抽出部１４は、フレームメモリ１２に記録された輝度値より、前述した処理により輝点位置を抽出して図示しないメモリに記録する。
処理Ｓ５では、移動輝点抽出部１５は、参照データ記録部１３に記録されている路面に照射された輝点と、処理Ｓ４で抽出した輝点との位置を調べ、位置移動が有った輝点については図示しないメモリに記録する。
【００３１】
処理Ｓ６では、処理部２１は、処理Ｓ５で位置移動が有ったとして記録された輝点が有るか無いかを判定し、判定がＮＯの場合は処理Ｓ１に移り、ＹＥＳの場合は処理Ｓ７に移る。
処理Ｓ７では、物体位置算出部１６は、前述したように、式（１）の演算処理を行って、移動した輝点に対応する物体の３次元位置を算出する。
【００３２】
処理Ｓ８では、障害物判定部１７は、Ｉ／Ｏ２３より車両のハンドル舵角を取込み、車両の車行進路を予測する。
処理Ｓ９では、障害物判定部１７は、処理Ｓ７で算出された物体位置と処理Ｓ８で予測された進路予測より衝突の危険が有るか無いかを判定し、判定がＮＯの場合は処理Ｓ１に終り、ＹＥＳの場合は処理Ｓ１０に移って図示しないブザー等で警告する。
【００３３】
処理Ｓ１１では、障害物マーク部１８はフィルタ挿入除去部２１に指令して帯域フィルタ２０を撮像経路外に除去させる。
処理Ｓ１２ではカメラ１１は監視領域を撮像してフレームメモリ１２に記録する。
【００３４】
処理Ｓ１１で帯域フィルタ２０が除去されているので、処理Ｓ１２で撮像された画像は自然の撮像画像となる。
なお、処理Ｓ１１で帯域フィルタを除去するのと同時にレーザ光源１０ｂの発光を停止させるようにしても良い。レーザ光源１１ｂの発光を停止すると、処理Ｓ１２で撮像した画像にはスポットが現われず、より自然な画像となる。
【００３５】
処理Ｓ１３では、障害物マーク部１８は、フレームメモリ１２に記録されている画像を表示部１９に表示する。
処理Ｓ１４では、障害物マーク部１８は、処理Ｓ９で衝突の危険有りと判定された物体について、図１０に示すように、障害物マークを付ける。
【００３６】
障害物マーク付けの方法としては、処理Ｓ１２でフレームメモリ１２に記録した画像データ値を変更したり、記録されている画像データに重畳して行なう。
また、マーク付けには、図１０に示すように○印や、これらを連ねた棒状でマークし、またマーク色を障害物の高さで変化させる方法、また、検出された障害物の大きさ・形でマーク付けする等色々な方法が考えられる。
【００３７】
図１０は、図６（Ｂ）で示すスポットが移動した輝点位置に○印の障害物マークを付けるようにした例を示す。
つぎに、本発明の第２の実施例を図１１および図１２を参照して説明する。図１１は本発明の第２の実施例の構成図、図１２は同実施例の動作フローチャートである。
【００３８】
図１１において、３１および３２は車両の周辺を撮像するカメラ、３３および３４はカメラ３１および３２で撮像された画像信号を一時記録するフレームメモリ、３５はフレームメモリ３３および３４に記録されている画像データに対して、それぞれカメラ３１および３２のレンズの歪曲収差を補正して、それぞれメモリ（右）３８およびメモリ（左）３９に記録する歪曲収差補正部、３６はメモリ３８または３９に記憶されている画像信号の中の路面画像を除去する路面画像除去部、３７は物体のエッジを検出する物体エッジ検出部である。
【００３９】
また、４０は物体の位置を算出する物体位置算出部、４１は障害物判定部、４２は障害物マーク部、４３は表示部、４４はその他の処理を行なう処理部、４５はインタフェース（Ｉ／Ｏ）、４６は処理を実行するプロセッサ（ＣＰＵ）である。
【００４０】
まず歪曲収差補正部３５について説明する。
カメラ３１および３２で撮像された画像信号は、カメラ３１および３２のレンズに歪曲収差が有る場合は、以後説明する路面画像除去部３６、物体エッジ検出部３７、物体位置算出部４０の処理結果に誤差を発生する。
【００４１】
すなわち、カメラのレンズに収差が無い場合、格子状の模様を撮像した場合は図１３（Ａ）に示すように格子状の画像データがフレームメモリに記録される。しかし、レンズに収差が有る場合は、図１３（Ｂ）または（Ｃ）に示すように歪んだ格子状画像が記録される。
【００４２】
収差量は光軸の中心より点Ｐの距離の３乗に比例して大きくなる。
すなわち、図１３（Ｄ）に示すように、収差が無い時の点Ｐ（ｘ_０ ，ｙ_０ ）が収差によって点Ｐ′（ｘ，ｙ）に結像したものとすると、収差量Ｄは
Ｄ＝［（ｘ_０ −ｘ）^２ ＋（ｙ_０ −ｙ）^２ ］^０．５ ・・・（３）
となり、このＤはレンズの中心点より点Ｐまでの距離の３乗に比例する。すなわち、
［（ｘ_０ −ｘ）^２ ＋（ｙ_０ −ｙ）^２ ］^０．５ ＝ｋ［（ｘ_０ ^２−ｙ_０ ^２）^０．５ ］^３ ・・・（４）
ただしｋは比例定数
となる。
【００４３】
したがって、レンズに収差が存在する場合は、例えば

なる演算を行って歪曲収差を補正する。
【００４４】
歪曲収差補正部３５は、フレームメモリ３３および３４に記録されている画素データについて式（５）の演算を行ってレンズ収差の補正を行なって、それぞれ、メモリ３８および３９に記録する。
また式（５）の演算を行ってフレームメモリの画素の補正を行なった場合、補正後のメモリ３８および３９にデータぬけが生ずる。このようなデータぬけの画素に対しては隣接する画素のデータを基にして補間を行なう。
【００４５】
カメラ３１および３２は、第１の実施例と同様に、図５に示すように、車両後部に路面よりＨなる高さに設置されて、車両後方を撮像する。
そこで、以後の説明を容易にするため、図１４に示すように、カメラ設置点での座標をＸ′，Ｙ′およびＺ′で表わし、路面での座標をＸ，ＹおよびＹで表わすものとする。したがって、カメラの設置俯角θ_Ｓ ＝９０のときは、
Ｘ＝Ｘ′，Ｙ＝Ｙ′＋Ｈ，Ｚ＝Ｚ′ ・・・（６）
となる。
【００４６】
カメラ３１および３２のレンズの光軸は、図１５に示すように、両光軸ともＺ′軸と一致させ、距離Ｌだけ離れて設置される。
まず、物体位置算出部３０での物体位置の算出方法について説明する。
図１５において、Ｘ′軸は両カメラのレンズの中心点を結ぶ方向にとり、Ｘ′軸は路面と平行になるようにカメラ３１および３２を設置する。
【００４７】
Ｙ′軸は、Ｚ′軸とＸ′軸とに垂直となる軸で、路面に垂直な軸である。
以後の説明を容易にするため、Ｘ′，Ｙ′およびＺ′軸のＯ点はカメラ（左）３２のレンズの中心にとる。
このように設置されたカメラ３１および３２によって撮像され、メモリ３８および３９に記録されている点Ｐ（Ｘ_Ｐ ′，Ｙ_Ｐ ′，Ｚ_Ｐ ′）の記録を、それぞれＰ_Ｌ （ｘ_ＬＰ，ｙ_ＬＰ）およびＰ_Ｒ （ｘ_ＲＰ，ｙ_ＲＰ）とすると、点Ｐまでの距離Ｚ_Ｐ ′は
Ｚ_Ｐ ′＝Ｌｆ／（ｘ_ＬＰ−ｘ_ＲＰ） ・・・（７）
ただし、Ｌは両レンズの間隔
ｆはレンズの焦点距離
で表わされる。
【００４８】
したがって、物体位置算出部３０は、メモリ３９よりｘ_ＬＰを、またメモリ３８よりｘ_ＲＰを読取ってＺ_Ｐ ′を算出する。
また、点Ｐ（Ｘ_Ｐ ′，Ｙ_Ｐ ′，Ｚ_Ｐ ′）のＹ_Ｐ ′は、図１５より、

で表わされる。
【００４９】
また、点Ｐ（Ｘ_Ｐ ′，Ｙ_Ｐ ′，Ｚ_Ｐ ′）のＸ_Ｐ ′は、図１５より、

で表わされる。
【００５０】
カメラが図５で示すように、垂直方向よりθ_Ｓ 下方向に向けられている場合は補正が必要となる。
カメラの俯角がθ_Ｓ である場合、図１４に示すようになる。
Ｘ′，Ｙ′およびＺ′座標で表わした点Ｐの位置Ｘ_Ｐ ′，Ｙ_Ｐ ′およびＺ_Ｐ ′は路面座標Ｘ，ＹおよびＺで表わすと図１４で示すようになる。
【００５１】
したがって、式（７），（９）および（１１）で算出したＺ_Ｐ ′，Ｙ_Ｐ ′およびＸ_Ｐ ′を用いてＸ，ＹおよびＺ座標で表わした点Ｐの値をＸ_Ｐ ，Ｙ_Ｐ およびＺ_Ｐ とすると、
Ｘ_Ｐ ＝Ｘ_Ｐ ′ ・・・（１２）
Ｙ_Ｐ ＝Ｈ−Ｚ_Ｐ ′ｃｏｓ θ_Ｓ ＋Ｙ_Ｐ ′ｓｉｎ θ_Ｓ ・・・（１３）
Ｚ_Ｐ ＝Ｚ_Ｐ ′ｓｉｎ θ_Ｓ ＋Ｙ_Ｐ ′ｃｏｓ θ_Ｓ ・・・（１４）
となる。
【００５２】
また、カメラの左右の撮像角がθ_Ｒ の場合は、Ｙ軸を回転軸としてθ_Ｒ 回転する補正を行なう。
物体位置算出部３０は式（１２）〜（１４）の演算を行って物体位置を算出する。
【００５３】
つぎに路面画像除去部３６について説明する。
路面に白線や文字等が描かれている場合、この白線や文字は物体と判定されて距離が算出される。しかし、路面の白線や文字は車両の走行には何んら障害となるものでは無く、このような高さ０の物体は予め除去すれば物体の位置算出処理が簡単となる。
【００５４】
路面画像を除去する原理を図１６を参照して説明する。
図１６（Ａ）はカメラ３１で撮像され、メモリ３８に記録されている右画像を示す。図１６（Ａ）において、３０ａおよびｂは路面上に画かれた白線、また３０ｃは物体（ポール）を示している。
【００５５】
そこで、メモリ３８に記録されている右画像は全て高さ０、すなわち路面上に画かれている画像だと仮定する。この右画像を左カメラ３２が撮像したとして右画像を左画像に投影した画像を演算によって算出する（図１６（Ｂ））。
このようにして算出した投影画像を左画像（メモリ３９）に重畳すると図１６（Ｃ）のようになる。
【００５６】
すなわち、右側カメラ３１で撮像された画像を投影した場合、路面上に画かれている白線等の模様は左側カメラ３２で撮像された模様と位置、輝度共に一致し、物体が路面より高くなるに従って差が大きくなる。
したがって、図１６（Ｄ）に示すように、左画像データと投影画像データの差を求めることによって高さのある物体を構成する画素以外の路面を構成する画素の輝度値は０または０に近い値となり、所定しきい値以下を０とすれば、全て０となる。これにより高さのある部分のみが０以外の値として取り出されて路面画像を除去することができる。
【００５７】
そこで、次に右画像を全て高さ０とした投影画像の算出方法について説明する。
右画像の点Ｐ_Ｒ （ｘ_ＲＰ，ｙ_ＲＰ）に対応する投影画像の点をＰ_Ｌ ′（ｘ_ＬＰ′，ｙ_ＬＰ′）とする。
【００５８】
図７に示すように、カメラ座標のＸ′軸と路面座標のＸ軸は平行であり、また、カメラによって撮像する掃査線のｘ軸（図１５のｘ_Ｌ 軸およびｘ_Ｒ 軸）も共に平行であるとすると、同一物体を撮像した場合の撮像画像のｙ_Ｌ とｙ_Ｒ 値は一致する。
【００５９】
したがって、
ｙ_ＬＰ′＝ｙ_ＲＰ ・・・（１５）
となる。
また、ｘ_ＬＰ′は、画像の全てが路面上であるとすれば、式（１３）で示すＹ_Ｐ の値は０となり、
０＝Ｈ_Ｐ −Ｚ_Ｐ ′ｃｏｓ θ_Ｓ ＋Ｙ_Ｐ ′ｓｉｎ θ_Ｓ ・・・（１６）
となる。
【００６０】
そこで、式（１６）のＺ_Ｐ ′およびＹ_Ｐ ′に式（７）のＺ_Ｐ ′および式（９）のＹ_Ｐ ′を代入してｘ_ＬＰ′を求めると、
ｘ_ＬＰ′＝（Ｌｆｃｏｓ θ_Ｓ −Ｌｙ_ＲＰｓｉｎ θ_Ｓ ）／Ｈ＋ｘ_ＲＰ ・・・（１７）
となる。
【００６１】
路面画像除去部３６は式（１５）および（１７）の演算を行なって投影画像（図１６（Ｂ））を算出する。
投影画像が作成されると左画像、すなわちメモリ３９に記録されているデータとの差を求めて路面画像を除去する（図１６（Ｄ））。
【００６２】
つぎに物体エッジ検出部３７について説明する。
図１７（Ａ）は、図１６（Ａ）で説明したと同様な路面上に画かれた白線３０ａおよびｂと路面上の物体（ポール）３０ｃを左側カメラ３２が撮像し、メモリ３９に記録されている画像データを示す。
【００６３】
メモリ３９に記録されているｍ行ｎ列の画像データの輝度値Ｉ_ｍ，ｎ を水平方向に掃査し、

ただし、Ｅ_０ はしきい値
なる処理を行なって微分画像を求めると、図１７（Ｂ）に示すように、物体や路面に画かれている文字等の縦方向エッジ部分が“１”に、その他の部分は“０”となる。
【００６４】
このようにして求めた微分画像（図１７（Ｂ））と、前述した路面画像除去部１６で求めた差分画像（図１６（Ｄ））と重ね合せてアンドを取ると、図１７（Ｃ）で示す物体のエッジ部分のみが抽出される。
したがって、この抽出された物体のエッジ部分について物体の３次元位置を算出させるようにすることによって、演算処理を大幅に低減させることができる。
【００６５】
つぎに、図１２を参照して実施例の動作を説明する。
処理Ｓ２１では、処理部４４は、カメラ３１および３２で撮像した画像データを、それぞれフレームメモリ３３および３４に記録する。
処理Ｓ２２では、歪曲収差補正部３５は、フレームメモリ３３および３４に記録されているデータに対して、カメラ３１および３２のレンズの収差に対応する補正を行なって、それぞれ、メモリ３８および３９に記録する。
【００６６】
処理Ｓ２３では、物体エッジ検出部３７は、メモリ３９に記録されている左画像データを水平方向に掃査して、式（１８）で示す演算を行なわせて左画像の微分画像（図１７（Ｂ））を作成して図示しないメモリに記録する。
処理Ｓ２４では、路面画像除去部３６は、メモリ３８に記録されている右画像を路面上画像と仮定して左画像に投影した投影画像（図１６（Ｂ））を作成して図示しないメモリに記録する。
【００６７】
処理Ｓ２５では、路面画像除去部３６は、メモリ３８に記録されている左画像より処理Ｓ２４で作成した投影画像データを減算した差分画像（図１６（Ｄ））を作成して図示しないメモリに記録する。
処理Ｓ２６では、物体エッジ検出部３７は、処理Ｓ２３で作成した微分画像（図１７（Ｂ））と処理Ｓ２５で作成した差分画像（図１６（Ｄ））より物体のエッジを表わす物体エッジ画像（図１７（Ｃ））を作成して図示しないメモリに記録する。
【００６８】
処理Ｓ２７では、処理部４４は、処理Ｓ２６で作成した物体エッジ画像データのエッジ部分について、測定すべき物体の左右画像の対応点をメモリ３８および３９より抽出する。
処理Ｓ２８では、物体位置算出部３０は、処理Ｓ２７で抽出された点に対応する物体位置を算出する。
【００６９】
処理Ｓ２９では、障害物判定部４１は、処理Ｓ２８の処理結果より障害物が有るか否かを判定し、判定がＮＯの場合は処理Ｓ３０に移って、図示しないモニタにフレーメモリ３３または３４に記録されている画像を表示す。
処理Ｓ３１では、障害物判定部４１は、Ｉ／Ｏ４５を介して舵角信号を読込み、処理Ｓ３２に移って読込んだ舵角より車両が走行する軌跡を推定する。
【００７０】
処理Ｓ３３では、障害物判定部４１は、検出された障害物に対して車両が衝突の可能性が有るか否かを処理Ｓ３２の走行軌跡と比較して判定し、判定がＹＥＳの場合は処理Ｓ３４に、また、衝突の可能性が無い場合は処理Ｓ３０に移ってモニタに撮像画像を表示して処理Ｓ２１に移る。
【００７１】
処理Ｓ３４では、障害物判定部４１はブザーを鳴らして警報を発する。
処理Ｓ３５では、障害物マーク部４２は、フレームメモリ３３または３４に記録されている画像を表示部４３に表示する。
処理Ｓ３６では、障害物マーク部４２は、処理Ｓ３３で衝突の可能性が有りと判定された物体について、図１０に示すように障害物マークを付けて処理Ｓ２１に移る。
【００７２】
なお、障害物のマーク方法は第１の実施例で説明したと同様な方法で行なわれる。
なお、処理Ｓ３５での画像の表示をフレームメモリ３４に記録されている画像データで表示を行ない、処理Ｓ２３で物体エッジ検出部３７が検出した物体エッジ（図１７（Ｃ））を表示画像に重畳するようにすれば、障害物のマーク付けを容易に行なうことができる。
【００７３】
なお、以上説明した実施例では、右側画像より投影画像を作成して左側画像に重畳したが、左側画像より投影画像を作成して右側画像に重畳するようにしてもよい。
【００７４】
【発明の効果】
以上説明したように、本発明によれば次の効果が得られる。
車両周辺の監視領域を撮像して表示している画像の車両の走行に障害となる物体に対してマーク付けを行なうようにしたので、直ちに障害物の位置を認識することができ、安全にかつスムーズに車両の運転を行なうことができる。
【００７５】
また、投光器より照射するスポットを撮像して物体位置を算出する方法をとる場合は、帯域フィルタを除去して撮像した画像を表示するようにしたので、鮮明な画像が表示されマーク付けされた障害物を容易に認識することができる。
【図面の簡単な説明】
【図１】本発明の基本構成図である。
【図２】本発明の第１の実施例の構成図である。
【図３】同実施例の動作フローチャートである。
【図４】実施例の投光器のファイバグレイティングの説明図である。
【図５】実施例の投光器およびカメラの設置例を示す図である。
【図６】撮像スポットの説明図である。
【図７】撮像画像信号の一走査線上の輝度分布を示す図である。
【図８】撮像画像信号より輝点位置を算出するための説明図である。
【図９】輝点位置より物体位置を算出するための説明図である。
【図１０】表示例を示す図である。
【図１１】本発明の第２の実施例の構成図である。
【図１２】同実施例の動作フローチャートである。
【図１３】同実施例のレンズ収差補正の説明図である。
【図１４】同実施例のカメラ俯角補正の説明図である。
【図１５】同実施例の３次元位置測定説明図である。
【図１６】同実施例の路面画像除去の説明図である。
【図１７】同実施例の物体エッジ検出の説明図である。
【図１８】従来の表示例を示す図である。
【符号の説明】
１ 撮像手段
２ 表示手段
３ 物体位置算出手段
４ 障害物判定手段
５ 障害物マーク手段
１０ 投光器
１１，３１，３２ カメラ
１０ａ ファイバグレイティング（ＦＧ）
１０ｂ レーザ光源
１２，３３，３４ フレームメモリ
１３ 参照データ記録部
１４ 輝点抽出部
１５ 移動輝点抽出部
１６，４０ 物体位置算出部
１７，４１ 障害物判定部
１８，４２ 障害物マーク部
１９，４３ 表示部
２０ 帯域フィルタ
２１ フィルタ挿入除去部
２２，４４ 処理部
２３，４５ インタフェース（Ｉ／Ｏ）
２４，４６ プロセッサ（ＣＰＵ）
３５ 歪曲収差補正部
３６ 路面画像除去部
３７ 物体エッジ検出部
３８，３９ メモリ[0001]
[Industrial applications]
The present invention relates to a vehicle periphery monitoring device that is effectively applied to monitor the periphery of a vehicle such as an automobile and assist the driver in confirming the safety of driving the vehicle.
[0002]
[Prior art]
As described in Japanese Patent Application No. 4-170559, a conventional vehicle peripheral device moves a bright spot position where the same spot as an imaged bright spot of a spot irradiated on a road surface is reflected by an object and imaged. The position of the object is calculated to detect the presence of the obstacle.
[0003]
In other words, as shown in FIG. 5, a large number of spots are illuminated by a projector mounted on a vehicle onto a road surface where no object exists as shown in FIG. 6A, and the illuminated spots are imaged and recorded by a camera. In addition, as shown in FIG. 6B, when the road surface on which the object exists is imaged, the position of the imaged spot changes with respect to the spot irradiated on the object.
[0004]
The position of the object is calculated from the change amount of the spot position and is displayed on the display unit as shown in FIG.
In addition, a camera is installed in place of the projector, and the position of the object is calculated from both image positions captured by the two left and right cameras, and the object position is displayed on the display unit as shown in FIG. Was.
[0005]
[Problems to be solved by the invention]
As described above, the conventional vehicle periphery monitoring device calculates the movement amount of the bright spot position of the spot captured by the camera or the position of an object having a height higher than the difference between the image positions captured by the two cameras, and calculates the position of the vehicle. Obstacles that hinder driving are displayed on the display unit using graphic figures.
[0006]
For this reason, the display on the display unit is different from the image captured in the actual monitoring area, and the safety is not sufficiently confirmed, so that the monitoring is performed by switching between the graphic figure and the captured image. It took a long time to recognize which of the captured images the object corresponds to.
[0007]
SUMMARY OF THE INVENTION It is an object of the present invention to provide an improved vehicle periphery monitoring device in which an image of a monitoring area displayed on a monitor is marked with an obstacle to travel of the vehicle so that the position of the obstacle can be immediately recognized. And
[0008]
[Means for Solving the Problems]
Means adopted by the present invention to solve the above-described problem will be described with reference to FIG. FIG. 1 is a basic configuration diagram of the present invention.
Imaging means (1) for imaging a monitoring area around the vehicle;
A display unit (2) for displaying an image taken by the imaging unit (1);
An object position calculating means (3) for calculating the position of the object from the image data picked up by the image pickup means (1);
Obstacle determining means (4) for determining whether or not the object calculated by the object position calculating means (3) is an obstacle to running of the vehicle;
Obstacle marking means (5) for marking an obstacle in the captured image displayed on the display means (2) when the obstacle determining means (4) determines that the obstacle is an obstacle to the running of the vehicle. When,
Is provided.
[0009]
Further, the imaging means (1) images a plurality of spots irradiated from the light projector.
Further, for the image displayed by the display means (2), a bandpass filter passing light having the wavelength of the irradiation light of the spot is removed from the imaging path, and the image is imaged by the imaging means (1).
[0010]
Further, the imaging means (1) images the monitoring area with two left and right cameras,
Assuming that the images captured by one camera are all images drawn on the road surface, calculate the image projected on the image by the other camera,
Obtain a difference image by removing the road surface image by removing the projected image from the image captured by the other camera,
The position of the object is calculated by using an edge portion extracted by superimposing a differential image of the image captured by the other camera and the differential image and obtaining an AND.
[0011]
[Action]
The imaging means 1 images a monitoring area around the vehicle.
The object position calculator 3 calculates the position of the object from the image data captured by the imager 1.
[0012]
The obstacle determining means 4 determines whether or not the object calculated by the object position calculating means 3 becomes an obstacle to the running of the vehicle.
When the obstacle determination unit 4 determines that the traveling of the vehicle is an obstacle, the obstacle mark unit 5 marks an obstacle that is an obstacle in the image captured by the imaging unit 1 and displayed on the display unit 2.
[0013]
Further, when the image pickup means 1 picks up an image of a spot irradiated from the light projector and calculates the object position, the bandpass filter installed in the image pickup means 1 is removed outside the image pickup path and the picked up image is displayed on the display means 2. To mark obstacles.
As described above, since an object that is an obstacle to the running of the vehicle in the image displayed by capturing and displaying the monitoring area around the vehicle is marked, the position of the obstacle can be immediately recognized. It is possible to safely and smoothly drive the vehicle.
[0014]
Also, when the method of calculating the object position by imaging the spot irradiated by the projector is adopted, since the band-pass filter is removed and the captured image is displayed, a clear image is displayed and the obstacle marked is marked. An object can be easily recognized.
Further, the image capturing means (1) captures an image of the monitoring area with two cameras on the left and right, and assuming that all images captured by one camera are images drawn on a road surface, the image captured by the other camera is used. Calculate the image projected on the other camera, remove the projected image from the image captured by the other camera to obtain a difference image from which the road surface image is removed, and superimpose the differential image and the difference image of the image captured by the other camera. Since the position of the object is calculated by using the edge portion extracted by extracting the AND, the arithmetic processing can be greatly reduced.
[0015]
【Example】
A first embodiment of the present invention will be described with reference to FIGS. FIG. 2 is a block diagram of the first embodiment of the present invention, and FIG. 3 is an operation flowchart of the first embodiment.
2, reference numeral 10 denotes a projector, 11 denotes an imaging camera, 12 denotes a frame memory, 13 denotes a reference data recording unit, 14 denotes a bright spot extracting unit, 15 denotes a moving bright spot extracting unit, 16 denotes an object position calculating unit, and 17 denotes an obstacle. An object determining unit, 18 is an obstacle mark unit, 19 is a display unit, 20 is a bandpass filter, 21 is a filter insertion / removal unit that inserts or removes the bandpass filter 20 inside or outside the imaging path, 22 is a processing unit that performs other processing, 23 is an interface (I / O), and 24 is a processor (CPU) that executes all processes.
[0016]
The light projector 10 includes a laser light source 10b that emits laser light and a fiber grating (FG) 10a, and irradiates a monitoring area around the vehicle with a grid-like laser light spot.
As shown in FIG. 4, the FG 10a is formed by arranging about 100 optical fibers having a diameter of several tens of μm and a length of 10 mm in a sheet shape, and superposing two such sheets orthogonally.
[0017]
When laser light is incident on the FG 1a from the laser light source 10b as shown by an arrow, the laser light is condensed at the focal point of each optical fiber, and then spreads as a spherical wave while interfering. As a result, the projection surface Is irradiated with a grid-like spot.
The camera 11 captures an image of the spot irradiated by the light projector 10, and the captured image signal is temporarily recorded in the frame memory 12.
[0018]
The bandpass filter 20 installed in front of the camera 11 allows light having a wavelength near 830 nm, which is an infrared region, emitted by the laser light source 10b of the light projector 10 to pass.
The reason why the bandpass filter 20 is installed on the front of the camera 11 is that when the camera 11 captures an image of a spot irradiated by the projector 10, sunlight and illumination light besides the spot are incident on the camera 11 to mask the spot. Become like The bandpass filter 20 is provided to allow the light of the radiating spot to pass therethrough, remove the sun and illumination light, improve the S / N, and reliably extract the bright spot of the spot.
[0019]
Further, as shown in FIG. 5, the light projector 10 and the camera 11 are fixedly mounted on the ground at an angle θ with respect to a normal line on the ground.
FIG. 6 shows an image of the irradiated spot captured by the camera 11, FIG. 6 (A) shows a case where there is no object on the road surface, and FIG. 6 (B) shows a case where there is an object. ing.
[0020]
When the spot irradiated from the light projector 10 is irradiated on the object, the spot position of the image captured by the camera 11 moves. That is, in a place where no object exists, the spot position indicated by a circle in FIG. 6 (A) and the spot position indicated by a circle in FIG. 6 (B) coincide with each other. ● The mark positions do not match.
[0021]
The vehicle periphery monitoring device calculates the object position from the movement amount between the mark position and the mark position.
As shown in FIG. 6A, a bright spot position obtained from an imaged spot when no object exists on the road surface is recorded in the reference data recording unit 13 in advance. The bright spot position is determined by a bright spot extracting unit described below.
[0022]
First, the bright spot extracting unit 14 sequentially reads out each pixel in the horizontal direction (V axis) recorded by the camera 11 and recorded in the frame memory 12, and is determined based on the background luminance as shown in FIG. If the luminance is lower than the threshold value as compared with the threshold value, the luminance I of the pixel is set to O.
[0023]
When the above processing is performed on all the pixels recorded in the frame memory 12, only the portion where the irradiated spot is imaged can be extracted as shown in FIG. FIG. 8 shows one spot as a representative.
Next, as shown in FIG. 8, the bright point extraction unit 14 calculates the center of gravity position by weighting the coordinate position of each pixel with the luminance I for a pixel whose luminance value is enclosed by a circle. The center of gravity position is recorded as a bright spot position in a memory (not shown).
[0024]
The reference data recording unit 13 previously captures an image of a road surface without an object, and records the bright spot positions extracted by the bright spot extracting unit 14 described above.
The moving bright spot extracting unit 15 compares the bright spot position recorded in the reference data recording unit 13 with the position obtained by imaging the periphery of the vehicle, and does not illustrate the bright spot whose position is moving. Record in memory. That is, among the spots illuminated by the light projector 10, the bright spot obtained by imaging the spot illuminated on the object is extracted.
[0025]
The object position calculator 16 calculates the position of the object from the bright spot position.
FIG. 9 is an explanatory diagram for calculating the object position of the object position calculation unit 16. FIG. 9A shows X, Y, and Z with the origin at the point where the optical axis of the lens of the camera 11 intersects the road surface. FIG. 9 (B) is a representation in the Y and Z axis planes.
[0026]
L is the focal length of the lens of the camera 11, D is the distance between the center of the lens and the projector 10, θ is the angle of depression of the camera described with reference to FIG. 5, and H is the height of the center of the lens on the road surface.
Further, a point P (x, y, z) is a point where the spot from the light projector 10 irradiates the object, and a point P (u, v + δ) is the point P (x, y, z) where the object is illuminated by the camera 11. Bright point obtained by imaging, point P_A(X_A, Y_A, 0) is the spot position illuminated on the road surface when there is no object, point P_A(U, v) is the point P_A(X_A, Y_A, 0).
[0027]
As is clear from FIG. 9A, when the spot irradiated from the light projector 10 is irradiated on the object, the bright spot position is determined by the absence of the object, that is, the bright spot position obtained when the object is irradiated on the road surface. Move in the v-axis (horizontal direction) direction. When the spot is irradiated to a point lower than the road surface, such as a groove, the spot moves in the opposite direction.
[0028]
Therefore, the point P (x, y, z) is calculated according to the principle of triangulation.

The object position calculation unit 16 calculates the object positions x, y, and z according to equation (1).
[0029]
Next, the operation of the first embodiment will be described with reference to FIG.
In the process S1, the processing unit 22 instructs the filter insertion / removal unit 21 to insert the bandpass filter 20 into the imaging path on the front surface of the camera 11.
In the process S2, the camera 11 captures an image of the monitoring area around the vehicle, and the image data captured in the process S3 is recorded in the frame memory 12.
[0030]
In the processing S4, the bright spot extracting unit 14 extracts the bright spot position from the luminance value recorded in the frame memory 12 by the above-described processing, and records it in a memory (not shown).
In the processing S5, the moving luminescent spot extracting unit 15 checks the positions of the luminescent spot illuminated on the road surface recorded in the reference data recording unit 13 and the luminescent spot extracted in the processing S4, and there is a position shift. The bright spot is recorded in a memory (not shown).
[0031]
In the process S6, the processing unit 21 determines whether or not there is a bright spot recorded as having the position movement in the process S5. If the determination is NO, the process proceeds to the process S1, and if the determination is YES, the process S7 is performed. Move on to
In the process S7, the object position calculation unit 16 calculates the three-dimensional position of the object corresponding to the moved bright spot by performing the calculation processing of the equation (1) as described above.
[0032]
In the process S8, the obstacle determination unit 17 takes in the steering angle of the vehicle from the I / O 23 and predicts the vehicle traveling path.
In the process S9, the obstacle determination unit 17 determines whether there is a danger of a collision based on the object position calculated in the process S7 and the course prediction predicted in the process S8, and if the determination is NO, the process proceeds to the process S1. If the determination is YES, the process proceeds to step S10 to warn the user with a buzzer or the like (not shown).
[0033]
In the process S11, the obstacle mark unit 18 instructs the filter insertion / removal unit 21 to remove the bandpass filter 20 out of the imaging path.
In step S12, the camera 11 captures an image of the monitoring area and records it in the frame memory 12.
[0034]
Since the bandpass filter 20 has been removed in step S11, the image captured in step S12 is a natural captured image.
The emission of the laser light source 10b may be stopped simultaneously with the removal of the bandpass filter in the processing S11. When the emission of the laser light source 11b is stopped, no spot appears in the image captured in step S12, and the image becomes more natural.
[0035]
In the process S13, the obstacle mark unit 18 displays the image recorded in the frame memory 12 on the display unit 19.
In step S14, the obstacle mark unit 18 attaches an obstacle mark to the object determined to be in danger of collision in step S9, as shown in FIG.
[0036]
As a method of marking an obstacle, the image data value recorded in the frame memory 12 in the process S12 is changed or superimposed on the recorded image data.
In addition, as shown in FIG. 10, a method of changing the mark color by changing the mark color with the height of the obstacle, or the size of the detected obstacle,・ Various methods such as marking with shapes are conceivable.
[0037]
FIG. 10 shows an example in which an obstruction mark indicated by a circle is placed at the bright spot position where the spot shown in FIG. 6B has moved.
Next, a second embodiment of the present invention will be described with reference to FIGS. FIG. 11 is a block diagram of a second embodiment of the present invention, and FIG. 12 is an operation flowchart of the second embodiment.
[0038]
In FIG. 11, reference numerals 31 and 32 denote cameras for imaging the periphery of the vehicle, reference numerals 33 and 34 denote frame memories for temporarily recording image signals captured by the cameras 31 and 32, and reference numeral 35 denotes images recorded in the frame memories 33 and 34. The distortion correction unit 36 corrects the distortion of the lenses of the cameras 31 and 32 with respect to the data, and records the corrected data in the memory (right) 38 and the memory (left) 39, respectively. A road surface image removing unit 37 for removing the road surface image from the image signal present is an object edge detecting unit 37 for detecting the edge of the object.
[0039]
Reference numeral 40 denotes an object position calculation unit for calculating the position of the object, 41 denotes an obstacle determination unit, 42 denotes an obstacle mark unit, 43 denotes a display unit, 44 denotes a processing unit that performs other processing, and 45 denotes an interface (I / I / O). O) and 46 are processors (CPU) that execute processing.
[0040]
First, the distortion correction unit 35 will be described.
When the lenses of the cameras 31 and 32 have distortion, the image signals captured by the cameras 31 and 32 include the processing results of the road surface image removing unit 36, the object edge detecting unit 37, and the object position calculating unit 40 described below. Generates an error.
[0041]
That is, when there is no aberration in the camera lens and when a lattice pattern is imaged, lattice image data is recorded in the frame memory as shown in FIG. However, when the lens has an aberration, a distorted lattice image is recorded as shown in FIG. 13B or 13C.
[0042]
The amount of aberration increases in proportion to the cube of the distance of the point P from the center of the optical axis.
That is, as shown in FIG. 13D, the point P (x₀, Y₀) Is imaged at the point P ′ (x, y) by aberration, the amount of aberration D is
D = [(x₀-X)²+ (Y₀-Y)²]^0.5            ... (3)
Where D is proportional to the cube of the distance from the center point of the lens to the point P. That is,
[(X₀-X)²+ (Y₀-Y)²]^0.5= K [(x₀ ²-Y₀ ²)^0.5]³... (4)
Where k is a proportional constant
It becomes.
[0043]
Therefore, if there is aberration in the lens, for example,

The following calculation is performed to correct the distortion.
[0044]
The distortion correction unit 35 corrects the lens aberration by performing the operation of Expression (5) on the pixel data recorded in the frame memories 33 and 34, and records the corrected data in the memories 38 and 39, respectively.
Also, when the pixel of the frame memory is corrected by performing the operation of the equation (5), data loss occurs in the corrected memories 38 and 39. For such a pixel without data, interpolation is performed based on the data of the adjacent pixel.
[0045]
Similar to the first embodiment, the cameras 31 and 32 are installed at the rear of the vehicle at a height of H from the road surface, as shown in FIG. 5, and image the rear of the vehicle.
Therefore, in order to facilitate the following description, as shown in FIG. 14, the coordinates at the camera installation point are represented by X ', Y' and Z ', and the coordinates on the road surface are represented by X, Y and Y. I do. Therefore, the camera installation depression angle θ_S= 90
X = X ', Y = Y' + H, Z = Z '(6)
It becomes.
[0046]
As shown in FIG. 15, the optical axes of the lenses of the cameras 31 and 32 coincide with the Z ′ axis, and are installed at a distance L.
First, a method of calculating an object position in the object position calculation unit 30 will be described.
In FIG. 15, the X 'axis is set in the direction connecting the center points of the lenses of both cameras, and the cameras 31 and 32 are installed so that the X' axis is parallel to the road surface.
[0047]
The Y 'axis is an axis perpendicular to the Z' axis and the X 'axis, and is an axis perpendicular to the road surface.
In order to facilitate the following description, the point O on the X ', Y' and Z 'axes is set at the center of the lens of the camera (left) 32.
The points P (X) which are imaged by the cameras 31 and 32 installed in this way and are recorded in the memories 38 and 39._P', Y_P', Z_P′) Is P_L(X_LP, Y_LP) And P_R(X_RP, Y_RP), The distance Z to the point P_P′
Z_P'= Lf / (x_LP-X_RP) (7)
Where L is the distance between both lenses
f is the focal length of the lens
Is represented by
[0048]
Therefore, the object position calculator 30 obtains x_LPAnd x from memory 38_RPRead Z_P'.
Also, the point P (X_P', Y_P', Z_P') Y_P'From FIG.

Is represented by
[0049]
Also, the point P (X_P', Y_P', Z_P') X_P'From FIG.

Is represented by
[0050]
As shown in FIG._SIf it is directed downward, correction is required.
The angle of depression of the camera is θ_SIn this case, the result is as shown in FIG.
The position X of the point P represented by X ', Y' and Z 'coordinates_P', Y_P'And Z_P'Is represented by road surface coordinates X, Y and Z as shown in FIG.
[0051]
Therefore, Z calculated by equations (7), (9) and (11)_P', Y_P'And X_P′, The value of point P expressed by X, Y and Z coordinates_P, Y_PAnd Z_PThen
X_P= X_P'... (12)
Y_P= H-Z_P'Cos θ_S+ Y_P'Sin θ_S              ... (13)
Z_P= Z_P'Sin θ_S+ Y_P'Cos θ_S                  ... (14)
It becomes.
[0052]
Also, when the imaging angles of the left and right_RIn the case of, θ_RPerform rotation correction.
The object position calculation unit 30 calculates the object position by performing the calculations of Expressions (12) to (14).
[0053]
Next, the road surface image removing unit 36 will be described.
When a white line or a character is drawn on the road surface, the white line or the character is determined as an object, and the distance is calculated. However, white lines and characters on the road surface do not hinder the running of the vehicle at all, and if such an object having a height of 0 is removed in advance, the position calculation processing of the object can be simplified.
[0054]
The principle of removing the road surface image will be described with reference to FIG.
FIG. 16A shows a right image captured by the camera 31 and recorded in the memory 38. In FIG. 16A, 30a and 30b indicate white lines drawn on the road surface, and 30c indicates an object (pole).
[0055]
Therefore, it is assumed that all the right images recorded in the memory 38 have a height of 0, that is, images drawn on a road surface. Assuming that the right image is captured by the left camera 32, an image obtained by projecting the right image on the left image is calculated by calculation (FIG. 16B).
When the projection image thus calculated is superimposed on the left image (memory 39), the result is as shown in FIG.
[0056]
That is, when an image captured by the right camera 31 is projected, a pattern such as a white line drawn on the road surface matches the position and the luminance of the pattern captured by the left camera 32, and as the object becomes higher than the road surface, The difference increases.
Therefore, as shown in FIG. 16D, the luminance value of the pixels constituting the road surface other than the pixels constituting the tall object by calculating the difference between the left image data and the projection image data is 0 or close to 0. If all the values are equal to or smaller than the predetermined threshold value and are set to 0, all the values become 0. Thereby, only a portion having a height is extracted as a value other than 0, and the road surface image can be removed.
[0057]
Therefore, a method of calculating a projected image in which all the right images have a height of 0 will be described.
Point P in the right image_R(X_RP, Y_RP) Is the point of the projected image corresponding to P_L′ (X_LP', Y_LP').
[0058]
As shown in FIG. 7, the X 'axis of the camera coordinates and the X axis of the road surface coordinates are parallel, and the x axis of the sweep line imaged by the camera (x in FIG. 15)._LAxis and x_RAxes are also parallel, y of the captured image when the same object is captured_LAnd y_RThe values match.
[0059]
Therefore,
y_LP'= Y_RP                                          ... (15)
It becomes.
Also, x_LP′ Is Y given by equation (13) if all of the images are on the road surface._PBecomes 0,
0 = H_P-Z_P'Cos θ_S+ Y_P'Sin θ_S              ... (16)
It becomes.
[0060]
Therefore, Z in equation (16)_P'And Y_P'To the Z of equation (7)_P'And Y in equation (9)_P'And substitute x_LP′
x_LP'= (Lfcos θ_S-Ly_RPsin θ_S) / H + x_RP    ... (17)
It becomes.
[0061]
The road surface image removing unit 36 calculates the projection images (FIG. 16B) by performing the calculations of the equations (15) and (17).
When the projection image is created, the difference from the left image, that is, the data recorded in the memory 39 is obtained, and the road surface image is removed (FIG. 16D).
[0062]
Next, the object edge detector 37 will be described.
FIG. 17A shows white lines 30 a and 30 b drawn on the road surface and an object (pole) 30 c on the road surface which are similar to those described with reference to FIG. The image data is shown.
[0063]
The luminance value I of the image data of m rows and n columns recorded in the memory 39_{m, n}Sweeps horizontally,

Where E₀Is the threshold
When the differential image is obtained by performing the following processing, as shown in FIG. 17B, the vertical edge portion of the object or the character drawn on the road surface becomes “1”, and the other portions become “0”. Become.
[0064]
When the differential image (FIG. 17 (B)) obtained in this way and the difference image (FIG. 16 (D)) obtained by the above-described road surface image removing unit 16 are overlapped and ANDed, FIG. 17 (C) Only the edge portions of the object indicated by are extracted.
Therefore, by calculating the three-dimensional position of the object with respect to the extracted edge portion of the object, the arithmetic processing can be significantly reduced.
[0065]
Next, the operation of the embodiment will be described with reference to FIG.
In the process S21, the processing unit 44 records the image data captured by the cameras 31 and 32 in the frame memories 33 and 34, respectively.
In the process S22, the distortion correction unit 35 corrects the data recorded in the frame memories 33 and 34 in accordance with the aberrations of the lenses of the cameras 31 and 32, and records the data in the memories 38 and 39, respectively. I do.
[0066]
In the process S23, the object edge detecting section 37 sweeps the left image data recorded in the memory 39 in the horizontal direction, performs the operation represented by Expression (18), and differentiates the left image (see FIG. B)) is created and recorded in a memory (not shown).
In the process S24, the road surface image removing unit 36 creates a projection image (FIG. 16B) projected on the left image assuming that the right image recorded in the memory 38 is an image on the road surface and stores it in a memory (not shown). Record.
[0067]
In the process S25, the road surface image removing unit 36 creates a difference image (FIG. 16D) obtained by subtracting the projection image data created in the process S24 from the left image recorded in the memory 38, and records the difference image in a memory (not shown). I do.
In the process S26, the object edge detecting unit 37 uses the differential image (FIG. 17B) created in the process S23 and the difference image (FIG. 16D) created in the process S25 to create an object edge image (FIG. 16D) representing an object edge. 17 (C) is created and recorded in a memory (not shown).
[0068]
In step S27, the processing unit 44 extracts corresponding points of the left and right images of the object to be measured from the memories 38 and 39 for the edge portion of the object edge image data created in step S26.
In step S28, the object position calculator 30 calculates an object position corresponding to the point extracted in step S27.
[0069]
In the process S29, the obstacle determination unit 41 determines whether or not there is an obstacle based on the processing result of the process S28. If the determination is NO, the process proceeds to the process S30, and the monitor (not shown) stores the information in the frame memory 33 or 34. Display the recorded image.
In the process S31, the obstacle determination unit 41 reads the steering angle signal via the I / O 45, and proceeds to the process S32 to estimate the trajectory on which the vehicle travels based on the read steering angle.
[0070]
In the process S33, the obstacle determination unit 41 determines whether or not there is a possibility that the vehicle will collide with the detected obstacle by comparing with the traveling locus in the process S32. In S34, when there is no possibility of collision, the process proceeds to S30, where the captured image is displayed on the monitor, and the process proceeds to S21.
[0071]
In the process S34, the obstacle determining unit 41 sounds a buzzer and issues an alarm.
In the process S35, the obstacle mark unit 42 displays an image recorded in the frame memory 33 or 34 on the display unit 43.
In the process S36, the obstacle mark unit 42 attaches an obstacle mark to the object determined to have a possibility of collision in the process S33 as shown in FIG. 10 and proceeds to the process S21.
[0072]
The method of marking an obstacle is performed in the same manner as described in the first embodiment.
The display of the image in the process S35 is performed using the image data recorded in the frame memory 34, and the object edge (FIG. 17C) detected by the object edge detection unit 37 in the process S23 is superimposed on the display image. By doing so, it is possible to easily mark an obstacle.
[0073]
In the embodiment described above, a projection image is created from the right image and is superimposed on the left image. However, a projection image may be created from the left image and superimposed on the right image.
[0074]
【The invention's effect】
As described above, according to the present invention, the following effects can be obtained.
Since an image of the monitoring area around the vehicle is displayed and marked with an object that is an obstacle to the traveling of the vehicle, the position of the obstacle can be immediately recognized, and the safe and secure operation can be performed. The vehicle can be operated smoothly.
[0075]
Also, when the method of calculating the object position by imaging the spot irradiated by the projector is adopted, since the band-pass filter is removed and the captured image is displayed, a clear image is displayed and the obstacle marked is marked. An object can be easily recognized.
[Brief description of the drawings]
FIG. 1 is a basic configuration diagram of the present invention.
FIG. 2 is a configuration diagram of a first embodiment of the present invention.
FIG. 3 is an operation flowchart of the embodiment.
FIG. 4 is an explanatory diagram of fiber grating of the projector according to the embodiment.
FIG. 5 is a diagram illustrating an example of installation of a projector and a camera according to the embodiment.
FIG. 6 is an explanatory diagram of an imaging spot.
FIG. 7 is a diagram illustrating a luminance distribution on one scanning line of a captured image signal.
FIG. 8 is an explanatory diagram for calculating a bright spot position from a captured image signal.
FIG. 9 is an explanatory diagram for calculating an object position from a bright spot position.
FIG. 10 is a diagram showing a display example.
FIG. 11 is a configuration diagram of a second embodiment of the present invention.
FIG. 12 is an operation flowchart of the embodiment.
FIG. 13 is an explanatory diagram of lens aberration correction of the same embodiment.
FIG. 14 is an explanatory diagram of camera depression angle correction of the embodiment.
FIG. 15 is an explanatory diagram of three-dimensional position measurement according to the same embodiment.
FIG. 16 is an explanatory diagram of road surface image removal of the embodiment.
FIG. 17 is an explanatory diagram of object edge detection of the embodiment.
FIG. 18 is a diagram showing a conventional display example.
[Explanation of symbols]
1 imaging means
2 Display means
3 Object position calculation means
4 Obstacle judgment means
5 Obstacle mark means
10 Floodlight
11,31,32 camera
10a Fiber grating (FG)
10b laser light source
12, 33, 34 frame memory
13 Reference data recording unit
14 Bright spot extractor
15 Moving bright spot extractor
16,40 Object position calculation unit
17, 41 Obstacle judgment unit
18,42 Obstacle mark part
19, 43 display unit
20 band filter
21 Filter insertion and removal unit
22, 44 processing unit
23, 45 interface (I / O)
24, 46 processor (CPU)
35 Distortion aberration corrector
36 Road surface image remover
37 Object edge detector
38,39 memory

Claims (2)

Imaging means for imaging a monitoring area around the vehicle;
Display means for displaying an image taken by the imaging means;
Object position calculation means for calculating the position of the object from the image data imaged by the imaging means,
Obstacle determining means for determining whether the object calculated by the object position calculating means is an obstacle to running of the vehicle,
When it is determined that the obstacle is determined to be an obstacle to the running of the vehicle by the obstacle determining means, the obstacle determining means for marking an obstacle to the captured image displayed on the display means ,
The imaging means is for imaging a plurality of spots radiated from a projector, and for an image displayed by the display means, a bandpass filter passing light having a wavelength of irradiation light of the spot is removed from an imaging path. A vehicle periphery monitoring device characterized by being imaged by the imaging means . Imaging means for imaging a monitoring area around the vehicle;
Display means for displaying an image taken by the imaging means;
Object position calculation means for calculating the position of the object from the image data imaged by the imaging means,
Obstacle determining means for determining whether the object calculated by the object position calculating means is an obstacle to running of the vehicle,
When it is determined that the obstacle is determined to be an obstacle to the running of the vehicle by the obstacle determining means, the obstacle determining means for marking an obstacle to the captured image displayed on the display means ,
The imaging means captures an image of the monitoring area with two cameras on the left and right,
Assuming that the images captured by one camera are all images drawn on the road surface, calculate the image projected on the image by the other camera,
Obtain a difference image by removing the road surface image by removing the projected image from the image captured by the other camera,
The differential image of the image captured by the other camera and the difference image are overlapped with each other, and an AND is used to calculate the position of the object using an extracted edge portion.
A vehicle periphery monitoring device, characterized in that:

Images