JP2004056219A - Vehicle surroundings monitoring apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a vehicle surroundings monitoring apparatus capable of displaying a marker acting like a proper rough distance standard to a target at a certain height from the ground. <P>SOLUTION: The monitoring apparatus is provided with: a camera unit 1 for picking up a scene behind the vehicle; an arithmetic unit 11 that sets a marker base position on the ground apart from the vehicle tail end by a prescribed distance in three-dimensional coordinates and a renewed marker position resulting from vertically moving the marker base position to have the same height from the ground as that of the target the vehicle is approaching in reverse, and converts the coordinates of the marker base position and the marker position having a certain height from the ground into coordinates of a two-dimensional imaging face viewed from the camera unit 1; a marker generator 12 for generating an on-ground marker image and a marker image having the certain height from the ground on the respective positions subjected to the coordinate conversion; a superimposer 13 for superimposing the image picked up by the camera unit 1 on the image generated by the marker generator and outputting the superimposed image; and a display apparatus 12 for displaying an output of the superimposer. <P>COPYRIGHT: (C)2004,JPO

Description

【００１５】
図５にカメラ座標（ｘ’，ｙ’，ｚ’）と２次元の撮像面座標の関係を示す。撮像面座標（ｘ”，ｚ”）はカメラ装置のＣＣＤ素子撮像面の中心を原点Ｏ”とし、水平方向をｘ”軸、垂直方向をｚ”軸とする。カメラ座標から撮像面座標への座標変換はカメラ装置の焦点距離をｆとすると次式で表される。
ｘ”＝ｆｘ’／ｙ’　　…（５）
ｚ”＝ｆｚ’／ｙ’　　…（６）
そこで図４の３次元座標におけるマーカ画像化対象である例えば距離目安線５２’の一端の座標を（ｘ_Ｍ，ｙ_Ｍ，ｚ_Ｍ）とすると、（４）式から（６）式により撮像画面座標に変換され、次式で表される。
ｘ_Ｍ”＝ｆｘ／（ｙ_Ｍｃｏｓθ_Ｐ−ｚ_Ｍｓｉｎθ_Ｐ）　　　…（７）
ｚ_Ｍ”＝ｆ（ｙ_Ｍｓｉｎθ_Ｐ−（ｚ_Ｍ−ｚ_０）ｃｏｓθ_Ｐ）／（ｙ_Ｍｃｏｓθ_Ｐ−ｚ_Ｍｓｉｎθ_Ｐ）　　　…（８）
（７）式および（８）式において、ｚ_Ｍ＝０とした場合がマーカ画像化対象が地面上のマーカベースの撮像画面座標である。図３に示した延長線５０、予想進路線５１、距離目安線５２は撮像画面座標に変換され、これらのマーカは図６に示すようにディスプレイ２上に延長線３０、予想進路線３１、距離目安線３２として表示される。
同様にして延長線５０、予想進路線５１、距離目安線５２のような地面上のマーカベース位置を任意の地上高位置に垂直に平行移動しｚ＝ｚ_Ｍ（ｚ_Ｍ＞０）に設定した延長線５０’、予想進路線５１’、距離目安線５２’についても、撮像画面座標に変換でき、それを先の図２のように延長線４０、予想進路線４１、距離目安線４２とする。さらに、図２で示すように距離目安線３２、４２を高さ補助線４３で結ぶ。
【００１６】
上記マーカのうち予想進路線３１、４１、距離目安線３２、４２、高さ補助線４３は後進時の旋回半径に応じて可変である。そこで、車両後進時の操舵角の変化ごとにこれらのマーカ画像をその都度座標変換計算しているとマーカ画像の表示が遅れるので、演算部１１で予めテーブルルックアップ方式マーカ画像化データまたは操舵角をパラメータとした相関式の係数の形のマーカ画像化データとして計算して内部メモリ１９に記憶しておく。マーカ発生器１２は演算部１１からの操舵角情報に応じて、内部メモリ１９に記憶されたマーカ画像化データから適切なデータを読み出し、計算を行いリアルタイムの操舵角に対応するマーカ画像を発生する。
【００１７】
すなわち、操舵角を左一杯、直進、右一杯を含めて離散的に複数選び、離散的な操舵角ごとに予想進路線３１、４１、距離目安線３２、４２、高さ補助線４３を計算してマーカ画像化データとして、例えば１つのマーカに対して各直線部の起点、折れ点と終点の撮像面座標として記憶しておく。マーカ発生器１２は車両運転時の実際の操舵角に応じて、最寄りの予め計算された離散的な操舵角のマーカ画像化データを内部メモリ１９から読み出し、マーカ発生器１２で実際の操舵角に応じた位置のマーカ画像とするため内挿、外挿計算をしてマーカ画像を発生する。
【００１８】
先の図２は、これまで説明したマーカ発生の要領にしたがってカメラ装置１の画像にマーカ画像を重畳したディスプレイ２に表示される画像を示す。図２は地面上のマーカおよび地上高のあるマーカの両方を表示した例であり、この図では操舵角は０度であり延長線３０と予想進路線３１が重なっており、また延長線４０と予想進路線４１も重なっている。
ここで、地面上に対応するマーカとしての延長線３０、予想進路線３１、距離目安線３２と、地上高のあるマーカとしての延長線４０、予想進路線４１、距離目安線４２を異なる色で表示する。また、高さ補助線４３はこれらと異なる色で表示する。
さらに、マーカ画像に遠近感をつけるため、カメラ座標の原点に近いマーカの部位ほど太く、遠いマーカ部位ほど細く表示することにより、全体としてマーカ画像を遠近感、立体感のあるものとする。
【００１９】
上記第１の実施例において、カメラ装置１は本発明における撮像手段に、演算部１１は演算手段に、ディスプレイ２は表示手段に対応する。
【００２０】
次に本実施例における作用について説明する。図７は車両２０が後方に停止している他の車両２２に対して、後方に直進接近し、バックドア２１を開放できる距離を確保して駐車したい場合の比較説明図である。図中、車両２０のバックドア２１を開いた状態を破線で示している。図７の（ａ）は、地面上に対応するマーカのみ画面表示される場合を示す。距離Ｌ_Ｂはバックドア開放に必要な他車との距離を示す。Ａは車両２０後端から距離Ｌ_Ｂ後方の地面上のマーカである距離目安線３２の示す実際の位置を示し、Ｃは後進時に運転者が目標としている他の車両のバンパー位置を示す。ディスプレイ画面表示上で、距離目安線３２が目標Ｃの画像部位と重なるまで後進すると、実際には位置Ａは目標Ｃを通り過ぎ目標Ｃの下にもぐりこんでおり，距離Ｌ_Ｂを確保出来ていない。
【００２１】
これに対し図７の（ｂ）は地面上に対応するマーカおよび地上高のあるマーカの両方が画面表示される場合を示す。Ｂは車両２０後端から距離Ｌ_Ｂ後方の他の車両２２のバンパー高さＨ_Ｂに対応するマーカである、距離目安線４２の示す実際の位置を示す。ディスプレイ画面表示上で、目標Ｃの画像部位と距離目安線４２が重なった時に、位置Ｂは目標Ｃと同じ位置にあり、距離Ｌ_Ｂが確保できている。
【００２２】
本実施例によれば、地上高のあるマーカが地面上に対応するマーカとは別に表示されているので、地面からの高さのある目標に対して、２次元のディスプレイ画面上で相対距離を把握しやすい。特に、図２の例では、鉛直マーカである高さ補助線４３を距離目安線３２、４２の両端を結ぶように追加したので、マーカ画像が立体的に認識しやすくなっている。
さらに、距離目安線３２、４２を車両後部からの単位距離例えば１ｍごとに表示することによって、地上高のある目標部位との水平距離が後進運転中に認識しやすくなる。
【００２３】
図７および図２は、車両を後方に直進させるときの例であるが、図３のように後進しながらハンドルを切り他の車両を避ける場合にも、同様の効果がある。すなわち、予想進路線３１、４１と距離目安線３２、４２および高さ補助線４３が表示され、予想進路線４１は目標としている後方車両のバンパー高さと同じ高さを示し、かつ車両２０の後部車体の左右外側線を示しているので、この予想進路線４１が後方車両のバンパーより手前側に表示されていれば、この後方車両を避けて通過できることが把握できる。
【００２４】
一方、地上高のある目標、例えば他の車両のバンパー高さＨ_Ｂは車型によって異なる。これに対応するため、変形例として高さｚ_Ｍを運転者が可変に設定できるようにすることができる。
高さｚ_Ｍを運転者が可変に設定するため、ＭＣＵ４に接続された操作スイッチ６のマーカデータ入力モードを選択し、データ入力部８から予め複数の高さ設定データを入力し、演算部１１で予め個々の高さごとの地上高のあるマーカ（延長線４０、予想進路線４１、距離目安線４２、高さ補助線４３）のマーカ画像化データを計算し内部メモリ１９に記憶させる。操作スイッチ６にマーカ地上高選択機能をもたせ、例えばロータリ型選択スイッチのそれぞれの地上高選択位置に対応して記憶しておく。例えば（１）大型トラック用、（２）小型トラック・ＳＵＶ用、（３）乗用車用と３段階の設定ができるようにし、各々地上高が０．６ｍ、０．５ｍ、０．４ｍとすると、ほとんどの車両に対して対応できる。
通常使用時、運転者は操作スイッチ６のロータリ型選択スイッチを操作して所定のマーカ地上高を選択することにより、演算部１１はマーカ発生器１２に選択されたマーカ高さのデータを出力する。
これにより、目標とする他の車両の部位の地上高に応じた適切なマーカがディスプレイに表示され、地上高のある目標に対する、より正確な距離が把握できる。
【００２５】
また、高さｚ_Ｍの設定を運転者が可変に設定することもできる。
この場合、マーカの地上高を複数予め設定する前述の変形例に従い、延長線４０、予想進路線４１、距離目安線４２、高さ補助線４３を予め設定した地上高ごとに、前述のような操作で演算部１１で計算し、操作スイッチ６のロータリ型選択スイッチのマーカ高さごとに対応してマーカ画像データとして記憶しておいたものを利用する。操作スイッチ６のロータリ型選択スイッチにマーカ高さとして前述の（１）、（２）、（３）の位置に加えて、（４）「高さ入力」の選択位置を用意し、データ入力部８からマーカ地上高ｚ_Ｍを入力できるようにする。マーカ発生器１２は運転手が入力した地上高ｚ_Ｍに最寄りのマーカ画像化データを、内部メモリ１９から読み出し、目的の地上高のマーカ画像化データに内挿、外挿計算してマーカ画像を発生する。
これにより、運転者がディスプレイ２の画像中の目標対象としたいものが任意の地上高を有している場合に、より正確な地上高のマーカを発生できて、ディスプレイ画面上での距離把握がしやすくなる。
【００２６】
次に、第２の実施例について説明する。
本実施例では図１および図８に示すように、超音波センサを用いた物体検出器９を、後部バンパーのコーナ付近に配置してある。この物体検出器９はある程度の広がりをもって後方に音波を放射し、後方障害物からの反射信号を受信する。物体検出器９の受信信号は演算部１１に送られる。演算部１１は、物体検出器９の受信信号に対し所定の閾値を設けて、閾値より高い受信信号がないときは、車両２０の後方には障害物がないと判断する。演算部１１は、判断結果をマーカ発生器１２に送り地上高のあるマーカを発生させず延長線３０、予想進路線３１、距離目安線３２だけをカメラ装置１の撮影画像に重畳して表示する。
逆に、物体検出器９の受信信号によって、演算部１１が閾値より高い応答があると判断したときは、地上高のあるマーカ（延長線４０、予想進路線４１、距離目安線４２、高さ補助線４３）の内少なくとも予想進路線４１、距離目安線４２は延長線３０、予想進路線３１、距離目安線３２と共にマーカ発生させる。その他の構成は第１の実施例と同じである。
ここで、超音波センサの代わりに、超短波センサを用いてもよい。
本実施例では物体検出器９が本発明の物体検出手段に対応する。
【００２７】
第２の実施例によれば、車両２０の後方に物体が存在せず地面上の目標物を頼って後進する場合、地面上のマーカのみが表示され、地上高のあるマーカは表示されないので後方監視画像が見やすい。また、後方に物体がある場合のみ地上高のあるマーカも自動的に表示され後方監視画像が見やすくなる。
【００２８】
第３の実施例を説明する。この実施例では、第２の実施例における物体検出器９の代わりに測距センサ９’を演算部１１に接続してある。
図９に示すように後方監視用のカメラ装置１と同一場所に例えばレーザーレーダまたはフェーズドアレイレーダのようなビーム状に電磁波を放射して反射波を受信する測距センサ９’を設置する。測距センサ９’の受信信号は演算部１１に送られる。測距センサ９’はカメラ装置１の水平視野範囲内で、水平方向に例えば中央後方を含む右後方２方向、左後方２方向の計５方位を設定し、カメラ装置１の垂直視野範囲と略同等で車両２０の車体を含まない範囲でビーム２８を縦方向に走査し、一番左側後方から順次右後方へ選択方位を切換えて縦方向の走査を繰り返す。
【００２９】
演算部１１は、後方各方位の縦方向走査に対して、測距センサ９’の受信信号が連続的な距離の変化を示し、所定の閾値を越える距離のジャンプがない場合は、図９に示すような地面からのみの応答、または図１０に示すような地面から立ち上がった壁２９のような構造物と判断する。その場合、演算部１１は延長線３０、予想進路線３１、距離目安線３２だけをカメラ装置１からの画像に重畳して表示するようにマーカ発生器１２を制御する。
【００３０】
一方、後方に他の車両２２が存在する場合は、図１１に示すように後方各方位の縦方向走査の少なくとも１つで、測距センサ９’の受信信号において所定の閾値を越えた距離のジャンプを示す。例えばバンパーの下端を境として上側のビーム２８ａと下側のビーム２８ｂは、測距センサ９’の受信信号に対応する距離が不連続なジャンプをする。これにより車両２０の後方に地面から離れて高さを有する物体が存在することを検知できる。この時、演算部１１は測距センサ９’からの受信信号に対応する距離のジャンプを生じたときの縦方向走査時のビーム角度から図１１中の角度θを求め、距離のジャンプ量Ｌ_ＪとからＬ_Ｊｓｉｎθを計算すればバンパー高さＨ_Ｂが求まる。この結果に基づき、演算部１１は表示すべき地上高のあるマーカ設定高さＨ_Ｂをマーカ発生器１２に指令し、マーカ発生器１２はＨ_Ｂに応じて自動的にマーカ（延長線４０、予想進路線４１、距離目安線４２、高さ補助線４３）を前述の第１の実施例の変形例の場合と同様に内部メモリ１９に記憶されたマーカ画像化データから内挿、外挿により発生させる。
この時、延長線４０、予想進路線４１、距離目安線４２のみを表示し、このマーカが地上高のあるマーカであることを示すために地面上のマーカとは異なる色で表示する。
本実施例では測距センサ９’が本発明の距離検出手段に対応する。
【００３１】
第３の実施例によれば、ディスプレイ２に車両２０の後方に、距離センサ９’の受信信号から距離のジャンプが検出される、すなわち地上高のある障害物がある場合だけ、地上高のあるマーカだけを表示する。後方に障害物がないか距離のジャンプを生じるような障害物がない場合は地面上のマーカだけを表示する。
その結果、自動的にマーカ表示を切換えて見やすくすることができる。さらに、地上高のあるマーカ表示のとき、測定した障害物の高さに応じた地上高マーカとなるので、距離把握が正確にできる。
【００３２】
本実施例では、地上高のある障害物がある場合、地上高のあるマーカだけを表示するとしているが、第２の実施例のように、地上高のあるマーカ（延長線４０、予想進路線４１、距離目安線４２）の内少なくとも予想進路線４１、距離目安線４２と地面上のマーカ（延長線３０、予想進路線３１、距離目安線３２）および高さ補助線４３を共にマーカ発生させて、地面上のマーカとは色を変えてもよい。
【００３３】
第４の実施例として左前方監視について説明する。以下に構成作用は第１の実施例と異なる点のみを説明する。ここでは、右ハンドル車において、運転席から見にくい左前方を撮影するものとする。
カメラ装置１は図１に示した第１の実施例の車体後部の代りに、左ドア２５ａに取り付けられたミラー２４ａのケース内に装着されている。カメラ装置１は図示しないレンズを車体左の前輪周辺部を含む前方を撮影可能に向けてあり、ドアミラーケースの図示しない撮影窓にはガラスまたは透明樹脂のシールドを設けてカメラ装置１を保護し、レンズに水滴や汚れが付着しないようにしてある。演算部１１はマーカ発生器１２を制御して、左前方監視用マーカを発生させる。
【００３４】
図１２は本実施例によるマーカ画像化対象を示す。
マーカ画像化対象の第１は車両２０の左車体側端線５４で、ドアミラー２４ａの左先端から鉛直に下ろした地面上の位置を通り、車体前後方向と平行に引いた直線であり、この直線の前端は車両先端から余裕代としてさらに例えば１０ｃｍ前方に伸ばした位置としてある。
さらに第２のマーカ画像化対象の線は、この左車体側端線５４の前端から、右側に直角に短い直線を引きこれを左角車体先端線５５と呼ぶ。
第３のマーカ画像化対象は左停車余地ガイド線５６で、左側サイドドア全開時の先端から鉛直に下ろした地面上の位置を通り、車体前後方向と平行に前方に引いた直線である。
【００３５】
これらの３次元座標の原点はカメラ装置１の鉛直下の地面であり，カメラ装置１のレンズ位置を３次元座標で（０，０，ｚ_０）とし、３次元座標のＸ軸は運転席側が正、Ｙ軸は前進方向が正、Ｚ軸は地面より上方を正とする。カメラレンズの光軸は（０，０，ｚ_０）の位置にありＸ’軸回りのピッチ角θ_Ｐ（上側を正）であり、Ｚ軸回りのヨー角θ_Ｙ（右側を正）で、カメラは光軸周りのロール角を０度とする。ここで、カメラ装置１は車体前方方向よりやや右側に向いており，車両２０の左先端部を映すのに適するように固定されている。カメラ座標（ｘ’，ｙ’，ｚ’）はカメラレンズの中心を原点とし、前輪近傍が原点位置に来るようにカメラ装置１の光軸のピッチ角とヨー角を設定し、ｙ’軸はカメラ光軸で前方撮影方向を正とする。
【００３６】
カメラにヨー角θ_Ｙがかかっているので、その点式（４）の行列Ｔが異なるが、３次元座標からカメラの撮像面座標に変換する方法は基本的に第１の実施例と同じである。
こうして、第１の実施例と同様に演算部１１において、地面上に対応したマーカおよび地上高のあるマーカのマーカ画像化データを予め計算し、内部メモリ１９に記憶する。演算部１１の制御によってマーカ発生器１２が内部メモリ１９からデータを読み出し、マーカを生成する。
【００３７】
図１４は、図１３に示すように縦列駐車した状態から前方に停車している車両の後部をかすめて右側に出ようとする場合のマーカ例を示す。すなわち、ディスプレイには図１２に示した地面上の左車体側端線５４に対応する左車体側端線３４、左角車体先端線５５に対応する左角車体先端線３５、左停車余地ガイド線５６に対応する左停車余地ガイド線３６と、それらの線と平行に所定の高さＨ_Ｂ位置に垂直に移動したそれぞれの線に対応するマーカ（左車体側端線４４、左角車体先端線４５、左停車余地ガイド線４６）が表示される。さらに立体的に見やすいように左車体側端線３４と４４をおよび左停車余地ガイド線３６と４６をそれぞれ高さ補助線４７で結んでいる。
【００３８】
図１３においてＡ’はディスプレイ上の左角車体先端線３５に対応する実際の位置を示し、Ｂ’は左角車体先端線４５に対応する実際の位置を示す。前方に停車している他の車両２２のバンパーの高さ位置Ｈ_Ｂに対応した左角車体先端線４５のマーカがディスプレイ画面上で、前方の他の車両２２のバンパーより下にあれば、車両２２の後方をクリアできると判断できる。
【００３９】
本実施例は以上のように、２次元画面のディスプレイ上で、目標物と同じ地上高のあるマーカを表示しているので、画面上で目標との相対位置が容易に把握できる。
【００４０】
また、左停車余地ガイド線３６、４６を参照して左側に車体を寄せれば、左側の地上高のある物体（例えば他の車両）との間隔を、サイドドア２５ａ全開の余地を残して、最低限の距離で駐停車が可能となる。
【００４１】
地上高のあるマーカの高さ設定については、第１の実施例の２つの変形例と同様に予め設定した地上高からの選択、または任意の地上高入力も可能である。
その場合、地上高のあるマーカの高さ設定を選択可能または任意の数値入力可能とすることによって、より一層、ディスプレイ画面で精度の高い目標との距離把握ができる。
【００４２】
なお、本発明の第１の実施例から第３の実施例のいずれかと、第４の実施例とを組み合わせることもできる。
その場合、図１においてギアシフトポジションセンサ５は映像切換器３だけでなく、演算部１１にも接続され、変速ギア後進段を検知したとき、マーカ発生器１２に後方監視用のマーカを発生するように制御させ、カメラ画像を後方監視用カメラ装置からのものに切換える。また、後方監視用のカメラ装置と左前方監視用のカメラ装置の画像も切換えてスーパーインポーザ１３に送られる。さらには、演算部１１に接続された操作スイッチ６を用いて後方監視／左前方監視を切換え、演算部１１を介してカメラ装置の画像を切換えることができる。
これによって，本発明の後方監視と左前方監視を必要に応じて切換使用できる車両周辺監視装置が実現できる。
【図面の簡単な説明】
【図１】本発明の第１の実施例の全体構成を示す図である。
【図２】車両後方画像にマーカを重ね合わせた監視画像を示す図である。
【図３】後輪の予測通過軌跡の算出要領を示す説明図である。
【図４】３次元座標とカメラ座標の関係の説明図である。
【図５】カメラ座標と撮像面座標の関係の説明図である。
【図６】マーカ発生器で生成する地面上のマーカ画像例を示す説明図である。
【図７】第１の実施例の作用を示す比較説明図である。
【図８】第２の実施例を示す説明図である。
【図９】第３の実施例において、測距センサの地面からの反射信号を示す図である。
【図１０】第３の実施例において、測距センサの地面と壁からの反射信号を示す図である。
【図１１】第３の実施例において、測距センサの後方に車両が存在する場合の、反射信号を示す図である。
【図１２】第４の実施例におけるマーカ画像化対象を説明する図である。
【図１３】第４の実施例の作用説明図である。
【図１４】第４の実施例における監視画像を示す図である。
【符号の説明】
１　　　　カメラ装置
２　　　　ディスプレイ
３　　　　映像切換器
４　　　　モニタコントロールユニット
５　　　　ギアシフトポジションセンサ
６　　　　操作スイッチ
７　　　　操舵角センサ
８　　　　データ入力部
９　　　　物体検出器
９’　　　測距センサ
１１　　　演算部
１２　　　マーカ発生器
１３　　　スーパーインポーザ
１５　　　映像切換スイッチ
１６　　　ナビゲーションモジュール
１７　　　ＴＶチューナ
１８　　　ナビゲーション装置
１９　　　内部メモリ
２０　　　車両
２１　　　バックドア
２２　　　他の車両
２３　　　ステアリング
２４ａ　　ドアミラー
２５ａ　　サイドドア
２８、２８ａ、２８ｂ　　ビーム
２９　　　壁
３０、４０　　　延長線
３１、４１　　　予想進路線
３２、４２　　　距離目安線
３４、４４　　　左車体側端線
３５、４５　　　左角車体先端線
３６、４６　　　左停車余地ガイド線
４３、４７　　　高さ補助線
５０、５０’　　延長線
５１、５１’　　予想進路線
５２、５２’　　距離目安線
５４　　　左車体側端線
５５　　　左角車体先端線
５６　　　左停車余地ガイド線[0001]
TECHNICAL FIELD OF THE INVENTION
BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle periphery monitoring device that displays a situation around a vehicle that is difficult for a driver to see on a display device in the vehicle.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, various vehicle rear monitoring systems are known in which a camera is installed at the rear of a vehicle and an image of the rear that is hardly seen by a driver is displayed on a monitor screen in the vehicle. For example, the vehicle rear monitoring device disclosed in Japanese Patent Laid-Open No. 7-239999 switches the display on the display to an image behind the vehicle when the shift lever is operated to the back position, and furthermore, stores the stored marking points. The vehicle can be set back as a landmark.
[0003]
[Problems to be solved by the invention]
By the way, in the above-described vehicle rear monitoring device, all the marking points are indications for a predetermined distance behind the rear end position of the vehicle body on the ground. If the vehicle is located away from the ground such as the bumper of another vehicle behind the vehicle, that is, if the vehicle approaches a target at a certain height above the ground, the illusion that the marking point in the image is farther from the target than the actual position. Therefore, there is a possibility that appropriate judgment cannot be obtained.
SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to provide a vehicle periphery monitoring device capable of displaying a marking point that is an appropriate distance guide to a target having a ground clearance in view of the above problem.
[0004]
[Means for Solving the Problems]
For this reason, the present invention sets imaging means for photographing the periphery of a vehicle, and a ground height marker position having a predetermined ground height at a predetermined position in a horizontal direction with respect to the vehicle in three-dimensional coordinates, and Calculating means for converting the coordinates into two-dimensional imaging plane coordinates viewed from the imaging means, a marker generator for generating a ground height marker image at the coordinate-converted ground height marker position, and an image captured by the imaging means A superimposer for superimposing and outputting an image and an image generated by a marker generator is provided, and display means for displaying an output of the superimposer.
[0005]
【The invention's effect】
According to the present invention, a ground height marker position having a predetermined ground height at a predetermined position in the horizontal direction with respect to the vehicle in three-dimensional coordinates is coordinate-converted into a two-dimensional imaging plane coordinate as viewed from the imaging means, and there is a certain ground height. Since the marker is displayed as a marker, when the driver drives the vehicle while watching the target having the ground height in the image of the display means, the driver directly compares the position of the target ground height with the height of the corresponding ground height marker. The distance can be easily grasped.
[0006]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing a configuration of a first embodiment in which the rear of a vehicle is projected by a camera device and the image is displayed on a display installed in the vehicle. In FIG. 1, the image of the vehicle body is shown by broken lines to show the installation positions of the main parts.
In the present embodiment, for example, a liquid crystal display device as the display 2 shared with the navigation device 18 and the TV tuner 17 is disposed at the front of the vehicle compartment such as an instrument panel.
The camera device 1 is installed in the rear part of the vehicle 20 at substantially the center from the middle to the upper part.
The camera device 1 has a lens (not shown) directed obliquely downward so that the rear surface of the vehicle body can be photographed, and a glass or transparent resin shield is provided on a shooting window of a case (not shown) of the camera device 1 to protect the camera device 1. Water droplets and dirt do not adhere to the lens. The camera device 1 uses, for example, a CCD element on an imaging surface (not shown). The camera device 1 is connected to the superimposer 13.
[0007]
The steering angle sensor 7 provided on the steering wheel 23 of the vehicle and detecting the steering angle is connected to the calculation unit 11, and the calculation unit 11 controls the marker generator 12. The operation unit 11 is connected to a rotary-type, push-button-type or touch-panel-type operation switch 6 for control thereof, and a data input unit 8 of a numeric keyboard for data input.
The marker generator 12 is connected to the superimposer 13, and the superimposer 13 superimposes the camera image from the camera device 1 and the marker image from the marker generator 12 to form a monitoring image.
The arithmetic unit 11, the marker generator 12, and the superimposer 13 are configured using a microcomputer as the monitor control unit (hereinafter, referred to as "MCU") 4.
[0008]
The operation switch 6 connected to the calculation unit 11 has a marker data input mode selection function in addition to the power on / off function of the MCU 4. When the camera device 1 is installed in a vehicle in the marker data input mode selection state, the three-dimensional coordinate data of various markers generated by the marker generator 12 from the keyboard of the data input unit 8 and the camera device 1 will be described later. Position (0,0, z ₀ ), Optical axis pitch angle θ _P , Yaw angle θ _Y And the focal length f of the camera.
The calculation unit 11 performs coordinate conversion calculation as described later and stores the marker image data in the internal memory 19.
The marker generator 12 reads out marker imaging data from the internal memory 19 and generates a specific marker image in order to generate a designated marker according to a control command from the arithmetic unit 11.
[0009]
In the navigation device 18, the superimposer 13, the navigation module 16 and the TV tuner 17 are connected to the video switch 3 connected to the display 2, and the video switch 3 is connected to the image signal S <b> 1 from the super From the navigation information and the image signal S2 from the module 16 or the image signal S3 from the TV tuner 17, 1 is selected and transmitted to the display 2. That is, the display 2 serves as display means of the vehicle periphery monitoring device when displaying an image from the superimposer 13, and serves as display means of the navigation device 18 when displaying an image or the like from the navigation module 16, and an image from the TV tuner 17. Is displayed as TV display means.
The image switch 3 is connected to a rotary type, push button type or touch panel type operation switch 15 for control thereof and a gear shift position sensor 5 for detecting a reverse gear of the transmission.
[0010]
The operation of monitoring the rear of the vehicle using an image captured by the camera device 1 will be described. Here, the camera device 1 will be described on the assumption that a clear image is displayed on the display screen of the display 2 irrespective of the distance from the lens without influence of lens aberration or the like.
When the gear shift position sensor 5 detects the reverse gear of the transmission gear, the image switch 3 of the navigation device 18 outputs the image from the super imposer 13 irrespective of whether the navigation information, the image or the TV image is selected by the operation switch 15. Switch to.
The marker generator 12 of the MCU 4 uses the marker image data in the internal memory 19 as a marker on the ground, as an on-ground marker, an extension line 30 which is a straight extension of the left and right rear ends of the vehicle 20 to the rear of the vehicle body as shown in FIG. In accordance with the signal from 7, a variable guide line 31 at the left and right rear ends of the variable and a reference distance line 32 connecting the predicted track line 31 between the left and right are generated. Further, the marker generator 12 sets an extension line 40 corresponding to a predetermined height position from the ground, an expected course line 41, , A distance reference line 42 connecting the predicted course lines 41 and a height auxiliary line 43 connecting the predicted course lines 31 and 41 vertically. A marker image on which these markers are drawn is sent to the super imposer 13.
[0011]
Here, the marker generator 12 uses the marker imaging data stored in the internal memory 19 as it is for a fixed marker set in advance, for example, for the extension line 30. However, for example, the predicted course lines 31 and 41, the distance reference lines 32 and 42, and the height auxiliary line 43 displayed according to the signal of the steering angle sensor 7 are discretely calculated by the arithmetic unit 11 in advance when the camera is installed as described later. Marker imaging data corresponding to a typical steering angle is calculated and stored in the internal memory 19. The marker generator 12 reads out the marker imaging data of the nearest steering angle stored in advance from the internal memory 19 according to the actual steering angle at the time of driving the vehicle, and performs interpolation and extrapolation calculation to generate a marker image. I do.
[0012]
A method for converting the three-dimensional coordinates of the marker imaging target into the two-dimensional imaging screen coordinates when displayed on the display 2 in the present embodiment will be described in further detail. These coordinate conversion calculations are performed in the arithmetic unit 11 by switching the MCU 4 to the data input mode by the operation switch 6 and using the keyboard of the data input unit 8 as described above. First, a case where a marker base position on the ground is set will be described.
A description will be given of the vehicle as viewed from directly above as shown in FIG. Here, there are three types of markers to be formed into the marker images, an extension line 50 extending straight backward in the vehicle width, an expected course line 51 at the left and right rear ends when the vehicle moves backward, and a predetermined distance from the vehicle rear end, for example, 1 m. It is a distance reference line 52 which is a reference for the positions of 2 m and 3 m.
[0013]
The shape and position of the predicted course line 51 and the reference distance line 52 change depending on the steering angle, and are obtained in the following manner.
Now the steering angle is θ _S , Wheel base L _H Assuming that the steering gear ratio is n, the turning radius R at the midpoint between the left and right rear wheels is given by the following equation. Here, when turning right, θ _S Shall be positive.
R = L _H / Tan (θ _S / N) ... (1)
Note that, when turning, the influence of the side slip of the tire occurs at the square of the vehicle speed, but since the vehicle periphery monitoring device is used at a low speed of about 10 km / h or less, this effect can be ignored. When R is positive, the vehicle turns right, and when R is negative, the vehicle turns left. Therefore, if the rear tread is W, the turning radius R at the left rear wheel position is _L , Turning radius R at the right rear wheel position _R Is given by the following equation.
R _L = L _H / Tan (θ _S / N) + W / 2 (2)
R _R = L _H / Tan (θ _S / N) -W / 2 (3)
Next, from the rear end of the vehicle on the short radius side (left rear wheel side in FIG. 3) of the predicted course line 51, the distance guide line 52 is moved to the left and right predicted lines at a place where the circumference of the predicted course line is 1m, 2m, and 3m. It is set as a normal line connecting the routes 51. When the vehicle moves backward just behind, the extension line 50 and the expected course line 51 overlap, and the distance reference line 52 is parallel to the left and right direction of the vehicle body, and the left and right expected course lines 51 are 1 m, 2 m, and 3 m from the rear end of the vehicle. Set to tie.
In FIG. 3 and the equations (2) and (3), the extension line 50 and the predicted course line 51 are described by the wheel width. When forming a marker image, W is set as the vehicle width at the rear of the vehicle.
[0014]
Next, in order to make the marker image target composed of these lines a marker on the ground to be superimposed on the camera image, the image is converted into a screen viewed from the camera device position. That is, the three-dimensional coordinates with the origin directly below the camera device as the origin are converted into the coordinates of the imaging plane. The method will be described with reference to FIGS.
FIG. 4 shows three-dimensional coordinates (x, y, z) and camera coordinates (x ′, y ′, z) having the origin at the ground position directly below the camera device 1 fixed substantially at the center of the vehicle above the rear middle section of the vehicle. ') Shows the relationship. The X-axis of the three-dimensional coordinates is positive on the passenger side, the Y-axis is positive in the reverse direction, and the Z-axis is positive above the ground. Here, regarding the position of the camera device, the lens position is represented by three-dimensional coordinates (0, 0, z). ₀ ), And the optical axis of the camera lens is a pitch angle θ around the X ′ axis. _P (The upper side is positive), and the yaw angle around the Z axis and the roll angle around the optical axis are 0 degrees. The camera coordinates (x ′, y ′, z ′) have the origin at the center of the camera lens, the y ′ axis is the camera optical axis, and the backward shooting direction is positive.
At this time, the determinant of the coordinate transformation from the three-dimensional coordinates to the camera coordinates is as follows.
(Equation 1)

[0015]
FIG. 5 shows the relationship between the camera coordinates (x ′, y ′, z ′) and the two-dimensional imaging plane coordinates. The imaging plane coordinates (x ", z") are defined as the origin O "at the center of the imaging surface of the CCD device of the camera device, the x" axis in the horizontal direction, and the z "axis in the vertical direction. The coordinate transformation is represented by the following equation, where f is the focal length of the camera device.
x ″ = fx ′ / y ′ (5)
z ″ = fz ′ / y ′ (6)
Therefore, for example, the coordinates of one end of the distance reference line 52 ′ to be a marker image in the three-dimensional coordinates of FIG. _M , Y _M , Z _M ), The coordinates are converted into the imaging screen coordinates from the equations (4) to (6), and are represented by the following equations.
x _M "= Fx / (y _M cos θ _P -Z _M sin θ _P …… (7)
z _M "= F (y _M sin θ _P − (Z _M -Z ₀ ) Cosθ _P ) / (Y _M cos θ _P -Z _M sin θ _P …… (8)
In equations (7) and (8), z _M In the case where = 0, the marker imaging target is a marker-based imaging screen coordinate on the ground. The extension line 50, the predicted course line 51, and the distance reference line 52 shown in FIG. 3 are converted into the coordinates of the image pickup screen, and these markers are displayed on the display 2 as shown in FIG. It is displayed as a reference line 32.
Similarly, the marker base position on the ground such as the extension line 50, the predicted course line 51, and the distance reference line 52 is vertically translated to an arbitrary ground height position, and z = z _M (Z _M The extended line 50 ′, the predicted course line 51 ′, and the distance reference line 52 ′ set to> 0) can also be converted into the image-capturing screen coordinates, which can be converted into the extended line 40, the predicted course line 41, as shown in FIG. The distance reference line 42 is used. Further, as shown in FIG. 2, the distance reference lines 32 and 42 are connected by a height auxiliary line 43.
[0016]
Among the markers, the expected course lines 31 and 41, the distance reference lines 32 and 42, and the height auxiliary line 43 are variable according to the turning radius at the time of reversing. Therefore, if the coordinate conversion is calculated for each of these marker images each time the steering angle changes when the vehicle is moving backward, the display of the marker images is delayed. Is calculated as marker imaging data in the form of a coefficient of a correlation equation using as a parameter, and stored in the internal memory 19. The marker generator 12 reads out appropriate data from the marker imaging data stored in the internal memory 19 in accordance with the steering angle information from the arithmetic unit 11, performs calculation, and generates a marker image corresponding to a real-time steering angle. .
[0017]
That is, a plurality of steering angles are discretely selected including the full left, straight ahead, and full right, and the expected course lines 31, 41, the distance reference lines 32, 42, and the height auxiliary line 43 are calculated for each discrete steering angle. For example, the start point, the break point, and the end point of each straight line portion for one marker are stored as the imaging surface coordinates as marker imaging data. The marker generator 12 reads out the marker imaging data of the nearest previously calculated discrete steering angle from the internal memory 19 in accordance with the actual steering angle at the time of driving the vehicle, and the marker generator 12 converts the marker imaging data to the actual steering angle. Interpolation and extrapolation calculations are performed to generate a marker image at a position corresponding to the marker image.
[0018]
FIG. 2 shows an image displayed on the display 2 in which the marker image is superimposed on the image of the camera device 1 in accordance with the procedure of the marker generation described above. FIG. 2 shows an example in which both a marker on the ground and a marker with a height above the ground are displayed. In this figure, the steering angle is 0 degree, the extension line 30 and the expected course line 31 overlap, and the extension line 40 The expected courses 41 also overlap.
Here, the extension line 30, the predicted course line 31, and the reference distance line 32 as the marker corresponding to the ground, and the extension line 40, the predicted course line 41, and the reference distance line 42 as the marker having the ground clearance are different colors. indicate. The height auxiliary line 43 is displayed in a color different from these.
Furthermore, in order to give a sense of perspective to the marker image, the marker portion closer to the origin of the camera coordinates is displayed thicker and the marker portion farther away is displayed thinner, so that the marker image as a whole has perspective and three-dimensionality.
[0019]
In the first embodiment, the camera device 1 corresponds to the image pickup unit in the present invention, the calculation unit 11 corresponds to the calculation unit, and the display 2 corresponds to the display unit.
[0020]
Next, the operation of the present embodiment will be described. FIG. 7 is a comparative explanatory diagram in the case where the vehicle 20 approaches the other vehicle 22 stopped rearward straight ahead and parks while securing a distance enough to open the back door 21. In the drawing, a state where the back door 21 of the vehicle 20 is opened is indicated by a broken line. FIG. 7A shows a case where only the marker corresponding to the ground is displayed on the screen. Distance L _B Indicates the distance from another vehicle required to open the back door. A is the distance L from the rear end of the vehicle 20 _B The actual position indicated by the distance reference line 32, which is a marker on the ground behind, is shown, and C indicates the bumper position of the other vehicle targeted by the driver during reverse travel. When the distance guide line 32 moves backward on the display screen until the distance reference line 32 overlaps the image portion of the target C, the position A actually passes through the target C and goes under the target C, and the distance L _B Has not been secured.
[0021]
On the other hand, FIG. 7B shows a case where both the marker corresponding to the ground and the marker having the ground clearance are displayed on the screen. B is a distance L from the rear end of the vehicle 20 _B Bumper height H of another vehicle 22 behind _B Indicates the actual position indicated by the distance reference line 42, which is a marker corresponding to. When the image portion of the target C and the distance reference line 42 overlap on the display screen display, the position B is at the same position as the target C and the distance L _B Has been secured.
[0022]
According to the present embodiment, a marker having a height above the ground is displayed separately from a corresponding marker on the ground, so that a relative distance on a two-dimensional display screen with respect to a target having a height above the ground is determined. Easy to grasp. In particular, in the example of FIG. 2, since the height auxiliary line 43 as a vertical marker is added so as to connect both ends of the distance reference lines 32 and 42, the marker image can be easily recognized three-dimensionally.
Furthermore, by displaying the distance reference lines 32 and 42 at a unit distance from the rear of the vehicle, for example, every 1 m, it becomes easier to recognize the horizontal distance to a target part having a ground clearance during reverse driving.
[0023]
FIGS. 7 and 2 show an example in which the vehicle is made to go straight back. However, a similar effect can be obtained when the driver turns the steering wheel while avoiding another vehicle as shown in FIG. That is, the predicted course lines 31 and 41, the distance reference lines 32 and 42, and the height auxiliary line 43 are displayed. The predicted course line 41 indicates the same height as the bumper height of the target rear vehicle and the rear part of the vehicle 20. Since the left and right outer lines of the vehicle body are shown, if the predicted course line 41 is displayed on the front side of the bumper of the rear vehicle, it can be understood that the vehicle can pass by avoiding the rear vehicle.
[0024]
On the other hand, a target having a ground clearance, for example, the bumper height H of another vehicle _B Varies by vehicle type. To cope with this, as a modification, the height z _M Can be variably set by the driver.
Height z _M The driver selects the marker data input mode of the operation switch 6 connected to the MCU 4 and inputs a plurality of height setting data from the data input unit 8 in advance. Marker imaging data of markers (extended line 40, predicted course line 41, distance reference line 42, height auxiliary line 43) having a ground height for each height is calculated and stored in the internal memory 19. The operation switch 6 is provided with a marker ground height selection function, and is stored, for example, corresponding to each ground height selection position of a rotary type selection switch. For example, (1) for large trucks, (2) for small trucks and SUVs, and (3) for passenger cars, three levels can be set, and the ground clearances are 0.6 m, 0.5 m, and 0.4 m, respectively. Compatible with most vehicles.
In normal use, the driver operates the rotary type selection switch of the operation switch 6 to select a predetermined marker ground height, so that the arithmetic unit 11 outputs data of the selected marker height to the marker generator 12. .
As a result, an appropriate marker according to the ground height of another target vehicle part is displayed on the display, and a more accurate distance to a target having a ground height can be grasped.
[0025]
Also, height z _M Can be variably set by the driver.
In this case, according to the above-described modification in which a plurality of ground heights of the markers are set in advance, the extension line 40, the predicted course line 41, the distance reference line 42, and the height auxiliary line 43 are set for each of the predetermined ground heights as described above. The data calculated by the operation unit 11 by operation and stored as marker image data corresponding to each marker height of the rotary type selection switch of the operation switch 6 is used. In addition to the above-mentioned positions (1), (2) and (3) as marker heights on the rotary type selection switch of the operation switch 6, a selection position of (4) "height input" is prepared, and a data input section is provided. Marker ground height z from 8 _M So that you can enter The marker generator 12 calculates the ground height z input by the driver. _M Is read out from the internal memory 19, and the marker image data is generated by interpolation and extrapolation to the target marker height image data.
Thereby, when the driver wants to set the target in the image on the display 2 to have an arbitrary ground clearance, a marker of a more accurate ground clearance can be generated, and the distance on the display screen can be grasped. Easier to do.
[0026]
Next, a second embodiment will be described.
In this embodiment, as shown in FIGS. 1 and 8, an object detector 9 using an ultrasonic sensor is disposed near a corner of a rear bumper. The object detector 9 emits a sound wave backward with a certain spread, and receives a reflected signal from a rear obstacle. The reception signal of the object detector 9 is sent to the calculation unit 11. The calculation unit 11 sets a predetermined threshold value for the reception signal of the object detector 9, and when there is no reception signal higher than the threshold value, determines that there is no obstacle behind the vehicle 20. The calculation unit 11 sends the determination result to the marker generator 12 and superimposes only the extension line 30, the estimated course line 31, and the distance reference line 32 on the image captured by the camera device 1 without generating a marker having a ground clearance. .
Conversely, when the calculation unit 11 determines from the received signal of the object detector 9 that there is a response higher than the threshold, the marker having the ground clearance (the extension line 40, the estimated course line 41, the distance reference line 42, the height Among the auxiliary lines 43), at least the predicted route line 41 and the distance reference line 42 are generated with markers along with the extension line 30, the predicted route line 31, and the distance reference line 32. Other configurations are the same as those of the first embodiment.
Here, an ultrashort wave sensor may be used instead of the ultrasonic sensor.
In this embodiment, the object detector 9 corresponds to the object detecting means of the present invention.
[0027]
According to the second embodiment, when there is no object behind the vehicle 20 and the vehicle 20 moves backward by relying on the target on the ground, only the marker on the ground is displayed, and the marker with the ground height is not displayed. Easy to see surveillance images. Also, only when there is an object behind, a marker having a height above the ground is automatically displayed, so that the rear monitoring image can be easily viewed.
[0028]
A third embodiment will be described. In this embodiment, a distance measuring sensor 9 'is connected to the calculation unit 11 instead of the object detector 9 in the second embodiment.
As shown in FIG. 9, a distance measuring sensor 9 'that radiates electromagnetic waves in a beam form and receives reflected waves, such as a laser radar or a phased array radar, is installed at the same place as the camera device 1 for monitoring the rear. The signal received by the distance measuring sensor 9 ′ is sent to the calculation unit 11. The distance measuring sensor 9 ′ sets a total of five directions in the horizontal direction within the horizontal field of view of the camera device 1, for example, two right rear directions including the center rear and two left rear directions, and is substantially the same as the vertical field range of the camera device 1. The beam 28 is scanned in the vertical direction within a range that does not include the vehicle body of the vehicle 20, and the selected direction is sequentially switched from the leftmost rear to the right rear to repeat the vertical scanning.
[0029]
The arithmetic unit 11 determines that the received signal of the distance measuring sensor 9 ′ indicates a continuous change in distance for the vertical scanning in each rearward direction, and when there is no jump of a distance exceeding a predetermined threshold value, FIG. It is determined that the response is only from the ground as shown, or a structure such as the wall 29 rising from the ground as shown in FIG. In this case, the calculation unit 11 controls the marker generator 12 so that only the extension line 30, the estimated course line 31, and the distance reference line 32 are displayed in a manner superimposed on the image from the camera device 1.
[0030]
On the other hand, when there is another vehicle 22 behind, as shown in FIG. 11, in at least one of the vertical scans in each of the rear directions, the distance of the distance exceeding a predetermined threshold in the reception signal of the distance measurement sensor 9 'is used. Indicates a jump. For example, the upper beam 28a and the lower beam 28b jump from the lower end of the bumper at a distance corresponding to the reception signal of the distance measuring sensor 9 ', which is discontinuous. This makes it possible to detect the presence of an object having a height away from the ground behind the vehicle 20. At this time, the calculation unit 11 obtains the angle θ in FIG. 11 from the beam angle at the time of vertical scanning when a distance jump corresponding to the reception signal from the distance measurement sensor 9 ′ occurs, and the distance jump amount L _J And from L _J If you calculate sinθ, bumper height H _B Is found. Based on this result, the calculation unit 11 calculates a marker set height H having a ground height to be displayed. _B To the marker generator 12, and the marker generator 12 _B (The extension line 40, the predicted course line 41, the distance reference line 42, and the height auxiliary line 43) are automatically stored in the internal memory 19 in the same manner as in the above-described modification of the first embodiment. It is generated by interpolation and extrapolation from the marker image data.
At this time, only the extension line 40, the expected course line 41, and the distance reference line 42 are displayed, and the marker is displayed in a different color from the marker on the ground to indicate that the marker is a marker having a ground clearance.
In this embodiment, the distance measuring sensor 9 'corresponds to the distance detecting means of the present invention.
[0031]
According to the third embodiment, a jump in the distance is detected from the reception signal of the distance sensor 9 ′ behind the vehicle 20 on the display 2, that is, only when there is an obstacle having a height above the ground, the display 2 has a height above the ground. Display only markers. If there is no obstacle behind or an obstacle that may cause a distance jump, only the marker on the ground is displayed.
As a result, the marker display can be automatically switched to make it easier to see. Further, when a marker with a ground height is displayed, the marker is a ground height marker corresponding to the measured height of the obstacle, so that the distance can be accurately grasped.
[0032]
In this embodiment, when there is an obstacle having a ground clearance, only a marker having a ground clearance is displayed. However, as in the second embodiment, a marker having a ground clearance (extension line 40, predicted course line) is used. 41, at least a predicted route line 41, a distance guide line 42, a marker on the ground (extension line 30, a predicted route line 31, a distance guide line 32) and a height auxiliary line 43 are generated. Thus, the color of the marker on the ground may be changed.
[0033]
A description will be given of a left front monitoring as a fourth embodiment. Hereinafter, only the points of the configuration and operation different from those of the first embodiment will be described. Here, it is assumed that, in a right-hand drive vehicle, the left front which is difficult to see from the driver's seat is photographed.
The camera device 1 is mounted in a case of a mirror 24a attached to a left door 25a instead of the rear portion of the vehicle body of the first embodiment shown in FIG. The camera device 1 is provided with a lens (not shown) so that the front of the vehicle body including the left front wheel peripheral portion can be photographed. A glass or transparent resin shield is provided on a photographing window (not shown) of the door mirror case to protect the camera device 1, Water droplets and dirt do not adhere to the lens. The calculation unit 11 controls the marker generator 12 to generate a left front monitoring marker.
[0034]
FIG. 12 shows a marker imaging target according to the present embodiment.
The first to be marker-imaged is a left vehicle body side end line 54 of the vehicle 20, which is a straight line that passes through a position on the ground vertically lowered from the left end of the door mirror 24a and is parallel to the vehicle body front-rear direction. The front end of the vehicle is further extended, for example, 10 cm forward from the front end of the vehicle as a margin.
Further, as the second marker imaging target line, a short straight line is drawn rightward at a right angle from the front end of the left vehicle body side end line 54, and this is referred to as a left corner vehicle body front line 55.
The third marker imaging target is a left parking space guide line 56, which is a straight line that passes through a position on the ground vertically lowered from the tip when the left side door is fully opened, and is drawn forward in parallel with the vehicle body front-rear direction.
[0035]
The origin of these three-dimensional coordinates is the ground directly below the camera device 1, and the lens position of the camera device 1 is represented by three-dimensional coordinates (0, 0, z). ₀ ), The X-axis of the three-dimensional coordinates is positive on the driver's seat side, the Y-axis is positive in the forward direction, and the Z-axis is positive above the ground. The optical axis of the camera lens is (0,0, z ₀ ) And the pitch angle θ around the X 'axis _P (The upper side is positive) and the yaw angle θ about the Z axis _Y (The right is positive), and the camera sets the roll angle around the optical axis to 0 degree. Here, the camera device 1 faces slightly rightward from the front of the vehicle body, and is fixed so as to be suitable for displaying the left front end portion of the vehicle 20. The camera coordinates (x ', y', z ') have the origin at the center of the camera lens, and set the pitch angle and yaw angle of the optical axis of the camera device 1 so that the vicinity of the front wheel is at the origin position. The forward shooting direction in the camera optical axis is positive.
[0036]
Camera yaw angle θ _Y , The matrix T of the point expression (4) is different, but the method of converting the three-dimensional coordinates to the imaging plane coordinates of the camera is basically the same as in the first embodiment.
In this way, similarly to the first embodiment, the arithmetic unit 11 calculates in advance the marker imaging data of the marker corresponding to the ground and the marker having the ground clearance, and stores it in the internal memory 19. The marker generator 12 reads data from the internal memory 19 under the control of the arithmetic unit 11 and generates a marker.
[0037]
FIG. 14 shows an example of a marker in a case where the rear portion of a vehicle stopped forward is grabbed to go to the right side from the state of parallel parking as shown in FIG. That is, the display shows on the display a left vehicle body end line 34 corresponding to the left vehicle body end line 54 on the ground, a left corner vehicle body front line 35 corresponding to the left corner vehicle body front line 55, and a left stop room guide line, as shown in FIG. A left parking guide line 36 corresponding to 56 and a predetermined height H parallel to these lines _B Markers (left vehicle body side end line 44, left corner vehicle body end line 45, left stopping room guide line 46) corresponding to the respective lines that have moved vertically to the position are displayed. Further, the left vehicle body side end lines 34 and 44 and the left parking space guide lines 36 and 46 are connected by height auxiliary lines 47, respectively, so as to be easily viewed three-dimensionally.
[0038]
In FIG. 13, A ′ indicates the actual position corresponding to the left corner vehicle body tip line 35 on the display, and B ′ indicates the actual position corresponding to the left corner vehicle body tip line 45. The height position H of the bumper of another vehicle 22 stopped ahead _B If the marker of the left corner vehicle front end line 45 corresponding to is located below the bumper of another vehicle 22 ahead on the display screen, it can be determined that the rear of the vehicle 22 can be cleared.
[0039]
As described above, in this embodiment, since a marker having the same height as the target is displayed on the display of the two-dimensional screen, the relative position with respect to the target can be easily grasped on the screen.
[0040]
Also, if the vehicle body is moved to the left with reference to the left stopping room guide lines 36 and 46, the space between the vehicle body and the object with a ground clearance on the left (for example, another vehicle) is set, leaving room for the side door 25a to be fully opened. Parking and stopping at a minimum distance are possible.
[0041]
As for the setting of the height of the marker having the ground height, selection from a predetermined ground height or input of an arbitrary ground height is possible as in the two modifications of the first embodiment.
In this case, by setting the height setting of a marker having a ground height that can be selected or an arbitrary numerical value can be input, it is possible to more accurately grasp the distance to the target on the display screen.
[0042]
It should be noted that any one of the first to third embodiments of the present invention can be combined with the fourth embodiment.
In this case, in FIG. 1, the gear shift position sensor 5 is connected not only to the video switching unit 3 but also to the calculation unit 11 so as to generate a backward monitoring marker in the marker generator 12 when detecting the reverse gear of the transmission gear. , And switches the camera image to that from the rear monitoring camera device. The images of the camera device for monitoring the rear and the camera device for monitoring the left front are also switched and sent to the superimposer 13. Further, it is possible to switch between rear monitoring and left front monitoring using the operation switch 6 connected to the arithmetic unit 11, and to switch the image of the camera device via the arithmetic unit 11.
As a result, the vehicle periphery monitoring device according to the present invention can be switched between the rear monitoring and the left front monitoring as required.
[Brief description of the drawings]
FIG. 1 is a diagram showing an overall configuration of a first embodiment of the present invention.
FIG. 2 is a diagram showing a monitoring image in which a marker is superimposed on a vehicle rear image.
FIG. 3 is an explanatory diagram showing a procedure for calculating a predicted passage locus of a rear wheel.
FIG. 4 is an explanatory diagram of a relationship between three-dimensional coordinates and camera coordinates.
FIG. 5 is an explanatory diagram of a relationship between camera coordinates and imaging plane coordinates.
FIG. 6 is an explanatory diagram showing an example of a marker image on the ground generated by a marker generator.
FIG. 7 is a comparative explanatory view showing the operation of the first embodiment.
FIG. 8 is an explanatory diagram showing a second embodiment.
FIG. 9 is a diagram showing a reflection signal from the ground of the distance measuring sensor in the third embodiment.
FIG. 10 is a diagram showing reflection signals from the ground and a wall of the distance measuring sensor in the third embodiment.
FIG. 11 is a diagram showing a reflection signal when a vehicle is present behind a distance measuring sensor in the third embodiment.
FIG. 12 is a diagram illustrating a marker imaging target in a fourth embodiment.
FIG. 13 is an operation explanatory view of the fourth embodiment.
FIG. 14 is a diagram illustrating a monitoring image according to a fourth embodiment.
[Explanation of symbols]
1 Camera device
2 Display
3 Video switch
4 Monitor control unit
5 Gear shift position sensor
6 Operation switch
7 Steering angle sensor
8 Data input section
9 Object detector
9 'ranging sensor
11 Operation part
12 Marker generator
13 Super Imposer
15 Video switch
16 Navigation module
17 TV tuner
18 Navigation device
19 Internal memory
20 vehicles
21 Backdoor
22 Other vehicles
23 Steering
24a door mirror
25a side door
28, 28a, 28b beam
29 walls
30, 40 extension
31, 41 Estimated course
32, 42 Distance guide line
34, 44 Left body side end line
35, 45 Left corner body tip line
36, 46 Left stop room guide line
43, 47 Height auxiliary line
50, 50 'extension
51, 51 'Estimated course
52, 52 'Distance guide line
54 Left body side end line
55 Left corner body tip line
56 Left stop room guide line

Claims (13)

Imaging means for photographing around the vehicle;
A ground height marker position having a predetermined ground height at a predetermined position in the horizontal direction with respect to the vehicle in three-dimensional coordinates is set, and the coordinate of the ground height marker position is converted into a two-dimensional imaging plane coordinate viewed from the imaging means. Computing means for performing
A marker generator that generates a ground height marker image at the coordinate-converted ground height marker position,
A superimposer that superimposes and outputs an image captured by the imaging unit and an image generated by the marker generator,
Display means for displaying an output of the superimposer. The arithmetic means moves the marker base position set on the ground at the predetermined position in the horizontal direction with respect to the vehicle in the three-dimensional coordinates by the predetermined ground height in the vertical direction, and sets the ground height marker position to The vehicle periphery monitoring device according to claim 1, wherein: 3. The vehicle periphery monitoring device according to claim 2, wherein the arithmetic unit sets a plurality of the marker base positions at predetermined intervals along a predicted course of the vehicle. The arithmetic means converts the marker base position into a two-dimensional imaging plane coordinate viewed from the imaging means,
The vehicle surroundings monitoring device according to claim 2, wherein the marker generator further generates a ground marker image at the marker base position after the coordinate conversion. The vehicle periphery monitoring device according to claim 4, wherein the marker generator further generates a height auxiliary line image connecting end points corresponding to upper and lower sides of the ground marker image and the ground height marker image. An object detection unit that sends out a signal in the direction of the predetermined position, receives a signal reflected from the object, and detects the object,
The marker generator generates the ground marker image and the ground height marker image when the reception signal of the object detection unit is equal to or more than a predetermined threshold, and the reception signal of the object detection unit is lower than a predetermined threshold. 6. The vehicle periphery monitoring device according to claim 4, wherein only the on-ground marker image is generated in the case. A signal for scanning up and down in the direction of the predetermined position is transmitted, and a distance detector is provided for detecting a distance to the object by receiving a signal reflected from the object, and the marker generator is detected by the distance detector. When the change in the distance in the scanning direction is larger than a predetermined threshold, at least the ground height marker image is generated, and when the change in the distance detected by the distance detecting means in the scanning direction is equal to or smaller than a predetermined threshold. The vehicle surroundings monitoring device according to claim 4 or 5, wherein only the ground marker image is generated. The vehicle periphery according to any one of claims 1 to 7, wherein the arithmetic unit includes a data input unit, and can input an arbitrary value as the predetermined position and the predetermined ground clearance. Monitoring device. 2. The apparatus according to claim 1, wherein the calculation unit sets a plurality of ground height marker positions, stores the plurality of ground height marker positions in an internal memory, and is capable of selecting an arbitrary ground height marker image position from the stored positions. 3. 8. The vehicle periphery monitoring device according to any one of 8. The vehicle periphery monitoring device according to any one of claims 1 to 9, wherein the predetermined position is a vehicle body side surface. The vehicle periphery monitoring device according to any one of claims 1 to 9, wherein the predetermined position is a position corresponding to a tip of the side door when the side door is fully opened. The vehicle periphery monitoring device according to any one of claims 1 to 9, wherein the predetermined position is a front end or a rear end of the vehicle body. The vehicle periphery monitoring device according to any one of claims 1 to 9, wherein the predetermined position is a position corresponding to a front end of the back door when the back door is fully opened.

Images