JP3747599B2 - Obstacle detection device for vehicle - Google Patents

Description

なお、式（６）において、テンプレートの画像の各画素の輝度値をＡｉｊ、画像Ｂの各画素の輝度値をＢｉｊとする。
ここで、微分画像を用いるのは、検出対象物である先行車は水平エッジをもつものが多く、先行車が背景より強調され検出しやすくするためである。
【００４０】
図１５は、正規化相関法による画像Ａ、Ｂ間のマッチング位置の検出方法の説明図である。
ここでは２台のカメラ１、２をそれぞれのＹ軸が同一線上にのるように配置してあるので、画像Ｂ内において、画像Ａのテンプレート（ウインドウ４０）と同じＸ軸の位置で、探索位置をＹ軸方向に順次ずらした複数の位置検出対象画像を求める。そして、図１５の（ｂ）に取り出して示すテンプレートと（ｃ）に示す画像Ｂから取り出した（ｄ１）から（ｄ３）に示す各位置検出対象画像との比較から、類似度の最も高い位置検出対象画像を求めることになる。
【００４１】
図中、（ｄ２）はテンプレートと類似度最大の位置で検出対象画像を示し、（ｄ１）はそれより上方の位置で、（ｄ３）はそれより下方の位置での検出対象画像を示している。このように画像Ａに対し各ウインドウに対応したウインドウが画像Ｂに作られる。その両ウインドウのＹ軸位置を示すｙａとｙｂはウインドウのマッチング位置となる。
【００４２】
ステップ１０５では、距離演算部８が各ウインドウにおいてウインドウ内に撮像された物体までの距離を式（１）を用いて演算する。物体が図１６に示すように画像Ａと画像Ｂのウインドウ内に同じ位置を占めるので、物体のマッチング位置差は、すなわちウインドウのマッチング位置差（ｙｂ−ｙａ）である。したがってウインドウのマッチング位置を用い、式（１）に代入することによりウインドウ内の物体までの距離が算出される。
【００４３】
上記のような処理を全てのウインドウについて行なって、ウインドウ毎にその内部に撮像されている物体までの距離を求めると、ステップ１０６において連続して同じ距離値が算出されたウインドウのかたまりを検出し、そのウインドウのかたまりが複数ある場合、距離の一番近いものを選んで先行車を含む検出領域として検出する。この検出領域に先行車と同じ距離の路面部分も含まれるため、先行車の距離を示すｙｃより上段に同じ距離が検出されるウインドウのかたまりの両側のウインドウのＸ座標位置内から下段ウインドウを抽出して先行車の領域として検出する。
【００４４】
またここでは先行車が遠く撮像される場合、先行車画像が小さく、ウインドウ位置をもって車両の幅として検出すると、精度が低下する。それを防ぐため、図１７に示すようにウインドウのかたまりの両側にあるウインドウ（ａ）に対して（ｂ）のように細分化した小ウインドウｗｓを用いて上記と同じようにマッチングしてウインドウの距離を算出する。そしてウインドウのかたまりと同じ距離の小ウインドウを用いて先行車として検出する。なお図１７においては右側のウインドウを示している。ｘｃｒはその小ウインドウの位置で車両のサイド位置として検出されたものである。
【００４５】
またウインドウを細分化する代わりに図１８のようなウインドウから縦方向エッジｔ、ｖを検出し、車両のテクスチャ、色または輝度が先行車と同様である否かを判断して先行車の両サイド位置を検出することも可能である。これにより先行車が小さく撮像されても精度を落とさずに先行車の位置特定ができる。
【００４６】
ステップ１０７では、ウインドウのかたまりが検出できたかをチェックし、できなかった場合、先行車なしとしてステップ１０１に戻り、次の画像を入力して上記の処理を繰り返す。ウインドウのかたまりを検出できた場合、先行車ありとしてウインドウで演算された距離値を先行車までの距離検出値として記憶しステップ１０８へ進む。
【００４７】
ステップ１０８では、先行車の走行レーンについての判断を行なう。自車と同一走行レーンであるか否かを走行レーン判断部１０が判断する。これは、図６に示したように、ウインドウのかたまりの距離と同じ位置ｙｃをラインとしてステップ１０２で検出された２つの白線と交わる２つの点を検出する。そしてウインドウのかたまりがこの２つの交点の中にあるかどうかを判断して先行車が自車と同一走行レーンであるか否かを判断する。自車と異なる走行レーンである場合、ステップ１０１へ戻り、上記処理を再び行なう。検出した先行車が自車と同一走行レーンである場合、ステップ１０９へ進む。
【００４８】
ステップ１０９では、先行車の検出距離と同じ位置ｙｃのラインから、ラインと白線の交点ｘｌｌ（左側白線）、ｘｌｒ（右側白線）を検出する。また車両を検出するウインドウのかたまりの両側のウインドウ位置を検出する。それらを式（４）に代入して、走行レーン上の横方向位置を算出する。
上記のように、ステレオ画像処理により、先行車までの距離と走行レーン内での横方向位置を検出することができる。
【００４９】
本実施例は以上のように構成され、ステレオ画像の一方に画像を分割するウインドウを設定し、各ウインドウと最も類似度の高いウインドウの位置を他方の画像から検出し三角測量原理に基づいた距離演算をし、隣接するウインドウから同じ距離演算値が演算されたウインドウのかたまりを先行車として検出する。そして先行車として認識されたウインドウのかたまりと走行レーンの位置を比較して自車と同一走行レーンかの判断を経て先行車の白線に対する横方向位置を検出するので、従来と比べて距離のみでなく横方向位置も算出される。またカメラが１種類で、かつモノクロでも充分に用が足り、処理が簡単になるとともに、検出環境を選ばないという効果が得られる。
【００５０】
次に、本発明の第２の実施例について説明する。
この実施例は、上記第１の実施例で検出された先行車の位置変動を観測してその挙動判断を行なうようにしたものである。
図１９は、本実施例の構成を示すブロック図であり、図９に示した第１の実施例のブロック図に車両挙動判定部１２を設けたものである。車両挙動判定部１２はメモリを有し横方向位置検出部１１で検出された先行車の横方向位置を記憶し、その位置の時間的変動により先行車の挙動を判定する。
【００５１】
またこの実施例では自車走行レーンのほかに隣接走行レーンについても検出するため、その機能を備えた走行レーン判断部１００を用いる。その他は第１の実施例と同様である。
走行レーン判断部１００の内容は以下の処理の流れにおいて説明する。
【００５２】
図２０は上記構成における処理の流れを示すフローチャートである。
ステップ１０１〜ステップ１０７までは第１の実施例と同様に、カメラ１、２からのステレオ画像を入力して画像メモリ３、４に記憶する。白線検出部５が第１の実施例と同様にステレオ画像の一方から自車の左右２本白線を検出しフィッテング処理によりデータ化する。
【００５３】
ウインドウ設定部６によってウインドウ設定された一方の画像をマッチング位置検出部７が正規化相関法により他方の画像から類似度の最も高いウインドウの位置を検出する。先行車検出部９が距離演算部８において距離演算されたウインドウのうち、同じ距離値が算出されかつ隣接するウインドウのかたまりから先行車の領域を検出する。
【００５４】
ステップ２０８において、画像Ａから走行レーン判断部１００がウインドウのかたまりの距離と同じＹ軸位置上の白線位置を検出する。ここでは自車走行レーンのほかに隣車線の白線の位置も検出する。
すなわち図２１に示すように自車が２車線道路を走行している場合、白線が３本であるので、それら全てがカメラ１、２の撮像範囲内にあれば、左車線を走行している場合では、距離Ｚ前方にある白線は、図２２のように先行車が検出されるＹ＝ｙｃライン上にｘｌ、ｘｒ、ｘｒ２の位置に撮像される。先行車とそれらとの位置判断で先行車の自車走行レーンを判定することができる。
図２３は自車走行レーンと右隣接車線上から先行車が同時に撮像されている画像を示す図である。Ｍは自車走行レーン、Ｎは右隣接車線上の先行車である。これらはウインドウのかたまり（太線部分）によって検出される。
【００５５】
ステップ２０８では以下のように先行車の走行レーン判断を行なう。
ラインＹ＝ｙｃと白線の近似線の交点から自車走行レーン上の白線位置ｘｌｌ、ｘｌｒを求める。白線間の距離Ｗｒは通常３．５ｍ程度であることから、図２５のように、距離Ｚ離れたところで右隣接レーンの外側の白線の画像上の位置ｘｌｒは、式（７）により求めることができる。
ｘｌｒ＝ｘｌｌ＋（ｆ×Ｗｒ）／Ｚ （７）
Ｚは車両までの距離検出値、ｆはカメラの焦点距離、Ｗｒは白線幅である。
ここでは、ｘｌｌは自車走行レーン上の右側白線の位置である。
【００５６】
ウインドウのかたまりが自車走行レ−ンの白線内にある場合、自車と同じ走行レ−ン上にあるとして、ステップ２０９へ進む。自車走行レーンでないと判定された場合、走行レーン判断部１００は式（７）に基づいて仮想的に右走行レーンの外側の白線位置ｘｌｒの位置を算出する。ウインドウのかたまりが自車の右白線と仮想白線の間にあるかどうかを判定する。なお、車間距離が近い場合、ｘｌｒは画像の右外に出てしまう場合が考えられが、そのような位置のときは、ｘｌｒがｘｌｌの外側にあることを考慮したうえ、判定を行なう。
そして、判定の結果は、ウインドウのかたまりが右走行レーンにある場合、先行車が右隣接車線上にあり、ステップ２１０へ進む。ウインドウのかたまりが右走行レーンでない場合、先行車が左隣車線上にあるとしてステップ２１１へ進む。
【００５７】
ステップ２０９において、図２４に示すようにＹ＝ｙｃライン上の位置ｘｌｌあるいはｘｌｒに対し、先行車を検出するウインドウのかたまりＭのサイド位置ｘｃｌもしくはｘｃｒからの白線に対する画像距離ｘＬＬ、ｘＬＲのいずれかの一方を算出する。本実施例ではｘＬＲを算出する。
【００５８】
ステップ２１０は、ステップ２０９と同じように右走行レーンの白線Ｒ１とＲ２に対する画像距離ｘＬＬもしくはｘＬＲを図２５から算出する。
ステップ２１１は、図２６のように左走行レーンの白線Ｌ１、Ｌ２に対するｘＬＬもしくはｘＬＲを算出する。
【００５９】
ステップ２１２では、前記式（４）を用いて画像上の横方向位置（ｘＬＬ、ｘＬＲ）から先行車の走行路上の横方向位置（ＬＬ、ＬＲ）を算出しメモリに記憶する。そして、横方向位置の時間的変動により先行車の挙動を判断する。図２７、２８、２９、３０はそれぞれ右白線に対する横方向位置ＬＲの時間的変化をプロットしたグラフである。図上には最小二乗誤差直線が示されている。各横方向位置を最小二乗誤差直線にフィッティングして誤差を求める。誤差が所定値内の場合最小二乗誤差直線の傾きで先行車の挙動を判定する。すなわち図２７では走行にしたがって先行車の右白線に対する横方向位置ＬＲは殆ど変化しないため、レ−ンをキ−プしながら走行していると判断できる。
【００６０】
また図２８では徐々に横方向位置ＬＲが長くなるため左車線への変更と判断できる。
図２９は右車線への変更を示している。
また図３０では最小二乗誤差直線に対して振動が大きいため、蛇行していると判断できる。
【００６１】
本実施例は以上のように構成され、先行車の白線に対する横方向距離および距離変化を検出し挙動判断をするようにしたので、ウインカの点滅を判断しなくても車線変更が検出できる。またウインカの点滅判断では困難な蛇行等の検出も行なえ居眠りなど推定も可能になる。
【００６２】
次に、第３の実施例について説明する。この実施例は図１９に示すブロック図の構成において、先行車検出部９を車両挙動検出部１２に接続したものである。車両挙動検出部１２は第２実施例で説明したように先行車の横方向位置の時間的変化で挙動を判断するが、この実施例ではさらに道路状況認識手段として先行車検出部９の先行車の距離から路面状況を判断する機能が付加されている。
【００６３】
車両挙動検出部１２はまず図３１に示すようにウインドウのかたまりの下段のウインドウＰを一つのウインドウとして取り出す。ウインドウＰを図３２のように縦方向に複数の小ウインドウｗｔに細分し、小ウインドウを用いて先行車の下端位置ｙｃｄを求める。また小ウインドウを設ける代わりに図３３のように横エッジで区分される域内の色や輝度が先行車と一致するかどうかの判断で車両の下端位置ｙｃｄを算出することもできる。ｙｃｄは先行車の下端位置であり、ウインドウのマッチングによって得られた先行車位置ｙｃと同じもので、直接算出することにより精度が高くなっている。
【００６４】
先行車の下端位置ｙｃｄとそのときの車両の距離Ｚが分かれば、図３４に示す撮像原理でカメラ１の光軸１ｂに対する実際の高さｙｄは、式（８）により求められる。
ｙｄ＝ｙｃｄ・Ｚ／ｆ （８）
ｙｄの検出を時間毎に行ない、メモリに保存させる。過去数回分のｙｄの変動を見ることで、先行車の縦方向の運動を認識し道路状況を判断する。
【００６５】
図３５〜４０はｙｄの時間的変化をプロットしたグラフである。
各グラフには第２の実施例と同じように最小二乗誤差直線が求められている。図３５では、最小二乗誤差直線の傾きが０に近いので、前方は平坦道路と判断する。
図３６では、最小二乗誤差直線の傾きが正なので、ｙｄが大きくなって行くことを示し、前方道路が下だり坂と判断する。
【００６６】
図３７では、最小二乗誤差直線の傾きが負なので、ｙｄが小さくなって行くことを示し前方道路が登ぼり坂と判断する。
図３８では、最小二乗誤差直線に対して、ｙｄの誤差が大きいので、前方道路が凸凹道と判断する。
図３９および図４０は、最小二乗誤差直線の傾きが大きく変化するので、前方道路上に石や穴または段差があると判断する。
本実施例は以上のように構成され、先行車の位置と距離によりカメラに対する上下位置を演算し、その位置の変動で路面状況が判断されるから、先行車を検出する際のデータを利用することができ、演算負担にならずに処理できる。
【００６７】
【発明の効果】
以上説明した通り、請求項１によれば入力画像を複数のウインドウに区切り、ウインドウ毎に求めた距離値を元に、同じ距離値が演算される隣接ウインドウのかたまりから先行車の領域を検出するとともに距離を演算する。また片方の画像から白線を検出先行車と白線の位置関係を求め、横方向位置を検出するようにしたので、赤外線カメラなどが不要で、装置の構成が簡単になるとともに先行車の挙動判断を行なうことが可能になる。
そしてさらに、先行車挙動判定手段が設けられるから、先行車の横方向位置の時間的変化が検出され、先行車の挙動判定ができる。例えば左白線や右白線に接近する場合、先行車が車線変更を行なっていると判断できる。またこの判断はウインカの点滅判断によらないため、ウインカの点滅検出も不要で、環境色や輝度に影響されない効果が得られる。ウインカをつけない車線変更を行なう先行車についても判断ができる。
【００６８】
請求項２記載の発明では、道路状況認識手段が設けられ、先行車の検出距離から先行車の画像位置を演算し、画像位置の時間的変化により前方路面状況を認識するようにしたので、例えば先行車の画像位置が下がったときに先行車が下だり坂を走行し始めると判断できる。これによって自車を減速制御するなどの道路状況に対応した車両制御が行なえる。
【００６９】
請求項３記載の発明では、白線を検出する画像領域を先行車より下方の画像領域とすることで、先行車に遮られた遠方白線は検出対象から除外され処理するデータが減少するとともに誤検出が減少される効果が得られる。
【図面の簡単な説明】
【図１】ステレオ画像処理による距離算出手法を説明するための説明図である。
【図２】画素と焦点距離の関係を示す説明図である。
【図３】マッチングによるウインドウ設定の説明図である。
【図４】先行車検出の説明図である。
【図５】距離と画像上の位置関係の説明図である。
【図６】先行車の走行レーン判断の説明図である。
【図７】先行車の横方向位置検出の説明図である。
【図８】画像位置と横方向位置検出の説明図である。
【図９】第１の実施例の構成を示すブロック図である。
【図１０】第１の実施例における処理の流れを示すフローチャートである。
【図１１】白線を検出するための説明図である。
【図１２】ウインドウ設定の説明図である。
【図１３】微分処理の効果を示す図である。
【図１４】先行車を検出ためのウインドウ設定の説明する図である。
【図１５】正規化相関法によるステレオ画像間でのマッチング位置の検出方法を示す説明図である。
【図１６】ウインドウ内の画像とウインドウの位置関係を示す図である。
【図１７】ウインドウを細分化する説明図である。
【図１８】撮像内容による先行車のサイド部の検出説明図である。
【図１９】第２の実施例の構成を示すブロック図である。
【図２０】第２の実施例における処理の流れを示すフローチャートである。
【図２１】隣車線の検出原理を示す図である。
【図２２】隣車線の撮像位置を示す図である。
【図２３】自車走行レーンと隣車線上に先行車が検出された状態を示す図である。
【図２４】自車走行レーン上の先行車の横方向位置検出の説明図である。
【図２５】右隣車線上の先行車の横方向位置検出の説明図である。
【図２６】左隣車線上の先行車の横方向位置検出の説明図である。
【図２７】先行車両がレーンキープ時の横方向位置の変化を示すグラフである。
【図２８】先行車両が左車線へ変更時の横方向位置の変化を示すグラフである。
【図２９】先行車両が左車線へ変更時の横方向位置の変化を示すグラフである。
【図３０】先行車両が蛇行しているときの横方向位置の変化を示すグラフである。
【図３１】先行車の下端位置検出の説明図である。
【図３２】ウインドウを細分化する説明図である。
【図３３】撮像内容による先行車の下端の検出説明図である。
【図３４】先行車の下端位置とカメラの光軸の距離関係を示す図である。
【図３５】先行車両が平坦地を走行しているときのグラフである。
【図３６】先行車両が下り坂を走行しているときのグラフである。
【図３７】先行車両が登り坂を走行しているときのグラフである。
【図３８】先行車両が凹凸な道を走行しているときのグラフである。
【図３９】先行車両が走行している道路上に石等の障害物があるときのグラフである。
【図４０】先行車両が走行している道路上に穴などがあるときのグラフである。
【符号の説明】
１、２ カメラ
１ａ、２ａ ＣＣＤ
１ｂ、２ｂ 光軸
３、４ 画像メモリ
５ 白線検出部（白線検出手段）
６ ウインドウ設定部（ウインドウ設定手段）
７ マッチング位置検出部（マッチング位置検出手段）
８ 距離演算部（距離演算手段）
９ 先行車検出部（先行車検出手段）
１０、１００ 走行レーン判断部（走行レーン判断手段）
１１ 横方向位置検出部（横方向位置検出手段）
１２ 車両挙動判定部（車両挙動判定手段）
Ｃ 先行車
Ｅ ウインドウのかたまり
ｆ 焦点距離
Ｄ 眼間距離
Ｌ１、Ｌ２ レンズ
Ｗ ウインドウ
ｗｄ ウインドウ
Ｗｒ 道路幅
ｙａ、ｙｂ マッチング位置
ｙｃ 先行車の撮像位置
ｘｒ Ｘ軸上のウインドウ設定位置
ｘｗ ウインドウの横幅
ｙｗ ウインドウの縦幅
ｙｒ Ｙ軸上のウインドウ設定位置
Ｚ 撮像距離[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an obstacle detection device for a vehicle that performs position detection and behavior determination of a preceding vehicle by image processing.
[0002]
[Prior art]
Conventionally, as an obstacle detection device for a vehicle, for example, there is one described in Japanese Patent Laid-Open No. 2-300741. This uses a thermal image and a color image to detect obstacles, detects a muffler in the thermal image, and detects a vehicle by detecting an orange or red turn signal from the color image diagonally up and down from the muffler position. It is.
[0003]
[Problems to be solved by the invention]
However, in the conventional apparatus as described above, a special infrared camera is used to obtain a thermal image, and two types of cameras must be used together with a camera that captures a color image. Becomes large and expensive.
Also, if the muffler is not warm, or if the temperature is high and the vehicle body is heated by sunlight, it is difficult to detect the muffler, and the detection position of the orange and red turn signals cannot be determined, and the preceding vehicle cannot be detected. There is a drawback.
[0004]
Furthermore, because the positional relationship between the muffler and the turn signal differs depending on the distance to the vehicle and the type of motorcycle, truck, etc., the turn signal cannot be detected even if the muffler is detected. As a result, the heated part or orange or red The vehicle is detected only by detecting this portion, and the detection result is uncertain.
[0005]
In addition, since the detection of the turn signal is detected by color, it is difficult to detect the turn signal in an orange vehicle. For the same reason, there is a problem that an orange reflector such as a delineator has a color close to that of a winker, so that it is highly likely to be erroneously detected as a winker.
[0006]
In the above conventional example, only the detection of the vehicle is performed, the position on the travel lane of the vehicle is not determined, and the behavior of the vehicle cannot be determined. Even if it is possible to make a determination such as changing the lane by detecting the blinking state of the turn signal, there is a vehicle that changes the lane without turning on the turn signal, and the determination result is uncertain.
Further, since the lane is not detected, it is impossible to determine which lane the preceding vehicle is to change to, so there is a problem that it is not possible to determine whether the preceding vehicle is an obstacle to the traveling of the host vehicle.
[0007]
In addition, even when trying to determine this, no white line appears on the thermal image. The color image is also subjected to contrast adjustment so that the orange and red colors of the blinkers can be easily detected, and the image is difficult to detect the white line, which makes it difficult to determine the driving lane.
In view of the above-described conventional problems, this embodiment provides a vehicle obstacle detection device that does not require a thermal image or a color image, can accurately specify the traveling position of a preceding vehicle, and can perform behavior determination. It is aimed.
[0008]
[Means for Solving the Problems]
  For this reason, the invention described in claim 1 includes two cameras that are arranged with a predetermined interocular distance in which the respective optical axes are parallel to each other,
An image memory for storing road images captured by the two cameras as first and second images;
White line detecting means for detecting a white line in the first image;
Window setting means for setting a window so as to divide a predetermined area in the first image into a plurality of sections;
Matching position detecting means for detecting the position of the window having the highest similarity with each window from the second image;
Distance calculating means for calculating a distance to an object imaged in each window based on the position on the image where a window having a high degree of similarity exists between the two images and the positional relationship between the two cameras; ,
A preceding vehicle detection means for detecting a block of windows in which the same distance value is calculated and the window is adjacent as a detection region of the preceding vehicle and setting the distance value of the window as the detection distance of the preceding vehicle;
Travel lane determining means for determining the travel lane of the preceding vehicle based on the positional relationship between the cluster of windows and the white line;
Lateral position detecting means for detecting the lateral position of a preceding vehicle on the travel lane;,
Preceding vehicle behavior determining means connected to the lateral position detecting means for detecting temporal changes in the lateral position of the preceding vehicle and determining the behavior of the preceding vehicle;It was supposed to have.
[0009]
  The invention according to claim 2Road condition recognition connected to the preceding vehicle detecting means instead of the preceding vehicle behavior determining means, calculating the image position of the preceding vehicle from the detection distance of the preceding vehicle, and recognizing the front road surface condition by temporal change of the image position meansIs assumed to be provided.
  Claim 3In the described invention, the white line detection means sets the image area below the preceding vehicle as the white line detection area.
[0010]
[Action]
In the first aspect of the present invention, the two cameras form a stereo camera, and the first and second images having parallax are captured and stored in the image memory. A white line is detected from the first image by the white line detecting means. The window setting means performs window setting so that the area is divided into predetermined areas in the first image. The matching position detection means detects the position of the window having the highest similarity with each window from the second image by matching processing.
[0011]
The distance calculation means calculates the distance to the object imaged in the window by triangulation based on the position of the window having the highest similarity between the first and second images and the positional relationship between the two cameras. To do. When the same distance value is calculated in adjacent windows, it can be determined that the same object is imaged in those windows, so that the preceding vehicle detection means calculates the same distance value and the windows of adjacent windows are calculated. A cluster is detected as a detection area of the preceding vehicle. The distance value of the window is the detection distance of the preceding vehicle. The traveling lane determining means determines the lane in which the preceding vehicle is traveling based on the positional relationship between the cluster of windows and the white line. The lateral position detecting means detects the position of the white line on the same distance as the preceding vehicle, and detects the lateral position of the preceding vehicle with respect to the white line.
[0012]
Since the preceding vehicle is detected on the stereo image in this way, it is sufficient even for a monochrome image, and heat, the color of the vehicle, etc. are not an influencing factor, and the effect of selecting the use environment can be obtained. In addition, since the white line is detected to determine the driving lane of the preceding vehicle, and the lateral position of the preceding vehicle with respect to the white line is detected, functions such as autonomous driving control and approach warning can be easily built, and the time of the vehicle position By judging the change, the behavior can be judged and higher-dimensional travel control can be performed.
[0013]
  And furthermore,Since the preceding vehicle behavior determining means is provided, a temporal change in the lateral position of the preceding vehicle is detected, and the behavior of the preceding vehicle can be determined. For example, when approaching the left white line or the right white line, it can be determined that the preceding vehicle is changing lanes. Further, since this determination is not based on the blinker blink determination, it is not necessary to detect blinker blinking, and an effect that is not affected by the environmental color or brightness can be obtained. Judgment can also be made for a preceding vehicle that changes lanes without turning the turn signal.
[0014]
  In invention of Claim 2, a road condition recognition means is provided,Since the image position of the preceding vehicle is calculated from the detection distance of the preceding vehicle and the front road surface condition is recognized by the temporal change in the image position, for example, when the image position of the preceding vehicle decreases, the preceding vehicle moves downhill. It can be determined that the car will start running. As a result, vehicle control corresponding to road conditions such as deceleration control of the host vehicle can be performed.
[0015]
  Claim 3In the described invention, the image area for detecting the white line is set as the image area below the preceding vehicle, so that the far white line blocked by the preceding vehicle is excluded from the detection target, and data to be processed is reduced and false detection is reduced. Effects can be obtained.
[0016]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described by way of examples.
Here, the principle of detecting the position of a preceding vehicle that is a front obstacle by image processing of a stereo image obtained by a stereo camera will be described before specifically describing the apparatus according to the embodiment.
[0017]
FIG. 1 is a diagram illustrating a triangulation principle for obtaining a distance Z to a preceding vehicle C using a stereo image obtained by a stereo camera. Here, two cameras 1 and 2 constituting a stereo camera are mounted on the vehicle. The cameras 1 and 2 are CCD cameras each having lenses L1 and L2 having the same focal length f and CCDs 1a and 2a arranged so that the distance from each lens to the imaging surface is the focal length f.
[0018]
In the cameras 1 and 2, the imaging surfaces of the CCDs 1 a and 2 a are positioned in the same vertical plane, and the Y axis that is the vertical reference axis of each imaging surface, that is, the imaging surface axis of the CCD 1 a and the imaging surface axis of the CCD 2 a ( YA, YB) are matched, the optical axes 1b, 2b of the lenses L1, L2 are parallel to each other, and the interocular distance (distance between the optical axes 1b, 2b) D is set up and down to be a predetermined value. Has been.
[0019]
In a stereo image composed of two images obtained by imaging the front of the vehicle with two cameras 1 and 2 whose focal length f and interocular distance D are known and whose optical axes 1b and 2b are parallel to each other, two images If the Y coordinates ya and yb of the matching position (the position of the most similar image in FIG. 1 and the position of the rear edge of the preceding vehicle C in FIG. 1) can be obtained, The distance Z can be obtained from the following equation (1).
Z = f * D / (yb-ya) (1)
Here, the units of f, ya and yb are the pixels of the CCDs 1a and 2a, and the units of D and Z are mm. In general, the focal length f is often expressed in mm, but the focal length f in the equation (1) is calculated in units of pixels.
[0020]
Here, a method of obtaining the focal length f in units of pixels will be described with reference to FIG. In FIG. 2A, an object C ′ having a known width W (mm) is imaged by one of two cameras 1 and 2 (here, camera 1) at a distance Z (mm). (B) of the figure shows the image A obtained at that time. In the following description, the first image captured by the camera 1 is A, and the second image captured by the camera 2 is B.
[0021]
The focal length f in units of pixels is obtained by imaging an object C ′ having a known width W (mm) at a distance Z (mm), and an image A (or image B) obtained at this time. By detecting the width xw (pixel) of the above object C ′ by image processing such as edge detection, it can be obtained by the following equation (2).
f = xw × Z / W (2)
Here, the unit of xw is a pixel, and the units of Z and W are mm.
[0022]
FIG. 3 shows a result obtained by cutting a predetermined area in the image A for each window and obtaining an imaging position of the same image in the image B using a characteristic edge of an object imaged in each window W. FIG. (A) is an image A with a window cut off, and (b) is an image B showing a window from which the same object is obtained.
If the matching position (Y-axis direction) between the A and B images is known for each window, the distances of all objects within the area partitioned into windows can be found by using Equation (1).
[0023]
The distance calculated for each window is the distance to an object having a characteristic part such as an edge imaged in the window, so if the same distance is obtained in adjacent windows, those windows are displayed in those windows. Can determine that the same object is being imaged. Therefore, a group of windows in which the same distance value is detected can be detected as an imaged object.
[0024]
Since the lower part of the preceding vehicle is imaged with its lower part connected to the road surface, as shown in FIG. 3B, the window in which the same distance as the preceding vehicle is detected includes the road surface part (thick line window). In order to remove the road surface portion, the positions of both side windows of the upper window not including the road surface are detected from the cluster of windows in which the same distance is detected as shown in FIG. If the extraction area is determined, a cluster E (thick line window) of a window not including a road surface portion can be detected as a preceding vehicle.
[0025]
Next, detection of the detected lateral position of the preceding vehicle will be described.
FIG. 5 shows a state in which the rear lower edge of the preceding vehicle C is imaged at a distance Z (here, imaged by the camera 1). As can be seen from this figure, the position in the Y-axis direction on the image where the lower edge of the preceding vehicle C detected at the distance Z is imaged can be obtained by equation (3).
yc = [(h−H) · f] / Z (3)
Here, H is the height from the road surface to the lower edge of the preceding vehicle, h is the height from the road surface to the center of the camera lens, f is the focal length, and Z is the inter-vehicle distance.
[0026]
Since the distance Z is detected by window matching, the Y coordinate position yc on the image can be obtained by the above equation (3). By drawing a line of Y = yc on the image as shown in FIG. 6, the image position (Y axis) at the same distance as the preceding vehicle is determined. Therefore, the point (xll, xlr) where the line and the white line intersect becomes the white line position on the same distance as the preceding vehicle. If the positional relationship between the preceding vehicle and the intersection of the white line is determined on the line, it can be determined whether the preceding vehicle is the own vehicle traveling lane or the adjacent traveling lane.
[0027]
The lateral position of the vehicle is determined by the positional relationship with the travel lane. First, as shown in FIG. 7, the intersection positions (xll, yc) and (xlr, yc) of two white lines that intersect the line Y = yc are calculated. Therefore, the distance on the image from the white line to both sides of the preceding vehicle is
The distance xLL with respect to the left white line is calculated by xcl-xll.
The distance xLR with respect to the right white line is calculated by xcr-xlr.
However, xcl and xcr are the X-axis positions of a group of windows indicating the width of the vehicle.
[0028]
Since the preceding vehicle is imaged as shown in FIG. 8, the relationship between the image distances xLL and xLR on the CCD 1a and the actual distances LL and LR on the travel lane is expressed by Expression (4).
LL = xLL · Z / f
LR = xLR · Z / f (4)
Here, f is the focal length, and Z is the inter-vehicle distance. C is a preceding vehicle.
As a result, the lateral position of the preceding vehicle with respect to the white line is detected.
As described above, the distance to the preceding vehicle is detected by the stereo image processing using the stereo image, and the position in the horizontal direction with respect to the white line is specified in the travel lane using one image.
[0029]
FIG. 9 is a block diagram showing the configuration of the first embodiment of the present invention.
The two cameras 1 and 2 constitute a stereo camera, and each of the lenses L1 and L2 having the same focal length f and the CCD 1a arranged so that the distance from each lens to the imaging surface becomes the focal length f. And a CCD camera having 2a. The cameras 1 and 2 are attached to predetermined parts on the vehicle suitable for photographing the front.
As shown in FIG. 1, in the cameras 1 and 2, the imaging surfaces of the CCDs 1a and 2a are located in the same vertical plane, and the vertical reference axes (YA axis and YB axis) of the imaging surfaces coincide with each other. The optical axes 1b and 2b of L1 and L2 are horizontal and parallel to each other, and are arranged side by side with a predetermined interocular distance D.
[0030]
Video signals captured by the cameras 1 and 2 are output and stored in the image memories 3 and 4 as digital images. The white line detection unit 5 performs image processing on the image A from the image memory 3 and detects two white lines surrounding the vehicle. The window setting unit 6 captures the image A from the image memory 3 and sets a plurality of windows of the same size so as to divide the screen into screen portions excluding areas such as the sky where no vehicle can exist.
[0031]
The matching position detection unit 7 superimposes the window on the differential image of the image A, detects the position of the window having the highest similarity with each window from the image B, and stores the matching position of the window between the two images. The distance calculation unit 8 calculates the distance to the imaging object in the window based on the matching position of each window, the focal length f of the lenses of the cameras 1 and 2 and the interocular distance D. The preceding vehicle detection unit 9 calculates the same distance and detects a cluster of adjacent windows as the preceding vehicle.
[0032]
The traveling lane determination unit 10 determines the traveling lane of the preceding vehicle based on the positional relationship on the image between the position of the window cluster and the white line detected by the white line detection unit 5.
The lateral position detection unit 11 detects the position of the white line that is on the same distance as the detection distance of the preceding vehicle. The window positions on both sides of the window cluster are detected, and the lateral distance from the white line is calculated as the positions of both sides of the preceding vehicle. The actual distance on the road is calculated from the distance on the image by the perspective relationship of the camera.
[0033]
FIG. 10 is a flowchart showing the flow of processing of this embodiment.
First, in step 101, stereo images, that is, images A and B output from the cameras 1 and 2, respectively, are input and stored in the image memories 3 and 4, respectively.
In step 102, the white line detection unit 5 performs image processing on one of the images A and B (image A stored in the image memory 3 in this embodiment) to detect a white line.
Here, since it is considered that only the own vehicle traveling lane is detected, two white lines on the left and right sides of the own vehicle are detected.
[0034]
11 and 12 are explanatory diagrams of the principle of white line detection. FIG. 11A is a plan view showing a state when the monocular camera (camera 1) is imaging a white line at the center of the road, and FIG. 11B is a side view thereof. When the vehicle is running between two white lines, the two white lines appear on both the left and right sides of the image. Assuming that the white line ahead is a straight line and the vehicle is running in the middle of the two white lines, as shown in FIG. It can obtain | require by Formula (5) by scientific calculation.
xr = (Wr · f) / (Z · 2), yr = (h · f) / Z (5)
Here, Wr is the width of the road surface, h is the height from the road surface to the center of the camera lens, f is the focal length, Z is the inter-vehicle distance, and xr and yr are the positions on the image.
[0035]
The position xr, yr on the image is obtained for a plurality of distance values Z based on the formula (5), and the window setting position for detecting the white line is determined. At each position, for example, as shown in FIG. A window Wd centered on yr is set. The size of the window Wd is determined so that the white line does not deviate in anticipation of the case where the vehicle is not in the center of the white line.
[0036]
Then, a differential operation is performed in each window Wd to detect a place where positive and negative edges appear as a pair on the left and right sides, and the center position of the positive and negative edges on the X axis is calculated and stored as a white line detection point. A white line is detected by connecting the detection points from each window so as to be further fitted. As a method of fitting, the technique introduced in Japanese Patent Application No. 3-3145 and Japanese Patent Application No. 4-171240 can be used.
[0037]
In addition, since the white line ahead of the vehicle is often hidden by the preceding vehicle, only the range below the image from the position yc of the preceding vehicle can be limited to the white line detection range.
This reduces the amount of data to be processed and reduces the time required for white line detection. Further, it is less likely that the edge of the preceding vehicle is erroneously detected as the edge of the white line, and the determination of the vehicle position becomes more accurate.
[0038]
Next, in step 103, the window setting unit 6 detects an area such as the sky in one of the images A and B (image A in the present embodiment), and a window having a predetermined size in accordance with a portion excluding the area. Set.
In step 104, the matching position detection unit 7 inputs the original image of the image A (and the image B) shown in FIG. 13A from the image memory 3 and the image memory 4 and performs a differentiation process so as to highlight the horizontal edge. A standing differential image (b) is created (hereinafter, for the sake of simplicity, the differential image is also referred to as an image A and an image B under the name of the original image).
[0039]
The image A is divided into windows 40 each having a width xw horizontally and a width yw vertically as shown in FIG. 14 at the window position set in step 103. Using each divided window 40 as a template, a position having the highest similarity to the image of the template from image B is obtained by the normalized correlation method using equation (6), and the matching position of the window is calculated.
[Expression 1]

In Equation (6), the brightness value of each pixel of the template image is Aij, and the brightness value of each pixel of the image B is Bij.
Here, the reason why the differential image is used is that the preceding vehicle, which is the detection target, often has a horizontal edge, and the preceding vehicle is emphasized from the background to facilitate detection.
[0040]
FIG. 15 is an explanatory diagram of a method for detecting a matching position between images A and B by the normalized correlation method.
Here, since the two cameras 1 and 2 are arranged so that their Y axes are on the same line, search is performed in the image B at the same position of the X axis as the template of the image A (window 40). A plurality of position detection target images whose positions are sequentially shifted in the Y-axis direction are obtained. Then, the position detection with the highest degree of similarity is made by comparing the template shown in FIG. 15B and the position detection target images shown in (d1) to (d3) extracted from the image B shown in (c). The target image is obtained.
[0041]
In the figure, (d2) shows a detection target image at a position having the maximum similarity to the template, (d1) shows a detection target image at a position above it, and (d3) shows a detection target image at a position below it. . In this way, a window corresponding to each window is created in the image B for the image A. The ya and yb indicating the Y-axis positions of the two windows are the matching positions of the windows.
[0042]
In step 105, the distance calculation unit 8 calculates the distance to the object imaged in the window in each window using Expression (1). Since the object occupies the same position in the window of the image A and the image B as shown in FIG. 16, the object matching position difference is the window matching position difference (yb-ya). Therefore, the distance to the object in the window is calculated by substituting into the equation (1) using the matching position of the window.
[0043]
When the above processing is performed for all windows and the distance to the object imaged in each window is obtained for each window, a block of windows for which the same distance value has been continuously calculated in step 106 is detected. When there are a plurality of chunks of the window, the one with the closest distance is selected and detected as a detection region including the preceding vehicle. Since the detection area also includes a road surface portion having the same distance as the preceding vehicle, the lower window is extracted from the X coordinate positions of the windows on both sides of the cluster of windows where the same distance is detected above yc indicating the distance of the preceding vehicle. Then, it is detected as the area of the preceding vehicle.
[0044]
Here, when the preceding vehicle is imaged far away, the preceding vehicle image is small, and if the window position is detected as the width of the vehicle, the accuracy decreases. In order to prevent this, as shown in FIG. 17, the window (a) on both sides of the window cluster is matched in the same manner as described above using the small window ws subdivided as shown in FIG. Calculate the distance. And it detects as a preceding vehicle using the small window of the same distance as the cluster of windows. FIG. 17 shows the right window. xcr is detected as the side position of the vehicle at the position of the small window.
[0045]
Further, instead of subdividing the window, the vertical edges t and v are detected from the window as shown in FIG. 18, and it is determined whether the vehicle texture, color or brightness is the same as that of the preceding vehicle. It is also possible to detect the position. As a result, even if the preceding vehicle is imaged small, the position of the preceding vehicle can be specified without reducing accuracy.
[0046]
In step 107, it is checked whether or not a cluster of windows has been detected. If not, it is determined that there is no preceding vehicle, the process returns to step 101, the next image is input, and the above processing is repeated. If the cluster of windows can be detected, the distance value calculated in the window with the presence of the preceding vehicle is stored as the distance detection value to the preceding vehicle, and the routine proceeds to step 108.
[0047]
In step 108, a determination is made regarding the travel lane of the preceding vehicle. The traveling lane determining unit 10 determines whether the traveling lane is the same as the own vehicle. As shown in FIG. 6, two points that intersect the two white lines detected in step 102 are detected using the same position yc as the distance of the cluster of windows as a line. Then, it is determined whether or not a group of windows is at these two intersections, and it is determined whether or not the preceding vehicle is in the same travel lane as the own vehicle. If the travel lane is different from the own vehicle, the process returns to step 101 and the above process is performed again. When the detected preceding vehicle is the same traveling lane as the own vehicle, the process proceeds to step 109.
[0048]
In step 109, intersections xll (left white line) and xlr (right white line) of the line and the white line are detected from the line at the same position yc as the detection distance of the preceding vehicle. Further, the window positions on both sides of the cluster of windows for detecting the vehicle are detected. By substituting them into equation (4), the lateral position on the travel lane is calculated.
As described above, the distance to the preceding vehicle and the lateral position in the travel lane can be detected by stereo image processing.
[0049]
This embodiment is configured as described above, sets a window for dividing an image into one of stereo images, detects the position of the window having the highest similarity to each window from the other image, and based on the triangulation principle Calculation is performed, and a cluster of windows in which the same distance calculation value is calculated from adjacent windows is detected as a preceding vehicle. Then, the position of the driving lane is compared with the mass of the window recognized as the preceding vehicle, and the lateral position of the preceding vehicle with respect to the white line is detected after determining whether it is the same traveling lane as the own vehicle. The horizontal position is also calculated. In addition, since only one type of camera can be used even in monochrome, the processing becomes simple and the detection environment can be selected.
[0050]
Next, a second embodiment of the present invention will be described.
In this embodiment, the behavior of the preceding vehicle detected in the first embodiment is observed and its behavior is judged.
FIG. 19 is a block diagram showing the configuration of the present embodiment, in which the vehicle behavior determination unit 12 is provided in the block diagram of the first embodiment shown in FIG. The vehicle behavior determination unit 12 has a memory, stores the lateral position of the preceding vehicle detected by the lateral position detection unit 11, and determines the behavior of the preceding vehicle based on temporal variation of the position.
[0051]
In this embodiment, in addition to the own vehicle traveling lane, an adjacent traveling lane is also detected, so the traveling lane determining unit 100 having the function is used. Others are the same as the first embodiment.
The contents of the traveling lane determination unit 100 will be described in the following processing flow.
[0052]
FIG. 20 is a flowchart showing the flow of processing in the above configuration.
In steps 101 to 107, stereo images from the cameras 1 and 2 are input and stored in the image memories 3 and 4 as in the first embodiment. As in the first embodiment, the white line detection unit 5 detects the left and right two white lines of the vehicle from one of the stereo images and converts it into data by fitting processing.
[0053]
The matching position detection unit 7 detects the position of the window having the highest similarity from the other image using the normalized correlation method for one image set by the window setting unit 6. The preceding vehicle detection unit 9 detects the area of the preceding vehicle from a group of adjacent windows in which the same distance value is calculated among the windows whose distances are calculated by the distance calculation unit 8.
[0054]
In step 208, the travel lane determining unit 100 detects a white line position on the Y-axis position that is the same as the distance of the cluster of windows from the image A. Here, the position of the white line in the adjacent lane is also detected in addition to the vehicle lane.
That is, as shown in FIG. 21, when the vehicle is traveling on a two-lane road, there are three white lines, so if all of them are within the imaging range of cameras 1 and 2, the vehicle is traveling in the left lane. In this case, the white line ahead of the distance Z is imaged at positions xl, xr, and xr2 on the Y = yc line where the preceding vehicle is detected as shown in FIG. The own vehicle traveling lane of the preceding vehicle can be determined by determining the position of the preceding vehicle and the preceding vehicle.
FIG. 23 is a diagram illustrating an image in which a preceding vehicle is simultaneously captured from the own vehicle traveling lane and the right adjacent lane. M is the own vehicle lane, and N is the preceding vehicle on the right adjacent lane. These are detected by a block of windows (thick line portion).
[0055]
In step 208, the traveling lane of the preceding vehicle is determined as follows.
White line positions xll and xlr on the vehicle traveling lane are obtained from the intersection of the line Y = yc and the approximate line of the white line. Since the distance Wr between the white lines is usually about 3.5 m, the position xlr on the image of the white line outside the right adjacent lane at a distance Z away as shown in FIG. it can.
xlr = xll + (f × Wr) / Z (7)
Z is the detected distance to the vehicle, f is the focal length of the camera, and Wr is the white line width.
Here, xll is the position of the white line on the right side of the vehicle lane.
[0056]
If the cluster of windows is within the white line of the host vehicle travel lane, the process proceeds to step 209, assuming that it is on the same travel lane as the host vehicle. When it is determined that the vehicle is not the own vehicle travel lane, the travel lane determination unit 100 virtually calculates the position of the white line position xlr outside the right travel lane based on Expression (7). It is determined whether the window block is between the right white line and the virtual white line of the vehicle. Note that when the inter-vehicle distance is short, xlr may come out of the right side of the image. However, at such a position, the determination is performed in consideration of the fact that xlr is outside xll.
As a result of the determination, if the cluster of windows is in the right lane, the preceding vehicle is on the right adjacent lane, and the process proceeds to step 210. If the cluster of windows is not the right lane, the process advances to step 211 assuming that the preceding vehicle is on the left lane.
[0057]
In step 209, as shown in FIG. 24, for the position xll or xlr on the Y = yc line, one of the image distances xLL and xLR with respect to the white line from the side position xcl or xcr of the cluster M of the window for detecting the preceding vehicle. One of these is calculated. In this embodiment, xLR is calculated.
[0058]
In step 210, the image distance xLL or xLR for the white lines R1 and R2 of the right lane is calculated from FIG. 25 in the same manner as in step 209.
In step 211, xLL or xLR is calculated for the white lines L1 and L2 of the left lane as shown in FIG.
[0059]
In step 212, the lateral position (LL, LR) on the travel path of the preceding vehicle is calculated from the lateral position (xLL, xLR) on the image using the equation (4) and stored in the memory. Then, the behavior of the preceding vehicle is determined from the temporal variation of the lateral position. 27, 28, 29, and 30 are graphs plotting temporal changes in the lateral position LR with respect to the right white line. The least square error line is shown on the figure. An error is obtained by fitting each lateral position to a least square error line. When the error is within a predetermined value, the behavior of the preceding vehicle is determined based on the slope of the least square error straight line. That is, in FIG. 27, since the lateral position LR with respect to the right white line of the preceding vehicle hardly changes as the vehicle travels, it can be determined that the vehicle is traveling while keeping the lane.
[0060]
In FIG. 28, since the lateral position LR gradually increases, it can be determined that the vehicle is changed to the left lane.
FIG. 29 shows the change to the right lane.
In FIG. 30, since the vibration is large with respect to the least square error line, it can be determined that the meander is meandering.
[0061]
Since the present embodiment is configured as described above and detects the lateral distance and distance change with respect to the white line of the preceding vehicle to determine the behavior, a lane change can be detected without determining blinker blinking. In addition, it is possible to detect meandering and the like, which are difficult to determine by blinking blinkers, and to estimate snoozing.
[0062]
Next, a third embodiment will be described. In this embodiment, the preceding vehicle detection unit 9 is connected to the vehicle behavior detection unit 12 in the configuration of the block diagram shown in FIG. As described in the second embodiment, the vehicle behavior detection unit 12 determines the behavior based on the temporal change in the lateral position of the preceding vehicle. In this embodiment, the preceding vehicle of the preceding vehicle detection unit 9 is further used as road condition recognition means. The function to judge the road surface condition from the distance of is added.
[0063]
First, as shown in FIG. 31, the vehicle behavior detection unit 12 takes out the lower window P of the window cluster as one window. The window P is subdivided into a plurality of small windows wt in the vertical direction as shown in FIG. 32, and the lower end position ycd of the preceding vehicle is obtained using the small windows. Further, instead of providing a small window, the lower end position ycd of the vehicle can also be calculated by determining whether the color and brightness in the region divided by the horizontal edge match those of the preceding vehicle as shown in FIG. ycd is the lower end position of the preceding vehicle, which is the same as the preceding vehicle position yc obtained by the window matching, and is highly accurate by direct calculation.
[0064]
If the lower end position ycd of the preceding vehicle and the distance Z of the vehicle at that time are known, the actual height yd with respect to the optical axis 1b of the camera 1 can be obtained by Expression (8) based on the imaging principle shown in FIG.
yd = ycd · Z / f (8)
The detection of yd is performed every time and stored in the memory. By looking at the fluctuation of yd for the past several times, the movement of the preceding vehicle in the vertical direction is recognized and the road condition is judged.
[0065]
FIGS. 35 to 40 are graphs plotting a temporal change of yd.
In each graph, a least square error straight line is obtained as in the second embodiment. In FIG. 35, since the slope of the least square error line is close to 0, it is determined that the road ahead is a flat road.
In FIG. 36, since the slope of the least square error straight line is positive, it indicates that yd increases and it is determined that the road ahead is a downhill.
[0066]
In FIG. 37, since the slope of the least-squares error line is negative, it indicates that yd becomes smaller and it is determined that the road ahead is uphill.
In FIG. 38, since the error of yd is large with respect to the least square error straight line, it is determined that the road ahead is an uneven road.
39 and 40, since the slope of the least square error line changes greatly, it is determined that there are stones, holes, or steps on the road ahead.
The present embodiment is configured as described above, and calculates the vertical position with respect to the camera based on the position and distance of the preceding vehicle, and the road surface condition is determined based on the variation in the position. Therefore, data when detecting the preceding vehicle is used. Can be processed without burdening computation.
[0067]
【The invention's effect】
  As described above, according to claim 1, the input image is divided into a plurality of windows, and based on the distance value obtained for each window, the area of the preceding vehicle is detected from a group of adjacent windows in which the same distance value is calculated. And calculate the distance. Also, the white line is detected from one of the images. The positional relationship between the preceding vehicle and the white line is obtained and the lateral position is detected. This eliminates the need for an infrared camera, etc. It becomes possible to do.
  And furthermore,Since the preceding vehicle behavior determining means is provided, a temporal change in the lateral position of the preceding vehicle is detected, and the behavior of the preceding vehicle can be determined. For example, when approaching the left white line or the right white line, it can be determined that the preceding vehicle is changing lanes. Further, since this determination is not based on the blinker blink determination, it is not necessary to detect blinker blinking, and an effect that is not affected by the environmental color or brightness can be obtained. Judgment can also be made for a preceding vehicle that changes lanes without turning the turn signal.
[0068]
  In invention of Claim 2, a road condition recognition means is provided,Since the image position of the preceding vehicle is calculated from the detection distance of the preceding vehicle and the front road surface condition is recognized by the temporal change in the image position, for example, when the image position of the preceding vehicle decreases, the preceding vehicle moves downhill. It can be determined that the car will start running. As a result, vehicle control corresponding to road conditions such as deceleration control of the host vehicle can be performed.
[0069]
  Claim 3In the described invention, the image area for detecting the white line is set as the image area below the preceding vehicle, so that the far white line blocked by the preceding vehicle is excluded from the detection target, and data to be processed is reduced and false detection is reduced. Effects can be obtained.
[Brief description of the drawings]
FIG. 1 is an explanatory diagram for explaining a distance calculation method by stereo image processing;
FIG. 2 is an explanatory diagram illustrating a relationship between a pixel and a focal length.
FIG. 3 is an explanatory diagram of window setting by matching;
FIG. 4 is an explanatory diagram of detection of a preceding vehicle.
FIG. 5 is an explanatory diagram of a positional relationship between a distance and an image.
FIG. 6 is an explanatory diagram of traveling lane determination of a preceding vehicle.
FIG. 7 is an explanatory diagram of lateral position detection of a preceding vehicle.
FIG. 8 is an explanatory diagram of image position and lateral position detection.
FIG. 9 is a block diagram showing the configuration of the first embodiment.
FIG. 10 is a flowchart showing the flow of processing in the first embodiment.
FIG. 11 is an explanatory diagram for detecting a white line.
FIG. 12 is an explanatory diagram of window settings.
FIG. 13 is a diagram showing the effect of differential processing.
FIG. 14 is a diagram illustrating window setting for detecting a preceding vehicle.
FIG. 15 is an explanatory diagram illustrating a method for detecting a matching position between stereo images by a normalized correlation method.
FIG. 16 is a diagram illustrating a positional relationship between an image in a window and the window.
FIG. 17 is an explanatory diagram of subdividing a window.
FIG. 18 is an explanatory diagram of detection of a side portion of a preceding vehicle based on imaging content.
FIG. 19 is a block diagram showing a configuration of a second exemplary embodiment.
FIG. 20 is a flowchart showing the flow of processing in the second embodiment.
FIG. 21 is a diagram showing the detection principle of the adjacent lane.
FIG. 22 is a diagram illustrating an imaging position of an adjacent lane.
FIG. 23 is a diagram showing a state in which a preceding vehicle is detected on the own vehicle travel lane and the adjacent lane.
FIG. 24 is an explanatory diagram of detection of a lateral position of a preceding vehicle on the own vehicle travel lane.
FIG. 25 is an explanatory diagram of detection of a lateral position of a preceding vehicle on the right adjacent lane.
FIG. 26 is an explanatory diagram of detection of a lateral position of a preceding vehicle on the left adjacent lane.
FIG. 27 is a graph showing a change in lateral position when the preceding vehicle is in lane keeping.
FIG. 28 is a graph showing a change in lateral position when the preceding vehicle is changed to the left lane.
FIG. 29 is a graph showing a change in lateral position when the preceding vehicle is changed to the left lane.
FIG. 30 is a graph showing changes in lateral position when the preceding vehicle is meandering.
FIG. 31 is an explanatory diagram of detection of a lower end position of a preceding vehicle.
FIG. 32 is an explanatory diagram of subdividing a window.
FIG. 33 is an explanatory diagram of detection of a lower end of a preceding vehicle based on imaging content.
FIG. 34 is a diagram showing the distance relationship between the lower end position of the preceding vehicle and the optical axis of the camera.
FIG. 35 is a graph when the preceding vehicle is traveling on a flat ground.
FIG. 36 is a graph when the preceding vehicle is traveling downhill.
FIG. 37 is a graph when the preceding vehicle is traveling uphill.
FIG. 38 is a graph when the preceding vehicle is traveling on an uneven road.
FIG. 39 is a graph when there is an obstacle such as a stone on the road on which the preceding vehicle is traveling.
FIG. 40 is a graph when there is a hole or the like on the road on which the preceding vehicle is traveling.
[Explanation of symbols]
1, 2 camera
1a, 2a CCD
1b, 2b Optical axis
3, 4 Image memory
5 White line detector (white line detection means)
6 Window setting section (window setting means)
7 Matching position detector (matching position detector)
8 Distance calculation part (distance calculation means)
9 Leading vehicle detection unit (leading vehicle detection means)
10, 100 Travel lane determination unit (travel lane determination means)
11 Lateral position detector (lateral position detector)
12 Vehicle behavior determination unit (vehicle behavior determination means)
C preceding car
E lumps of windows
f Focal length
D Interocular distance
L1, L2 lens
W window
wd window
Wr Road width
ya, yb matching position
yc Imaging position of the preceding vehicle
xr Window setting position on the X axis
width of xw window
yw window height
yr Window setting position on the Y axis
Z imaging distance

Claims (3)

Two cameras arranged in parallel with each other and with a predetermined interocular distance;
An image memory for storing road images captured by the two cameras as first and second images;
White line detecting means for detecting a white line in the first image;
Window setting means for setting a window so as to divide a predetermined area in the first image into a plurality of sections;
Matching position detecting means for detecting the position of the window having the highest similarity with each window from the second image;
Distance calculating means for calculating a distance to an object imaged in each window based on the position on the image where a window having a high degree of similarity exists between the two images and the positional relationship between the two cameras; ,
A preceding vehicle detection means for detecting a block of windows in which the same distance value is calculated and the window is adjacent as a detection region of the preceding vehicle and setting the distance value of the window as the detection distance of the preceding vehicle;
Travel lane determining means for determining the travel lane of the preceding vehicle based on the positional relationship between the cluster of windows and the white line;
The lateral position detecting means for detecting the lateral position of the preceding vehicle on the traveling lane,
Vehicle obstacle detection, comprising: preceding vehicle behavior determining means connected to the lateral position detecting means for detecting temporal changes in the lateral position of the preceding vehicle to determine the behavior of the preceding vehicle apparatus. Two cameras arranged in parallel with each other and with a predetermined interocular distance;
An image memory for storing road images captured by the two cameras as first and second images;
White line detecting means for detecting a white line in the first image;
Window setting means for setting a window so as to divide a predetermined area in the first image into a plurality of sections;
Matching position detecting means for detecting the position of the window having the highest similarity with each window from the second image;
Distance calculating means for calculating a distance to an object imaged in each window based on the position on the image where a window having a high degree of similarity exists between the two images and the positional relationship between the two cameras; ,
A preceding vehicle detection means for detecting a block of windows in which the same distance value is calculated and the window is adjacent as a detection region of the preceding vehicle and setting the distance value of the window as the detection distance of the preceding vehicle;
Travel lane determining means for determining the travel lane of the preceding vehicle based on the positional relationship between the cluster of windows and the white line;
Lateral position detection means for detecting the lateral position of a preceding vehicle on the travel lane;
Road condition recognizing means connected to the preceding vehicle detecting means, calculating an image position of the preceding vehicle from a detection distance of the preceding vehicle, and recognizing a front road surface condition by a temporal change of the image position. Obstacle detection device for vehicles. 3. The obstacle detection device for a vehicle according to claim 1, wherein the white line detection means uses an image area below the preceding vehicle as a white line detection area .

Images