JP3588728B2 - On-street line detector in front of vehicle - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a detection apparatus by which an object in front of a vehicle on a road, especially on a curve road, can be detected in an early stage by computing the approximate line of the connection of all first-scene-originated density edge points and second-scene- originated density edge points on road-system coordinates. SOLUTION: A first scene which is photographed by a first camera 16b contains a precedent vehicle and the like on a vehicle where the apparatus is mounted running lane at a short distance of 10 to 30m, and its resolution is high. In second scenes which are photographed by a second camera 26b and a third camera 36b , the resolution of a road surface and a vehicle at long distances of 30 to 60m and 60 to 90m is high. Positions of density edge points on both photographed scene screens are converted, by coordinate conversion means 100 to 300, into an identical road-system coordinate position in which the horizontal projection line of respective vehicle front and rear axles is regarded as one axle, and a density edge point distribution on the top view of a road surface is obtained. A line which approximates the connection of all density edge points originated from both scenes of road-system coordinates is computed 100, and the approximate line of an on-road white line from a short distance up to a long distance is obtained. When the approximate line is used as a reference to set a search region in order to detect a preceding vehicle, an object in front of the vehicle on a road, especially on a curve road, can be detected in an early stage.

Description

で算出される値、に変換し、データテーブルＨＧｎ（ここではｎ＝１）の、位置（ｒ，ｔ）対応のアドレスのデータを１インクレメントする。Ｘ−Ｙ座標上の一点は極座標ρ−θに変換すると極座標ρ−θ上では曲線となるので、前記変換された位置（ｒ，ｔ）は、ｔ＝０〜３１のそれぞれに対応する３２個（それらの連なりが曲線）となる。データテーブルＨＧｎの、これらの位置（ｒ，ｔ）に割り当てたアドレスのデータが１インクレメントされるので、各アドレスのデータは、Ｘ−Ｙ座標上の特徴点それぞれの、ρ−θ座標上の曲線が通る度数を表わすことになる。
【００６９】
上記（５）式を変形すると次の（５ａ）式となる。
【００７０】

これは次の（５ｂ）式で表わすことができる。
【００７１】

ここで、ａ＝（Ｔｅ−Ｔｓ）／Ｔｄ，ｂ＝Ｔｓ，ｃ＝Ｒｓ，ｄ＝Ｒｄ／（Ｒｅ−Ｒｓ）である。
【００７２】
この（５）式において、（Ｘｃｈ，Ｙｃｈ）は、極座標原点のＸ−Ｙ座標上の位置であり、ここでは
Ｘｃｈ＝２５５、Ｙｃｈ＝３５０
である。Ｘｓは特徴点の、Ｘ−Ｙ座標上のＸ座標値、ＹｓはＹ座標値である。
【００７３】
ここで、
θ＝〔（ｔ・（Ｔｅ−Ｔｓ）／Ｔｄ）＋Ｔｓ〕
と表わすと、
（Ｔｅ−Ｔｓ）／Ｔｄ＝（π／２）／３２、
Ｔｓ＝０
であるので、

であり、ｔ＝０〜３１であるので、θ＝０，θ＝π／６４，θ＝２π／６４，θ＝３π／６４，・・・，３１π／６４と、π／６４を最小単位として３２点のρ値ｒが算出される。すなわちこの第１ハフ変換では、ウィンドウ２の右半分の領域の特徴点各点が、θがπ／６４を最小単位とし、かつ０以上π／２未満の範囲の極座標平面上の位置（ｒ，ｔ）に変換される。
【００７４】
上記（５）式の、Ｒｄ／（Ｒｅ−Ｒｓ）は、
Ｒｄ／（Ｒｅ−Ｒｓ）＝３２／２５６
である。これは、θ＝π／６４をｔの１単位としているので、すなわち、π／２を３２分割しているので、これに対応してρの範囲０〜２５６も同様に３２分割するものである。すなわちｒの一単位は２５６／３２である。
【００７５】
要約すると、「ハフ変換（ＨＧ１）」では、（５）式は具体的には次の（５−１）式である。
【００７６】

これを（５ｂ）式の形に変形して、

で表わすと、ａ_１＝π／６４，ｂ_１＝０，ｃ_１＝０，ｄ_１＝１／８ である。
【００７７】
このように第１ハフ変換では、ρ−θ極座標のθの単位をπ／６４と粗いものとし、しかもρの単位を２５６／３２と粗いものとしているので、１つの特徴点（Ｘｓ，Ｙｓ）をρ−θ極座標に変換する演算回数（ｔの切換回数３２すなわち（１）式の演算を行なう回数）が極く少く、ｒデータビット数が少く、特徴点各点をρ−θ極座標へ変換する速度が速く、かつ、度数カウント処理速度が速い。
【００７８】
ＣＰＵ１１は、第１回のハフ変換である「ハフ変換（ＨＧ１）」を終了すると、第１回の「最大点探索」を実行する。
【００７９】
第１回の「最大点探索」
前述のデータテーブルＨＧｎ（ここではｎ＝１）の各アドレス（ｒ，ｔ）のデータ（度数）を順次に読出して、読出したデータをレジスタＧｍのデータと比較して、読出しデータの方が大きいと読出しデータをレジスタＧｍに更新メモリし、かつこの時読出しデータのアドレス（ｒ，ｔ）のｒはレジスタｒｍ１に、ｔはレジスタｔｍ１に更新メモリする。データテーブルＨＧｎのすべてのアドレスに対してこのような処理を終了すると、レジスタＧｍにはデータテーブルＨＧｎにある度数の最大値が、レジスタｒｍ１，ｔｍ１には該最大値があるアドレス（ｒｍ１，ｔｍ１）が格納されていることになる。
【００８０】
ＣＰＵ１１は、第１回の「最大点探索」を終了すると、ｒ−ｔ極座標系で表現された前記アドレス（ｒｍ１，ｔｍ１）を、Ｘ−Ｙ座標系画面に対応するρ−θ極座標系アドレス（Ｒｍ１，Ｔｍ１）に変換する。ここでは、
Ｒｍ１＝ｒｍ１・（Ｒｅ１−Ｒｓ１）／Ｒｄ１ ・・・（６）
Ｔｍ１＝ｔｍ１・（Ｔｅ１−Ｔｓ１）／Ｔｄ１ ・・・（７）
を算出して、算出したＲｍ１をレジスタＲｍ１に、算出したＴｍ１をレジスタＴｍ１に格納する。 （Ｒｅ１−Ｒｓ１）＝２５６
Ｒｄ１＝３２
（Ｔｅ１−Ｔｓ１）＝π／２
Ｔｄ１＝３２
であり、（６）式は、Ｒｍ１＝８・ｒｍ１ を意味し、（７）式は、Ｔｍ１＝（π／６４）・ｔｍ１
を意味する。アドレス（Ｒｍ１，Ｔｍ１）をＸ−Ｙ座標系への変換式に代入すると、補正画像メモリの画像データを表示する画面上の、ウィンドウ２（又はレ−ン検出ウィンド）の右半分の領域にある、ある直線（低密度ハフ変換６７により検出した直線：以下第１回検出の直線と称す）を示す式が得られる。
【００８１】
次にＣＰＵ１１は、イメージメモリ１５ａのある領域に割り当てたデータテーブルＨＧｎ（ｒ，ｔ）のデータをクリアする。次に第２回のハフ変換である「ハフ変換（ＨＧ２）」の変換パラメータを設定する。前述の第１回のハフ変換ではθの範囲Ｔｓ〜Ｔｅを０〜π／２、単位（Ｔｅ−Ｔｓ）／Ｔｄをπ／６４とし、ρの範囲Ｒｓ〜Ｒｅは０〜２５６、単位Ｒｄ／（Ｒｅ−Ｒｓ）を１／８としたが、ここでは、θおよびρの範囲を、前記（Ｒｍ１，Ｔｍ１）を中心とする小範囲に設定し、θおよびρの単位も小さく設定する。具体的には、
Ｒｓ＝８（ｒｍ１−２）
Ｒｅ＝８（ｒｍ１＋２）
Ｒｄ＝３２
Ｔｓ＝（π／６４）（ｔｍ１−２）
Ｔｅ＝（π／６４）（ｔｍ１＋２）
Ｔｄ＝３２
を設定する。なお、ｒｍ１＝Ｒｍ１／８，ｔｍ１＝Ｔｍ１／（π／６４）である。そして第２回の「ハフ変換（ＨＧ２）」を行なう。
【００８２】
「ハフ変換（ＨＧ２）」
この第２回の「ハフ変換（ＨＧ２）」の内容は、第１回の「ハフ変換（ＨＧ１）」と同様であるが、演算式（前述の（５）式）の内容が異なる。すなわち前記（５）式は具体的には次の（５−２）式となる。
【００８３】

これを（５ｂ）式の形に変形して、

で表わすと、前述の第１回の変換で、
ａ_１＝（π／６４）， ｂ_１＝０， ｃ_１＝０， ｄ_１＝１／８
に対して、
ａ_２＝（π／５１２），ｂ_２＝π（ｔｍ１−２）／６４，ｃ_２＝８（ｒｍ１−２），ｄ_２＝１
となる。
【００８４】
ｔ＝０〜３１であるので、
θ＝ π（ｔｍ１−２）／６４，
θ＝ （π／５１２）＋π（ｔｍ１−２）／６４，
θ＝ ２（π／５１２）＋π（ｔｍ１−２）／６４，
θ＝ ３（π／５１２）＋π（ｔｍ１−２）／６４，
・
・
・
θ＝３１（π／５１２）＋π（ｔｍ１−２）／６４，
と、（π／５１２）を最小単位として３２点のθに対応するρ値ｒが算出される。すなわちこの第２回のハフ変換では、ウィンドウ２の右半分の領域の特徴点各点が、θが（π／５１２）を最小単位（ｔの１単位がπ／５１２）とし、かつθ＝π（ｔｍ１−２）／６４以上θ＝３１（π／５１２）＋π（ｔｍ１−２）／６４以下の範囲の極座標平面上の位置（ｒ，ｔ）に変換される。ρの範囲は８（ｒｍ１−２）以上８（ｒｍ１＋２）以下で、ρの単位はこの範囲を３２分割するものである。すなわちｒの一単位は３２／３２＝（１／１）＝１である。したがって、第２回のハフ変換はウィンドウ２（又はレ−ン検出ウィンドウ）の右半分の特徴点を、第１のハフ変換よりも、狭い範囲かつ高い密度の極座標に変換するものである。特徴点の１つの極座標変換では、この第２ハフ変換でもｔ＝０〜３１、すなわち３２回の演算を行なうので、第１ハフ変換と処理時間は同程度である。１つの特徴点（Ｘｓ，Ｙｓ）をρ−θ極座標に変換する演算回数が極く少く、ｒデータビット数が少く、特徴点各点をρ−θ極座標へ変換する速度が速く、かつ、度数カウント処理速度が速い。
【００８５】
ＣＰＵ１１は、第２回のハフ変換である「ハフ変換（ＨＧ２）」を終了すると、第２回の「最大点探索」を実行する。その内容は前述の、第１回の「最大点探索」と同様である。この処理を終了すると、レジスタＧｍにはデータテーブルＨＧｎにある度数の最大値が、レジスタｒｍ２，ｔｍ２には該最大値があるアドレス（ｒｍ２，ｔｍ２）が格納されていることになる。
【００８６】
ＣＰＵ１１は、第２回の「最大点探索」を終了すると、ｒ−ｔ極座標系で表現された前記アドレス（ｒｍ２，ｔｍ２）を、Ｘ−Ｙ座標系画面に対応するρ−θ極座標系アドレス（Ｒｍ２，Ｔｍ２）に変換する。演算式は、
Ｒｍ２＝ｒｍ２・（Ｒｅ２−Ｒｓ２）／Ｒｄ２＋Ｒｓ２ ・・・（８）
Ｔｍ２＝ｔｍ２・（Ｔｅ２−Ｔｓ２）／Ｔｄ２＋Ｔｓ２ ・・・（９）
である。算出したＲｍ２をレジスタＲｍ２に、算出したＴｍ２をレジスタＴｍ２に格納する。ここでは、
（Ｒｅ２−Ｒｓ２）＝３２
Ｒｄ２＝３２
（Ｔｅ２−Ｔｓ２）＝π／１６
Ｔｄ２＝３２
であり、（８）式は、具体的には、
Ｒｍ２＝ｒｍ２＋８ｒｍ１−１６ ・・・（８−１）
であり、（９）式は、具体的には、
Ｔｍ２＝ｔｍ２・（π／５１２）＋（π／６４）・（ｔｍ１−２） ・・・（９−１）
を意味する。アドレス（Ｒｍ２，Ｔｍ２）をＸ−Ｙ座標系への変換式に代入すると、補正画像メモリの画像データを表示する画面上の、ウィンドウ２（又はレ−ン検出ウィンドウ）の右半分の領域にある、ある直線（中密度ハフ変換により検出した直線：以下第２回検出の直線と称す）を示す式が得られる。
【００８７】
次にＣＰＵ１１は、イメージメモリ１５ａのある領域に割り当てたデータテーブルＨＧｎ（ｒ，ｔ）のデータをクリアする。次に第３回のハフ変換である「ハフ変換（ＨＧ３）」の変換パラメータを設定する。ここでは、θおよびρの範囲を、第２回の「ハフ変換（ＨＧ２）」の場合よりも、前記（Ｒｍ２，Ｔｍ２）を中心とする更に小さい範囲に設定し、θおよびρの単位も小さく設定する。具体的には、
Ｒｓ＝ｒｍ２＋８ｒｍ１−１８
Ｒｅ＝ｒｍ２＋８ｒｍ１−１４
Ｒｄ＝３２
Ｔｓ＝（π／５１２）ｔｍ２＋（π／６４）ｔｍ１−９π／２５６
Ｔｅ＝（π／５１２）ｔｍ２＋（π／６４）ｔｍ１−７π／２５６
Ｔｄ＝３２
を設定する。なお、ｒｍ２，ｔｍ２は、Ｒｍ２，Ｔｍ２に対して上記（８−１）式，（９−１）式で規定されるものである。そして第３回の「ハフ変換（ＨＧ３）」を行なう。
【００８８】
「ハフ変換（ＨＧ３）」
この第３回の「ハフ変換（ＨＧ３）」の内容は、前述の第２回の「ハフ変換（ＨＧ２）」と同様であるが、演算式（前述の（５）式）の内容が異なる。前記（５）式は、この第３回の「ハフ変換（ＨＧ３）」では、具体的には次の（５−３）式となる。
【００８９】

これを（５ｂ）式の形に変形して、

で表わすと、前述の第１回および第２回の変換で、
ａ_１＝（π／６４）， ｂ_１＝０， ｃ_１＝０， ｄ_１＝１／８
ａ_２＝（π／５１２），ｂ_２＝π（ｔｍ１−２）／６４，ｃ_２＝８（ｒｍ１−２），ｄ_２＝１
に対して、
ａ_３＝（π／４０９６），ｂ_３＝（π／５１２）ｔｍ２＋（π／６４）ｔｍ１−９π／２５６，
ｃ_３＝ｒｍ２＋８ｒｍ１−１８，ｄ_３＝８
となる。
【００９０】
ｔ＝０〜３１であるので、
θ＝ （π／５１２）ｔｍ２＋（π／６４）ｔｍ１−９π／２５６，
θ＝ （π／４０９６）＋（π／５１２）ｔｍ２＋（π／６４）ｔｍ１−９π／２５６，
θ＝ ２（π／４０９６）＋（π／５１２）ｔｍ２＋（π／６４）ｔｍ１−９π／２５６，
θ＝ ３（π／４０９６）＋（π／５１２）ｔｍ２＋（π／６４）ｔｍ１−９π／２５６，
・
・
・
θ＝３１（π／４０９６）＋（π／５１２）ｔｍ２＋（π／６４）ｔｍ１−９π／２５６，
と、（π／４０９６）を最小単位として３２点のθ（ｔ＝０〜３１）に対応するρ値ｒが算出される。すなわちこの第３回のハフ変換では、ウィンドウ２（又はレ−ン検出ウィンドウ）の右半分の領域の特徴点各点が、θが（π／４０９６）を最小単位（ｔの１単位がπ／４０９６）とし、かつθ＝（π／５１２）ｔｍ２＋（π／６４）ｔｍ１−９π／２５６以上 θ＝３１（π／４０９６）＋（π／５１２）ｔｍ２＋（π／６４）ｔｍ１−９π／２５６以下 の範囲の極座標平面上の位置（ｒ，ｔ）に変換される。ρの範囲はＲｓ＝ｒｍ２＋８ｒｍ１−１８ 以上 Ｒｅ＝ｒｍ２＋８ｒｍ１−１４ 以下で、ρの単位はこの範囲を３２分割するものである。すなわちｒの一単位は４／３２＝（１／８）である。したがって、第３回のハフ変換はウィンドウ２（又はレ−ン検出ウィンドウ）の右半分の特徴点を、第２回のハフ変換よりも、更に狭い範囲かつ高い密度の極座標に変換するものである。特徴点の１つの極座標変換では、この第３回のハフ変換でもｔ＝０〜３１であるので、３２回の演算を行なうので、第２回のハフ変換と処理時間は同程度である。１つの特徴点（Ｘｓ，Ｙｓ）をρ−θ極座標に変換する演算回数が極く少く、ｒデータビット数が少く、特徴点各点をρ−θ極座標へ変換する速度が速く、かつ、度数カウント処理速度が速い。
【００９１】
ＣＰＵ１１は、第３回のハフ変換である「ハフ変換（ＨＧ３）」を終了すると、第３回の「最大点探索」を実行する。その内容は前述の、第２回の「最大点探索」と同様である。この処理を終了すると、レジスタＧｍにはデータテーブルＨＧｎにある度数の最大値が、レジスタｒｍ３，ｔｍ３には該最大値があるアドレス（ｒｍ３，ｔｍ３）が格納されていることになる。
【００９２】
ＣＰＵ１１は、第３回の「最大点探索」を終了すると、ｒ−ｔ極座標系で表現された前記アドレス（ｒｍ３，ｔｍ３）を、Ｘ−Ｙ座標系画面に対応するρ−θ極座標系アドレス（Ｒｍ３，Ｔｍ３）に変換する。演算式は、
Ｒｍ３＝ｒｍ３・（Ｒｅ３−Ｒｓ３）／Ｒｄ３＋Ｒｓ３・・・（１０）
Ｔｍ３＝Ｔｍ３・（Ｔｅ３−Ｔｓ３）／Ｔｄ３＋Ｔｓ３・・・（１１）
である。算出したＲｍ３をレジスタＲｍ３に、算出したＴｍ３をレジスタＴｍ３に格納する。ここでは、
（Ｒｅ３−Ｒｓ３）＝４
Ｒｄ３＝３２
（Ｔｅ３−Ｔｓ３）＝π／１２８
Ｔｄ３＝３２
であり、（１０）式は、具体的には、
Ｒｍ３＝（１／８）・ｒｍ３＋ｒｍ２＋８ｒｍ１−１８ ・・・（１０−１）
であり、（７）式は、具体的には、
Ｔｍ３＝（π／１２８）・ｔｍ３＋（π／５１２）ｔｍ２＋（π／６４）ｔｍ１−９π／２５６ ・・・（１０−１）
を意味する。アドレス（Ｒｍ３，Ｔｍ３）をＸ−Ｙ座標系への変換式に代入すると、補正画像メモリの画像データを表示する画面上の、ウィンドウ２（又はレ−ン検出ウィンドウ）の右半分の領域にある、ある直線（高密度ハフ変換により検出した直線：以下第３回検出の直線と称す）を示す式が得られる。以上で「直線当てはめ」を終了したことになる。
【００９３】
以上で、ウィンドウ２（又はレ−ン検出ウィンドウ）の右半分領域にある画像中の直線（最も代表的な直線）を表わす直線（第３回検出の直線）を得たことになる。ウィンドウ２（又はレ−ン検出ウィンドウ）は、自車レーンの左，右端の白線を検出するに適した領域に設定されており、ウィンドウ２（又はレ−ン検出ウィンドウ）の右半分領域に右端の白線像があると、第３回検出の直線はこの白線を近似する直線である確率が高い。したがって、上述の「直線当てはめ」は、ウィンドウ２（又はレ−ン検出ウィンドウ）の右半分領域での、自車レーンの右端白線の検出である。
【００９４】
ＣＰＵ１１は、次に、第３回検出の直線を表わすデータ（Ｒｍ３，Ｔｍ３）のＲｍ３をレジスタＲｍＲに、Ｔｍ３をレジスタＴｍＲに格納する。すなわち自車レーンの右端白線を表わす直線を表わすデータをレジスタＲｍＲ，ＴｍＲに格納する。
【００９５】
左白線検出
ＣＰＵ１１は次に、以上に説明したウィンドウ２（又はレ−ン検出ウィンドウ）の右半分領域に関する処理と同様な処理を、ウィンドウ２（又はレ−ン検出ウィンドウ）の左半分領域に施して、自車レーンの左端白線を表わす直線を検出し、それを表わすデータをレジスタＲｍＬ，ＴｍＬに格納する。以上で「左右白線検出」Ｄ３を終了する。ＣＰＵ１１は次に、「無限遠点計算」Ｄ４を行なう。
【００９６】
Ｄ４．「無限遠点計算」Ｄ４
ここではまず、「直線交点計算」で、レジスタＲｍＲ，ＴｍＲのデータが表わす直線（自車走行レーン右端白線と推定した）と、レジスタＲｍＬ，ＴｍＬのデータが表わす直線（自車走行レーン左端白線と推定した）との交点（Ｘｃ，Ｙｃ）を算出する。次に、算出した交点（Ｘｃ，Ｙｃ）が、レジスタＸｖ，Ｙｖのデ−タが表わす無限遠点（Ｘｖ，Ｙｖ）を中心とする横６０画素×縦６０画素の領域内に存在するかをチェックする。この領域内であると、今回求めた交点（Ｘｃ，Ｙｃ）が無限遠点である信頼性が高いので、無限遠点データ（Ｘｃ，Ｙｃ）を、今回求めた交点（Ｘｃ，Ｙｃ）に１／８の重み付けをし、これまでの無限遠点データ（Ｘｃ，Ｙｃ）に７／８の重み付けをして加算した値に更新する。そして、無限遠点追跡に失敗した回数をカウントするためのレジスタＮｖｅをクリアする。
【００９７】
今回算出した交点（Ｘｃ，Ｙｃ）が、レジスタＸｖ，Ｙｖのデ−タが表わす無限遠点（Ｘｖ，Ｙｖ）を中心とする前記横６０画素×縦６０画素の領域内にないときには、無限遠点追跡が失敗（今回の交点算出がエラー又はこれまでの無限遠点データ（Ｘｖ，Ｙｖ）がエラー）であるとして、レジスタＮｖｅの内容を１インクレメントし、レジスタＮｖｅの内容が５になったかをチェックする。５になっていると、今回と過去４回の計５回連続して交点算出がエラーであったことになり、これは現在保待している無限遠点データ（Ｘｃ，Ｙｃ）がエラーであると見なして、無限遠点データ（Ｘｃ，Ｙｃ）を今回算出した交点（Ｘｃ，Ｙｃ）に更新する。
【００９８】
Ａ４．「ρ，θ＆無限遠点のフィルタリング」Ａ４
ここでＣＰＵ１１は、「左右白線検出」（Ｄ３）で検出した、直線を表わすパラメ−タρ，θおよび「無限遠点計算」（Ｄ４）で算出した直線交点座標（Ｘｖ，Ｙｖ）をフィルタリングする。すなわち、
ρＬ＝（３・ρＬ＋ＲｍＬ）／４ ・・・（１１Ｌ）
ρＲ＝（３・ρＲ＋ＲｍＲ）／４ ・・・（１１Ｒ）
θＬ＝（３・θＬ＋ＴｍＬ）／４ ・・・（１２Ｌ）
θＲ＝（３・θＲ＋ＴｍＲ）／４ ・・・（１２Ｒ）
を算出して、レジスタρＬ，ρＲ，θＬ，θＲに格納する。上記４式のカッコ内のρＬ，ρＲ，θＬ，θＲはレジスタρＬ，ρＲ，θＬ，θＲの前回値、ＲｍＬ等はレジスタＲｍＬ等のデ−タである。次に、算出したρＬ，ρＲで表わされる直線とθＬ，θＲで表わされる直線の交点（Ｘｖ，Ｙｖ）を算出し、ＸｖをレジスタＸｖに、ＹｖをレジスタＹｖに書込む。
【００９９】
Ｄ５．「左右白線間隔（ＷＬ）計算」Ｄ５
ＣＰＵ１１は次に、自車レーンの右白線と左白線との間隔（レーン幅）ＷＬを算出する。これにおいては、カメラ１６ｂの視野中心線（図１７の２点鎖線）が路面と交わる位置（画面上では画面の中心点）Ｌ１での、画面上の右端白線（ρＲ，θＲ）のＸ位置を路面上位置ＸＲに変換し、画面上の左端白線（ρＬ，θＬ）のＸ位置を路面上位置ＸＬに変換して、間隔ＷＬ＝ＸＲ−ＸＬを、次のように算出する：
ＸＲ＝（Ｓｙ×Ｈｃ×ｓｉｎθＲ）／（Ｓｘ×ｃｏｓθＲ） ・・・（１３Ｒ）
ＸＬ＝（Ｓｙ×Ｈｃ×ｓｉｎρＲ）／（Ｓｘ×ｃｏｓρＲ） ・・・（１３Ｌ）
ＷＬ＝ＸＲ−ＸＬ ・・・（１４）
なお、図１７のＳｘおよびＳｙはそれぞれカメラ１６ｂの横方向および縦方向のスケールファクタであり、Ｈｃは、カメラ１６ｂのレンズの中心の、路面からの高さである。
【０１００】
ＣＰＵ１１は次に、算出した、路面上のレーン間隔ＷＬが正しい（自車レーン検出成功）かをチェックする（Ｄ６）。すなわち、基準値ＷＬ３Ｄに対するＷＬの偏差が、許容範囲ＤｗＬ内であるかをチェックする。この実施例では、日本の高速道路のレーン幅が３．５±０．２ｍであるので、基準値ＷＬ３Ｄ＝３．５ｍに、許容値ＤｗＬ＝０．３ｍに定めている。
【０１０１】
基準値ＷＬ３Ｄに対するＷＬの偏差が許容範囲ＤｗＬ内にあると、自車レーン検出に成功しているとして、これを表わす情報「１」をレジスタＦＬに書込み（６１）、基準値ＷＬ３Ｄに対するＷＬの偏差が許容範囲ＤｗＬを外れていると、自車レーン検出に失敗しているとして、これを表わす情報「０」をレジスタＦＬに書込む（６２）。以上で、「自車レーン検出」を終了したことになり、自車レーン検出が成功していると、レジスタＦＬのデータは「１」である。
【０１０２】
「自車レーン検出」Ｄを終了するとＣＰＵ１１は、レジスタＦＬのデータをチェックしてそれが「１」であると、「カ−ブ推定」Ｇ１を実行する。すなわち、
Ｒ＝（Ｌ１／２）・ｓｉｎＡ ・・・（１５）
でカ−ブ半径Ｒを算出する。Ｌ１は、カメラ１６ｂからその視野中心線が水平路面と交わる点までの距離（Ｌ１＝１５ｍ）、Ａはカメラ１６ｂのステアリング角（図１１のＡ５で算出する）である。次に、算出したカ−ブ半径が２０００ｍ以下（カ−ブしている）かをチェックして（Ｇ２）、そうであると「特徴点選定」Ｇ３を実行する。
【０１０３】
Ｇ３．「特徴点選定」Ｇ３
ＣＰＵ１１は、ウィンドウ２の領域内かつｙ軸方向で無限遠点（Ｙｖ）までの、レジスタρＲ，θＲのデ−タρＲ，θＲが表わす直線上の画素を中心とする、ｙ方向±設定半幅値（路面上で±８０ｃｍ以内）、かつｘ方向±５画素以内の特徴点を、右白線（ここでは道路がカ−ブしているので右曲線）を表わす特徴点として抽出する。同様に、ウィンドウ２の領域内かつｙ軸方向で無限遠点（Ｙｖ）までの、レジスタρＬ，θＬのデ−タρＬ，θＬが表わす直線上の画素を中心とする、ｙ方向±設定半幅値（路面上で±８０ｃｍ以内）、かつｘ方向±５画素以内の特徴点を、左白線（ここでは道路がカ−ブしているので左曲線）を表わす特徴点として抽出する。
【０１０４】
Ｇ４Ａ．「座標変換」Ｇ４Ａ
次にＣＰＵ１１は、「特徴点選定」Ｇ３で抽出した特徴点の座標を、路面の３次元座標系（Ｘ，Ｙ，Ｚ）に変換する。Ｘは水平左右方向，Ｙは垂直上下方向，ＺはＸ，Ｙに直交する水平方向（車両前方向）である。カメラ１６ｂの光学モデルの側面および平面を図１８の（ａ）および（ｂ）に示す。なお、路面は水平な平面（Ｚ軸に平行）と見なす。図１８の（ａ）において、路面上の点Ｐが上述の特徴点の１つｙｐを意味するとする：
ｔａｎ（α−θ）＝ｈｃ／Ｚｐ ・・・（１６）
ｔａｎθ＝Ｂ／ｆ ・・・（１７）
であり、これらの関係より、点ＰのＺ軸座標値Ｚｐは、
Ｚｐ＝（ｆ＋Ｂ・ｔａｎα）・ｈｃ／（ｆ・ｔａｎα−Ｂ） ・・・（１８）
となる。また、カメラ１６ｂの下向き角度αは、
ｔａｎα＝（ｙｏ−ＶＰ）／ｆ ・・・（１９）
である。（１９）式のｔａｎαを算出し、これを（１８）式に代入してＺｐを算出することができる。ここで、ｙｏはカメラ座標系（画面座標系）の視野中心位置，ｙｐは特徴点の位置，Ｂは、カメラ系のｙ座標上での、視野中心位置に対する特徴点の位置差、ＶＰはカメラ系のｙ座標上での、道路座標系の水平線を投影した位置である。
【０１０５】
毎回、距離計算にｔａｎ関数を使用すると、処理時間が長くなるため、近似計算を考える。関数ｔａｎ（β）と角度βとの関係は、βが０に近いときには、ｔａｎβ＝βと近似できるため、（１６）式，（１７）式は、
α−θ＝ｈｃ／Ｚｐ ・・・（１６ａ）
θ＝Ｂ／ｆ＝（ｙｏ−ｙｐ）／ｆ ・・・（１７ａ）
α＝（ｙｏ−ＶＰ）／ｆ ・・・（１９ａ）
となり、（１８）式は、
Ｚｐ＝ｆ・ｈｃ／（ｙｐ−ＶＰ） ・・・（１８ａ）
と近似できる。ＣＰＵ１１は、この（１８ａ）式の演算で、特徴点のカメラ系のｙ座標値を、道路系のＺ軸座標値Ｚｐに変換する。
【０１０６】
図１８の（ｂ）を参照すると、路面上の点Ｐ（画面上の特徴点の対応位置）と結像面との関係は、
ｔａｎθ＝Ｘｐ／Ｚｐ ・・・（２０）
ｔａｎθ＝ｘｐ／ｆ ・・・（２１）
となる。ｘｐが、カメラ座標系での、特徴点のｘ座標値である。これら（２０），（２１）式より、道路座標系の点ＰのＸ座標値Ｘｐは、
Ｘｐ＝Ｚｐ・ｘｐ／ｆ ・・・（２２）
と算出できる。ＣＰＵ１１は、この（２２）式で表わされる演算で、Ｘｐを算出する。なお、道路座標系（Ｘ，Ｙ，Ｚ）で、特徴点のＹ座標値は、計算処理を単純化するために、すべて０としている。
【０１０７】
Ｇ５Ａ．「特徴点の回転補正」Ｇ５Ａ
レジスタＡ，Ｂ，Ｃには、後述の「カメラステアリング角の算出」Ａ５で算出した、カメラ１６ｂ，２６ｂ，３６ｂのステアリング角度Ａ，Ｂ，Ｃ（図５）が格納される。「特徴点の回転補正」Ｇ５でＣＰＵ１１は、上述のように道路系座標に変換した特徴点位置デ−タ（Ｚｐ，Ｘｐ）を、垂直軸Ｙ（図１８の（ａ））を中心にＡだけ回転した値に変換する。これは、カメラの視野中心線から見た特徴点の方向を、車両の前後軸から見た特徴点の方向に変換することを意味する。すなわち、この回転補正前の特徴点位置デ−タ（Ｚｐ，Ｘｐ）は、Ｚ軸がカメラの視野中心線の水平面投影線と合致する道路座標系のものであるが、上述の回転補正は、特徴点位置デ−タ（Ｚｐ，Ｘｐ）を、Ｚ軸が車両前後軸の水平面投影線に合致する道路座標系のものに変換する。
【０１０８】
以上に説明した、図９の「補正画像メモリイニシャライズ」Ｃ６から図１０を経て図１１の「特徴点の回転補正」Ｇ５までの上述のＣＰＵ１１の処理と同様な処理を、ＣＰＵ２１およびＣＰＵ３１が、それぞれカメラ２６ｂおよび３６ｂで撮影して得た画像デ−タを対象に、実行する。
【０１０９】
Ｔ３Ａ．「補正デ−タの集積」Ｔ３Ａ
次にＣＰＵ１１は、上述のように回転補正を施した特徴点位置デ−タ（右白線対応のものと左白線対応のもの、計２グル−プ）を、メモリ１５ａの特定領域に割り当てたレ−ン領域デ−タテ−ブルの、カメラ１６ｂ撮影デ−タ領域（右白線用領域と左白線用領域）に書込む。
【０１１０】
そして、ＣＰＵ間シリアルデ−タ通信ラインを介して、ＣＰＵ２１に、それが上述のように回転補正を施した特徴点位置デ−タの転送を指示しかつＤＭＡ装置１４に、ＤＭＡ転送装置２４から送られて来るデ−タの、レ−ン領域デ−タテ−ブルのカメラ２６ｂ撮影デ−タ領域への書込みを指示する。ＣＰＵ２１はＣＰＵ１１のこの指示に応じてＤＭＡ装置２４に、回転補正を施した特徴点位置デ−タを格納しているメモリ領域を指定してデ−タ転送を指示する。ＤＭＡ装置２４はこの指示に応答して該メモリ領域のデ−タを読出してＤＭＡデ−タラインに送出し、ＤＭＡ装置１４がそれをメモリ１５ａに書込む。ＤＭＡ２４が送出終了を報知するとＣＰＵ２１は転送終了をＣＰＵ１１に報知する。
【０１１１】
この報知を受けるとＣＰＵ１１は、ＣＰＵ間シリアルデ−タ通信ラインを介して、ＣＰＵ３１に、それが上述のように回転補正を施した特徴点位置デ−タの転送を指示しかつＤＭＡ装置１４に、ＤＭＡ転送装置２４から送られて来るデ−タの、レ−ン領域デ−タテ−ブルのカメラ３６ｂ撮影デ−タ領域への書込みを指示する。ＣＰＵ３１はＣＰＵ１１のこの指示に応じてＤＭＡ装置３４に、回転補正を施した特徴点位置デ−タを格納しているメモリ領域を指定してデ−タ転送を指示する。ＤＭＡ装置３４はこの指示に応答して該メモリ領域のデ−タを読出してＤＭＡデ−タラインに送出し、ＤＭＡ装置１４がそれをメモリ１５ａに書込む。ＤＭＡ３４が送出終了を報知するとＣＰＵ２１は転送終了をＣＰＵ１１に報知する。
【０１１２】
なお、ＣＰＵ２１又はＣＰＵ３１が、ステップＧ２で推定カ−ブ半径Ｒが２０００ｍ超を判定していた場合には、ステップＧ３，Ｇ４Ａ，Ｇ５Ａを実行していないので、該ＣＰＵ２１又はＣＰＵ３１はＣＰＵ１１にデ−タなしを報知する。この場合にはそこでＣＰＵ１１は、後述の「ρ，θの座標変換」Ｇ４Ｂに進む。
Ｇ６Ａ．「曲線検出（最小自乗）」Ｇ６Ａ
ＣＰＵ１１は次に、上述のレ−ン領域デ−タテ−ブルの右白線用領域群の特徴点位置デ−タ（カメラ１６ｂ，２６ｂ，３６ｂそれぞれで得た特徴点の集合）に対して、最小自乗法を適用して、それらの特徴点の分布が表わす円弧（右白線の近似曲線）を算出する。すなわち、Ｚ方向で車両に近い側の円弧端のＸ，Ｚ座標値，遠い側の円弧端のＸ，Ｚ座標値，円弧中心のＸ，Ｚ座標値および半径Ｒ_Ｒを算出する。これらの座標値はいずれも、この段階では、車両の前後軸の水平面投影線をＺ軸とする道路座標系のものである。このとき、特徴点のサンプル数（存在数）をカウントする。そして同様に、上述のレ−ン領域デ−タテ−ブルの左白線用領域群の特徴点位置デ−タ（カメラ１６ｂ，２６ｂ，３６ｂそれぞれで得た特徴点の集合）に対して、最小自乗法を適用して、それらの特徴点の分布が表わす円弧（左白線の近似曲線）を算出し、特徴点のサンプル数をカウントする。
【０１１３】
Ｇ７．「曲線検出の成否判定」Ｇ７
ＣＰＵ１１は次に、特徴点サンプル数（２組）および半径Ｒ（２組Ｒ_Ｒ，Ｒ_Ｌ）をチェックして、特徴点サンプル数の１つでも所定値未満であると、又は、カ−ブ半径Ｒの一方が２０００ｍを越えると、曲線検出不成立と見なして、レ−ンを直線と決定（判定）する。
【０１１４】
Ｇ９．「カ−ブＲのフィルタリング」Ｇ９
特徴点サンプル数のいずれもが所定値以上でしかもカ−ブ半径Ｒ_Ｒ，Ｒ_Ｌのいずれもが２０００ｍ以下であると、曲線検出成立と見なして、今回検出した半径ＲＲ_Ｒ，Ｒ_Ｌのそれぞれをフィルタリング処理して、半径レジスタＲＲ，ＲＬに更新書込みする。すなわち、今回算出した半径Ｒが１５０ｍ以下のときには係数Ｋｗを１６に、１５０ｍ超過３００ｍ以下のときには係数Ｋｗを８に、３００ｍを越えるときには係数Ｋｗを４に定めて、
ＲＲ＝〔（Ｋｗ−１）・ＲＲ＋Ｒ_Ｒ〕／Ｋｗ ・・・（２３Ｒ）
ＲＬ＝〔（Ｋｗ−１）・ＲＬ＋Ｒ_Ｌ〕／Ｋｗ ・・・（２３Ｌ）
を算出し、算出したＲＲおよびＲＬをレジスタＲＲ，ＲＬに更新書込みする。（Ｋｗ−１）・ＲＲ，（Ｋｗ−１）・ＲＬのＲＲおよびＲＬは、それぞれレジスタＲＲおよびＲＬの、更新書込み前のデ−タ値である。Ｋｗを上述のように定めるので、算出したカ−ブ半径Ｒ_Ｒ，Ｒ_Ｌが大きいときには、フィルタリング処理値ＲＲ，ＲＬに対する今回算出の半径Ｒ_Ｒ，Ｒ_Ｌの寄与比（重み）が大きく、算出値に対するフィルタリング処理値の応答性が高い。算出したカ−ブ半径Ｒ_Ｒ，Ｒ_Ｌが小さいときには、フィルタリング処理値ＲＲ，ＲＬに対する今回算出の半径Ｒ_Ｒ，Ｒ_Ｌの寄与比（重み）が小さく、算出値に対するフィルタリング処理値の応答性は低い。カ−ブ半径に対応して定められるカメラ１６ｂ，２６ｂ，３６ｂのステアリング角Ａ，Ｂ，Ｃは、カ−ブ半径に略反比例し、ホストＭＰＵ１８が行なうカ−ブ対応の制御の重要度がカ−ブ半径に略反比例し、カ−ブ検出又は算出の誤差が大きい場合やそれらの信頼性が低い場合には、カ−ブ半径が小さいときにカメラが走行レ−ンを見失なうとか、ホストＭＰＵ１８のカ−ブ対応制御がエラ−となるとか、制御の信頼性が低下する。これを改善するために、上述のように、フィルタリング処理値を、大きい算出値Ｒ_Ｒ，Ｒ_Ｌに対しては高応答（Ｋｗ小）に、小さい算出値Ｒ_Ｒ，Ｒ_Ｌに対しては低応答（Ｋｗ大）に定めた。
【０１１５】
Ｇ４Ｂ．「ρ，θの座標変換」Ｇ４Ｂ
先の「カ−ブ推定」Ｇ１で推定したカ−ブ半径がＲが２０００ｍを越える場合、先の「補正デ−タの集積」Ｔ３Ａで、ＣＰＵ２１又はＣＰＵ３１が転送デ−タなしを報知して来た場合、あるいは、「曲線検出の成否判定」Ｇ７で曲線検出不成立と判定した場合、ＣＰＵ１１は、先の「ρ，θ＆無限遠点のフィルタリング」Ａ４で得た走行レ−ンの左右白線を近似する直線を規定するデ−タ（ρＬ，θＬ），（ρＲ，θＲ）を、カメラの視野中心線の水平面投影線をＺ軸とする道路座標系（Ｘ，Ｙ，Ｚ）の直線を規定するデ−タ（Ｒｏａｄ−ρＬ，Ｒｏａｄ−θＬ），（Ｒｏａｄ−ρＲ，Ｒｏａｄ−θＲ）に変換する。ＣＰＵ２１およびＣＰＵ３１も、先の「カ−ブ推定」Ｇ１で推定したカ−ブ半径Ｒが２０００ｍを越える場合、同様に変換処理を行なう。
【０１１６】
Ｇ５Ｂ．「回転補正」Ｇ５Ｂ
ＣＰＵ１１は、上記変換で得たデ−タ（Ｒｏａｄ−ρＬ，Ｒｏａｄ−θＬ），（Ｒｏａｄ−ρＲ，Ｒｏａｄ−θＲ）に、カメラ１６ｂのステアリング角Ａ相当の回転補正を加えた、車両前後軸を基準とした直線規定デ−タ（ρＬ_Ｒ，θＬ_Ｒ），（ρＲ_Ｒ，θＲ_Ｒ）を算出する。すなわち、カメラの視野中心線の水平面投影線をＺ軸とする道路座標系の直線規定デ−タ（Ｒｏａｄ−ρＬ，Ｒｏａｄ−θＬ），（Ｒｏａｄ−ρＲ，Ｒｏａｄ−θＲ）を、車両の前後軸の水平面投影線をＺ軸とする道路座標系の直線規定デ−タ（ρＬ_Ｒ，θＬ_Ｒ），（ρＲ_Ｒ，θＲ_Ｒ）に変換する。ＣＰＵ２１およびＣＰＵ３１も同様に変換処理を行なう。
【０１１７】
Ｔ３Ｂ．「補正デ−タの集積」Ｔ３Ｂ
ＣＰＵ１１は、ＣＰＵ間シリアルデ−タ通信ラインを介して、ＣＰＵ２１およびＣＰＵ３１より、それらが変換処理した直線規定デ−タ（ρＬ_Ｒ，θＬ_Ｒ），（ρＲ_Ｒ，θＲ_Ｒ）を得る。
【０１１８】
なお、ＣＰＵ２１又はＣＰＵ３１が、ステップＧ２で推定カ−ブＲが２０００ｍ以下を判定していた場合には、ステップＧ４Ｂ，Ｇ５Ｂを実行していないので、該ＣＰＵ２１又はＣＰＵ３１は、ＣＰＵ１１に該当デ−タなしを報知する。この場合にはＣＰＵ１１は、そのまま次の「直線検出」Ｇ６Ｂに進む。
【０１１９】
Ｇ６Ｂ．「直線検出」Ｇ６Ｂ
ＣＰＵ１１は、自己が算出してセ−ブしている直線規定デ−タ（ρＬ_Ｒ，θＬ_Ｒ），（ρＲ_Ｒ，θＲ_Ｒ）ならびにＣＰＵ２１およびＣＰＵ３１が転送して来た同様な直線規定デ−タの３者（ただし、ＣＰＵ１１のものだけ（１者のみ）の場合があり、またＣＰＵ２１又はＣＰＵ３１のものとＣＰＵ１１のもの合せて２者の場合もある）が表わす２グル−プの直線（右白線グル−プと左白線グル−プ）の各グル−プを代表する（近似する）右代表直線および左代表直線を規定するデ−タ（ρＲ_Ｒｍ，θＲ_Ｒｍ）および（ρＬ_Ｒｍ，θＬ_Ｒｍ）を算出する。これは重み付け平均で行ない、ＣＰＵ１１のものには８／１６の重み付けを、ＣＰＵ２１のものには５／１６の重み付けを、ＣＰＵ３１のものには３／１６の重み付けをする。
【０１２０】
Ｇ１０．「左右レ−ンの推定」Ｇ１０
ＣＰＵ１１は、「カ−ブＲのフィルタリング」Ｇ９からこの「左右レ−ンの推定」Ｇ１０に進んだときには、「カ−ブＲのフィルタリング」Ｇ９で得た半径レジスタＲＲのデ−タとレジスタＲＬのＲＬを比較して右曲りか左曲りかを判定して、右曲りのときには、ＲＲ−ＷＬを右隣接レ−ンの右白線のカ−ブ半径として、ＲＬ＋ＷＬを左隣接レ−ンの左白線のカ−ブ半径として算出し、左曲りのときには、ＲＲ＋ＷＬを右隣接レ−ンの右白線のカ−ブ半径として、ＲＬ−ＷＬを左隣接レ−ンの左白線のカ−ブ半径として算出する。ＷＬは、レジスタＷＬのデ−タであり、「左右白線間隔（ＷＬ）計算」Ｄ５（図１０）でＣＰＵ１１が算出した自車走行レ−ン幅である。
【０１２１】
「直線検出」Ｇ６Ｂからこの「左右レ−ンの推定」Ｇ１０に進んだときには、「直線検出」Ｇ６Ｂで算出した、右代表直線（を規定するデ−タρＲ_Ｒｍ，θＲ_Ｒｍ）を右にＷＬ分平行移動した直線を規定するデ−タ（ρＲＲ_Ｒ，θＲＲ_Ｒ）ならびに左代表直線（を規定するデ−タρＬ_Ｒｍ，θＬ_Ｒｍ）を左にＷＬ分平行移動した直線を規定するデ−タ（ρＬＬ_Ｒ，θＬＬ_Ｒ）を算出する。
【０１２２】
Ｇ１２．「車両検索エリアの設定」Ｇ１２
曲線検出成立と判定している場合には、「左右レ−ンの推定」Ｇ１０で算出した、右隣接レ−ンの右白線に相当する円弧および左隣接レ−ンの左白線に相当する円弧それぞれの、図３（道路座標系であり車両の前後軸の水平面投影線がＺ軸）に示す１０ｍ，Ｌ１＝１５ｍ，３０ｍ，Ｌ２＝４５ｍ，６０ｍ，Ｌ３＝７５ｍおよび９０ｍ（いずれもＺ軸値）の、それぞれの地点の位置デ−タ（Ｚ，Ｘ）を算出する。
【０１２３】
曲線検出不成立と判定している場合には、「左右レ−ンの推定」Ｇ１０で算出した、右隣接レ−ンの右白線に相当する直線および左隣接レ−ンの左白線に相当する直線それぞれの、図３に示す１０ｍ，Ｌ１＝１５ｍ，３０ｍ，Ｌ２＝４５ｍ，６０ｍ，Ｌ３＝７５ｍおよび９０ｍの、それぞれの地点の位置デ−タ（Ｚ，Ｘ）を算出する。
【０１２４】
これにより、カメラ１６ｂ（ＣＰＵ１１）に撮影が割り当てられた領域（１０〜３０ｍの範囲）に、道路領域（自車走行レ−ン＋右隣接レ−ン＋左隣接レ−ン）を規定する６点（１０ｍ地点で左右各１点，１５ｍ地点で左右各１点および３０ｍ地点で左右各１点）が定められたことになる。
【０１２５】
同様に、カメラ２６ｂ（ＣＰＵ２１）に撮影が割り当てられた領域（３０〜６０ｍの範囲）に、道路領域（自車走行レ−ン＋右隣接レ−ン＋左隣接レ−ン）を規定する６点（３０ｍ地点で左右各１点，４５ｍ地点で左右各１点および６０ｍ地点で左右各１点）が定められたことになる。
【０１２６】
更に同様に、カメラ３６ｂ（ＣＰＵ３１）に撮影が割り当てられた領域（６０〜９０ｍの範囲）に、道路領域（自車走行レ−ン＋右隣接レ−ン＋左隣接レ−ン）自車両走行レ−ン領域を規定する６点（６０ｍ地点で左右各１点，７５ｍ地点で左右各１点および９０ｍ地点で左右各１点）が定められたことになる。
【０１２７】
Ｇ１２．「車両検索エリアの座標逆変換」Ｇ１２
ＣＰＵ１１は、カメラ１６ｂ（ＣＰＵ１１）に撮影が割り当てられた領域の上記６点を、カメラ１６ｂのステアリング角Ａ対応の補正を施してから、すなわちカメラ１６ｂの視野中心線の水平面投影線をＺ軸とする道路座標系の位置に変換してから、カメラ１６ｂの撮影画面上の座標値に逆変換し、同様に、カメラ２６ｂ（ＣＰＵ２１）および３６ｂ（ＣＰＵ３１）に撮影が割り当てられた領域の各６点を、各カメラの撮影画面上の座標値に逆変換する。
【０１２８】
Ｔ４．「車両検索エリア内特徴点の集積」Ｔ４
ここでＣＰＵ１１は、ＣＰＵ間シリアルデ−タ通信ラインを介して、ＣＰＵ２１に、カメラ２６ｂ（ＣＰＵ２１）に撮影が割り当てられた領域の上記６点の上記逆変換した座標値を与えて、該６点で囲まれる領域の特徴点の摘出，座標変換（カメラ視野中心線の水平面投影線をＺ軸とする道路座標系の位置への変換）および回転補正（車両前後軸の水平面投影線をＺ軸とする道路座標系の位置への変換）を指示し、ＣＰＵ３１には、カメラ３６ｂ（ＣＰＵ３１）に撮影が割り当てられた領域の上記６点の逆変換した座標値を与えて、該６点で囲まれる領域の特徴点の摘出，座標変換および回転補正を指示する。そしてＣＰＵ１１は、カメラ１６ｂ（ＣＰＵ１１）に撮影が割り当てられた領域の上記６点の上記逆変換した座標値に基づいて、該６点が囲む領域（折れ線で囲まれる）の、メモリ１５ａ上の特徴点を摘出し、摘出した特徴点に対して、上述の「座標変換」Ｇ４Ａと同様な座標変換処理を実行し、更に、上述の「特徴点の回転補正」Ｇ５Ａと同様な回転補正処理を実行する。このようにして得た特徴点デ−タは、メモリ１５ａの特定領域に割り当てたレ−ン領域デ−タテ−ブルの、カメラ１６ｂ撮影デ−タ領域に書込む。ＣＰＵ２１およびＣＰＵ３１も同様な処理を行なう。
【０１２９】
次にＣＰＵ１１は、ＣＰＵ間シリアルデ−タ通信ラインを介して、ＣＰＵ２１に、このように処理した特徴点デ−タの、メモリ１５ａへのＤＭＡ転送を指示する。このＤＭＡ転送が終了するとＣＰＵ１１は、ＣＰＵ３１に、上述のように処理した特徴点デ−タの、メモリ１５ａへのＤＭＡ転送を指示する。これらのＤＭＡ転送の詳細は、前述の「補正デ−タの集積」Ｔ３Ａの中で説明したＤＭＡ転送と同様である。
【０１３０】
Ｈ．「先行車両認識＆測距」Ｈ
メモリ１５ａの特定領域に割り当てたレ−ン領域デ−タテ−ブルの、カメラ１６ｂ撮影デ−タ領域，カメラ２６ｂ撮影デ−タ領域およびカメラ３６ｂ撮影デ−タ領域の全領域に渡って、特徴点の分布を検索する。すなわち、車両前後軸の水平面投影線をＺ軸とする道路座標系で、自車走行レ−ン，右隣接レ−ンおよび左隣接レ−ンの３レ−ン（道路面領域）に分布する特徴点の、Ｘ方向（左右方向）分布数（Ｚ軸投影ヒストグラム）を検索する。これにおいては、道路面領域をＺ方向で下（車両直近）側から、左から右に水平走査（Ｘ方向走査）して、その走査線上にある特徴点の数をカウントし、カウント値を走査線Ｎｏ．（Ｚ位置）対応でセ−ブする。このＸ走査を道路面領域のＺ方向で下（車両直近）から上側（遠方）へ行なう。
【０１３１】
道路面領域についてこれ（Ｚ軸投影ヒストグラムの作成）を完了すると、特徴点の水平カウント値（Ｘ方向分布数）が設定値以上のＺ方向連続幅（車両に近い始端Ｚ位置と終端Ｚ位置）を、Ｚ方向で下（車両直近）側から検索し、それがあると、そのＺ方向連続幅のＺ始端からＺ終端までの、道路面領域の特徴点を、Ｘ方向で左端から垂直（Ｚ方向）に走査して右端まで、特徴点の垂直分布数を検索（Ｘ軸投影ヒストグラムを作成）する。そしてこのＸ軸投影ヒストグラムから、特徴点の垂直カウント値（Ｚ方向分布数）が設定値以上のＸ方向連続幅（始端Ｘ位置と終端Ｘ位置）を、Ｘ方向で左（左隣接レ−ンの左白線相当位置）側から検索し、それがあると、その中央点（始端Ｘ位置と終端Ｘ位置の中間点（Ｘｔ）を算出し、垂直走査幅（前記水平カウント値が設定値以上のＺ方向連続幅の中間点（Ｚｔ）を算出し、車両デ−タテ−ブルに、（Ｘｔ，Ｚｔ）を書込む。特徴点の水平カウント値が設定値以上の第１のＺ方向連続幅について、左端から右端まで、上述の垂直カウント値が設定値以上のｘ方向連続幅の検索を終了すると、Ｚ軸投影ヒストグラム上の、車両直近から遠方への次の、水平カウント値が設定値以上の第２のＺ方向連続幅を検索する。
【０１３２】
このようにして、まずＺ軸投影ヒストグラムによりＺ方向の、他車両が存在する可能性がある領域を、自車両直近側から遠方側に（Ｚ方向に）順次に検索し、１つの該当領域を検索したときにその領域を、自車両の左方側（左隣接レ−ン）から右方側に（Ｘ方向に）、他車両が存在する可能性がある領域を検索し、それがあると、検索した領域の中心デ−タ（Ｘｔ，Ｚｔ）を、車両デ−タテ−ブルに書込む。これを道路面領域全面について終了すると、カメラ１６ｂ，２６ｂおよび３６ｂで撮影した路面（自車走行レ−ン＋推定の左隣接レ−ン＋推定の右隣性レ−ン）上の存在車両（正確には、車両でない場合もある。例えば貨物車からの落下物）の道路座標系上の位置（Ｘｔ，Ｚｔ）が車両デ−タテ−ブルにあることになる。
【０１３３】
次にＣＰＵ１１は、車両デ−タテ−ブル上の各位置（Ｘｔ，Ｚｔ）が、左隣接レ−ン（推定レ−ン）上か，それと自車走行レ−ンとの間（両レ−ンにまたがる）か、自車走行レ−ン上か、それと右隣接レ−ンとの間か、あるいは、右隣接レ−ン（推定レ−ン）上か、識別し、この区分を表わすデ−タを、車両デ−タテ−ブルの各位置情報（Ｘｔ，Ｚｔ）に付加する。以下においては位置情報（Ｘｔ，Ｚｔ）＋区分デ−タを、他車両デ−タと称す。
【０１３４】
Ｋ．「出力」Ｋ
ＣＰＵ１１は、曲線検出成立の場合にはカ−ブ情報（カ−ブ半径（ＲＲ＋ＲＬ）／２）および車両デ−タテ−ブルの他車両デ−タを、通信コントロ−ラ１７を介してホストＭＰＵ１８に転送する。曲線検出不成立の場合には、直線情報（直線であることを表わすデ−タ）と車両デ−タテ−ブルの他車両デ−タを転送する。
【０１３５】
Ａ５．「カメラステアリング角の算出」Ａ５
ＣＰＵ１１は、車両デ−タテ−ブルの、他車両デ−タの中から、区分デ−タが自車走行レ−ン，それと左又は右隣性レ−ンとの間である車両位置の、距離（Ｚｔ）が１０〜３０ｍの範囲内であって、しかも最短のものを検索する。それがあると、該検索した車両位置（Ｘｔ，Ｚｔ）にカメラ１６ｂの視野中心線（の路面上投影線）を向けるためのステアリング角Ａを算出する。該当する車両位置が無かったときには、自車走行レ−ンの、車両よりＬ１前方（Ｚ位置＝Ｌ１）の幅ＷＬの中央（左，右白線の中間点）にカメラ１６ｂの視野中心線（の路面上投影線）を向けるためのステアリング角Ａを算出する。そして、レジスタＡｎのデータＡｎをレジスタＡに書込み、レジスタＡｎに、今回算出したステアリング角Ａを書込む。
【０１３６】
次にＣＰＵ１１は、車両デ−タテ−ブルの、他車両デ−タの中から、区分デ−タが自車走行レ−ン，それと左又は右隣性レ−ンとの間である車両位置の、距離（Ｚｔ）が３０〜６０ｍの範囲内であって、しかも最短のものを検索する。それがあると、該検索した車両位置（Ｘｔ，Ｚｔ）にカメラ２６ｂの視野中心線（の路面上投影線）を向けるためのステアリング角Ｂを算出する。該当する車両位置が無かったときには、自車走行レ−ンの、車両よりＬ２前方（Ｚ位置＝Ｌ２）の幅ＷＬの中央（左，右白線の中間点）にカメラ２６ｂの視野中心線（の路面上投影線）を向けるためのステアリング角Ｂを算出する。そして、レジスタＢｎのデータＢｎをレジスタＢに書込み、レジスタＢｎに、今回算出したステアリング角Ｂを書込む。
【０１３７】
更にＣＰＵ１１は、車両デ−タテ−ブルの、他車両デ−タの中から、区分デ−タが自車走行レ−ン，それと左又は右隣性レ−ンとの間である車両位置の、距離（Ｚｔ）が６０〜９０ｍの範囲内であって、しかも最短のものを検索する。それがあると、該検索した車両位置（Ｘｔ，Ｚｔ）にカメラ３６ｂの視野中心線（の路面上投影線）を向けるためのステアリング角Ｃを算出する。該当する車両位置が無かったときには、自車走行レ−ンの、車両よりＬ３前方（Ｚ位置＝Ｌ３）の幅ＷＬの中央（左，右白線の中間点）にカメラ３６ｂの視野中心線（の路面上投影線）を向けるためのステアリング角Ｃを算出する。そして、レジスタＣｎのデータＣｎをレジスタＣに書込み、レジスタＣｎに、今回算出したステアリング角Ｃを書込む。
【０１３８】
Ｔ５．「カメラステアリング角の送信」Ｔ５
ＣＰＵ１１は、ＣＰＵ間シリアルデ−タ通信ライン介して、ＣＰＵ２１に、レジスタＢｎのデ−タＢｎを送信し、ＣＰＵ３１にはレジスタＣｎのデ−タＣｎを送信する。
【０１３９】
ＣＰＵ１１，ＣＰＵ２１およびＣＰＵ３１は、先に説明した「カメラステアリング出力」Ａ１で、デ−タＡｎ，ＢｎおよびＣｎを、それぞれステアリングコントロ−ラ１６ｅ，２６ｅおよび３６ｅに与え、これらのコントロ−ラが、カメラ１６ｂ，２６ｂおよび３６ｂの視野中心線（の水平面投影線）が車両前後軸（の水平面投影線）となす角がＡｎ，ＢｎおよびＣｎとなるようにカメラを回転駆動する。これにより、前方車両があるときには、カメラは前方車両を追跡し、前方車両がないときには、自車走行レ−ンの幅中央を追跡することになる。
【０１４０】
「カメラステアリング角の送信」Ｔ５を終了するとＣＰＵ１１は、図９の「画像入力を指令」Ｔ１に進む。これ以降の説明は、ここまでの説明の繰返しになるので、省略する。
【図面の簡単な説明】
【図１】本発明の一実施例の概要を示すブロック図である。
【図２】図１に示すテレビカメラ１６ｂの撮影画面を示す平面図である。
【図３】図１に示すテレビカメラ１６ｂ，２６ｂおよび３６ｂの撮影範囲（車両前方方向）を示す平面図である。
【図４】（ａ）は図１に示すテレビカメラ１６ｂ，２６ｂおよび３６ｂの撮影範囲（左右方向）を示すグラフ、（ｂ）は量子化誤差を示すグラフである。
【図５】図１に示すテレビカメラ１６ｂ，２６ｂおよび３６ｂの撮影範囲およびステアリング角Ａ，Ｂ，Ｃを示す平面図である。
【図６】図１に示す第１画像処理装置１００の構成を示すブロック図である。
【図７】図１に示す第２画像処理装置２００の構成を示すブロック図である。
【図８】図１に示す第３画像処理装置３００の構成を示すブロック図である。
【図９】図６に示すＣＰＵ１１の画像デ−タ処理および先行車両検出の内容の一部を示すフロ−チャ−トである。
【図１０】図６に示すＣＰＵ１１の画像デ−タ処理および先行車両検出の内容の一部を示すフロ−チャ−トである。
【図１１】図６に示すＣＰＵ１１の画像デ−タ処理および先行車両検出の内容の残部を示すフロ−チャ−トである。
【図１２】（ａ）は、図９に示す「特徴点検出ウィンドウ１セット」Ｃ１の内容を示すフロ−チャ−トである。（ｂ）は、図１に示すテレビカメラ１６ｂの撮影画面上の特徴点検出ウィンドウ１の領域を示す平面図である。
【図１３】（ａ）は、図９に示す「ボンネット検出」Ｃ３の内容を示すフロ−チャ−トである。（ｂ）は、図１に示すテレビカメラ１６ｂの撮影画面上の特徴点の分布と、「直線あてはめ」により検出する直線を示す平面図である。
【図１４】（ａ）は、図１に示すテレビカメラ１６ｂの撮影画面上のロ−ル角およびパン移動量を示す平面図、（ｂ）は図９に示す「画像回転平行移動」Ｃ７で用いる計算式を示す。
【図１５】図９に示す「補間」Ｃ８で注目画素の画像デ−タを算出するために参照する画素を示す平面図である。
【図１６】図１０に示す「レ−ン検出ウィンドウ設定」Ａ３で設定されるテレビカメラ１６ｂの撮影画面上のレ−ン検出ウィンドウを示す平面図である。
【図１７】図１に示すテレビカメラ１６ｂ内のレンズおよび撮像素子と車両前方の先行車両との幾何学的関係を示す側面概要図である。
【図１８】（ａ）は図１に示すテレビカメラ１６ｂ内のレンズおよび撮像素子と車両前方の先行車両との幾何学的関係を示す側面概要図、（ｂ）は平面概要図である。
【符号の説明】
１１：ＣＰＵ １２：ＲＯＭ
１３：ＲＡＭ １５〜１６：入出力ポ−ト
１５ａ：イメ−ジメモリ １６ａ：カメラコントロ−ラ
１６ｂ：テレビカメラ １６ｃ：Ａ／Ｄコンバ−タ
１６ｄ：回動機構 １６ｅ：ステアリングコントロ−ラ
１７：通信コントロ−ラ １８：ホストＭＰＵ[0001]
TECHNICAL FIELD OF THE INVENTION
BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to detection of an object in front of a vehicle, and in particular, but not exclusively, to photograph a scene including a road surface in front of the vehicle on a vehicle, and to display a traveling lane on the road surface on a photographing screen. The present invention relates to a road object detection device that detects a vehicle (a white line that separates the vehicle) and a preceding vehicle (or an obstacle ahead) and detects a distance of the own vehicle to the preceding vehicle.
[0002]
[Prior art]
One example of this type of detection device is disclosed in Japanese Patent Application Laid-Open No. 64-15605. In this case, an angle of 45 degrees with respect to the x-axis of image data representing a scene including a road surface in front of the vehicle and obtained by a photographing device mounted on the vehicle on an x, y orthogonal coordinate system screen is shown. A differential value in the direction to be formed is calculated, and a position on the x, y orthogonal coordinates that is equal to or greater than a set value is set as a feature point, a series of feature points is processed into a thin line, and a thin line having a predetermined length or more is extracted. The thin line that satisfies the conditions set based on the characteristics of the white line on the screen when the front is photographed from the TV camera is recognized as the white line on the road surface. That is, two straight lines representing the left and right end white lines of the lane in which the vehicle is traveling are obtained. The intersection of the two straight lines is determined, the presence or absence of an object is detected in a region (triangular region) between the two straight lines from the intersection to the host vehicle. Is calculated as the following distance). If the lane on which the vehicle is traveling (the vehicle traveling lane) is a straight line, all preceding vehicles on the lane are present in the triangular area (even if the roof is off, The bottom is in the area). However, since the two straight lines approximating the left and right end white lines are determined based on the left and right end white line images at positions near the camera, if the own vehicle lane curves before and after the own vehicle, or When the vehicle is curved ahead, the intersection of the two straight lines, which can be regarded as an infinity point in the case of a straight lane, is to the left of the far point of the own vehicle lane (the own vehicle lane is at the right). In the case of a curve) or to the right (in the case of a curve to the left). That is, the intersection deviates to the left from the far point of the vehicle lane when the vehicle lane curves to the right, and shifts to the right when the vehicle lane curves to the left. For this reason, where there is a curve, there is a high probability that the detection of a forward object (preceding vehicle or obstacle), particularly an object located far away, will fail if only the object search is performed within the triangular area.
[0003]
Japanese Patent Application Laid-Open No. 1-276214 discloses that an index along a road is photographed by a camera on a vehicle to generate vehicle position information on the vehicle, and the optimal route is determined by comparing the vehicle position information with the destination information. Two-dimensional camera for controlling the vehicle speed and wheel steering according to the determined route, and for changing the shooting direction of the camera horizontally and vertically to capture an indicator along the road; A posture change mechanism is proposed. However, this proposal is based on the premise that a road exists along the determined optimum route and that no preceding vehicle exists on the road. Actually, it is necessary to track the actual road (confirm the presence), and to reliably detect the presence or absence of the preceding vehicle and the inter-vehicle distance.
[0004]
Japanese Patent Application Laid-Open No. Hei 6-12435 discloses that in order to track a preceding vehicle with a camera, the focal length and zoom are controlled so that the image of the preceding vehicle has a substantially constant size on the screen. A tracking device that controls the attitude of a camera in a horizontal direction and a vertical direction so as to be substantially in the center at the top is presented. The following control is premised on the detection or identification of the first preceding vehicle, and the search for the next preceding vehicle when the following vehicle disappears from the front of the own vehicle following the road. There is no. In addition, there is no technique for dealing with erroneous following when a preceding vehicle following the vehicle is off the road and deviates from the traveling direction of the own vehicle.
[0005]
[Problems to be solved by the invention]
The detection of the traveling lane on the road and / or the preceding vehicle requires a long search distance because the vehicle is traveling. For example, when performing automatic control of the following distance based on the detection of a preceding vehicle, to put it simply, while traveling at a speed of about 40 km / h, a search up to about 30 m ahead may provide some safety. However, in the case of about 80 km, it may be necessary to detect the preceding vehicle over a long section of 50 m or more. In any case, since the range of the vehicle traveling speed is wide, it is preferable to automatically detect whether or not a preceding vehicle exists in a long section of 50 m or more assuming high-speed traveling.
[0006]
However, at a short distance, it is preferable to detect the vehicle at a wide angle in order to prepare for a lateral approach or a lane change of an adjacent vehicle, and it is better to set the camera to a wide field of view (wide angle). And the detection accuracy deteriorates. If a narrow field of view (telephoto) is set for long-distance detection, a short distance (especially an adjacent lane) leaks, and the nearest vehicle (especially an adjacent lane) cannot be detected. In addition, when the road is curved, the tip of the curve goes off the screen, and the vehicle ahead of the curve cannot be recognized.
[0007]
Therefore, when the preceding vehicle traveling on the own vehicle lane is definitely specified, the control for tracking it as a target is relatively simple, but the road condition (car) before that is controlled. (Vehicle / straight path), the presence or absence of a preceding vehicle, etc., and the detection or identification of a vehicle or object ahead of the own vehicle lane, and whether or not the preceding vehicle or object is on the own vehicle lane. Confirmation has a difficult problem, and a technique for detecting, specifying, or confirming with high accuracy is desired.
[0008]
A first object of the present invention is to detect an object ahead of a road, particularly ahead of a curve road as early as possible (while it is far away), and to improve the detection accuracy. And
[0009]
[Means for Solving the Problems]
(1) A road line detection device in front of a vehicle according to a first embodiment of the present invention includes a first scene having a wide viewing angle including a short-distance road surface and a narrow viewing angle including a long-distance road surface in front of the vehicle. Photographing apparatus for photographing the second scene (16b, 16d / 26b, 26d / 36b, 36d);
Feature point detecting means (100-300) for detecting density edge points (feature points) on the photographic images of the first scene and the second scene;
Positions of the density edge points (feature points) on the screens of the first scene and the second scene are the same as the horizontal plane projection line of the longitudinal axis of the vehicle as one axis (Z axis). Coordinate value conversion means (100 to 300) for converting to a position on road system coordinates;
Approximation line detection means (100) for calculating a line approximating the entire series of the density edge points originating from the first scene and the density edge points originating from the second scene on the road system coordinates;
Is provided. In addition, in order to facilitate understanding, symbols or corresponding items attached to corresponding elements of the embodiments shown in the drawings and described below are added for reference in parentheses.
[0010]
According to this, since the first scene has a wide range in the left-right direction at a short distance in front of the vehicle, the own vehicle traveling lane at a short distance (10 to 30 m) and a preceding lane on the lane. The vehicles include, of course, vehicles (side vehicles) slightly ahead of the own vehicle on the adjacent lane, and their resolution is high on the photographing screen of the first scene. Therefore, based on the image data of the first scene photographed image, the left and right white lines defining the traveling lane of the vehicle can be reliably detected.
[0011]
On the other hand, the second scene is for photographing a long distance in front of the vehicle (30 to 60 m, 60 to 90 m) with a narrow viewing angle. And the resolution of the vehicle is high. Therefore, a white line on a distant road surface can be detected with high accuracy based on the image data of the second scene photographed image.
[0012]
The photographing devices (16b, 16d / 26b, 26d / 36b, 36d) photograph the first scene and the second scene, and the coordinate value conversion means (100 to 300) performs the first scene. The positions on the screen of the density edge points (feature points) of the photographing screens of the second scene and the second scene are respectively on the same road system coordinates with one horizontal plane projection line (Z axis) of the longitudinal axis of the vehicle. Is obtained, a density edge point distribution in a form in which the road surface in front of the vehicle is viewed from above, that is, a density edge point distribution on the plan view of the road surface is obtained. This edge point distribution has high accuracy (resolution) even at a distance (30 to 60 m, 60 to 90 m) from the vehicle (since it is photographed as the second scene with a narrow viewing angle).
[0013]
Then, the approximate line detecting means (100) calculates a line which approximates the entire series of the density edge points originating from the first scene and the density edge points originating from the second scene on the road system coordinates. Therefore, the obtained line is an approximate line of a road white line extending over a long distance ranging from a short distance (10 to 30 m) in front of the vehicle to a long distance (30 to 60 m, 60 to 90 m). Since the resolution is high from 10 to 30 m) to the distant places (30 to 60 m, 60 to 90 m), a line with high reliability of approximation can be obtained. The lines obtained in this manner (usually, two lines approximating the right white line and the left white line defining the own vehicle traveling lane) are used as a reference for setting a search area for detecting a preceding vehicle. Thus, the preceding vehicle detection accuracy is improved. When the road is curved, the curve ahead can be detected at an early stage, and the accuracy and reliability of the calculation of the curve radius R are improved.
[0014]
BEST MODE FOR CARRYING OUT THE INVENTION
(2) The on-street line detection device in front of the vehicle according to the second embodiment of the present invention has a short distance (10 to 30 m) in front of the vehicle that forms a first angle (A) in the horizontal direction with respect to the longitudinal axis of the vehicle. A first scene having a wide viewing angle including a road surface and a long distance (30 to 60 m, 60 to 90 m) in front of the vehicle forming a second angle (B, C) in the horizontal direction with respect to the longitudinal axis of the vehicle. Photographing devices (16b, 16d / 26b, 26d / 36b, 36d) for photographing the second scene with a narrow viewing angle including the road surface
Feature point detecting means (100-300) for detecting density edge points (feature points) on the photographic images of the first scene and the second scene;
A first coordinate for converting a position on the screen of the density edge point of the photographing screen of the first scene into a position on a road system coordinate having the horizontal plane projection line of the center line of the visual field of the first scene as one axis. Value conversion means (100);
A second coordinate for converting the position on the screen of the density edge point of the photographing screen of the second scene into a position on a road system coordinate having the horizontal plane projection line of the center line of the visual field of the second scene as one axis. Value conversion means (200, 300);
The position of the density edge point converted by the first coordinate value conversion means (100) is rotated by a first angle (A), and the horizontal plane projection line of the front and rear axis of the vehicle is defined as one axis (Z axis) on road system coordinates. First rotation correcting means (100) for converting the position into a position;
The horizontal plane projection line of the front-rear axis of the vehicle obtained by rotating the position of the density edge point converted by the second coordinate value conversion means (200, 300) by the second angle (B, C) is defined as one axis (Z axis). Second rotation correcting means (200, 300) for converting to a position on a road system coordinate to be changed;
The entire sequence of the density edge points originating from the first scene and the density edge points originating from the second scene on the road system coordinates with the horizontal projection line of the longitudinal axis of the vehicle as one axis (Z axis). Approximation line detection means (100) for calculating a line approximating
The first angle (A) is changed to track the density edge point corresponding to the approximate line on the first scene, and the density edge point corresponding to the approximate line is tracked on the second scene. And a photographing direction control means (11, 21, 31, 16e, 26e, 36e, 100) for changing the second angle (B).
[0015]
If the photographing direction of the photographing device with respect to the longitudinal axis of the vehicle is fixed, the distant road surface deviates from the first and second scenes (especially the second scene) when the road is curved. Although the possibility is high, the photographing direction control means (11, 21, 31, 16e, 26e, 36e, 100) traces the density edge point corresponding to the approximate line, and the first angle (A) and the second angle ( (B) and (C) are changed, so that both the first scene and the second scene always include the road white line, and the possibility that the road white line is off the screen even when the road is curved is reduced. I do. That is, the reliability of detecting a distant white line on a curved road is high.
[0016]
(3) In one embodiment of the above (2), the photographing direction control means (11, 21, 31, 16e, 26e, 36e, 100) is provided at substantially the midpoint between two approximate lines on the first scene. Is defined as the center of the first scene, and the second angle is defined as the center of the second scene substantially at the midpoint between two approximation lines on the second scene. (B, C) is defined. According to this, both the screens of the first scene and the second scene have a right white line and a left white line that define the own vehicle traveling lane in substantially the right half and the left half of the screen. The reliability of tracking a white line on the road, particularly on a curved road, is high.
[0017]
(4) In one embodiment of the above (2), the apparatus further comprises an object (preceding vehicle / road obstacle) existing between the two approximate lines on the first scene and the second scene. Forward object detecting means (100) for detecting;
In the photographing direction control means (11, 21, 31, 16e, 26e, 36e, 100), the front object detection means (100) detects an object (preceding vehicle / road obstacle) in the first scene area. In some cases, the first angle (A) is set such that the object is substantially at the center of the first scene, and when no object is detected, the substantially middle point of two approximate lines on the first scene is defined as the first scene. The first angle (A) at the center of the vehicle is determined, and when the front object detection means (100) detects an object (preceding vehicle / road obstacle) in the second scene area, the object is regarded as a second object. The second angle, which is substantially the center of the second scene, is determined, and when no detection is performed, the substantially middle point of the two approximate lines on the second scene is substantially the center of the second scene. Two angles (A) are determined.
[0018]
According to this, the first angle (A) is changed so as to track a preceding vehicle when there is a preceding vehicle in the first scene area, and to track the preceding vehicle when there is a preceding vehicle in the second scene area. Then, the second angle (B) is changed. That is, the preceding vehicle is tracked. However, since the detection of the right white line and the left white line defining the own vehicle traveling lane is performed prior to the preceding vehicle detection, when the preceding vehicle deviates from the own vehicle traveling lane, the traveling lane white line is automatically detected. Tracking. When a preceding vehicle enters the vehicle running lane, the preceding vehicle is tracked.
[0019]
(5) In one embodiment of the present invention, the approximate line detecting means (100) calculates the entire series of density edge points as an arc by the least square method. Thus, when the road is curved, the radius of the curve can be easily obtained.
[0020]
(6) In the third embodiment of the present invention, the photographing device (16b, 16d / 26b, 26d / 36b, 36d) has a first wide viewing angle for photographing a short distance (10 to 30 m) road surface in front of the vehicle. A first rotary drive device (16d) for rotating the camera (16b) and the first camera (16b) in a horizontal direction with respect to the longitudinal axis of the vehicle to rotate the camera; To change the shooting direction of the second camera (26b, 36b) having a narrow viewing angle for shooting the road surface of 60 m to 60 m, and the second camera (26b, 36b) in the horizontal direction with respect to the longitudinal axis of the vehicle. And a second rotation driving device (26d, 36d) for driving the rotation.
[0021]
(7) In one embodiment common to the first to third embodiments, the photographing device (16b, 16d / 26b, 26d / 36b, 36d) photographs a road surface at a short distance (10 to 30 m) ahead of the vehicle. A first rotation driving device (16d) for rotating the first camera (16b) having a wide viewing angle and rotating the first camera (16b) in a horizontal direction with respect to the front-rear axis of the vehicle to rotate the camera; The second camera (26b) having a medium viewing angle for photographing a road surface at a medium distance (30 to 60m) and the second camera (26b) in a horizontal direction with respect to the longitudinal axis of the vehicle are rotationally driven to change the photographing direction. A second rotary driving device (26d), a third camera (36b) having a narrow viewing angle for photographing a road surface at a long distance (60 to 90m) ahead of the vehicle, and a third camera (36b) in a horizontal direction with respect to the vehicle longitudinal axis. of Third rotating device for rotating the camera to change the shading direction (36d), including.
[0022]
Other objects and features of the present invention will become apparent from the following description of embodiments with reference to the drawings.
[0023]
【Example】
FIG. 1A shows the configuration of an embodiment of the present invention. The first to third television cameras 16b, 26b and 36b supported by the rotating mechanisms 16d, 26d and 36d are connected to the first to third image processing devices 100 to 300, respectively. Each of the rotating mechanisms 16d, 26d and 36d has a built-in electric motor and a speed reducer for rotational drive, and a camera (16b, 26b, 36b) is fixed to an output rotation shaft of the speed reducer.
[0024]
The first to third rotating mechanisms 16d, 26d and 36d are supported by a single support frame and installed near the upper center of the front window in the vehicle MCR as shown in FIG. 1 (b). Each of the first to third television cameras 16b, 26b and 36b photographs a scene in front of the vehicle and outputs an analog signal of 512 × 512 pixels per frame.
[0025]
The first camera 16b is a wide-angle camera, and projects an area of approximately 24 m × 24 m, which is 30 m ahead of the camera and orthogonal to the center line of the visual field, to 512 × 512 pixels. The center line of the visual field intersects the road surface approximately 15 m in front of the vehicle on a horizontal road surface. The first camera 16b is focused on the intersection. If the photographing screen of the first camera 16b is, for example, the one shown in FIG. 2 and the road surface is horizontal, the road surface at Y = 150 on the screen is 30 m in front of the vehicle, and the road surface at Y = 350 is 10 m in front of the vehicle. is there. White line detection and vehicle detection described later are performed in the range of Y = 150 to 350 (feature point detection window 2).
[0026]
The second camera 26b is a standard angle-of-view camera, and projects an area of approximately 19m x 19m, which is 60m ahead of the camera and orthogonal to the center line of the visual field, on 512 x 512 pixels. The center line of the visual field intersects the road surface approximately 45 m in front of the vehicle on a horizontal road surface. The point of intersection is focused by the second camera 26b. On the photographing screen of the second camera 26b, the road surface at Y = 150 is 60 m in front of the vehicle, and the road surface at Y = 350 is 30 m in front of the vehicle. White line detection and vehicle detection described later are performed in the range of Y = 150 to 350 (feature point detection window 2).
[0027]
The third camera 36b is a telephoto camera, and projects an area of approximately 13 m × 13 m, which is 90 m ahead and orthogonal to the center line of the visual field, to 512 × 512 pixels. The center line of the visual field intersects the road surface approximately 75 m in front of the vehicle on a horizontal road surface. The third camera 36b is focused on the intersection. On the shooting screen of the third camera 36b, the road surface at Y = 150 is 90 m ahead of the vehicle, and the road surface at Y = 350 is 60 m ahead of the vehicle. White line detection and vehicle detection described later are performed in the range of Y = 150 to 350 (feature point detection window 2).
[0028]
Each of the first to third cameras 16b, 26b, and 36b has a distance in front of the vehicle and a photographing range (entire screen), and an area for detecting a white line and detecting a vehicle (Y = 150 to 350: an area of the feature point detection window 2). ) Is shown in FIG. FIG. 4A shows the relationship between the vehicle front distance and the photographing width of each of the first to third cameras 16b, 26b, and 36b, and FIG. The relationship of the error σ of the distance measurement on the shooting screen is shown.
[0029]
When the road is curved, a camera with a narrower viewing angle is more likely to photograph a direction deviating from the road surface. Therefore, in this embodiment, the cameras 16b, 26b and 36b are supported by the rotating mechanisms 16d, 26d and 36d, and the lane detection 10 m to 90 m in front of the vehicle and the forward lane detection are performed mainly by the CPU 11 described later of the image processing apparatus 100. When the vehicle is detected and the preceding vehicle is not detected, the cameras 16b, 26b and 36b are individually steered so that the center of the field of view of the cameras 16b, 26b and 36b is aligned with the center of the lane as shown in FIG. . A camera having (detected) a vehicle in the field of view (Y = 150 to 350 described above) steers the vehicle so that the vehicle is positioned at the center of the field of view.
[0030]
FIG. 6 shows a configuration of the first image processing apparatus 100. The image processing apparatus 100 is mainly composed of a microcomputer (hereinafter referred to as a CPU) 11, a read-only memory (ROM) 12 storing a control program, and a parameter being processed on its bus line. A read / write memory (RAM) 13 for storing data and input / output ports (I / O) 15 and 16 to which various components are connected are connected. An analog image signal (video signal) output from the first camera 16b is supplied to an A / D converter 16c. In the A / D converter 16c, the analog image signal from the first camera 16b is converted into digital data (gradation 0) of 256 gradations (gradation 0 is a black level and gradation 255 is a white level) for each pixel. (Data) and gives it to the image memory 15a. The image memory 15a has a gradation data storage area for several pages, assuming that an area for storing gradation data of one screen (512 × 512 pixels) is one page, and one bit information (2 bits). It has a binary data storage area for storing value data for several screens.
[0031]
The CPU 11 controls the aperture of the first camera 16b, the output level of the image signal, and the like via the television camera controller 16a, and synchronizes the input / output of the A / D converter 16c and the writing process of the image memory 15a. Control. The DMA device 14 is connected to the image memory 15a, and writes image data sent from the image processing device 200 or 300 into a memory area designated by the CPU 11.
[0032]
The electric motor of the turning mechanism 16d is connected to (a motor driver of) a steering controller 16e, and the CPU 11 gives a target angle to the controller 16e, and the controller 16e is driven by the turning mechanism 16d. The electric motor of the rotation mechanism 16d is driven forward (rightward) or reversely driven (leftward) so that the steering angle A of the first camera 16b is set to the target angle An, so that the steering angle A of the first camera 16b is increased. When the angle matches the target angle An, the electric motor is stopped. The steering angle A is an angle formed by the center line of the camera visual field (the projection line on the horizontal plane) with respect to the front-rear axis of the vehicle (the projection line on the horizontal plane). When both coincide, the steering angle = 0. is there. In this embodiment, the steering range (adjustment range of the angle A) is 60 ° in the right direction (+) and 60 ° in the left direction (−).
[0033]
7 and 8 show the configurations of the second image processing device 200 and the third image processing device 300. The configuration of these image processing systems 200 and 300 is substantially the same as the configuration of the first image processing apparatus 100. However, the DMA devices 24 and 34 of the second and third image processing devices 200 and 300 read the image data in the areas designated by the CPUs 21 and 31 from the memories 25a and 25b, respectively, and read the image data of the first image processing device 100. The DMA device 14 writes the transferred image data in an area designated by the CPU 11 in the memory 15a.
[0034]
The CPU 11 of the first image processing apparatus 100 is connected to the serial data communication lines of the CPUs 21 and 31 of the second and third image processing apparatuses 200 and 300. Is instructed to the CPUs 21 and 31 (slaves) of the second and third image processing apparatuses 200 and 300. The CPU 11 detects a travel lane (white line) on an image 10 to 90 m in front of the vehicle based on the captured image data of the first camera 16b and the image data transferred from 200 and 300. And a vehicle on the traveling lane (which may be a preceding vehicle / an obstacle on the road). If there is a detected vehicle, the steering angle for setting the detected vehicle at the center of the screen. If not, the steering angle for setting the center of the traveling lane at the center of the screen is calculated for each of the first to third cameras, and the curve information (curve radius) and the detected vehicle information (front position) are calculated. The data is transferred to a host microcomputer (computer system mainly composed of a CPU: hereinafter referred to as an MPU) 18 via a communication controller 17. The steering angle An addressed to the first camera 16b is given to the steering controller 16e, and the steering angles Bn, Cn addressed to the second and third cameras 26b, 36b are transferred to the CPUs 21 and 31 via serial communication lines. These CPUs 21 and 31 provide the received steering angles Bn and Cn to the steering controllers 26e and 36e.
[0035]
The MPU 18 is a computer system that performs vehicle speed control (constant speed running control / inter-vehicle distance control) and vehicle traveling direction control (brake pressure control for each wheel brake). The curve information is used for vehicle speed control and vehicle traveling direction control.
[0036]
FIGS. 9, 10 and 11 show an outline of the processing operation of the CPU 11. FIG. Many parts of the processing operation of the CPU 11 are common or similar to the processing operation of the CPU 1 of the object detection device in front of the vehicle presented in JP-A-6-213660 (title of the invention: method of detecting an approximate straight line of an image). It is known. Therefore, in FIGS. 9 to 11, blocks representing processing operations substantially the same as the known processing operations are indicated by thick lines (thick solid lines or thick broken lines), and only the outline of the contents is described below. Blocks representing processing operations not disclosed in the above-mentioned gazettes of the CPU 11 of this embodiment are represented by thin lines (thin solid lines or thin broken lines).
[0037]
Blocks representing processing operations executed by the CPUs 21 and 31 in the same manner as the CPU 11 are indicated by solid lines (thick solid lines or thin solid lines), and blocks representing processing operations executed only by the CPU 11 are indicated by broken lines (thick broken lines or thin broken lines). did. Hereinafter, the processing operation of the CPU 11 will be described with reference to FIGS. 9 to 11, and the processing operation of the CPUs 21 and 22 will also be described.
[0038]
First, refer to FIG. When the power of the first image processing apparatus 100 is turned on, the CPU 11 performs initialization of input / output ports, initialization of internal registers and internal counters (initialization A), and then executes the second and third steps. The CPU 21 or 31 of the third image processing apparatus 200 or 300 is instructed to read an image (T1).
[0039]
Next, the CPU 11 converts the analog image signal of one screen into a digital signal at a predetermined cycle and writes the digital image signal into the input data memory area of the image memory 15a (image input B). Similarly, the CPUs 21 and 31 also write one screen of image data in the image memories 25a and 35a in response to the image reading command.
[0040]
When the image data of one screen is written into the memory, the CPU 11, CPU 21 and CPU 31 give the target steering angles A, B and C to the steering controllers 16e, 26e and 36e, respectively (A1). The target steering angles A, B, and C are written in registers A, B, and C, respectively, and the initial value is 0. After executing “calculation of camera steering angle” (A4) described later, It is the calculated value.
[0041]
Next, only the CPU 11 executes “one set of feature point detection windows” (C1) to “calculation of roll angle θr and pan movement amount Xp” (C5). These contents will be described below.
[0042]
C1. “One set of feature point detection windows” (FIG. 12)
This content is shown in FIG. As shown in FIG. 12 (b), the upper left of the photographing screen of the first camera 16b (image data is gradation data in the input data memory) is defined as the origin (0, 0), and one point is determined. An image area having (0,350) and another point (511,511) as diagonal corners is defined as a feature point detection area (window 1). That is, the X coordinate value 0 of the upper left corner of the area to be determined is set to the register XUL, the Y coordinate value 350 is set to the register YUL, the X coordinate value 511 of the lower right corner of the area is set to the register XLR, and Y The coordinate value 511 is written in the register YLR (step 1 in FIG. 12A; the word "step" is omitted in parentheses, and only the step number is described). The value indicated by the data in each of these registers is hereinafter represented by the register symbol itself.
[0043]
In response to the "feature point detection (UP)" process C2 described below, the above-mentioned "feature point detection window set" C1 uses a block (feature) indicated by a two-dot chain line as shown in FIG. This means that the point detection window 1) has been designated as the processing target area. As shown in FIG. 12B, this area sufficiently includes a front edge image of the hood of the vehicle. Next, "feature point detection (UP)" C2 is executed.
[0044]
C2. "Feature point detection (UP)" C2
Here, from the pixel (0, 509) two pixels above the lower left corner (0, 511) of the feature point detection window 1 in the Y direction, the Y direction coordinate value is gradually reduced to 350 from the bottom to the top in the Y direction. , The pixels distributed in the direction (on the straight line of X = 0) are sequentially designated as the target pixel, and the pixel (Xs, Ys + 1) one pixel lower than the target pixel (Xs, Ys). ) And the sum of the luminances represented by the gradation data of the pixel (Xs, Ys + 2) two pixels below, and the pixel (Xs, Ys−1) one pixel above the pixel of interest (Xs, Ys) and two pixels above. The absolute value of the difference between the sum of the luminances represented by the gradation data in the input data memory of the pixel (Xs, Ys-2), that is, the absolute value of the gradation differential value in the vertical direction (Y direction) I do. This is defined as a gradation differential value D (Xs, Ys) of the target pixel point (Xs, Ys). The tone differential value D (Xs, Ys) is the absolute value of the difference between the image density sum of the upper two pixels and the image density sum of the lower two pixels of the target point (Xs, Ys). It represents a change in density in the vertical direction (Y) at the boundary, and has a large value at the contour line extending in the horizontal direction of each image on the screen. It is checked whether the gradation differential value D (Xs, Ys) is equal to or greater than a threshold value Th1 (set value). If the difference value is equal to or greater than Th1, the binary data assigned to a certain area of the image memory 15a is determined. "1" is written to the address E (Xs, Ys) corresponding to the target pixel point (Xs, Ys) in the table. This information “1” means that the image is a contour line extending in the horizontal direction. When such processing is performed for one line (X = 0, Y = 509 to 350) in the Y direction, the scanning line is shifted to the right and the next one line (X = 1, Y = 509 to 350) Perform the same processing. Similarly, the same processing is performed up to the last line (X = 511, Y = 509 to 350).
[0045]
By the above processing, the binary data table E (Xs, Ys), Xs = 0 to 511, and Ys = 509 to 350 in the image memory 15a are stored in the feature point detection window 1 (b in FIG. 12). Of the image extending in the horizontal direction has been written. Since the feature point detection window 1 is set in an area including the leading edge image of the hood, the binary data table E (Xs, Ys) includes information representing the leading edge of the hood. In this embodiment, the leading edge of the hood is regarded as a straight line, and the straight line is detected in the next “bonnet detection” C3.
[0046]
C3. "Bonnet detection" C3 (Fig. 13)
First, the outline is explained. Here, a straight line is detected using a known “Hough transform”. The Hough transform converts information (for example, the binary data table E (Xs, Ys)) expressed in a rectangular coordinate system (XY coordinate system; for example, a pixel distribution in the feature point detection window 1) into: It is a method of expressing in a polar coordinate system (ρ-θ coordinate system),
ρ = Xcos (θ) + Ysin (θ)
Is converted. One point (pixel) in the XY coordinate system becomes a certain curve in the ρ-θ polar coordinate system (an orthogonal coordinate plane in which ρ and θ are assigned to one of the two orthogonal axes and the other), and lies on a straight line in the XY coordinate system. When each point is represented by each curve in the ρ-θ coordinate system, these curves intersect at one point. By substituting the intersection into the conversion formula to the XY coordinate system, a straight line formula in the XY coordinate system is obtained.
[0047]
Therefore, the coordinates having the feature point information “1” on the binary data table E (Xs, Ys) are converted into ρ-θ polar coordinate values, and the intersection of the curves is obtained in the ρ-θ polar coordinate system. By substituting the intersection into the conversion formula to the XY coordinate system, a straight line connecting the feature points can be obtained. However, a straight line simply connecting two points can be directly obtained in the XY coordinate system, and there is no point in performing conversion to the ρ-θ polar coordinate system. However, the contour of the bonnet on the screen is such that the tip contour of the actual hood is curved, or the bonnet is curved or rough due to the light hitting the bonnet, etc. When the bonnet contour is approximated by a straight line, if a straight line connecting two points having the characteristic point information “1” on the binary data table E (Xs, Ys) is simply obtained, the approximation error is too large. However, it is possible that a contour line is extracted from a non-bonnet. As shown in FIG. 12B and FIG. 2, the bonnet contour is distributed over a wide area, so that the feature point information “1” is included in the binary data table E (Xs, Ys). There are many. Therefore, a large number of straight lines connecting two points (pixels) with the “1” are obtained. When a straight line descending to the right representing a large number of obtained straight line groups is determined, the straight line best approximates the right half of the contour line of the hood tip (FIG. 13B). Similarly, when a straight line descending to the left representing a large number of obtained straight line groups is determined, the straight line best approximates the left half of the bonnet tip contour (FIG. 13B). However, if such statistical processing is performed in the XY coordinate system, the processing becomes extremely complicated. However, when the coordinates (feature points) having the feature point information “1” on the binary data table E (Xs, Ys) are converted to the ρ-θ polar coordinate system using the Hough transform, each feature point is represented. The intersections of the curves are concentrated at two points (corresponding to the right half contour and the left half contour of the hood). Therefore, the feature points of the binary data table E (Xs, Ys) are subjected to the Hough transform, and the frequency at which the curve passes (the number of curves passing through one point) is counted at each point on the ρ-θ polar coordinate system. Is the largest point and the next largest point are selected, and each selected point is substituted into the conversion formula to the XY coordinate system to obtain a straight line expression representing the right half and the left half of the contour line of the hood. Is obtained. In this embodiment, the contour of the bonnet tip is detected according to such a logic, that is, by Hough transform and counting the number of times the curve passes through each point on the ρ-θ polar coordinate system.
[0048]
By the way, the count of the degree of intersecting curve at each point on the ρ-θ polar coordinate system is determined by each point of the entire ρ-θ polar coordinate system having a size corresponding to the size of the binary data table E (Xs, Ys). In this case, the count processing of the number of times the curve passes becomes enormous, and the straight line detection processing takes a long time. Therefore, in this embodiment, the lowermost left and right intermediate point (X = 255, Y = 511) of the binary data table E (Xs, Ys) is determined as the origin (0, 0) of the ρ-θ polar coordinate system. And the binary data table E (Xs, Ys) is divided into right and left, and the right half is subjected to the first Hough transform to obtain the characteristic points of the binary data table E (Xs, Ys). Is converted to a low-density (large sampling pitch) ρ-θ polar coordinate system, and the frequency at which the curve passes through each point of the polar coordinate system (the number of curves passing through one point) is counted. Prediction point). Next, in the second Hough transform, a small range ρ-θ region centered on the first prediction point is set, and the binary data table E ( Xs, Ys) are converted, the frequency at which the curve passes through each point of the ρ-θ polar coordinate system is counted, and the point having the maximum frequency (second predicted point) is obtained. Then, in the third Hough transform, a ρ-θ region with a smaller range centered on the second prediction point is set, and the binary data table E ( Xs, Ys) are converted, the frequency at which the curve passes through each point of the ρ-θ polar coordinate system is counted, and the point having the maximum frequency (third predicted point) is obtained. Then, the straight line represented by the third prediction point is determined as the right half of the contour line of the hood tip (12R to 14R in FIG. 13). For the left half of the binary data table E (Xs, Ys), the first Hough transform, the second Hough transform, and the third Hough transform are similarly performed to obtain a straight line representing the left half of the bonnet tip contour. (12L to 14L in FIG. 13).
[0049]
Next, the content of the “bonnet detection” C3 will be described in detail with reference to FIG. First, parameters for detecting a straight line approximating the right half of the contour line of the bonnet tip are set (12R). Then, a point (R _m3 , T _m3 ) Is detected (13R). R _m3 Is the ρ value of the ρ-θ polar coordinate system, T _m3 Is the θ value. The coordinate value of the obtained point is stored in the register R _mR , T _mR (14R). A similar process is performed to detect a straight line approximating the left half of the bonnet tip contour (12L to 14L). The details of the "straight line fitting" 13R and 13L for detecting a straight line by the Hough transform are the same as those disclosed in Japanese Patent Application No. 5-7017, and a detailed description thereof will be omitted. The "straight line fit" 13R detects a straight line in the right half area of the window 1 (FIG. 2), and the "straight line fit" 13L detects a straight line in the left half area.
[0050]
Each of the straight lines obtained by the “straight line fitting” 13R and 13L is a point (R _mR , T _mR ) And (R _mL , T _mL ) And expressing these straight lines in an XY coordinate system,
R _mR = Xcos (T _mR ) + Ysin (T _mR ) and
R _mL = Xcos (T _mL ) + Ysin (T _mL )
It becomes. These straight lines are as shown in FIG.
[0051]
C4. "Line intersection calculation" C4 (FIG. 14)
After terminating the above “bonnet detection” C3, the CPU 11 calculates the intersection (Xc, Yc) of these straight lines in the “straight line intersection calculation” C4 as follows:
Xc = Xch + [RmR × sinTmL−RmL × sinTmR] / [cosTmR × sinTmL−cosTmL × sinTmR] (1)
yc = Ych + [RmR × cosTmL−RmL × cosTmR] / [sinTmR × cosTmL−sinTmL × cosTmR] (2)
In this calculation, Xch is the center position in the X direction of the area of the binary data table E (Xs, Ys) (Xch = 255), Ych is the lowermost position (Ych = 509), and the point (Xch, Ych) The parentheses are defined at the polar coordinate origin in the above-described straight line detection.
[0052]
C5. "Calculation of roll angle and pan movement amount" C5 (FIG. 14)
Here, the roll angle is, as shown in FIG. 14A, an angle formed between a straight line (vertical axis of the vehicle) that bisects the angle at which the detected two straight lines intersect with each other and the Y axis of the photographing screen. θr, the angle between the center line of view of the first camera 16b and the longitudinal axis of the vehicle when they are projected on a horizontal plane, ie, the roll direction of the center line of view of the first camera 16b with respect to the longitudinal axis of the vehicle. Is the deviation angle. The pan movement amount Xp is a lateral shift amount of the center of the bonnet tip (the intersection of the two straight lines) with respect to the vertical axis dividing the screen into two equal parts on the shooting screen. When calculating the inter-vehicle distance with respect to a vehicle (preceding vehicle) in front of the vehicle in the traveling direction, if the roll angle θr or the pan movement amount Xp is large, the vehicle deviates from the traveling direction of the vehicle. The probability that a certain vehicle or object is erroneously recognized as a vehicle or the like on the own vehicle lane (the lane on which the own vehicle is running) increases. To reduce this probability, the mounting angle of the camera 16b may be adjusted so that both the roll angle θr and the pan movement amount Xp become zero, but this adjustment is difficult and troublesome. Therefore, a certain roll angle θr and pan movement amount Xp cannot be avoided. Therefore, in this embodiment, the roll angle θr and the pan movement amount Xp are adjusted to zero on the screen by “screen calibration” C. In order to calculate the adjustment allowance, the processing of the above-described C1 to C4 is performed. And in the "roll angle, pan movement amount calculation" C5, the roll angle θr and the pan movement amount Xp are calculated and stored in the roll angle register θr and the pan movement amount register Xp:
θr = (TmL + TmR) / 2−π / 2 (3)
Xp = Xc-256 (4).
[0053]
The CPU 11 transmits the calculated roll angle θr and pan movement amount Xp to the CPUs 21 and 31 via the inter-CPU serial data communication line (T2 in FIG. 9).
[0054]
C6-8. "Image Correction" C6-8
The CPUs 11, 21, and 31 execute "image correction" C6 to C8. The contents will be described with respect to the CPU 11 (the same applies to the processing of the CPUs 21 and 31). The CPU 11 performs the "image correction" including the following "correction image memory initialization" C6, "image rotation / translation" C7, and "interpolation" C8. Thus, the address (x, y in FIG. 14B) of the gradation data of each pixel of the input data memory (gradation data storage area for one screen) of the image memory 15a is Address of the screen coordinate system representing the captured image when θr = 0 and Xp = 0, where x ′ and x ′ in FIG. 14B are used. y '), and writes it into a corrected image memory (a gradation data storage area for one screen of the image memory 15a) corresponding to the input data memory (C6, C7).
[0055]
C6. "Corrected image memory initialization" C6
When such rotation and parallel movement are performed, pixels having no gradation data are generated on the corrected screen. In the "correction image memory initialization" C6, "-1" which does not exist in the captured image data (gradation data) is stored in the "correction image memory initialization" C6 so that such pixels can be recognized later. Write the data to be represented.
[0056]
C7. "Image rotation translation" C7
In this case, each of the addresses (x, y) of the input data memory; x = 0 to 511, y = 0 to 511 is represented by a screen coordinate system (corrected image) representing a captured image at θr = 0, Xp = 0. (X ', y'); x '= 0 to 511, y' = 0 to 511, and the gradation data of the address (x, y) of the input data memory is The address (x, y) is written to the converted address (x ', y') in the corrected image memory. As a result, if the gradation data of the corrected image memory is temporarily displayed on the two-dimensional display, both the roll angle θr and the pan movement amount Xp shown in FIG. 14A are substantially zero, as shown in FIG. The image is displayed. However, this "image rotation / translation" may cause an address (pixel = blank pixel) where no gradation data exists in the corrected image memory. In this case, the above-mentioned "corrected image" By the "memory initialization" C6, data indicating "-1" remains.
[0057]
C8. "Interpolation" C8
In the "interpolation" C8, an average value of the gradation data of the four surrounding pixels is assigned to a pixel having no gradation data. If any of the surrounding four pixels is a blank pixel, this assignment is not performed. In this "interpolation" C8, the addresses (pixels) of the corrected image memory are sequentially designated, and it is determined whether the designated pixel (pixel of interest) is a blank pixel (memory data indicating "-1"). If the target pixel is a blank pixel, as shown in FIG. 15, gradation data is stored in the pixel B on the left, the pixel C on the right, the pixel A on the upper side, and the pixel D on the lower side. It is checked in this order whether any one of these four pixels is a blank pixel. Then, the processing for the target pixel is terminated (leaving the blank pixel as it is), and the target pixel is moved to the next. When it is checked whether or not there is gradation data for each of the four pixels, if there is gradation data, it is added to the contents of the accumulation register Is, and the obtained sum is updated in the accumulation register Is. Then, the content of the count register Ns is incremented by one. When this is completed for all four pixels, the average value Ic (x, y) of the gradation data of these pixels is calculated and stored in the target pixel.
[0058]
With the above, the "screen calibration" C (FIG. 3) is completed, and the CPUs 11, 21, and 31 then determine whether the data in the register Nve is 0 (success in tracking the point at infinity: described later). Is checked (A2 in FIG. 10). In the later-described “calculation of infinity point” (D4), when the left and right white lines defining the own vehicle traveling lane have been successfully detected and their intersection (referred to as infinity point) has been successfully calculated. , The data of the register Nve is 0. If the data of the register Nve is 0, the CPU 11 executes "lane detection window setting" (A3), and if the data of the register Nve is not 0, "set the feature point detection window 2" (A3). Execute D1). The CPUs 21 and 31 execute the same processing.
[0059]
A3. "Lane detection window setting" (A3)
Here, in order to remove a background image (disturbance) such as a tree and a road shoulder from a scene in front of the vehicle and detect an edge point of only a white line, a window for limiting an edge detection range is set. In order to increase the processing speed of the white line detection, data defining a plurality of lane detection windows corresponding to the lane shape is stored in the ROM 12, and the previous "calculation of infinity point" (D4) At the infinity point (Xv, Yv) calculated in the above and the left and right white lines calculated in the “calculation of the left and right white line (WL)” (D5) on the starting positions (xL, yL) and (xr, yr) on the screen. Based on this, a lane detection window is determined.
[0060]
The shape of the lane detection window is an inverted V-shape as shown in FIG. 16, having a total of 45 patterns, each having 5 vertices and 3 left and right ends. The window shape of each pattern is determined from the following viewpoint so that a white line can always be captured.
(1) Window width (Wc, WL, Wr: FIG. 16)
The width that can absorb the fluctuation of the tilt angle of the camera 16b due to the pitching of the vehicle
(2) Vertex coordinates (C ₀ ~ C ₄ : Fig. 16)
The infinity point moving on the screen is always captured in the window by the yaw angle between the vehicle and the white line, the curve path, and the steering of the camera 16b.
(3) Left and right end coordinates (L0, L1, L2, r0, r1, r2: FIG. 16)
Due to the yaw angle between the vehicle and the white line, the curve path, and the steering of the cameras 16b, 26b, 36b, even if the position of the lane projected on the left and right ends on the screen changes, the white line is captured in the window. .
[0061]
The dimensions (in pixel units) shown in FIG. 16 are as follows:
Pitch of window vertices C0 to C4 (C _n-1 -C _n )
Camera 16b: 50, Camera 26b: 70, Camera 36b: 80
The center C2 is the center of the screen in the x direction.
Pitch of window left and right ends L0-L2, r0-r2
Camera 16b: 60, Camera 26b: 70, Camera 36b: 80
L1 and r1 are reference lane widths, and the end of the white line (a straight line approximating the white line) that appears when the vehicle is running in the center when the lane is horizontal and straight, at the screen edge. Is a position,
Window vertex width Wc:
Camera 16b: 50, Camera 26b: 70, Camera 36b: 80
Window left and right edge widths WL, Wr:
Camera 16b: 60, camera 26b: 70, camera 36b: 80.
[0062]
In the "lane detection window setting" (A3), the CPU 11 measures a point (one Ci of C0 to C4) closest to the x coordinate value (Xv) of the previously detected infinity point among C0 to C4. Then, the x-coordinate value and the y-coordinate value of the infinity point detected last time are determined as the vertices of the reference position (inverted v-shaped line) of the lane detection window, and the left and right white lines detected last time are displayed on the screen. A point (one Li of L0 to L2, one ri of r0 to r2) that is closest to the y coordinate values (yL) and (yr) of the start position (xL, yL) and (xr, yr) is searched, The x-coordinate value of the start position detected last time and the y-coordinate value of the detected point are set to the left and right ends of the reference position (inverted v-shaped line) of the window, and the three points thus determined and Wc, WL, Wr are used. An area (lane detection window) on the screen to be defined is determined. The memory to which the determined lane detection window is applied is a corrected image memory.
[0063]
D1. "2 sets of feature point detection windows" D1
This content is similar to the above-mentioned "set of feature point detection windows 1" C1, but the size and position of window 2 are different from window 1, and the memory to which window 2 is applied is a corrected image memory. As shown in FIG. 2, window 2 is set above window 1.
[0064]
D2. "Feature point detection (UP)" D2
When the window 2 is set, a feature point in the window 2 on the corrected image memory is detected. This content is the same as the above-mentioned "feature point detection (UP)" C2, and the "1" information representing the feature point in the "feature point detection (UP)" D2 also includes the binary data table E (Xs, Ys). However, the difference is that the detection area is the window 2 and the data to be processed is the data on the corrected image memory.
[0065]
When a lane detection window is set, a feature point in the area inside the lane detection window on the corrected image memory is detected.
[0066]
D3. "Left and right white line detection" D3
Here, a straight line approximating the right end white line and a straight line approximating the left end white line of the own vehicle lane are detected by processing similar to the above-mentioned "bonnet detection" C3. However, since window 2 (or the lane detection window) is different in size and position from window 1, the polar coordinate origin is different (FIG. 2).
[0067]
"Line Fit"
Note that "straight line fit" is applied to the right half of window 2 (or the lane detection window). Here, first, the feature points on the binary data table E (Xs, Ys) are converted into the ρ-θ polar coordinate system, and the frequency count value through which the curve of each point (r, t) of this coordinate system passes is calculated. The data table HGn (r, t) assigned to a certain area of the image memory 15a to be written, that is, n = 1 in this case, is cleared. Next, Hough transformation parameters (conversion area and Hough transformation density) are set, and the first Hough transformation (HG1) is performed.
[0068]
"Hough transform (HG1)"
In this case, the characteristic point ("" in the right half of the area corresponding to the window 2 (or the lane detection window) in one screen corresponding to the corrected image memory of the binary data table E (Xs, Ys) is used. 1) Check if there is information). Every time there is a feature point, the point is defined on the X-Y coordinate at the origin (Xch = 255, Ych = 350), θ is in the range of 0 to π / 2, and the unit of θ is (π / 2). × (1/32) position (r, t) on polar coordinates ρ-θ, t = 0 to 31, r

And the data at the address corresponding to the position (r, t) in the data table HGn (here, n = 1) is incremented by one. When one point on the XY coordinate is converted into the polar coordinate ρ-θ, it becomes a curve on the polar coordinate ρ-θ. Therefore, the converted positions (r, t) are 32 points corresponding to t = 0 to 31 respectively. (These series are curves). Since the data of the addresses assigned to these positions (r, t) in the data table HGn is incremented by one, the data of each address is represented on the ρ-θ coordinate of each feature point on the XY coordinate. It represents the frequency through which the curve passes.
[0069]
When the above equation (5) is modified, the following equation (5a) is obtained.
[0070]

This can be expressed by the following equation (5b).
[0071]

Here, a = (Te−Ts) / Td, b = Ts, c = Rs, and d = Rd / (Re−Rs).
[0072]
In the equation (5), (Xch, Ych) is the position on the XY coordinate of the polar coordinate origin, and here,
Xch = 255, Ych = 350
It is. Xs is the X coordinate value of the feature point on the XY coordinate, and Ys is the Y coordinate value.
[0073]
here,
θ = [(t · (Te−Ts) / Td) + Ts]
Which means that
(Te−Ts) / Td = (π / 2) / 32,
Ts = 0
So

And t = 0 to 31, so that θ = 0, θ = π / 64, θ = 2π / 64, θ = 3π / 64,..., 31π / 64, and π / 64 as a minimum unit The ρ values r of 32 points are calculated. In other words, in the first Hough transform, each feature point in the right half area of the window 2 has a position (r, r, r) on the polar coordinate plane in which θ has a minimum unit of π / 64 and is 0 or more and less than π / 2. t).
[0074]
Rd / (Re-Rs) in the above formula (5) is:
Rd / (Re-Rs) = 32/256
It is. This is because θ = π / 64 is one unit of t, that is, π / 2 is divided into 32, and correspondingly, the range 0 to 256 of ρ is similarly divided into 32. . That is, one unit of r is 256/32.
[0075]
In summary, in the “Hough transform (HG1)”, the expression (5) is specifically the following expression (5-1).
[0076]

This is transformed into the form of equation (5b),

In the expression, a ₁ = Π / 64, b ₁ = 0, c ₁ = 0, d ₁ = 1/8.
[0077]
As described above, in the first Hough transform, the unit of θ of the ρ-θ polar coordinate is set to be coarse as π / 64, and the unit of ρ is set to be coarse as 256/32, so that one feature point (Xs, Ys) Is converted into ρ-θ polar coordinates (the number of times t is switched 32, ie, the number of times of performing the operation of equation (1)) is very small, the number of r data bits is small, and each feature point is converted into ρ-θ polar coordinates. And the frequency count processing speed is fast.
[0078]
When terminating the “Hough transform (HG1)” that is the first Hough transform, the CPU 11 executes the first “maximum point search”.
[0079]
First "Maximum point search"
The data (frequency) of each address (r, t) of the above-mentioned data table HGn (here, n = 1) is sequentially read, and the read data is compared with the data of the register Gm, and the read data is larger. And the read data are updated in the register Gm, and at this time, the read data address (r, t) is updated in the register rm1 and t in the register tm1. When such processing is completed for all the addresses of the data table HGn, the maximum value of the frequency in the data table HGn is stored in the register Gm, and the address (rm1, tm1) where the maximum value is stored in the registers rm1 and tm1. Is stored.
[0080]
When the first “maximum point search” is completed, the CPU 11 replaces the address (rm1, tm1) expressed in the rt polar coordinate system with the ρ-θ polar coordinate system address (ρ-θ polar coordinate system corresponding to the XY coordinate system screen). Rm1, Tm1). here,
Rm1 = rm1 · (Re1-Rs1) / Rd1 (6)
Tm1 = tm1 · (Te1-Ts1) / Td1 (7)
Is calculated, and the calculated Rm1 is stored in the register Rm1, and the calculated Tm1 is stored in the register Tm1. (Re1-Rs1) = 256
Rd1 = 32
(Te1-Ts1) = π / 2
Td1 = 32
Equation (6) means Rm1 = 8 · rm1 and equation (7) means Tm1 = (π / 64) · tm1
Means When the addresses (Rm1, Tm1) are substituted into the conversion formula for the XY coordinate system, the address (Rm1, Tm1) is in the right half area of the window 2 (or the lane detection window) on the screen displaying the image data of the corrected image memory. An equation indicating a certain straight line (a straight line detected by the low-density Hough transform 67: hereinafter, referred to as a first detection straight line) is obtained.
[0081]
Next, the CPU 11 clears the data of the data table HGn (r, t) allocated to a certain area of the image memory 15a. Next, a conversion parameter of “Hough transform (HG2)” which is the second Hough transform is set. In the first Hough transform described above, the range Ts to Te of θ is 0 to π / 2, the unit (Te−Ts) / Td is π / 64, the range Rs to Re of ρ is 0 to 256, and the unit Rd / Although (Re−Rs) is set to 、, here, the range of θ and ρ is set to a small range centered on the above (Rm1, Tm1), and the units of θ and ρ are also set to be small. In particular,
Rs = 8 (rm1-2)
Re = 8 (rm1 + 2)
Rd = 32
Ts = (π / 64) (tm1-2)
Te = (π / 64) (tm1 + 2)
Td = 32
Set. Note that rm1 = Rm1 / 8 and tm1 = Tm1 / (π / 64). Then, the second “Hough transform (HG2)” is performed.
[0082]
"Hough transform (HG2)"
The content of the second “Hough transform (HG2)” is the same as the content of the first “Hough transform (HG1)”, but the content of the arithmetic expression (the above formula (5)) is different. That is, the equation (5) is specifically the following equation (5-2).
[0083]

This is transformed into the form of equation (5b),

In the first conversion described above,
a ₁ = (Π / 64), b ₁ = 0, c ₁ = 0, d ₁ = 1/8
Against
a ₂ = (Π / 512), b ₂ = Π (tm1-2) / 64, c ₂ = 8 (rm1-2), d ₂ = 1
It becomes.
[0084]
Since t = 0 to 31,
θ = π (tm1-2) / 64,
θ = (π / 512) + π (tm1-2) / 64,
θ = 2 (π / 512) + π (tm1-2) / 64,
θ = 3 (π / 512) + π (tm1-2) / 64,
・
・
・
θ = 31 (π / 512) + π (tm1-2) / 64,
And the ρ value r corresponding to θ at 32 points is calculated using (π / 512) as the minimum unit. That is, in the second Hough transform, each feature point in the right half area of the window 2 has a minimum unit of θ (π / 512) (one unit of t is π / 512) and θ = π The position is converted to a position (r, t) on the polar coordinate plane in a range of not less than (tm1-2) / 64 and not more than θ = 31 (π / 512) + π (tm1-2) / 64. The range of ρ is 8 (rm1-2) or more and 8 (rm1 + 2) or less, and the unit of ρ divides this range into 32. That is, one unit of r is 32/32 = (1/1) = 1. Therefore, the second Hough transform converts the feature points in the right half of the window 2 (or the lane detection window) into polar coordinates having a narrower range and a higher density than the first Hough transform. In one polar coordinate transformation of a feature point, t = 0 to 31, ie, 32 calculations are performed in the second Hough transformation, so that the processing time is almost the same as that of the first Hough transformation. The number of operations for converting one feature point (Xs, Ys) to ρ-θ polar coordinates is extremely small, the number of r data bits is small, the speed for converting each feature point to ρ-θ polar coordinates is fast, and the frequency High count processing speed.
[0085]
When the “Hough transform (HG2)” that is the second Hough transform ends, the CPU 11 executes the second “maximum point search”. The contents are the same as those in the first "maximum point search" described above. When this process ends, the register Gm stores the maximum value of the frequency in the data table HGn, and the registers rm2 and tm2 store the addresses (rm2 and tm2) where the maximum value exists.
[0086]
When the second “maximum point search” is completed, the CPU 11 replaces the address (rm2, tm2) expressed in the rt polar coordinate system with the ρ-θ polar coordinate system address (corresponding to the XY coordinate system screen). Rm2, Tm2). The operation expression is
Rm2 = rm2 · (Re2-Rs2) / Rd2 + Rs2 (8)
Tm2 = tm2 · (Te2-Ts2) / Td2 + Ts2 (9)
It is. The calculated Rm2 is stored in the register Rm2, and the calculated Tm2 is stored in the register Tm2. here,
(Re2-Rs2) = 32
Rd2 = 32
(Te2-Ts2) = π / 16
Td2 = 32
Equation (8) is specifically expressed as
Rm2 = rm2 + 8rm1-16 (8-1)
Equation (9) is, specifically,
Tm2 = tm2 · (π / 512) + (π / 64) · (tm1-2) (9-1)
Means When the address (Rm2, Tm2) is substituted into the conversion formula to the XY coordinate system, the address (Rm2, Tm2) is in the right half area of window 2 (or the lane detection window) on the screen displaying the image data of the corrected image memory. , A straight line (a straight line detected by the medium-density Hough transform: hereinafter referred to as a second detection straight line) is obtained.
[0087]
Next, the CPU 11 clears the data of the data table HGn (r, t) allocated to a certain area of the image memory 15a. Next, a conversion parameter of “Hough transform (HG3)” which is the third Hough transform is set. Here, the range of θ and ρ is set to a smaller range centered on (Rm2, Tm2) than in the case of the second “Hough transform (HG2)”, and the units of θ and ρ are also smaller. Set. In particular,
Rs = rm2 + 8rm1-18
Re = rm2 + 8rm1-14
Rd = 32
Ts = (π / 512) tm2 + (π / 64) tm1-9π / 256
Te = (π / 512) tm2 + (π / 64) tm1-7π / 256
Td = 32
Set. Note that rm2 and tm2 are defined by the above equations (8-1) and (9-1) with respect to Rm2 and Tm2. Then, the third “Hough transform (HG3)” is performed.
[0088]
"Hough transform (HG3)"
The content of the third “Hough transform (HG3)” is the same as the content of the second “Hough transform (HG2)”, but the content of the arithmetic expression (the above formula (5)) is different. In the third “Hough transform (HG3)”, the equation (5) is specifically the following equation (5-3).
[0089]

This is transformed into the form of equation (5b),

In the first and second conversions described above,
a ₁ = (Π / 64), b ₁ = 0, c ₁ = 0, d ₁ = 1/8
a ₂ = (Π / 512), b ₂ = Π (tm1-2) / 64, c ₂ = 8 (rm1-2), d ₂ = 1
Against
a ₃ = (Π / 4096), b ₃ = (Π / 512) tm2 + (π / 64) tm1-9π / 256
c ₃ = Rm2 + 8rm1-18, d ₃ = 8
It becomes.
[0090]
Since t = 0 to 31,
θ = (π / 512) tm2 + (π / 64) tm1-9π / 256
θ = (π / 4096) + (π / 512) tm2 + (π / 64) tm1-9π / 256
θ = 2 (π / 4096) + (π / 512) tm2 + (π / 64) tm1-9π / 256
θ = 3 (π / 4096) + (π / 512) tm2 + (π / 64) tm1-9π / 256
・
・
・
θ = 31 (π / 4096) + (π / 512) tm2 + (π / 64) tm1-9π / 256
Then, the ρ value r corresponding to 32 points θ (t = 0 to 31) is calculated using (π / 4096) as the minimum unit. That is, in the third Hough transform, each feature point in the right half area of the window 2 (or the lane detection window) has θ of (π / 4096) as the minimum unit (one unit of t is π / 4096) and θ = (π / 512) tm2 + (π / 64) tm1-9π / 256 or more θ = 31 (π / 4096) + (π / 512) tm2 + (π / 64) tm1-9π / 256 or less Is converted to a position (r, t) on the polar coordinate plane in the range of The range of ρ is Rs = rm2 + 8rm1-18 or more and Re = rm2 + 8rm1-14 or less, and the unit of ρ divides this range into 32. That is, one unit of r is 4/32 = (1/8). Therefore, the third Hough transform converts the feature points in the right half of window 2 (or the lane detection window) into polar coordinates having a smaller area and a higher density than the second Hough transform. . In one polar coordinate transformation of a feature point, since t = 0 to 31 in this third Hough transformation, 32 calculations are performed, so that the processing time is almost the same as that of the second Hough transformation. The number of operations for converting one feature point (Xs, Ys) to ρ-θ polar coordinates is extremely small, the number of r data bits is small, the speed for converting each feature point to ρ-θ polar coordinates is fast, and the frequency High count processing speed.
[0091]
When the “Hough transform (HG3)” that is the third Hough transform ends, the CPU 11 executes the third “maximum point search”. The contents are the same as in the above-described second "maximum point search". When this process ends, the register Gm stores the maximum value of the frequency in the data table HGn, and the registers rm3 and tm3 store the addresses (rm3 and tm3) where the maximum value exists.
[0092]
When completing the third “maximum point search”, the CPU 11 replaces the address (rm3, tm3) expressed in the rt polar coordinate system with the ρ-θ polar coordinate system address ( Rm3, Tm3). The operation expression is
Rm3 = rm3 · (Re3-Rs3) / Rd3 + Rs3 (10)
Tm3 = Tm3 · (Te3-Ts3) / Td3 + Ts3 (11)
It is. The calculated Rm3 is stored in the register Rm3, and the calculated Tm3 is stored in the register Tm3. here,
(Re3-Rs3) = 4
Rd3 = 32
(Te3-Ts3) = π / 128
Td3 = 32
Equation (10) is, specifically,
Rm3 = (1/8) · rm3 + rm2 + 8rm1-18 (10-1)
Equation (7) is, specifically,
Tm3 = (π / 128) · tm3 + (π / 512) tm2 + (π / 64) tm1-9π / 256 (10-1)
Means When the address (Rm3, Tm3) is substituted into the conversion formula to the XY coordinate system, the address (Rm3, Tm3) is in the right half area of the window 2 (or the lane detection window) on the screen displaying the image data of the corrected image memory. , A straight line (a straight line detected by the high-density Hough transform: hereinafter referred to as a third detection straight line) is obtained. This is the end of “straight line fitting”.
[0093]
Thus, a straight line (a straight line of the third detection) representing a straight line (the most representative straight line) in the image in the right half area of the window 2 (or the lane detection window) is obtained. The window 2 (or the lane detection window) is set to an area suitable for detecting the left and right white lines of the vehicle lane, and the right end of the right half area of the window 2 (or the lane detection window). , There is a high probability that the straight line detected for the third time is a straight line approximating this white line. Therefore, the above-mentioned "straight line fitting" is the detection of the rightmost white line of the own vehicle lane in the right half area of the window 2 (or the lane detection window).
[0094]
Next, the CPU 11 stores Rm3 of the data (Rm3, Tm3) representing the straight line of the third detection in the register RmR, and stores Tm3 in the register TmR. That is, data representing a straight line representing the rightmost white line of the own vehicle lane is stored in the registers RmR and TmR.
[0095]
Left white line detection
Next, the CPU 11 performs the same processing as the above-described processing on the right half area of the window 2 (or the lane detection window) on the left half area of the window 2 (or the lane detection window). A straight line representing the leftmost white line of the vehicle lane is detected, and data representing the straight line is stored in registers RmL and TmL. Thus, the “left and right white line detection” D3 is completed. Next, the CPU 11 performs an "infinity point calculation" D4.
[0096]
D4. "Calculation of infinity point" D4
Here, first, in the “straight line intersection calculation”, a straight line represented by the data of the registers RmR and TmR (estimated as the white line at the right end of the vehicle running lane) and a straight line represented by the data of the registers RmL and TmL (the white line at the left end of the vehicle running lane (Xc, Yc) is calculated. Next, it is determined whether or not the calculated intersection (Xc, Yc) exists in an area of 60 horizontal pixels × 60 vertical pixels centered at the infinity point (Xv, Yv) represented by the data of the registers Xv, Yv. To check. Within this region, the intersection (Xc, Yc) obtained this time has a high reliability of being a point at infinity, so the infinity point data (Xc, Yc) is added to the intersection (Xc, Yc) obtained this time by one. / 8 is weighted and updated to a value obtained by adding 7/8 to the infinity point data (Xc, Yc) so far. Then, the register Nve for counting the number of times the infinite point tracking has failed is cleared.
[0097]
When the intersection (Xc, Yc) calculated this time is not within the area of 60 pixels horizontally × 60 pixels vertically centered at the point of infinity (Xv, Yv) represented by the data of the registers Xv and Yv, Assuming that the point tracking has failed (this intersection calculation is an error or the infinite point data (Xv, Yv) so far is an error), the content of the register Nve is incremented by 1 and the content of the register Nve becomes 5 Check If it is 5, this means that the intersection calculation has been in error for a total of five consecutive times, this time and the past four times, and this is because the infinity point data (Xc, Yc) currently held is an error. Assuming that there is, the infinity point data (Xc, Yc) is updated to the intersection (Xc, Yc) calculated this time.
[0098]
A4. “Filtering of ρ, θ & infinity point” A4
Here, the CPU 11 filters the parameters ρ and θ representing the straight line detected in “Left and Right White Line Detection” (D3) and the coordinates (Xv, Yv) of the straight line intersection calculated in “Calculate Infinity Point” (D4). . That is,
ρL = (3 · ρL + RmL) / 4 (11 L)
ρR = (3 · ρR + RmR) / 4 (11R)
θL = (3 · θL + TmL) / 4 (12 L)
θR = (3 · θR + TmR) / 4 (12R)
Is calculated and stored in the registers ρL, ρR, θL, θR. ΡL, ρR, θL, and θR in parentheses in the above four equations are the previous values of the registers ρL, ρR, θL, and θR, and RmL and the like are data of the register RmL and the like. Next, the intersection (Xv, Yv) of the straight line represented by the calculated ρL, ρR and the straight line represented by θL, θR is calculated, and Xv is written into the register Xv, and Yv is written into the register Yv.
[0099]
D5. "Left and right white line interval (WL) calculation" D5
Next, the CPU 11 calculates an interval (lane width) WL between the right white line and the left white line of the own vehicle lane. In this case, the X position of the rightmost white line (ρR, θR) on the screen at the position (center point of the screen on the screen) L1 where the center line of the visual field of the camera 16b (the two-dot chain line in FIG. 17) intersects the road surface is shown. The distance WL = XR−XL is calculated as follows by converting the X position of the leftmost white line (ρL, θL) on the screen into the road position XL.
XR = (Sy × Hc × sin θR) / (Sx × cos θR) (13R)
XL = (Sy × Hc × sinρR) / (Sx × cosρR) (13L)
WL = XR-XL (14)
Note that Sx and Sy in FIG. 17 are the horizontal and vertical scale factors of the camera 16b, respectively, and Hc is the height of the center of the lens of the camera 16b from the road surface.
[0100]
Next, the CPU 11 checks whether the calculated lane interval WL on the road surface is correct (self-vehicle lane detection succeeded) (D6). That is, it is checked whether the deviation of WL from the reference value WL3D is within the allowable range DwL. In this embodiment, since the lane width of the Japanese expressway is 3.5 ± 0.2 m, the reference value WL3D = 3.5 m and the allowable value DwL = 0.3 m.
[0101]
If the deviation of the WL from the reference value WL3D is within the allowable range DwL, it is determined that the own vehicle lane has been successfully detected, and information “1” indicating this is written to the register FL (61), and the deviation of the WL from the reference value WL3D is determined. Is out of the allowable range DwL, it is determined that the own vehicle lane detection has failed, and information "0" indicating this is written to the register FL (62). As described above, the “vehicle lane detection” is completed, and if the own vehicle lane detection is successful, the data in the register FL is “1”.
[0102]
When the "vehicle lane detection" D is completed, the CPU 11 checks the data in the register FL, and if it is "1", executes the "curve estimation" G1. That is,
R = (L1 / 2) · sinA (15)
Is used to calculate the curve radius R. L1 is the distance (L1 = 15 m) from the camera 16b to the point where the visual field center line intersects the horizontal road surface, and A is the steering angle of the camera 16b (calculated by A5 in FIG. 11). Next, it is checked whether or not the calculated curve radius is 2000 m or less (curved) (G2), and if so, "feature point selection" G3 is executed.
[0103]
G3. "Feature point selection" G3
The CPU 11 sets the center of a pixel on the straight line represented by the data ρR and θR of the registers ρR and θR in the area of the window 2 and to the point at infinity (Yv) in the y-axis direction. A feature point (within ± 80 cm on the road surface) and within ± 5 pixels in the x direction is extracted as a feature point representing a right white line (here, a right curve because the road is curved). Similarly, within the area of the window 2 and up to the point at infinity (Yv) in the y-axis direction, the center of a pixel on the straight line represented by the data ρL, θL of the registers ρL, θL, ± the set half-width value in the y direction A feature point (within ± 80 cm on the road surface) and within ± 5 pixels in the x direction is extracted as a feature point representing a left white line (left curve because the road is curved here).
[0104]
G4A. "Coordinate transformation" G4A
Next, the CPU 11 converts the coordinates of the feature points extracted in the “feature point selection” G3 into a three-dimensional coordinate system (X, Y, Z) of the road surface. X is a horizontal left-right direction, Y is a vertical up-down direction, and Z is a horizontal direction (vehicle front direction) orthogonal to X and Y. The side and plane of the optical model of the camera 16b are shown in FIGS. The road surface is regarded as a horizontal plane (parallel to the Z axis). In FIG. 18 (a), it is assumed that a point P on the road surface means one of the above feature points yp:
tan (α−θ) = hc / Zp (16)
tan θ = B / f (17)
From these relationships, the Z-axis coordinate value Zp of the point P is
Zp = (f + B · tan α) · hc / (f · tan α−B) (18)
It becomes. The downward angle α of the camera 16b is
tanα = (yo-VP) / f (19)
It is. The tan α in the equation (19) is calculated, and this is substituted into the equation (18) to calculate Zp. Here, yo is the center position of the visual field in the camera coordinate system (screen coordinate system), yp is the position of the feature point, B is the position difference of the characteristic point with respect to the center position of the visual field on the y coordinate of the camera system, and VP is the camera. This is the position where the horizontal line of the road coordinate system is projected on the y coordinate of the system.
[0105]
Each time the use of the tan function in the distance calculation increases the processing time, so an approximate calculation is considered. Since the relationship between the function tan (β) and the angle β can be approximated as tanβ = β when β is close to 0, the expressions (16) and (17) are
α−θ = hc / Zp (16a)
θ = B / f = (yo−yp) / f (17a)
α = (yo-VP) / f (19a)
And equation (18) is
Zp = f · hc / (yp−VP) (18a)
Can be approximated. The CPU 11 converts the camera-system y-coordinate value of the feature point into a road-system Z-axis coordinate value Zp by the calculation of the expression (18a).
[0106]
Referring to FIG. 18B, the relationship between a point P on the road surface (corresponding position of a feature point on the screen) and the imaging plane is as follows.
tan θ = Xp / Zp (20)
tan θ = xp / f (21)
It becomes. xp is the x coordinate value of the feature point in the camera coordinate system. From these equations (20) and (21), the X coordinate value Xp of the point P in the road coordinate system is
Xp = Zp · xp / f (22)
Can be calculated. The CPU 11 calculates Xp by the calculation represented by the equation (22). In the road coordinate system (X, Y, Z), all the Y coordinate values of the feature points are set to 0 in order to simplify the calculation process.
[0107]
G5A. "Feature point rotation correction" G5A
The registers A, B, and C store the steering angles A, B, and C of the cameras 16b, 26b, and 36b (FIG. 5) calculated in "Calculation of Camera Steering Angle" A5 described later. In "rotation correction of feature point" G5, the CPU 11 converts the feature point position data (Zp, Xp) converted into the road system coordinates as described above with respect to the vertical axis Y ((a) in FIG. 18). Convert to a value rotated by just This means that the direction of the feature point viewed from the center line of the field of view of the camera is converted into the direction of the feature point viewed from the longitudinal axis of the vehicle. That is, the feature point position data (Zp, Xp) before the rotation correction is based on the road coordinate system in which the Z axis matches the horizontal projection line of the center line of view of the camera. The feature point position data (Zp, Xp) is converted to a road coordinate system in which the Z axis coincides with the horizontal projection line of the vehicle longitudinal axis.
[0108]
The CPU 21 and the CPU 31 perform the same processing as the above-described processing from the “corrected image memory initialization” C6 in FIG. 9 to the “correction of feature point rotation” G5 in FIG. The processing is executed for image data obtained by photographing with the cameras 26b and 36b.
[0109]
T3A. "Acquisition of correction data" T3A
Next, the CPU 11 allocates the characteristic point position data (two groups corresponding to the right white line and one corresponding to the left white line) subjected to the rotation correction as described above to a specific area of the memory 15a. The data is written in the shooting data area (the area for the right white line and the area for the left white line) of the camera 16b of the data area table.
[0110]
Then, via the serial data communication line between the CPUs, the CPU 21 instructs the CPU 21 to transfer the feature point position data subjected to the rotation correction as described above, and sends the data to the DMA device 14 from the DMA transfer device 24. Instructs the camera 26b to write the received data of the lane area data table into the photographing data area. In response to this instruction from the CPU 11, the CPU 21 instructs the DMA device 24 to specify the memory area storing the rotation-corrected feature point position data and to perform data transfer. In response to this instruction, the DMA device 24 reads the data in the memory area and sends it to the DMA data line, and the DMA device 14 writes it to the memory 15a. When the DMA 24 notifies the end of transmission, the CPU 21 notifies the CPU 11 of the end of transfer.
[0111]
Upon receiving this notification, the CPU 11 instructs the CPU 31 via the inter-CPU serial data communication line to transfer the feature point position data subjected to the rotation correction as described above, and to the DMA device 14, Instructs the camera 36b to write the data sent from the DMA transfer device 24 to the shooting data area of the lane area data table. In response to this instruction from the CPU 11, the CPU 31 instructs the DMA device 34 to specify the memory area storing the rotation-corrected feature point position data and to perform data transfer. In response to this instruction, the DMA device 34 reads the data in the memory area and sends it to the DMA data line, and the DMA device 14 writes it to the memory 15a. When the DMA 34 notifies the end of the transmission, the CPU 21 notifies the CPU 11 of the end of the transfer.
[0112]
If the CPU 21 or the CPU 31 determines in step G2 that the estimated curve radius R exceeds 2000 m, the CPU 21 or the CPU 31 does not execute the steps G3, G4A, and G5A. Notify that there is no data. In this case, the CPU 11 proceeds to “coordinate conversion of ρ, θ” G4B described later.
G6A. "Curve detection (least square)" G6A
Next, the CPU 11 compares the characteristic point position data (a set of characteristic points obtained by the cameras 16b, 26b and 36b) of the right white line area group in the lane area data table with a minimum value. The square method (approximate curve of the right white line) represented by the distribution of these characteristic points is calculated by applying the square method. That is, the X and Z coordinate values of the arc end closer to the vehicle in the Z direction, the X and Z coordinate values of the arc end farther from the vehicle, the X and Z coordinate values of the arc center, and the radius R _R Is calculated. At this stage, all of these coordinate values are of a road coordinate system in which the horizontal projection line of the front-rear axis of the vehicle is the Z-axis. At this time, the number of samples (existence number) of feature points is counted. In the same manner, with respect to the feature point position data (a set of feature points obtained by the cameras 16b, 26b, and 36b) of the left white line area group in the lane area data table, a minimum An arc (approximate curve of the left white line) represented by the distribution of these feature points is calculated by applying the multiplication method, and the number of samples of the feature points is counted.
[0113]
G7. "Curve detection success / failure determination" G7
Next, the CPU 11 determines the number of feature point samples (two sets) and the radius R (two sets R _R , R _L ), If at least one of the feature point sample numbers is less than a predetermined value, or if one of the curve radii R exceeds 2000 m, it is determined that curve detection is not established and the lane is determined to be a straight line. (judge.
[0114]
G9. "Filtering of curve R" G9
Each of the number of feature point samples is equal to or greater than a predetermined value, and the curve radius R _R , R _L Is 2,000 m or less, it is considered that curve detection is established, and the radius RR detected this time is _R , R _L Are subjected to a filtering process and updated in the radius registers RR and RL. That is, the coefficient Kw is set to 16 when the radius R calculated this time is 150 m or less, the coefficient Kw is set to 8 when the radius is more than 150 m and 300 m or less, and the coefficient Kw is set to 4 when the radius R exceeds 300 m.
RR = [(Kw-1) .RR + R _R ] / Kw (23R)
RL = [(Kw-1) · RL + R _L ] / Kw (23L)
Is calculated, and the calculated RR and RL are updated and written in the registers RR and RL. RR and RL of (Kw-1) .RR and (Kw-1) .RL are the data values of registers RR and RL before updating and writing, respectively. Since Kw is determined as described above, the calculated curve radius R _R , R _L Is large, the radius R of the current calculation with respect to the filtering values RR and RL _R , R _L Is large, and the responsiveness of the filtering value to the calculated value is high. Calculated curve radius R _R , R _L Is smaller, the radius R of the present calculation with respect to the filtering values RR and RL _R , R _L Is small, and the responsiveness of the filtering value to the calculated value is low. The steering angles A, B, and C of the cameras 16b, 26b, and 36b determined corresponding to the radius of the curve are substantially inversely proportional to the radius of the curve, and the importance of the control for the curve performed by the host MPU 18 is important. -If the error in the curve detection or calculation is large or the reliability is low, the camera may lose track of the running lane when the curve radius is small. If the curve corresponding control of the host MPU 18 becomes an error, the reliability of the control is reduced. In order to improve this, as described above, the filtering value is set to a large calculated value R. _R , R _L , A high response (Kw small) and a small calculated value R _R , R _L Was set to low response (large Kw).
[0115]
G4B. “Coordinate transformation of ρ and θ” G4B
If the radius of the curve estimated by the "Estimation of curve" G1 exceeds 2000 m, the CPU 21 or 31 notifies the absence of transfer data by the "accumulation of correction data" T3A. If it has come, or if it is determined that the curve detection is not established in the "determination of success / failure of curve detection" G7, the CPU 11 determines the left and right white lines of the traveling lane obtained in the previous "filtering of ρ, θ & infinity point" A4. The data (ρL, θL) and (ρR, θR) that define the straight line to be approximated are defined as a straight line in the road coordinate system (X, Y, Z) with the horizontal plane projection line of the center line of view of the camera as the Z axis. (Road-.rho.L, Road-.theta.L) and (Road-.rho.R, Road-.theta.R). The CPU 21 and the CPU 31 also perform the conversion processing in the case where the curve radius R estimated in the above-mentioned "Estimate curve" G1 exceeds 2000 m.
[0116]
G5B. "Rotation correction" G5B
The CPU 11 applies the rotation correction corresponding to the steering angle A of the camera 16b to the data (Road-ρL, Road-θL) and (Road-ρR, Road-θR) obtained by the above conversion, and calculates the longitudinal axis of the vehicle. Straight line definition data (ρL _R , ΘL _R ), (ΡR _R , ΘR _R ) Is calculated. That is, the straight line defining data (Road-ρL, Road-θL) and (Road-ρR, Road-θR) of the road coordinate system with the horizontal plane projection line of the camera's visual field center line as the Z-axis are represented by the longitudinal axis of the vehicle. Of the road coordinate system (ρL _R , ΘL _R ), (ΡR _R , ΘR _R ). The CPU 21 and the CPU 31 similarly perform the conversion process.
[0117]
T3B. "Acquisition of correction data" T3B
The CPU 11 sends straight-line definition data (ρL) converted by the CPU 21 and the CPU 31 from the CPU 21 and the CPU 31 via the inter-CPU serial data communication line. _R , ΘL _R ), (ΡR _R , ΘR _R Get)
[0118]
If the CPU 21 or the CPU 31 determines in step G2 that the estimated curve R is 2000 m or less, the CPU 21 or the CPU 31 does not execute the steps G4B and G5B. Notify none. In this case, the CPU 11 proceeds to the next “straight line detection” G6B.
[0119]
G6B. "Line detection" G6B
The CPU 11 calculates and saves the straight line definition data (ρL _R , ΘL _R ), (ΡR _R , ΘR _R ) And similar straight-line prescribed data transferred by the CPU 21 and the CPU 31 (however, there is a case where only the CPU 11 (only one person) is provided. A right representative straight line and a left representative straight line representing (approximately) representative of each of the two groups of straight lines (the right white line group and the left white line group) represented by the two groups may be defined. Data (ρR _R m, θR _R m) and (ρL _R m, θL _R m) is calculated. This is performed by weighted averaging. The CPU 11 is weighted by 8/16, the CPU 21 is weighted by 5/16, and the CPU 31 is weighted by 3/16.
[0120]
G10. "Estimation of left and right lane" G10
When the CPU 11 proceeds from the "curve R filtering" G9 to the "left and right lane estimation" G10, the data of the radius register RR and the register RL obtained in the "curve R filtering" G9. Is determined by comparing the RLs of the right and left turns. In the case of a right turn, RR-WL is the radius of the curve of the right white line of the right adjacent lane, and RL + WL is the left of the left adjacent lane. It is calculated as the radius of the curve of the white line, and when turning left, RR + WL is the radius of the curve of the right white line of the right adjacent lane, and RL-WL is the radius of the curve of the left white line of the left adjacent lane. calculate. WL is the data of the register WL, and is the own vehicle traveling lane width calculated by the CPU 11 in the "calculation of white line spacing (WL)" D5 (FIG. 10).
[0121]
When the process proceeds from the "straight line detection" G6B to the "estimation of right and left lanes" G10, the data ρR defining the right representative straight line calculated by the "straight line detection" G6B is used. _R m, θR _R m) to the right which translates to the right by the amount of WL (ρRR _R , ΘRR _R ) And the left representative straight line (data ρL _R m, θL _R m) that defines a straight line that is translated to the left by WL. _R , ΘLL _R ) Is calculated.
[0122]
G12. "Setting of vehicle search area" G12
If it is determined that the curve has been detected, an arc corresponding to the right white line of the right adjacent lane and an arc corresponding to the left white line of the left adjacent lane calculated in "Estimation of left and right lane" G10. 10m, L1 = 15m, 30m, L2 = 45m, 60m, L3 = 75m and 90m (all Z axis values) shown in FIG. ), The position data (Z, X) of each point is calculated.
[0123]
If it is determined that the curve detection is not established, a straight line corresponding to the right white line of the right adjacent lane and a straight line corresponding to the left white line of the left adjacent lane calculated in "Estimation of right and left lane" G10. The position data (Z, X) of each point of 10 m, L1 = 15 m, 30 m, L2 = 45 m, 60 m, L3 = 75 m and 90 m shown in FIG. 3 is calculated.
[0124]
Thus, the road area (the own vehicle traveling lane + the right adjacent lane + the left adjacent lane) is defined in the area (range of 10 to 30 m) to which the camera 16b (CPU 11) is assigned to capture the image. This means that a point (one point on each side at 10 m, one point on each side at 15 m, and one point on each side at 30 m) is determined.
[0125]
Similarly, a road area (own vehicle traveling lane + right adjacent lane + left adjacent lane) is defined in an area (range of 30 to 60 m) where photographing is allocated to the camera 26b (CPU 21). A point (one point on each side at 30 m, one point on each side at 45 m, and one point on each side at 60 m) is determined.
[0126]
Similarly, a road area (own vehicle running lane + right adjacent lane + left adjacent lane) own vehicle running is set in an area (range of 60 to 90 m) where photographing is allocated to the camera 36b (CPU 31). Six points defining the lane area (one point on each side at 60 m, one point on each side at 75 m, and one point on each side at 90 m) are defined.
[0127]
G12. "Reverse coordinate transformation of vehicle search area" G12
The CPU 11 corrects the above-mentioned six points in the area to which the photographing is assigned to the camera 16b (CPU 11) in accordance with the steering angle A of the camera 16b, that is, sets the horizontal projection line of the visual field center line of the camera 16b as the Z axis. And then inversely converted to coordinate values on the photographing screen of the camera 16b, and similarly, each of the six points in the area where photographing is assigned to the cameras 26b (CPU 21) and 36b (CPU 31). Is inversely transformed into coordinate values on the photographing screen of each camera.
[0128]
T4. "Accumulation of feature points in vehicle search area" T4
Here, the CPU 11 provides the CPU 21 with the inversely transformed coordinate values of the six points in the area to which the camera 26b (CPU 21) is assigned to capture via the inter-CPU serial data communication line. Extraction of feature points in the enclosed area, coordinate transformation (conversion of the center line of the camera's field of view to the position in the road coordinate system with the horizontal plane as the Z axis), and rotation correction (the horizontal plane of the vehicle longitudinal axis as the Z axis) To the position in the road coordinate system), and gives the CPU 31 the inversely transformed coordinate values of the above-mentioned six points in the area to which the camera 36b (CPU 31) is assigned to capture, so that the area surrounded by the six points is given. , Extraction of coordinate points, coordinate conversion and rotation correction are instructed. Based on the inversely transformed coordinate values of the six points in the area to which the camera 16b (CPU 11) is assigned to shoot, the CPU 11 determines the characteristics of the area (enclosed by a polygonal line) surrounded by the six points on the memory 15a. The points are extracted, and the extracted coordinate points are subjected to the same coordinate conversion processing as the above-described “coordinate conversion” G4A, and further, the same rotation correction processing as the above-described “characteristic point rotation correction” G5A is executed. I do. The feature point data obtained in this way is written into the camera 16b shooting data area of the lane area data table allocated to the specific area of the memory 15a. The CPU 21 and the CPU 31 perform similar processing.
[0129]
Next, the CPU 11 instructs the CPU 21 via the CPU serial data communication line to perform a DMA transfer of the feature point data thus processed to the memory 15a. When the DMA transfer is completed, the CPU 11 instructs the CPU 31 to perform a DMA transfer of the feature point data processed as described above to the memory 15a. The details of these DMA transfers are the same as those of the DMA transfer described in the above-mentioned "Correction Data Integration" T3A.
[0130]
H. "Advance vehicle recognition &ranging" H
The characteristics of the lane area data table allocated to the specific area of the memory 15a are all over the camera 16b image data area, the camera 26b image data area, and the camera 36b image data area. Find the distribution of points. In other words, in the road coordinate system in which the horizontal projection line of the vehicle front-rear axis is the Z axis, the vehicle is distributed in three lanes (road surface area) of the traveling lane, the right adjacent lane, and the left adjacent lane. The X-direction (left-right direction) distribution number (Z-axis projection histogram) of the feature points is searched. In this method, the road surface area is horizontally scanned (scanning in the X direction) from left to right in the Z direction from the lower side (close to the vehicle), the number of feature points on the scanning line is counted, and the count value is scanned. Line No. Save (Z position). This X scanning is performed from below (close to the vehicle) to above (far) in the Z direction of the road surface area.
[0131]
When this (creation of the Z-axis projection histogram) is completed for the road surface area, the continuous width in the Z direction where the horizontal count value (the number of distributions in the X direction) of the feature point is equal to or greater than the set value (the starting Z position and the ending Z position near the vehicle) Is searched from the lower side (closest to the vehicle) in the Z direction, and when it is found, the feature points of the road surface region from the Z start end to the Z end of the Z direction continuous width are vertically (Z Direction) to search the number of vertical distributions of feature points up to the right end (create an X-axis projection histogram). From the X-axis projection histogram, the vertical width of the feature point (the number of distributions in the Z direction) is equal to or greater than the set value, and the continuous width in the X direction (the starting X position and the ending X position) is shifted to the left in the X direction (left adjacent lane). From the left white line equivalent position), and if there is, the center point (intermediate point (Xt) between the start X position and the end X position) is calculated, and the vertical scanning width (when the horizontal count value is equal to or more than the set value) The intermediate point (Zt) of the continuous width in the Z direction is calculated, and (Xt, Zt) is written in the vehicle data table. From the left end to the right end, when the search for the continuous width in the x direction in which the above-described vertical count value is equal to or greater than the set value is completed, the next horizontal count value in the Z-axis projection histogram from the vehicle nearest to far away is equal to or greater than the set value. The second Z-direction continuous width is searched.
[0132]
In this manner, first, a region in which there is a possibility that another vehicle is present in the Z direction is sequentially searched from the near side of the own vehicle to the far side (in the Z direction) using the Z-axis projection histogram, and one corresponding region is searched. When the search is performed, the area is searched for from the left side (left adjacent lane) to the right side (in the X direction) of the own vehicle, and an area where there is a possibility that another vehicle is present is searched. Then, the center data (Xt, Zt) of the searched area is written in the vehicle data table. When this is completed for the entire road surface area, the vehicles existing on the road surface (the own vehicle traveling lane + the estimated left adjacent lane + the estimated right adjacent lane) photographed by the cameras 16b, 26b and 36b. To be precise, the vehicle may not be a vehicle, for example, the position (Xt, Zt) on the road coordinate system of a falling object from a freight vehicle is in the vehicle data table.
[0133]
Next, the CPU 11 determines whether each position (Xt, Zt) on the vehicle data table is on the left adjacent lane (estimated lane) or between it and the own vehicle traveling lane (both lanes). Data on the vehicle running lane, between it and the right adjacent lane, or on the right adjacent lane (estimated lane). Is added to each position information (Xt, Zt) of the vehicle data table. Hereinafter, the position information (Xt, Zt) + section data is referred to as other vehicle data.
[0134]
K. "Output" K
When curve detection is established, the CPU 11 transmits the curve information (curve radius (RR + RL) / 2) and other vehicle data to the host MPU 18 via the communication controller 17. Transfer to If the curve detection is not established, the straight line information (data representing a straight line) and the vehicle data table and other vehicle data are transferred.
[0135]
A5. "Calculation of camera steering angle" A5
The CPU 11 determines, from among the other vehicle data in the vehicle data table, the vehicle position in which the divided data is between the own vehicle traveling lane and the left or right neighbor lane. The shortest distance (Zt) within the range of 10 to 30 m is searched. If there is, the steering angle A for directing (the projection line on the road surface of) the visual field center line of the camera 16b to the searched vehicle position (Xt, Zt) is calculated. When there is no corresponding vehicle position, the center line of the visual field of the camera 16b is located at the center (the midpoint between the left and right white lines) of the width WL of the host vehicle traveling lane L1 ahead of the vehicle (Z position = L1). A steering angle A for directing a projection line on a road surface) is calculated. Then, the data An of the register An is written to the register A, and the steering angle A calculated this time is written to the register An.
[0136]
Next, the CPU 11 determines, from the other vehicle data in the vehicle data table, the vehicle position where the divided data is between the own vehicle traveling lane and the left or right neighbor lane. , The distance (Zt) is within the range of 30 to 60 m and the shortest is searched. When there is, the steering angle B for directing (the projected line on the road surface of) the visual field center line of the camera 26b to the searched vehicle position (Xt, Zt) is calculated. When there is no corresponding vehicle position, the center line of the visual field of the camera 26b is located at the center (the middle point between the left and right white lines) of the width WL of the own vehicle traveling lane L2 ahead of the vehicle (Z position = L2). A steering angle B for directing a projection line on a road surface) is calculated. Then, the data Bn of the register Bn is written to the register B, and the steering angle B calculated this time is written to the register Bn.
[0137]
Further, the CPU 11 determines, from among the other vehicle data in the vehicle data table, the vehicle position in which the divided data is between the own vehicle traveling lane and the left or right adjacent lane. , The distance (Zt) is in the range of 60 to 90 m and the shortest one is searched. When there is, the steering angle C for directing (the projection line on the road surface of) the visual field center line of the camera 36b to the searched vehicle position (Xt, Zt) is calculated. When there is no corresponding vehicle position, the center line of the visual field of the camera 36b is located at the center (the midpoint between the left and right white lines) of the width WL of the own vehicle traveling lane L3 ahead of the vehicle (Z position = L3). A steering angle C for directing a projection line on a road surface) is calculated. Then, the data Cn of the register Cn is written to the register C, and the steering angle C calculated this time is written to the register Cn.
[0138]
T5. "Transmission of camera steering angle" T5
The CPU 11 transmits the data Bn of the register Bn to the CPU 21 and transmits the data Cn of the register Cn to the CPU 31 via the inter-CPU serial data communication line.
[0139]
The CPU 11, CPU 21 and CPU 31 supply the data An, Bn and Cn to the steering controllers 16e, 26e and 36e, respectively, in the "camera steering output" A1 described above, and these controllers control the camera. The camera is rotationally driven such that the angles formed by (the horizontal projection lines of) the visual field center lines of 16b, 26b and 36b with (the horizontal projection lines of) the vehicle longitudinal axis are An, Bn and Cn. As a result, when there is a forward vehicle, the camera tracks the forward vehicle, and when there is no forward vehicle, the camera tracks the center of the width of the own vehicle traveling lane.
[0140]
When the “transmission of the camera steering angle” T5 ends, the CPU 11 proceeds to the “command for image input” T1 in FIG. The following description will be omitted because it is a repetition of the above description.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an outline of an embodiment of the present invention.
FIG. 2 is a plan view showing a shooting screen of a television camera 16b shown in FIG.
FIG. 3 is a plan view showing a shooting range (a front direction of the vehicle) of the television cameras 16b, 26b, and 36b shown in FIG.
4A is a graph showing a shooting range (left and right directions) of the television cameras 16b, 26b and 36b shown in FIG. 1, and FIG. 4B is a graph showing a quantization error.
FIG. 5 is a plan view showing shooting ranges and steering angles A, B, and C of the television cameras 16b, 26b, and 36b shown in FIG.
FIG. 6 is a block diagram showing a configuration of a first image processing apparatus 100 shown in FIG.
FIG. 7 is a block diagram showing a configuration of a second image processing apparatus 200 shown in FIG.
FIG. 8 is a block diagram showing a configuration of a third image processing device 300 shown in FIG.
9 is a flowchart showing a part of the contents of image data processing and preceding vehicle detection by the CPU 11 shown in FIG. 6;
FIG. 10 is a flowchart showing a part of the contents of image data processing and preceding vehicle detection by the CPU 11 shown in FIG. 6;
FIG. 11 is a flowchart showing the rest of the contents of image data processing and preceding vehicle detection by CPU 11 shown in FIG. 6;
FIG. 12A is a flowchart showing the contents of "one set of feature point detection windows" C1 shown in FIG. FIG. 2B is a plan view showing an area of a feature point detection window 1 on a shooting screen of the television camera 16b shown in FIG.
FIG. 13A is a flowchart showing the contents of "bonnet detection" C3 shown in FIG. FIG. 2B is a plan view illustrating a distribution of feature points on a shooting screen of the television camera 16b illustrated in FIG. 1 and a straight line detected by “straight line fitting”.
14A is a plan view showing a roll angle and a pan movement amount on a photographing screen of the television camera 16b shown in FIG. 1, and FIG. 14B is an "image rotation parallel movement" C7 shown in FIG. The calculation formula used is shown.
FIG. 15 is a plan view showing pixels referred to for calculating image data of a target pixel in “interpolation” C8 shown in FIG. 9;
FIG. 16 is a plan view showing a lane detection window on the shooting screen of the television camera 16b set by “lane detection window setting” A3 shown in FIG. 10;
17 is a schematic side view showing a geometric relationship between a lens and an image sensor in the television camera 16b shown in FIG. 1 and a preceding vehicle ahead of the vehicle.
18A is a schematic side view showing a geometric relationship between a lens and an image sensor in the television camera 16b shown in FIG. 1 and a preceding vehicle ahead of the vehicle, and FIG. 18B is a schematic plan view.
[Explanation of symbols]
11: CPU 12: ROM
13: RAM 15-16: I / O ports
15a: Image memory 16a: Camera controller
16b: TV camera 16c: A / D converter
16d: Rotating mechanism 16e: Steering controller
17: Communication controller 18: Host MPU

Claims (7)

A photographing device for photographing a first scene having a wide viewing angle including a short-distance road surface and a second scene having a narrow viewing angle including a long-distance road surface in front of the vehicle;
Feature point detecting means for detecting density edge points on the photographed screens of the first scene and the second scene;
The positions on the screen of the density edge points of the photographing screens of the first scene and the second scene are respectively converted into positions on the same road system coordinates with one horizontal projection line of the front-rear axis of the vehicle as one axis. Coordinate value converting means; and
Approximation line detection means for calculating a line approximating the entire series of the density edge points originating from the first scene and the density edge points originating from the second scene on the road system coordinates;
A road line detection device in front of a vehicle, comprising: A first scene having a wide viewing angle including a short distance road surface in front of the vehicle at a first angle in a horizontal direction with respect to the longitudinal axis of the vehicle, and a second angle in a horizontal direction with respect to the longitudinal axis of the vehicle A photographing device for photographing a second scene having a narrow viewing angle including a long distance road surface ahead of the vehicle;
Feature point detecting means for detecting density edge points on the photographed screens of the first scene and the second scene;
A first coordinate for converting a position on the screen of the density edge point of the photographing screen of the first scene into a position on a road system coordinate having the horizontal plane projection line of the center line of the visual field of the first scene as one axis. Value conversion means;
A second coordinate for converting the position on the screen of the density edge point of the photographing screen of the second scene into a position on a road system coordinate having the horizontal plane projection line of the center line of the visual field of the second scene as one axis. Value conversion means;
First rotation correction means for converting the position of the density edge point converted by the first coordinate value conversion means by a first angle into a position on road system coordinates having a horizontal plane projection line of the longitudinal axis of the vehicle as one axis;
The second rotation correction for converting the position of the density edge point converted by the second coordinate value converting means by a second angle into a position on road system coordinates having the horizontal projection line of the front-rear axis of the vehicle as one axis. means;
A line approximating the entire series of the density edge points originating from the first scene and the density edge points originating from the second scene on the road system coordinates with the horizontal projection line of the longitudinal axis of the vehicle as one axis. Approximate line detection means for calculating
Changing the first angle to track the density edge point on the first scene corresponding to the approximate line, and tracking the density edge point on the second scene corresponding to the approximate line. Photographing direction control means for changing the second angle;
A road line detection device in front of a vehicle, comprising: The photographing direction control means determines the first angle with the substantially middle point of the two approximate lines on the first scene being the center of the first scene, and the two approximate lines on the second scene. 3. The on-road line detecting device in front of a vehicle according to claim 2, wherein the second angle is set such that a substantially middle point of the second scene is the center of the second scene. The apparatus further comprises: forward object detecting means for detecting an object present between the two approximate lines on the first scene and the second scene, respectively.
When the forward object detecting means detects an object in the first scene area, the photographing direction control means determines the first angle with the object being substantially at the center of the first scene, and does not detect the first angle. In some cases, the first angle is set such that a substantially middle point between the two approximate lines on the first scene is the center of the first scene, and the front object detecting means detects an object in the second scene area. When the object is detected, the second angle is set so that the object is substantially at the center of the second scene. 3. The on-street line detection device in front of a vehicle according to claim 2, wherein the second angle is set as the center of the vehicle. The road line detection device according to claim 1, wherein the approximate line detection means calculates an entire series of the density edge points by a least square method using an arc. The photographing apparatus includes a first camera having a wide viewing angle for photographing a short-distance road surface in front of the vehicle and a first rotational drive for rotating the first camera to change a photographing direction of the first camera in a horizontal direction with respect to a longitudinal axis of the vehicle. Apparatus and second rotary drive for rotating the second camera having a narrow viewing angle for photographing a long distance road surface in front of the vehicle and for rotating the second camera in a horizontal direction with respect to the longitudinal axis of the vehicle 6. The object detection device in front of a vehicle according to claim 1, further comprising a device. The photographing apparatus includes a first camera having a wide viewing angle for photographing a short-distance road surface in front of the vehicle and a first rotational drive for rotating the first camera to change a photographing direction of the first camera in a horizontal direction with respect to a longitudinal axis of the vehicle. A second camera having a medium viewing angle for photographing a middle-distance road surface in front of the vehicle and a second rotation driving device for rotating the camera in order to change the photographing direction of the second camera in a horizontal direction with respect to the longitudinal axis of the vehicle; And a third camera having a narrow viewing angle for photographing a road surface at a long distance in front of the vehicle and a third rotation driving device for rotating the third camera in order to change the imaging direction of the third camera in a horizontal direction with respect to the longitudinal axis of the vehicle. The object detection device in front of the vehicle according to claim 1, wherein the object detection device includes:

Images