JP4752158B2 - Environment complexity calculation device, environment recognition degree estimation device and obstacle alarm device - Google Patents

Description

【００６６】
指標Ｃ１は輝度分割ブロックの数、指標Ｃ２は赤色系分割ブロックの数を示している。また、α、β、γは、重み付け係数であり、それぞれ所定の値をとる。
【００６７】
ここで、指標Ｃ１の演算について説明する。なお、マイクロコンピュータ７０は、上述したように領域Ｂ、領域Ｃ及び領域Ｄについて指標Ｃ１を演算するが、ここでは、領域Ｂについて図１１に示す撮像画像を用いて指標Ｃ１を演算することについて説明する。
【００６８】
マイクロコンピュータ７０は、障害物撮影用ＣＣＤカメラ２１を駆動させ、例えば図１１に示すような撮像画像を得る。そして、当該撮像画像全体を輝度分割ブロックとして設定する。
【００６９】
マイクロコンピュータ７０は、設定された輝度分割ブロック内において各画素の輝度値の最大値と最小値の差を演算する。マイクロコンピュータ７０は、輝度値の最大値と最小値の差が所定の閾値以上であるかを判定し、上記差が所定の閾値を超えているときは輝度分割ブロックを４つに分割する。このとき、マイクロコンピュータ７０は、縦横ほぼ同じ画素数で構成されるように、かつ可能な限り大きくするように、輝度分割ブロックを分割するのが好ましい。
【００７０】
撮像画像の輝度値が例えば図１２（Ａ）に示すようになっている場合、マイクロコンピュータ７０は、輝度値の最大値（１８０）と最小値（０００）の差を演算する。そして、輝度値の最大値と最小値の差（１８０）が閾値（例えば４０）以上であるかを判定し、ここでは上記差が閾値（４０）を超えているので、図１２（Ａ）に示すように輝度分割ブロックを４つに分割する。
【００７１】
マイクロコンピュータ７０は、分割された各輝度分割ブロック内において各画素の輝度値の最大値と最小値の差が所定の閾値以下になるまで、前記輝度分割ブロックを４つに分割し、新たな輝度分割ブロックを設定することを繰り返す。
【００７２】
例えば、図１２（Ａ）に示す輝度分割ブロックＢＫ１は４×４画素で構成され、輝度値の最大値は１８０、輝度値の最小値は２０である。マイクロコンピュータ７０は、輝度分割ブロックＢＫ１の輝度値の最大値と最小値の差（１６０）を求め、上記差が閾値（４０）以上であるので、図１２（Ｂ）に示すように、輝度分割ブロックＢＫ１を４つ（ＢＫ２，ＢＫ３，ＢＫ４，ＢＫ５）に分割する。
【００７３】
そして、マイクロコンピュータ７０は、輝度分割ブロックＢＫ２の輝度値の最大値と最小値の差（３９）は閾値（４０）以上でないので、輝度分割ブロックＢＫ２に対しては分割を行わない。一方、輝度分割ブロックＢＫ３，ＢＫ４，ＢＫ５については、輝度値の最大値と最小値の差は閾値（４０）以上であるので、図１２（Ｃ）に示すように、各輝度分割ブロックＢＫを分割する。
【００７４】
このような処理を経て、マイクロコンピュータ７０は、各画素の輝度値の最大値と最小値の差が所定の閾値（４０）以下になった輝度分割ブロックの数を示す指標Ｃ１を求める。これにより、マイクロコンピュータ７０は、輝度分割ブロックを構成する各画素の輝度値を均質にする。
【００７５】
なお、指標Ｃ１は、各画素の輝度値の最大値と最小値の差が所定の閾値以下になった輝度分割ブロックの数に限定されるものではない。例えば、指標Ｃ１は、撮像画像の輝度値の分散値であってもよい。また、指標Ｃ１は、撮像画像の２次元高速フーリエ変換（ＦＦＴ：Ｆａｓｔ Ｆｏｕｒｉｅｒ Ｔｒａｎｓｆｏｒｍ）の高周波成分の総和であってもよい。
【００７６】
また、マイクロコンピュータ７０は、図１２（Ｃ）に示すように、輝度分割ブロックＢＫが１×１画素になるまで分割したが、輝度分割ブロックＢＫを分割して予め定めた大きさ（例えば、４×４画素）になったときに、分割を終了してもよい。
【００７７】
つぎに、指標Ｃ２の演算について説明する。なお、マイクロコンピュータ７０は、上述したように所定領域毎に赤色系分割ブロックの数を示すＣ２を演算するが、ここでは、領域Ｂについて、図１３（Ａ）に示す撮像画像を用いて指標Ｃ２を演算することについて説明する。
【００７８】
マイクロコンピュータ７０は、障害物撮影用ＣＣＤカメラ２１を駆動させ、例えば図１３（Ａ）に示すような撮像画像を得る。そして、当該撮像画像から明度０．３以上、彩度０．１以上を抽出して、例えば図１３（Ｂ）に示すような撮像画像を得る。さらに、マイクロコンピュータ７０は、０．１以下又は０．７５以上の色相を抽出すると、図１３（Ｃ）に示すような撮像画像を得る。
【００７９】
マイクロコンピュータ７０は、図１３（Ｃ）に示すような撮像画像に対して、同一の赤色系分割ブロック内が赤色系画素のみで占めるようにブロック分割を繰り返し、図１３（Ｄ）に示すように、赤色系画素（１）又は赤色系画素以外の画素（０）で構成された赤色系分割ブロックを得る。そして、マイクロコンピュータ７０は、赤色系画素のみで構成された赤色系分割ブロックの数である指標Ｃ２を求める。
【００８０】
マイクロコンピュータ７０は、求められた輝度分割ブロックの数を示す指標Ｃ１及び赤色系分割ブロックの数を示す指標Ｃ２を用いて、上述した式（１）に従って環境の複雑度Ｃを演算して、ステップＳＴ２３に移行する。なお、以上のようにして演算された複雑度Ｃの値は、例えば図１４に示すように、時刻によって変化する。
【００８１】
なお、指標Ｃ２は、赤色系分割ブロックの数に限定されるものではなく、例えば、次のような値でもよい。
【００８２】
マイクロコンピュータ７０は、所定の領域の撮像画像に対して、マンセル色票において色票が２．５ＲＰ，５ＲＰ，７．５ＲＰ，１０ＲＰ，２．５Ｒ，５Ｒ，７．５Ｒ，１０Ｒ，２．５ＹＲ，５ＹＲに属し、クロマ２以上、明度３以上の赤色系の画素のみを抽出してもよい。そして、同一の分割ブロック内が上記赤色系画素のみ、赤色系画素以外の画素のみで構成されるまで、上記分割ブロックを分割することを繰り返し、得られた分割ブロックの数を指標Ｃ２とすればよい。
【００８３】
また、マイクロコンピュータ７０は、所定の領域の撮像画像に対して、マンセル色票において色票が２．５ＲＰ，５ＲＰ，７．５ＲＰ，１０ＲＰ，２．５Ｒ，５Ｒ，７．５Ｒ，１０Ｒ，２．５ＹＲ，５ＹＲに属し、クロマ２以上、明度３以上の赤色系の画素のみを抽出し、上記赤色系の画素の数を指標Ｃ２としてもよい。あるいは、撮像画像の中央から上述した各赤色系画素の距離の総和を指標Ｃ２としてもよい。
【００８４】
ステップＳＴ２３では、マイクロコンピュータ７０は、環境認識度合を判定するための閾値ｔｈを設定する。閾値ｔｈは、車両外部の環境照度によって異なるように設定される。その理由は、複雑度Ｃの値が一定であっても、車両外部が明るかったり暗くなったりすると、ドライバは交通環境が複雑と感じたり単純と感じることがあるからである。
【００８５】
マイクロコンピュータ７０には、例えば図１５に示すように、環境照度に対する閾値ｔｈを記述した閾値テーブルが記憶されている。図１５によると、閾値ｔｈは、環境照度がゼロからＳ１までの場合は最小値（一定）となり、環境照度がＳ１からＳ２まで大きくなるに従って一定の割合で大きくなる。そして、閾値ｔｈは、環境照度がＳ２からＳ３までの場合は最大値（一定）となり、環境照度がＳ３より大きくなるに従って一定の割合で小さくなる。なお、図１５に示す閾値テーブルは本実施の形態の一例であり、本発明はこれに限定されるものではない。
【００８６】
そこで、マイクロコンピュータ７０は、上記閾値テーブルを参照して、照度センサ４１によって検出された環境照度に基づいて、閾値ｔｈを設定する。さらに、マイクロコンピュータ７０は、ドライバの操作入力値やドライバの姿勢変化に応じて、閾値ｔｈの最大値を再設定することができる。
【００８７】
最初に、マイクロコンピュータ７０は、スライダパネル５０に設けられたスライダ５１の入力値に応じて、閾値ｔｈの最大値を設定する。例えば、ドライバがスライダ５１を「強」の方にスライドしている場合は、図１５に示すように、マイクロコンピュータ７０は閾値ｔｈの最大値を大きく設定する。逆に、ドライバがスライダ５１を「弱」の方にスライドしている場合は、マイクロコンピュータ７０は閾値ｔｈの最大値を小さく設定する。なお、スライダ５１が「強」と「弱」の中間にある場合は、マイクロコンピュータ７０は、環境照度に基づいて設定されたデフォルトの閾値ｔｈのままでよい。
【００８８】
次に、マイクロコンピュータ７０は、ドライバの姿勢変化に応じて、閾値ｔｈの最大値を設定する。その理由は、ドライバは、頻繁に姿勢の変化がある場合では種々の環境を認識し易い傾向があり、姿勢の変化がない場合では環境を認識しにくい傾向があるからである。そこで、マイクロコンピュータ７０は、交通環境の認識のしやすさとドライバの姿勢変化の関係を考慮して、以下のように閾値ｔｈを設定する。
【００８９】
マイクロコンピュータ７０は、例えば上述したステップＳＴ２１においてドライバの姿勢変化が「小」であると判定した場合、図１６に示すように、閾値ｔｈの最大値を小さく設定する。逆に、例えばステップＳＴ２１においてドライバの姿勢変化が「大」であると判定した場合は、閾値ｔｈの最大値を大きく設定する。なお、マイクロコンピュータ７０は、ドライバの姿勢変化が「中」であると判定した場合は、環境照度に基づいて設定されたデフォルトの閾値ｔｈのままでよい。
【００９０】
このように、マイクロコンピュータ７０は、環境照度に対応する閾値ｔｈが記述された閾値テーブルと、スライダパネル５０に設けられたスライダ５１の入力値と、更にドライバの姿勢変化と、に基づいて閾値ｔｈを設定すると、ステップＳＴ２４に移行する。
【００９１】
ステップＳＴ２４では、マイクロコンピュータ７０は、図１７に示すように、ステップＳＴ２２で演算された複雑度Ｃと、ステップＳＴ２３で設定された閾値ｔｈとを比較する。そして、複雑度Ｃが閾値ｔｈを超えていなかったときは、環境は複雑でない、すなわち単純であると判定する。また、複雑度Ｃが閾値ｔｈを超えていたときは、環境は複雑であると判定する。
【００９２】
マイクロコンピュータ７０は、障害物撮影用ＣＣＤカメラ２１で得られた撮像画像が例えば図１８に示すような前方風景の場合は、交通環境が「単純」であると判定し、例えば図１９に示すような前方風景の場合は、交通環境が「複雑」であると判定する。そして、サブルーチン処理を終了して、図３に示すステップＳＴ３に移行する。
【００９３】
（ステップＳＴ３）
ステップＳＴ３では、マイクロコンピュータ７０は、ドライバに対して障害物の警報を行う。ここでは、領域毎に警報要請フラグがあるかを判定し、優先順位の高いフラグを用いてドライバに所定の警報を行う。なお、ここにいう警報要請フラグとは、上記所定領域について障害物等の存在によりドライバに警報を行う必要があることを示すフラグをいう。そして、マイクロコンピュータ７０は、具体的には図２０に示すステップＳＴ３１からステップＳＴ４２までの処理を実行する。
【００９４】
ステップＳＴ３１では、マイクロコンピュータ７０は、領域Ａの警報要請フラグを生成するか否かを判定する。ここでは、具体的には図２１に示すステップＳＴ５１からステップＳＴ５４までのサブルーチンを実行する。
【００９５】
ステップＳＴ５１では、マイクロコンピュータ７０は、領域Ａにおいて障害物を検出したか、又は現在の自車位置が障害物出現範囲内にあるかを判定し、いずれか一方の条件を満たすときはステップＳＴ５２に移行し、いずれの条件も満たさないときはサブルーチン処理を終了する。
【００９６】
ステップＳＴ５２では、マイクロコンピュータ７０は、上述したステップＳＴ１３で検出された自車状態に基づいて、障害物に対して自車が回避行動をとっているかを判定し、回避行動をとっているときはステップＳＴ５３に移行し、回避行動をとっていないときはサブルーチン処理を終了する。
【００９７】
ステップＳＴ５３では、マイクロコンピュータ７０は、ステップＳＴ１において検出された障害物に関する情報及び自車状態に基づいて、自車が障害物を回避することが不可能であるかを判定する。そして、例えばドライバがブレーキペダルを踏んだり、ハンドルを操作して、障害物に対する回避行動をとっていたとしても、自車が障害物を回避することができないときはステップＳＴ５４に移行し、自車が障害物を回避することができるときはサブルーチン処理を終了する。
【００９８】
ステップＳＴ５４では、マイクロコンピュータ７０は、領域Ａにおける警報要請フラグを発生してサブルーチン処理を終了する。マイクロコンピュータ７０は、このようなサブルーチン処理を終了すると、図２０に示すステップＳＴ３２に移行する。
【００９９】
ステップＳＴ３２では、マイクロコンピュータ７０は、領域Ｂ，Ｃ及びＤのそれぞれに対して警報要請フラグを生成するか否かの判定を行う。具体的には図２２に示すステップＳＴ６１からステップＳＴ６４までのサブルーチン処理を実行する。なお、ここではいずれの領域でも同じ処理を行うので、領域Ｂの処理を例に挙げて説明する。
【０１００】
ステップＳＴ６１では、マイクロコンピュータ７０は、領域Ｂにおいて障害物を検出したか、又は現在の自車位置が障害物出現範囲内にあるかを判定し、いずれか一方の条件を満たすときはステップＳＴ６２に移行し、いずれの条件も満たさないときはサブルーチン処理を終了する。
【０１０１】
ステップＳＴ６２では、マイクロコンピュータ７０は、上述したステップＳＴ２において領域Ｂは複雑であると判定したか、すなわち、ステップＳＴ２において領域Ｂの複雑度Ｃが閾値ｔｈを超えたかを判定する。そして、領域Ｂの複雑度Ｃが閾値ｔｈを超えたときはステップＳＴ６３に移行し、閾値ｔｈを超えていないときはサブルーチン処理を終了する。
【０１０２】
ステップＳＴ６３では、マイクロコンピュータ７０は、障害物又は障害物出現範囲の場所が初めて警報を発する場所であるかを判定し、初めて警報を発する場所であるときはステップＳＴ６４に移行し、初めて警報を発する場所でないときはサブルーチン処理を終了する。
【０１０３】
ステップＳＴ６４では、マイクロコンピュータ７０は、領域Ｂにおける警報要請フラグを発生してサブルーチン処理を終了する。マイクロコンピュータ７０は、以上のようなサブルーチン処理を終了すると、図２０に示すステップＳＴ３３に移行する。
【０１０４】
ステップＳＴ３３では、マイクロコンピュータ７０は、領域Ａの警報要請フラグがあるか否かを判定し、領域Ａの警報要請フラグがあったときはステップＳＴ３９に移行し、領域Ａの警報要請フラグがなかったときはステップＳＴ３４に移行する。
【０１０５】
ステップＳＴ３４では、マイクロコンピュータ７０は、領域Ｃ又は領域Ｄの警報要請フラグがあるか否かを判定する。そして、各領域の少なくとも一方に警報要請フラグがあったときはステップＳＴ３８に移行し、各領域のいずれにも警報要請フラグがないときはステップＳＴ３５に移行する。
【０１０６】
ステップＳＴ３５では、マイクロコンピュータ７０は、領域Ｂの警報要請フラグがあるか否かを判定し、領域Ｂの警報要請フラグがあったときはステップＳＴ３６に移行し、領域Ｂの警報要請フラグがなかったときはサブルーチン処理を終了する。
【０１０７】
このように、マイクロコンピュータ７０は、ステップＳＴ３３からステップＳＴ３５の順に処理を実行することで、領域Ａ、領域Ｃ又は領域Ｄ、領域Ｂの順に優先順位を設定し、優先順位の高い領域から障害物の警報を行っている。
【０１０８】
ステップＳＴ３６では、マイクロコンピュータ７０は、前回の判定処理で領域Ａの警報要請フラグがあったかを判定し、領域Ａの警報要請フラグがあったときはステップＳＴ３７に移行し、領域Ａの警報要請フラグがなかったときはサブルーチン処理を終了する。
【０１０９】
ステップＳＴ３７では、マイクロコンピュータ７０は、ドライバに領域Ｂの警報を行う。マイクロコンピュータ７０は、ドライバに対する警報として、「前方障害物に注意してください。」、「交差点に近づきます。歩行者の飛び出しに注意してください。」などをＬＣＤ６１に表示させたり、音声によりスピーカ６２から出力させて、サブルーチン処理を終了する。
【０１１０】
また、ステップＳＴ３４で領域Ｃ、領域Ｄの少なくとも一方の警報要請フラグがあったと判定して移行したときのステップＳＴ３８では、マイクロコンピュータ７０は、警報要請フラグのあった領域について、ドライバに対して警報を行う。警報要請の一例としては、ステップＳＴ３７と同様である。
【０１１１】
一方、ステップＳＴ３３で領域Ａの警報要請フラグがあったと判定して移行したときのステップＳＴ３９では、領域Ａの警報要請フラグが所定数以上あるか否かを判定し、警報要請フラグが所定数以上あるときはステップＳＴ４０に移行し、警報要請フラグが所定数以上ないときはステップＳＴ４２に移行する。
【０１１２】
ステップＳＴ４０では、マイクロコンピュータ７０は、自車状態検出部３０により検出された車速が所定値以上であるかを判定し、車速が所定値以上であるときはステップＳＴ４１に移行し、車速が所定値以上でないときはステップＳＴ４２に移行する。
【０１１３】
ステップＳＴ４１では、マイクロコンピュータ７０は、ドライバに対して領域Ａにおける第１警報を行う。マイクロコンピュータ７０は、ドライバに対する第１警報として、例えば「衝突注意。減速してください。」などをＬＣＤ６１に表示させたり、音声によりスピーカ６２から出力させて、サブルーチン処理を終了する。
【０１１４】
ステップＳＴ４２では、マイクロコンピュータ７０は、ドライバに対して領域Ａにおける第２警報を行う。マイクロコンピュータ７０は、ドライバに対する第２警報として、例えば「障害物を回避してください。」などをＬＣＤ６１に表示させたり、音声によりスピーカ６２から出力させて、サブルーチン処理を終了する。
【０１１５】
マイクロコンピュータ７０は、ステップＳＴ３のサブルーチン処理を終了すると、ドライバに対する警報を終了する。このように、マイクロコンピュータ７０は、各領域の警報要請フラグに優先順位を設定して処理を行うことによって、ドライバに対する警報を最小限に抑制し、情報過多になってドライバが混乱することを防止することができる。
【０１１６】
以上のように、第１の実施の形態に係る障害物警報装置１は、移動障害物を自動的に認識し、移動障害物の出現可能範囲を推定し、更に、移動障害物の出現場所において、ドライバの交通環境の認識度合を考慮することによって、ドライバを混乱させることなく、移動障害物をドライバに認識させることができる。また、障害物警報装置１は、ドライバに提示するべき情報量が多いときは、自車の車速を低下させるように促して、事故を未然に防止することができる。
【０１１７】
障害物警報装置１は、図１０に示した領域Ａから領域Ｄのいずれにおいても障害物を検出した場合には、自車直前の領域Ａを最優先し、領域Ａに存在する障害物に対して警報を行う。これにより、障害物警報装置１は、直前に迫った危機を回避することができる。
【０１１８】
［他の実施の形態］
つぎに、本発明の他の実施の形態について説明する。なお、第１の実施の形態と重複する箇所については同一の符号を付し、詳細な説明は省略する。
【０１１９】
（撮像画像の分割の態様）
第１の実施の形態では、マイクロコンピュータ７０は、図１０に示すように、撮像領域を領域Ａから領域Ｄまでの４つに分割し、領域Ｂ、領域Ｃ及び領域Ｄにおいて指標Ｃ１及び指標Ｃ２を演算していた
これに対して、マイクロコンピュータ７０は、例えば図２３（Ａ）に示す撮像画像を、図２３（Ｂ）に示すように４つの領域に分割してもよい。つまり、マイクロコンピュータ７０は、長方形状の撮像画像を２つの対角線に沿って分割することによって、撮像画像を４つの領域に分割している。そして、第１の実施の形態と同様に、各領域について指標Ｃ１及び指標Ｃ２を演算すればよい。
【０１２０】
また、マイクロコンピュータ７０は、例えば図２４（Ａ）に示す撮像画像を、図２４（Ｂ）に示すように、領域（１）、２つの領域（２）、領域（３）の４つの領域に分割してもよい。ここで、領域（１）は、自車の前方直前の車両を示す領域である。領域（２）は、領域（１）の両外側の領域である。領域（３）は、領域（１）及び領域（２）以外の領域である。
【０１２１】
マイクロコンピュータ７０は、領域（１）については輝度分割ブロックを１×１画素まで分割し、領域（２）については２×２画素まで分割し、領域（３）については２×２画素まで分割して、各領域で指標Ｃ１を演算することができる。さらに、マイクロコンピュータ７０は、各領域において赤色系分割ブロックを１×１画素まで分割して、図２４（Ｃ）に示す画像を得て、各領域で指標Ｃ２を演算することができる。
【０１２２】
さらに、マイクロコンピュータ７０は、撮像画像を次のように分割することもできる。例えば図２５に示すように、撮像画像を「画面下部」、「画面中央」、「画面左方」、「画面右方」に分割してもよい。「画面下部」は、撮像画像の下側の縦方向約１／５から１／４までの長方形状の領域である。「画面中央」は、三角形状の領域であり、その三角形の一辺が「画面下部」に隣接し、その一辺の対角が撮像画面の上端に位置している。「画面左方」は、「画面下部」及び「画面中央」以外の領域のうち左側の台形状の領域である。「画面右方」は、「画面下部」及び「画面中央」以外の領域のうち右側の台形状の領域である。
【０１２３】
画面中央については、図２６に示すように、さらに分割してもよい。
【０１２４】
（閾値ｔｈの設定）
マイクロコンピュータ７０は、第１の実施の形態のように、ドライバ個人によって操作されたスライダ５１の入力値に従って閾値ｔｈを設定する場合に限らず、例えば複数のドライバによって操作されたスライダ５１の各入力値に従って設定するようにしてもよい。
【０１２５】
図２７は、１０名のドライバに１０枚の撮像画像を提示した場合に、ドライバによって入力された環境の認識度合をプロットした図である。マイクロコンピュータ７０は、例えばこれらの値の平均を演算することで妥当な閾値ｔｈを設定することができる。
【０１２６】
（複雑度Ｃの他の演算手法）
マイクロコンピュータ７０は、式（１）に従って複雑度Ｃを演算し、複雑度Ｃと閾値ｔｈとを比較することによって環境認識度合を推定していたが、本発明はこれに限定されるものではない。
【０１２７】
マイクロコンピュータ７０は、例えば図２８に示すように、指標Ｃ１及び指標Ｃ２によって決定される認識度合を示す認識度合マップを記憶してもよい。このとき、マイクロコンピュータ７０は、第１の実施の形態と同様にして指標Ｃ１及び指標Ｃ２を演算し、上記認識度合マップを参照して環境の認識度合を推定すればよい。例えば、マイクロコンピュータ７０は、（Ｃ１，Ｃ２）＝（ａ，ｂ）のときは環境の認識度合が高いと判定し、（Ｃ１，Ｃ２）＝（ｃ，ｄ）のときは環境の認識度合が低いと判定することができる。
【０１２８】
（３次元空間配置）
マイクロコンピュータ７０は、撮像画像を予め所定領域に分割し、所定領域毎に、第１の実施の形態と同様に指標Ｃ１及び指標Ｃ２を演算し、さらに、指標Ｃ１・Ｃ２を演算する。そして、指標Ｃ１、指標Ｃ２、指標Ｃ１・Ｃ２を、図２９に示すように、３次元ベクトル空間に配置し、各画素上での移動ベクトルの値（大きさ、向き）を複雑度Ｃとして用いることもできる。
【０１２９】
このとき、例えば図３０に示すように、３次元ベクトル空間内に、環境の認識度合が低いと推定される第１の領域と、環境の認識度合が高いと推定される第２の領域とを設けてもよい。マイクロコンピュータ７０は、上記３次元空間を用いて求められた複雑度Ｃが上記領域のいずれに属するかを判定し、複雑度Ｃが第１の領域にあるとドライバは環境認識の度合が低いと推定し、複雑度Ｃが第２の領域にあるとドライバは環境認識の度合が高いと推定することができる。
【０１３０】
（ニューラルネット）
マイクロコンピュータ７０は、ニューラルネットを用いて環境の認識度合を推定してもよい。図３１は、ニューラルネットを用いて環境の認識度合を推定するマイクロコンピュータ７０の機能的な構成を示すブロック図である。
【０１３１】
マイクロコンピュータ７０は、更新可能なデータベースを有する変換関数からなるニューラルネット７１と、ドライバの環境の認識度合を推定する認識度合推定部７２と、入力値から推定値を減算する処理を行う演算器７３と、を備えている。なお、ニューラルネット７１の代わりに、統計的手法によって学習機能を備えたものであってもよい
演算器７３は、図２に示したスライダ５１の入力値から、認識度合推定部７２で推定された環境の認識度合の推定値を減算する。ニューラルネット７１は、演算器７３により減算された減算値と、複雑度Ｃとに基づいて逐次学習し、学習済みの複雑度Ｃを認識度合推定部７２に供給する。認識度合推定部７２は、ニューラルネット７１からの学習済みの複雑度Ｃを用いて、ドライバ毎に環境の認識度合を推定すればよい。
【０１３２】
この結果、マイクロコンピュータ７０は、ドライバの実際の環境認識を考慮して逐次学習し、ドライバ個人の視覚特性に応じて、交通環境の認識度合を正確に推定することができる。
【０１３３】
【発明の効果】
本発明に係る環境複雑度演算装置は、前記撮像手段で生成された撮像画像に含まれる輝度の変化の分布、前記撮像画像に含まれる赤色系画素の分布の少なくとも一方に基づいて、環境の複雑度を演算することによって、環境の認識のしにくさ、すなわち環境の複雑さを示す複雑度を求めることができる。
【０１３４】
本発明に係る環境認識度合推定装置は、複雑度演算装置で演算された複雑度に基づいて環境の認識度合を推定することによって、観察者の環境の認識度合を客観的かつ正確に推定することができる。
【０１３５】
本発明に係る障害物警報装置は、障害物の位置を検出し、検出された障害物の位置を含む所定領域が環境の認識度合が低いと推定されたときに、前記障害物に関する警報を行うことによって、環境の認識度合の低い領域に障害物があるときに当該障害物の存在を観察者に警報することができる。一方、環境の認識度合の高い領域に障害物があるときに警報を行わないので、観察者が余計な警報によって煩わしさを受けることを防止することができる。
【図面の簡単な説明】
【図１】本発明の実施の形態に係る障害物警報装置の構成を示すブロック図である。
【図２】障害物警報装置に備えられたスライダパネルの構成を示す図である。
【図３】障害物警報装置に備えられたマイクロコンピュータの動作手順のメインルーチンを示すフローチャートである。
【図４】メインルーチンにおけるステップＳＴ１の具体的な動作手順を示すフローチャートである。
【図５】データキャリアとデータキャリアリーダの概要を説明するための図である。
【図６】マイクロコンピュータに記憶された出現範囲推定テーブルを示す図である。
【図７】ＬＣＤに表示される障害物表示画面を示す図である。
【図８】メインルーチンにおけるステップＳＴ２の具体的な動作手順を示すフローチャートである。
【図９】障害物表示画面を領域Ａから領域Ｄに分割した状態を説明する図である。
【図１０】障害物表示画面を模式的に領域Ａから領域Ｄに分割した状態を説明する図である。
【図１１】領域Ｂで指標Ｃ１を演算するときの撮像画像の一例を示す図である。
【図１２】撮像画像の輝度値の一例を示す図である。
【図１３】撮像画像の一例を示す図である。
【図１４】複雑度Ｃが時刻によって変化する状態を説明する図である。
【図１５】環境照度に対する閾値が記述された閾値テーブルを示す図である。
【図１６】ドライバの姿勢変化に応じて閾値ｔｈが変化する状態を説明する図である。
【図１７】複雑度Ｃと閾値ｔｈとを比較して環境の認識度合を推定するための図である。
【図１８】交通環境が単純である風景画像の一例を示す図である。
【図１９】交通環境が複雑である風景画像の一例を示す図である。
【図２０】メインルーチンにおけるステップＳＴ３の具体的な動作手順を示すフローチャートである。
【図２１】サブルーチンにおけるステップＳＴ３１の具体的な動作手順を示すフローチャートである。
【図２２】サブルーチンにおけるステップＳＴ３２の具体的な動作手順を示すフローチャートである。
【図２３】撮像画像の分割の他の手法を説明するための図である。
【図２４】撮像画像の分割の他の手法を説明するための図である。
【図２５】撮像画像の分割の他の手法を説明するための図である。
【図２６】画面中央の分割の一例を示す図である。
【図２７】１０名のドライバに１０枚の撮像画像を提示した場合に、ドライバによって入力された環境の認識度合をプロットした図である。
【図２８】指標Ｃ１及び指標Ｃ２によって決定される認識度合を示す認識度合マップを示す図である。
【図２９】指標Ｃ１、指標Ｃ２、指標Ｃ１・Ｃ２からなる３次元ベクトル空間を示す図である。
【図３０】３次元ベクトル空間内に、環境の認識度合が低いと推定される第１の領域と、環境の認識度合が高いと推定される第２の領域とを設けた状態を説明する図である。
【図３１】ニューラルネットを用いて環境の認識度合を推定するマイクロコンピュータの機能的な構成を示すブロック図である。
【符号の説明】
１ 障害物警報装置
１０ インフラ情報検出部
２０ 障害物情報検出部
４０ 環境情報検出部
６０ 障害物情報出力部
７０ マイクロコンピュータ[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an environment complexity calculation device, an environment recognition degree estimation device, and an obstacle alarm device, and more particularly, to an environment complexity calculation device and an environment recognition degree that support a driver's obstacle recognition according to the traffic environment around the vehicle. The present invention relates to an estimation device and an obstacle alarm device.
[0002]
[Prior art and problems to be solved by the invention]
When driving a vehicle, it is important to confirm that the vehicle is jumping out from the left and right when the vehicle is going straight ahead, and to check the crossing objects on the left and right in the direction of travel when the vehicle is turning left and right. It is. As the moving body that jumps out or crosses from the left and right, various things such as a pedestrian, a two-wheeled vehicle, and a four-wheeled vehicle can be considered.
[0003]
Japanese Patent Laid-Open No. 3-260813 proposes an environment recognition device that detects such a moving body. The environment recognition apparatus obtains contour information by extracting a difference in brightness information and a closed section region, and recognizes a moving object from the contour information. However, the lighting conditions surrounding the vehicle change from moment to moment, and because of the influence of ambient light, it is rare that the contour can be extracted completely and clearly. Therefore, since the environment recognition apparatus performs image recognition using only the contour information, the response to environmental changes is weak due to the small amount of information. In particular, when detecting an obstacle or the like mounted on a vehicle, it is difficult to say that a moving body can be detected optimally because the influence described above is greatly affected. Further, the process of directly extracting the closed section from the brightness information is difficult to perform because it is easily affected by shadows, disturbance light, and the like, and the brightness changed by the disturbance light directly affects the change of the center of gravity and area.
[0004]
In addition, detection methods in which internal information is added to contour information, such as template matching and texture matching, have been proposed. However, there are problems such as it takes time for processing and it takes time to prepare a large amount of information about the template in advance.
[0005]
Under such a background, for example, in recent years, a technique derived from a technique for compressing image information and capturing camera image information as a distribution of brightness data has been proposed.
[0006]
In Japanese Patent Laid-Open No. 11-142168, principal component analysis, which is one of multivariate analyses, is performed, and imaging information is mapped to an information space spanned by normalized principal component features, and data in the information space is mapped. There has been proposed a technique in which an image recognition technique is improved by analyzing a behavior. This technique performs more robust environment recognition using normalized principal component feature values using vehicle motion information and operation information. Japanese Patent Laid-Open No. 2000-19259 proposes a technique for recognizing an object using visible light image information and infrared image information other than visible light.
[0007]
However, these techniques have difficulty in recognizing a moving object when there are many factors that hinder recognition of an object such as a guardrail or a stopped vehicle. For example, there is a problem that it is impossible to recognize a pedestrian on a sidewalk or intersection other than on a road surface on which a vehicle passes, a pedestrian jumping out from a waiting place for a bicycle or between stopped vehicles.
[0008]
In contrast, for example, in JP-A-11-301343, a vehicle that predicts the appearance probability of a moving obstacle by environment (road shape) recognition using not only the obstacle itself but also a camera (visible light, infrared). Lighting devices have been proposed. Japanese Patent Laid-Open No. 11-232569 proposes a pedestrian warning system having a device for transmitting to the vehicle side whether or not a pedestrian has approached an intersection or the like. Furthermore, in Japanese Patent Laid-Open No. 6-144129, without actively recognizing a moving obstacle, a plurality of caution items prepared for the same cautionary state are selected at random and the driver is selected. There has been proposed an attention-assisting assisting device that alerts the user by informing.
[0009]
In this way, moving obstacle recognition support that supports obstacle recognition to the driver by performing automatic recognition of moving obstacles, considering the appearance probability of moving objects, or approaching pedestrians is proposed Has been.
[0010]
However, it is the driver itself that ultimately recognizes the moving obstacle. If the driver's visual perception and cognitive characteristics are not taken into account, too much information is presented to the driver, which may confuse the driver. In addition, the method of presenting attention items at random according to the location is inefficient although the amount of information presentation is small.
[0011]
The present invention has been proposed to solve the above-described problems, and is an environment complexity calculation device and environment that can efficiently and accurately support obstacle recognition without presenting excessive information to the driver. It is an object of the present invention to provide a recognition degree estimation device and an obstacle warning device.
[0012]
[Means for Solving the Problems]
  According to the first aspect of the present invention, an imaging unit that generates a captured image based on imaging light from an environment, and a captured image generated by the imaging unitSetting a luminance division block, and repeatedly dividing the luminance division block and setting a new luminance division block until it is determined that the luminance value of each pixel is uniform within the set luminance division block, Based on the number of luminance division blocks in which the luminance value of each pixel is uniform for the captured imageAnd complexity calculation means for calculating the complexity of the environment.
[0013]
  In the first aspect of the present invention, it is preferable that the imaging means is installed so as to image the traffic environment in front of the front seat of the vehicle, for example..
[0015]
  DuplicateThe coarseness calculation means sets a luminance division block for the captured image. The luminance division block set first shows an area for calculating the complexity. Then, in the luminance division block, the luminance division block is divided until it is determined that the luminance value of each pixel is uniform, and the divided luminance division block is set as a new luminance division block. repeat. Here, the brightness value of each pixel being homogeneous is not limited to the case where the brightness values of each pixel match, and it is sufficient that the difference between the maximum value of brightness values and the minimum value of brightness values is equal to or less than a predetermined value. That is, it is only necessary that the variation in the luminance value of each pixel is a predetermined value or less.
[0016]
When it is determined that the luminance values of the pixels in all the luminance division regions are uniform, the number of all these luminance division blocks is obtained as the environmental complexity. The number of such luminance division blocks corresponds to the spatial frequency. That is, if the total number is large, there are many high-frequency components, and the captured image is complicated. Further, when the total number is small, the high-frequency component is small and the captured image is simple. Thus, the complexity of the environment can be obtained by obtaining the number of luminance division blocks.
[0017]
  Claim2The described inventionImaging means for generating a captured image based on imaging light from the environment;A red-based divided block is set in the captured image generated by the imaging unit, and the red-based divided block is divided into new red colors until the set red-based divided block is configured by only predetermined red-based pixels. Repeat the setting of the system division block, and calculate the complexity of the environment based on the number of red system division blocks composed only of the predetermined red system pixels for the captured image.And a complexity calculation means.It is characterized by that.
[0018]
  Claim2In the described invention, the complexity calculation means sets a red-based divided block for the captured image. The red-type divided block set first indicates an area for calculating the complexity. The red-based divided block is divided until the red-based divided block is composed of only predetermined red-based pixels. Then, the divided red-based divided block is set as a new red-based divided block, and this process is repeated. If all the red divided blocks are composed of only predetermined red pixels, the number of all these red divided blocks is determined as the environmental complexity. Red has a characteristic that is more attractive than green, and the number of red-based divided blocks indicates the complexity of the image. Therefore, the complexity of the environment can be obtained by calculating the number of the red divided blocks.
[0019]
  Claim3The described inventionImaging means for generating a captured image based on imaging light from the environment;A luminance division block is set in the captured image generated by the imaging unit, and the luminance division block is divided into new ones until it is determined that the luminance value of each pixel is uniform within the set luminance division block. Repeating the setting of the luminance division block, calculating the number of luminance division blocks in which the luminance values of the respective pixels are uniform for the captured image, and adding the red division block to the captured image generated by the imaging means Until the set red-based divided block is composed of only predetermined red-based pixels, and repeatedly setting the new red-based divided block by dividing the red-based divided block into the captured image. On the other hand, the number of red-based divided blocks composed of only the predetermined red-based pixels is calculated, and the number of red-colored divided blocks and the number of red-based divided blocks are calculated based on the number of red-colored divided blocks. To calculate the complexity of theAnd a complexity calculation means.It is characterized by that.
[0020]
  Claim3In the described invention, the complexity calculation means is the claim.1The number of luminance division blocks is determined in the same manner as described in the invention, and further,2The number of red-type divided blocks is obtained in the same manner as described above, and the complexity of the environment is obtained based on the total number of these blocks. The complexity of the environment is calculated by multiplying the number of luminance division blocks, the number of red division blocks, the product of the number of luminance division blocks and the number of red division blocks by a predetermined weighting factor, Is preferred.
[0021]
  Claim4The invention described is from claim 13Any one ofenvironmentA complexity computing device; and environment recognition degree estimating means for estimating the environment recognition degree based on the complexity computed by the complexity computing device.
[0022]
  Claim4In the described invention,environmentSince the complexity calculated by the complexity calculation device indicates the complexity of the captured image of the environment, it is used as a parameter for estimating the degree of recognition of the environment of the observer. That is, the environment recognition degree estimation means can estimate the environment recognition degree based on the complexity.
[0023]
  Claim5The described invention is claimed.4In the described invention, the environment recognition degree estimation means estimates that the degree of recognition of the environment is low when the complexity calculated by the complexity calculation apparatus is equal to or greater than a threshold, and is calculated by the complexity calculation apparatus. When the complexity is smaller than the threshold, it is estimated that the degree of recognition of the environment is high.
[0024]
  Claim5In the described invention, a threshold value that serves as a reference value for the degree of environmental recognition is set. In other words, the threshold value is a complexity value indicating an intermediate between a complex environment and a simple environment. Therefore, by using this threshold, the environment recognition level estimation means estimates that the environment recognition level is low when the complexity is equal to or higher than the threshold, and that the environment recognition level is high when the complexity is lower than the threshold. Can be estimated.
[0025]
  Claim6The invention described in claim 5RecordIn the described invention, the apparatus further comprises illuminance detection means for detecting the illuminance of the environment, and the environment recognition degree estimation means comprises:Based on a threshold table describing the threshold corresponding to the illuminance of the environment,The threshold value corresponding to the illuminance detected by the illuminance detection means is set.YouIt is characterized by that.
[0026]
  Claim6In the described invention, even if the complexity is constant, when the outside of the vehicle becomes bright or dark, the observer may feel that the environment is complicated or simple. Therefore, in order to correct the change in the degree of recognition of the environment due to such a change in illuminance, a threshold table in which threshold values corresponding to the illuminance are described in advance is used..Thereby, when the illuminance is detected by the illuminance detection means, a threshold value corresponding to the detected illuminance can be set. And the recognition degree of an environment can be estimated using the set threshold value. As a result, it is possible to accurately estimate the degree of recognition of the environment regardless of the illuminance.
[0027]
  Claim7The described invention is claimed.6In the described invention, the apparatus further includes posture change detection means for detecting a change in the posture of the observer, and the environment recognition degree estimation means increases as the posture change of the observer detected by the posture change detection means increases.The above described in the threshold tableThresholdMaximum value ofIs set larger, and as the posture change of the observer detected by the posture change detection means becomes smallerSaidThresholdMaximum value ofSet to smallerYouIt is characterized by that.
[0028]
  Claim7In the described invention, the observer tends to recognize the environment when the posture change is large. The environmental recognition level estimation means can be used as the posture change of the observer increases.Described in the threshold tableThresholdMaximum value ofIs set to a large value, and the threshold becomes smaller as the observer's posture changesMaximum value ofBy setting small, it is possible to accurately estimate the degree of recognition of the environment so as to match the actual visual characteristics of the observer.
[0029]
  Claim8The invention described in claim 1 estimates the degree of recognition of the environment for each predetermined area.4From7The environment recognition degree estimation device according to any one of the above, obstacle position detection means for detecting the position of an obstacle, and a predetermined area including the position of the obstacle detected by the obstacle detection means, Alarm means for giving an alarm regarding the obstacle when the degree of environment recognition is estimated to be low by the degree estimating means.
[0030]
  Claim8In the described invention, the obstacle position detecting means detects the position of the obstacle. At this time, the environment recognition degree estimation device estimates the degree of recognition of the environment in the predetermined area including the position where the obstacle is detected. Here, when the recognition degree of the environment is high, the observer can sufficiently recognize the obstacle detected by the obstacle position detecting means. On the other hand, if the recognition degree of the environment is low, the observer may not be able to recognize the obstacle detected by the obstacle position detection means. Therefore, the alarm means issues an alarm regarding the obstacle when it is estimated that the predetermined area including the position of the obstacle has a low degree of environment recognition. As a result, the observer can avoid an accident by recognizing in advance that there is an obstacle even if there is an obstacle in a region where the degree of environmental recognition is low. The alarm means can alert the observer by outputting an image or sound.
  The alarm means does not perform an alarm regarding the obstacle when it is estimated that the predetermined area including the position of the obstacle has a high recognition degree of the environment. Thus, the observer can recognize the obstacle when there is an obstacle in a region where the degree of recognition of the environment is high, and is not bothered by an extra alarm.
[0031]
  Claim9The described invention is claimed.8In the described invention, the environment recognition degree estimation device divides an area excluding the lower part of the captured image into a plurality of estimation areas, estimates an environment recognition degree for each of the divided estimation areas, and the warning means When the position of the obstacle detected by the obstacle position detecting means is in any of the estimated areas, and the environment recognition degree of the estimated area is estimated to be low by the environment recognition degree estimating means, the priority A warning regarding an obstacle existing in a high estimated area is provided.
[0032]
  Claim8In the described invention, the environment recognition degree estimation device divides an area excluding the lower part of the captured image into a plurality of estimation areas, and estimates the environment recognition degree for each of the divided estimation areas. That is, the recognition degree of the environment is not estimated for the lower part of the captured image. The reason is that the lower part of the imaging screen shows an image of the environment immediately before the observer, and it is meaningless to estimate the degree of recognition of the environment.
[0033]
Whether the warning means has estimated that the position of the obstacle detected by the obstacle position detection means is in any of the estimated areas, and that the environment recognition degree of the estimation area is low by the environment recognition degree estimation means Determine. Here, all estimated regions where obstacles are detected and the degree of environmental recognition is low are selected. Then, an alarm indicating that there is an obstacle is given to an estimated area having a high priority among the selected designated areas. This makes it possible to avoid a major accident by letting the observer be aware of the most dangerous area, and not to inform the observer about the less dangerous area so that the observer is overconfused due to excessive information. Can be prevented.
[0034]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.
[0035]
[First Embodiment]
The first embodiment of the present invention can be applied to, for example, the obstacle alarm device 1 having the configuration shown in FIG. The obstacle alarm device 1 includes an infrastructure information detection unit 10 that detects infrastructure information from infrastructure outside the vehicle, an obstacle information detection unit 20 that detects information about roads and surrounding obstacles, An own vehicle state detection unit 30 for detecting an operation state, an environment information detection unit 40 for detecting an environment outside the driver and the vehicle, a slider panel 50 for inputting visual characteristics indicating the degree of environment recognition of the driver, and a fault An obstacle information output unit 60 that outputs object information and a microcomputer 70 that performs overall control based on information detected by each unit are provided.
[0036]
The infrastructure information detection unit 10 includes a GPS receiving circuit 11 that receives a GPS (Global Positioning System) signal, a DVD drive 12 that reads map information recorded on a DVD disc, an approach information receiving circuit 13 that receives approach information, It has.
[0037]
The GPS receiving circuit 11 receives a GPS signal having time and GPS satellite position information via the GPS antenna 11 a and supplies the GPS signal to the microcomputer 70 via the data bus 5. The DVD drive 12 reads the map information from the DVD disk based on the position information where the vehicle is currently traveling, and supplies it to the microcomputer 70 via the data bus 5. The approach information receiving circuit 13 receives approach information transmitted from a data carrier reader, which will be described later, and supplies it to the microcomputer 70 via the data bus 5.
[0038]
The obstacle information detection unit 20 includes an obstacle photographing CCD camera 21 for photographing obstacles on and around the road, and an infrared camera 22 for photographing obstacles on and around the road with infrared rays. A radar transceiver 23 for recognizing a front obstacle.
[0039]
The obstacle photographing CCD camera 21 and the infrared camera 22 are installed so that an object in the vehicle front direction can be imaged. The obstacle photographing CCD camera 21 and the infrared camera 22 supply the photographed photographed image to the microcomputer 70 via the data bus 5. In order to recognize a front obstacle, the radar transceiver 23 sharply squeezes a pulsed optical radar with respect to the obstacle and transmits it in a two-dimensional direction, and receives the optical radar reflected by the obstacle. The radar transmitter / receiver 23 is not limited to transmitting / receiving an optical radar, but may be a transmitter / receiver of radio radar.
[0040]
The own vehicle state detection unit 30 includes, for example, a wheel speed sensor, a steering angle sensor, a throttle valve sensor, a master cylinder oil pressure sensor, a yaw rate sensor, a longitudinal acceleration sensor, a lateral acceleration sensor, and the like. The own vehicle state detection unit 30 detects the vehicle speed, the steering angle, the accelerator operation amount, the brake operation amount, the yaw rate, the roll rate, the pitch rate, the longitudinal acceleration, the lateral acceleration, and the blinker operation amount. Supply.
[0041]
The environment information detection unit 40 includes an illuminance sensor 41 that detects illuminance outside the vehicle, and a driver photographing CCD camera 42 that photographs a driver. Here, only one driver photographing CCD camera 42 is provided. However, in order to easily detect a change in the posture of the driver, three CCD cameras that capture the front front, right front, and left front of the driver are provided. It may be provided.
[0042]
The slider panel 50 is used to input a visual characteristic indicating the complexity of the external environment according to the driver's subjectivity. As shown in FIG. 2, the slider panel 50 has a slider 51 that can move left and right. For example, when the driver tends to overlook the traffic environment, that is, when determining that it is difficult to recognize the traffic environment outside the vehicle, the driver operates the slider 51 in the “weak” direction. Further, the driver operates the slider 51 in the “strong” direction when the traffic environment can be seen well, that is, when it is easy to recognize the traffic environment outside the vehicle. Thereby, as will be described in detail later, the microcomputer 70 can estimate the degree of environment recognition in consideration of the current visual characteristics of the driver.
[0043]
The obstacle information output unit 60 includes an LCD (Liquid Crystal Display) 61 that outputs obstacle information using images, and a speaker 62 that outputs obstacle information using voice.
[0044]
The microcomputer 70 stores a CPU (Central Processing Unit) (not shown), a RAM (Random Access Memory) that is a data work area, a program that executes various routine processes to be described later, an appearance range estimation table, and a threshold table. ROM (Read Only Memory). The microcomputer 70 estimates obstacles that may appear based on information from each unit, estimates the degree of environment recognition of the driver, and performs obstacle information alarms.
[0045]
(Main routine)
In the obstacle alarm device 1 configured as described above, the microcomputer 70 executes the processing from step ST1 to step ST3 shown in FIG. Here, the process from step ST1 to step ST3 will be briefly described first, and then specific subroutines for each process will be described.
[0046]
The microcomputer 70 detects the position of the host vehicle based on the information detected by the infrastructure information detection unit 10 and estimates an obstacle that may appear in the traveling direction of the host vehicle (step ST1). Then, the captured image of the environment in the traveling direction of the host vehicle is divided, and the environment recognition degree indicating whether the driver can easily recognize each divided area is estimated (step ST2). Finally, the microcomputer 70 issues an alarm about the obstacle to the driver by an image or sound as necessary based on the obstacle information that may appear and the degree of environment recognition for each predetermined area (step). ST3).
[0047]
(Step ST1)
In step ST1, the microcomputer 70 executes processing from step ST11 to step ST14 shown in FIG. 4 specifically to estimate an obstacle that may appear in the traveling direction of the host vehicle.
[0048]
In step ST11, the microcomputer 70 detects the position of the host vehicle, and also detects a moving obstacle around the host vehicle position and outside the driver's field of view. Here, for example, the infrastructure described below is used.
[0049]
For example, as shown in FIG. 5, the pedestrian always carries an article (for example, a mobile phone) provided with the data carrier 16. In addition, a data carrier 16 is provided in a two-wheeled vehicle or a four-wheeled vehicle. The data carrier 16 can output several types of approach information, and the pedestrian, two-wheeled vehicle, and four-wheeled vehicle data carrier 16 output different approach information. On the other hand, a data carrier reader 17 for receiving approach information is installed at intersections and pedestrian crossings with poor visibility. When the data carrier 16 approaches, the data carrier reader 17 receives the approach information and transmits the approach information to the obstacle alarm device 1.
[0050]
When the access information transmitted from the data carrier reader 17 is received by the access information receiving circuit 13, the microcomputer 70 of the obstacle alarm device 1 can specify the location of the obstacle, and the obstacle is a pedestrian, Whether the vehicle is a two-wheeled vehicle or a four-wheeled vehicle can be recognized.
[0051]
Then, the microcomputer 70 is based on the GPS signal received by the GPS receiving circuit 11, the map information read from the DVD drive 12, and the approach information received by the approach information receiving circuit 13. In addition to detecting the position, a moving obstacle around the vehicle position and outside the driver's field of view is detected, and the type of the moving obstacle (person, two-wheeled vehicle, or four-wheeled vehicle) is recognized.
[0052]
Further, the microcomputer 70 refers to the appearance range estimation table shown in FIG. 6 and estimates a range where a moving obstacle may appear on the road around the own vehicle based on the type of the moving obstacle. For example, when the microcomputer 70 recognizes a four-wheeled vehicle as a moving obstacle, the microcomputer 70 estimates that the four-wheeled vehicle may appear within a range of 10 m from the current position in the traveling direction. When a pedestrian is recognized as a moving obstacle, it is estimated that the pedestrian may appear within a radius of 1 m from the current position.
[0053]
The microcomputer 70 sets a coordinate system (hereinafter referred to as “own vehicle peripheral coordinate system”) composed of a travel direction axis of the host vehicle and an axis orthogonal to the travel direction axis, and sets the range of possible occurrence of moving obstacles. After describing in the peripheral coordinate system, the process proceeds to step ST12.
[0054]
In step ST12, the microcomputer 70 detects an obstacle within the driver's field of view. Here, the microcomputer 70 drives the obstacle photographing CCD camera 21 and the infrared camera 22 to acquire a captured image generated by the obstacle photographing CCD camera 21 and the infrared camera 22. Then, by obtaining the difference between the images generated by the respective cameras, a moving obstacle is detected by obtaining an image in which a person or a car that is particularly problematic in terms of safety is sufficiently contrasted with the others.
[0055]
Further, the microcomputer 70 obtains a distance image of the front obstacle of the host vehicle based on the round-trip time of the optical radar from transmission to reception by the radar transceiver 23 to recognize the front obstacle. The microcomputer 70 describes the moving obstacle recognized in this way in the own vehicle peripheral coordinate system, and proceeds to step ST13.
[0056]
In step ST <b> 13, the microcomputer 70 detects the own vehicle state based on the information from the own vehicle state detection unit 30. Here, as described in, for example, Japanese Patent Application Laid-Open No. 11-301343, the microcomputer 70 includes a vehicle speed, a steering angle, an accelerator operation amount, a brake operation amount, a yaw rate, a roll rate, a pitch rate, a longitudinal acceleration, a lateral acceleration, The acceleration and turn signal operation amount are detected, the traveling direction of the host vehicle is estimated, the estimation result of the traveling direction of the host vehicle is described in the host vehicle peripheral coordinate system, and the process proceeds to step ST14.
[0057]
In step ST14, the microcomputer 70 only has information on moving obstacles that may appear within the predicted travel range of the host vehicle with respect to the host vehicle peripheral coordinate system obtained by the processes from step ST11 to step ST13. And remove other information. And if the microcomputer 70 displays the information regarding the moving obstruction described in the own vehicle periphery coordinate system on the LCD 61 as an obstruction display screen, it will escape from a subroutine and will transfer to step ST2 shown in FIG.
[0058]
Here, for example, as shown in FIG. 7, the obstacle display screen is a map of the surrounding area where the vehicle is traveling, the current position of the vehicle, the predicted travel range of the vehicle, and a moving obstacle (human or Car etc.) and its possible range of appearance. The size of the moving obstacle displayed on the obstacle display screen indicates a range where the moving obstacle may appear.
[0059]
(Step ST2)
In step ST2, the microcomputer 70 specifically executes the processes from step ST21 to step ST24 shown in FIG. 8 in order to estimate the degree to which the driver recognizes the surrounding environment of the vehicle.
[0060]
In step ST <b> 21, the microcomputer 70 detects a change in the driver's posture using information from the environment information detection unit 40. That is, the microcomputer 70 obtains a difference between frames of the captured image from the driver photographing CCD camera 42 and detects the difference value of the captured image as the moving area amount of the driver.
[0061]
The microcomputer 70 counts “1” when the moving area amount is equal to or larger than the predetermined threshold, and does not count when the moving area amount is smaller than the predetermined threshold. Then, for example, a count value for the past 5 minutes is obtained, and when the count value is less than the threshold value TH1, it is determined that the posture change is “small”, and the count value is greater than or equal to the threshold value TH1 and less than the threshold value TH2 (> TH1). Is determined to be “medium”, and when the count value is equal to or greater than the threshold value TH2, it is determined that the posture change is “large”. Moreover, the microcomputer 70 acquires the illuminance outside the vehicle detected by the illuminance sensor 41, and proceeds to step ST22.
[0062]
In step ST22, the microcomputer 70 calculates the complexity C of the environment for each predetermined area of the obstacle display screen shown in FIG. The complexity C here refers to a parameter used to estimate the degree of recognition of the driver's traffic environment.
[0063]
First, the microcomputer 70 divides the obstacle and the region where the appearance is possible in the obstacle display screen obtained in step ST1 into four in the distance and direction. Here, as shown in FIG. 9, the obstacle display screen is divided into an area A, an area B, an area C, and an area D. As a result, the obstacle display screen includes an area A that is a near-front area of the host vehicle, an area B that is adjacent to the area A and is a far-front area of the host vehicle, an area A, and A region C that is adjacent to B and is a region at the far left front of the host vehicle, and a region D that is adjacent to the regions A and B and is a region at the far right front of the host vehicle. . Obstacle display screen area A to area D are typically divided as shown in FIG.
[0064]
Next, the microcomputer 70 calculates the complexity C of the traffic environment for each of the region B, the region C, and the region D based on the following equation (1).
[0065]
[Expression 1]

[0066]
The index C1 indicates the number of luminance divided blocks, and the index C2 indicates the number of red-based divided blocks. Further, α, β, and γ are weighting coefficients, each having a predetermined value.
[0067]
Here, the calculation of the index C1 will be described. As described above, the microcomputer 70 calculates the index C1 for the regions B, C, and D. Here, the calculation of the index C1 for the region B using the captured image shown in FIG. 11 will be described. To do.
[0068]
The microcomputer 70 drives the obstacle photographing CCD camera 21 to obtain a captured image as shown in FIG. 11, for example. Then, the entire captured image is set as a luminance division block.
[0069]
The microcomputer 70 calculates the difference between the maximum value and the minimum value of the luminance values of each pixel in the set luminance division block. The microcomputer 70 determines whether the difference between the maximum value and the minimum value of the luminance value is equal to or greater than a predetermined threshold value, and divides the luminance division block into four when the difference exceeds the predetermined threshold value. At this time, it is preferable that the microcomputer 70 divides the luminance division block so as to be configured with substantially the same number of pixels in the vertical and horizontal directions and as large as possible.
[0070]
For example, when the brightness value of the captured image is as shown in FIG. 12A, the microcomputer 70 calculates the difference between the maximum value (180) and the minimum value (000) of the brightness values. Then, it is determined whether the difference (180) between the maximum value and the minimum value of the luminance value is a threshold value (for example, 40) or more. Here, the difference exceeds the threshold value (40). As shown, the luminance division block is divided into four.
[0071]
The microcomputer 70 divides the luminance division block into four until the difference between the maximum value and the minimum value of the luminance value of each pixel in the divided luminance division blocks is equal to or less than a predetermined threshold value, and obtains a new luminance. Repeat setting the division block.
[0072]
For example, the luminance division block BK1 shown in FIG. 12A is composed of 4 × 4 pixels, the maximum luminance value is 180, and the minimum luminance value is 20. The microcomputer 70 obtains the difference (160) between the maximum value and the minimum value of the luminance value of the luminance division block BK1, and since the difference is equal to or greater than the threshold value (40), as shown in FIG. The block BK1 is divided into four (BK2, BK3, BK4, BK5).
[0073]
The microcomputer 70 does not divide the luminance division block BK2 because the difference (39) between the maximum value and the minimum value of the luminance values of the luminance division block BK2 is not equal to or greater than the threshold value (40). On the other hand, with respect to the luminance divided blocks BK3, BK4, and BK5, the difference between the maximum value and the minimum value of the luminance values is equal to or greater than the threshold (40), so that each luminance divided block BK is divided as shown in FIG. To do.
[0074]
Through such processing, the microcomputer 70 obtains an index C1 indicating the number of luminance division blocks in which the difference between the maximum and minimum luminance values of each pixel is equal to or less than a predetermined threshold (40). Thereby, the microcomputer 70 makes the luminance values of the pixels constituting the luminance division block uniform.
[0075]
The index C1 is not limited to the number of luminance division blocks in which the difference between the maximum and minimum luminance values of each pixel is equal to or less than a predetermined threshold value. For example, the index C1 may be a variance value of the luminance value of the captured image. In addition, the index C1 may be a sum of high-frequency components of a two-dimensional fast Fourier transform (FFT) of the captured image.
[0076]
Further, as shown in FIG. 12C, the microcomputer 70 divides the luminance division block BK until it becomes 1 × 1 pixel, but the luminance division block BK is divided to have a predetermined size (for example, 4 The division may be terminated when (× 4 pixels) is reached.
[0077]
Next, the calculation of the index C2 will be described. As described above, the microcomputer 70 calculates C2 indicating the number of red-based divided blocks for each predetermined area. Here, for the area B, the index C2 is obtained using the captured image shown in FIG. The operation will be described.
[0078]
The microcomputer 70 drives the obstacle imaging CCD camera 21 to obtain a captured image as shown in FIG. Then, a brightness of 0.3 or more and a saturation of 0.1 or more are extracted from the captured image to obtain a captured image as shown in FIG. 13B, for example. Further, when the microcomputer 70 extracts a hue of 0.1 or less or 0.75 or more, a captured image as shown in FIG. 13C is obtained.
[0079]
The microcomputer 70 repeats block division for the captured image as shown in FIG. 13C so that only the red pixels occupy the same red division block, as shown in FIG. 13D. Then, a red-based divided block composed of red-based pixels (1) or pixels (0) other than red-based pixels is obtained. Then, the microcomputer 70 obtains an index C2 that is the number of red-based divided blocks composed of only red-based pixels.
[0080]
The microcomputer 70 calculates the complexity C of the environment according to the above-described equation (1) using the index C1 indicating the number of obtained luminance divided blocks and the index C2 indicating the number of red-based divided blocks. Move on to ST23. Note that the value of the complexity C calculated as described above changes depending on time as shown in FIG. 14, for example.
[0081]
The index C2 is not limited to the number of red-based divided blocks, and may be the following value, for example.
[0082]
The microcomputer 70 has a color chart of 2.5RP, 5RP, 7.5RP, 10RP, 2.5R, 5R, 7.5R, 10R, 2.5YR, in the Munsell color chart for a captured image of a predetermined area. Only red pixels belonging to 5YR and having chroma 2 or higher and brightness 3 or higher may be extracted. Then, until the same divided block is composed of only the red pixels and only pixels other than the red pixels, the division block is repeatedly divided, and the number of obtained divided blocks is set as the index C2. Good.
[0083]
Further, the microcomputer 70 has a color chart of 2.5RP, 5RP, 7.5RP, 10RP, 2.5R, 5R, 7.5R, 10R, 2. 5YR and 5YR, and only red pixels having chroma 2 or more and lightness 3 or more may be extracted, and the number of red pixels may be used as the index C2. Alternatively, the sum of the distances of the red pixels described above from the center of the captured image may be used as the index C2.
[0084]
In step ST23, the microcomputer 70 sets a threshold th for determining the degree of environment recognition. The threshold th is set to be different depending on the ambient illuminance outside the vehicle. The reason is that even if the value of complexity C is constant, the driver may feel that the traffic environment is complex or simple when the outside of the vehicle becomes bright or dark.
[0085]
For example, as shown in FIG. 15, the microcomputer 70 stores a threshold value table describing the threshold value th for the ambient illuminance. According to FIG. 15, the threshold th is the minimum value (constant) when the ambient illuminance is from zero to S1, and increases at a constant rate as the ambient illuminance increases from S1 to S2. The threshold th is the maximum value (constant) when the environmental illuminance is from S2 to S3, and decreases at a constant rate as the environmental illuminance increases from S3. Note that the threshold table shown in FIG. 15 is an example of the present embodiment, and the present invention is not limited to this.
[0086]
Therefore, the microcomputer 70 sets the threshold th based on the environmental illuminance detected by the illuminance sensor 41 with reference to the threshold table. Furthermore, the microcomputer 70 can reset the maximum value of the threshold th in accordance with the operation input value of the driver and the change in the posture of the driver.
[0087]
First, the microcomputer 70 sets the maximum value of the threshold th according to the input value of the slider 51 provided on the slider panel 50. For example, when the driver slides the slider 51 toward “strong”, the microcomputer 70 sets the maximum value of the threshold th large as shown in FIG. On the other hand, when the driver slides the slider 51 toward “weak”, the microcomputer 70 sets the maximum value of the threshold th small. If the slider 51 is between “strong” and “weak”, the microcomputer 70 may remain at the default threshold th set based on the ambient illuminance.
[0088]
Next, the microcomputer 70 sets the maximum value of the threshold th in accordance with the change in the posture of the driver. The reason is that the driver tends to recognize various environments when there is frequent posture change, and tends to be difficult to recognize the environment when there is no posture change. Therefore, the microcomputer 70 sets the threshold th as follows in consideration of the relationship between the ease of recognition of the traffic environment and the change in the posture of the driver.
[0089]
For example, when the microcomputer 70 determines that the change in the posture of the driver is “small” in step ST21 described above, the microcomputer 70 sets the maximum value of the threshold th as shown in FIG. On the other hand, for example, when it is determined in step ST21 that the change in the posture of the driver is “large”, the maximum value of the threshold th is set large. When the microcomputer 70 determines that the change in the posture of the driver is “medium”, the microcomputer 70 may keep the default threshold th set based on the environmental illuminance.
[0090]
As described above, the microcomputer 70 determines the threshold th based on the threshold value table in which the threshold value th corresponding to the ambient illuminance is described, the input value of the slider 51 provided in the slider panel 50, and the change in the attitude of the driver. Is set, the process proceeds to step ST24.
[0091]
In step ST24, as shown in FIG. 17, the microcomputer 70 compares the complexity C calculated in step ST22 with the threshold th set in step ST23. When the complexity C does not exceed the threshold th, it is determined that the environment is not complicated, that is, simple. When the complexity C exceeds the threshold th, it is determined that the environment is complex.
[0092]
The microcomputer 70 determines that the traffic environment is “simple” when the captured image obtained by the obstacle photographing CCD camera 21 is, for example, a forward scenery as shown in FIG. 18. For example, as shown in FIG. 19. In the case of a forward scenery, it is determined that the traffic environment is “complex”. Then, the subroutine processing is terminated, and the process proceeds to step ST3 shown in FIG.
[0093]
(Step ST3)
In step ST3, the microcomputer 70 issues an obstacle warning to the driver. Here, it is determined whether there is an alarm request flag for each area, and a predetermined alarm is given to the driver using a flag having a high priority. The warning request flag here refers to a flag indicating that it is necessary to give a warning to the driver due to the presence of an obstacle or the like in the predetermined area. Specifically, the microcomputer 70 executes processing from step ST31 to step ST42 shown in FIG.
[0094]
In step ST31, the microcomputer 70 determines whether or not to generate a warning request flag for the area A. Specifically, the subroutine from step ST51 to step ST54 shown in FIG. 21 is executed.
[0095]
In step ST51, the microcomputer 70 determines whether an obstacle has been detected in the area A or whether the current vehicle position is within the obstacle appearance range, and if either condition is satisfied, the microcomputer 70 proceeds to step ST52. When the process proceeds and none of the conditions are satisfied, the subroutine processing is terminated.
[0096]
In step ST52, the microcomputer 70 determines whether the vehicle is taking an avoidance action against the obstacle based on the own vehicle state detected in step ST13 described above. When the process proceeds to step ST53 and the avoidance action is not taken, the subroutine process is terminated.
[0097]
In step ST53, the microcomputer 70 determines whether it is impossible for the own vehicle to avoid the obstacle based on the information on the obstacle detected in step ST1 and the own vehicle state. For example, even if the driver steps on the brake pedal or operates the steering wheel to take an avoidance action against the obstacle, if the own vehicle cannot avoid the obstacle, the process proceeds to step ST54. When the obstacle can be avoided, the subroutine processing is terminated.
[0098]
In step ST54, the microcomputer 70 generates an alarm request flag in the area A and ends the subroutine processing. When the microcomputer 70 completes such a subroutine process, the microcomputer 70 proceeds to step ST32 shown in FIG.
[0099]
In step ST32, the microcomputer 70 determines whether to generate an alarm request flag for each of the regions B, C, and D. Specifically, the subroutine processing from step ST61 to step ST64 shown in FIG. 22 is executed. Here, since the same processing is performed in any region, the processing in region B will be described as an example.
[0100]
In step ST61, the microcomputer 70 determines whether an obstacle has been detected in the region B or whether the current vehicle position is within the obstacle appearance range, and if either condition is satisfied, the microcomputer 70 proceeds to step ST62. When the process proceeds and none of the conditions are satisfied, the subroutine processing is terminated.
[0101]
In step ST62, the microcomputer 70 determines whether or not the region B is complicated in step ST2 described above, that is, whether or not the complexity C of the region B exceeds the threshold th in step ST2. When the complexity C of the region B exceeds the threshold th, the process proceeds to step ST63, and when it does not exceed the threshold th, the subroutine processing is terminated.
[0102]
In step ST63, the microcomputer 70 determines whether the place of the obstacle or the obstacle appearance range is a place where an alarm is issued for the first time, and when it is a place where an alarm is issued for the first time, the microcomputer 70 proceeds to step ST64 and issues an alarm for the first time. If it is not a place, the subroutine processing is terminated.
[0103]
In step ST64, the microcomputer 70 generates an alarm request flag in the area B and ends the subroutine processing. When the microcomputer 70 completes the above subroutine processing, the microcomputer 70 proceeds to step ST33 shown in FIG.
[0104]
In step ST33, the microcomputer 70 determines whether or not there is an alarm request flag for area A. If there is an alarm request flag for area A, the microcomputer 70 proceeds to step ST39 and there is no alarm request flag for area A. If so, the process proceeds to step ST34.
[0105]
In step ST34, the microcomputer 70 determines whether or not there is an alarm request flag for area C or area D. Then, when there is an alarm request flag in at least one of the areas, the process proceeds to step ST38, and when there is no alarm request flag in any of the areas, the process proceeds to step ST35.
[0106]
In step ST35, the microcomputer 70 determines whether or not there is an alarm request flag for area B. If there is an alarm request flag for area B, the microcomputer 70 proceeds to step ST36 and there is no alarm request flag for area B. If so, the subroutine processing is terminated.
[0107]
As described above, the microcomputer 70 sets the priority order in the order of the area A, the area C or the area D, and the area B by executing the processes in the order from step ST33 to step ST35. The warning is performed.
[0108]
In step ST36, the microcomputer 70 determines whether or not there is an alarm request flag for area A in the previous determination process. If there is an alarm request flag for area A, the microcomputer 70 proceeds to step ST37, and the alarm request flag for area A is set. If not, the subroutine processing is terminated.
[0109]
In step ST <b> 37, the microcomputer 70 issues a warning for area B to the driver. The microcomputer 70 displays a warning to the driver on the LCD 61 such as “Please be careful about obstacles ahead” and “Be close to the intersection. Then, the subroutine processing is terminated.
[0110]
Further, in step ST38 when it is determined that there is at least one warning request flag for area C or area D in step ST34, the microcomputer 70 issues a warning to the driver regarding the area having the warning request flag. I do. An example of the alarm request is the same as step ST37.
[0111]
On the other hand, in step ST39 when it is determined that there is an alarm request flag for area A in step ST33, it is determined whether the alarm request flag for area A is equal to or greater than a predetermined number, and the alarm request flag is equal to or greater than a predetermined number. If there is, the process proceeds to step ST40, and if there is not a predetermined number of warning request flags, the process proceeds to step ST42.
[0112]
In step ST40, the microcomputer 70 determines whether or not the vehicle speed detected by the own vehicle state detection unit 30 is equal to or higher than a predetermined value. When the vehicle speed is higher than or equal to a predetermined value, the microcomputer 70 proceeds to step ST41 and the vehicle speed is equal to the predetermined value. If not, the process proceeds to step ST42.
[0113]
In step ST41, the microcomputer 70 issues a first alarm in the area A to the driver. As the first alarm for the driver, the microcomputer 70 displays, for example, “Collision warning. Please decelerate” on the LCD 61 or outputs from the speaker 62 by voice, and ends the subroutine processing.
[0114]
In step ST42, the microcomputer 70 gives a second alarm in the area A to the driver. The microcomputer 70 displays, for example, “Avoid obstacles” on the LCD 61 as the second alarm for the driver, or outputs it from the speaker 62 by voice, and the subroutine processing is terminated.
[0115]
When the microcomputer 70 completes the subroutine processing in step ST3, the microcomputer 70 terminates the alarm for the driver. As described above, the microcomputer 70 sets the priority order to the alarm request flag of each area and performs processing, thereby suppressing the alarm to the driver to the minimum and preventing the driver from being confused due to excessive information. can do.
[0116]
As described above, the obstacle alarm device 1 according to the first embodiment automatically recognizes a moving obstacle, estimates the possible range of appearance of the moving obstacle, and further at the place where the moving obstacle appears. By considering the recognition degree of the traffic environment of the driver, the driver can recognize the moving obstacle without confusing the driver. In addition, the obstacle alarm device 1 can prevent accidents by encouraging the vehicle speed of the host vehicle to be reduced when the amount of information to be presented to the driver is large.
[0117]
When the obstacle warning device 1 detects an obstacle in any of the areas A to D shown in FIG. 10, the area A immediately before the host vehicle is given the highest priority, and the obstacle existing in the area A is detected. Alarm. Thereby, the obstacle alarm device 1 can avoid the crisis that has just approached.
[0118]
[Other embodiments]
Next, another embodiment of the present invention will be described. In addition, the same code | symbol is attached | subjected about the location which overlaps with 1st Embodiment, and detailed description is abbreviate | omitted.
[0119]
(Division of captured image)
In the first embodiment, as shown in FIG. 10, the microcomputer 70 divides the imaging region into four regions from region A to region D, and indexes C1 and C2 in region B, region C, and region D. Was calculating
On the other hand, for example, the microcomputer 70 may divide the captured image shown in FIG. 23A into four regions as shown in FIG. That is, the microcomputer 70 divides the captured image into four regions by dividing the rectangular captured image along two diagonal lines. Then, similarly to the first embodiment, the index C1 and the index C2 may be calculated for each region.
[0120]
Further, the microcomputer 70, for example, converts the captured image shown in FIG. 24A into four areas, that is, an area (1), two areas (2), and an area (3) as shown in FIG. It may be divided. Here, the region (1) is a region indicating the vehicle immediately before the host vehicle. Region (2) is a region on both outer sides of region (1). Region (3) is a region other than region (1) and region (2).
[0121]
The microcomputer 70 divides the luminance division block into 1 × 1 pixels for the region (1), divides up to 2 × 2 pixels for the region (2), and divides up to 2 × 2 pixels for the region (3). Thus, the index C1 can be calculated in each region. Furthermore, the microcomputer 70 can divide the red-based divided block into 1 × 1 pixels in each region, obtain an image shown in FIG. 24C, and calculate the index C2 in each region.
[0122]
Furthermore, the microcomputer 70 can also divide the captured image as follows. For example, as shown in FIG. 25, the captured image may be divided into “bottom of screen”, “center of screen”, “left of screen”, and “right of screen”. The “lower part of the screen” is a rectangular area from about 1/5 to 1/4 in the vertical direction below the captured image. The “center of the screen” is a triangular area, and one side of the triangle is adjacent to the “lower part of the screen”, and the diagonal of the one side is located at the upper end of the imaging screen. The “left side of the screen” is a trapezoidal region on the left side of the region other than “the lower part of the screen” and “the center of the screen”. The “right side of the screen” is a trapezoidal region on the right side of the region other than the “bottom of the screen” and the “center of the screen”.
[0123]
The center of the screen may be further divided as shown in FIG.
[0124]
(Setting of threshold th)
The microcomputer 70 is not limited to the case where the threshold th is set according to the input value of the slider 51 operated by the driver as in the first embodiment. For example, each input of the slider 51 operated by a plurality of drivers is used. You may make it set according to a value.
[0125]
FIG. 27 is a diagram in which the recognition degree of the environment input by the driver is plotted when ten captured images are presented to ten drivers. The microcomputer 70 can set an appropriate threshold th by, for example, calculating the average of these values.
[0126]
(Other calculation methods of complexity C)
The microcomputer 70 calculates the complexity C according to the equation (1) and compares the complexity C with the threshold th to estimate the degree of environment recognition. However, the present invention is not limited to this. .
[0127]
For example, as shown in FIG. 28, the microcomputer 70 may store a recognition degree map indicating the recognition degrees determined by the indices C1 and C2. At this time, the microcomputer 70 may calculate the index C1 and the index C2 in the same manner as in the first embodiment, and estimate the environment recognition level with reference to the recognition level map. For example, the microcomputer 70 determines that the degree of recognition of the environment is high when (C1, C2) = (a, b), and the degree of recognition of the environment when (C1, C2) = (c, d). It can be determined that the value is low.
[0128]
(3D space layout)
The microcomputer 70 divides the captured image into predetermined areas in advance, calculates the index C1 and the index C2 for each predetermined area as in the first embodiment, and further calculates the indices C1 and C2. Then, the index C1, the index C2, and the indices C1 and C2 are arranged in a three-dimensional vector space as shown in FIG. 29, and the value (magnitude, direction) of the movement vector on each pixel is used as the complexity C. You can also.
[0129]
At this time, for example, as shown in FIG. 30, a first area that is estimated to have a low degree of environmental recognition and a second area that is estimated to have a high degree of environmental recognition are included in a three-dimensional vector space. It may be provided. The microcomputer 70 determines which of the areas the complexity C obtained using the three-dimensional space belongs, and if the complexity C is in the first area, the driver has a low degree of environmental recognition. Assuming that the complexity C is in the second region, the driver can estimate that the degree of environment recognition is high.
[0130]
(Neural network)
The microcomputer 70 may estimate the degree of environment recognition using a neural network. FIG. 31 is a block diagram showing a functional configuration of the microcomputer 70 that estimates the degree of recognition of the environment using a neural network.
[0131]
The microcomputer 70 includes a neural network 71 composed of a conversion function having an updatable database, a recognition degree estimation unit 72 that estimates the recognition degree of the driver environment, and an arithmetic unit 73 that performs a process of subtracting the estimated value from the input value. And. Instead of the neural network 71, a learning function may be provided by a statistical method.
The computing unit 73 subtracts the estimated value of the environment recognition level estimated by the recognition level estimation unit 72 from the input value of the slider 51 shown in FIG. The neural network 71 sequentially learns based on the subtraction value subtracted by the computing unit 73 and the complexity C, and supplies the learned complexity C to the recognition degree estimation unit 72. The recognition degree estimation unit 72 may estimate the environment recognition degree for each driver using the learned complexity C from the neural network 71.
[0132]
As a result, the microcomputer 70 can sequentially learn in consideration of the driver's actual environment recognition, and can accurately estimate the degree of recognition of the traffic environment according to the driver's individual visual characteristics.
[0133]
【The invention's effect】
The environmental complexity computing device according to the present invention is based on at least one of a luminance change distribution included in a captured image generated by the imaging unit and a red pixel distribution included in the captured image. By calculating the degree, it is possible to obtain the difficulty of recognizing the environment, that is, the complexity indicating the complexity of the environment.
[0134]
The environment recognition degree estimation device according to the present invention estimates the recognition degree of the environment of the observer objectively and accurately by estimating the recognition degree of the environment based on the complexity calculated by the complexity calculation device. Can do.
[0135]
The obstacle alarm device according to the present invention detects the position of an obstacle, and performs an alarm regarding the obstacle when a predetermined area including the detected obstacle position is estimated to have a low degree of environmental recognition. Thus, when there is an obstacle in an area where the degree of environmental recognition is low, the observer can be warned of the presence of the obstacle. On the other hand, since an alarm is not issued when there is an obstacle in a region where the degree of environmental recognition is high, it is possible to prevent the observer from being bothered by an extra alarm.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of an obstacle alarm device according to an embodiment of the present invention.
FIG. 2 is a diagram showing a configuration of a slider panel provided in the obstacle alarm device.
FIG. 3 is a flowchart showing a main routine of an operation procedure of a microcomputer provided in the obstacle alarm device.
FIG. 4 is a flowchart showing a specific operation procedure of step ST1 in the main routine.
FIG. 5 is a diagram for explaining an outline of a data carrier and a data carrier reader;
FIG. 6 is a diagram showing an appearance range estimation table stored in a microcomputer.
FIG. 7 is a diagram showing an obstacle display screen displayed on the LCD.
FIG. 8 is a flowchart showing a specific operation procedure of step ST2 in the main routine.
FIG. 9 is a diagram illustrating a state in which an obstacle display screen is divided from region A to region D.
FIG. 10 is a diagram for explaining a state in which an obstacle display screen is divided into a region A and a region D schematically.
11 is a diagram illustrating an example of a captured image when calculating an index C1 in a region B. FIG.
FIG. 12 is a diagram illustrating an example of a luminance value of a captured image.
FIG. 13 is a diagram illustrating an example of a captured image.
FIG. 14 is a diagram illustrating a state in which complexity C changes with time.
FIG. 15 is a diagram illustrating a threshold value table in which threshold values for environmental illuminance are described.
FIG. 16 is a diagram illustrating a state in which a threshold th changes according to a change in the posture of a driver.
FIG. 17 is a diagram for estimating the recognition degree of the environment by comparing the complexity C and the threshold th.
FIG. 18 is a diagram illustrating an example of a landscape image with a simple traffic environment.
FIG. 19 is a diagram illustrating an example of a landscape image having a complicated traffic environment.
FIG. 20 is a flowchart showing a specific operation procedure of step ST3 in the main routine.
FIG. 21 is a flowchart showing a specific operation procedure of step ST31 in the subroutine.
FIG. 22 is a flowchart showing a specific operation procedure of step ST32 in the subroutine.
FIG. 23 is a diagram for explaining another method for dividing a captured image;
FIG. 24 is a diagram for explaining another technique for dividing a captured image;
FIG. 25 is a diagram for explaining another technique for dividing a captured image;
FIG. 26 is a diagram illustrating an example of division at the center of the screen.
FIG. 27 is a diagram plotting the recognition degree of the environment input by a driver when ten captured images are presented to ten drivers.
FIG. 28 is a diagram showing a recognition degree map showing the recognition degrees determined by the indicators C1 and C2.
FIG. 29 is a diagram illustrating a three-dimensional vector space including an index C1, an index C2, and indices C1 and C2.
FIG. 30 is a diagram illustrating a state in which a first region estimated to have a low degree of environmental recognition and a second region estimated to have a high degree of environmental recognition are provided in a three-dimensional vector space. It is.
FIG. 31 is a block diagram showing a functional configuration of a microcomputer that estimates the degree of recognition of an environment using a neural network.
[Explanation of symbols]
1 Obstacle alarm device
10 Infrastructure information detector
20 Obstacle information detector
40 Environmental Information Detection Unit
60 Obstacle information output section
70 Microcomputer

Claims (9)

Imaging means for generating a captured image based on imaging light from the environment;
A luminance division block is set in the captured image generated by the imaging unit, and the luminance division block is divided into new ones until it is determined that the luminance value of each pixel is uniform within the set luminance division block. Complexity calculating means for calculating the complexity of the environment based on the number of luminance divided blocks in which the luminance value of each pixel is uniform with respect to the captured image, repeatedly setting luminance divided blocks ;
Environmental complexity computing device. Imaging means for generating a captured image based on imaging light from the environment;
A red-based divided block is set in the captured image generated by the imaging unit, and the red-based divided block is divided into new red colors until the set red-based divided block is configured by only predetermined red-based pixels. Repetitively setting a system division block, and for the captured image, a complexity calculation means for calculating the complexity of the environment based on the number of red system division blocks composed of only the predetermined red system pixels ;
Environmental complexity computation device equipped with. Imaging means for generating a captured image based on imaging light from the environment;
A luminance division block is set in the captured image generated by the imaging unit, and the luminance division block is divided into new ones until it is determined that the luminance value of each pixel is uniform within the set luminance division block. Repeating the setting of the luminance division block, calculating the number of luminance division blocks in which the luminance values of the respective pixels are uniform for the captured image, and adding the red division block to the captured image generated by the imaging means Until the set red-based divided block is composed of only predetermined red-based pixels, and repeatedly setting the new red-based divided block by dividing the red-based divided block into the captured image. On the other hand, the number of red-based divided blocks composed of only the predetermined red-based pixels is calculated, and the number of red-colored divided blocks and the number of red-based divided blocks are calculated based on the number of red-colored divided blocks. A complexity calculating means for calculating a complexity of,
Environmental complexity computation device equipped with. The environmental complexity computing device according to any one of claims 1 to 3 ,
An environment recognition degree estimating means for estimating an environment recognition degree based on the complexity computed by the complexity computing device;
An environment recognition degree estimation device comprising: The environment recognition degree estimation means estimates that the degree of recognition of the environment is low when the complexity calculated by the complexity calculator is equal to or greater than a threshold, and the complexity calculated by the complexity calculator is greater than the threshold. The environment recognition level estimation device according to claim 4, wherein when it is small, the environment recognition level is estimated to be high. It further comprises illuminance detection means for detecting the illuminance of the environment,
The environment recognition degree estimating means, on the basis of the threshold table to which the threshold has been described corresponding to the illuminance of the environment, characterized in that you set the threshold value corresponding to the detected illuminance by the illuminance detecting means 5. Symbol mounting environment recognition degree estimation device. It further comprises posture change detecting means for detecting the posture change of the observer,
The environment recognition degree estimation means sets the maximum value of the threshold value described in the threshold value table as the observer's posture change detected by the posture change detection means increases, and is detected by the posture change detection means. observers of environment recognition degree estimation apparatus according to claim 6, wherein the posture change is characterized that you set smaller the maximum value of the threshold in accordance reduced. The environment recognition degree estimation device according to any one of claims 4 to 7 , which estimates an environment recognition degree for each predetermined area;
Obstacle position detecting means for detecting the position of the obstacle;
A warning unit that issues a warning about the obstacle when the predetermined area including the position of the obstacle detected by the obstacle detection unit is estimated by the environment recognition degree estimation unit to have a low degree of environment recognition;
Obstacle alarm device with. The environment recognition degree estimation device divides an area excluding a lower part of a captured image into a plurality of estimation areas, estimates an environment recognition degree for each divided estimation area,
The warning means is estimated that the position of the obstacle detected by the obstacle position detecting means is in any of the estimated areas, and the environment recognition degree of the estimated area is low by the environment recognition degree estimating means. The obstacle alarm device according to claim 8 , wherein an alarm regarding an obstacle existing in an estimated area having a high priority is issued.

Images