JP2004347471A - Preceding running car detection device - Google Patents

Preceding running car detection device Download PDF

Info

Publication number
JP2004347471A
JP2004347471A JP2003145235A JP2003145235A JP2004347471A JP 2004347471 A JP2004347471 A JP 2004347471A JP 2003145235 A JP2003145235 A JP 2003145235A JP 2003145235 A JP2003145235 A JP 2003145235A JP 2004347471 A JP2004347471 A JP 2004347471A
Authority
JP
Japan
Prior art keywords
preceding vehicle
vehicle
radar
same
recognizing means
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
JP2003145235A
Other languages
Japanese (ja)
Other versions
JP4265931B2 (en
Inventor
Takashige Kaji
孝慈 梶
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Subaru Corp
Original Assignee
Fuji Heavy Industries Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fuji Heavy Industries Ltd filed Critical Fuji Heavy Industries Ltd
Priority to JP2003145235A priority Critical patent/JP4265931B2/en
Publication of JP2004347471A publication Critical patent/JP2004347471A/en
Application granted granted Critical
Publication of JP4265931B2 publication Critical patent/JP4265931B2/en
Anticipated expiration legal-status Critical
Expired - Fee Related legal-status Critical Current

Links

Images

Landscapes

  • Traffic Control Systems (AREA)
  • Optical Radar Systems And Details Thereof (AREA)
  • Image Analysis (AREA)
  • Measurement Of Optical Distance (AREA)
  • Controls For Constant Speed Travelling (AREA)
  • Image Processing (AREA)
  • Radar Systems Or Details Thereof (AREA)

Abstract

<P>PROBLEM TO BE SOLVED: To correctly detect an ahead running car as a target by using an image information of a stereo-camera, and a radar information of a laser or millimeter waves. <P>SOLUTION: The steps of the detection device are as follows: the S105 determines whether the horizontal positions of the preceding running cars by the stereo-camera 4 and that of the preceding running car by the laser radar are coincident; the S112 determines that the preceding running cars are not the same, if the positions are not coincident; if they are not the same, after reflecting the variation of theoretical distances to each distance by the stereo-camera, the S108 determines whether the distance by the stereo camera 4 and the distance to the existent preceding running cars by the laser radar are the same; the S111 determines that the preceding running cars are the same, if the distances are same; the S109 determines whether the speed of the cars are same, if the cars are not the same; the S111 determines that the preceding running cars are same, if the speeds are the same; the S110 compares the ID number of the preceding running car with that of the previous time, if the cars are not the same; S111 determines that the preceding running cars are the same, if the ID numbers are the same; and S112 determines that the preceding running cars are not the same, if the ID numbers are not the same. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

【０００１】
【発明の属する技術分野】
本発明は、ステレオカメラからの画像情報とレーザやミリ波等のレーダ情報とを基に前方に存在する先行車両を認識する先行車両検出装置に関する。
【０００２】
【従来の技術】
従来より、車間距離検出装置として、レーザレーダや、ミリ波レーダを用いた車間距離自動維持運転システム（Ａｄａｐｔｉｖｅ Ｃｒｕｉｓｅ Ｃｏｎｔｒｏｌ）が実用化されている。しかし、レーザレーダやミリ波レーダ等のレーダ情報による車間検出装置は進行路推定方法が前方環境を視覚的に広く捉えようとするものではないため、先行車両の誤認識やロストが度々発生する。
【０００３】
これに対し、特開平８−３２９３９３号公報では、カメラを付加し、カメラで検出した先行車両の存在領域（水平方向のみ）にレーザレーダで検出した車両が重なっていれば、先行車両と判断する方式にて先行車両を確実に検出する方法を提案している。
【０００４】
【特許文献１】
特開平８−３２９３９３号公報
【０００５】
【発明が解決しようとする課題】
しかしながら、上述の先行技術では、カメラからの出力とレーザレーダからの出力の横位置（水平方向位置）の比較しか行っておらず、また、カメラによる立体物検出も行っていないため、カメラによる検出結果が本当に立体物であり、先行車両なのかの判定が不確実である。例えば単眼カメラの場合、レンズ表面の先行車両の走行領域に相当する箇所にゴミが付着した場合、カメラは先行車両と誤認識する可能性がある。そして、横位置比較しか行っていないため、誤認識した方向に存在するレーザレーダの情報を先行車両として認識してしまう可能性がある。
【０００６】
また、レーザレーダの換わりにミリ波レーダを用いた場合、ミリ波レーダはオーバーパスにより先先行車（先行車両の前方に存在する先行車両）を検出してしまうことがある。その時、先行車両と先先行車の横位置は略重なっているため、従来技術による先行車両検出装置では先先行車を先行車両として検出してしまい、ＡＣＣの誤制御を生じてしまう虞がある。
【０００７】
本発明は上記事情に鑑みてなされたもので、ステレオカメラからの画像情報とレーザやミリ波等のレーダ情報とを用いて、目標とする先行車両を正確に検出することが可能な先行車両検出装置を提供することを目的としている。
【０００８】
【課題を解決するための手段】
上記目的を達成するため請求項１記載の本発明による先行車両検出装置は、車両前方の画像情報を基に前方に存在する立体物と自車両との距離データを検出し先行車両を認識する画像先行車両認識手段と、車両前方のレーダ情報を基に前方に存在する立体物と自車両との距離データを検出し先行車両を認識するレーダ先行車両認識手段と、上記画像先行車両認識手段で認識した先行車両と上記レーダ先行車両認識手段で認識した先行車両とを比較して、これら先行車両が存在する水平方向位置が略等しく、且つ、これら先行車両までの上記自車両からの距離が略等しい場合、これら先行車両は同一の先行車両であると判断する先行車両判断手段とを備えたことを特徴としている。
【０００９】
また、請求項２記載の本発明による先行車両検出装置は、請求項１記載の先行車両検出装置において、上記先行車両判断手段は、上記画像先行車両認識手段で認識した先行車両と上記レーダ先行車両認識手段で認識した先行車両とを比較する際、上記画像先行車両認識手段で上記先行車両を認識する場合に発生する距離のばらつきと、上記画像先行車両認識手段と上記レーダ先行車両認識手段との取り付け位置関係で発生する距離のばらつきとの少なくともどちらかの補正を行い比較することを特徴としている。
【００１０】
更に、請求項３記載の本発明による先行車両検出装置は、請求項１又は請求項２に記載の先行車両検出装置において、上記先行車両判断手段は、上記画像先行車両認識手段で認識した先行車両と上記レーダ先行車両認識手段で認識した先行車両とを比較して、これら先行車両が存在する水平方向位置は略等しく、且つ、これら先行車両までの上記自車両からの距離が略等しくない場合は、これら先行車両の速度を比較して該先行車両速度が一致する場合は、これらの先行車両は同一の先行車両であると判断することを特徴としている。
【００１１】
また、請求項４記載の本発明による先行車両検出装置は、請求項１乃至請求項３の何れか一つに記載の先行車両検出装置において、上記画像先行車両認識手段と上記レーダ先行車両認識手段は、検出した上記立体物毎に識別番号を設定するものであって、上記先行車両判断手段は、上記画像先行車両認識手段で認識した先行車両と上記レーダ先行車両認識手段で認識した先行車両とを比較して、これら先行車両が存在する水平方向位置は略等しく、且つ、これら先行車両までの上記自車両からの距離が略等しくない場合は、これら先行車両の認識番号が前回に同一の先行車両であると判断した先行車両の認識番号と一致する場合は、これら先行車両は同一の先行車両であると判断することを特徴としている。
【００１２】
すなわち、上記請求項１記載の先行車両検出装置は、画像先行車両認識手段で車両前方の画像情報を基に前方に存在する立体物と自車両との距離データを検出し先行車両を認識し、レーダ先行車両認識手段で車両前方のレーダ情報を基に前方に存在する立体物と自車両との距離データを検出し先行車両を認識する。そして、先行車両判断手段は、画像先行車両認識手段で認識した先行車両とレーダ先行車両認識手段で認識した先行車両とを比較して、これら先行車両が存在する水平方向位置が略等しく、且つ、これら先行車両までの自車両からの距離が略等しい場合、これら先行車両は同一の先行車両であると判断する。このように、画像先行車両認識手段で検出した立体物の先行車両とレーダ先行車両認識手段で検出した立体物の先行車両を、水平方向位置と距離とで比較し同一の先行車両の特定を行うため、目標とする先行車両を正確に検出することが可能となる。
【００１３】
また、上記請求項２記載の本発明による先行車両検出装置は、請求項１記載の先行車両検出装置において、先行車両判断手段は、画像先行車両認識手段で認識した先行車両とレーダ先行車両認識手段で認識した先行車両とを比較する際、画像先行車両認識手段で先行車両を認識する場合に発生する距離のばらつきと、画像先行車両認識手段とレーダ先行車両認識手段との取り付け位置関係で発生する距離のばらつきとの少なくともどちらかの補正を行い比較することで、より正確な比較が行える。
【００１４】
更に、上記請求項３記載の本発明による先行車両検出装置は、請求項１又は請求項２に記載の先行車両検出装置において、先行車両判断手段は、画像先行車両認識手段で認識した先行車両とレーダ先行車両認識手段で認識した先行車両とを比較して、これら先行車両が存在する水平方向位置は略等しく、且つ、これら先行車両までの自車両からの距離が略等しくない場合は、これら先行車両の速度を比較して該先行車両速度が一致する場合は、これらの先行車両は同一の先行車両であると判断する。特に、路肩の停止車両の脇をすり抜ける際、追従している先行車両と停止車両の位置が近接している場合、先行車両を誤認識する場合があるが、このように先行車両の速度で比較することにより、誤認識を防止する。
【００１５】
また、上記請求項４記載の本発明による先行車両検出装置は、請求項１乃至請求項３の何れか一つに記載の先行車両検出装置において、画像先行車両認識手段とレーダ先行車両認識手段は、検出した立体物毎に識別番号を設定するものであって、先行車両判断手段は、画像先行車両認識手段で認識した先行車両とレーダ先行車両認識手段で認識した先行車両とを比較して、これら先行車両が存在する水平方向位置は略等しく、且つ、これら先行車両までの自車両からの距離が略等しくない場合は、これら先行車両の認識番号が前回に同一の先行車両であると判断した先行車両の認識番号と一致する場合は、これら先行車両は同一の先行車両であると判断する。たとえ、今回、先行車両の特定が困難な状況となっても、前回までの情報を利用して、特定できるようにする。
【００１６】
【発明の実施の形態】
以下、図面に基づいて本発明の実施の形態を説明する。
図１乃至図６は本発明の実施の形態を示し、図１は先行車両検出装置を有する車間距離自動維持運転システムの概略構成図、図２は先行車両検出手順を示すフローチャート、図３はステレオカメラとレーザレーダの取り付け位置の違いと障害物検出の水平方向位置の違いの説明図、図４はステレオカメラとレーザレーダの取り付け位置の詳細説明図、図５は先行車両に対するステレオカメラで検出する領域とレーザレーダで検出する領域の説明図、図６はステレオカメラによる計測距離と理論誤差の説明図である。
【００１７】
図１において、符号１は自動車等の車両（自車両）で、この車両１には、ＡＣＣシステム２が搭載されている。このＡＣＣシステム２は、主として、走行制御ユニット３、ステレオカメラ４、レーザ投光ユニット５、先行車両検出装置６とで構成され、定速走行制御状態のときは運転者が設定した車速を保持した状態で走行し、追従走行制御状態のときは目標車速を先行車両の車速に設定し、先行車両に対して一定車間距離を保持した状態で走行する。
【００１８】
ステレオカメラ４は、ステレオ光学系として例えば電荷結合素子（ＣＣＤ）等の固体撮像素子を用いた１組の（左右の）ＣＣＤカメラで構成され、これら左右のＣＣＤカメラは、それぞれ車室内の天井前方に一定の間隔をもって取り付けられ、車外の対象を異なる視点からステレオ撮像し、先行車両検出装置６に入力する。
【００１９】
また、レーザ投光ユニット５は、車両１のバンパー構造材の上部に配設され、レーザビームの投射・受光と左右方向への走査機能を有するレーザヘッドを備えて構成され、先行車両検出装置６と接続されている。そして、このレーザ投光ユニット５からはレーザビームが水平に投射され、道路表面より高い位置にある立体物のみが検出されると共に、このレーザ投光ユニット５からのレーザビームは左右方向に走査され、所定の走査範囲で一定の間隔毎にレーザビームが投光・受光されて距離を検出する動作が繰り返され、立体物の二次元分布が計測される。
【００２０】
また、自車両１には、車速を検出する車速センサ７が設けられており、この車速は走行制御ユニット３と先行車両検出装置６とに入力される。
【００２１】
先行車両検出装置６は、ステレオカメラ４からの画像、レーザ投光ユニット５による走査結果、及び、車速が入力され、ステレオカメラ４からの画像に基づく前方立体物との距離と先行車両、レーザ投光ユニット５による走査結果に基づく前方立体物との距離と先行車両とをそれぞれ独立して求め、これら先行車両を後述する先行車両検出手順を示すフローチャートに従って統合し、最終的に先行車両を判断し、走行制御ユニット３に出力する。
【００２２】
ここで、先行車両検出装置６における、ステレオカメラ４からの画像の処理は、例えば以下のように行われる。まず、ステレオカメラ４のＣＣＤカメラによって撮像した左右１組のステレオ画像対を処理して画像全体に渡る３次元の距離分布を算出し、その距離分布情報から、道路形状や複数の立体物の３次元位置を高速で検出する。そして、検出した道路形状と各立体物の位置を比較して、走行路上の自車両直前の先行車両を特定し、自車両と先行車両との車間距離を求め、また、車間距離の変化及び車速センサ７で検出した自車速に基づいて、先行車速を算出する。このようにステレオカメラ４と先行車両検出装置６は、車両前方の画像情報を基に前方に存在する立体物と自車両との距離データを検出し先行車両を認識する画像先行車両認識手段としての機能を有している。尚、ステレオカメラ４からの画像を基に検出した各立体物には、予め決められたルール（例えば、自車両１の走行路上の近くの立体物から他の走行路上の立体物にいくに従って順に大きくなるようにナンバリングしていく）に基づいて識別番号（ＩＤ番号）が付与される。
【００２３】
また、先行車両検出装置６における、レーザレーダによる立体物及び先行車両の検出は、例えば、レーザ投光ユニット５によるレーザビームの投光受光の所要時間から各立体物までの距離を計算し、また、レーザビームを走査する方向から物体の二次元の位置を計算してレーザ距離データとして保存する。そして、各立体物との車間距離の変化及び車速センサ７で検出した自車速に基づいて各立体物の中から、自車両１の前方に存在する先行車両を特定する。このようにレーザ投光ユニット５と先行車両検出装置６は、車両前方のレーダ情報を基に前方に存在する立体物と自車両との距離データを検出し先行車両を認識するレーダ先行車両認識手段としての機能を有している。尚、ステレオカメラ４からの画像を基に検出した各立体物と同様に、レーザ投光ユニット５で検出した各立体物には、予め決められたルール（例えば、自車両１の走行路上の近くの立体物から他の走行路上の立体物にいくに従って順に大きくなるようにナンバリングしていく）に基づいて識別番号（ＩＤ番号）が付与される。
【００２４】
走行制御ユニット３は、運転者の操作入力によって設定される走行速度を維持するよう定速走行制御を行なう定速走行制御の機能、及び自車両と先行車両の車間距離を一定に保持した状態で走行する追従走行制御の機能を実現するもので、ステアリングコラムの側部等に設けられた定速走行操作レバーに連結される複数のスイッチ類で構成された定速走行スイッチ８、先行車両検出装置６、車速センサ７等が接続されている。定速走行スイッチ８は、定速走行時の目標車速を設定する車速セットスイッチ、主に目標車速を下降側へ変更設定するコーストスイッチ、主に目標車速を上昇側へ変更設定するリジュームスイッチ等で構成されている。更に、この定速走行操作レバーの近傍には、走行制御のＯＮ／ＯＦＦを行うメインスイッチ（図示せず）が配設されている。
【００２５】
運転者が図示しないメインスイッチをＯＮし、定速走行操作レバーにより、希望する速度をセットすると、定速走行スイッチ８からの信号が走行制御ユニット３に入力され、車速センサ７で検出した車速が、運転者のセットした設定車速に収束するように、スロットルアクチュエータ９を駆動させてスロットル弁１０の開度をフィードバック制御し、自車両を自動的に定速状態で走行させる。
【００２６】
又、走行制御ユニット３は、定速走行制御を行っている際に、先行車両検出装置６にて先行車両を認識し、先行車両の速度が自車両の設定した目標速度以下の場合には、先行車両に対して一定の車間距離を保持した状態で走行する追従走行制御へ自動的に切換えられる。
【００２７】
車両の走行制御が追従走行制御へ移行すると、先行車両検出装置６で求めた自車両と先行車両との車間距離及び先行車速と、車速センサ７で検出した自車速とに基づき適切な車間距離の目標値を設定し、車間距離が目標値になるように、スロットルアクチュエータ９へ駆動信号を出力して、スロットル弁１０の開度をフィードバック制御し、先行車両に対して一定車間距離を保持した状態で追従走行させる。
【００２８】
次に、先行車両検出装置６における先行車両検出手順を、図２に示すフローチャートで説明する。図２に示すルーチンは所定時間（例えば５０ｍｓｅｃ）毎に起動され、先ず、ステップ（以下「Ｓ」と略称）１０１で、ステレオカメラ４、レーザ投光ユニット５等の各センサのチェックが行われ、これらセンサの何れかに異常がある場合には、そのままプログラムを終了する。
【００２９】
一方、Ｓ１０１で各センサが正常であることが確認されるとＳ１０２に進み、ステレオカメラ４からの画像を基に得られる様々な情報、すなわち、自車両１前方の先行車両に関するデータ（自車両と先行車両との車間距離、先行車速、前方の各立体物それぞれのＩＤ番号と先行車両のＩＤ番号等）を読み込む。
【００３０】
次いで、Ｓ１０３に進み、レーザレーダにより得られる様々な情報、すなわち、自車両１前方の先行車両に関するデータ（自車両と先行車両との車間距離、先行車速、前方の各立体物それぞれのＩＤ番号と先行車両のＩＤ番号等）を読み込む。
【００３１】
その後、Ｓ１０４に進み、ステレオカメラ４とレーザ投光ユニット５の取り付け位置の違いによる角度補正を行う。すなわち、図３に示すように、ステレオカメラ４とレーザ投光ユニット５で同一先行車両の同一部分を検出する時、これらの取り付け位置が異なっていると、この部分はステレオカメラ４では角度ａで検出されるが、レーザ投光ユニット５では角度ｂで検出されることになる。
【００３２】
従って、図４に示すように、レーザ投光ユニット５がステレオカメラ４の前方１９００ｍｍの位置に左側に１７０ｍｍづらして、共に正面を向くように配設されているものとすると、取り付け位置の違いにより生じる理論的な先行車両角度差は、以下の（１）式で算出される。レーザ投光ユニット５による先行車両距離をＬｂ、レーザ投光ユニット５による先行車両角度をθｂ、理論式より算出されるステレオカメラ４の先行車両角度をθａ、レーザ投光ユニット５とステレオカメラ４との車幅方向ずれ量をＸ（＝１７０ｍｍ）、レーザ投光ユニット５とステレオカメラ４との前後方向ずれ量をＹ（＝１９００ｍｍ）として、
θａ＝ｔａｎ−１（（Ｌ・ｓｉｎθ−Ｘ）／Ｌ・ｃｏｓθ） …（１）
ここで、
θ＝ｔａｎ−１（Ｌｂ・ｓｉｎθ／（Ｌｂ・ｃｏｓθｂ＋Ｙ））
Ｌ＝（Ｌｂ・ｃｏｓθｂ＋Ｙ）／ｃｏｓθ
である。
【００３３】
次いで、Ｓ１０５に進むと、ステレオカメラ４とレーザレーダとによる先行車両の水平方向位置が一致しているか否か判定される。
【００３４】
具体的には、図５に示すように、ステレオカメラ４からの画像で検出される先行車両の左エッジ角度情報はＳＬの領域で抽出され、右エッジ角度情報はＳＲの領域で抽出される。従って、ＳＬからＳＲの間のＬｓの間にレーザレーダで検出した先行車両の水平方向の角度情報が存在しているか否か判定される。
【００３５】
Ｓ１０５の判定の結果、ＳＬからＳＲの間のＬｓの間にレーザレーダで検出した先行車両の水平方向の角度情報が存在していない場合、すなわち、ステレオカメラ４とレーザレーダとによる先行車両の水平方向位置が一致していない場合は、Ｓ１１２に進み、ステレオカメラ４による先行車両とレーザレーダとによる先行車両は不一致と判定しプログラムを抜ける。
【００３６】
逆に、Ｓ１０５の判定の結果、ＳＬからＳＲの間のＬｓの間にレーザレーダで検出した先行車両の水平方向の角度情報が存在している場合、すなわち、ステレオカメラ４とレーザレーダとによる先行車両の水平方向位置が一致している場合は、Ｓ１０６に進む。
【００３７】
Ｓ１０６では、ステレオカメラ４の距離の理論的距離ばらつきの計算を行い、ステレオカメラ４からの画像を基に得られる自車両と先行車両との車間距離の値に予め反映させる。例えば、ステレオカメラ４の仕様により発生する理論的な距離誤差（距離Ｄにおける距離誤差）ΔＤは、以下の（２）式により算出される。焦点距離をＦ、ＣＣＤの水平長と垂直長を共にｄｘ、ステレオカメラ４のカメラ間隔をＬ、走査線数をＭ、物体の高さをＨ、認識物体として映る４×４画素ブロック数をＮとして、
ΔＤ＝（Ｄ・Ｄ・ｄｘ）／（（Ｆ・Ｌ−Ｄ・ｄｘ）・Ｎ１／２）…（２）
ここで、
Ｎ＝（（Ｍ／４）・Ｆ・Ｈ・２）／（ｄｘ・Ｄ）
こうして、上述の（２）式を用いて求められる理論的な距離誤差ΔＤは、図６に示すように得られる。
【００３８】
Ｓ１０６で理論的距離ばらつきの計算を行い、ステレオカメラ４からの画像を基に得られる自車両と先行車両との車間距離の値に反映させた後は、Ｓ１０７に進み、ステレオカメラ４の距離の外環境変化によるばらつきの計算を行い、ステレオカメラ４からの画像を基に得られる自車両と先行車両との車間距離の値に予め反映させる。この外環境の変化によるばらつきは、例えば、急激な輝度変化等により発生する誤差であり、予め実験等により設定しておいた値である。
【００３９】
その後、Ｓ１０８に進み、ステレオカメラ４とレーザレーダとによる先行車両の存在する距離が一致するか否か判定する。
【００４０】
このＳ１０８での判定の結果、ステレオカメラ４とレーザレーダとによる先行車両の存在する距離が一致する場合はＳ１１１に進み、ステレオカメラ４とレーザレーダとによる先行車両は同一の車両であるとして先行車両の検出を終え、プログラムを抜ける。
【００４１】
逆に、ステレオカメラ４とレーザレーダとによる先行車両の存在する距離が不一致の場合は、Ｓ１０９に進み、ステレオカメラ４とレーザレーダとによる先行車両の速度が一致するか否か判定する。
【００４２】
このＳ１０９の判定の結果、ステレオカメラ４とレーザレーダとによる先行車両の速度が一致する場合はＳ１１１に進み、ステレオカメラ４とレーザレーダとによる先行車両は同一の車両であるとして先行車両の検出を終え、プログラムを抜ける。
【００４３】
逆に、ステレオカメラ４とレーザレーダとによる先行車両の速度が不一致の場合はＳ１１０に進み、ステレオカメラ４による先行車両のＩＤ番号とレーザレーダによる先行車両のＩＤ番号が前回設定したＩＤ番号と一致するか否か判定し、一致する場合はＳ１１１に進み、ステレオカメラ４とレーザレーダとによる先行車両は同一の車両であるとして先行車両の検出を終え、プログラムを抜ける。
【００４４】
逆に、ステレオカメラ４による先行車両のＩＤ番号とレーザレーダによる先行車両のＩＤ番号が前回設定した先行車両のＩＤ番号と不一致であればＳ１１２に進み、ステレオカメラ４による先行車両とレーザレーダとによる先行車両は不一致と判定しプログラムを抜ける。尚、この場合、前回設定した先行車両のＩＤ番号と一致する方の先行車両を今回の先行車両とする。
【００４５】
こうして、上述の先行車両検出手順からも明らかなように、先行車両検出装置６は先行車両判断手段としての機能を有している。
【００４６】
このように本実施の形態によれば、ステレオカメラ４で検出した先行車両とレーザレーダで検出した先行車両を、水平方向位置と距離とで比較し同一の先行車両の特定を行うため、目標とする先行車両を正確に検出することが可能となる。
【００４７】
また、本実施の形態では、ステレオカメラ４で認識した先行車両とレーザレーダで認識した先行車両とを比較する際、ステレオカメラ４で先行車両を認識する場合に発生する距離のばらつき（距離誤差及び外環境変化により発生する誤差）と、ステレオカメラ４とレーザ投光ユニット５との取り付け位置関係で発生する距離のばらつきとを反映して比較することで、より正確な比較が行えるようになっている。
【００４８】
更に、本実施の形態は、ステレオカメラ４で認識した先行車両とレーザレーダで認識した先行車両とを比較して、これら先行車両が存在する水平方向位置は略等しく、且つ、これら先行車両までの自車両からの距離が略等しくない場合は、これら先行車両の速度を比較して先行車速が一致する場合に、これらの先行車両は同一の先行車両であると判断する。このため、特に、路肩の停止車両の脇をすり抜ける際、追従している先行車両と停止車両の位置が近接している場合、先行車両を誤認識する場合があるが、このように先行車両の速度で比較することにより、誤認識が確実に防止される。
【００４９】
また、本実施の形態によれば、今回のＩＤ番号と前回のＩＤ番号とを比較して先行車両の一致を判断するので、たとえ、今回、先行車両の特定が困難な状況となっても、前回までのＩＤ番号の情報を利用して、正確に先行車両を特定できる。
【００５０】
尚、本実施の形態では、レーダ情報として、レーザレーダを用いた例で説明しているが、ミリ波レーダを用いた場合も略同様に実現できる。この場合、ミリ波レーダでは、オーバーパスによる先行車両の誤検出の問題があるが、ステレオカメラ４により認識した先行車両を基準として用いることにより、ミリ波レーダが間違った先行車両を認識していることを検出することが可能になる。
【００５１】
また、本実施の形態では、水平方向位置比較と距離比較、速度比較を行った後、条件に一致する対象物がまだ複数存在する場面において、選択した先行車情報をＡＣＣシステム２に用いる場合、どの対象物の情報を使用してもＡＣＣ制御には影響を及ぼさないことより、どの対象物を先行車両として選択しても構わないものとする。
【００５２】
更に、先行車両のＩＤ番号を出力情報として持っている場合で、ステレオカメラ４が外環境の急激な明るさの変化等で、一時的（１〜２秒程度）に先行車両を見失ってしまう場面では、それまでのステレオカメラ４とレーザレーダから判断された先行車情報のＩＤ情報を記憶させておき、ステレオカメラ４からの情報が得られなくなった時点で、レーザレーダの情報の中から、記憶させておいたＩＤ番号と同一のＩＤ番号の先行車両を選択させ、一時的（１〜２秒程度）に補間させても良い。
【００５３】
【発明の効果】
以上説明したように本発明によれば、ステレオカメラからの画像情報とレーザやミリ波等のレーダ情報とを用いて、目標とする先行車両を正確に検出することが可能となる。
【図面の簡単な説明】
【図１】先行車両検出装置を有する車間距離自動維持運転システムの概略構成図
【図２】先行車両検出手順を示すフローチャート
【図３】ステレオカメラとレーザレーダの取り付け位置の違いと障害物検出の水平方向位置の違いの説明図
【図４】ステレオカメラとレーザレーダの取り付け位置の詳細説明図
【図５】先行車両に対するステレオカメラで検出する領域とレーザレーダで検出する領域の説明図
【図６】ステレオカメラによる計測距離と理論誤差の説明図
【符号の説明】
１ 自車両
２ ＡＣＣシステム
３ 走行制御ユニット
４ ステレオカメラ（画像先行車両認識手段）
５ レーザ投光ユニット（レーダ先行車両認識手段）
６ 先行車両検出装置（画像先行車両認識手段、レーダ先行車両認識手段、先行車両判断手段）[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a preceding vehicle detection device that recognizes a preceding vehicle existing ahead based on image information from a stereo camera and radar information such as laser and millimeter waves.
[0002]
[Prior art]
Conventionally, as an inter-vehicle distance detection device, an inter-vehicle distance automatic maintenance operation system (Adaptive Cruise Control) using a laser radar or a millimeter wave radar has been put to practical use. However, in an inter-vehicle detection device based on radar information such as a laser radar or a millimeter-wave radar, a traveling route estimating method does not attempt to visually grasp the environment in front of the vehicle, and erroneous recognition or loss of a preceding vehicle often occurs.
[0003]
On the other hand, in Japanese Patent Application Laid-Open No. 8-329393, a camera is added, and if the vehicle detected by the laser radar overlaps the existing area (only in the horizontal direction) of the preceding vehicle detected by the camera, the vehicle is determined to be the preceding vehicle. It proposes a method for reliably detecting the preceding vehicle by using the method.
[0004]
[Patent Document 1]
JP-A-8-329393
[0005]
[Problems to be solved by the invention]
However, in the above-described prior art, only the horizontal position (horizontal position) of the output from the camera and the output from the laser radar is compared, and the three-dimensional object detection by the camera is not performed. The result is really a three-dimensional object, and it is uncertain whether it is a preceding vehicle. For example, in the case of a monocular camera, if dust adheres to a portion of the lens surface corresponding to the traveling area of the preceding vehicle, the camera may erroneously recognize the preceding vehicle. Since only the lateral position comparison is performed, there is a possibility that the information of the laser radar present in the erroneously recognized direction may be recognized as the preceding vehicle.
[0006]
When a millimeter wave radar is used instead of a laser radar, the millimeter wave radar may detect a preceding vehicle (preceding vehicle existing ahead of the preceding vehicle) due to an overpass. At this time, since the lateral positions of the preceding vehicle and the preceding vehicle substantially overlap, the preceding vehicle detection device according to the related art may detect the preceding vehicle as the preceding vehicle and cause erroneous ACC control.
[0007]
The present invention has been made in view of the above circumstances, and detects a preceding vehicle that can accurately detect a target preceding vehicle by using image information from a stereo camera and radar information such as laser and millimeter waves. It is intended to provide a device.
[0008]
[Means for Solving the Problems]
In order to achieve the above object, a preceding vehicle detection device according to the present invention detects distance data between a three-dimensional object present ahead and a host vehicle based on image information ahead of the vehicle and recognizes the preceding vehicle. Preceding vehicle recognizing means, radar preceding vehicle recognizing means for detecting distance data between a three-dimensional object present ahead and the host vehicle based on radar information ahead of the vehicle and recognizing the preceding vehicle, and recognizing by the image preceding vehicle recognizing means. Comparing the preceding vehicle and the preceding vehicle recognized by the radar preceding vehicle recognizing means, the positions of the preceding vehicles in the horizontal direction are substantially equal, and the distances to the preceding vehicles from the own vehicle are substantially equal. In this case, the preceding vehicle is provided with a preceding vehicle determining means for determining that the preceding vehicle is the same preceding vehicle.
[0009]
According to a second aspect of the present invention, there is provided a preceding vehicle detecting apparatus according to the first aspect, wherein the preceding vehicle determining means includes the preceding vehicle recognized by the image preceding vehicle recognizing means and the radar preceding vehicle. When comparing the preceding vehicle recognized by the recognizing means, when the image preceding vehicle recognizing means recognizes the preceding vehicle, the variation in distance that occurs and the difference between the image preceding vehicle recognizing means and the radar preceding vehicle recognizing means. It is characterized in that at least one of the corrections and the variation in the distance generated due to the mounting positional relationship is corrected and compared.
[0010]
Further, according to a third aspect of the present invention, there is provided a preceding vehicle detecting apparatus according to the first or second aspect, wherein the preceding vehicle determining means includes the preceding vehicle recognized by the image preceding vehicle recognizing means. And the preceding vehicle recognized by the radar preceding vehicle recognizing means, the horizontal position where these preceding vehicles exist is substantially equal, and the distance from the own vehicle to these preceding vehicles is not substantially equal. The speeds of these preceding vehicles are compared, and when the speeds of the preceding vehicles match, the preceding vehicles are determined to be the same preceding vehicle.
[0011]
According to a fourth aspect of the present invention, there is provided a preceding vehicle detecting apparatus according to any one of the first to third aspects, wherein the image preceding vehicle recognizing means and the radar preceding vehicle recognizing means are provided. Is to set an identification number for each detected three-dimensional object, the preceding vehicle determining means, the preceding vehicle recognized by the image preceding vehicle recognition means and the preceding vehicle recognized by the radar preceding vehicle recognition means If the preceding vehicles are located at substantially the same horizontal position and the distances from the host vehicle to the preceding vehicles are not substantially equal, the identification numbers of these preceding vehicles are the same as those of the preceding vehicle. When the identification number of the preceding vehicle determined to be a vehicle matches, the preceding vehicle is determined to be the same preceding vehicle.
[0012]
That is, the preceding vehicle detection device according to claim 1 detects distance data between a three-dimensional object existing ahead and the own vehicle based on image information ahead of the vehicle by image preceding vehicle recognition means, and recognizes the preceding vehicle. Based on radar information ahead of the vehicle, radar preceding vehicle recognition means detects distance data between the three-dimensional object existing ahead and the own vehicle to recognize the preceding vehicle. Then, the preceding vehicle determining means compares the preceding vehicle recognized by the image preceding vehicle recognizing means with the preceding vehicle recognized by the radar preceding vehicle recognizing means, and the horizontal positions at which these preceding vehicles exist are substantially equal, and When the distances from these vehicles to the preceding vehicles are substantially equal, it is determined that these preceding vehicles are the same preceding vehicle. In this way, the preceding vehicle of the three-dimensional object detected by the image preceding vehicle recognition means and the preceding vehicle of the three-dimensional object detected by the radar preceding vehicle recognition means are compared by the horizontal position and the distance, and the same preceding vehicle is specified. Therefore, it is possible to accurately detect the target preceding vehicle.
[0013]
According to a second aspect of the present invention, there is provided a preceding vehicle detecting apparatus according to the first aspect, wherein the preceding vehicle determining means includes a preceding vehicle recognized by the image preceding vehicle recognizing means and a radar preceding vehicle recognizing means. When comparing with the preceding vehicle recognized in the above, it occurs due to the variation in the distance that occurs when the image preceding vehicle recognition means recognizes the preceding vehicle and the mounting positional relationship between the image preceding vehicle recognition means and the radar preceding vehicle recognition means. A more accurate comparison can be made by performing at least one of the corrections and the comparison with the variation in the distance.
[0014]
Further, the preceding vehicle detecting device according to the present invention according to claim 3 is the preceding vehicle detecting device according to claim 1 or 2, wherein the preceding vehicle determining means is configured to determine the preceding vehicle recognized by the image preceding vehicle recognizing means. Compared with the preceding vehicle recognized by the radar preceding vehicle recognizing means, if the horizontal position where these preceding vehicles exist is substantially equal and the distance from the own vehicle to these preceding vehicles is not substantially equal, the preceding If the vehicle speeds are compared and the preceding vehicle speeds match, it is determined that these preceding vehicles are the same preceding vehicle. In particular, when passing by the side of a stopped vehicle on the shoulder of the road, if the position of the following vehicle and the stopped vehicle is close to each other, the preceding vehicle may be erroneously recognized. By doing so, erroneous recognition is prevented.
[0015]
According to a fourth aspect of the present invention, there is provided a preceding vehicle detecting apparatus according to any one of the first to third aspects, wherein the image preceding vehicle recognizing means and the radar preceding vehicle recognizing means are provided. The identification number is set for each detected three-dimensional object, and the preceding vehicle determining means compares the preceding vehicle recognized by the image preceding vehicle recognition means with the preceding vehicle recognized by the radar preceding vehicle recognition means, When the horizontal positions where these preceding vehicles exist are substantially equal, and the distances from the own vehicle to these preceding vehicles are not substantially equal, it is determined that the identification numbers of these preceding vehicles are the same preceding vehicle in the last time. If the identification numbers of the preceding vehicles match, it is determined that these preceding vehicles are the same preceding vehicle. Even if it is difficult to identify the preceding vehicle this time, the preceding vehicle can be identified using the information up to the previous time.
[0016]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
1 to 6 show an embodiment of the present invention. FIG. 1 is a schematic configuration diagram of an inter-vehicle distance automatic maintenance driving system having a preceding vehicle detecting device, FIG. 2 is a flowchart showing a preceding vehicle detecting procedure, and FIG. FIG. 4 is an explanatory diagram of a difference between a mounting position of a camera and a laser radar and a difference of a horizontal position of obstacle detection, FIG. 4 is a detailed explanatory diagram of a mounting position of a stereo camera and a laser radar, and FIG. FIG. 6 is an explanatory diagram of an area and an area detected by a laser radar. FIG. 6 is an explanatory view of a distance measured by a stereo camera and a theoretical error.
[0017]
In FIG. 1, reference numeral 1 denotes a vehicle (own vehicle) such as an automobile, and the vehicle 1 has an ACC system 2 mounted thereon. The ACC system 2 mainly includes a travel control unit 3, a stereo camera 4, a laser projection unit 5, and a preceding vehicle detection device 6, and holds a vehicle speed set by a driver in a constant-speed travel control state. The vehicle travels in the following state, and in the following traveling control state, the target vehicle speed is set to the vehicle speed of the preceding vehicle, and the vehicle travels while maintaining a constant inter-vehicle distance with respect to the preceding vehicle.
[0018]
The stereo camera 4 is composed of a set of (left and right) CCD cameras using a solid-state imaging device such as a charge-coupled device (CCD) as a stereo optical system. , And a stereo image of an object outside the vehicle from different viewpoints is input to the preceding vehicle detection device 6.
[0019]
The laser projection unit 5 is provided above the bumper structural member of the vehicle 1 and includes a laser head having a function of projecting / receiving a laser beam and scanning in the left-right direction. Is connected to Then, a laser beam is projected horizontally from the laser projecting unit 5 to detect only a three-dimensional object located at a position higher than the road surface, and the laser beam from the laser projecting unit 5 is scanned in the left-right direction. The operation of projecting and receiving a laser beam at regular intervals in a predetermined scanning range and detecting the distance is repeated, and the two-dimensional distribution of the three-dimensional object is measured.
[0020]
The host vehicle 1 is provided with a vehicle speed sensor 7 for detecting a vehicle speed. The vehicle speed is input to the travel control unit 3 and the preceding vehicle detection device 6.
[0021]
The preceding vehicle detection device 6 receives the image from the stereo camera 4, the scanning result by the laser projection unit 5, and the vehicle speed, and inputs the distance from the three-dimensional object ahead based on the image from the stereo camera 4, the preceding vehicle, and the laser projection. The distance to the three-dimensional object ahead and the preceding vehicle based on the scanning result by the optical unit 5 are obtained independently, and these preceding vehicles are integrated according to a flowchart showing a preceding vehicle detection procedure described later, and finally the preceding vehicle is determined. , To the travel control unit 3.
[0022]
Here, the processing of the image from the stereo camera 4 in the preceding vehicle detection device 6 is performed, for example, as follows. First, a pair of left and right stereo images captured by the CCD camera of the stereo camera 4 is processed to calculate a three-dimensional distance distribution over the entire image. Detects dimensional position at high speed. Then, by comparing the detected road shape with the position of each three-dimensional object, the preceding vehicle immediately before the own vehicle on the traveling road is specified, the inter-vehicle distance between the own vehicle and the preceding vehicle is obtained, and the change in the inter-vehicle distance and the vehicle speed are determined. The preceding vehicle speed is calculated based on the own vehicle speed detected by the sensor 7. As described above, the stereo camera 4 and the preceding vehicle detection device 6 detect distance data between the three-dimensional object present ahead and the host vehicle based on the image information ahead of the vehicle and serve as image preceding vehicle recognition means for recognizing the preceding vehicle. Has a function. Each of the three-dimensional objects detected based on the image from the stereo camera 4 includes a predetermined rule (for example, in order from a three-dimensional object on the traveling road of the host vehicle 1 to a three-dimensional object on another traveling road in order. An identification number (ID number) is assigned based on the numbering of the identification numbers.
[0023]
The detection of the three-dimensional object and the preceding vehicle by the laser radar in the preceding vehicle detection device 6 is performed, for example, by calculating the distance to each three-dimensional object from the time required for emitting and receiving the laser beam by the laser projection unit 5, Then, the two-dimensional position of the object is calculated from the scanning direction of the laser beam and stored as laser distance data. Then, based on the change in the inter-vehicle distance with each three-dimensional object and the own vehicle speed detected by the vehicle speed sensor 7, the preceding vehicle existing ahead of the own vehicle 1 is specified from among the three-dimensional objects. As described above, the laser projecting unit 5 and the preceding vehicle detecting device 6 detect the distance data between the three-dimensional object present ahead and the own vehicle based on the radar information ahead of the vehicle and recognize the preceding vehicle by the radar preceding vehicle recognizing means. As a function. In addition, like the three-dimensional objects detected based on the image from the stereo camera 4, each of the three-dimensional objects detected by the laser projecting unit 5 includes a predetermined rule (for example, a nearby rule on the traveling path of the vehicle 1). Are sequentially numbered from the three-dimensional object to the three-dimensional object on another traveling road so as to increase in order.).
[0024]
The traveling control unit 3 performs a constant speed traveling control function of performing a constant speed traveling control so as to maintain a traveling speed set by a driver's operation input, and a state in which a distance between the host vehicle and the preceding vehicle is kept constant. A constant-speed traveling switch 8 comprising a plurality of switches connected to a constant-speed traveling operation lever provided on a side portion of a steering column or the like, which realizes a function of following and traveling traveling control, and a preceding vehicle detection device. 6, a vehicle speed sensor 7 and the like are connected. The constant speed traveling switch 8 is a vehicle speed set switch for setting a target vehicle speed during constant speed traveling, a coast switch for mainly changing and setting the target vehicle speed to the down side, a resume switch for mainly changing and setting the target vehicle speed to the up side, and the like. It is configured. Further, a main switch (not shown) for turning ON / OFF the traveling control is disposed near the constant-speed traveling operation lever.
[0025]
When the driver turns on a main switch (not shown) and sets a desired speed with a constant speed traveling operation lever, a signal from the constant speed traveling switch 8 is input to the traveling control unit 3 and the vehicle speed detected by the vehicle speed sensor 7 is reduced. Then, the throttle actuator 9 is driven to feedback-control the opening of the throttle valve 10 so that the vehicle speed converges to the set vehicle speed set by the driver, and the host vehicle automatically runs at a constant speed.
[0026]
Further, the traveling control unit 3 recognizes the preceding vehicle by the preceding vehicle detection device 6 during the constant speed traveling control, and when the preceding vehicle speed is equal to or less than the target speed set by the own vehicle, The control is automatically switched to the follow-up traveling control in which the vehicle travels while maintaining a constant inter-vehicle distance with respect to the preceding vehicle.
[0027]
When the traveling control of the vehicle shifts to the following traveling control, an appropriate inter-vehicle distance is determined based on the inter-vehicle distance and the preceding vehicle speed between the own vehicle and the preceding vehicle determined by the preceding vehicle detection device 6 and the own vehicle speed detected by the vehicle speed sensor 7. A state in which a target value is set, a drive signal is output to the throttle actuator 9 so that the inter-vehicle distance becomes the target value, the opening of the throttle valve 10 is feedback-controlled, and a constant inter-vehicle distance is maintained with respect to the preceding vehicle. To follow.
[0028]
Next, a preceding vehicle detection procedure in the preceding vehicle detection device 6 will be described with reference to a flowchart shown in FIG. The routine shown in FIG. 2 is started every predetermined time (for example, 50 msec). First, in step (hereinafter abbreviated as “S”) 101, each sensor such as the stereo camera 4 and the laser projection unit 5 is checked. If any of these sensors is abnormal, the program ends.
[0029]
On the other hand, if it is confirmed in S101 that each sensor is normal, the process proceeds to S102, and various information obtained based on the image from the stereo camera 4, that is, data on the preceding vehicle ahead of the own vehicle 1 (the own vehicle and the The distance between the vehicle and the preceding vehicle, the preceding vehicle speed, the ID number of each preceding three-dimensional object, and the ID number of the preceding vehicle are read.
[0030]
Next, in S103, various information obtained by the laser radar, that is, data on the preceding vehicle ahead of the own vehicle 1 (the distance between the own vehicle and the preceding vehicle, the preceding vehicle speed, the ID number of each preceding three-dimensional object and The ID number of the preceding vehicle is read.
[0031]
Then, the process proceeds to S104, in which the angle is corrected based on the difference between the mounting positions of the stereo camera 4 and the laser projection unit 5. That is, as shown in FIG. 3, when the same part of the same preceding vehicle is detected by the stereo camera 4 and the laser projection unit 5 when the mounting positions thereof are different, this part is at an angle a in the stereo camera 4. However, it is detected at the angle b in the laser projection unit 5.
[0032]
Therefore, as shown in FIG. 4, if the laser projection unit 5 is arranged at a position of 1900 mm in front of the stereo camera 4 with a leftward distance of 170 mm and both faces forward, there is a difference in the mounting position. The resulting theoretical preceding vehicle angle difference is calculated by the following equation (1). The preceding vehicle distance by the laser projection unit 5 is Lb, the preceding vehicle angle by the laser projection unit 5 is θb, the preceding vehicle angle of the stereo camera 4 calculated by the theoretical formula is θa, and the laser projection unit 5 and the stereo camera 4 X (= 170 mm) in the vehicle width direction and Y (= 1900 mm) in the front-rear direction between the laser projection unit 5 and the stereo camera 4.
θa = tan -1 ((L · sin θ−X) / L · cos θ) (1)
here,
θ = tan -1 (Lb · sin θ / (Lb · cos θb + Y))
L = (Lb · cos θb + Y) / cos θ
It is.
[0033]
Next, in S105, it is determined whether or not the horizontal position of the preceding vehicle by the stereo camera 4 and the laser radar is the same.
[0034]
Specifically, as shown in FIG. 5, the left edge angle information of the preceding vehicle detected in the image from the stereo camera 4 is extracted in the SL area, and the right edge angle information is extracted in the SR area. Therefore, it is determined whether there is horizontal angle information of the preceding vehicle detected by the laser radar during Ls between SL and SR.
[0035]
As a result of the determination in S105, when there is no horizontal angle information of the preceding vehicle detected by the laser radar during Ls between SL and SR, that is, when the level of the preceding vehicle is determined by the stereo camera 4 and the laser radar. If the direction positions do not match, the process proceeds to S112, where it is determined that the preceding vehicle by the stereo camera 4 and the preceding vehicle by the laser radar do not match, and the program exits.
[0036]
Conversely, as a result of the determination in S105, if there is horizontal angle information of the preceding vehicle detected by the laser radar during Ls between SL and SR, that is, the preceding vehicle by the stereo camera 4 and the laser radar If the horizontal positions of the vehicles match, the process proceeds to S106.
[0037]
In S106, the theoretical distance variation of the distance of the stereo camera 4 is calculated, and is reflected in advance on the value of the inter-vehicle distance between the own vehicle and the preceding vehicle obtained based on the image from the stereo camera 4. For example, a theoretical distance error (distance error at the distance D) ΔD generated by the specifications of the stereo camera 4 is calculated by the following equation (2). The focal length is F, the horizontal and vertical lengths of the CCD are both dx, the camera interval of the stereo camera 4 is L, the number of scanning lines is M, the height of the object is H, and the number of 4 × 4 pixel blocks reflected as a recognized object is N. As
ΔD = (D · D · dx) / ((F · LD−dx) · N 1/2 )… (2)
here,
N = ((M / 4) .F.H.2) / (dx.D)
Thus, the theoretical distance error ΔD obtained by using the above equation (2) is obtained as shown in FIG.
[0038]
After calculating the theoretical distance variation in S106 and reflecting it on the value of the inter-vehicle distance between the own vehicle and the preceding vehicle obtained based on the image from the stereo camera 4, the process proceeds to S107, where the distance of the stereo camera 4 is calculated. The variation due to the change in the external environment is calculated, and the calculated variation is reflected in advance on the value of the inter-vehicle distance between the own vehicle and the preceding vehicle obtained based on the image from the stereo camera 4. The variation due to the change in the external environment is an error generated due to, for example, a sudden change in luminance, and is a value set in advance by an experiment or the like.
[0039]
Thereafter, the process proceeds to S108, and it is determined whether or not the distances at which the preceding vehicle exists between the stereo camera 4 and the laser radar coincide.
[0040]
As a result of the determination in S108, if the distances at which the preceding vehicle exists with the stereo camera 4 and the laser radar match, the process proceeds to S111, and the preceding vehicle determined by the stereo camera 4 and the laser radar is the same vehicle. And exit the program.
[0041]
Conversely, when the distances at which the preceding vehicle exists between the stereo camera 4 and the laser radar do not match, the process proceeds to S109, and it is determined whether or not the speeds of the preceding vehicle using the stereo camera 4 and the laser radar match.
[0042]
As a result of the determination in S109, if the speeds of the preceding vehicle by the stereo camera 4 and the laser radar match, the process proceeds to S111, and it is determined that the preceding vehicle by the stereo camera 4 and the laser radar is the same vehicle and the detection of the preceding vehicle is performed. Finish and exit the program.
[0043]
Conversely, if the speeds of the preceding vehicle by the stereo camera 4 and the laser radar do not match, the process proceeds to S110, and the ID number of the preceding vehicle by the stereo camera 4 and the ID number of the preceding vehicle by the laser radar match the previously set ID number. It is determined whether or not the process is to be performed. If they match, the process proceeds to S111, where it is determined that the preceding vehicle by the stereo camera 4 and the laser radar is the same vehicle, detection of the preceding vehicle is completed, and the program exits.
[0044]
Conversely, if the ID number of the preceding vehicle by the stereo camera 4 and the ID number of the preceding vehicle by the laser radar do not match the previously set ID number of the preceding vehicle, the process proceeds to S112, and the process proceeds to S112. The preceding vehicle is determined to be inconsistent and exits the program. In this case, the preceding vehicle that matches the ID number of the preceding vehicle set last time is set as the current preceding vehicle.
[0045]
Thus, as is clear from the preceding vehicle detection procedure described above, the preceding vehicle detection device 6 has a function as a preceding vehicle determination unit.
[0046]
As described above, according to the present embodiment, the preceding vehicle detected by the stereo camera 4 and the preceding vehicle detected by the laser radar are compared with each other by the horizontal position and the distance to specify the same preceding vehicle. It is possible to accurately detect the preceding vehicle that is running.
[0047]
Further, in the present embodiment, when comparing the preceding vehicle recognized by the stereo camera 4 with the preceding vehicle recognized by the laser radar, the dispersion of the distance (distance error and distance) occurring when the stereo camera 4 recognizes the preceding vehicle is compared. More accurate comparison can be performed by making a comparison by reflecting an error caused by a change in the external environment) and a variation in the distance caused by the positional relationship between the stereo camera 4 and the laser projection unit 5. I have.
[0048]
Further, in the present embodiment, the preceding vehicle recognized by the stereo camera 4 is compared with the preceding vehicle recognized by the laser radar. When the distances from the own vehicle are not substantially equal, the speeds of the preceding vehicles are compared, and if the preceding vehicle speeds match, it is determined that these preceding vehicles are the same preceding vehicle. For this reason, especially when passing the side of the stopped vehicle on the road shoulder, if the position of the following vehicle and the stopped vehicle is close to each other, the preceding vehicle may be erroneously recognized. By comparing the speeds, erroneous recognition is reliably prevented.
[0049]
Further, according to the present embodiment, since the current ID number is compared with the previous ID number to determine the match of the preceding vehicle, even if it is difficult to identify the preceding vehicle this time, The preceding vehicle can be accurately specified using the information of the ID number up to the previous time.
[0050]
Although the present embodiment has been described using an example in which a laser radar is used as the radar information, a case where a millimeter-wave radar is used can be realized in substantially the same manner. In this case, the millimeter-wave radar has a problem of erroneous detection of a preceding vehicle due to an overpass, but the millimeter-wave radar recognizes an incorrect preceding vehicle by using the preceding vehicle recognized by the stereo camera 4 as a reference. Can be detected.
[0051]
Further, in the present embodiment, in the case where the selected preceding vehicle information is used for the ACC system 2 in a scene in which a plurality of objects matching the conditions still exist after performing the horizontal position comparison, the distance comparison, and the speed comparison, Since the ACC control is not affected by using the information of any object, any object may be selected as the preceding vehicle.
[0052]
Furthermore, when the ID number of the preceding vehicle is provided as output information, the stereo camera 4 may temporarily (about 1 to 2 seconds) lose sight of the preceding vehicle due to a sudden change in brightness of the external environment. Then, the ID information of the preceding vehicle information determined by the stereo camera 4 and the laser radar up to that time is stored, and when the information from the stereo camera 4 cannot be obtained, the information of the laser radar is stored. A preceding vehicle having the same ID number as the previously set ID number may be selected and interpolated temporarily (about 1 to 2 seconds).
[0053]
【The invention's effect】
As described above, according to the present invention, it is possible to accurately detect a target preceding vehicle by using image information from a stereo camera and radar information such as laser and millimeter waves.
[Brief description of the drawings]
FIG. 1 is a schematic configuration diagram of an inter-vehicle distance automatic maintenance driving system having a preceding vehicle detection device.
FIG. 2 is a flowchart showing a preceding vehicle detection procedure;
FIG. 3 is an explanatory diagram of a difference between a mounting position of a stereo camera and a laser radar and a difference of a horizontal position of obstacle detection.
FIG. 4 is a detailed explanatory view of a mounting position of a stereo camera and a laser radar.
FIG. 5 is an explanatory diagram of an area detected by a stereo camera and an area detected by a laser radar for a preceding vehicle.
FIG. 6 is an explanatory diagram of a distance measured by a stereo camera and a theoretical error.
[Explanation of symbols]
1 own vehicle
2 ACC system
3 Travel control unit
4 Stereo camera (image preceding vehicle recognition means)
5 Laser projection unit (Radar preceding vehicle recognition means)
6 preceding vehicle detection device (image preceding vehicle recognition means, radar preceding vehicle recognition means, preceding vehicle determination means)

Claims (4)

車両前方の画像情報を基に前方に存在する立体物と自車両との距離データを検出し先行車両を認識する画像先行車両認識手段と、
車両前方のレーダ情報を基に前方に存在する立体物と自車両との距離データを検出し先行車両を認識するレーダ先行車両認識手段と、
上記画像先行車両認識手段で認識した先行車両と上記レーダ先行車両認識手段で認識した先行車両とを比較して、これら先行車両が存在する水平方向位置が略等しく、且つ、これら先行車両までの上記自車両からの距離が略等しい場合、これら先行車両は同一の先行車両であると判断する先行車両判断手段と、
を備えたことを特徴とする先行車両検出装置。Image preceding vehicle recognizing means for detecting distance data between the three-dimensional object present ahead and the own vehicle based on image information ahead of the vehicle and recognizing the preceding vehicle;
Radar preceding vehicle recognizing means for detecting distance data between a three-dimensional object present ahead and the host vehicle based on radar information ahead of the vehicle and recognizing the preceding vehicle;
By comparing the preceding vehicle recognized by the image preceding vehicle recognizing means with the preceding vehicle recognized by the radar preceding vehicle recognizing means, the positions of the preceding vehicles in the horizontal direction are substantially equal, and Preceding vehicle determining means for determining that these preceding vehicles are the same preceding vehicle when the distances from the own vehicle are substantially equal;
A preceding vehicle detection device comprising: 上記先行車両判断手段は、上記画像先行車両認識手段で認識した先行車両と上記レーダ先行車両認識手段で認識した先行車両とを比較する際、上記画像先行車両認識手段で上記先行車両を認識する場合に発生する距離のばらつきと、上記画像先行車両認識手段と上記レーダ先行車両認識手段との取り付け位置関係で発生する距離のばらつきとの少なくともどちらかの補正を行い比較することを特徴とする請求項１記載の先行車両検出装置。When the preceding vehicle determining means compares the preceding vehicle recognized by the image preceding vehicle recognizing means with the preceding vehicle recognized by the radar preceding vehicle recognizing means, the preceding vehicle recognizing means recognizes the preceding vehicle. Wherein at least one of a variation in a distance occurring in the image and a variation in a distance occurring due to an attachment positional relationship between the image preceding vehicle recognizing means and the radar preceding vehicle recognizing means is corrected and compared. 2. The preceding vehicle detection device according to claim 1. 上記先行車両判断手段は、上記画像先行車両認識手段で認識した先行車両と上記レーダ先行車両認識手段で認識した先行車両とを比較して、これら先行車両が存在する水平方向位置は略等しく、且つ、これら先行車両までの上記自車両からの距離が略等しくない場合は、これら先行車両の速度を比較して該先行車両速度が一致する場合は、これらの先行車両は同一の先行車両であると判断することを特徴とする請求項１又は請求項２に記載の先行車両検出装置。The preceding vehicle determining means compares the preceding vehicle recognized by the image preceding vehicle recognizing means with the preceding vehicle recognized by the radar preceding vehicle recognizing means, and the horizontal positions at which these preceding vehicles exist are substantially equal, and If the distances from the host vehicle to the preceding vehicles are not substantially equal, the speeds of the preceding vehicles are compared, and if the preceding vehicle speeds match, it is determined that these preceding vehicles are the same preceding vehicle. The preceding vehicle detection device according to claim 1 or 2, wherein the determination is made. 上記画像先行車両認識手段と上記レーダ先行車両認識手段は、検出した上記立体物毎に識別番号を設定するものであって、
上記先行車両判断手段は、上記画像先行車両認識手段で認識した先行車両と上記レーダ先行車両認識手段で認識した先行車両とを比較して、これら先行車両が存在する水平方向位置は略等しく、且つ、これら先行車両までの上記自車両からの距離が略等しくない場合は、これら先行車両の認識番号が前回に同一の先行車両であると判断した先行車両の認識番号と一致する場合は、これら先行車両は同一の先行車両であると判断することを特徴とする請求項１乃至請求項３の何れか一つに記載の先行車両検出装置。The image preceding vehicle recognizing means and the radar preceding vehicle recognizing means set an identification number for each of the detected three-dimensional objects,
The preceding vehicle determining means compares the preceding vehicle recognized by the image preceding vehicle recognizing means with the preceding vehicle recognized by the radar preceding vehicle recognizing means, and the horizontal positions at which these preceding vehicles exist are substantially equal, and If the distance to the preceding vehicle from the host vehicle is not substantially equal, if the identification number of the preceding vehicle matches the identification number of the preceding vehicle that was previously determined to be the same preceding vehicle, The preceding vehicle detection device according to any one of claims 1 to 3, wherein the vehicles are determined to be the same preceding vehicle.
JP2003145235A 2003-05-22 2003-05-22 Leading vehicle detection device Expired - Fee Related JP4265931B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2003145235A JP4265931B2 (en) 2003-05-22 2003-05-22 Leading vehicle detection device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2003145235A JP4265931B2 (en) 2003-05-22 2003-05-22 Leading vehicle detection device

Publications (2)

Publication Number Publication Date
JP2004347471A true JP2004347471A (en) 2004-12-09
JP4265931B2 JP4265931B2 (en) 2009-05-20

Family

ID=33532470

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2003145235A Expired - Fee Related JP4265931B2 (en) 2003-05-22 2003-05-22 Leading vehicle detection device

Country Status (1)

Country Link
JP (1) JP4265931B2 (en)

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006240454A (en) * 2005-03-02 2006-09-14 Daihatsu Motor Co Ltd Object recognition device and object recognition method for vehicle
JP2006293835A (en) * 2005-04-13 2006-10-26 Toyota Motor Corp Object detection apparatus
JP2007226680A (en) * 2006-02-24 2007-09-06 Toyota Motor Corp Object detection system
JP2010501952A (en) * 2007-04-19 2010-01-21 ローベルト ボッシュ ゲゼルシャフト ミット ベシュレンクテル ハフツング Driver assistance system and method for determining the validity of an object
JP2011520704A (en) * 2008-05-23 2011-07-21 ヴアブコ・ゲゼルシヤフト・ミツト・ベシユレンクテル・ハフツング Driver assistance system
DE102015207206A1 (en) 2014-04-25 2015-10-29 Honda Motor Co., Ltd. OBJECT DETECTION DEVICE
US20150324652A1 (en) * 2014-05-09 2015-11-12 Honda Motor Co., Ltd. Object recognition apparatus
EP2927710A3 (en) * 2014-03-13 2015-12-30 Ricoh Company, Ltd. Ranging system, information processing apparatus, information processing method and program thereof
WO2016047460A1 (en) * 2014-09-24 2016-03-31 株式会社デンソー Object detection device
US9836964B2 (en) 2011-07-26 2017-12-05 Toyota Jidosha Kabushiki Kaisha Vehicle identification system and vehicle identification device
KR20180018767A (en) 2015-07-27 2018-02-21 닛산 지도우샤 가부시키가이샤 Object detection method and object detection apparatus
WO2019039279A1 (en) * 2017-08-22 2019-02-28 ソニー株式会社 Signal processing device, signal processing method, program, moving body, and signal processing system
WO2019239775A1 (en) * 2018-06-15 2019-12-19 日立オートモティブシステムズ株式会社 Vehicle object sensing device
US11391840B2 (en) 2018-06-25 2022-07-19 Ricoh Company, Ltd. Distance-measuring apparatus, mobile object, distance-measuring method, and distance measuring system

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6476831B2 (en) 2013-12-26 2019-03-06 株式会社リコー Parallax calculation system, information processing apparatus, information processing method, and program
JP6379646B2 (en) 2014-05-09 2018-08-29 株式会社リコー Information processing apparatus, measurement method, and program
JP6716857B2 (en) 2014-12-26 2020-07-01 株式会社リコー Mobile object, measuring system, measuring method and program
EP3514759A1 (en) 2018-01-23 2019-07-24 Ricoh Company, Ltd. Image processing method, image processing apparatus, on-board device, moving body, and system

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06230115A (en) * 1993-02-01 1994-08-19 Toyota Motor Corp Vehicular gap detector
JPH08271247A (en) * 1995-03-30 1996-10-18 Mitsubishi Electric Corp Device for estimating alignment error of sensor
JPH0991595A (en) * 1995-09-25 1997-04-04 Mitsubishi Motors Corp Obstacle recognition device
JP2000131432A (en) * 1998-10-23 2000-05-12 Honda Motor Co Ltd Method for detecting obstruction to vehicle
JP2000329852A (en) * 1999-05-17 2000-11-30 Nissan Motor Co Ltd Obstacle recognition device
JP2002099906A (en) * 2000-09-22 2002-04-05 Mazda Motor Corp Object-recognizing device
JP2003121547A (en) * 2001-10-18 2003-04-23 Fuji Heavy Ind Ltd Outside-of-vehicle monitoring apparatus
JP2003139858A (en) * 2001-11-02 2003-05-14 Fuji Heavy Ind Ltd Outside-vehicle monitoring apparatus
JP2003151094A (en) * 2001-11-15 2003-05-23 Fuji Heavy Ind Ltd Device for monitoring outside of vehicle
JP2003168197A (en) * 2001-11-30 2003-06-13 Hitachi Ltd Method and device for recognizing traveling circumstance
JP2004037239A (en) * 2002-07-03 2004-02-05 Fuji Heavy Ind Ltd Identical object judging method and system, and misregistration correcting method and system
JP2004117071A (en) * 2002-09-24 2004-04-15 Fuji Heavy Ind Ltd Vehicle surroundings monitoring apparatus and traveling control system incorporating the same

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06230115A (en) * 1993-02-01 1994-08-19 Toyota Motor Corp Vehicular gap detector
JPH08271247A (en) * 1995-03-30 1996-10-18 Mitsubishi Electric Corp Device for estimating alignment error of sensor
JPH0991595A (en) * 1995-09-25 1997-04-04 Mitsubishi Motors Corp Obstacle recognition device
JP2000131432A (en) * 1998-10-23 2000-05-12 Honda Motor Co Ltd Method for detecting obstruction to vehicle
JP2000329852A (en) * 1999-05-17 2000-11-30 Nissan Motor Co Ltd Obstacle recognition device
JP2002099906A (en) * 2000-09-22 2002-04-05 Mazda Motor Corp Object-recognizing device
JP2003121547A (en) * 2001-10-18 2003-04-23 Fuji Heavy Ind Ltd Outside-of-vehicle monitoring apparatus
JP2003139858A (en) * 2001-11-02 2003-05-14 Fuji Heavy Ind Ltd Outside-vehicle monitoring apparatus
JP2003151094A (en) * 2001-11-15 2003-05-23 Fuji Heavy Ind Ltd Device for monitoring outside of vehicle
JP2003168197A (en) * 2001-11-30 2003-06-13 Hitachi Ltd Method and device for recognizing traveling circumstance
JP2004037239A (en) * 2002-07-03 2004-02-05 Fuji Heavy Ind Ltd Identical object judging method and system, and misregistration correcting method and system
JP2004117071A (en) * 2002-09-24 2004-04-15 Fuji Heavy Ind Ltd Vehicle surroundings monitoring apparatus and traveling control system incorporating the same

Cited By (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006240454A (en) * 2005-03-02 2006-09-14 Daihatsu Motor Co Ltd Object recognition device and object recognition method for vehicle
JP4518978B2 (en) * 2005-03-02 2010-08-04 ダイハツ工業株式会社 Vehicle object recognition device and object recognition method
JP2006293835A (en) * 2005-04-13 2006-10-26 Toyota Motor Corp Object detection apparatus
JP4534846B2 (en) * 2005-04-13 2010-09-01 トヨタ自動車株式会社 Object detection device
JP2007226680A (en) * 2006-02-24 2007-09-06 Toyota Motor Corp Object detection system
US7825849B2 (en) 2006-02-24 2010-11-02 Toyota Jidosha Kabushiki Kaisha Object detecting apparatus and method for detecting an object
JP4595833B2 (en) * 2006-02-24 2010-12-08 トヨタ自動車株式会社 Object detection device
JP2010501952A (en) * 2007-04-19 2010-01-21 ローベルト ボッシュ ゲゼルシャフト ミット ベシュレンクテル ハフツング Driver assistance system and method for determining the validity of an object
JP2011520704A (en) * 2008-05-23 2011-07-21 ヴアブコ・ゲゼルシヤフト・ミツト・ベシユレンクテル・ハフツング Driver assistance system
CN104554215B (en) * 2008-05-23 2022-04-15 威伯科有限公司 Driver assistance system
US9836964B2 (en) 2011-07-26 2017-12-05 Toyota Jidosha Kabushiki Kaisha Vehicle identification system and vehicle identification device
EP2927710A3 (en) * 2014-03-13 2015-12-30 Ricoh Company, Ltd. Ranging system, information processing apparatus, information processing method and program thereof
DE102015207206A1 (en) 2014-04-25 2015-10-29 Honda Motor Co., Ltd. OBJECT DETECTION DEVICE
US9921304B2 (en) 2014-04-25 2018-03-20 Honda Motor Co., Ltd. Object detection apparatus
US20150324652A1 (en) * 2014-05-09 2015-11-12 Honda Motor Co., Ltd. Object recognition apparatus
DE102015208599A1 (en) 2014-05-09 2015-11-12 Honda Motor Co., Ltd. Object detection device
CN105083161A (en) * 2014-05-09 2015-11-25 本田技研工业株式会社 Object recognition apparatus and method for recognizing objects
US9778352B2 (en) 2014-05-09 2017-10-03 Honda Motor Co., Ltd. Object recognition apparatus
WO2016047460A1 (en) * 2014-09-24 2016-03-31 株式会社デンソー Object detection device
US10049577B2 (en) 2014-09-24 2018-08-14 Denso Corporation Object detection apparatus
JP2016066182A (en) * 2014-09-24 2016-04-28 株式会社デンソー Object detector
KR20180018767A (en) 2015-07-27 2018-02-21 닛산 지도우샤 가부시키가이샤 Object detection method and object detection apparatus
US10269127B2 (en) 2015-07-27 2019-04-23 Nissan Motor Co., Ltd. Object detecting method and object detecting device
WO2019039279A1 (en) * 2017-08-22 2019-02-28 ソニー株式会社 Signal processing device, signal processing method, program, moving body, and signal processing system
JPWO2019039279A1 (en) * 2017-08-22 2020-10-15 ソニー株式会社 Signal processing equipment, signal processing methods, programs, mobiles, and signal processing systems
US11328444B2 (en) 2017-08-22 2022-05-10 Sony Corporation Signal processing apparatus, signal processing method, program, mobile object, and signal processing system
JP7160040B2 (en) 2017-08-22 2022-10-25 ソニーグループ株式会社 Signal processing device, signal processing method, program, moving object, and signal processing system
WO2019239775A1 (en) * 2018-06-15 2019-12-19 日立オートモティブシステムズ株式会社 Vehicle object sensing device
US11914028B2 (en) 2018-06-15 2024-02-27 Hitachi Astemo, Ltd. Object detection device for vehicle
US11391840B2 (en) 2018-06-25 2022-07-19 Ricoh Company, Ltd. Distance-measuring apparatus, mobile object, distance-measuring method, and distance measuring system

Also Published As

Publication number Publication date
JP4265931B2 (en) 2009-05-20

Similar Documents

Publication Publication Date Title
JP4265931B2 (en) Leading vehicle detection device
US6744380B2 (en) Apparatus for monitoring area adjacent to vehicle
US10922561B2 (en) Object recognition device and vehicle travel control system
US11097726B2 (en) Lane keeping assist system and method for improving safety in preceding vehicle follower longitudinal control
JP3860061B2 (en) Outside-of-vehicle monitoring device and travel control device equipped with this out-of-vehicle monitoring device
US11367291B2 (en) Traffic signal display estimation system
CN109219546B (en) Parking assistance device and parking assistance method
EP1564703A1 (en) Vehicle driving assist system
JPH09142236A (en) Periphery monitoring method and device for vehicle, and trouble deciding method and device for periphery monitoring device
US20030078730A1 (en) Monitor system of vehicle outside and method of monitoring same
US6834254B2 (en) Monitor system of vehicle outside and the method thereof
JP2004098717A (en) Scene monitor device and travel control device equipped with the same
JP2008123462A (en) Object detector
GB2313971A (en) Obstacle tracking by moving vehicle
JP2004117071A (en) Vehicle surroundings monitoring apparatus and traveling control system incorporating the same
JP7088137B2 (en) Traffic light information management system
JP2006011570A (en) Camera calibration method and camera calibration device
JP6194245B2 (en) Traffic light recognition device
JP3440956B2 (en) Roadway detection device for vehicles
JPH1069598A (en) Collision preventing device for vehicle
JP4518978B2 (en) Vehicle object recognition device and object recognition method
JP2006004188A (en) Obstacle recognition method and obstacle recognition device
JP7232005B2 (en) VEHICLE DRIVING ENVIRONMENT DETECTION DEVICE AND DRIVING CONTROL SYSTEM
JP4376147B2 (en) Obstacle recognition method and obstacle recognition device
JPH1068777A (en) On-vehicle preceding car detecting device

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20060512

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20080529

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20080617

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20080725

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20080902

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20081030

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20081125

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20090119

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20090210

A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20090212

R150 Certificate of patent or registration of utility model

Ref document number: 4265931

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

Free format text: JAPANESE INTERMEDIATE CODE: R150

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20120227

Year of fee payment: 3

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20120227

Year of fee payment: 3

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20130227

Year of fee payment: 4

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20140227

Year of fee payment: 5

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

S531 Written request for registration of change of domicile

Free format text: JAPANESE INTERMEDIATE CODE: R313531

R350 Written notification of registration of transfer

Free format text: JAPANESE INTERMEDIATE CODE: R350

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

S533 Written request for registration of change of name

Free format text: JAPANESE INTERMEDIATE CODE: R313533

R350 Written notification of registration of transfer

Free format text: JAPANESE INTERMEDIATE CODE: R350

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

LAPS Cancellation because of no payment of annual fees