JPH10283592A - On-vehicle drive supporting device - Google Patents

On-vehicle drive supporting device

Info

Publication number
JPH10283592A
JPH10283592A JP9103797A JP9103797A JPH10283592A JP H10283592 A JPH10283592 A JP H10283592A JP 9103797 A JP9103797 A JP 9103797A JP 9103797 A JP9103797 A JP 9103797A JP H10283592 A JPH10283592 A JP H10283592A
Authority
JP
Japan
Prior art keywords
vehicle
obstacle
contact
road
possibility
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
JP9103797A
Other languages
Japanese (ja)
Other versions
JP3827804B2 (en
Inventor
Hideaki Tsuchiya
英明 土屋
Atsushi Ikeda
敦 池田
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Subaru Corp
Original Assignee
Fuji Heavy Industries Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fuji Heavy Industries Ltd filed Critical Fuji Heavy Industries Ltd
Priority to JP09103797A priority Critical patent/JP3827804B2/en
Publication of JPH10283592A publication Critical patent/JPH10283592A/en
Application granted granted Critical
Publication of JP3827804B2 publication Critical patent/JP3827804B2/en
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Landscapes

  • Traffic Control Systems (AREA)

Abstract

PROBLEM TO BE SOLVED: To accurately decide the propriety of proceeding a narrow road and to surely prevent contact with an obstacle. SOLUTION: Each signal is inputted to a controller 5 from CCD cameras 4, sensors 6, 7, 9 and 10 and a mirror switch 11, an image recognizing part 21 acquires an environment in a proceeding direction of self-vehicle with three- dimensional image data of a distance distribution, a road shape and obstacle recognizing part 22 three-dimensionally recognizes the shapes of a road and obstacles, a road and obstacle position calculating and storing part 23 calculates and stores the positions of the road and obstacles on a three-dimensional coordinate, a self-vehicle position calculating and storing part 24 calculates and stores the position of self-vehicle 1 on the three-dimensional coordinate, and a self- vehicle estimated course calculating part 25 calculates estimated course of the vehicle 1 on the three-dimensional coordinate from a driving state. A contact deciding part 26 calculates contact possibilities of obstacles and the vehicle 1 on a flat plane and in a height direction based on the estimated course and a vehicle external form, generally decides the contacts of the obstacles and the vehicle 1 and performs output control to an alarm generating part 12, a sound generating 13, and 1st and 2nd displaying parts 14 and 15.

Description

【発明の詳細な説明】DETAILED DESCRIPTION OF THE INVENTION

【0001】[0001]

【発明の属する技術分野】本発明は、ガードレール、側
壁、駐車車両等の障害物との接触の可能性についての正
確な情報を提供して狭路等への進入、及びに狭路等の走
行が容易に行なえるようにドライバの運転を支援する車
両用運転支援装置に関する。
BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention provides accurate information on the possibility of contact with obstacles such as guardrails, side walls, parked vehicles, etc. to enter narrow roads and travel on narrow roads. The present invention relates to a vehicle driving support device that supports driving of a driver so that driving can be easily performed.

【0002】[0002]

【従来の技術】近年、車両の安全性の向上を図るため、
積極的にドライバの運転操作を支援する総合的な運転支
援システム(ADA;Active Drive Assist system)が
開発されている。このADAシステムは、車両の走行環
境情報や自車両の走行状態から先行車両との衝突、障害
物との接触、車線逸脱等の様々な可能性を推定して、安
全を維持できないと予測される場合に、ドライバに対し
て報知、その他制御等を行なうものである。
2. Description of the Related Art In recent years, in order to improve the safety of vehicles,
2. Description of the Related Art A comprehensive driving assistance system (ADA; Active Drive Assist system) has been developed that actively supports the driving operation of a driver. This ADA system estimates various possibilities such as a collision with a preceding vehicle, a contact with an obstacle, a lane departure, and the like from the traveling environment information of the vehicle and a traveling state of the own vehicle, and is predicted to be unable to maintain safety. In this case, notification and other control are performed to the driver.

【0003】上記車両の走行環境情報を得るための装置
としては、レーザ・レーダ装置等が従来より公知である
が、最近では車両に搭載した複数のカメラにより捉えた
車両前方の風景や物体の画像情報を処理して、道路、交
通環境を実用上十分な精度と時間で三次元的に認識する
ことが可能になってきている。
As a device for obtaining the traveling environment information of the vehicle, a laser radar device and the like have been conventionally known, but recently, an image of a landscape or an object in front of the vehicle captured by a plurality of cameras mounted on the vehicle. It has become possible to process information to three-dimensionally recognize roads and traffic environments with sufficient accuracy and time for practical use.

【0004】上記ADAシステムの機能の一つである狭
路進入の可否の判定や、障害物との接触防止を図る機能
を有するものとして、駐車空間を決定し、この駐車空間
の情報に基づく様々な駐車補助制御を行って駐車の際の
ドライバの負担を軽減するものがある。
A parking space is determined as having a function of judging whether or not to enter a narrow road, which is one of the functions of the ADA system, and a function of preventing contact with an obstacle. There is a vehicle that performs a simple parking assist control to reduce the burden on the driver during parking.

【0005】この駐車補助制御装置では駐車空間を正確
に決定するため、例えば、特開平6−274796号公
報に示されるような技術が提案されている。この技術
は、車体後部のCCDカメラにて得た画像データを距離
データに変換して既駐車車両のコーナーを検出し、さら
に距離データを用いて算出される駐車入口直線及び検出
したコーナー点に基づき、既駐車車両の側面を近似した
直線等を算出して誘導すべき駐車空間を決定するもので
ある。
In order to accurately determine a parking space in this parking assist control device, for example, a technique as disclosed in Japanese Patent Application Laid-Open No. 6-274796 has been proposed. This technology converts the image data obtained by the CCD camera at the rear of the vehicle body into distance data, detects the corner of the parked vehicle, and further calculates the parking entrance straight line calculated using the distance data and the detected corner point. And calculating a straight line or the like approximating the side of the parked vehicle to determine a parking space to be guided.

【0006】[0006]

【発明が解決しようとする課題】しかしながら、既駐車
車両の側面を高さ方向の凹凸の分布のない直線等で近似
するだけでは、既駐車車両が例えば、横にドアミラー等
の部分的な突起部を有する際に有効に対処することが難
しい。すなわち、障害物と自車両との接触は、障害物の
外形の高さ方向の凹凸の分布と、自車両の高さ方向の凹
凸の分布によっても決定されるものであって、例え障害
物に一部突出している部分があったとしても、その部分
の高さの自車両の外形が凹んでいたり、あるいはその部
分の高さに自車両の外形が無く衝突する可能性が無い場
合がある。
However, simply approximating the side surface of the parked vehicle with a straight line or the like having no unevenness distribution in the height direction, the parked vehicle may be, for example, partially projected such as a side mirror such as a side mirror. It is difficult to effectively deal with the problem. That is, the contact between the obstacle and the own vehicle is determined by the distribution of the unevenness in the height direction of the outer shape of the obstacle and the distribution of the unevenness in the height direction of the own vehicle. Even if there is a part that protrudes, the outer shape of the host vehicle at the height of the part may be depressed, or there may be no possibility that the height of the part has the outer shape of the host vehicle and collide.

【0007】また、障害物の外形を上から投影した線で
代表して、この障害物の横を自車両が走行可能か否かの
処理も狭路進入の可否の判定においてよく用いられる手
法であるが、例えば、図9に示すような自車両1がトラ
ック2の横を通過しようとする際に、このトラック2の
ドアミラー2aは高さ方向の一部であっても、上からの
投影だけでは上記トラック2の上から下まで突起が存在
するように表現されるため、自車両1の高さが上記トラ
ック2のドアミラー2aより低いにもかかわらず、通行
できないと判定されてしまう可能性がある。
[0007] Also, as a representative of the outline of the obstacle as a line projected from above, a process for determining whether or not the own vehicle can travel beside the obstacle is a method often used in determining whether or not to enter a narrow road. However, for example, when the own vehicle 1 as shown in FIG. 9 tries to pass beside the truck 2, even if the door mirror 2a of the truck 2 is a part of the height direction, only the projection from above is performed. In this case, since the protrusions are expressed from the top to the bottom of the truck 2, there is a possibility that it is determined that the vehicle cannot pass even though the height of the host vehicle 1 is lower than the door mirror 2 a of the truck 2. is there.

【0008】本発明は上記事情に鑑みてなされたもの
で、狭路進入の可否の判定や、障害物との接触防止を図
るにあたり、狭路進入の可否の判定が正確に行え、確実
に障害物との接触を防止してドライバの運転を有効に支
援する車両用運転支援装置を提供することを目的として
いる。
The present invention has been made in view of the above circumstances. In order to determine whether or not to enter a narrow road and to prevent contact with an obstacle, it is possible to accurately determine whether or not to enter a narrow road, and to reliably determine whether to enter a narrow road. It is an object of the present invention to provide a driving support device for a vehicle that prevents a contact with an object and effectively supports driving of a driver.

【0009】[0009]

【課題を解決するための手段】上記目的を達成するため
本発明による車両用運転支援装置は、自車両の進行方向
の環境を距離分布の三次元画像データとして得る画像認
識手段と、上記三次元画像データから上記自車両の進行
方向の道路の形状と障害物を三次元に認識する道路形状
・障害物認識手段と、上記認識した道路と障害物の予め
設定しておいた三次元空間での位置を算出して記憶する
道路・障害物位置算出記憶手段と、上記道路・障害物と
相対的な位置にある上記自車両の上記予め設定しておい
た三次元空間での位置を算出して記憶する自車両位置算
出記憶手段と、上記自車両の運転状態から上記自車両の
上記予め設定しておいた三次元空間での予想進路を算出
する自車両予想進路算出手段と、上記自車両の予想進路
と予め記憶しておいた車両外形に基づき上記障害物と上
記自車両との平面上での接触可能性と高さ方向での接触
可能性を求め、これらの結果で上記障害物と上記自車両
との接触を判定する接触判定手段と、上記接触判定手段
での接触判定結果に基づき運転者に報知する報知手段と
を備えたものであり、望ましくは、上記道路・障害物位
置算出記憶手段は、道路面より下の障害物の情報を検出
する手段を有し、上記予め設定しておいた三次元空間で
の位置を算出して記憶するように形成し、また、望まし
くは、上記接触判定手段は、上記自車両の予め設定して
おいた各状態での複数の上記車両外形の情報を有し、上
記各設定状態に応じて対応する車両外形情報で接触の判
定を行うように形成し、さらに望ましくは、上記報知手
段は、上記障害物と上記自車両との状態を、俯瞰図と上
記自車両後方からの投影図の少なくとも一方の視覚的表
示部と、音声で報知する音声発生部と、警報音で報知す
る警報音発生部の少なくとも一つで形成したものであ
る。
According to the present invention, there is provided a driving support system for a vehicle, comprising: image recognition means for obtaining an environment in a traveling direction of a host vehicle as three-dimensional image data of a distance distribution; Road shape / obstacle recognition means for three-dimensionally recognizing the shape of the road and obstacles in the traveling direction of the vehicle from the image data; and a preset three-dimensional space of the recognized roads and obstacles. A road / obstacle position calculating / storing means for calculating and storing a position; calculating a position in the preset three-dimensional space of the host vehicle at a position relative to the road / obstacle; Self-vehicle position calculation storage means for storing; self-vehicle predicted course calculation means for calculating an expected course of the self-vehicle in the preset three-dimensional space from the driving state of the self-vehicle; Memorize in advance the expected course Based on the obtained vehicle outline, the possibility of contact between the obstacle and the own vehicle on a plane and the possibility of contact in the height direction are determined, and based on these results, the contact between the obstacle and the own vehicle is determined. Contact determination means, and a notification means for notifying the driver based on the result of the contact determination by the contact determination means, desirably, the road / obstacle position calculation storage means, It has means for detecting information on an obstacle, and is formed so as to calculate and store the position in the three-dimensional space set in advance, and preferably, the contact determination means is provided in the vehicle. It has a plurality of information of the vehicle outer shape in each state set in advance, and is formed so as to determine the contact with the vehicle outer shape information corresponding to each set state, more preferably, The notifying unit is configured to determine whether the obstacle is in contact with the vehicle. The state is formed by at least one of a visual display unit of at least one of an overhead view and a projection view from the rear of the host vehicle, a voice generation unit that reports by sound, and an alarm sound generation unit that reports by an alarm sound. It is.

【0010】上記構成により、画像認識手段で自車両の
進行方向の環境を距離分布の三次元画像データとして得
て、道路形状・障害物認識手段で上記三次元画像データ
から上記自車両の進行方向の道路の形状と障害物を三次
元に認識し、道路・障害物位置算出記憶手段で上記認識
した道路と障害物の予め設定しておいた三次元空間での
位置を算出して記憶する。また、自車両位置算出記憶手
段で上記道路・障害物と相対的な位置にある上記自車両
の上記予め設定しておいた三次元空間での位置を算出し
て記憶する。自車両予想進路算出手段では、上記自車両
の運転状態から上記自車両の上記予め設定しておいた三
次元空間での予想進路を算出する。そして、接触判定手
段が、上記自車両の予想進路と予め記憶しておいた車両
外形に基づき上記障害物と上記自車両との平面上での接
触可能性と高さ方向での接触可能性を求め、これらの結
果で上記障害物と上記自車両との接触を判定して、上記
接触判定手段での接触判定結果に基づき報知手段は必要
な情報を運転者に報知する。上記道路・障害物位置算出
記憶手段は、道路面より下の障害物の情報をも検出し、
上記予め設定しておいた三次元空間での位置を算出して
記憶するように形成すれば、特に道路面より下の側溝、
縁石等の位置認識も正確に行えて好ましい。また、上記
接触判定手段は、上記自車両の予め設定しておいた各状
態での複数の上記車両外形の情報を有し、上記各設定状
態に応じて対応する車両外形情報で接触の判定を行うよ
うに形成すれば、車両の様々な状態に合った適切な車両
外形情報で接触の判定が行えるため好ましい。さらに、
上記報知手段は、上記障害物と上記自車両との状態を、
俯瞰図と上記自車両後方からの投影図の少なくとも一方
の視覚的表示部と、音声で報知する音声発生部と、警報
音で報知する警報音発生部の少なくとも一つで形成すれ
ば、ドライバに対しての報知が素早く行え、ドライバも
状況認識が的確に行えるので好ましい。
With the above arrangement, the environment in the traveling direction of the own vehicle is obtained as three-dimensional image data of the distance distribution by the image recognition means, and the traveling direction of the own vehicle is obtained from the three-dimensional image data by the road shape / obstacle recognition means. The shape and obstacle of the road are recognized three-dimensionally, and the road and obstacle position calculation storage means calculates and stores the positions of the recognized road and obstacle in a preset three-dimensional space. Further, the own vehicle position calculation storage means calculates and stores the position of the own vehicle at a position relative to the road / obstacle in the preset three-dimensional space. The own vehicle expected route calculating means calculates an expected route of the own vehicle in the preset three-dimensional space from the driving state of the own vehicle. Then, the contact determination means determines the possibility of contact between the obstacle and the own vehicle on a plane and the possibility of contact in the height direction based on the predicted course of the own vehicle and the vehicle shape stored in advance. Then, the contact between the obstacle and the host vehicle is determined based on these results, and the notifying unit notifies the driver of necessary information based on the contact determination result by the contact determining unit. The road / obstacle position calculation storage means also detects information on obstacles below the road surface,
If it is formed so as to calculate and store the position in the previously set three-dimensional space, especially the gutter below the road surface,
It is preferable that the position of a curb or the like can be accurately recognized. Further, the contact determination means has a plurality of pieces of information on the vehicle outer shape in each preset state of the own vehicle, and determines the contact with the vehicle outer shape information corresponding to each of the set states. It is preferable that the contact is determined so that the contact can be determined based on appropriate vehicle outer shape information suitable for various states of the vehicle. further,
The notifying unit is configured to indicate a state of the obstacle and the own vehicle,
If formed from at least one of a bird's-eye view and at least one of a projection view from the rear of the host vehicle, a voice generation unit that notifies by voice, and at least one of an alarm sound generation unit that notifies by an alarm sound, This is preferable because notification can be performed quickly and the driver can accurately recognize the situation.

【0011】[0011]

【発明の実施の形態】以下、図面に基づいて本発明の実
施の形態を説明する。図2は車両用運転支援装置の概略
構成図で、この図2において、符号1は自動車等の車両
(自車両)であり、この自車両1に、狭路進入の可否の
判定や、障害物との接触防止を図る機能を一つの機能と
して有し、ドライバの運転を支援する車両用運転支援装
置3が搭載されている。以下、本発明の実施の形態で
は、車両用運転支援装置3の狭路進入の可否の判定や、
障害物との接触防止を図る機能の部分についてのみ説明
し、他の機能の部分については説明を省略する。
Embodiments of the present invention will be described below with reference to the drawings. FIG. 2 is a schematic configuration diagram of the vehicle driving support device. In FIG. 2, reference numeral 1 denotes a vehicle (own vehicle) such as an automobile. The vehicle has a function of preventing contact with the vehicle as one function, and is equipped with a vehicle driving support device 3 that supports driving of a driver. Hereinafter, in the embodiment of the present invention, it is determined whether the vehicle driving assistance device 3 can enter a narrow road,
Only the part of the function for preventing contact with an obstacle will be described, and the description of the other function parts will be omitted.

【0012】上記車両用運転支援装置3は、ステレオ光
学系として例えば電荷結合素子(CCD)等の固体撮像
素子を用いた1組の(左右の)CCDカメラ4を有し、
これら左右のCCDカメラ4は、それぞれ車室内の天井
前方に一定の間隔をもって取り付けられ、車外の対象を
異なる視点からステレオ撮像するようになっている。そ
して、上記1組のCCDカメラ4で撮像した自車両1の
走行(進入)方向の映像信号は、制御装置5に入力され
るようになっている。
The vehicle driving support device 3 has a set of (left and right) CCD cameras 4 using a solid-state image sensor such as a charge-coupled device (CCD) as a stereo optical system.
These left and right CCD cameras 4 are respectively mounted at predetermined intervals in front of the ceiling in the vehicle cabin so as to stereoscopically image an object outside the vehicle from different viewpoints. Then, a video signal of the traveling (entering) direction of the own vehicle 1 captured by the set of CCD cameras 4 is input to the control device 5.

【0013】また、上記車両用運転支援装置3には、上
記自車両1の4隅に超音波センサ6,7(左右前端側超
音波センサ6,左右後端側超音波センサ7)が配設され
ている。上記超音波センサ6,7は、図3に示すよう
に、道路下の斜め側方の形状を検出するようにそれぞれ
設けられており、図4に示すような状況、すなわち上記
自車両1の道路下斜め側方に側溝8等がある場合に、こ
の側溝8等の位置が正確に検出されるようになってい
る。
The vehicle driving support device 3 has ultrasonic sensors 6 and 7 (front left and right ultrasonic sensors 6 and rear left and right ultrasonic sensors 7) at four corners of the vehicle 1. Have been. As shown in FIG. 3, the ultrasonic sensors 6 and 7 are provided so as to detect a shape of a diagonal side under the road, respectively, and a situation as shown in FIG. When there is a side groove 8 or the like on the lower oblique side, the position of the side groove 8 or the like is accurately detected.

【0014】上記超音波センサ6,7は、例えば、上記
各超音波センサ6,7の超音波振動子(図示せず)が発
振器及び受信器を兼ねて構成される周知のものであり、
指向性の強い音波を所定の範囲内で方向を変えながら発
し超音波振動子の発振開始時刻を検出し、次に反射波入
力時刻を検出して両時刻の差と音波を発する方向から上
記各超音波センサ6,7と道路下斜め側方の障害物(側
溝8等)との間の距離と方向を検出するもので、これら
上記超音波センサ6,7からの信号は上記制御装置5に
入力されるようになっている。
The ultrasonic sensors 6 and 7 are, for example, well-known ultrasonic transducers (not shown) of each of the ultrasonic sensors 6 and 7 configured as an oscillator and a receiver.
The directional strong sound wave is emitted while changing the direction within a predetermined range, the oscillation start time of the ultrasonic transducer is detected, then the reflected wave input time is detected, and the difference between the two times and the direction in which the sound wave is emitted are detected. It detects the distance and direction between the ultrasonic sensors 6 and 7 and obstacles (side grooves 8 and the like) obliquely below the road, and signals from the ultrasonic sensors 6 and 7 are sent to the control device 5. Is to be entered.

【0015】さらに、上記車両用運転支援装置3は、上
記自車両1の速度を検出する車速センサ9、ハンドル角
を検出するハンドル角センサ10およびドアミラー1a
の格納・復帰を指令するミラースイッチ11からの各信
号が上記制御装置5に入力されるように形成されてお
り、上記制御装置5は上述の各情報(CCDカメラ4か
らの映像信号、超音波センサ6,7、車速センサ9、ハ
ンドル角センサ10、およびミラースイッチ11からの
各信号)に基づいて狭路進入の可否の判定や、障害物と
の接触防止を図る機能を達成すべく報知手段としての警
報発生部12、音声発生部13、第1の表示部14、第
2の表示部15に制御出力するように構成されている。
Further, the vehicle driving support device 3 includes a vehicle speed sensor 9 for detecting the speed of the host vehicle 1, a steering wheel angle sensor 10 for detecting a steering wheel angle, and a door mirror 1a.
Each signal from the mirror switch 11 for instructing storage / return of the image data is input to the control device 5, and the control device 5 transmits the above information (the video signal from the CCD camera 4, Notification means for determining whether or not to enter a narrow road and preventing contact with an obstacle based on the signals from the sensors 6 and 7, the vehicle speed sensor 9, the steering wheel angle sensor 10, and the mirror switch 11. The control output is performed to an alarm generation unit 12, a voice generation unit 13, a first display unit 14, and a second display unit 15 as the above.

【0016】上記警報発生部12は、例えばブザー等で
あり、進入ができない寸法の狭路に走行していった場合
や、走行を続けると障害物との接触の可能性が有る場合
に上記制御装置5からの出力信号により警報音を発して
ドライバに報知するようになっている。
The alarm generation unit 12 is, for example, a buzzer or the like, and performs the above-described control when the vehicle travels on a narrow road that cannot be entered, or when there is a possibility of contact with an obstacle if the vehicle continues to travel. A warning sound is issued by an output signal from the device 5 to notify the driver.

【0017】上記音声発生部13は、上記制御装置5か
らの出力信号に応じて予め録音しておいた音声の中(例
えば、「前方は通過困難」、「左前方注意」、「左側方
注意」、「右前方注意」、「右側方注意」…等)から所
定の音声を選択し、車室内に設けた図示しないスピーカ
から車室内に音声を発するように形成されている。
The voice generating unit 13 includes a voice recorded in advance in accordance with an output signal from the control unit 5 (for example, “passing ahead”, “caution left front”, “caution left”. , "Right attention", "right attention", etc.), and a speaker (not shown) provided in the cabin emits sound into the cabin.

【0018】上記第1の表示部14と上記第2の表示部
15は、上記制御装置5からの出力信号に応じて、車内
に設けたそれぞれのモニタに、自車両1と障害物との位
置関係を視覚的に表示するものである。
The first display unit 14 and the second display unit 15 display the positions of the vehicle 1 and obstacles on respective monitors provided in the vehicle in accordance with an output signal from the control device 5. The relationship is visually displayed.

【0019】すなわち、上記第1の表示部14と上記第
2の表示部15のそれぞれのモニタは、図5に示すよう
に隣り合って、1モニタ2画面上で設けられ、上記第1
の表示部14では自車両1を上から見た俯瞰図を表示す
る画面上に、自車両1と障害物(図5ではトラック2と
電柱17を例にしている)との平面上での接触可能性を
表示するようになっている。また、上記第2の表示部1
5では自車両1を後方から投影した図をを表示する画面
上に、自車両1と障害物との高さ方向での接触可能性を
表示するようになっている。尚、画面上に映し出される
障害物の形状は、予めメモリ内に記憶しておいた物体形
状の中から選択して表示するものであっても、上記CC
Dカメラ4で撮影した映像をそのまま用いるものであっ
てもどちらでも良い。
That is, the respective monitors of the first display unit 14 and the second display unit 15 are provided adjacent to each other on one monitor and two screens as shown in FIG.
Is displayed on a screen displaying a bird's-eye view of the host vehicle 1 from above, and a contact between the host vehicle 1 and an obstacle (in FIG. 5, the truck 2 and the telephone pole 17 are illustrated) on a plane. It shows the possibility. Further, the second display unit 1
In 5, the possibility of contact between the host vehicle 1 and an obstacle in the height direction is displayed on a screen that displays a view of the host vehicle 1 projected from behind. Note that the shape of the obstacle projected on the screen may be selected from the object shapes stored in the memory in advance and displayed.
The video taken by the D camera 4 may be used as it is, or either may be used.

【0020】上記警報発生部12、音声発生部13、お
よび第1の表示部14と第2の表示部15は、上記制御
装置5によってドライバが効果的に認識できるように作
動されるようになっており、例えば自車両1と障害物と
の状況を、常時、上記第1の表示部14と第2の表示部
15で提供し、所定の距離から障害物へ向けて走行する
場合に上記音声発生部13による情報提供を行ない、障
害物との接触可能性が所定に高まった場合に上記警報発
生部12による警報が行われるようになっている。
The alarm generator 12, the voice generator 13, and the first display unit 14 and the second display unit 15 are operated by the control device 5 so that the driver can recognize them effectively. For example, the status of the host vehicle 1 and the obstacle is always provided on the first display unit 14 and the second display unit 15, and when the vehicle travels from a predetermined distance to the obstacle, the sound is output. The information is provided by the generating unit 13, and when the possibility of contact with an obstacle increases to a predetermined level, the alarm is generated by the alarm generating unit 12.

【0021】上記制御装置5は、マイクロコンピュータ
とその周辺回路で形成され、図1に示すように、画像認
識部21、道路形状・障害物認識部22、道路・障害物
位置算出記憶部23、自車両位置算出記憶部24、自車
両予想進路算出部25、接触判定部26で主に構成され
ている。
The control device 5 is formed by a microcomputer and its peripheral circuits. As shown in FIG. 1, an image recognition unit 21, a road shape / obstacle recognition unit 22, a road / obstacle position calculation storage unit 23, It mainly comprises a host vehicle position calculation storage unit 24, a host vehicle predicted course calculation unit 25, and a contact determination unit 26.

【0022】上記画像認識部21は、上記CCDカメラ
4で撮像した自車両の進入方向の環境の1組のステレオ
画像対に対し、対応する位置のずれ量から三角測量の原
理によって画像全体に渡る距離情報を求める処理を行な
って、三次元の距離分布を表す距離画像を生成して上記
道路形状・障害物認識部22に出力する画像認識手段と
して形成されている。
The image recognizing unit 21 covers the whole image of a set of stereo images of the environment in the approach direction of the vehicle captured by the CCD camera 4 based on the amount of displacement of the corresponding position based on the principle of triangulation. It is formed as image recognition means for performing a process of obtaining distance information, generating a distance image representing a three-dimensional distance distribution, and outputting the generated distance image to the road shape / obstacle recognition unit 22.

【0023】上記道路形状・障害物認識部22は、上記
画像認識部21からの三次元画像データから上記自車両
1の進入方向の道路の形状を認識し、同時に障害物を三
次元的に認識して上記道路・障害物位置算出記憶部23
に出力する道路形状・障害物認識手段として形成されて
いる。
The road shape / obstacle recognition unit 22 recognizes the shape of the road in the approach direction of the vehicle 1 from the three-dimensional image data from the image recognition unit 21, and at the same time recognizes the obstacle three-dimensionally. And the road / obstacle position calculation storage unit 23
It is formed as a road shape / obstacle recognizing means for outputting to the user.

【0024】即ち、個々の物体の三次元距離データによ
り、物体が重なっている場合にも容易に分離して、何で
あるかを認識することができ、これにより白線、ガード
レール、縁石等が抽出される。そして例えば白線を折れ
線で近似して、左右の折れ線で囲まれた範囲を自車線と
判断し、この自車線のデータの水平成分で道路のカーブ
を、垂直成分で道路の上りや下りを検出して、手前から
遠方に向かって道路形状を三次元的に検出するのであ
る。
That is, based on the three-dimensional distance data of the individual objects, even if the objects are overlapped, it is possible to easily separate them and recognize what they are, thereby extracting white lines, guardrails, curbs and the like. You. Then, for example, the white line is approximated by a polygonal line, the area surrounded by the left and right polygonal lines is determined to be the own lane, the curve of the road is detected by the horizontal component of the data of the own lane, and the ascending or descending of the road is detected by the vertical component. Thus, the road shape is three-dimensionally detected from the near side to the far side.

【0025】また、画像データから障害となる立体物の
みを選別し、この選別された物体が自車両1に対しての
障害物であるのかを判断すると共に、その障害物までの
距離(相対的な距離)、大きさ、左右端の位置を検出す
るのであり、この道路形状と障害物のデータが上記道路
・障害物位置算出記憶部23に出力されるようになって
いる。
Further, only the three-dimensional object that becomes an obstacle is selected from the image data, it is determined whether or not the selected object is an obstacle to the host vehicle 1, and the distance (relative) to the obstacle is determined. The data of the road shape and the obstacle is output to the road / obstacle position calculation storage unit 23.

【0026】上記道路・障害物位置算出記憶部23は、
上記道路形状・障害物認識部22で抽出した道路形状と
障害物を予め設定しておいた三次元座標上の位置を算出
して記憶する道路・障害物位置算出記憶手段として形成
されている。上記予め設定しておいた三次元座標は、自
車両1と障害物とが平面、高さ方向で十分に収まる範囲
で形成されている。
The road / obstacle position calculation storage unit 23 includes:
The road shape / obstacle recognition unit 22 is formed as a road / obstacle position calculation storage unit that calculates and stores a position on three-dimensional coordinates in which the road shape and the obstacle extracted by the road shape / obstacle recognition unit 22 are set in advance. The previously set three-dimensional coordinates are formed in a range where the host vehicle 1 and the obstacle can sufficiently be accommodated in the plane and height directions.

【0027】また、上記道路・障害物位置算出記憶部2
3には、前記超音波センサ6,7からの信号が入力さ
れ、道路面より下にある側溝あるいは縁石等の位置が正
確に三次元座標に表現されるようになっている。
Further, the road / obstacle position calculation storage unit 2
Signals from the ultrasonic sensors 6 and 7 are input to 3 so that the position of a gutter or curb below the road surface is accurately represented in three-dimensional coordinates.

【0028】上記自車両位置算出記憶部24は、上記道
路・障害物位置算出記憶部23で道路形状と障害物の位
置を設定した三次元座標上に、上記道路・障害物と相対
的な位置にある自車両1の位置を算出して記憶する自車
両位置算出記憶手段としてのもので、道路形状と障害物
の位置および自車両1の位置が設定された三次元座標の
データは上記自車両予想進路算出部25に出力されるよ
うになっている。
The self-vehicle position calculation storage unit 24 stores the relative position of the road / obstacle on the three-dimensional coordinates in which the road shape and the position of the obstacle are set by the road / obstacle position calculation storage unit 23. The three-dimensional coordinate data in which the shape of the road, the position of the obstacle, and the position of the own vehicle 1 are set are used as the own vehicle position calculation storage means for calculating and storing the position of the own vehicle 1. The information is output to the predicted course calculation unit 25.

【0029】尚、上記道路・障害物位置算出記憶部23
と上記自車両位置算出記憶部24で形成する座標は、例
えば、上記道路形状・障害物認識部22で得られる新た
な自車両から見た道路・障害物の相対位置の座標を、前
回までに計算した道路・障害物の座標に車両運動を考慮
して繰り返し加えていくことにより形成するようにして
も良い。これはすなわち、ハンドル角により自車両1の
直進・旋回を判断し、直進の場合、車両移動量(例えば
車速と経過時間とで算出する)を前回までに計算した全
ての道路形状・障害物の座標の車両前後方向成分に加算
し、また、旋回の場合は、移動量とハンドル角から自車
両1の回転中心と回転角を求め、前回までに計算した全
ての道路形状・障害物の座標を上記回転中心に対して回
転させる。この処理後、記憶してある全ての道路形状・
障害物の座標のなかで記憶領域外に出たもののデータと
前記CCDカメラ4の視野内にあるデータを消去し、新
たに上記CCDカメラ4の視野内で計算された相対位置
の座標を記憶データに追加するものである。
The road / obstacle position calculation storage unit 23
And the coordinates formed by the host vehicle position calculation storage unit 24 are, for example, the coordinates of the relative position of the road / obstacle viewed from the new host vehicle obtained by the road shape / obstacle recognition unit 22 by the previous time. It may be formed by repeatedly adding the calculated road / obstacle coordinates in consideration of the vehicle motion. In other words, the straight traveling / turning of the own vehicle 1 is determined based on the steering wheel angle, and in the case of straight traveling, the vehicle movement amount (for example, calculated by the vehicle speed and the elapsed time) is calculated for all road shapes and obstacles calculated up to the previous time. In the case of a turn, the rotation center and rotation angle of the vehicle 1 are obtained from the movement amount and the steering wheel angle, and the coordinates of all road shapes and obstacles calculated up to the previous time are calculated. Rotate with respect to the rotation center. After this processing, all the stored road shapes and
The data of the obstacles that have come out of the storage area and the data in the field of view of the CCD camera 4 are deleted, and the coordinates of the relative position newly calculated in the field of view of the CCD camera 4 are stored in the storage data. Is to be added to

【0030】上記自車両予想進路算出部25は、自車両
予想進路算出手段としてのもので、前記車速センサ9か
らの車両速度、前記ハンドル角センサ10からのハンド
ル角、上記自車両位置算出記憶部24からの自車両1と
道路・障害物の位置を示す三次元座標のデータを基に、
この三次元座標上で自車両1がどのような進路で走行さ
れるかを、自車両1の車両諸元で予め設定しておいた車
両の運動方程式等により求めて予想するようになってお
り、求めた予想進路は上記接触判定部26に出力され
る。
The predicted vehicle route calculating section 25 serves as a predicted vehicle route calculating means, and includes a vehicle speed from the vehicle speed sensor 9, a steering wheel angle from the steering wheel angle sensor 10, and a vehicle position calculating and storing section. Based on the three-dimensional coordinate data indicating the position of the vehicle 1 and the road / obstacle from 24,
The path on which the vehicle 1 travels on the three-dimensional coordinates is determined by predicting the vehicle's equation of motion or the like set in advance in the vehicle specifications of the vehicle 1. , And the obtained predicted course is output to the contact determination unit 26.

【0031】上記接触判定部26は、上記自車両予想進
路算出部25で設定した上記自車両1の予想進路と前記
ミラースイッチ11からの信号が入力され、 上記自車
両1の予想進路と上記ミラースイッチ11からの信号で
選択される予め記憶しておいた自車両1の車両外形に基
づき障害物と自車両1との平面上での接触可能性と高さ
方向での接触可能性を求め、これらの結果で上記障害物
と自車両1との接触を判定し、必要に応じて上記警報発
生部12、音声発生部13、および第1の表示部14と
第2の表示部15に出力してドライバに報知する接触判
定手段として形成されている。
The contact determination section 26 receives the predicted path of the host vehicle 1 and the signal from the mirror switch 11 set by the predicted path calculation section 25 of the host vehicle, and inputs the predicted path of the host vehicle 1 and the mirror. Based on the vehicle shape of the host vehicle 1 stored in advance, which is selected by a signal from the switch 11, the possibility of contact between the obstacle and the host vehicle 1 on a plane and the possibility of contact in the height direction are determined. Based on these results, the contact between the obstacle and the host vehicle 1 is determined, and output to the alarm generation unit 12, the voice generation unit 13, and the first display unit 14 and the second display unit 15 as necessary. As a contact determination means for notifying the driver.

【0032】すなわち、接触判定は自車両1の車両外形
に基づき予め設定しておいた接触判定外形線を上記予想
進路に従って移動させ、この移動した接触外形線と障害
物とが接触するか否かで行われるようになっている。
That is, the contact determination is performed by moving a contact determination outline set in advance based on the vehicle outline of the vehicle 1 in accordance with the predicted course, and determining whether or not the moved contact outline contacts an obstacle. It is to be performed in.

【0033】平面上での接触可能性を判定するための上
記接触判定外形線は、上面から見てドアミラー1aの復
帰・格納の際の突出を考慮するとともに、車両バンパー
の形状に合わせてドライバの感覚に沿って形成され、図
6(b)に示すように、ドアミラー1aを復帰した状態
の接触判定外形線31bと、図7(b)に示すように、
ドアミラー1aを格納した状態の接触判定外形線32b
の2種類が設定され、上記ミラースイッチ11からの信
号でどちらかが選択されるようになっている。
The contact determination outline for determining the possibility of contact on a plane takes into account the protrusion of the door mirror 1a when the door mirror 1a is returned and stored when viewed from above, and matches the shape of the vehicle bumper. As shown in FIG. 6B, the contact determination outer shape line 31b is formed in a state where the door mirror 1a is returned, and as shown in FIG.
Contact determination outline 32b with door mirror 1a stored
Are set, and one of them is selected by a signal from the mirror switch 11.

【0034】また、高さ方向での接触可能性を判定する
ための上記接触判定外形線は、正面から見てドアミラー
1aの復帰・格納の際の突出を考慮して車両形状をドラ
イバの感覚に沿って形成され、図6(a)に示すよう
に、ドアミラー1aを復帰した状態の接触判定外形線3
1aと、図7(a)に示すように、ドアミラー1aを格
納した状態の接触判定外形線32aの2種類が設定され
ており、上記ミラースイッチ11からの信号でどちらか
が選択されるようになっている。
In addition, the contact determination outline for determining the possibility of contact in the height direction is such that the shape of the vehicle can be felt by the driver in consideration of the projection when the door mirror 1a is returned and stored when viewed from the front. As shown in FIG. 6 (a), the contact determination outline 3 in a state where the door mirror 1a is returned.
As shown in FIG. 7A, two types of contact determination outline lines 32 a in a state where the door mirror 1 a is stored are set, and one of them is selected by a signal from the mirror switch 11. Has become.

【0035】上記接触判定部26で、例えば平面上での
ドアミラー1aを復帰した自車両1の障害物との接触可
能性を判定する場合には、図8に示すように、上記接触
判定外形線31bを上記自車両予想進路算出部25で求
めた予想進路に基づき内輪差等を考慮して移動させ、移
動させるエリア内に障害物が有る場合は平面上の判定で
は接触すると結果を出力するものである。また、平面上
でのドアミラー1aを格納した自車両1の障害物との接
触可能性の判定、高さ方向でのドアミラー1aを復帰・
格納した自車両1の障害物との接触可能性の判定も、特
に図示しないが同様に行われるようになっている。
When the contact determination unit 26 determines, for example, the possibility of contact with the obstacle of the host vehicle 1 after returning the door mirror 1a on a plane, as shown in FIG. 31b is moved in consideration of an inner wheel difference or the like based on the predicted course calculated by the own vehicle predicted course calculation unit 25, and when there is an obstacle in an area to be moved, a result is output when it is touched in the determination on a plane. It is. In addition, the possibility of contact with the obstacle of the vehicle 1 storing the door mirror 1a on the plane is determined, and the door mirror 1a is returned in the height direction.
The stored possibility of contacting the own vehicle 1 with an obstacle is determined in a similar manner, not shown.

【0036】そして、上記接触判定部26での平面上で
の接触可能性の判定結果は前記第1の表示部14の画面
上に表示され、高さ方向での接触可能性の判定結果は前
記第2の表示部15の画面上に表示される。ここで、例
えば図5に示すように、平面上での接触可能性の判定結
果が接触の可能性が有りとなっても、高さ方向での接触
可能性の判定結果は接触の可能性が無いとの結果となっ
ているため、上記接触判定部26は自車両1と障害物と
は接触しないと判定する。すなわち、上記接触判定部2
6は、平面上での接触可能性の判定結果と高さ方向での
接触可能性の判定結果の両方が接触可能性有りの場合に
自車両1と障害物とは接触する可能性ありと判定し、上
記警報発生部12および上記音声発生部13に対して出
力してドライバに報知するようになっている。
The result of the determination of the possibility of contact on a plane by the contact determination unit 26 is displayed on the screen of the first display unit 14, and the result of determination of the possibility of contact in the height direction is It is displayed on the screen of the second display unit 15. Here, for example, as shown in FIG. 5, even if the determination result of the possibility of contact on the plane indicates that there is a possibility of contact, the result of determination of the possibility of contact in the height direction indicates that the possibility of contact is high. Since there is no result, the contact determination unit 26 determines that the own vehicle 1 does not contact the obstacle. That is, the contact determination unit 2
6 indicates that there is a possibility that the own vehicle 1 and the obstacle may come into contact with each other when both the determination result of the possibility of contact on the plane and the determination result of the possibility of contact in the height direction indicate that there is a possibility of contact. Then, the warning is output to the alarm generator 12 and the voice generator 13 to notify the driver.

【0037】次に、上記構成による車両用運転支援装置
の作用について説明する。まず、車両が走行する際、C
CDカメラ4で撮像された自車両1の走行(進入)方向
の映像信号、超音波センサ6,7による道路下斜め側方
の障害物(側溝8等)との距離を示す信号、車速センサ
9による自車両速度の信号、ハンドル角センサ10によ
るハンドル角の信号、およびミラースイッチ11による
自車両1のドアミラー1aの復帰・格納の状態を示す信
号の各信号が制御装置5に入力される。
Next, the operation of the vehicle driving support device having the above configuration will be described. First, when the vehicle travels, C
A video signal of the traveling direction of the own vehicle 1 captured by the CD camera 4, a signal indicating a distance to an obstacle (e.g., a side groove 8) obliquely below the road by the ultrasonic sensors 6 and 7, a vehicle speed sensor 9. , A signal indicating a state of returning and storing the door mirror 1a of the own vehicle 1 by the mirror switch 11 are input to the control device 5.

【0038】上記制御装置5では、画像認識部21にお
いて、上記CCDカメラ4で撮像した自車両の進入方向
の環境の1組のステレオ画像対に対し、対応する位置の
ずれ量から三角測量の原理によって画像全体に渡る距離
情報を求める処理を行なって、三次元の距離分布を表す
距離画像を生成して道路形状・障害物認識部22に出力
する。
In the control device 5, in the image recognition section 21, the principle of triangulation is calculated based on the amount of displacement of the corresponding position for a pair of stereo images of the environment in the approach direction of the vehicle captured by the CCD camera 4. A distance image representing a three-dimensional distance distribution is generated and output to the road shape / obstacle recognition unit 22.

【0039】上記道路形状・障害物認識部22は、上記
画像認識部21からの三次元画像データから上記自車両
1の進入方向の道路の形状を認識し、同時に障害物を三
次元的に認識して道路・障害物位置算出記憶部23に出
力する。
The road shape / obstacle recognition unit 22 recognizes the shape of the road in the approach direction of the vehicle 1 from the three-dimensional image data from the image recognition unit 21, and at the same time recognizes the obstacle three-dimensionally. And outputs it to the road / obstacle position calculation storage unit 23.

【0040】上記道路・障害物位置算出記憶部23は、
上記道路形状・障害物認識部22で抽出した道路形状と
障害物を予め設定しておいた三次元座標上の位置を算出
して記憶する。また、上記道路・障害物位置算出記憶部
23には、前記超音波センサ6,7からの信号が入力さ
れ、道路面より下にある側溝あるいは縁石等の位置が正
確に三次元座標上に表現される。
The road / obstacle position calculation storage unit 23
The road shape and obstacles extracted by the road shape / obstacle recognition unit 22 are calculated and stored in a predetermined three-dimensional coordinate position. The signals from the ultrasonic sensors 6 and 7 are input to the road / obstacle position calculation storage unit 23, and the position of a gutter or curb below the road surface is accurately represented on three-dimensional coordinates. Is done.

【0041】次いで、自車両位置算出記憶部24は、上
記道路・障害物位置算出記憶部23で道路形状と障害物
の位置を設定した三次元座標上に、上記道路・障害物と
相対的な位置にある自車両1の位置を算出して記憶し、
この道路形状と障害物の位置および自車両1の位置が設
定された三次元座標のデータは自車両予想進路算出部2
5に出力される。
Next, the own vehicle position calculation storage unit 24 stores, on the three-dimensional coordinates in which the road shape and the obstacle position are set by the road / obstacle position calculation storage unit 23, relative to the road / obstacle. Calculate and store the position of the host vehicle 1 at the position,
The data of the three-dimensional coordinates in which the road shape, the position of the obstacle, and the position of the own vehicle 1 are set are calculated by the own vehicle predicted course calculating unit 2.
5 is output.

【0042】上記自車両予想進路算出部25は、前記車
速センサ9からの車両速度、前記ハンドル角センサ10
からのハンドル角、上記自車両位置算出記憶部24から
の自車両1と道路・障害物の位置を示す三次元座標のデ
ータを基に、この三次元座標上で自車両1がどうのよう
な進路で走行されるかを、自車両1の車両諸元で予め設
定しておいた車両の運動方程式等により求めて予想し、
この求めた予想進路は接触判定部26に出力される。
The host vehicle predicted course calculating section 25 calculates the vehicle speed from the vehicle speed sensor 9 and the steering wheel angle sensor 10.
Based on the three-dimensional coordinate data indicating the steering wheel angle from the vehicle position and the position of the vehicle 1 and the road / obstacle from the vehicle position calculation storage unit 24 described above. It is determined whether or not the vehicle is traveling on the course by calculating the equation of motion of the vehicle set in advance in the vehicle specifications of the own vehicle 1, and
The obtained predicted course is output to the contact determination unit 26.

【0043】上記接触判定部26は、上記自車両予想進
路算出部25で設定した上記自車両1の予想進路と前記
ミラースイッチ11からの信号が入力され、 上記自車
両1の予想進路と上記ミラースイッチ11からの信号で
選択される予め記憶しておいた自車両1の車両外形(ド
アミラー1aが復帰された状態か、あるいは格納された
状態か)に基づき障害物と自車両1との平面上での接触
可能性と高さ方向での接触可能性を求め、これらの結果
で上記障害物と自車両1との接触を判定し、必要に応じ
て上記警報発生部12、音声発生部13、および第1の
表示部14と第2の表示部15に出力してドライバに報
知する。
The contact determination section 26 receives the signal from the mirror switch 11 and the predicted path of the host vehicle 1 set by the predicted path calculation section 25 for the own vehicle, and inputs the predicted path of the host vehicle 1 and the mirror. On the plane between the obstacle and the host vehicle 1 based on the stored vehicle outer shape of the host vehicle 1 (whether the door mirror 1a is restored or stored) selected by a signal from the switch 11. The contact possibility in the height direction and the contact possibility in the above are determined, and the contact between the obstacle and the host vehicle 1 is determined based on these results, and the alarm generation unit 12, the sound generation unit 13, The information is output to the first display unit 14 and the second display unit 15 to notify the driver.

【0044】すなわち、接触判定は自車両1の車両外形
に基づき予め設定しておいた接触判定外形線を上記予想
進路に従って移動させ、この移動した接触外形線と障害
物とが接触するか否かで行われる。
That is, the contact determination is performed by moving a contact determination contour set in advance based on the vehicle contour of the vehicle 1 in accordance with the predicted course, and determining whether or not the moved contact contour contacts an obstacle. Done in

【0045】上記接触判定部26で、例えば平面上での
ドアミラー1aを復帰した自車両1の障害物との接触可
能性を判定する場合には、図8に示すように、接触判定
外形線31bを上記自車両予想進路算出部25で求めた
予想進路に基づき内輪差等を考慮して移動させ、移動さ
せるエリア内に障害物が有る場合は平面上の判定では接
触すると結果を出力する。また、平面上でのドアミラー
1aを格納した自車両1の障害物との接触可能性の判
定、高さ方向でのドアミラー1aを復帰・格納した自車
両1の障害物との接触可能性の判定も同様に行われる。
When the contact judging section 26 judges, for example, the possibility of contact with the obstacle of the host vehicle 1 after returning the door mirror 1a on a plane, as shown in FIG. Is moved in consideration of the inner wheel difference and the like based on the predicted route obtained by the predicted vehicle route calculating section 25, and if there is an obstacle in the area to be moved, the result is output if it is touched in the determination on the plane. Also, the possibility of contact with the obstacle of the own vehicle 1 storing the door mirror 1a on the plane and the possibility of contact with the obstacle of the own vehicle 1 returning and storing the door mirror 1a in the height direction are determined. Is performed similarly.

【0046】そして、上記接触判定部26での平面上で
の接触可能性の判定結果は前記第1の表示部14の画面
上に表示され、高さ方向での接触可能性の判定結果は前
記第2の表示部15の画面上に表示される。ここで、例
えば図5に示すように、平面上での接触可能性の判定結
果が接触の可能性が有りとなっても、高さ方向での接触
可能性の判定結果は接触の可能性が無いとの結果となっ
ているため、上記接触判定部26は自車両1と障害物と
は接触しないと判定する。すなわち、上記接触判定部2
6は、平面上での接触可能性の判定結果と高さ方向での
接触可能性の判定結果の両方が接触可能性有りの場合に
自車両1と障害物とは接触する可能性ありと判定し、上
記警報発生部12および上記音声発生部13に対して出
力してドライバに報知する。
The result of the determination of the possibility of contact on a plane by the contact determination unit 26 is displayed on the screen of the first display unit 14, and the result of determination of the possibility of contact in the height direction is It is displayed on the screen of the second display unit 15. Here, for example, as shown in FIG. 5, even if the determination result of the possibility of contact on the plane indicates that there is a possibility of contact, the result of determination of the possibility of contact in the height direction indicates that the possibility of contact is high. Since there is no result, the contact determination unit 26 determines that the own vehicle 1 does not contact the obstacle. That is, the contact determination unit 2
6 indicates that there is a possibility that the own vehicle 1 and the obstacle may come into contact with each other when both the determination result of the possibility of contact on the plane and the determination result of the possibility of contact in the height direction indicate that there is a possibility of contact. Then, it outputs to the alarm generation unit 12 and the voice generation unit 13 to notify the driver.

【0047】上記警報発生部12、音声発生部13、お
よび第1の表示部14と第2の表示部15は、上記制御
装置5の上記接触判定部26によってドライバが効果的
に認識できるように作動される。
The alarm generation unit 12, the voice generation unit 13, and the first display unit 14 and the second display unit 15 are arranged so that the driver can be effectively recognized by the contact determination unit 26 of the control device 5. Activated.

【0048】すなわち、自車両1と障害物との状況は、
常時、上記第1の表示部14と第2の表示部15で提供
され、所定の距離から障害物へ向けて走行する場合に上
記音声発生部13による予め用意しておいた所定の音声
による情報提供を行ない、障害物との接触可能性が所定
に高まった場合に上記警報発生部12によるブザー音等
による警報が行われる。
That is, the situation between the vehicle 1 and the obstacle is as follows.
Always provided by the first display unit 14 and the second display unit 15 and provided by a predetermined voice prepared by the voice generation unit 13 when traveling from a predetermined distance to an obstacle. The warning is provided by a warning sound or the like by the warning generator 12 when the possibility of contact with an obstacle is increased to a predetermined level.

【0049】このように本発明の実施の形態によれば、
自車両の障害物との接触可能性を、平面上での接触可能
性と高さ方向での接触可能性を求め、これら両方の結果
で判定するので、狭路進入の可否の判定や障害物との接
触判定が正確に行える。
As described above, according to the embodiment of the present invention,
The possibility of contact with the obstacle of the host vehicle is determined by determining the possibility of contact on a plane and the possibility of contact in the height direction, and judgment is made based on both of these results. Can be accurately determined.

【0050】また、超音波センサで道路面より下の障害
物の情報も取り込んで接触判定するので、進入しようと
する通路にある略全ての障害物について対応でき、一層
正確な判定ができる。
Further, since information on obstacles below the road surface is taken in by the ultrasonic sensor, contact determination is made, so that almost all obstacles in the path to be entered can be dealt with, and more accurate determination can be made.

【0051】さらに、ドアミラーの復帰・格納状態で変
化する車両外形も考慮することで、例えばドアミラーを
格納すれば通行できる場合等も確実にドライバに報知で
きる。
Further, by taking into account the vehicle outer shape that changes when the door mirror is returned and retracted, it is possible to reliably inform the driver of, for example, a case where the door mirror can be retracted if the door mirror is retracted.

【0052】また、モニターの画面に、障害物と上記自
車両との状態を、俯瞰図と自車両後方からの投影図で視
覚的に示すことにより、ドライバが現在の状態を素早く
感覚的に理解して判断しやすい。
Also, the state of the obstacle and the host vehicle is visually shown on the monitor screen in a bird's-eye view and a projection view from the rear of the host vehicle, so that the driver can quickly and intuitively understand the current state. Easy to judge.

【0053】さらに、自車両と障害物との状況は、常
時、上記第1の表示部と第2の表示部で提供され、所定
の距離から障害物へ向けて走行する場合に音声発生部に
よる予め用意しておいた所定の音声による情報提供を行
ない、障害物との接触可能性が所定に高まった場合に警
報発生部によるブザー音等による警報が行われるため、
ドライバが現在の状態をいち早く理解して判断しやす
い。
Further, the status of the own vehicle and the obstacle is always provided on the first display unit and the second display unit, and when the vehicle runs from a predetermined distance toward the obstacle, the sound generation unit outputs the information. Information is provided by a predetermined voice prepared in advance, and when the possibility of contact with an obstacle increases to a predetermined level, an alarm such as a buzzer sound by an alarm generation unit is issued,
It is easy for the driver to quickly understand the current state and make a judgment.

【0054】また、自車両と障害物との接触判定に用い
る接触判定外形線は、ドアミラーの形状とバンパーの形
状を考慮して形成されるため、ドライバの車両感覚に合
った接触判定が行われる。そして、この接触判定外形線
を内輪差等考慮して移動して接触判定に用いるため、正
確な判定が行える。
Further, since the contact determination outline used for determining the contact between the host vehicle and the obstacle is formed in consideration of the shape of the door mirror and the shape of the bumper, the contact determination matching the driver's sense of the vehicle is performed. . Then, the contact determination outline is moved in consideration of the inner ring difference or the like and used for the contact determination, so that accurate determination can be performed.

【0055】尚、本発明の実施の形態では、道路面より
下の障害物の情報を超音波センサで得るようにしている
が、他の例えばCCDカメラ等で情報を得るようにして
も良い。
In the embodiment of the present invention, the information on the obstacle below the road surface is obtained by the ultrasonic sensor. However, the information may be obtained by another device such as a CCD camera.

【0056】また、本発明の実施の形態では、車両が前
進する場合を制御する例を示しているが、車両が後進す
る場合も車両の進行方向(後方)をCCDカメラで検出
して同様に制御できる。
Further, in the embodiment of the present invention, an example in which the case where the vehicle moves forward is controlled, but when the vehicle moves backward, the traveling direction (rearward) of the vehicle is detected by the CCD camera and similarly. Can control.

【0057】[0057]

【発明の効果】以上説明したように本発明によれば、自
車両の障害物との接触可能性を、平面上での接触可能性
と高さ方向での接触可能性を求め、これら両方の結果で
判定するので、狭路進入の可否の判定が正確に行え、確
実に障害物との接触を防止してドライバの運転を有効に
支援することができる。また、道路面より下の障害物の
情報をも検出して接触判定することにより、道路面より
下の側溝、縁石等の位置認識も正確に行えて信頼性が向
上する。さらに、車両の予め設定しておいた各状態に応
じて対応する車両外形情報で接触判定を行うことで、車
両の様々な状態に合った適切な車両外形情報で接触の判
定が行える。また、障害物と自車両との状態を、俯瞰図
と自車両後方からの投影図の少なくとも一方の視覚的表
示部と、音声で報知する音声発生部と、警報音で報知す
る警報音発生部の少なくとも一つで形成すれば、ドライ
バに対しての報知が素早く行え、ドライバも状況認識が
的確に行える。
As described above, according to the present invention, the possibility of contact with an obstacle of the host vehicle is determined based on the possibility of contact on a plane and the possibility of contact in the height direction. Since the determination is made based on the result, it is possible to accurately determine whether or not the vehicle can enter a narrow road, and it is possible to surely prevent a contact with an obstacle and effectively support the driving of the driver. In addition, by detecting information on obstacles below the road surface and making contact determination, the position of a gutter, a curb and the like below the road surface can be accurately recognized, and reliability is improved. Further, by performing the contact determination based on the vehicle outline information corresponding to each preset state of the vehicle, the contact can be determined based on the vehicle outline information appropriate for various states of the vehicle. Further, at least one of a bird's-eye view and a projected view from behind the host vehicle, a visual display unit, a sound generating unit notifying by sound, and an alarm sound generating unit notifying by an alarm sound the state of the obstacle and the host vehicle. If at least one of them is formed, the driver can be quickly notified, and the driver can accurately recognize the situation.

【図面の簡単な説明】[Brief description of the drawings]

【図1】車両用運転支援装置の機能ブロック図FIG. 1 is a functional block diagram of a vehicle driving assistance device.

【図2】車両用運転支援装置の概略構成図FIG. 2 is a schematic configuration diagram of a vehicle driving assistance device.

【図3】超音波センサの取り付け説明図FIG. 3 is an explanatory view of mounting an ultrasonic sensor.

【図4】側溝のある状況での走行の一例を示す説明図FIG. 4 is an explanatory diagram showing an example of traveling in a situation with a gutter.

【図5】表示部の表示の説明図FIG. 5 is an explanatory diagram of a display on a display unit.

【図6】ドアミラーを復帰させた状態での接触判定外形
線の説明図
FIG. 6 is an explanatory diagram of a contact determination outline when the door mirror is returned.

【図7】ドアミラーを格納した状態での接触判定外形線
の説明図
FIG. 7 is an explanatory diagram of a contact determination outline when the door mirror is stored.

【図8】実際の予想進路に基づく接触判定外形線の説明
FIG. 8 is an explanatory diagram of a contact determination outline based on an actual predicted course;

【図9】トラックの横を通過する状況での走行の一例を
示す説明図
FIG. 9 is an explanatory diagram showing an example of traveling in a situation of passing the side of a truck.

【符号の説明】[Explanation of symbols]

1 自車両 1a 自車両のドアミラー 2 トラック(障害物) 2a トラックのドアミラー 3 車両用運転支援装置 4 CCDカメラ 5 制御装置 6 左右前端側超音波センサ 7 左右後端側超音波センサ 8 側溝 9 車速センサ 10 ハンドル角センサ 11 ミラースイッチ 12 警報発生部(報知手段) 13 音声発生部(報知手段) 14 第1の表示部(報知手段) 15 第2の表示部(報知手段) 21 画像認識部(画像認識手段) 22 道路形状・障害物認識部(道路形状・障害物認
識手段) 23 道路・障害物位置算出記憶部(道路・障害物位
置算出記憶手段) 24 自車両位置算出記憶部(自車両位置算出記憶手
段) 25 自車両予想進路算出部(自車両予想進路算出手
段) 26 接触判定部(接触判定手段)
DESCRIPTION OF SYMBOLS 1 Own vehicle 1a Own vehicle door mirror 2 Truck (obstacle) 2a Truck door mirror 3 Vehicle driving support device 4 CCD camera 5 Control device 6 Left and right front end ultrasonic sensor 7 Left and right rear end ultrasonic sensor 8 Side groove 9 Vehicle speed sensor REFERENCE SIGNS LIST 10 handle angle sensor 11 mirror switch 12 alarm generation unit (notification unit) 13 sound generation unit (notification unit) 14 first display unit (notification unit) 15 second display unit (notification unit) 21 image recognition unit (image recognition) 22) Road shape / obstacle recognition unit (road shape / obstacle recognition unit) 23 Road / obstacle position calculation storage unit (road / obstacle position calculation storage unit) 24 Own vehicle position calculation storage unit (own vehicle position calculation) Storage means) 25 own-vehicle predicted route calculation unit (own-vehicle predicted route calculation unit) 26 contact determination unit (contact determination unit)

Claims (4)

【特許請求の範囲】[Claims] 【請求項1】 自車両の進行方向の環境を距離分布の三
次元画像データとして得る画像認識手段と、上記三次元
画像データから上記自車両の進行方向の道路の形状と障
害物を三次元に認識する道路形状・障害物認識手段と、
上記認識した道路と障害物の予め設定しておいた三次元
空間での位置を算出して記憶する道路・障害物位置算出
記憶手段と、上記道路・障害物と相対的な位置にある上
記自車両の上記予め設定しておいた三次元空間での位置
を算出して記憶する自車両位置算出記憶手段と、上記自
車両の運転状態から上記自車両の上記予め設定しておい
た三次元空間での予想進路を算出する自車両予想進路算
出手段と、上記自車両の予想進路と予め記憶しておいた
車両外形に基づき上記障害物と上記自車両との平面上で
の接触可能性と高さ方向での接触可能性を求め、これら
の結果で上記障害物と上記自車両との接触を判定する接
触判定手段と、上記接触判定手段での接触判定結果に基
づき運転者に報知する報知手段とを備えたことを特徴と
する車両用運転支援装置。
An image recognizing means for obtaining an environment in a traveling direction of an own vehicle as three-dimensional image data of a distance distribution, and a three-dimensional image of a road shape and an obstacle in the traveling direction of the own vehicle from the three-dimensional image data. Road shape and obstacle recognition means to recognize,
Road / obstacle position calculation and storage means for calculating and storing the positions of the recognized roads and obstacles in a preset three-dimensional space; Own vehicle position calculation storage means for calculating and storing the position of the vehicle in the predetermined three-dimensional space; and the predetermined three-dimensional space of the own vehicle based on the driving state of the own vehicle Own vehicle predicted course calculating means for calculating the predicted course of the vehicle, and the possibility of contact between the obstacle and the self-vehicle on a plane based on the predicted course of the self-vehicle and the vehicle outline stored in advance. Contact determination means for determining the possibility of contact in the vehicle direction, and determining the contact between the obstacle and the host vehicle based on these results; and notification means for notifying the driver based on the result of the contact determination by the contact determination means. Vehicle driving support characterized by comprising: Apparatus.
【請求項2】 上記道路・障害物位置算出記憶手段は、
道路面より下の障害物の情報を検出する手段を有し、上
記予め設定しておいた三次元空間での位置を算出して記
憶することを特徴とする請求項1記載の車両用運転支援
装置。
2. The road / obstacle position calculation storage means,
2. The vehicle driving support according to claim 1, further comprising means for detecting information of an obstacle below a road surface, wherein the predetermined position in the three-dimensional space is calculated and stored. apparatus.
【請求項3】 上記接触判定手段は、上記自車両の予め
設定しておいた各状態での複数の上記車両外形の情報を
有し、上記各設定状態に応じて対応する車両外形情報で
接触の判定を行うことを特徴とする請求項1又は請求項
2記載の車両用運転支援装置。
3. The contact judging means has a plurality of information on the vehicle outer shape in each preset state of the own vehicle, and makes contact with the vehicle outer shape information corresponding to each set state. The vehicle driving support device according to claim 1 or 2, wherein the determination is made.
【請求項4】 上記報知手段は、上記障害物と上記自車
両との状態を、俯瞰図と上記自車両後方からの投影図の
少なくとも一方の視覚的表示部と、音声で報知する音声
発生部と、警報音で報知する警報音発生部の少なくとも
一つで形成したことを特徴とする請求項1,2,3のい
ずれか一つに記載の車両用運転支援装置。
4. The notifying means includes a visual display unit for at least one of a bird's-eye view and a projection view from the rear of the host vehicle, and a sound generating unit for notifying the state of the obstacle and the host vehicle by voice. The driving assistance device for a vehicle according to any one of claims 1, 2, and 3, wherein the driving assistance device is formed by at least one of an alarm sound generation unit that notifies by an alarm sound.
JP09103797A 1997-04-09 1997-04-09 Vehicle driving support device Expired - Lifetime JP3827804B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP09103797A JP3827804B2 (en) 1997-04-09 1997-04-09 Vehicle driving support device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP09103797A JP3827804B2 (en) 1997-04-09 1997-04-09 Vehicle driving support device

Publications (2)

Publication Number Publication Date
JPH10283592A true JPH10283592A (en) 1998-10-23
JP3827804B2 JP3827804B2 (en) 2006-09-27

Family

ID=14015316

Family Applications (1)

Application Number Title Priority Date Filing Date
JP09103797A Expired - Lifetime JP3827804B2 (en) 1997-04-09 1997-04-09 Vehicle driving support device

Country Status (1)

Country Link
JP (1) JP3827804B2 (en)

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001006097A (en) * 1999-06-25 2001-01-12 Fujitsu Ten Ltd Device for supporting driving for vehicle
US6411867B1 (en) 1999-10-27 2002-06-25 Fujitsu Ten Limited Vehicle driving support system, and steering angle detection device
JP2006099409A (en) * 2004-09-29 2006-04-13 Denso Corp Navigation system for avoiding deviation due to contact
JP2006131166A (en) * 2004-11-09 2006-05-25 Alpine Electronics Inc Drive supporting device
JP2006155615A (en) * 2004-11-26 2006-06-15 Daimler Chrysler Ag Lane deviation warning system by discrimination of driveway edge mark and structural boarder of driveway edge
JP2007142765A (en) * 2005-11-17 2007-06-07 Auto Network Gijutsu Kenkyusho:Kk Vehicle periphery monitoring system
US7366595B1 (en) 1999-06-25 2008-04-29 Seiko Epson Corporation Vehicle drive assist system
JP2013246483A (en) * 2012-05-23 2013-12-09 Kokusai Kogyo Co Ltd Obstacle detection method and obstacle detection system
KR20140083783A (en) * 2012-12-26 2014-07-04 현대모비스 주식회사 Assistant system for backward driving of vehicle
US9731717B2 (en) 2014-10-27 2017-08-15 Hyundai Motor Company Driver assistance apparatus and method for operating the same
JP2018081363A (en) * 2016-11-14 2018-05-24 スズキ株式会社 Driving support device
KR20180068484A (en) * 2016-12-14 2018-06-22 현대자동차주식회사 Apparatus and method for controlling narrow road driving of vehicle
WO2019049733A1 (en) * 2017-09-07 2019-03-14 日立建機株式会社 Safe driving assistance device
CN111698496A (en) * 2019-03-11 2020-09-22 株式会社三丰 Measurement result display device and storage medium
CN113071418A (en) * 2021-04-28 2021-07-06 中国第一汽车股份有限公司 Vehicle rearview mirror control method and device, electronic equipment and storage medium
CN113391368A (en) * 2021-06-30 2021-09-14 山东国瑞新能源有限公司 Road exploration method and equipment based on virtual imaging technology
US20220101022A1 (en) * 2020-09-29 2022-03-31 Rivian Ip Holdings, Llc Vehicle cliff and crevasse detection systems and methods
CN114550131A (en) * 2022-01-29 2022-05-27 北京百度网讯科技有限公司 Electronic map processing method and device, electronic equipment and storage medium
CN114537425A (en) * 2022-02-28 2022-05-27 重庆长安汽车股份有限公司 Detection early warning method and device for front deceleration strip and ditch bank during driving and vehicle
CN114771408A (en) * 2022-03-28 2022-07-22 海尔(深圳)研发有限责任公司 Top-mounted air conditioner for vehicle and control method thereof
US11804135B2 (en) 2019-11-28 2023-10-31 Mitsubishi Electric Corporation Object recognition apparatus, object recognition method, and computer readable medium

Cited By (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7366595B1 (en) 1999-06-25 2008-04-29 Seiko Epson Corporation Vehicle drive assist system
JP2001006097A (en) * 1999-06-25 2001-01-12 Fujitsu Ten Ltd Device for supporting driving for vehicle
JP4723703B2 (en) * 1999-06-25 2011-07-13 富士通テン株式会社 Vehicle driving support device
US7640108B2 (en) 1999-06-25 2009-12-29 Fujitsu Ten Limited Vehicle drive assist system
US7640107B2 (en) 1999-06-25 2009-12-29 Fujitsu Ten Limited Vehicle drive assist system
US6411867B1 (en) 1999-10-27 2002-06-25 Fujitsu Ten Limited Vehicle driving support system, and steering angle detection device
US6567726B2 (en) 1999-10-27 2003-05-20 Fujitsu Ten Limited Vehicle driving support system, and steering angle detection device
JP2006099409A (en) * 2004-09-29 2006-04-13 Denso Corp Navigation system for avoiding deviation due to contact
JP4573242B2 (en) * 2004-11-09 2010-11-04 アルパイン株式会社 Driving assistance device
JP2006131166A (en) * 2004-11-09 2006-05-25 Alpine Electronics Inc Drive supporting device
JP2006155615A (en) * 2004-11-26 2006-06-15 Daimler Chrysler Ag Lane deviation warning system by discrimination of driveway edge mark and structural boarder of driveway edge
JP2007142765A (en) * 2005-11-17 2007-06-07 Auto Network Gijutsu Kenkyusho:Kk Vehicle periphery monitoring system
JP2013246483A (en) * 2012-05-23 2013-12-09 Kokusai Kogyo Co Ltd Obstacle detection method and obstacle detection system
KR20140083783A (en) * 2012-12-26 2014-07-04 현대모비스 주식회사 Assistant system for backward driving of vehicle
US9731717B2 (en) 2014-10-27 2017-08-15 Hyundai Motor Company Driver assistance apparatus and method for operating the same
US10407060B2 (en) 2014-10-27 2019-09-10 Hyundai Motor Company Driver assistance apparatus and method for operating the same
JP2018081363A (en) * 2016-11-14 2018-05-24 スズキ株式会社 Driving support device
KR20180068484A (en) * 2016-12-14 2018-06-22 현대자동차주식회사 Apparatus and method for controlling narrow road driving of vehicle
WO2019049733A1 (en) * 2017-09-07 2019-03-14 日立建機株式会社 Safe driving assistance device
JP2019049776A (en) * 2017-09-07 2019-03-28 日立建機株式会社 Safety-drive supporting device
US10967789B2 (en) 2017-09-07 2021-04-06 Hitachi Construction Machinery Co., Ltd. Safe driving assistance device
CN111698496A (en) * 2019-03-11 2020-09-22 株式会社三丰 Measurement result display device and storage medium
US11804135B2 (en) 2019-11-28 2023-10-31 Mitsubishi Electric Corporation Object recognition apparatus, object recognition method, and computer readable medium
US20220101022A1 (en) * 2020-09-29 2022-03-31 Rivian Ip Holdings, Llc Vehicle cliff and crevasse detection systems and methods
CN113071418A (en) * 2021-04-28 2021-07-06 中国第一汽车股份有限公司 Vehicle rearview mirror control method and device, electronic equipment and storage medium
CN113391368A (en) * 2021-06-30 2021-09-14 山东国瑞新能源有限公司 Road exploration method and equipment based on virtual imaging technology
CN114550131A (en) * 2022-01-29 2022-05-27 北京百度网讯科技有限公司 Electronic map processing method and device, electronic equipment and storage medium
CN114537425A (en) * 2022-02-28 2022-05-27 重庆长安汽车股份有限公司 Detection early warning method and device for front deceleration strip and ditch bank during driving and vehicle
CN114537425B (en) * 2022-02-28 2024-05-24 重庆长安汽车股份有限公司 Detection and early warning method and device for front deceleration strip and ditch during driving and vehicle
CN114771408A (en) * 2022-03-28 2022-07-22 海尔(深圳)研发有限责任公司 Top-mounted air conditioner for vehicle and control method thereof
WO2023185183A1 (en) * 2022-03-28 2023-10-05 海尔(深圳)研发有限责任公司 Top-mounted air conditioner for vehicle and control method therefor

Also Published As

Publication number Publication date
JP3827804B2 (en) 2006-09-27

Similar Documents

Publication Publication Date Title
JP3827804B2 (en) Vehicle driving support device
JP5620472B2 (en) Camera system for use in vehicle parking
JP6115576B2 (en) Vehicle travel control device
JP4687160B2 (en) Perimeter monitoring device
CN202686359U (en) Narrow road detection device
JP3358709B2 (en) Driving support device for vehicles
US8872919B2 (en) Vehicle surrounding monitoring device
JP6120371B2 (en) Automatic parking control device and parking assist device
JP3853542B2 (en) Image processing apparatus, image processing method, and navigation apparatus
JP4847051B2 (en) Vehicle surrounding monitoring method and system
JP6808017B2 (en) Driving support device
JP2001195699A (en) Vehicle circumference monitor device and recording medium for stored with vehicle collision danger judgement processing program
JP2007233864A (en) Dead angle support information notification device and program
JP4223320B2 (en) Vehicle driving support device
JP2007226666A (en) Driving support method and driving support device
KR20210127267A (en) Vehicle and method for controlling thereof
JPH1116097A (en) Operation supporting device for vehicle
JP2020177292A (en) Vehicle remote indication system
JP2001109999A (en) Driving support device for vehicle
US11794809B1 (en) Vehicle and trailer wheel path collision detection and alert
JP2007124097A (en) Apparatus for visually recognizing surrounding of vehicle
JPH1069598A (en) Collision preventing device for vehicle
KR102199743B1 (en) Driver assistance system and method for providing blind spot image
JP2021126950A (en) Parking support device
JP2019064317A (en) Display device for vehicle

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20040318

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20060323

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20060404

A521 Written amendment

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20060601

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20060627

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20060705

R150 Certificate of patent or registration of utility model

Free format text: JAPANESE INTERMEDIATE CODE: R150

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20090714

Year of fee payment: 3

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20100714

Year of fee payment: 4

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20110714

Year of fee payment: 5

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20110714

Year of fee payment: 5

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20120714

Year of fee payment: 6

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20120714

Year of fee payment: 6

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20130714

Year of fee payment: 7

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

S531 Written request for registration of change of domicile

Free format text: JAPANESE INTERMEDIATE CODE: R313531

R350 Written notification of registration of transfer

Free format text: JAPANESE INTERMEDIATE CODE: R350

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

EXPY Cancellation because of completion of term