JP2017182768A - Collision prevention apparatus - Google Patents

Collision prevention apparatus Download PDF

Info

Publication number
JP2017182768A
JP2017182768A JP2016191731A JP2016191731A JP2017182768A JP 2017182768 A JP2017182768 A JP 2017182768A JP 2016191731 A JP2016191731 A JP 2016191731A JP 2016191731 A JP2016191731 A JP 2016191731A JP 2017182768 A JP2017182768 A JP 2017182768A
Authority
JP
Japan
Prior art keywords
pedestrian
vehicle
collision
host vehicle
lane
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
JP2016191731A
Other languages
Japanese (ja)
Other versions
JP6837708B2 (en
Inventor
陽介 栗原
Yosuke Kurihara
陽介 栗原
増田 基
Motoi Masuda
基 増田
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Daihatsu Motor Co Ltd
Original Assignee
Daihatsu Motor Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Daihatsu Motor Co Ltd filed Critical Daihatsu Motor Co Ltd
Publication of JP2017182768A publication Critical patent/JP2017182768A/en
Application granted granted Critical
Publication of JP6837708B2 publication Critical patent/JP6837708B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Abstract

PROBLEM TO BE SOLVED: To prevent an automatic brake from unnecessarily operating by inhibiting the operation of the brake in the case of a crossing pedestrian ahead of the vehicle itself having no risk of colliding with the vehicle.SOLUTION: A collision prevention apparatus 1 performs: determining, in the state of recognizing a pedestrian crossing ahead of a vehicle itself from an image taken with a camera 2, that a current point in time is operation timing for an automatic brake to be operated on the basis of a vehicular speed detected by a vehicular speed sensor 3 and a vertical distance between the vehicle itself and the pedestrian, and deriving collision prediction time until the vehicle itself collides with the pedestrian from the vehicular speed detected by the vehicular speed sensor 3 and a derived moving speed of the pedestrian in a crossing direction of the pedestrian; and determining that the vehicle itself is predicted to collide with the pedestrian within a first operation-permissible range of a given width that is smaller than an estimated travel route of the vehicle itself from a center thereof on the basis of the derived collision prediction time, and executing control to cause the automatic brake to operate, in the case of determining that a current position of the pedestrian is within a second operation-permissible range of a given distance on left and right sides relative to a center of the estimated travel route of the vehicle itself.SELECTED DRAWING: Figure 1

Description

本発明は、自車両前方の歩行者を検知して、検知した前記歩行者との衝突を防止する衝突回避装置に関する。   The present invention relates to a collision avoidance device that detects a pedestrian in front of the host vehicle and prevents a collision with the detected pedestrian.

従来、自車両前方の歩行者を検知して、歩行者の動きを予測して自車両と衝突するおそれがある場合に、検知した歩行者との衝突を防止する装置が開発されており、特に自車両前方の横断歩行者が自車両と衝突するおそれがあるかどうかを的確に判断して、衝突を未然に防止できるようにしたものが提案されている(特許文献1参照)。   Conventionally, a device has been developed that detects a pedestrian in front of the host vehicle, predicts the pedestrian's movement, and may collide with the host vehicle. There has been proposed one that can accurately determine whether or not a crossing pedestrian in front of the host vehicle may collide with the host vehicle so that the collision can be prevented in advance (see Patent Document 1).

特開2015−32154号公報(段落0010〜0018および図1〜図3参照)Japanese Patent Laying-Open No. 2015-32154 (see paragraphs 0010 to 0018 and FIGS. 1 to 3)

ところで、従来の横断歩行者との衝突防止策では、横断歩行者の速度と自車両の車速とに基づき、横断歩行者との衝突可能性を判断することが一般に行われているが、歩行者と自車両との距離が大きい場合に、歩行者が横断方向ではない別方向に移動したり、横断歩行者ではない障害物を歩行者と誤認したりするなどして、衝突を回避すべき横断歩行者ではないにも拘らず、衝突回避装置である自動ブレーキが不要に作動してドライバに違和感を与えるという問題がある。   By the way, in conventional collision prevention measures with a crossing pedestrian, it is generally performed to determine the possibility of a collision with a crossing pedestrian based on the speed of the crossing pedestrian and the vehicle speed of the own vehicle. When the distance between the vehicle and the host vehicle is large, the pedestrian may move in a different direction that is not a crossing direction, or a crossing that should avoid a collision by misidentifying an obstacle that is not a crossing pedestrian as a pedestrian. In spite of not being a pedestrian, there is a problem that the automatic brake, which is a collision avoidance device, operates unnecessarily and gives the driver a sense of incongruity.

本発明は、自車両前方の横断歩行者のうち、自車両と衝突するおそれがある歩行者を的確に検知し、衝突のおそれがない場合には自動ブレーキを作動させず、自動ブレーキの不要な作動を未然に防止できるようにすることを目的とする。   The present invention accurately detects pedestrians that may collide with the host vehicle among crossing pedestrians in front of the host vehicle, and does not activate the automatic brake when there is no risk of collision, and does not require an automatic brake. The purpose is to prevent the operation in advance.

上記した目的を達成するために、本発明の衝突回避装置は、自車両前方の歩行者を検知して、検知した前記歩行者との衝突を防止する衝突回避装置において、自車両前方のカメラ画像の画像認識処理により前方に存在する障害物が歩行者であるかどうかを認識する認識手段と、自車両の車速を検出する車速検出手段と、前記車速検出手段による検出車速および前記カメラ画像に基づき、前記自車両と前記歩行者との相対距離および前記歩行者の横断方向への移動速度を導出する導出手段と、前記障害物との衝突を回避すべき作動タイミングに達したときに自動ブレーキを作動させる制動手段と、少なくとも前記導出手段による前記相対距離から、現在時点が前記作動タイミングであると判断し、前記車速検出手段による検出車速および前記導出手段による前記移動速度から自車両の中心から車幅より小さい所定幅の範囲内において自車両と前記歩行者との衝突が予測されると判断し、前記歩行者の現在位置が自車両の自車の推定進路中心に対して所定距離内に位置すると判断したときに、前記制動手段を作動させる制御を実行する制御手段とを備えることを特徴としている(請求項1)。   In order to achieve the above-described object, a collision avoidance device according to the present invention detects a pedestrian in front of the host vehicle and prevents a collision with the detected pedestrian. Based on the recognition means for recognizing whether the obstacle present ahead is a pedestrian by the image recognition processing, the vehicle speed detection means for detecting the vehicle speed of the host vehicle, the vehicle speed detected by the vehicle speed detection means and the camera image Derivation means for deriving a relative distance between the host vehicle and the pedestrian and a moving speed of the pedestrian in the transverse direction, and automatic braking when reaching an operation timing at which a collision with the obstacle should be avoided. From the braking means to be operated and at least the relative distance by the deriving means, it is determined that the current time point is the operation timing, and the vehicle speed detected by the vehicle speed detecting means and the guide speed are determined. It is determined that a collision between the own vehicle and the pedestrian is predicted within a predetermined width smaller than the vehicle width from the center of the own vehicle based on the moving speed of the vehicle, and the current position of the pedestrian is the own vehicle of the own vehicle And control means for executing control for operating the braking means when it is determined that the position is within a predetermined distance with respect to the estimated course center.

また、前記カメラ画像に基づいて前記自車両が走行している車線である自車線の領域を検出する自車線検出手段をさらに備え、前記制御手段は、少なくとも前記導出手段による前記相対距離から、現在時点が前記作動タイミングであると判断し、前記車速検出手段による検出車速および前記導出手段による前記移動速度から自車両の中心から自車の推定進路内より小さい所定幅の範囲内において自車両と前記歩行者との衝突が予測されると判断し、前記歩行者の現在位置が自車の推定進路中心に対して所定距離内に位置すると判断し、前記歩行者が前記自車線検出手段による前記自車線の領域内に位置すると判断したときに、前記制動手段を作動させる制御を実行するようにしてもよい(請求項2)。   Further, the vehicle further comprises own lane detecting means for detecting an area of the own lane that is a lane in which the own vehicle is traveling based on the camera image, and the control means is configured to detect at least a current distance from the relative distance by the deriving means. It is determined that the time is the operation timing, and the vehicle and the vehicle are within a predetermined width smaller than the estimated vehicle speed from the center of the vehicle based on the vehicle speed detected by the vehicle speed detection unit and the moving speed by the derivation unit. It is determined that a collision with a pedestrian is predicted, the current position of the pedestrian is determined to be within a predetermined distance with respect to the estimated course center of the own vehicle, and the pedestrian is When it is determined that the vehicle is located within the lane region, control for operating the braking means may be executed (claim 2).

請求項1にかかる発明によれば、少なくとも導出手段による相対距離から、現在時点が制動手段の作動タイミングであると判断し、車速検出手段による検出車速および導出手段による歩行者の移動速度から自車両の中心から車幅より小さい所定幅の範囲内において自車両と歩行者との衝突が予測されると判断し、歩行者の現在位置が自車の推定進路中心に対して所定距離内に位置すると判断するという3つの判断条件により横断歩行者を認識することによって、これら3つの判断条件がすべて成立したときに、その歩行者は、障害物ではなく明らかに自車両の前方を横断すると判断する歩行者であって自車両との衝突が予測されるものと判断することができる。したがって、上記した3つの判断条件が成立すれば横断歩行者との衝突のおそれがあるとして、制動手段を作動させることにより、自車両の前方を横断する歩行者との衝突を未然に防止することができ、3つの判断条件の1つでも不成立であれば歩行者との衝突のおそれがないとして、制動手段を不要に作動させないようにすることができる。   According to the first aspect of the present invention, it is determined that the current time point is the operation timing of the braking means from at least the relative distance by the deriving means, and the own vehicle is determined from the vehicle speed detected by the vehicle speed detecting means and the moving speed of the pedestrian by the deriving means. If it is determined that a collision between the host vehicle and the pedestrian is predicted within a predetermined range smaller than the vehicle width from the center of the vehicle, and the current position of the pedestrian is located within a predetermined distance with respect to the estimated course center of the host vehicle By recognizing a crossing pedestrian based on the three judgment conditions of judging, when all of these three judgment conditions are satisfied, the pedestrian clearly judges to cross the front of the host vehicle instead of an obstacle. Therefore, it can be determined that a collision with the own vehicle is predicted. Therefore, if the above three determination conditions are satisfied, there is a risk of a collision with a crossing pedestrian, so that a collision with a pedestrian crossing the front of the host vehicle can be prevented by operating the braking means. If one of the three judgment conditions is not satisfied, the braking means can be prevented from being operated unnecessarily, assuming that there is no possibility of a collision with a pedestrian.

請求項2にかかる発明によれば、横断歩行者が自車線の外側において赤信号で立ち止まることが予想されるという制動手段の作動が不要な状況において、制動手段の誤作動を防止することができる。また、このような誤作動を防止することで、衝突回避装置の品質をさらに向上させることができる。   According to the second aspect of the present invention, it is possible to prevent malfunction of the braking means in a situation where the operation of the braking means that a crossing pedestrian is expected to stop at a red signal outside the own lane is unnecessary. . Moreover, the quality of the collision avoidance device can be further improved by preventing such a malfunction.

本発明に係る衝突回避装置の第1実施形態のブロック図である。1 is a block diagram of a first embodiment of a collision avoidance device according to the present invention. 第1実施形態の動作説明図である。It is operation | movement explanatory drawing of 1st Embodiment. 第1実施形態の動作説明用フローチャートである。It is a flowchart for operation | movement description of 1st Embodiment. 本発明に係る衝突回避装置の第2実施形態の動作説明図である。It is operation | movement explanatory drawing of 2nd Embodiment of the collision avoidance apparatus which concerns on this invention. 第2実施形態の動作説明用のフローチャートである。It is a flowchart for operation | movement description of 2nd Embodiment.

<第1実施形態>
本発明に係る衝突回避装置の第1実施形態について、図1ないし図3を参照して詳細に説明する。なお、本実施形態でいう前後、左右とはシートに着座した状態で見た前後、左右を意味する。
<First Embodiment>
A first embodiment of a collision avoidance device according to the present invention will be described in detail with reference to FIGS. 1 to 3. Note that front and rear and left and right in the present embodiment mean front and rear and left and right as viewed in a seated state on the seat.

図1に示すように、衝突回避装置1は、自車両前方を撮影するカメラ2と、自車両の車速を検出する車速検出手段である車速センサ3と、マイクロコンピュータ構成のECU(Electronic Control Unit)4と、ECU4の制御により自動ブレーキをかけるブレーキアクチュエータ5と、ステアリングの操舵角を検出する舵角センサ6とを備える。   As shown in FIG. 1, the collision avoidance device 1 includes a camera 2 that captures the front of the host vehicle, a vehicle speed sensor 3 that is a vehicle speed detection unit that detects the vehicle speed of the host vehicle, and an ECU (Electronic Control Unit) having a microcomputer configuration. 4, a brake actuator 5 that applies automatic braking under the control of the ECU 4, and a steering angle sensor 6 that detects the steering angle of the steering.

ECU4は、カメラ2によるカメラ画像を取り込み、例えば内蔵メモリに格納された歩行者、各種車両、道路構造物等の照合用画像パターンとカメラ画像とを照合していわゆるパターン認識による画像認識処理を行い、自車両の前方に存在する障害物が歩行者であるかどうかを認識する認識手段としての機能を有する。   The ECU 4 captures a camera image from the camera 2 and performs image recognition processing by so-called pattern recognition by collating image images for collation such as pedestrians, various vehicles and road structures stored in the built-in memory with camera images. , It has a function as a recognition means for recognizing whether an obstacle existing in front of the host vehicle is a pedestrian.

また、ECU4は、自車両の前方に存在する障害物が歩行者であると認識すると、車速センサ3による検出車速およびカメラ2によるカメラ画像に基づき、自車両と歩行者と進行方向における相対距離(以下、縦距離ともいう)および歩行者の横断方向への移動速度を導出する導出手段としての機能を有する。   When the ECU 4 recognizes that the obstacle existing in front of the host vehicle is a pedestrian, the ECU 4 determines the relative distance in the traveling direction between the host vehicle and the pedestrian based on the detected vehicle speed by the vehicle speed sensor 3 and the camera image by the camera 2. (Hereinafter also referred to as a longitudinal distance) and a function as a derivation means for deriving the moving speed of the pedestrian in the transverse direction.

そして、自車両と歩行者等の障害物との縦距離が、所定の減速度で自車両を制動した(ブレーキをかけた)ときに衝突を回避可能な限界である衝突を回避すべき作動タイミングに達したときにブレーキアクチュエータ5を制御して自動ブレーキを作動可能にする機能を有し、係るECU4の機能およびブレーキアクチュエータ5により、本発明における制動手段が構成される。   The operation timing at which the vertical distance between the host vehicle and an obstacle such as a pedestrian is the limit at which the host vehicle can be avoided when the host vehicle is braked (brake) at a predetermined deceleration. The brake actuator 5 is controlled to enable automatic braking, and the function of the ECU 4 and the brake actuator 5 constitute the braking means in the present invention.

ところで、ECU4は不要な自動ブレーキを作動させないようにするために、以下のような制御を行う。すなわち、図2に示すように、カメラ2によるカメラ画像から自車両Cの前方を横断する歩行者Pを認識した状態で、(1)車速センサ3による検出車速、自車両Cと歩行者Pとの縦距離のほか、検出車速から算出される自車両Cの現在加減速度および自車両Cと歩行者Pとの相対速度などから、現在時点が自動ブレーキを作動させるべき作動タイミングであると判断し、かつ、(2)自車両Cと歩行者Pとの縦距離、自車両Cと歩行者Pとの相対速度などから自車両Cが歩行者Pと衝突するまでの衝突予測時間(TTC)を導出し、導出した衝突予測時間(TTC)に基づき、図2に示す自車両Cの中心から車幅より小さい所定幅である第1の作動許可範囲Wの範囲内において自車両Cと歩行者Pとの衝突が予測されると判断し、かつ、(3)歩行者Pの現在位置が、図2に示す自車両Cの自車の推定進路内Lよりも外側であって左右に所定距離である第2の作動許可範囲D内に位置すると判断したときに、自車両Cの前方を横断する歩行者Pが、図2に現在位置、TTC、横方向速度などから導出する×印で示す衝突予測位置で自車両Cと衝突するおそれがあるとして、自動ブレーキを作動させる制御を実行するようになっている。   By the way, the ECU 4 performs the following control in order not to activate an unnecessary automatic brake. That is, as shown in FIG. 2, in the state where the pedestrian P crossing the front of the host vehicle C is recognized from the camera image of the camera 2, (1) the vehicle speed detected by the vehicle speed sensor 3, the host vehicle C and the pedestrian P In addition to the vertical distance, the current acceleration / deceleration of the own vehicle C calculated from the detected vehicle speed and the relative speed between the own vehicle C and the pedestrian P are determined to be the operation timing at which the automatic brake should be activated. And (2) a predicted collision time (TTC) until the own vehicle C collides with the pedestrian P from the vertical distance between the own vehicle C and the pedestrian P, the relative speed between the own vehicle C and the pedestrian P, and the like. Based on the derived predicted collision time (TTC), the own vehicle C and the pedestrian P within the first operation permission range W that is a predetermined width smaller than the vehicle width from the center of the own vehicle C shown in FIG. And that the collision is predicted, and ( ) When it is determined that the current position of the pedestrian P is located outside the estimated course L of the host vehicle C of the host vehicle C shown in FIG. In addition, it is assumed that a pedestrian P crossing the front of the host vehicle C may collide with the host vehicle C at a predicted collision position indicated by an X in FIG. 2 derived from the current position, TTC, lateral speed, and the like. Control for operating the brake is executed.

このように、ECU4により、自動ブレーキを作動させる制御を実行すべき上記(1)〜(3)の3つの判断条件がすべて成立したときに、障害物ではなく明らかに自車両の前方を横断すると判断する歩行者であって自車両との衝突が予測されるものと判断することができ、3つの判断条件が成立すれば横断歩行者との衝突のおそれがあるとして、ブレーキアクチュエータ5を制御して自動ブレーキを作動させて、自車両の前方を横断する歩行者との衝突を未然に防止する一方、3つの判断条件の1つでも不成立であれば歩行者との衝突のおそれがないとして、ブレーキアクチュエータ5を非作動に制御して自動ブレーキを不要に作動させない。   As described above, when all of the above three determination conditions (1) to (3) that should execute the control for operating the automatic brake are satisfied, the ECU 4 clearly crosses the front of the host vehicle instead of the obstacle. It can be determined that a collision with the own vehicle is predicted, and if the three determination conditions are satisfied, the brake actuator 5 is controlled so that there is a possibility of a collision with a crossing pedestrian. The automatic brake is activated to prevent a collision with a pedestrian crossing the front of the vehicle, while if any one of the three judgment conditions is not established, there is no possibility of a collision with the pedestrian. The brake actuator 5 is controlled to be inactive so that the automatic brake is not operated unnecessarily.

このとき、上記した(3)の判断条件の成否を判断する際の基準となる自車両の自車の推定進路中心CLに対して左右に設定すべき第2の作動許可範囲Dは、自車両Cと歩行者Pとの縦距離、横断する歩行者Pの横移動速度に応じて可変するのが望ましい。例えば、歩行者Pの横移動速度が速いときには、或いは、自車両Cと歩行者Pとの縦距離が長いときには、歩行者Pは自車両Cの中心前方における衝突予測位置に早く到達する可能性があることから、衝突回避対象となる歩行者Pの検知領域を広げるために第2の作動許可範囲Dを広くする。一方、歩行者Pの横移動速度が遅いときには、或いは、自車両Cと歩行者Pとの縦距離が短いときには、歩行者Pが自車両Cの中心前方における衝突予測位置に到達するのが遅くなるため、衝突回避対象となる歩行者Pの検知領域を広げる必要がない。なお、歩行者Pの移動速度、縦距離に対して、第2の作動許可範囲Dをどのように設定すればよいかを予め実験的に検証しておき、歩行者のPの移動速度、縦距離と作動許可範囲Dとの関係をマップ化してECU4の内蔵メモリ等に記憶させておき、現在の歩行者Pの移動速度、縦距離に対する最適な作動許可範囲Dを読み出して上記した(3)の判断条件の比較基準にすればよい。   At this time, the second operation permission range D to be set to the left and right with respect to the estimated course center CL of the own vehicle as a reference when judging whether or not the judgment condition (3) described above is satisfied is the own vehicle It is desirable to vary according to the vertical distance between C and the pedestrian P and the lateral movement speed of the pedestrian P crossing. For example, when the lateral movement speed of the pedestrian P is fast, or when the vertical distance between the host vehicle C and the pedestrian P is long, the pedestrian P may reach the predicted collision position in front of the center of the host vehicle C early. Therefore, the second operation permission range D is widened in order to widen the detection area of the pedestrian P that is a collision avoidance target. On the other hand, when the lateral movement speed of the pedestrian P is slow, or when the vertical distance between the host vehicle C and the pedestrian P is short, the pedestrian P is slow to reach the predicted collision position in front of the center of the host vehicle C. Therefore, it is not necessary to expand the detection area of the pedestrian P that is a collision avoidance target. In addition, it is verified experimentally beforehand how to set the second operation permission range D with respect to the moving speed and the vertical distance of the pedestrian P, and the moving speed and the vertical length of the pedestrian P are verified. The relationship between the distance and the operation permission range D is mapped and stored in the built-in memory of the ECU 4, etc., and the optimum operation permission range D with respect to the current moving speed and vertical distance of the pedestrian P is read and described above (3) It may be used as a reference for comparison of the determination conditions.

次に、ECU4の制御動作について、図3のフローチャートを参照して説明する。いま、図3に示すように、ECU4により、上記したようなカメラ画像に基づく画像認識処理により、自車両の前方に存在する障害物が歩行者であるかどうかの判定を行い(ステップS1)、この判定結果がYESであれば、上記した(1)の判断条件に関する現在時点が自動ブレーキを作動させるべき作動タイミングであるかどうか判定する(ステップS2)。   Next, the control operation of the ECU 4 will be described with reference to the flowchart of FIG. Now, as shown in FIG. 3, the ECU 4 determines whether or not the obstacle present in front of the host vehicle is a pedestrian by the image recognition process based on the camera image as described above (step S1). If this determination result is YES, it is determined whether or not the current time point regarding the determination condition (1) described above is the operation timing at which the automatic brake should be operated (step S2).

そして、ステップS2の判定結果がYESであれば、上記した(1)の判断条件が成立したとして次のステップS3に移行し、導出した衝突予測時間(TTC)から、上記した(2)の判断条件に関する自車両の中心から車幅より小さい所定幅である第1の作動許可範囲W(図2参照)の範囲内において自車両と歩行者との衝突が予測されるかどうかの判定がなされる(ステップS3)。   If the determination result in step S2 is YES, the process proceeds to the next step S3 on the assumption that the above-described determination condition (1) is satisfied, and the above-described determination (2) is made from the derived collision prediction time (TTC). It is determined whether or not a collision between the host vehicle and a pedestrian is predicted within a first operation permission range W (see FIG. 2) that is a predetermined width smaller than the vehicle width from the center of the host vehicle regarding the conditions. (Step S3).

このステップS3の判定結果がYESであれば、上記した(2)の判断条件も成立したとして次のステップS4に移行し、上記した(3)の判断条件に関する自車両の左右所定距離の第2の作動許可範囲D(図2参照)内に歩行者が位置するかどうかの判定がなされ(ステップS4)、この判定結果がYESであれば、上記した(3)の判断条件が成立して、(1)〜(3)の3つの判断条件すべてが成立し、図2中の×印で示す衝突予測位置で自車両と横断歩行者が衝突するおそれがあると判断されたとして次のステップS5に移行する。   If the determination result in step S3 is YES, it is determined that the above-described determination condition (2) is also established, and the process proceeds to the next step S4. It is determined whether or not the pedestrian is located within the permitted operation range D (see FIG. 2) (step S4). If the determination result is YES, the determination condition of (3) described above is satisfied, If all three judgment conditions (1) to (3) are satisfied and it is judged that there is a possibility that the host vehicle and the crossing pedestrian may collide at the collision predicted position indicated by the cross in FIG. Migrate to

そして、ステップS4の判定結果がYESであれば、ステップS5において自動ブレーキ作動対象物と判断され(ステップS5)、ステップS4の判定結果がNOであれば、ステップS1,S2,S3の判定結果がNOの場合と共に、自動ブレーキ作動非対象物と判断され(ステップS6)、その後、車速センサ3による車速、方向指示器、シフトレンジなどの各種スイッチ情報、舵角センサ6による操舵角、他システムの作動状態などの車両側条件が確認されて車両側作動条件を満たしているかどうかの判定がなされ(ステップS7)、この判定結果がYESであれば自動ブレーキが作動され(ステップS8)、判定結果がNOであれば自動ブレーキは非作動となり(ステップS9)、その後動作は終了する。   And if the determination result of step S4 is YES, it will be judged in step S5 that it is an automatic brake actuating object (step S5), and if the determination result of step S4 is NO, the determination result of step S1, S2, S3 will be. Together with the case of NO, it is determined that the vehicle is not subject to automatic brake operation (step S6). Thereafter, various switch information such as the vehicle speed by the vehicle speed sensor 3, the direction indicator, the shift range, the steering angle by the steering angle sensor 6, the other system The vehicle side conditions such as the operating state are confirmed to determine whether or not the vehicle side operating conditions are satisfied (step S7). If this determination result is YES, the automatic brake is operated (step S8), and the determination result is If NO, the automatic brake is deactivated (step S9), and then the operation ends.

したがって、上記した第1実施形態によれば、上記した(1)〜(3)の3つの判断条件が成立すれば横断歩行者との衝突のおそれがあるとして、自動ブレーキを作動させることにより、自車両の前方を横断する歩行者との衝突を未然に防止することができる。   Therefore, according to the first embodiment described above, if the three determination conditions (1) to (3) described above are satisfied, it is assumed that there is a risk of a collision with a crossing pedestrian, and by operating the automatic brake, A collision with a pedestrian crossing the front of the host vehicle can be prevented in advance.

さらに、3つの判断条件の1つでも不成立であれば歩行者との衝突のおそれがないとして、制動手段を不要に作動させないようにすることができ、例えば歩行者が図2に示す第2の作動許可範囲Dの範囲よりも外側にいる場合に、当該歩行者との衝突のおそれがないにも拘らず自動ブレーキがむやみに作動してドライバが煩わしく感じることを防止できる。   Furthermore, if any one of the three judgment conditions is not satisfied, the braking means can be prevented from being operated unnecessarily because there is no risk of a collision with the pedestrian. When the vehicle is outside the operation permission range D, it is possible to prevent the driver from feeling uncomfortable due to the automatic brake being operated unnecessarily even though there is no risk of collision with the pedestrian.

また、上記した(3)の判断条件の基準となる自車両の自車の推定進路中心CLに対して左右に設定する所定距離の第2の作動許可範囲Dを、自車両Cと歩行者Pとの縦距離、横断する歩行者Pの移動速度に応じて可変するようにしたため、移動速度、縦距離の異なる横断歩行車に対する自動ブレーキの作動条件を絞ることができ、衝突回避の精度のいっそうの向上を図ることが可能になる。   In addition, the second operation permission range D of a predetermined distance set to the left and right with respect to the estimated course center CL of the own vehicle, which is the criterion of the determination condition of (3) described above, is defined as the own vehicle C and the pedestrian P. Because it can be changed according to the vertical distance and the moving speed of the pedestrian P crossing, it is possible to narrow down the automatic brake operating conditions for crossing walking cars with different moving speeds and vertical distances, and to improve the accuracy of collision avoidance Can be improved.

<第2実施形態>
上記した第1実施形態では、ECU4は、不要な自動ブレーキの作動を回避するために、(1)〜(3)の判断条件を用いたが、さらに条件を付加して第2実施形態としてもよい。
Second Embodiment
In the first embodiment described above, the ECU 4 uses the determination conditions (1) to (3) in order to avoid unnecessary automatic brake operation. Good.

ECU4が、カメラ画像から前方に歩行者を認識し、(1)〜(3)の判断条件のすべてを満たしていた場合であっても、自動ブレーキの作動が不要の場合がある。例えば、自車両の前方に横断歩行者が検出された場合であっても、その歩行者の進行方向の信号が赤信号であった場合は、自車両の走行する車線(自車線)を横断せずに、停止する蓋然性が高い。このような状況で自動ブレーキを作動させるのはお節介となる可能性が高い。   Even when the ECU 4 recognizes a pedestrian forward from the camera image and satisfies all the determination conditions (1) to (3), the automatic brake operation may not be necessary. For example, even if a crossing pedestrian is detected in front of the host vehicle, if the pedestrian's traveling direction signal is a red signal, the lane (own lane) in which the host vehicle travels must be crossed. Without stopping, there is a high probability of stopping. It is highly likely that operating the automatic brake in this situation will be a nuisance.

そこで、図4に示すように、自車両が進行している車線(自車線)の領域(範囲)と横断歩行者との位置関係を判断条件に加える。自車線の範囲の検出は、ECU4がカメラ2によるカメラ画像を取り込み、自車両の進行方向の左右両側で、自車両の進行方向と平行方向に引かれた白線7を検出する。ECU4による白線7の検出方法としては、例えば、取り込んだカメラ画像を2値化した上で、エッジ検出処理を行うことにより検出する。このとき、進行方向の左右両側で白線7を検出できた場合は、これらの白線7の間隔(自車線の幅LW)が、自車両の車幅よりも広い一定幅である場合、ECU4は、当該両白線7が自車線を規定する白線7であって、両白線7に挟まれた領域を自車線の領域として検出する。このように、ECU4が、カメラ画像に基づいて自車線の領域を検出する機能が、本発明の自車線検出手段に相当する。   Therefore, as shown in FIG. 4, the positional relationship between the area (range) of the lane (own lane) in which the host vehicle is traveling and the crossing pedestrian is added to the determination condition. In the detection of the range of the own lane, the ECU 4 captures the camera image from the camera 2 and detects the white line 7 drawn in the direction parallel to the traveling direction of the host vehicle on both the left and right sides of the traveling direction of the host vehicle. As a method for detecting the white line 7 by the ECU 4, for example, the detected camera image is binarized and then detected by performing edge detection processing. At this time, if the white lines 7 can be detected on both the left and right sides in the traveling direction, if the distance between the white lines 7 (the width LW of the own lane) is a certain width wider than the vehicle width of the own vehicle, the ECU 4 The both white lines 7 are white lines 7 defining the own lane, and an area sandwiched between the two white lines 7 is detected as an area of the own lane. Thus, the function in which the ECU 4 detects the area of the own lane based on the camera image corresponds to the own lane detecting means of the present invention.

そして、ECU4は、(1)〜(3)の判断条件のすべてを満たした際、(4)当該横断歩行者が自車線の領域内にいると判断した場合に、自動ブレーキを作動させる制御を実行し、自車線の領域外にいた場合は、自動ブレーキを作動させる制御を実行しない(後述する未指定走行モード)。なお、ECU4は、自車線の認定ができない場合は、(1)〜(3)の判断条件のすべてを満たせば、自動ブレーキを作動させる制御を実行する。自車線を規定するための白線7の検出方法は、適宜変更することができる。   When the ECU 4 satisfies all of the determination conditions (1) to (3), (4) when it is determined that the crossing pedestrian is in the area of the own lane, the ECU 4 performs control for operating the automatic brake. If it is executed and the vehicle is outside the area of the own lane, the control for operating the automatic brake is not executed (undesignated travel mode described later). Note that if the vehicle lane cannot be recognized, the ECU 4 executes control for operating the automatic brake if all of the determination conditions (1) to (3) are satisfied. The detection method of the white line 7 for defining the own lane can be changed as appropriate.

なお、左側走行の場合は、判断条件(1)〜(3)のすべてを満たす横断歩行者が赤信号で停止するのは、自車両進行方向の左側であるため、左側走行の場合は、自車線の左側に位置する歩行者と左側の白線7の位置関係を比較して、(4)の判断条件の成否を判断してもよい。一方、右側走行の場合は、自車線の右側に位置する歩行者と右側の白線7の位置関係を比較して、(4)の判断条件の成否を判断してもよい。また、左側走行モード、右側走行モード、未指定走行モードの3種類を設け、(4)の判断条件の成否の判断方法を切り換え設定できるようにしてもよい。   In the case of left-handed driving, a crossing pedestrian that satisfies all of the determination conditions (1) to (3) stops at a red light on the left side in the traveling direction of the own vehicle. The success or failure of the determination condition (4) may be determined by comparing the positional relationship between the pedestrian positioned on the left side of the lane and the white line 7 on the left side. On the other hand, in the case of right-side traveling, the success or failure of the determination condition (4) may be determined by comparing the positional relationship between the pedestrian located on the right side of the own lane and the white line 7 on the right side. Further, three types of the left traveling mode, the right traveling mode, and the non-designated traveling mode may be provided so that the determination method of success / failure of the determination condition of (4) can be switched.

次に、本例のECU4の制御動作について、図5のフローチャートを参照して説明する。なお、ステップS1〜S4の処理については、上記した第1実施形態の処理(図3のステップS1〜ステップS4参照)と同じであるため、説明を省略する。   Next, the control operation of the ECU 4 of this example will be described with reference to the flowchart of FIG. In addition, about the process of step S1-S4, since it is the same as the process (refer step S1-step S4 of FIG. 3) of above-described 1st Embodiment, description is abbreviate | omitted.

ステップS4の判定結果がYESであれば、一定幅の自車線が検出されたか否かの判定がなされ(ステップS10)、この判定がYESであれば、歩行者の現在位置が自車線の領域内にあるか否かが判定される(ステップS11)。この判定結果がYESの場合は、ステップS10で一定幅の自車線が検出されなかった場合(ステップS10でNOの場合)とともに、自動ブレーキ作動対象物と判断される(ステップS5)。一方、ステップS11の判定結果がNOであれば、ステップS1,S2,S3,S4の判定結果がNOの場合と共に、自動ブレーキ作動非対象物と判断される(ステップS6)。その後、車速センサ3による車速、方向指示器、シフトレンジなどの各種スイッチ情報、舵角センサ6による操舵角、他システムの作動状態などの車両側条件が確認されて車両側作動条件を満たしているかどうかの判定がなされ(ステップS7)、この判定結果がYESであれば自動ブレーキが作動され(ステップS8)、判定結果がNOであれば自動ブレーキは非作動となり(ステップS9)、その後動作は終了する。   If the determination result in step S4 is YES, it is determined whether or not a certain lane has been detected (step S10). If this determination is YES, the current position of the pedestrian is within the area of the own lane. Is determined (step S11). If the determination result is YES, it is determined that the vehicle is subject to automatic braking operation (step S5), along with the case where the own lane having a certain width is not detected in step S10 (NO in step S10). On the other hand, if the determination result of step S11 is NO, it is determined that the determination result of steps S1, S2, S3, and S4 is NO and the automatic brake operation non-target object (step S6). After that, vehicle condition such as vehicle speed, direction indicator, shift range and other switch information by vehicle speed sensor 3, steering angle by steering angle sensor 6, operating state of other system, etc. is confirmed and vehicle-side operating condition is satisfied If the determination result is YES, the automatic brake is activated (step S8). If the determination result is NO, the automatic brake is deactivated (step S9), and then the operation ends. To do.

したがって、上記した第2実施形態によれば、横断歩行者が自車線の外側において赤信号で立ち止まることが予想されるという自動ブレーキの作動が不要な状況において、自動ブレーキの誤作動を防止することができる。また、このような誤作動を防止することで、衝突回避装置1の品質をさらに向上させることができる。   Therefore, according to the second embodiment described above, it is possible to prevent the automatic brake from malfunctioning in a situation where the operation of the automatic brake is unnecessary, in which the crossing pedestrian is expected to stop at the red signal outside the own lane. Can do. Moreover, the quality of the collision avoidance device 1 can be further improved by preventing such a malfunction.

なお、本発明は上記した実施形態に限定されるものではなく、その趣旨を逸脱しない限りにおいて上述したもの以外に種々の変更を行なうことが可能である。   The present invention is not limited to the above-described embodiment, and various modifications other than those described above can be made without departing from the spirit of the present invention.

例えば、上記した実施形態における歩行者かどうかの認識は、上記したパターン認識による画像認識処理に限るものではない。   For example, the recognition as to whether or not the person is a pedestrian in the above-described embodiment is not limited to the above-described image recognition processing by pattern recognition.

また、車速センサ3による検出車速およびカメラ2によるカメラ画像に基づき、自車両と歩行者と進行方向における相対距離(縦距離)および歩行者の横断方向への移動速度を導出するようにしたが、相対距離(縦距離)および歩行者の横断方向への移動速度の導出方法はこれに限るものではない。   Further, based on the vehicle speed detected by the vehicle speed sensor 3 and the camera image by the camera 2, the relative distance (vertical distance) in the traveling direction between the host vehicle and the pedestrian and the moving speed of the pedestrian in the transverse direction are derived. The method for deriving the relative distance (vertical distance) and the moving speed of the pedestrian in the transverse direction is not limited to this.

また、上記した実施形態では、自車両と歩行者との縦距離、横断歩行者の移動速度に応じて、図2に示す第2の作動許可範囲Dを可変にするとしたが、自車両に配置された舵角センサにより検出される舵角に基づくステアリングの操作状態に応じて第2の作動許可範囲Dを可変するようにしてもよく、例えば左方向にステアリング操作しているときには自車両の左側の作動許可範囲Dを広げ、右方向にステアリング操作しているときには自車両の右側の作動許可範囲Dを広げるとよい。こうすると、自車両のステアリングの操作状態に応じて横断歩行車に対する自動ブレーキの作動条件を絞ることができて衝突回避の精度の向上を図ることが可能になる。   In the above-described embodiment, the second operation permission range D shown in FIG. 2 is made variable according to the vertical distance between the own vehicle and the pedestrian and the moving speed of the crossing pedestrian. The second operation permission range D may be varied in accordance with the steering operation state based on the steering angle detected by the steering angle sensor. For example, when the steering operation is performed in the left direction, the left side of the host vehicle When the steering operation is performed in the right direction, the operation permission range D on the right side of the host vehicle may be expanded. If it carries out like this, the operating condition of the automatic brake with respect to a crosswalk can be narrowed down according to the operation state of the steering of the own vehicle, and it becomes possible to aim at the improvement of the precision of collision avoidance.

1 …衝突回避装置
2 …カメラ(認識手段)
3 …車速センサ(車速検出手段)
4 …ECU(認識手段、導出手段、制動手段、制御手段、自車線検出手段)
5 …ブレーキアクチュエータ
1 ... Collision avoidance device 2 ... Camera (recognition means)
3 ... Vehicle speed sensor (vehicle speed detection means)
4 ... ECU (recognition means, derivation means, braking means, control means, own lane detection means)
5 ... Brake actuator

Claims (2)

自車両前方の歩行者を検知して、検知した前記歩行者との衝突を防止する衝突回避装置において、
自車両前方のカメラ画像の画像認識処理により前方に存在する障害物が歩行者であるかどうかを認識する認識手段と、
自車両の車速を検出する車速検出手段と、
前記車速検出手段による検出車速および前記カメラ画像に基づき、前記自車両と前記歩行者との相対距離および前記歩行者の横断方向への移動速度を導出する導出手段と、
前記障害物との衝突を回避すべき作動タイミングに達したときに自動ブレーキを作動させる制動手段と、
少なくとも前記導出手段による前記相対距離から、現在時点が前記作動タイミングであると判断し、前記車速検出手段による検出車速および前記導出手段による前記移動速度から自車両の中心から自車の推定進路内より小さい所定幅の範囲内において自車両と前記歩行者との衝突が予測されると判断し、前記歩行者の現在位置が自車の推定進路中心に対して所定距離内に位置すると判断したときに、前記制動手段を作動させる制御を実行する制御手段と
を備えることを特徴とする衝突回避装置。
In a collision avoidance device that detects a pedestrian in front of the host vehicle and prevents a collision with the detected pedestrian,
Recognizing means for recognizing whether an obstacle existing ahead is a pedestrian by image recognition processing of a camera image in front of the host vehicle;
Vehicle speed detection means for detecting the vehicle speed of the host vehicle;
Deriving means for deriving a relative distance between the host vehicle and the pedestrian and a moving speed in the transverse direction of the pedestrian based on the vehicle speed detected by the vehicle speed detecting means and the camera image;
Braking means for activating an automatic brake when an operation timing to avoid a collision with the obstacle is reached;
At least from the relative distance by the deriving means, it is determined that the current time point is the operation timing, and from the center of the own vehicle from the vehicle speed detected by the vehicle speed detecting means and the moving speed by the deriving means from the estimated course of the own vehicle When it is determined that a collision between the host vehicle and the pedestrian is predicted within a small predetermined width, and the current position of the pedestrian is determined to be within a predetermined distance with respect to the estimated course center of the host vehicle. And a control means for executing control for operating the braking means.
前記カメラ画像に基づいて前記自車両が走行している車線である自車線の領域を検出する自車線検出手段をさらに備え、
前記制御手段は、少なくとも前記導出手段による前記相対距離から、現在時点が前記作動タイミングであると判断し、前記車速検出手段による検出車速および前記導出手段による前記移動速度から自車両の中心から自車の推定進路内より小さい所定幅の範囲内において自車両と前記歩行者との衝突が予測されると判断し、前記歩行者の現在位置が自車の推定進路中心に対して所定距離内に位置すると判断し、前記歩行者が前記自車線検出手段による前記自車線の領域内に位置すると判断したときに、前記制動手段を作動させる制御を実行することを特徴とする請求項1に記載の衝突回避装置。
Further comprising own lane detection means for detecting an area of the own lane that is the lane in which the own vehicle is traveling based on the camera image;
The control means determines that the current time point is the operation timing from at least the relative distance by the deriving means, and determines from the center of the own vehicle from the vehicle speed detected by the vehicle speed detecting means and the moving speed by the deriving means. It is determined that a collision between the host vehicle and the pedestrian is predicted within a range of a predetermined width smaller than the estimated course of the vehicle, and the current position of the pedestrian is located within a predetermined distance with respect to the estimated course center of the host vehicle. 2. The collision according to claim 1, wherein when the pedestrian determines that the pedestrian is positioned within the area of the own lane by the own lane detecting means, the control for operating the braking means is executed. Avoidance device.
JP2016191731A 2016-03-23 2016-09-29 Collision avoidance device Active JP6837708B2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2016058241 2016-03-23
JP2016058241 2016-03-23

Publications (2)

Publication Number Publication Date
JP2017182768A true JP2017182768A (en) 2017-10-05
JP6837708B2 JP6837708B2 (en) 2021-03-03

Family

ID=60007090

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2016191731A Active JP6837708B2 (en) 2016-03-23 2016-09-29 Collision avoidance device

Country Status (1)

Country Link
JP (1) JP6837708B2 (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113291300A (en) * 2020-02-21 2021-08-24 丰田自动车株式会社 Driving assistance system
CN113291299A (en) * 2020-02-21 2021-08-24 丰田自动车株式会社 Driving assistance system
CN113401082A (en) * 2021-08-02 2021-09-17 姜春诗 Self-judging automobile safety automatic braking system and method
CN113715812A (en) * 2020-05-26 2021-11-30 丰田自动车株式会社 Driving support device
CN114512027A (en) * 2020-11-16 2022-05-17 长城汽车股份有限公司 Anti-collision control method and device
JP2022086637A (en) * 2020-11-30 2022-06-09 トヨタ自動車株式会社 Driving support system
US20230373477A1 (en) * 2016-12-22 2023-11-23 Toyota Jidosha Kabushiki Kaisha Collision avoidance support device

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009101720A (en) * 2007-10-19 2009-05-14 Mitsubishi Electric Corp Collision damage reduction device for vehicle
JP2009271766A (en) * 2008-05-08 2009-11-19 Hitachi Ltd Obstacle detection device for automobile
JP2015024713A (en) * 2013-07-25 2015-02-05 トヨタ自動車株式会社 Collision determination device

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009101720A (en) * 2007-10-19 2009-05-14 Mitsubishi Electric Corp Collision damage reduction device for vehicle
JP2009271766A (en) * 2008-05-08 2009-11-19 Hitachi Ltd Obstacle detection device for automobile
JP2015024713A (en) * 2013-07-25 2015-02-05 トヨタ自動車株式会社 Collision determination device

Cited By (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20230373477A1 (en) * 2016-12-22 2023-11-23 Toyota Jidosha Kabushiki Kaisha Collision avoidance support device
JP2021135523A (en) * 2020-02-21 2021-09-13 トヨタ自動車株式会社 Driving support system
CN113291299A (en) * 2020-02-21 2021-08-24 丰田自动车株式会社 Driving assistance system
JP7259780B2 (en) 2020-02-21 2023-04-18 トヨタ自動車株式会社 driving support system
CN113291300A (en) * 2020-02-21 2021-08-24 丰田自动车株式会社 Driving assistance system
US11932241B2 (en) 2020-02-21 2024-03-19 Toyota Jidosha Kabushiki Kaisha Driving assist system
US20210261120A1 (en) * 2020-02-21 2021-08-26 Toyota Jidosha Kabushiki Kaisha Driving assist system
CN113291300B (en) * 2020-02-21 2024-01-30 丰田自动车株式会社 Driving assistance system
US11787398B2 (en) * 2020-02-21 2023-10-17 Toyota Jidosha Kabushiki Kaisha Driving assist system
JP2021135510A (en) * 2020-02-21 2021-09-13 トヨタ自動車株式会社 Driving support system
CN113291299B (en) * 2020-02-21 2024-01-02 丰田自动车株式会社 Driving assistance system
JP7388238B2 (en) 2020-02-21 2023-11-29 トヨタ自動車株式会社 driving assistance system
CN113715812A (en) * 2020-05-26 2021-11-30 丰田自动车株式会社 Driving support device
CN113715812B (en) * 2020-05-26 2024-02-06 丰田自动车株式会社 Driving assistance device
CN114512027A (en) * 2020-11-16 2022-05-17 长城汽车股份有限公司 Anti-collision control method and device
JP2022086637A (en) * 2020-11-30 2022-06-09 トヨタ自動車株式会社 Driving support system
JP7347405B2 (en) 2020-11-30 2023-09-20 トヨタ自動車株式会社 driving assistance system
CN113401082A (en) * 2021-08-02 2021-09-17 姜春诗 Self-judging automobile safety automatic braking system and method

Also Published As

Publication number Publication date
JP6837708B2 (en) 2021-03-03

Similar Documents

Publication Publication Date Title
JP6837708B2 (en) Collision avoidance device
US9963128B2 (en) Travel assistance device
EP3366540B1 (en) Information processing apparatus and non-transitory computer-readable recording medium
CN108541325B (en) Driving support device and driving support method
JP6404634B2 (en) Consistent behavior generation of predictive advanced driver assistance systems
JP4933962B2 (en) Branch entry judgment device
JP5163991B2 (en) Vehicle speed control method in complex traffic situations
WO2016159288A1 (en) Target presence determination method and device
US10850737B2 (en) Driving lane changing apparatus and driving lane changing method
US9592827B2 (en) Collision mitigation apparatus and collision mitigation program
CN108602494B (en) Vehicle control device and vehicle control method
EP3476676B1 (en) System and method for performing autonomous emergency braking
JP6828429B2 (en) Vehicle collision avoidance support device and vehicle collision avoidance support method
KR20170050433A (en) Vehicle Control System and Method Thereof
JP5202741B2 (en) Branch entry judgment device
US9290172B2 (en) Collision mitigation device
JP5655297B2 (en) Object detection device, vehicle safety system equipped with object detection device
JP6913602B2 (en) Information processing equipment and programs
CN113272197A (en) Device and method for improving an assistance system for lateral vehicle movement
JP2010097400A (en) Collision prevention device
JP6733616B2 (en) Vehicle control device
JP7035862B2 (en) Driving support device
JP7289591B2 (en) Tracking control device
JP2016159758A (en) Driving assist system
KR20220029862A (en) Collision prevention control system and methof for vehicle

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20190805

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20200423

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20200512

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20200708

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20200929

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20201126

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20210209

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20210209

R150 Certificate of patent or registration of utility model

Ref document number: 6837708

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250