JP2020071705A - Concentrated state estimation device - Google Patents
Concentrated state estimation device Download PDFInfo
- Publication number
- JP2020071705A JP2020071705A JP2018205777A JP2018205777A JP2020071705A JP 2020071705 A JP2020071705 A JP 2020071705A JP 2018205777 A JP2018205777 A JP 2018205777A JP 2018205777 A JP2018205777 A JP 2018205777A JP 2020071705 A JP2020071705 A JP 2020071705A
- Authority
- JP
- Japan
- Prior art keywords
- gaze
- driver
- point
- state
- driving
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 claims description 28
- 238000000034 method Methods 0.000 claims description 15
- 230000004434 saccadic eye movement Effects 0.000 claims description 7
- 238000012790 confirmation Methods 0.000 description 5
- 238000012544 monitoring process Methods 0.000 description 4
- 210000005252 bulbus oculi Anatomy 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000007796 conventional method Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 210000001508 eye Anatomy 0.000 description 2
- 125000002066 L-histidyl group Chemical group [H]N1C([H])=NC(C([H])([H])[C@](C(=O)[*])([H])N([H])[H])=C1[H] 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000004459 microsaccades Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000004461 rapid eye movement Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 230000003945 visual behavior Effects 0.000 description 1
Landscapes
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
- Eye Examination Apparatus (AREA)
- Traffic Control Systems (AREA)
Abstract
Description
本発明は、集中状態推定装置、方法、及びプログラムに係り、特に、運転中のドライバの状態を推定するための集中状態推定装置、方法、及びプログラムに関する。 The present invention relates to a concentrated state estimation device, method, and program, and more particularly to a concentrated state estimation device, method, and program for estimating the state of a driver who is driving.
従来より、車両を運転するドライバの状態を判別する技術がある。 Conventionally, there is a technique for determining the state of a driver who drives a vehicle.
例えば、ドライバの眼のサッカード運動の回数からドライバの注意力に関するドライバ状態として、「注意良好状態」、「漫然状態」、及び「注意散漫状態」を判別する技術がある（特許文献１参照）。 For example, there is a technology that determines a “good attention state”, a “distracted state”, and a “distracted state” as the driver state related to the driver's attention from the number of saccade movements of the driver's eyes (see Patent Document 1). ..
また、例えば、サッカード運動のうちマイクロサッカードの増加、すなわち振幅の小さなサッカードが割合として増加しているか、ということから、ドライバの先行車に対する運転の注意状態が低い「漫然状態」を判別する技術がある（特許文献２参照）。 In addition, for example, it is determined whether the driver's attention to the preceding vehicle is low or "absent" because the microsaccade in the saccade movement is increasing, that is, the saccade having a small amplitude is increasing. There is a technique for doing so (see Patent Document 2).
また、例えば、視線に基づいた運転集中度の判定技術として、顔向きと視線のずれからのドライバの集中度を判定する技術がある（特許文献３参照）。 Further, for example, as a technique for determining the degree of driver's concentration based on the line of sight, there is a technique for determining the degree of driver's concentration based on the difference between the face direction and the line of sight (see Patent Document 3).
例えば、従来の特許文献３の技術では、視線に基づいた運転集中度の判定技術として、顔向きと視線のずれから集中度の判定をしていた。しかし、従来技術では、走行環境に対する注意の有無が判断できない、という問題があった。 For example, in the conventional technique of Patent Document 3, as a technique for determining the degree of concentration of driving based on the line of sight, the degree of concentration is determined from the difference between the face direction and the line of sight. However, the conventional technique has a problem that it cannot be determined whether or not attention is paid to the running environment.
本発明は、上記事情を鑑みて成されたものであり、複数の注視状態を考慮して、ドライバの運転に対する集中状態を推定できる集中状態推定装置、方法及びプログラムを提供することを目的とする。 The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a concentrated state estimation device, method, and program capable of estimating a concentrated state of a driver with respect to driving in consideration of a plurality of gaze states. ..
上記目的を達成するために、第１の発明に係る集中状態推定装置は、予め求められた前方注視時間後に自車両が到達されると推定される地点を、前方注視点として設定する前方注視点設定手段と、前記自車両の周辺に関するデータに基づいて、ドライバが運転時に注意を払うべき点である運転注意点を検出する運転注意点検出手段と、前記ドライバの視線、及び注視の継続時間を検出するドライバ視線検出手段と、設定した前記前方注視点と、検出した前記運転注意点と、検出した前記ドライバの視線とに基づいて、前記ドライバの注視の状態が、複数種類の注視状態の何れであるかを判別する注視点判別手段と、前記判別された注視状態の各々の割合と、検出された前記注視の継続時間とに基づいて、前記ドライバの運転に対する集中状態を推定する集中状態推定手段と、を含んで構成されている。 In order to achieve the above object, a concentration state estimation device according to a first aspect of the present invention sets a point at which a vehicle is estimated to arrive after a predetermined forward gaze time as a forward gaze point. Based on the setting means and the data about the surroundings of the own vehicle, the driver's attention detecting means for detecting the driver's attention, which is the point that the driver should pay attention when driving, the driver's line of sight, and the duration of the gaze. Based on the driver's line-of-sight detecting means for detecting, the set front gaze point, the detected driving attention point, and the detected line of sight of the driver, the gaze state of the driver is one of a plurality of gaze states. Based on the ratio of each of the determined gaze state and the detected duration of the gaze state, the driver's focus state on driving is determined. Is configured to include a centralized state estimation means estimates that the.
第２の発明に係る集中状態推定方法は、前方注視点設定手段が、前方注視点を設定するステップと、運転注意点検出手段が、前記自車両の周辺に関するデータに基づいて、ドライバが運転時に注意を払うべき点である運転注意点を検出するステップと、ドライバ視線検出手段が、前記ドライバの視線、及び注視の継続時間を検出するステップと、注視点判別手段が、設定した前記前方注視点と、検出した前記運転注意点と、検出した前記ドライバの視線とに基づいて、前記ドライバの注視の状態が、複数種類の注視状態の何れであるかを判別するステップと、集中状態推定手段が、前記判別された注視状態の各々の割合と、検出された前記注視の継続時間とに基づいて、前記ドライバの運転に対する集中状態を推定するステップと、を含んで実行することを特徴とする。 In the concentrated state estimating method according to the second aspect of the present invention, the front gazing point setting means sets the front gazing point, and the driving caution point detecting means, based on the data related to the surroundings of the own vehicle, when the driver is driving. The step of detecting a driver's attention point that should be paid attention, the driver's line-of-sight detection means detecting the driver's line-of-sight and the duration of the gaze, and the gaze point determination means set the front gaze point. A step of determining which of a plurality of types of gaze states the gaze state of the driver is, based on the detected driving precautions and the detected line of sight of the driver; A step of estimating a concentration state of the driver with respect to driving, based on a ratio of each of the determined gaze states and a detected duration of the gaze state. Characterized in that row.
第３の発明に係るプログラムは、コンピュータを、第１の発明に記載の集中状態推定装置の各部として機能させるためのプログラムである。 A program according to a third invention is a program for causing a computer to function as each unit of the concentrated state estimating device according to the first invention.
本発明の集中状態推定装置、方法、及びプログラムによれば、複数の注視状態を考慮して、ドライバの運転に対する集中状態を推定できる。 According to the concentration state estimation device, method, and program of the present invention, it is possible to estimate the concentration state for the driver's driving in consideration of a plurality of gaze states.
まず本発明の実施の形態の前提となる技術について説明する。 First, a technique on which the embodiment of the present invention is based will be described.
自動車運転中のドライバには、１．５から３秒後に自車両が到達するポイントを目標到達点（前方注視点）として注視する性質が知られている。 It is known that a driver who is driving a car looks at a point where the own vehicle arrives after 1.5 to 3 seconds as a target reaching point (forward gazing point).
［参考文献１］特開２０１０−１７０１８７号公報 [Reference 1] Japanese Patent Laid-Open No. 2010-170187
また、上記のような前方注視行動は、眼球だけではなく顔を注視対象に向けることも知られている。顔向きと眼球の一致した視線の方向から目標経路に沿った運転に集中している状態が推定でき、自動運転状態においても集中状態のレベルが高いと判断できる。 It is also known that the above-mentioned forward gaze action directs not only the eyeball but also the face toward the gaze target. It can be estimated that the driver is concentrating on the driving along the target route from the direction of the line of sight where the face direction and the eyeball match, and it can be determined that the concentration level is high even in the automatic driving state.
ただし、安全運転のためには、周囲の環境に目配せして、危険の有無を確認する「分散的注意機能」が必要となる。例えば、自動運転車の自動運転の際には、自動運転の状態から手動運転に復帰する際に、ドライバの分散的注意機能が十分に働いている安全な状態で復帰することが望ましい。 However, in order to drive safely, it is necessary to have a "distributed caution function" that allows you to pay attention to the surrounding environment and check for danger. For example, in the case of automatic driving of an autonomous vehicle, when returning from the state of automatic driving to manual driving, it is desirable to return in a safe state in which the decentralized attention function of the driver is sufficiently working.
そこで、本発明の実施の形態では、自動運転車の自動運転の際に、ドライバの分散的注意機能を考慮して、ドライバの運転に対する準備の状態がどの程度であるかを表す集中状態を推定する。集中状態は、走行環境に応じて発生する危険に対処する際にドライバが行う注視の状態が複数種類の注視状態のいずれであるかを判別し、注視状態の各々の割合と、注視の継続時間を用いて推定を行う。なお、以下に説明する本実施の形態では、自動運転車を対象として集中状態を推定する場合を例に説明するが、これに限定されるものではない。例えば、通常の自動車の場合に適用した場合でも、手動運転時において、ドライバが、運転中にどの程度集中しているかを表す集中度を推定することもできる。 Therefore, in the embodiment of the present invention, when the automatic driving of the self-driving vehicle is taken into consideration, the concentrated state indicating how much the driver is prepared for driving is estimated in consideration of the decentralized attention function of the driver. To do. The concentration state determines which of a plurality of types of gaze states the driver performs when dealing with a danger that occurs depending on the driving environment, and determines the proportion of each gaze state and the duration of the gaze. Estimate using. In the present embodiment described below, the case where the concentrated state is estimated for an autonomous vehicle will be described as an example, but the present invention is not limited to this. For example, even when applied to an ordinary automobile, it is possible to estimate the degree of concentration, which indicates how much the driver is concentrated during driving during manual driving.
以下、図面を参照して本発明の実施の形態を詳細に説明する。 Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
＜本発明の実施の形態に係る集中状態推定装置の構成＞ <Configuration of concentrated state estimation device according to embodiment of the present invention>
次に、本発明の実施の形態に係る集中状態推定装置の構成について説明する。本発明の実施の形態では、集中状態推定装置は、自動運転車に搭載され、ドライバの集中状態を推定するものとする。 Next, the configuration of the concentrated state estimation device according to the embodiment of the present invention will be described. In the embodiment of the present invention, the concentration state estimation device is installed in an autonomous vehicle and estimates the concentration state of the driver.
図１に示すように、本発明の実施の形態に係る集中状態推定装置１００は、ＣＰＵと、ＲＡＭと、後述する集中状態推定処理ルーチンを実行するためのプログラムや各種データを記憶したＲＯＭと、を含むコンピュータで構成することが出来る。この集中状態推定装置１００は、機能的には図１に示すようにセンサ１０と、車載カメラ１２と、演算部２０と、出力部５０とを備えている。 As shown in FIG. 1, a concentrated state estimation device 100 according to an embodiment of the present invention includes a CPU, a RAM, a ROM that stores programs and various data for executing a concentrated state estimation processing routine described later, Can be configured with a computer including. This concentrated state estimation device 100 functionally includes a sensor 10, a vehicle-mounted camera 12, a calculation unit 20, and an output unit 50 as shown in FIG. 1.
センサ１０は、レーザーレーダ等のセンサであり、自車両の周辺環境をセンシングすることで、周辺環境の物体の種類、及び位置をセンサ検出データとして検出し、運転注意点検出部３２に出力する。周辺環境の物体としては、道路標識、周辺車両、及び歩行者などである。また、周辺環境の物体として、自動運転に用いる地図情報に基づいて検出される交通信号も挙げられる。交通信号の場合には、周辺環境の物体の種類は、交通信号の状態となる。 The sensor 10 is a sensor such as a laser radar, which detects the type and position of an object in the surrounding environment as sensor detection data by sensing the surrounding environment of the vehicle, and outputs the sensor detection data to the driving caution point detection unit 32. Objects in the surrounding environment include road signs, surrounding vehicles, and pedestrians. In addition, as an object of the surrounding environment, a traffic signal detected based on map information used for automatic driving can also be mentioned. In the case of a traffic signal, the type of object in the surrounding environment is the state of the traffic signal.
車載カメラ１２は、コックピットに設置されたドライバを撮影するための内部カメラ、及び自車両の周辺を監視するための周辺監視カメラである。車載カメラ１２は、内部カメラによって、ドライバの顔向きや眼球の向きなどがわかるドライバの動画像を撮影し、ドライバ視線検出部３４に出力する。車載カメラ１２は、周辺監視カメラによって、前方車両や歩行者などを含みうる周辺環境の動画像を撮影し、前方注視点設定部３０、及び注視点判別部３６に出力する。 The vehicle-mounted camera 12 is an internal camera for shooting a driver installed in the cockpit and a peripheral monitoring camera for monitoring the surroundings of the vehicle. The vehicle-mounted camera 12 captures a moving image of the driver, which shows the driver's face orientation, eyeball orientation, etc., by the internal camera, and outputs it to the driver's line-of-sight detection unit 34. The vehicle-mounted camera 12 captures a moving image of the surrounding environment, which may include a vehicle in front, a pedestrian, etc., by the surrounding monitoring camera, and outputs it to the front gazing point setting unit 30 and the gazing point determination unit 36.
演算部２０は、前方注視点設定部３０と、運転注意点検出部３２と、ドライバ視線検出部３４と、注視点判別部３６と、集中状態推定部３８とを含んで構成されている。 The calculation unit 20 includes a front gazing point setting unit 30, a driving attention point detection unit 32, a driver line-of-sight detection unit 34, a gazing point determination unit 36, and a concentration state estimation unit 38.
前方注視点設定部３０は、前方注視点を設定する。前方注視点は、自車両の前に前方車両がない場合と、自車両が前方車両に追従する場合とで、それぞれ設定する。前方車両がない場合は、現地点から所定の到達時間後に自車両が到達すると推定される地点を、ドライバが運転の際に注視している路面上のポイントである前方注視点と推定して設定する。ドライバが運転する場合、自車両が現地点から前方注視点に到達する時間は、通常１．５から３秒であり、この時間は走行する道路の曲率が大きいほど、車速が大きいほど、小さくなる。そのため、前方注視点設定部３０は、車載カメラ１２（周辺監視カメラ）で撮影した周辺環境の動画像から道路の形状を識別し、道路曲率や車速に応じて適切に到達時間を設定する。前方注視点設定部３０は、設定した前方注視時間に応じて推定される地点を前方注視点に設定する。また、前方注視点設定部３０は、前方車両が前方注視点に相当する車間内に存在し、自車両が前方車両に追従する場合には、ドライバは前方車両の位置を注視することから、周辺環境の動画像から前方車両を識別し、識別した前方車両の位置に前方注視点を設定する。なお、前方注視点の設定にはセンサ１０で検出したセンサ検出データを用いてもよい。 The front gazing point setting unit 30 sets a front gazing point. The forward gazing point is set when the front vehicle is not in front of the host vehicle and when the host vehicle follows the front vehicle. If there is no vehicle in front, the point estimated to be reached by the own vehicle after a predetermined arrival time from the local point is set by presuming to be the forward gazing point, which is the point on the road surface that the driver is gazing at when driving. To do. When the driver drives, the time for the host vehicle to reach the gazing point ahead from the local point is usually 1.5 to 3 seconds. This time becomes smaller as the curvature of the road on which the vehicle travels increases and the vehicle speed increases. .. Therefore, the front gazing point setting unit 30 identifies the shape of the road from the moving image of the surrounding environment captured by the vehicle-mounted camera 12 (surrounding camera), and sets the arrival time appropriately according to the road curvature and the vehicle speed. The forward gaze point setting unit 30 sets a point estimated according to the set forward gaze time as the forward gaze point. In addition, the front gaze point setting unit 30 determines that the driver is gazing at the position of the front vehicle when the front vehicle is inside the vehicle space corresponding to the front gaze point and the host vehicle follows the front vehicle. A front vehicle is identified from a moving image of the environment, and a front gazing point is set at the identified position of the front vehicle. The sensor detection data detected by the sensor 10 may be used for setting the forward gazing point.
運転注意点検出部３２は、センサ検出データを含む自車両の周辺に関するデータに基づいて、ドライバが運転時に注意を払うべき、安全確認が必要となる点である運転注意点を検出する。自車両の周辺に関するデータは、センサ検出データや、自車両のミラー位置などである。運転注意点検出部３２は、センサ１０で検出したセンサ検出データに含まれる周辺環境の物体の中から危険因子となる物体の位置や、車両のミラー位置などの運転注意点を検出する。なお、自車両周辺の危険因子については、参考文献２の記載の危険判定プログラムに基づき、精度よく検出することができる。なお、運転注意点の検出には、車載カメラ１２（周辺監視カメラ）で撮影した周辺環境の動画像を用いてもよい。 The driving caution detection unit 32 detects a driving caution, which is a point that the driver needs to pay attention to when driving, that is, a safety confirmation, based on data related to the surroundings of the vehicle including the sensor detection data. The data related to the area around the host vehicle is sensor detection data, the mirror position of the host vehicle, and the like. The driving caution point detection unit 32 detects driving caution points such as the position of an object that is a risk factor and the mirror position of the vehicle among the objects in the surrounding environment included in the sensor detection data detected by the sensor 10. The risk factors around the host vehicle can be accurately detected based on the risk determination program described in Reference 2. Note that a moving image of the surrounding environment captured by the vehicle-mounted camera 12 (surrounding camera) may be used to detect the driving caution.
［参考文献２］特開２０１１−１９８２６６号公報 [Reference 2] Japanese Unexamined Patent Publication No. 2011-198266
ドライバ視線検出部３４は、車載カメラ１２（内部カメラ）で撮影したドライバの動画像から、ドライバの視線、及び注視の継続時間を検出する。人間の視線は、ある物標を注視し続ける「注視」の状態と急速な眼球運動によって他の物標に視線が移動する「サッカード」の状態を繰り返すことが知られている。ドライバ視線検出部３４では、「注視」の状態における視線に加え、「サッカード」の発生タイミングを区切りとする注視の継続時間も出力する。 The driver's line-of-sight detection unit 34 detects the line of sight of the driver and the duration of the gaze from the moving image of the driver taken by the vehicle-mounted camera 12 (internal camera). It is known that the line of sight of a human repeats a "gaze" state in which a certain target is continuously gazed and a "saccade" state in which the gaze moves to another target due to rapid eye movement. In addition to the line of sight in the “gaze” state, the driver's line of sight detection unit 34 also outputs the gaze duration, which is delimited by the occurrence timing of the “saccade”.
注視点判別部３６は、ドライバの注視の状態が、複数種類の注視状態の何れであるかを判別する。判別は、前方注視点設定部３０で設定した前方注視点と、運転注意点検出部３２で検出した運転注意点と、ドライバ視線検出部３４で検出したドライバの視線とに基づいて行う。複数種類の注視状態は、前方注視点を注視している状態、運転注意点を注視している状態、及び前方注視点及び運転注意点のいずれでもない非運転注意点を注視している状態を含む。 The gaze point determination unit 36 determines which of a plurality of types of gaze states the gaze state of the driver is. The determination is performed based on the front gazing point set by the front gazing point setting unit 30, the driving attention point detected by the driving attention point detection unit 32, and the driver's line of sight detected by the driver line-of-sight detection unit 34. There are multiple types of gaze states: a state of gazing at the front gazing point, a state of gazing at a driving caution, and a state of gazing at a non-driving gazing that is neither a front gazing point nor a driving caution. Including.
集中状態推定部３８は、注視点判別部３６で判別された注視状態の各々の割合と、ドライバ視線検出部３４で検出された注視の継続時間とに基づいて、ドライバの運転に対する集中状態を推定し、推定結果を出力部５０に出力する。 The concentration state estimation unit 38 estimates the concentration state with respect to the driving of the driver based on the ratio of each of the gaze states determined by the gaze point determination unit 36 and the duration of the gaze detected by the driver gaze detection unit 34. Then, the estimation result is output to the output unit 50.
以下、集中状態推定部３８において、集中状態を推定する手法の詳細を説明する。 Hereinafter, the detail of the method of estimating the concentration state in the concentration state estimation unit 38 will be described.
集中状態の推定のために、周辺の環境ごとに、注視の各状態の割合に対する基準、及び注視の継続時間に対する基準が予め定められているものとする。 In order to estimate the concentration state, it is assumed that a standard for the proportion of each state of gaze and a standard for the duration of the gaze are predetermined for each surrounding environment.
集中状態推定部３８は、自車両の周辺の環境に対する基準の各々と、注視の各状態の割合と、注視の継続時間とに基づいて、集中状態を推定する。集中状態推定部３８では、基準を満たす場合に、集中状態の推定値を高く推定し、基準を満たさない場合に集中状態の推定値を低く推定する。 The concentration state estimation unit 38 estimates the concentration state based on each of the criteria for the environment around the own vehicle, the ratio of each state of gaze, and the duration of the gaze. The concentrated state estimating unit 38 estimates the estimated value of the concentrated state to be high when the criterion is satisfied, and estimates the estimated value of the concentrated state to be low when the criterion is not satisfied.
注視の各状態の割合に対する基準、及び注視の継続時間に対する基準は、ドライブシミュレータの実験等によって定めるようにすればよい。例えば、注視の各状態の割合に対する基準については、単位時間あたりの各状態のそれぞれの割合の範囲や、運転注意点に対する注視の回数などについて定める。注視の継続時間に対する基準については、前方注視点を注視している状態や、運転注意点を注視している状態について、一回ごとの注視の継続時間が一定時間以上か、また、非運転注意点の注視の継続時間が一定時間以下か、などを定める。 The standard for the proportion of each state of gaze and the standard for the duration of the gaze may be set by an experiment of a drive simulator. For example, as a reference for the ratio of each state of gaze, the range of each ratio of each state per unit time, the number of times of gaze with respect to driving precautions, and the like are determined. As for the criteria for the duration of gaze, for the state of gazing at the front gaze point and the state of gazing at driving precautions, whether the duration of each gaze is a certain time or more, Determine whether the duration of the point gaze is less than a certain time.
例えば、ドライブシミュレータの実験において、故意にわき見を誘発するような場面や、非運転注視点を注視させるタスクなどを与えることで、ドライバの視線分布や注視継続時間の変化を加えることができる。その上で、先行車が急減速したり、歩行者が横断したりするような危険な場面を発生させ、その反応の速度や正確さで集中状態を対応付けることができ、この結果を用いて視線分布等から集中状態を推定するアルゴリズムを求める。 For example, in an experiment of a drive simulator, it is possible to add a change in the driver's gaze distribution or gaze duration by giving a scene that deliberately induces a sideways look, a task to gaze at a non-driving gaze point, and the like. On top of that, a dangerous scene, such as a preceding vehicle suddenly decelerating or a pedestrian crossing, is generated, and the concentration state can be associated with the speed and accuracy of the reaction. Find an algorithm for estimating the concentration state from the distribution.
周辺の環境ごとに基準を定める理由は、例えば、周辺の環境には、前方車両が少なく道路幅も広く見通しのよい環境である場合や、前方車両や周辺の建物などが多く見通しが悪い環境である場合など、周辺の環境が様々である場合が想定されるためである。そのため、周辺の環境に応じた適切な集中状態を推定するために、これらの周辺の環境ごとに基準を定める。例えば、見通しが悪い環境であれば、運転注意点に対する注視の回数が一定回数以上必要とし、前方注視点や運転注意点に対する注視の継続時間が一定の時間は必要とするように基準を定める。 The reason for setting the standard for each surrounding environment is, for example, that the surrounding environment is one where there are few vehicles in front and the road width is wide and there is good visibility, or when there are many vehicles in front and surrounding buildings and the visibility is poor. This is because it is assumed that the surrounding environment is various, such as in some cases. Therefore, in order to estimate an appropriate concentration state according to the surrounding environment, a standard is set for each of these surrounding environments. For example, in an environment where visibility is poor, the standard is set such that the number of gazes for the driving attention point needs to be a certain number of times or more, and the duration of the gaze for the front gaze point and the driving attention point needs a certain time.
集中状態推定部３８は、どの周辺環境の基準を用いるかを決定するため、センサ１０の車載カメラ１２（周辺監視カメラ）で撮影した周辺環境の動画像に基づいて、自車両の周辺の環境を識別する。そして、集中状態推定部３８は、識別した自車両の周辺の環境に対応する基準の各々に対する、各状態の割合、及び各状態の注視の一回ごとの注視の継続時間に基づいて、集中状態を推定する。 The concentrated state estimating unit 38 determines the surrounding environment based on the moving image of the surrounding environment captured by the vehicle-mounted camera 12 (surrounding monitoring camera) of the sensor 10 in order to determine which surrounding environment standard to use. Identify. Then, the concentrated state estimation unit 38, based on the ratio of each state to each of the criteria corresponding to the environment around the identified own vehicle, and the duration of the gaze for each state of gaze, the concentrated state. To estimate.
以下、図２、及び図３を参照して、集中状態の推定の一例を説明する。 Hereinafter, an example of estimation of the concentration state will be described with reference to FIGS. 2 and 3.
図２は、市街地運転中のドライバの視線及び顔向きの時間変化を示している。図２の例では、ドライバは、前方注視点に顔を向けて自動運転しており、周囲の安全確認のためにミラーや周辺を短時間、眼球のみで注視していることがわかる。視線が０［ｄｅｇ］に近い状態が前方注視点を注視している状態と捉えることができる。また、視線の位置が左右いずれかに振れている破線の丸で囲った箇所が、運転注意点を注視している状態と捉えることができる。図２の例では、安全確認のとき、ルームミラーなどの運転注意点を注視している注視状態と判別できる。従来技術（特許文献３参照）では、顔向きと視線のずれから集中度（集中状態）を判定しており、図２のように、安全確認のために顔向きと視線がずれてしまう視認行動は、推定上外乱として扱われてしまう。これに対し、本発明の実施の形態に係る手法を用いれば、顔向きと視線が一致しているか否かにかかわらず、前方注視点又は運転注意点を注視している状態について、高い集中状態の推定値を出力することができる。 FIG. 2 shows temporal changes in the line of sight and face direction of the driver while driving in the city. In the example of FIG. 2, it can be seen that the driver is automatically driving with his face facing the front gazing point and gazing at the mirror and the surroundings for a short time only with his / her eyes to confirm the surrounding safety. A state in which the line of sight is close to 0 [deg] can be regarded as a state of gazing at the forward gazing point. In addition, a portion surrounded by a broken-line circle in which the position of the line of sight is swaying to the left or right can be regarded as a state where the driver is paying attention to the driving caution. In the example of FIG. 2, at the time of safety confirmation, it can be determined that the driver is watching the driving precautions such as the rearview mirror. In the prior art (see Patent Document 3), the degree of concentration (concentration state) is determined from the deviation between the face direction and the line of sight, and as shown in FIG. 2, the visual behavior in which the face direction and the line of sight are deviated for safety confirmation. Is treated as an external disturbance. On the other hand, by using the method according to the embodiment of the present invention, regardless of whether or not the face direction and the line of sight are the same, the state of gazing at the forward gaze point or the driving attention point is a high concentration state. The estimated value of can be output.
図３は、図２とは別の運転場面でのドライバの視線の動きを示した結果である。ドライバの視線は、注視（止まる）とサッカード（動く）の繰り返しとなっており、この場面の注視点を抽出すると、前方車両の軌跡を前方注視点として注視したあと、目立ち度の高い非運転注意点を注視、さらに前方注視点に戻り、車列の前方の状況を確認するための運転注意点を注視している。本発明の実施の形態に係る手法では、非運転注意点を注視している時間や、１回の運転注意点の注視継続時間などから集中状態を推定することができる。 FIG. 3 is a result showing the movement of the driver's line of sight in a driving scene different from that in FIG. The driver's line of sight is repeated gaze (stop) and saccade (moving). When the gaze point of this scene is extracted, the gaze of the vehicle ahead is gazed at as the front gaze point, and then the non-driving with high visibility He is paying attention to the points of caution and then returning to the point of interest in front of him to watch the points of caution for driving in order to confirm the situation in front of the train. With the method according to the embodiment of the present invention, it is possible to estimate the concentration state from the time during which the non-driving attention point is gazed at, the gaze duration of one driving attention point, and the like.
以上のように、本発明の実施の形態では、ドライバが運転時に注意を払うべき安全確認が必要となる個所を運転注意点として検出し、この運転注意点への注視行動がどの程度の割合を占めるかと、１回の注視行動の継続時間によって、「分散的注意機能」という観点も含めた集中状態の推定を可能とする。 As described above, in the embodiment of the present invention, the driver detects a point that requires safety confirmation that should be paid attention when driving as a driver's caution point, and determines the proportion of the gaze behavior to the driver's caution point. Whether or not it occupies and the duration of one gaze action makes it possible to estimate the concentration state including the viewpoint of "distributed attention function".
また、比較的交通量の少ない自動車専用道などの環境を走行する場合には、基準において、前方注視点への注視の継続時間は長くなる傾向となることは許容される。しかし、運転注意点の増加に伴い、前方注視点への注視の継続時間は短くなり、運転注意点への注視割合が増加しないと、分散的注意機能が不十分であると判断され、集中状態の推定値は低下する。同様に、非運転注意点への注視時間が長い場合も、分散的注意機能の低下と判断され、集中状態の推定値は低下することとなる。 Further, when traveling in an environment such as an automobile road where the traffic volume is relatively small, it is permissible that the duration of the gazing toward the forward gazing point tends to be long as a standard. However, as the number of driving precautions increases, the duration of the gaze to the forward gaze point becomes shorter, and unless the proportion of gaze to the driving attention points increases, it is determined that the decentralized attention function is inadequate, and the concentration state The estimate of is lower. Similarly, even when the time taken to pay attention to the non-driving attention point is long, it is determined that the decentralized attention function is deteriorated, and the estimated value of the concentration state is decreased.
＜本発明の実施の形態に係る集中状態推定装置の作用＞ <Operation of concentrated state estimation device according to embodiment of the present invention>
次に、本発明の実施の形態に係る集中状態推定装置１００の作用について説明する。集中状態推定装置１００は、図４に示す集中状態推定処理ルーチンを実行する。なお、集中状態推定装置１００は、センサ１０のセンサ取得データ、及び車載カメラ１２で撮影した、ドライバの動画像、及び周辺環境の動画像を適宜取得しているものとする。 Next, the operation of the concentrated state estimation device 100 according to the embodiment of the present invention will be described. The concentrated state estimation device 100 executes a concentrated state estimation processing routine shown in FIG. It is assumed that the concentration state estimation device 100 appropriately acquires the sensor acquisition data of the sensor 10, the moving image of the driver captured by the vehicle-mounted camera 12, and the moving image of the surrounding environment.
まず、ステップＳ１００では、前方注視点設定部３０は、前方注視点を設定する。 First, in step S100, the front gazing point setting unit 30 sets a front gazing point.
次に、ステップＳ１０２では、運転注意点検出部３２は、センサ１０で検出したセンサ検出データを含む自車両の周辺に関するデータに基づいて、ドライバが運転時に注意を払うべき、安全確認が必要となる点である運転注意点を検出する。 Next, in step S102, the driving caution detection unit 32 needs to perform safety confirmation, which the driver should pay attention to when driving, based on the data related to the surroundings of the vehicle including the sensor detection data detected by the sensor 10. Detect the driving caution, which is the point.
ステップＳ１０４では、ドライバ視線検出部３４は、車載カメラ１２（内部カメラ）で撮影したドライバの動画像から、ドライバの視線、及び注視の継続時間を検出する。 In step S104, the driver's line-of-sight detection unit 34 detects the line of sight of the driver and the duration of the gaze from the moving image of the driver taken by the vehicle-mounted camera 12 (internal camera).
ステップＳ１０６では、注視点判別部３６は、ドライバの注視の状態が、複数種類の注視状態の何れであるかを判別する。判別は、前方注視点設定部３０で設定した前方注視点と、運転注意点検出部３２で検出した運転注意点と、ドライバ視線検出部３４で検出したドライバの視線とに基づいて行う。複数種類の注視状態は、前方注視点を注視している状態、運転注意点を注視している状態、及び前方注視点及び運転注意点のいずれでもない非運転注意点を注視している状態を含む。 In step S106, the gazing point determination unit 36 determines which of a plurality of types of gazing states the driver's gazing state is. The determination is performed based on the front gazing point set by the front gazing point setting unit 30, the driving attention point detected by the driving attention point detection unit 32, and the driver's line of sight detected by the driver line-of-sight detection unit 34. There are multiple types of gaze states: a state of gazing at the front gazing point, a state of gazing at a driving caution, and a state of gazing at a non-driving gazing that is neither a front gazing point nor a driving caution. Including.
ステップＳ１０８では、集中状態推定部３８は、注視点判別部３６で判別された注視状態の各々の割合と、ドライバ視線検出部３４で検出された注視の継続時間とに基づいて、ドライバの運転に対する集中状態を推定し、推定結果を出力部５０に出力する。周辺の環境ごとに、注視の各状態の割合に対する基準、及び注視の継続時間に対する基準が予め定められているものとし、自車両の周辺の環境に対する基準の各々に対する、注視の各状態の割合と、注視の継続時間とに基づいて、集中状態を推定する。 In step S108, the concentration state estimation unit 38 performs the driver's driving based on the ratio of each of the gaze states determined by the gaze point determination unit 36 and the duration of the gaze detected by the driver gaze detection unit 34. The concentration state is estimated and the estimation result is output to the output unit 50. For each surrounding environment, the standard for the ratio of each state of gaze, and the standard for the duration of the gaze shall be set in advance, and the ratio of each state of gaze to each of the standards for the surrounding environment of the own vehicle and , The concentration state is estimated based on the duration of gaze.
以上説明したように、本発明の実施の形態に係る集中状態推定装置によれば、ドライバの運転に対する集中状態を推定できる。 As described above, according to the concentrated state estimation device according to the embodiment of the present invention, it is possible to estimate the concentrated state for the driving of the driver.
なお、本発明は、上述した実施の形態に限定されるものではなく、この発明の要旨を逸脱しない範囲内で様々な変形や応用が可能である。 The present invention is not limited to the above-described embodiments, and various modifications and applications can be made without departing from the scope of the present invention.
例えば、上述した実施の形態では、周辺の環境に対する基準を設けて集中状態を推定する場合を例に説明したが、これに限定されるものではない。例えば、生体情報に対する基準を設けて、ドライバの生体情報を更に考慮して集中状態を推定するようにしてもよい。 For example, in the above-described embodiment, the case where the concentration state is estimated by setting the reference for the surrounding environment has been described as an example, but the present invention is not limited to this. For example, a criterion for biometric information may be provided, and the biometric information of the driver may be further considered to estimate the concentration state.
また、上述した実施の形態では、複数種類の注視状態として、前方注視点を注視している状態、運転注意点を注視している状態、及び前方注視点及び運転注意点のいずれでもない非運転注意点を注視している状態を判別する場合を例に説明したが、これに限定されるものではない。例えば、前方注視点や運転注意点の周辺を注視している状態など他の状態を判別してもよい。 Further, in the above-described embodiment, as a plurality of types of gaze states, a state of gazing a front gazing point, a state of gazing a driving caution, and a non-driving that is neither a gazing front nor a driving caution. The case where the state in which attention is paid to attention is determined has been described as an example, but the present invention is not limited to this. For example, other states such as a state of gazing at the front gazing point or the vicinity of the driving caution may be determined.
１０ センサ
１２ 車載カメラ
２０ 演算部
３０ 前方注視点設定部
３２ 運転注意点検出部
３４ ドライバ視線検出部
３６ 注視点判別部
３８ 集中状態推定部
５０ 出力部
１００ 集中状態推定装置
10 sensor 12 vehicle-mounted camera 20 arithmetic unit 30 front gazing point setting unit 32 driving attention point detection unit 34 driver line-of-sight detection unit 36 gazing point determination unit 38 concentration state estimation unit 50 output unit 100 concentration state estimation device
Claims (6)
自車両の周辺に関するデータに基づいて、ドライバが運転時に注意を払うべき点である運転注意点を検出する運転注意点検出手段と、
前記ドライバの視線、及び注視の継続時間を検出するドライバ視線検出手段と、
設定した前記前方注視点と、検出した前記運転注意点と、検出した前記ドライバの視線とに基づいて、前記ドライバの注視の状態が、複数種類の注視状態の何れであるかを判別する注視点判別手段と、
前記判別された注視状態の各々の割合と、検出された前記注視の継続時間とに基づいて、前記ドライバの運転に対する集中状態を推定する集中状態推定手段と、
を含む集中状態推定装置。 Front gazing point setting means for setting a front gazing point,
Driving caution point detecting means for detecting a driving caution point, which is a point that the driver should pay attention when driving, based on the data about the vicinity of the own vehicle;
A driver's line-of-sight detection means for detecting the line-of-sight of the driver and the duration of the gaze;
Based on the set forward gaze point, the detected driving attention point, and the detected line of sight of the driver, a gaze point that determines which of a plurality of types of gaze states the gaze state of the driver is Discriminating means,
Based on each ratio of the determined gaze state and the detected duration of the gaze, a concentrated state estimation means for estimating a concentrated state for driving the driver,
Centralized state estimation device including.
前記集中状態推定手段は、前記自車両の周辺の環境に対する前記基準の各々と、前記判別された注視状態の各々の割合と、前記注視の継続時間とに基づいて、前記集中状態を推定する請求項１又は請求項２に記載の集中状態推定装置。 According to a predetermined surrounding environment, the standard for the proportion of each state of the gaze, and the standard for the duration of the gaze is predetermined,
The concentration state estimating means estimates the concentration state based on each of the criteria for the environment around the own vehicle, the ratio of each of the determined gaze states, and the duration of the gaze. The concentrated state estimation device according to claim 1 or 2.
自車両の周辺に関するデータに基づいて、ドライバが運転時に注意を払うべき点である運転注意点を検出するステップと、
前記ドライバの視線、及び注視の継続時間を検出するステップと、
設定した前記前方注視点と、検出した前記運転注意点と、検出した前記ドライバの視線とに基づいて、前記ドライバの注視の状態が、複数種類の注視状態の何れであるかを判別するステップと、
前記判別された注視状態の各々の割合と、検出された前記注視の継続時間とに基づいて、前記ドライバの運転に対する集中状態を推定するステップと、
を含む集中状態推定方法。 Setting a forward gaze point,
A step of detecting a driving caution point, which is a point that the driver should pay attention when driving, based on the data related to the vicinity of the own vehicle;
Detecting the driver's line of sight and the duration of gaze;
A step of determining which of a plurality of types of gaze states the gaze state of the driver is based on the set front gaze point, the detected driving attention point, and the detected line of sight of the driver; ,
Estimating a state of concentration with respect to the driving of the driver, based on each of the determined gaze states and the detected duration of the gaze,
Concentrated state estimation method including.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2018205777A JP7164275B2 (en) | 2018-10-31 | 2018-10-31 | Centralized state estimator |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2018205777A JP7164275B2 (en) | 2018-10-31 | 2018-10-31 | Centralized state estimator |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| JP2020071705A true JP2020071705A (en) | 2020-05-07 |
| JP7164275B2 JP7164275B2 (en) | 2022-11-01 |
Family
ID=70549580
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| JP2018205777A Active JP7164275B2 (en) | 2018-10-31 | 2018-10-31 | Centralized state estimator |
Country Status (1)
| Country | Link |
|---|---|
| JP (1) | JP7164275B2 (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2010032424A1 (en) * | 2008-09-18 | 2010-03-25 | 学校法人中部大学 | Sleepiness signal detector |
| JP2018063489A (en) * | 2016-10-11 | 2018-04-19 | 国立大学法人金沢大学 | Operator state estimation device |
| JP2018097515A (en) * | 2016-12-12 | 2018-06-21 | 株式会社デンソーアイティーラボラトリ | Drive assisting device, drive assisting method, and program thereof |
-
2018
- 2018-10-31 JP JP2018205777A patent/JP7164275B2/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2010032424A1 (en) * | 2008-09-18 | 2010-03-25 | 学校法人中部大学 | Sleepiness signal detector |
| JP2018063489A (en) * | 2016-10-11 | 2018-04-19 | 国立大学法人金沢大学 | Operator state estimation device |
| JP2018097515A (en) * | 2016-12-12 | 2018-06-21 | 株式会社デンソーアイティーラボラトリ | Drive assisting device, drive assisting method, and program thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| JP7164275B2 (en) | 2022-11-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3735365B1 (en) | Primary preview region and gaze based driver distraction detection | |
| US10009580B2 (en) | Method for supplementing a piece of object information assigned to an object and method for selecting objects in surroundings of a vehicle | |
| JP5718942B2 (en) | Apparatus and method for assisting safe operation of transportation means | |
| JP6350145B2 (en) | Face orientation detection device and vehicle warning system | |
| JP5184596B2 (en) | Armpit judging device | |
| JP4722777B2 (en) | Obstacle recognition judgment device | |
| JP6565408B2 (en) | Vehicle control apparatus and vehicle control method | |
| JPH08178712A (en) | Rambling-drive detection apparatus | |
| JP2017033542A (en) | Information display | |
| KR101999079B1 (en) | Display device control method and display device | |
| US20180037162A1 (en) | Driver assistance system | |
| US11008012B2 (en) | Driving consciousness estimation device | |
| CN114072865A (en) | Information processing apparatus, mobile apparatus, method, and program | |
| JP4882587B2 (en) | Vehicle information providing device | |
| JP2019199251A (en) | Method and driver assistance system for assisting driver in driving vehicle | |
| US20240000354A1 (en) | Driving characteristic determination device, driving characteristic determination method, and recording medium | |
| KR20200133858A (en) | Autonomous driving apparatus and method | |
| JP6259680B2 (en) | Safety confirmation determination device and driving support device | |
| JP2005063105A (en) | Device for determining inattentive driving | |
| EP3270764B1 (en) | Situational awareness monitoring method and apparatus | |
| JP2021140423A (en) | Vehicle controller | |
| JP5261045B2 (en) | Vehicle driving support device | |
| JP4345526B2 (en) | Object monitoring device | |
| US11685384B2 (en) | Driver alertness detection method, device and system | |
| JP2020071705A (en) | Concentrated state estimation device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| A621 | Written request for application examination |
Free format text: JAPANESE INTERMEDIATE CODE: A621 Effective date: 20210728 |
|
| A977 | Report on retrieval |
Free format text: JAPANESE INTERMEDIATE CODE: A971007 Effective date: 20220628 |
|
| A131 | Notification of reasons for refusal |
Free format text: JAPANESE INTERMEDIATE CODE: A131 Effective date: 20220705 |
|
| A521 | Request for written amendment filed |
Free format text: JAPANESE INTERMEDIATE CODE: A523 Effective date: 20220829 |
|
| TRDD | Decision of grant or rejection written | ||
| A01 | Written decision to grant a patent or to grant a registration (utility model) |
Free format text: JAPANESE INTERMEDIATE CODE: A01 Effective date: 20220920 |
|
| A61 | First payment of annual fees (during grant procedure) |
Free format text: JAPANESE INTERMEDIATE CODE: A61 Effective date: 20221018 |
|
| R150 | Certificate of patent or registration of utility model |
Ref document number: 7164275 Country of ref document: JP Free format text: JAPANESE INTERMEDIATE CODE: R150 |