JP2011116210A - Gaze determination device - Google Patents

Gaze determination device Download PDF

Info

Publication number
JP2011116210A
JP2011116210A JP2009274457A JP2009274457A JP2011116210A JP 2011116210 A JP2011116210 A JP 2011116210A JP 2009274457 A JP2009274457 A JP 2009274457A JP 2009274457 A JP2009274457 A JP 2009274457A JP 2011116210 A JP2011116210 A JP 2011116210A
Authority
JP
Japan
Prior art keywords
angle
frequency distribution
face
face orientation
driver
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
JP2009274457A
Other languages
Japanese (ja)
Other versions
JP5042296B2 (en
Inventor
Shinsuke Ueda
信介 植田
Akio Takahashi
昭夫 高橋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Honda Motor Co Ltd
Original Assignee
Honda Motor Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Honda Motor Co Ltd filed Critical Honda Motor Co Ltd
Priority to JP2009274457A priority Critical patent/JP5042296B2/en
Publication of JP2011116210A publication Critical patent/JP2011116210A/en
Application granted granted Critical
Publication of JP5042296B2 publication Critical patent/JP5042296B2/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
  • Image Processing (AREA)
  • Traffic Control Systems (AREA)
  • Image Analysis (AREA)

Abstract

<P>PROBLEM TO BE SOLVED: To automatically and accurately determine an object to be gazed by a driver. <P>SOLUTION: This gaze determination device 10 includes: a frequency distribution detection part 22 for detecting the frequency distribution of face direction angles corresponding to predetermined ranges further to the lateral outer sides than pillars provided to both ends of the windshield of a vehicle; a frequency distribution storage part 23 which divides the predetermined ranges into first areas in contact with the pillars near in the direction from the pillars to right and left door mirrors, respectively, and second areas further to the lateral outer sides than the first areas, respectively, which stores the results of the detection of the frequency distribution for each first area and second area; an intermediate angle detection part 25 for detecting the intermediate angle between the face direction angle at which the frequency distribution is maximum in the first area and the face direction angle at which the frequency distribution is maximum in the second area; and a gaze object determination part 26 for determining that the object to be gazed is a door mirror when the face direction angle is present in the angle range between the angle corresponding to each pillar and the intermediate angle. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

本発明は、注視判定装置に関する。   The present invention relates to a gaze determination apparatus.

従来、例えば運転者の顔を撮影した撮影画像上において、運転者が左右のドアミラー又はルームミラーを見たときの眼の位置を検出し、検出結果に基づき脇見許容時間に応じた所定の画像領域の左右端を決定する脇見検出装置が知られている（例えば、特許文献１参照）。
また、従来、例えば運転者により左右のアウターミラーが操作されているときの運転者の顔向き角度を予め検出して記憶しておき、記憶した顔向き角度に基づき、運転中の顔向き角度を判定し、運転者が余所見をしているか否かを判定する余所見判定装置が知られている（例えば、特許文献２参照）。 Conventionally, for example, on a photographed image obtained by photographing a driver's face, the position of the eye when the driver looks at the left and right door mirrors or room mirrors is detected, and a predetermined image area corresponding to the allowable time for looking aside based on the detection result There is known an armpit detection device that determines the left and right ends of the image (for example, see Patent Document 1).
Conventionally, for example, the driver's face angle when the left and right outer mirrors are operated by the driver is detected and stored in advance, and the face angle during driving is determined based on the stored face angle. There is known an aftersight determination device that determines and determines whether or not a driver is making an observation (see, for example, Patent Document 2).

特許第４３０５０５７号公報Japanese Patent No. 4305057 特開２００９−９２４４号公報JP 2009-9244 A

ところで、上記従来技術に係る脇見検出装置においては、運転者の眼の位置又は視線を検出することが困難な場合には運転者の脇見を検知することが出来ないという問題が生じる。
また、上記従来技術に係る余所見判定装置においては、運転者により左右のアウターミラーが操作されているときの運転者の顔向き角度が予め検出されていない場合には、運転者が余所見をしているか否かを判定することが出来ないという問題が生じる。 By the way, in the above-described prior art aside detection device, when it is difficult to detect the position or line of sight of the driver's eyes, there arises a problem that the driver's aside cannot be detected.
Further, in the above-described prior art determination device according to the prior art, when the driver's face angle when the left and right outer mirrors are operated by the driver is not detected in advance, the driver makes a redundant look. There arises a problem that it cannot be determined whether or not.

本発明は上記事情に鑑みてなされたもので、運転者の注視対象を自動的かつ的確に判定することが可能な注視判定装置を提供することを目的とする。   The present invention has been made in view of the above circumstances, and an object thereof is to provide a gaze determination apparatus capable of automatically and accurately determining a driver's gaze target.

上記課題を解決して係る目的を達成するために、本発明の第１態様に係る注視判定装置は、車両の運転席に着座した運転者の顔を撮像して顔画像を出力する撮像手段（例えば、実施の形態での乗員カメラ１１）と、該撮像手段から出力された前記顔画像に基づき前記運転者の顔向き角度を検知して検知結果を出力する顔向き検知手段（例えば、実施の形態での顔向き検知部２１）と、該顔向き検知手段から出力された前記顔向き角度の検知結果に基づいて前記運転者の注視対象を判定して判定結果を出力する注視対象判定手段（例えば、実施の形態での注視対象判定部２６）とを備える注視判定装置であって、前記顔向き検知手段から出力された前記顔向き角度の検知結果から、前記車両のフロントガラス（例えば、実施の形態でのフロントガラス３１）両端に設けられた各ピラー（例えば、実施の形態でのピラー３２）よりも左右の外側方向の所定範囲内に対応する前記顔向き角度の頻度分布を検出して検出結果を出力する頻度分布検出手段（例えば、実施の形態での頻度分布検出部２２）と、前記所定範囲を前記各ピラーから左右のドアミラー（例えば、実施の形態でのドアミラー３３）に向かう方向の近傍で前記各ピラーに接する第１領域と該第１領域よりも左右の外側方向の第２領域とに区分し、前記頻度分布検出手段から出力された前記頻度分布の検出結果を前記第１領域及び前記第２領域毎に記憶する記憶手段（例えば、実施の形態での頻度分布記憶部２３）と、前記記憶手段に記憶された前記頻度分布の検出結果から、前記第１領域及び前記第２領域毎に前記頻度分布が最大となる前記顔向き角度を検出して検出結果を出力する最大頻度検出手段（例えば、実施の形態での最大頻度検出部２４）と、前記最大頻度検出手段から出力された前記顔向き角度の前記検出結果から、前記第１領域にて前記頻度分布が最大となる前記顔向き角度と前記第２領域にて前記頻度分布が最大となる前記顔向き角度との略中間の角度を検出して検出結果を出力する中間角度検出手段（例えば、実施の形態での中間角度検出部２５）とを備え、前記注視対象判定手段は、前記顔向き検知手段から出力された前記顔向き角度の検知結果において、前記顔向き角度が前記各ピラーに対応する角度と前記略中間の角度との間の角度範囲に存在する場合には、前記注視対象がドアミラーであると判定する。   In order to solve the above-described problems and achieve the object, the gaze determination apparatus according to the first aspect of the present invention is an imaging unit that images a driver's face seated in a driver's seat of a vehicle and outputs a face image. For example, the occupant camera 11) in the embodiment and the face orientation detection means (for example, implementation) that detects the driver's face orientation angle based on the face image output from the imaging means and outputs a detection result. Face orientation detection unit 21) in the form, and gaze target determination means for determining the driver's gaze target based on the detection result of the face orientation angle output from the face orientation detection means and outputting the determination result ( For example, a gaze determination apparatus including the gaze target determination unit 26) according to the embodiment, wherein the vehicle windshield (for example, implementation) is detected based on the detection result of the face direction angle output from the face direction detection unit. Front in the form of The lath 31) detects the frequency distribution of the face orientation angle corresponding to a predetermined range in the left and right outer directions with respect to the pillars provided at both ends (for example, the pillar 32 in the embodiment), and outputs a detection result. The frequency distribution detecting means (for example, the frequency distribution detecting unit 22 in the embodiment) and the predetermined ranges in the vicinity of the respective directions from the respective pillars to the left and right door mirrors (for example, the door mirror 33 in the embodiment). The first region and the second region are divided into a first region in contact with the pillar and a second region on the left and right outside of the first region, and the frequency distribution detection result output from the frequency distribution detector is used as the first region and the second region. From the storage means for storing each area (for example, the frequency distribution storage unit 23 in the embodiment) and the detection result of the frequency distribution stored in the storage means, the first area and the second area Frequency Maximum frequency detection means (for example, maximum frequency detection unit 24 in the embodiment) that detects the face orientation angle at which the maximum is detected and outputs a detection result, and the face orientation angle output from the maximum frequency detection means From the detection result, a substantially intermediate angle between the face orientation angle at which the frequency distribution becomes maximum in the first region and the face orientation angle at which the frequency distribution becomes maximum in the second region is detected. Intermediate angle detection means (for example, the intermediate angle detection unit 25 in the embodiment) for outputting the detection result, and the gaze target determination means detects the face orientation angle output from the face orientation detection means. As a result, when the face orientation angle is in an angle range between the angle corresponding to each pillar and the substantially intermediate angle, it is determined that the gaze target is a door mirror.

本発明の第１態様に係る注視判定装置によれば、運転者がドアミラーの調整操作などの機器操作を行なわない場合であっても、自動的にドアミラーの位置を運転者の顔向きから検出することができ、利便性を向上させることができる。
また、所定範囲内に対応する顔向き角度の頻度分布の蓄積により第１領域および第２領域を設定することから、例えば運転者の姿勢が変化した場合などであっても、自動的に追従して適切な第１領域および第２領域を設定することができる。 According to the gaze determination apparatus according to the first aspect of the present invention, the position of the door mirror is automatically detected from the face direction of the driver even when the driver does not perform an apparatus operation such as a door mirror adjustment operation. And convenience can be improved.
In addition, since the first region and the second region are set by accumulating the frequency distribution of the face orientation angles corresponding to the predetermined range, for example, even when the driver's posture changes, it automatically follows. Thus, the appropriate first area and second area can be set.

本発明の実施の形態に係る注視判定装置の構成図である。It is a lineblock diagram of a gaze judging device concerning an embodiment of the invention. 本発明の実施の形態に係る第１領域Ａ及び第２領域Ｂの例を示す図である。It is a figure which shows the example of the 1st area | region A and the 2nd area | region B which concern on embodiment of this invention. 本発明の実施の形態に係る各第１領域Ａ及び第２領域Ｂでの顔向き角度の頻度分布の例を示す図である。It is a figure which shows the example of the frequency distribution of the face direction angle in each 1st area | region A and 2nd area | region B which concerns on embodiment of this invention. 本発明の実施の形態に係る注視判定装置の動作を示すフローチャートである。It is a flowchart which shows operation | movement of the gaze determination apparatus which concerns on embodiment of this invention. 図４に示す判定角度設定の処理を示すフローチャートである。It is a flowchart which shows the process of the determination angle setting shown in FIG.

以下、本発明の注視判定装置の一実施形態について添付図面を参照しながら説明する。
本実施の形態による注視判定装置１０は、例えば図１に示すように、乗員カメラ１１と、処理装置１２とを備えて構成され、さらに、処理装置１２は、顔向き検知部２１と、頻度分布検出部２２と、頻度分布記憶部２３と、最大頻度検出部２４と、中間角度検出部２５と、注視対象判定部２６とを備えて構成されている。 Hereinafter, an embodiment of a gaze determination apparatus according to the present invention will be described with reference to the accompanying drawings.
As shown in FIG. 1, for example, the gaze determination device 10 according to the present embodiment includes an occupant camera 11 and a processing device 12. The processing device 12 further includes a face orientation detection unit 21, a frequency distribution, and the like. The detection unit 22, the frequency distribution storage unit 23, the maximum frequency detection unit 24, the intermediate angle detection unit 25, and the gaze target determination unit 26 are configured.

乗員カメラ１１は、少なくとも車両の運転席に着座した運転者の顔を、撮像対象として、例えば可視光領域または赤外線領域にて撮像可能であって、運転者の顔を含む顔画像を出力する。   The occupant camera 11 can capture at least a driver's face seated in the driver's seat of the vehicle as an imaging target, for example, in a visible light region or an infrared region, and outputs a face image including the driver's face.

処理装置１２の顔向き検知部２１は、乗員カメラ１１から出力された顔画像に基づき、例えば運転者の顔を検知対象とした所定の認識処理を行ない、認識した顔に基づき、所定の基準方向（例えば、運転席の正面方向など）に対する運転者の顔向きの角度（顔向き角度）を検知して検知結果を出力する。   The face orientation detection unit 21 of the processing device 12 performs a predetermined recognition process based on the face image output from the occupant camera 11, for example, with the driver's face as a detection target, and a predetermined reference direction based on the recognized face. The angle of the driver's face (for example, the front direction of the driver's seat) is detected and the detection result is output.

頻度分布検出部２２は、顔向き検知部２１から出力された運転者の顔向き角度の頻度分布を、例えば図２に示すように、車両のフロントガラス３１両端に設けられたピラー３２よりも左右の外側方向の所定範囲（学習範囲）内に対して検出して検出結果を出力する。
例えば、頻度分布検出部２２は、所定範囲（学習範囲）内に対応する角度範囲で運転者の顔向き角度が所定時間以上に亘って停留した場合に、この顔向き角度の頻度の度数を増大更新する。 The frequency distribution detection unit 22 displays the frequency distribution of the driver's face angle output from the face direction detection unit 21 on the left and right sides of the pillars 32 provided at both ends of the windshield 31 of the vehicle, for example, as shown in FIG. Is detected within a predetermined range (learning range) in the outer direction of the output and a detection result is output.
For example, the frequency distribution detection unit 22 increases the frequency of the frequency of the face orientation when the driver's face orientation angle is stopped for a predetermined time or more in an angle range corresponding to a predetermined range (learning range). Update.

頻度分布記憶部２３は、頻度分布検出部２２から出力された所定範囲内での運転者の顔向き角度の頻度分布を、例えばピラー３２から左右のドアミラー３３に向かう方向の近傍でピラー３２に接する第１領域Ａでの頻度分布と、この第１領域Ａよりも左右の外側方向の第２領域Ｂでの頻度分布とに区分して記憶する。
なお、第１領域Ａは、少なくとも左右のドアミラー３３を内部に含む領域であって、第２領域Ｂは、少なくとも左右のドアミラー３３よりも左右の外側方向であって、左右のドアガラス３４の一部を含む領域である。 The frequency distribution storage unit 23 contacts the pillar 32 with the frequency distribution of the driver's face orientation angle within a predetermined range output from the frequency distribution detection unit 22, for example, in the vicinity of the direction from the pillar 32 toward the left and right door mirrors 33. The frequency distribution in the first area A and the frequency distribution in the second area B in the left and right outer direction than the first area A are stored separately.
The first area A is an area that includes at least the left and right door mirrors 33 inside, and the second area B is at least the left and right outer directions of the left and right door mirrors 33, and is one of the left and right door glasses 34. It is an area including a part.

最大頻度検出部２４は、頻度分布記憶部２３に記憶された各領域Ａ，Ｂ毎において、例えば頻度分布の標本数が所定閾値以上に到達した以後に、顔向き角度の頻度の度数が増大更新される都度に、運転者の顔向き角度の頻度分布が最大となる顔向き角度を検出して検出結果を出力する。   The maximum frequency detection unit 24 increases and updates the frequency of the face orientation angle in each of the regions A and B stored in the frequency distribution storage unit 23, for example, after the number of frequency distribution samples reaches a predetermined threshold or more. Each time, the face orientation angle that maximizes the frequency distribution of the driver's face orientation angle is detected, and the detection result is output.

中間角度検出部２５は、例えば図３に示すように、最大頻度検出部２４から出力された各領域Ａ，Ｂ毎での運転者の顔向き角度の頻度分布が最大となる顔向き角度（最大頻度角度）θＡ，θＢの略中間の角度、例えば中点の角度（（θＡ＋θＢ）／２）を検出して、この検出結果を判定角度として出力する。   For example, as shown in FIG. 3, the intermediate angle detection unit 25 has a face direction angle (maximum of the frequency distribution of the driver's face direction angle for each of the regions A and B output from the maximum frequency detection unit 24. Frequency angle) An approximately intermediate angle between θA and θB, for example, a midpoint angle ((θA + θB) / 2) is detected, and the detection result is output as a determination angle.

注視対象判定部２６は、中間角度検出部２５から出力された判定角度に基づき、顔向き検知部２１から出力された運転者の顔向き角度が、左右のピラー３２に対応する顔向き角度と判定角度との間の角度範囲に存在するか否かを判定し、この角度範囲内に存在すると判定した場合には、運転者の注視対象が左右のドアミラー３３であると判定して、この判定結果を出力する。   The gaze target determination unit 26 determines that the driver's face direction angle output from the face direction detection unit 21 is the face direction angle corresponding to the left and right pillars 32 based on the determination angle output from the intermediate angle detection unit 25. It is determined whether or not it exists in the angle range between the angles, and if it is determined that it exists in this angle range, it is determined that the driver's gaze target is the left and right door mirrors 33, and this determination result Is output.

この実施の形態による注視判定装置１０は上記構成を備えており、次に、この注視判定装置１０の動作について説明する。   The gaze determination apparatus 10 according to this embodiment has the above-described configuration. Next, the operation of the gaze determination apparatus 10 will be described.

先ず、例えば図４に示すステップＳ０１においては、車両の運転席に着座した運転者の顔画像を取得する。
次に、ステップＳ０２においては、顔画像に基づき、運転者の顔向き角度を検出する。
次に、ステップＳ０３においては、運転者の顔向きの方向が左（例えば、運転席の正面から左）であるか否かを判定する。
この判定結果が「ＹＥＳ」の場合には、ステップＳ０４に進み、このステップＳ０４においては、左のドアミラー３３に対して、後述する判定角度設定の処理を行なう。
一方、この判定結果が「ＮＯ」の場合には、ステップＳ０５に進み、このステップＳ０５においては、右のドアミラー３３に対して、後述する判定角度設定の処理を行なう。 First, for example, in step S01 shown in FIG. 4, a face image of the driver seated in the driver's seat of the vehicle is acquired.
Next, in step S02, the driver's face orientation angle is detected based on the face image.
Next, in step S03, it is determined whether the direction of the driver's face is left (for example, left from the front of the driver's seat).
If this determination is “YES”, the flow proceeds to step S 04, and in this step S 04, a determination angle setting process described later is performed on the left door mirror 33.
On the other hand, if this determination is “NO”, the flow proceeds to step S 05, and in this step S 05, determination angle setting processing described later is performed on the right door mirror 33.

次に、ステップＳ０６においては、顔向き検知部２１から出力された運転者の顔向き角度が、左右のピラー３２に対応する顔向き角度と判定角度との間の角度範囲に存在するか否かを判定する。
この判定結果が「ＹＥＳ」の場合には、ステップＳ０７に進み、このステップＳ０７においては、運転者の注視対象がドアミラー３３であると判定して、エンドに進む。
一方、この判定結果が「ＮＯ」の場合には、ステップＳ０８に進み、このステップＳ０８においては、運転者の注視対象がドアミラー３３以外であると判定して、エンドに進む。 Next, in step S06, whether or not the driver's face orientation angle output from the face orientation detector 21 is within an angle range between the face orientation angle corresponding to the left and right pillars 32 and the determination angle. Determine.
If this determination result is "YES", the process proceeds to step S07, and in this step S07, it is determined that the driver's gaze target is the door mirror 33, and the process proceeds to the end.
On the other hand, if the determination result is “NO”, the process proceeds to step S08. In step S08, it is determined that the driver's gaze target is other than the door mirror 33, and the process proceeds to the end.

以下に、上述したステップＳ０４，Ｓ０５での判定角度設定の処理について説明する。
先ず、例えば図５に示すステップＳ１１においては、運転者の顔向き角度は車両のフロントガラス３１両端に設けられたピラー３２よりも左右の外側方向の所定範囲（学習範囲）内に対応する角度であるか否かを判定する。
この判定結果が「ＮＯ」の場合には、エンドに進む。
一方、この判定結果が「ＹＥＳ」の場合には、ステップＳ１２に進む。 The determination angle setting process in steps S04 and S05 described above will be described below.
First, for example, in step S11 shown in FIG. 5, the driver's face direction angle is an angle corresponding to a predetermined range (learning range) in the lateral direction outside the pillars 32 provided at both ends of the windshield 31 of the vehicle. It is determined whether or not there is.
If this determination is “NO”, the flow proceeds to the end.
On the other hand, if this determination is “YES”, the flow proceeds to step S12.

次に、ステップＳ１２においては、所定範囲（学習範囲）内に対応する角度範囲で運転者の顔向き角度が所定時間以上に亘って停留したか否かを判定する。
この判定結果が「ＮＯ」の場合には、エンドに進む。
一方、この判定結果が「ＹＥＳ」の場合には、ステップＳ１３に進む。
次に、ステップＳ１３においては、運転者の顔向き角度が第１領域Ａ内に対応する角度範囲内であるか否かを判定する。
この判定結果が「ＹＥＳ」の場合には、ステップＳ１４に進み、このステップＳ１４においては、第１領域Ａの頻度分布、つまり顔向き角度の頻度の度数を増大更新する。
一方、この判定結果が「ＮＯ」の場合には、ステップＳ１５に進み、このステップＳ１５においては、第２領域Ｂの頻度分布、つまり顔向き角度の頻度の度数を増大更新する。 Next, in step S12, it is determined whether or not the driver's face angle has been stopped for a predetermined time or more in an angle range corresponding to a predetermined range (learning range).
If this determination is “NO”, the flow proceeds to the end.
On the other hand, if the determination is “YES”, the flow proceeds to step S13.
Next, in step S13, it is determined whether or not the driver's face angle is within an angle range corresponding to the first area A.
If the determination result is “YES”, the process proceeds to step S14, and in this step S14, the frequency distribution of the first region A, that is, the frequency of the face orientation angle is increased and updated.
On the other hand, if this determination is “NO”, the flow proceeds to step S 15, and in this step S 15, the frequency distribution of the second region B, that is, the frequency of the face orientation angle is increased and updated.

次に、ステップＳ１６においては、各領域Ａ，Ｂ毎において頻度分布の標本数が所定閾値以上に到達したか否かを判定する。
この判定結果が「ＮＯ」の場合には、エンドに進む。
一方、この判定結果が「ＹＥＳ」の場合には、ステップＳ１７に進む。
そして、ステップＳ１７においては、頻度分布の標本数が所定閾値以上に到達した以後での運転者の顔向き角度の頻度分布が最大となる顔向き角度を検出する。 Next, in step S16, it is determined whether or not the number of frequency distribution samples has reached a predetermined threshold value or more for each of the regions A and B.
If this determination is “NO”, the flow proceeds to the end.
On the other hand, if this determination is “YES”, the flow proceeds to step S17.
In step S17, the face orientation angle at which the frequency distribution of the driver's face orientation angle after the number of samples of the frequency distribution reaches a predetermined threshold or more is detected.

次に、ステップＳ１８においては、両領域Ａ，Ｂにおいて運転者の顔向き角度の頻度分布が最大となる顔向き角度の検出が完了したか否かを判定する。
この判定結果が「ＮＯ」の場合には、エンドに進む。
一方、この判定結果が「ＹＥＳ」の場合には、ステップＳ１９に進む。
そして、ステップＳ１９においては、各領域Ａ，Ｂ毎での運転者の顔向き角度の頻度分布が最大となる顔向き角度（最大頻度角度）θＡ，θＢの中点の角度（（θＡ＋θＢ）／２）を検出して、この検出結果を判定角度として設定し、エンドに進む。 Next, in step S18, it is determined whether or not the detection of the face orientation angle that maximizes the frequency distribution of the driver's face orientation angle in both areas A and B is completed.
If this determination is “NO”, the flow proceeds to the end.
On the other hand, if the determination is “YES”, the flow proceeds to step S19.
In step S19, the angle at the midpoint between the face orientation angles (maximum frequency angles) θA and θB ((θA + θB) / 2) at which the frequency distribution of the driver's face orientation angle in each of the regions A and B is maximized. ) Is detected, the detection result is set as a determination angle, and the process proceeds to the end.

上述したように、本実施の形態による注視判定装置１０によれば、運転者がドアミラー３３の調整操作などの機器操作を行なわない場合であっても、自動的にドアミラー３３の位置を運転者の顔向きから検出することができ、利便性を向上させることができる。
また、所定範囲内に対応する顔向き角度の頻度分布の蓄積により第１領域Ａおよび第２領域Ｂを設定することから、例えば運転者の姿勢が変化した場合などであっても、自動的に追従して適切な第１領域Ａおよび第２領域Ｂを設定することができる。 As described above, according to the gaze determination device 10 according to the present embodiment, the position of the door mirror 33 is automatically set to the position of the driver even when the driver does not perform an apparatus operation such as an adjustment operation of the door mirror 33. It can be detected from the face direction, and convenience can be improved.
In addition, since the first area A and the second area B are set by accumulating the frequency distribution of the face orientation angles corresponding to the predetermined range, for example, even when the driver's posture changes, for example, It is possible to follow and set the appropriate first area A and second area B.

なお、上述した実施の形態においては、例えば車両の始動時などにおいて、例えばイグニッションがＯＮとされてから判定角度が設定されるまでの期間においては、単に、運転者の顔向き角度が第１領域Ａに対応する角度範囲内に存在するか否かに応じて、運転者の注視対象がドアミラー３３であるか否かを判定してもよい。   In the above-described embodiment, for example, when the vehicle is started, for example, during the period from when the ignition is turned on until the determination angle is set, the driver's face angle is simply the first region. Whether or not the driver's gaze target is the door mirror 33 may be determined according to whether or not the angle is within an angle range corresponding to A.

１０ 注視判定装置
１１ 乗員カメラ（撮像手段）
２１ 顔向き検知部（顔向き検知手段）
２２ 頻度分布検出部（頻度分布検出手段）
２３ 頻度分布記憶部（記憶手段）
２４ 最大頻度検出部（最大頻度検出手段）
２５ 中間角度検出部（中間角度検出手段）
２６ 注視対象判定部（注視対象判定手段）
３１ フロントガラス
３２ ピラー
３３ ドアミラー 10 Gaze determination device 11 Crew camera (imaging means)
21 Face orientation detection unit (face orientation detection means)
22 Frequency distribution detection unit (frequency distribution detection means)
23 Frequency distribution storage unit (storage means)
24 Maximum frequency detection unit (maximum frequency detection means)
25 Intermediate angle detector (intermediate angle detector)
26 Gaze target determination unit (Gaze target determination means)
31 windshield 32 pillar 33 door mirror

Claims (1)

車両の運転席に着座した運転者の顔を撮像して顔画像を出力する撮像手段と、該撮像手段から出力された前記顔画像に基づき前記運転者の顔向き角度を検知して検知結果を出力する顔向き検知手段と、該顔向き検知手段から出力された前記顔向き角度の検知結果に基づいて前記運転者の注視対象を判定して判定結果を出力する注視対象判定手段とを備える注視判定装置であって、
前記顔向き検知手段から出力された前記顔向き角度の検知結果から、前記車両のフロントガラス両端に設けられた各ピラーよりも左右の外側方向の所定範囲内に対応する前記顔向き角度の頻度分布を検出して検出結果を出力する頻度分布検出手段と、
前記所定範囲を前記各ピラーから左右のドアミラーに向かう方向の近傍で前記各ピラーに接する第１領域と該第１領域よりも左右の外側方向の第２領域とに区分し、前記頻度分布検出手段から出力された前記頻度分布の検出結果を前記第１領域及び前記第２領域毎に記憶する記憶手段と、
前記記憶手段に記憶された前記頻度分布の検出結果から、前記第１領域及び前記第２領域毎に前記頻度分布が最大となる前記顔向き角度を検出して検出結果を出力する最大頻度検出手段と、
前記最大頻度検出手段から出力された前記顔向き角度の前記検出結果から、前記第１領域にて前記頻度分布が最大となる前記顔向き角度と前記第２領域にて前記頻度分布が最大となる前記顔向き角度との略中間の角度を検出して検出結果を出力する中間角度検出手段とを備え、
前記注視対象判定手段は、前記顔向き検知手段から出力された前記顔向き角度の検知結果において、前記顔向き角度が前記各ピラーに対応する角度と前記略中間の角度との間の角度範囲に存在する場合には、前記注視対象がドアミラーであると判定することを特徴とする注視判定装置。 An imaging unit that images a driver's face seated in a driver's seat of the vehicle and outputs a face image; and a detection result obtained by detecting the driver's face orientation angle based on the face image output from the imaging unit. A gaze comprising: a face orientation detection means for outputting; and a gaze target judgment means for judging the driver's gaze target based on the detection result of the face orientation angle output from the face orientation detection means and outputting a judgment result. A determination device,
From the detection result of the face orientation angle output from the face orientation detection means, the frequency distribution of the face orientation angle corresponding to a predetermined range in the outer left and right directions with respect to the pillars provided at both ends of the windshield of the vehicle. A frequency distribution detecting means for detecting the output and outputting a detection result;
The predetermined range is divided into a first region in contact with each pillar in the vicinity of the direction from each pillar to the left and right door mirrors and a second region in the left and right outer direction with respect to the first region, and the frequency distribution detecting means Storage means for storing the detection result of the frequency distribution output from each of the first region and the second region;
Maximum frequency detection means for detecting the face orientation angle at which the frequency distribution is maximized for each of the first area and the second area from the detection result of the frequency distribution stored in the storage means and outputting the detection result When,
From the detection result of the face orientation angle output from the maximum frequency detector, the face orientation angle at which the frequency distribution is maximized in the first region and the frequency distribution is maximized in the second region. An intermediate angle detecting means for detecting a substantially intermediate angle with the face orientation angle and outputting a detection result;
In the detection result of the face direction angle output from the face direction detection unit, the gaze target determination unit has an angle range between the angle corresponding to each pillar and the substantially intermediate angle in the detection result of the face direction angle. If present, the gaze determination apparatus determines that the gaze target is a door mirror.
JP2009274457A 2009-12-02 2009-12-02 Gaze determination device Expired - Fee Related JP5042296B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2009274457A JP5042296B2 (en) 2009-12-02 2009-12-02 Gaze determination device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2009274457A JP5042296B2 (en) 2009-12-02 2009-12-02 Gaze determination device

Publications (2)

Publication Number Publication Date
JP2011116210A true JP2011116210A (en) 2011-06-16
JP5042296B2 JP5042296B2 (en) 2012-10-03

Family

ID=44282079

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2009274457A Expired - Fee Related JP5042296B2 (en) 2009-12-02 2009-12-02 Gaze determination device

Country Status (1)

Country Link
JP (1) JP5042296B2 (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013030078A (en) * 2011-07-29 2013-02-07 Secom Co Ltd Face image authentication apparatus
US20130076881A1 (en) * 2011-09-26 2013-03-28 Honda Motor Co., Ltd. Facial direction detecting apparatus
WO2013051307A1 (en) * 2011-10-06 2013-04-11 本田技研工業株式会社 Warning device
JP2015085719A (en) * 2013-10-28 2015-05-07 株式会社デンソーアイティーラボラトリ Gazing object estimation device
JP2021077139A (en) * 2019-11-11 2021-05-20 マツダ株式会社 Driver state estimating device

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH09238905A (en) * 1996-03-05 1997-09-16 Nissan Motor Co Ltd Apparatus for measurement of direction of visual axis
JP2001357498A (en) * 2000-06-15 2001-12-26 Mazda Motor Corp Information providing device for vehicle
JP2005136541A (en) * 2003-10-29 2005-05-26 Sharp Corp Apparatus and method for stereoscopic image display
JP2007069756A (en) * 2005-09-07 2007-03-22 Toyota Motor Corp Vehicle input operation restricting device
JP2007225282A (en) * 2004-03-31 2007-09-06 Pioneer Electronic Corp Information presentation device, information presentation program, and information presentation method or the like
JP2008013070A (en) * 2006-07-06 2008-01-24 Toyota Motor Corp Vehicular display device
JP2008037201A (en) * 2006-08-03 2008-02-21 Toyota Motor Corp Drive assist device
JP2008162550A (en) * 2007-01-04 2008-07-17 Toyota Motor Corp External environment display device

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH09238905A (en) * 1996-03-05 1997-09-16 Nissan Motor Co Ltd Apparatus for measurement of direction of visual axis
JP2001357498A (en) * 2000-06-15 2001-12-26 Mazda Motor Corp Information providing device for vehicle
JP2005136541A (en) * 2003-10-29 2005-05-26 Sharp Corp Apparatus and method for stereoscopic image display
JP2007225282A (en) * 2004-03-31 2007-09-06 Pioneer Electronic Corp Information presentation device, information presentation program, and information presentation method or the like
JP2007069756A (en) * 2005-09-07 2007-03-22 Toyota Motor Corp Vehicle input operation restricting device
JP2008013070A (en) * 2006-07-06 2008-01-24 Toyota Motor Corp Vehicular display device
JP2008037201A (en) * 2006-08-03 2008-02-21 Toyota Motor Corp Drive assist device
JP2008162550A (en) * 2007-01-04 2008-07-17 Toyota Motor Corp External environment display device

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013030078A (en) * 2011-07-29 2013-02-07 Secom Co Ltd Face image authentication apparatus
US20130076881A1 (en) * 2011-09-26 2013-03-28 Honda Motor Co., Ltd. Facial direction detecting apparatus
WO2013051307A1 (en) * 2011-10-06 2013-04-11 本田技研工業株式会社 Warning device
CN103843047A (en) * 2011-10-06 2014-06-04 本田技研工业株式会社 Warning device
JP5661942B2 (en) * 2011-10-06 2015-01-28 本田技研工業株式会社 Alarm device
US9221386B2 (en) 2011-10-06 2015-12-29 Honda Motor Co., Ltd. Warning device for judging whether occupant is visually distracted
JP2015085719A (en) * 2013-10-28 2015-05-07 株式会社デンソーアイティーラボラトリ Gazing object estimation device
JP2021077139A (en) * 2019-11-11 2021-05-20 マツダ株式会社 Driver state estimating device
JP7276083B2 (en) 2019-11-11 2023-05-18 マツダ株式会社 Driver state estimation device

Also Published As

Publication number Publication date
JP5042296B2 (en) 2012-10-03

Similar Documents

Publication Publication Date Title
JP4888838B2 (en) Driver imaging device and driver imaging method
JP5042296B2 (en) Gaze determination device
US20180335633A1 (en) Viewing direction detector and viewing direction detection system
JP5092776B2 (en) Gaze direction detection device and gaze direction detection method
EP3566904B1 (en) Image display apparatus
JP5093611B2 (en) Vehicle periphery confirmation device
KR102091101B1 (en) Image display apparatus
WO2015079657A1 (en) Viewing area estimation device
JP2010160606A (en) Information presentation system for vehicle
JP2016168877A (en) Visual recognition device for vehicle
WO2019187283A1 (en) Image processing device, image display system, and image processing method
CN109415020B (en) Luminance control device, luminance control system and luminance control method
JP2018022958A (en) Vehicle display controller and vehicle monitor system
JP2011087006A (en) Display device for vehicle
JP2017091013A (en) Driving support device
JP2009248812A (en) Driving assistance device
JP2010179817A (en) Anti-dazzling device for vehicle
WO2018154857A1 (en) Vehicle display control device, vehicle display control system, vehicle display control method, and program
JP2010076480A (en) Driving assistance system for vehicle
JP2017181588A (en) Projection device, shield, projection method and projection program
JP2018162030A (en) Display device for vehicle
KR102010407B1 (en) Smart Rear-view System
KR20150073676A (en) Apparatus and method for correcting distortion of vehicle
JP2017061216A (en) On-board imaging system, vehicle and imaging method
JP2018084767A (en) Display control device, control method, program, and storage medium

Legal Events

Date Code Title Description
TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20120612

A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20120614

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20120710

R150 Certificate of patent or registration of utility model

Ref document number: 5042296

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

Free format text: JAPANESE INTERMEDIATE CODE: R150

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20150720

Year of fee payment: 3

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

LAPS Cancellation because of no payment of annual fees