JP2022038225A - 顔情報取得装置および顔情報取得方法 - Google Patents
顔情報取得装置および顔情報取得方法 Download PDFInfo
- Publication number
- JP2022038225A JP2022038225A JP2020142607A JP2020142607A JP2022038225A JP 2022038225 A JP2022038225 A JP 2022038225A JP 2020142607 A JP2020142607 A JP 2020142607A JP 2020142607 A JP2020142607 A JP 2020142607A JP 2022038225 A JP2022038225 A JP 2022038225A
- Authority
- JP
- Japan
- Prior art keywords
- face
- driver
- information
- input
- information acquisition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title description 11
- 238000012937 correction Methods 0.000 claims abstract description 55
- 230000006399 behavior Effects 0.000 claims description 28
- 230000001815 facial effect Effects 0.000 claims description 24
- 230000008921 facial expression Effects 0.000 claims description 7
- 238000003384 imaging method Methods 0.000 abstract description 2
- 210000001747 pupil Anatomy 0.000 description 13
- 230000011514 reflex Effects 0.000 description 11
- 238000004891 communication Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 238000013527 convolutional neural network Methods 0.000 description 4
- 230000008451 emotion Effects 0.000 description 3
- 210000000887 face Anatomy 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 125000002066 L-histidyl group Chemical group [H]N1C([H])=NC(C([H])([H])[C@](C(=O)[*])([H])N([H])[H])=C1[H] 0.000 description 1
- 238000004378 air conditioning Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 210000003128 head Anatomy 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
- B60K35/10—Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
- B60K35/20—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor
- B60K35/21—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor using visual output, e.g. blinking lights or matrix displays
- B60K35/22—Display screens
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
- B60K35/65—Instruments specially adapted for specific vehicle types or users, e.g. for left- or right-hand drive
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/014—Hand-worn input/output arrangements, e.g. data gloves
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/166—Detection; Localisation; Normalisation using acquisition arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/174—Facial expression recognition
- G06V40/176—Dynamic expression
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
- B60K2360/149—Instrument input by detecting viewing direction not otherwise provided for
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
- B60K2360/20—Optical features of instruments
- B60K2360/21—Optical features of instruments using cameras
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
- B60K2360/741—Instruments adapted for user detection
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2540/00—Input parameters relating to occupants
- B60W2540/225—Direction of gaze
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/01—Indexing scheme relating to G06F3/01
- G06F2203/011—Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- General Engineering & Computer Science (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
- Multimedia (AREA)
- Combustion & Propulsion (AREA)
- Chemical & Material Sciences (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Mathematical Physics (AREA)
- Automation & Control Theory (AREA)
- Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
- Image Processing (AREA)
Abstract
Description
5 顔情報取得装置
531 挙動取得部
532 入力画面表示部
533 補正情報取得部
Claims (6)
- 車両の運転者の顔を撮影した複数の顔画像のそれぞれから前記運転者の顔の挙動を表す顔情報を取得する挙動取得部と、
前記運転者から情報の入力を受け付ける入力欄を有する画面を前記車両に搭載された表示部に表示させる入力画面表示部と、
前記複数の顔画像のうち前記画面が前記表示部に表示されている間に生成された顔画像から、前記顔情報を補正するための補正情報を取得する補正情報取得部と、
を備える顔情報取得装置。 - 前記入力画面表示部は、前記入力欄が前記表示部の一端および他端に配置される画面を前記表示部に表示させ、
前記補正情報取得部は、前記表示部の一端に配置された前記入力欄が前記運転者からの情報の入力を受け付けている間に生成された顔画像から第1の補正情報を取得し、前記表示部の他端に配置された前記入力欄が前記運転者からの情報の入力を受け付けている間に生成された顔画像から第2の補正情報を取得する、請求項1に記載の顔情報取得装置。 - 前記顔情報は、前記運転者の視線方向を表す情報であり、
前記補正情報は、前記運転者の右目の視線方向と左目の視線方向との差異を表す情報である、
請求項1または2に記載の顔情報取得装置。 - 前記顔情報は、前記運転者の視線方向を表す情報であり、
前記補正情報は、前記運転者の目の3次元位置を表す情報である、
請求項1または2に記載の顔情報取得装置。 - 前記顔情報は、前記運転者の表情を表す情報であり、
前記補正情報は、前記運転者の無表情時の表情を表す情報である、
請求項1に記載の顔情報取得装置。 - 車両の運転者の顔を撮影した複数の顔画像のそれぞれから前記運転者の顔の挙動を表す顔情報を取得し、
前記運転者から情報の入力を受付ける入力欄を有する画面を前記車両に搭載された表示部に表示させ、
前記複数の顔画像のうち前記画面が前記表示部に表示されている間に生成された顔画像から、前記顔情報を補正するための補正情報を取得する、
ことを含む顔情報取得方法。
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2020142607A JP7367637B2 (ja) | 2020-08-26 | 2020-08-26 | 顔情報取得装置および顔情報取得方法 |
US17/406,311 US20220066551A1 (en) | 2020-08-26 | 2021-08-19 | Apparatus and method for obtaining facial information |
CN202110952306.7A CN114103961B (zh) | 2020-08-26 | 2021-08-19 | 面部信息获取装置以及面部信息获取方法 |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2020142607A JP7367637B2 (ja) | 2020-08-26 | 2020-08-26 | 顔情報取得装置および顔情報取得方法 |
Publications (2)
Publication Number | Publication Date |
---|---|
JP2022038225A true JP2022038225A (ja) | 2022-03-10 |
JP7367637B2 JP7367637B2 (ja) | 2023-10-24 |
Family
ID=80358563
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
JP2020142607A Active JP7367637B2 (ja) | 2020-08-26 | 2020-08-26 | 顔情報取得装置および顔情報取得方法 |
Country Status (3)
Country | Link |
---|---|
US (1) | US20220066551A1 (ja) |
JP (1) | JP7367637B2 (ja) |
CN (1) | CN114103961B (ja) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2023104608A (ja) * | 2022-01-18 | 2023-07-28 | トヨタ自動車株式会社 | ドライバ監視装置、ドライバ監視用コンピュータプログラム及びドライバ監視方法 |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2013125307A (ja) * | 2011-12-13 | 2013-06-24 | Panasonic Corp | 計測対象抽出装置、顔形状推定装置、計測対象抽出方法および顔形状推定方法 |
WO2015125243A1 (ja) * | 2014-02-19 | 2015-08-27 | 三菱電機株式会社 | 表示制御装置、表示制御装置の表示制御方法、視線方向検出システムおよび視線方向検出システムのキャリブレーション制御方法 |
WO2016139850A1 (ja) * | 2015-03-05 | 2016-09-09 | ソニー株式会社 | 情報処理装置、制御方法、およびプログラム |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2014125380A2 (en) * | 2013-02-14 | 2014-08-21 | The Eye Tribe Aps | Systems and methods of eye tracking calibration |
US9189095B2 (en) * | 2013-06-06 | 2015-11-17 | Microsoft Technology Licensing, Llc | Calibrating eye tracking system by touch input |
EP3292024A4 (en) * | 2015-05-06 | 2018-06-20 | Magna Mirrors of America, Inc. | Vehicle vision system with blind zone display and alert system |
JP6693489B2 (ja) * | 2017-11-15 | 2020-05-13 | オムロン株式会社 | 情報処理装置、運転者モニタリングシステム、情報処理方法、及び情報処理プログラム |
-
2020
- 2020-08-26 JP JP2020142607A patent/JP7367637B2/ja active Active
-
2021
- 2021-08-19 CN CN202110952306.7A patent/CN114103961B/zh active Active
- 2021-08-19 US US17/406,311 patent/US20220066551A1/en active Pending
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2013125307A (ja) * | 2011-12-13 | 2013-06-24 | Panasonic Corp | 計測対象抽出装置、顔形状推定装置、計測対象抽出方法および顔形状推定方法 |
WO2015125243A1 (ja) * | 2014-02-19 | 2015-08-27 | 三菱電機株式会社 | 表示制御装置、表示制御装置の表示制御方法、視線方向検出システムおよび視線方向検出システムのキャリブレーション制御方法 |
WO2016139850A1 (ja) * | 2015-03-05 | 2016-09-09 | ソニー株式会社 | 情報処理装置、制御方法、およびプログラム |
Also Published As
Publication number | Publication date |
---|---|
CN114103961A (zh) | 2022-03-01 |
CN114103961B (zh) | 2023-12-12 |
JP7367637B2 (ja) | 2023-10-24 |
US20220066551A1 (en) | 2022-03-03 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10311735B2 (en) | Vehicle display system and method of controlling vehicle display system | |
US9785235B2 (en) | Display control apparatus, display control method of display control apparatus, and eye gaze direction detection system | |
US9696814B2 (en) | Information processing device, gesture detection method, and gesture detection program | |
JPWO2010035472A1 (ja) | 視線方向判定装置及び視線方向判定方法 | |
CN110031975B (zh) | 在车辆中校准增强现实眼镜的方法和系统与增强现实眼镜 | |
JPWO2013008305A1 (ja) | 瞼検出装置 | |
WO2019176492A1 (ja) | 算出システム、情報処理装置、運転支援システム、指標算出方法、コンピュータプログラム、及び記憶媒体 | |
JP6669182B2 (ja) | 乗員監視装置 | |
CN109415020B (zh) | 辉度控制装置、辉度控制系统以及辉度控制方法 | |
JP6708152B2 (ja) | 運転者状態推定装置、及び運転者状態推定方法 | |
CN114103961B (zh) | 面部信息获取装置以及面部信息获取方法 | |
JP2017126151A (ja) | 視線検出装置および視線検出方法 | |
CN113879321B (zh) | 驾驶员监视装置及驾驶员监视方法 | |
KR101976498B1 (ko) | 차량용 제스처 인식 시스템 및 그 방법 | |
CN109866683A (zh) | 车辆用驾驶支援装置 | |
JP2016170688A (ja) | 運転支援装置及び運転支援システム | |
JP2021189852A (ja) | 状態判定装置および状態判定方法 | |
US20230394702A1 (en) | Device, method, and computer program for estimating seat position | |
JP7374373B2 (ja) | 体格判定装置および体格判定方法 | |
US20240029454A1 (en) | Line of sight estimating device, storage medium for storing computer program for line of sight estimation, and line of sight estimating method | |
JP2023097182A (ja) | ドライバ挙動判定装置、ドライバ挙動判定方法及びドライバ挙動判定用コンピュータプログラム | |
JP7301256B2 (ja) | 顔向き判定装置および顔向き判定方法 | |
CN114103991B (zh) | 行驶控制装置以及行驶控制方法 | |
JP2023085085A (ja) | 覚醒度判定装置 | |
CN114004889A (zh) | 一种融合头部姿态的三维眼球移动方向判断方法 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
A621 | Written request for application examination |
Free format text: JAPANESE INTERMEDIATE CODE: A621 Effective date: 20220610 |
|
A977 | Report on retrieval |
Free format text: JAPANESE INTERMEDIATE CODE: A971007 Effective date: 20230425 |
|
A131 | Notification of reasons for refusal |
Free format text: JAPANESE INTERMEDIATE CODE: A131 Effective date: 20230509 |
|
A521 | Request for written amendment filed |
Free format text: JAPANESE INTERMEDIATE CODE: A523 Effective date: 20230629 |
|
TRDD | Decision of grant or rejection written | ||
A01 | Written decision to grant a patent or to grant a registration (utility model) |
Free format text: JAPANESE INTERMEDIATE CODE: A01 Effective date: 20230912 |
|
A61 | First payment of annual fees (during grant procedure) |
Free format text: JAPANESE INTERMEDIATE CODE: A61 Effective date: 20230925 |
|
R151 | Written notification of patent or utility model registration |
Ref document number: 7367637 Country of ref document: JP Free format text: JAPANESE INTERMEDIATE CODE: R151 |