JP7011578B2 - 運転行動を監視する方法及びシステム - Google Patents

運転行動を監視する方法及びシステム Download PDF

Info

Publication number
JP7011578B2
JP7011578B2 JP2018508721A JP2018508721A JP7011578B2 JP 7011578 B2 JP7011578 B2 JP 7011578B2 JP 2018508721 A JP2018508721 A JP 2018508721A JP 2018508721 A JP2018508721 A JP 2018508721A JP 7011578 B2 JP7011578 B2 JP 7011578B2
Authority
JP
Japan
Prior art keywords
features
video
driver
face
video data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2018508721A
Other languages
English (en)
Japanese (ja)
Other versions
JP2018528536A (ja
JP2018528536A5 (enExample
Inventor
タムラカール，アミール
ホ，グレゴリー
サルター，デビッド
ホアン，ジーフア
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
SRI International Inc
Original Assignee
SRI International Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by SRI International Inc filed Critical SRI International Inc
Publication of JP2018528536A publication Critical patent/JP2018528536A/ja
Publication of JP2018528536A5 publication Critical patent/JP2018528536A5/ja
Application granted granted Critical
Publication of JP7011578B2 publication Critical patent/JP7011578B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G06V40/167—Detection; Localisation; Normalisation using comparisons between temporally consecutive images
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/02—Alarms for ensuring the safety of persons
    • G08B21/06—Alarms for ensuring the safety of persons indicating a condition of sleep, e.g. anti-dozing alarms

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)
  • Time Recorders, Dirve Recorders, Access Control (AREA)
  • Traffic Control Systems (AREA)
JP2018508721A 2015-08-31 2016-08-30 運転行動を監視する方法及びシステム Active JP7011578B2 (ja)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201562212272P 2015-08-31 2015-08-31
US62/212,272 2015-08-31
PCT/US2016/049480 WO2017040519A1 (en) 2015-08-31 2016-08-30 Method and system for monitoring driving behaviors

Publications (3)

Publication Number Publication Date
JP2018528536A JP2018528536A (ja) 2018-09-27
JP2018528536A5 JP2018528536A5 (enExample) 2021-01-28
JP7011578B2 true JP7011578B2 (ja) 2022-01-26

Family

ID=58188346

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2018508721A Active JP7011578B2 (ja) 2015-08-31 2016-08-30 運転行動を監視する方法及びシステム

Country Status (4)

Country Link
US (1) US10769459B2 (enExample)
EP (1) EP3345127A4 (enExample)
JP (1) JP7011578B2 (enExample)
WO (1) WO2017040519A1 (enExample)

Families Citing this family (44)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6320661B2 (ja) * 2016-02-18 2018-05-09 三菱電機株式会社 搭乗者検出装置、搭乗者検出システムおよび搭乗者検出方法
KR101951565B1 (ko) * 2016-10-18 2019-02-22 서울대학교산학협력단 자세 및 작업 감지를 통한 안구 건조증 경고 시스템
US11279279B2 (en) 2016-12-22 2022-03-22 Sri International Driver monitoring and response system
JP6558388B2 (ja) * 2017-03-14 2019-08-14 オムロン株式会社 画像処理装置
US10380413B2 (en) * 2017-07-13 2019-08-13 Robert Bosch Gmbh System and method for pose-invariant face alignment
CN109803583A (zh) * 2017-08-10 2019-05-24 北京市商汤科技开发有限公司 驾驶状态监控方法、装置和电子设备
JP2019040465A (ja) * 2017-08-25 2019-03-14 トヨタ自動車株式会社 行動認識装置，学習装置，並びに方法およびプログラム
CN107704836B (zh) * 2017-10-17 2021-10-08 电子科技大学 基于物体检测的疲劳驾驶检测方法
CN109960986A (zh) 2017-12-25 2019-07-02 北京市商汤科技开发有限公司 人脸姿态分析方法、装置、设备、存储介质以及程序
US20190236360A1 (en) * 2018-01-30 2019-08-01 Mashgin Inc. Feedback loop for image-based recognition
CN110569795B (zh) * 2018-03-13 2022-10-14 腾讯科技（深圳）有限公司 一种图像识别方法、装置以及相关设备
CN108960065B (zh) * 2018-06-01 2020-11-17 浙江零跑科技有限公司 一种基于视觉的驾驶行为检测方法
JP6870660B2 (ja) * 2018-06-08 2021-05-12 トヨタ自動車株式会社 ドライバ監視装置
CN110633593A (zh) * 2018-06-21 2019-12-31 北京嘀嘀无限科技发展有限公司 一种恶性事件预测方法及系统
KR102503939B1 (ko) * 2018-09-28 2023-02-28 한국전자통신연구원 얼굴 이미지 비식별화 장치 및 방법
CN109389069B (zh) 2018-09-28 2021-01-05 北京市商汤科技开发有限公司 注视点判断方法和装置、电子设备和计算机存储介质
EP3891691A4 (en) * 2018-12-07 2022-11-09 Warner Bros. Entertainment Inc. TRIP CONFIGURABLE CONTENT
FR3090172B1 (fr) * 2018-12-13 2020-11-20 Continental Automotive France Procédé de détermination d’un niveau de vigilance d’un conducteur
CN111382642A (zh) * 2018-12-29 2020-07-07 北京市商汤科技开发有限公司 人脸属性识别方法及装置、电子设备和存储介质
FR3091602A1 (fr) * 2019-01-07 2020-07-10 Big Data Sante Procédé de création d’avatars pour protéger des données sensibles
JP7149202B2 (ja) * 2019-02-25 2022-10-06 株式会社日立ソリューションズ 行動分析装置および行動分析方法
US10850702B2 (en) * 2019-03-18 2020-12-01 Pony Ai Inc. Vehicle seat belt monitoring
CN110414421B (zh) * 2019-07-25 2023-04-07 电子科技大学 一种基于连续帧图像的行为识别方法
CN110852190B (zh) * 2019-10-23 2022-05-20 华中科技大学 一种融合目标检测与手势识别的驾驶行为识别方法及系统
US11687778B2 (en) 2020-01-06 2023-06-27 The Research Foundation For The State University Of New York Fakecatcher: detection of synthetic portrait videos using biological signals
WO2021156914A1 (ja) * 2020-02-03 2021-08-12 三菱電機株式会社 注意方向判定装置および注意方向判定方法
WO2021240671A1 (ja) * 2020-05-27 2021-12-02 三菱電機株式会社 ジェスチャ検出装置およびジェスチャ検出方法
WO2021240668A1 (ja) * 2020-05-27 2021-12-02 三菱電機株式会社 ジェスチャ検出装置およびジェスチャ検出方法
KR102475961B1 (ko) * 2020-06-09 2022-12-12 주식회사 에프에스솔루션 이미지 처리 방법, 및 이를 이용하는 장치 및 프로그램
CN111860259B (zh) * 2020-07-10 2024-07-09 广东正扬传感科技股份有限公司 驾驶检测模型的训练、使用方法、装置、设备及介质
US11776319B2 (en) * 2020-07-14 2023-10-03 Fotonation Limited Methods and systems to predict activity in a sequence of images
CN114648712B (zh) * 2020-12-18 2023-07-28 抖音视界有限公司 视频分类方法、装置、电子设备及计算机可读存储介质
US11468617B2 (en) * 2021-03-10 2022-10-11 International Business Machines Corporation Selective redaction of images
US20220368830A1 (en) * 2021-05-11 2022-11-17 Bendix Commercial Vehicle Systems Llc Presence and Location Based Driver Recording System
CN113657153B (zh) * 2021-07-08 2024-04-19 西安理工大学 改进EfficientNet用于安全带检测的方法
JP7479575B2 (ja) * 2021-07-09 2024-05-08 三菱電機株式会社 体格推定装置および体格推定方法
US12026953B2 (en) 2021-09-13 2024-07-02 Verizon Patent And Licensing Inc. Systems and methods for utilizing machine learning for vehicle detection of adverse conditions
CN114663863A (zh) * 2022-02-24 2022-06-24 北京百度网讯科技有限公司 图像处理方法、装置、电子设备和计算机存储介质
JP7735948B2 (ja) * 2022-07-08 2025-09-09 トヨタ自動車株式会社 画像記憶システムおよび画像記憶方法
GB2630744A (en) * 2023-06-05 2024-12-11 Univ Nanyang Tech Driver monitoring camera
US12559088B2 (en) * 2023-08-31 2026-02-24 Nissan North America, Inc. Driver assistance based on pose detection
KR20260008120A (ko) * 2023-12-26 2026-01-15 엘지전자 주식회사 서버
WO2025172420A1 (en) * 2024-02-16 2025-08-21 Sony Group Corporation Vehicle, network node and method
CN119540927B (zh) * 2024-11-13 2025-10-21 重庆市铜梁区公安局 一种酒后驾驶行为预警识别方法及系统

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002352228A (ja) 2001-05-25 2002-12-06 Toshiba Corp 画像処理システム及びこれを用いた運転支援システム
JP2009217633A (ja) 2008-03-11 2009-09-24 Toyota Motor Corp 車両用飲酒運転防止装置
JP2010204847A (ja) 2009-03-02 2010-09-16 Toyota Motor Corp 運転者状態検出装置及びこれを用いた衝突早期検出装置
US20150161505A1 (en) 2013-12-11 2015-06-11 Volvo Car Corporation Method of Programming a Neural Network Computer
JP2015162087A (ja) 2014-02-27 2015-09-07 株式会社デンソー 車載システム、情報処理装置、及びプログラム
JP2016503220A (ja) 2013-01-18 2016-02-01 マイクロソフト テクノロジー ライセンシング，エルエルシー ジェスチャ認識のためのパーツ及び状態検出

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2546415B2 (ja) * 1990-07-09 1996-10-23 トヨタ自動車株式会社 車両運転者監視装置
JP4738852B2 (ja) 2005-03-15 2011-08-03 富士通テン株式会社 運転支援装置及び運転支援システム
DE102006037156A1 (de) * 2006-03-22 2007-09-27 Volkswagen Ag Interaktive Bedienvorrichtung und Verfahren zum Betreiben der interaktiven Bedienvorrichtung
US7940962B2 (en) * 2007-08-03 2011-05-10 Delphi Technologies, Inc. System and method of awareness detection
JP2011123529A (ja) * 2009-12-08 2011-06-23 Sony Corp 情報処理装置、情報処理方法、およびプログラム
KR101276871B1 (ko) * 2009-12-14 2013-06-18 안동대학교 산학협력단 차량 충돌 방지 장치 및 방법
US8636361B2 (en) * 2011-07-20 2014-01-28 National Taiwan University Learning-based visual attention prediction system and method thereof
CN103020594B (zh) 2012-12-03 2016-01-20 清华大学苏州汽车研究院（吴江） 利用在线学习消除驾驶人个体差异性的疲劳状态检测方法
US9589357B2 (en) * 2013-06-04 2017-03-07 Intel Corporation Avatar-based video encoding
US10210761B2 (en) * 2013-09-30 2019-02-19 Sackett Solutions & Innovations, LLC Driving assistance systems and methods
US9361510B2 (en) 2013-12-13 2016-06-07 Intel Corporation Efficient facial landmark tracking using online shape regression method
US20160191958A1 (en) * 2014-12-26 2016-06-30 Krush Technologies, Llc Systems and methods of providing contextual features for digital communication

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002352228A (ja) 2001-05-25 2002-12-06 Toshiba Corp 画像処理システム及びこれを用いた運転支援システム
JP2009217633A (ja) 2008-03-11 2009-09-24 Toyota Motor Corp 車両用飲酒運転防止装置
JP2010204847A (ja) 2009-03-02 2010-09-16 Toyota Motor Corp 運転者状態検出装置及びこれを用いた衝突早期検出装置
JP2016503220A (ja) 2013-01-18 2016-02-01 マイクロソフト テクノロジー ライセンシング，エルエルシー ジェスチャ認識のためのパーツ及び状態検出
US20150161505A1 (en) 2013-12-11 2015-06-11 Volvo Car Corporation Method of Programming a Neural Network Computer
JP2015162087A (ja) 2014-02-27 2015-09-07 株式会社デンソー 車載システム、情報処理装置、及びプログラム

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
松本 吉央，リアルタイム顔・視線計測システムの開発と知的インタフェースへの応用，情報処理学会論文誌，日本，社団法人情報処理学会，2006年10月15日，第４７巻 Ｎｏ．ＳＩＧ１５(CVIM 16)，10～21ページ

Also Published As

Publication number Publication date
US10769459B2 (en) 2020-09-08
JP2018528536A (ja) 2018-09-27
EP3345127A1 (en) 2018-07-11
EP3345127A4 (en) 2019-08-21
WO2017040519A1 (en) 2017-03-09
US20180239975A1 (en) 2018-08-23

Similar Documents

Publication Publication Date Title
JP7011578B2 (ja) 運転行動を監視する方法及びシステム
US20220058407A1 (en) Neural Network For Head Pose And Gaze Estimation Using Photorealistic Synthetic Data
Shariff et al. Event cameras in automotive sensing: A review
Wang et al. A survey on driver behavior analysis from in-vehicle cameras
US10387725B2 (en) System and methodologies for occupant monitoring utilizing digital neuromorphic (NM) data and fovea tracking
JP6695503B2 (ja) 車両の運転者の状態を監視するための方法及びシステム
CN111695402B (zh) 用于标注3d点云数据中人体姿态的工具和方法
US20190246036A1 (en) Gesture- and gaze-based visual data acquisition system
EP4042318A1 (en) System and method of generating a video dataset with varying fatigue levels by transfer learning
Ryan et al. Real-time multi-task facial analytics with event cameras
García et al. Driver monitoring based on low-cost 3-D sensors
Poon et al. YOLO-based deep learning design for in-cabin monitoring system with fisheye-lens camera
JP2023552105A (ja) 人間行動のオクルージョン対応予測
KR20250154377A (ko) 확장 현실 시스템들을 위한 오브젝트들의 포즈 예측
Kang et al. Head pose-aware regression for pupil localization from a-Pillar cameras
CN117416364A (zh) 驾驶相关的增强虚拟场
KR20230071593A (ko) 인공 지능을 이용하여 운전자의 주시 상태를 판단하는 차량 장치 및 그 제어 방법
WO2023234882A1 (en) System and method for lossless synthetic anonymization of the visual data
Mihai et al. Using dual camera smartphones as advanced driver assistance systems: Navieyes system architecture
US12371072B2 (en) Operator take-over prediction
US12437438B2 (en) Estimating body poses based on images and head-pose information
Torstensson et al. In-vehicle driver and passenger activity recognition
Jia An Analysis ofDriver Cognitive Distraction
CN116152790A (zh) 一种安全带检测方法及装置
Peláez Coronado et al. Driver monitoring based on low-cost 3-D sensors

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20190823

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20190823

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20200908

A524 Written submission of copy of amendment under article 19 pct

Free format text: JAPANESE INTERMEDIATE CODE: A524

Effective date: 20201204

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20210525

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20211221

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20220114

R150 Certificate of patent or registration of utility model

Ref document number: 7011578

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250