WO2021112048A1 - 運転分析装置、及び運転分析方法 - Google Patents
運転分析装置、及び運転分析方法 Download PDFInfo
- Publication number
- WO2021112048A1 WO2021112048A1 PCT/JP2020/044557 JP2020044557W WO2021112048A1 WO 2021112048 A1 WO2021112048 A1 WO 2021112048A1 JP 2020044557 W JP2020044557 W JP 2020044557W WO 2021112048 A1 WO2021112048 A1 WO 2021112048A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- driver
- unit
- analysis
- function
- processing unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
- A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
- A61B3/113—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for determining or recording eye movement
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
- B60K35/10—Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
- B60K35/20—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor
- B60K35/21—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor using visual output, e.g. blinking lights or matrix displays
- B60K35/23—Head-up displays [HUD]
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
- B60K35/20—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor
- B60K35/26—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor using acoustic output
- B60K35/265—Voice
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
- B60K35/20—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor
- B60K35/28—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
- B60K35/20—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor
- B60K35/29—Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
- B60K35/65—Instruments specially adapted for specific vehicle types or users, e.g. for left- or right-hand drive
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
- B60K35/65—Instruments specially adapted for specific vehicle types or users, e.g. for left- or right-hand drive
- B60K35/654—Instruments specially adapted for specific vehicle types or users, e.g. for left- or right-hand drive the user being the driver
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
- B60W40/09—Driving style or behaviour
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—Two-dimensional [2D] image generation
- G06T11/20—Drawing from basic elements
- G06T11/26—Drawing of charts or graphs
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/77—Determining position or orientation of objects or cameras using statistical methods
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/62—Extraction of image or video features relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/766—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using regression, e.g. by projecting features on hyperplanes
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
- G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/16—Anti-collision systems
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
- B60K2360/16—Type of output information
- B60K2360/178—Warnings
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
- B60K2360/18—Information management
- B60K2360/186—Displaying information according to relevancy
- B60K2360/1868—Displaying information according to relevancy according to driving situations
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
- B60K2360/20—Optical features of instruments
- B60K2360/21—Optical features of instruments using cameras
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
- B60K2360/741—Instruments adapted for user detection
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20072—Graph-based image processing
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20076—Probabilistic image processing
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30268—Vehicle interior
Definitions
- This disclosure relates to a technique for analyzing the state of a driver from an image of the driver.
- Patent Document 1 describes a technique for identifying and evaluating the characteristics of driver's health deterioration by detecting the direction of the driver's eyes and head.
- the characteristics of poor health include perceptual or visual distraction of the driver and increased workload of the driver.
- the percentage of the center of the road which is the ratio of looking at the front of the road from the total of the time when the driver looks at the front of the road, which is the direction in which the driver's eyes are typically facing, and the time when he looks at the area other than the front of the road ( Hereinafter, PRC) is calculated. Then, distraction and deterioration of health are evaluated using PRC.
- One aspect of the present disclosure is to provide a technique for analyzing various driver states including intentional driver behavior from a driver image.
- One aspect of the present disclosure is an operation analyzer, which includes an information generation unit, an acquisition unit, an extraction unit, and an analysis unit.
- the information generation unit uses the line-of-sight direction indicating which of the plurality of preset visual regions the driver's line of sight is directed to, and the open / closed state of the driver's eyes as detection elements, and the time series of the detection elements.
- the acquisition unit acquires evaluation data from the time series of the detection elements using a time window having a preset time width.
- the extraction unit extracts a plurality of feature quantities including at least the result of totaling the appearance frequency for each detection element from the evaluation data.
- the analysis unit analyzes at least one of the driver's tendency and the driver's state by using a function that inputs at least a part of the plurality of feature quantities extracted by the extraction unit.
- the analysis is performed without specially looking at the front direction, so that it is possible to accurately determine the state in which the driver intentionally looks at other than the front direction, and the driver is distracted. It is possible to suppress misidentification as such.
- the accuracy of the analysis on the driver state such as deterioration of health and distraction can be improved.
- the driving analysis device 1 shown in FIG. 1 is mounted on a vehicle and analyzes the driver's state and the like by analyzing the driver's face direction and line-of-sight direction.
- the driver's condition to be analyzed may usually include drowsiness and drowsiness.
- the operation analyzer 1 includes a camera 10 and a processing unit 20.
- the operation analyzer 1 may include an HMI unit 30.
- the processing unit 20, the camera 10 and the HMI unit 30 may be directly connected or may be connected via an in-vehicle network such as a CAN.
- CAN is a registered trademark and is an abbreviation for Controller Area Network.
- the camera 10 for example, a known CCD image sensor, CMOS image sensor, or the like can be used.
- the camera 10 is arranged so that, for example, the face of the driver seated in the driver's seat of the vehicle is included in the imaging range.
- the camera 10 periodically executes imaging and outputs the captured image data to the processing unit 20.
- the processing unit 20 includes a microcomputer having a CPU 20a and, for example, a semiconductor memory such as RAM or ROM (hereinafter, memory 20b).
- the processing unit 20 includes a driving analysis unit 21 and a learning unit 22 as blocks representing functions realized by the CPU 20a executing a program. The processing contents of the driving analysis unit 21 and the learning unit 22 will be described later.
- the HMI unit 30 includes an audio output unit 31, a meter display unit 32, and a HUD display unit 33.
- HMI is an abbreviation for human-machine interface.
- the voice output unit 31 is a device that generates a warning sound or a voice through a speaker mounted on the vehicle.
- the meter display unit 32 is a device for displaying measurement results such as vehicle speed, engine speed, and remaining fuel amount measured in the vehicle.
- the HUD display unit 33 is a device that projects various information by projecting an image onto a windshield or a combiner. HUD is an abbreviation for head-up display. At least one display screen of the meter display unit 32 and the HUD display unit 33 is provided with an area for presenting the analysis result by the operation analysis unit 21.
- the processing unit 20 acquires an image for one frame from the camera 10.
- the face detection process is a process of detecting a face region, which is a region in which a face is captured, from the image acquired in S10.
- pattern matching can be used, but the face detection process is not limited to pattern matching.
- the face detection process for example, the portion indicated by the frame W in FIG. 3 is detected as a face region.
- the feature point detection process is a process of detecting a plurality of facial feature points necessary for specifying the imaged face orientation and eye condition using the image of the face region extracted in S20.
- the facial feature points characteristic parts in the contours of the eyes, nose, mouth, ears, face and the like are used.
- a plurality of face feature points indicated by circles in FIG. 3 are detected.
- the processing unit 20 executes the gaze area / face orientation detection process.
- the gaze area / face orientation detection process is based on the plurality of face feature points detected in S30, and uses the peripheral image of the eyes detected from the image of the face area, the direction in which the driver gazes, and the open / closed state of the driver's eyes. , And the process of detecting the face orientation of the driver and storing the detection result in the memory 20b.
- the memory 20b stores at least the detection results for the number of frames corresponding to the maximum width of the time window described later.
- the gaze direction of the driver is expressed by dividing the range visible to the driver during driving into a plurality of regions (hereinafter referred to as the visual region) and which visual region the driver is gazing at.
- the visual region As shown in FIG. 4, in the viewing area, the left side mirror (hereinafter, left mirror) E1, the front E2, the room mirror (hereinafter, rear mirror) E3, the meter E4, and the right side mirror (hereinafter, right mirror) E5. , Console E6, and hand E7 may be included.
- the method of dividing the visual region is not limited to the above E1 to E7, and a finer divided visual region may be used, or a visual region divided according to an angle or the like viewed from the driver may be used. ..
- the open / closed state of the driver indicates whether or not any visual region is being viewed by closing the eyes. This state is referred to as eye closure E8.
- the state information of the visual areas E1 to E7 is represented by a binary value of 1 when it is determined that the driver is gazing at the corresponding visual area and 0 when it is determined that the driver is not gazing.
- the state information of the closed eye E8 is also represented by a binary value of 1 when the driver is closed and 0 when the driver is not closed.
- the state information of the visual areas E1 to E7 and the eye closed E8 one of them is selectively set to 1 and the other is set to 0 at any time.
- the driver's face orientation is an angle indicating how much the face orientation is rotated or tilted in each of the yaw direction ⁇ 1, the pitch direction ⁇ 2, and the roll direction ⁇ 3.
- the yaw direction ⁇ 1 represents an angle in the direction of rotation about the neck.
- the pitch direction ⁇ 2 represents the inclination angle of the face in the front-back direction.
- the roll direction ⁇ 3 represents the lateral inclination angle of the face.
- the driver's face position may be detected.
- eye states E1 to E8 are collectively referred to as eye states E1 to E8, and the yaw direction ⁇ 1, the pitch direction ⁇ 2, and the roll direction ⁇ 3 representing the driver's face orientation are collectively referred to as face states ⁇ 1 to ⁇ 3.
- the method proposed by the applicant of the present application in Japanese Patent Application No. 2019-108028 can be used to detect feature points and gaze directions using a regression function.
- the processing unit 20 extracts evaluation data from the time-series data of the eye states E1 to E8 and the facial states ⁇ 1 to ⁇ 3 stored in the memory 20b using a time window.
- the time window is set to a predetermined time width in the past with reference to the current time.
- the time width may be set to a different length depending on the analysis target. For example, when it is used for detecting forward carelessness, it is set to a relatively short time width T1 (for example, about 3 seconds). When used for checking the surrounding environment, set a longer time width T2 (for example, about 15 seconds). When used for detecting drowsiness, a longer time width T3 (for example, about 30 seconds) is set.
- the raw data of the gaze area shown in FIG. 6 is generated by arranging the eye states having a value of 1 among the eye states E1 to E8 along the time axis.
- the time widths T1 to T3 of the time window may be switched according to the application that uses the analysis result in the operation analysis process, or a plurality of types of time widths T1 to T3 may be used at the same time.
- the raw face orientation data is generated by arranging the angles of the face states ⁇ 1 to ⁇ 3 represented with reference to the traveling direction of the vehicle along the time axis.
- the processing unit 20 extracts the feature amount using the extracted evaluation data.
- the feature amount may include the result of totaling the frequency at which the values of the eye states E1 to E8 become 1 (that is, the number of image frames) for each eye state E1 to E8.
- the feature amount may include the duration of the state in one eye state, the transition interval, and the like. Further, the frequency and transition interval of the transition from the first eye state to the second eye state (for example, front ⁇ right mirror, etc.) may be included.
- the feature amounts include the eye closure time per unit time (hereinafter, PERCLOS), the number of eye closures per unit time (hereinafter, eye closure frequency), and the eye closure cycle, which are calculated by focusing on the eye closure E8 in the raw data of the gaze area. , And the average closing time per time may be included.
- the feature amount is based on the facial states ⁇ 1 to ⁇ 3, with the predetermined angle range based on the front direction as the range and the outside of the predetermined angle range as the outside range, and the feature amount is within the range and outside the range, respectively.
- the result of totaling the applicable frequencies may be included.
- the predetermined angle range may be set to the same angle on both sides with respect to the front direction, or may be set to different angles on both sides with respect to the front direction as shown in FIG.
- the feature amount may include an average value of face-facing angles within a range and a dispersion value.
- the feature amount may include a preset frequency for each typical face position, a moving speed when the face position changes, and the like.
- FIG. 9 is a graph showing how the feature amount obtained by aggregating the eye states E1 to E8 changes with the passage of time.
- the processing unit 20 executes an analysis process for analyzing the state of the driver and the like using the feature amount extracted in S60.
- the processing unit 20 presents the analysis result in S70 to the driver via the HMI unit 30 and ends the processing.
- a method of presenting the analysis result an image may be used or a voice may be used.
- S10 to S40 correspond to the information generation unit
- S50 corresponds to the acquisition unit
- S60 corresponds to the extraction unit
- S70 corresponds to the analysis unit
- S80 corresponds to the display unit.
- the feature amount to be extracted for example, the aggregated value when the gaze direction is the front mirror E2, the left mirror E1, the right mirror E5, and the rearview mirror E3 may be used. Further, as the feature amount to be extracted, the aggregated value when the face orientation is outside the predetermined range of the yaw direction ⁇ 1 (hereinafter, large yaw) may be used. Further, as the feature amount to be extracted, PERCLOS, which is a feature amount based on the eye closure E8, the eye closure time, the average eye closure time, and the like may be used.
- the average value of the data for all the drivers aggregated in advance may be used, or the data for some drivers extracted under a certain condition may be used.
- the average value of may be used.
- the statistical data corresponds to the reference value. Gender, age, driving experience, etc. may be used as the conditions used for extracting the driver. Further, regardless of the driver, the area, time zone, weather, etc. may be used as the conditions used for data extraction.
- the analysis result as shown in FIG. 12 can be obtained.
- the driver's visual behavior is characterized by weak confirmation of the surroundings of the vehicle, because the front gaze is prominent and the mirror gaze is small.
- the radar chart on the right side in FIG. 12 it can be seen that the driver is looking aside and is careless in the front as a characteristic of the driver's visual behavior because the gaze in the front tends to be very small.
- Example 2 Analysis using a function
- the analysis process in S70 for example, as shown in FIG. 13, at least a part of the feature amount generated in S60 is set as the input information group IE, and the driver which is the analysis result using the function prepared in advance is used. You may output the tendency and the state of the driver.
- a certain driver state is output as an analysis result by using a regression function using gradient boosting will be described. That is, when analyzing a plurality of driver states, a different regression function is used for each driver state to be analyzed.
- the input information group IE that is the input of the function may include the eight feature quantities used for the radar chart described above.
- the driver state that is the output of the function may include "normal”, “sleepiness”, and “dumb driving”. Further, when the driver tendency is output, as shown in FIG. 12, "more / less gaze forward”, “more / less gaze at the mirror”, and the like may be included.
- the input information group IE is continuously extracted frame by frame using a so-called sliding window in which the time window is applied while shifting the time by one frame of the image.
- the analysis process is executed for each of the input information groups IE extracted for each frame.
- the function outputs the score vector A as the analysis result.
- the score vector A has an element that becomes 1 when it corresponds to the driver state to be analyzed, and an element that becomes 1 when it does not correspond. That is, the score vector A is a vector having two elements a 1 and a 2, and is represented by the equation (1).
- the score vector A takes a one-hot format in which the value of one of the elements is large and the value of the other element is small, and the recognition result (that is, the analysis target) is finally based on the element having the larger value. Corresponds / does not correspond to the driver state) is obtained.
- the initial value A (0) of the score vector is given, and the score vector A is corrected by the correction amount R to obtain the score vector A representing the analysis result of the driver state. calculate.
- correction amount R is obtained by causing the input information group I E cut out by the time window to act on the correction function F I , as shown in the equation (3).
- Correction function F I is a function of applying the additive model of the regression function using the gradient boosting.
- Such regression functions include, for example, "One Millisecond Face Alignment with an Ensemble of Regression Trees” Vahid Kazemi and Josephine Sullivan, The IEEE Conference on CVPR, 2014,1867-1874 (hereinafter, Reference 1), and "Greedy”. Function Approximation: A gradient boosting machine "Jerome H. Friedman, The Annals of Statistics Volume 29, Number 5 (2001), 1189-1232 (hereinafter, Reference 2), etc.
- Correction function F I is a function recursively value is determined using the I-number of regression tree RT 1 ⁇ RT I prepared in advance, as defined in (4) equation.
- I is an integer greater than or equal to 1.
- Regression trees RT 1 to RT I are a set of weak hypotheses that have a tree structure.
- F 0 is the initial value of the correction function F I
- ⁇ is a learning rate and is set to 0 ⁇ ⁇ 1. By reducing the value of ⁇ , overfitting is suppressed.
- the I regression trees RT 1 to RT I all have the same structure.
- the regression tree RT i a binary tree that sequentially branches the nodes into two is used.
- the node that becomes the branch point of the branch of the regression tree RT i is called the normal node 41
- the node that becomes the leaf of the regression tree RT i is called the terminal node 42.
- the comparison data X k and the threshold value TH k are associated with the kth normal node 41.
- the comparison data X k and the threshold value TH k are usually used for branch determination to determine which lower node to branch from the node 41.
- Specific values r i1 to r i 8 of the regression amount G i calculated by the regression tree RT i are associated with each of the terminal nodes 42.
- the number of normal nodes 41 excluding the terminal node 42 is K.
- K is an integer greater than or equal to 1.
- any one of M feature quantities X 1 to X M belonging to the input information group IE may be used.
- M is an integer greater than or equal to 1.
- the result of performing some calculation on a plurality of feature quantities selected from M feature quantities X 1 to X M may be used.
- the calculation result for the feature amount is the comparison data X k , it is necessary to select the feature amount having the same unit as shown in FIG.
- the feature quantities that use the aggregated values of the eye states E1 to E8 as they are are count values, it is possible to use the calculation result between the aggregated values of the eye states E1 to E8 as the comparison data X k. is there.
- the feature amounts representing PERCLOS, the eye closing frequency, and the average eye closing time represent time and ratio, and it is impossible to use the calculation result with the feature amount using the aggregated value as it is as the comparison data X k. is there.
- the normal node 41 or the terminal of the next layer depends on whether the comparison data X k is smaller than the threshold TH k. Which branch to reach node 42 is determined. That is, according to the comparison result at each normal node 41, the process of sequentially selecting the branches of the regression tree RT i is repeated to reach any of the plurality of terminal nodes 42. Any of the regression quantities r i1 , r i2 , r i3 , ... Associated with the reached end node 42 becomes the value of the regression function G i , and eventually becomes a part of the value of the correction function F I.
- the initial value A (0) of the score vector A, the initial value F 0 of the correction function F I , and the parameters defining the regression tree RT i (that is, the comparison data X 1 to X K , Thresholds TH 1 to TH K , regression quantities r i1 , r i2 , r i3 , ...) need to be prepared in advance.
- a (0) , F 0 , X 1 to X K , TH 1 to TH K , r i1 , r i2 , r i3 , ... are collectively referred to as analytical parameters.
- the analysis parameters are generated when the learning unit 22 of the processing unit 20 executes the learning process.
- the learning process does not necessarily have to be executed by the processing unit 20, and may be executed by a device other than the processing unit 20.
- the memory 20b stores time-series data (hereinafter, learning time-series data) of the eye states E1 to E8 and the face states ⁇ 1 to ⁇ 3 prepared for learning.
- the learning time series data is data obtained by driving an unspecified number of drivers in various situations.
- the correct answer label of the driver state is given in advance to the learning time series data on a frame-by-frame basis. As shown in FIG. 17, for example, the correct answer label collects the subjective evaluation and the objective evaluation of the driver's state in parallel with the collection of the learning time series data, and estimates the driver's state from the subjective evaluation and the objective evaluation.
- the subjective evaluation is, for example, “I felt drowsy”, “I was drowsy”, etc., and the objective evaluation was "I was yawning”, “I was tilted my neck”, “I was looking only at the front” and so on.
- the processing unit 20 uses each of the N input information groups IE cut out using the time window from the learning time-series data stored in the memory 20b as learning data, and scores for each learning data.
- the correct answer values of the vector A (hereinafter referred to as teacher data) A 1 to A N are generated.
- N is an integer greater than or equal to 1.
- the teacher data A 1 to A N are generated by referring to the correct answer label assigned to each frame of the image included in the training data.
- the processing unit 20 calculates the initial value A (0) of the score vector.
- the initial value A of the score vector (0) may be, for example, using the average value of the teacher data A 1 ⁇ A N generated by S110.
- the processing unit 20 calculates a modified residual ⁇ A n , which is the difference between the teacher data An and the initial value A (0) of the score vector, for each of the N training data according to the equation (5). To do.
- an initial value F 0 of the correction function F I used for calculating the correction amount R is calculated using equation (6).
- equation (6) when the distance between the modified residual ⁇ A n in each training data and an arbitrary vector V having the same dimension as the score vector A is summed for all the training data, the total value is minimized. It means that the vector V to be used is set to the initial value F 0 of the correction function F I.
- the processing unit 20 initializes the regression tree index i used for identifying the regression trees RT 1 to RT I to 1.
- the processing unit 20 calculates the remaining correction amounts c 1 to c N for each training data using the equation (7).
- the processing unit 20 uses the training data, comparison data X 1 ⁇ X K and the threshold value TH 1 regression by sequentially setting the ⁇ TH K tree RT i used in each ordinary node 41 of regression tree RT i To generate.
- the method described in Section 2.3.2 of Reference 1 described above may be used.
- the comparison data X 1 to X K may be randomly selected from all the selectable comparison data candidates, or the optimum one may be searched for by brute force.
- selecting the comparison data X 1 to X K by such learning selects the comparison data associated with the feature having a high contribution rate in the analysis of the driver state from all the selectable comparison data candidates. , Means to select automatically.
- the processing unit 20 the comparison data X 1 ⁇ X K selected in S170, the index for classifying training data, obtained a value close to the rest correction amount c 1 ⁇ c N in all training data
- the regression tree RT i is generated by setting the threshold values TH 1 to TH K so as to be obtained. That is, to produce a regression function G i implemented by regression tree RT i.
- the processing unit 20 uses the regression function G i generated in S180, the equation (8), and updates the correction function F i.
- the processing unit 20 increments the regression tree index i by 1.
- the processing unit 20 determines whether or not i> I. If a negative judgment is made in S210, the process is returned to S160 in order to create a new regression tree RT i, and if a positive judgment is made in S210, the process is terminated.
- I pieces of the regression function G 1 ⁇ G I for one correction function F I i.e., I-number of regression tree RT 1 ⁇ RT I
- the analysis process executed by the processing unit 20 in S70 will be described with reference to the flowchart of FIG.
- the analysis process may be performed every time an image is acquired in S10, that is, every frame, or every plurality of frames.
- the processing unit 20 initializes the regression tree index i to 1.
- the processing unit 20 sets the regression tree RT i identified by the regression tree index i, the input information group IE acquired in S60, and the determination parameter of the regression tree RTi calculated in the learning process.
- the value of the regression function G i is obtained using Eq. (8), and the correction function F i is updated using Eq. (8).
- the processing unit 20 increments the regression tree index i by 1.
- the processing unit 20 determines whether or not i> I. That is, whether or not used to calculate all the correction function F I of the I-number of regression tree RT 1 ⁇ RT I.
- the processing unit 20 returns the process to S320 when a negative determination is made in S340, and proceeds to the process to S350 when an affirmative determination is made in S340.
- the process of S320 ⁇ S340, finally (4) the correction function F I indicated in formula is calculated, (3) As shown in equation, the value obtained by this correction function F I is the initial score vector It is the correction amount R for the value A (0).
- the processing unit 20 generates the corrected score vector A using the equation (2).
- the processing unit 20 extracts an element having a larger value (hereinafter, maximum score SC) from the modified score vector A.
- the processing unit 20 determines whether or not the extracted maximum score SC is larger than the preset threshold value THs.
- THs is a threshold value for determining the reliability of the analysis result. If SC> THs, the processing unit 20 advances the processing to S380, and if SC ⁇ THs, the processing proceeds to S390.
- the processing unit 20 outputs the analysis result associated with the element having the maximum score SC, that is, whether or not it corresponds to the driver state of the analysis target, and ends the processing.
- the processing unit 20 invalidates the analysis result and ends the process, assuming that the analysis result shown in the score vector A is unreliable.
- the feature quantities extracted from the eye states E1 to E8 and the facial states ⁇ 1 to ⁇ 3 are used to analyze whether or not the driver state corresponds to the driver state to be analyzed, and the analysis result is presented to the driver. ..
- the analysis is performed without specially looking at the front direction, so it is possible to accurately determine the state in which the driver intentionally looks at something other than the front direction, and misidentifies it as distraction. Can be suppressed.
- a radar chart is used as a method of presenting the analysis result, and statistical data such as the average value of an unspecified number of drivers is presented together with the feature amount extracted for the driver itself. Therefore, the driver can easily compare the driver itself with others. As a result, the driver can easily understand the tendency and characteristics of the driver himself / herself during driving.
- At least a part of the feature amount is set as the input information group IE , and it is estimated whether or not it corresponds to the driver state to be analyzed by using the regression function generated by the gradient boosting. Output as analysis result. Therefore, it is possible to generate a regression function and perform highly accurate analysis on the relationship between the driver state and the feature quantity without any specialized knowledge.
- gradient boosting is used to generate a regression function (that is, a regression tree) used for analysis.
- the second embodiment is different from the first embodiment in that a random forest is used for generating the regression function.
- the state information for learning is stored in the memory 20b as in the case of the first embodiment.
- the correct answer label of the driver state is given in advance to the state information for learning on a frame-by-frame basis.
- the correct label of the driver state the state in which the driver does not feel drowsy (hereinafter, normal) is 0, and the state in which the driver feels drowsiness (hereinafter, drowsiness) is 1.
- the processing unit 20 similarly to the processing in S310, the N input information group I E which is cut out using a time window as learning data, for each learning data, teacher data A 1 is correct value ⁇ AN is generated.
- the teacher data takes a value of 0 or 1.
- the processing unit 20 initializes the regression tree index i to 1.
- the processing unit 20 extracts M feature quantities X 1 to X M from each of the training data.
- the M features are extracted from a larger number of feature candidates than M in different combination patterns for each regression tree index i.
- the processing unit 20 a feature quantity X 1 ⁇ X M extracted in S530, using as index for classifying training data at each node included in the regression tree RT i, to generate a regression tree RTi.
- each of the learning data assigned to the ordinary node 41 of interest calculates an error Q m for each lower node using sorted learning data, and generates the error Q m regression tree RT i by the selected threshold a m so as to minimize. That is, to produce a regression function G i implemented by regression tree RT i.
- the error Q m is calculated as follows. That is, of the two lower nodes, the first node is assigned an allocation value of 0 representing "normal", the second node is assigned an allocation value of 1 representing "sleepiness", and the learning assigned to each node. Let the squared error between the teacher data of the data and the assigned value of the node be the error Q m .
- the processing unit 20 increments the regression tree index i by 1.
- the processing unit 20 determines whether or not the regression tree index i is larger than the total number I of the regression trees RTi used to generate the regression function. If i> I, the processing unit 20 ends the processing, if i ⁇ I, the processing returns to S530, and in order to generate a new regression tree, the processing unit 20 starts the processing again from the selection of M feature quantities. ..
- I regression trees RT 1 to RT I are generated. Specifically, the comparison data X ik and the threshold value a ik set for each of the normal nodes 41 constituting the regression tree RT i and the output value set for each of the terminal nodes 42 are generated as analysis parameters. Will be done. Note that k is an index that identifies the individual nodes that make up the regression tree RT i.
- FIG. 20 relates to an analysis process executed by the processing unit 20 of the second embodiment in place of the analysis process of the first embodiment described with reference to FIG. 18 in order to realize the function as the operation analysis unit 21. This will be described using the flowchart of. In the analysis process, the analysis parameters generated by the above-mentioned learning process are used.
- the processing unit 20 initializes the regression tree index i to 1, and initializes the total score SC to 0.
- the processing unit 20 extracts the feature quantities X 1 to X M used as comparison data in the regression tree RT i from the input information group IE.
- the processing unit 20 acquires the output value r by sequentially tracing the regression tree RT i using the feature quantities X 1 to X M extracted in S720.
- the processing unit 20 updates the total score SC by adding the output value r to the total score SC.
- the processing unit 20 increments the regression tree index i by 1.
- the processing unit 20 determines whether or not the regression tree index i is larger than the total number I of the regression trees RT i used to generate the regression function, and if i> I, the processing shifts to S770. If i ⁇ I, the process returns to S720.
- the processing unit 20 determines whether or not the total score SC is larger than the preset threshold value THs, shifts the processing to S780 if SC> THs, and performs the processing S790 if SC ⁇ THs. Move to.
- the processing unit 20 outputs as an analysis result that it corresponds to the driver state of the analysis target, that is, "sleepiness", and ends the processing.
- the processing unit 20 outputs as an analysis result that the driver state to be analyzed does not correspond to the driver state and the driver state corresponds to "normal", and ends the process.
- the driver state which is the analysis result is presented to the driver, but the content presented to the driver is not limited to the driver state.
- warnings and advice appropriate for the analyzed driver status may be presented to the driver.
- the presentation method is not limited to the image, and voice may be used.
- the regression tree is used to generate the function used for the analysis process, but instead of the regression tree, for example, a support vector machine, a neural network as shown in FIG. 22, or the like is used. May be good.
- the analysis result by the analysis process is presented to the driver via the HMI unit 30, but the method of using the analysis result is not limited to the presentation to the driver.
- the analysis result may be used, for example, for vehicle control for driving support.
- the feature quantities extracted from the eye states E1 to E8 and the face states ⁇ 1 to ⁇ 3 extracted from the driver's face image are used in the analysis process, but the information used in the analysis process is described above. It is not limited to the feature amount.
- the feature amount extracted from the information representing the driving operation of the driver, the information representing the behavior of the vehicle, the information representing the surrounding condition of the vehicle, and the like may be used for the analysis process.
- the processing unit 20 and its method described in the present disclosure are dedicated provided by configuring a processor and memory programmed to perform one or more functions embodied by a computer program. It may be realized by a computer. Alternatively, the processing unit 20 and its method described in the present disclosure may be realized by a dedicated computer provided by configuring the processor with one or more dedicated hardware logic circuits. Alternatively, the processing unit 20 and its method described in the present disclosure are a combination of a processor and memory programmed to perform one or more functions and a processor composed of one or more hardware logic circuits. It may be realized by one or more dedicated computers configured by. The computer program may also be stored on a computer-readable non-transitional tangible recording medium as an instruction executed by the computer. The method for realizing the functions of each unit included in the processing unit 20 does not necessarily include software, and all the functions may be realized by using one or a plurality of hardware.
- a plurality of functions possessed by one component in the above embodiment may be realized by a plurality of components, or one function possessed by one component may be realized by a plurality of components. .. Further, a plurality of functions possessed by the plurality of components may be realized by one component, or one function realized by the plurality of components may be realized by one component. Further, a part of the configuration of the above embodiment may be omitted. In addition, at least a part of the configuration of the above embodiment may be added or replaced with the configuration of the other above embodiment.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mechanical Engineering (AREA)
- Transportation (AREA)
- Chemical & Material Sciences (AREA)
- Combustion & Propulsion (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Medical Informatics (AREA)
- Ophthalmology & Optometry (AREA)
- Evolutionary Computation (AREA)
- Databases & Information Systems (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Veterinary Medicine (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- Public Health (AREA)
- Biomedical Technology (AREA)
- Automation & Control Theory (AREA)
- Mathematical Physics (AREA)
- Heart & Thoracic Surgery (AREA)
- Probability & Statistics with Applications (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Traffic Control Systems (AREA)
- Image Analysis (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US17/826,403 US12272159B2 (en) | 2019-12-02 | 2022-05-27 | Driving analysis device and driving analysis method for analyzing driver tendency |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2019-218107 | 2019-12-02 | ||
| JP2019218107A JP7172968B2 (ja) | 2019-12-02 | 2019-12-02 | 運転分析装置、及び運転分析方法 |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US17/826,403 Continuation US12272159B2 (en) | 2019-12-02 | 2022-05-27 | Driving analysis device and driving analysis method for analyzing driver tendency |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2021112048A1 true WO2021112048A1 (ja) | 2021-06-10 |
Family
ID=76220640
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/JP2020/044557 Ceased WO2021112048A1 (ja) | 2019-12-02 | 2020-11-30 | 運転分析装置、及び運転分析方法 |
Country Status (3)
| Country | Link |
|---|---|
| US (1) | US12272159B2 (cg-RX-API-DMAC7.html) |
| JP (1) | JP7172968B2 (cg-RX-API-DMAC7.html) |
| WO (1) | WO2021112048A1 (cg-RX-API-DMAC7.html) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114066297A (zh) * | 2021-11-24 | 2022-02-18 | 西南交通大学 | 一种高速铁路行车调度员工作状态识别方法 |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2021144419A (ja) * | 2020-03-11 | 2021-09-24 | いすゞ自動車株式会社 | 安全運転判定装置 |
| JP7803742B2 (ja) * | 2022-02-28 | 2026-01-21 | パイオニア株式会社 | 情報処理装置、情報処理方法、プログラム及び記録媒体 |
| WO2024146692A1 (en) | 2023-01-05 | 2024-07-11 | Xylon d.o.o. | Visual distraction detection method in driving monitoring system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH10272960A (ja) * | 1997-03-31 | 1998-10-13 | Mitsubishi Motors Corp | 覚醒度推定装置 |
| JP2009261516A (ja) * | 2008-04-23 | 2009-11-12 | Toyota Motor Corp | 覚醒度推定装置 |
| JP2017211882A (ja) * | 2016-05-26 | 2017-11-30 | トヨタ自動車株式会社 | 車両の自動運転制御システム |
| JP2018075255A (ja) * | 2016-11-10 | 2018-05-17 | 株式会社デンソー | 状態推定装置、及び状態推定方法 |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AU2003211065A1 (en) | 2002-02-19 | 2003-09-09 | Volvo Technology Corporation | System and method for monitoring and managing driver attention loads |
| JP4551766B2 (ja) | 2002-10-15 | 2010-09-29 | ボルボ テクノロジー コーポレイション | 被験者の頭及び目の動きを分析する方法及び装置 |
| JP2004192345A (ja) * | 2002-12-11 | 2004-07-08 | Nissan Motor Co Ltd | 運転者状態検出装置 |
| CN108537161A (zh) * | 2018-03-30 | 2018-09-14 | 南京理工大学 | 一种基于视觉特性的驾驶分心检测方法 |
| JP7120938B2 (ja) | 2019-02-04 | 2022-08-17 | 株式会社Soken | 電池システムおよび二次電池の制御方法 |
| JP2021089479A (ja) | 2019-12-02 | 2021-06-10 | 株式会社デンソー | 意識判定装置、及び意識判定方法 |
-
2019
- 2019-12-02 JP JP2019218107A patent/JP7172968B2/ja active Active
-
2020
- 2020-11-30 WO PCT/JP2020/044557 patent/WO2021112048A1/ja not_active Ceased
-
2022
- 2022-05-27 US US17/826,403 patent/US12272159B2/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH10272960A (ja) * | 1997-03-31 | 1998-10-13 | Mitsubishi Motors Corp | 覚醒度推定装置 |
| JP2009261516A (ja) * | 2008-04-23 | 2009-11-12 | Toyota Motor Corp | 覚醒度推定装置 |
| JP2017211882A (ja) * | 2016-05-26 | 2017-11-30 | トヨタ自動車株式会社 | 車両の自動運転制御システム |
| JP2018075255A (ja) * | 2016-11-10 | 2018-05-17 | 株式会社デンソー | 状態推定装置、及び状態推定方法 |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114066297A (zh) * | 2021-11-24 | 2022-02-18 | 西南交通大学 | 一种高速铁路行车调度员工作状态识别方法 |
| CN114066297B (zh) * | 2021-11-24 | 2023-04-18 | 西南交通大学 | 一种高速铁路行车调度员工作状态识别方法 |
Also Published As
| Publication number | Publication date |
|---|---|
| JP2021089480A (ja) | 2021-06-10 |
| JP7172968B2 (ja) | 2022-11-16 |
| US12272159B2 (en) | 2025-04-08 |
| US20220284718A1 (en) | 2022-09-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| TWI741512B (zh) | 駕駛員注意力監測方法和裝置及電子設備 | |
| Li et al. | Modeling of driver behavior in real world scenarios using multiple noninvasive sensors | |
| JP7172968B2 (ja) | 運転分析装置、及び運転分析方法 | |
| CN109740466A (zh) | 广告投放策略的获取方法、计算机可读存储介质 | |
| CN102934458B (zh) | 兴趣度估计装置以及兴趣度估计方法 | |
| US10185869B2 (en) | Filter and shutter based on image emotion content | |
| Król et al. | A novel machine learning analysis of eye-tracking data reveals suboptimal visual information extraction from facial stimuli in individuals with autism | |
| JP5771127B2 (ja) | 注目度推定装置およびそのプログラム | |
| Dua et al. | AutoRate: How attentive is the driver? | |
| EP3113073A1 (en) | Determination device, determination method, and non-transitory storage medium | |
| Chengula et al. | Improving road safety with ensemble learning: Detecting driver anomalies using vehicle inbuilt cameras | |
| JP5225870B2 (ja) | 情動分析装置 | |
| Baccour et al. | Comparative analysis of vehicle-based and driver-based features for driver drowsiness monitoring by support vector machines | |
| JP2007006427A (ja) | 映像監視装置 | |
| JP2010131401A (ja) | 被験者の頭及び目の動きを分析する方法及び装置 | |
| CN109657550B (zh) | 一种疲劳度检测方法及装置 | |
| Li et al. | Classification of eye movement and its application in driving based on a refined pre-processing and machine learning algorithm | |
| JP7552698B2 (ja) | 画像処理装置、画像処理方法、及び、記録媒体 | |
| CN119502943B (zh) | 基于复杂情景的驾驶行为规范方法、系统、介质及设备 | |
| CN118447486A (zh) | 状态确定方法、装置、车辆及计算机可读存储介质 | |
| Hecht et al. | Information constrained control analysis of eye gaze distribution under workload | |
| JP7439932B2 (ja) | 情報処理システム、データ蓄積装置、データ生成装置、情報処理方法、データ蓄積方法、データ生成方法及び記録媒体、並びに、データベース | |
| JP7120258B2 (ja) | 行動認識装置、及び行動認識方法 | |
| CN113413134A (zh) | 疲劳识别方法、装置、电子设备及存储介质 | |
| Palazzi et al. | Where should you attend while driving? |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20895821 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 20895821 Country of ref document: EP Kind code of ref document: A1 |