WO2025004257A1 - 眠気推定装置および眠気推定方法 - Google Patents

眠気推定装置および眠気推定方法 Download PDF

Info

Publication number
WO2025004257A1
WO2025004257A1 PCT/JP2023/024104 JP2023024104W WO2025004257A1 WO 2025004257 A1 WO2025004257 A1 WO 2025004257A1 JP 2023024104 W JP2023024104 W JP 2023024104W WO 2025004257 A1 WO2025004257 A1 WO 2025004257A1
Authority
WO
WIPO (PCT)
Prior art keywords
drowsiness
noise factor
unit
score
related information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2023/024104
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
高貴 上野
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mitsubishi Electric Corp
Original Assignee
Mitsubishi Electric Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mitsubishi Electric Corp filed Critical Mitsubishi Electric Corp
Priority to JP2025529114A priority Critical patent/JP7843933B2/ja
Priority to PCT/JP2023/024104 priority patent/WO2025004257A1/ja
Publication of WO2025004257A1 publication Critical patent/WO2025004257A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/18—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state for vehicle drivers or machine operators
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems

Definitions

  • This disclosure relates to a drowsiness estimation device and a drowsiness estimation method.
  • the present disclosure has been made to solve the above-mentioned problems, and aims to provide a drowsiness estimation device that, when estimating the drowsiness of an occupant of a moving body, prevents the accuracy of estimating the drowsiness of the occupant from decreasing due to the occupant's behavior exhibiting characteristics similar to those observed when drowsiness occurs.
  • the drowsiness estimation device includes a sensing unit that acquires drowsiness-related information indicating a state related to drowsiness of the occupant for each frame based on frames of an image capturing an image of the face of an occupant of a moving body, a first noise factor detection unit that detects a noise factor behavior by the occupant that is a behavior involving eye movement similar to a behavior caused by drowsiness based on the drowsiness-related information acquired by the sensing unit, a feature calculation unit that calculates a drowsiness estimation feature amount for estimating the drowsiness of the occupant based on post-noise factor removal drowsiness-related information obtained after excluding the drowsiness-related information that was the basis for the detection of the noise factor behavior by the first noise factor detection unit from the drowsiness-related information acquired by the sensing unit, a drowsiness score calculation unit that calculates a drowsiness score using the d
  • the present disclosure when estimating the drowsiness of an occupant of a moving body, it is possible to prevent a decrease in the accuracy of estimating the drowsiness of the occupant due to the occupant engaging in behavior that exhibits characteristics similar to those observed when drowsiness occurs.
  • FIG. 1 is a diagram illustrating an example of the configuration of a drowsiness estimation device according to a first embodiment.
  • 4 is a flowchart for explaining an operation of the drowsiness estimation device according to the first embodiment.
  • This is a flowchart for explaining details of the feature calculation process in embodiment 1, in which the feature calculation unit identifies drowsiness-related information after noise factors have been removed based on the exclusion flag that the sensing result selection unit has assigned to the drowsiness-related information to be excluded, and calculates features for estimating drowsiness based on the identified drowsiness-related information after noise factors have been removed.
  • FIG. 13 is a flowchart for explaining details of the feature calculation process in embodiment 1 when the feature calculation unit calculates features for estimating drowsiness based on drowsiness-related information after noise factor removal output from the sensing result selection unit.
  • 5A and 5B are diagrams illustrating an example of a hardware configuration of a drowsiness estimation device according to embodiment 1.
  • FIG. 11 is a diagram illustrating an example of the configuration of a drowsiness estimation device according to a second embodiment.
  • 10 is a flowchart for explaining an operation of a drowsiness estimation device according to a second embodiment.
  • FIG. 13 is a diagram illustrating an example of the configuration of a drowsiness estimation device according to a third embodiment.
  • FIG. 13 is a flowchart for explaining an operation of a drowsiness estimation device according to a third embodiment.
  • 1 is a diagram showing an example of the configuration of a drowsiness estimation device that combines the configurations of a drowsiness estimation device according to a first embodiment, a drowsiness estimation device according to a second embodiment, and a drowsiness estimation device according to a third embodiment.
  • FIG. 1 is a flowchart for explaining the operation of a drowsiness estimation device that combines the configurations of a drowsiness estimation device according to embodiment 1, a drowsiness estimation device according to embodiment 2, and a drowsiness estimation device according to embodiment 3.
  • FIG. 1 is a diagram showing an example of the configuration of a drowsiness estimation device 1 according to the first embodiment.
  • the drowsiness estimation device 1 of embodiment 1 is connected to an imaging device 2, and estimates the drowsiness of a person (hereinafter referred to as the “subject”) whose drowsiness is to be estimated based on an image captured by the imaging device 2.
  • the subject is assumed to be a driver of a vehicle (not shown).
  • the drowsiness estimation device 1 according to the first embodiment is mounted on the vehicle.
  • the driver of the vehicle is also simply referred to as the "driver”.
  • the imaging device 2 is mounted on a vehicle.
  • the imaging device 2 is installed in the center of the dashboard, on an A-pillar, on a meter panel, or the like of the vehicle so as to be able to capture an image of at least the face of the driver.
  • the imaging device 2 may be shared with a so-called "Driver Monitoring System (DMS)."
  • the imaging device 2 is a visible light camera or an infrared camera.
  • the imaging device 2 is provided with a light source (not shown) that irradiates an area including the driver's face with infrared light for imaging.
  • the light source is, for example, an LED (Light Emitting Diode).
  • the imaging device 2 outputs the captured image (hereinafter referred to as the “captured image”) to the drowsiness estimation device 1 .
  • the drowsiness estimation device 1 includes a sensing unit 11 , a first noise factor detection unit 12 , a feature calculation unit 13 , a drowsiness score calculation unit 14 , and a drowsiness estimation unit 15 .
  • the feature amount calculation unit 13 includes a sensing result selection unit 131 .
  • the sensing unit 11 acquires information indicating a state related to the driver's drowsiness (hereinafter referred to as "drowsiness-related information") based on a captured image of the driver's face.
  • the sensing unit 11 acquires the captured image frame by frame from the imaging device 2.
  • the sensing unit 11 acquires the drowsiness-related information for each frame.
  • the process of acquiring drowsiness-related information performed by the sensing unit 11 is referred to as the "sensing process.”
  • the state related to the driver's drowsiness includes how open the driver's eyes are (in other words, the degree of eyelid opening), how open the driver's mouth is (in other words, the degree of mouth opening), the position of the driver's facial feature points, the driver's line of sight, the driver's facial orientation, the driver's head position, etc.
  • the facial feature points are the corners and corners of the eyes, points on the outer circumference of the eyes, etc., and the positions of the driver's facial feature points are indicated, for example, by coordinates on the captured image.
  • the driver's facial orientation is represented, for example, by an angle based on the time when the driver faces forward in the traveling direction of the vehicle (0 degrees).
  • the driver's head position is represented by coordinates in real space.
  • the sensing unit 11 can calculate the driver's facial orientation and head position from the captured image.
  • what state is determined to be a state related to driver drowsiness is determined in advance by an administrator or the like.
  • the sensing unit 11 may detect a state related to the driver's drowsiness by using a known image recognition technique and acquire drowsiness-related information.
  • the sensing unit 11 outputs the acquired drowsiness-related information to the first noise factor detection unit 12.
  • the sensing unit 11 may, for example, associate the drowsiness-related information with a frame of the captured image from which the drowsiness-related information was acquired, and output the information to the first noise factor detection unit 12.
  • the drowsiness-related information output by the sensing unit 11 to the first noise factor detection unit 12, or information in which the drowsiness-related information is associated with a frame of the captured image from which the drowsiness-related information was acquired is also referred to as a "sensing result.” It is assumed that each frame of the captured image is assigned information indicating the image capture date and time.
  • the first noise factor detection unit 12 detects a driver's behavior involving eye movement (hereinafter referred to as a "noise factor behavior”) that is similar to a behavior caused by drowsiness, based on the drowsiness-related information acquired by the sensing unit 11.
  • a noise factor behavior a driver's behavior involving eye movement
  • the process of detecting a noise factor performed by the first noise factor detection unit 12 is referred to as a "first noise factor detection process.”
  • noise-causing behaviors include behavior of looking downward (so-called downward gaze) and behavior of squinting the eyes.
  • the driver may look downward when viewing the meter.
  • the driver may make a so-called "frowning face.”
  • the so-called frowning face is also called a "suffering face.”
  • people generally narrow their eyes when making a “suffering face.”
  • people generally narrow their eyes when they smile.
  • the above-mentioned behaviors of looking down, making a distressed face, or smiling can be said to be noise-causing behaviors similar to behaviors caused by drowsiness, such as covering the eyes with the eyelids or closing the eyes.
  • the first noise cause detection unit 12 detects such noise causing behavior. It should be noted that the type of behavior that is to be regarded as a noise causing behavior is determined in advance by an administrator or the like.
  • the first noise factor detection unit 12 detects that the driver has performed a noise-causing behavior if the degree of eyelid opening of the driver is below a predetermined threshold value (hereinafter referred to as the ⁇ eyelid opening degree determination threshold value'') based on the drowsiness-related information.
  • a predetermined threshold value hereinafter referred to as the ⁇ eyelid opening degree determination threshold value''
  • the first noise factor detection unit 12 may detect that the driver has performed a noise factor behavior when, for example, based on time-series drowsiness-related information, the degree of eyelid opening of the driver becomes smaller than the degree of eyelid opening based on the immediately preceding drowsiness-related information by a preset threshold value or more (hereinafter referred to as "threshold value for determining eyelid opening degree difference").
  • the first noise factor detection unit 12 stores the sensing results acquired from the sensing unit 11 in time series in a storage unit (not shown).
  • the first noise factor detection unit 12 can identify the immediately preceding drowsiness-related information based on the sensing results stored in the storage unit.
  • the storage unit is provided in a location that can be referenced by the drowsiness estimation device 1.
  • the eyelid opening degree determination threshold and the eyelid opening degree difference determination threshold are appropriately set by an administrator or the like and stored in the storage unit.
  • the drowsiness-related information includes at least the degree to which the driver's eyelids are open.
  • the first noise factor detection unit 12 may detect the occurrence of a noise causing behavior by the driver using a model that has previously learned a "patient face" (hereinafter referred to as a "machine learning model").
  • the machine learning model that the first noise factor detection unit 12 uses to detect the noise causing behavior by the driver is also referred to as a "first machine learning model”.
  • the first machine learning model is a machine learning model such as a Support Vector Machine (SVM), a Random Forest, a LightGBM (Light Gradient Boosting Machine), or a convolutional neural network.
  • the first machine learning model is a model that receives information indicating the positions of feature points of a person's face on an image, and outputs information indicating whether the person is making a restless face.
  • the first machine learning model is generated in advance and stored in a storage unit.
  • the first noise factor detection unit 12 detects whether the driver is making a distressed face by inputting drowsiness-related information into a first machine learning model and obtaining information indicating whether the driver is making a distressed face.
  • the drowsiness-related information includes at least position information of characteristic points of the driver's face on the captured image.
  • the first noise factor detection unit 12 detects that the driver has performed a noise-causing behavior when the driver's gaze direction is below a predetermined threshold value (hereinafter referred to as the "gaze direction determination threshold value”) based on the drowsiness-related information.
  • the first noise factor detection unit 12 may detect, for example, that the driver has engaged in noise-causing behavior when the driver's line of sight changes downward by more than a predetermined angle (hereinafter referred to as the "downward gaze determination angle") during a predetermined period (hereinafter referred to as the "downward gaze determination period”) based on time-series drowsiness-related information.
  • the gaze direction determination threshold, the downward gaze determination period, and the downward gaze determination angle are appropriately set by an administrator or the like according to the installation position and angle of view of the imaging device 2, and are stored in the memory unit.
  • the drowsiness-related information includes at least the driver's line of sight.
  • the first noise factor detection unit 12 detects that a noise factor behavior has been performed by the driver, it outputs information regarding the detected noise factor behavior (hereinafter referred to as ⁇ noise factor behavior information'') to the feature calculation unit 13 together with the sensing results obtained from the sensing unit 11.
  • the noise causing behavior information includes information that the first noise factor detection unit 12 has detected the noise causing behavior and drowsiness-related information that is the basis for detecting the noise causing behavior by the first noise factor detection unit 12.
  • the noise causing behavior information may further include information that can identify the type of the noise causing behavior detected by the first noise factor detection unit 12 (e.g., looking downward, squinting, etc.).
  • the feature calculation unit 13 calculates features for estimating the driver's drowsiness (hereinafter referred to as "features for drowsiness estimation”) based on drowsiness-related information obtained by the sensing unit 11 after excluding the drowsiness-related information that was the basis for the first noise factor detection unit 12 to detect noise-causing behavior (hereinafter referred to as “drowsiness-related information after noise factor removal”).
  • featuresiness estimation drowsiness-related information obtained by the sensing unit 11 after excluding the drowsiness-related information that was the basis for the first noise factor detection unit 12 to detect noise-causing behavior
  • the process of calculating drowsiness estimation feature amounts performed by the feature amount calculation unit 13 is referred to as a "feature amount calculation process.”
  • the sensing result selection unit 131 provided in the feature calculation unit 13 assigns an exclusion flag to drowsiness-related information acquired by the sensing unit 11 that is to be excluded when calculating the feature for drowsiness estimation, based on the noise factor behavior information related to the noise factor behavior detected by the first noise factor detection unit 12.
  • the sensing result selection unit 131 selects, from among the drowsiness-related information included in the sensing result, drowsiness-related information that is the basis for the first noise factor detection unit 12 to detect the noise factor behavior as drowsiness-related information to be excluded. Then, the sensing result selection unit 131 assigns an exclusion target flag to the drowsiness-related information to be excluded in the sensing result. As described above, the first noise factor detection unit 12 outputs the sensing result acquired from the sensing unit 11 together with the noise factor behavior information to the feature calculation unit 13.
  • the sensing result selection unit 131 can identify drowsiness-related information to be excluded from the noise factor behavior information and the sensing result output from the first noise factor detection unit 12.
  • the sensing result selection unit 131 assigns an exclusion target flag to drowsiness-related information included in the sensing result output from the first noise factor detection unit 12 if the drowsiness-related information is to be excluded. Then, the sensing result selection unit 131 outputs the sensing result after assigning the exclusion target flag to the drowsiness-related information to be excluded to the feature calculation unit 13.
  • the feature calculation unit 13 excludes, from the drowsiness-related information acquired by the sensing unit 11, the drowsiness-related information to which the sensing result selection unit 131 has assigned an exclusion target flag, thereby identifying the drowsiness-related information after excluding the drowsiness-related information that was the basis for the first noise factor detection unit 12 to detect the noise factor behavior from the drowsiness-related information acquired by the sensing unit 11, i.e., the drowsiness-related information after noise factor removal.
  • the feature amount calculation unit 13 calculates a feature amount for estimating drowsiness based on the identified drowsiness-related information after noise factors are removed. In other words, the feature calculation unit 13 calculates features for estimating drowsiness based on drowsiness-related information that has not been assigned an exclusion flag among the drowsiness-related information obtained from the sensing unit 11 via the first noise factor detection unit 12.
  • the feature amount for estimating drowsiness includes, for example, the degree of eyelid opening, the degree of opening, the driver's facial direction, the driver's head position, the driver's line of sight, the driver's PERCLOS (Percent of the time eyes are closed per unit time), the number of times the driver blinks, or the driver's blink rate.
  • the feature amount calculation unit 13 calculates the above-mentioned feature amount for estimating drowsiness based on the drowsiness-related information after noise factor removal.
  • the feature calculation unit 13 stores the sensing result output from the sensing result selection unit 131, and calculates the drowsiness estimation feature based on the stored sensing result and on a predetermined number of past drowsiness-related information.
  • the feature calculation unit 13 may calculate features for estimating drowsiness based on, for example, stored drowsiness-related information after noise factors have been removed and acquired during a predetermined period in the past (hereinafter referred to as the "feature calculation target period").
  • the feature calculation unit 13 calculates the drowsiness estimation feature based on the drowsiness-related information after noise factors are removed that has been acquired over the past three minutes.
  • the drowsiness-related information after noise factor removal drowsiness-related information acquired based on the captured image acquired from the imaging device 2 is excluded from drowsiness-related information to be excluded.
  • drowsiness-related information that is the basis for the first noise factor detection unit 12 to detect the noise factor behavior is excluded from drowsiness-related information acquired by the sensing unit 11 based on the imaging device 2.
  • the feature amount calculation unit 13 does not use drowsiness-related information acquired based on a frame of the captured image in which the driver performing the noise factor behavior is captured, for calculating the feature amount for drowsiness estimation. More specifically, the feature amount calculation unit 13 does not use drowsiness-related information acquired based on a frame of the captured image in which the driver performing the noise factor behavior is captured, for calculating the feature amount for drowsiness estimation.
  • the feature calculation unit 13 outputs the drowsiness-related information, more specifically, information associating the drowsiness-related information after noise factor removal with the calculated feature for estimating drowsiness (hereinafter referred to as "feature information") to the drowsiness score calculation unit 14.
  • the sensing result selection unit 131 assigns an exclusion flag to the drowsiness-related information acquired by the sensing unit 11, and the feature calculation unit 13 identifies drowsiness-related information after noise factors have been removed based on the exclusion flag, and calculates features for estimating drowsiness from the identified drowsiness-related information after noise factors have been removed.
  • the feature amount calculation unit 13 may calculate the drowsiness estimation feature amount by other methods.
  • the sensing result selection unit 131 identifies drowsiness-related information to be excluded based on noise-causing behavior information regarding noise-causing behavior detected by the first noise factor detection unit 12, it excludes the identified drowsiness-related information to be excluded from the drowsiness-related information included in the sensing result and acquired by the sensing unit 11, and selects the excluded drowsiness-related information as drowsiness-related information after noise factor removal.
  • the sensing result selection unit 131 selects the drowsiness-related information after noise factor removal, it outputs the sensing result including the selected drowsiness-related information after noise factor removal (hereinafter referred to as the “sensing result after noise removal”) to the feature calculation unit 13.
  • the feature amount calculation unit 13 calculates a feature amount for estimating drowsiness, based on the drowsiness-related information after noise factor removal output from the sensing result selection unit 131 .
  • the feature calculation unit 13 calculates features for estimating drowsiness based on drowsiness-related information selected by the sensing result selection unit 131 from the drowsiness-related information acquired from the sensing unit 11 via the first noise factor detection unit 12 and included in the sensing result after noise removal.
  • the feature amount calculation unit 13 may calculate the drowsiness estimation feature amount in this manner.
  • the drowsiness score calculation unit 14 calculates the drowsiness score using the feature for drowsiness estimation calculated by the feature calculation unit 13.
  • the drowsiness score calculation unit 14 can identify the feature for drowsiness estimation calculated by the feature calculation unit 13 from the feature information output from the feature calculation unit 13.
  • the drowsiness score calculated by the drowsiness score calculation unit 14 is a score indicating the degree of drowsiness of the driver, which is used for estimating the drowsiness of the driver.
  • the drowsiness score is expressed as a value ranging from "0" to "100", and a higher drowsiness score indicates a higher degree of drowsiness of the driver.
  • the estimation of the drowsiness of the driver using the drowsiness score is performed by the drowsiness estimation unit 15.
  • the process of calculating the sleepiness score performed by the sleepiness score calculation unit 14 is referred to as a "sleepiness score calculation process.”
  • the drowsiness score calculation unit 14 calculates the drowsiness score using, for example, a machine learning model that has previously learned the drowsiness score.
  • the machine learning model used by the drowsiness score calculation unit 14 to calculate the drowsiness score is also referred to as a “second machine learning model.”
  • the second machine learning model is a machine learning model such as a Support Vector Machine (SVM), a Random Forest, a Light Gradient Boosting Machine (LightGBM), or a convolutional neural network.
  • SVM Support Vector Machine
  • Random Forest Random Forest
  • Light Gradient Boosting Machine Light Gradient Boosting Machine
  • the second machine learning model receives, for example, a feature for drowsiness estimation as an input and outputs a drowsiness score.
  • the second machine learning model is generated in advance and stored in a location that can be referenced by the drowsiness score calculation unit 14, such as a storage unit.
  • the drowsiness score calculation unit 14 calculates the drowsiness score by inputting the features for estimating drowsiness into the second machine learning model to obtain the drowsiness score.
  • the drowsiness score calculation unit 14 may calculate the drowsiness score based on, for example, a rule for calculating the drowsiness score that is set in advance (hereinafter referred to as a "rule for calculating the drowsiness score").
  • the sleepiness score calculation rules are generated in advance by an administrator or the like, and are stored in a location that can be referenced by the sleepiness score calculation unit 14, such as a memory unit.
  • the rules for calculating the drowsiness score include calculation rules for the drowsiness score based on the number of blinks in a set period of time, such as "if the number of blinks in the past three minutes is 20 or more, the drowsiness score is set to 60.”
  • the drowsiness score calculation unit 14 outputs the calculated driver's drowsiness score to the drowsiness estimation unit 15.
  • the drowsiness estimation unit 15 estimates the drowsiness of the driver based on the drowsiness score calculated by the drowsiness score calculation unit 14.
  • the driver's drowsiness estimated by the drowsiness estimation unit 15 may be expressed in multiple states, such as "awake,”"weakdrowsiness,” and “strong drowsiness,” or it may be expressed as a binary value of "drowsy” or “not drowsy,” or it may be a continuous value indicating the degree of drowsiness.
  • the process of estimating the driver's drowsiness performed by the drowsiness estimation unit 15 is referred to as a "drowsiness estimation process.”
  • the drowsiness estimation unit 15 estimates the driver's drowsiness using, for example, a machine learning model that has learned drowsiness in advance.
  • the machine learning model used by the drowsiness estimation unit 15 to estimate the driver's drowsiness is also referred to as a “third machine learning model.”
  • the third machine learning model is a machine learning model such as a Support Vector Machine (SVM), a Random Forest, a Light Gradient Boosting Machine (LightGBM), or a convolutional neural network.
  • SVM Support Vector Machine
  • Random Forest Random Forest
  • Light Gradient Boosting Machine Light Gradient Boosting Machine
  • the third machine learning model receives a drowsiness score as input and outputs information indicating drowsiness.
  • the information indicating drowsiness is, for example, information indicating a plurality of states of the driver's drowsiness, information indicating "drowsy" or "not drowsy", or a continuous value indicating the degree of drowsiness.
  • the third machine learning model is generated in advance and stored in a location that can be referenced by the drowsiness estimation unit 15, such as a memory unit.
  • the drowsiness estimation unit 15 estimates the driver's drowsiness by inputting the drowsiness score into the third machine learning model to obtain information indicating drowsiness.
  • the drowsiness estimation unit 15 may estimate the drowsiness of the driver based on, for example, a preset rule for estimating the drowsiness of the driver (hereinafter referred to as a "drowsiness estimation rule").
  • the drowsiness estimation rules are generated in advance by an administrator or the like, and are stored in a location that can be referenced by the drowsiness estimation unit 15, such as a memory unit.
  • the rules for drowsiness estimation include, for example, conditions that correspond to the range of drowsiness scores and information indicating multiple states of drowsiness of the driver, such as "if the drowsiness score is between “0" and “50”, it is considered “awake”, if the drowsiness score is between “50” and “60”, it is considered “weak drowsiness”, and if the drowsiness score is "60” or above, it is considered “strong drowsiness”, "if the drowsiness score is "60” or above, it is considered “drowsy”, and if the drowsiness score is less than “60", it is considered “not drowsy”, or a calculation formula for the degree of drowsiness based on the drowsiness score.
  • the drowsiness estimation unit 15 outputs the estimation result of the driver's drowsiness (hereinafter referred to as the “drowsiness estimation result”) to an external device (not shown) of the drowsiness estimation device 1 .
  • the drowsiness estimation unit 15 outputs the drowsiness estimation result to an alarm device mounted on the vehicle.
  • the alarm device outputs an alarm when the drowsiness estimation result indicates that the driver is drowsy.
  • the drowsiness estimation unit 15 may store the drowsiness estimation result in a storage unit.
  • FIG. 2 is a flowchart for explaining the operation of the drowsiness estimation device 1 according to the first embodiment. For example, when the power supply of the vehicle is turned on and a captured image is output from the imaging device 2, the drowsiness estimation device 1 repeats the operation shown in the flowchart of FIG. 2 until the power supply of the vehicle is turned off.
  • the sensing unit 11 acquires a captured image of the driver's face from the imaging device 2, and performs a sensing process to acquire drowsiness-related information based on the acquired captured image (step ST10).
  • the sensing unit 11 outputs the sensing result to the first noise factor detection unit 12 .
  • the first noise factor detection unit 12 performs a first noise factor detection process to detect noise factor behavior by the driver that is similar to behavior caused by drowsiness, based on the drowsiness-related information acquired by the sensing unit 11 in step ST10 (step ST20).
  • the first noise factor detection unit 12 detects that a noise causing behavior has been performed by the driver, it outputs noise causing behavior information to the feature calculation unit 13 together with the sensing result acquired from the sensing unit 11 .
  • the first noise factor detection unit 12 may output information to the feature calculation unit 13 to the effect that it has not detected that a noise-causing behavior has occurred, or it may not output anything to the feature calculation unit 13.
  • the feature amount calculation unit 13 performs a feature amount calculation process to calculate a feature amount for drowsiness estimation based on drowsiness-related information after noise factor removal after excluding the drowsiness-related information that was the basis for the first noise factor detection unit 12 to detect the noise factor behavior in step ST20 from the drowsiness-related information acquired by the sensing unit 11 in step ST10 (step ST30).
  • the feature amount calculation unit 13 performs a feature amount calculation process to calculate a feature amount for drowsiness estimation based on the drowsiness-related information after noise factor removal after excluding the drowsiness-related information that was the basis for the first noise factor detection unit 12 to detect the noise factor behavior from the drowsiness-related information acquired by the sensing unit 11.
  • the feature amount calculation unit 13 outputs the feature amount information to the drowsiness score calculation unit 14 .
  • the drowsiness score calculation unit 14 performs a drowsiness score calculation process to calculate a drowsiness score using the features for drowsiness estimation calculated by the feature calculation unit 13 in step ST30 (step ST40).
  • the drowsiness score calculation unit 14 outputs the calculated drowsiness score of the driver to the drowsiness estimation unit 15 .
  • the drowsiness estimation unit 15 performs a drowsiness estimation process to estimate the drowsiness of the driver based on the drowsiness score calculated by the drowsiness score calculation unit 14 in step ST40 (step ST50).
  • the drowsiness estimation unit 15 outputs a result of estimating the driver's drowsiness.
  • FIG. 3 is a flowchart for explaining an example of details of the process of step ST30 in FIG. More specifically, Figure 3 is a flowchart for explaining the details of the feature calculation process in embodiment 1, in which the feature calculation unit 13 identifies drowsiness-related information after noise factors have been removed based on the exclusion target flag that the sensing result selection unit 131 has assigned to the drowsiness-related information to be excluded, and calculates features for estimating drowsiness based on the identified drowsiness-related information after noise factors have been removed.
  • the sensing result selection unit 131 Based on noise factor behavior information regarding the noise factor behavior detected by the first noise factor detection unit 12, the sensing result selection unit 131 assigns an exclusion target flag to drowsiness-related information acquired by the sensing unit 11 that is to be excluded when calculating the features for estimating drowsiness (step ST301). The sensing result selection unit 131 assigns an exclusion target flag to the drowsiness-related information to be excluded, and outputs the sensing results to the feature calculation unit 13.
  • the feature calculation unit 13 Based on the sensing results output from the sensing result selection unit 131 in step ST301, the feature calculation unit 13 excludes drowsiness-related information to be excluded, to which the sensing result selection unit 131 has assigned an exclusion flag, from the drowsiness-related information acquired by the sensing unit 11, thereby identifying drowsiness-related information after noise factors have been removed (step ST302).
  • the feature amount calculation unit 13 calculates a feature amount for estimating drowsiness, based on the drowsiness-related information after noise factor removal identified in step ST302 (step ST303).
  • the feature amount calculation unit 13 outputs the feature amount information to the drowsiness score calculation unit 14 .
  • FIG. 4 is a flowchart for explaining another example of the details of the process of step ST30 in FIG. More specifically, Figure 4 is a flowchart for explaining the details of the feature calculation process in embodiment 1 when the feature calculation unit 13 calculates features for estimating drowsiness based on drowsiness-related information after noise factor removal output from the sensing result selection unit 131.
  • the sensing result selection unit 131 When the sensing result selection unit 131 identifies drowsiness-related information to be excluded based on noise-causing behavior information regarding the noise-causing behavior detected by the first noise factor detection unit 12, it excludes the identified drowsiness-related information to be excluded from the drowsiness-related information included in the sensing result and acquired by the sensing unit 11, and selects the drowsiness-related information after the exclusion as drowsiness-related information after noise factor removal (step ST311). When the sensing result selection unit 131 selects the drowsiness-related information after noise factor removal, the sensing result selection unit 131 outputs the sensing result after noise removal to the feature calculation unit 13.
  • the feature amount calculation unit 13 calculates a feature amount for estimating drowsiness, based on the drowsiness related information after noise factor removal output from the sensing result selection unit 131 in step ST311 (step ST312).
  • the feature amount calculation unit 13 outputs the feature amount information to the drowsiness score calculation unit 14 .
  • the drowsiness estimation device 1 acquires drowsiness-related information for each frame based on frames of the captured image of the driver's face, and detects noise-causing behavior by the driver that is similar to behavior caused by drowsiness based on the acquired drowsiness-related information.
  • the drowsiness estimation device 1 calculates drowsiness estimation feature quantities based on drowsiness-related information after noise factors have been removed, which is obtained after excluding the drowsiness-related information that was the basis for detecting the noise-causing behavior from the acquired drowsiness-related information.
  • the drowsiness estimation device 1 then calculates a drowsiness score using the drowsiness estimation feature quantities, and estimates the driver's drowsiness based on the calculated drowsiness score.
  • the drowsiness estimation device 1 calculates drowsiness estimation feature quantities based on the drowsiness-related information after noise factor removal, and estimates the driver's drowsiness based on the drowsiness score calculated using the drowsiness estimation feature quantities. Therefore, when estimating the drowsiness of a driver, the drowsiness estimation device 1 can prevent a decrease in the estimation accuracy of the driver's drowsiness due to the driver's noise-causing behavior.
  • the drowsiness estimation device 1 can prevent a decrease in the estimation accuracy of the driver's drowsiness due to a feature amount that becomes noise being used in estimating the driver's drowsiness, and can perform a highly accurate drowsiness estimation.
  • the features that become noise are features calculated based on an image capturing the face of a driver who is engaging in noise-causing behavior, and features calculated based on an image capturing the face of a driver who is not engaging in noise-causing behavior are necessary for accurate estimation of the driver's drowsiness.
  • the drowsiness estimation device 1 Even if the driver performs a noise-causing behavior such as looking down, the drowsiness estimation device 1 removes only the feature amount that becomes noise due to the noise-causing behavior, and performs drowsiness estimation of the driver using the feature amount after the feature amount that becomes noise has been removed as a feature amount for drowsiness estimation. In other words, even if the driver performs a noise-causing behavior, the drowsiness estimation device 1 does not stop estimating the driver's drowsiness itself, but performs drowsiness estimation using the necessary feature amount.
  • the drowsiness estimation device 1 can prevent the estimation accuracy of the driver's drowsiness from decreasing due to the feature amount that becomes noise being used to estimate the driver's drowsiness, and can perform drowsiness estimation with high accuracy.
  • the drowsiness estimation device 1 includes the processing circuit 1001 for performing control to estimate the drowsiness of a driver based on an image acquired from an imaging device 2, using feature amounts excluding feature amounts calculated due to the driver performing a noise factor behavior as feature amounts for drowsiness estimation.
  • the processing circuitry 1001 may be dedicated hardware as shown in FIG. 5A, or may be a processor 1004 executing a program stored in a memory as shown in FIG. 5B.
  • the processing circuit 1001 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination of these.
  • ASIC Application Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array
  • the processing circuit is a processor 1004
  • the functions of the sensing unit 11, first noise factor detection unit 12, feature calculation unit 13, drowsiness score calculation unit 14, and drowsiness estimation unit 15 are realized by software, firmware, or a combination of software and firmware.
  • the software or firmware is written as a program and stored in memory 1005.
  • the processor 1004 executes the functions of the sensing unit 11, first noise factor detection unit 12, feature calculation unit 13, drowsiness score calculation unit 14, and drowsiness estimation unit 15 by reading and executing the program stored in memory 1005.
  • the drowsiness estimation device 1 includes a memory 1005 for storing a program that, when executed by the processor 1004, results in the execution of steps ST10 to ST50 in FIG. 2 described above.
  • memory 1005 may be, for example, a non-volatile or volatile semiconductor memory such as a RAM, a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), or an EEPROM (Electrically Erasable Programmable Read-Only Memory), or a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, or a DVD (Digital Versatile Disc).
  • a non-volatile or volatile semiconductor memory such as a RAM, a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), or an EEPROM (Electrically Erasable Programmable Read-Only Memory), or a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, or a DVD (Digital Versatile Disc).
  • the functions of the sensing unit 11, the first noise factor detection unit 12, the feature amount calculation unit 13, the drowsiness score calculation unit 14, and the drowsiness estimation unit 15 may be partially realized by dedicated hardware and partially realized by software or firmware.
  • the functions of the sensing unit 11 may be realized by a processing circuit 1001 as dedicated hardware, and the functions of the first noise factor detection unit 12, the feature amount calculation unit 13, the drowsiness score calculation unit 14, and the drowsiness estimation unit 15 may be realized by the processor 1004 reading and executing a program stored in the memory 1005.
  • the drowsiness estimation device 1 also includes devices such as the imaging device 2, an input interface device 1002 that performs wired or wireless communication, and an output interface device 1003.
  • the drowsiness estimation device 1 is an on-board device mounted in a vehicle, and the sensing unit 11, the first noise factor detection unit 12, the feature calculation unit 13, the drowsiness score calculation unit 14, and the drowsiness estimation unit 15 are provided in the drowsiness estimation device 1.
  • some of the sensing unit 11, the first noise factor detection unit 12, the feature calculation unit 13, the drowsiness score calculation unit 14, and the drowsiness estimation unit 15 may be mounted on the vehicle's in-vehicle device, and the others may be provided on a server connected to the in-vehicle device via a network, and the drowsiness estimation system may be configured with the in-vehicle device and the server.
  • the sensing unit 11, the first noise factor detection unit 12, the feature calculation unit 13, the drowsiness score calculation unit 14, and the drowsiness estimation unit 15 may all be provided in the server.
  • the subject is a vehicle driver as an example, but this is merely one example.
  • the subject may be a vehicle occupant other than the driver.
  • the subject may also be an occupant, including the driver, of a moving body other than a vehicle, such as a bus, train, or airplane.
  • the drowsiness estimation device 1 according to embodiment 1 can be applied as a drowsiness estimation device that estimates the drowsiness of an occupant of a moving body other than a vehicle.
  • the drowsiness estimation device 1 is configured to include a sensing unit 11 that acquires drowsiness-related information indicating a state related to the occupant's drowsiness for each frame based on frames of an image capturing the face of an occupant of a moving body; a first noise factor detection unit 12 that detects noise factor behavior by the occupant, which is behavior involving eye movement similar to behavior caused by drowsiness, based on the drowsiness-related information acquired by the sensing unit 11; a feature calculation unit 13 that calculates drowsiness estimation feature amounts for estimating the occupant's drowsiness based on drowsiness-related information after noise factor removal is removed from the drowsiness-related information acquired by the sensing unit 11, which is the basis for the drowsiness-related information that the first noise factor detection unit 12 detected the noise factor behavior; a drowsiness score calculation unit 14 that calculates a drowsiness score using the drow
  • the drowsiness estimation device 1 can prevent a decrease in the accuracy of estimating the drowsiness of the occupant due to the occupant's behavior exhibiting characteristics similar to those observed when drowsiness occurs.
  • Embodiment 2 Among the driving conditions of a moving body, there are driving conditions in which it is assumed that the occupant is unlikely to become drowsy. If it is estimated that the occupant is drowsy in such a driving state, this is an erroneous estimation (overestimation). If a warning is issued based on the erroneous estimation, the warning is an over-alarm and may be annoying to the occupant.
  • the second noise factor detection unit 16 acquires vehicle information from the vehicle information acquisition device 3, and detects a vehicle driving condition (hereinafter referred to as a "noise factor driving condition") that is expected to be unlikely to cause drowsiness in the driver based on the acquired vehicle information.
  • a vehicle driving condition hereinafter referred to as a "noise factor driving condition”
  • the process of detecting a noise cause driving condition performed by the second noise cause detection unit 16 is referred to as a "second noise cause detection process.”
  • noise-causing driving conditions examples include a state in which the vehicle is traveling at a low speed, a state in which the turn signal is used frequently while the vehicle is traveling, a state in which the brakes are used frequently, or a state in which there is a large change in the steering wheel angle of the vehicle. It should be noted that the vehicle driving conditions that are to be considered as noise-causing driving conditions are determined in advance by an administrator or the like.
  • the vehicle information conditions for determining that the vehicle is in a noise factor driving state are, for example, that the vehicle speed is equal to or lower than a predetermined threshold, that the frequency of use of the turn signal in a predetermined period is equal to or higher than a predetermined threshold, that the frequency of use of the brake in a predetermined period is equal to or higher than a predetermined threshold, and that the amount of change in the steering wheel angle in a predetermined period is equal to or higher than a predetermined threshold.
  • the second noise factor detection unit 16 stores the vehicle information acquired from the vehicle information acquisition device 3 in a chronological order in a storage unit or the like. Based on the stored vehicle information, the second noise factor detection unit 16 can determine changes in the vehicle's driving conditions, such as changes in the steering angle.
  • the second noise factor detection unit 16 When the second noise factor detection unit 16 detects that the vehicle's driving state is a noise factor driving state, it outputs information regarding the detected noise factor driving state (hereinafter referred to as ⁇ noise factor driving state information'') to the score correction unit 17 together with the vehicle information acquired from the vehicle information acquisition device 3.
  • the noise factor driving condition information includes information that a noise factor driving condition has been detected by the second noise factor detection unit 16.
  • the noise factor driving condition information may further include information that can identify the type of noise factor driving condition detected by the second noise factor detection unit 16 (low speed driving, frequent use of blinkers, frequent use of brakes, large changes in steering angle, etc.).
  • the score correction unit 17 corrects the drowsiness score calculated by the drowsiness score calculation unit 14 when the second noise factor detection unit 16 detects a noise factor driving state.
  • the drowsiness score calculation unit 14 outputs the calculated drowsiness score of the driver to the score correction unit 17 .
  • the score correction unit 17 corrects the drowsiness score in accordance with predetermined rules (hereinafter referred to as "score correction rules").
  • the score correction rules are set in advance by an administrator or the like, and are stored in a location that can be referenced by the score correction unit 17, such as a storage unit.
  • the process of correcting the drowsiness score performed by the score correction unit 17 is referred to as a "drowsiness score correction process.”
  • the score correction rule is set to "multiply the drowsiness score by 0.5 when a noise-causing driving state is detected.”
  • the drowsiness score calculated by the drowsiness score calculation unit 14 is 90.
  • the second noise factor detection unit 16 detects that the driving state is a noise factor state in which the turn signal is used frequently because the frequency of turn signal use in a preset period is equal to or greater than a preset threshold value.
  • the vehicle may frequently change lanes.
  • the drowsiness score calculation unit 14 has calculated a high drowsiness score of "90" may indicate that the feature calculation unit 13 has calculated a feature that may cause the drowsiness score to be calculated high due to some behavior of the driver, such as the driver checking left and right when frequently changing lanes, which is regarded as an eye-closing action (i.e., an action when drowsiness occurs).
  • the drowsiness score calculated by the drowsiness score calculation unit 14 may be incorrect.
  • the score correction unit 17 corrects the drowsiness score to "45".
  • the drowsiness estimation unit 15 estimates the driver as "drowsy" when the drowsiness score exceeds "50.” In this case, if the drowsiness score is "90,” the drowsiness estimation unit 15 estimates the driver as "drowsy.” Then, for example, an alarm device (not shown) outputs an alarm based on the drowsiness estimation result of the drowsiness estimation unit 15 that the driver is "drowsy.” In the above-described example, if the score correction unit 17 does not correct the drowsiness score, a warning about drowsiness would be output to the driver even though it is assumed that the driver is not feeling drowsy.
  • the drowsiness estimation unit 15 can prevent overestimation of the driver's drowsiness, and can suppress excessive warnings as described above.
  • the above-mentioned specific example is merely one example.
  • the score correction rule may be set to a rule different from the example given above.
  • a score correction rule may be set so that the degree of correction of the drowsiness score varies depending on the type of noise-causing driving condition. For example, the score correction rule may be set to "when a noise-causing driving state in which the turn signal is used frequently is detected, the drowsiness score is multiplied by 0.5, and when a noise-causing driving state in which the turn signal is used frequently is detected, the drowsiness score is multiplied by 0.7.”
  • the score correction unit 17 outputs the corrected drowsiness score (hereinafter referred to as the “corrected drowsiness score”) to the drowsiness estimation unit 15 .
  • the drowsiness estimation unit 15 estimates the drowsiness of the driver based on the corrected drowsiness score.
  • FIG. 7 is a flowchart for explaining the operation of the drowsiness estimation device 1a according to the second embodiment.
  • the drowsiness estimation device 1a repeats the operation shown in the flowchart of FIG. 7 until the vehicle power is turned off.
  • the specific operations in the processing of steps ST10 to ST40 and step ST50 in Figure 7 are similar to the specific operations in the processing of steps ST10 to ST40 and step ST50 in Figure 2, which have already been explained in embodiment 1, so the same step numbers are used and duplicate explanations are omitted.
  • the second noise factor detection unit 16 acquires vehicle information from the vehicle information acquisition device 3, and performs a second noise factor detection process to detect a noise factor driving condition based on the acquired vehicle information (step ST60).
  • the second noise factor detection unit 16 detects that the vehicle's driving state is a noise factor driving state, it outputs noise factor driving state information to the score correction unit 17 together with the vehicle information acquired from the vehicle information acquisition device 3.
  • the second noise factor detection unit 16 may output information to the score correction unit 17 to the effect that it did not detect that the vehicle is in a noise factor driving state, or it may not output anything to the score correction unit 17.
  • the score correction unit 17 When the second noise factor detection unit 16 detects a noise factor driving state in step ST60, the score correction unit 17 performs a drowsiness score correction process to correct the drowsiness score calculated by the drowsiness score calculation unit 14 (step ST45). The score correction unit 17 outputs the corrected drowsiness score to the drowsiness estimation unit 15 .
  • the processes of steps ST10 to ST40 and the process of step ST60 are performed in parallel, but this is merely an example.
  • the drowsiness estimation device 1a may execute the process of step ST60 after the processes of steps ST10 to ST40. It is sufficient that the processes of steps ST10 to ST40 and step ST60 have been performed before the process of step ST45 is performed.
  • the drowsiness estimation device 1a acquires drowsiness-related information for each frame based on frames of an image capturing the driver's face, and detects noise-causing behavior by the driver that is similar to behavior caused by drowsiness based on the acquired drowsiness-related information.
  • the drowsiness estimation device 1a calculates drowsiness estimation feature quantities based on drowsiness-related information after noise factor removal, which is obtained after excluding drowsiness-related information that was the basis for detecting the noise-causing behavior from the acquired drowsiness-related information.
  • the drowsiness estimation device 1a then calculates a drowsiness score using the drowsiness estimation feature quantities, and estimates the driver's drowsiness based on the calculated drowsiness score. Furthermore, when a noise-causing driving state is detected based on the vehicle information, the drowsiness estimation device 1 a corrects the calculated drowsiness score. When the drowsiness estimation device 1 a corrects the drowsiness score, the drowsiness estimation device 1 a estimates the driver's drowsiness based on the corrected drowsiness score.
  • the drowsiness estimation device 1a can prevent the accuracy of the estimation of the driver's drowsiness from decreasing due to the driver engaging in noise-causing behavior, and more specifically, when estimating the driver's drowsiness, can prevent the accuracy of the estimation of the driver's drowsiness from decreasing due to features that become noise being used to estimate the driver's drowsiness, thereby enabling highly accurate drowsiness estimation and preventing over-estimation of the driver's drowsiness.
  • the hardware configuration of the drowsiness estimation device 1a according to the second embodiment is similar to the hardware configuration of the drowsiness estimation device 1 described in the first embodiment with reference to FIGS. 5A and 5B, and therefore is not illustrated.
  • the functions of the sensing unit 11, the first noise factor detection unit 12, the feature calculation unit 13, the drowsiness score calculation unit 14, the drowsiness estimation unit 15, the second noise factor detection unit 16, and the score correction unit 17 are realized by the processing circuit 1001.
  • the drowsiness estimation device 1a includes the processing circuit 1001 for controlling the estimation of the drowsiness of the driver based on the captured image acquired from the imaging device 2, using feature amounts excluding feature amounts calculated due to the driver performing a noise factor behavior as feature amounts for drowsiness estimation, and for controlling the correction of the drowsiness score based on the vehicle information acquired from the vehicle information acquisition device 3.
  • the processing circuit 1001 reads out and executes a program stored in the memory 1005, thereby executing the functions of the sensing unit 11, the first noise factor detection unit 12, the feature amount calculation unit 13, the drowsiness score calculation unit 14, the drowsiness estimation unit 15, the second noise factor detection unit 16, and the score correction unit 17.
  • the drowsiness estimation device 1a includes a memory 1005 for storing a program that, when executed by the processing circuit 1001, results in the execution of steps ST10 to ST60 in Fig. 7 described above. It can also be said that the program stored in the memory 1005 causes a computer to execute the procedures or methods of the sensing unit 11, the first noise factor detection unit 12, the feature amount calculation unit 13, the drowsiness score calculation unit 14, the drowsiness estimation unit 15, the second noise factor detection unit 16, and the score correction unit 17.
  • the drowsiness estimation device 1a includes devices such as an imaging device 2 and a vehicle information acquisition device 3, as well as an input interface device 1002 and an output interface device 1003 that perform wired or wireless communication.
  • the drowsiness estimation device 1a is an on-board device mounted in a vehicle, and the sensing unit 11, the first noise factor detection unit 12, the feature calculation unit 13, the drowsiness score calculation unit 14, the drowsiness estimation unit 15, the second noise factor detection unit 16, and the score correction unit 17 are provided in the drowsiness estimation device 1a.
  • the sensing unit 11, the first noise factor detection unit 12, the feature calculation unit 13, the drowsiness score calculation unit 14, the drowsiness estimation unit 15, the second noise factor detection unit 16, and the score correction unit 17 may be mounted on the vehicle's in-vehicle device, and the others may be provided on a server connected to the in-vehicle device via a network, and the drowsiness estimation system may be configured with the in-vehicle device and the server.
  • the sensing unit 11, the first noise factor detection unit 12, the feature calculation unit 13, the drowsiness score calculation unit 14, the drowsiness estimation unit 15, the second noise factor detection unit 16, and the score correction unit 17 may all be provided on the server.
  • the subject is a vehicle driver as an example, but this is merely one example.
  • the subject may be a vehicle occupant other than the driver.
  • the subject may also be an occupant, including the driver, of a moving body other than a vehicle, such as a bus, train, or airplane.
  • the drowsiness estimation device 1a according to embodiment 2 can be applied as a drowsiness estimation device that estimates the drowsiness of an occupant of a moving body other than a vehicle.
  • the drowsiness estimation device 1a is configured to include a sensing unit 11 that acquires drowsiness-related information indicating a state related to the occupant's drowsiness for each frame based on frames of an image capturing the face of an occupant of a moving body; a first noise factor detection unit 12 that detects noise factor behavior by the occupant, which is behavior involving eye movement similar to behavior caused by drowsiness, based on the drowsiness-related information acquired by the sensing unit 11; a feature calculation unit 13 that calculates drowsiness estimation feature amounts for estimating the occupant's drowsiness based on drowsiness-related information after noise factor removal is removed from the drowsiness-related information acquired by the sensing unit 11, which is the basis for the drowsiness-related information that the first noise factor detection unit 12 detected the noise factor behavior; a drowsiness score calculation unit 14 that calculates a drowsiness score using the d
  • the drowsiness estimation device 1a can prevent a decrease in the accuracy of estimating the drowsiness of the occupant due to the occupant's behavior exhibiting characteristics similar to those observed when drowsiness occurs.
  • the drowsiness estimation device 1a includes a second noise factor detection unit 16 that detects a noise factor driving state, which is a driving state of the mobile body that is assumed to be unlikely to cause drowsiness in the occupant, based on mobile body information related to the mobile body, and a score correction unit 17 that corrects the drowsiness score calculated by the drowsiness score calculation unit 14 when the second noise factor detection unit 16 detects a noise factor driving state, and the drowsiness estimation unit 15 is configured to estimate the drowsiness of the occupant based on the corrected drowsiness score after correction by the score correction unit 17 when the score correction unit 17 corrects the drowsiness score calculated by the drowsiness score calculation unit 14. Therefore, the drowsiness estimation device 1a can prevent overestimation of the occupant's drowsiness.
  • a noise factor driving state which is a driving state of the mobile body that is assumed to be unlikely to cause drowsiness in the occupant, based on
  • the drowsiness estimation device detects a noise-causing driving state based on vehicle information, and when it detects that a noise-causing driving state exists, it corrects the drowsiness score to prevent erroneous estimation (overestimation) of drowsiness.
  • a malfunction of the sensing unit may also be considered as an event that may lead to an erroneous estimation of drowsiness. For example, in a situation where a driver of a vehicle has normal eye behavior and drives with his/her eyes open, the driver may be erroneously detected as having his/her eyes closed due to an error in sensing processing by the sensing unit, and erroneous drowsiness-related information may be acquired.
  • the feature amount calculation unit calculates a drowsiness estimation feature amount based on the erroneous drowsiness-related information, for example, a drowsiness estimation feature amount that may estimate "drowsiness present" may be calculated. As a result, the drowsiness estimation unit may overestimate the driver's drowsiness.
  • the drowsiness score is corrected in consideration of the possibility of an error in the sensing process by such a sensing unit.
  • the subject is also assumed to be a driver of a vehicle.
  • FIG. 8 is a diagram showing an example of the configuration of a drowsiness estimation device 1b according to the third embodiment.
  • the same components as those of the drowsiness estimation device 1 described in the first embodiment with reference to FIG. 1 are denoted by the same reference numerals, and duplicated descriptions will be omitted.
  • the drowsiness estimation device 1 b according to the third embodiment differs from the drowsiness estimation device 1 according to the first embodiment in that a third noise factor detection unit 18 and a score correction unit 17 are provided.
  • the third noise factor detection unit 18 detects the occurrence of an event (hereinafter referred to as ⁇ noise factor sensing'') in which it is presumed that the sensing unit 11 has erroneously acquired drowsiness-related information due to an erroneous detection of a state related to occupant drowsiness, based on the drowsiness-related information acquired by the sensing unit 11.
  • the sensing unit 11 outputs the sensing result to the first noise factor detection unit 12 and the third noise factor detection unit 18 .
  • the process of detecting noise factor sensing performed by the third noise factor detection unit 18 is referred to as a "third noise factor detection process.”
  • noise factor sensing is erroneous acquisition of drowsiness-related information caused by the sensing unit 11 erroneously detecting that the driver has his/her eyes closed when the eyes are actually open, in other words, that the degree of eyelid opening is low. If such erroneous acquisition of drowsiness-related information occurs, it may lead to an erroneous estimation of the driver's drowsiness when the drowsiness estimation unit 15 estimates the driver's drowsiness based on the drowsiness-related information.
  • the third noise factor detection unit 18 detects whether or not the noise factor sensing is occurring depending on whether or not drowsiness-related information that can be determined as an extremely long period of continuous eye closure has been acquired, for example, based on the time-series drowsiness-related information acquired by the sensing unit 11.
  • the third noise factor detection unit 18 detects that the noise factor sensing is occurring when drowsiness-related information that can be determined as an extremely long period of continuous eye closure has been acquired.
  • the third noise factor detection unit 18 detects whether or not the noise factor sensing is occurring depending on whether or not drowsiness-related information that can be determined to indicate an extremely large number of blinks has been acquired, for example, based on the time-series drowsiness-related information acquired by the sensing unit 11.
  • the third noise factor detection unit 18 detects that the noise factor sensing is occurring when drowsiness-related information that can be determined to indicate an extremely large number of blinks has been acquired.
  • the third noise factor detection unit 18 detects the driver's blinks by a known blink detection method based on the drowsiness-related information in time series.
  • the third noise factor detection unit 18 detects blinks a preset number of times (hereinafter referred to as the "blink count determination threshold") or more in a preset period (hereinafter referred to as the "blink determination period”), the third noise factor detection unit 18 detects that drowsiness-related information that can be determined as extremely many blinks has been acquired, that is, that noise factor sensing has occurred.
  • the blink determination period and the blink count determination threshold are determined in advance by an administrator or the like, and are stored in a location that the third noise factor detection unit 18 can refer to.
  • the above example is merely one example, and the conditions under which the third noise factor detection unit 18 detects that noise factor sensing is occurring based on the drowsiness-related information acquired by the sensing unit 11 are appropriately determined in advance by an administrator, etc.
  • the third noise factor detection unit 18 When the third noise factor detection unit 18 detects that noise factor sensing is occurring, it outputs information regarding the detected noise factor sensing (hereinafter referred to as ⁇ noise factor sensing information'') to the score correction unit 17 together with the sensing results.
  • the noise factor sensing information includes information that the third noise factor detection unit 18 has detected that noise factor sensing is occurring.
  • the noise factor sensing information may further include information that can identify the type of noise factor sensing (extremely continuous eye closure, extremely frequent blinking, etc.) that the third noise factor detection unit 18 has detected as occurring.
  • the score correction unit 17 corrects the drowsiness score calculated by the drowsiness score calculation unit 14 based on the noise factor sensing.
  • the drowsiness score calculation unit 14 outputs the calculated drowsiness score of the driver to the score correction unit 17 .
  • the score correction unit 17 corrects the drowsiness score in accordance with a predetermined score correction rule.
  • the process of correcting the drowsiness score performed by the score correction unit 17 is referred to as a "drowsiness score correction process.”
  • the score correction rule is set to "multiply the drowsiness score by 0.5 when the occurrence of noise factor sensing is detected.”
  • the drowsiness score calculated by the drowsiness score calculation unit 14 is "90.”
  • the third noise factor detection unit 18 detects, based on the drowsiness-related information acquired from the sensing unit 11, that a state in which the eyelid opening degree is equal to or less than the eyelid opening degree for determining continuous eye closure has continued for a period for determining continuous eye closure (for example, 10 seconds), and therefore that noise factor sensing has occurred, which is the acquisition of drowsiness-related information that may be regarded as extremely long continuous eye closure.
  • the drowsiness score calculation unit 14 calculated a high drowsiness score of "90" despite this abnormal state suggests that the abnormal state of continuous eye closure is regarded as a state in which drowsiness occurs, and that the feature calculation unit 13 may have calculated feature amounts that could result in a high drowsiness score. In other words, the drowsiness score calculated by the drowsiness score calculation unit 14 may be incorrect. In this case, the score correction unit 17 corrects the drowsiness score to "45".
  • the drowsiness estimation unit 15 estimates that the driver is "drowsy" when the drowsiness score exceeds "50.” In this case, if the drowsiness score is "90,” the drowsiness estimation unit 15 estimates that the driver is “drowsy.” Then, for example, an alarm device (not shown) outputs an alarm based on the drowsiness estimation result of "drowsiness" by the drowsiness estimation unit 15. In the above-described example, if the score correction unit 17 does not correct the drowsiness score, a warning about drowsiness would be output to the driver even though it is assumed that the driver is not feeling drowsy.
  • the above-mentioned specific example is merely one example.
  • the score correction rule may be set to a rule different from the example given above.
  • a score correction rule may be set so that the degree of correction of the drowsiness score varies depending on the type of noise factor sensing.
  • the score correction rule may be set to: "When noise factor sensing is detected, which is an erroneous acquisition of drowsiness-related information that may be an extremely long continuous eye closure, the drowsiness score is multiplied by 0.5; when noise factor sensing is detected, which is an erroneous acquisition of drowsiness-related information that may be an extremely large number of blinks, the drowsiness score is multiplied by 0.7.”
  • the score correction unit 17 outputs the corrected drowsiness score to the drowsiness estimation unit 15 .
  • the drowsiness estimation unit 15 estimates the drowsiness of the driver based on the corrected drowsiness score.
  • FIG. 9 is a flowchart for explaining the operation of the drowsiness estimation device 1b according to the third embodiment.
  • the drowsiness estimation device 1b repeats the operation shown in the flowchart of FIG. 9 until the vehicle power is turned off.
  • the specific operations in the processing of steps ST10 to ST40 and step ST50 in Figure 9 are similar to the specific operations in the processing of steps ST10 to ST40 and step ST50 in Figure 2, which have already been explained in embodiment 1, so the same step numbers are used and duplicate explanations are omitted.
  • the third noise factor detection unit 18 acquires drowsiness-related information from the sensing unit 11, and performs a third noise factor detection process to detect the occurrence of noise factor sensing based on the acquired drowsiness-related information (step ST70).
  • the third noise factor detection unit 18 detects the occurrence of noise factor sensing, it outputs the noise factor sensing information to the score correction unit 17 together with the drowsiness-related information from the sensing unit 11 .
  • the third noise factor detection unit 18 may output information to the score correction unit 17 to the effect that it did not detect the occurrence of noise factor sensing, or it may not output anything to the score correction unit 17.
  • the score correction unit 17 When the third noise factor detection unit 18 detects noise factor sensing in step ST70, the score correction unit 17 performs a drowsiness score correction process to correct the drowsiness score calculated by the drowsiness score calculation unit 14 based on the noise factor sensing (step ST45). The score correction unit 17 outputs the corrected drowsiness score to the drowsiness estimation unit 15 .
  • the processes of steps ST10 to ST40 and the process of step ST70 are performed in parallel, but this is merely an example.
  • the drowsiness estimation device 1b may execute the process of step ST70 after the processes of steps ST10 to ST40. It is sufficient that the processes of steps ST10 to ST40 and step ST70 have been performed before the process of step ST45 is performed.
  • the drowsiness estimation device 1b acquires drowsiness-related information for each frame based on frames of captured images of the driver's face, and detects noise-causing behavior by the driver that is similar to behavior caused by drowsiness based on the acquired drowsiness-related information.
  • the drowsiness estimation device 1b calculates drowsiness estimation feature quantities based on drowsiness-related information after noise factor removal, which is obtained by excluding the drowsiness-related information that was the basis for detecting the noise-causing behavior from the acquired drowsiness-related information.
  • the drowsiness estimation device 1b calculates a drowsiness score using the drowsiness estimation feature quantities, and estimates the driver's drowsiness based on the calculated drowsiness score. Furthermore, when the drowsiness estimation device 1b detects the occurrence of noise factor sensing based on the drowsiness-related information, the drowsiness estimation device 1b corrects the drowsiness score. When the drowsiness estimation device 1b corrects the drowsiness score, the drowsiness estimation device 1b estimates the driver's drowsiness based on the corrected drowsiness score.
  • the drowsiness estimation device 1b can prevent the accuracy of the estimation of the driver's drowsiness from decreasing due to the driver engaging in noise-causing behavior, and more specifically, when estimating the driver's drowsiness, can prevent the accuracy of the estimation of the driver's drowsiness from decreasing due to features that become noise being used to estimate the driver's drowsiness, thereby enabling highly accurate drowsiness estimation and preventing over-estimation of the driver's drowsiness.
  • the hardware configuration of the drowsiness estimation device 1b according to the third embodiment is similar to the hardware configuration of the drowsiness estimation device 1 described in the first embodiment with reference to FIGS. 5A and 5B, and therefore is not illustrated.
  • the functions of the sensing unit 11, the first noise factor detection unit 12, the feature calculation unit 13, the drowsiness score calculation unit 14, the drowsiness estimation unit 15, the score correction unit 17, and the third noise factor detection unit 18 are realized by the processing circuit 1001.
  • the drowsiness estimation device 1b includes the processing circuit 1001 for controlling the estimation of the drowsiness of the driver based on the captured image acquired from the imaging device 2, using the feature amount excluding the feature amount calculated due to the driver performing a noise factor behavior as the drowsiness estimation feature amount, and for controlling the correction of the drowsiness score based on the drowsiness-related information acquired based on the captured image.
  • the processing circuit 1001 reads out and executes the programs stored in the memory 1005, thereby executing the functions of the sensing unit 11, the first noise factor detection unit 12, the feature amount calculation unit 13, the drowsiness score calculation unit 14, the drowsiness estimation unit 15, the score correction unit 17, and the third noise factor detection unit 18.
  • the drowsiness estimation device 1b includes a memory 1005 for storing a program that, when executed by the processing circuit 1001, results in the execution of steps ST10 to ST50 and step ST70 in Fig. 7 described above. It can also be said that the program stored in the memory 1005 causes a computer to execute the procedures or methods of the sensing unit 11, the first noise factor detection unit 12, the feature amount calculation unit 13, the drowsiness score calculation unit 14, the drowsiness estimation unit 15, the score correction unit 17, and the third noise factor detection unit 18.
  • the drowsiness estimation device 1b includes devices such as an imaging device 2, an input interface device 1002 that performs wired or wireless communication, and an output interface device 1003.
  • the drowsiness estimation device 1b is an on-board device mounted in a vehicle, and the sensing unit 11, the first noise factor detection unit 12, the feature calculation unit 13, the drowsiness score calculation unit 14, the drowsiness estimation unit 15, the score correction unit 17, and the third noise factor detection unit 18 are provided in the drowsiness estimation device 1b.
  • the sensing unit 11, the first noise factor detection unit 12, the feature calculation unit 13, the drowsiness score calculation unit 14, the drowsiness estimation unit 15, the score correction unit 17, and the third noise factor detection unit 18 may be mounted on the vehicle's in-vehicle device, and the others may be provided on a server connected to the in-vehicle device via a network, and the drowsiness estimation system may be configured with the in-vehicle device and the server.
  • the sensing unit 11, the first noise factor detection unit 12, the feature calculation unit 13, the drowsiness score calculation unit 14, the drowsiness estimation unit 15, the score correction unit 17, and the third noise factor detection unit 18 may all be provided on the server.
  • the subject is a vehicle driver as an example, but this is merely an example.
  • the subject may be a vehicle occupant other than the driver.
  • the subject may also be an occupant, including the driver, of a moving body other than a vehicle, such as a bus, train, or airplane.
  • the drowsiness estimation device 1b according to embodiment 2 can be applied as a drowsiness estimation device that estimates the drowsiness of an occupant of a moving body other than a vehicle.
  • the drowsiness estimation device 1b is configured to include a sensing unit 11 that acquires drowsiness-related information indicating a state related to the occupant's drowsiness for each frame based on frames of an image capturing the face of an occupant of a moving body; a first noise factor detection unit 12 that detects noise factor behavior by the occupant, which is behavior involving eye movement similar to behavior caused by drowsiness, based on the drowsiness-related information acquired by the sensing unit 11; a feature calculation unit 13 that calculates drowsiness estimation feature amounts for estimating the occupant's drowsiness based on drowsiness-related information after noise factor removal is removed from the drowsiness-related information acquired by the sensing unit 11, which is the basis for the drowsiness-related information that the first noise factor detection unit 12 detected the noise factor behavior; a drowsiness score calculation unit 14 that calculates a drowsiness score using the d
  • the drowsiness estimation device 1b can prevent a decrease in the accuracy of estimating the drowsiness of the occupant due to the occupant's behavior exhibiting characteristics similar to those observed when drowsiness occurs.
  • the drowsiness estimation device 1b includes a third noise factor detection unit 18 that detects the occurrence of noise factor sensing, which is an event in which the sensing unit 11 is estimated to have erroneously acquired drowsiness-related information due to an erroneous detection of a state related to the drowsiness of the occupant based on the drowsiness-related information acquired by the sensing unit 11, and a score correction unit 17 that corrects the drowsiness score calculated by the drowsiness score calculation unit 14 when the third noise factor detection unit 18 detects the occurrence of noise factor sensing, and the drowsiness estimation unit 15 is configured to estimate the drowsiness of the occupant based on the corrected drowsiness score after correction by the score correction unit 17 when the score correction unit 17 corrects the drowsiness score calculated by the drowsiness score calculation unit 14. Therefore, the drowsiness estimation device 1b can prevent overestimation of the occupant's drowsiness.
  • the configuration of the drowsiness estimation device may be a combination of the configurations of the drowsiness estimation device 1 according to the above-mentioned embodiment 1, the drowsiness estimation device 1a according to the embodiment 2, and the drowsiness estimation device 1b according to the embodiment 3.
  • FIG. 10 is a diagram showing an example configuration of a drowsiness estimation device 1c that combines the configurations of the drowsiness estimation device 1 according to embodiment 1, the drowsiness estimation device 1a according to embodiment 2, and the drowsiness estimation device 1b according to embodiment 3.
  • Figure 11 is a flowchart for explaining the operation of a drowsiness estimation device 1c that combines the configurations of a drowsiness estimation device 1 according to embodiment 1, a drowsiness estimation device 1a according to embodiment 2, and a drowsiness estimation device 1b according to embodiment 3.
  • the drowsiness estimation device 1c includes a sensing unit 11 that acquires drowsiness-related information indicating a state related to drowsiness of an occupant of a moving body (e.g., a driver) for each frame of an image captured of the face of the occupant, and a first noise factor detection unit 12 that detects a noise-causing behavior of the occupant based on the drowsiness-related information acquired by the sensing unit 11.
  • a sensing unit 11 that acquires drowsiness-related information indicating a state related to drowsiness of an occupant of a moving body (e.g., a driver) for each frame of an image captured of the face of the occupant
  • a first noise factor detection unit 12 that detects a noise-causing behavior of the occupant based on the drowsiness-related information acquired by the sensing unit 11.
  • the present invention includes a feature amount calculation unit 13 that calculates a feature amount for estimating drowsiness of an occupant based on drowsiness-related information obtained by removing drowsiness-related information that is the basis for detecting a noise-causing behavior by a first noise factor detection unit 12 from the drowsiness-related information acquired by a sensing unit 11, a drowsiness score calculation unit 14 that calculates a drowsiness score using the feature amount for drowsiness estimation calculated by the feature amount calculation unit 13, a drowsiness estimation unit 15 that estimates the drowsiness of an occupant based on the drowsiness score calculated by the drowsiness score calculation unit 14, and a second noise factor detection unit 16 that detects a noise factor driving state that is assumed to be unlikely to cause drowsiness in an occupant (e.g., a driver) based on moving body information (vehicle information) related to a moving body (e.g., a vehicle).
  • the vehicle is equipped with a third noise factor detection unit 18 that detects the occurrence of noise factor sensing, which is an event in which the sensing unit 11 is presumed to have erroneously acquired drowsiness-related information due to an erroneous detection of a state related to the occupant's drowsiness, based on the drowsiness-related information acquired by the sensing unit 11, and a score correction unit 17 that corrects the drowsiness score calculated by the drowsiness score calculation unit 14 based on the noise factor driving state or the occurrence of noise factor sensing when the second noise factor detection unit 16 detects a noise factor driving state or when the third noise factor detection unit 18 detects the occurrence of noise factor sensing, and the drowsiness estimation unit 15 can be configured to estimate the drowsiness of the occupant based on the corrected drowsiness score after correction by the score correction unit 17.
  • the drowsiness estimation unit 15 can be configured to estimate the drowsiness of the occupant based
  • the drowsiness estimation device 1c can prevent the accuracy of the estimation of the occupant's drowsiness from being reduced due to the occupant engaging in noise-causing behavior, and more specifically, when estimating the occupant's drowsiness, can prevent the accuracy of the estimation of the occupant's drowsiness from being reduced due to features that become noise being used to estimate the occupant's drowsiness, thereby enabling highly accurate drowsiness estimation and preventing over-estimation of the occupant's drowsiness.
  • the score correction rule used by the score correction unit 17 when correcting the drowsiness score is set to a rule that takes into account both the noise factor driving state and the occurrence of noise factor sensing.
  • the score correction rule is set to: "If a noise factor driving state is detected, the drowsiness score is multiplied by 0.5; if the occurrence of noise factor sensing is detected, the drowsiness score is multiplied by 0.5.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Surgery (AREA)
  • Psychiatry (AREA)
  • Psychology (AREA)
  • Social Psychology (AREA)
  • Educational Technology (AREA)
  • Developmental Disabilities (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Biomedical Technology (AREA)
  • Child & Adolescent Psychology (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Hospice & Palliative Care (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Automation & Control Theory (AREA)
  • Mathematical Physics (AREA)
  • Transportation (AREA)
  • Mechanical Engineering (AREA)
  • General Physics & Mathematics (AREA)
  • Traffic Control Systems (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
  • Image Analysis (AREA)
PCT/JP2023/024104 2023-06-29 2023-06-29 眠気推定装置および眠気推定方法 Ceased WO2025004257A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2025529114A JP7843933B2 (ja) 2023-06-29 2023-06-29 眠気推定装置および眠気推定方法
PCT/JP2023/024104 WO2025004257A1 (ja) 2023-06-29 2023-06-29 眠気推定装置および眠気推定方法

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2023/024104 WO2025004257A1 (ja) 2023-06-29 2023-06-29 眠気推定装置および眠気推定方法

Publications (1)

Publication Number Publication Date
WO2025004257A1 true WO2025004257A1 (ja) 2025-01-02

Family

ID=93937975

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/024104 Ceased WO2025004257A1 (ja) 2023-06-29 2023-06-29 眠気推定装置および眠気推定方法

Country Status (2)

Country Link
JP (1) JP7843933B2 (https=)
WO (1) WO2025004257A1 (https=)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008212298A (ja) * 2007-03-01 2008-09-18 Toyota Central R&D Labs Inc 眠気判定装置及びプログラム
JP2011125620A (ja) * 2009-12-21 2011-06-30 Toyota Motor Corp 生体状態検出装置
JP2011229741A (ja) * 2010-04-28 2011-11-17 Toyota Motor Corp 眠気度推定装置および眠気度推定方法
JP2016115118A (ja) * 2014-12-15 2016-06-23 アイシン精機株式会社 下方視判定装置および下方視判定方法
WO2019159229A1 (ja) * 2018-02-13 2019-08-22 三菱電機株式会社 誤検出判定装置及び誤検出判定方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008212298A (ja) * 2007-03-01 2008-09-18 Toyota Central R&D Labs Inc 眠気判定装置及びプログラム
JP2011125620A (ja) * 2009-12-21 2011-06-30 Toyota Motor Corp 生体状態検出装置
JP2011229741A (ja) * 2010-04-28 2011-11-17 Toyota Motor Corp 眠気度推定装置および眠気度推定方法
JP2016115118A (ja) * 2014-12-15 2016-06-23 アイシン精機株式会社 下方視判定装置および下方視判定方法
WO2019159229A1 (ja) * 2018-02-13 2019-08-22 三菱電機株式会社 誤検出判定装置及び誤検出判定方法

Also Published As

Publication number Publication date
JP7843933B2 (ja) 2026-04-10
JPWO2025004257A1 (https=) 2025-01-02

Similar Documents

Publication Publication Date Title
Schmidt et al. Eye blink detection for different driver states in conditionally automated driving and manual driving using EOG and a driver camera
JP6998564B2 (ja) 覚醒度推定装置及び覚醒度推定方法
EP2564777B1 (en) Method for classification of eye closures
JP6219952B2 (ja) ドライバの運転状態を判断する方法
CN110765807B (zh) 驾驶行为分析、处理方法、装置、设备和存储介质
US11430231B2 (en) Emotion estimation device and emotion estimation method
US11453401B2 (en) Closed eye determination device
US20090322506A1 (en) Method and apparatus for driver state detection
JP6754368B2 (ja) 車両のドライバの疲労状態及び／又は睡眠状態を判定するための装置によって実施される方法
JP7550879B2 (ja) 睡眠検出装置及び睡眠検出システム
US20170345276A1 (en) Method and apparatus for recognising microsleep in a driver of a vehicle
JP6662352B2 (ja) 居眠り運転警報装置
JPWO2020255238A1 (ja) 情報処理装置、プログラム及び情報処理方法
US12295731B2 (en) Driver availability detection device and driver availability detection method
JP7192668B2 (ja) 覚醒度判定装置
US20200234161A1 (en) Abnormality detection device and abnormality detection method
JP7046748B2 (ja) 運転者状態判定装置および運転者状態判定方法
US12240469B2 (en) Occupant state detection device and occupant state detection method
WO2023017595A1 (ja) 乗員状態判定装置、乗員状態判定方法、および、乗員状態判定システム
CN120942356A (zh) 一种应用于商用车的疲劳驾驶预警处理系统及方法
JP7843933B2 (ja) 眠気推定装置および眠気推定方法
US11893807B2 (en) Method for determining a level of alertness of a driver
JP7612118B2 (ja) 眠気推定装置および眠気推定方法
JP7704008B2 (ja) 運転者状態判定方法及びその装置
CN107809952A (zh) 用于在使用眼睛张开度的情况下对眨眼事件和仪表察看进行区分的方法和装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23943653

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2025529114

Country of ref document: JP