WO2022124163A1 - Dispositif d'éveil et procédé de commande de stimulus d'éveil - Google Patents

Dispositif d'éveil et procédé de commande de stimulus d'éveil Download PDF

Info

Publication number
WO2022124163A1
WO2022124163A1 PCT/JP2021/044132 JP2021044132W WO2022124163A1 WO 2022124163 A1 WO2022124163 A1 WO 2022124163A1 JP 2021044132 W JP2021044132 W JP 2021044132W WO 2022124163 A1 WO2022124163 A1 WO 2022124163A1
Authority
WO
WIPO (PCT)
Prior art keywords
pattern
driver
stimulus
drowsiness level
awakening
Prior art date
Application number
PCT/JP2021/044132
Other languages
English (en)
Japanese (ja)
Inventor
有華里 伊藤
真紀子 杉浦
彩子 小谷
政治 河合
Original Assignee
株式会社デンソー
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社デンソー filed Critical 株式会社デンソー
Publication of WO2022124163A1 publication Critical patent/WO2022124163A1/fr

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems

Definitions

  • This disclosure relates to an awakening device for awakening a person sitting in the driver's seat and an awakening stimulus control method.
  • Patent Document 1 discloses an awakening device that generates a predetermined pattern of vibration when a driver's drowsiness is detected. Vibration is an awakening stimulus to awaken the driver.
  • the driver's drowsiness level may drop when the awakening device generates an arousal stimulus.
  • the drowsiness level may not be improved even if a predetermined pattern of arousal stimulus is given. Based on such circumstances, it is conceivable to measure the drowsiness level of the driver at regular intervals and give a pattern stimulus according to the measured drowsiness level.
  • the cycle of switching the stimulation pattern is too long, it may cause annoyance. For example, even when the drowsiness level is lowered, the previous pattern is continued until a predetermined time elapses. Further, even if the switching cycle of the stimulation pattern is made too short, the stimulation pattern is frequently changed, which may cause trouble for the driver.
  • the present disclosure has been made based on this circumstance, and an object thereof is to provide an awakening device capable of reducing the risk of causing trouble to the driver.
  • the awakening device for achieving the purpose includes a driver status acquisition unit that acquires information indicating the driver status from at least one of a sensor mounted on the vehicle and a sensor mounted on the driver's body.
  • a stimulus control unit that generates an arousal stimulus with a stimulus pattern corresponding to the drowsiness level determined by the drowsiness level determination unit is provided, and the stimulus control unit sequentially switches and executes a plurality of types of stimulus patterns. It is configured to adjust the timing of switching the stimulus pattern from the current pattern to another stimulus pattern based on the state of the driver after switching to the current pattern, which is the currently executing stimulus pattern. ing.
  • the timing for changing the stimulus pattern is changed based on the state of the driver after the stimulus pattern currently being executed is started. According to such a configuration, it is possible to accelerate or delay the timing of strengthening or weakening the arousal stimulus or changing the type of the arousal stimulus according to the state of the driver. As a result, it is possible to reduce the risk of causing trouble to the driver.
  • the awakening stimulus control method for achieving the above object is to acquire information indicating the driver's state from at least one of a sensor mounted on the vehicle and a sensor mounted on the driver's body. , Judgment among a plurality of pre-prepared stimulus patterns in which at least one of the characteristics and intensity of the arousal stimulus to be generated is different from the determination of the drowsiness level of the driver based on the acquired information indicating the state of the driver.
  • Awakening stimulus is generated by a stimulus pattern according to the drowsiness level, multiple types of stimulus patterns are sequentially switched and executed, and the state of the driver after switching to the current pattern, which is the stimulus pattern currently being executed. Includes changing the timing of switching the stimulus pattern from the current pattern to another stimulus pattern based on.
  • the above method corresponds to the control method of the arousal stimulus carried out by the above-mentioned awakening device. That is, according to the above method, the same effect is obtained by the same action as the above-mentioned awakening device.
  • FIG. 1 is a diagram showing an example of a schematic configuration of the awakening device 1 according to the present disclosure.
  • the awakening device 1 of the present disclosure is a device for guiding a driver to an awakening state, and is mounted on, for example, a vehicle Hv and used.
  • the driver in the present disclosure refers to, for example, a person sitting in the driver's seat.
  • the expression "driver" is not limited to a person who actually performs a part or all of the driving operation.
  • the description of a driver refers to a person who should receive the authority of driving operation from the automatic driving system during automatic driving.
  • the vehicle Hv may be a remote-controlled vehicle that is remotely controlled by an operator existing outside the vehicle.
  • the operator here refers to a person who has the authority to control the vehicle by remote control from the outside of the vehicle.
  • the awakening device 1 of the present disclosure is used as a part of a cockpit system that accepts an operation of the operator.
  • the level indicated by "automatic driving” in the present disclosure may be, for example, equivalent to level 3 defined by the American Society of Automotive Engineers of Japan (SAE International), or may be level 4 or higher.
  • Level 3 refers to the level at which the system executes all operation tasks within the operational design domain (ODD), while the operation authority is transferred from the system to the user in an emergency.
  • ODD operational design domain
  • the ODD defines conditions under which automatic driving can be executed, such as the traveling position being in a highway.
  • the user is required to be able to respond promptly when there is a request for a change of operation from the system.
  • the person who takes over the driving operation may be an operator.
  • Level 3 corresponds to so-called conditional automated driving.
  • Level 4 is a level at which the system can perform all driving tasks except under specific circumstances such as unresponsive roads and extreme environments.
  • Level 5 is the level at which the system can perform all driving tasks in any environment.
  • Level 4 or higher automated driving refers to the level at which the automated driving device performs all driving tasks, that is, the automated level at which the driver's seat occupants are allowed to sleep.
  • the awakening device 1 is used by being connected to various in-vehicle devices such as a driver status monitor (hereinafter, DSM: Driver Status Monitor) 21.
  • DSM Driver Status Monitor
  • the awakening device 1 is connected to a DSM 21, an input device 22, a display 23, a light emitting device 31, an aroma shooter 32, a dialogue device 33, an air conditioner 34, and a vibration generator 35.
  • the awakening device 1 can be used by being directly or indirectly connected to a head-up display (HUD) 36, a speaker 37, a window motor 38, and the like.
  • HUD head-up display
  • the awakening device 1 is also connected to various sensors / devices (not shown in FIG. 1) via the in-vehicle network Nw, which is a communication network constructed in the vehicle.
  • the awakening device 1 is input with the detection results of various in-vehicle sensors via the in-vehicle network Nw.
  • the in-vehicle sensor include sensors that detect vehicle speed, acceleration, steering angle, shift position, accelerator depression amount, brake depression amount, and the like.
  • In-vehicle sensors also include sensors / switches that detect the operating state of the parking brake and the power state of the vehicle Hv.
  • the awakening device 1 and the in-vehicle device may be connected by a dedicated line or may be connected via the in-vehicle network Nw. Further, an ECU (Electronic Control Unit) may be interposed between the awakening device 1 and the in-vehicle device.
  • ECU Electronic Control Unit
  • the DSM21 is a device that sequentially detects the user's state based on the user's face image.
  • the DSM 21 includes, for example, a near-infrared light source, a near-infrared camera, and a control module for controlling them.
  • the DSM 21 is installed in a posture in which the near-infrared camera faces the direction in which the headrest of the driver's seat is present, for example, on the upper surface of the steering column portion, the upper surface of the instrument panel, or the like.
  • the DSM 21 uses a near-infrared camera to photograph the head of the driver irradiated with near-infrared light by a near-infrared light source.
  • the image captured by the near-infrared camera is image-analyzed by the control module.
  • the control module extracts driver state information, which is information indicating the driver's state, such as the opening of the driver's eyes, from the captured image input from the near-infrared camera.
  • the DSM 21 outputs the driver state information extracted from the driver's face image to the awakening device 1.
  • the driver status information includes, for example, the direction of the driver's face, the direction of the line of sight, the opening of the eyes, the opening of the mouth, and the like.
  • the opening of the eyes can be rephrased as the degree of opening of the eyelids.
  • the information indicating the opening degree of the eyes corresponds to the eye opening degree information.
  • the DSM 21 may be configured to detect swelling, inattentiveness, outbreak of illness, etc. based on the time course of the opening degree of the eyes, the facial expression, the orientation of the face, and the like.
  • the driver status information can include sleepiness, inattentiveness, poor physical condition, and the like.
  • the DSM21 can detect yawning based on the pattern of mouth opening over time.
  • the awakening device 1 may have a function of detecting the state of the driver based on the image analysis.
  • the DSM 21 may be configured to be able to output the driver's face image to the awakening device 1.
  • the functional arrangement between the DSM 21 and the awakening device 1 can be changed as appropriate.
  • the input device 22 is an operating member for receiving a driver's instruction to the awakening device 1.
  • the input device 22 may be a mechanical switch (so-called steer switch) provided on the spoke portion of the steering wheel, or may be a voice input device that recognizes the utterance content of the driver.
  • the input device 22 may be a touch panel laminated on the display panel of the display 23 provided on the instrument panel.
  • the display 23 for example, a center display provided in the central region of the instrument panel in the vehicle width direction can be adopted.
  • the display 23 may be a meter display.
  • the input device 22 may be a driver's smartphone.
  • the touch panel and display of the smartphone possessed by the driver can be used as the input device 22 and the display 23.
  • the light emitting device 31 is a device that generates light that can be visually recognized by the driver.
  • the light emitting device 31 includes, for example, an LED as a light source.
  • the light emitting device 31 is provided at a position within the driver's field of view, such as a spoke portion of the steering wheel, an upper surface portion of the instrument panel, and a ceiling portion in front of the driver's seat.
  • the light generated by the light emitting device 31 corresponds to an arousal stimulus for vision.
  • the awakening stimulus is a stimulus that the driver can feel with all five senses, and is a stimulus that awakens the driver.
  • the light emitting device 31 can change the brightness, the color of light, and the like.
  • the light emitting device 31 can periodically change the brightness. When the brightness changes periodically, the light emitting device 31 can adjust the length of the cycle, the waveform, the amount of change in the brightness per unit time, and the like.
  • the aroma shooter 32 injects the fragrance into the passenger compartment of the vehicle Hv.
  • the driver perceives the scent of the sprayed fragrance.
  • the scent generated by the aroma shooter 32 corresponds to the arousal stimulus for the sense of smell.
  • the aroma shooter 32 can change the type of scent, the intensity of the scent, and the like.
  • the aroma shooter 32 may be integrated with the air conditioner 34.
  • the dialogue device 33 interacts with the driver. That is, the dialogue device 33 recognizes the voice produced by the driver, creates an answer to the driver's voice, and pronounces the answer.
  • the dialogue device 33 for example, uses artificial intelligence to perform a dialogue.
  • the dialogue provided by the dialogue device 33 corresponds to the arousal stimulus corresponding to the auditory sense.
  • the air conditioner 34 generates cold air as air conditioning air in the vehicle interior of the vehicle Hv.
  • Cold wind corresponds to arousal stimuli for cold and tactile sensations.
  • the air conditioner 34 can change the temperature, air volume, and mode of the cold air.
  • the auto mode is a mode in which the blowing direction of cold air is changed by a preset algorithm.
  • the driver face mode is a mode in which cold air is always blown toward the driver's face.
  • the vibration generator 35 generates vibration based on the control signal from the awakening device 1.
  • the vibration generator 35 is embedded in the seating surface and the backrest portion of the driver seat. Vibration corresponds to arousal stimuli.
  • the HUD 36 is a device that projects a virtual image that can be perceived by the user by projecting image light onto a predetermined area of the windshield based on control signals and video data input from the awakening device 1 and the navigation device.
  • the HUD 36 displays an image superimposed on the landscape in front of the vehicle Hv.
  • Such a HUD 36 can generate a light stimulus by displaying a predetermined image.
  • the light stimulus corresponds to the arousal stimulus for vision.
  • the device that outputs the optical stimulus based on the image does not have to be HUD36.
  • An image-based optical stimulus may be generated from a meter display or a center display.
  • the HUD 36 can play a role as an information presenting device that presents various information such as the operating state of the awakening device 1, the break proposal, and the state of the driver recognized by the system.
  • Information on the operating state of the awakening device 1 includes, for example, activation information and continuation information of the awakening device 1.
  • the activation information is information for notifying the driver that the awakening stimulus will be activated from now on, and is expressed by, for example, a text or an icon image.
  • the activation information is displayed before the arousal stimulus occurs.
  • the continuation information is displayed when the arousal stimulus is ongoing.
  • the continuation information informs the driver that the arousal stimulus is continuing.
  • the continuation information may include an icon image or text outlining the stimulus currently being output.
  • the outline of the stimulus includes the types of stimuli such as light, sound, vibration, odor, and air conditioning.
  • the continuation information may include advance notice information indicating the type and intensity of the stimulus output in the next term.
  • the break proposal is a display of the content that proposes to the driver to take a break.
  • the break proposal is displayed on any one or more of a plurality of display devices such as the HUD 36 and the display 23.
  • the HUD 36 displays, for example, an icon image imitating a coffee cup or an icon image for urging a nap when parked, as a break proposal.
  • the driver state is, for example, information indicating the drowsiness level of the driver recognized by the awakening device 1 described later.
  • the HUD 36 displays an icon image or text indicating the drowsiness level under the control of the awakening device 1.
  • the driver state may include the degree of fatigue and the degree of indiscriminateness.
  • the presentation of various information is not limited to the HUD 36, and may be performed using a meter display, a center information display, or the like.
  • the speaker 37 generates sound in the vehicle interior of the vehicle Hv. Speech corresponds to arousal stimuli for hearing. There are two types of voice: music voice and alarm voice. The expression with voice includes mere sound.
  • the window motor 38 is a motor for changing the opening degree of the window glass (so-called door window) of the door.
  • the window motor 38 can be, for example, a motor for opening and closing a door window for a driver's seat.
  • the window motor 38 opens and closes the driver's seat door window based on the control signal input from the awakening device 1.
  • another ECU such as a body ECU may be interposed between the awakening device 1 and the window motor 38.
  • a light emitting device 31 an aroma shooter 32, a dialogue device 33, an air conditioner 34, a vibration generator 35, a HUD 36, a speaker 37, and a window motor 38.
  • a stimulus generator Is also referred to as a stimulus generator.
  • the in-vehicle device that communicates with the awakening device 1 is not limited to the above.
  • a biosensor that senses the biometric information of the driver may be connected to the awakening device 1.
  • the biosensor is, for example, a heart rate sensor that measures a heart rate.
  • Biological sensors also include sensors that detect blood pressure, electrocardiogram, pulse wave, sweating amount, body temperature, heat dissipation from the human body, respiratory rhythm, and respiratory depth.
  • the biosensor may be built in the driver's seat or may be provided in the steering.
  • millimeter wave radars that detect the driver's heart rate, body movement, and posture can also be included in the biosensor.
  • the biosensor may be a wearable device 40 that is used by being attached to, for example, a wrist of a driver.
  • various shapes such as a wristband type, a wristwatch type, a ring type, a glasses type, and an earphone type can be adopted.
  • the wearable device 40 is configured to be able to communicate with the awakening device 1 via a communication device 39 mounted on the vehicle Hv.
  • the connection mode between the wearable device 40 and the communication device 39 may be a wired connection or a wireless connection.
  • the communication device 39 and the wearable device 40 are configured to carry out wireless communication conforming to short-range wireless communication standards such as Bluetooth (registered trademark) and Wi-Fi (registered trademark).
  • a smartphone paired with each of the communication device 39 and the wearable device 40 may be interposed between the communication device 39 and the wearable device 40.
  • the awakening device 1 is a device that operates various stimulus generators based on an input signal from the DSM 21 in order to maintain the awake state of the driver or guide the driver to the awake state.
  • the awakening device 1 is configured as a computer including a processor 11, a RAM (Random Access Memory) 12, a storage 13, a communication interface 14 (I / O in the figure), a bus line connecting these configurations, and the like.
  • the processor 11 is an arithmetic core such as a CPU (Central Processing Unit).
  • the processor 11 executes various processes by accessing the RAM 12.
  • the RAM 12 is a volatile memory.
  • the storage 13 has a configuration including a non-volatile storage medium such as a flash memory.
  • the storage 13 stores an awakening stimulus control program, which is a program for making the computer function as the awakening device 1. Executing the awakening stimulus control program by the processor 11 corresponds to executing the awakening stimulus control method corresponding to the awakening stimulus control program.
  • the communication interface 14 is a circuit for the awakening device 1 to communicate with another device.
  • the communication interface 14 may be realized by using an analog circuit element, an IC, or the like.
  • the awakening device 1 provides each functional unit shown in FIG. 2 by the processor 11 executing the awakening stimulus control program stored in the storage 13. That is, the awakening device 1 includes a sensor information acquisition unit F1, a drowsiness level determination unit F2, a stimulus control unit F3, a display processing unit F4, an end determination unit F5, an operation reception unit F6, and a pattern management unit F7.
  • the stimulus control unit F3 includes a change determination unit F31 and a pattern selection unit F32 as finer functional units.
  • the sensor information acquisition unit F1 is configured to acquire information (that is, determination material) for determining the drowsiness level of the driver from the DSM21, a wearable device as a biological sensor, and the like.
  • the drowsiness level is a parameter indicating the degree of drowsiness.
  • Information that can be used to determine the drowsiness level includes, for example, the opening of the eyes, the frequency of blinking, the degree of variation in the blinking interval, the direction of the driver's face, the direction of the line of sight, the posture, and the frequency of yawning. There is a degree of wobbling of the handle angle.
  • the sensor information acquisition unit F1 sequentially acquires various information for the drowsiness level determination unit F2 to determine the drowsiness level from a predetermined sensor group including the DSM 21.
  • the material for determining drowsiness level can also be called a driver feature that can be used to evaluate drowsiness level.
  • the sensor information acquisition unit F1 corresponds to the driver status acquisition unit.
  • the sensor information acquisition unit F1 can also acquire the state (on / off) of the driving power source, vehicle speed, acceleration, steering angle, accelerator depression amount, brake depression amount, and the like.
  • the traveling power source is a power source for traveling the vehicle Hv, and refers to an ignition power source when the vehicle Hv is a gasoline-powered vehicle.
  • the driving power source refers to the system main relay.
  • the drowsiness level determination unit F2 determines the drowsiness level of the driver based on various information acquired by the sensor information acquisition unit F1.
  • a case where the drowsiness level is determined by dividing it into 6 stages from 0 to 5 will be described as an example.
  • level 1 corresponds to a state in which the driver is not sleepy at all
  • level 1 corresponds to a state in which the driver is slightly sleepy, in other words, the driver himself / herself is not aware of drowsiness.
  • Level 2 is a state in which the driver is a little sleepy and can be aware of drowsiness.
  • Level 3 corresponds to a sleepy state
  • level 4 corresponds to a fairly sleepy state.
  • Level 5 corresponds to almost / completely sleeping.
  • the awakening device 1 shall perform the same control as in the case of drowsiness level 3 in the case of drowsiness level 4 to 5. Therefore, in the following, the description in the case of level 4 to level 5 will be omitted.
  • a control signal for executing MRM is output to the ECU that controls the running of the vehicle Hv. Is also good.
  • the MRM is, for example, a control in which the vehicle is decelerated at a predetermined deceleration while issuing an alarm to the surroundings, and the vehicle is stopped in the traveling lane or near the shoulder of the road.
  • the control signal for executing the MRM may be a signal indicating that the driver is sleeping.
  • the drowsiness level determination unit F2 has the degree of eye opening, the degree of fluctuation of the degree of eye opening over time, the difference in the degree of opening of the left and right eyes, the speed of movement of the line of sight, the cycle and speed of blinking (blinking), the frequency of yawning, and the like. Is used in combination to determine the drowsiness level. In general, the stronger the drowsiness, the smaller the amount of eye opening tends to be. Therefore, the drowsiness level determination unit F2 may determine that the smaller the eye opening amount, the higher the drowsiness level.
  • the drowsiness level determination unit F2 may determine the drowsiness level higher as the moving speed of the line of sight and the speed of blinking are smaller.
  • the blink cycle tends to be unstable. Therefore, the drowsiness level determination unit F2 may evaluate the drowsiness level based on the degree of stability of the blink cycle, for example, the dispersion value of the blink cycle within a certain time.
  • the drowsiness level determination unit F2 determines the amount of heat released from the driver's body, the tendency and distribution of changes in body surface temperature, the presence or absence of deep breathing, the presence or absence of vertical movement of the shoulders, and whether or not the face is facing downward or upward.
  • the drowsiness level may be determined based on the condition. Further, the drowsiness level may be determined higher as the change in the speed of the vehicle Hv and the change in the steering angle are smaller.
  • the awakening device 1 has a timer function for measuring a predetermined time.
  • the awakening device 1 uses the timer function to calculate the elapsed time from the start of each operation pattern, the duration of the closed eye state in which the driver is closed, and the like.
  • the closed eye state here means, for example, a state in which the degree of eye opening is equal to or less than a predetermined threshold value.
  • the closed eye state can include a state in which the eyes are completely closed and a state in which the eyes are slightly open, such as a state in which the eyes are almost closed.
  • the awakening device 1 is provided with operation pattern groups G0 (A), G0 (B), G1 (A), G1 (B), G2, and G3 as shown in FIG. There is.
  • the operation pattern group G0 (A) is an operation pattern group corresponding to a state in which no awakening stimulus is output.
  • the operation pattern group G0 (A) can be set when the drowsiness level is 0. For example, it is applied immediately after the start of operation of the operation pattern group G0 (A). Further, the operation pattern group G0 (A) is applied, for example, when the drowsiness level is maintained at 0 for a predetermined release time after the drowsiness level drops from 1 to 0.
  • the release time can be, for example, 45 seconds or 1 minute.
  • the actuated pattern group G0 (A) can be referred to as a non-actuated group. Further, the state in which the operation pattern group G0 (A) is set can also be called a stimulation stop mode.
  • the operation pattern groups other than the operation pattern group G0 (A), that is, the groups G1 (A), G1 (B), G2, and G3 each include a plurality of operation patterns having different types and combinations of arousal stimuli to be generated. That is, the groups G1 (A), G1 (B), G2, and G3 are operation pattern groups that intermittently or continuously generate some kind of stimulus. Therefore, the groups G1 (A), G1 (B), G2, and G3 can be referred to as a stimulus generation group.
  • the state in which the stimulus generation group is selected can be referred to as a stimulus generation mode. In the stimulus generation group, as shown in FIG. 3, the type of arousal stimulus to be generated is determined.
  • the operation pattern corresponds to the stimulation pattern.
  • Types of arousal stimuli include light emission, scent, dialogue, cold air, vibration, and voice.
  • the light emission is an arousal stimulus of light generated by the light emitting device 31 and the HUD 36.
  • the scent is an arousal stimulus generated by the aroma shooter 32.
  • Dialogue is an arousal stimulus generated by the dialogue device 33.
  • the cold air is, for example, an arousal stimulus generated by the air conditioner 34. Cold air may be given to the driver by opening the driver's door window using the window motor 38.
  • Vibration is an arousal stimulus generated by the vibration generator 35.
  • the voice is an arousal stimulus generated by the speaker 37.
  • the characteristics of the arousal stimulus are determined for each type of arousal stimulus.
  • the light arousal stimuli generated by the light emitting device 31 are classified into light emitting devices 1 to 4.
  • Emissions 1 to 4 have different characteristics from each other.
  • Features include, for example, the color of light, the brightness, the operation pattern of light emission, the place where light is generated, a device that generates light, and the like.
  • the intensity of the stimulus in other words, the effect of awakening the driver, is higher in light emission 2 than in light emission 1, higher in light emission 3 than in light emission 2, and higher in light emission 4 than in light emission 3.
  • the scent awakening stimulus is divided into scents 1 to 4.
  • the scents 1 to 4 have different characteristics from each other.
  • Features include, for example, the type of scent, the intensity of the scent, and the like.
  • the effect of awakening the driver that is, the intensity of the stimulus is higher in scent 2 than in scent 1, higher in scent 3 than in scent 2, and higher in scent 4 than in scent 3.
  • the cold air awakening stimulus consists of three elements: cold air temperature, air volume, and mode.
  • the temperature of the cold air is auto or low temperature. Auto is to set the temperature by a preset algorithm.
  • the air volume is auto or strong. Auto is to set the air volume by a preset algorithm.
  • the mode is the above-mentioned auto mode or the driver face mode. When the temperature, air volume, and mode of the cold air are all automatic, it means that the awakening stimulus of the cold air is not generated.
  • the effect of awakening the driver is higher when the temperature of the cold air is low than when the temperature of the cold air is auto.
  • the effect of awakening the driver is higher when the air volume of the cold air is strong than when the air volume of the cold air is automatic.
  • the effect of awakening the driver is higher in the driver face mode than in the auto mode.
  • a plurality of types of alarms such as alarms 1, 2, and 3 may be prepared according to the loudness and frequency of the sound.
  • the intensity of the arousal stimulus is set to increase in the order of alarms 1, 2, and 3.
  • Group G0 (B) corresponding to the stimulus generation group includes operation patterns P00 and P01.
  • group G0 (B) When the group G0 (B) is applied, the time zone of the operation pattern P00 and the time zone of the operation pattern P01 are alternately repeated. Awakening stimulus does not occur in the time zone of the operation pattern P00.
  • an arousal stimulus associated with the operation pattern P01 in FIG. 3 is generated, for example, light emission 1 and scent 1.
  • Group G0 (B) is applied, for example, when the drowsiness level drops from 1 or 2 to 0, as shown in FIG. It should be noted that generating a certain awakening stimulus corresponds to operating a device capable of outputting the awakening stimulus.
  • the operation pattern group G1 (A) includes operation patterns P10, P11, P12, P13, and P14 in which at least one of the characteristics and the intensity of the arousal stimulus to be generated is different.
  • the time of the operation pattern P14 is sequentially passed from the time zone of the operation pattern P10, the time zone of the operation pattern P11, the time zone of the operation pattern P12, and the time zone of the operation pattern P13. It shifts to the band one by one.
  • the time zone of the operation pattern P14 ends, the time zone of the operation pattern P10 is restored again. That is, while the group G1 (A) is applied, the operation patterns P11 to 14 are repeated in order.
  • the awakening stimulus does not occur in the time zone of the operation pattern P10.
  • the awakening stimulus associated with each operation pattern is generated in FIG.
  • Such group G1 (A) is applied, for example, when the drowsiness level rises from 0 to 1 as shown in FIG.
  • the operation pattern group G1 (B) includes operation patterns P15 and P16.
  • the group G0 (B) is applied, the time zone of the operation pattern P15 and the time zone of the operation pattern P16 are alternately repeated. No arousal stimulus occurs during the time zone of the operation pattern P15.
  • an arousal stimulus associated with the operation pattern P16 in FIG. 3 is generated, for example, an image display on the HUD 36.
  • Such group G1 (B) is applied, for example, when the drowsiness level drops from 2 to 1.
  • the operation pattern group G2 includes operation patterns P20, P21, P22, and P23.
  • the time zone of the operation pattern P20 is sequentially passed through the time zone of the operation pattern P21 and the time zone of the operation pattern P22, and then sequentially shifts to the time zone of the operation pattern P23.
  • the time zone of the operation pattern P23 ends, the time zone of the operation pattern P20 is restored again. No arousal stimulus occurs during the time zone of the operation pattern P20.
  • the awakening stimulus associated with each operation pattern is generated in FIG.
  • Such group G2 is applied, for example, when the drowsiness level is determined to be 2.
  • the operation pattern group G3 includes operation patterns P30, P31, P32, and P33.
  • the time zone of the operation pattern P30, the time zone of the operation pattern P31, and the time zone of the operation pattern P32 are sequentially passed, and then the time zone of the operation pattern P33 is sequentially shifted.
  • the time zone of the operation pattern P33 ends, the time zone of the operation pattern P30 is restored again. No arousal stimulus occurs during the time zone of the operation pattern P30.
  • the awakening stimulus associated with each operation pattern is generated in FIG.
  • Such an operation pattern group G3 is applied, for example, when the drowsiness level is determined to be 3.
  • the group G3 does not automatically transition to another group.
  • the driver's break operation it is possible to adopt that the shift position of the vehicle is set to the parking position, the parking brake is set to on, the driving power of the vehicle Hv is set to off, and the like. can.
  • the standard duration T1 which is the duration of each operation pattern, is, for example, 45 seconds.
  • the standard duration T1 may be, for example, 30 seconds, 50 seconds, 60 seconds, 90 seconds, or the like. If the standard duration T1 is too long, the driver becomes accustomed to the stimulus output in the operation pattern, and it is difficult to obtain the awakening effect. Further, if the standard duration T1 is too short, the output mode of the stimulus is frequently changed, which may cause trouble for the driver. It is preferable that the standard duration T1 is set to 40 seconds or more and 60 seconds or less in order to secure the awakening effect and reduce the annoyance at the same time.
  • the standard duration T1 corresponds to the first hour.
  • the stimulus control unit F3 executes, as a basic operation, changing the operation pattern at a cycle corresponding to the standard duration T1.
  • the operation patterns P00, P10, P15, P20, and P30 located at the head of the stimulus generation operation pattern are all set to rest patterns that do not output awakening stimuli, but the present invention is not limited to this.
  • the head pattern of each group may be set to output some kind of arousal stimulus.
  • the pause duration which is the standard duration T1 of the pause pattern, may be set shorter than the stimulation duration, which is the standard duration T1 of the operation pattern that generates the stimulus.
  • the pause duration can be set to a length such as 15 seconds, 20 seconds, and 30 seconds that the driver can recognize that it is a pause period and that the increase in drowsiness can be suppressed during the pause period.
  • the rest duration is set to about half of the stimulation duration.
  • the stimulus generation control process is a process of controlling the operation of various stimulus generators so as to maintain the driver's awake state or lead the driver to the awake state.
  • the stimulus generation control process includes steps S100 to S113.
  • the stimulus generation control process shown in FIG. 6 is started when a predetermined start event occurs.
  • the start event for example, the ignition of the vehicle Hv is turned on, the vehicle starts running, the start instruction is input by the driver, and the like can be adopted.
  • the remaining time from the automatic driving mode of level 4 or higher to the timing of switching to the manual driving mode is less than a predetermined threshold, which is also a start event. Can be adopted as.
  • the remaining time until the timing of switching to the manual operation mode can be calculated based on the control plan of the remaining distance and the traveling speed until leaving the ODD such as a highway.
  • the stimulus generation control process may be started when the drowsiness level of the driver changes from a state of 0 to a state of 1 or more by the drowsiness level determination unit F2. In that case, steps S100 to S101 can be omitted.
  • the operation reception unit F6 gives an instruction of the driver to the generation mode of the awakening stimulus, as will be described separately.
  • the process of accepting operations is also executed at any time.
  • the flowchart shown in FIG. 6 can be understood as a flowchart showing an outline of the operation of the awakening device 1.
  • step S100 the sensor information acquisition unit F1 acquires sensor values of various state quantities as a material for determining the drowsiness level and moves to S101. It should be noted that the process of sequentially acquiring the drowsiness level determination material by the sensor information acquisition unit F1 is sequentially executed in parallel with the processing flow described later. Since step S100 is a step in which the sensor information acquisition unit F1 acquires information indicating the driver status, it can be called a driver status acquisition step.
  • step S101 the drowsiness level determination unit F2 determines the drowsiness level of the driver based on various information acquired by the sensor information acquisition unit F1. Then, the display processing unit F4 displays the determined drowsiness level on at least one of the HUD 36 and the display 23, and proceeds to step S102.
  • step S101 can be called a drowsiness level determination step.
  • the drowsiness level determination unit F2 sequentially determines the drowsiness level of the driver in a predetermined determination cycle such as 5 seconds, 10 seconds, and 15 seconds even after step S101.
  • the determination result of the drowsiness level determination unit F2 is stored in, for example, the RAM 12 for a certain period of time.
  • the determination results of a plurality of drowsiness levels having different determination times can be sorted and saved in the order of the determination time so that the latest data of the determination time is at the beginning.
  • the storage period of the drowsiness level determination result can be, for example, 2 minutes or 5 minutes.
  • step S102 the pattern selection unit F32 selects an operation pattern group according to the drowsiness level determined in step S101 from the operation pattern groups prepared in advance exemplified in FIGS. 3 and 4. For example, when the drowsiness level determined in step S101 is 0, the pattern selection unit F32 sets the operation pattern group G0 (A). When the drowsiness level determined in step S101 is level 1, the pattern selection unit F32 sets the operation pattern group G1 (A). When the drowsiness level determined in step S101 is level 2, the pattern selection unit F32 sets the operation pattern group G2. When the drowsiness level determined in step S101 is level 3 or higher, the pattern selection unit F32 sets the operation pattern group G3. In addition, when the drowsiness level is level 0, the pattern selection unit F32 selects the operation pattern group G0 (A).
  • Such step S102 can be called a stimulation pattern selection step or an operation pattern selection step.
  • step S104 the stimulus control unit F3 starts the generation of arousal stimuli according to the operation pattern group set in step S103.
  • the operation pattern group G1 (A) is selected, the output of the awakening stimulus corresponding to the operation pattern P10 located at the head of the list of a plurality of operation patterns constituting the operation pattern group G1 (A) is started. do.
  • the operation pattern P10 is used as a pause pattern here, if the operation pattern P10 includes, for example, the generation of light emission 2, a control signal is output to the light emitting device 31 to start generating light emission of a predetermined operation pattern.
  • the arousal stimulus is not output.
  • the generation of the arousal stimulus corresponding to the selected group is caused by the group G0 (A) being selected in the subsequent processing or at the end timing in steps S112 and S113.
  • the stimulus control unit F3 activates a timer and starts measuring the elapsed time from the start of the current pattern, which is the currently executing operation pattern.
  • the display processing unit F4 uses the HUD 36 to present information regarding the operating state of the awakening device 1 or the break proposal. The information type to be presented is determined for each operation pattern as shown in the rightmost column of FIG.
  • step S105 the count value of the timer is referred to, and it is determined whether or not the predetermined intermediate confirmation time T2 has elapsed since the start of the current pattern.
  • the intermediate confirmation time T2 may be a value shorter than the standard duration T1 and is set to, for example, 30 seconds. Of course, the intermediate confirmation time T2 may be 25 seconds, 40 seconds, or the like.
  • the intermediate confirmation time T2 is preferably set longer than half of the standard duration T1.
  • the intermediate confirmation time T2 corresponds to the second time.
  • step S105 If the elapsed time from the start of the current pattern is equal to or longer than the intermediate confirmation time T2, affirmative determination is made in step S105 and the process proceeds to step S106. On the other hand, if the intermediate confirmation time T2 has not yet elapsed since the start of the current pattern, step S105 is repeated while sequentially determining whether or not the end condition is satisfied in step S112.
  • the termination conditions include, for example, that the driving power of the vehicle Hv has been turned off, that the vehicle has arrived at the destination, that the driver has given an termination instruction, and the like. If the end condition is satisfied, this process ends.
  • step S106 it is determined whether or not the driver's state satisfies the predetermined temporary switching condition when the elapsed time reaches the intermediate confirmation time T2 after starting the current pattern, which is the operation pattern currently being executed. do.
  • the elapsed time since the start of the current pattern can be measured by the timer function described above.
  • the time point at which the elapsed time from the start of the current pattern becomes the intermediate confirmation time T2 is also described as the intermediate confirmation time point.
  • the temporary switching condition is a condition for temporarily changing the output mode of the arousal stimulus according to the state of the driver.
  • the group switching condition is a condition for suddenly switching to an operation pattern group corresponding to a higher drowsiness level.
  • the group switching condition can be understood as an intensity changing condition for strengthening the arousal stimulus to be generated from one viewpoint.
  • the group switching condition may be, for example, a case where the total value of the time during which the driver is in the closed eye state in the period from the start of the current pattern to the intermediate confirmation time T2 is equal to or greater than the predetermined emergency switching threshold value. can.
  • the emergency switching threshold value can be 5 seconds, 10 seconds, or the like. Further, the emergency switching threshold value can be set to 15% to 30% of the intermediate confirmation time T2.
  • the emergency switching threshold value may be set to a different value depending on the determination value of the current drowsiness level. For example, when the drowsiness level is 1, the emergency switching threshold value may be 3 seconds, while when the drowsiness level is 2, the emergency switching threshold value may be 5 seconds. It can be said that the group switching condition corresponds to the case where the current pattern does not work for the driver and the drowsiness level deteriorates.
  • the intra-group switching condition is a condition for switching to the next operation pattern in the currently selected group.
  • the intra-group switching condition can be understood as a type change condition for immediately changing the type of awakening stimulus without changing the intensity of the awakening stimulus to be generated.
  • the intra-group switching condition can be said to correspond to the case where the current pattern does not change the drowsiness level, that is, the drowsiness level has not deteriorated but has not been improved.
  • the posture, eye opening degree, eye movement speed, blink (blink) cycle stability, blink speed, etc. at the time of intermediate confirmation are improved compared to the start of the current pattern. If not, it is determined that the intra-group switching condition is satisfied.
  • the case where the degree of eye opening is improved corresponds to the case where the degree of eye opening is increased
  • the case where the posture is improved means the case where the direction of the face approaches from the downward direction or the upward direction to the front direction. This event is a sign of improved drowsiness.
  • step S106 if at least one of the group switching condition and the intra-group switching condition is satisfied, the process proceeds to step S107. On the other hand, if neither the group switching condition nor the intra-group switching condition is satisfied, step S106 is negatively determined and the process proceeds to step S108. If neither the group switching condition nor the intra-group switching condition is satisfied, for example, when the drowsiness level is lowered, or when the drowsiness level is improving but the level value is lowered, a significant change occurs. Refers to the case where it is not.
  • step S107 the process according to the satisfied temporary switching condition is executed.
  • the pattern selection unit F32 selects an operation pattern group one step higher than the currently selected operation pattern group.
  • the operation pattern group one step higher is an operation pattern group whose associated drowsiness level is one step higher.
  • G1 (A) or G1 (B) G2 is selected. If the currently selected operation pattern group is G2, G3 is selected. If the currently selected operation pattern group is G0 (A) or G0 (B), G1 (A) is selected.
  • the pattern selection unit F32 selects the next operation pattern in the currently selected operation pattern group. For example, when the currently selected operation pattern is P11, P12 is selected. If the currently selected operation pattern is P21, P22 is selected. The same applies to other cases.
  • step S107 When the selection process in step S107 is completed, the process returns to step S103, and the output of the arousal stimulus corresponding to the operation pattern group and operation pattern selected in step S107 is started. Then, the processes after step S104 are sequentially executed. That is, when the temporary switching condition is satisfied, the operation pattern is changed in a hurry without waiting for the expiration of the standard duration T1.
  • step S108 the count value of the timer is referred to, and it is determined whether or not the standard duration T1 has elapsed since the start of the current pattern. If the elapsed time from the start of the current pattern is equal to or longer than the standard duration T1, affirmative determination is made in step S108, and the process proceeds to step S109. On the other hand, if the standard duration T1 has not yet elapsed since the start of the current pattern (step S108 NO), step S108 is repeated while sequentially determining whether or not the end condition is satisfied in step S113. Step S113 is the same process as step S112.
  • step S109 the change determination unit F31 determines the change in the drowsiness level based on the latest determination value of the drowsiness level stored in the RAM 12.
  • the change in drowsiness level is the change in the latest drowsiness level, which is the most recently determined drowsiness level, with respect to the start drowsiness level, which is the drowsiness level measured at the start of the current pattern.
  • the start drowsiness level corresponds to the drowsiness level determined immediately after step S103.
  • the drowsiness level used in each determination step such as step S109 may be the average value of the determination results (so-called moving average value) or the mode value within the latest predetermined time.
  • the drowsiness level can be the average value or the mode value of the drowsiness level within the last 20 seconds.
  • the drowsiness level statistically determined by the drowsiness level at each time point within the latest predetermined time as a population is also referred to as a movement statistical value of the drowsiness level. According to the configuration in which the movement statistics are used to determine whether or not the drowsiness level has changed, the risk of weakening or strengthening the arousal stimulus due to the instantaneous change can be reduced.
  • Elevated means that the latest drowsiness level is higher than the starting drowsiness level.
  • Decline means that the latest drowsiness level is lower than the start drowsiness level.
  • Identical means that the latest drowsiness level is the same as the starting drowsiness level.
  • step S109 is negatively determined and the process proceeds to step S110.
  • step S111 the pattern selection unit F32 selects the next operation pattern of the current operation pattern group and returns to step S103.
  • the pattern selection unit F32 selects the mode of arousal stimulation, that is, the operation pattern group, based on the latest drowsiness level and the change direction of the drowsiness level. If the drowsiness level is elevated, select an operation pattern group according to the latest drowsiness level. Specifically, when the latest drowsiness level after the increase in drowsiness level is 1, the pattern selection unit F32 sets the operation pattern group G1 (A). Further, when the latest drowsiness level after the increase in drowsiness level is 2, the pattern selection unit F32 sets the operation pattern group G2. When the latest drowsiness level is level 3 or higher, the pattern selection unit F32 sets the operation pattern group G3.
  • the pattern selection unit F32 sets the operation pattern group G1 (B). Further, when the latest drowsiness level after the drowsiness level is lowered is 0, the pattern selection unit F32 sets the operation pattern group G0 (B). As described above, here, as an example, once the drowsiness level rises to level 3, the drowsiness level is maintained at level 3 regardless of the determination result of the drowsiness level. Therefore, the drowsiness level does not drop from level 3 to level 2.
  • the group selection process in step S111 is completed, the process returns to step S103.
  • the awakening device 1 of the present embodiment includes an operation reception unit F6 for acquiring the driver's opinion on the current pattern based on the driver's operation signal detected and output by the input device 22.
  • the operation reception unit F6 cooperates with the display processing unit F4 to acquire the driver's opinion or instruction operation for the current pattern. For example, as shown in FIG. 7, the display processing unit F4 displays various button images (B1 to B5) for acquiring the driver's opinion on the current pattern on the display 23. Then, the operation reception unit F6 acquires the driver's opinion and instruction based on the selection operation for the button image.
  • the button image selection operation can be detected based on the correspondence between the display position information of various button images and the user's touch position information with respect to the display 23.
  • Button B1 shown in FIG. 7 is a skip button, which corresponds to a button for the driver to instruct to switch to the next operation pattern.
  • the pattern management unit F7 registers the operation pattern in which the skip operation has been performed in the RAM 12 or the storage 13 as a skip pattern.
  • the operation of selecting the skip button B1 is also referred to as a skip operation.
  • the awakening device 1 of the present embodiment is configured to omit the execution from the next time with respect to the operation pattern registered in the skip pattern. For example, when the skip operation is performed on the operation pattern P12, the operation patterns other than the operation pattern P12 in the group G1 (A) are cyclically executed. Specifically, the operation patterns P10, P11, P13, and P14 are executed in this order, and when the time zone of the operation pattern P14 ends, the operation pattern P10 returns to the time zone again.
  • a plurality of operation patterns belonging to one group may be registered as skip patterns. However, if all the operation patterns belonging to one group are allowed to be registered in the skip pattern, the group will not function. Further, when there is only one operation pattern belonging to one group that is not registered in the skip pattern, a single operation pattern will continue to be executed. If one operation pattern is continued endlessly, the driver becomes accustomed to the arousal stimulus provided by the operation pattern, and there is a concern that it becomes difficult to obtain the awakening effect.
  • the pattern management unit F7 of the present disclosure refuses to register a new skip pattern when the number of operation patterns not registered in the skip pattern is two or less in the same group. do.
  • the pattern management unit F7 discards the registration of the skip pattern having the oldest registration time and then creates a new skip pattern. May be registered.
  • the pattern management unit F7 controls the registration state of the skip pattern so that at least two or more operation patterns are cyclically executed.
  • the temporary skip operation itself may be configured to be acceptable. By performing the skip operation, the effect of stimulating the user can be expected.
  • the temporary skip operation corresponds to an operation of transitioning to the next operation pattern without setting the skip pattern.
  • Button B2 is a good button for registering the current pattern as a favorite operation pattern.
  • the pattern management unit F7 registers the operation pattern being executed at that time in the favorite pattern.
  • the favorite pattern can also be called a highly evaluated operation pattern.
  • Whether or not it is registered in the favorite pattern can be used as a parameter for increasing the appearance rate when the operation pattern is randomly executed, for example.
  • the operation pattern registered in the favorite pattern may have a set value of the standard duration T1 for the operation pattern longer than a normal value by a predetermined amount.
  • the extension time can be, for example, 5 seconds or 10 seconds.
  • Button B3 is a low evaluation button for inputting that the current pattern is not the driver's preference.
  • the pattern management unit F7 registers the operation pattern being executed at that time in the low evaluation pattern.
  • the low rating button B3 can be selected, for example, if you don't like it very much or feel annoyed, although it is not enough to skip it.
  • the pattern management unit F7 may register the operation pattern in which the skip operation is performed as a low evaluation pattern.
  • the skip button B1 may also serve as the low evaluation button B3.
  • the set value of the standard duration T1 for the operation pattern may be shorter than the normal value by a predetermined amount.
  • the shortening amount can be, for example, 5 seconds or 10 seconds.
  • the operation pattern registered in the low evaluation pattern may reduce the intensity of the stimulus by a predetermined amount.
  • buttons B4 and B5 are buttons for adjusting the intensity of stimulation.
  • the button B4 is a down button B4 for weakening a predetermined amount of stimulation. Reducing the stimulus is equivalent to reducing the volume and frequency of voice. Further, in the case of light, it corresponds to reducing the brightness or darkening the hue. In the case of vibration, it corresponds to reducing the amplitude of vibration or lengthening the vibration interval.
  • the button B5 is an up button B5 for strengthening a predetermined amount of stimulation.
  • some instruction operations may be configured to be acceptable based on the output signal of the steering switch.
  • the skip operation and the up operation may be detected based on the fact that a predetermined steering switch is pressed.
  • a switch corresponding to some button images may be provided on the steering wheel.
  • the mode of registering to the skip pattern, favorite pattern, low evaluation pattern, etc. based on the user operation is disclosed, but the present invention is not limited to this.
  • comfort / discomfort may be determined, and various operation patterns may be automatically / semi-automatically registered based on the determination result of the emotion. The same can be applied to adjusting the intensity of the stimulus.
  • the operation pattern judged to be pleasant is automatically registered in the favorite pattern.
  • the operation pattern determined to be unpleasant may be automatically registered in the skip pattern and the low evaluation pattern.
  • automatic registration here refers to registration without asking for driver permission.
  • Semi-automatic registration refers to inquiring whether to register with the driver and registering if the driver approves.
  • the information used as a material for acquiring the driver's emotions for arousal stimuli such as comfort / discomfort is not limited to the driver's facial expression. It may be determined whether or not the driver feels uncomfortable based on the driver's gesture, speech, and degree of improvement in drowsiness level. In addition, based on the degree of improvement in the drowsiness level, an effective pattern that is an effective operation pattern for improving the drowsiness level and a weak effect pattern that is an operation pattern that is not so effective may be specified.
  • the awakening device 1 determines the driver's emotions (pleasant / unpleasant) for each operation pattern based on the driver operation signal input from the input device 22 and the analysis result of the driver's face image, and determines the frequency of occurrence of the operation pattern and the frequency of occurrence of the operation pattern. Execution conditions, duration, and stimulus intensity may be adjusted.
  • the DSM 21 may be provided or the awakening device 1 may be provided with a function of analyzing the face image and determining whether or not the driver feels uncomfortable.
  • the setting information for each operation pattern is stored separately for each driver, in other words, for each user.
  • the user as a driver may be identified based on the identification information of the key used for unlocking the vehicle Hv, the face image, the voiceprint, the fingerprint, and the like.
  • Various methods can be used to identify the driver.
  • the vehicle Hv is a vehicle provided for the car sharing service
  • the driver can be specified based on the usage reservation information of the vehicle Hv.
  • the pattern setting information for each user may be stored in the storage 13 or may be stored in a server on the cloud.
  • the stimulus control unit F3 having the above configuration can change the timing of switching the stimulus pattern from the current pattern to another pattern from the originally planned time based on the state of the driver after switching to the current pattern. That is, whether or not to switch the operation pattern is changed according to the state of the driver at a predetermined timing during execution of one operation pattern. Specifically, as shown in FIG. 8, when a certain operation pattern A is being executed and the driver's state satisfies a predetermined temporary switching condition when the intermediate confirmation time T2 elapses, the pattern A To a predetermined pattern B.
  • the driver's eye closure time is equal to or greater than the predetermined emergency switching threshold during the period from the start of the current pattern to the elapse of the intermediate confirmation time T2
  • the continuation of the current pattern is interrupted and the driver is drowsy one step higher.
  • Switch to the group corresponding to the level. Switching the operation pattern when the intermediate confirmation time T2 has elapsed corresponds to accelerating the end timing of the current pattern.
  • the sleepiness level is not observed in the same group.
  • This configuration corresponds to a configuration in which if the current pattern does not produce an effect, it is interrupted in the middle and an awakening stimulus of another operation pattern within the same group is output. According to this configuration, it is possible to shorten the duration of the operation pattern that does not contribute to arousal. As a result, it is possible to shorten the time required to reach the operation pattern that contributes to the driver's awakening, and it becomes easier to lead the driver to the awake state.
  • the cycle in which each operation pattern is switched is basically the standard duration T1. Since the standard duration T1 is set to be relatively longer than the intermediate confirmation time T2, it is possible to reduce the risk of causing trouble to the driver due to frequent switching of the operation pattern.
  • the working pattern group G2 includes arousal stimuli of cold air not included in the previous group.
  • the previous group is an operation pattern group before the transition, and here refers to the group G1 (A).
  • the cold wind arousal stimulus corresponds to an additional awakening stimulus.
  • the driver will experience additional arousal stimuli and will be more likely to notice that the arousal stimuli change as the drowsiness level rises.
  • the operation pattern group G2 is set to generate an awakening stimulus of cold air in some time zones. According to the setting mode in which the cold air is temporarily generated, the driver is less likely to feel the awakening stimulus annoyingly as compared with the case where the awakening stimulus of the cold air is generated during all the periods constituting the operation pattern group.
  • the awakening device 1 when the drowsiness level rises from level 2 to level 3 while the operation pattern group G2 is set, the awakening device 1 newly sets the operation pattern group G3.
  • the working pattern group G3 includes a vibrating arousal stimulus as an additional arousal stimulus.
  • the working pattern group G2 corresponds to the aspect before the drowsiness level rises. By experiencing the vibration as an additional wakefulness stimulus, the driver can easily notice that the wakefulness stimulus changes as the drowsiness level rises.
  • the awakening device 1 newly sets the operation pattern group G1 (B).
  • the working pattern group G1 (B) includes a light stimulus by HUD36 instead of the arousal stimulus included in the working pattern group G2.
  • Photostimulation by HUD36 corresponds to new arousal stimuli not included in the previous group.
  • the operation pattern group G2 includes a light stimulus by the light emitting device 31.
  • the light stimulus by the HUD 36 is the same type of awakening stimulus as the light stimulus by the light emitting device 31, but can be said to be an awakening stimulus having different characteristics such as appearance and generation position. Due to the different characteristics, the driver can experience changes in the arousal stimulus.
  • the operation pattern groups G0 (A), G0 (B), G1 (A), G1 (B), G2, and G3 include a resting operation pattern in which arousal stimulation does not occur.
  • the rest pattern By including the rest pattern, the arousal stimulus provided by other operation patterns stands out, and a further awakening effect can be expected while reducing the risk of causing annoyance.
  • the mode of switching the operation pattern at the time of intermediate confirmation based on the condition that the temporary switching condition is satisfied has been disclosed, but the present invention is not limited to this.
  • the duration of the current pattern may be extended by a predetermined time from the standard duration T1 based on the improvement in the drowsiness level of the driver at the time of the interim confirmation or the observation of a sign thereof.
  • the pattern management unit F7 is currently in the case where the drowsiness level of the driver is improved or a sign thereof is observed based on the history information of the driver state from the start of the current pattern to the intermediate confirmation time. Register the pattern as a valid pattern. Then, when the current pattern is registered in the effective pattern, the stimulus control unit F3 initially determines the end timing of the current pattern based on the standard duration T1 on condition that the user does not feel uncomfortable with the current pattern. It will be later than planned.
  • the effective pattern corresponds to the operation pattern determined to be effective for the awakening of the driver. Whether or not it is an effective pattern can be determined, for example, by comparing the drowsiness level and the driver state at the start and end of the operation pattern.
  • the stimulus control unit F3 may determine whether or not the current pattern is effective for the driver's awakening, instead of the pattern management unit F7.
  • the functional arrangement in the awakening device 1 can be changed as appropriate.
  • the duration of the current pattern is set from the standard duration T1. May be extended for a predetermined time.
  • the extension time can be, for example, 10 seconds.
  • the group switching condition and the intra-group switching condition are not limited to the above-mentioned examples.
  • the group switching condition can be a case where the number of times the driver yawns in the period from the start of the current pattern to the intermediate confirmation time T2 is, for example, three times or more, which is equal to or greater than a predetermined threshold value. .. That is, when the number of yawning executions in the period from the start of the current pattern to the intermediate confirmation time T2 is equal to or greater than a predetermined value, it is determined that the group switching condition is satisfied.
  • the switching condition within the group can be a case where the judgment value of the drowsiness level is constant in the period from the start of the current pattern to the intermediate confirmation time T2. That is, when the determination value of the drowsiness level is constant in the period from the start of the current pattern to the intermediate confirmation time T2, it is determined that the intra-group switching condition is satisfied.
  • the above assumed scene corresponds to a state in which the drowsiness level is increasing or decreasing.
  • the rising and falling drowsiness levels suggest that the driver's drowsiness is located near the boundaries of those levels.
  • the effect of lowering the drowsiness level of the driver can be expected by outputting the arousal stimulus corresponding to the relatively higher drowsiness level.
  • the above configuration has an air-conditioning stimulus pattern that outputs air-conditioned air from the air-conditioning device mounted on the vehicle as a stimulus pattern, and a window portion that applies outside air as an awakening stimulus to the driver by opening the door window for the driver's seat. It corresponds to the configuration that can adopt the open pattern. Based on the determination that the drowsiness level has not improved at the timing when the intermediate confirmation time T2 has elapsed while the stimulus control unit F3 is executing the air conditioning stimulus pattern, the window opening pattern is used instead of the air conditioning stimulus pattern from the next time. May be configured to perform. The driver's feelings can be different between the air-conditioned air and the external wind, which is the wind from the door window.
  • a window opening pattern may be adopted instead of the air conditioning stimulating pattern. According to this configuration, it becomes easy to accept even a driver who is not good at air-conditioned air.
  • the window opening pattern is controlled so as not to be automatically adopted in rainy weather.
  • the window opening pattern is automatically excluded from the options. Is preferable. If the weather conditions satisfy the above conditions, the driver may be configured to present a confirmation screen as to whether or not the window opening pattern may be executed. Whether or not the window opening pattern may be automatically executed may be registered by the driver via a predetermined setting screen.
  • cold air may be output from the air conditioner 34 as an awakening stimulus
  • the driver may be given a tactile stimulus having different characteristics by opening the door window of the driver's seat. If the door window opens suddenly, the driver may be confused. Therefore, when adopting the opening of the door window as an arousal stimulus, it is preferable to notify the driver in advance.
  • the pattern selection unit F32 may skip each group. For example, when the operation pattern group is set to G1 (A) and the skip operation is performed, the operation pattern group may be switched from G1 (A) to G2. Further, the operation receiving unit F6 may be configured to be able to receive a group skip operation for changing the operation pattern group itself and an individual skip operation for changing the operation pattern within the same operation pattern. For example, a button image corresponding to a group skip operation and a button image for accepting an individual skip operation may be displayed on the display 23, respectively, and configured to detect those instruction operations based on the user's touch position. .. The group skip operation and the individual skip operation may be configured to be detected by voice input.
  • control different from the pleasantly determined operation pattern may be applied to the operation pattern determined to be unpleasant.
  • the stimulus control unit F3 may not execute the operation pattern determined to be unpleasant until the driver gives permission.
  • the stimulus of the operation pattern determined to be unpleasant may be output in a mode in which the intensity is suppressed.
  • the operation pattern determined to be unpleasant may be configured not to be executed when the drowsiness level is less than the predetermined value, but to be executed when the drowsiness level is equal to or higher than the predetermined value.
  • the predetermined value here can be 3 or 2.
  • the operation pattern determined to be unpleasant may not be executed, and the operation pattern determined to be unpleasant may be executed when the drowsiness level is 2 or more. Execution / non-execution of a certain operation pattern can be realized by automatically / manually editing the operation pattern group.
  • the above-mentioned control does not include the operation pattern determined to be unpleasant in the operation pattern group corresponding to the drowsiness level of 0 to 1, while the operation pattern group having the drowsiness level of 2 or more includes the operation pattern determined to be unpleasant.
  • Each actuation pattern group may be configured such that the higher the drowsiness level, the higher the proportion of actuation patterns determined to be unpleasant.
  • the operation pattern group G3 corresponding to the drowsiness level 3 or higher may be composed of only the operation patterns judged to be unpleasant. According to the configuration in which the appearance ratio of the operation pattern judged to be unpleasant is increased as the drowsiness level is increased, the effect of further lowering the drowsiness level of the driver can be expected.
  • the awakening device 1 may be configured to temporarily stop the output of the awakening stimulus based on a predetermined operation by the driver. In that case, the awakening device 1 accepts the restart operation of the driver, or restarts the output of the awakening stimulus based on the elapse of the predetermined pause release time.
  • the awakening device 1 may be configured so that the intensity of the awakening stimulus can be temporarily weakened based on the driver operation. In that case, the awakening device 1 may accept the driver's restoration operation, or the awakening device 1 may restore the intensity of the arousal stimulus based on the fact that the drowsiness level is not improved.
  • the driver operation for stopping the output of the arousal stimulus or weakening the intensity is canceled.
  • the skip operation may be canceled.
  • the operation invalid threshold value can be, for example, 3 or 4. Even when the drowsiness level is equal to or higher than the predetermined operation invalid threshold value, the driver operation in the direction of increasing the stimulus intensity, the favorite registration operation, and the like may be configured to be acceptable.
  • the pattern management unit F7 may specify a weak effect pattern, which is an operation pattern in which the arousal effect is weak, by comparing the drowsiness level and the driver state at the start and end of the current pattern. For example, the driver state at the start and the driver state at the end can be compared, and an operation pattern in which no significant change in the improvement direction is observed can be registered as a weak effect pattern.
  • the driver state here is not limited to the determination value of the drowsiness level, but can also include the posture, the degree of eye opening, the movement speed of the line of sight, the stability of the blink (blink) cycle, the speed of the blink, and the like.
  • the stimulus control unit F3 may increase the intensity of the stimulus or combine another type of arousal stimulus with respect to such a weak effect pattern. Further, the stimulus control unit F3 may change the execution frequency, the execution condition, and the standard duration T1 for the weak effect pattern. For example, the stimulus control unit F3 may shorten the standard duration T1 by a predetermined amount for the operation pattern determined to be a weak effect pattern, as compared with other operation patterns. Further, the stimulus control unit F3 may adopt the operation pattern determined to be a weak effect pattern less frequently than other operation patterns. For example, it may be controlled to be executed once every two laps.
  • the execution condition may include that the elapsed time from the selection of the current operation pattern group is within a predetermined time.
  • the weak effect pattern is also executed in the initial stage after the group is selected.
  • a predetermined time has elapsed from the group selection, only the operation pattern other than the weak effect pattern is executed. According to such a configuration, only the operation pattern having a relatively strong effect is executed as time goes by, and the effect of lowering the drowsiness level of the driver can be expected.
  • the awakening device 1 may randomly arrange a completely different operation pattern at a position (time zone) recognized as a weak effect operation pattern.
  • a position time zone
  • the awakening device 1 may randomly arrange a completely different operation pattern at a position (time zone) recognized as a weak effect operation pattern.
  • the display processing unit F4 may present an option for the driver to input the reason.
  • the pattern management unit F7 saves the skip reason in association with the operation pattern.
  • the stimulus control unit F3 may execute the operation pattern in a mode in which the stimulation intensity is increased by a predetermined amount for the operation pattern for which the stimulus is too weak as the reason for skipping. Further, the stimulus control unit F3 may execute the operation pattern in a mode in which the stimulus intensity is weakened by a predetermined amount for the operation pattern for which the stimulus is too strong as the reason for skipping.
  • ⁇ About the configuration of the operation pattern for each group> The combination of operation patterns and the execution order of each group can be changed as appropriate.
  • the pause pattern is arranged at the head of each group, but the present invention is not limited to this.
  • the pause pattern may be placed at the end of the group instead of at the beginning. According to the configuration in which the rest pattern is placed at the beginning, the driver can notice that the drowsiness level and the mode have changed by experiencing the time when the arousal stimulus does not occur.
  • adjacent operation pattern groups include non-overlapping types of arousal stimuli.
  • the actuation pattern group G1 (B) may include a type of arousal stimulus not included in the actuation pattern group G2.
  • Examples of different types of arousal stimuli include dialogue awakening stimuli, vibration awakening stimuli, voice awakening stimuli, and the like.
  • the driver can experience the change in the arousal stimuli. As a result, it is possible to reduce the risk that the driver will become accustomed to the arousal stimulus. As a result, the risk that the drowsiness level that is about to fall will rise again can be reduced.
  • the awakening device 1 and its method described in the present disclosure may be realized by a dedicated computer constituting a processor programmed to perform one or more functions embodied by a computer program. Further, the apparatus and the method thereof described in the present disclosure may be realized by using a dedicated hardware logic circuit. Further, the apparatus and method thereof described in the present disclosure may be realized by one or more dedicated computers configured by a combination of a processor for executing a computer program and one or more hardware logic circuits. Further, the computer program may be stored in a computer-readable non-transitional tangible recording medium as an instruction executed by the computer.
  • the means and / or functions provided by the awakening device 1 and the like can be provided by the software recorded in the actual memory device and the computer, software only, hardware only, or a combination thereof that execute the software.
  • some or all of the functions included in the awakening device 1 may be realized as hardware.
  • a mode in which a certain function is realized as hardware includes a mode in which one or more ICs are used.
  • the awakening device 1 may be realized by using an MPU, a GPU, or a DFP (Data Flow Processor) instead of the CPU.
  • the awakening device 1 may be realized by combining a plurality of types of arithmetic processing devices such as a CPU, an MPU, and a GPU.
  • the awakening device 1 may be realized as a system-on-chip (SoC).
  • SoC system-on-chip
  • various processing units may be realized by using FPGA (Field-Programmable Gate Array) or ASIC (Application Specific Integrated Circuit).
  • Various programs may be stored in a non-transitionary tangible storage medium.
  • various storage media such as HDD (Hard-disk Drive), SSD (Solid State Drive), flash memory, and SD (Secure Digital) card can be adopted.
  • the non-transitional substantive recording medium also includes a ROM such as EPROM (Erasable Programmable Read Only Memory).
  • the plurality of functions possessed by one component in the above embodiment may be realized by a plurality of components, or one function possessed by one component may be realized by a plurality of components. Further, a plurality of functions possessed by the plurality of components may be realized by one component, or one function realized by the plurality of components may be realized by one component. In addition, a part of the configuration of the above embodiment may be omitted. Further, at least a part of the configuration of the above embodiment may be added or replaced with the configuration of the other above embodiment.
  • various forms such as a system having the awakening device 1 as a component are also included in the scope of the present disclosure.
  • a program for making a computer function as an awakening device 1 a non-transitional actual recording medium such as a semiconductor memory in which this program is recorded, and the like are also included in the scope of the present disclosure.

Landscapes

  • Engineering & Computer Science (AREA)
  • Automation & Control Theory (AREA)
  • Human Computer Interaction (AREA)
  • Transportation (AREA)
  • Mechanical Engineering (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Traffic Control Systems (AREA)

Abstract

Un dispositif d'éveil (1) comprend une pluralité de configurations de fonctionnement pour chaque niveau de somnolence. Le dispositif d'éveil (1) est configuré de sorte que, si le niveau de somnolence est constant, la configuration de fonctionnement correspondant au niveau de somnolence actuel soit commutée et exécutée à chaque durée standard prescrite (T1). Si l'état d'un conducteur, à un moment auquel un temps de confirmation intermédiaire prescrit (T2) depuis le début de la configuration de fonctionnement en cours d'exécution s'est écoulé, satisfait une condition de commutation spéciale prescrite, le dispositif d'éveil (1) commute la configuration de fonctionnement à une configuration prescrite sans attendre que la durée standard (T1) se soit écoulée.
PCT/JP2021/044132 2020-12-11 2021-12-01 Dispositif d'éveil et procédé de commande de stimulus d'éveil WO2022124163A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2020-206063 2020-12-11
JP2020206063A JP7540319B2 (ja) 2020-12-11 2020-12-11 覚醒装置

Publications (1)

Publication Number Publication Date
WO2022124163A1 true WO2022124163A1 (fr) 2022-06-16

Family

ID=81973936

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/044132 WO2022124163A1 (fr) 2020-12-11 2021-12-01 Dispositif d'éveil et procédé de commande de stimulus d'éveil

Country Status (2)

Country Link
JP (1) JP7540319B2 (fr)
WO (1) WO2022124163A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116726223A (zh) * 2023-05-08 2023-09-12 深圳盈特创智能科技有限公司 一种智能香薰机控制电路、装置及控制方法
CN118253009A (zh) * 2024-04-11 2024-06-28 电子科技大学 基于闭环声刺激和光生物调控的睡眠剥夺系统及使用方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9135803B1 (en) * 2014-04-17 2015-09-15 State Farm Mutual Automobile Insurance Company Advanced vehicle operator intelligence system
JP2017059043A (ja) * 2015-09-17 2017-03-23 トヨタ自動車株式会社 車両用覚醒制御システム
WO2017221603A1 (fr) * 2016-06-21 2017-12-28 株式会社デンソー Appareil de maintien de la vigilance
WO2019026553A1 (fr) * 2017-08-02 2019-02-07 株式会社デンソー Dispositif de maintien de vigilance
WO2019073708A1 (fr) * 2017-10-12 2019-04-18 株式会社デンソー Dispositif d'assistance à la conduite véhiculaire
JP2019159711A (ja) * 2018-03-12 2019-09-19 矢崎総業株式会社 覚醒度維持装置

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5056067B2 (ja) 2007-02-26 2012-10-24 株式会社デンソー 居眠り警報装置
JP6965727B2 (ja) 2017-12-21 2021-11-10 株式会社デンソー 居眠り警報装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9135803B1 (en) * 2014-04-17 2015-09-15 State Farm Mutual Automobile Insurance Company Advanced vehicle operator intelligence system
JP2017059043A (ja) * 2015-09-17 2017-03-23 トヨタ自動車株式会社 車両用覚醒制御システム
WO2017221603A1 (fr) * 2016-06-21 2017-12-28 株式会社デンソー Appareil de maintien de la vigilance
WO2019026553A1 (fr) * 2017-08-02 2019-02-07 株式会社デンソー Dispositif de maintien de vigilance
WO2019073708A1 (fr) * 2017-10-12 2019-04-18 株式会社デンソー Dispositif d'assistance à la conduite véhiculaire
JP2019159711A (ja) * 2018-03-12 2019-09-19 矢崎総業株式会社 覚醒度維持装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116726223A (zh) * 2023-05-08 2023-09-12 深圳盈特创智能科技有限公司 一种智能香薰机控制电路、装置及控制方法
CN118253009A (zh) * 2024-04-11 2024-06-28 电子科技大学 基于闭环声刺激和光生物调控的睡眠剥夺系统及使用方法

Also Published As

Publication number Publication date
JP7540319B2 (ja) 2024-08-27
JP2022093011A (ja) 2022-06-23

Similar Documents

Publication Publication Date Title
CN105015445B (zh) 用于对车辆的驾驶员进行与人相关的辅助的方法和系统
JP6965727B2 (ja) 居眠り警報装置
WO2022124163A1 (fr) Dispositif d'éveil et procédé de commande de stimulus d'éveil
CN108430818B (zh) 用于操作机动车辆的方法，驾驶员辅助系统和机动车辆
US11453414B2 (en) Method and device for adapting a driving strategy of an at least partially automated transportation vehicle
JP4727688B2 (ja) 覚醒度推定装置
JP5842857B2 (ja) 覚醒度改善装置
CN104908687B (zh) 用于允许在机动车辆中的提高效能的小睡的方法
JP5155805B2 (ja) 睡眠制御装置
WO2019087507A1 (fr) Système de suppression de conduite inattentive
CN112568904B (zh) 车辆交互方法、装置、计算机设备及存储介质
JP6776681B2 (ja) ドライバ状態判定装置、及びドライバ状態判定プログラム
CN114599542A (zh) 用于在车辆中在行驶期间控制第一车辆功能和第二车辆功能以对车辆乘员产生作用的方法
CN114572226B (zh) 基于驾驶员状态的车辆控制方法、装置、车辆及存储介质
JP2011118831A (ja) 覚醒支援装置
JP6958982B2 (ja) 覚醒機能を備えた電子装置、覚醒プログラムおよび覚醒方法
JP7087286B2 (ja) 覚醒維持装置
JP6593247B2 (ja) 安全運転支援装置
TWI818395B (zh) 用於監控車輛駕駛員的活動的駕駛員監控方法及用於執行該方法的車輛
JP6673070B2 (ja) ドライバ状態誘導装置、及びドライバ状態誘導プログラム
JP2022179108A (ja) 覚醒支援システム
JP7380622B2 (ja) 覚醒システムおよび覚醒制御装置
JP7151603B2 (ja) 覚醒装置
JP7251466B2 (ja) 状態改善装置
JP2023017603A (ja) 覚醒支援装置及び覚醒支援方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21903267

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21903267

Country of ref document: EP

Kind code of ref document: A1