WO2022124163A1

WO2022124163A1 - Awakening device, and awakening stimulus control method

Info

Publication number: WO2022124163A1
Application number: PCT/JP2021/044132
Authority: WO
Inventors: 有華里伊藤; 真紀子杉浦; 彩子小谷; 政治河合
Original assignee: 株式会社デンソー
Priority date: 2020-12-11
Filing date: 2021-12-01
Publication date: 2022-06-16
Also published as: JP7540319B2; JP2022093011A

Abstract

An awakening device (1) comprises a plurality of operation patterns for each drowsiness level. The awakening device (1) is configured so that if the drowsiness level is constant, the operation pattern corresponding to the current drowsiness level is switched and executed at each prescribed standard time duration (T1). If the state of a driver at a timing at which a prescribed intermediate confirmation time (T2) from the start of the operation pattern currently being executed has elapsed satisfies a prescribed special switching condition, the awakening device (1) switches the operation pattern to a prescribed pattern without waiting for the standard time duration (T1) to elapse.

Description

Awakening device, awakening stimulus control method

Cross-reference of related applications

This application is based on Patent Application No. 2020-206063 filed in Japan on December 11, 2020, and the contents of the basic application are incorporated by reference as a whole.

This disclosure relates to an awakening device for awakening a person sitting in the driver's seat and an awakening stimulus control method.

Patent Document 1 discloses an awakening device that generates a predetermined pattern of vibration when a driver's drowsiness is detected. Vibration is an awakening stimulus to awaken the driver.

Japanese Unexamined Patent Publication No. 2018-55527

The driver's drowsiness level may drop when the awakening device generates an arousal stimulus. In addition, the drowsiness level may not be improved even if a predetermined pattern of arousal stimulus is given. Based on such circumstances, it is conceivable to measure the drowsiness level of the driver at regular intervals and give a pattern stimulus according to the measured drowsiness level.

Here, if the cycle of switching the stimulation pattern is too long, it may cause annoyance. For example, even when the drowsiness level is lowered, the previous pattern is continued until a predetermined time elapses. Further, even if the switching cycle of the stimulation pattern is made too short, the stimulation pattern is frequently changed, which may cause trouble for the driver.

The present disclosure has been made based on this circumstance, and an object thereof is to provide an awakening device capable of reducing the risk of causing trouble to the driver.

The awakening device for achieving the purpose includes a driver status acquisition unit that acquires information indicating the driver status from at least one of a sensor mounted on the vehicle and a sensor mounted on the driver's body. A plurality of prepared devices in which at least one of the characteristics and the intensity of the arousal stimulus to be generated is different from the sleepiness level determination unit that determines the sleepiness level of the driver based on the information indicating the driver status acquired by the driver status acquisition unit. Among the stimulus patterns of the above, a stimulus control unit that generates an arousal stimulus with a stimulus pattern corresponding to the drowsiness level determined by the drowsiness level determination unit is provided, and the stimulus control unit sequentially switches and executes a plurality of types of stimulus patterns. It is configured to adjust the timing of switching the stimulus pattern from the current pattern to another stimulus pattern based on the state of the driver after switching to the current pattern, which is the currently executing stimulus pattern. ing.

According to the above configuration, the timing for changing the stimulus pattern is changed based on the state of the driver after the stimulus pattern currently being executed is started. According to such a configuration, it is possible to accelerate or delay the timing of strengthening or weakening the arousal stimulus or changing the type of the arousal stimulus according to the state of the driver. As a result, it is possible to reduce the risk of causing trouble to the driver.

Further, the awakening stimulus control method for achieving the above object is to acquire information indicating the driver's state from at least one of a sensor mounted on the vehicle and a sensor mounted on the driver's body. , Judgment among a plurality of pre-prepared stimulus patterns in which at least one of the characteristics and intensity of the arousal stimulus to be generated is different from the determination of the drowsiness level of the driver based on the acquired information indicating the state of the driver. Awakening stimulus is generated by a stimulus pattern according to the drowsiness level, multiple types of stimulus patterns are sequentially switched and executed, and the state of the driver after switching to the current pattern, which is the stimulus pattern currently being executed. Includes changing the timing of switching the stimulus pattern from the current pattern to another stimulus pattern based on.

The above method corresponds to the control method of the arousal stimulus carried out by the above-mentioned awakening device. That is, according to the above method, the same effect is obtained by the same action as the above-mentioned awakening device.

The reference numerals in parentheses described in the claims indicate the correspondence with the specific means described in the embodiment described later as one embodiment, and do not limit the technical scope of the present disclosure. do not have.

It is a block diagram which shows the structure of awakening apparatus 1. It is a functional block diagram of the awakening device 1. It is a figure which shows the setting example of the operation pattern. It is a figure which shows the setting example of the transition source and the transition destination of the operation pattern group. It is a figure which shows the other setting example of the operation pattern. It is a flowchart for demonstrating the operation of the awakening apparatus 1. It is a figure which shows the display screen example for accepting a driver operation. It is a figure for demonstrating the timing when the awakening apparatus 1 switches an operation pattern.

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. FIG. 1 is a diagram showing an example of a schematic configuration of the awakening device 1 according to the present disclosure.

<Introduction>
The awakening device 1 of the present disclosure is a device for guiding a driver to an awakening state, and is mounted on, for example, a vehicle Hv and used. The driver in the present disclosure refers to, for example, a person sitting in the driver's seat. The expression "driver" is not limited to a person who actually performs a part or all of the driving operation. The description of a driver refers to a person who should receive the authority of driving operation from the automatic driving system during automatic driving. Further, the vehicle Hv may be a remote-controlled vehicle that is remotely controlled by an operator existing outside the vehicle. The operator here refers to a person who has the authority to control the vehicle by remote control from the outside of the vehicle. When an operator is assumed as a driver, the awakening device 1 of the present disclosure is used as a part of a cockpit system that accepts an operation of the operator.

The level indicated by "automatic driving" in the present disclosure may be, for example, equivalent to level 3 defined by the American Society of Automotive Engineers of Japan (SAE International), or may be level 4 or higher. Level 3 refers to the level at which the system executes all operation tasks within the operational design domain (ODD), while the operation authority is transferred from the system to the user in an emergency. The ODD defines conditions under which automatic driving can be executed, such as the traveling position being in a highway. At level 3, the user is required to be able to respond promptly when there is a request for a change of operation from the system. The person who takes over the driving operation may be an operator. Level 3 corresponds to so-called conditional automated driving. Level 4 is a level at which the system can perform all driving tasks except under specific circumstances such as unresponsive roads and extreme environments. Level 5 is the level at which the system can perform all driving tasks in any environment. Level 4 or higher automated driving refers to the level at which the automated driving device performs all driving tasks, that is, the automated level at which the driver's seat occupants are allowed to sleep.

<About the system configuration including the awakening device 1>
As shown in FIG. 1, the awakening device 1 is used by being connected to various in-vehicle devices such as a driver status monitor (hereinafter, DSM: Driver Status Monitor) 21. For example, the awakening device 1 is connected to a DSM 21, an input device 22, a display 23, a light emitting device 31, an aroma shooter 32, a dialogue device 33, an air conditioner 34, and a vibration generator 35. Further, the awakening device 1 can be used by being directly or indirectly connected to a head-up display (HUD) 36, a speaker 37, a window motor 38, and the like.

In addition, the awakening device 1 is also connected to various sensors / devices (not shown in FIG. 1) via the in-vehicle network Nw, which is a communication network constructed in the vehicle. For example, the awakening device 1 is input with the detection results of various in-vehicle sensors via the in-vehicle network Nw. Examples of the in-vehicle sensor include sensors that detect vehicle speed, acceleration, steering angle, shift position, accelerator depression amount, brake depression amount, and the like. In-vehicle sensors also include sensors / switches that detect the operating state of the parking brake and the power state of the vehicle Hv.

The awakening device 1 and the in-vehicle device may be connected by a dedicated line or may be connected via the in-vehicle network Nw. Further, an ECU (Electronic Control Unit) may be interposed between the awakening device 1 and the in-vehicle device.

The DSM21 is a device that sequentially detects the user's state based on the user's face image. The DSM 21 includes, for example, a near-infrared light source, a near-infrared camera, and a control module for controlling them. The DSM 21 is installed in a posture in which the near-infrared camera faces the direction in which the headrest of the driver's seat is present, for example, on the upper surface of the steering column portion, the upper surface of the instrument panel, or the like. The DSM 21 uses a near-infrared camera to photograph the head of the driver irradiated with near-infrared light by a near-infrared light source. The image captured by the near-infrared camera is image-analyzed by the control module. The control module extracts driver state information, which is information indicating the driver's state, such as the opening of the driver's eyes, from the captured image input from the near-infrared camera. The DSM 21 outputs the driver state information extracted from the driver's face image to the awakening device 1.

The driver status information includes, for example, the direction of the driver's face, the direction of the line of sight, the opening of the eyes, the opening of the mouth, and the like. The opening of the eyes can be rephrased as the degree of opening of the eyelids. The information indicating the opening degree of the eyes corresponds to the eye opening degree information. Further, the DSM 21 may be configured to detect swelling, inattentiveness, outbreak of illness, etc. based on the time course of the opening degree of the eyes, the facial expression, the orientation of the face, and the like. The driver status information can include sleepiness, inattentiveness, poor physical condition, and the like. In addition, the DSM21 can detect yawning based on the pattern of mouth opening over time. The awakening device 1 may have a function of detecting the state of the driver based on the image analysis. In that case, the DSM 21 may be configured to be able to output the driver's face image to the awakening device 1. The functional arrangement between the DSM 21 and the awakening device 1 can be changed as appropriate.

The input device 22 is an operating member for receiving a driver's instruction to the awakening device 1. The input device 22 may be a mechanical switch (so-called steer switch) provided on the spoke portion of the steering wheel, or may be a voice input device that recognizes the utterance content of the driver. Further, the input device 22 may be a touch panel laminated on the display panel of the display 23 provided on the instrument panel. As the display 23, for example, a center display provided in the central region of the instrument panel in the vehicle width direction can be adopted. The display 23 may be a meter display. Further, the input device 22 may be a driver's smartphone. For example, the touch panel and display of the smartphone possessed by the driver can be used as the input device 22 and the display 23.

The light emitting device 31 is a device that generates light that can be visually recognized by the driver. The light emitting device 31 includes, for example, an LED as a light source. The light emitting device 31 is provided at a position within the driver's field of view, such as a spoke portion of the steering wheel, an upper surface portion of the instrument panel, and a ceiling portion in front of the driver's seat. The light generated by the light emitting device 31 corresponds to an arousal stimulus for vision.

The awakening stimulus is a stimulus that the driver can feel with all five senses, and is a stimulus that awakens the driver. The light emitting device 31 can change the brightness, the color of light, and the like. The light emitting device 31 can periodically change the brightness. When the brightness changes periodically, the light emitting device 31 can adjust the length of the cycle, the waveform, the amount of change in the brightness per unit time, and the like.

The aroma shooter 32 injects the fragrance into the passenger compartment of the vehicle Hv. The driver perceives the scent of the sprayed fragrance. The scent generated by the aroma shooter 32 corresponds to the arousal stimulus for the sense of smell. The aroma shooter 32 can change the type of scent, the intensity of the scent, and the like. The aroma shooter 32 may be integrated with the air conditioner 34.

The dialogue device 33 interacts with the driver. That is, the dialogue device 33 recognizes the voice produced by the driver, creates an answer to the driver's voice, and pronounces the answer. The dialogue device 33, for example, uses artificial intelligence to perform a dialogue. The dialogue provided by the dialogue device 33 corresponds to the arousal stimulus corresponding to the auditory sense.

The air conditioner 34 generates cold air as air conditioning air in the vehicle interior of the vehicle Hv. Cold wind corresponds to arousal stimuli for cold and tactile sensations. The air conditioner 34 can change the temperature, air volume, and mode of the cold air. There are two modes, an auto mode and a driver face mode. The auto mode is a mode in which the blowing direction of cold air is changed by a preset algorithm. The driver face mode is a mode in which cold air is always blown toward the driver's face.

The vibration generator 35 generates vibration based on the control signal from the awakening device 1. The vibration generator 35 is embedded in the seating surface and the backrest portion of the driver seat. Vibration corresponds to arousal stimuli.

The HUD 36 is a device that projects a virtual image that can be perceived by the user by projecting image light onto a predetermined area of the windshield based on control signals and video data input from the awakening device 1 and the navigation device. The HUD 36 displays an image superimposed on the landscape in front of the vehicle Hv. Such a HUD 36 can generate a light stimulus by displaying a predetermined image. The light stimulus corresponds to the arousal stimulus for vision. The device that outputs the optical stimulus based on the image does not have to be HUD36. An image-based optical stimulus may be generated from a meter display or a center display.

Further, the HUD 36 can play a role as an information presenting device that presents various information such as the operating state of the awakening device 1, the break proposal, and the state of the driver recognized by the system. Information on the operating state of the awakening device 1 includes, for example, activation information and continuation information of the awakening device 1.

The activation information is information for notifying the driver that the awakening stimulus will be activated from now on, and is expressed by, for example, a text or an icon image. The activation information is displayed before the arousal stimulus occurs. The continuation information is displayed when the arousal stimulus is ongoing. The continuation information informs the driver that the arousal stimulus is continuing. The continuation information may include an icon image or text outlining the stimulus currently being output. The outline of the stimulus includes the types of stimuli such as light, sound, vibration, odor, and air conditioning. In addition, the continuation information may include advance notice information indicating the type and intensity of the stimulus output in the next term.

The break proposal is a display of the content that proposes to the driver to take a break. The break proposal is displayed on any one or more of a plurality of display devices such as the HUD 36 and the display 23. Under the control of the awakening device 1, the HUD 36 displays, for example, an icon image imitating a coffee cup or an icon image for urging a nap when parked, as a break proposal.

The driver state is, for example, information indicating the drowsiness level of the driver recognized by the awakening device 1 described later. For example, the HUD 36 displays an icon image or text indicating the drowsiness level under the control of the awakening device 1. In addition, the driver state may include the degree of fatigue and the degree of indiscriminateness. The presentation of various information is not limited to the HUD 36, and may be performed using a meter display, a center information display, or the like.

The speaker 37 generates sound in the vehicle interior of the vehicle Hv. Speech corresponds to arousal stimuli for hearing. There are two types of voice: music voice and alarm voice. The expression with voice includes mere sound.

The window motor 38 is a motor for changing the opening degree of the window glass (so-called door window) of the door. The window motor 38 can be, for example, a motor for opening and closing a door window for a driver's seat. The window motor 38 opens and closes the driver's seat door window based on the control signal input from the awakening device 1. In addition, another ECU such as a body ECU may be interposed between the awakening device 1 and the window motor 38. When the window motor 38 opens the door window for the driver's seat, the wind from outside the vehicle interior hits the driver's face. Such external winds can be an arousal stimulus for the driver.

Hereinafter, for convenience, devices that directly or indirectly stimulate the driver, such as a light emitting device 31, an aroma shooter 32, a dialogue device 33, an air conditioner 34, a vibration generator 35, a HUD 36, a speaker 37, and a window motor 38. Is also referred to as a stimulus generator.

The in-vehicle device that communicates with the awakening device 1 is not limited to the above. For example, a biosensor that senses the biometric information of the driver may be connected to the awakening device 1. The biosensor is, for example, a heart rate sensor that measures a heart rate. Biological sensors also include sensors that detect blood pressure, electrocardiogram, pulse wave, sweating amount, body temperature, heat dissipation from the human body, respiratory rhythm, and respiratory depth. The biosensor may be built in the driver's seat or may be provided in the steering. In addition, by transmitting and receiving millimeter waves as exploration waves toward the driver's seat, millimeter wave radars that detect the driver's heart rate, body movement, and posture can also be included in the biosensor.

Further, the biosensor may be a wearable device 40 that is used by being attached to, for example, a wrist of a driver. As the wearable device 40, various shapes such as a wristband type, a wristwatch type, a ring type, a glasses type, and an earphone type can be adopted. The wearable device 40 is configured to be able to communicate with the awakening device 1 via a communication device 39 mounted on the vehicle Hv. The connection mode between the wearable device 40 and the communication device 39 may be a wired connection or a wireless connection. For example, the communication device 39 and the wearable device 40 are configured to carry out wireless communication conforming to short-range wireless communication standards such as Bluetooth (registered trademark) and Wi-Fi (registered trademark). A smartphone paired with each of the communication device 39 and the wearable device 40 may be interposed between the communication device 39 and the wearable device 40.

<About the configuration of the awakening device 1>
The awakening device 1 is a device that operates various stimulus generators based on an input signal from the DSM 21 in order to maintain the awake state of the driver or guide the driver to the awake state. The awakening device 1 is configured as a computer including a processor 11, a RAM (Random Access Memory) 12, a storage 13, a communication interface 14 (I / O in the figure), a bus line connecting these configurations, and the like. There is. The processor 11 is an arithmetic core such as a CPU (Central Processing Unit). The processor 11 executes various processes by accessing the RAM 12. The RAM 12 is a volatile memory.

The storage 13 has a configuration including a non-volatile storage medium such as a flash memory. The storage 13 stores an awakening stimulus control program, which is a program for making the computer function as the awakening device 1. Executing the awakening stimulus control program by the processor 11 corresponds to executing the awakening stimulus control method corresponding to the awakening stimulus control program.

Further, in the storage 13, a plurality of operation pattern groups in which the generation patterns of arousal stimuli are grouped for each drowsiness level are registered. Each of the plurality of action pattern groups is set to provide arousal stimuli according to the drowsiness level described later. The details of the operation pattern group will be described later. The communication interface 14 is a circuit for the awakening device 1 to communicate with another device. The communication interface 14 may be realized by using an analog circuit element, an IC, or the like.

The awakening device 1 provides each functional unit shown in FIG. 2 by the processor 11 executing the awakening stimulus control program stored in the storage 13. That is, the awakening device 1 includes a sensor information acquisition unit F1, a drowsiness level determination unit F2, a stimulus control unit F3, a display processing unit F4, an end determination unit F5, an operation reception unit F6, and a pattern management unit F7. The stimulus control unit F3 includes a change determination unit F31 and a pattern selection unit F32 as finer functional units.

The sensor information acquisition unit F1 is configured to acquire information (that is, determination material) for determining the drowsiness level of the driver from the DSM21, a wearable device as a biological sensor, and the like. The drowsiness level is a parameter indicating the degree of drowsiness. Information that can be used to determine the drowsiness level includes, for example, the opening of the eyes, the frequency of blinking, the degree of variation in the blinking interval, the direction of the driver's face, the direction of the line of sight, the posture, and the frequency of yawning. There is a degree of wobbling of the handle angle. The sensor information acquisition unit F1 sequentially acquires various information for the drowsiness level determination unit F2 to determine the drowsiness level from a predetermined sensor group including the DSM 21. The material for determining drowsiness level can also be called a driver feature that can be used to evaluate drowsiness level. The sensor information acquisition unit F1 corresponds to the driver status acquisition unit.

The sensor information acquisition unit F1 can also acquire the state (on / off) of the driving power source, vehicle speed, acceleration, steering angle, accelerator depression amount, brake depression amount, and the like. The traveling power source is a power source for traveling the vehicle Hv, and refers to an ignition power source when the vehicle Hv is a gasoline-powered vehicle. When the vehicle Hv is an electric vehicle or a hybrid vehicle, the driving power source refers to the system main relay.

The drowsiness level determination unit F2 determines the drowsiness level of the driver based on various information acquired by the sensor information acquisition unit F1. In the present embodiment, as an example, a case where the drowsiness level is determined by dividing it into 6 stages from 0 to 5 will be described as an example. The smaller the value of drowsiness level, the higher the degree of arousal. For example, level 1 corresponds to a state in which the driver is not sleepy at all, and level 1 corresponds to a state in which the driver is slightly sleepy, in other words, the driver himself / herself is not aware of drowsiness. Level 2 is a state in which the driver is a little sleepy and can be aware of drowsiness. Level 3 corresponds to a sleepy state, and level 4 corresponds to a fairly sleepy state. Level 5 corresponds to almost / completely sleeping.

In the present embodiment, as an example, the awakening device 1 shall perform the same control as in the case of drowsiness level 3 in the case of drowsiness level 4 to 5. Therefore, in the following, the description in the case of level 4 to level 5 will be omitted. However, in the case of drowsiness levels 4 to 5, in addition to the same control as level 3, for example, a control signal for executing MRM (Minimum Risk Maneuver) is output to the ECU that controls the running of the vehicle Hv. Is also good. The MRM is, for example, a control in which the vehicle is decelerated at a predetermined deceleration while issuing an alarm to the surroundings, and the vehicle is stopped in the traveling lane or near the shoulder of the road. The control signal for executing the MRM may be a signal indicating that the driver is sleeping.

A variety of methods can be used to determine the drowsiness level. For example, the drowsiness level determination unit F2 has the degree of eye opening, the degree of fluctuation of the degree of eye opening over time, the difference in the degree of opening of the left and right eyes, the speed of movement of the line of sight, the cycle and speed of blinking (blinking), the frequency of yawning, and the like. Is used in combination to determine the drowsiness level. In general, the stronger the drowsiness, the smaller the amount of eye opening tends to be. Therefore, the drowsiness level determination unit F2 may determine that the smaller the eye opening amount, the higher the drowsiness level. In addition, the stronger the drowsiness, the lower the moving speed of the line of sight and the speed of blinking. Therefore, the drowsiness level determination unit F2 may determine the drowsiness level higher as the moving speed of the line of sight and the speed of blinking are smaller. In addition, the blink cycle tends to be unstable. Therefore, the drowsiness level determination unit F2 may evaluate the drowsiness level based on the degree of stability of the blink cycle, for example, the dispersion value of the blink cycle within a certain time. In addition, the drowsiness level determination unit F2 determines the amount of heat released from the driver's body, the tendency and distribution of changes in body surface temperature, the presence or absence of deep breathing, the presence or absence of vertical movement of the shoulders, and whether or not the face is facing downward or upward. The drowsiness level may be determined based on the condition. Further, the drowsiness level may be determined higher as the change in the speed of the vehicle Hv and the change in the steering angle are smaller.

The details of the stimulus control unit F3, the change determination unit F31, the pattern selection unit F32, the display processing unit F4, the end determination unit F5, the operation reception unit F6, and the pattern management unit F7 will be described below. In addition, the awakening device 1 has a timer function for measuring a predetermined time. The awakening device 1 uses the timer function to calculate the elapsed time from the start of each operation pattern, the duration of the closed eye state in which the driver is closed, and the like. The closed eye state here means, for example, a state in which the degree of eye opening is equal to or less than a predetermined threshold value. The closed eye state can include a state in which the eyes are completely closed and a state in which the eyes are slightly open, such as a state in which the eyes are almost closed.

<About the operation pattern group>
As an example in the present embodiment, the awakening device 1 is provided with operation pattern groups G0 (A), G0 (B), G1 (A), G1 (B), G2, and G3 as shown in FIG. There is. The operation pattern group G0 (A) is an operation pattern group corresponding to a state in which no awakening stimulus is output. The operation pattern group G0 (A) can be set when the drowsiness level is 0. For example, it is applied immediately after the start of operation of the operation pattern group G0 (A). Further, the operation pattern group G0 (A) is applied, for example, when the drowsiness level is maintained at 0 for a predetermined release time after the drowsiness level drops from 1 to 0. The release time can be, for example, 45 seconds or 1 minute. The actuated pattern group G0 (A) can be referred to as a non-actuated group. Further, the state in which the operation pattern group G0 (A) is set can also be called a stimulation stop mode.

The operation pattern groups other than the operation pattern group G0 (A), that is, the groups G1 (A), G1 (B), G2, and G3 each include a plurality of operation patterns having different types and combinations of arousal stimuli to be generated. That is, the groups G1 (A), G1 (B), G2, and G3 are operation pattern groups that intermittently or continuously generate some kind of stimulus. Therefore, the groups G1 (A), G1 (B), G2, and G3 can be referred to as a stimulus generation group. The state in which the stimulus generation group is selected can be referred to as a stimulus generation mode. In the stimulus generation group, as shown in FIG. 3, the type of arousal stimulus to be generated is determined. The operation pattern corresponds to the stimulation pattern.

Types of arousal stimuli include light emission, scent, dialogue, cold air, vibration, and voice. The light emission is an arousal stimulus of light generated by the light emitting device 31 and the HUD 36. The scent is an arousal stimulus generated by the aroma shooter 32. Dialogue is an arousal stimulus generated by the dialogue device 33. The cold air is, for example, an arousal stimulus generated by the air conditioner 34. Cold air may be given to the driver by opening the driver's door window using the window motor 38. Vibration is an arousal stimulus generated by the vibration generator 35. The voice is an arousal stimulus generated by the speaker 37.

In addition, in each operation pattern group, the characteristics of the arousal stimulus are determined for each type of arousal stimulus. As shown in FIG. 3, the light arousal stimuli generated by the light emitting device 31 are classified into light emitting devices 1 to 4. Emissions 1 to 4 have different characteristics from each other. Features include, for example, the color of light, the brightness, the operation pattern of light emission, the place where light is generated, a device that generates light, and the like. The intensity of the stimulus, in other words, the effect of awakening the driver, is higher in light emission 2 than in light emission 1, higher in light emission 3 than in light emission 2, and higher in light emission 4 than in light emission 3.

The scent awakening stimulus is divided into scents 1 to 4. The scents 1 to 4 have different characteristics from each other. Features include, for example, the type of scent, the intensity of the scent, and the like. The effect of awakening the driver, that is, the intensity of the stimulus is higher in scent 2 than in scent 1, higher in scent 3 than in scent 2, and higher in scent 4 than in scent 3.

The cold air awakening stimulus consists of three elements: cold air temperature, air volume, and mode. The temperature of the cold air is auto or low temperature. Auto is to set the temperature by a preset algorithm. The air volume is auto or strong. Auto is to set the air volume by a preset algorithm. The mode is the above-mentioned auto mode or the driver face mode. When the temperature, air volume, and mode of the cold air are all automatic, it means that the awakening stimulus of the cold air is not generated. The effect of awakening the driver is higher when the temperature of the cold air is low than when the temperature of the cold air is auto. The effect of awakening the driver is higher when the air volume of the cold air is strong than when the air volume of the cold air is automatic. The effect of awakening the driver is higher in the driver face mode than in the auto mode.

There are two types of voice awakening stimuli, "on" and "alarm" as an example. On corresponds to the state of outputting music as an arousal stimulus. The alarm outputs an alarm as an awakening stimulus. The effect of awakening the driver is higher with the alarm than with the on. A plurality of types of alarms such as

alarms

1, 2, and 3 may be prepared according to the loudness and frequency of the sound. The intensity of the arousal stimulus is set to increase in the order of

alarms

1, 2, and 3.

Group G0 (B) corresponding to the stimulus generation group includes operation patterns P00 and P01. When the group G0 (B) is applied, the time zone of the operation pattern P00 and the time zone of the operation pattern P01 are alternately repeated. Awakening stimulus does not occur in the time zone of the operation pattern P00. In the time zone of the operation pattern P01, an arousal stimulus associated with the operation pattern P01 in FIG. 3 is generated, for example, light emission 1 and scent 1. Group G0 (B) is applied, for example, when the drowsiness level drops from 1 or 2 to 0, as shown in FIG. It should be noted that generating a certain awakening stimulus corresponds to operating a device capable of outputting the awakening stimulus.

The operation pattern group G1 (A) includes operation patterns P10, P11, P12, P13, and P14 in which at least one of the characteristics and the intensity of the arousal stimulus to be generated is different. When the group G1 (A) is set, the time of the operation pattern P14 is sequentially passed from the time zone of the operation pattern P10, the time zone of the operation pattern P11, the time zone of the operation pattern P12, and the time zone of the operation pattern P13. It shifts to the band one by one. When the time zone of the operation pattern P14 ends, the time zone of the operation pattern P10 is restored again. That is, while the group G1 (A) is applied, the operation patterns P11 to 14 are repeated in order. It should be noted that the awakening stimulus does not occur in the time zone of the operation pattern P10. In the time zone of the operation patterns P11 to P14, the awakening stimulus associated with each operation pattern is generated in FIG. Such group G1 (A) is applied, for example, when the drowsiness level rises from 0 to 1 as shown in FIG.

The operation pattern group G1 (B) includes operation patterns P15 and P16. When the group G0 (B) is applied, the time zone of the operation pattern P15 and the time zone of the operation pattern P16 are alternately repeated. No arousal stimulus occurs during the time zone of the operation pattern P15. In the time zone of the operation pattern P16, an arousal stimulus associated with the operation pattern P16 in FIG. 3 is generated, for example, an image display on the HUD 36. Such group G1 (B) is applied, for example, when the drowsiness level drops from 2 to 1.

The operation pattern group G2 includes operation patterns P20, P21, P22, and P23. When the operation pattern group G2 is set, the time zone of the operation pattern P20 is sequentially passed through the time zone of the operation pattern P21 and the time zone of the operation pattern P22, and then sequentially shifts to the time zone of the operation pattern P23. When the time zone of the operation pattern P23 ends, the time zone of the operation pattern P20 is restored again. No arousal stimulus occurs during the time zone of the operation pattern P20. In the time zone of the operation patterns P21 to P23, the awakening stimulus associated with each operation pattern is generated in FIG. Such group G2 is applied, for example, when the drowsiness level is determined to be 2.

The operation pattern group G3 includes operation patterns P30, P31, P32, and P33. When the group G2 is set, the time zone of the operation pattern P30, the time zone of the operation pattern P31, and the time zone of the operation pattern P32 are sequentially passed, and then the time zone of the operation pattern P33 is sequentially shifted. When the time zone of the operation pattern P33 ends, the time zone of the operation pattern P30 is restored again. No arousal stimulus occurs during the time zone of the operation pattern P30. In the time zone of the operation patterns P31 to P33, the awakening stimulus associated with each operation pattern is generated in FIG. Such an operation pattern group G3 is applied, for example, when the drowsiness level is determined to be 3.

Once the drowsiness level rises to level 3, the drowsiness level remains at 3 until the driver's resting motion is detected. Therefore, the group G3 does not automatically transition to another group. As the driver's break operation, it is possible to adopt that the shift position of the vehicle is set to the parking position, the parking brake is set to on, the driving power of the vehicle Hv is set to off, and the like. can.

The standard duration T1, which is the duration of each operation pattern, is, for example, 45 seconds. The standard duration T1 may be, for example, 30 seconds, 50 seconds, 60 seconds, 90 seconds, or the like. If the standard duration T1 is too long, the driver becomes accustomed to the stimulus output in the operation pattern, and it is difficult to obtain the awakening effect. Further, if the standard duration T1 is too short, the output mode of the stimulus is frequently changed, which may cause trouble for the driver. It is preferable that the standard duration T1 is set to 40 seconds or more and 60 seconds or less in order to secure the awakening effect and reduce the annoyance at the same time. The standard duration T1 corresponds to the first hour. The stimulus control unit F3 executes, as a basic operation, changing the operation pattern at a cycle corresponding to the standard duration T1.

In the example shown in FIG. 3, the operation patterns P00, P10, P15, P20, and P30 located at the head of the stimulus generation operation pattern are all set to rest patterns that do not output awakening stimuli, but the present invention is not limited to this. As illustrated in FIG. 5, the head pattern of each group may be set to output some kind of arousal stimulus. The pause duration, which is the standard duration T1 of the pause pattern, may be set shorter than the stimulation duration, which is the standard duration T1 of the operation pattern that generates the stimulus. For example, the pause duration can be set to a length such as 15 seconds, 20 seconds, and 30 seconds that the driver can recognize that it is a pause period and that the increase in drowsiness can be suppressed during the pause period. For example, the rest duration is set to about half of the stimulation duration.

<About the operation of the awakening device 1>
Here, the stimulus generation control process executed by the awakening device 1 will be described with reference to FIG. The stimulus generation control process is a process of controlling the operation of various stimulus generators so as to maintain the driver's awake state or lead the driver to the awake state. Here, as an example, it is assumed that the stimulus generation control process includes steps S100 to S113. Of course, the number of steps constituting the stimulus generation control process and the process order can be changed as appropriate. The stimulus generation control process shown in FIG. 6 is started when a predetermined start event occurs.

As the start event, for example, the ignition of the vehicle Hv is turned on, the vehicle starts running, the start instruction is input by the driver, and the like can be adopted. In addition, when the vehicle Hv is equipped with an automatic driving function of level 4 or higher, the remaining time from the automatic driving mode of level 4 or higher to the timing of switching to the manual driving mode is less than a predetermined threshold, which is also a start event. Can be adopted as. The remaining time until the timing of switching to the manual operation mode can be calculated based on the control plan of the remaining distance and the traveling speed until leaving the ODD such as a highway.

In addition, for example, the stimulus generation control process may be started when the drowsiness level of the driver changes from a state of 0 to a state of 1 or more by the drowsiness level determination unit F2. In that case, steps S100 to S101 can be omitted.

Further, while the stimulator 1 is executing the predetermined operation pattern, in parallel with the processing flow shown in FIG. 6, the operation reception unit F6 gives an instruction of the driver to the generation mode of the awakening stimulus, as will be described separately. The process of accepting operations is also executed at any time. The flowchart shown in FIG. 6 can be understood as a flowchart showing an outline of the operation of the awakening device 1.

First, in step S100, the sensor information acquisition unit F1 acquires sensor values of various state quantities as a material for determining the drowsiness level and moves to S101. It should be noted that the process of sequentially acquiring the drowsiness level determination material by the sensor information acquisition unit F1 is sequentially executed in parallel with the processing flow described later. Since step S100 is a step in which the sensor information acquisition unit F1 acquires information indicating the driver status, it can be called a driver status acquisition step.

In step S101, the drowsiness level determination unit F2 determines the drowsiness level of the driver based on various information acquired by the sensor information acquisition unit F1. Then, the display processing unit F4 displays the determined drowsiness level on at least one of the HUD 36 and the display 23, and proceeds to step S102. Such step S101 can be called a drowsiness level determination step.

The drowsiness level determination unit F2 sequentially determines the drowsiness level of the driver in a predetermined determination cycle such as 5 seconds, 10 seconds, and 15 seconds even after step S101. The determination result of the drowsiness level determination unit F2 is stored in, for example, the RAM 12 for a certain period of time. The determination results of a plurality of drowsiness levels having different determination times can be sorted and saved in the order of the determination time so that the latest data of the determination time is at the beginning. The storage period of the drowsiness level determination result can be, for example, 2 minutes or 5 minutes.

In step S102, the pattern selection unit F32 selects an operation pattern group according to the drowsiness level determined in step S101 from the operation pattern groups prepared in advance exemplified in FIGS. 3 and 4. For example, when the drowsiness level determined in step S101 is 0, the pattern selection unit F32 sets the operation pattern group G0 (A). When the drowsiness level determined in step S101 is level 1, the pattern selection unit F32 sets the operation pattern group G1 (A). When the drowsiness level determined in step S101 is level 2, the pattern selection unit F32 sets the operation pattern group G2. When the drowsiness level determined in step S101 is level 3 or higher, the pattern selection unit F32 sets the operation pattern group G3. In addition, when the drowsiness level is level 0, the pattern selection unit F32 selects the operation pattern group G0 (A). Such step S102 can be called a stimulation pattern selection step or an operation pattern selection step.

In step S104, the stimulus control unit F3 starts the generation of arousal stimuli according to the operation pattern group set in step S103. For example, when the operation pattern group G1 (A) is selected, the output of the awakening stimulus corresponding to the operation pattern P10 located at the head of the list of a plurality of operation patterns constituting the operation pattern group G1 (A) is started. do. Although the operation pattern P10 is used as a pause pattern here, if the operation pattern P10 includes, for example, the generation of light emission 2, a control signal is output to the light emitting device 31 to start generating light emission of a predetermined operation pattern.

If group G0 (A) is selected, the arousal stimulus is not output. When a group other than the group G0 (A) is selected, the generation of the arousal stimulus corresponding to the selected group is caused by the group G0 (A) being selected in the subsequent processing or at the end timing in steps S112 and S113. Continue until it is determined to be present. Further, in step S104, the stimulus control unit F3 activates a timer and starts measuring the elapsed time from the start of the current pattern, which is the currently executing operation pattern. Further, the display processing unit F4 uses the HUD 36 to present information regarding the operating state of the awakening device 1 or the break proposal. The information type to be presented is determined for each operation pattern as shown in the rightmost column of FIG.

In step S105, the count value of the timer is referred to, and it is determined whether or not the predetermined intermediate confirmation time T2 has elapsed since the start of the current pattern. The intermediate confirmation time T2 may be a value shorter than the standard duration T1 and is set to, for example, 30 seconds. Of course, the intermediate confirmation time T2 may be 25 seconds, 40 seconds, or the like. The intermediate confirmation time T2 is preferably set longer than half of the standard duration T1. The intermediate confirmation time T2 corresponds to the second time.

If the elapsed time from the start of the current pattern is equal to or longer than the intermediate confirmation time T2, affirmative determination is made in step S105 and the process proceeds to step S106. On the other hand, if the intermediate confirmation time T2 has not yet elapsed since the start of the current pattern, step S105 is repeated while sequentially determining whether or not the end condition is satisfied in step S112.

The termination conditions include, for example, that the driving power of the vehicle Hv has been turned off, that the vehicle has arrived at the destination, that the driver has given an termination instruction, and the like. If the end condition is satisfied, this process ends.

In step S106, it is determined whether or not the driver's state satisfies the predetermined temporary switching condition when the elapsed time reaches the intermediate confirmation time T2 after starting the current pattern, which is the operation pattern currently being executed. do. The elapsed time since the start of the current pattern can be measured by the timer function described above. Hereinafter, the time point at which the elapsed time from the start of the current pattern becomes the intermediate confirmation time T2 is also described as the intermediate confirmation time point. The temporary switching condition is a condition for temporarily changing the output mode of the arousal stimulus according to the state of the driver.

In this embodiment, as an example, it is assumed that a group switching condition and an intra-group switching condition are set as temporary switching conditions. The group switching condition is a condition for suddenly switching to an operation pattern group corresponding to a higher drowsiness level. The group switching condition can be understood as an intensity changing condition for strengthening the arousal stimulus to be generated from one viewpoint. The group switching condition may be, for example, a case where the total value of the time during which the driver is in the closed eye state in the period from the start of the current pattern to the intermediate confirmation time T2 is equal to or greater than the predetermined emergency switching threshold value. can. That is, when the total time of the closed eyes state in the period from the start of the current pattern to the intermediate confirmation time T2 is equal to or longer than the emergency switching threshold value, it is determined that the group switching condition is satisfied. The emergency switching threshold value can be 5 seconds, 10 seconds, or the like. Further, the emergency switching threshold value can be set to 15% to 30% of the intermediate confirmation time T2. The emergency switching threshold value may be set to a different value depending on the determination value of the current drowsiness level. For example, when the drowsiness level is 1, the emergency switching threshold value may be 3 seconds, while when the drowsiness level is 2, the emergency switching threshold value may be 5 seconds. It can be said that the group switching condition corresponds to the case where the current pattern does not work for the driver and the drowsiness level deteriorates.

The intra-group switching condition is a condition for switching to the next operation pattern in the currently selected group. From one viewpoint, the intra-group switching condition can be understood as a type change condition for immediately changing the type of awakening stimulus without changing the intensity of the awakening stimulus to be generated. The intra-group switching condition can be said to correspond to the case where the current pattern does not change the drowsiness level, that is, the drowsiness level has not deteriorated but has not been improved.

For example, there is no change in drowsiness level, and the posture, eye opening degree, eye movement speed, blink (blink) cycle stability, blink speed, etc. at the time of intermediate confirmation are improved compared to the start of the current pattern. If not, it is determined that the intra-group switching condition is satisfied. It should be noted that the case where the degree of eye opening is improved corresponds to the case where the degree of eye opening is increased, and the case where the posture is improved means the case where the direction of the face approaches from the downward direction or the upward direction to the front direction. This event is a sign of improved drowsiness. When there is no change in the drowsiness level at the time of the intermediate confirmation and no sign of improvement in the drowsiness level is detected, it can be determined that the intra-group switching condition is satisfied.

In step S106, if at least one of the group switching condition and the intra-group switching condition is satisfied, the process proceeds to step S107. On the other hand, if neither the group switching condition nor the intra-group switching condition is satisfied, step S106 is negatively determined and the process proceeds to step S108. If neither the group switching condition nor the intra-group switching condition is satisfied, for example, when the drowsiness level is lowered, or when the drowsiness level is improving but the level value is lowered, a significant change occurs. Refers to the case where it is not.

In step S107, the process according to the satisfied temporary switching condition is executed. For example, when the group switching condition is satisfied, the pattern selection unit F32 selects an operation pattern group one step higher than the currently selected operation pattern group. The operation pattern group one step higher is an operation pattern group whose associated drowsiness level is one step higher. For example, when the currently selected operation pattern group is G1 (A) or G1 (B), G2 is selected. If the currently selected operation pattern group is G2, G3 is selected. If the currently selected operation pattern group is G0 (A) or G0 (B), G1 (A) is selected.

Further, when the in-group switching condition is satisfied, the pattern selection unit F32 selects the next operation pattern in the currently selected operation pattern group. For example, when the currently selected operation pattern is P11, P12 is selected. If the currently selected operation pattern is P21, P22 is selected. The same applies to other cases.

When the selection process in step S107 is completed, the process returns to step S103, and the output of the arousal stimulus corresponding to the operation pattern group and operation pattern selected in step S107 is started. Then, the processes after step S104 are sequentially executed. That is, when the temporary switching condition is satisfied, the operation pattern is changed in a hurry without waiting for the expiration of the standard duration T1.

In step S108, the count value of the timer is referred to, and it is determined whether or not the standard duration T1 has elapsed since the start of the current pattern. If the elapsed time from the start of the current pattern is equal to or longer than the standard duration T1, affirmative determination is made in step S108, and the process proceeds to step S109. On the other hand, if the standard duration T1 has not yet elapsed since the start of the current pattern (step S108 NO), step S108 is repeated while sequentially determining whether or not the end condition is satisfied in step S113. Step S113 is the same process as step S112.

In step S109, the change determination unit F31 determines the change in the drowsiness level based on the latest determination value of the drowsiness level stored in the RAM 12. The change in drowsiness level is the change in the latest drowsiness level, which is the most recently determined drowsiness level, with respect to the start drowsiness level, which is the drowsiness level measured at the start of the current pattern. The start drowsiness level corresponds to the drowsiness level determined immediately after step S103. The drowsiness level used in each determination step such as step S109 may be the average value of the determination results (so-called moving average value) or the mode value within the latest predetermined time. For example, it can be the average value or the mode value of the drowsiness level within the last 20 seconds. The drowsiness level statistically determined by the drowsiness level at each time point within the latest predetermined time as a population is also referred to as a movement statistical value of the drowsiness level. According to the configuration in which the movement statistics are used to determine whether or not the drowsiness level has changed, the risk of weakening or strengthening the arousal stimulus due to the instantaneous change can be reduced.

As the judgment operation pattern of the change in drowsiness level, there are ascending, descending, and maintaining. Elevated means that the latest drowsiness level is higher than the starting drowsiness level. Decline means that the latest drowsiness level is lower than the start drowsiness level. Identical means that the latest drowsiness level is the same as the starting drowsiness level.

If the latest drowsiness level and the drowsiness level at the start are the same, the step S109 is negatively determined and the process proceeds to step S110. On the other hand, if the drowsiness level is rising or falling, the process proceeds to step S111. In step S110, the pattern selection unit F32 selects the next operation pattern of the current operation pattern group and returns to step S103.

In step S111, the pattern selection unit F32 selects the mode of arousal stimulation, that is, the operation pattern group, based on the latest drowsiness level and the change direction of the drowsiness level. If the drowsiness level is elevated, select an operation pattern group according to the latest drowsiness level. Specifically, when the latest drowsiness level after the increase in drowsiness level is 1, the pattern selection unit F32 sets the operation pattern group G1 (A). Further, when the latest drowsiness level after the increase in drowsiness level is 2, the pattern selection unit F32 sets the operation pattern group G2. When the latest drowsiness level is level 3 or higher, the pattern selection unit F32 sets the operation pattern group G3.

If the drowsiness level is decreasing, select the operation pattern group according to the latest drowsiness level. Specifically, when the latest drowsiness level after the drowsiness level drops is 1, the pattern selection unit F32 sets the operation pattern group G1 (B). Further, when the latest drowsiness level after the drowsiness level is lowered is 0, the pattern selection unit F32 sets the operation pattern group G0 (B). As described above, here, as an example, once the drowsiness level rises to level 3, the drowsiness level is maintained at level 3 regardless of the determination result of the drowsiness level. Therefore, the drowsiness level does not drop from level 3 to level 2. When the group selection process in step S111 is completed, the process returns to step S103.

<About pattern registration processing based on driver operation>
The awakening device 1 of the present embodiment includes an operation reception unit F6 for acquiring the driver's opinion on the current pattern based on the driver's operation signal detected and output by the input device 22.

The operation reception unit F6 cooperates with the display processing unit F4 to acquire the driver's opinion or instruction operation for the current pattern. For example, as shown in FIG. 7, the display processing unit F4 displays various button images (B1 to B5) for acquiring the driver's opinion on the current pattern on the display 23. Then, the operation reception unit F6 acquires the driver's opinion and instruction based on the selection operation for the button image. The button image selection operation can be detected based on the correspondence between the display position information of various button images and the user's touch position information with respect to the display 23.

Button B1 shown in FIG. 7 is a skip button, which corresponds to a button for the driver to instruct to switch to the next operation pattern. When the operation reception unit F6 detects that the skip button B1 has been touch-operated by the driver, the pattern management unit F7 registers the operation pattern in which the skip operation has been performed in the RAM 12 or the storage 13 as a skip pattern. For convenience, the operation of selecting the skip button B1 is also referred to as a skip operation.

The awakening device 1 of the present embodiment is configured to omit the execution from the next time with respect to the operation pattern registered in the skip pattern. For example, when the skip operation is performed on the operation pattern P12, the operation patterns other than the operation pattern P12 in the group G1 (A) are cyclically executed. Specifically, the operation patterns P10, P11, P13, and P14 are executed in this order, and when the time zone of the operation pattern P14 ends, the operation pattern P10 returns to the time zone again.

Note that a plurality of operation patterns belonging to one group may be registered as skip patterns. However, if all the operation patterns belonging to one group are allowed to be registered in the skip pattern, the group will not function. Further, when there is only one operation pattern belonging to one group that is not registered in the skip pattern, a single operation pattern will continue to be executed. If one operation pattern is continued endlessly, the driver becomes accustomed to the arousal stimulus provided by the operation pattern, and there is a concern that it becomes difficult to obtain the awakening effect.

Based on such circumstances, the pattern management unit F7 of the present disclosure refuses to register a new skip pattern when the number of operation patterns not registered in the skip pattern is two or less in the same group. do. Alternatively, when the number of operation patterns not registered in the skip pattern in the same group is two or less, the pattern management unit F7 discards the registration of the skip pattern having the oldest registration time and then creates a new skip pattern. May be registered. In any case, the pattern management unit F7 controls the registration state of the skip pattern so that at least two or more operation patterns are cyclically executed.

Even if the registration of the skip pattern itself is rejected, the temporary skip operation itself may be configured to be acceptable. By performing the skip operation, the effect of stimulating the user can be expected. The temporary skip operation corresponds to an operation of transitioning to the next operation pattern without setting the skip pattern.

Button B2 is a good button for registering the current pattern as a favorite operation pattern. When the operation reception unit F6 detects that the good button B2 is selected by the driver, the pattern management unit F7 registers the operation pattern being executed at that time in the favorite pattern. The favorite pattern can also be called a highly evaluated operation pattern.

Whether or not it is registered in the favorite pattern can be used as a parameter for increasing the appearance rate when the operation pattern is randomly executed, for example. Further, the operation pattern registered in the favorite pattern may have a set value of the standard duration T1 for the operation pattern longer than a normal value by a predetermined amount. The extension time can be, for example, 5 seconds or 10 seconds.

Button B3 is a low evaluation button for inputting that the current pattern is not the driver's preference. When the operation reception unit F6 detects that the low evaluation button B3 is selected by the driver, the pattern management unit F7 registers the operation pattern being executed at that time in the low evaluation pattern. The low rating button B3 can be selected, for example, if you don't like it very much or feel annoyed, although it is not enough to skip it. The pattern management unit F7 may register the operation pattern in which the skip operation is performed as a low evaluation pattern. The skip button B1 may also serve as the low evaluation button B3.

Whether or not it is registered in the low evaluation pattern can be used as a parameter for lowering the appearance rate when the operation pattern is randomly executed, for example. Further, for the operation pattern registered in the low evaluation pattern, the set value of the standard duration T1 for the operation pattern may be shorter than the normal value by a predetermined amount. The shortening amount can be, for example, 5 seconds or 10 seconds. In addition, the operation pattern registered in the low evaluation pattern may reduce the intensity of the stimulus by a predetermined amount.

Buttons B4 and B5 are buttons for adjusting the intensity of stimulation. For example, the button B4 is a down button B4 for weakening a predetermined amount of stimulation. Reducing the stimulus is equivalent to reducing the volume and frequency of voice. Further, in the case of light, it corresponds to reducing the brightness or darkening the hue. In the case of vibration, it corresponds to reducing the amplitude of vibration or lengthening the vibration interval. Further, the button B5 is an up button B5 for strengthening a predetermined amount of stimulation.

Note that some instruction operations may be configured to be acceptable based on the output signal of the steering switch. For example, the skip operation and the up operation may be detected based on the fact that a predetermined steering switch is pressed. In other words, a switch corresponding to some button images may be provided on the steering wheel. With such a configuration, the convenience of the driver can be enhanced.

Further, in the above, the mode of registering to the skip pattern, favorite pattern, low evaluation pattern, etc. based on the user operation is disclosed, but the present invention is not limited to this. By analyzing the driver's face image, comfort / discomfort may be determined, and various operation patterns may be automatically / semi-automatically registered based on the determination result of the emotion. The same can be applied to adjusting the intensity of the stimulus. For example, the operation pattern judged to be pleasant is automatically registered in the favorite pattern. Further, the operation pattern determined to be unpleasant may be automatically registered in the skip pattern and the low evaluation pattern. Note that automatic registration here refers to registration without asking for driver permission. Semi-automatic registration refers to inquiring whether to register with the driver and registering if the driver approves.

In addition, the information used as a material for acquiring the driver's emotions for arousal stimuli such as comfort / discomfort is not limited to the driver's facial expression. It may be determined whether or not the driver feels uncomfortable based on the driver's gesture, speech, and degree of improvement in drowsiness level. In addition, based on the degree of improvement in the drowsiness level, an effective pattern that is an effective operation pattern for improving the drowsiness level and a weak effect pattern that is an operation pattern that is not so effective may be specified. The awakening device 1 determines the driver's emotions (pleasant / unpleasant) for each operation pattern based on the driver operation signal input from the input device 22 and the analysis result of the driver's face image, and determines the frequency of occurrence of the operation pattern and the frequency of occurrence of the operation pattern. Execution conditions, duration, and stimulus intensity may be adjusted. The DSM 21 may be provided or the awakening device 1 may be provided with a function of analyzing the face image and determining whether or not the driver feels uncomfortable.

It is preferable that the setting information for each operation pattern is stored separately for each driver, in other words, for each user. The user as a driver may be identified based on the identification information of the key used for unlocking the vehicle Hv, the face image, the voiceprint, the fingerprint, and the like. Various methods can be used to identify the driver. Further, when the vehicle Hv is a vehicle provided for the car sharing service, the driver can be specified based on the usage reservation information of the vehicle Hv. The pattern setting information for each user may be stored in the storage 13 or may be stored in a server on the cloud.

<Effects of the embodiment>
The stimulus control unit F3 having the above configuration can change the timing of switching the stimulus pattern from the current pattern to another pattern from the originally planned time based on the state of the driver after switching to the current pattern. That is, whether or not to switch the operation pattern is changed according to the state of the driver at a predetermined timing during execution of one operation pattern. Specifically, as shown in FIG. 8, when a certain operation pattern A is being executed and the driver's state satisfies a predetermined temporary switching condition when the intermediate confirmation time T2 elapses, the pattern A To a predetermined pattern B.

For example, if the driver's eye closure time is equal to or greater than the predetermined emergency switching threshold during the period from the start of the current pattern to the elapse of the intermediate confirmation time T2, the continuation of the current pattern is interrupted and the driver is drowsy one step higher. Switch to the group corresponding to the level. Switching the operation pattern when the intermediate confirmation time T2 has elapsed corresponds to accelerating the end timing of the current pattern.

According to such a configuration, when the driver's drowsiness is progressing, the intensity of the arousal stimulus given to the driver can be promptly increased. As a result, it is possible to suppress an increase in the drowsiness level of the driver.

Further, in the above configuration, when the switching condition in the group is satisfied, that is, when the improvement of the drowsiness level is not observed at the timing when the intermediate confirmation time T2 has elapsed from the start of the current pattern, the sleepiness level is not observed in the same group. Switch to another operation pattern. This configuration corresponds to a configuration in which if the current pattern does not produce an effect, it is interrupted in the middle and an awakening stimulus of another operation pattern within the same group is output. According to this configuration, it is possible to shorten the duration of the operation pattern that does not contribute to arousal. As a result, it is possible to shorten the time required to reach the operation pattern that contributes to the driver's awakening, and it becomes easier to lead the driver to the awake state.

In addition, if the temporary switching condition is not satisfied at the time of the intermediate confirmation, the operation pattern is switched to another operation pattern when the standard duration T1 has elapsed. Therefore, the cycle in which each operation pattern is switched is basically the standard duration T1. Since the standard duration T1 is set to be relatively longer than the intermediate confirmation time T2, it is possible to reduce the risk of causing trouble to the driver due to frequent switching of the operation pattern.

In addition, when the drowsiness level rises from level 1 to level 2 while the operation pattern group G1 (A) is set, the awakening device 1 newly sets the operation pattern group G2. The working pattern group G2 includes arousal stimuli of cold air not included in the previous group. The previous group is an operation pattern group before the transition, and here refers to the group G1 (A). The cold wind arousal stimulus corresponds to an additional awakening stimulus. The driver will experience additional arousal stimuli and will be more likely to notice that the arousal stimuli change as the drowsiness level rises. Further, the operation pattern group G2 is set to generate an awakening stimulus of cold air in some time zones. According to the setting mode in which the cold air is temporarily generated, the driver is less likely to feel the awakening stimulus annoyingly as compared with the case where the awakening stimulus of the cold air is generated during all the periods constituting the operation pattern group.

Further, when the drowsiness level rises from level 2 to level 3 while the operation pattern group G2 is set, the awakening device 1 newly sets the operation pattern group G3. The working pattern group G3 includes a vibrating arousal stimulus as an additional arousal stimulus. The working pattern group G2 corresponds to the aspect before the drowsiness level rises. By experiencing the vibration as an additional wakefulness stimulus, the driver can easily notice that the wakefulness stimulus changes as the drowsiness level rises.

Further, when the drowsiness level drops from level 2 to level 1 while the operation pattern group G2 is set, the awakening device 1 newly sets the operation pattern group G1 (B). The working pattern group G1 (B) includes a light stimulus by HUD36 instead of the arousal stimulus included in the working pattern group G2. Photostimulation by HUD36 corresponds to new arousal stimuli not included in the previous group. When the operation pattern group G1 (B) is set, the driver feels that the arousal stimulus has changed significantly, and notices that the arousal stimulus changes as the drowsiness level decreases. That is, it can be recognized that the system follows the driver's state change to some extent. As a result, even if the arousal stimulus continues after the drowsiness level drops, the risk of the driver feeling annoyed can be reduced.

By the way, the operation pattern group G2 includes a light stimulus by the light emitting device 31. The light stimulus by the HUD 36 is the same type of awakening stimulus as the light stimulus by the light emitting device 31, but can be said to be an awakening stimulus having different characteristics such as appearance and generation position. Due to the different characteristics, the driver can experience changes in the arousal stimulus.

In addition, in the setting example shown in FIG. 3, the operation pattern groups G0 (A), G0 (B), G1 (A), G1 (B), G2, and G3 include a resting operation pattern in which arousal stimulation does not occur. By including the rest pattern, the arousal stimulus provided by other operation patterns stands out, and a further awakening effect can be expected while reducing the risk of causing annoyance.

Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments, and can be variously modified and implemented. For example, the configuration described as a supplement below can also be implemented in combination with the above-described embodiment.

<Supplement of system response according to the driver status at the time of intermediate confirmation>
In the above-described embodiment, the mode of switching the operation pattern at the time of intermediate confirmation based on the condition that the temporary switching condition is satisfied has been disclosed, but the present invention is not limited to this. The duration of the current pattern may be extended by a predetermined time from the standard duration T1 based on the improvement in the drowsiness level of the driver at the time of the interim confirmation or the observation of a sign thereof.

For example, the pattern management unit F7 is currently in the case where the drowsiness level of the driver is improved or a sign thereof is observed based on the history information of the driver state from the start of the current pattern to the intermediate confirmation time. Register the pattern as a valid pattern. Then, when the current pattern is registered in the effective pattern, the stimulus control unit F3 initially determines the end timing of the current pattern based on the standard duration T1 on condition that the user does not feel uncomfortable with the current pattern. It will be later than planned.

The effective pattern corresponds to the operation pattern determined to be effective for the awakening of the driver. Whether or not it is an effective pattern can be determined, for example, by comparing the drowsiness level and the driver state at the start and end of the operation pattern. The stimulus control unit F3 may determine whether or not the current pattern is effective for the driver's awakening, instead of the pattern management unit F7. The functional arrangement in the awakening device 1 can be changed as appropriate.

Further, even if the current pattern is registered in the favorite pattern by the intermediate confirmation time based on the driver operation signal input from the input device 22 and the driver's face image, the duration of the current pattern is set from the standard duration T1. May be extended for a predetermined time. The extension time can be, for example, 10 seconds.

<Supplement to specific examples of temporary switching conditions>
The group switching condition and the intra-group switching condition are not limited to the above-mentioned examples. For example, the group switching condition can be a case where the number of times the driver yawns in the period from the start of the current pattern to the intermediate confirmation time T2 is, for example, three times or more, which is equal to or greater than a predetermined threshold value. .. That is, when the number of yawning executions in the period from the start of the current pattern to the intermediate confirmation time T2 is equal to or greater than a predetermined value, it is determined that the group switching condition is satisfied.

The switching condition within the group can be a case where the judgment value of the drowsiness level is constant in the period from the start of the current pattern to the intermediate confirmation time T2. That is, when the determination value of the drowsiness level is constant in the period from the start of the current pattern to the intermediate confirmation time T2, it is determined that the intra-group switching condition is satisfied.

<Supplement to the switching timing of the operation pattern group>
Even if the drowsiness level of the driver rises at the timing when the intermediate confirmation time T2 has elapsed from the start of the current pattern, the current pattern may be maintained until the standard duration T1 elapses. However, in that case, even if the drowsiness level returns to the initial level at the timing when the standard duration T1 has elapsed, the operation pattern group corresponding to the highest drowsiness level observed on the way is switched. According to such a configuration, even if the drowsiness level is temporarily lowered at the timing when the standard duration T1 has elapsed, a relatively strong arousal stimulus can be output. The above assumed scene corresponds to a state in which the drowsiness level is increasing or decreasing. The rising and falling drowsiness levels suggest that the driver's drowsiness is located near the boundaries of those levels. When the drowsiness level changes near such a boundary, the effect of lowering the drowsiness level of the driver can be expected by outputting the arousal stimulus corresponding to the relatively higher drowsiness level.

<Change of cold air stimulation>
In addition, the above configuration has an air-conditioning stimulus pattern that outputs air-conditioned air from the air-conditioning device mounted on the vehicle as a stimulus pattern, and a window portion that applies outside air as an awakening stimulus to the driver by opening the door window for the driver's seat. It corresponds to the configuration that can adopt the open pattern. Based on the determination that the drowsiness level has not improved at the timing when the intermediate confirmation time T2 has elapsed while the stimulus control unit F3 is executing the air conditioning stimulus pattern, the window opening pattern is used instead of the air conditioning stimulus pattern from the next time. May be configured to perform. The driver's feelings can be different between the air-conditioned air and the external wind, which is the wind from the door window. Therefore, when the stimulation of the air-conditioned air does not work, it can be expected that the drowsiness level of the driver is improved by adopting the external wind as the stimulation of awakening. If it is presumed that the driver is not good at stimulating the air conditioning due to the facial expression of the driver or the like, a window opening pattern may be adopted instead of the air conditioning stimulating pattern. According to this configuration, it becomes easy to accept even a driver who is not good at air-conditioned air.

It is preferable that the window opening pattern is controlled so as not to be automatically adopted in rainy weather. In addition, in the season when the amount of fine particles such as pollen, yellow sand, and PM2.5 contained in the air (so-called scattering amount) exceeds a predetermined threshold, the window opening pattern is automatically excluded from the options. Is preferable. If the weather conditions satisfy the above conditions, the driver may be configured to present a confirmation screen as to whether or not the window opening pattern may be executed. Whether or not the window opening pattern may be automatically executed may be registered by the driver via a predetermined setting screen.

In addition, at drowsiness level 2, cold air may be output from the air conditioner 34 as an awakening stimulus, while at drowsiness level 3, the driver may be given a tactile stimulus having different characteristics by opening the door window of the driver's seat. If the door window opens suddenly, the driver may be confused. Therefore, when adopting the opening of the door window as an arousal stimulus, it is preferable to notify the driver in advance.

<Regarding the range of influence of skip operation>
When a skip operation is input, the pattern selection unit F32 may skip each group. For example, when the operation pattern group is set to G1 (A) and the skip operation is performed, the operation pattern group may be switched from G1 (A) to G2. Further, the operation receiving unit F6 may be configured to be able to receive a group skip operation for changing the operation pattern group itself and an individual skip operation for changing the operation pattern within the same operation pattern. For example, a button image corresponding to a group skip operation and a button image for accepting an individual skip operation may be displayed on the display 23, respectively, and configured to detect those instruction operations based on the user's touch position. .. The group skip operation and the individual skip operation may be configured to be detected by voice input.

<Awakening stimulus (operation pattern) judged to be unpleasant>
Based on the driver operation signal input from the input device 22 and the driver's face image, control different from the pleasantly determined operation pattern may be applied to the operation pattern determined to be unpleasant. For example, the stimulus control unit F3 may not execute the operation pattern determined to be unpleasant until the driver gives permission. Further, the stimulus of the operation pattern determined to be unpleasant may be output in a mode in which the intensity is suppressed.

In addition, the operation pattern determined to be unpleasant may be configured not to be executed when the drowsiness level is less than the predetermined value, but to be executed when the drowsiness level is equal to or higher than the predetermined value. The predetermined value here can be 3 or 2. For example, when the drowsiness level is 0 to 1, the operation pattern determined to be unpleasant may not be executed, and the operation pattern determined to be unpleasant may be executed when the drowsiness level is 2 or more. Execution / non-execution of a certain operation pattern can be realized by automatically / manually editing the operation pattern group. For example, the above-mentioned control does not include the operation pattern determined to be unpleasant in the operation pattern group corresponding to the drowsiness level of 0 to 1, while the operation pattern group having the drowsiness level of 2 or more includes the operation pattern determined to be unpleasant. Corresponds to that. Each actuation pattern group may be configured such that the higher the drowsiness level, the higher the proportion of actuation patterns determined to be unpleasant.

The operation pattern group G3 corresponding to the drowsiness level 3 or higher may be composed of only the operation patterns judged to be unpleasant. According to the configuration in which the appearance ratio of the operation pattern judged to be unpleasant is increased as the drowsiness level is increased, the effect of further lowering the drowsiness level of the driver can be expected.

<About response processing to driver operations>
The awakening device 1 may be configured to temporarily stop the output of the awakening stimulus based on a predetermined operation by the driver. In that case, the awakening device 1 accepts the restart operation of the driver, or restarts the output of the awakening stimulus based on the elapse of the predetermined pause release time.

Further, the awakening device 1 may be configured so that the intensity of the awakening stimulus can be temporarily weakened based on the driver operation. In that case, the awakening device 1 may accept the driver's restoration operation, or the awakening device 1 may restore the intensity of the arousal stimulus based on the fact that the drowsiness level is not improved.

When the drowsiness level of the driver is equal to or higher than the predetermined operation invalid threshold value, it is preferable that the driver operation for stopping the output of the arousal stimulus or weakening the intensity is canceled. Further, when the drowsiness level of the driver is equal to or higher than the operation invalidity threshold value, the skip operation may be canceled. The operation invalid threshold value can be, for example, 3 or 4. Even when the drowsiness level is equal to or higher than the predetermined operation invalid threshold value, the driver operation in the direction of increasing the stimulus intensity, the favorite registration operation, and the like may be configured to be acceptable.

<About automatic change processing of arousal stimulus>
The pattern management unit F7 may specify a weak effect pattern, which is an operation pattern in which the arousal effect is weak, by comparing the drowsiness level and the driver state at the start and end of the current pattern. For example, the driver state at the start and the driver state at the end can be compared, and an operation pattern in which no significant change in the improvement direction is observed can be registered as a weak effect pattern. The driver state here is not limited to the determination value of the drowsiness level, but can also include the posture, the degree of eye opening, the movement speed of the line of sight, the stability of the blink (blink) cycle, the speed of the blink, and the like.

The stimulus control unit F3 may increase the intensity of the stimulus or combine another type of arousal stimulus with respect to such a weak effect pattern. Further, the stimulus control unit F3 may change the execution frequency, the execution condition, and the standard duration T1 for the weak effect pattern. For example, the stimulus control unit F3 may shorten the standard duration T1 by a predetermined amount for the operation pattern determined to be a weak effect pattern, as compared with other operation patterns. Further, the stimulus control unit F3 may adopt the operation pattern determined to be a weak effect pattern less frequently than other operation patterns. For example, it may be controlled to be executed once every two laps.

In addition, the execution condition may include that the elapsed time from the selection of the current operation pattern group is within a predetermined time. As such, according to the configuration including the elapsed time from the selection of the operation pattern group within a predetermined time as the execution condition, the weak effect pattern is also executed in the initial stage after the group is selected. On the other hand, when a predetermined time has elapsed from the group selection, only the operation pattern other than the weak effect pattern is executed. According to such a configuration, only the operation pattern having a relatively strong effect is executed as time goes by, and the effect of lowering the drowsiness level of the driver can be expected.

Further, the awakening device 1 may randomly arrange a completely different operation pattern at a position (time zone) recognized as a weak effect operation pattern. By randomly adopting such an awakening stimulus that is not normally used, the fear that the driver gets used to the awakening stimulus can be reduced. In addition, by randomly adopting an awakening stimulus that is not normally used (that is, irregular), the driver can be surprised and a higher awakening effect can be expected.

<Regarding the intensity adjustment of the stimulus>
When the skip operation is performed, the display processing unit F4 may present an option for the driver to input the reason. As options for skipping reasons, it is assumed that the stimulus type is disliked, the stimulus is too weak, and the stimulus is too strong. When the skip reason can be acquired based on the user operation, the pattern management unit F7 saves the skip reason in association with the operation pattern. The stimulus control unit F3 may execute the operation pattern in a mode in which the stimulation intensity is increased by a predetermined amount for the operation pattern for which the stimulus is too weak as the reason for skipping. Further, the stimulus control unit F3 may execute the operation pattern in a mode in which the stimulus intensity is weakened by a predetermined amount for the operation pattern for which the stimulus is too strong as the reason for skipping.

<About the configuration of the operation pattern for each group>
The combination of operation patterns and the execution order of each group can be changed as appropriate. For example, in the example shown in FIG. 3, the pause pattern is arranged at the head of each group, but the present invention is not limited to this. The pause pattern may be placed at the end of the group instead of at the beginning. According to the configuration in which the rest pattern is placed at the beginning, the driver can notice that the drowsiness level and the mode have changed by experiencing the time when the arousal stimulus does not occur.

It is preferable that adjacent operation pattern groups include non-overlapping types of arousal stimuli. The actuation pattern group G1 (B) may include a type of arousal stimulus not included in the actuation pattern group G2. Examples of different types of arousal stimuli include dialogue awakening stimuli, vibration awakening stimuli, voice awakening stimuli, and the like. In the above case, since the operation pattern group G1 (B) includes different types of arousal stimuli, the driver can experience the change in the arousal stimuli. As a result, it is possible to reduce the risk that the driver will become accustomed to the arousal stimulus. As a result, the risk that the drowsiness level that is about to fall will rise again can be reduced.

<Addition (1)>
The awakening device 1 and its method described in the present disclosure may be realized by a dedicated computer constituting a processor programmed to perform one or more functions embodied by a computer program. Further, the apparatus and the method thereof described in the present disclosure may be realized by using a dedicated hardware logic circuit. Further, the apparatus and method thereof described in the present disclosure may be realized by one or more dedicated computers configured by a combination of a processor for executing a computer program and one or more hardware logic circuits. Further, the computer program may be stored in a computer-readable non-transitional tangible recording medium as an instruction executed by the computer. That is, the means and / or functions provided by the awakening device 1 and the like can be provided by the software recorded in the actual memory device and the computer, software only, hardware only, or a combination thereof that execute the software. For example, some or all of the functions included in the awakening device 1 may be realized as hardware. A mode in which a certain function is realized as hardware includes a mode in which one or more ICs are used. The awakening device 1 may be realized by using an MPU, a GPU, or a DFP (Data Flow Processor) instead of the CPU. The awakening device 1 may be realized by combining a plurality of types of arithmetic processing devices such as a CPU, an MPU, and a GPU. The awakening device 1 may be realized as a system-on-chip (SoC). Further, various processing units may be realized by using FPGA (Field-Programmable Gate Array) or ASIC (Application Specific Integrated Circuit). Various programs may be stored in a non-transitionary tangible storage medium. As a program storage medium, various storage media such as HDD (Hard-disk Drive), SSD (Solid State Drive), flash memory, and SD (Secure Digital) card can be adopted. The non-transitional substantive recording medium also includes a ROM such as EPROM (Erasable Programmable Read Only Memory).

The plurality of functions possessed by one component in the above embodiment may be realized by a plurality of components, or one function possessed by one component may be realized by a plurality of components. Further, a plurality of functions possessed by the plurality of components may be realized by one component, or one function realized by the plurality of components may be realized by one component. In addition, a part of the configuration of the above embodiment may be omitted. Further, at least a part of the configuration of the above embodiment may be added or replaced with the configuration of the other above embodiment.

In addition to the above-mentioned awakening device 1, various forms such as a system having the awakening device 1 as a component are also included in the scope of the present disclosure. For example, a program for making a computer function as an awakening device 1, a non-transitional actual recording medium such as a semiconductor memory in which this program is recorded, and the like are also included in the scope of the present disclosure.

Claims

A driver status acquisition unit (F1) that acquires information indicating the driver status from at least one of a sensor mounted on the vehicle and a sensor mounted on the driver's body, and a driver status acquisition unit (F1).
Based on the information indicating the state of the driver acquired by the driver state acquisition unit, the drowsiness level determination unit (F2) for determining the drowsiness level of the driver and the drowsiness level determination unit (F2)
Awakening stimulus is generated by the stimulus pattern corresponding to the drowsiness level determined by the drowsiness level determination unit among a plurality of prepared stimulus patterns in which at least one of the characteristics and the intensity of the awakening stimulus to be generated is different. It is equipped with a stimulus control unit (F3) to make it
The stimulus control unit
It is configured to sequentially switch and execute a plurality of types of the stimulation patterns.
Awakening configured to adjust the timing of switching the stimulus pattern from the current pattern to another stimulus pattern based on the state of the driver after switching to the current pattern which is the stimulus pattern currently being executed. Device.
The awakening device according to claim 1.
The stimulus control unit
While the basic operation is to change the stimulation pattern every cycle having a predetermined first hour length, while performing the operation.
The timing for changing the stimulus pattern is adjusted based on the state of the driver at the timing when the second time shorter than the first time has elapsed since the current pattern, which is the stimulus pattern currently being executed, has started. Awakening device configured to.
The awakening device according to claim 2.
The driver state acquisition unit is configured to acquire eye opening degree information indicating the degree of eye opening of the driver, which is determined based on an image captured by a camera installed in the vehicle interior.
The stimulus control unit
Based on the eye opening degree information acquired by the driver state acquisition unit, the eye closed state in which the driver closes his eyes within the period from the start of the current pattern to the elapse of the second time. Calculate the total time of
An awakening device configured to change the stimulation pattern without waiting for the lapse of the first time when the total time of the closed eyes is equal to or greater than a predetermined threshold value.
The awakening device according to claim 3.
A plurality of the stimulation patterns are prepared for each of the drowsiness levels.
The stimulus control unit is an arousal device configured to switch to the stimulus pattern associated with the higher drowsiness level when the total time of the closed eye state is equal to or longer than a predetermined threshold value.
The awakening device according to any one of claims 2 to 4.
A plurality of the stimulation patterns are prepared for each drowsiness level.
The stimulus control unit
Whether or not the drowsiness level of the driver is improved at the timing when the second time has elapsed from the start of the current pattern based on the information indicating the state of the driver acquired by the driver state acquisition unit. Judging,
If the drowsiness level is not improved, it is configured to switch to another stimulus pattern associated with the same drowsiness level as the current pattern without waiting for the passage of the first time. Awakening device.
The awakening device according to any one of claims 2 to 5.
The stimulus control unit
Based on the information indicating the state of the driver acquired by the driver state acquisition unit, the drowsiness level of the driver is higher than the drowsiness level by the time when the second time elapses from the start of the current pattern. Even if it rises to, the current pattern is maintained until the first time elapses, and at the timing when the first time elapses, it corresponds to the drowsiness level observed until the second time elapses. An arousal device configured to switch to another attached stimulus pattern.
The awakening device according to any one of claims 2 to 6.
A plurality of the stimulation patterns are prepared for each drowsiness level.
The stimulus that suits the driver's taste among the plurality of stimulus patterns based on at least one of the driver's operation signal input from a predetermined input device and the analysis result of the driver's face image. It is equipped with a pattern management unit (F7) that identifies favorite patterns that are patterns.
The stimulus control unit is at least one of the execution frequency of the stimulus pattern, the execution conditions, and the standard duration as the first time, depending on whether or not the pattern management unit determines that the favorite pattern is used. An awakening device that is configured to change one.
The awakening device according to any one of claims 2 to 6.
A plurality of the stimulation patterns are prepared for each drowsiness level.
Based on the history of the driver's state acquired by the driver state acquisition unit, a pattern for specifying an effective pattern, which is the stimulus pattern effective for improving the drowsiness level of the driver, from among a plurality of the stimulus patterns. Equipped with a management department (F7)
The stimulus control unit is at least one of the execution frequency of the stimulus pattern, the execution conditions, and the standard duration as the first time, depending on whether or not the pattern management unit determines that the effective pattern is used. An awakening device that is configured to change one.
The awakening device according to any one of claims 2 to 8.
A plurality of the stimulation patterns are prepared for each drowsiness level.
Based on the history information of the driver's state acquired by the driver state acquisition unit, a weak effect pattern, which is the stimulus pattern that does not contribute to the improvement of the drowsiness level of the driver, is determined from the plurality of stimulus patterns. Equipped with a pattern management unit (F7)
The stimulus control unit is at least one of the execution frequency of the stimulus pattern, the execution conditions, and the standard duration as the first time, depending on whether or not the pattern management unit determines the weak effect pattern. An awakening device configured to change one.
The awakening device according to any one of claims 1 to 9.
A plurality of the stimulation patterns are prepared for each drowsiness level.
It is equipped with an operation reception unit (F6) that accepts user operations for the current pattern.
The operation reception unit is configured to be able to receive a skip operation for terminating the current pattern and switching to another stimulus pattern as the user operation.
The stimulus control unit
It is configured to cyclically execute a plurality of the stimulation patterns corresponding to the drowsiness level determined by the drowsiness level determination unit.
When the skip operation is accepted, the current pattern is terminated, the current pattern is switched to another stimulation pattern corresponding to the current drowsiness level, and the stimulus pattern is switched to.
From the next time, the awakening device configured to cyclically execute the stimulus pattern other than the stimulus pattern on which the skip operation is performed among the plurality of the stimulus patterns.
The awakening device according to claim 10.
A pattern management unit (F7) for registering the stimulus pattern for which the skip operation has been performed as a skip pattern is provided.
The pattern management unit is an awakening device configured to manage the registration state of the skip pattern so that the number of the stimulation patterns not registered in the skip pattern is 2 or more for each drowsiness level.
The awakening device according to any one of claims 1 to 11.
A plurality of the stimulation patterns are prepared for each drowsiness level.
Among the plurality of stimulation patterns, the air conditioning stimulation pattern that outputs air conditioning air from the air conditioning device mounted on the vehicle and the outside air as the awakening stimulation by opening a predetermined amount of the door window for the driver's seat are given to the driver. The window opening pattern to hit is included,
As the current pattern, a pattern for determining whether or not the drowsiness level is improved by the air conditioning stimulation pattern based on the information indicating the state of the driver acquired by the driver state acquisition unit while executing the air conditioning stimulation pattern. Equipped with a management department (F7)
An awakening device configured to execute the window opening pattern instead of the air conditioning stimulation pattern based on the determination by the pattern management unit that the drowsiness level is not improved by the air conditioning stimulation pattern.
Obtaining information indicating the state of the driver from at least one of the sensor mounted on the vehicle and the sensor mounted on the driver's body (S100), and
Determining the drowsiness level of the driver based on the acquired information indicating the state of the driver (S101, S104) and
Among a plurality of prepared stimulus patterns in which at least one of the characteristics and the intensity of the wakefulness stimulus to be generated is different, the wakefulness stimulus is generated by the stimulus pattern according to the determined drowsiness level (S103). ,
To sequentially switch and execute a plurality of types of the stimulation patterns (S108 to S111), and
To change the timing of switching the stimulus pattern from the current pattern to another stimulus pattern based on the state of the driver after switching to the current pattern which is the stimulus pattern currently being executed (S105 to S107). Arousal stimulus control methods, including.