JP2022179108A - Awakening support system - Google Patents

Abstract

To provide an awakening support system capable of reducing a possibility of impairing comfort of occupants.SOLUTION: An in-vehicle system sequentially determines a drowsiness level and a fatigue level based on biological information and driving hours of a driver. An awakening support system Sy1 applies awakening stimulus such as light and cool air to the driver based on the event that the drowsiness level reaches a prescribed value or more. A fatigue mitigating system Sy2 applies fatigue mitigating stimulus alternately repeating warm heat and cooling to the driver based on the event that the fatigue level reaches a prescribed value or more. When both of the awakening support system Sy1 and the fatigue mitigating system Sy2 are operated, a mediation part F53 imposes a restriction on the operation of the fatigue mitigating system Sy2. For example, the mitigation part forbids an output of warm heat having contradictory attributes to cold air, as the awakening stimulus.SELECTED DRAWING: Figure 3

Description

TECHNICAL FIELD The present disclosure relates to a wake-up assistance system that outputs wake-up stimuli according to the drowsiness level of an occupant.

Patent Literature 1 discloses an awakening device that sequentially outputs a plurality of vibration patterns when drowsiness or lethargy of a user is detected.

Japanese Patent Application Laid-Open No. 2002-200001 discloses a fatigue relief device that provides a user with thermal stimulation by driving a heater built into a seat for the purpose of relieving fatigue of the user.

JP 2018-55527 A JP 2007-209446 A

Various stimuli such as cold wind, vibration, and light are assumed as stimuli for arousal. In the awakening device, a plurality of operation patterns with different stimulus strengths are prepared, and the operation pattern corresponding to the drowsiness level of the driver can be applied. On the other hand, as a stimulus for alleviating fatigue, heat and the like are assumed.

If a fatigue relief device that mainly outputs heat and a fatigue relief device that can output cold air as a kind of awakening stimulus are operated independently, the comfort of the occupants may be impaired due to the simultaneous output of contradictory stimuli. .

The present disclosure has been made based on the above studies, and one of its purposes is to provide an awakening support system capable of reducing the risk of impairing the comfort of the passenger.

The first awakening support system disclosed herein includes a drowsiness determination unit (F3) for estimating the drowsiness level of a passenger seated in a predetermined seat, and a plurality of types of predetermined sleepiness levels for improving the drowsiness level of the passenger. An awakening support unit (Sy1) that sequentially or parallelly generates awakening stimuli, and a fatigue alleviating unit (Sy2) that sequentially or parallelly generates a plurality of types of predetermined fatigue alleviating stimuli for improving the fatigue state of the occupant. and an arbitration unit (F53) that arbitrates the respective operations of the awakening support unit and the fatigue alleviation unit, and the arbitration unit is configured to change the operation mode of the fatigue alleviation unit according to the drowsiness level. .

According to the system described above, the operation of the fatigue relief section, and thus the mode of outputting the fatigue relief stimulus, is changed according to the drowsiness level. Accordingly, it is possible to reduce the possibility that conflicting stimuli are simultaneously output from the awakening support unit and the fatigue alleviation unit. Along with this, it is possible to reduce the risk of impairing the comfort of the occupant.

In addition, the second awakening support system disclosed herein includes a fatigue determination unit (F4) that determines the fatigue level of the occupant seated in a predetermined seat, and a plurality of types of sensors for improving the drowsiness level of the occupant. An awakening support unit (Sy1) that sequentially or parallelly generates predetermined awakening stimuli, and a fatigue relief unit (Sy2) that sequentially or parallelly generates a plurality of types of predetermined fatigue relief stimuli for improving the fatigue level of the occupant. ), and an arbitration unit (F53) that arbitrates the respective operations of the awakening support unit and the fatigue alleviation unit, and the arbitration unit changes the control mode of the awakening support unit according to the fatigue level of the occupant. It is configured.

According to the system described above, the operation of the awakening support unit and, in turn, the output mode of the awakening stimulus are changed according to the fatigue level, thereby reducing the possibility that conflicting stimuli are simultaneously output from the awakening support unit and the fatigue alleviating unit. As a result, it is possible to reduce the risk of impairing the comfort of the passenger.

It should be noted that the symbols in parentheses described in the claims indicate the corresponding relationship with specific means described in the embodiments described later as one aspect, and limit the technical scope of the present disclosure. is not.

1 is a block diagram showing an example of the overall configuration of an in-vehicle system IvS; FIG. 3 is a schematic diagram for explaining types of seat heaters 35. FIG. 2 is a functional block diagram of the stimulus control device 1; FIG. FIG. 10 is a diagram showing an example of an operation pattern that can be selected by the awakening support control unit F51; FIG. 10 is a diagram for explaining the operation of the awakening support system Sy1 when the first mode Md1 is applied; FIG. 10 is a diagram for explaining the operation of the awakening support system Sy1 when the second mode Md2 is applied; FIG. 4 is a diagram for explaining the operation of the fatigue relaxation system Sy2; FIG. 4 is a diagram for explaining an example of control of the operation of each system according to a combination of drowsiness level and fatigue level; FIG. 4 is a diagram for explaining an example of control of the operation of each system according to a combination of drowsiness level and fatigue level; FIG. 11 is a diagram for explaining the operation of an arbitration unit F53; It is a figure which shows an example of the apparatus which can output the stimulus of an attribute common to a wakefulness support system and a fatigue relief system, and its control mode. FIG. 10 is a diagram showing an example of control when the awakening support system Sy1 and the fatigue alleviation system Sy2 are alternately operated.

Embodiments of the present disclosure will be described below with reference to the drawings. FIG. 1 is a diagram showing an example of a schematic configuration of a stimulus control device 1 according to the present disclosure.

<Introduction>
The stimulus control device 1 of the present disclosure is a device that integrally controls the output mode of an arousal stimulus, which is a stimulus for leading the driver to an arousal state, and a fatigue-relieving stimulus for relieving the driver's fatigue, It is installed and used in various vehicles. Both the arousal stimulus and the fatigue alleviating stimulus are stimuli that the driver can perceive through sensory organs corresponding to sight, touch, pressure, pain, cold, warmth, hearing, and smell. For example, vibration, light, heat, cold wind, etc., can serve as arousal stimulus or fatigue alleviation stimulus depending on the output mode thereof.

Note that the driver in the present disclosure refers to, for example, a person sitting in the driver's seat, that is, an occupant in the driver's seat. The expression "driver" is not limited to the person who actually carries out part or all of the driving operation. The description of the driver refers to a person who should receive driving operation authority from the automatic driving system during automatic driving. Further, the vehicle to which the present disclosure is applied may be a remotely controlled vehicle that is remotely controlled by an operator present outside the vehicle. The operator here refers to a person who has the authority to remotely control the vehicle from outside the vehicle. The stimulus control device 1 of the present disclosure is also applicable to cockpit systems that include operator seats. The concept of drivers can include operators.

<Regarding the system configuration including the stimulus control device 1>
As shown in FIG. 1, the stimulus control device 1 is used in connection with various in-vehicle devices such as the DSM 21 . For example, the stimulation control device 1 is connected to a DSM 21, an input device 22, a display 23, a HUD 24, a light emitting device 31, a speaker 32, an aroma shooter 33, an air conditioner 34, a seat heater 35, an SVS 36, a massager 37, and a vibration generator 38. ing. DSM as a member name is an abbreviation for Driver Status Monitor, and HUD means Head-Up Display. SVS is an abbreviation for Seat Ventilation System. The stimulus control device 1 is configured to be able to communicate with the wearable device 40 via the communication device 39 .

The stimulation control apparatus 1 is also connected to various sensors/devices (not shown in FIG. 1) via an in-vehicle network Nw, which is a communication network built in the vehicle. For example, the stimulus control device 1 receives detection results from various in-vehicle sensors via an in-vehicle network Nw.

Vehicle-mounted sensors include, for example, various sensors for detecting vehicle conditions such as vehicle speed, acceleration, steering angle, shift position, accelerator depression amount, and brake depression amount. In-vehicle sensors also include sensors/switches that detect the operating state of the parking brake and the power state of the vehicle. A detection signal of the seating sensor for each seat is also input to the stimulus control device 1 . A seat sensor is a sensor that detects that a person is seated, and is provided for each seat, for example. The seating sensors can be, for example, pressure sensors embedded in the seating surface of each seat. The seating sensor may be a camera that captures an image of the interior of the vehicle, or may be a millimeter wave radar. A seat belt sensor that detects the wearing state of the seat belt can also be used as the seating sensor.

The stimulus control device 1 and various in-vehicle devices may be connected by a dedicated line, or may be connected via an in-vehicle network Nw. Further, an ECU (Electronic Control Unit) may be interposed between the stimulus control device 1 and the in-vehicle device. For example, the stimulus control device 1 may be connected to each of the aroma shooter 33, the air conditioner 34, the seat heater 35, and the SVS 36 via an air conditioning ECU. For convenience, the configuration including the stimulation control device 1 and the on-vehicle device connected to the stimulation control device 1 is referred to as an on-vehicle system IvS.

The DSM 21 is a device that sequentially detects the user's state based on the user's face image. The DSM 21 includes, for example, a near-infrared light source, a near-infrared camera, and a control module for controlling them. The DSM 21 is installed, for example, on the upper surface of the steering column or the upper surface of the instrument panel, with the near-infrared camera facing the direction in which the headrest of the driver's seat exists. The DSM 21 uses a near-infrared camera to photograph the driver's head irradiated with near-infrared light from the near-infrared light source. An image captured by the near-infrared camera is image-analyzed by the control module. The control module extracts driver state information, which is information indicating the state of the driver, such as the degree of opening of the driver's eyes, from the captured image input from the near-infrared camera. The DSM 21 outputs driver state information extracted from the driver's facial image to the stimulus control device 1 .

The driver state information includes, for example, the orientation of the driver's face, the direction of the line of sight, the degree of opening of the eyes, the degree of opening of the mouth, the posture, and the like. The degree of opening of the eyes can be rephrased as the degree of opening of the eyelids. The information indicating the degree of eye opening corresponds to eye opening degree information. Further, the DSM 21 may be configured to detect sleeping, looking aside, the occurrence of illness, etc., based on at least one of changes in eye opening degree over time, facial expression, and face direction. good. Driver status information may include sleeping, looking aside, feeling unwell, and the like. Additionally, the DSM 21 may detect yawning based on the pattern of changes in mouth opening over time. Note that the stimulus control device 1 may have the function of detecting the state of the driver based on the image analysis. In that case, the DSM 21 may be configured to output the driver's face image to the stimulus control device 1 . The functional arrangement between the DSM 21 and the stimulus control device 1 can be changed as appropriate.

The input device 22 is an operating member for receiving a driver's instruction to the stimulus control device 1 . The input device 22 may be a mechanical switch (so-called steer switch) provided on the spoke portion of the steering wheel, or may be a voice input device that recognizes the content of the driver's speech. Also, the input device 22 may be a touch panel laminated on the display panel of the display 23 . For the display 23, for example, a center display provided in the center region of the instrument panel in the vehicle width direction or in the center console can be adopted. The display 23 may be a so-called meter display arranged in the area in front of the driver's seat. Note that the input device 22 may be the driver's smart phone. For example, the touch panel and display of a smart phone owned by the driver can be used as the input device 22 . The input device 22 outputs to the in-vehicle network Nw an electrical signal corresponding to the operation performed by the user on the device as a user operation signal. The user operation signal includes information indicating the details of the user's operation.

The HUD 24 is a device that displays a virtual image that can be perceived by the user by projecting image light onto a predetermined area of the windshield based on control signals and video data input from the stimulus control device 1, navigation device, or the like. The HUD 24 displays an image superimposed on the scenery in front of the vehicle. In addition, the HUD 24 plays a role as an information presentation device that presents various information such as the operating state of the stimulus control device 1, the driver's recognized state, and suggestions for breaks based on input signals from the stimulus control device 1. I can carry it. The information about the operating state of the stimulus control device 1 includes, for example, activation information and continuation information of the stimulus control device 1 .

The activation information is information for notifying the driver that the awakening stimulus will be activated from now on, and is represented by, for example, text or an icon image. Wake-up information is displayed before the arousal stimulus occurs. Continuation information is displayed when the arousal stimulus continues. Continuation information informs the driver that the arousal stimulus is continuing. Continuation information may include an icon image or text outlining the stimulus currently being output. The summary of stimuli includes types of stimuli such as light, sound, vibration, smell, and air conditioning. In addition, the continuation information may include advance notice information indicating the type and strength of the next stimulus to be output.

The driver state refers to, for example, a sleepiness level determined by a sleepiness determination unit F3, which will be described later, or a driver's fatigue level determined by a fatigue determination unit F4. The HUD 24, under the control of the stimulus controller 1, displays icon images or text indicating sleepiness and fatigue levels. The rest proposal is a display of contents suggesting that the driver take a rest. A break suggestion is displayed on any one or more of a plurality of display devices such as the HUD 24 and the display 23, for example. Under the control of the stimulus control device 1, the HUD 24 displays an icon image imitating a coffee cup or an icon image urging the user to park and take a nap, for example, as a suggestion for a break. Various information may be presented using not only the HUD 24 but also a meter display, a center information display, or the like.

The light-emitting device 31 is a device that generates light visible to the driver. The light emitting device 31 includes, for example, a plurality of LEDs (light emitting diodes) as light sources. The light emitting device 31 is provided at a position within the driver's field of vision, such as the spoke portion of the steering wheel, the upper surface portion of the instrument panel, or the ceiling portion in front of the driver's seat.

The light emitting device 31 is configured to be able to adjust the luminance, color of light, and the like. The light emitting device 31 can periodically change the luminance. When the luminance changes periodically, the light emitting device 31 can adjust the period length, the waveform, the amount of luminance change per unit time, and the like. The light generated by the light emitting device 31 corresponds to a visual stimulus. For example, bluish-white light having a luminance above a predetermined threshold corresponds to a visual arousal stimulus. White light and red light also correspond to arousal stimuli. In addition, light such as orange, blue, and green with suppressed or fluctuating luminance can correspond to light expected to have a relaxing effect and, in turn, can correspond to a stimulus for alleviating fatigue.

The speaker 32 generates sound inside the vehicle. The expression "voice" includes mere sounds such as warning sounds, musical sounds, and natural sounds. Natural sounds can include the sound of flowing water, the sound of waves, the sound of rain, and the sounds of animals such as birds. Speech corresponds to auditory stimuli. The speaker 32 corresponds to a device that outputs auditory stimulation. An output pattern in which the interval between sounds is equal to or less than a predetermined value can be used as an arousal stimulus. On the other hand, natural sounds and the like can be employed as fatigue-relieving stimuli.

The aroma shooter 33 injects fragrance into the passenger compartment. The driver perceives the scent of the sprayed perfume. The scent generated by the aroma shooter 33 corresponds to an arousal stimulus for the sense of smell. The aroma shooter 33 can change the type of fragrance, the strength of the fragrance, the ejection time of the fragrance, and the like. Note that the aroma shooter 33 may be integrated with the air conditioner 34 .

The air conditioner 34 selectively generates cold air and warm air as conditioned air in the vehicle compartment. Cold wind responds to the arousal stimulus using cold sensation and tactile sensation (pressure sensation). The air conditioner 34 can change the temperature, air volume, and mode of cold air. Modes include an auto mode and a face mode. Auto mode is a mode in which the blowing direction of cool air is changed by a preset algorithm. The face mode is a mode in which cold air is continuously blown toward the driver's face.

The seat heater 35 is, for example, an electric heater such as a PTC (Positive Temperature Coefficient) heater, and heats the seat St and, by extension, the seated occupant using heat generated by current. The heat output by the seat heater 35 corresponds to the thermal stimulus. In addition, in this embodiment, a driver is assumed as an object to which various stimuli are applied. Therefore, the seat St corresponds to the driver's seat. Also, the description of the seated person refers to the driver.

The seat heater 35 is built in the backrest portion of the seat St serving as the driver's seat, the headrest, and the seating portion. As the seat heater 35, as shown in FIG. 2, there are a waist heater 35A, an upper heater 35B, and a neck heater 35C. The lumbar heater 35A is a seat heater 35 provided at a predetermined position in the backrest that is assumed to come into contact with the lumbar region of the seated person. The upper heater 35B is a seat heater 35 provided in the backrest above the lumbar heater 35A, for example, at a position where the shoulders and shoulder blades of the seated person come into contact. The neck heater 35C is a heater for warming the neck of the seated person, and is arranged at the upper end of the backrest or the headrest. In a normal driving posture, the neck of the driver is often not in contact with the seat St (that is, is floating). Therefore, the neck heater 35C of the present disclosure is configured as a so-called radiant heater that emits infrared rays (heat) as a more suitable example.

It should be noted that the in-vehicle system IvS does not need to include all the devices described above as the seat heater 35 . A part of the seat heater 35 may be omitted, and the waist heater 35A and the upper heater 35B may be integrated as a back heater. As the seat heater 35, a seat surface heater may be provided on the seat so as to warm the thighs of the seated person.

The SVS 36 is a device that cools the seated person's body using a blower 361 built into the seat, for example, the backrest. Cooling of the back of the driver by the SVS 36 corresponds to the cold stimulus. The SVS 36 is configured to draw in air from the seat surface through grooves formed in the seat pad and blow it out in a predetermined direction, such as the lower part of the seat, sideways, or upward, so as not to cause discomfort to passengers in other seats. According to the suction type, the heat trapped in the seat St can be released to give a cool feeling to the seated occupant. In addition, according to the suction type, the cool air output from the air conditioner 34 is sucked along the human body, so a cooling effect that wraps the body of the seated person can be expected. The SVS 36 may be of a type that blows air toward the seated person. Even with the method of blowing air, the body surface of the seated person can be cooled by blowing air. Also, the SVS 36 may be configured to be switchable between a suction mode and a blowout mode by operating the blower 361 in reverse rotation. Note that the SVS 36 basically only blows air, and does not adjust the temperature of the air itself.

The massager 37 is a device that relaxes the body of the seated person by vibrating a vibration generator provided on the seat St in a predetermined pattern or by driving movable rollers. The massager 37 is configured to use an actuator such as a motor to apply tactile stimuli such as pushing, kneading, rubbing, tapping, and shaking to the body of the seated person.

The vibration generator 38 generates vibration based on control signals from the stimulation control device 1 . The vibration generator 38 is configured to vibrate a seat belt, for example. Vibrations correspond to arousal stimuli. Note that the vibration generator 38 may apply vibration to the back of the seated person. Vibration generator 38 may be integrated with massager 37 .

Hereinafter, for the sake of convenience, a device that directly or indirectly stimulates the driver, such as the light emitting device 31, will also be referred to as a stimulus generator.

The communication device 39 is a device for communication connection between the wearable device 40 and the stimulation control device 1 . The connection mode between the wearable device 40 and the communication device 39 may be wired connection or wireless connection. For example, the communication device 39 and the wearable device 40 are configured to perform wireless communication conforming to short-range wireless communication standards such as Bluetooth (registered trademark) and Wi-Fi (registered trademark). A smart phone paired with each may be interposed between the communication device 39 and the wearable device 40 .

The wearable device 40 is a device that is worn on, for example, the wrist of the driver and used, and includes, for example, a biosensor that senses the biometric information of the driver. The biosensor is, for example, a heart rate sensor that measures heart rate. Biosensors include sensors that detect blood pressure, electrocardiogram, pulse wave, amount of perspiration, body temperature, amount of heat released from the human body, rhythm of breathing, and depth of breathing. As the wearable device 40, various shapes such as a wristband type, a wristwatch type, a ring type, an eyeglass type, and an earphone type can be adopted. The wearable device 40 is configured to be able to mutually communicate with the stimulus control apparatus 1 via a communication device 39 mounted on the vehicle.

The in-vehicle device that communicates with the stimulus control device 1 is not limited to the above examples. An in-vehicle biosensor may be connected to the stimulus control device 1 . The in-vehicle biosensor may be built in the driver's seat St, or may be provided in the steering wheel. In addition, millimeter-wave radar that detects the driver's heart rate, body movement, and posture can also be included in the in-vehicle biosensor by transmitting and receiving millimeter waves as search waves toward the driver's seat.

<Regarding the configuration of the stimulus control device 1>
The stimulus control device 1 is a device that operates various stimulus generators based on an input signal from the DSM 21 and a user's operation signal output from the input device 22 . The stimulus control device 1 is configured as a computer including a processor 11, a RAM (Random Access Memory) 12, a storage 13, a communication interface 14 (I/O), and a bus line connecting these components. The processor 11 is, for example, a calculation core such as a CPU (Central Processing Unit). The processor 11 accesses the RAM 12 to perform various processes. RAM 12 is a volatile memory.

The storage 13 is configured to include a non-volatile storage medium such as flash memory. The storage 13 stores a stimulus control program, which is a program for causing a computer to function as the stimulus control device 1 . Execution of the stimulus control program by the processor 11 corresponds to execution of a stimulus control method corresponding to the stimulus control program.

Further, in the storage 13, a plurality of output patterns of arousal stimulation are registered. Each of the plurality of output patterns differs in the combination and intensity of the arousal stimulus to be output. Details of the stimulus output pattern, that is, the stimulus pattern will be described separately later. The communication interface 14 is a circuit for the stimulation control device 1 to communicate with other devices. The communication interface 14 may be implemented using an analog circuit element, an IC, or the like.

The stimulus control device 1 provides each functional unit shown in FIG. 3 by executing a stimulus control program stored in the storage 13 by the processor 11 . That is, the stimulus control device 1 includes an operation signal acquisition unit F1, a sensor information acquisition unit F2, a drowsiness determination unit F3, a fatigue determination unit F4, an integrated control unit F5, and a display processing unit F6. The integrated control unit F5 includes, as more detailed functional units, an awakening support control unit F51, a fatigue alleviation control unit F52, and an arbitration unit F53. In addition, the stimulus control device 1 has a timer function for measuring a predetermined time. The stimulus control device 1 uses the timer function to specify the elapsed time from the start of outputting each stimulus.

The operation signal acquisition unit F1 acquires a user operation signal from the input device 22 . That is, the information indicating the content of the user's operation on the user's input device 22 is acquired. Specifically, the operation signal acquisition unit F1 acquires an instruction to start, stop, or change the pattern of fatigue-relieving stimulation. Further, the operation signal acquisition unit F1 acquires an instruction to start, stop, or change the pattern of output of the arousal stimulus.

The sensor information acquisition unit F2 acquires information (that is, judgment material) for judging the drowsiness level of the driver from the DSM 21, the wearable device 40 as a biosensor, or the like. The sleepiness level is a parameter representing the degree of sleepiness. Information that can be used to determine sleepiness level includes, for example, the degree of eye opening, frequency of blinking, degree of variation in intervals between blinking, face orientation, line-of-sight direction, posture, frequency of yawning, and steering wheel angle. and the degree of fluctuation. The sensor information acquisition unit F2 sequentially acquires various types of information for the drowsiness determination unit F3 to determine the drowsiness level from a predetermined sensor group including the DSM21. Drowsiness level predictors can also be referred to as driver characteristics that can be used to assess drowsiness levels. The sensor information acquisition section F2 corresponds to the driver state acquisition section.

Further, the sensor information acquisition unit F2 acquires information (that is, judgment material) for judging the fatigue level of the driver from the DSM 21 or the like. The fatigue level is a parameter representing the degree of fatigue (that is, the degree of fatigue). Information that can be used as information for determining the degree of fatigue includes, for example, the behavior of the driver, as well as the driver's facial expression, complexion, heartbeat, body temperature, driving time, and the like. The driver's behavior that can be used to determine the degree of fatigue includes, for example, the movement of the shoulder, the movement of rubbing the shoulder, the movement of changing the posture, the movement of tilting the neck, and the like.

The driving time is the elapsed time from the start of driving, and is specified using, for example, a timer that measures the time after the driving power supply is turned on, or the time when the driving power supply is turned on. sell. The power supply for running is a power supply for running the vehicle, and indicates an ignition power supply when the vehicle is a gasoline vehicle. When the vehicle is an electric vehicle or a hybrid vehicle, the running power supply refers to the system main relay. As an index of the degree of fatigue, it is also possible to use the amount of decrease in the skin temperature of the peripheral area relative to the start of running.

The drowsiness determination unit F3 determines the drowsiness level of the driver based on various information acquired by the sensor information acquisition unit F2. In the present embodiment, as an example, a case where the drowsiness level is classified into four stages from 0 to 3 and determined will be described. A smaller sleepiness level corresponds to a higher degree of wakefulness. For example, level 0 corresponds to a state in which the driver is not sleepy at all, and level 1 corresponds to a state in which the driver is slightly sleepy, in other words, the driver himself/herself is not aware of sleepiness. Level 2 corresponds to a state in which the driver seems to be a little sleepy and the driver himself/herself can be aware of sleepiness. Level 3 corresponds to a sleepy state. As the drowsiness level, level 4 corresponding to a very sleepy state, level 5 corresponding to almost/completely asleep, etc. may be provided.

Various methods can be used as a method for determining the sleepiness level. For example, the drowsiness determination unit F3 determines the degree of eye openness, the degree of fluctuation over time in the degree of eye openness, the difference in the degree of openness of the left and right eyes, the movement speed of the line of sight, the cycle and speed of blinking (blinking), the frequency of yawning, and the like. Used in combination to determine drowsiness level. In general, there is a tendency that the stronger the drowsiness, the smaller the amount of eye opening. Therefore, the drowsiness determination unit F3 may determine that the drowsiness level is higher as the amount of eye opening is smaller. Also, the more drowsy, the slower the line of sight movement speed and the blinking speed. Therefore, the drowsiness determination unit F3 may determine the drowsiness level to be higher as the line-of-sight movement speed and blink speed are lower. In addition, the higher the drowsiness level, the more unstable the blinking cycle tends to be. Therefore, the drowsiness determination unit F3 may evaluate the drowsiness level based on the degree of stability of the blinking cycle, for example, the variance value of the blinking cycle within a certain period of time. In addition, the drowsiness determination unit F3 determines the amount of heat radiation from the driver's body, the tendency and distribution of changes in body surface temperature, the presence or absence of deep breathing, the presence or absence of vertical movement of the shoulders, and whether the face is facing downward or upward. The drowsiness level may be determined based on, for example.

The fatigue determination unit F4 determines the fatigue level of the driver based on various information acquired by the sensor information acquisition unit F2. In this embodiment, as an example, a case where the fatigue level is divided into two stages from 0 to 1 and determined will be described. A smaller value of the fatigue level corresponds to a lower level of fatigue. For example, level 0 corresponds to no fatigue, and level 1 corresponds to fatigue. Such a fatigue determination part F4 corresponds to a configuration for determining the presence or absence of fatigue. Of course, the fatigue level may be evaluated in three stages or more. For example, level 1 may be assigned to a slightly tired state, and level 2 may be assigned to a very tired state.

Various methods can be used to determine the fatigue level. For example, the fatigue determination unit F4 determines that the fatigue level is 1 when the driving time is equal to or greater than a predetermined fatigue determination threshold. The fatigue determination threshold for driving time can be set to, for example, 10 minutes, 30 minutes, or 1 hour. The fatigue determination unit F4 sets the fatigue level from 0 to 1 based on whether the frequency of moving the shoulder, the frequency of massaging the shoulder, the frequency of changing the posture, and the frequency of tilting the neck/head are equal to or greater than a predetermined threshold. Also good. Setting the fatigue level to 1 corresponds to determining that the driver is tired. The fatigue determination unit F4 may determine the fatigue level by using a combination of driver behavior, driver's facial expression, complexion, heart rate, body temperature, driving time, and the like. For example, if the driver's expression or complexion is not good, the degree of fatigue may be evaluated as high. Alternatively, if the heartbeat of the driver is higher than the normal value, or if the body temperature is higher than normal, the degree of fatigue may be determined to be high. In addition, it may be determined that the driver is in a fatigued state based on the fact that the number of right and left turns after the start of travel exceeds a predetermined value, or that the driver has traveled on a mountain road for a predetermined time or longer. The fatigue determination unit F4 corresponds to a configuration that determines the fatigue level of the driver based on at least one of the output of the biosensor and the travel history from the start of driving.

Based on the drowsiness level being 1 or higher, the awakening support control unit F51 outputs the awakening stimulus in a pattern according to the drowsiness level. A configuration including the awakening support control unit F51 and a stimulus output device capable of outputting an awakening stimulus forms an awakening support system Sy1. Note that generating a certain awakening stimulus corresponds to operating a device capable of outputting the awakening stimulus. For the sake of convenience, outputting an awakening stimulus is also referred to as activation for awakening. A drowsiness level of 1 or more corresponds to an awakening support operation condition for operating the awakening support system Sy1. Further, level 1 corresponds to the awakening support threshold here. The awakening support system Sy1 or the awakening support control unit F51, which is the center thereof, corresponds to the awakening support unit.

As shown in FIG. 4 as an example, the stimulus control device 1 is provided with three modes for outputting stimuli for supporting wakefulness. That is, the awakening support control unit F51 is configured to be capable of selectively adopting the first mode Md1, the second pattern, and the third mode Md3 based on the current drowsiness level judgment value, the past drowsiness level history, and the like. ing. The first mode Md1 is a mode that is mainly applied when the drowsiness level is one. The second mode Md2 is a mode applied when the drowsiness level is two. The third mode Md3 is a mode applied when the drowsiness level is 3.

Types of arousal stimuli include light, scent, cool air, vibration, and sound. Light is a visual arousal stimulus generated by the light emitting device 31 . The scent is an awakening stimulus generated by the aroma shooter 33 . Cold air is an awakening stimulus generated by the air conditioner 34, for example. Vibration is an arousal stimulus that vibration generator 38 applies to the driver through the seat belt. Sound is an arousal stimulus generated by the speaker 32 .

The arousal stimulation of light generated by the light emitting device 31 is classified into type 1 (#1 in the figure) and type 2 (#2) according to the characteristics of the light. The characteristics of light include, for example, the color of light, luminance, how light flickers, whether or not there is fluctuation, and where light is generated. For example, type 1 may be light blue light and type 2 may be pale light. The intensity of stimulation, in other words, the effect of awakening the driver is set to be higher in type 2 than in type 1. For example, the higher the brightness, the higher the stimulus intensity. For example, the brightness of type 2 light is set to 1.2 times the brightness of type 1 light. Arousal stimulation of light can be realized as illumination by the light emitting device 31 . In addition, HUD24 may output the illumination as a light stimulus. The description of the light emitting device 31 below can be replaced with the HUD 24 . Further, the HUD 24 and the light emitting device 31 may be used together as an output source of optical stimulation.

Aroma arousal stimuli are classified into type 1 (#1 in the figure) and type 2 (#2) according to the characteristics of the aroma. Fragrance characteristics include, for example, the type of fragrance, the strength of the fragrance, and the like. The effect of awakening the driver, that is, the intensity of stimulation is set to be higher in Type 2 than in Type 1. For example, the type 1 scent may be a lemon scent and the type 2 scent may be a mint scent.

The cold wind awakening stimulus includes three elements: cold wind temperature, wind volume, and mode. Cool air temperature settings include Auto, Semi-Lo, and Lo. Auto is to set the temperature according to a preset algorithm. Low refers to the lowest temperature that the air conditioner 34 can output, such as 18 degrees or 17.5 degrees. Semi-low refers to a temperature that is a predetermined amount above the minimum temperature, such as 20 degrees. The air volume is auto or strong. Auto is to set the air volume by a preset algorithm. Strong refers to a level of 70% or more of the maximum air volume that the air conditioner 34 can output. The mode is the above-described auto mode or face mode. The effect of awakening the driver is higher as the temperature of the cool air is lower. The effect of awakening the driver increases as the air volume of the cool air increases. Further, the effect of awakening the driver is higher in the face mode than in the auto mode.

Arousal stimulation of vibration can be classified into three levels of weak, medium, and strong according to the strength (amplitude) of vibration. Of course, as the characteristics of the vibration stimulus, it is possible to adopt not only the strength but also the rhythm and the place where the vibration is applied. Three or more types of vibration may be prepared, or only one type may be provided.

Two types of audio awakening stimuli are prepared, for example, "music" for outputting a predetermined melody and "alarm" for outputting a warning sound. It should be noted that a plurality of types of alarms, such as type 1 (#1) and type 2 (#2), may be prepared according to the volume and frequency of the sound. The intensity as an arousal stimulus is set so that type 2 is higher than type 1.

The first mode Md1 includes a plurality of stimulus patterns P11 to P14 each having different features or combinations of arousal stimuli to be generated. The stimulation pattern P11 is a pattern for turning on Type 1 illumination (light), and the stimulation pattern P12 is a pattern for outputting Type 1 scent. The stimulation pattern P13 is a pattern for outputting a predetermined melody, and the stimulation pattern P14 is a pattern for outputting semi-low cold wind. Illumination (light) of the patterns P11-P12 may be realized using the HUD 24, as described above.

When the first mode Md1 is applied, the awakening support system Sy1 sequentially executes the stimulation pattern P11, the stimulation pattern P12, and the stimulation pattern P13 as shown in FIG. 5, and executes the stimulation pattern P14. When the time period of the stimulation pattern P14 ends, the time period of the stimulation pattern P11 is resumed. That is, while the first mode Md1 is applied, the stimulation patterns P11-14 are repeated in order.

The second mode Md2 includes a plurality of stimulus patterns P21 to P23 each having different features or combinations of arousal stimuli to be generated. The stimulation pattern P21 is a pattern in which type 2 illumination, face mode cold wind, medium level vibration, and type 1 alarm are output in parallel. The stimulus pattern P22 is a pattern in which type 2 illumination, type 1 fragrance, auto mode cool air, and type 1 alarm are output in parallel. The stimulation pattern P23 is a pattern in which type 2 illumination and face mode cold wind are output in parallel. When the second mode Md2 is applied, the awakening support system Sy1 sequentially executes stimulation patterns P21 to P23 as shown in FIG. When the time period of the stimulation pattern P23 ends, the time period of the stimulation pattern P21 is resumed. That is, the stimulation patterns P21-P23 are repeated during the second mode Md2. The illumination (light) of the patterns P21-P23 and P31-P33 may be realized using the HUD24.

A third mode Md3 includes a plurality of stimulation patterns P31-P33. The third mode Md3 is the same as the second mode Md2 except that the scent stimulus is changed from type 1 to type 2, the alarm type is changed from 1 to 2, and a break is suggested. can do. Of course, the configuration of the third mode Md3 may be completely different from that of the second mode Md2. The number of stimulation patterns constituting each mode described above and the content of each stimulation pattern can be changed as appropriate.

The pattern duration Tp, which is the duration of each stimulation pattern, is, for example, 45 seconds. The pattern continuation time Tp may be, for example, 30 seconds, 50 seconds, 60 seconds, or the like. If the pattern continuation time Tp is too long, the driver will become accustomed to the stimulus output by the operation pattern, and it will be difficult to obtain an awakening effect. On the other hand, if the pattern duration Tp is too short, the stimulus output mode will be changed frequently, which may annoy the driver. It is preferable that the pattern continuation time Tp is set to 40 seconds or more and 60 seconds or less in order to ensure both the awakening effect and the reduction of annoyance. The pattern continuation time Tp corresponds to the first time. As long as the drowsiness level does not change, the wakefulness support control unit F51 performs, as a basic operation, changing the stimulation pattern at a cycle corresponding to the pattern duration time Tp.

The awakening support control unit F51 has a stop mode in which no awakening stimulus is output. A state in which one of the first mode Md1, the second mode Md2, and the third mode Md3 is set with respect to the stop mode can be called an arousal stimulus output mode. Suspension mode applies when the drowsiness level is set to zero. When the output of the arousal stimulus is started or the stimulus output mode is changed, the arousal support control unit F51 stops the output of the arousal stimulus for about 10 seconds at the beginning and guides the user to change the stimulus output mode. Image or sound is used.

In addition, the awakening support control unit F51 may have an arbitration mode. The arbitration mode is a mode in which the intensity of some stimuli is increased or weakened or some types of stimulus outputs are stopped in accordance with the operation of the fatigue alleviation system Sy2. Details of the arbitration mode will be described separately later.

The awakening support control unit F51 outputs, for example, a control signal for outputting an awakening stimulus associated with the current stimulation pattern to the arbitration unit F53. Alternatively, the awakening support control unit F51 operates in the mode instructed by the arbitration unit F53. Interaction between the awakening support control unit F51 and the arbitration unit F53 will be separately described later.

When the fatigue level is 1 or higher, the fatigue relaxation control unit F52 outputs the fatigue relaxation stimulus in a pattern according to the fatigue level. A configuration including the fatigue relaxation control section F52 and a stimulus output device capable of outputting a fatigue relaxation stimulus forms a fatigue relaxation system Sy2. Note that generating a certain fatigue-relieving stimulus corresponds to operating a device capable of outputting the stimulus. A fatigue level of 1 corresponds to a fatigue mitigation operating condition for operating the fatigue mitigation system Sy2. Also, level 1 corresponds to the threshold for fatigue mitigation. The fatigue relaxation system Sy2 or the fatigue relaxation control section F52, which is the center thereof, corresponds to the fatigue relaxation section.

As an example, as shown in FIG. 7, when the fatigue level is set to 1 or higher, the fatigue relaxation control unit F52 operates the massager 37 for a predetermined massage time Tmg, and then alternately activates the seat heater 35 and the SVS 36. Start the hot and cold cycle to drive.

The hot/cold cycle includes a hot term in which the seat heater 35 is operated for a predetermined hot time Twm and a cold term in which the SVS 36 is operated for a predetermined cold time Tcl. The warm time Twm is, for example, 3.5 minutes, 4 minutes, or 5 minutes. The cold time Tcl is preferably shorter than the hot time Twm, and can be, for example, 3 minutes, 2.5 minutes, or 1.5 minutes. The warming time Twm may be configured so that the user can set it within 3 minutes to 5 minutes. Also, the cooling time Tcl may be configured so that the user can set it within a range of 1.5 minutes to 3 minutes. The heating and cooling cycle is configured so that one cycle is within 5 to 7 minutes, such as 6.5 minutes. According to the hot-cold cycle, blood flow is improved due to the pumping action of repeated expansion and contraction of blood vessels due to heat, and an effect of relieving fatigue can be expected. Also, by performing a massage before the hot and cold cycle, the effect of enhancing the fatigue recovery effect due to the hot and cold cycle can be expected. If the hot time Twm or cold time Tcl is too short, it will be difficult to obtain the pump action. Therefore, from the viewpoint of alleviating fatigue, it is preferable to set the warm time Twm to 1 minute or longer and the cold time Tcl to 30 seconds or longer.

The seat heaters 35 driven during the warming time Twm may be all or part of the waist heater 35A, the upper heater 35B, and the neck heater 35C. For example, only the upper heater 35B may be the seat heater 35 that operates in the warm-cool cycle. The seat heater 35 to be operated during the warming time Twm may be configured so that the user can specify it via a predetermined setting change screen.

Further, as a more preferable example, the fatigue relaxation control unit F52 operates the light emitting device 31 in conjunction with the driving of the massager 37, the seat heater 35, and the SVS36. For example, the fatigue relaxation control unit F52 causes the light emitting device 31 to emit green illumination while the massager 37 is being driven. Further, when the seat heater 35 is being driven, the fatigue relaxation control unit F52 turns on yellowish orange (amber) or amber illumination so that the brightness fluctuates at each lighting location. According to this configuration, the driver can visually feel the heat, and it is expected that the heat given by the seat heater 35 will improve the effect of relieving fatigue. Furthermore, the fatigue relaxation control unit F52 causes the light emitting device 31 to emit blue or white illumination while the SVS 36 is being driven. According to the configuration in which blue or white light is lit/blinks when the SVS 36 is driven, it is possible to provide a visual cooling sensation and also an awakening effect can be expected. Note that the illumination during fatigue relief may be executed as an option when the drowsiness level is 1.

When the fatigue level is set to 1, the fatigue relaxation control unit F52 outputs, to the arbitration unit F53, a control signal for outputting a fatigue relaxation stimulus corresponding to each time, using the control mode as basic control. . Alternatively, the awakening support control unit F51 operates in the mode instructed by the arbitration unit F53. The interaction between the fatigue relaxation control section F52 and the arbitration section F53 will be separately described later.

The arbitration unit F53 arbitrates the operations of the awakening support system Sy1 and the fatigue alleviation system Sy2. The arbitration unit F53, for example, compares the content of the stimulus that the awakening support control unit F51 is to output and the content of the stimulus that is to be output by the fatigue alleviation control unit F52, and has a predetermined relationship in which the two compete. In some cases, the output of the fatigue-relieving stimulus is stopped or its intensity is weakened. For example, when there is a competitive relationship, there is a case where a stimulus with an opposite attribute is added. Reverse attributes are, for example, application of heat (that is, heating) and cooling. Warm-color illumination such as yellow-orange and cool air/drive of SVS 36 can also be included in the relationship of reverse attributes. Details of the arbitration unit F53 will be described separately later.

The display processing unit F6 displays images on the HUD 24 or the display 23, such as the operating state of the stimulus control device 1, the recognized state of the driver, and a proposal for taking a break. For example, the display processing unit F6 displays an icon image indicating that the driver is in a fatigued state on the display 23 when the fatigue level is determined to be 1 or higher. Further, when the sleepiness level is determined to be 1 or higher, the display processing unit F6 displays an icon image corresponding to the sleepiness level on the display 23 . When the drowsiness level is 3 or higher, the display processing unit F6 displays an image related to a proposal for rest on the display 23 or the HUD 24 .

<Regarding an operation example of the arbitration unit F53>
The arbitration unit F53 arbitrates the operation of each system when both the awakening support system Sy1 and the fatigue alleviation system Sy2 operate, in other words, when both the drowsiness level and the fatigue level are 1 or higher.

Methods of mediation include, for example, the following policies (A) to (E).
(A) Stop the fatigue relaxation system Sy2.
(B) Drive both systems while degenerating the function/operation of the fatigue mitigation system Sy2.
(C) Allow each system to output only stimuli with common attributes.
(D) Separate the time periods for operating each system.
(E) Increase the intensity of the arousal stimulus or lengthen the pattern duration without changing the operation of the fatigue relief system Sy2.

The policy (B) includes a policy (B1) that gives priority to the awakening support system Sy1 when the fatigue alleviating system Sy2 is outputting a stimulation of the same type or target site as the awakening stimulation to be output. . The policy (B) also includes a policy (B2) of stopping or weakening the output of fatigue-relieving stimulation that causes drowsiness. Policy (C) corresponds to operating each system alternately. Policy (D) is divided into a pattern (D1) in which the awakening support time is set longer than the fatigue relief time and a pattern (D2) in which the awakening support time is set shorter than the fatigue relief time. can do. The awakening support time refers to the time during which the awakening support system Sy1 is operated. The fatigue relaxation time refers to the time to operate the fatigue relaxation system Sy2.

Policies (A), (B), (D1), and (E) correspond to policies that limit the operation of the fatigue reduction system Sy2 in order to prioritize the awakening support system Sy1 over the fatigue reduction system Sy2. By prioritizing awakening over alleviating fatigue, it is possible to prevent the driver from becoming drowsy. Also, in any policy, it is possible to reduce the possibility that competing stimuli are output at the same time. Along with this, it is also possible to reduce the fear of giving the driver annoyance.

<Regarding the Arbitration Unit F53 When Policy (A) is Adopted>
FIG. 8 is a diagram showing a control example of the awakening support system Sy1 and the fatigue alleviation system Sy2 according to the combination of the drowsiness level and the fatigue level based on the policy (A). When both the drowsiness level and the fatigue level are 0, the awakening support system Sy1 and the fatigue alleviation system Sy2 do not operate.

When the drowsiness level is 0 and the fatigue level is 1, the awakening support system Sy1 does not try to output the awakening stimulus. Therefore, the arbitration unit F53 does not perform special arbitration processing, and operates the fatigue relaxation system Sy2 as usual. When the drowsiness level is 1 or more and the fatigue level is 1, the arbitration unit F53 either invalidates the instruction signal for the fatigue relaxation control unit F52, or instructs the fatigue relaxation control unit F52 to to stop controlling the As a result, when the drowsiness level is 1 or higher, the operation of the fatigue alleviating system Sy2 is stopped, and only the awakening support system Sy1 operates. By stopping the fatigue alleviating system Sy2, it is possible to prevent the driver from becoming drowsy due to the relaxing effect of massage and heat. The expression that a certain fatigue-relieving stimulus is not output includes the arbitration unit F53 invalidating the output instruction signal for the stimulus output from the fatigue-relaxation control unit F52, and the fatigue-relaxation control unit F52 under the control of the arbitration unit F53. stops outputting the output instruction signal.

<Regarding the Arbitration Unit F53 When Policy (B) is Adopted>
FIG. 9 is a diagram showing a control example of the awakening support system Sy1 and the fatigue alleviation system Sy2 according to the combination of the drowsiness level and the fatigue level based on the policy (B). When the drowsiness level is 1 or more, the operation of the fatigue alleviation system Sy2 is restricted according to the height of the drowsiness level.

For example, the arbitration unit F53 performs arbitration so as not to output fatigue relief stimulations of the same type as awakening stimulations as control corresponding to policy (B1). Massage, vibration, and illumination are examples of fatigue-relief stimulation and awakening stimulation in which the types of stimulation overlap. Also, the warm/cold cycle by the seat heater 35 and the SVS 36 and the cool air by the air conditioner 34 correspond to the same type of stimulation. Stimuli of the same type can be understood as stimuli that are sensed by the same sensory organs. If the same type of stimulus is mixed and output, there is a possibility that the intended effect cannot be obtained. In particular, if two stimuli of the same type and opposite attributes, such as heat and cold wind, are output at the same time, the effect may be weakened.

Based on such circumstances, the arbitration unit F53 may stop the driving of the massager 37, the illumination of the fatigue relaxation system, and the hot/cold cycle when both the fatigue level and the drowsiness level are 1 or higher. The fatigue-relieving illumination refers to illumination linked to driving of the seat heater 35 and the SVS 36 . In this case, the driver is provided with vibration or illumination as a stimulus for awakening according to the execution schedule of the stimulus pattern for awakening. FIG. 10 is a diagram conceptually showing the operation of the arbitration unit F53 in response to the illumination lighting instruction by the awakening support control unit F51 and the fatigue alleviation control unit F52. The arbitration unit F53 gives priority to stimuli for awakening and outputs the stimuli of the same type.

The arbitration unit F53 may limit the output of fatigue-relieving stimuli whose types overlap (conflict) with arousal stimuli. may always apply. Output restriction here means no output.

Further, as a control corresponding to the policy (B2), the arbitration unit F53 shortens the warming time Twm of the heating/cooling cycle by a predetermined amount when the fatigue level is 1 or more and the drowsiness level is 1, Set the cooling time Tcl longer. This is because the heat provided by the seat heater 35 is a factor of drowsiness. Further, when the drowsiness level is 2 or higher, the arbitration unit F53 stops driving the seat heater 35 . When the fatigue level is 1 or higher and the sleepiness level is 2 or higher, the SVS 36 operates intermittently.

Note that the arbitration unit F53 may reduce the operating range of the seat heater 35 step by step according to the drowsiness level. For example, when the fatigue level is 1, the arbitration unit F53 may stop the operation of only the neck heater 35C when the drowsiness level is 1, and stop the upper heater 35B and the neck heater 35C when the drowsiness level is 2. . Such a control mode corresponds to control to reduce the number of seat heaters 35 to be driven or the area of the portion to which heat is applied as the drowsiness level increases.

The arbitration unit F53 may stop the operation of the upper heater 35B and the neck heater 35C when the drowsiness level is 1, and stop the operation of all the seat heaters 35 when the drowsiness level is 2. Such arbitration control is equivalent to control of stopping part of the seat heaters 35 when the drowsiness level is 1, and stopping all seat heaters 35 when the drowsiness level is 2. It should be noted that the SVS 36 that imparts a cool sensation has the same attributes as the cool air from the air conditioner 34 . In other words, the SVS 36 is a device that outputs a stimulus that does not compete with cold wind as an arousal stimulus. Therefore, the SVS 36 may allow intermittent operation. As for the waist heater 35A, since it heats a part (waist) away from the face to which cold air is blown as an arousal stimulus, there is little mutual influence even if both are applied at the same time. Therefore, even if the drowsiness level is 2 or higher, the waist heater 35A may be operated intermittently as a heat/cool cycle.

As another aspect, when the fatigue level is 1 and the drowsiness level is 1, the arbitration unit F53 may operate the fatigue alleviation system Sy2 in the basic operation, while changing the operation of the awakening support system Sy1 from the basic operation. For example, regarding the awakening support system Sy1, activations for awakening other than cool air are cyclically output during the warm term. Then, when the cold term comes, the cold wind may be output in the face mode for a predetermined pattern duration time Tp. According to this configuration, it is possible to provide a cool feeling at once, so a further awakening effect can be expected. In addition, since the air conditioning device 34 does not output a cold wind stimulus during the warm term, it is possible to reduce the possibility that conflicting stimuli are output at the same time, thereby impairing the driver's comfort. In the case of synchronizing the cold air output from the SVS 36 and the air conditioner 34, considering that the driver's body cools down quickly, the cool air output time may be shorter than the predetermined pattern duration Tp by about 10 seconds. good.

<Regarding the Arbitration Unit F53 When Policy (C) is Adopted>
Common attribute stimuli for fatigue relief and awakening include, for example, combinations of cold air from the air conditioner 34 and the SVS 36 , massage from the massager 37 and vibration from the vibration generator 38 . Cool-color illumination can also include stimulation of attributes common to fatigue relief and alertness. When both the fatigue level and the drowsiness level are 1 or higher, the arbitration unit F53 outputs only the stimulus output instruction signal having the common attribute described above, which is input from the awakening support control unit F51 and the fatigue alleviation control unit F52, to the subsequent stage. It may be configured to output.

Alternatively, when both the fatigue level and the drowsiness level are 1 or higher, the arbitration unit F53 may output a predetermined instruction signal to the awakening support control unit F51 and the fatigue alleviation control unit F52 to operate in the arbitration mode. . In the arbitration mode, the awakening support control unit F51 and the fatigue alleviation control unit F52 can perform control for awakening support and control for alleviating fatigue using only the above-described common attribute stimulus. Specifically, the awakening support control unit F51 can sequentially output cool air, vibration, and type 1 or type 2 illumination in the arbitration mode. The fatigue relaxation controller F52 can drive the massager 37 only once or intermittently in the arbitration mode. Moreover, the fatigue relaxation control unit F52 may intermittently drive the SVS 36 in the arbitration mode. FIG. 11 summarizes the devices operated as stimulus output sources by the awakening support control unit F51 and the fatigue alleviation control unit F52 in policy (C) and their operation modes. The parameters that constitute the intensity of stimulation, such as the intensity of vibration and the intensity of cold wind, can be set to be larger as the drowsiness level is higher.

<Regarding the Arbitration Unit F53 When Policy (D) is Adopted>
FIG. 12 is a diagram showing a control example of the awakening support system Sy1 and the fatigue alleviation system Sy2 according to the combination of the drowsiness level and the fatigue level based on the policy (D). When both the drowsiness level and the fatigue level are 1 or higher, the arbitration unit F53 alternately operates the awakening support system Sy1 and the fatigue alleviation system Sy2. For example, Trf shown in FIG. 12 represents the fatigue relief time, which is the time to operate the fatigue relief system Sy2, and Tas represents the awakening support time, which is the time to operate the awakening support system Sy1. FIG. 12 illustrates a case where the awakening support time Tas is set longer than the fatigue alleviation time Tfr. It is preferable that the fatigue relaxation time Tfr is mainly set to an integer multiple of one period of the hot and cold cycle, for example, one time. The massage may be performed only for the first time, or may be performed every time. When massage is performed, the fatigue relaxation time Tfr may be set to a value obtained by adding the massage time Tmg to the time for one cycle of the hot and cold cycle. The awakening support time Tas can be set to a length that is an integral multiple of the pattern continuation time Tp within a range longer than one time the fatigue relief time Tfr.

Of course, as another aspect, the awakening support time Tas may be set shorter than the fatigue relief time Tfr. For example, when the fatigue level is higher than the sleepiness level, such as when the fatigue level is 2 and the sleepiness level is 1, the fatigue relief time Tfr may be set longer than the awakening support time Tas. Further, when the drowsiness level is 1, the fatigue relief time Tfr is set equal to or longer than the awakening support time Tas, and when the drowsiness level is 2 or more, the awakening support time Tas is set longer than the fatigue relief time Tfr. You can make it longer. The arbitration unit F53 may be configured to shorten the fatigue relaxation time Tfr as the drowsiness level increases. When dynamically changing the fatigue relaxation time Tfr, the length of each hot/cold term is adjusted so that the ratio between the hot time Twm and the cold time Tcl set as the basic operation is maintained. is preferred. The arbitration unit F53 may change the control target values (outputs) of the seat heater 35 and the SVS 36 so that the amount of heat generated and the amount of heat absorbed per unit time increase as the fatigue relaxation time Tfr is shortened.

<Regarding the Arbitration Unit F53 When Policy (E) is Adopted>
Policy (E) is to increase the intensity of the awakening stimulus of the awakening support system Sy1 or to control the increase in drowsiness of the driver due to the heat while operating the fatigue alleviating system Sy2 in its basic operation. This corresponds to the technical idea of lengthening the The adjustment of the intensity of the awakening stimulus and the change of the pattern duration can be executed by the awakening support control unit F51 based on the instruction from the arbitration unit F53, for example. More specifically, the awakening support control unit F51 can perform the following adjustment based on the fact that the arbitration mode is set based on the instruction from the arbitration unit F53.

For example, in the arbitration mode, when the fatigue level is 1 and the drowsiness level is 1, the awakening support control unit F51 increases the intensity of each stimulus by a predetermined amount ( for example, 1.2 times). Note that the case where the fatigue level is 0 and the drowsiness level is 1 corresponds to the case where the awakening support system Sy1 operates alone without the fatigue alleviating system Sy2 operating in parallel. For example, when the fatigue level is 1 and the drowsiness level is 2, the intensity of each stimulus is increased by a predetermined amount compared to when the fatigue level is 0 and the drowsiness level is 2.

Further, when the fatigue level is 1 and the sleepiness level is 1, the awakening support control unit F51 increases the pattern duration Tp by a predetermined amount (for example, 10 seconds) than when the fatigue level is 0 and the sleepiness level is 1. as long as possible). When the fatigue level is 1 and the sleepiness level is 2, the awakening support control unit F51 sets the pattern duration Tp to a predetermined amount (for example, about 10 seconds) than when the fatigue level is 0 and the sleepiness level is 2. You can make it longer. Instead of lengthening the pattern duration time Tp for all stimulation patterns, it is also possible to lengthen only the stimulation pattern including the output of the cool air.

Thus, when the two systems operate in parallel, the arbitration unit F53 may strengthen the awakening stimulus or extend the pattern duration time more than when the awakening support system Sy1 operates alone. . According to this control mode, it is possible to reduce the possibility that the driver's drowsiness level will increase due to the fatigue alleviation system Sy2.

<Other Examples of Operation of the Stimulus Control Device 1>
The processing of the arbitration unit F53 described above can also be applied when the drowsiness level is determined to be from 0 to 1 or more while the fatigue alleviation system Sy2 is in operation, It is also applicable when it is determined to be 1 or more.

If the drowsiness level is already 1 or more when the fatigue level transitions from 0 to 1, the start of the operation of the fatigue alleviation system Sy2 may be put on hold until the drowsiness level becomes 0 or 1. The arbitration unit F53 suspends the operation of the fatigue reduction system Sy2 when the sleepiness level is 1 or more, and when the sleepiness level drops from 2 to 1 or from 1 to 0, the fatigue relaxation is stopped. System Sy2 may be operated.

When a fellow passenger is present, the display processing unit F6 displays an image of the sleepiness level and fatigue level determination results in a display area that is difficult for the fellow passenger to see. According to the said structure, it can be made hard to give a fellow passenger anxiety. Note that the position that is difficult to see from the fellow passenger refers to, for example, the meter display and the HUD 24 . Whether or not there is a fellow passenger can be specified based on the detection signal of the seating sensor in the front passenger seat or rear seat. That is, when a signal indicating that a person is seated is input from at least one of the seating sensors corresponding to the seats other than the driver's seat, it is assumed that there is a fellow passenger, and the driver's state is determined. The HUD 24 or meter display is adopted as the display destination of the recognition result. When there is no fellow passenger, the display position of the recognition result of the driver's state is arbitrary, and may be, for example, the center display. This configuration corresponds to a configuration that changes the display destination of the recognition result of the system for the driver state according to the presence or absence of a fellow passenger.

Further, the display processing unit F6 proposes a break based on the fact that the sleepiness level is 3 when the fatigue level is 0, and when the fatigue level is 1 or more, the sleepiness level is 1 or Even in the state of 2, the rest proposal may be implemented. In this way, while the drowsiness level is normally 3 and a break is urged, if the driver is in a fatigued state, the drowsiness level that urges the driver to take a break is lowered, thereby making it possible to propose an early break. Accordingly, it is possible to reduce the possibility that the driver will continue to drive in a state of fatigue and drowsiness.

The integrated control unit F5 may be configured to control the operating states of the awakening support system Sy1 and the fatigue alleviation system Sy2 based on user instructions. When the user inputs an instruction to activate/deactivate either the awakening support system Sy1 or the fatigue alleviating system Sy2, the integrated control unit F5 gives priority to the user's instruction. For example, when the integrated control unit F5 acquires an operation signal for operating the awakening support system Sy1 or the fatigue alleviation system Sy2, regardless of the drowsiness level or the fatigue level judgment value, the integrated control unit F5 drives the instructed system. sell.

More specifically, when the input device 22 inputs an activation instruction signal for the awakening support system Sy1, the integrated control unit F5 operates the awakening support system Sy1 regardless of the drowsiness level or the fatigue level determination value. Let At that time, if the fatigue relaxation system Sy2 is in operation, the fatigue relaxation system Sy2 may be stopped.

In addition, when the input device 22 inputs an activation instruction signal for the fatigue relief system Sy2, the integrated control unit F5 operates the fatigue relief system Sy2 regardless of the drowsiness level or the fatigue level determination value. At that time, if the awakening support system Sy1 is in operation, the awakening support system Sy1 may be stopped. Alternatively, it may be operated in an arbitration mode such as weakening the operation of the awakening support system Sy1.

Furthermore, when the fatigue relaxation system Sy2 is in operation and a stop instruction signal for the fatigue relaxation system Sy2 is input, the integrated control unit F5 stops the fatigue relaxation system Sy2 regardless of the fatigue level determination value. The same applies to the awakening support system Sy1.

Note that the startup instruction signal for a certain system corresponds to an operation signal indicating that an operation for driving the system has been performed. A stop instruction signal for a certain system corresponds to an operation signal indicating that an operation for stopping the drive of the system has been performed. A start instruction signal and a stop instruction signal are input from the input device 22 to the stimulus control device 1 based on the user's operation or speech on the input device 22 .

In this manner, the integrated control unit F5 maximizes the driver's desire when a start instruction signal for either the awakening support system Sy1 or the fatigue relief system Sy2 is input, or when a stop instruction signal for either of them is input. Prioritize. By giving top priority to the driver's wishes in this way, the driver's convenience or comfort can be improved. Further, when the integrated control unit F5 erroneously determines the drowsiness level or the fatigue level, it becomes possible to correct the system behavior based on the driver's instruction operation.

In a state in which the drowsiness level is determined to be 2 or 3 or higher, the integrated control unit F5 performs drowsiness notification processing before/after stopping the awakening support system Sy1 when a stop instruction signal for the awakening support system Sy1 is input. may be implemented. The drowsiness notification process corresponds to a process of outputting an image display or an audio output indicating that high drowsiness is detected. Specifically, the stimulus control device 1 makes an announcement "You seem to be very sleepy. Is it OK to stop the awakening support system?" An opportunity may be provided to choose to stop/continue operation of the assistance system Sy1. In addition, after stopping the awakening support system Sy1 based on the driver's instruction operation, an announcement such as "You seem to be very sleepy, so please take a break early" will be made, and the screen of the display 23/HUD 24 will be displayed. You can also put out a shot icon. As a result, safe driving can be encouraged even when the awakening support system Sy1 is stopped when the drowsiness level is high.

Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments, and various modifications described below are also included in the technical scope of the present disclosure. Various modifications can be made without departing from the scope of the present invention. For example, the following various supplements and modifications can be implemented in combination as appropriate within a range that does not cause technical contradiction. It should be noted that members having the same functions as those of the members described above are given the same reference numerals, and explanation thereof may be omitted. Also, when only a part of the configuration is mentioned, the above description can be applied to the other parts.

In the above, we have described the configuration for adding an arousal stimulus to the occupant in the driver's seat (that is, the driver) or adding a stimulus to relieve fatigue. may That is, the present disclosure may be applied to passenger seats and rear seats. When the present disclosure is applied to the front passenger seat, the above description of the driver can be implemented by replacing the front passenger seat occupant. The present disclosure can also be applied in parallel to both the driver's seat and the passenger's seat. The same is true for the rear seats.

An example of the control mode of the arbitration unit F53 in a system capable of implementing wake-up support and fatigue relief for the passenger in the front passenger seat will be described below. The following discussion is equally applicable to rear seat occupants.

The drowsiness determination unit F3 described above determines the drowsiness level of the front passenger seat occupant based on biometric information such as the face image of the front passenger seat occupant. Further, the fatigue determination unit F4 determines the fatigue level of the front passenger seat occupant based on the biological information of the front passenger seat passenger. The awakening support system Sy1 and the fatigue alleviation system Sy2 are configured to be able to apply various stimuli to the front passenger as well as the driver. For example, a seat heater 35 and an SVS 36 are also provided on the front passenger seat.

If the drowsiness level of the front passenger seat occupant is 1 or higher at the point in time when the fatigue level of the front passenger seat occupant transitions from 0 to 1, the arbitration unit F53 may preferentially implement the fatigue alleviation system Sy2. Further, if the drowsiness level of the occupant in the front passenger seat is 1 or higher at the time when the fatigue level of the occupant in the front passenger seat transitions from 0 to 1, the arbitration part F53 may drive the seat heater 35 in the front passenger seat to promote drowsiness. . This is because the occupant in the front passenger seat does not necessarily have to stay up. Which of fatigue relief and awakening support should be prioritized may be registered in advance as an arbitration policy between the awakening support system Sy1 and the fatigue alleviation system Sy2 for the front passenger seat occupant. Alternatively, the HUD 24 or the like may be used to present options to the driver so that the driver can select, via the input device 22, which of fatigue relief and wakefulness support should be prioritized.

If the arbitration policy of various stimulation outputs for the front passenger seat occupant is set to give priority to awakening assistance, the fatigue relief system Sy2 is operated when the drowsiness level falls below a predetermined value (for example, 0). be able to. The drowsiness level for releasing the operation restriction of the fatigue alleviation system Sy2 may be 0 or 1.

Note that if the drowsiness levels of both the driver and the front passenger are 1 or higher, the front passenger may also be prioritized for awakening assistance. This is because the effect of reducing (improving) the drowsiness level of the driver through dialogue with the front passenger seat occupant can be expected by the control for maintaining the awake state of the front passenger seat passenger. On the other hand, when the drowsiness level of the driver is 0 and the drowsiness level of the passenger is 1 or more and the fatigue level is 1, the fatigue relief system Sy2 is preferentially operated for the passenger. can be As described above, the mediation policy for the passenger in the front passenger seat may be switched between fatigue relief and wakefulness support according to the drowsiness level of the driver.

In addition, when the drowsiness level of both the driver and the front passenger seat occupant is 2 and the front passenger seat occupant is activated by the fatigue relief system Sy2, it is possible to add a stimulus for fatigue relief in a manner that does not make the front passenger seat occupant sleepy. preferable. This is because the driver can be expected to wake up through dialogue by extending the time the passenger in the front passenger seat is awake. It should be noted that fatigue relief that makes it difficult to fall asleep includes, for example, stopping the output of warm stimuli, shortening the warm term, and reducing the number of seat heaters 35 that are driven during the warm term.

<Additional notes>
The apparatus, systems, and techniques described in the present disclosure may be implemented by a special purpose computer comprising a processor programmed to perform one or more functions embodied by the computer program. . The apparatus and techniques described in this disclosure may also be implemented using dedicated hardware logic. Additionally, the apparatus and techniques described in this disclosure may be implemented by one or more special purpose computers configured in combination with a processor executing a computer program and one or more hardware logic circuits. For example, some or all of the functions provided by the stimulus control device may be implemented as hardware. Implementation of a function as hardware includes implementation using one or more ICs. A CPU, an MPU, a GPU, a DFP (Data Flow Processor), or the like can be used as a processor (arithmetic core). Also, some or all of the functions of the stimulus control device may be implemented by combining multiple types of arithmetic processing devices. Some or all of the functions of the stimulation control device may be implemented using a system-on-chip (SoC), FPGA, ASIC, or the like. FPGA stands for Field-Programmable Gate Array. ASIC is an abbreviation for Application Specific Integrated Circuit.

Computer programs may also be stored as computer-executable instructions on a computer-readable, non-transitory tangible storage medium. A HDD (Hard-disk Drive), an SSD (Solid State Drive), a flash memory, an SD (Secure Digital) card, or the like can be used as a program storage medium.

In addition to the stimulus control device described above, various forms such as a system including the stimulus control device as a component are also included in the scope of the present disclosure. For example, a program for causing a computer to function as a stimulus control device, a form of a non-transitional substantive recording medium such as a semiconductor memory in which the program is recorded, and the like are also included in the scope of the present disclosure.

1 stimulation control device, 21 DSM (biological sensor), 22 input device, 23 display (display device), 24 HUD (display device), 34 air conditioner, 35 seat heater, 36 SVS, 361 blower, F1 operation signal acquisition unit, F2 sensor information acquisition unit, F3 drowsiness determination unit, F4 fatigue determination unit, F5 integrated control unit, F6 display processing unit, F51 awakening support control unit, F52 fatigue alleviation control unit, F53 arbitration unit, Sy1 awakening support system (awakening support unit ), Sy2 fatigue relaxation system (fatigue relaxation part)

Claims (14)

a drowsiness determination unit (F3) for estimating the drowsiness level of an occupant sitting on a predetermined seat;
an awakening support unit (Sy1) that sequentially or in parallel generates a plurality of types of predetermined awakening stimuli for improving the drowsiness level of the occupant;
a fatigue relief section (Sy2) that sequentially or in parallel generates a plurality of types of predetermined fatigue relief stimuli for improving the fatigue state of the occupant;
An arbitration unit (F53) that arbitrates the respective operations of the awakening support unit and the fatigue alleviation unit,
The awakening support system, wherein the arbitration unit is configured to change the operation mode of the fatigue alleviation unit according to the drowsiness level. The awakening support system according to claim 1,
The awakening support system, wherein the seat is a driver's seat and the passenger is a driver's seat passenger. The awakening support system according to claim 1,
The wake-up support system, wherein the arbitration unit is configured not to output the fatigue-relieving stimulus when the drowsiness level is equal to or higher than a wake-up support threshold that is a threshold for generating the wake-up stimulus. The awakening support system according to any one of claims 1 to 3,
The plurality of types of fatigue relaxation stimuli include the same type as the awakening stimulus,
When the drowsiness level is equal to or higher than a wakefulness support threshold that is a threshold for generating the wakefulness stimulus, the arbitration unit selects the fatigue-relieving stimulus of the same type as the wakefulness stimulus among the plurality of types of the fatigue-relieving stimulus. Awakening support system that stops the output of The awakening support system according to any one of claims 1 to 4,
Among the plurality of types of fatigue-relieving stimuli, one having a conflicting attribute with any of the plurality of types of awakening stimuli is included,
A fatigue determination unit (F4) that determines the fatigue level of the occupant based on the output of the biosensor or the driving history from the start of driving,
When the drowsiness level is equal to or higher than a threshold for awakening support, which is a threshold for generating the awakening stimulus, and the fatigue level is equal to or higher than a threshold for fatigue relief, which is a threshold for generating the fatigue alleviating stimulus. (2) an awakening support system that prohibits the output of the fatigue relief stimulus having an attribute conflicting with any one of the plurality of types of the awakening stimulus; The awakening support system according to any one of claims 1 to 5,
The arbitration unit is configured to alternately operate the fatigue relief unit and the awakening support unit when a condition for operating both the fatigue relief unit and the awakening support unit is satisfied. system. The awakening support system according to any one of claims 1 to 6,
The fatigue alleviating section is configured to selectively output a warm stimulus and a cold stimulus as the fatigue alleviating stimulus,
The arbitration unit is configured to stop the output of the warm stimulus when the drowsiness level transitions to or above a wake-up support threshold, which is a threshold for generating the wake-up stimulus, while the fatigue alleviation unit is in operation. Awakening support system. The awakening support system according to any one of claims 1 to 7,
The fatigue relief section can output heat from a seat heater (35) built in the seat as the fatigue relief stimulus, and uses a blower (361) built in the seat to heat the occupant from the seat side. It is configured to give a cool feeling to
When the drowsiness level is equal to or higher than a threshold for awakening support, which is a threshold for generating the awakening stimulus, the arbitration unit prohibits the fatigue alleviating unit from outputting the heat, while the operation of the blower is prohibited. A wake support system configured to allow. The awakening support system according to any one of claims 1 to 7,
A fatigue determination unit (F4) that determines the fatigue level of the occupant based on at least one of the output of the biosensor or the running history from the start of driving,
The fatigue alleviating section is configured to output the fatigue alleviating stimulus on condition that the fatigue level is equal to or greater than a fatigue alleviating threshold, which is a threshold for generating the fatigue alleviating stimulus,
When the drowsiness level is equal to or higher than a threshold for awakening support that is a threshold for generating the awakening stimulus, the arbitration unit maintains the drowsiness level at a predetermined level even if the fatigue level is equal to or higher than the fatigue alleviation threshold. A wake-up support system configured not to output the fatigue-relieving stimulus until the value becomes equal to or less than the value. The awakening support system according to claim 2,
A display processing unit (F6) for displaying the state of the driver's seat occupant on a display device (23, 24),
The awakening assistance system, wherein the display processing unit is configured to change a display position regarding the state of the driver's seat occupant depending on whether or not there is a fellow passenger. The awakening support system according to any one of claims 1 to 10,
The drowsiness determination unit determines the drowsiness level of a fellow passenger who is an occupant of a predetermined seat other than the driver's seat in addition to the driver's seat occupant as the occupant,
The awakening support unit is configured to be capable of outputting the awakening stimulus to the fellow passenger,
The fatigue alleviating section is configured to selectively output a warm stimulus and a cold stimulus as the fatigue alleviating stimulus to the fellow passenger,
A fatigue determination unit (F4) that determines the fatigue level of the fellow passenger based on the biological information of the fellow passenger,
The arbitration unit preferentially outputs the awakening stimulus over the fatigue relieving stimulus to the driver's seat occupant, and prioritizes the fatigue relieving stimulus over the awakening stimulus to the fellow passenger. A wakefulness support system configured to output. The awakening support system according to any one of claims 1 to 11,
The drowsiness determination unit determines the drowsiness level of a fellow passenger who is an occupant of a predetermined seat other than the driver's seat in addition to the driver's seat occupant as the occupant,
The awakening support unit is configured to be capable of outputting the awakening stimulus to the fellow passenger,
The fatigue relief section is configured to be capable of outputting the fatigue relief stimulus to the fellow passenger,
The fatigue relief section is configured to selectively output a warm stimulus and a cold stimulus,
A fatigue determination unit (F4) that determines the fatigue level of the fellow passenger based on the biological information of the fellow passenger,
The arbitration unit determines that the drowsiness level of the fellow passenger is equal to or higher than a wakefulness support threshold, which is a level at which the wakefulness stimulus should be output, and that the fatigue level of the fellow passenger is a fatigue level at which the fatigue mitigation stimulus should be output. An awakening support system configured to output the warm stimulus to the fellow passenger with priority over the cold stimulus when the stimulus is equal to or greater than the relaxation threshold. The awakening support system according to any one of claims 1 to 11,
An operation signal acquisition unit (F1) for acquiring from an input device (22) an operation signal, which is a signal corresponding to the occupant's operation for switching the operation state of the awakening support unit and the fatigue relief unit,
A wake-up support system configured to perform control according to the operation signal regardless of the drowsiness level determination value when the operation signal acquisition unit acquires the operation signal. A fatigue determination unit (F4) that determines the fatigue level of an occupant sitting on a predetermined seat;
an awakening support unit (Sy1) that sequentially or in parallel generates a plurality of types of predetermined awakening stimuli for improving the drowsiness level of the occupant;
a fatigue relief section (Sy2) that sequentially or in parallel generates a plurality of types of predetermined fatigue relief stimuli for improving the fatigue level of the occupant;
An arbitration unit (F53) that arbitrates the respective operations of the awakening support unit and the fatigue alleviation unit,
The awakening support system, wherein the arbitration unit is configured to change a control mode of the awakening support unit according to the fatigue level of the passenger.

Images