JP2018151752A - Awakening support device, method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an awakening support device for urging awakening of a driver of a vehicle.SOLUTION: An awakening support device includes: a driver state determination section for determining whether a driver is drowsy on the basis of detection information of a driver state detection sensor to detect a state of a driver of a vehicle; a stimulation selection section for selecting a stimulation to be used for an awakening support of a driver among multiple kinds of stimulations prepared in accordance with multiple kinds of environments on the basis of detection information of an environment sensor to detect states of multiple kinds of environments in the circumference of a driver when the driver state determination section determines that a driver is drowsy; and a stimulation information output section for outputting stimulation information to generate a stimulation selected by the stimulation selection section to a stimulation generation device capable of generating multiple kinds of stimulations.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a wake-up support device, a method, and a program for urging a driver to wake up.

  For example, the driver of the vehicle is required to be sufficiently awake while driving the vehicle. In recent years, a vehicle equipped with an automatic driving control device that automatically controls the driving operation of the vehicle has also been proposed. However, in the present situation, the driving operation is not completely left to the automatic driving control device. Awakening is required.

  Not only the driver of the vehicle, but also the driver or operator of the machine used in the manufacturing facility or plant, etc. is also sufficiently awake while driving and operating the machine etc. It is required to do.

  Under such circumstances, for example, a device for awakening the driver's sleepiness, in other words, an awakening support device that promotes the driver's awakening has been proposed. The wake-up support device is basically a device that obtains a driving state of a vehicle, a driver's biological information and video, and gives a stimulus to the driver when necessary based on the information. Examples of the stimulus given to the driver include sound, vibration, light, and scent.

  For example, Patent Document 1 discloses an example of a wake-up support device that gives a vibration stimulus to a driver by vibrating a driver's seat. Patent Document 2 discloses an example of a wake-up support device that gives a driver a sound stimulus by generating sound from a speaker.

Japanese Patent No. 5482644 Japanese Patent No. 5056067

  Such awakening support devices are currently under development in methods and techniques for detecting driver drowsiness, stimuli given to drivers, and methods of giving stimuli, etc., and awakening support devices with various specifications and configurations Has been proposed.

  The present invention provides a novel wake-up support device that promotes driver wake-up.

  In order to solve the above-described problem, a first aspect of the present invention is an awakening support device that promotes a driver's awakening. The awakening support device includes a driver state determination unit that determines whether or not the driver is drowsy based on detection information of a driver state detection sensor that detects the state of the driver, and the driver When the state determination unit determines that the driver is drowsy, the driver's awakening support is based on detection information of an environmental sensor that detects a plurality of types of environmental conditions around the driver. A stimulus selection unit that selects a stimulus to be used from among a plurality of types of stimuli prepared in advance in association with the plurality of types of environments, and a stimulus generation device capable of generating the plurality of types of stimuli, A stimulus information output unit that outputs stimulus information for generating the stimulus selected by the stimulus selection unit.

  A wake-up support device according to a second aspect of the present invention relates to the wake-up support device according to the first aspect, wherein the stimulus selection unit uses the detection information of the environment sensor and the detection information for each of the plurality of types of environments. Based on the storage table that associates the effectiveness information indicating the effectiveness of the stimulus set in advance in association with each other, the effectiveness information of the stimulus corresponding to the detection information obtained by the environmental sensor is acquired, and the effectiveness The information is compared to select the most suitable stimulus for the driver's arousal support.

  A wake-up support device according to a third aspect of the present invention relates to the wake-up support device according to the first aspect, wherein the stimulus information output unit has a plurality of stimuli selected by the stimulus selection unit. The stimulus information is output to the stimulus generator so as to generate a plurality of stimuli sequentially or simultaneously.

  A wake-up support device according to a fourth aspect of the present invention relates to the wake-up support device according to the first aspect, wherein the stimulus selection unit uses the detection information of the environment sensor as a variable for each of the plurality of types of environments. By calculating according to the function of the effectiveness information representing the effectiveness of the vehicle, the effectiveness information of the stimulus corresponding to the detection information obtained by the environmental sensor is obtained, and the driver's awakening is compared by comparing the effectiveness information Choose the best stimulus for support.

  According to the first aspect of the present invention, the wake-up support device includes a stimulus generating device capable of generating a plurality of types of stimuli when the vehicle driver is determined to be drowsy. The stimulus information for generating the stimulus selected based on the environment is output. For this reason, the type of stimulus selected based on the environment around the driver is generated. Thereby, it can be expected that the driver's awakening is effectively promoted as compared with the case where other types of stimulation are generated.

  According to the second aspect of the present invention, the arousal support device uses the storage table of the detection information of the environment sensor and the effectiveness information of the stimulus, and selects the optimum stimulus selected based on the environment around the driver. Is output to the stimulus generator. For this reason, the optimal stimulus selected based on the surrounding environment of the driver is generated. Thereby, it can be expected that the driver's awakening is most effectively promoted as compared with the case where other types of stimulation are generated.

  According to the third aspect of the present invention, when the number of stimuli selected based on the surrounding environment of the driver is plural, the wake-up support device generates the plural stimuli in order. Information is output to the stimulus generator. Even if a certain stimulus does not effectively act on the driver's arousal support, it can be expected that another stimulus effectively acts on the driver's arousal support.

  According to the fourth aspect of the present invention, the wakefulness support device selects the optimum selected based on the environment around the driver by the calculation according to the function of the stimulus effectiveness information using the detection information of the environment sensor as a variable. Stimulation information for generating a simple stimulus is output to the stimulus generator. For this reason, the optimal stimulus selected based on the surrounding environment of the driver is generated. Thereby, it can be expected that the driver's awakening is most effectively promoted as compared with the case where other types of stimulation are generated.

  That is, according to each aspect of the present invention, it is possible to provide a novel arousal support device, method, and program for encouraging the driver to awaken.

It is a figure showing the whole automatic driving control system composition provided with the driving support device containing the awakening support device concerning one embodiment of this invention. It is a figure which shows the correspondence of each sensor contained in each generating apparatus and environmental sensor which are contained in the stimulus generator shown in FIG. It is a block diagram which shows the function structure of the awakening assistance apparatus which concerns on one Embodiment of this invention. It is a flowchart which shows the procedure and control content of the awakening assistance by the awakening assistance apparatus shown in FIG. It is a flowchart which shows the procedure and control content of another awakening assistance by the awakening assistance apparatus shown in FIG.

Embodiments according to the present invention will be described below with reference to the drawings.
[One Embodiment]
(Constitution)
FIG. 1 is a diagram showing an overall configuration of an automatic driving control system according to an embodiment of the present invention. This automatic driving control system is mounted on a vehicle 1 such as a passenger car.

  The vehicle 1 includes, as basic equipment, a power unit 2 including a power source and a transmission, and a steering device 3 equipped with a steering wheel 3a. An engine and / or a motor is used as the power source.

  The vehicle 1 is configured to be able to travel in either a manual operation mode or an automatic operation mode.

  The manual driving mode is a mode in which the vehicle 1 is driven mainly by a driver's manual driving operation, for example. The manual operation mode includes, for example, an operation mode for driving the vehicle based only on the driver's driving operation, and an operation mode for performing driving operation support control for supporting the driving operation of the driver while mainly driving the driver's driving operation. Is included.

  In the driving operation support control, for example, the steering torque is assisted so that the driver's steering becomes an appropriate steering amount based on the curvature of the curve when the vehicle 1 is traveling on the curve. The driving operation support control includes control for assisting a driver's accelerator operation (for example, operation of an accelerator pedal) or brake operation (for example, operation of a brake pedal), manual steering (manual operation of steering), and manual speed adjustment (speed). Adjustment manual operation) is also included. In manual steering, the vehicle 1 is steered mainly by the driver's operation of the steering wheel 3a. In manual speed adjustment, the speed of the vehicle is adjusted mainly by the driver's accelerator operation or brake operation.

  Note that the driving operation support control does not include control for forcibly intervening in the driving operation of the driver to automatically drive the vehicle. In other words, in the manual driving mode, the driving operation of the driver is reflected in the driving of the vehicle within a preset allowable range, but forcibly intervenes in the driving of the vehicle under certain conditions (for example, deviation from the lane of the vehicle). Control to do is not included.

  On the other hand, the automatic driving mode is a mode that realizes a driving state in which the vehicle automatically travels along the road on which the vehicle travels, for example. The automatic driving mode includes, for example, a driving state in which the vehicle automatically travels toward a preset destination without driving by the driver. In the automatic driving mode, it is not always necessary to automatically control all of the vehicle, and the driving state in which the driving operation of the driver is reflected in the driving of the vehicle within the preset allowable range is also included in the automatic driving mode. That is, the automatic driving mode includes control for forcibly intervening in driving of the vehicle under certain conditions, while reflecting the driving operation of the driver in driving of the vehicle within a preset allowable range.

  In FIG. 1, the vehicle 1 is also provided with an automatic driving control device 5 for executing driving control in the automatic driving mode. The automatic driving control device 5 acquires sensing information from the steering sensor 11, the accelerator pedal sensor 12, the brake pedal sensor 13, the GPS receiver 14, the gyro sensor 15, and the vehicle speed sensor 16, respectively. The automatic driving control device 5 is a peripheral monitoring system that monitors these sensing information, route information generated by a navigation system (not shown), traffic information acquired by road-to-vehicle communication, and positions and movements of surrounding people and vehicles. The vehicle 1 is automatically controlled based on the information obtained by the above.

  Automatic control includes, for example, automatic steering (automatic steering operation) and automatic speed adjustment (automatic driving of speed). Automatic steering is an operating state in which the steering device 3 is automatically controlled. Automatic steering includes LKA (Lane Keeping Assist). For example, the LKA automatically controls the steering device 3 so that the vehicle 1 does not deviate from the traveling lane even when the driver does not perform the steering operation. Even when LKA is being executed, the driver's steering operation may be reflected in the steering of the vehicle in a range where the vehicle 1 does not deviate from the travel lane (allowable range). Note that automatic steering is not limited to LKA.

  Automatic speed adjustment is an operating state in which the speed of the vehicle 1 is automatically controlled. Automatic speed adjustment includes ACC (Adaptive Cruise Control). For example, when there is no preceding vehicle ahead of the vehicle 1, ACC performs constant speed control that causes the vehicle 1 to travel at a constant speed at a preset speed, and when the preceding vehicle exists ahead of the vehicle 1. Is a follow-up control that adjusts the vehicle speed of the vehicle 1 in accordance with the inter-vehicle distance from the preceding vehicle. The automatic operation control device 5 decelerates the vehicle 1 according to the driver's brake operation (for example, operation of the brake pedal) even when ACC is being executed. In addition, the automatic operation control device 5 can perform the driver's accelerator operation (for example, accelerator) up to a preset maximum allowable speed (for example, the maximum speed legally determined on the traveling road) even during execution of ACC. The vehicle can be accelerated according to the pedal operation. The automatic speed adjustment is not limited to ACC but also includes CC (Cruise Control).

  In FIG. 1, the vehicle 1 also includes a driving support device 6. The driving support device 6 may have various functions. For example, the driving support device 6 cooperates with a driver monitoring function and a notification device that constitute a driver monitoring system that cooperates with a driver state detection sensor to monitor the driver and acquire the state of the driver. Warning generation function that constitutes a warning generation system that issues a warning according to the state of the driver, operation mode switching control function that constitutes an operation mode switching control system that controls switching of the operation mode according to the state of the driver, biological information It may have a wake-up support function that constitutes a wake-up support system that works with the sensor and the stimulus output device to wake up the driver according to the driver's state.

  The driver's condition is, for example, whether the driver is concentrating on driving, whether the driver is looking aside, whether the driver is drowsy, Whether or not the vehicle is in an automatic driving state or not can be various. The determination of the driver's state is performed as follows, for example.

  First, a driver's monitoring image is acquired by the driver state detection sensor, and based on the acquired monitoring image, the driver's eye opening, blink frequency, eye movement, etc. are detected, and the driver's line of sight is detected. Recognize direction, dwell time, driver's arousal level, etc. Next, the state of the driver is determined by comparing the direction of the driver's line of sight, the residence time, the driver's arousal level, and the like with preset threshold values.

  In this embodiment, the driving support device 6 includes at least a driver monitoring unit 6a that performs the driver monitoring function and an awakening support unit 6b that performs the awakening support function. The driver monitoring unit 6a constitutes a driver monitoring system 8 in cooperation with a driver camera 7 as a driver state detection sensor. In addition, the awakening support unit 6b can generate a plurality of types of stimuli prepared in advance in association with the environmental sensor 9A that detects a plurality of types of environmental conditions around the driver of the vehicle 1 and the plurality of types of environments. The arousal support system 10 is configured to cooperate with the stimulus generation device 9B to promote the driver's arousal by stimulation. In addition, the driver monitoring unit 6a and the awakening support unit 6b constitute a waking support device 6A. In other words, the driving support device 6 includes the awakening support device 6A.

  Therefore, the state of the driver determined by the driver monitoring unit 6a includes at least a state whether or not the driver is drowsy.

  The driver camera 7 is installed at a position in front of the driver, such as on the dashboard, for example, and images the driver and outputs the video signal to the driving support device 6.

  The stimulus generator 9B includes a plurality of generators that respectively generate a plurality of types of stimuli. The environment sensor 9A includes a plurality of sensors corresponding to a plurality of types of stimulus generators of the stimulus generator 9B. FIG. 2 is a diagram illustrating an example of a correspondence relationship between each generation device included in the stimulus generation device 9B and each sensor included in the environment sensor 9A.

  The stimulus generator 9B includes, for example, a sound generator 9B1 that generates a warning sound and a sound in order to urge the driver to awaken. The sound generator 9B1 can use, for example, the voice output function of the navigation system, but may be configured with any device that generates sound.

  The stimulus generator 9B also includes a light generator 9B2 that generates light, for example, to promote the driver's awakening. For example, the room lamp of the vehicle 1 can be used as the light generation device 9B2, but any device may be used as long as it can generate light.

  The stimulus generator 9B further includes, for example, a vibration generator 9B3 that generates vibration in order to promote the driver's awakening. The vibration generator 9B3 is embedded in, for example, the seat seat of the vehicle 1, but the installation location is not limited thereto.

  The stimulus generator 9B further includes a scent generator 9B4 that generates a scent, for example, in order to promote the driver's awakening. For example, an air purifier or a diffuser can be used as the scent generating device 9B4.

  In addition, the stimulus generator 9B is not limited to these generators, and may include another generator that generates another stimulus in order to promote the driver's awakening.

  The environmental sensor 9A includes a noise sensor 9A1 that detects noise around the vehicle 1 and in the vehicle in order to detect whether or not the sound generator 9B1 of the stimulus generator 9B effectively operates.

  The environmental sensor 9A also includes a solar radiation sensor 9A2 that detects the intensity of sunlight with respect to the vehicle 1 in order to detect whether or not the light generator 9B2 of the stimulus generator 9B works effectively. The solar radiation sensor 9A2 may be used also as, for example, a solar radiation sensor used for controlling an automatic air conditioner.

  The environmental sensor 9A further includes a vibration sensor 9A3 that detects the vibration of the vehicle 1 in order to detect whether or not the vibration generator 9B3 of the stimulus generator 9B effectively operates.

  The environment sensor 9A further includes a window sensor 9A4 that detects the opening / closing amount of the window of the vehicle 1 in order to detect whether or not the scent generation device 9B4 of the stimulus generation device 9B effectively operates.

  In addition, the environmental sensor 9A has a sensor for detecting whether or not the other stimulus generator effectively works corresponding to the other stimulus generator of the stimulus generator 9B. Also good.

  The arousal support device 6A is a device that cooperates with the environment sensor 9A and the stimulus generation device 9B to promote the driver's arousal when necessary, for example, when the driver is drowsy. It is configured. FIG. 3 is a block diagram showing the functional configuration.

  The awakening support device 6 </ b> A includes a control unit 61, an input / output interface unit 62, and a storage unit 63.

  The input / output interface unit 62 converts the input analog signal into digital data and outputs the digital data to the control unit 61, and converts the digital data output from the control unit 61 into an analog signal and outputs the analog signal. Specifically, the input / output interface unit 62 receives the video signal output from the driver camera 7, converts it into digital data, and outputs it to the control unit 61. The input / output interface unit 62 also converts the detection signal output from the environment sensor 9 </ b> A into digital data and outputs the digital data to the control unit 61. The input / output interface unit 62 further converts the stimulus information data output from the control unit 61 into a signal and outputs the signal to the stimulus generator 9B.

  The storage unit 63 uses a non-volatile memory that can be written and read as needed, such as a solid state drive (SSD) and a hard disk drive (HDD), as a storage medium. The storage unit 63 includes a driver monitoring video storage unit 631, a driver state storage unit 632, a stimulus information storage unit 633, and a selection result storage unit 634 as storage areas used for carrying out this embodiment. ing. The storage unit 63 is not limited to a nonvolatile memory, and may be configured by a volatile memory such as a RAM except for the stimulus information storage unit 633, for example.

  The stimulus information storage unit 633 stores, for each stimulus that can be generated by the stimulus generator 9B, information related to the effectiveness of each stimulus set in advance in association with the detection information of each sensor of the environmental sensor 9A corresponding to the stimulus. doing. For example, the stimulus information storage unit 633 includes, as information related to the effectiveness of each stimulus, detection information of each sensor of the environment sensor 9A and effectiveness information indicating the effectiveness of the stimulus set in advance in association with the value. You may have a storage table showing correspondence. Or instead of having such a storage table, the stimulus information storage unit 633 represents the effectiveness of each stimulus with the detection information of each sensor of the environmental sensor 9A as a variable x as information on the effectiveness of each stimulus. A function f (x) of validity information may be stored.

  For example, assuming that construction is being performed near the vehicle 1, the greater the detected value of the noise level around the vehicle 1, the lower the effectiveness of stimulation using sound. . In addition, it is set such that the stronger the sunlight, the lower the effectiveness of stimulation using light. Assuming that the vehicle 1 is traveling on a rough road, the greater the vibration detection level, the lower the effectiveness of stimulation using vibration. Assuming that the outside air is taken in, the effectiveness of the stimulation using the scent becomes lower as the opening of the window is larger. As described above, the effectiveness of the stimulus changes according to the state of the environment around the driver of the vehicle 1.

  The control unit 61 has a CPU (Central Processing Unit) and a program memory that constitute a computer. The control unit 61 includes a driver monitoring video acquisition unit 611, a driver state determination unit 612, a stimulus selection unit 613, and a stimulus information output unit 614 as control functions necessary for carrying out this embodiment. ing. All of these control functions are realized by causing the CPU to execute a program stored in the program memory.

  The driver monitoring video acquisition unit 611 takes in the driver monitoring video information of the driver output from the driver camera 7 via the input / output interface unit 62, and drives the captured driver monitoring video information in the storage unit 63. Stored in the person monitoring video storage unit 631.

  The driver state determination unit 612 reads the driver monitoring video information from the driver monitoring video storage unit 631 at preset time intervals, and each time the driver is drowsy based on the driver monitoring video information. The process which determines whether it exists is performed. The driver state determination unit 612 stores information representing the determination result in the driver state storage unit 632 of the storage unit 63.

  The determination as to whether or not the driver is drowsy is performed, for example, as follows. First, the driver state determination unit 612 detects the driver's eye opening degree, blinking frequency, eye movement, and the like based on the driver monitoring video information, and recognizes the driver's arousal level. Next, the driver state determination unit 612 compares the driver's arousal level with a preset threshold value. If the driver's arousal level is higher than the threshold, the driver state determination unit 612 determines that the driver is not drowsy. Conversely, if the driver's arousal level is lower than the threshold, It is determined that the driver is drowsy.

  The stimulus selection unit 613 determines whether the driver is likely to sleep. For this reason, the stimulus selection unit 613 reads information representing the determination result of whether or not the driver is drowsy from the driver state storage unit 632. When the information indicating the determination result indicates that the driver of the vehicle 1 is drowsy, the stimulus selection unit 613 determines that wake-up support is necessary. When it is determined that awakening support is necessary, the stimulus selection unit 613 is also set in advance in association with the detection information of each sensor of the environment sensor 9A and the detection information stored in the stimulus information storage unit 633. Based on the information on the effectiveness, the stimulus used for the driver's arousal support is selected. The stimulus selection unit 613 causes the selection result storage unit 634 to store type information indicating the type of the selected stimulus.

  Based on the selection result stored in the selection result storage unit 634, the stimulus information output unit 614 outputs stimulus information for generating a stimulus used for assisting the driver to awaken to the stimulus generator 9B, and the stimulus Is generated in the stimulus generator 9B.

(Operation example)
Next, an operation example of the awakening support apparatus configured as described above will be described. FIG. 4 is a flowchart showing the overall control procedure and control contents.

(1) Monitoring start When driving is started, the driver monitoring system 8 starts monitoring the driver in step S1. Driver monitoring continues, for example, during driving. The driver is monitored as follows, for example.

  When driving is started, the driver camera 7 is activated, and continuously captures a predetermined range including the driver's face and outputs the video signal. In this state, the awakening support device 6A takes in the driver monitoring video information output from the driver camera 7 via the input / output interface unit 62 under the control of the driver monitoring video acquisition unit 611, and takes the driver in The monitoring video information is stored in the driver monitoring video storage unit 631 of the storage unit 63.

  In addition, you may perform a driver | operator's imaging intermittently at a predetermined time interval. In addition, the driver camera 7 or the input / output interface unit 62 may encode the video signal according to a predetermined encoding method. In this way, it is possible to reduce the amount of driver monitoring video information and save the storage capacity of the driver monitoring video storage unit 631.

  When the acquisition of the driver monitoring video information is started, the awakening support device 6 </ b> A determines the state of the driver every time when a certain period of time has been determined under the control of the driver state determination unit 612. Do. The determination time interval of the driver may be set to a short interval of about 1 second so that a substantially continuous determination can be performed, for example, or set to a relatively long interval of 10 to 30 seconds. May be.

  The driver state determination unit 612 reads the driver monitoring video information from the driver monitoring video storage unit 631 at a preset time interval. Next, every time the driver monitoring video information is read, the driver state determination unit 612 determines whether or not the driver is drowsy based on the driver monitoring video information. The driver state determination unit 612 stores information indicating the determination result in the driver state storage unit 632 of the storage unit 63 in association with information indicating the determination timing, for example, time stamp information.

(2) Determination of drowsiness When monitoring of the driver is started, the wakefulness support device 6A determines whether or not the driver is drowsy in step S2 under the control of the stimulus selection unit 613, that is, the driver. Process to determine whether or not is sleepy.

  As described above, the driver state determination unit 612 determines the state of the driver at regular time intervals, and stores information indicating the determination result in the driver state storage unit 632. The determination result of the driver's state includes information representing the determination result as to whether or not the driver is drowsy.

  The stimulus selection unit 613 reads information representing a determination result of whether or not the driver is drowsy from the driver state storage unit 632 at regular time intervals. If the information representing the determination result indicates that the driver is not drowsy, the process proceeds to step S6 for determining the end of driving. On the other hand, when the information indicating the determination result indicates that the driver is drowsy, the process proceeds to step S3 for selecting an optimal stimulus.

(3) Selection of Optimal Stimulus The stimulus selection unit 613 takes in the detection information of each sensor of the environment sensor 9A in step S3. The stimulus selection unit 613 also reads information related to the effectiveness of each corresponding stimulus from the stimulus information storage unit 633 based on the detection information of each sensor of the environmental sensor 9A, and acquires the effectiveness information representing the effectiveness of each stimulus. To do. For example, the stimulus selection unit 613 stores each stimulus based on a storage table that is stored in advance in the stimulus information storage unit 633 and represents a correspondence relationship between detection information of each sensor of the environment sensor 9A and effectiveness information of each stimulus. You may acquire the validity information. Alternatively, instead of reading the effectiveness information of each stimulus from the stimulus information storage unit 633, the stimulus selection unit 613 uses the detection information of each sensor of the environment sensor 9A stored in the stimulus information storage unit 633 in advance as the variable x. A function f (x) of effectiveness information representing the effectiveness of each stimulus to be read may be read from the stimulus information storage unit 633, and the effectiveness information of each stimulus may be acquired by calculation according to the function f (x). . The stimulus selection unit 613 selects the optimal stimulus, that is, the stimulus having the highest effectiveness by comparing the effectiveness information of all the stimuli, and stores the selected stimulus type information in the selection result storage unit 634. Remember.

(4) Stimulus information output In step S4, the stimulus information output unit 614 reads information indicating the selection result of the stimulus selection unit 613, that is, type information indicating the type of optimal stimulus from the selection result storage unit 634, and outputs the stimulus. Is output to the stimulus generator 9B.

  The stimulus generator 9B generates an optimum stimulus selected by the stimulus selector 613 based on the input stimulus information. Thus, since the optimal stimulus selected based on the surrounding environment of the driver is generated, the driver's awakening is the most effective among the awakening support using the stimulus that can be generated by the stimulus generating device 9B. You can expect to be inspired by

(6) Sleepiness Re-determination After outputting the stimulus information from the stimulus information output unit 614, after a preset period has elapsed, the stimulus selection unit 613 sufficiently improves the driver's sleepiness in step S5. It is determined again whether or not. This re-determination is performed by the same process as step S2 for determining whether or not the driver is drowsy.

  That is, the stimulus selection unit 613 reads information representing a determination result of whether or not the driver is drowsy from the driver state storage unit 632. If the information indicating the result indicates that the driver is not drowsy, it is determined that the drowsiness of the driver has been sufficiently improved, and the process proceeds to step S2 for determining whether the driver is drowsy. Return. On the other hand, when the information indicating the result indicates that the driver is drowsy, the process proceeds to step S6 for determining the end of driving.

(9) Driving Completion Determination In step S6, the awakening support device 6A determines whether driving has ended. If it is determined that the driving has not ended, the process returns to step S2 for determining the driver's sleepiness. On the other hand, when it is determined that the driving is finished, the control of the awakening support is finished.

(Another operation example)
Next, another example of operation of the awakening support device configured as described above will be described. FIG. 5 is a flowchart showing the overall control procedure and control contents.

(1) Monitoring start When driving is started, the driver monitoring system 8 starts monitoring the driver in step T1. Driver monitoring continues, for example, during driving. The driver monitoring process is the same as step S1 in the above-described operation example.

(2) Determination of sleepiness When monitoring of the driver is started, the wakefulness support device 6A determines whether or not the driver is drowsy in step T2 under the control of the stimulus selection unit 613, that is, the driver. Process to determine whether or not is sleepy. The process of determining the driver's sleepiness is the same as step S2 in the above-described operation example.

  When the information indicating the determination result indicates that the driver is not drowsy, the process proceeds to step T10 for determining the end of driving. On the other hand, when the information indicating the determination result indicates that the driver is drowsy, the process proceeds to the process of step T3 for selecting an appropriate stimulus.

(3) Selection of stimulus The stimulus selection unit 613 takes in the detection information of each sensor of the environment sensor 9A in step T3. The stimulus selection unit 613 also reads information related to the effectiveness of each corresponding stimulus from the stimulus information storage unit 633 based on the detection information of each sensor of the environmental sensor 9A, and acquires the effectiveness information representing the effectiveness of each stimulus. To do. As described in the previous operation example, for example, the stimulus selection unit 613 associates the detection information of each sensor of the environmental sensor 9A and the effectiveness information of each stimulus, which are stored in advance in the stimulus information storage unit 633. The effectiveness information of each stimulus may be acquired based on a storage table representing the relationship. Alternatively, instead of reading the effectiveness information of each stimulus from the stimulus information storage unit 633, the stimulus selection unit 613 uses the detection information of each sensor of the environment sensor 9A stored in the stimulus information storage unit 633 in advance as the variable x. A function f (x) of effectiveness information representing the effectiveness of each stimulus to be read may be read from the stimulus information storage unit 633, and the effectiveness information of each stimulus may be acquired by calculation according to the function f (x). . Subsequently, the stimulus selection unit 613 compares the effectiveness information of all the stimuli with a preset threshold value, determines that the effectiveness information is appropriate if the effectiveness information is equal to or greater than the threshold value, and the effectiveness information is less than the threshold value. If so, it is determined to be inappropriate.

  Next, in step T4, the stimulus selection unit 613 determines whether there is an appropriate stimulus, that is, whether there is an appropriate stimulus. If there is an appropriate stimulus, the process proceeds to step T6 for determining the number of appropriate stimuli. On the other hand, if there is no appropriate stimulus, the process proceeds to step T5 for selecting the optimum stimulus.

  When there is no appropriate stimulus, the stimulus selection unit 613 selects an optimal stimulus, that is, a stimulus having the highest effectiveness by comparing all pieces of effectiveness information in Step T5, and selects the selected stimulus. Is stored in the selection result storage unit 634.

  When there is an appropriate stimulus, the stimulus selecting unit 613 determines whether or not there are a plurality of appropriate stimuli in Step T6. When the number of appropriate stimuli is not plural, that is, when there is only one appropriate stimulus, the single appropriate stimulus is selected, and the type information indicating the type of the selected stimulus is selected. The data is stored in the storage unit 634. In this case, the single appropriate stimulus selected is considered the optimal stimulus. On the other hand, when there are a plurality of appropriate stimuli, all the appropriate stimuli are selected and the type information of each selected stimulus is stored in the selection result storage unit 634.

(4a) Output of Single Stimulus Information and Re-Determination of Sleepiness Step T6 for determining the presence or absence of an appropriate stimulus, as a result of the determination in step T4, or for determining the number of appropriate stimuli If the number of appropriate stimuli is one as a result of the determination, the stimulus information output unit 614 obtains the type information indicating the type of the stimulus selected by the stimulus selection unit 613 from the selection result storage unit 634 in Step T7. The stimulus information for reading and generating the stimulus represented by the type information in the stimulus generator 9B is output to the stimulus generator 9B.

  The stimulus generator 9B generates an optimum stimulus selected by the stimulus selector 613 based on the input stimulus information. As described above, since the optimal stimulus selected based on the environmental condition around the driver of the vehicle 1 is generated, the driver's awakening support using the stimulus that can be generated by the stimulus generation device 9B. Awakening can be expected to be most effectively promoted.

  Following the output of the stimulation information by the stimulation information output unit 614, after a preset period has elapsed, the stimulation selection unit 613 determines again whether or not the driver's sleepiness has been sufficiently improved in step T8. . This re-determination is performed by the same process as step T2 for determining whether or not the driver is drowsy.

  That is, the stimulus selection unit 613 reads information representing a determination result of whether or not the driver is drowsy from the driver state storage unit 632. When the information indicating the read determination result indicates that the driver is not drowsy, it is determined that the drowsiness of the driver has been sufficiently improved, and it is determined whether or not the driver is drowsy step T2 Return to the process. On the other hand, when the information indicating the result indicates that the driver is still drowsy, the process proceeds to step T10 for determining the end of driving.

(4b) Output of a plurality of stimulus information and re-determination of sleepiness On the other hand, if the number of appropriate stimuli is plural as a result of the determination in step T6 for determining the number of appropriate stimuli, the stimulus information output unit 614 In step T9, the plurality of pieces of stimulus information are read so that the type information indicating the type of the stimulus selected by the stimulus selecting unit 613 is read from the selection result storage unit 634, and the appropriate types of these stimuli are sequentially generated once. Are sequentially output to the stimulus generator 9B. At this time, for example, a plurality of types of appropriate stimuli may be ordered according to their effectiveness, and stimulus information may be output according to the order. In addition, you may make it output irritation | stimulation information in order to generate the said multiple types of irritation | stimulation simultaneously.

  In order to support the driver's arousal, the optimal stimulus selected based on the environmental conditions around the driver of the vehicle 1 is not necessarily optimal for the driver's arousal support depending on the driver's constitution and physical condition. Not always. In some cases, a stimulus other than the optimum stimulus selected based on the state of the environment around the driver of the vehicle 1 acts more effectively on the driver's arousal support. When a plurality of appropriate stimuli are generated in order, it can be expected that any of the plurality of appropriate stimuli effectively acts on the driver's arousal support.

  Following the output of each stimulus information by the stimulus information output unit 614, after a preset period has elapsed, the stimulus selection unit 613 determines again whether or not the driver's drowsiness has been sufficiently improved. This re-determination is performed by the same process as step T2 for determining whether or not the driver is drowsy.

  That is, the stimulus selection unit 613 reads information representing a determination result of whether or not the driver is drowsy from the driver state storage unit 632. When the information indicating the result indicates that the driver is not drowsy, it is determined that the drowsiness of the driver has been sufficiently improved, and the process of Step T2 is performed to determine whether the driver is drowsy. Return. On the other hand, if the information indicating the result indicates that the driver is still drowsy, if there is stimulus information to be output next, the next stimulus information is output. If there is no stimulus information to be output, the process proceeds to step T10 for determining the end of driving.

(9) Driving end determination In step T10, the awakening support device 6A determines whether driving has ended. If it is determined that the driving has not ended, the process returns to the process of step T2 for determining the driver's sleepiness. On the other hand, when it is determined that the driving is finished, the control of the awakening support is finished.

(effect)
As described above in detail, in the embodiment of the present invention, the driver state determination unit 612 determines the driver state based on the detection information of the driver camera 7 that is a driver state detection sensor that detects the driver state. Further, it is determined by the stimulus selection unit 613 whether or not the driver is drowsy. When the determination result indicates that the driver is drowsy, a stimulus selected based on the state of the environment around the driver of the vehicle is generated.

  When the optimal stimulus is generated, it can be expected that the driver's awakening is most effectively promoted in the awakening support using the stimulus that can be generated by the stimulus generation device 9B.

  When a plurality of appropriate stimuli are generated, it can be expected that other stimuli effectively act on the driver's arousal support even if the optimal stimulus does not act effectively on the driver's arousal support.

[Other Embodiments]
In the embodiment, the case where the driver state detection sensor is configured by the driver camera 7 and the state of the driver is determined based on the video signal including the driver's face obtained by the driver camera 7 has been described as an example. However, the driver state detection sensor is not limited to the driver camera 7, and is configured by a biological sensor that acquires biological information of the driver, and is detected by a biological signal obtained by the biological sensor, for example, a pulse wave sensor or a heart rate sensor. The state of the driver may be determined based on the pulse wave signal or heartbeat signal of the driver or a signal representing the vertical movement of the diaphragm detected by the pressure sensor.

  In the embodiment, the stimulation generators 9B1 to 9B4 included in the stimulus generator 9B and the sensors 9A1 to 9A4 included in the environment sensor 9A correspond to each other on a one-to-one basis. However, the present invention is not limited to this, and a plurality of sensors may correspond to one stimulus generator. For example, in addition to the window sensor 9A4, an odor sensor may correspond to the scent generating device 9B4.

  In the embodiment, an example of supporting the driver's awakening when the driver becomes drowsy during manual driving of the vehicle has been described. However, the present invention is not limited to such a situation, and the driver can be used in various situations. You may make it support awakening. For example, the following situation may be used as a situation for supporting the driver's awakening.

  Unlike the current situation where it is desirable to leave driving operations to fully automatic driving control devices from the viewpoint of safety, the future without worrying to leave driving operations to fully automatic driving control devices In the situation where it is necessary to switch to manual driving for a while during automatic driving of the vehicle, the sleepiness is such that it is determined that the driver is sleeping or that it is difficult for the driver to manually drive the vehicle When the driver is hosting, the driver's awakening may be supported.

  In addition, the vehicle type, the function of the automatic driving control device, the control function and control procedure of the wake-up support device, and the control content can be variously modified and implemented without departing from the gist of the present invention.

  In short, the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Further, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, you may combine suitably the component covering different embodiment.

A part or all of the above embodiment can be described as in the following supplementary notes, but is not limited thereto.
(Appendix 1)
A wake-up support device that promotes driver wake-up, comprising: a memory having a driver state storage unit, a stimulus information storage unit, and a selection result storage unit; and at least one hardware processor coupled to the memory ,
The at least one hardware processor comprises:
Based on the detection information of the driver state detection sensor that detects the state of the driver, it is determined whether or not the driver is drowsy, and information indicating the determination result is stored in the driver state storage unit. Let
When the information indicating the determination result is read from the driver state storage unit, and the information indicating the determination result indicates that the driver is drowsy, a plurality of types of environmental conditions around the driver The detection information of the environment sensor for detecting the detection information and the plurality of types of stimulations stored in advance in the stimulation information storage unit in advance in association with the plurality of types of environments in advance in association with the detection information. Based on the information on the set effectiveness, the stimulus used for the driver's arousal support is selected from the plurality of types of stimuli, and the type information indicating the type of the selected stimulus is stored in the selection result To remember
The type information is read from the selection result storage unit, and the stimulus information for generating the stimulus represented by the type information is output to the stimulus generator capable of generating the plurality of types of stimuli. Awakening support device.
(Appendix 2)
An awakening support method executed by a device that prompts the driver to awaken,
A hardware processor is used to determine whether or not the driver is drowsy based on detection information of a driver state detection sensor that detects the state of the driver, and information indicating the determination result is stored in a memory. Remember
When the information indicating the determination result is read from the memory using the hardware processor, and the information indicating the determination result indicates that the driver is drowsy, a plurality of types around the driver The detection information of the environmental sensor that detects the state of the environment and the plurality of types of stimuli that are stored in advance in the memory in association with the plurality of types of environment in association with the detection information Based on information on the effectiveness set in advance, a stimulus to be used for the driver's arousal support is selected from the plurality of types of stimuli, and type information indicating the type of the selected stimulus is stored in the memory. Remember,
Using the hardware processor, the type information is read from the memory, and stimulus information for generating a stimulus represented by the type information is output to a stimulus generator capable of generating the plurality of types of stimuli. Awakening support method.

DESCRIPTION OF SYMBOLS 1 ... Vehicle, 2 ... Power unit, 3 ... Steering device, 3a ... Steering wheel, 5 ... Automatic driving control device, 6 ... Driving support device, 6A ... Awakening support device, 6a ... Driver monitoring part, 6b ... Awakening support part, DESCRIPTION OF SYMBOLS 7 ... Driver camera, 8 ... Driver monitoring system, 9A ... Environmental sensor, 9A1-9A4 ... Sensor, 9A1 ... Noise sensor, 9A2 ... Solar radiation sensor, 9A3 ... Vibration sensor, 9A4 ... Window sensor, 9B ... Stimulus generator, 9B1 to 9B4 ... generating device, 9B1 ... sound generating device, 9B2 ... light generating device, 9B3 ... vibration generating device, 9B4 ... fragrance generating device, 10 ... arousal support system, 11 ... steering sensor, 12 ... accelerator pedal sensor, 13 ... Brake pedal sensor, 14 ... GPS receiver, 15 ... Gyro sensor, 16 ... Vehicle speed sensor, 61 ... Control unit, 62 ... Input / output interface Ace unit, 63 ... storage unit, 611 ... driver monitoring video acquisition unit, 612 ... driver state determination unit, 613 ... stimulus selection unit, 614 ... stimulus information output unit, 631 ... driver monitoring video storage unit, 632 ... driving Person state storage unit, 633... Stimulation information storage unit, 634... Selection result storage unit.

Claims (6)

A wake-up support device that promotes driver wake-up,
A driver state determination unit that determines whether or not the driver is drowsy based on detection information of a driver state detection sensor that detects the state of the driver;
When the driver state determination unit determines that the driver is drowsy, based on detection information of an environmental sensor that detects a plurality of types of environmental conditions around the driver, the driver A stimulus selection unit that selects a stimulus to be used for awakening support from a plurality of types of stimuli prepared in advance in association with the plurality of types of environments;
An arousal support device comprising: a stimulus information output unit that outputs stimulus information for generating the stimulus selected by the stimulus selection unit with respect to the stimulus generation device capable of generating the plurality of types of stimuli.   For each of the plurality of types of environments, the stimulus selection unit is based on a storage table in which detection information of the environment sensor and validity information indicating the effectiveness of a stimulus set in advance in association with the detection information are associated with each other. The stimulus effectiveness information corresponding to the detection information obtained by the environmental sensor is acquired, and the stimulus information that is optimal for assisting the driver's arousal is selected by comparing the effectiveness information. Awakening support device.   The stimulus information output unit outputs the stimulus information to the stimulus generator so as to generate a plurality of stimuli sequentially or simultaneously when the number of stimuli selected by the stimulus selection unit is plural. The awakening support apparatus according to claim 1.   The stimulus selection unit, for each of the plurality of types of environment, the detection obtained by the environment sensor by calculation according to a function of effectiveness information representing the effectiveness of the stimulus using the detection information of the environment sensor as a variable The wakefulness support apparatus according to claim 1, wherein the stimulus effectiveness information corresponding to the information is acquired, and the stimulus information that is most suitable for the driver's wakeup support is selected by comparing the effectiveness information. An awakening support method executed by a device that prompts the driver to awaken,
A driver state determination step for determining whether or not the driver is drowsy based on detection information of a driver state detection sensor that detects the state of the driver;
Stimulus used to support the driver's arousal based on detection information of an environmental sensor that detects a plurality of types of environmental conditions around the driver when it is determined that the driver is drowsy A stimulus selection step of selecting from a plurality of types of stimuli prepared in advance in association with the plurality of types of environments;
An awakening support method, comprising: a stimulus information generation step of outputting stimulus information for generating the stimulus selected in the stimulus selection step with respect to the stimulus generator capable of generating the plurality of types of stimuli.   The program which makes a computer perform the function of each part which the awakening assistance apparatus as described in any one of Claims 1 thru | or 4 comprises, or the process of each step of the awakening assistance method of Claim 5.