JP6776681B2 - Driver status determination device and driver status determination program - Google Patents

Description

The disclosure according to this specification relates to a driver state determination device for determining a driver state and a driver state determination program.

Conventionally, there is known a technique of determining, for example, a decrease in arousal level as a state of a driver and performing an actuation to increase the arousal level of the driver. For example, the arousal maintenance device disclosed in Patent Document 1 is a stimulus having an awakening effect when the driver's drowsiness level is lowered and the driver's movement, biological signal, vehicle behavior, etc. associated with drowsiness are further detected. To the driver. In this way, by giving a stimulus having an awakening effect at a timing that matches the subjectivity and feeling of the driver, the driver can accept the stimulus without discomfort.

Japanese Patent No. 4518083

Now, the inventor of the present disclosure has focused on the fact that a driver with reduced alertness becomes distracted before falling into a drowsy or drowsy state. In such a distracted state, the driver is likely to look aside. Therefore, the inventor of the present disclosure has come up with the idea of determining a distracted state based on inattentiveness and carrying out an actuation that enhances the arousal level of the driver.

In the process of developing such technology, there are two types of inattentiveness in a distracted state: inattentiveness due to interest in a state where alertness is relatively maintained, and inattentiveness due to boredom in a state where alertness is relatively low. Was newly found. However, Patent Document 1 does not disclose any technique for detecting a distracted state or discriminating an inattentive state in a distracted state. When it is not possible to discriminate inattentiveness in such a distracted state, the actuation for the driver who inattentive in the distracted state has to be the same.

As a result, there is a risk that the driver who looks aside from the interest will be too stimulated by the actuation, and the driver's consciousness will be raised to the hypervigilance state. On the other hand, the driver who looked aside due to boredom lacked the stimulation by the actuation, and it could be difficult to restore the driver's consciousness to a state where he could drive correctly.

The present disclosure has been made in view of the above problems, and an object thereof is to provide a technique for appropriately distinguishing a state of distraction in a driver and contributing to the implementation of an actuation in an appropriate manner. It is in.

In order to achieve the above object, one of the disclosed aspects is an automatic driving function that can support or substitute the driving operation by the driver, and the mode of the act that enhances the alertness of the driver can be changed according to the state of the driver. A driver status determination device that determines the driver's status in a vehicle (A) equipped with various actuation function units (70, 270), and acquires inattentive information indicating the inattentive state of the driver. Units (31, 35), arousal level information acquisition unit (33, 36) that acquires arousal level information indicating the driver's arousal level, and arousal level that is equal to or less than the preset calm arousal level based on the arousal level information. determining a reduction in, an determination unit for determining whether or not the inattentive based on the inattentive information occurs when it is determined that a further inattentive, does not drop below undisturbed wakefulness objective醒度determines that inattentive occurs by curiosity if a determination unit inattention by boring occurs when the objective醒度has decreased to not more than calm alertness and (37), looking aside If it is determined that there is to output to a judgment result indicating whether the inattention by boring or inattention by curiosity to actuation function unit, be determined not to be further inattentive, calm alertness Below, when the arousal level is reduced, the output unit (38) that outputs the determination result indicating the decrease in the arousal level to the actuation function unit and the operation information indicating the operating state of the automatic operation function are displayed. comprising the activation information acquisition unit for acquiring (32, 232), a determination unit, if the automatic cruise function is activated, the driver while mitigating the determining criterion is inattentive state, driving the driver determining the reduction of alertness with calm alertness in order to implement the actuation to maintain alternately ready to actuation function unit, it is a driver determination device.

In addition, another disclosed aspect is an automatic driving function capable of assisting or substituting the driving operation by the driver, and an actuation in which the mode of the act that enhances the arousal level of the driver can be changed according to the state of the driver. In the vehicle (A) equipped with the functional unit (70, 270), a driver status determination program for determining the driver status, the step (S101) of acquiring the inattentive information indicating the inattentive state of the driver, and In the step (S106) of acquiring the arousal level information indicating the driver's arousal level, it was determined based on the arousal level information that the arousal level was lower than the preset calm arousal level, and inattentiveness occurred based on the inattentive information. a determining whether further when it is determined that the driver is looking aside, it is determined that inattention by curiosity occurs when the arousal level has not dropped below calm alertness, arousal degree is a step (S107, S108) determines that the inattentive by boring occurs if you drop below calm wakefulness, when it is determined that the driver is looking aside, the bored or inattentive by curiosity the determination result indicating whether the inattention and outputted to the actuation function unit by further be determined not looking aside, the awakening degree if the degree of awakening below calm alertness is decreased A step (S109) for outputting a determination result indicating a decrease in the value to the actuation function unit and a step for acquiring operation information indicating the operation state of the automatic operation function are provided to at least one processor (30b, 210b). When the automatic driving function is activated, the actuation function unit is provided with an actuation that keeps the driver in a state where the driver can change driving while relaxing the criteria for determining that the driver is in an inattentive state. determining the reduction of alertness with calm alertness order to be a driver condition determination program.

In these aspects, when the distracted driver looks aside, the inattentive information and the alertness information are combined to distinguish between the inattentiveness due to interest and the inattentiveness due to boredom. If the determination result of such inattentiveness is output to the actuation device, the actuation device can appropriately increase the arousal level of the driver by changing the mode of the actuation. Therefore, the driver state determination device or the driver state determination program can accurately determine the state of distraction in the driver and contribute to the implementation of the actuation in an appropriate manner.

The reference numbers in parentheses are merely examples of the correspondence with the specific configuration in the embodiment described later, and do not limit the technical scope at all.

It is a block diagram which shows the whole image of the in-vehicle device including the actuation system by 1st Embodiment. It is a figure which shows typically the concept of the state management of the driver arousal degree. It is a figure which shows the functional block constructed in the state determination apparatus. It is a flowchart which shows the detail of the attention distraction degree determination processing carried out by a state determination apparatus. It is a figure which shows the functional block constructed in the state guidance apparatus. It is a flowchart which shows the detail of the guidance determination processing carried out by a state guidance apparatus. It is a figure which shows an example of the 1st actuary which guides a driver's line of sight. It is a figure which shows an example of the 2nd actument which guides both the driver's line of sight and consciousness. It is a figure which shows an example of the third actument which induces the consciousness of a driver. It is a figure which compares and shows the feature of each actument. It is a block diagram which shows the whole image of the in-vehicle device including the actuation device by 2nd Embodiment. It is a figure which shows the functional block constructed in the actuation device. It is a figure which shows the functional block constructed in the state determination apparatus of 3rd Embodiment.

Hereinafter, a plurality of embodiments of the present disclosure will be described with reference to the drawings. In addition, duplicate description may be omitted by assigning the same reference numerals to the corresponding components in each embodiment. When only a part of the configuration is described in each embodiment, the configurations of the other embodiments described above can be applied to the other parts of the configuration. Further, not only the combination of the configurations specified in the description of each embodiment, but also the configurations of a plurality of embodiments can be partially combined even if the combination is not specified. Further, it is assumed that the unspecified combination of the configurations described in the plurality of embodiments and modifications is also disclosed by the following description.

(First Embodiment)
The actuation system 10 of FIG. 1 according to the first embodiment of the present disclosure is composed of a state determination device 30, a state guidance device 70, and the like. The actuation system 10 guides the driver's driving state to the correct state by increasing the driver's arousal level by acting toward the driver of the vehicle A.

As shown in FIG. 2, the arousal level of the driver may deviate from the state suitable for driving due to fatigue due to long-time driving, continuation of monotonous driving operation, and the like. For example, when the driver's alertness decreases, he / she becomes in a state of drowsiness or strong drowsiness. Further, when the driver's alertness is excessively increased, the driver's alertness becomes a hypervigilance state in which it is difficult to concentrate on driving due to feeling frustrated and impatient. In addition, the driver is in a distracted state before falling into a distracted state due to decreased alertness. The actuation system 10 of FIG. 1, which will be described below, mainly restores the driver's consciousness in a distracted state appropriately, thereby guiding the driver to an awake state suitable for driving without making the driver hyper-awakened. To do.

The actuation system 10 is mounted on the vehicle A together with the driver monitor system 11, the in-vehicle device group 13, the wearable device 19, the automatic driving ECU 20, the display 80, the voice output device 85, the tactile presenter 87, and the like.

The driver monitor system 11 shown in FIGS. 1 and 3 includes an in-vehicle camera 12, a control unit, and the like. The in-vehicle camera 12 is, for example, a near-infrared camera having a near-infrared light source. The in-vehicle camera 12 is installed in the vehicle interior of the vehicle A in a posture in which the image pickup direction is directed toward the driver's seat side. The in-vehicle camera 12 photographs the face of the driver irradiated with near-infrared light by a near-infrared light source. The control unit detects the driver's line-of-sight direction, face-facing direction, eye opening degree, and the like by analyzing the image captured by the in-vehicle camera 12. The driver monitor system 11 sequentially provides the detected information to the state determination device 30.

The in-vehicle device group 13 includes a large number of control devices and a large number of sensors mounted on the vehicle A. As an example, the in-vehicle device group 13 includes the vehicle speed, steering angle, brake operation amount, turn signal operating state, acceleration, distance from the vehicle in front, the shape of the road in progress, and peripheral vehicles running in parallel. Detects information such as the relative position of. The in-vehicle device group 13 sequentially provides various detected information to the state determination device 30.

The wearable device 19 is a device that can detect the biological signal of the driver by being attached to the driver. As an example, the wearable device 19 detects information such as the heart rate and body temperature of the worn driver. The wearable device 19 sequentially provides the detected information to the state determination device 30 by wireless communication. The wearable device 19 may be a wristwatch-type device worn on the driver's wrist, or a glasses-type device worn on the driver's face.

The automatic operation ECU (Electronic Control Unit) 20 is mainly composed of a computer having a CPU, RAM, and ROM. The automatic driving ECU 20 can support or act as a driver's driving operation by formulating a driving plan for the vehicle A and controlling the control system of the vehicle A according to the formulated traveling plan. For example, an automatic operation start button is connected to the automatic operation ECU 20. Based on the input to the automatic operation start button by the passenger such as the driver, the automatic operation ECU 20 changes the operation state of the automatic operation function from the off state (hereinafter, "manual operation mode") to the on state (hereinafter, "automatic operation mode"). ”). The automatic operation ECU 20 can sequentially provide the operation information indicating the operation state of the automatic operation function, that is, the automatic operation mode or the manual operation mode, to the automatic operation ECU 20 and the state guidance device 70. At least a part of the information provided to the state determination device 30 from the in-vehicle device group 13 such as the shape of the road and the relative position of the surrounding vehicles is not from the in-vehicle device group 13 but from the automatic driving ECU 20. May be provided to.

The display 80, the voice output device 85, and the tactile presenter 87 shown in FIG. 1 are configured to present various information related to the vehicle A to the boarding of the vehicle A including the driver. These configurations may be configured to be pre-mounted on the vehicle A, or may be temporarily mounted on the vehicle A by being brought into the vehicle interior by a passenger such as a driver.

The display 80 presents information through the visual sense of a driver or the like by means of a display object such as an image and an icon displayed on the display surface (see FIG. 10). The display 80 includes, for example, a center display 81, a head-up display 82, a room mirror monitor 83, and a mirror indicator 84 (see FIG. 7).

The center display 81 is arranged in the center of the upper surface of the instrument panel. The center display 81 displays various images in color on the display surface. The head-up display 82 projects a light onto the projection area 82a partitioned by the windshield to superimpose a virtual image on the foreground. The rearview mirror monitor 83 is integrally formed with a rearview mirror provided in the vehicle interior. The rearview mirror monitor 83 displays various images in color on the reflecting surface of the rearview mirror. The mirror indicator 84 is integrally formed with an external rearview mirror installed on the left and right doors of the vehicle A. The mirror indicator 84 emits and displays an icon having a predetermined shape on the reflective surface of the rearview mirror outside the vehicle body.

The voice output device 85 presents information through the hearing of a driver or the like by means of a notification sound or a voice message reproduced in the vehicle interior (see FIG. 10). The audio output device 85 has a plurality of speakers. A plurality of speakers are installed in the vehicle interior in an arrangement surrounding the driver. The voice output device 85 can make the driver hear a notification sound, a voice message, or the like from an arbitrary direction by selecting a speaker that outputs sound from a plurality of speakers. The speaker used in the audio output device 85 may have a configuration capable of outputting a directional sound that can be heard only by the driver, for example.

The tactile presenter 87 presents information to the driver through tactile sensation by vibrating or displacing the portion in contact with the driver (see FIG. 10). The tactile presenter 87 has, for example, a steering vibration portion embedded in the rim portion of the steering wheel and a seat vibration portion embedded in the seat surface or backrest of the driver's seat. In addition, the tactile presenter 87 may have, for example, an active footrest that changes its posture in the front-rear and left-right directions of the vehicle A.

Next, the configurations of the state determination device 30 and the state guidance device 70 will be described in order.

The state determination device 30 shown in FIGS. 1 and 3 determines the state of the driver, and provides the state guidance device 70 with the determination result obtained by the determination. The state determination device 30 is provided with a state determination processing unit 30a. The state determination processing unit 30a is an electronic circuit mainly composed of a microcomputer having a CPU 30b, a RAM 30c, a memory 30d, and the like. The memory 30d is a non-transitory tangible storage medium in which information can be read by the CPU 30b. The CPU 30b can execute various programs stored in the memory 30d.

The state determination device 30 causes the CPU 30b to execute the state determination program stored in the memory 30d. As a result, the state determination device 30 is constructed with an inattentive determination device 31, an operation information acquisition unit 32, an arousal level determination device 33, a distraction determination device 34, and the like as functional blocks related to the driver state determination.

The inattentive determination device 31 acquires the line-of-sight and face orientation information provided by the driver monitor system 11 and generates information for determining whether or not the driver is inattentive. In order to easily handle the position viewed by the driver, the inattentive determination device 31 is preset with a provision that divides the visible range in front of the driver into a plurality of areas (“1” to “7”).

In this regulation, for example, the range of the windshield is divided into three in the horizontal direction, and numbers "1" to "3" are assigned to each. Further, the area under the windshield is set to "4" around the center console and "5" around the steering wheel. Further, the left and right sides of the windshield are set to "6" and "7", respectively. The inattentive determination device 31 determines which region the driver is looking at in the predetermined regions "1" to "7" by integrating the information on the line of sight and the face orientation.

In addition, when the driver's viewing position (hereinafter referred to as "line-of-sight position") is continuously deviated from the front area of "1" to "3", the inattentive determination device 31 is deviated from the front. (Hereinafter, "fixed time for inattentiveness") is measured. The inattentive determination device 31 provides the distraction determination device 34 with a number indicating the driver's line-of-sight position and an inattentive fixed time as inattentive information indicating the driver's inattentive state.

The operation information acquisition unit 32 acquires operation information indicating the operation state of the automatic operation function from the automatic operation ECU 20. The operation information acquisition unit 32 provides the acquired operation information to the alertness determination device 33 and the distraction determination device 34.

The arousal level determination device 33 is information for determining a decrease in the arousal level of the driver based on the information on the degree of eye opening provided by the driver monitoring system 11, various information provided by the in-vehicle device group 13 and the wearable device 19. , Specifically, it generates a determination alertness. The judgment arousal level becomes a higher value as the driver's arousal level increases, and becomes a lower value as the driver's arousal level decreases.

The arousal degree determination device 33 extracts the opening / closing eye pattern, the eyelid opening / closing time, and the eye opening amount from the transition of the eye opening degree provided by the driver monitor system 11. The arousal level determination device 33 calculates the driver's arousal level low when the frequency of blinking increases, the closing time of eyes at the blinking increases, the amount of eye opening decreases, and the like.

In addition, the arousal level determination device 33 determines the decrease in the arousal level based on the operation information of the driving operation input to the vehicle A. The arousal degree determination device 33 monitors the history of operation information linked to the driver's driving operation, for example, accelerator operation and braking operation estimated from the vehicle speed and acceleration, and steering operation estimated from the steering angle. It is possible to detect changes in the operation pattern of.

As a result, the alertness determination device 33 compares the change in the inter-vehicle distance with the vehicle in front with each history of the driver's accelerator operation and brake operation, thereby delaying the response of the accelerator operation or the brake operation to the change in the inter-vehicle distance. Can be detected. Therefore, the alertness determination device 33 can calculate the driver's alertness lower as the response delay between the accelerator operation and the brake operation increases.

Further, the alertness determination device 33 can detect a delay in starting the steering operation when entering a curve by comparing the road shape with the history of the driver's steering operation. Therefore, the alertness determination device 33 can calculate the driver's alertness lower as the start delay of the steering operation increases.

Further, the alertness determination device 33 calculates the driver's alertness lower as the input frequency of the correction rudder decreases in the straight section. Further, even when it is estimated that the monotonous driving operation is continued based on the change of the road shape and the vehicle speed, the decrease in the arousal level of the driver is estimated.

Further, the alertness determination device 33 estimates the driving load caused by a complicated road environment or a mixture of roads. In the scene where the driving operation is supposed to be busy under a high driving load, the alertness determination device 33 is insufficient in each operation of the driver based on the operation information, or the heart rate or the like remains low based on the biological signal. In some cases, the driver's alertness is calculated low.

The arousal level determination device 33 acquires a determination arousal level that quantitatively evaluates the arousal level of the driver by appropriately combining the above-mentioned information. For example, the arousal level determination device 33 suspends the use of the operation information for the arousal level determination in the automatic operation mode in which the automatic operation function is operating based on the operation information acquired from the operation information acquisition unit 32. Then, the judgment alertness is calculated using only the eye opening degree and the biological signal. The arousal level determination device 33 provides the acquired determination arousal level to the distraction determination device 34 as arousal level information indicating a decrease, maintenance, increase, etc. of the driver's arousal level.

The distraction determination device 34 determines the degree of distraction of the driver by combining the inattentive information provided by the operation information acquisition unit 32 and the arousal level information provided by the arousal level determination device 33. The distraction determination device 34 is further constructed with an inattentive information acquisition unit 35, an arousal level information acquisition unit 36, a determination unit 37, an output unit 38, and the like as sub-function blocks for determining the degree of distraction. ..

The inattentive information acquisition unit 35 acquires the line-of-sight information and the inattentive fixing time from the inattentive determination device 31 as the inattentive information of the driver. The arousal level information acquisition unit 36 acquires the determination arousal level from the awakening level determination device 33 as the driver's arousal level information.

The determination unit 37 determines the occurrence of inattentiveness in the driver based on the inattentive information acquired by the inattentive information acquisition unit 35. When the driver is inattentive, the determination unit 37 determines the type of inattentiveness of the driver based on the arousal level information acquired by the arousal level information acquisition unit 36 (see FIG. 2). Specifically, the determination unit 37 determines that the inattentiveness when the determination arousal level is not lowered is the inattentiveness based on interest. On the other hand, the determination unit 37 determines that the inattentiveness when the arousal level is low is the inattentiveness due to boredom.

Based on the operation information acquired by the operation information acquisition unit 32, the determination unit 37 relaxes the determination criteria for determining that the driver is in the inattentive state when the automatic operation ECU 20 is in the automatic operation mode. In addition, even if it is not determined that the inattentiveness has occurred based on the inattentive information, the determination unit 37 determines that the alertness is distracted due to the decrease in the arousal level when the determination arousal level is lowered. ..

The output unit 38 sequentially outputs the determination result by the determination unit 37 toward the state guidance device 70. The output unit 38 determines whether the inattentiveness is due to interest or boredom (hereinafter, "inattentiveness determination result"), or the determination result in which the arousal level is lowered when no inattentiveness occurs (hereinafter, "inattentiveness determination result"). The arousal level determination result) can be provided to the state guidance device 70.

The details of the distraction degree determination process performed by the distraction determination device 34 will be described with reference to FIGS. 1 and 3 based on FIG. The attention distraction degree determination process is started, for example, based on the fact that the shift position of the vehicle A is set to "D (drive)" and the vehicle A is in a state where it can travel forward. The attention distraction degree determination process is repeatedly started until the shift position of the vehicle A is changed from "D".

In S101, the inattentive information is acquired from the inattentive determination device 31. Then, based on the inattentive information acquired from the inattentive determination device 31, it is determined whether or not the line-of-sight position is out of the region of "1" to "3", that is, from the front. If it is determined in S101 that the line-of-sight position of the driver is out of the region of "1" to "3", the process proceeds to S104. On the other hand, if it is determined in S101 that the line-of-sight position of the driver is in any of the regions "1" to "3", the process proceeds to S102.

In S102, the determination arousal level is acquired from the arousal level determination device 33, and the determination arousal level is compared with the preset calm arousal level. The calm arousal level is a threshold value for determining whether or not the driver's arousal level has decreased in the distracted state (see FIG. 2). The calm arousal level is preset, for example, based on evaluation data that objectively evaluates a decrease in arousal level in a plurality of subjects. When it is determined in S102 that the determination arousal level is higher than the calm arousal level, it is estimated that the driver's arousal level is sufficiently maintained, and the process returns to S101. On the other hand, if it is determined in S102 that the determination arousal level is equal to or less than the calm arousal level, the process proceeds to S103. In S103, an awakening degree determination result indicating that the awakening degree of the driver has decreased is generated, and the process proceeds to S109.

On the other hand, in S104 when it is determined in S101 that the line-of-sight position of the driver is not in front, it is determined whether or not the driver is in operation checking the side. Specifically, in S104, when the line-of-sight position is "6" or "7" and the turn signal is operating, it is estimated that the operation to confirm the side is in progress, and the process returns to S101. .. On the other hand, if the line-of-sight position is not "6" or "7", or if the turn signal is not operating, it is presumed that the side confirmation operation is not in progress, and the process proceeds to S105.

In S105, the inattentive fixed time indicated by the inattentive information acquired in S101 is compared with the preset inattentive determination time. The inattentive determination time may be, for example, a constant value or a value that is changed according to the vehicle condition. For example, the inattentive determination time may be shortened as the vehicle speed of the vehicle A increases. In addition, when the automatic driving function is activated, the criteria for inattentive determination may be relaxed by controlling the inattentive determination time to be adjusted longer. When it is determined in S105 that the inattentive fixing time is equal to or less than the inattentive determination time, it is determined that no inattentiveness has occurred, and the process returns to S101. On the other hand, if it is determined in S105 that the inattentive fixing time exceeds the inattentive determination time, it is determined that inattentiveness has occurred, and the process proceeds to S106.

In S106, the determination arousal level is acquired from the arousal level determination device 33 as in S102, and the determination arousal level and the calm arousal level are compared. If it is determined in S106 that the determination arousal level is higher than the calm arousal level, the process proceeds to S107. In S107, with respect to the inattentiveness determined to occur in S105, an inattentiveness determination result indicating that the inattentiveness is due to the driver's interest is generated, and the process proceeds to S109. On the other hand, if it is determined in S106 that the determination arousal level is equal to or less than the calm arousal level, the process proceeds to S108. In S108, with respect to the inattentiveness determined to occur in S105, an inattentiveness determination result indicating that the inattentiveness is due to the boredom of the driver is generated, and the process proceeds to S109.

In S109, the determination result generated by any one of the processes S103, S107, and S108 is output to the state guidance device 70, and a series of distraction determination processes is completed.

The state guidance device 70 shown in FIGS. 1 and 5 executes an actuation using the display 80, the voice output device 85, and the tactile presenter 87. The state guidance device 70 can change the mode of the actuation according to the state of the driver. The state guidance device 70 guides the driver's operating state to the correct state by the actuation.

The state guidance device 70 is provided with an actuation control unit 70a. The actuation control unit 70a is an electronic circuit mainly composed of a microcomputer having a CPU 70b, a RAM 70c, a memory 70d, and the like. The memory 70d is a non-transitory tangible storage medium in which information can be read by the CPU 70b. The CPU 70b can execute various programs stored in the memory 70d.

The state guidance device 70 causes the CPU 70b to execute the state guidance program stored in the memory 70d. As a result, the state guidance device 70 has a determination result acquisition unit 71, an operation information acquisition unit 72, a first implementation unit 73, a second implementation unit 74, and a third implementation unit 75 as functional blocks related to driver state guidance. Etc. are constructed.

The determination result acquisition unit 71 acquires, from the output unit 38 (see FIG. 3), an inattentive determination result indicating whether the driver's inattentiveness is an inattentive look due to interest or boredom. In addition, the determination result acquisition unit 71 further acquires the arousal level determination result indicating the decrease in the arousal level of the driver who is not looking aside from the output unit 38.

Similar to the operation information acquisition unit 32 (see FIG. 3) of the state determination device 30, the operation information acquisition unit 72 acquires operation information indicating the operation state of the automatic operation function from the automatic operation ECU 20. The operation information acquisition unit 72 mainly provides the operation information of the automatic operation function to the first implementation unit 73.

Based on the inattentive determination result acquired by the determination result acquisition unit 71, the first implementation unit 73 implements the first actuation when the driver makes an inattentive look based on interest. The first actuation guides the driver's line of sight in a specific direction (eg, front). The first actuation is said to be less likely to stimulate the driver's consciousness than the second and third actuations described later. In addition, the first execution unit 73 interrupts the execution of the first actuation when the automatic driving function is the automatic driving mode in which the driver's driving operation is performed on behalf of the driver based on the operation information. As a result, while the vehicle A is traveling by the automatic driving function, the actuation is not performed even if the driver who maintains the arousal level looks aside.

The second implementation unit 74 executes the second actuation when the driver looks aside due to boredom based on the inattentive determination result acquired by the determination result acquisition unit 71. The second actuation guides the driver's line of sight in a specific direction in the same manner as the first actuation. In addition, the second activation stimulates the driver's consciousness more than the first actuation, thereby inducing a higher level of alertness.

Based on the arousal level determination result acquired by the determination result acquisition unit 71, the third implementation unit 75 implements the third actuation when the arousal level of the driver who is not looking aside drops to a distracted state. The third actuation is performed in a manner that stimulates the driver's consciousness more than the first actuation, and like the second actuation, the driver's consciousness can be guided to a state of high arousal level.

The details of the guidance determination process performed by the state guidance device 70 will be described with reference to FIGS. 1 and 5 based on FIG. The guidance determination process is started based on the fact that the shift position of the vehicle A is set to "D", as in the distraction determination process (see FIG. 4), until the shift position is changed from "D". It is started repeatedly.

In S121, a process of acquiring an inattentive determination result or an arousal degree determination result provided by the state determination device 30 is performed. Then, it is determined whether or not any of the inattentive determination result and the arousal degree determination result could be obtained. If it is determined in S121 that none of the determination results has been acquired, S121 is repeated and the acquisition of the determination result is awaited. Then, by acquiring either the inattentive determination result or the arousal level determination result, the process proceeds from S121 to S122.

When the determination result acquired from the state determination device 30 is an inattentive determination result indicating the occurrence of inattentiveness due to interest, the process proceeds from S122 to S123. In S123, the implementation of the first actuation is determined, and a series of guidance determination processes are completed. According to S123, the first actuation by the first implementation unit 73 is executed.

On the other hand, when the determination result acquired from the state determination device 30 is an inattentive determination result indicating the occurrence of inattentiveness due to boredom, the process proceeds to S125 after the determinations in S122 and S124. In S125, the implementation of the second actuation is determined, and a series of guidance determination processes is completed. According to S125, the second actuation by the second implementation unit 74 is executed.

Further, when the determination result acquired from the state determination device 30 is the alertness determination result indicating the driver's distracted state, the process proceeds to S127 after the determinations of S122, S124, and S126. In S127, the implementation of the third actuation is determined, and a series of guidance determination processes are completed. According to S127, the third actuation by the third implementation unit 75 is executed. The determination of S126 may be omitted.

Next, the details of the situations carried out by the respective implementing units 73 to 75 will be described with reference to FIGS. 7 to 10 and with reference to FIG.

In the first actuation shown in FIGS. 7 and 10, information is presented for guiding the line of sight of the driver looking aside to a specific position (for example, the front). When the first actuation is performed, the driver's consciousness is presumed to be awake. Therefore, the first actuation is carried out with the policy of not stimulating the driver's consciousness and not bothering the driver.

In the first actuation, the audio output device 85 reproduces the first induction sound. The voice output device 85 controls each speaker so that the driver recognizes a virtual sound source so that the first guidance sound can be heard from the guidance destination (for example, the front) where the driver's line of sight is to be guided. The volume of the first induction sound is made smaller than the volume of the second induction sound reproduced by the voice output device 85 in the second actuation. This first induction sound is, for example, content such as music, and is said to have a slower rhythm than the second induction sound. In addition, the pitch of the first induction sound is set lower than the pitch of the second induction sound. As described above, the content of the first induction sound is such that it is difficult to stimulate the driver.

In the first actuation, the line-of-sight guidance animation that guides the line-of-sight of the driver who looks aside is displayed on the display 80 as content. The prominence of the first display object 61 displayed on the display 80 in this line-of-sight guidance animation is based on the prominence of the second display object 62 (see FIG. 8) displayed on the display 80 in the second actuation. Is also set low. As a result, the visibility of the first display object 61 is lower than the visibility of the second display object 62.

The line-of-sight guidance animation in the first actuation is, for example, a moving image in which an arrow-shaped image is displayed on a center display 81, a room mirror monitor 83, a mirror indicator 84, or the like, and these arrow-shaped images are moved or blinked. The tip of each arrow-shaped image is directed to the guidance destination where the driver's line of sight is desired to be guided.

Information presentation by the tactile presenter 87 in the first actuation is performed, for example, by vibrating the steer vibrating portion. The vibration of the steer vibrating part in the first actuation is weaker than the vibration of the steer vibrating part in the second and third actuations described later. The seat vibrating part embedded in the driver's seat is not used in the first actuation.

In the second and third actions shown in FIGS. 8 and 9, information is presented for inducing the driver's consciousness to an awake state or shifting to a break. When the second and third actuations are performed, it is estimated that the driver's alertness is reduced. Therefore, the second and third actuations are based on the policy of actively stimulating the driver's consciousness by strongly appealing the current state to the driver using emphasized sounds, displays, vibrations, etc. Be conscious.

The volume of the second induction sound reproduced by the voice output device 85 in the second act shown in FIGS. 8 and 10 is made higher than the volume of the first induction sound. Like the first guidance sound, the second guidance sound is reproduced in the vehicle interior so that the driver can hear it from the direction in which the line of sight is guided. The second induction sound is content such as a warning sound. As the second induction sound, a sound source having a faster rhythm than the first induction sound and having a higher pitch than the first induction sound is used. As described above, the second induction sound is said to easily cause the driver to feel alert.

In the second actuation, a line-of-sight guidance animation having a content different from that of the first actuation is displayed on the display 80. The size and movement of the second display object 62 displayed in the line-of-sight guidance animation of the second actuation is set to be larger than that of the first display object 61 (see FIG. 7). The second display object 62 is, for example, a concentric elliptical image straddling the projection area 82a, the rearview mirror monitor 83, and the center display 81. The center of each ellipse is set at the guidance destination where the driver's line of sight is to be guided. In the line-of-sight guidance animation, for example, the driver's line of sight is guided to the center by continuously reducing each ellipse toward the center. As described above, in the line-of-sight guidance animation of the second actuation, by moving the second display object 62, which has higher prominence and visibility than the first display object 61, not only the driver's line-of-sight guidance but also consciousness Is also guided.

Information presentation by the tactile presenter 87 in the second actuation is performed by vibrating both the steer vibrating portion and the seat vibrating portion. The strength of the vibration of each vibrating part is set stronger than the vibration of the steer vibrating part in the first actuation. The tactile presenter 87 stimulates the driver by the vibration of each vibrating portion to improve the alertness.

In the third actuation shown in FIGS. 9 and 10, the audio output device 85 reproduces the second induction sound as in the second actuation. In addition, the tactile presenter 87 strongly vibrates the steer vibrating portion and the seat vibrating portion. On the other hand, in the third actuation, as the third display object 63, an icon requesting a driver status notification or warning, or a break is displayed on the center display 81, the projection area 82a, and the like. Like the second display object 62 (see FIG. 8), each icon of the third display object 63 is displayed in a size larger than that of the first display object 61 (see FIG. 7), and the first display object 61 It is said to be a form with higher prominence and visibility than. Each icon of the third display object 63 may be displayed on the center display 81, the projection area 82a, or the like as a part of the second display object 62 together with the line-of-sight guidance animation or between the line-of-sight guidance animations.

In the first embodiment described so far, when the distracted driver looks aside, the inattentive information and the alertness information are combined to distinguish between the inattentiveness due to interest and the inattentiveness due to boredom. .. If the determination result of such inattentiveness is output to the state guidance device 70, the state guidance device 70 can appropriately increase the arousal level of the driver by changing the mode of the actuation. Therefore, the state determination device 30 can accurately determine the state of distraction in the driver and contribute to the implementation of the actuation in an appropriate manner.

In addition, in the first embodiment, the decrease in the arousal level of the driver is determined based on the operation information of the driving operation by the driver. In this way, characteristic changes due to a decrease in alertness appear in the driving operation. Therefore, by using the history of operation information such as accelerator operation, brake operation, and steering operation, it is possible to accurately determine the decrease in the driver's arousal level from the change in the driving pattern.

Further, in the first embodiment, when the automatic driving function is activated, the use of the operation information for determining the decrease in alertness is interrupted. As described above, the state of the driver has substantially no effect on the operation information when the automatic driving function is operating. Therefore, by stopping the use of the operation information for the arousal level determination, the decrease in the arousal level of the driver can be accurately determined even during automatic driving.

Further, in the first embodiment, when the automatic driving function is activated, the criterion for determining that the driver is in the inattentive state is relaxed. As a result, the first actuation and the second actuation are less likely to occur. Based on the above, it is possible to prevent the driver from feeling annoyed by the actions frequently performed on the driver who is released from the driving operation in the vehicle A during automatic driving. In addition, when the arousal level is lowered, the actuation is surely carried out. Therefore, the effect of keeping the driver in a state where the driver can change driving is surely exhibited.

In the first embodiment, the inattentiveness determination device 31 and the inattentive information acquisition unit 35 correspond to the "inattentiveness information acquisition unit", and the arousal level determination device 33 and the arousal level information acquisition unit 36 become the "alertness information acquisition unit". Equivalent to. Further, the state determination device 30 corresponds to the "driver state determination device", the CPU 30b corresponds to the "processor", and the state guidance device 70 corresponds to the "actuation function unit".

(Second Embodiment)
The second embodiment of the present disclosure shown in FIGS. 11 and 12 is a modification of the first embodiment. The actuation device 210 of the second embodiment is an electronic control device that integrates the functions of both the state determination device 30 (see FIG. 1) and the state guidance device 70 (see FIG. 1) of the first embodiment. The actuation device 210 is provided with a processing unit 210a or the like having at least one CPU 210b, a RAM 210c, a memory 210d, or the like. The memory 210d is a non-transitory tangible storage medium in which information can be read by the CPU 210b. The CPU 210b can execute the state determination program and the state guidance program stored in the memory 210d.

According to the execution of the state determination program, the actuation device 210 includes an inattentive determination device 31, an arousal level determination device 33, a distraction determination device 34, and an operation information acquisition unit 232, which are substantially the same as those in the first embodiment. At least built. The distraction determination device 34 includes an inattentive information acquisition unit 35, an arousal level information acquisition unit 36, a determination unit 37, and an output unit 38. The operation information acquisition unit 232 corresponds to the operation information acquisition unit 32 (see FIG. 3) of the first embodiment, and outputs the operation information acquired from the automatic operation ECU 20 to the distraction determination device 34 and the state inducer 270.

Further, according to the execution of the state guidance program, at least the state inducer 270 is constructed in the actuation device 210. The state inducer 270 includes a determination result acquisition unit 71, a first implementation unit 73, a second implementation unit 74, and a third implementation unit 75, which are substantially the same as those in the first embodiment.

Even if each function of determining the driver state and guiding is realized by one actuation device 210 as in the second embodiment described so far, the same effect as that of the first embodiment can be obtained. It is possible. Therefore, the actuation device 210 can accurately determine the state of distraction in the driver and perform the actuation in an appropriate manner. In the second embodiment, the actuation device 210 corresponds to the "driver state determination device", the CPU 210b corresponds to the "processor", and the state inducer 270 corresponds to the "actuation function unit".

(Third Embodiment)
The third embodiment of the present disclosure shown in FIG. 13 is another modification of the first embodiment. The state determination device 330 according to the third embodiment is connected to the driver monitor system 311 corresponding to the driver monitor system 11 (see FIG. 1) of the first embodiment. The driver monitor system 311 is provided with an inattentive determination circuit 331 having substantially the same function as the inattentive determination device 31 (see FIG. 3) of the first embodiment. The inattentive determination circuit 331 uses the line-of-sight and face orientation information to generate inattentive information including at least a number indicating the driver's line-of-sight position and an inattentive fixing time, and provides the state determination device 330 with the inattentive information. In addition, the driver monitor system 311 generates information indicating the degree of eye opening of the driver and provides it to the state determination device 330.

The state determination device 330 constructs an operation information acquisition unit 32, an arousal level determination device 33, a distraction determination device 34, etc., which are substantially the same as those in the first embodiment, by executing the state determination program by the CPU 30b (see FIG. 1). The distraction determination device 34 includes an inattentive information acquisition unit 35, an arousal level information acquisition unit 36, a determination unit 37, and an output unit 38. On the other hand, the state determination device 330 does not have a functional block corresponding to the inattentive determination device 31 (see FIG. 3) of the first embodiment.

The arousal level determination device 33 uses the information indicating the driver's eye opening degree acquired from the inattentive determination circuit 331 together with various information acquired from the in-vehicle device group 13 and the wearable device 19 to calculate the driver's determination arousal level. The distraction determination device 34 can acquire inattentive information from the inattentive determination circuit 331 by the inattentive information acquisition unit 35. The distraction determination device 34 generates an inattentive determination result or an arousal degree determination result in the determination unit 37, and outputs the determination result from the output unit 38 to the state guidance device 70.

As in the third embodiment described so far, the generation of inattentive information does not have to be performed by the state determination device 330. If the state determination device 330 can acquire the inattentive information, the state determination device 330 can generate the inattentive determination result or the arousal degree determination result as in the first embodiment. Therefore, even the state determination device 330 of the third embodiment can accurately determine the state of distraction in the driver and contribute to the implementation of the actuation in an appropriate mode. In the third embodiment, only the inattentive information acquisition unit 35 corresponds to the "inattentive information acquisition unit", and the state determination device 330 corresponds to the "driver state determination device".

(Other embodiments)
Although a plurality of embodiments according to the present disclosure have been described above, the present disclosure is not construed as being limited to the above embodiments, and is applied to various embodiments and combinations without departing from the gist of the present disclosure. can do.

By providing the arousal level determination device, the state determination device of the above embodiment can determine the arousal level of the driver from the operation information and the like and generate the arousal level information. However, the generation of alertness information does not have to be performed by the state determination device. If the state determination device can acquire the arousal level information from another in-vehicle device or the like, the state determination device can generate an inattentive determination result or an arousal level determination result as in the above embodiment.

The state determination device of the above embodiment can cooperate with the automatic driving ECU mounted on the vehicle, and the control contents are changed such as relaxing the judgment criteria of inattentiveness during automatic driving. However, the state determination device may be configured not to cooperate with the automatic operation ECU. The state determination device is mounted on a vehicle that is not equipped with an automatic driving function, and can determine the type of inattentiveness of a driver who is in a distracted state.

In the above embodiment, a rule for dividing the front area of the driver into a plurality of areas and specifying the line-of-sight position of the driver by the number of each area is set in the inattentive determination device. As described above, the method of specifying the line-of-sight position preset in the inattentive determination device can be changed as appropriate. Further, the method of dividing the front area can be changed as appropriate. For example, the range of the windshield can be divided not only in the horizontal direction but also in the vertical direction. With such a region setting method, it may be possible to detect not only horizontal inattentiveness but also vertical inattentiveness, for example.

The functional unit that implements the driver state determination program of the present disclosure can also be provided by hardware and software different from the above-described configuration, or a combination thereof. For example, a processing circuit provided in the driver monitor system may implement a driver state determination method for determining the type of inattentiveness in a distracted state.

Further, the storage medium for storing the driver state determination program may be a semiconductor memory such as a flash memory or a storage device such as a hard disk drive. Further, the storage medium may have a configuration built in the state determination device, or may be provided in the form of a memory card or the like and may be inserted into a card slot provided in the state determination device.

Vehicle A, 210 actuation device (driver status determination device), 30,330 status determination device (driver status determination device), 30b, 210b CPU (processor), 31 inattentive judgment device (inattentive information acquisition unit), 32,232 operation Information acquisition unit, 33 alertness determination device (alertness information acquisition unit), 35 inattentive information acquisition unit, 36 alertness information acquisition unit, 37 judgment unit, 38 output unit, 70 state guidance device (actuation function unit), 270 State inducer (actuation function part)

Claims (4)

It is equipped with an automatic driving function that can support or substitute driving operations by the driver, and an actuation function unit (70,270) that can change the mode of the actuation that enhances the arousal level of the driver according to the state of the driver. A driver state determination device that determines the state of the driver in the vehicle (A).
The inattentive information acquisition unit (31, 35) for acquiring the inattentive information indicating the inattentive state of the driver, and
The arousal level information acquisition unit (33, 36) that acquires the arousal level information indicating the arousal level of the driver, and
It is a determination unit that determines a decrease in arousal level that is equal to or lower than a preset calm arousal level based on the arousal level information, and determines whether or not inattentiveness has occurred based on the inattentiveness information. when it is determined that there was determined that inattention by curiosity occurs when the objective醒度does not drop below the undisturbed wakefulness, when the sense醒度is lowered below the undisturbed wakefulness Is a determination unit (37) that determines that inattentiveness has occurred due to boredom, and
If it is determined that the driver is looking aside it is toward a judgment result indicating whether the inattention by boring or inattention by curiosity Previous Symbol Actuation function unit output, is determined not to be the more inattentive However, when the arousal level is lower than the calm arousal level, the output unit (38) that outputs the determination result indicating the decrease in the arousal level to the actuation function unit ,
The operation information acquisition unit (32,232) for acquiring the operation information indicating the operation state of the automatic operation function is provided.
The determination unit, the field automatic cruise function is activated if, while the driver relaxes the a determination criterion is inattentive state, the actuation to maintain the driver operating alternately ready Accession determining the reduction of alertness with the calm alertness order to practice the Chueshon functional unit, driver condition determining device. The driver state according to claim 1, wherein the arousal level information acquisition unit acquires the arousal level information by determining a decrease in the arousal level of the driver based on the operation information of the driving operation input to the vehicle. Judgment device. The driver state determination device according to claim 2 , wherein the arousal level information acquisition unit interrupts the use of the operation information for determining a decrease in arousal level when the automatic driving function is operating. It is equipped with an automatic driving function that can support or substitute driving operations by the driver, and an actuation function unit (70,270) that can change the mode of the actuation that enhances the arousal level of the driver according to the state of the driver. A driver status determination program that determines the status of the driver in the vehicle (A).
In the step (S101) of acquiring the inattentive information indicating the inattentive state of the driver,
In the step (S106) of acquiring the arousal level information indicating the arousal level of the driver,
It is a step of determining a decrease in arousal level that is equal to or lower than a preset calm arousal level based on the arousal level information, and determining whether or not inattentiveness has occurred based on the inattentiveness information, and further inspecting. when it is determined that, tedious if the degree of awakening is determined that inattention by curiosity occurs when not drop below the undisturbed alertness, perception醒度is lowered below the undisturbed wakefulness Steps (S107, S108) for determining that inattentiveness has occurred due to
If it is determined that the driver is looking aside it is toward a judgment result indicating whether the inattention by boring or inattention by curiosity Previous Symbol Actuation function unit output, is determined not to be the more inattentive However, when the arousal level is lower than the calm arousal level, the step (S109) of outputting the determination result indicating the decrease in the arousal level to the actuation function unit is performed.
A step of acquiring operation information indicating the operation state of the automatic operation function, and
Is run by at least one processor (30b, 210b)
When the automatic cruise function is activated, while the driver relaxes the a determination criterion is inattentive state, the actuation to maintain the driver operating alternately ready to said actuation function unit determining the reduction of alertness with the calm alertness order to practice, driver condition determination program.

Images