JP6291787B2 - Vehicle display device - Google Patents

Description

  The present invention relates to a display device for a vehicle that presents to the occupant that the vehicle is moving in a traveling direction according to the estimated occupant's intention.

  With this type of device, driving assistance that detects the intersection where the car traveling on the route turns right and left and automatically operates the blinker of the host vehicle when the blinker operation is not performed at a predetermined distance from the intersection. An apparatus is known (Patent Document 1).

JP 2011-221573 A

However, the conventional vehicle-mounted system operates the blinker uniformly regardless of the occupant's intention when the blinker is not operated at a point a predetermined distance before the intersection. there is a problem that feel uncomfortable.

The problem to be solved by the present invention is to prevent the occupant from feeling uncomfortable with the operation of the in-vehicle system.

  The present invention detects the biological information of an occupant who operates the host vehicle, and the occupant makes a decision from the result of the evaluation value of the detected biological information and the evaluation reference value that evaluates the predetermined decision-making state of the occupant. If the estimated direction of travel differs from the current direction of travel of the host vehicle when the travel timing is determined and the estimated travel direction is different from the current travel direction of the host vehicle when approaching a predetermined distance or less to the branch point detected on the route traveled by the host vehicle The above problem is solved by presenting the estimated traveling direction as an execution task at the timing determined as the decision-making timing.

According to the present invention, the control content related to the automatic driving control is presented as an execution task at the timing when the occupant's intention is determined in consideration of the occupant's intention, so the occupant feels uncomfortable with the operation of the in-vehicle system. You can avoid remembering . Thereby, the reliability of the passenger | crew with respect to the automatic driving | running control which an in-vehicle system performs can be raised.

1 is a configuration diagram of a travel control system including an automatic driving information providing apparatus according to an embodiment of the present invention. It is a figure which shows the 1st example of the task information of the automatic driving function which the automatic driving information provision apparatus of this embodiment presents. It is a figure which shows the 2nd example of the task information of the automatic driving function which the automatic driving information provision apparatus of this embodiment shows. It is a figure which shows the 3rd example of the task information of the automatic driving function which the automatic driving information provision apparatus of this embodiment shows. It is a figure which shows the 4th example of the task information of the automatic driving function which the automatic driving information provision apparatus of this embodiment shows. It is a figure which shows the 5th example of the task information of the automatic driving function which the automatic driving information provision apparatus of this embodiment shows. It is a 1st figure for demonstrating the micro saccade motion of the passenger | crew (human) eye which the biometric information detection apparatus of this embodiment detects. It is a 2nd figure for demonstrating the micro saccade motion of the passenger | crew (human) eye which the biometric information detection apparatus of this embodiment detects. It is a 1st figure for demonstrating the electrocardiogram information of the passenger | crew (human being) which the biological information detection apparatus of this embodiment detects. It is a 2nd figure for demonstrating the electrocardiogram information of the passenger | crew (human being) which the biometric information detection apparatus of this embodiment detects. It is a flowchart figure which shows the control procedure of the traveling control system which concerns on this embodiment. It is a 1st figure for demonstrating the relationship between the task information of the automatic driving function which the automatic driving information provision apparatus which concerns on this embodiment shows, and the behavior of the own vehicle. It is a 2nd figure for demonstrating the relationship between the task information of the automatic driving function which the automatic driving information provision apparatus which concerns on this embodiment shows, and the behavior of the own vehicle. It is a 3rd figure for demonstrating the relationship between the task information of the automatic driving function which the automatic driving information provision apparatus which concerns on this embodiment shows, and the behavior of the own vehicle.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the present embodiment, a case where the automatic driving information providing apparatus according to the present invention is applied to an in-vehicle travel control system 1000 will be described as an example.

  FIG. 1 is a block configuration diagram of a travel control system 1000 including an automatic driving information providing apparatus 100 according to the present embodiment. As shown in FIG. 1, a driving support system 1000 according to this embodiment includes an automatic driving control device 200 that controls automatic driving of the host vehicle, an automatic driving information providing device 100 that presents information related to automatic driving, and the operation of the vehicle. A vehicle controller 60 such as an ECU that controls the whole, a navigation device 70 that searches a route from the current position to the destination based on the map information 71, a winker device 81, a vehicle speed sensor 82, an acceleration sensor 83, etc. Including various devices 80. Each of these devices is connected by a CAN (Controller Area Network) or other in-vehicle LAN, and exchanges information with each other.

  As the automatic driving control device 200, the vehicle controller 60, the navigation device 70, and the various in-vehicle devices 80 according to the present embodiment, the automatic driving control device, the vehicle controller, the navigation device, and various in-vehicle devices known at the time of filing the present application are appropriately used. can do.

  The automatic driving control device 200 of the present embodiment is a device that automatically drives the vehicle. The route on which the vehicle travels is not particularly limited, and may be a route from the current position to the destination designated by the occupant, or may be a preset automatic driving section. In the present embodiment, the route to the destination input by the occupant via the vehicle-mounted navigation device 70 or the like is recognized, and drive control is performed to drive the vehicle on the route. In addition, the traveling of the host vehicle is controlled in consideration of surrounding obstacle information acquired from various sensors and images captured by cameras, traffic rules such as signals, and the like.

  As shown in FIG. 1, the automatic driving information providing apparatus 100 according to the present embodiment includes a control device 10, a biological information detection device 20, and a presentation device 30.

Hereinafter, each device will be described.
First, the presentation device 30 will be described. The presentation device 30 of this embodiment includes a touch panel display 31 having an information input function and a speaker 32. The presentation device 30 of the present embodiment only needs to be able to present information related to traveling, such as the traveling direction of the host vehicle, and can also use existing vehicle equipment such as lights and blinkers. Further, in this example, the display 31 that accepts touch panel type input information is adopted, but a separate input device 33 may be provided around the steering wheel. The input device 33 may be a voice input device including a microphone.

  The presentation device 30 of the present embodiment presents task information of the automatic driving function when the automatic driving function of the automatic driving control device 200 included in the host vehicle is operating. The task information in the present embodiment is information on work (control contents) being executed or about to be executed by the automatic operation function of the control device (computer) of the automatic control device 200. The task information is defined for each job (control content). Task information of this embodiment includes tasks such as start, acceleration, deceleration, and stop (control contents), right turn, left turn, lane change to the right lane, lane change to the left lane (control contents), overtaking, etc. Includes tasks.

  In the present embodiment, the presentation device 30 is located on the branch road before the branch road detected on the route traveled by the host vehicle, that is, when the distance between the current position of the host vehicle and the branch point approaches a predetermined distance or less. A task for moving the host vehicle in a traveling direction presumed to be intended by a passenger operating the host vehicle is presented.

  The traveling direction estimated to be intended by the occupant is the traveling direction of the vehicle driven by the occupant. There is no particular limitation on the method of obtaining the traveling direction estimated to be intended by the occupant. For example, the traveling direction at the current position calculated by the navigation device 70 based on the destination input by the occupant may be used. Further, based on the number of travel histories (or travel history frequencies) stored for each road (link) that the occupant has traveled in the past, the traveling direction when traveling toward a route with a higher number of travel histories (or travel history frequencies) May be.

  The presentation device 30 according to the present embodiment refers to the map information 71 of the navigation device 70, extracts a branch point that exists on the route from the current position to the estimated destination, and is within a predetermined distance from the branch point. When approaching, the task to move the vehicle in the direction of travel is presented. The presentation device 30 according to the present embodiment presents a task for moving the host vehicle in the traveling direction by indicating the traveling direction in which the host vehicle is to be moved. This is because if the direction of travel is shown to the occupant who is driving the vehicle, the occupant can understand that the vehicle will move in that direction.

  The example of the presentation aspect of "the task which moves the own vehicle to the advancing direction" in this embodiment is demonstrated based on FIG. 2A-FIG. 2E.

  As shown in FIG. 2A, the presentation device 30 of the present embodiment indicates a “task for moving the host vehicle in a predetermined traveling direction” by six arrows. An arrow with a symbol of 1 indicates a task for moving the host vehicle straight. The arrow with the symbol 2 indicates a task for changing the vehicle to the right lane. An arrow marked with 3 indicates a task for changing the vehicle to the left lane. An arrow marked with 4 indicates a task for turning the host vehicle to the right at the next intersection. An arrow with a symbol of 5 indicates a task of turning the host vehicle to the left at the next intersection. An arrow with a reference numeral 6 indicates a task for temporarily stopping the host vehicle. The presentation device 30 of the present embodiment displays so that the arrows with the symbols 1 to 6 can be identified according to the task to be executed. The display mode is not particularly limited as long as each arrow can be identified.

As illustrated in FIG. 2B, the presentation device 30 according to the present embodiment highlights and displays “Execution task to be executed immediately after”. The information of “execution task executed immediately after” is a task that is different from the task currently executed by the automatic driving control apparatus 200 and is executed immediately after the current time. That is, the “execution task” is not the control content currently executed by the automatic driving control apparatus 200 but the control content to be executed from now.
In this example, the arrow of reference numeral 2 corresponding to the execution task is displayed in a relatively emphasized manner compared to other arrows. In addition, a task to be executed next to the execution task may be displayed. When the “turn right” indicated by the arrow corresponding to the reference numeral 4 is executed after the “lane change to the right lane” indicated by the arrow corresponding to the reference numeral 2, the arrow indicated by the reference numeral 2 is emphasized as shown in FIG. 2B. While displaying, the arrow of the code | symbol 4 is displayed so that identification is possible. As shown in the figure, the display of the arrow 4 is different from the display of the arrow 2. In the present embodiment, the arrow 4 indicated as a task being executed is highlighted so as to be relatively conspicuous relative to the arrow 2 indicated as a task executed next to the execution task.

  Also, as shown in FIG. 2C, the presentation device 30 of the present embodiment has a task other than the currently executed task (reference numeral 1), the task to be executed immediately (reference numeral 2), and the task to be executed next (reference numeral 4). The arrows corresponding to the tasks (reference numerals 3, 5, 6) may be grayed out and displayed (not displayed).

  Furthermore, as shown in FIG. 2D, the presentation device 30 according to the present embodiment displays the task of turning right (reference 4) after changing the lane (reference 2) to the right lane from the straight traveling state. You can choose to go straight, rejecting the policy of automatic driving. In this case, it is possible to instruct a straight advance by touching the arrow of reference numeral 1 indicating a task going straight ahead. This instruction information is sent to the automatic operation control device 200. The automatic driving control device 200 causes the vehicle controller 60 to execute a straight traveling control in accordance with a straight traveling command intended by the occupant.

  As shown in FIG. 2E, when the task of turning right from the straight traveling state (reference numeral 4) is displayed, the occupant checks the situation of his / her surroundings by visual recognition, and the oncoming vehicle 30 Can be selected. In this case, the right turn task can be temporarily stopped by touching the arrow 6 indicating the task to be paused. This temporary stop instruction information is sent to the automatic operation control apparatus 200. The automatic driving control device 200 causes the vehicle controller 60 to execute travel control for deceleration or stop according to a temporary stop command that is intended by the occupant.

  Next, the biological information detection apparatus 20 will be described. The biological information detection apparatus 20 of the present embodiment detects biological information of an occupant who operates the host vehicle. The biological information detection apparatus 20 according to the present embodiment detects information on fixation micromotion that is one of the eye movements of an occupant (human). The biological information detection apparatus 20 according to the present embodiment detects electrocardiographic information, which is electrical information associated with the motion of the occupant (human) heart (myocardium). Specifically, the biological information detection device 20 includes an eye movement measurement device 21 that measures the eye movement of the occupant and an electrocardiographic system 22 that measures the electrocardiogram information of the occupant.

  The eye movement measuring device 21 will be described. The eye movement measuring device 21 detects fixation micromotion of the occupant's eye, particularly microsaccade movement. It is known that the human eyeball keeps moving minutely continuously when the line of sight is directed to the same place. This is called fixation micromotion. In general, fixation microtremor includes tremor movement, drift movement, microsaccade movement (also called flick movement). Of these, the micro saccade movement is known to have a close relationship with human decision-making activities.

  The eye movement measurement device 21 of the biological information detection device 20 of the present embodiment images the occupant's eye and acquires information on its movement over time. The eye movement measuring device 21 includes a camera having a CCD (Charge Coupled Device) image sensor, a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and the like. The camera images the occupant's eyeball over time. The biological information detection apparatus 20 according to the present embodiment detects biological information related to the movement of the eyeball from the captured image of the camera, and sends the detected information to the control apparatus 10 described later. The configuration of the eye movement measurement device 21 of the present embodiment is not particularly limited, and an eye movement measurement device known at the time of filing can be used.

  The electrocardiograph 22 of this embodiment will be described. The electrocardiograph 22 detects electrocardiogram information (for example, an electrocardiogram) indicating the electrical activity status of the occupant's heart. The configuration of the electrocardiograph 22 of the present embodiment is not particularly limited, and an electrocardiograph known at the time of filing can be used.

  Finally, the control device 10 of the automatic driving information providing device 100 of the present embodiment will be described. The control device 10 according to the present embodiment automatically executes a ROM (Read Only Memory) 12 in which a program for presenting task information of the automatic driving function at a predetermined timing is stored, and a program stored in the ROM 12. A CPU (Central Processing Unit) 11 as an operation circuit that functions as the driving information providing device 100 and a RAM (Random Access Memory) 13 that functions as an accessible storage device are provided.

  The control device 10 of the automatic driving information providing apparatus 100 according to the present embodiment includes a biological information acquisition function, an evaluation value derivation function, a decision making timing determination function, and a presentation control function. The control device 10 of the present embodiment is a computer that executes each function by cooperation of software for realizing the above function and the above-described hardware.

  Below, each function which the control apparatus 10 of the automatic driving information provision apparatus 100 implement | achieves is each demonstrated.

  First, the biological information acquisition function of the control device 10 will be described. The control device 10 of the present embodiment acquires, as biological information, information on the eye movements of the occupant operating the host vehicle detected by the biological information detecting device 21 of the biological information detecting device 20. The control device 10 of the present embodiment acquires the occupant (human) electrocardiographic information detected by the electrocardiograph 22 of the biological information detection device 20 as biological information.

Next, the evaluation value derivation function of the control device 10 will be described. The control device 10 calculates an evaluation value of the biological information from the acquired biological information.
In this embodiment, the control means 10 derives the evaluation value of the biological information on the movement of the eyeball related to the micro saccade from the biological information on the information on the eye movement of the occupant. Specifically, the control apparatus 10 determines the size of the microsaccade of the occupant's eyeball (the amplitude of the movement of the eyeball), the occurrence frequency of the microsaccade of the occupant's eyeball, and the microsaccade of the occupant's eyeball from the biological information related to the microsaccade motion One or more of the lengths of time for which a saccade appears is derived as an evaluation value of biological information regarding the eyeball.

First, the evaluation value of the biological information related to the movement of the eyeball will be described.
FIG. 3A shows the human eye EY. As mentioned earlier, the human eyeball is constantly moving. When the human eye looks at a single gazing point, it makes minute repetitive movements. When the human eye is observed over time, the eyeball repeatedly moves in a minute range of the movement angle θ with reference to a predetermined position such as the observation point o. FIG. 3B shows information on the movement of the reference point over time, such as the central part (black eye part) of the pupil of the eyeball. The biological information related to the eyeball shown in FIG. 3B includes eyeball movement data MS1 to MS6 when viewing the gazing point.

  As described above, fixation microtremor which is one of eye movements includes tremor motion, drift motion, and microsaccade motion. Tremor movement is high frequency (about 90Hz) and small amplitude (<1 min of arc), drift movement is low speed (~ 6 min of arc / sec) movement The microsaccade motion is a high-speed (1-10 deg / sec) jumping motion. The micro saccade movement of the eyeball, which is one of the “high-speed involuntary eye movements of a certain angle or less”, is a movement with an angle of 1 deg or less and an angular velocity of 0.1 to 10 deg / sec. The tremor motion is characterized by a smaller amplitude than the micro saccade motion, and the drift motion is characterized by a lower angular velocity (moves slowly) than the micro saccade motion. For this reason, the data MS4 to M6 whose amplitude is smaller than the predetermined threshold 1 is excluded from the biological information regarding the movement of the eyeball as shown in FIG. 3B, and the amplitude is equal to or larger than the predetermined threshold 2 (may be equal to the threshold 1). Data MS1 to MS3 can be extracted as information on the microsaccade movement of the eyeball. Similarly, the data MS4 to M6 whose angular velocity is smaller than the predetermined threshold 3 are excluded from the biological information relating to the movement of the eyeball as shown in FIG. 3B, and the angular velocity is equal to or higher than the predetermined threshold 4 (may be equal to the threshold 3). Data MS1 to MS3 can be extracted as information on the microsaccade movement of the eyeball.

  Regarding the information of the micro saccade movement extracted in this manner, the control device 10 uses the same information on the movement of the eyeball shown in FIG. 3B as the magnitude (amplitude θ) of the movement angle θ of the movement of the eyeball in FIG. The residence time T during which the line of sight stays at the gazing point, and the frequency (number of times / hour) at which microsaccades determined as “high-speed minute involuntary eye movements below a certain angle” are calculated. The magnitude of the movement angle θ of the eyeball movement, the residence time T, and the frequency of occurrence of the microsaccade are used for the calculation process of the timing of decision of the occupant. The method for deriving evaluation values of these biological information is not particularly limited, and a method known at the time of filing can be used as appropriate.

Next, an evaluation value of biological information related to heart motion will be described.
In the present embodiment, the control device 10 derives the heart rate (R) interval (RRI: RR Interval) shown in FIG. 4A as an evaluation value from the acquired electrocardiogram information. RRI is an interval between R waves (electrocardiograms of the heart when the left ventricle contracts) having the maximum amplitude in the electrocardiogram signal. The control device 10 divides the heartbeat (R) interval (RRI: R-R Interval) and / or the heartbeat interval (RRI) and the variance value (RRV) of the RRI from the fluctuation time series data (electrocardiogram) by the RRI. CVRR (the following formula: Formula 1) representing the fluctuation rate is derived as an evaluation value of biological information related to the heart. RRI and its average value are shown on the upper side of FIG. 4B. RRI is indicated by a solid line, and the average value is indicated by a broken line. CVRR (the following formula: Formula 1) representing a variation rate obtained by dividing the RRI dispersion value (RRV) by RRI and the average value thereof are shown on the upper side of FIG. 4B. CVRR is indicated by a solid line, and the average value is indicated by a broken line. The interval between heartbeats (R) (RRI: R-R Interval) and the RRI variance (RRV) are used for the calculation process of the occupant's decision making timing.

Subsequently, the decision making timing determination function of the control device 10 of the present embodiment will be described.
The control device 10 according to the present embodiment compares the derived evaluation value of the biological information with a preset evaluation reference value for evaluating the decision-making state of the occupant, and the occupant makes a decision based on the comparison result. Determine the timing of the visit.

  First, a method for determining the timing at which an occupant made a decision using biological information relating to the movement of the eyeball will be described.

  By the way, it is known that when humans make their own decisions on a selection task, a characteristic change occurs in eyeball fixation micromotion. Usually, there is no significant change in the amplitude and frequency of the microsaccade movement before and after the movement of the line of sight, but it is known that when the decision on the selection task is made, the amplitude and frequency of the microsaccade change. In addition, when making a decision for a selection task in which a human selects one of a plurality of candidates, the dwell time of the line of sight toward the candidate that he / she wants to select is not selected. It is known that it tends to be longer than the dwell time of the line of sight toward (candidate). For example, if the occupant's gaze dwell time toward one gaze point is longer than a predetermined value, it can be seen that the candidate corresponding to that gaze point is selected in the occupant's brain.

  As described above, the timing at which the amplitude and frequency of the micro saccade of the occupant's eyeball are reduced below a predetermined value, and the timing at which the occupant's gaze retention time is evaluated to be longer than the predetermined value are the timing at which the occupant made a decision. It can be judged that.

  For this reason, the control apparatus 10 of this embodiment is the timing when the occurrence frequency of the micro saccade of the occupant's eyeball changes to a value lower than the preset evaluation reference value of the occurrence frequency, and the magnitude of the movement angle of the occupant's eyeball. Is a timing at which the movement angle is changed to a value lower than a preset evaluation reference value of the moving angle, or a timing at which the dwell time at which the line of sight stays at the same gaze point is evaluated to be longer than a preset dwell time evaluation reference value Is determined to be the timing when the occupant makes a decision. The evaluation reference value of the present embodiment is set so that a change point where the occurrence frequency of the micro saccade of the occupant's eyeball is significantly reduced and a change point where the magnitude of the movement angle of the occupant's eyeball is significantly reduced can be extracted. It is preferable. Moreover, it is preferable to set the reference evaluation value of the staying time during which the line of sight stays at the gazing point as a time during which it can be determined whether or not the decision is made. Although not particularly limited, according to the experimental results of the “intention selection task” by the inventors, it is derived that the staying time of the line of sight becomes 1.2 to 1.5 seconds or more when making a decision. Although not particularly limited, the reference evaluation value of the residence time in the present embodiment is set to about 1.0 to 1.7 seconds.

  Note that the evaluation reference value is experimentally defined because it has influences such as the detection accuracy of the biological information detection device 20, the control content of the automatic driving control device 200, and the level of difficulty of an occupant's decision in driving operation. It is preferable. Specifically, the frequency of microsaccades of the occupant's eye obtained experimentally, the size of the movement angle of the occupant's eye, the average value, the peak value, the median value of the dwell time when the line of sight stays at the gazing point, etc. A representative value may be defined as a reference evaluation value.

  Based on the biological information relating to the movement of the eyeball shown in FIG. 3B, a method for determining the timing at which the occupant made a decision will be described.

  First, a determination method based on the movement angle θ of the eyeball movement will be described. Here, for convenience of explanation, it is assumed that all of the eye movement data MS1 to MS6 shown in FIG. 3B satisfy the definition conditions of the microsaccade movement. The maximum width of the movement angle θ1 of the eyeball movement shown in the eyeball movement data MS1 to MS3 is larger than the maximum width of the movement angle θ2 of the eyeball movement shown in the eyeball movement data MS4 to MS6. Assume that the evaluation reference value θTH of the moving angle is defined so as to satisfy the condition of θ2 <θTH <θ1. The movement angle θ1 of the eye movement shown in the data MS1 to MS3 is higher than the evaluation reference value θTH, but the movement angle θ2 of the eye movement shown in the data MS4 to MS6 is lower than the evaluation reference value θTH. The control device 10 allows the occupant to determine the timing Td when the movement angle of each occupant's eyeball changes from the data MS1 to MS3 higher than the evaluation reference value θTH to the data MS4 to 6 lower than the evaluation reference value θTH. It is determined that it is time to make a decision.

  Second, a determination method based on the occurrence frequency of microsaccades will be described. Here, for convenience of explanation, it is assumed that the eye movement data MS1 to MS3 shown in FIG. 3B satisfy the micro saccade definition conditions, and the eye movement data MS4 to MS6 do not satisfy the micro saccade definition conditions. To do. The time T when the eye movement data MS1 to MS3 is detected is substantially the same as the time T when the eye movement data MS4 to MS6 are detected. That is, the occurrence frequency of the micro saccade at the time T from the start timing T0 to the timing Td is 3 / hour T, whereas the occurrence frequency of the micro saccade at the time Te from the time Td until the detection of the data MS6 is completed. Is 0 / hour T. In the control device 10, the occupant made a decision at the timing Td at which the occurrence frequency of the microsaccade changed from the data MS1 to MS3 higher than the evaluation reference value KTH to the data MS4 to 6 lower than the evaluation reference value KTH. Judge that it is timing.

  Third, a method for determining the timing of decision making based on the dwell time of the line of sight will be described. The eye movement data MS1 to MS6 shown in FIG. 3B are eye movements when the line of sight is directed to different gazing points. In the determination method of the decision making timing based on the staying time of the line of sight, the data MS1 to MS6 that are the source of the determination may not be limited to the data of the micro saccade movement. When the eye gaze dwell time Ts1 in the eye movement data MS1, the gaze dwell time Ts2 in the eye movement data MS2, and the gaze dwell time Ts3 in the eye movement data MS3 are compared, the dwell time Although the time Ts1 and the residence time Ts2 are substantially equal, the residence time Ts3 is significantly longer than Ts1 and Ts2. The control device 10 measures the residence time of the data MS that is sequentially generated, and compares the length of the residence time with the evaluation reference value. When it is determined that the occupant's line-of-sight dwell time Ts3 is longer than the evaluation reference value, the control device 10 makes a decision at the timing Td when the detection of the eye movement data MS3 is completed. Judge that it is timing.

  Next, a method for determining the timing at which the occupant made a decision using biometric information related to the electrocardiogram information of the heart will be described.

  The heart rate interval (RRI), the RRI variance value (RRV), and the CVRR representing the rate of change obtained by dividing the RRI variance value (RRV) by the RRI have been described with reference to FIGS. 4A and 4B.

The balance of the sympathetic nerve and the parasympathetic nerve of the autonomic nervous system is reflected in the feature amount of the biological information related to the electrocardiographic information of the heart. It is known that the sympathetic nerve dominates when a human is inferring a choice task, that is, during the examination, and the parasympathetic nervous system dominates when making a decision based on the result of the examination . That is, with this change from sympathetic nerve dominance to parasympathetic dominance, RRI increases (heart rate decreases) and CVRR decreases. In this embodiment, the decision timing based on the activity change of the autonomic nervous system can be determined by continuously monitoring the increase in RRI or the decrease in CVRR.

  Although not particularly limited, in this embodiment, both RRI and CVRR calculate a moving average several seconds before (approximately 3 to 5 seconds, a maximum of 30 seconds) from the present, and the increase / decrease rate of the value is a threshold value (the average length). If it exceeds (set accordingly), it is evaluated that it has started to increase or decrease. In the example shown in FIG. 4B, it can be determined that the decision has been made at the timing Td when the moving average of the RRI increases. Similarly, in the example shown in FIG. 4B, it can be determined that a decision has been made at timing Td when the moving average of CVRR decreases. Note that changes in RRI and CVRR are said to occur earlier than decision making, so that the maximum and minimum appear after decision making.

  In addition, the power spectrum is obtained from the frequency analysis of the electrocardiogram, and the activity state of the autonomic nervous system can be known from the balance between the high frequency component (HF: 0.15 to 0.4 Hz) and the low frequency component (0.05 to 0.15 Hz). Are known. In the present embodiment, RRI (heart rate variability time series) data is acquired and power spectrum analysis is performed. The result is integrated in a section of 0.05 to 0.15 Hz (LF) and 0.15 to 0.40 Hz (HF), and the integrated values are set to LF and HF, respectively. The value of LF / HF is used as the evaluation value. If the value of LF / HF is larger than the evaluation reference value, it can be determined that the sympathetic nerve is dominant, and if the value of LF / HF is smaller than the evaluation reference value, it can be determined that the parasympathetic nerve is dominant. This method can also estimate the passenger's decision making timing.

  Finally, the presentation control function of the control device 10 of the automatic driving information providing device 100 of this embodiment will be described.

  The control device 10 of the present embodiment extracts a scene in which the traveling direction estimated to be intended by the occupant is different from the current traveling direction of the host vehicle, that is, the scene in which the host vehicle performs right / left turn or lane change operation control. Then, the control device 10 presents, as an execution task executed by the automatic driving function, the control contents for moving the host vehicle in the traveling direction estimated to be intended by the occupant at the timing when the occupant is determined to have made a decision. 30. The presentation method may be an output by an image or an output by sound.

  Next, a task information display method of the automatic driving information providing apparatus 100 according to the present embodiment will be specifically described with reference to FIG. FIG. 5 is a flowchart illustrating a control procedure of the travel control system 1000 according to the embodiment. This process is executed when the automatic driving function is operating.

  As shown in FIG. 8, first, in step S <b> 10, the control device 10 determines whether or not the branch point detected on the route traveled by the host vehicle has approached a predetermined distance or less. This process is executed when a branch point is approached. In step S <b> 11, the control device 10 acquires from the navigation device 70 the traveling direction estimated to be intended by the occupant operating the host vehicle at the branch point. The control device 10 acquires the biological information of the occupant who operates the host vehicle from the biological information detection device 20.

  In step S12, the control device 10 derives an evaluation value from the biological information. In step S13, the control device 10 compares the evaluation value of the biometric information with a preset evaluation reference value for evaluating the occupant's decision-making state, and the timing at which the occupant makes a decision based on the comparison result. Judging.

  In step S14, the control device 10 determines whether or not it is the execution task presentation timing. Specifically, the control device 10 determines whether or not the traveling direction estimated to be intended by the occupant is a scene different from the current traveling direction of the host vehicle. That is, when the automatic driving control apparatus 200 tries to advance the vehicle along the predicted route, it is determined whether a right / left turn or a traveling lane change is necessary at the approaching branch point. Currently, the vehicle is going straight, but if it is necessary to turn left or right or change the driving lane at the approaching branch point, the execution task will be changed from straight ahead to right or left turn, etc. Presentation is required. For this reason, the control apparatus 10 determines that it is the execution task presentation timing when the traveling direction estimated by the occupant is different from the current traveling direction of the host vehicle at the branch point.

  In step S <b> 15, the control device 10 presents the control contents for moving the host vehicle in the traveling direction estimated to be intended by the occupant at the timing determined by the decision making timing determination function as an execution task executed by the automatic driving function. Let the device 30 present it.

  In step S16, it is determined whether the execution task has been accepted by the occupant. When the execution task is not accepted, the process proceeds to step S18, and the control device 10 receives a change input or a pause input from the user for the presented execution task. As shown in FIGS. 2D and 2E, the user can input an intended traveling direction and pause when the presented execution task is not accepted.

  If the presented execution task is accepted, the process proceeds to step S17, and the automatic driving control apparatus 200 determines whether or not the environment is such that the task can actually be executed. Specifically, the automatic driving control device 200 determines an interval from the preceding vehicle, the presence / absence of an oncoming vehicle, and the like. In step S19, the automatic driving control apparatus 200 executes the accepted execution task or the execution task changed by the user.

The automatic driving information providing apparatus 100 according to the present embodiment performs the “execution task” to be performed next by the automatic driving function at a timing before the “execution task” is actually performed, and the biological information of the occupant Is displayed as an “execution task” to be performed in the near future (within several seconds) from the present time at the timing when it is determined that the occupant has made a decision. When humans act, the automatic driving information providing apparatus 100 according to the present embodiment is based on the fact that a decision on behavior is made at a timing prior to actual behavior. the operation content, presented in the timing of the decision-making of the previous timing than its operation is actually performed.

Thus, the occupant can feel the vehicle operation under the control of the automatic driving control device 200 having a determination system (control device) different from the occupant as if it were an operation based on his / her own determination. Thereby, a passenger | crew does not feel discomfort with respect to the automatic traveling control of the own vehicle. Furthermore, it is possible to make the passenger feel a sense of unity with the host vehicle having an automatic driving function.

Then, based on FIGS. 6-8, the automatic driving | operation information provision operation | movement of this embodiment is demonstrated.
FIG. 6 is a diagram showing task information displayed until the host vehicle V reaches the intersection Q, which is a branch point on the route on which the host vehicle travels, for each of the points P1 to P4 according to the order in which the host vehicle passes. It is. The direction of travel presumed to be intended by the occupant operating the vehicle at the branch point is indicated by a solid line. A route on which the host vehicle can travel, and a route other than the estimated traveling direction is indicated by a broken line. In this process, the timing for determining the decision of the occupant is only when the vehicle behavior changes from the current running state (straight-ahead state) (turning task: right turn / left turn / lane change / stop). That is, when the vehicle returns from the turning state to the straight traveling state, the execution task is presented in accordance with the timing of the vehicle behavior without making a determination of the occupant's decision making timing.

The process for estimating the timing of occupant decision-making uses a time headway (THW) corresponding to the distance between the preceding vehicle and the host vehicle calculated from the current speed, and the timing at which the vehicle behavior changes at a branch point or the like A period from 7 seconds before to 1.2 seconds before (the timing at which the vehicle-mounted system that performs automatic driving for turning right or left by automatic traveling starts turning the steering wheel). If the timing of decision making cannot be determined before 1.2 seconds, unless there is a task change instruction, the display is switched to the execution task display at that time (1.2 seconds before).

  Task information TK1 is presented before point P1 7 to 10 seconds before arrival at intersection Q. Since the task being executed is straight running and there is no need to make a right or left turn at this time, the next task is not displayed in the task information T1. The task information TK1 includes a task for changing the lane to the right lane that may be selected and a task for pause. The time to the intersection Q can be calculated from the distance between the current position and the position of the intersection Q (obtained from the map information 71) and the vehicle speed of the own vehicle (obtained from the vehicle speed sensor 82).

  The task information TK2 is presented in front of the point P2 where the right turn operation arriving at the intersection Q becomes a limit after passing the point P1. Since the task being executed is straight running and there is no need to make a right or left turn at this time, the next task is not displayed in the task information TK2. The task information TK2 includes a task for changing lanes to the right lane that may be selected, a task for turning right and left, and a task for pause.

  The task information TK3 is presented in front of the point P3 5 to 7 seconds before the left turn starting to arrive at the intersection Q after passing through the point P2. Since the task being executed is straight running and there is no need to make a right or left turn at this time, the next task is not displayed in the task information TK3. The task information TK3 includes a task for changing the lane to the right lane that may be selected, a task for turning right and left, and a task for pause.

  After passing through the point P3, task information TK4 is presented in front of the point P4 1.2 to 3 seconds before the left turn starting to arrive at the intersection Q. The task being executed is straight running, and the next task is a left turn (indicated by a hatched arrow). The task information TK4 includes a task for changing the lane to the right lane where there is a possibility of selection, and a task for pause.

  Before passing through the point P4 and before arriving at the intersection Q, task information TK5 is presented. The task being executed is a left turn, and the next task is a straight run (indicated by a hatched arrow). The task information TK5 includes a task for changing the lane to the right lane where there is a possibility of selection, and a task for pause.

  The task information TK6 is presented in front of the point R after turning the intersection Q to the left. The task being executed is straight running and there is no next task, so it is not displayed. The task information TK6 includes a suspended task that may be selected.

  FIG. 7 is a time chart showing the behavior of the vehicle, the timing of the occupant's decision making, the display timing of the next task, and the display timing of the execution task. As shown in FIG. 7, when trying to move the vehicle along the route intended by the occupant, the timing of T1 at which the occupant's intention switches from straight travel to left turn than the timing T2 at which the vehicle behavior switches from straight travel to left turn. Faster. In the present embodiment, the display is switched between the execution task and the next task at the timing of actual vehicle behavior. That is, it is presented as an execution task for executing the left turn task in accordance with the timing when the occupant's intention is determined from going straight to turning left. By the way, the left turn task is displayed as the next task before the occupant's intention is decided from straight ahead to left turn, but at the timing of decision making, the left turn task is switched as the display of the execution task to be performed immediately after . In other words, a task for causing the automatic driving control device 200 to execute a control content (for example, right / left turn) different from the current control content (for example, straight travel) is executed immediately after the occupant's intention is determined. To present to the crew.

  The contents of the execution task newly displayed after switching and the occupant's decision-making contents match at the occupant's decision-making timing. You can confirm that it is reflected. As described above, it is possible to confirm that the automatic driving control according to the user's own intention is performed at the timing of the decision making, so that the occupant's confidence in the automatic driving control device 200 can be increased.

As shown in FIG. 7, the occupant's decision making timing is estimated using the biological information, and the execution task display is switched at the timing when the decision making is determined. Thus, although the actual left tasks by the automatic operation control apparatus 200 is executed is a timing T2, the automatic operation information providing apparatus 100 of the present embodiment, prior to the this, the timing of the occupant decisions T1 The left turn task is presented to the occupant. Thus, the timing at which the occupant decides to make a left turn can be matched with the presentation timing to the effect that the left turn task is executed. It becomes possible to recognize at the timing of this, and it does not feel strange when actually executed.

  FIG. 8 is a diagram illustrating a difference between a conventional automatic driving information presentation method and the automatic driving information presentation method of the present embodiment.

  As shown in FIG. 8, in Comparative Example 1 according to the conventional technique, when the vehicle is in a straight traveling state, an arrow in a straight traveling direction indicating the operation being performed is presented as shown in B3. When the vehicle transitions to the left turn state, as shown in B2, a left arrow indicating the operation being performed is presented. Furthermore, when the vehicle changes to a straight-ahead state after turning left, an arrow in a straight-ahead direction indicating the operation being performed is presented as shown in B1.

  On the other hand, in Example 1 of the present embodiment, when the vehicle is in a straight traveling state, an arrow in a straight traveling direction indicating the operation being performed is presented as indicated by A3. The difference from the first comparative example is that in the first example, the execution task for the left turn is the execution task as shown in A2 several seconds before the vehicle actually starts turning (about 1.0 to 2.0 seconds). A left-pointing arrow indicating is shown in advance. When the vehicle transitions to a straight-ahead state after a left turn, an arrow in a straight-ahead direction indicating the operation being performed is presented as indicated by A1.

  When the automatic driving operation is executed, the occupant monitors whether or not the task that the in-vehicle automatic driving control device 200 is going to meet his / her intention. When a human executes a task based on his / her own decision, the decision is made before the start of task execution. The time from decision making to the start of task execution is generally said to be 1.2 seconds or longer. Similarly, the occupant of the vehicle on which the automatic driving control device 200 is mounted is determined to perform the task before the automatic driving control device 200 executes the task. In the present embodiment, the display of the running task is switched in accordance with the timing at which the occupant determines the intention regarding the traveling of the host vehicle. Thereby, since the timing which determines the task which a passenger | crew performs, and the display timing of an execution task correspond, the monitoring burden of a passenger | crew's task can be reduced.

  Since this invention is comprised as mentioned above and acts as mentioned above, there exist the following effects.

[1] According to the automatic driving information providing apparatus 100 of the present embodiment of the present invention, information related to automatic driving control is presented in advance in consideration of the timing of occupant decision-making, so automatic driving control is performed. It is possible to prevent the passenger from feeling uncomfortable with the operation of the vehicle at the time. Thereby, the reliability of the passenger | crew with respect to the automatic driving | running control which an in-vehicle system performs can be raised.

[2] According to the automatic driving information providing apparatus 100 of the present embodiment according to the present invention, the evaluation value of the biological information is set as the magnitude of the movement angle of the movement of the occupant's eyeball, the occurrence frequency of the micro saccade of the occupant's eyeball, The timing at which the occupant's intention is determined can be appropriately determined from these evaluation values .

  [3] According to the automatic driving information providing apparatus 100 of the present embodiment according to the present invention, the timing at which the evaluation value of the biological information is changed to a value lower than the evaluation reference value is determined as the timing for occupant decision making. Thus, the timing at which the occurrence frequency of the micro saccade of the occupant's eyeball changes to a value lower than the preset evaluation frequency evaluation value, the evaluation of the movement angle at which the magnitude of the movement angle of the occupant's eyeball is set in advance. The timing at which the occupant makes a decision can be appropriately determined from the timing at which the value changes to a value lower than the reference value.

  [4] According to the automatic driving information providing apparatus 100 of the present embodiment according to the present invention, the residence time in which the line of sight stays at the same gazing point is set, and the timing at which the occupant's intention is determined from these evaluation values is appropriately determined. it can.

  [5] According to the automatic driving information providing apparatus 100 of the present embodiment of the present invention, the timing at which the evaluation value of the biological information is changed to a value larger than the evaluation reference value is determined as the timing for occupant decision making. Accordingly, it is possible to appropriately determine the timing at which the occupant made a decision from the timing at which the staying time during which the line of sight stays at the same gazing point is longer than the preset evaluation reference value of the staying time.

  The embodiment described above is described for facilitating the understanding of the present invention, and is not described for limiting the present invention. Therefore, each element disclosed in the above embodiment is intended to include all design changes and equivalents belonging to the technical scope of the present invention.

  That is, in the present specification, the automatic driving information providing apparatus 100 will be described as an example of the automatic driving information providing apparatus according to the present invention, but the present invention is not limited to this.

  Further, in this specification, as one aspect of the automatic driving information providing apparatus according to the present invention, automatic driving information provision including a control device 10 including a CPU 11, a ROM 12, and a RAM 13, a biological information detection device 20, and a presentation device 30 is provided. The apparatus 100 will be described as an example, but the present invention is not limited to this.

  Further, in the present specification, as one aspect of the automatic driving information providing apparatus having the presenting means, the biological information acquiring means, the evaluation value deriving means, the decision making timing determining means, and the presentation control means according to the present invention, A control device 10 that executes a biometric information acquisition function, an evaluation value derivation function, a decision determination timing determination function, and a presentation control function, a presentation device 30 that includes the presentation function, and biological information that the control device 10 acquires. The automatic driving information providing apparatus 100 including the biological information detecting apparatus 20 to be detected will be described as an example, but the present invention is not limited to this.

  Moreover, in this specification, the automatic driving information provision apparatus 100 which is one aspect | mode of the automatic driving information provision apparatus which concerns on this invention, the automatic driving control apparatus 200, the vehicle controller 60, the navigation apparatus 70, various apparatuses 80, and The driving support system 1000 including the above will be described as an example, but the present invention is not limited to this.

DESCRIPTION OF SYMBOLS 1000 ... Driving control system 100 ... Automatic driving | operation information provision apparatus 10 ... Control apparatus 20 ... Biological information detection apparatus 21 ... Eye movement measuring apparatus 22 ... Electrocardiograph 30 ... Presentation apparatus 31 ... Display 32 ... Speaker 33 ... Input device 60 ... Vehicle Controller 70 ... Navigation device 71 ... Map information 80 ... Various devices

Claims (5)

When the self-driving function of the host vehicle is operating, the task of executing the current traveling direction of the host vehicle when approaching a predetermined distance or less to a branch point detected on the route on which the host vehicle travels In an automatic driving information providing apparatus provided with a presenting means for presenting to the occupant, and presenting the traveling direction of the host vehicle at the branch point to the occupant as a next task ,
Biometric information acquisition means for acquiring biometric information of an occupant operating the host vehicle;
Evaluation value deriving means for deriving an evaluation value of the biological information;
The evaluation value of the biometric information is compared with a preset evaluation standard value for evaluating the occupant's decision-making state, and the occupant selects the traveling direction as the next task from the result of the comparison Decision decision timing judging means for judging the decision timing ,
The presenting means, when the next task determined at the decision making timing is different from the execution task that is the current traveling direction of the host vehicle , the next task at the decision making timing. As an automatic driving information providing device to present to the passenger. The biological information is eye movement,
The evaluation value of the biological information is one or more of a magnitude of a movement angle of movement of the occupant's eyeball and a frequency of occurrence of micro saccades of the occupant's eyeball. Automatic driving information providing device. The decision timing judgment means is
3. The timing at which the evaluation value of the biological information changes to a value lower than the preset evaluation reference value is determined as the timing at which the occupant makes a decision. The information providing apparatus according to the description. The biological information is eye movement,
The automatic driving information providing apparatus according to claim 1, wherein the evaluation value of the biological information is a residence time in which the line of sight stays at the same point of sight. The decision timing judgment means is
The timing at which the evaluation value of the biological information is evaluated to be greater than the preset evaluation reference value is determined to be the timing at which the occupant has made a decision. The information providing apparatus according to the description.