JP6620775B2 - Display device and display method - Google Patents

Description

  The present invention relates to an apparatus and a method for displaying a countdown state until a driving mode of a vehicle traveling in an automatic driving mode is switched to a manual driving mode.

  In recent years, as a driving mode of a vehicle, in addition to a manual driving mode in which the vehicle is driven based on a driving operation of the driver, an automatic driving mode in which the vehicle is driven along a predetermined route without depending on the driving operation of the driver. Development is underway. The automatic driving mode includes, for example, navigation system information using GPS (Global Positioning System), traffic information acquired by road-to-vehicle communication, and information on a surrounding monitoring system that monitors the position and movement of surrounding people and vehicles. Originally, the vehicle can be automatically driven by controlling a power unit, a steering device, a brake, and the like (see, for example, Patent Document 1).

  Such an automatic driving mode is expected to have an effect of reducing the burden of driving operation of the driver and alleviating traffic congestion.

  For example, highway is equipped with road-to-vehicle communication, no signal, no pedestrians, lanes are surely provided on one side, and most driving operations are relatively monotonous. This is considered to be a section suitable for driving in the automatic driving mode. However, it is still difficult to drive in the automatic driving mode on a general road where road-to-vehicle communication is not established, there is only one lane on both sides, or the lane is disappearing.

  Therefore, even when driving in the automatic driving mode on the highway, when leaving the highway, it is necessary to switch from the automatic driving mode to the manual driving mode before reaching the general road.

  For example, if the vehicle is traveling on an expressway in the automatic driving mode according to the travel plan, the vehicle can always grasp the distance from the current position to the exit of the expressway and the general road. The time required to leave the highway and reach a general road can also be estimated. And, for example, when the time to reach the general road is 60 seconds, “I will enter the general road soon. After 60 seconds, switch to manual operation mode. Please let me know "is output. In addition, countdown displays such as “60 seconds remaining”, “59 seconds remaining”,... Are displayed from a screen such as a car navigation screen, a tablet screen, or a meter screen.

  On the other hand, even when driving on the highway in the automatic driving mode, it may be impossible to continue the automatic driving mode due to bad weather such as fog or heavy rain, because the external sensor of the vehicle does not operate normally It is possible. In such a case, it is determined by the vehicle that the automatic operation mode cannot be continued, and “Automatic operation mode cannot be continued due to bad weather, so switch to manual operation mode after 60 seconds. Please complete the voice announcement "is output. In this case as well, countdown displays such as “60 seconds remaining”, “59 seconds remaining”,... Are generally made from a screen such as a car navigation screen, a tablet screen, or a meter screen.

  Note that the time for preparation for switching from the automatic operation mode to the manual operation mode is appropriately set individually by an automobile manufacturer or the like, and the above-described 60 seconds is an example and is not limited. .

  Before the countdown is completed, the driver must complete a series of preparations necessary for manual driving, such as taking a driving posture, facing forward, grasping the steering wheel, and adjusting the accelerator.

JP, 2015-141053, A

  However, such voice announcements and countdown displays have the following problems.

  For example, the time of 60 seconds seems to be sufficiently long as the time for completing a series of preparations necessary for manual operation as described above. However, some drivers may not feel that 60 seconds is a sufficient length, but rather a very short length.

  On the other hand, for drivers who are comfortably immersed in the state of being released from driving tension by automatic driving, do not have the consciousness that automatic driving is about to end, and are not ready for manual driving at all, There may be a risk of a light panic due to a sudden announcement “... switch to manual operation mode after 60 seconds”.

  For such a driver, for example, the time of 60 seconds feels very short, and it does not have to be impatient. Furthermore, when the countdown display starts in such a state, the user is distracted. In particular, a screen such as a car navigation screen, a tablet screen, or a meter screen on which the countdown is generally displayed is not in front of the driver's line of sight, which may cause the driver to look aside.

  The countdown is a period during which the driver should prepare for manual driving. However, in this case, rather than preparing for manual driving in the meantime, rather than being bothered and glanced at the display itself made on the screen, when it is noticed, the countdown ends and manual driving is completed. There is also a concern that the operation mode may be switched to the manual operation mode in a state where the preparation is not ready.

  In this case, there is a problem that the safe operation to the manual operation mode cannot be performed and the operation may become unstable as soon as the manual operation mode is switched.

  The present invention has been made in view of such circumstances, and displays the countdown state until switching from the automatic operation mode to the manual operation mode so as not to hinder the driver's preparation for manual operation. It is an object of the present invention to provide a display device and a display method for enabling a driver to surely carry out preparation for manual driving while recognizing the time until switching.

  In order to achieve the above object, the present invention takes the following measures.

  That is, the first aspect of the present invention is an apparatus for displaying a countdown state until the driving mode of a vehicle traveling in the automatic driving mode is switched to the manual driving mode, and the vehicle driver's A detection unit that detects the line-of-sight direction, and a display control unit that controls the indicator on the windshield of the vehicle so that the indicator indicating the countdown state is displayed on a portion that does not block the line-of-sight direction detected by the detection unit With.

  According to a second aspect of the present invention, in the display device according to the first aspect, the display control unit is a part on the windshield of the vehicle that does not block the line-of-sight direction detected by the detection unit, and The indicator is controlled to be displayed within a predetermined distance from the intersection with the line-of-sight direction on the windshield.

  According to a third aspect of the present invention, in the display device according to the first aspect, when the detection unit detects that the line-of-sight direction has changed, the display control unit detects the position of the indicator on the windshield. Is controlled to change in accordance with the change.

  According to a fourth aspect of the present invention, there is provided a device for displaying a countdown state until the driving mode of a vehicle traveling in the automatic driving mode is switched to the manual driving mode, and the gaze direction of the driver of the vehicle In a part that does not block the line-of-sight direction detected by the detection unit on the display panel, a display panel provided between the vehicle windshield and the driver, A display control unit that controls to display an indicator indicating a countdown state.

  According to a fifth aspect of the present invention, in the display device according to the fourth aspect, the display control unit displays the indicator within a predetermined distance from an intersection with the line-of-sight direction on the display panel. Control.

  According to a sixth aspect of the present invention, in the display device according to the fourth aspect, when the detection unit detects that the line-of-sight direction has changed, the display control unit detects the position of the indicator on the display panel. Is controlled to change in accordance with the change.

  According to a seventh aspect of the present invention, in the display device according to any one of the first to sixth aspects, the detection unit is a camera that images the driver. From video information captured by the camera, The apparatus further includes a determination unit that determines the line-of-sight direction.

  According to an eighth aspect of the present invention, in the display device according to any one of the first to seventh aspects, the projector includes a projector that projects and displays the indicator, and the display control unit controls the projector.

  According to a ninth aspect of the present invention, in the display device according to any one of the first to eighth aspects, the indicator includes a numerical display indicating the countdown state.

  According to a tenth aspect of the present invention, in the display device according to any one of the first to ninth aspects, the indicator has a donut shape disposed around the line-of-sight direction with the line-of-sight direction as a center. And the display control unit displays the donut-shaped indicator at the start of the countdown and stops displaying the indicator at the end of the countdown, according to the progress of the countdown. Control so that the part is gradually hidden.

  An eleventh aspect of the present invention is a method for displaying a countdown state until a driving mode of a vehicle traveling in an automatic driving mode is switched to a manual driving mode, and the line-of-sight direction of the driver of the vehicle And an indicator indicating the countdown state is displayed on a portion of the windshield of the vehicle that does not obstruct the detected line-of-sight direction.

  A twelfth aspect of the present invention is a method for displaying a countdown state until a driving mode of a vehicle traveling in an automatic driving mode is switched to a manual driving mode, and the line-of-sight direction of the driver of the vehicle And an indicator indicating the countdown state is displayed on a portion of the display panel provided between the windshield of the vehicle and the driver that does not obstruct the detected line-of-sight direction.

  According to the first and eleventh aspects of the present invention, the driver needs to start preparation for switching from the automatic operation mode to the manual operation mode by starting display of the indicator on the windshield. It can be recognized that there is. In addition, since the indicator indicates the countdown state, the driver can grasp the time until the switching to the manual operation mode is executed. Moreover, since the indicator does not block the driver's line-of-sight direction, it does not impede forward confirmation by the driver and does not impair safety.

  According to the second and fifth aspects of the present invention, the indicator is not displayed on the intersection with the line-of-sight direction on the windshield, but is displayed at an appropriate distance away from the intersection. The driver can easily recognize the countdown state by not obstructing the direction and viewing the indicator in the distance.

  According to the third and sixth aspects of the present invention, even if the driver's line-of-sight direction changes, the position where the indicator is displayed also moves in conjunction with it, so the driver can change the line-of-sight direction. Even if it is changed, it is possible to easily recognize the countdown state.

  According to the fourth and twelfth aspects of the present invention, the driver is manually operated from the automatic operation mode by starting the display of the indicator on the display panel provided between the windshield and the driver. It can be recognized that preparation for switching to the mode needs to be started. Further, since the indicator indicates the countdown state, the driver can grasp the time until switching to the manual operation mode is executed. Moreover, since the indicator does not block the driver's line-of-sight direction, it does not impede forward confirmation by the driver and does not impair safety.

  According to the seventh aspect of the present invention, it is possible to accurately determine the driver's line-of-sight direction by imaging the driver with a camera.

  According to the eighth aspect of the present invention, the indicator can be projected from the projector.

  According to the ninth aspect of the present invention, by displaying a numerical value as an indicator, the driver can accurately grasp the time until the switching to the manual operation mode is executed.

  According to the tenth aspect of the present invention, the driver can easily detect the progress of the countdown because the driver can sense that the area of the annular portion decreases without directly looking at the indicator.

It is a figure which shows the example of whole structure of the automatic driving | operation control system provided with the driving | operation mode switching control system which implement | achieves the display apparatus which concerns on this embodiment. It is a schematic diagram which illustrates the inside of the driver's seat of the vehicle 1 in this embodiment. It is a block diagram which shows the function structural example of the operation mode switching control system in this embodiment. It is a schematic diagram which shows an example of the indicator from which a shape changes according to countdown. It is a figure which shows the example of a display of the indicator displayed as a numerical value. It is a schematic diagram which shows the example of a display of the indicator which moves according to the change of a driver | operator's gaze direction. It is a flowchart (1/2) which shows the operation example of the operation mode switching control system in this embodiment. It is a flowchart (2/2) which shows the operation example of the operation mode switching control system in this embodiment. It is a schematic diagram which shows the example of arrangement | positioning of the transparent display panel arrange | positioned between a windshield and a driver | operator. It is a block diagram which shows the function structural example of the operation mode switching control system in a modification.

  Embodiments according to the present invention will be described below with reference to the drawings.

(Constitution)
A display device to which a display method according to an embodiment of the present invention is applied will be described.

  FIG. 1 is a diagram illustrating an overall configuration example of an automatic operation control system including an operation mode switching control system 10 that realizes a display device according to the present embodiment. This automatic driving control system is mounted on a vehicle 1 such as a passenger car.

  The vehicle 1 includes, as basic equipment, a power unit 2 including a power source and a transmission, and a steering device 3 equipped with a steering wheel 4, and further includes a manual operation mode and an automatic operation mode as operation modes. . An engine and / or a motor is used as the power source.

  The manual driving mode is a mode in which the vehicle 1 is driven mainly by a manual driving operation of a driver (hereinafter also referred to as “driver”). The manual operation mode includes, for example, an operation mode for driving the vehicle based only on the driver's driving operation, and an operation mode for performing driving operation support control for supporting the driving operation of the driver while mainly driving the driver's driving operation. Is included.

  In the driving operation support control, for example, the steering torque is assisted so that the driver's steering becomes an appropriate steering amount based on the curvature of the curve when the vehicle 1 is traveling on the curve. The driving operation support control includes control for assisting the driver's accelerator operation (for example, operation of the accelerator pedal) or brake operation (for example, operation of the brake pedal), manual steering (manual operation of steering), and manual speed adjustment ( Manual operation of speed adjustment is also included. In manual steering, the vehicle 1 is steered mainly by the driver's operation of the steering wheel 4. In manual speed adjustment, the speed of the vehicle is adjusted mainly by the driver's accelerator operation or brake operation.

  Note that the driving operation support control does not include control for forcibly intervening in the driving operation of the driver and causing the vehicle 1 to automatically travel. In other words, in the manual driving mode, the driving operation of the driver is reflected in the driving of the vehicle within a preset allowable range, but forcibly intervenes in the driving of the vehicle under certain conditions (for example, deviation from the lane of the vehicle). Control to do is not included.

  On the other hand, the automatic driving mode is a mode for realizing an operating state in which the vehicle 1 is automatically driven along a road on which the vehicle 1 is traveling, for example. The automatic driving mode includes, for example, a driving state in which the vehicle 1 is automatically driven toward a preset destination without driving by the driver. In the automatic driving mode, it is not always necessary to automatically control all of the vehicle 1, and the driving state in which the driving operation of the driver is reflected in the traveling of the vehicle 1 within a preset allowable range is also included in the automatic driving mode. That is, the automatic driving mode includes a control in which the driving operation of the driver is reflected in the traveling of the vehicle 1 within a preset allowable range, but forcibly intervenes in the traveling of the vehicle under a certain condition.

  In FIG. 1, reference numeral 5 denotes an automatic operation control device for executing operation control in the automatic operation mode. The automatic driving control device 5 acquires sensing data from the steering sensor 11, the accelerator pedal sensor 12, the brake pedal sensor 13, the GPS receiver 14, the gyro sensor 15, and the vehicle speed sensor 16, respectively. These sensing data, route information generated by a navigation system (not shown), traffic information acquired by road-to-vehicle communication, information obtained by a peripheral monitoring system that monitors the position and movement of surrounding people and vehicles Based on this, the traveling of the vehicle 1 is automatically controlled.

  Automatic control includes, for example, automatic steering (automatic steering operation) and automatic speed adjustment (automatic driving of speed). Automatic steering is an operating state in which the steering device 3 is automatically controlled. Automatic steering includes LKA (Lane Keeping Assist). For example, the LKA automatically controls the steering device 3 so that the vehicle 1 does not deviate from the traveling lane even when the driver does not perform the steering operation. Even when the LKA is being executed, the driver's steering operation may be reflected in the steering of the vehicle within a range (allowable range) in which the vehicle 1 does not deviate from the traveling lane. Note that automatic steering is not limited to LKA.

  Automatic speed adjustment is an operating state in which the speed of the vehicle 1 is automatically controlled. Automatic speed adjustment includes ACC (Adaptive Cruise Control). For example, when there is no preceding vehicle ahead of the vehicle 1, ACC performs constant speed control that causes the vehicle 1 to travel at a constant speed at a preset speed, and when the preceding vehicle exists ahead of the vehicle 1. Is a follow-up control that adjusts the vehicle speed of the vehicle 1 in accordance with the inter-vehicle distance from the preceding vehicle. The automatic operation control device 5 decelerates the vehicle 1 according to the driver's brake operation (for example, operation of the brake pedal) even when ACC is being executed. In addition, the automatic operation control device 5 can perform the driver's accelerator operation (for example, accelerator) up to a preset maximum allowable speed (for example, the maximum speed legally determined on the traveling road) even during execution of ACC. The vehicle 1 can be accelerated according to the pedal operation. The automatic speed adjustment is not limited to ACC but also includes CC (Cruise Control).

  In this embodiment, the automatic operation control system includes an operation mode switching control system 10 as a device for performing switching control between the manual operation mode and the automatic operation mode. The operation mode switching control system 10 realizes the display device according to the present embodiment. The display device according to the present embodiment is a device for displaying a countdown state until the operation mode of the vehicle 1 traveling in the automatic operation mode is switched to the manual operation mode. Therefore, hereinafter, the control at the time of switching from the automatic operation mode to the manual operation mode performed in the operation mode switching control system 10 will be described, and the description of the control at the time of switching from the manual operation mode to the automatic operation mode will be omitted. .

  The operation mode switching control system 10 includes an operation mode switching control device 6, a driver camera 7, a projector 8, and a speaker 9.

  FIG. 2 is a schematic view illustrating the inside of the driver's seat of the vehicle 1 in the present embodiment.

  FIG. 3 is a block diagram showing a functional configuration example of the operation mode switching control system 10 in the same embodiment.

  The driver camera 7 is arranged at a place where the driver can be imaged, for example, on the dashboard 70, images the driver, and outputs the video signal F to the operation mode switching control device 6.

  The speaker 9 outputs audio information for notifying the start of preparation for switching from the automatic operation mode to the manual operation mode based on the message information E output from the operation mode switching control device 6. The voice information is, for example, “Switch to manual operation mode after 60 seconds. Complete preparations for manual operation by that time”. In addition, at the end of switching from the automatic operation mode to the manual operation mode, audio information such as “switching to the manual operation mode has been completed” may be output.

  The projector 8 is disposed on the dashboard 70, for example, and switches from the automatic operation mode to the manual operation mode based on the projection pattern information I output from the operation mode switching control device 6 and the driver's line-of-sight direction information H. An indicator indicating the state of the countdown made in response to the start of preparation for the projection is projected on the windshield 72. The indicator may be a numerical display corresponding to the count, or may be image information such that a part thereof is gradually hidden according to the countdown. Alternatively, for example, an hourglass display in which sand is dropped according to a countdown may be used. The driver gaze direction information H is used to determine the position on the windshield 72 where the indicator is projected.

  The operation mode switching control device 6 is a device that comprehensively controls switching of operation modes, and includes a control unit 61, an input / output interface unit 62, and a storage unit 63.

  The input / output interface unit 62 receives the manual operation switching request information A output from the automatic operation control device 5 and outputs it to the control unit 61. When the driving mode of the vehicle 1 is switched from the automatic driving mode to the manual driving mode, the automatic driving control device 5 requests the manual driving switching at the timing of starting preparation for switching (for example, 60 seconds before switching to the manual driving mode). Information A is output.

  A specific example of the timing at which the automatic driving control device 5 outputs the manual driving switching request information A will be described until the vehicle 1 traveling in the automatic driving mode in the automatic driving mode according to the preset driving plan enters the general road. In the meantime, the case of switching to the manual operation mode will be described as an example.

  The automatic operation control device 5 always knows the position of the vehicle 1 by the GPS receiver 14 and the speed of the vehicle 1 by the vehicle speed sensor 16. Therefore, it is necessary to drive in the manual operation mode by taking into account a preset travel plan, the position of the vehicle 1 grasped by the GPS receiver 14, and the speed of the vehicle 1 grasped by the vehicle speed sensor 16. The time required to reach a general road (for example, another 300 seconds) is calculated. Further, the time required for preparation for switching by subtracting the time (for example, 60 seconds) required for preparation for switching from the automatic operation mode to the manual operation mode from the required time (for example, 300-60 = 240 seconds later). Then, at this timing (for example, after 240 seconds), manual operation switching request information A is output.

  The time required for preparation for switching from the automatic operation mode to the manual operation mode (hereinafter also referred to as “preparation time”) is not limited to 60 seconds, and may be another time. In addition, for example, automatic driving control such as 60 seconds when switching to manual driving mode by exiting from a highway, 30 seconds when switching due to bad weather, and 4 seconds when switching due to an emergency. The apparatus 5 may set different preparation times according to each case.

  The manual operation switching request information A includes both reason information S indicating the reason for switching and preparation time information t.

  The reason information S is set, for example, as S1 for switching from a highway to a general road, S2 for switching due to bad weather, S3 for switching due to an emergency, and S3 for other cases. .

  The preparation time information t indicates the length of the preparation time in units of seconds (for example, 60, 30, 4, etc.).

  The input / output interface unit 62 receives the manual operation switching request information A including the reason information S and the preparation time information t as described above, and outputs it to the control unit 61.

  The input / output interface unit 62 also receives the video signal F output from the driver camera 7 and outputs it to the control unit 61 as its digital data (driver monitoring video data) G.

  The input / output interface unit 62 also outputs the projection pattern information I and the driver's line-of-sight direction information H output from the control unit 61 to the projector 8. The input / output interface unit 62 further outputs the message information E output from the control unit 61 to the speaker 9. Furthermore, the operation mode switching control signal L output from the control unit 61 is output to the automatic operation control device 5.

  The storage unit 63 uses a non-volatile memory that can be written and read as needed, such as an SSD (Solid State Drive) or HDD (Hard Disk Drive), as a storage medium, and is used to realize this embodiment. As storage areas, a message format information storage unit 631, a driver video storage unit 632, and a projection pattern storage unit 633 are provided.

  The control unit 61 has a central processing unit (CPU) and a program memory that constitute a computer, and as operation functions necessary for realizing the present embodiment, an operation mode switching signal receiving unit 610, a message information creating unit 611, A driver video acquisition unit 612, a driver gaze direction determination unit 613, a projection pattern control unit 614, and an operation mode switching signal output unit 615 are provided. Each of these control functions is realized by causing the CPU to execute a program stored in the program memory.

  When receiving the manual operation switching request information A output from the input / output interface unit 62, the operation mode switching signal receiving unit 610 generates an activation signal B for starting the operation of the message information creating unit 611. Then, the activation signal B is output to the message information creation unit 611 and the driver gaze direction determination unit 613 together with the reason information S and the preparation time information t included in the manual operation switching request information A.

  The message information creation unit 611 receives the activation signal B, reason information S, and preparation time information t output from the operation mode switching signal reception unit 610. Then, message format information C corresponding to the reason information S is acquired from the message format information storage unit 631.

  The message format information storage unit 631 stores message format information C corresponding to the reason information S in advance. For example, the message format information C1 corresponding to the reason information S1 is “I will soon enter the general road. After t seconds, switch to manual operation mode. By that time, please complete preparation for manual operation” The message format information C2 corresponding to the reason information S2 is “Automatic operation mode cannot be continued due to bad weather, so switch to manual operation mode after t seconds. Complete preparation for manual operation by that time” The message format information C3 corresponding to the reason information S3 is “switching to the manual operation mode after t seconds due to the occurrence of an emergency. Complete preparation for manual operation immediately”.

  Thereafter, the message information creation unit 611 applies the value of the preparation time information t to “t” of the acquired message format information C. Thus, for example, if the reason information S1 and the preparation time t (= 60), “I will enter the general road soon. After 60 seconds, switch to manual operation mode. By then, preparation for manual operation is completed. Create message information E. The message information creation unit 611 outputs message information E to the input / output interface unit 62.

  The input / output interface unit 62 receives the message information E output from the message information creation unit 611 and outputs it to the speaker 9. As a result, a voice message will be output from the speaker 9: “Soon you will enter the general road. After 60 seconds, switch to manual operation mode. By that time, please complete preparations for manual operation”. Become.

  Various message information E can be created by adding new message format information C to the message format information storage unit 631 or updating the existing message format information C in the message format information storage unit 631. ing.

  The driver video acquisition unit 612 takes in the driver monitoring video data G from the input / output interface unit 62 and stores the driver monitoring video data G in the driver video storage unit 632.

  The driver's line-of-sight direction determination unit 613 receives the activation signal B, reason information S, and preparation time information t output from the operation mode switching signal reception unit 610. Then, in response to the activation signal B, acquisition of the driver monitoring video data G stored in the driver video storage unit 632 is started, and the driver's line-of-sight direction is detected based on the acquired driver monitoring video data G. Then, the driver's line-of-sight direction information H, which is the detection result, is output to the projection pattern control unit 614 together with the reason information S and the preparation time information t. The processing until the driver's line-of-sight direction is determined by the driver's line-of-sight direction determination unit 613 based on the driver monitoring video data G is very short in time. Implemented and can be considered near real-time.

  Further, in the above, the driver monitoring video data G from the driver video acquisition unit 612 is temporarily stored in the driver video storage unit 632, and the driver line-of-sight direction determination unit 613 receives the driver video storage unit 632 from the driver video storage unit 632. It is assumed that the monitoring video data G is acquired. However, the driver gaze direction determination unit 613 directly acquires the driver monitoring video data G from the driver video acquisition unit 612, and detects the driver's gaze direction based on the directly acquired driver monitoring video data G. You may make it do. In this case, the driver video storage unit 632 can be omitted.

  The projection pattern control unit 614 receives the driver gaze direction information H, the reason information S, and the preparation time information t output from the driver gaze direction determination unit 613. In response to this, the countdown from the preparation time (for example, 60 seconds) defined by the preparation time information t is started. The countdown is normally performed every second, but is not limited thereto.

  The projection pattern control unit 614 further acquires the projection pattern information I from the projection pattern storage unit 633 according to the reason information S, and outputs the projection pattern information I to the input / output interface unit 62 together with the driver's line-of-sight direction information H. . The input / output interface unit 62 outputs the projection pattern information I and the driver sight line direction information H to the projector 8.

  The projection pattern information I is information defining an indicator indicating the countdown state projected from the projector 8, and the indicator may be a numerical value corresponding to the count, and a part of the indicator is gradually hidden according to the countdown. Such an image may be used.

  The projector 8 directs the projection direction in the direction specified by the driver's line-of-sight direction information H, and projects the indicator defined by the projection pattern information I.

  For example, as illustrated in FIG. 2, the indicator Y has a donut shape formed between a circle having a radius r <b> 1 centering on an intersection point X between the driver's line-of-sight direction and the windshield 72 and a circle having a radius r <b> 2. There may be. In this case, as shown in FIG. 4, the projection pattern information I defines a donut-shaped indicator Y that gradually decreases the area of the annular portion in accordance with the countdown. That is, the definition is such that the donut-shaped annular portion gradually disappears and the donut shape disappears when the countdown ends.

  The radius r1 is a short distance that does not hinder the driver's forward confirmation.

  According to such a projection pattern, the indicator Y does not obstruct the driver's line-of-sight direction, so it does not hinder forward confirmation by the driver, and the driver does not directly look at the indicator Y, so Since the decrease in area can be detected, the progress of the countdown is easily recognized.

  Note that the donut-shaped indicator Y is not limited to the one that is hidden in the clockwise direction as shown in FIG. 4 according to the countdown. For example, the donut-shaped indicator Y is hidden in the counterclockwise direction according to the countdown. Projection pattern information I can also be defined.

  The shape of the indicator Y is not limited to the donut shape, and may be, for example, a vertically or horizontally long bar shape. The projection pattern information I is defined such that the length of the bar decreases according to the countdown. You can also Or it can also be set as the hourglass display displayed so that sand may fall according to countdown, for example.

  FIG. 5 is a diagram showing a display example of the indicator Z that is displayed numerically instead of image information. Even when a numerical display is used as the indicator Z, a numerical value indicating a countdown is displayed at a distance r1 from the intersection X between the driver's line-of-sight direction and the windshield 72 so as not to inhibit the driver's forward confirmation. Shall be displayed. In this case, the projection pattern defines an indicator that is a numerical value that decreases in number according to a countdown.

  Even with such a projection pattern, the indicator Z does not obstruct the driver's line-of-sight direction, and thus does not hinder forward confirmation by the driver, and the driver reads the numerical value at a distance without looking directly at the indicator Z. Can easily recognize the progress of the countdown.

  Note that when the driver's line-of-sight direction changes, it is photographed by the driver camera 7, and as a result, the driver's line-of-sight direction H input to the projector 8 also changes, so the positions where the indicators Y and Z are displayed. Also, as shown in FIGS. 6A and 6B, it moves in conjunction therewith. Therefore, even if the driver changes the line-of-sight direction, the driver similarly recognizes the progress of the countdown.

  The projection pattern storage unit 633 stores projection pattern information I.

  Various projection patterns can be defined by adding new projection pattern information I to the projection pattern storage unit 633 or by updating the existing projection pattern information I in the projection pattern storage unit 632. Yes.

  The projection pattern control unit 614 can acquire arbitrary projection pattern information I from the projection pattern storage unit 633, but is set by default or is selected in advance by the driver. It is preferable to obtain I. This is because the driver may be confused if the projection is performed according to different projection pattern information I each time the countdown progresses every time the automatic operation mode is switched to the manual operation mode. Therefore, any projection pattern information I is set by default, and the projection pattern control unit 614 may continue to acquire the projection pattern information I set by default unless there is a change request from the driver. .

  In order to make the driver recognize the progress of the countdown more clearly, the colors of the indicators X and Y may be changed according to the countdown.

  Furthermore, the color of the indicator is not limited to the same color. For example, in order to make the driver know the progress of the countdown more clearly and specifically, the first 1/3 of the indicator that is hidden is projected in blue, and then hidden. It is also possible to project the 1/3 region to be displayed in yellow and finally project the 1/3 region that is not displayed in red.

  As described above, the operation mode switching control system 10 functions as a display device for displaying a countdown state until the operation mode of the vehicle traveling in the automatic operation mode is switched to the manual operation mode.

  When the countdown ends, the projection pattern control unit 614 outputs an operation signal K to the operation mode switching signal output unit 615.

  In response to this, the operation mode switching signal output unit 615 outputs an operation mode switching control signal L for requesting switching from the automatic operation mode to the manual operation mode of the operation mode to the input / output interface unit 62. As described above, the input / output interface unit 62 outputs the operation mode switching control signal L to the automatic operation control device 5.

  When receiving the operation mode switching control signal L, the automatic operation control device 5 switches the operation mode from the automatic operation mode to the manual operation mode.

(motion)
Next, the operation of the display device according to this embodiment will be described. As described above, the display device according to the present embodiment is realized by the operation mode switching control system 10. Therefore, the operation of the operation mode switching control system 10 will be described below. The operation description of the display device according to the present embodiment is included in the operation description of the operation mode switching control system 10.

  7 and 8 are flowcharts showing an operation example of the operation mode switching control system 10 in the present embodiment.

  In the driving mode switching control system 10 according to the present embodiment, a driver is always imaged by the driver camera 7 arranged at a place where the driver can be imaged, for example, on the dashboard 70, and the video signal F is displayed. And output to the operation mode switching control device 6. The driver monitoring video data G, which is digital data of the video signal F, is output to the driver video acquisition unit 612 by the input / output interface 62, and further from the driver video acquisition unit 612 to the driver video storage unit 632. Is output and stored in the driver video storage unit 632.

  On the other hand, when it is necessary to switch from the state in which the vehicle 1 is traveling in the automatic operation mode to the manual operation mode, the automatic operation control device 5 starts the operation mode switching control system at the timing to start preparation for switching. Manual operation switching request information A is output to 10. The manual operation switching request information A includes reason information S indicating the reason for switching and preparation time information t. The manual operation switching request information A is received by the input / output interface unit 62 and output to the operation mode switching signal receiving unit 610 (S1).

  When the operation mode switching signal receiving unit 610 receives the manual operation switching request information A output from the input / output interface unit 62, an activation signal B for starting the operation of the message information creating unit 611 is generated. The activation signal B is output from the operation mode switching signal receiving unit 610 to the message information creating unit 611 and the driver gaze direction determining unit 613 together with the reason information S and the preparation time information t included in the manual driving switching request information A. (S2).

  The message information creation unit 611 receives the activation signal B, reason information S, and preparation time information t output from the operation mode switching signal reception unit 610. The message information creation unit 611 further acquires message format information C corresponding to the reason information S from the message format information storage unit 631 (S3). For example, the message format information C1 corresponding to the reason information S1 is “I will soon enter the general road. After t seconds, switch to manual operation mode. By that time, please complete preparation for manual operation”. .

  Note that by adding new message format information C to the message format information storage unit 631 or updating the existing message format information C in the message format information storage unit 631, the message information creation unit 611 can perform various operations. Message information E can be created.

  In the message information creation unit 611, the value of the preparation time information t is applied to “t” of the message format information C. Thus, for example, if the reason information S1 and the preparation time information t (= 60), “I will enter the general road soon. After 60 seconds, switch to manual operation mode. The message information E “Please complete” is created (S4).

  Such message information E is output from the message information creation unit 611 to the input / output interface unit 62, and further output from the input / output interface unit 62 to the speaker 9. As a result, a voice message is output from the speaker 9 “Soon it will enter the general road. After 60 seconds, switch to manual operation mode. By that time, please complete preparations for manual operation” (S5). ).

  On the other hand, the driver's line-of-sight direction determination unit 613 receives the activation signal B, reason information S, and preparation time information t output from the operation mode switching signal reception unit 610. In response to the activation signal B, driver monitoring video data G is acquired from the driver video storage unit 632, and the driver's line-of-sight direction is detected based on the acquired driver monitoring video data G. Then, the driver's line-of-sight direction information H as the detection result is output to the projection pattern control unit 614 together with the reason information S and the preparation time information t (S6). The processing until the driver's line-of-sight direction is determined by the driver's line-of-sight direction determination unit 613 based on the driver monitoring video data G is very short in time. Implemented and considered near real time.

  The driver gaze direction determination unit 613 directly acquires the driver monitoring video data G from the driver video acquisition unit 612, and detects the driver's gaze direction based on the directly acquired driver monitoring video data G. You may make it do.

  The driver's line-of-sight direction information H, the reason information S, and the preparation time information t, which are detection results output from the driver's line-of-sight direction determination unit 613, are received by the projection pattern control unit 614.

  In the projection pattern control unit 614, the projection pattern information I is acquired from the projection pattern storage unit 633 according to the reason information S, and the projection pattern information I is output to the input / output interface unit 62 together with the driver gaze direction information H. . Further, the projection pattern information I and the driver sight line direction information H are output to the projector 8 by the input / output interface unit 62. In response to this, the projection direction of the projector 8 is directed to the direction specified by the driver gaze direction information H. Thereafter, projection of the indicator is started from the projector 8 according to the projection pattern information I (S7).

  The operations in steps S3 to S5 and the operations in steps S6 to S7 are performed in parallel. In step S5, a voice message is output from the speaker 9, and in step S7, projection of an indicator is started. Is almost the same time.

  Thereafter, the projection pattern control unit 614 starts counting down from the preparation time (for example, 60 seconds) defined by the preparation time information t (S8).

  For example, as illustrated in FIG. 2, the indicator Y has a donut shape formed between a circle having a radius r <b> 1 centering on an intersection point X between the driver's line-of-sight direction and the windshield 72 and a circle having a radius r <b> 2. There may be. The radius r1 is a sufficiently short distance that does not hinder the driver's forward confirmation.

  In this case, as illustrated in FIG. 4, when the countdown proceeds (S9), the annular portion of the donut-shaped indicator Y gradually disappears according to the countdown, and the countdown ends (S10: Yes). The indicator Y disappears (S11).

  According to such a projection pattern, the indicator Y does not obstruct the driver's line-of-sight direction, so it does not hinder forward confirmation by the driver, and the driver does not directly look at the indicator Y, so Since it can be sensed that the area decreases, the progress of the countdown can be easily recognized.

  In addition, as shown in FIG. 5, even when displaying an indicator Z that is a numerical value instead of image information, the forward confirmation of the driver is not hindered from the intersection point X between the driver's line-of-sight direction and the windshield 72. A numerical value indicating a countdown is displayed at a location sufficiently short distance r1, and the numerical value is also decreased according to the countdown. When the countdown is completed, “0” is displayed, and thereafter, the numerical value display disappears.

  Even with such a projection pattern, the indicator Z does not obstruct the driver's line-of-sight direction, and thus does not hinder forward confirmation by the driver, and the driver reads the numerical value at a distance without looking directly at the indicator Z. Therefore, the progress of the countdown can be easily recognized.

  When the driver's line-of-sight direction changes, it is photographed by the driver camera 7, and as a result, the driver's line-of-sight direction H input to the projector 8 also changes, but the position where the indicator Y is displayed is As shown in FIGS. 6 (a) and 6 (b), since the vehicle moves in conjunction with it, the driver can recognize the progress of the countdown in the same manner even if the line of sight is changed.

  Thus, the operation mode switching control system 10 functions as a display device for displaying the countdown state until the operation mode of the vehicle traveling in the automatic operation mode is switched to the manual operation mode, and the indicator Y is projected. Is started, the indicator Y is gradually hidden, the countdown is progressed, and the indicator Y is hidden, so that the driver knows the end of the countdown. .

  Similarly, even in the case of the indicator Z that displays a numerical value, the start of the countdown is grasped by the driver by starting the projection of the indicator Z. In addition, when the numerical value of the indicator Z changes, the progress of the countdown is grasped by the driver. Furthermore, when the numerical value of the indicator Z indicates zero and is not displayed, the end of the countdown is recognized by the driver.

  When the countdown ends in this way, the projection pattern control unit 614 outputs the operation signal K to the operation mode switching signal output unit 615. In response to this, the operation mode switching signal output unit 615 outputs an operation mode switching control signal L for requesting switching from the automatic operation mode to the manual operation mode to the input / output interface unit 62, and further inputs / outputs. The data is output from the interface unit 62 to the automatic operation control device 5. As a result, the automatic operation control device 5 switches from the automatic operation mode to the manual operation mode (S12).

(effect)
As described above, the operation mode switching control system 10 realizing the display device according to the present embodiment displays the countdown state until the operation mode of the vehicle 1 traveling in the automatic operation mode is switched to the manual operation mode. be able to.

  That is, by starting projection of the indicators Y and Z, the driver can be made aware that it is necessary to start preparation for switching from the automatic operation mode to the manual operation mode. Further, as the countdown progresses, the driver can recognize the progress of the countdown by hiding a part of the indicator Y or changing the numerical value of the indicator Z. Furthermore, by making the indicator Y non-displayed or displaying zero as the numerical value of the indicator Z and making it non-display immediately after that, the driver can be made aware of the end of the countdown.

  In addition, since the indicators Y and Z do not block the driver's line of sight, they do not hinder forward confirmation by the driver, and the driver can change the indicators Y and Z without looking directly at the indicators Y and Z. Since it can be seen from a distance, it is possible to easily recognize the progress of the countdown.

  Further, when the driver's line-of-sight direction changes, it is photographed by the driver camera 7, and as a result, the driver's line-of-sight direction H input to the projector 8 also changes, so indicators Y and Z are displayed. The position also moves in conjunction with it, as shown in FIGS. 6 (a) and 6 (b). Therefore, the driver can recognize the progress of the countdown in the same manner even if the line-of-sight direction is changed.

  From the above, the driver can complete preparations for manual operation with a margin before the countdown is completed, and can realize safe handover to manual operation.

(Modification)
In the above embodiment, the configuration in which the indicators Y and Z are projected and displayed on the windshield 72 from the projector 8 has been described. However, the present invention is not limited to such a configuration, and as shown in FIG. A transparent display panel 74 may be provided between the windshield 72 and the driver, and indicators Y and Z may be projected and displayed on the transparent display panel 74 instead of the windshield 72.

  In the above-described embodiment, the configuration in which the indicators Y and Z are projected using the projector 8 has been described. However, the present invention is not limited to such a configuration. As shown in FIG. 10, on the windshield 72 functioning as the display 80 according to the driver's line-of-sight direction information H and the projection pattern information I output from the projection pattern control unit 614 output from the input / output interface unit 62. It is also possible to realize without using the projector 8 by causing the indicators Y and Z to be displayed on the screen.

  The best mode for carrying out the present invention has been described above with reference to the accompanying drawings, but the present invention is not limited to such a configuration. Within the scope of the invented technical idea of the scope of claims, a person skilled in the art can conceive of various changes and modifications. The technical scope of the present invention is also applicable to these changes and modifications. It is understood that it belongs to.

[Appendix 1]
A device for displaying a countdown state until the driving mode of the vehicle running in the automatic driving mode is switched to the manual driving mode,
A detection unit and a control processor;
The control processor is
Let the detection unit detect the gaze direction of the driver of the vehicle,
A display device that controls so that an indicator indicating the countdown state is displayed on a portion of the windshield of the vehicle that does not block the line-of-sight direction detected by the detection unit.

[Appendix 2]
A device for displaying a countdown state until the driving mode of the vehicle running in the automatic driving mode is switched to the manual driving mode,
A detector, a display panel provided between the windshield of the vehicle and the driver, and a control processor,
The control processor is
Let the detection unit detect the gaze direction of the driver of the vehicle,
A display device that controls so that an indicator indicating the countdown state is displayed on a portion of the display panel that does not block the line-of-sight direction detected by the detection unit.

[Appendix 3]
A method for displaying a countdown state until a driving mode of a vehicle traveling in an automatic driving mode is switched to a manual driving mode,
Using the detection unit, the line-of-sight direction of the driver of the vehicle is detected,
The display method of controlling using the processor so that the indicator which shows the state of the countdown is displayed on the portion of the windshield of the vehicle which does not obstruct the detected line-of-sight direction.

[Appendix 4]
A method for displaying a countdown state until a driving mode of a vehicle traveling in an automatic driving mode is switched to a manual driving mode,
Using the detection unit, the line-of-sight direction of the driver of the vehicle is detected,
Using the processor, an indicator indicating the countdown state is displayed on a portion of the display panel provided between the windshield of the vehicle and the driver that does not obstruct the detected line-of-sight direction. To control the display method.

  DESCRIPTION OF SYMBOLS 1 ... Vehicle, 2 ... Power unit, 3 ... Steering device, 4 ... Steering wheel, 5 ... Automatic driving control device, 6 ... Driving mode switching control device, 7 ... Speaker, 8 ... Projector, 10 ... Driving mode switching control system, 11 ... steering sensor, 12 ... accelerator pedal sensor, 13 ... brake pedal sensor, 14 ... GPS receiver, 15 ... gyro sensor, 16 ... vehicle speed sensor, 61 ... control unit, 62 ... input / output interface unit, 63 ... storage unit, 70 DESCRIPTION OF SYMBOLS ... Dashboard, 72 ... Windshield, 74 ... Display panel, 80 ... Display, 100 ... Driving mode switching control system, 610 ... Driving mode switching signal receiving unit, 611 ... Message information creating unit, 612 ... Driver image acquiring unit, 613 ... Driver gaze direction determination unit, 614 ... Projection pattern control , 615 ... operation mode switching signal output unit, 631 ... message format information storage unit, 632 ... driver video storage unit, 633 ... projection pattern storage unit.

Claims (11)

A device for displaying a countdown state until the driving mode of the vehicle running in the automatic driving mode is switched to the manual driving mode,
A detection unit for detecting a gaze direction of the driver of the vehicle;
Wherein on the windshield of the vehicle, the site does not block the detected gaze direction by the detecting unit, and a display control unit the indicator indicating the status of the countdown is controlled to be displayed,
The indicator has a donut shape centered on the line-of-sight direction and arranged to surround the line-of-sight direction,
As the countdown progresses, the display control unit gradually displays the annular portion in the donut shape so that the donut-shaped indicator is displayed at the start of the countdown and the indicator is not displayed at the end of the countdown. A display device that is controlled so as to be hidden.   The display control unit is a part on the windshield of the vehicle that does not block the line-of-sight direction detected by the detection unit, and within a predetermined distance from the intersection with the line-of-sight direction on the windshield. The display device according to claim 1, wherein the indicator is controlled to be displayed.   2. The display control unit according to claim 1, wherein when the detection unit detects that the line-of-sight direction has changed, the display control unit controls the position of the indicator on the windshield to change in accordance with the change. The display device described. A device for displaying a countdown state until the driving mode of the vehicle running in the automatic driving mode is switched to the manual driving mode,
A detection unit for detecting a gaze direction of the driver of the vehicle;
A display panel provided between the vehicle windshield and the driver;
Wherein on the display panel, at a site that does not block the detected gaze direction by the detecting unit, and a display control unit the indicator indicating the status of the countdown is controlled to be displayed,
The indicator has a donut shape centered on the line-of-sight direction and arranged to surround the line-of-sight direction,
As the countdown progresses, the display control unit gradually displays the annular portion in the donut shape so that the donut-shaped indicator is displayed at the start of the countdown and the indicator is not displayed at the end of the countdown. A display device that is controlled so as to be hidden.   The display device according to claim 4, wherein the display control unit controls the indicator to be displayed within a predetermined distance from an intersection with the line-of-sight direction on the display panel.   5. The display control unit according to claim 4, wherein when the detection unit detects that the line-of-sight direction has changed, the display control unit controls the position of the indicator on the display panel to change in accordance with the change. The display device described. The detection unit is a camera that images the driver,
The display device according to claim 1, further comprising a determination unit that determines the line-of-sight direction from video information captured by the camera. A projector for projecting and displaying the indicator;
The display device according to claim 1, wherein the display control unit controls the projector.   The display device according to claim 1, wherein the indicator includes a numerical display indicating the countdown state. A method for displaying a countdown state until a driving mode of a vehicle traveling in an automatic driving mode is switched to a manual driving mode,
Detecting the gaze direction of the driver of the vehicle,
Controlling the indicator on the windshield of the vehicle that does not obstruct the detected line-of-sight direction so that an indicator indicating the countdown state is displayed .
The indicator has a donut shape centered on the line-of-sight direction and arranged to surround the line-of-sight direction,
The control is such that the donut-shaped indicator gradually appears as the countdown progresses so that the donut-shaped indicator is displayed at the start of the countdown and the indicator is not displayed at the end of the countdown. A display method including controlling to be hidden. A method for displaying a countdown state until a driving mode of a vehicle traveling in an automatic driving mode is switched to a manual driving mode,
Detecting the gaze direction of the driver of the vehicle,
And windshield of the vehicle, on the display panel provided between the driver, the portion does not block the detected gaze direction, the indicator indicating the status of the countdown is controlled to be displayed Including
The indicator has a donut shape centered on the line-of-sight direction and arranged to surround the line-of-sight direction,
The controlling is such that the donut-shaped indicator gradually appears as the countdown progresses so that the donut-shaped indicator is displayed at the start of the countdown and the indicator is not displayed at the end of the countdown. A display method including controlling to be hidden.