JP2019077340A - Head-up display device - Google Patents

Abstract

To provide a head-up display device capable of giving an occupant a sense of security that an automatic driving system is normally operated.SOLUTION: A head-up display device 4 is communicably connected to an automatic driving section performing automatic driving of a vehicle 1. The head-up display device includes: an automatic driving information acquisition section 300 which acquires automatic driving related to information on the vehicle 1; a side rear information acquisition section 400 which acquires side rear information on the vehicle 1; a display section 10 which displays a display image M; and a control section 100 which controls the display image M. When a lane change of the vehicle 1 is performed, the control section 10 generates a side rear image 120 in the display image M on the basis of the side rear information so as to give an occupant 3 a sense of security that the automatic driving section recognizes a side rear situation and an automatic driving system is normally operated.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a head-up display device.

  As a display device for a vehicle, by projecting a display image on a light transmitting member such as a windshield of a vehicle, a virtual image is made visible to a passenger sitting in a driver's seat using light of the display image reflected by the windshield There is a so-called head up display (HUD) device. For example, the image display system for vehicles disclosed by patent document 1 produced | generated the traveling action which the own vehicle in automatic driving | running | working is trying to take next as a display image, and projected it on the windshield.

Japanese Patent Publication No. 2016-182891

  However, in the configuration described in Patent Document 1 above, since the passenger is not notified of the side rear situation, it is not possible to know whether the automatic driving system recognizes the side rear situation, giving the passenger a sense of anxiety or discomfort. There was a point.

  It is an object of the present invention to provide a head-up display device that gives the passenger a sense of security that the automatic driving system is operating normally by displaying the situation behind the side in lane change during automatic driving. Do.

The head-up display device of the present invention is
A head-up display device communicably connected to an automatic driving unit for automatically driving a vehicle, comprising:
An automatic driving information acquisition unit that acquires automatic driving related information including lane change of the vehicle;
A side rear information acquisition unit that acquires side rear information of the vehicle;
A display unit for displaying a display image;
A control unit that controls the display image;
Equipped with
The control unit generates a side rear image on the display image based on the side rear information when the lane change of the vehicle is performed.
It is characterized by

  According to the present invention, it is possible to give a passenger a sense of security that the automatic driving system is operating normally.

It is a figure explaining the structure of the HUD apparatus of the 1st thru | or 6th embodiment of this invention, and the virtual image which this HUD apparatus displays. It is a figure which shows the structure of the HUD apparatus of the 1st thru | or 6th embodiment of this invention. It is a figure which shows the example of a display of the virtual image which the HUD apparatus which concerns on said 1st embodiment displays. It is a figure which shows the example of a display of the virtual image which the HUD apparatus which concerns on the said 2nd embodiment displays. It is a figure which shows the example of a display of the virtual image which the HUD apparatus which concerns on the said 3rd embodiment displays. It is a figure which shows the example of a display of the virtual image which the HUD apparatus which concerns on the said 4th embodiment displays. It is a figure which shows the example of a display of the virtual image which the HUD apparatus which concerns on the said 5th embodiment displays. It is a flowchart which shows the main operation | movement procedure of the HUD apparatus of the 1st thru | or 6th embodiment of this invention.

  The preferred embodiments described below are used to facilitate understanding of the present invention. Accordingly, one of ordinary skill in the art should note that the present invention is not unduly limited by the embodiments described below.

  FIG. 1 shows a schematic view of the configuration of a head-up display device (hereinafter also referred to as a HUD device) 4 according to an embodiment of the present invention. The HUD device 4 includes a display unit 10, a flat mirror 21, a curved mirror 22, and a housing 30, and the housing 30 includes a case 31 and a transparent cover 32. The HUD device 4 reflects the display light L representing the display image M displayed by the display unit 10 with the flat mirror 21 and the curved surface mirror 22 and is a windshield made of a light transmitting member of the vehicle 1 on which the HUD device 4 is mounted. The virtual image V is emitted to 2 and the passenger 3 (usually, the driver of the vehicle 1) visually recognizes the virtual image V so as to be superimposed on the real scenery. Further, in the coordinate axes shown in FIG. 1, the z-axis positive direction represents the front direction of the vehicle 1, the y-axis positive direction represents the upper side in the vertical direction, and the x-axis positive direction (upper vertical direction with respect to the drawing) The left direction of the vehicle 1 is shown.

  FIG. 2 shows a block diagram of the HUD device 4. The HUD device 4 includes a display unit 10, a control unit 100, a vehicle information acquisition unit 200, an automatic driving information acquisition unit 300, a side rear information acquisition unit 400, and a front information acquisition unit 500.

  The display unit 10 includes a liquid crystal panel module, and displays the display image M generated by the control unit 100. In addition, as another embodiment, a self light emitting display panel module such as an organic EL (Electro Luminescence) element may be used, and a reflection type such as DMD (Digital Micromirror Device) or LCoS (Liquid Crystal on Silicon) (registered trademark) may be used. It may be a display panel module, or may be a scanning display device or the like that scans laser light.

  The control unit 100 includes a circuit, and the circuit includes at least one processor (e.g., a central processing unit (CPU), at least one application specific integrated circuit (ASIC), and the like). And / or at least one semiconductor integrated circuit such as at least one field-programmable gate array (FPGA). At least one processor may perform all or part of the functions of HUD device 4 shown in FIG. 2 by reading one or more instructions from at least one computer readable tangible storage medium. The recording medium includes any type of magnetic medium such as hard disk, any type of optical medium such as CD and DVD, any type of semiconductor memory such as volatile memory, and non-volatile memory. Volatile memory includes DRAM and SRAM, and non-volatile memory includes ROM and NVRAM. A semiconductor memory is also a semiconductor circuit that is part of a circuit with at least one processor. The ASIC is an integrated circuit customized to perform all or part of the functions of the HUD device 4 shown in FIG. 2, and the FPGA is all or part of the functions of the HUD device 4 shown in FIG. An integrated circuit designed to perform The control unit 100 performs image processing based on information from the vehicle information acquisition unit 200, the automatic driving information acquisition unit 300, the side rear information acquisition unit 400, and the forward information acquisition unit 500, and generates and controls the display image M. .

  Vehicle information acquisition unit 200 transmits vehicle information of vehicle 1 (for example, speed, blinking of direction indicator, remaining amount of traveling energy, traveling distance) via a communication interface (not shown) such as CAN (Control Area Network) or in-vehicle Ethernet. It is acquired and output to the control unit 100. In addition, the vehicle information acquisition unit 200 may output the vehicle information of the vehicle 1 according to an automatic driving unit (not shown).

  The automatic driving information acquisition unit 300 acquires automatic driving related information including at least a lane change of the vehicle 1 from an automatic driving unit (not shown) via a communication interface (not shown) such as CAN or in-vehicle Ethernet, and outputs it to the control unit 100. .

  The automatic driving system of the vehicle is based on the technical level in the Society of Automotive Engineers (SAE) J3016 and the public-private ITS concept and road map 2017 (Advanced Information Communication Network Society Promotion Strategy Headquarters, Public-Private Data Utilization Promotion Strategy Conference). Are defined in six levels from level 0 to level 5. In the above-mentioned roadmap, the SAE Level 3 or higher automatic driving system is called "Advanced Automatic Driving System", and the SAE Level 4 or 5 Automatic Driving System is called "Fully Automatic Driving System". The “automatic operation” in the present specification includes the above-mentioned level 1 to level 5 automatic operation systems.

  Side rear information acquisition unit 400 is configured of a stereo camera or a single-eye camera, captures the side rear of vehicle 1, analyzes the captured image data, acquires the side rear information of vehicle 1, and outputs it to control unit 100 . The side rear information includes an actual landscape behind the side of the vehicle 1, an object (usually another vehicle), object information on the object (for example, an image of the object, a position of the object, a velocity of the object, And the like) and the like. In addition, as another embodiment, the control unit 100 acquires side rear information of the vehicle 1 from remote sensing such as LiDAR (Light Detection and Ranging), inter-vehicle communication such as V2X (Vehicle to X), and road-vehicle communication. May be output. Further, the side rear information acquisition unit 400 may output the side rear information of the vehicle 1 according to an automatic driving unit (not shown).

  The forward information acquisition unit 500 is composed of a stereo camera and a monocular camera, captures an image of the front of the vehicle 1, analyzes the captured image data, and outputs the forward information of the vehicle 1 to the control unit 100. Note that as another example, forward information of the vehicle 1 is detected from remote sensing such as LiDAR (Light Detection and Ranging), inter-vehicle communication such as V2X (Vehicle to X), or road-vehicle communication. It may be output. Further, the forward information acquisition unit 500 may output the forward information of the vehicle 1 in accordance with an automatic driving unit (not shown).

  Next, display and operation of the HUD device 4 will be described using FIGS. 3 to 8.

  FIG. 3 shows a first embodiment of the HUD device 4. The control unit 100 determines whether the vehicle 1 changes lanes from the automatic driving related information of the automatic driving information acquisition unit 300. When the vehicle 1 changes lanes, the control unit 100 generates the side rear image 120 based on the side rear information of the side rear information acquisition unit 400. The control unit 100 generates the display image M such that the side rear image 120 is disposed in the display area 110.

  FIG. 4 shows a second embodiment of the HUD device 4. The control unit 100 determines whether the vehicle 1 changes lanes from the automatic driving related information of the automatic driving information acquisition unit 300. When the vehicle 1 changes lanes, the control unit 100 generates the side rear image 120 based on the side rear information of the side rear information acquisition unit 400. The control unit 100 determines whether there is an object that affects the lane change of the vehicle 1 on the side rear based on the side rear information. For example, if there is an object in the side rear information and the position of the object is close to the vehicle 1 or if the speed of the object is faster than the vehicle 1, it is assumed that the object affecting the lane change of the vehicle 1 exists. judge. When the object 122 affecting the lane change of the vehicle 1 is present, the alert image 130 for the object 122 is generated in the side rear image 120. The alerting image 130 is positioned, for example, to overlap the object 122. The control unit 100 generates the display image M such that the side rear image 120 is disposed in the display area 110.

  The alerting image 130 is an image showing the passenger 3 alerting the object 122, and the expression may be an appropriate one selected from signs or the like, or may be characters or images. Good.

  FIG. 5 shows a third embodiment of the HUD device 4. The control unit 100 determines whether the vehicle 1 changes lanes from the automatic driving related information of the automatic driving information acquisition unit 300. When the vehicle 1 changes lanes, the control unit 100 generates the side rear image 120 based on the side rear information of the side rear information acquisition unit 400. The control unit 100 determines whether there is an object that affects the lane change of the vehicle 1 on the side rear based on the side rear information. For example, if there is an object in the side rear information and the position of the object is close to the vehicle 1 or if the speed of the object is faster than the vehicle 1, it is assumed that the object affecting the lane change of the vehicle 1 exists. judge. When the target object 122 affecting the lane change of the vehicle 1 is present, a predicted position image 140 indicating the predicted position of the vehicle 1 after the lane change is generated in the side rear image 120. The predicted position image 140 is located, for example, at a relative position (rear of the object 122) of the side rear image 120 with the object 122. The control unit 100 generates the display image M such that the side rear image 120 is disposed in the display area 110.

  The predicted position image 140 is an image showing the predicted position of the vehicle 1 after the lane change to the passenger 3, and the expression thereof may be represented by a figure or may be based on the image of the vehicle 1 .

  FIG. 6 shows a fourth embodiment of the HUD device 4. The control unit 100 determines whether the vehicle 1 changes lanes from the automatic driving related information of the automatic driving information acquisition unit 300. When the vehicle 1 changes lanes, the control unit 100 generates the side rear image 120 based on the side rear information of the side rear information acquisition unit 400. The control unit 100 determines whether there is an object that affects the lane change of the vehicle 1 on the side rear based on the side rear information. For example, when there is no object in the side rear information, or when the position of the object is far from the vehicle 1 or when the speed of the object is slower than the vehicle 1, the object that affects the lane change of the vehicle 1 is present It decides that it does not do. If there is no object that affects the lane change of the vehicle 1, a direction indication image 150 indicating the direction of the lane change is generated in the side rear image 120. The control unit 100 may control to blink the direction indication image 150 in conjunction with the blinking of the direction indicator acquired from the vehicle information acquisition unit 200. The control unit 100 generates the display image M such that the side rear image 120 is disposed in the display area 110.

  The direction indication image 150 is an image that shows the passenger 3 the direction of the lane change of the vehicle 1, and the expression may be represented by a graphic or may be based on the indicator of the vehicle 1.

  FIG. 7 shows a fifth embodiment of the HUD device 4. The control unit 100 determines whether the vehicle 1 changes lanes from the automatic driving related information of the automatic driving information acquisition unit 300. When the vehicle 1 changes lanes, the control unit 100 generates the side rear image 120 based on the side rear information of the side rear information acquisition unit 400. The side rear image 120 includes the boundary 121 on the outer periphery of the side rear image 120. The control unit 100 determines whether there is an object that affects the lane change of the vehicle 1 on the side rear based on the side rear information. For example, when there is no object in the side rear information, or when the position of the object is far from the vehicle 1 or when the speed of the object is slower than the vehicle 1, the object that affects the lane change of the vehicle 1 is present It decides that it does not do. When there is no target object that affects the lane change of the vehicle 1, control is performed to display the boundary 121 in a blinking manner. The control unit 100 may control to blink the boundary 121 in conjunction with the blinking of the direction indicator acquired from the vehicle information acquisition unit 200. The control unit 100 generates the display image M such that the side rear image 120 is disposed in the display area 110.

  In the sixth embodiment of the HUD device 4, the control unit 100 acquires, from the automatic driving related information of the automatic driving information acquisition unit 300, information indicating that the vehicle 1 has started lane change and has been completed. The control unit 100 may further acquire information indicating that it is in the middle of a lane change, the time required for the lane change, and the remaining time until the completion of the lane change from the automatic driving related information. The control unit 100 controls to stop the generation of the side rear image 120 between the start of the lane change of the vehicle 1 and the completion of the lane change.

  FIG. 8 is a flow chart showing main operation procedures in the first to sixth embodiments of the HUD device 4.

  In step S11, the control unit 100 acquires, from the automatic driving information acquisition unit 300, automatic driving related information including lane change of the vehicle 1.

  In step S12, the control unit 100 determines whether the vehicle 1 changes lanes based on the automatic driving related information. In step S12, if the vehicle 1 does not change lanes (No), the process ends. If the vehicle 1 changes lanes (Yes) in step S12, the process proceeds to step S13. Do.

  In step S13, the control unit 100 acquires side rear information from the side rear information acquisition unit 400.

  In step S14, the control unit 100 generates a side rear image 120 based on the side rear information.

  In step S <b> 15, the control unit 100 determines whether or not there is an object affecting the lane change of the vehicle 1 behind the side of the vehicle 1 based on the side rear information. "If there is an object that affects the lane change of the vehicle 1 (Yes)", the process proceeds to step S21, and if "an object that affects the lane change of the vehicle 1 does not exist (No)", the process proceeds to step S31. Migrate to

  In step S <b> 21, the control unit 100 generates a warning image 130 as a side rear image 120 based on the side rear information.

  In step S <b> 22, the control unit 100 generates a predicted position image 140 as a side rear image 120 based on the side rear information.

  In step S23, the control unit 100 generates the display image M such that the side rear image 120 is disposed in the display area 110.

  In step S24, the control unit 100 outputs the display image M to the display unit 10. The display unit 10 displays the display image M. After step S24, the process proceeds to step S13.

  In step S31, the control unit 100 generates the direction indication image 150 as the side rear image 120, and blinks the direction indication image 150 in conjunction with the blinking of the direction indicator acquired from the vehicle information acquisition unit 200. Control.

  In step S32, the control unit 100 performs control to blink the boundary 121 of the side rear image 120 in conjunction with the blinking of the direction indicator acquired from the vehicle information acquisition unit 200.

  In step S33, the control unit 100 generates a display image M such that the side rear image 120 is disposed in the display area 110.

  In step S34, the control unit 100 outputs the display image M to the display unit 10. The display unit 10 displays the display image M.

  In step S35, the control unit 100 determines whether to cancel the generation of the side rear image 120 from the automatic driving related information. In the case of not canceling generation of the side rear image 120 (in the case of No), the process proceeds to step S13, and in the case of stopping generation of the side rear image 120 in the case of step S35 (in the case of Yes) Transition.

  In step S36, the control unit 100 cancels the generation of the side rear image 120.

  The present invention described above has the following effects.

  In the first embodiment, when the automatic driving unit of the vehicle 1 changes lanes, the side rear image 120 is displayed on the virtual image V of the HUD device 4. By visually recognizing the side rear image 120, the passenger 3 can obtain a sense of security that the automatic driving system is operating normally.

  In the second embodiment, when there is an object affecting the lane change behind the vehicle 1, the alert image 130 is displayed in the side rear image 120. By visually recognizing the side rear image 120 and the alerting image 130, the passenger 3 obtains a sense of security that the automatic driving system is operating normally and the target object affecting the lane change is recognized. be able to.

  In the third embodiment, a predicted position image 140 indicating the position of the vehicle 1 after the lane change is displayed in the side rear image 120. By visually recognizing the side rear image 120 and the predicted position image 140, the passenger 3 obtains a sense of security that the automatic driving system is operating normally and that the future position of the vehicle can be known. it can.

  In the fourth embodiment, the direction indication image 150 is displayed in the side rear image 120 and blinks in conjunction with the direction indicator. The passenger 3 can obtain a sense of security that the automatic driving system is operating normally by visually recognizing the side rear image 120 and the direction indication image 150, and can receive it as a natural information presentation. .

  In the fifth embodiment, the boundary 121 of the side rear image 120 blinks in conjunction with the direction indicator. The passenger 3 can obtain a sense of security that the automatic driving system is operating normally by visually recognizing the side rear image 120 and the boundary 121, and can receive it as a natural information presentation.

  In the sixth embodiment, the control unit 100 stops the generation of the side rear image 120 between the start and the completion of the lane change of the vehicle 1. The passenger 3 can obtain a sense of security that the automatic driving system is operating normally by the disappearance of the side rear image 120 along with the lane change of the vehicle 1, and the line-of-sight return to the front of the vehicle 1 Can be made faster.

  As a modification of the first embodiment, the control unit 100 may generate the side rear image 120 in the direction in which the vehicle 1 changes lanes. For example, when the vehicle 1 changes lanes in the x-axis positive direction (left direction), the side rear image 120 is generated on the left side of the center of the display area 110 and the vehicle 1 changes lanes in the x-axis negative direction (right direction) In this case, the side rear image 120 is generated to the right of the center of the display area 110.

  Note that, as a modification of the third embodiment, the control unit 100 may generate the predicted position image 140 even when there is no target object affecting the lane change at the rear side of the vehicle 1.

  Note that, as a modification of the sixth embodiment, the control unit 100 appropriately adjusts the timing of stopping the generation of the side rear image 120 from the start of the lane change of the vehicle 1 to the completion of the lane change. Good.

  As a modification of the above-mentioned embodiment, after Step S24 and in the case where the judgment in Step S35 is "No" in the flow chart of FIG. 8, the process may proceed to Step S11.

  In the above embodiment, the control unit 100 controls the vehicle information of the vehicle information acquisition unit 200, the automatic driving related information of the automatic driving information acquisition unit 300, the side rear information of the side rear information acquisition unit 400, and the front information acquisition unit 500. It is also possible to generate another information image (for example, the speed of the vehicle 1 and navigation to the destination) from the front information of and to generate the display image M in combination with the side rear image 120.

  In addition, although the windshield type HUD apparatus was used as a structure of the said embodiment, it is applicable also to a combiner type HUD apparatus.

  The present invention is not limited by the above embodiments and the drawings. Changes (including deletion of components) can be made as appropriate without changing the scope of the present invention.

  The head-up display device of the present invention is suitable for vehicles, particularly four-wheeled vehicles.

1 ... vehicle 2 ... windshield 3 ... passenger 4 ... head up display device (HUD device)
DESCRIPTION OF SYMBOLS 10 ... Display part 21 ... Plane mirror 22 ... Curved surface mirror 30 ... Housing | casing 100 ... Control part 110 ... Display area 120 ... Side back image 121 ... Boundary 122 ・. .. Object (other vehicles)
130 ... alerting image 140 ... predicted position image 150 ... direction indication image 200 ... vehicle information acquisition unit 300 ... automatic operation unit 400 ... backward information acquisition unit 500 ... forward information Acquisition unit M: Display image L: Display light V: Virtual image

Claims (6)

A head-up display device communicably connected to an automatic driving unit for automatically driving a vehicle, comprising:
An automatic driving information acquisition unit that acquires automatic driving related information including lane change of the vehicle;
A side rear information acquisition unit that acquires side rear information of the vehicle;
A display unit for displaying a display image;
A control unit that controls the display image;
Equipped with
The control unit generates a side rear image on the display image based on the side rear information when the lane change of the vehicle is performed.
Head-up display device characterized in that. The control unit
It is determined from the side rear information whether there is an object that affects the lane change of the vehicle, and if it exists, a warning image is generated on the side rear image,
The head-up display device according to claim 1, characterized in that: The control unit
Generating a predicted position image of the vehicle after the lane change in the side rear image;
The head-up display device according to any one of claims 1 to 2, characterized in that: The control unit
Generating a direction indication image that blinks in conjunction with the direction indicator of the vehicle on the side rear image;
The head-up display device according to any one of claims 1 to 3, characterized in that: The control unit
The boundary of the side rear image is controlled to blink in conjunction with the turn indicator of the vehicle,
The head-up display device according to any one of claims 1 to 4, characterized in that: The control unit
Between the start and the completion of the lane change of the vehicle, stopping the generation of the side rear image;
The head-up display device according to any one of claims 1 to 5, characterized in that:

Images