JP2019207632A - Display device - Google Patents

Abstract

To provide a display device capable of satisfactorily informing information assumed to have a high priority when informing a plurality of information about the front of a vehicle.SOLUTION: A display device displays a virtual image V that overlaps a front landscape of a vehicle to a driver. The display device can display a first image C1 corresponding to a first object M1, and can display a second image C2 corresponding to a second object M2. The display device can display the first image C1 in an AR mode and a standing mode that is visually recognized by standing on the driver side with respect to the AR mode, and can display the second image C2 in a standing mode and an AR mode that is visually recognized by tilting toward a front road R with respect to the standing mode. When both the first object M1 and the second object M2 are specified, the display device fades out while changing the first image C1 displayed in the AR mode to the standing mode, and the display device fades in the second image C2 while changing the AR mode from the standing mode.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a display device.

  As a conventional display device, Patent Document 1 discloses a display device that displays an image that is visually recognized by a viewer (mainly a driver of a vehicle) as a virtual image that overlaps the front landscape of the vehicle. The display device disclosed in Patent Literature 1 controls the relative height or the like of a notification image that notifies the presence of a specific target in accordance with the approach of the specific target in front of the vehicle.

JP, 2015-221633, A

  For example, when there are a plurality of objects having different distances from the vehicle in front of the vehicle, the technique disclosed in Patent Document 1 causes the notification image to be deformed in various modes according to the distance to each object. However, there is room for improvement in notifying information that is assumed to have a higher priority for the viewer among the information of each object in front of the vehicle.

  The present invention has been made in view of the above circumstances, and provides a display device capable of satisfactorily informing information assumed to have high priority when informing a plurality of information ahead of a vehicle. Objective.

In order to achieve the above object, a display device according to the present invention includes:
A display device that displays a superimposed image that is visually recognized by a viewer as a virtual image that overlaps a front landscape of a vehicle,
Object information acquisition means for acquiring object information indicating at least the position of the object, which is information relating to an object existing around the vehicle;
Based on the object information, the position of the first object and the position of the second object closer to the vehicle than the first object are acquired, display control of the superimposed image is performed, and display of the superimposed image is performed. Display control means capable of displaying a first image at a position corresponding to the first object and displaying a second image at a position corresponding to the second object in the region;
The display control means includes
The first image can be displayed in a first special mode visually recognized along the road ahead of the vehicle, and a first predetermined mode that stands up to the viewer side from the first special mode and is visually recognized.
The second image can be displayed in a second predetermined mode and a second special mode that is visually recognized by tilting toward the front road side than the second predetermined mode,
When both the first object and the second object exist, the first image displayed in the first special mode is faded out while changing from the first special mode to the first predetermined mode. Then, an image transition process for fading in the second image while changing from the second special mode to the second predetermined mode is executed.

  ADVANTAGE OF THE INVENTION According to this invention, when alerting | reporting several information ahead of a vehicle, the information assumed that a priority is high can be alert | reported favorably.

It is a figure for demonstrating the mounting aspect to the vehicle of the head-up display (HUD) apparatus which concerns on one Embodiment of this invention. It is a block diagram of the display system for vehicles. It is a flowchart which shows an example of a display control process. (A)-(c) is a figure which shows the example of a transition of the 1st image displayed on the position corresponding to a front object, and the 2nd image displayed on the position corresponding to an interruption object. (A) And (b) is a timing chart for demonstrating the transition of a 1st image and a 2nd image.

  A display device according to an embodiment of the present invention will be described with reference to the drawings.

  The display device according to the present embodiment is a HUD (Head-Up Display) device 10 included in the vehicle display system 100 shown in FIG. As shown in FIG. 1, the HUD device 10 is provided inside a dashboard 2 of the vehicle 1, and not only information related to the vehicle 1 (hereinafter referred to as vehicle information) but also information other than vehicle information is integrated. The driver D is notified. The vehicle information includes not only information on the vehicle 1 itself but also information on the outside of the vehicle 1 related to the operation of the vehicle 1.

  The vehicle display system 100 is a system configured in the vehicle 1, and includes a HUD device 10, a peripheral information acquisition unit 40, a front information acquisition unit 50, a line-of-sight detection unit 60, and an ECU (Electronic Control Unit) 70. With.

  As shown in FIG. 1, the HUD device 10 emits display light L toward the windshield 3 subjected to the combiner process. The display light L reflected by the windshield 3 goes to the driver D side. The driver D can visually recognize the image represented by the display light L as the virtual image V in front of the windshield 3 by placing the viewpoint in the eye box E. That is, the HUD device 10 displays the virtual image V at the front position of the windshield 3. Thereby, the driver D can observe the virtual image V superimposed on the front scenery.

  The HUD device 10 includes the display unit 20 and the control device 30 shown in FIG. 2 and a reflection unit (not shown).

  The display unit 20 displays a superimposed image that is visually recognized by the driver D as a virtual image V under the control of the control device 30. The display unit 20 includes, for example, a TFT (Thin Film Transistor) type LCD (Liquid Crystal Display), a backlight that illuminates the LCD from behind, and the like. The backlight is composed of, for example, an LED (Light Emitting Diode). The display unit 20 generates the display light L when the LCD illuminated by the backlight displays an image under the control of the control device 30. The generated display light L is emitted toward the windshield 3 after being reflected by the reflecting portion. The reflection part is composed of, for example, two mirrors, a folding mirror and a concave mirror. The folding mirror folds the display light L emitted from the display unit 20 and directs it toward the concave mirror. The concave mirror reflects the light L toward the windshield 3 while expanding the display light L from the folding mirror. Thereby, the virtual image V visually recognized by the driver D is an enlarged image displayed on the display unit 20. Note that the number of mirrors constituting the reflecting portion can be arbitrarily changed according to the design.

  In the following description, the display unit 20 displaying an image visually recognized by the driver D as the virtual image V is also referred to as “displaying a superimposed image”. In addition, the fact that the control device 30 performs display control of the display unit 20 is also referred to as “performs display control of a superimposed image”. In addition, the display unit 20 is not limited to the one using an LCD as long as it can display a superimposed image, such as OLED (Organic Light Emitting Diodes), DMD (Digital Micromirror Device), and LCOS (Liquid Crystal On Silicon). A display device may be used.

  The control device 30 includes a microcomputer that controls the overall operation of the HUD device 10, and includes a control unit 31, a ROM (Read Only Memory) 32, and a RAM (Random Access Memory) 33. Moreover, the control apparatus 30 is provided with the input / output circuit for communicating with a drive circuit and the various systems in the vehicle 1 as a structure which is not shown in figure.

  The ROM 32 stores an operation program and various image data in advance. The RAM 33 temporarily stores various calculation results and the like. The control unit 31 includes a CPU (Central Processing Unit) 31a that executes an operation program stored in the ROM 32, and a GPU (Graphics Processing Unit) 31b that executes image processing in cooperation with the CPU. In particular, the ROM 32 stores an operation program for executing display control processing described later. A part of the control unit 31 may be configured by an ASIC (Application Specific Integrated Circuit). Moreover, the structure of the control apparatus 30 and the control part 31 is arbitrary as long as the function demonstrated below is satisfied.

  The control unit 31 drives and controls the display unit 20. For example, the control unit 31 drives and controls the backlight of the display unit 20 with the CPU 31a, and drives and controls the LCD of the display unit 20 with the GPU 31b that operates in cooperation with the CPU 31a.

  The CPU 31a of the control unit 31 controls the superimposed image based on various image data stored in the ROM 32 in cooperation with the GPU 31b. The GPU 31b determines the control content of the display operation of the display unit 20 based on the display control command from the CPU 31a. The GPU 31 b reads image part data necessary for configuring one screen to be displayed on the display unit 20 from the ROM 32 and transfers it to the RAM 33. Also, the GPU 31b uses the RAM 33 to create picture data for one screen based on the image part data and various image data received from the outside of the HUD device 10 by communication. Then, when the GPU 31b completes the picture data for one screen in the RAM 33, the GPU 31b transfers the picture data to the display unit 20 in accordance with the update timing of the image. Thereby, the superimposed image visually recognized by the driver D as a virtual image V is displayed on the display unit 20. In addition, a layer is assigned in advance to each image constituting an image visually recognized as the virtual image V, and the control unit 31 can individually control display of each image.

  In addition, the control unit 31 determines the display position of the content image (the first image C1 and the second image C2 shown in FIGS. 4A to 4C) in the display area of the virtual image V and the display distance of the content image. Control. Note that the dotted frame indicated by the virtual image V in FIGS. 4A to 4C indicates the display area of the virtual image V, and the content image is an image that is visually recognized as a virtual image in the display area. This is synonymous with displaying the content image in the display area of the superimposed image on the display unit 20 that displays the superimposed image visually recognized by the driver D as the virtual image V.

  The control of the display position means that the control unit 31 controls the display unit 20 so that an image is displayed at an arbitrary position in the display region of the virtual image V as viewed from the driver D. The display distance control is a control of how far the content image is displayed from the vehicle 1 when viewed from the driver D. That is, the control of the display distance is that the control unit 31 controls the display unit 20 so that the content image is visually recognized at a position facing a predetermined distance in front of the vehicle 1. The control of the display distance is realized by the control unit 31 being able to control the display of the content image in the display area of the virtual image V in a manner having a depth. For example, in order to display a content image with the depth, 3D image data is stored in the ROM 32 in advance, and predetermined objects (first object M1 and second object M2) that can be calculated from object information or the like. ) And the distance from the vehicle 1 to the predetermined object, the content image is controlled to be drawn using a perspective method (for example, a one-point perspective method). Further, by arranging the display surface of the display unit 20 (an LCD display surface or a screen in the case of using DMD or LCOS) with an inclination, a virtual surface on which the virtual image V is displayed (on the display surface of the display unit 20). The content image may be given a depth by setting the corresponding surface) to be tilted forward with respect to the vertical direction of the vehicle 1. In this case, in consideration of the projection of the image desired to be viewed from the viewpoint of the driver D onto the virtual surface, the control unit 31 may perform drawing control of the content image in a manner having a depth. As the viewpoint position of the driver D, the control unit 31 may use an assumed viewpoint position stored in advance in the ROM 32, or may use the line-of-sight information from the line-of-sight detection unit 60.

  In this embodiment, in particular, since the content image can be displayed with a depth as described above, the control unit 31 can display the first image C1 and the second image C2 shown in FIGS. The image C2 can be displayed in an AR (Augmented Reality) mode.

  Here, the first image C1 is an image displayed at a position corresponding to a first object M1 as a forward object, which will be described later, and is an image for notifying the presence of the first object M1. 4A and 4B show an example in which the first object M1 is a preceding vehicle of the vehicle 1 (hereinafter also referred to as the host vehicle 1). Note that the “position corresponding to the first object M1” as the display position of the first image C1 is not limited to a position that is visually recognized by being superimposed on the first object M1, and is shown in FIGS. 4A and 4B. As shown, it may be a position in the vicinity of the first object M1. The same applies to the “position corresponding to the second object M2” as the display position of the second image C2 described below.

  Further, the second image C2 is an image displayed at a position corresponding to a second object M2 as an interrupting object to be described later, and is an image for notifying the presence of the second object M2. 4B and 4C show an example in which the second object M2 is an interrupted vehicle that has interrupted between the first object M1 as the preceding vehicle and the host vehicle 1. Note that the first image C1 and the second image C2 may have any form as long as the presence of the corresponding object can be alerted, and may be, for example, a character, a symbol, a graphic, an icon, or a combination thereof. .

  The first image C1 in the AR mode is visually recognized by the driver D along the front road R as a real scene, as shown in FIG. The first image C1 in the AR mode is drawn by using the perspective method (for example, one-point perspective method) by the control unit 31 so as to be along the forward road R specified based on traffic information and forward image data described later, for example. Is done. Here, an image drawn using the perspective method and viewed along the front road R is set as a perspective degree “100%” (see FIG. 5B described later). On the other hand, an image that is visually recognized by getting up to the driver D side with respect to the front road R and being viewed in front of the driver D is defined as “0%”. The degree between 100% and 0% of the perspective corresponds to the degree of rising from the front road R toward the driver D in the visually recognized image. In such a case, the first image C1 in the AR mode is displayed with a perspective degree of 100%, for example. Moreover, the control part 31 can change the 1st image C1 of AR mode to the standing mode which stands | starts up and is visually recognized from the driver | operator D side rather than AR mode, as shown in FIG.4 (b). ing. The perspective degree of the first image C1 in the standing manner may be smaller than the perspective degree of the first image C1 in the AR manner.

  As shown in FIG. 4B, the second image C2 in the AR mode is visually recognized by the driver D along the front road R as a real scene. For example, the second degree C2 has a perspective degree smaller than 100%. Is set. Moreover, the control part 31 can change the 2nd image C2 of AR mode to the standing mode which stands | starts up and is visually recognized from the driver | operator side rather than AR mode, as shown in FIG.4 (c). Yes. The second image C2 in the standing manner is displayed with, for example, a perspective degree of 0%, that is, facing the driver D.

  As described above, each of the first image C1 and the first image C2 is capable of image transition between the AR mode and the standing mode under the control of the control unit 31. However, in this embodiment, as described above, the AR mode (first special mode) of the first image C1 is different from the AR mode (second special mode) of the second image C2. Further, the stand-up mode (first predetermined mode) of the first image C1 is different from the stand-up mode (second predetermined mode) of the second image C2.

  Returning to FIG. 2, the CPU 31 a of the control unit 31 communicates with each of the peripheral information acquisition unit 40, the front information acquisition unit 50, the line-of-sight detection unit 60, and the ECU 70. As the communication, for example, a communication method such as CAN (Controller Area Network), Ethernet, MOST (Media Oriented Systems Transport), or LVDS (Low Voltage Differential Signaling) is applicable.

  The peripheral information acquisition unit 40 acquires information about the periphery (external) of the vehicle 1, and communicates between the vehicle 1 and a wireless network (V2N: Vehicle To cellular Network), and communicates between the vehicle 1 and another vehicle (V2V). : Vehicle To Vehicle), communication between the vehicle 1 and pedestrians (V2P: Vehicle To Pedestrian), and communication between the vehicle 1 and roadside infrastructure (V2I: Vehicle To roadside Infrastructure). That is, the peripheral information acquisition unit 40 enables communication by V2X (Vehicle To Everything) between the vehicle 1 and the outside of the vehicle 1.

  For example, (i) the peripheral information acquisition unit 40 includes a communication module that can directly access a WAN (Wide Area Network), an external device (such as a mobile router) that can access the WAN, an access point of a public wireless LAN (Local Area Network), and the like. A communication module for communication is provided, and Internet communication is performed. The peripheral information acquisition unit 40 includes a GPS controller that calculates the position of the vehicle 1 based on a GPS (Global Positioning System) signal received from an artificial satellite or the like. With these configurations, communication by V2N is enabled. (Ii) In addition, the peripheral information acquisition unit 40 includes a wireless communication module compliant with a predetermined wireless communication standard, and performs communication using V2V or V2P. (Iii) In addition, the peripheral information acquisition unit 40 has a communication device that wirelessly communicates with roadside infrastructure, and is installed as an infrastructure from a base station of a driving safety support system (DSSS), for example. Object information and traffic information are acquired via the roadside apparatus. As a result, communication by V2I becomes possible.

  In this embodiment, the surrounding information acquisition unit 40 acquires object information indicating the position, size, attributes, and the like of various objects such as a vehicle, a traffic light, and a pedestrian existing outside the host vehicle 1 by using the V2I. 31. The object information is not limited to V2I, and may be acquired by any communication of V2X. In addition, the surrounding information acquisition unit 40 acquires traffic information including the position and shape of the surrounding road of the vehicle 1 by V2I and supplies the traffic information to the control unit 31. Further, based on information from the GPS controller of the peripheral information acquisition unit 40, the control unit 31 calculates the position of the host vehicle 1.

  The forward information acquisition unit 50 is, for example, a stereo camera that captures a landscape in front of the vehicle 1 or a LIDAR (Laser Imaging Detection And Ranging) that measures the distance from the vehicle 1 to an object positioned in front of the vehicle 1. A distance measuring sensor, a sonar that detects an object located in front of the vehicle 1, an ultrasonic sensor, a millimeter wave radar, and the like are included.

  In this embodiment, the forward information acquisition unit 50 transmits forward image data indicating a forward landscape image captured by a stereo camera, data indicating a distance to an object measured by a distance measuring sensor, and other detection data to the CPU 31a. To do.

  The line-of-sight detection unit 60 includes, for example, an imaging unit (for example, a stereo camera) that captures the face of the driver D and generates imaging data, and an image processing unit that performs image processing of the captured data. Is detected. For example, the line-of-sight detection unit 60 detects the line-of-sight direction of the driver D by a known image analysis method such as a pattern matching method, and transmits detection data indicating the detection result to the CPU 31a.

  The CPU 31a refers to the table data stored in advance in the ROM 32 in which the line-of-sight direction and the position of the visual target are associated with each other, and identifies the target that the driver D is currently viewing. The line-of-sight direction may be detected based on, for example, either eye of the driver D or may be detected as the center of gravity of both eyes. If the CPU 31a can identify the target that the driver D is viewing, the detection data is arbitrary. Further, the line-of-sight detection unit 60 may detect the direction of the face of the driver D by a known image analysis method, and use the detected data as detection data. In this case, the CPU 31a refers to the table data stored in advance in the ROM 32 in which the face orientation and the position of the visual recognition target are associated with each other, and identifies the target that the driver D is currently viewing. . In addition, you may obtain | require the position of the visual recognition object by numerical formula from a gaze direction, without using table data. Data indicating the mathematical formula may be stored in the ROM 32 in advance.

  The line-of-sight detection unit 60 may be an imaging unit, an electrooculogram sensor, or a motion sensor mounted on a wearable device worn by the driver D on the head. The line-of-sight direction and the face direction can be detected from the imaging data obtained by the imaging means, as described above. In addition, the electrooculogram sensor detects the line-of-sight direction of the driver D based on the measured electrooculogram. Moreover, a motion sensor consists of 1 or a combination of an acceleration sensor, a gyro sensor, and a geomagnetic sensor, for example, and detects the direction of the driver | operator's D face.

  Note that the CPU 31a may be configured to detect the line-of-sight direction and the face direction of the driver D based on signals from the imaging means and various sensors. As long as the CPU 31a can identify the target that the driver D is viewing, the detection method and detection configuration of the driver D's line-of-sight direction and face orientation are arbitrary.

  ECU70 controls each part of the vehicle 1, for example, transmits the vehicle speed information which shows the present vehicle speed of the vehicle 1 to CPU31a. Note that the CPU 31a may acquire vehicle speed information from a vehicle speed sensor. In addition, the ECU 70 transmits a measured amount such as the engine speed, warning information of the vehicle 1 itself (fuel reduction, engine oil pressure abnormality, etc.) and other vehicle information to the CPU 31a. Based on the information acquired from the ECU 70, the CPU 31a can display an image indicating the vehicle speed, the engine speed, various warnings, and the like on the display unit 20 via the GPU 31b. That is, as shown in FIGS. 4A to 4C, in addition to the first image C1 and the second image C2, an image V1 indicating vehicle information can be displayed in the display area of the virtual image V. Note that the image V1 is displayed with a degree of perspective of 0% so that it can be visually recognized as facing the driver D, for example. Further, the image V1 may include, for example, an image indicating traffic information (for example, a speed limit) acquired by V2I via the peripheral information acquisition unit 40.

(Display control processing)
Next, a display control process for performing display for notifying the presence of various objects existing in front of the host vehicle 1 will be described with reference to FIG. The display control process is executed by the control unit 31 and is continuously executed in a state where the ignition of the vehicle 1 is turned on, for example.

  When the display control process is started, first, the control unit 31 acquires object information from the peripheral information acquisition unit 40 (step S1). Subsequently, the control unit 31 determines whether or not there is a front object (first object M1) located in front of the host vehicle 1 based on the acquired object information (step S2).

  In step S2, for example, the control unit 31 determines that the object is a virtual image V viewed from the driver D based on the position of the object indicated by the object information acquired by the V2I or V2V from the roadside infrastructure via the peripheral information acquisition unit 40. When it is determined that the object is located within the display area, it is determined that there is a front object (first object M1). Note that the method for identifying the presence of the forward object is not limited to this. For example, the object information from the peripheral information acquisition unit 40 is not limited to V2I or V2V, and may be acquired by any communication (at least one of V2N, V2V, and V2P) of V2X. Further, the control unit 31 may specify the front object based on the information acquired from the front information acquisition unit 50. Specifically, based on the forward image data, for example, a forward object may be specified by a known image analysis method such as a pattern matching method, or a detection signal from a ranging sensor, sonar, ultrasonic sensor, or millimeter wave radar The forward object may be specified based on the above. Even when the front object is specified based on the front image data and the detection signal, it can be said that the control unit 31 obtains object information because the position of the front object is calculated in the calculation process.

When it is determined that there is a front object (step S2; Yes), the control unit 31 determines whether or not the second image C2 is being displayed (step S3). When the second image C2 is not being displayed (step S3; No), as shown in FIG. 4A, the control unit 31 takes the first position at the position corresponding to the front object (first object M1) specified in step S2. One image C1 is displayed in the AR mode (step S4).
On the other hand, when it is determined that there is no forward object (step S2; No), the control unit 31 hides the first image C1 (erase from the display area of the virtual image V) (step S5). After execution of step S5, the control unit 31 executes again from the process of step S1.

  Following execution of step S4 or when the second image C2 is being displayed (step S3; Yes), the control unit 31 determines whether or not there is an interrupting object (second object M2) (step S6). .

  Here, the interrupting object refers to an object that is positioned between or about to be positioned between the vehicle 1 and the front object. The control unit 31 identifies the object based on the object information in the same manner as in step S2. For example, (a) at least a part of the object is located in the travel lane of the host vehicle 1, or (b) At least part of the object has entered the display area of the virtual image V, (c) At least part of the object has entered the field of view of the driver D visible through the windshield 3, ) When it is specified that the object has behaved into the driving lane, the display area, or the view area, the object is specified as an interrupted object (second object M2). (It is determined that there is an interrupting object). When the interrupting object is specified based on the behavior of the object as in (D), only the behavior of the object in front of the host vehicle 1 can be specified only by the information from the front information acquisition unit 50. 31 preferably uses information from the peripheral information acquisition unit 40. This is because there are interrupting objects appearing from the side or the back of the own vehicle 1 (such as a vehicle overtaking the own vehicle 1 or a motorcycle running alongside the own vehicle 1).

  When it is determined that there is an interrupted object (step S6; Yes), the control unit 31 determines whether or not “the first image C1 is being displayed and the second image C2 is in a non-display state” (step S6). S7). When it is not “the first image C1 is being displayed and the second image C2 is not being displayed” (step S7; No), the control unit 31 executes again from the process of step S1.

  When the first image C1 is being displayed and the second image C2 is in the non-display state (step S7; Yes), the control unit 31 gradually erases the first image C1 displayed in the AR mode. Then, an image transition process for gradually displaying the second image C2 is executed (step S8). Specifically, in the image transition process, the control unit 31 raises the first image C1 displayed in the AR mode from the AR mode as shown in order in FIGS. 4 (a) to 4 (c). The second image C2 is faded in while changing from the AR mode to the standing mode. For example, the control unit 31 determines the content image (first image) according to the distance from the own vehicle 1 to the identified objects (first object M1, second object M2) and the position and attribute of the object indicated by the object information. The display control of the content image may be performed based on data for generating (C1, second image C2) (data indicating a table, a mathematical expression, an image part, etc.). The data may be stored in the ROM 32 in advance.

Here, the states shown in FIGS. 4A to 4C will be described in order.
FIG. 4A shows an example in which a first image C <b> 1 in the AR mode is displayed corresponding to the first object M <b> 1 as the preceding vehicle of the host vehicle 1. FIG. 4B shows a state in which the second object M2 as another vehicle has entered between the first object M1 and the host vehicle 1. When the second object M2 is positioned in this manner, the first image C1 for emphasizing the presence of the first object M1 is lost because it overlaps the first image C1 in the AR mode. Therefore, in step S8, the first image C1 in the AR mode is faded out while gradually changing the perspective degree to the standing mode. At the same time, the second image C2 for emphasizing the presence of the second object M2 as an interrupting object is faded in while gradually changing the degree of perspective to a standing state. As a result, as shown in FIG. 4C, the second image C2 for the second object M2 that has been interrupted in front of the host vehicle 1 can be displayed so as to be viewed in front of the driver D.

  Subsequent to step S8, the control unit 31 acquires line-of-sight information indicating the line-of-sight direction of the driver D from the line-of-sight detection unit 60, and based on the acquired line-of-sight information, the driver D visually recognizes the second image C2 in the standing state. It is determined whether or not (step S9).

  When it is determined in step S9 that the driver D is viewing the second image C2 in the standing state (step S9; Yes), the control unit 31 displays the second image C2 in the display area of the virtual image V before the determination. A visibility ensuring process of moving to the end side position is executed (step S10). The visibility ensuring process can suppress the second image C <b> 2 presumed to have been recognized by the driver D from hindering the driver D from viewing the front landscape. Note that the second image C2 may be moved to a position closer to the end side of the display area of the virtual image V than before the determination, and the size of the second image C2 may be reduced more than before the determination. The visibility of the second image C2 may be lowered than before (for example, the display brightness and saturation of the second image C2 may be reduced, the transparency may be increased, etc.).

  After the execution of the process of step S10 or when the driver D has not visually recognized the second image C2 in the standing state (step S9; No), the control unit 31 executes again from the process of step S1.

  When it is determined in step S6 that there is no interrupted object (step S6; No), the control unit 31 advances the process to step S11. If there is an interrupted object (second object M2) that has already been identified in the previous step S6, the control unit 31 determines that the second object M2 is between the vehicle 1 and the first object M1. What is necessary is just to discriminate | determine that there is no interruption object, when the movement state which shows having come off can be specified. The movement state is, for example, that a predetermined part of the second object M2 is located outside the traveling lane of the own vehicle 1, or that the second object M2 exhibits a behavior that deviates from the traveling lane of the own vehicle 1. In step S11, the control unit 31 determines whether or not “the second image C2 is being displayed and the first image C1 is not being displayed”. When the “second image C2 is being displayed and the first image C1 is not being displayed” is not in effect (step S11; No), the control unit 31 executes the process from step S1 again.

  When the second image C2 is being displayed and the first image C1 is in a non-display state (step S11; Yes), the control unit 31 gradually deletes the second image C2 and gradually deletes the first image C1. (Step S12). Specifically, the control unit 31 causes the second image C2 to fade out while changing from the standing mode to the AR mode, and causes the first image C1 to fade in while changing from the standing mode to the AR mode. That is, display control is performed to reversely follow the image transition described in step S8. By doing so, the driver D's consciousness can be smoothly directed to the first object M1. After execution of the process of step S12, the control unit 31 executes the process from step S1 again. The display control process described above is continuously executed until, for example, the HUD device 10 is turned off.

5A and 5B show timing charts of transition examples of the first image C1 and the second image C2.
FIG. 5A shows the visibility of the first image C1 and the second image C2 over time. Here, the visibility is, for example, display luminance. In the figure, the portion where the visibility is maximum is a luminance set according to a predetermined luminance or an external light environment, and the virtual image V is displayed. The image displayed in the area indicates the luminance that is clearly visible to the driver D. On the other hand, the portion where the visibility is minimum in the figure shows a state in which the image is not displayed. The visibility here may be transparency (maximum visibility when transparency is 0%, minimum visibility when transparency is 100%).
FIG. 5B shows the degree of parsing of the first image C1 and the second image C2 over time corresponding to FIG. t1 is the timing at which the interrupting object (second object M2) is specified, and corresponds to the timing at which Yes is determined in step S6.

  In the state “0 ≦ t <t1” before the interrupted object is specified, the first image C1 in the AR mode with a perspective degree of 100% visually recognized along the front road R is at a position corresponding to the first object M1. indicate.

  “T1 ≦ t <t4” after specifying the interrupted object corresponds to the image transition processing in step S8.

At “t1”, the visibility of the first image C1 starts to be lowered and fade-out is started. In addition, in order for the driver D to intuitively pay attention to the information in front, the degree of the first image C1 starts to be lowered.
Display of the second image C2 at a position corresponding to the second object M2 (interrupt object) is started at “t2” when the visibility of the first image C1 is lower than the predetermined value TH. From this time, the second image C2 becomes visible from the non-display state, but it is preferable to set the perspective of the second image C2 at t2 to a value lower than 100%. For example, by making the change rate of the second image C2 the same as the first image C1 (by making the “tilt” the same as shown in FIG. 5B), a smooth change in the perspective is realized. It is possible to suppress the driver D from feeling uncomfortable due to the change in the perspective.
At “t3”, the visibility of the first image C1 becomes 0 (non-display), so the parsing control of the first image C1 is ended here. On the other hand, the increase in the visibility of the second image C2 continues until the visibility becomes maximum at “t4”. At this time, the display of the second image C2 is controlled so that the degree of perspective is 0%.

  The image control in step S12 may be considered retrospectively from the time series shown in FIG. Further, in the example shown in FIG. 5, t2 and t3 are separated, but “t2 = t3” may be used. By so doing, the display control described above can be performed on either the first object M1 or the second object M2, so that it is possible to avoid complicated display control and display appearance. Further, in the example shown in FIG. 5, the change rate (slope) of the degree of perspective is matched between the first image C1 and the second image C2, but the change rate (slope) of the degree of perspective of the second image C2 is the first. The rate of change in the degree of parsing of the image C1 may be larger. By so doing, it is possible to further increase the degree of attention to the second object M2 as the interrupting object.

  In addition, this invention is not limited by the above embodiment, a modified example, and drawing. Changes (including deletion of components) can be made as appropriate without departing from the scope of the present invention.

  The first object M1 and the second object M2 are not limited to vehicles, and may be pedestrians, bicycles, and other moving objects. Note that the first object M1 may be an object that does not move (for example, a part of the front road R). For example, a route guidance image (an arrow-shaped image or an image showing a travel lane) that is visually recognized by being superimposed on a part of the front road R as the first object M1 is the first image C1, and the first image C1. When the second object M2 as an interrupted object is generated between the display position of the vehicle and the host vehicle 1, the display control process may be performed.

  In the above description, the AR mode (first special mode) of the first image C1 and the AR mode (second special mode) of the second image C2 have different degrees of perspective, and the standing mode (first mode) of the first image C1. Although the predetermined aspect) and the standing aspect (second predetermined aspect) of the second image C2 have been shown as examples having different degrees of perspective, the degree of perspective of each aspect is arbitrary. For example, the AR mode of the second image C2 may be set to 100%, and the standing mode of the first image C1 may be set to 0%. Further, “perceived along the front road R” is not limited to the perspective degree 100%, and if the driver D can recognize that the image is displayed along the front road R, the degree of perspective It may be a predetermined degree smaller than 100%. In addition, the display between the AR mode and the standing mode in the first image C1 and the second image C2 does not have to use a strict perspective method using a vanishing point or the like. It is arbitrary if a change between a state visually recognized along the line and a state visually recognized by itself can be felt.

  The projection target of the display light L is not limited to the windshield 3 but may be a combiner configured by a plate-shaped half mirror, a hologram element, or the like. Further, the display device that executes the above display control processing is not limited to the HUD device 10. The display device may be configured as a head mounted display (HMD) that is mounted on the head of the driver D of the vehicle 1. And in the image which HMD displays as a virtual image, you may perform display control of a content image (1st image C1, 2nd image C2) by the method similar to the above-mentioned. That is, the display device is not limited to the one mounted on the vehicle 1 and may be any device used in the vehicle 1.

The HUD device 10 (an example of a display device) described above displays a superimposed image that is visually recognized by the driver D (an example of a viewer) as a virtual image V that overlaps with the scenery in front of the vehicle 1.
The HUD device 10 includes object information acquisition means (for example, a control unit 31 that executes the process of step S1) that acquires information about an object existing around the vehicle 1 and indicates at least the position of the object, and the object Based on the information, the first object M1 and the second object M2 located closer to the vehicle 1 than the first object M1 are acquired, display control of the superimposed image is performed, and the first object is displayed in the display area of the superimposed image. Display control means capable of displaying the first image C1 at a position corresponding to M1 and displaying the second image C2 at a position corresponding to the second object M2 (for example, the control unit 31 for executing the processes of steps S8 and S12, Display unit 20).
The display control means includes a first special mode (AR mode) in which the first image C1 is visually recognized along the front road R of the vehicle 1, and a first image in which the first image C1 is visually recognized by rising from the first special mode. The second special mode (AR) can be displayed in a predetermined mode (standing mode), and the second image C2 is visually recognized by being tilted to the front road R side with respect to the second predetermined mode (standing mode). Display).
When both the first object M1 and the second object M2 exist, the display control process fades out while changing the first image C1 displayed in the first special mode from the first special mode to the first predetermined mode. In addition, an image transition process for fading in the second image C2 while changing from the second special mode to the second predetermined mode is executed.
Since it did in this way, when alerting | reporting the some information ahead of the vehicle 1, the information assumed that a priority is high can be alert | reported favorably.

In the above description, an example is shown in which the control unit 31 of the HUD device 10 specifies the presence of the first object M1 and the second object M2 in steps S2 and S6 of the display control process, but is not limited thereto. The control unit 31 specifies specific data indicating that the presence of the first object M1 and the second object M2 has been specified from any configuration outside the HUD device 10 (for example, the peripheral information acquisition unit 40 and the front information acquisition unit 50). And the determination process of step S2 or S6 may be executed based on the acquired specific data.
In addition, the control unit 31 determines that the second object M2 is between the host vehicle 1 and the first object M1 from any configuration outside the HUD device 10 (for example, the peripheral information acquisition unit 40 and the front information acquisition unit 50). Interrupt state information indicating that an interrupt has occurred can be acquired, and image transition processing may be executed in response to the acquisition of interrupt state information during display of the first image C1. Note that “the control unit 31 acquires the interrupt state information” is not only acquired from the outside of the HUD device 10 as described above, but also as described with reference to FIG. It also includes specifying the state and generating data indicating the identification result.
In addition, the control unit 31 determines whether the second object M2 is between the host vehicle 1 and the first object M1 from any configuration outside the HUD device 10 (for example, the peripheral information acquisition unit 40 and the front information acquisition unit 50). It is possible to acquire movement state information indicating that the image is detached, and change the second image C2 from the second predetermined mode to the second special mode in response to the acquisition of the movement state information during the display of the second image C2. While fading out, the first image C1 may be faded in while changing from the first predetermined mode to the first special mode. Note that “the control unit 31 acquires the movement state information” is not only acquired from the outside of the HUD device 10 as described above, but also the control unit 31 itself determines the movement state as described with reference to FIG. It includes specifying and generating data indicating the specified result.
That is, the identification of the first object M1 and the second object M2 and the identification of the interruption state and the movement state of the second object M2 itself can be performed by any configuration outside the HUD device 10 (for example, the peripheral information acquisition unit 40 or the front It may be performed by the information acquisition unit 50). And the specific data which shows these specific results may be contained in the object information which the control part 31 acquires by step S1.

  In the above description, in order to facilitate understanding of the present invention, descriptions of known technical matters are omitted as appropriate.

DESCRIPTION OF SYMBOLS 100 ... Display system for vehicles 10 ... HUD apparatus 20 ... Display part 30 ... Control apparatus 31 ... Control part (31a ... CPU, 31b ... GPU)
32 ... ROM, 33 ... RAM
40 ... Peripheral information acquisition unit 50 ... Forward information acquisition unit 60 ... Gaze detection unit 70 ... ECU
DESCRIPTION OF SYMBOLS 1 ... Vehicle, 2 ... Dashboard, 3 ... Windshield D ... Driver, E ... Eye box, L ... Display light V ... Virtual image V1 ... Image which shows vehicle information M1 ... 1st object, M2 ... 2nd object C1 ... 1st image, C2 ... 2nd image R ... Road ahead

Claims (5)

A display device that displays a superimposed image that is visually recognized by a viewer as a virtual image that overlaps a front landscape of a vehicle,
Object information acquisition means for acquiring object information indicating at least the position of the object, which is information relating to an object existing around the vehicle;
Based on the object information, the position of the first object and the position of the second object closer to the vehicle than the first object are acquired, display control of the superimposed image is performed, and display of the superimposed image is performed. Display control means capable of displaying a first image at a position corresponding to the first object and displaying a second image at a position corresponding to the second object in the region;
The display control means includes
The first image can be displayed in a first special mode visually recognized along the road ahead of the vehicle, and a first predetermined mode that stands up to the viewer side from the first special mode and is visually recognized.
The second image can be displayed in a second predetermined mode and a second special mode that is visually recognized by tilting toward the front road side than the second predetermined mode,
When both the first object and the second object exist, the first image displayed in the first special mode is faded out while changing from the first special mode to the first predetermined mode. Performing an image transition process for fading in the second image while changing the second image from the second special mode to the second predetermined mode;
Display device. The display control means can acquire interrupt status information indicating that the second object has interrupted between the vehicle and the first object, and the interrupt status is displayed during the display of the first image. Executing the image transition process in response to the acquisition of information;
The display device according to claim 1. The display control means can acquire movement state information indicating that the second object is out of the space between the vehicle and the first object, and has acquired the movement state information during the display of the second image. Accordingly, the second image is faded out while changing from the second predetermined mode to the second special mode, and the first image is changed from the first predetermined mode to the first special mode. While fading in,
The display device according to claim 1. When the display control unit acquires line-of-sight information indicating the line-of-sight direction of the viewer from the line-of-sight detection unit, and specifies that the viewer has viewed the second image based on the line-of-sight information, the second image Is moved to a position closer to the end of the display area than before being specified,
The display device according to claim 1. The first special mode and the second special mode are modes using a perspective method.
The display device according to claim 1.