JP7236674B2 - Display device - Google Patents

Description

TECHNICAL FIELD The present disclosure relates to, for example, a vehicle-mounted display device.

A head-up display (hereinafter also referred to as HUD) is known as a display device. A HUD projects an image onto a translucent display medium, presents the image to the user so as to be superimposed on an object that can be seen through the display medium, and realizes a so-called augmented reality (AR) display. can.

Some in-vehicle HUDs present information such as driving support information to the driver in the form of a virtual image superimposed on the actual scenery in front of the windshield. This type of display device is disclosed in Patent Documents 1 and 2, for example.

JP-A-7-257228 JP 2018-045103 A

By the way, since the AR display is performed in the front field of view, which is most important for driving, it does not interfere with the driver's driving. Although various ideas have been conventionally made for such a display, they are still insufficient.

The present disclosure has been made in consideration of the above points, and provides a display device capable of performing superimposed display that does not hinder driving.

One aspect of the display device of the present disclosure is
an image forming unit;
a projection unit capable of projecting an image formed by the image forming unit as a virtual image visible to a user;
a sensor that detects position information for determining a projection position of the virtual image projected by the projection unit;
An image display device comprising
The projection area by the projection unit is a first projection area in which a first image formed by the image forming unit is superimposed on the real image as the virtual image, and a position different from the first area. and a second projection area in which the second image formed by the image forming unit is displayed in a non-superimposed manner,
whether the first image and the second image having the same meaning are superimposed on the first projection area as the first image according to the sensing reliability of the sensor, or It is possible to switch whether the second image is displayed in a non-superimposed manner in the second projection area.

According to the present disclosure, whether the first image and the second image having the same meaning are displayed in a superimposed manner or displayed in a non-superimposed manner is switched according to the sensing reliability of the sensor. AR display that does not hinder driving can be realized.

FIG. 2 is a diagram showing an example of mounting a display device according to an embodiment on a vehicle; 4A and 4B are diagrams each showing an example of a region onto which light is projected by the display device according to the embodiment; A diagram showing an example of displaying a virtual image so as to overlap the foreground Block diagram showing a configuration example of a display device Diagram showing an example of correct virtual image displayed A diagram showing an example in which the wrong virtual image is displayed Diagram showing display area of HUD A diagram showing a display example in a non-superimposed display area FIGS. 9A and 9B are diagrams showing examples of display in a superimposed display area and display in a non-superimposed display area of images having the same meaning; FIG. 9A is a diagram showing display in the superimposed display area; FIG. Diagram showing the display of Flowchart for explaining switching operation between superimposed display and non-superimposed display Flowchart for explaining switching operation between superimposed display and non-superimposed display

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<1> Schematic Configuration of Display Device FIG. 1 is a diagram showing an example of mounting a display device 100 according to an embodiment of the present disclosure on a vehicle 200 .

Display device 100 in the present embodiment is embodied as a vehicle-mounted head-up display (HUD). Display device 100 is attached near the upper surface of dashboard 220 of vehicle 200 .

The display device 100 projects light onto a region D10 of the windshield (so-called windshield) 210, which is within the driver's field of vision, indicated by a dashed line. A portion of the projected light is transmitted through the windshield 210 while another portion is reflected by the windshield 210 . This reflected light is directed to the eyes of the driver. The reflected light that enters the driver's eyes is a virtual image that looks like an image of an object on the opposite side of the windshield 210 (outside the vehicle 200) against the background of the real object seen through the windshield 210. perceive as Vi.

FIG. 2 is a diagram showing an example of region D10, which is a region onto which light is projected by display device 100 in the present embodiment.

The area D10 is located below the windshield 210 on the driver's seat side, as shown, for example, in FIG. 2 as an area surrounded by a dashed line. Display device 100 attached to dashboard 220 projects an image onto windshield 210 by projecting light onto area D10 as shown in FIG. As a result, a virtual image Vi that looks like an image of an object outside the vehicle 200 to the driver is generated.

FIG. 3 is a diagram showing an example of a virtual image generated by display device 100 according to the present embodiment, and an example of superimposition of this virtual image with the scenery in front of vehicle 200 seen from the driver of vehicle 200 during travel. be.

Overall FIG. 3 schematically shows a portion of the scenery within the field of view of the driver (not shown) while driving the vehicle 200 . However, the dashed frame indicating the area D10 where the image is projected from the display device 100 is shown for convenience of explanation of the present embodiment, and does not exist and is perceived by the driver. Also, the arrow image with reference sign V10 is an AR (Augmented Reality) route, which is an example of the virtual image Vi generated by the display device 100 and perceived by the driver.

As shown in FIG. 3, the virtual image V10 is displayed so as to be superimposed on the scenery that can actually be seen within the field of view of the driver. In practice, the virtual image V10 is superimposed and displayed on the road. As a result, the driver is guided to run on the strip-shaped area indicated by the virtual image V10.

FIG. 4 is a block diagram showing a configuration example of the display device 100. As shown in FIG.

The display device 100 has a map information acquisition unit 101 , a position detection unit 102 , a distance measurement unit 103 , a vehicle behavior detection unit 104 , a viewpoint detection unit 105 , an image formation unit 110 , a display control unit 120 , a HUD 130 and a determination unit 150 .

The map information acquisition unit 101 acquires map information including information representing topography, road shapes, etc. in absolute coordinate system coordinates. The map information acquired by the map information acquisition unit 101 may be stored in a map information storage medium mounted on the vehicle 200, or may be acquired through communication with an external device. good. In the case of this embodiment, the map information acquisition unit 101 is a so-called navigation system, and acquires a route from the current location to the destination. The map information acquisition unit 101 outputs map information and route information to the image forming unit 110 .

The position detection unit 102 is embodied by a GPS receiver, a gyroscope, a vehicle speed sensor, etc., and detects the current position of the vehicle 200 .

A distance measuring unit 103 is a radar, a camera, an infrared sensor, or the like, and detects the presence or absence of an object and the distance to the object.

The vehicle behavior detection unit 104 is embodied by a gyroscope, a suspension stroke sensor, a vehicle speed sensor, an acceleration sensor, and the like, and detects physical quantities that indicate the behavior of the vehicle.

The viewpoint detection unit 105 captures an image of the driver's eye using, for example, an infrared camera, and performs image processing on the image of the captured eye to measure the coordinates of the position of the driver's eye in the vehicle coordinate system. A detection result by the viewpoint detection unit 105 is output to the display control unit 120 .

The image forming unit 110 forms an image based on the virtual image Vi based on input signals from the map information acquiring unit 101 , the position detecting unit 102 , the distance measuring unit 103 and the vehicle behavior detecting unit 104 .

Based on the image formed by the image forming unit 110 and the viewpoint information, the display control unit 120 controls the light source unit, the scanning unit, the screen driving unit, and the like, which constitute the HUD 130, so that the area D10 of the windshield is displayed. A virtual image Vi is displayed.

Display control unit 120 and HUD 130 function as projection unit 160 capable of projecting an image formed by image forming unit 110 as a virtual image superimposed on a real image seen by the user (driver in this embodiment). The display control unit 120 and the HUD 130 also function as a projection unit 160 capable of projecting the image formed by the image forming unit 110 as a non-overlapping virtual image that does not overlap the real image seen by the user.

In addition to these, in the case of the present embodiment, the position detection unit 102 and the distance measurement unit 103 output sensing reliability information to the determination unit 150 as indicated by the dotted line in the drawing. The determination unit 150 performs determination processing, which will be described later, based on the reliability information, and outputs the determination result to the image forming unit 110 . Image forming unit 110 switches between superimposed display and non-superimposed display based on the determination result.

<2> Association display in superimposed area and non-superimposed area FIGS. 5 and 6 show an example of the forward field of view that the driver can see. FIG. 5 is an example in which a correct virtual image V10 is displayed, and FIG. 6 is an example in which an incorrect virtual image V10 is displayed. In the correct display example of FIG. 5, a virtual image V10 is displayed in front of the traveling direction to instruct the vehicle to travel behind the obstacle X1. On the other hand, in the incorrect display example of FIG. 6, the virtual image V10 is displayed in front of the obstacle X1. One of the reasons for such incorrect display is that the accuracy of sensing signals output from the position detection unit 102 and the distance measurement unit 103 to the image forming unit 110 is poor.

Therefore, the display device 100 of the present embodiment inputs the sensing reliability information output from the position detection unit 102 and the distance measurement unit 103 to the determination unit 150, and based on the reliability information, the display displayed on the HUD 130 You can switch images.

In other words, depending on the reliability of the sensors (the position detection unit 102 and the distance measurement unit 103) that obtain position information for determining the projection position of the virtual image projected by the projection unit 150, superimposition display is performed or non-display is performed. It is designed to select whether to perform superimposed display. As a result, it is possible to prevent an incorrect (inaccurate) virtual image V10 as shown in FIG. 6 from being displayed.

FIG. 7 shows the display area of the HUD 130. As shown in FIG. The display area of the HUD is different from the superimposed display area (first projection area) 10 in which the first image formed by the image forming unit 110 is superimposed on the real image as a virtual image, and the first area. and a non-superimposed display area (second projection area) 20 in which the second image formed by the image forming unit 110 is displayed in a non-superimposed manner.

In the case of the present embodiment, the superimposed display area (first projection area) 10 is located in front of the driver. In other words, the superimposed display area (first projection area) 10 is located in the main visual field of the driver. In the embodiment, the non-superimposed display area (second projection area) 20 is located below and adjacent to the superimposed display area (first projection area) 10 . Note that the superimposed display area (first projection area) 10 and the non-superimposed display area (second projection area) 20 do not necessarily have to be adjacent to each other. The non-superimposed display area 20 may be called an ST (Static) display area. .

A second image having the same meaning as the first image that was not displayed in the superimposed display area 10 is displayed in the non-superimposed display area 20 . Also, in the non-superimposed display area 20, a numerical image representing vehicle behavior such as vehicle speed is displayed.

FIG. 8 is a diagram showing a display example in the non-superimposed display area 20. As shown in FIG. The virtual image V10 displayed in the superimposed display area 10 in FIGS. 5 and 6 is switched and displayed as the image W10 in the non-superimposed display area in FIG. A trigger for switching from superimposed display to non-superimposed display may be, for example, a decrease in the reliability of sensing as described above, or may be an operation by the driver.

FIG. 9 shows an example of display of images having the same meaning in the superimposed display area 10 and display in the non-superimposed display area 20 . In the display example in the superimposed display area 10 of FIG. 9A, an elliptical virtual image V10 is displayed at the foot of the pedestrian to alert the driver. In the display example in the non-superimposed display area 20 of FIG. 9B, a human-shaped image W10 representing a pedestrian is displayed. Then, as described above, the display shown in FIG. 9A and the display shown in FIG. 9B are switched using, for example, the reliability of sensing or the driver's operation as a trigger.

Thus, in the present embodiment, images having the same meaning are displayed in the superimposed display area 10 when the reliability of sensing is high, and displayed non-superimposed when the reliability of sensing is low. Displayed in area 20 . Accordingly, when the reliability of sensing is low, it is possible to prevent displaying a virtual image with poor positional accuracy with respect to the real image in the superimposed display area 10 . In addition, when the driver sees the image W10 displayed in the non-superimposed display area 20, it is not currently displayed in the superimposed display area 10, but the information actually meant by the image W10 can be conveyed to the driver. Furthermore, it is possible to recognize that the sensing process is running.

Incidentally, such switching control between superimposed display and non-superimposed display can also be performed by the driver's operation via an operation unit (not shown). For example, if the driver does not want to display the content in a superimposed manner, the content can be displayed as an icon image in a non-superimposed manner without being superimposed.

Here, the inventors of the present invention have found that when the first image and the second image having the same meaning are switched and displayed in the superimposed display area 10 and the non-superimposed display area 20, the superimposed display is performed. If the image displayed in the area 10 and the image displayed in the non-superimposed display area are associated with each other, the driver can immediately understand the meaning of the content, which is very convenient. .

Based on this consideration, in the present embodiment, when the first image and the second image having the same meaning are switched and displayed in the superimposed display area 10 and the non-superimposed display area 20, the superimposed An association process is performed on the first image displayed in the display area 10 and the second image displayed in the non-superimposed display area 20 . In practice, the process of forming the first image and the second image associated with each other is performed by the image forming section 110 .

In this embodiment, the following method is presented as a method of associating the first image and the second image. For association, any one of the following methods may be used, or a plurality of methods may be used in combination.

Association method 1)
A first image and a second image having the same meaning are associated by color. For example, the first image and the second image have the same color. For example, the elliptical virtual image V10 in FIG. 9 is made yellow, and the human-shaped image W10 in FIG. 9B is made yellow.

Association method 2)
A first image and a second image having the same meaning are related by shape. For example, the first image and the second image have the same or similar shape. For example, FIGS. 5 and 8 are examples corresponding to this, and the virtual image V10 (FIG. 5) and the image W10 (FIG. 8) have similar shapes.

Association method 3)
A first image and a second image having the same meaning are associated by position. For example, when the virtual image V10 of FIG. 9A is displayed at the leftmost position of the screen, the image W10 of FIG. 9B is also displayed at the leftmost position of the screen.

Association method 5)
After temporarily displaying the first image and the second image having the same meaning in both the superimposed display area 10 and the non-superimposed display area 20, the display is switched to either one.

<3> Switching control between superimposed display and non-superimposed display As described above, the display device 100 of the present embodiment uses the same The meaningful first image and second image (pedestrian in the example of FIG. 9) are superimposed on the first projection area (superimposed display area 10) as the first image (elliptical in the example of FIG. 9). It is switched whether to display or to non-superimpose display in the second projection area (non-superimposition display area 20) as the second image (human-shaped icon in the example of FIG. 9).

FIG. 10 is a flowchart for explaining switching operation between superimposed display and non-superimposed display.

In step S1, sensing is performed by the position detection unit 102, the distance measurement unit 103, and the like. When there is a sensing signal, the display device 100 performs processing for displaying an image corresponding to the sensing result.

Specifically, when there is a sensing signal, the process moves from step S2 to step S3. In step S3, the determination unit 150 determines whether or not the reliability of the sensing signal exceeds a predetermined threshold. If the reliability exceeds the threshold, the virtual image V10 is displayed in the superimposed display area 10 in step S4. On the other hand, if the reliability is equal to or less than the threshold, the process moves to step S5 and the image W10 is displayed in the non-superimposed display area. Specifically, in the example of FIG. 9, when the reliability exceeds the threshold, the image forming unit 110 forms an elliptical image as shown in FIG. It is displayed as a virtual image V10. On the other hand, when the reliability is equal to or less than the threshold, the image forming unit 110 forms a human-shaped image as shown in FIG. 9B, and this image is displayed in the non-superimposed display area 20 as the image W10.

After step S4 or step S5, the process returns to step S2. If it is determined in step S2 that there is no sensing signal, the display processing of the content (for example, pedestrian) ends.

In this way, when the reliability of sensing is low, it is possible to prevent an incorrect (in other words, poor positional accuracy) superimposed display as shown in FIG. You can prevent things from being passed on.

By the way, when the superimposed display and the non-superimposed display are switched based on the sensing reliability, if the sensing reliability changes frequently in a short time, the superimposed display and the non-superimposed display are switched frequently. Since it is performed, there is a possibility that the display may be troublesome for the driver. For example, when the vehicle 200 travels along a tree-lined road and bright and dark places are frequently switched, the reliability of the sensor changes frequently, so the superimposed display and non-superimposed display are frequently switched. likely to be done.

Therefore, in this embodiment, the following method is presented as an improvement measure for switching control.

When the sensing reliability exceeds a predetermined threshold for a predetermined period of time (for example, 0.5 seconds), superimposed display is performed, and otherwise, non-superimposed display is performed. FIG. 11, in which parts corresponding to those in FIG. 10 are denoted by the same reference numerals, is a flow chart corresponding to this switching control method. FIG. 11 differs from FIG. 10 in the determination criteria of step S11. When the reliability of sensing frequently changes in a short time, the process moves from step S11 to step S5. As a result, even if the reliability of sensing frequently changes in a short period of time, the switching between the superimposed display and the non-superimposed display is not frequently performed, so that annoying display can be suppressed.

Incidentally, this switching control method is an example in which the reliability condition for switching from superimposed display to non-superimposed display and the reliability condition for switching from non-superimposed display to superimposed display are different. can be said. By varying the switching conditions in this way, it is possible to prevent adverse effects due to frequent switching.

It should be noted that it is very dangerous to make a mistake in superimposed display, so the reliability condition for switching from non-superimposed display to superimposed display should be stricter than the reliability condition for switching from superimposed display to non-superimposed display. preferable.

Moreover, it is preferable that the second image once displayed in a non-superimposed state is not switched to a superimposed display until the information signified by the second image becomes invalid. For example, if the second image is an image for guiding a left turn as shown in FIG. 8, the superimposed display is not switched until the vehicle finishes turning the intersection. Also, for example, if the second image is an image for warning a pedestrian as shown in FIG. 9B, the superimposed display is not switched until the pedestrian is removed from the warning target. By doing so, it is possible to prevent the content from being superimposed and displayed suddenly, and the driver does not have to be surprised.

<4> Summary As described above, according to the present embodiment, the first image and the Either the second image (pedestrian in the example of FIG. 9) is superimposed on the first projection area (superimposed display area 10) as the first image (elliptical in the example of FIG. 9), or image (in the example of FIG. 9, a human-shaped icon) is switched to non-superimposed display in the second projection area (non-superimposed display area 20). A superimposed display can be avoided, and an AR display that does not hinder driving can be realized.

Further, by devising the conditions for switching between superimposed display and non-superimposed display, it is possible to realize AR display that does not impede the driving of the driver.

The above-described embodiments are merely examples of specific implementations of the present invention, and the technical scope of the present invention should not be construed to be limited by these. That is, the present invention can be embodied in various forms without departing from its spirit or essential characteristics.

In the above-described embodiments, the case where the display device of the present disclosure is applied to an in-vehicle HUD has been described, but the present invention is not limited to this. It is widely applicable to display devices that switch between display and non-superimposed display.

Each process of the above-described embodiments can be implemented by software, hardware, or software in cooperation with hardware. Each functional block used in the description of the above embodiments is partially or wholly realized as an LSI, which is an integrated circuit, and each process described in the above embodiments is partially or wholly implemented as It may be controlled by one LSI or a combination of LSIs. An LSI may be composed of individual chips, or may be composed of one chip so as to include some or all of the functional blocks. The LSI may have data inputs and outputs. LSIs are also called ICs, system LSIs, super LSIs, and ultra LSIs depending on the degree of integration. The method of circuit integration is not limited to LSI, but may be realized by a dedicated circuit, a general-purpose processor, or a dedicated processor. Further, an FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connections and settings of the circuit cells inside the LSI may be used. The present disclosure may be implemented as digital or analog processing. Furthermore, if an integration technology that replaces the LSI appears due to advances in semiconductor technology or another derived technology, the technology may naturally be used to integrate the functional blocks. Application of biotechnology, etc. is possible.

The display device of the present invention is suitable for, for example, an in-vehicle HUD.

REFERENCE SIGNS LIST 10 superimposed display area 20 non-superimposed display area 100 display device 101 map information acquisition unit 102 position detection unit 103 distance measurement unit 104 vehicle behavior detection unit 105 viewpoint detection unit 110 image formation unit 120 display control unit 130 HUD (head-up display)
150 Determination Unit 160 Projection Unit 200 Vehicle 210 Windshield 220 Dashboard Vi, V10 Virtual Image W10 Icon Image

Claims (4)

an image forming unit;
a projection unit capable of projecting an image formed by the image forming unit as a virtual image visible to a user;
A display device mounted on a vehicle ,
The image forming unit forms the image based on an input signal from a sensor that detects the current location of the vehicle or the position of an object ,
The projection area by the projection unit is a first projection area in which a first image formed by the image forming unit is superimposed on a real image as the virtual image, and a position different from the first area. a second projection area positioned so that the second image formed by the image forming unit is displayed in a non-superimposed manner;
whether the first image and the second image having the same meaning are superimposed on the first projection area as the first image according to the sensing reliability of the sensor, or switching whether to display the second image in a non-superimposed manner in the second projection area;
The first image is displayed in the first projection area when the reliability exceeds a predetermined threshold for a predetermined period of time or more; a second image is displayed,
display device. The second image once displayed in a non-superimposed manner is not switched to the superimposed display until the information implied by the second image becomes invalid.
The display device according to claim 1 . an image forming unit;
a projection unit capable of projecting an image formed by the image forming unit as a virtual image visible to a user;
A display device mounted on a vehicle,
The image forming unit forms the image based on an input signal from a sensor that detects the current location of the vehicle or the position of an object,
The projection area by the projection unit is a first projection area in which a first image formed by the image forming unit is superimposed on a real image as the virtual image, and a position different from the first area. a second projection area positioned so that the second image formed by the image forming unit is displayed in a non-superimposed manner;
whether the first image and the second image having the same meaning are superimposed on the first projection area as the first image according to the sensing reliability of the sensor, or switching whether to display the second image in a non-superimposed manner in the second projection area;
The second image once displayed in a non-superimposed manner is not switched to the superimposed display until the information implied by the second image becomes invalid.
display device. The first image is displayed in the first projection area when the reliability exceeds a predetermined threshold, and the second image is displayed in the second projection area when the reliability is less than or equal to the predetermined threshold. a second image is displayed,
The display device according to claim 3.

Images