JP7338632B2 - Display device - Google Patents

Description

The present disclosure relates to display devices.

2. Description of the Related Art As a conventional display device, Patent Literature 1 discloses one that displays an image visually recognized by a viewer (mainly a driver of a vehicle) as a virtual image superimposed on the scenery in front of the vehicle. The display device disclosed in Patent Literature 1 controls the relative height and the like of a notification image that notifies a forward target according to the distance from the vehicle to the forward target.

JP 2015-221633 A

If the technology disclosed in Japanese Patent Laid-Open No. 2002-200001 is used to simultaneously display a plurality of notification images as described above, the same display control is performed for each notification image according to the distance from the vehicle. However, there is room for improvement in effectively displaying one of the plurality of notification images that is assumed to have a higher priority for the viewer.

The present disclosure has been made in view of the above circumstances, and an object thereof is to provide a display device capable of effectively displaying a notification image assumed to have a high priority among a plurality of notification images.

In order to achieve the above object, the display device of the present disclosure includes:
A display device that displays a superimposed image that is visually recognized by a viewer as a virtual image that overlaps the scenery in front of the vehicle,
Acquisition means for acquiring a distance from the vehicle to a front object in front of the vehicle;
display control means for two-dimensionally controlling the display of a notification image that notifies a forward target within the display area of the superimposed image;
Storage means for storing reference control information for determining, in accordance with the distance acquired by the acquisition means, a display ratio of the notification image with respect to a predetermined reference size that decreases as the distance increases. and
The display control means is
As the notification image, a plurality of images including a first image and a second image for notifying a forward target having a longer distance than the first image can be displayed,
When the display ratio of the first image and the second image is changed and controlled based on the reference control information and the display ratio of the first image becomes equal to or greater than the first threshold, Enlargement processing is executed to display the two images at a ratio larger than the display ratio determined based on the reference control information.

According to the display device of the present disclosure, it is possible to effectively display a notification image that is assumed to have a high priority among a plurality of notification images.

(a) is a diagram showing an example of a mounting mode of a head-up display (HUD) device according to an embodiment of the present invention on a vehicle, and (b) is a diagram for explaining a two-dimensionally displayed notification image. It is a diagram. 1 is a block diagram of a vehicle display system; FIG. 9 is a flowchart showing notification image special control processing; (a) is a diagram of a function indicated by reference control information, and (b) and (c) are diagrams showing specific examples of notification images. (a) to (c) are diagrams for explaining notification image special control processing. (a) to (c) are diagrams showing transition examples of notification images when the notification image special control process is executed. (a) to (c) are diagrams for explaining a modification of the enlargement process. (a) to (c) are diagrams showing modified examples of the function indicated by the reference control information.

A display device according to an embodiment of the present invention will be described with reference to the drawings.

The display device according to the present embodiment is a HUD (Head-Up Display) device 10 included in the vehicle display system 100 shown in FIG. The HUD device 10 is provided inside the dashboard 2 of the vehicle 1, as shown in FIG. The driver 4 is notified. The vehicle information includes not only information on the vehicle 1 itself, but also information on the outside of the vehicle 1 related to the operation of the vehicle 1 .

The vehicle display system 100 is a system configured in the vehicle 1, and includes a HUD device 10, a peripheral information acquisition unit 40, a forward information acquisition unit 50, a line-of-sight detection unit 60, and an ECU (Electronic Control Unit) 70. and a car navigation (car navigation) device 80.

The HUD device 10 emits display light L toward the windshield 3, as shown in FIG. 1(a). The display light L reflected by the windshield 3 goes toward the driver 4 side. The driver 4 can visually recognize the image represented by the display light L displayed in front of the windshield 3 as a virtual image V by placing the viewpoint in the eyebox E. As a result, the driver 4 can observe the virtual image V superimposed on the front scenery.

The HUD device 10 displays a virtual image V two-dimensionally on a virtual plane set in front of the vehicle 1, as shown in FIG. 1(b). The virtual plane is set at a position a predetermined distance P (for example, about 5 to 10 m) ahead of the vehicle 1 from the visual point position 4a when the driver 4 places his/her visual point in the eye box E. The virtual plane corresponds to the image display plane of the display unit 20 . The virtual plane and the eyebox E are set based on the size of the display plane and the optical system configured by various mirrors in the HUD device 10 and the windshield 3 .

The HUD device 10 includes a display section 20 and a control device 30 shown in FIG. 2, and a reflection section (not shown).

The display unit 20 displays a superimposed image that is visually recognized by the driver 4 as a virtual image V under the control of the control device 30 . The display unit 20 has, for example, a TFT (Thin Film Transistor) type LCD (Liquid Crystal Display), a backlight for illuminating the LCD from behind, and the like. The backlight is composed of, for example, LEDs (Light Emitting Diodes). Under the control of the control device 30, the display unit 20 generates the display light L by displaying an image on the LCD illuminated by the backlight. The generated display light L is emitted toward the windshield 3 after being reflected by the reflecting portion. The reflecting section is composed of, for example, two mirrors, a folding mirror and a concave mirror. The folding mirror folds the display light L emitted from the display unit 20 and directs it toward the concave mirror. The concave mirror magnifies the display light L from the folding mirror and reflects it toward the windshield 3. - 特許庁As a result, the virtual image V visually recognized by the driver 4 is an enlarged version of the image displayed on the display unit 20 . The number of mirrors forming the reflecting section can be arbitrarily changed according to the design.

In addition, hereinafter, the display unit 20 displaying an image visually recognized by the driver 4 as the virtual image V is also referred to as “displaying a superimposed image”. Further, the display control of the display unit 20 by the control device 30 is also referred to as “executing the display control of the superimposed image”. Further, the display unit 20 is not limited to those using an LCD as long as it can display a superimposed image. A display device may be used.

The control device 30 is composed of a microcomputer that controls the overall operation of the HUD device 10 , and includes a control section 31 , a ROM (Read Only Memory) 32 and a RAM (Random Access Memory) 33 . In addition, the control device 30 includes a drive circuit and an input/output circuit for communicating with various systems in the vehicle 1 as configurations not shown.

The ROM 32 pre-stores an operation program and various image data. The RAM 33 temporarily stores various calculation results and the like. The control unit 31 includes a CPU (Central Processing Unit) 31a that executes an operation program stored in the ROM 32, and a GDC (Graphics Display Controller) 31b that executes image processing in cooperation with the CPU 31a. The GDC 31b includes, for example, a GPU (Graphics Processing Unit), FPGA (Field-Programmable Gate Array), ASIC (Application Specific Integrated Circuit), and the like. In particular, the ROM 32 stores an operation program for carrying out a notification image special control process, which will be described later, and data of the reference control information BD. The configurations of the control device 30 and the control unit 31 are arbitrary as long as the functions described below are satisfied.

The control unit 31 drives and controls the display unit 20 . For example, the control unit 31 drives and controls the backlight of the display unit 20 with the CPU 31a, and drives and controls the LCD of the display unit 20 with the GDC 31b that operates in cooperation with the CPU 31a.

The CPU 31 a of the control unit 31 cooperates with the GDC 31 b to control superimposed images based on various image data stored in the ROM 32 . The GDC 31b determines the control contents of the display operation of the display section 20 based on the display control command from the CPU 31a. The GDC 31b reads from the ROM 32 the image parts data required to compose one screen displayed on the display unit 20 and transfers it to the RAM 33. FIG. The GDC 31b also uses the RAM 33 to create picture data for one screen based on image parts data and various image data received from the outside of the HUD device 10 through communication. When the picture data for one screen is completed in the RAM 33, the GDC 31b transfers the picture data to the display section 20 in synchronization with the image update timing. As a result, the superimposed image visually recognized by the driver 4 is displayed as the virtual image V on the display unit 20 . A layer is assigned in advance to each image that constitutes an image that is visually recognized as the virtual image V, and the control unit 31 can perform individual display control of each image.

Further, the control unit 31 controls the display position and size of the content image within the virtual image V display area. In particular, in this embodiment, the control unit 31 controls the size of a notification image, which will be described later, included in the content image according to the distance from the vehicle 1 to the forward target to be notified by the notification image. The function and control of the notification image will be detailed later. 1(b) and FIGS. 6(a) to 6(c), the dotted frame indicated by the virtual image V indicates the display area of the virtual image V, and the content image is visually recognized as the virtual image V within the display area. This is an image that will be This is synonymous with displaying the content image within the display area of the superimposed image on the display unit 20 that displays the superimposed image as the virtual image V that is visually recognized by the driver 4 .

Further, the control unit 31 communicates with each of the peripheral information acquisition unit 40 , the forward information acquisition unit 50 , the line-of-sight detection unit 60 , the ECU 70 and the car navigation device 80 . Communication methods such as CAN (Controller Area Network), Ethernet, MOST (Media Oriented Systems Transport), and LVDS (Low Voltage Differential Signaling) are applicable as the communication.

The peripheral information acquisition unit 40 acquires information on the periphery (external) of the vehicle 1, and includes communication between the vehicle 1 and a wireless network (V2N: Vehicle To cellular Network), communication between the vehicle 1 and other vehicles (V2V (Vehicle To Vehicle), communication between the vehicle 1 and pedestrians (V2P: Vehicle To Pedestrian), and communication between the vehicle 1 and roadside infrastructure (V2I: Vehicle To Roadside Infrastructure). In other words, the peripheral information acquisition unit 40 enables communication between the vehicle 1 and the outside of the vehicle 1 by V2X (Vehicle To Everything).

For example, (i) the peripheral information acquisition unit 40 is connected to a communication module that can directly access a WAN (Wide Area Network), an external device that can access the WAN (such as a mobile router), a public wireless LAN (Local Area Network) access point, or the like. Equipped with a communication module for communication, etc., and performs Internet communication. The peripheral information acquisition unit 40 also includes a GPS controller that calculates the position of the vehicle 1 based on a GPS (Global Positioning System) signal received from an artificial satellite or the like. These configurations enable communication by V2N. (ii) The peripheral information acquisition unit 40 includes a wireless communication module conforming to a predetermined wireless communication standard, and performs V2V or V2P communication. (iii) In addition, the peripheral information acquisition unit 40 has a communication device that wirelessly communicates with roadside infrastructure, for example, from a base station of a driving safety support system (DSSS) installed as infrastructure Object information and traffic information around the vehicle 1 are acquired via the roadside wireless device. This enables V2I communication.

In this embodiment, the peripheral information acquisition unit 40 acquires object information indicating the positions, sizes, attributes, etc. of various objects such as vehicles, traffic lights, and pedestrians existing outside the vehicle 1 by V2I, and the control unit 31 supply to Note that the object information is not limited to V2I, and may be acquired by any communication among V2X. In addition, the peripheral information acquisition unit 40 acquires traffic information including the position and shape of roads around the vehicle 1 by V2I, and supplies the information to the control unit 31 . Also, the control unit 31 calculates the position of the vehicle 1 based on information from the GPS controller of the surrounding information acquisition unit 40 . These various types of information include information indicating the details of the forward target, which will be described later, and information for the control unit 31 to specify the distance from the vehicle 1 to the forward target.

The forward information acquisition unit 50 is, for example, a stereo camera that captures the scenery in front of the vehicle 1, and a distance measurement such as LIDAR (Laser Imaging Detection And Ranging) that measures the distance from the vehicle 1 to an object located in front of the vehicle 1. It is composed of a sensor, a sonar for detecting an object positioned in front of the vehicle 1, an ultrasonic sensor, a millimeter wave radar, and the like.

In this embodiment, the forward information acquisition unit 50 transmits forward image data representing a forward scenery image captured by a stereo camera, data representing the distance to an object measured by a range sensor, and other detection data to the control unit 31. Send to These various types of information include information indicating the details of the forward target, which will be described later, and information for the control unit 31 to specify the distance from the vehicle 1 to the forward target.

The line-of-sight detection unit 60 includes, for example, imaging means (for example, a stereo camera) that captures an image of the face of the driver 4 and generates image data, and an image processing unit that performs image processing on the image data. to detect For example, the line-of-sight detection unit 60 detects the line-of-sight direction of the driver 4 by a known image analysis method such as a pattern matching method, and transmits detection data indicating the detection result to the CPU 31a.

The CPU 31a refers to table data stored in advance in the ROM 32, in which the line-of-sight direction and the position of the visually recognized object are associated with each other, and identifies the object that the driver 4 is currently visually recognizing. Note that the line-of-sight direction may be detected based on either of the eyes of the driver 4, or may be detected as the center of gravity of the eyes. As long as the CPU 31a can specify the target visually recognized by the driver 4, any detection data can be used. Further, the line-of-sight detection unit 60 may detect the orientation of the face of the driver 4 by a known image analysis method and use it as detection data. In this case, the CPU 31a may identify the target currently viewed by the driver 4 by referring to the table data stored in advance in the ROM 32, in which the orientation of the face and the position of the target to be viewed are associated with each other. . It should be noted that the position of the visual recognition target may be obtained from the line-of-sight direction using a formula without using the table data. Data representing the formula may be stored in the ROM 32 in advance.

Also, the line-of-sight detection unit 60 may be imaging means mounted on a wearable device worn on the head of the driver 4, an electrooculogram sensor, or a motion sensor. It is possible to detect the line-of-sight direction and the face orientation in the same manner as described above, based on the imaging data obtained by the imaging means. Further, the electro-oculogram sensor detects the line-of-sight direction of the driver 4 based on the measured electro-oculography. Also, the motion sensor is, for example, one or a combination of an acceleration sensor, a gyro sensor, and a geomagnetic sensor, and detects the face orientation of the driver 4 .

Note that the CPU 31a may be configured to detect the line-of-sight direction and face orientation of the driver 4 based on signals from imaging means and various sensors. As long as the CPU 31a can specify the target visually recognized by the driver 4, any detection method and detection configuration can be used for the line-of-sight direction and face direction of the driver 4. FIG.

The ECU 70 controls each part of the vehicle 1 and, for example, transmits vehicle speed information indicating the current vehicle speed of the vehicle 1 to the CPU 31a. Note that the CPU 31a may acquire vehicle speed information from a vehicle speed sensor. In addition, the ECU 70 transmits to the CPU 31a a measurement amount such as the engine speed, warning information of the vehicle 1 itself (low fuel, abnormal engine oil pressure, etc.), and other vehicle information. Based on the information acquired from the ECU 70, the CPU 31a can also cause the display unit 20 to display images indicating vehicle speed, engine speed, various warnings, etc. via the GDC 31b. That is, in the display area of the virtual image V, it is possible to display an image showing vehicle information other than the notification image, which will be described later.

The car navigation device 80 includes a GPS controller that calculates the position of the vehicle 1 based on GPS signals received from satellites or the like. The car navigation device 80 has a storage unit for storing map data, reads map data near the current position from the storage unit based on the position information from the GPS controller, and provides a guidance route to the destination set by the user. decide. Then, the car navigation device 80 outputs information about the current position of the vehicle 1 and the determined guide route to the control unit 31 . Further, the car navigation device 80 outputs information indicating the names and types of facilities in front of the vehicle 1 and the distance between the facilities and the vehicle 1 to the control unit 31 by referring to the map data. Map data includes road shape information (lanes, road width, number of lanes, intersections, curves, forks, etc.), regulatory information on road signs such as speed limits, and information on each lane when there are multiple lanes. Various types of information are associated with position data. The car navigation device 80 outputs these various types of information to the control section 31 . These various types of information include information indicating the details of the forward target, which will be described later, and information for the control unit 31 to specify the distance from the vehicle 1 to the forward target. The car navigation device 80 is not limited to one mounted on the vehicle 1, and communicates with the control unit 31 by wire or wirelessly. ).

(Function and control of notification image)
Here, the function and control of the notification image will be described. The notification image displayed within the display area of the virtual image V is an image for notifying a forward target in front of the vehicle 1 . Forward objects include not only real objects (hereinafter referred to as real objects) but also information in the scenery ahead of the vehicle 1 . The notification image can be broadly classified into a display in an "enhancement mode" that emphasizes a real object, and a display in a "visualization mode" that visualizes information that is not visually recognized in the scenery ahead.

The notification image in the emphasized mode is displayed at a position corresponding to a real object existing in front of the vehicle 1, such as a preceding vehicle, a pedestrian, a road sign, and a building. The emphasized notification image is, for example, superimposed on the real object or displayed in the vicinity of the real object, and emphasizes the existence of the real object and notifies the user. In other words, the “position corresponding to the actual object” as the display position of the notification image in the emphasized mode is not limited to the position where it is visually superimposed on the actual object, but may be a position in the vicinity of the actual object. Note that the mode of emphasis is arbitrary as long as it is a mode that does not hinder the visual recognition of the real object.

The notification image in the visualization mode is, for example, an image simulating a road sign for notifying various information at a place where no road sign is installed, or an image that the driver 4 presets in the car navigation device 80 or the like. It is an image showing a POI (Point of Interest) such as a destination or a facility. Note that the road signs here include all kinds of signs, such as guide signs, warning signs, regulatory signs, and instruction signs. In other words, the notification image that can be displayed in this embodiment includes an image simulating a road sign. Here, an example of a notification image in a visualization mode imitating a road sign is shown in FIGS. 4(b) and 4(c). The notification image shown in FIG. 4B is an image imitating a guide sign. The notification image shown in FIG. 4(c) is an image imitating a regulation sign indicating a speed limit.

The control unit 31 controls the display size of the notification image to decrease as the distance X from the vehicle 1 to the forward target increases (in other words, the display size of the notification image increases as the distance X decreases). Controls image display. This makes it possible to give the notification image a pseudo perspective.

The control unit 31 acquires the distance X to the object ahead based on information transmitted from at least one of the surrounding information acquiring unit 40, the forward information acquiring unit 50, and the car navigation device 80 (hereinafter referred to as a specific configuration). . Note that the control unit 31 may acquire information indicating the distance X from a specific configuration, or may acquire coordinate information of a forward object from a specific configuration and calculate the distance X based on the coordinate information. In particular, the distance X to the forward target, which is the notification target of the notification image in the visualization mode, is the representative position of the forward target specified based on the information from the surrounding information acquisition unit 40 or the car navigation device 80 (for example, the start of the restricted section position, or a position predetermined as a position suitable for road guidance).

The control unit 31 determines the display ratio Y based on the obtainable distance X and the reference control information BD as described above, and displays the notification image at the determined display ratio Y. FIG. The reference control information BD is data of a function F (numerical expression) that indicates the relationship between the distance X and the display ratio Y. FIG. The function F is, for example, an inverse proportion that can be represented by Y=α/X, as shown in FIG. 4(a). The constant α is determined by, for example, a predetermined distance P from the virtual plane to the eyebox E and a reference size, which will be described later.

The display ratio Y is the display ratio of the notification image to the reference size, and is represented by S/ _S0 , where S is the size to be displayed on the virtual surface and _S0 is the reference size. The standard size is stored in advance in the ROM 32 as the size of the notification image at the predetermined distance _X0 . It is arbitrary how the predetermined distance _X0 is set.

The standard size is predetermined according to the type of notification image. FIG. 4(b) shows a notification image of a standard size with a horizontal length of A and a vertical length of B as a rectangular notification image. FIG. 4(c) shows a circular notification image of a standard size with a horizontal length of C and a vertical length of D. FIG. Note that C=D if the notification image is a perfect circle, and C≠D if it is an ellipse. The length referred to here can be defined, for example, in pixels.

For example, when the display ratio Y determined by the control unit 31 is "2", the notification image shown in FIG. It will be displayed enlarged. Further, when the display ratio Y determined by the control unit 31 is "1/2", the notification image shown in FIG. It will be displayed after being reduced from the standard size. As described above, the enlargement or reduction of the notification image is performed while maintaining the aspect ratio of the standard size notification image. Note that the shape of the notification image is not limited to a rectangle or a circle, and may be other shapes such as triangles and polygons. The configuration of the notification image is arbitrary as long as it can notify information about the forward target, and may be, for example, characters, symbols, graphics, icons, or a combination thereof.

As shown in FIG. 1B, since the virtual image V is displayed two-dimensionally on the virtual plane, the notification image displayed within the display area of the virtual image V is also displayed two-dimensionally. For example, the virtual plane is set facing the driver 4 and the notification image is also displayed facing the driver 4 . By arranging the display surface of the display unit 20 (the display surface of the LCD or the screen in the case of using DMD or LCOS) with an inclination, the virtual plane is set to be inclined forward with respect to the vertical direction of the vehicle 1. You may Even in such a case, the notification image can be viewed directly facing the driver 4 by considering the projection from the obliquely set virtual plane to the front plane. Therefore, even when the virtual plane of the virtual image V is set obliquely, the concept of enlargement or reduction of the notification image visually recognized by the driver 4 is the same as described above.

As described above, the control unit 31 controls the display of the notification image. When there are multiple front targets, the control unit 31 can display multiple notification images (simultaneously) within the same period.
The notification image may be deleted when the display ratio of the notification image determined based on the reference control information BD reaches or exceeds the predetermined value TM. By doing this, when the vehicle 1 approaches a predetermined forward target, it is possible to avoid annoyance to the driver 4 due to excessively large display of the notification image for notifying the forward target. The predetermined value TM will be described later.

Here, among the plurality of notification images, the one that notifies the front object at the closest distance X from the vehicle 1 is the first image V1, and the one that notifies the front object at the distance X farther than the first image V1 is the second image. Two images V2 are used, and a third image V3 indicates a front object whose distance X is farther than that of the second image V2.
These plural notification images are displayed in descending order of display size by control based on the reference control information BD (hereinafter referred to as normal control), as shown in FIGS. 1 image V1, 2nd image V2, 3rd image V3. As the vehicle 1 travels forward, the distance X to the forward object gradually decreases, so the display sizes of the first image V1, the second image V2, and the third image V3 gradually increase. If this is explained using the function F shown in FIG. 5A, the points representing the first image V1, the second image V2, and the third image V3 on the curve represented by the function F move to the left. corresponds to
However, depending on the distance X, the second image V2 displayed after the first image V1 may become too small to be visually recognized only by the control by the normal control. In order to solve this problem, the control unit 31 executes notification image special control processing shown in FIG.

When a plurality of front objects are specified during normal control based on the reference control information BD, the control unit 31 performs display control of a plurality of notification images for notifying the front objects. Then, the control unit 31 repeatedly executes the notification image special control process at a predetermined control cycle during the period in which the plurality of notification images are displayed. That is, the notification image special control process is executed at least during the period in which both the first image V1 and the second image V2 are displayed.

(Announcement image special control processing)
First, the control unit 31 determines whether or not the display ratio of the first image V1 is equal to or greater than the first threshold value T1 (step S1). The first threshold value T1 is predetermined as a display ratio corresponding to the distance X when it is assumed that the forward object is sufficiently close to the vehicle 1 and that the forward object will soon pass, and is stored in the ROM 32. .

When the display ratio of the first image V1 is equal to or greater than the first threshold value T1 (step S1; Yes), the control section 31 executes the process of step S2. On the other hand, if the display ratio of the first image V1 is less than the first threshold value T1 (step S1; No), the control unit 31 controls the driver 4 to display the first image V1 based on the line-of-sight information acquired from the line-of-sight detection unit 60. is visually recognized (step S3).

When the driver 4 is not visually recognizing the first image V1 (step S3; No), the controller 31 terminates the notification image special control process, while the driver 4 is visually recognizing the first image V1. If so (step S3; Yes), the process of step S2 is executed. By doing so, even if the display ratio of the first image V1 is less than the first threshold value T1, it is assumed that the driver 4 recognizes the first image V1 and the importance of the first image V1 has already decreased. In this case, enlargement processing can be executed to ensure the visibility of the second image V2 as described later.

In step S2, the control unit 31 determines whether or not the conditions for erasing the first image V1 are satisfied. For example, when the display ratio of the first image V1 is equal to or greater than the predetermined value TM, the control unit 31 determines that the deletion condition is satisfied (step S2; Yes), and moves the first image V1 within the display range of the virtual image V. (step S4). The predetermined value TM is predetermined, for example, as a value greater than the first threshold value T1 and stored in the ROM 32 . After execution of step S4 or when the deletion condition for the first image V1 is not satisfied (step S2; No), the control unit 31 executes the process of step S5.

In step S5, the control unit 31 determines whether or not the display ratio of the second image V2 is less than the second threshold value T2. The second threshold T2 is a value smaller than the first threshold T1, and is the distance when it is assumed that the forward object is far from the vehicle 1 and the second image V2 showing the forward object is difficult for the driver 4 to visually recognize. A display ratio corresponding to X is determined in advance and stored in the ROM 32 .

If the display ratio of the second image V2 is equal to or greater than the second threshold value T2 (step S5; No), the control section 31 terminates the notification image special control process. In this case, the second image V2 is displayed according to the normal control based on the reference control information BD, that is, according to the function F.
On the other hand, when the display ratio of the second image V2 is less than the second threshold value T2 (step S5; Yes), the control unit 31 performs enlargement processing on the second image V2 (step S6).

In the enlargement process of step S6, the control unit 31 displays the second image V2 at a ratio larger than the display ratio R (see FIGS. 5B and 5C) determined based on the reference control information BD. In the enlargement process, the control unit 31 adjusts the function F (Y=α/X) based on the reference control information BD to a predetermined amount γ The display control of the second image V2 is performed using the special function Fs (Y=(α+γ)/X) increased by .

FIGS. 6(a) and 6(b) show an example of image transition when the enlargement process is executed. When the enlargement process is executed, the second image V2 displayed under normal control as shown in FIG. 6(a) is enlarged as shown in FIG. 6(b). By executing the enlargement process on the second image V2 in this way, the notification target of the first image V1 will soon pass, and the second image V2, which is assumed to have a higher importance of notification than the first image V1, is displayed. It can be easily visually recognized and displayed.

Further, when executing the enlargement process, the control unit 31 may display the second image V2 higher than before the execution when viewed from the driver 4 . In this way, the driver 4 can secure a field of view necessary for driving. Such control of moving and displaying the second image V2 upward along with the enlargement processing is particularly useful when the second image V2 is a notification image in a visualization form that resembles a road sign. This is because a normal road sign is displayed above the road surface ahead, so that the driver 4 can visually recognize the second image V2 as if he or she were looking at the actual road sign.

Further, when displaying the third image V3 that indicates a forward object whose distance X is farther than the second image V2, the control unit 31 is executing the enlargement process on the second image V2. However, the display of the third image V3 is continued under the control based on the reference control information BD. As a result, the second image V2 is relatively more conspicuous than the third image V3, and the second image V2 having a higher degree of notification importance than the third image V3 can be displayed effectively. . Note that if there are four or more notification images when the notification image special control process is executed, the notification image displayed smaller than the third image V3 (that is, the distance X is farther than the notification target of the third image V3) An image for notifying an object ahead) is also continued to be displayed under control based on the reference control information BD.

In addition, in step S1 or S2, the control unit 31 determines whether or not part of the first image V1 is out of the display area of the virtual image V, as shown in FIG. Enlargement processing may be executed if the image is present and if Yes is determined in step S5. Even in this way, it is possible to display the second image V2, which is assumed to have a higher importance of notification than the first image V1 because the notification target of the first image V1 will soon pass, to be displayed in an easily recognizable manner.

Subsequently, the control unit 31 determines whether or not a predetermined period of time (for example, several seconds) has passed since execution of the enlarging process was started (step S7). If the predetermined period has passed (step S7; Yes), the control unit 31 executes return processing for returning the display control of the second image V2 to the control based on the reference control information BD (step S8), and End special control processing.

FIGS. 6B and 6C show an example of image transition when the restoration process is executed. When the return process is executed, the second image V2 that has been enlarged and displayed as shown in FIG. 6(b) is displayed under control based on the reference control information BD as shown in FIG. 6(c).

Note that FIG. 6C shows an example in which the display ratio changes in the Y direction during execution of the enlargement processing and the return processing. may be changed. In this way, when the display ratio is gradually changed, the point (X, Y) indicating the second image V2 moves diagonally upward to the left with respect to the Y direction from the function F to the special function Fs when the enlargement process is executed. When the return process is executed, the function moves from the special function Fs to the function F diagonally downward to the left with respect to the Y direction.

If the predetermined period has not elapsed since the start of the enlargement process (step S7; No), the control unit 31, based on the line-of-sight information acquired from the line-of-sight detection unit 60, enlarges the second image V2 of the driver 4 being displayed. is visually recognized (step S9).

If the driver 4 is not viewing the second image V2 (step S9; No), the controller 31 terminates the notification image special control process while the driver 4 is viewing the second image V2. If so (step S9; Yes), the return process of step S8 is executed. By doing so, when it is assumed that the driver 4 recognizes the second image V2 being enlarged and displayed, the display control of the second image V2 can be returned to the control based on the reference control information BD. For the driver 4, it is possible to prevent the second image V2, whose importance has already decreased, from being continuously enlarged. The notification image special control processing has been described above.

In addition, this invention is not limited by the above embodiment and drawing. Modifications (including deletion of components) can be made as appropriate without changing the gist of the present invention. An example of modification will be described below.

In the enlargement processing described above, an example of controlling the second image V2 based on the special function Fs has been shown, but the present invention is not limited to this. For example, as shown in FIG. 7A, the display of the second image V2 may be controlled at a constant display ratio Y that does not depend on the distance X when the enlargement process is performed.
However, when the display of the second image V2 is controlled at a constant display ratio Y during execution of the enlargement process, ideally, as shown in FIG. is rarely located on the function F at the beginning of . Therefore, as shown in FIG. 7(b), from the start point of the return process, the return function Fb may be used to control the display of the second image V2. It can also be said that control using the return function Fb is an example of control of the display ratio of the second image V2 based on the reference control information BD. Further, as shown in FIG. 7C, even if the display ratio, which is a constant value until the start of the restoration process after the enlargement process is executed, is changed based on the function F from the start time. good. In these modified examples as well, the display ratio may be gradually changed during execution of the enlargement process and the return process, as described above.

As shown in FIG. 7A, when the second image V2 is controlled to be displayed at a constant display ratio Y that does not depend on the distance X during the enlargement process, the constant display ratio Y intersects the function F. The recovery process may be started at the point of time. In other words, the predetermined period from when the enlargement process is performed on the second image V2 to when the return process is started does not have to be a predetermined period.

In the above description, an example in which the special function Fs is obtained by increasing the proportionality constant of the function F and the return function Fb is obtained by decreasing the proportionality constant of the function F is shown, but the present invention is not limited to this. For example, the special function Fs may be obtained by translating the function F (Y=α/X) in the Y direction by +δs (δs is a constant) (Y=α/X+δs). Further, the return function Fb may be obtained by translating the function F (Y=α/X) in the Y direction by −δb (δb is a constant) (Y=α/X−δb).

In the above, an example in which the function F is inversely proportional has been shown, but by approximating the curve, the function F can be converted to a linear function with a constant slope (Y = αX + β ) may indicate a straight line.

Moreover, in the above description, the function F may be an algebraic function including a polynomial function or an elementary function that represents a curve. Also, the function F may be composed of a plurality of linear functions with different slopes. For example, as shown in FIG. 8(b), the function F may represent a curve in which the display ratio Y becomes larger than the direct proportional relationship as the distance X becomes larger. Further, as shown in FIG. 8B, the function F is represented by a plurality of straight lines, and the plurality of straight lines have a gentle slope when the distance is equal to or greater than a predetermined distance X compared to when the distance is less than X. may contain a straight line. Examples of the function F shown in FIGS. 8B and 8C are useful when it is difficult to see distant scenery during rain or thick fog, or when the notification image is difficult to see due to strong sunlight.

In the above description, the reference control information BD is data representing the formula of the function F. However, the reference control information BD is a table configured to be able to determine the display ratio Y corresponding to the acquired distance X. It may be data. In other words, if the reference control information BD can determine a display ratio Y that decreases as the distance X increases (in other words, a display ratio Y that increases as the distance X decreases) according to the obtained distance X, , may be data representing a formula or table data, and its configuration is arbitrary.

Further, in the above description, the control unit 31 determines that the deletion condition is satisfied when the display ratio of the first image V1 is equal to or greater than the predetermined value TM (step S2 of the notification image special control process shown in FIG. 3). ), but not limited to this. For example, when the object visually recognized by the driver 4 is the first image V1, the control unit 31 determines that the deletion condition is satisfied (step S2 of the notification image special control process shown in FIG. 3; Yes). You may

It goes without saying that part of the processing can be omitted, and predetermined processing can be changed or added in the notification image special control processing described above, as long as the second image V2 can be effectively displayed. . For example, at least one of step S3, steps S2 and S4, and step S9 may be omitted from the notification image special control process. Further, the process of step S5 may be omitted in the notification image special control process.

The projection target of the display light L is not limited to the windshield 3, and may be a combiner configured by a plate-like half mirror, a hologram element, or the like. Further, the display device that executes the notification image special control process described above is not limited to the HUD device 10 . The display device may be configured as a head mounted display (HMD) worn on the head of the driver 4 of the vehicle 1 . Then, in the image displayed as a virtual image by the HMD, display control of the notification image may be performed by the same method as described above. In other words, the display device is not limited to one mounted on the vehicle 1, and may be one used in the vehicle 1. FIG.

The display device (for example, the HUD device 10 ) described above displays a superimposed image that is viewed by the driver 4 (an example of a viewer) as a virtual image V that overlaps the scenery in front of the vehicle 1 .
The HUD device 10 has two acquisition means (for example, the control unit 31) that acquires the distance X from the vehicle 1 to the forward target in front of the vehicle 1, and a notification image that reports the forward target within the display area of the superimposed image. Display control means (for example, the control unit 31 or the display unit 20) that controls display in dimensions, and a display ratio Y of the notification image to a predetermined reference size that decreases as the distance X increases are acquired. Storage means (for example, ROM 32) for storing reference control information BD for determination according to the distance X obtained by the means.
The display control means can display a plurality of images including a first image V1 and a second image V2 for notifying an object in front of which the distance X is longer than the first image V1, as the notification image. Further, the display control means changes and controls the display ratio Y of each of the first image V1 and the second image V2 based on the reference control information BD. If the above is the case, it is determined whether or not the display ratio Y of the second image V2 is less than the second threshold T2, which is smaller than the first threshold T1. Then, when the display ratio Y of the second image V2 is equal to or greater than the second threshold value T2, the display control means controls the display ratio Y of the second image V2 based on the reference control information BD, and controls the display ratio Y of the second image V2. When the display ratio Y is less than the second threshold value T2, enlargement processing for displaying the second image V2 at a ratio larger than the display ratio R (see FIG. 5C) determined based on the reference control information BD. to run.
By performing the enlargement process on the second image V2 in this way, as described above, it is assumed that the notification target of the first image V1 will soon pass, and the importance of notification will be higher than that of the first image V1. It is possible to display the second image V2 in an easily visible manner. In other words, according to the display device described above, it is possible to effectively display the notification image which is assumed to have the highest priority among the plurality of notification images.

In the above description, descriptions of known technical matters have been omitted as appropriate to facilitate understanding of the present disclosure.

DESCRIPTION OF SYMBOLS 100... Display system for vehicles 10... HUD apparatus 20... Display part 30... Control apparatus 31... Control part (31a...CPU, 31b...GDC)
32... ROM, 33... RAM
40... Peripheral information acquisition unit 50... Forward information acquisition unit 60... Line of sight detection unit 70... ECU
80... Car navigation system BD... Reference control information F... Function, Fs... Special function, Fb... Return function T1... First threshold value, T2... Second threshold value, TM... Predetermined value V... Virtual image V1... First image, V2... Second image, V3... Third image 1... Vehicle, 2... Dashboard, 3... Windshield, 4... Driver E... Eye box, L... Display light

Claims (8)

A display device that displays a superimposed image that is visually recognized by a viewer as a virtual image that overlaps the scenery in front of the vehicle,
Acquisition means for acquiring a distance from the vehicle to a front object in front of the vehicle;
display control means for two-dimensionally controlling the display of a notification image that notifies a forward target within the display area of the superimposed image;
Storage means for storing reference control information for determining, in accordance with the distance acquired by the acquisition means, a display ratio of the notification image with respect to a predetermined reference size that decreases as the distance increases. and
The display control means is
As the notification image, a plurality of images including a first image and a second image for notifying a forward target having a longer distance than the first image can be displayed,
When the display ratio of the first image and the second image is changed and controlled based on the reference control information and the display ratio of the first image becomes equal to or greater than the first threshold, performing an enlargement process for displaying the two images at a ratio larger than the display ratio determined based on the reference control information;
display device. The display control means erases the first image when a display ratio of the first image, which has been changed and controlled based on the reference control information, reaches or exceeds a predetermined value larger than the first threshold .
The display device according to claim 1. The enlargement process is performed on condition that the display ratio of the second image is less than a second threshold that is smaller than the first threshold,
When the display ratio of the second image is equal to or greater than the second threshold and the enlargement process is not executed, the display control means controls the display ratio of the second image based on the reference control information.
The display device according to claim 1 or 2. The plurality of images includes a third image that reports a forward target having a longer distance than the second image,
The display control means controls the display ratio of the third image based on the reference control information even when executing the enlargement process.
4. The display device according to claim 1. When executing the enlargement process, the display control means displays the second image higher than before the execution when viewed from the viewer.
The display device according to any one of claims 1 to 4. The display control means restores the display ratio of the second image to the display ratio determined based on the reference control information after a predetermined period of time has elapsed since the enlargement process was executed.
The display device according to any one of claims 1 to 5. When the display control means specifies that the viewer has viewed the second image on which the enlargement processing has been performed based on the line-of-sight information indicating the line-of-sight direction of the viewer acquired from the line-of-sight detection means, 2 images are returned to the display ratio determined based on the reference control information;
The display device according to any one of claims 1 to 6. The notification image that can be displayed by the display control means includes an image simulating a road sign,
The display device according to any one of claims 1 to 7.

Images