WO2020022239A1

WO2020022239A1 - Display control device and head-up display device

Info

Publication number: WO2020022239A1
Application number: PCT/JP2019/028579
Authority: WO
Inventors: 貴生人川手
Original assignee: 日本精機株式会社
Priority date: 2018-07-24
Filing date: 2019-07-22
Publication date: 2020-01-30
Also published as: JPWO2020022239A1; JP7255596B2

Abstract

The present invention facilitates the recognition of a real object that has approached, and reduces image complexity. A first visibility adjustment unit 73 acquires the position of a real object 300, and gradually reduces the visibility of a relevant information image V1 as the distance to the real object 300 decreases, or as the real object 300 approaches the outer edge of a display region 11. A second visibility adjustment unit 74 acquires a gazing range GZ of a viewer, increases the visibility of the relevant information image V1 when it is determined that the relevant information image V1 was gazed at, and then gradually lowers the visibility on the basis of the travel distance of a vehicle or the elapsed time.

Description

Display control device, head-up display device

The present invention relates to a head-up display device used in a vehicle and superimposing a virtual image on a foreground of the vehicle for visual recognition, and a display control device used for the device.

In the head-up display device disclosed in Patent Document 1, when a real object existing in the foreground approaches, an emphasized image displayed for the real object is perceived as annoying. In addition, the visibility of the emphasized image displayed on the real object is reduced (in Patent Literature 1, the image is darkened or reduced).

JP-A-2015-11666

However, by reducing the visibility of the image with respect to the approaching real object, the complexity of the image can be reduced, but the possibility of missing the approaching real object increases.

The present invention has been made in view of the above problems, and has as its object to make it easy to recognize an approaching real object while reducing the complexity of an image.

A first aspect of the display control device is a display control device which is mounted on a vehicle and displays a related information image of a real object present in the foreground in a display area overlapping the foreground of the vehicle, Acquiring a position, as the distance to the real object is reduced, or as the real object approaches the outer edge of the display area, a first visibility adjustment unit that gradually reduces the visibility of the related information image, Obtain the gaze range of the viewer, when it is determined that the related information image has been watched, after increasing the visibility of the related information image, based on the travel distance of the vehicle, or elapsed time, gradually And a second visibility adjusting unit for lowering the visibility.

The real object approaching the vehicle has a relatively low importance for the viewer (the driver of the vehicle) because the risk of collision has been dealt with or the object is not of interest. Or, the degree of importance has been reduced). According to the first aspect, the visibility of the related information image decreases as the real object approaches the vehicle. Therefore, for the viewer (the driver of the vehicle), the real object which has avoided the danger of the collision and the real object which is not of interest. The visual attention to the related information image of the real object can be made difficult to direct, but if the viewer is interested in the real object approaching the vehicle or the related information image of this real object, Since the visibility of the related information image is temporarily increased based on gazing at the related information image, information on the real object of interest is easily recognized. Then, the visibility of the related information image temporarily increases, and then gradually decreases based on the traveling distance of the vehicle or the elapsed time. This can contribute to safe driving of the vehicle.

FIG. 1 is a block diagram functionally showing a configuration of a head-up display device according to an embodiment of the present invention. It is a figure showing the example of the virtual image which the head up display device of the above-mentioned embodiment displays. It is a flowchart which shows the 1st visibility adjustment process which the head-up display device of the said embodiment performs. It is a flowchart which shows the 2nd visibility adjustment process which the head-up display device of the said embodiment performs. FIG. 4 is a diagram illustrating an example of a virtual image when the first visibility adjustment process illustrated in FIG. 3 is performed. FIG. 5 is a diagram illustrating an example of a virtual image when the second visibility adjustment process illustrated in FIG. 4 is performed. FIG. 4 is a diagram illustrating a characteristic of visibility of a related information image with respect to a distance in the first visibility adjustment processing illustrated in FIG. 3. It is a figure which shows the characteristic of the visibility of the related information image with respect to time and travel distance when a 2nd visibility adjustment process is performed before a 1st visibility adjustment process is performed. It is a figure which shows the characteristic of the visibility of the related information image with respect to time and travel distance when a 2nd visibility adjustment process is performed after a 1st visibility adjustment process is performed. It is a figure which shows the modification of the characteristic of the visibility of the related information image with respect to time when a 2nd visibility adjustment process is performed before a 1st visibility adjustment process is performed. It is a figure which shows the modification of the characteristic of the visibility of the related information image with respect to the time when a 2nd visibility adjustment process is performed after a 1st visibility adjustment process is performed.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. The present invention is not limited by the following embodiments (including the contents of the drawings). Modifications (including deletion of components) can be made to the following embodiments. In the following description, well-known technical items are not described in order to facilitate understanding of the present invention.

FIG. 1 is a block diagram showing an overall configuration of a head-up display device (hereinafter, referred to as a HUD device 1) in the present embodiment. The HUD device 1 includes an image display unit 10 that displays a display image that is a source of a virtual image V (see FIG. 2), and a display unit 2 (for example, a front of a vehicle) that displays display light of the display image displayed by the image display unit 10. (A windshield), a relay optical unit 10a composed of one or a plurality of optical members such as a reflective optical system, a refractive optical system, and a diffractive optical system for appropriately expanding and projecting, and display control for controlling the display of the image display unit 10. And an apparatus 20.

For example, the image display unit 10 displays a display image by receiving a projector (not shown) using a reflective display device such as DMD or LCoS, and receives display light from the projector. (Not shown) and a screen (not shown) that emits light toward the relay optical unit 10a. The image display unit 10 generates a virtual image in front of the viewer by displaying a display image on a screen based on image data input from the image generation unit 80 (display control device 20). The image display unit 10 may be, for example, a transmissive display panel such as a liquid crystal display element, a self-luminous display panel such as an organic EL element, or a scanning display device that scans a laser beam. When the display unit 10 is configured by a projector such as a reflective display device or a scanning display device, a screen can be said to be a display surface of the image display unit 10. On the other hand, when the image display unit 10 includes a transmissive display panel or a self-luminous display panel, the display area of the panel can be said to be the display surface of the image display unit 10.

The display control device 20 replaces the first interface (the identification information acquisition unit 30, the position information acquisition unit 32, the distance information acquisition unit 34, the gaze position acquisition unit 40, and the vehicle state acquisition unit 50) with the first interface. Alternatively, together with the first interface, at least the second interface 60 for acquiring various information necessary for the processing of the display processing unit 70 and the display control of a related information image V1 (a type of virtual image V) described later are executed. A display processing unit 70.

The first interface (identification information acquisition unit 30, position information acquisition unit 32, distance information acquisition unit 34, gaze position acquisition unit 40, vehicle state acquisition unit 50), and second interface 60 can communicate with external devices. It is connected to. The external devices include a real object identification unit 31, a real object position detection unit 33, a real object distance detection unit 35, a line of sight detection unit 41, a position detection unit 51, a direction detection unit 52, a posture detection unit 53, and a cloud server (external server). ) 500, vehicle ECU 600, navigation system 700, other in-vehicle devices in the own vehicle (not shown), mobile devices in the own vehicle, other vehicles (vehicle-to-vehicle communication V2V), communication infrastructure on the road (road-to-vehicle communication V2I), walking Portable device (communication V2P between a vehicle and a pedestrian). The HUD device 1 acquires various types of information and image element data that is the source of the virtual image V from an external device via an interface, and displays the virtual image V according to the processing of the display processing unit 70. Note that part or all of the image element data is stored in a storage unit 71 described later, and an image generation unit 80 described later reads out the image element data stored in the storage unit 71 according to an instruction from the display processing unit 70, A mode in which image data for displaying a display image on which the virtual image V is based on the image display unit 10 may be generated.

The identification information acquisition unit 30 is an input interface for acquiring identification information of the real object 300 existing in the foreground 200 of the own vehicle. For example, the real information identification unit includes one or more cameras and sensors mounted on the own vehicle. The identification information is acquired from the unit 31 and output to the display processing unit 70. For example, the real object identifying unit 31 captures (detects) the foreground 200 of the own vehicle, analyzes the captured (detected) data, and generates identification information of the real object 300 of the foreground 200 of the own vehicle (not shown). Having. For example, the identification information acquired by the identification information acquisition unit 30 is the type of the real object 300. The type of the real object 300 is, for example, an obstacle (another vehicle, a person, an animal, a thing), a road sign, a road, a building, or the like, but is not limited thereto as long as it exists in the foreground 200 and can be identified. .

The position information acquisition unit 32 is an input interface for acquiring the position of the real object 300 present in the foreground 200 of the own vehicle, and for example, detects the position of a real object including one or more cameras and sensors mounted on the own vehicle. The position of the real object 300 is acquired from the unit 33 and output to the display processing unit 70. For example, the real object position detection unit 33 captures (detects) the foreground 200 of the own vehicle, analyzes the captured (detected) data, and generates a position analysis unit (not shown) that generates position information of the real object 300 of the foreground 200. Have. In this case, the position information of the real object 300 acquired by the position information acquisition unit 32 includes a two-dimensional (up and down direction and left and right direction) position viewed from the viewer, for example, The real coordinates or the coordinates of the real object 300 in the display area 11.

The distance information acquisition unit 34 is an input interface that acquires the distance to the real object 300 existing in the foreground 200 of the own vehicle, and is, for example, a real object distance including one or more cameras and sensors mounted on the own vehicle. The distance from the detection unit 35 to the real object 300 is obtained and output to the display processing unit 70. For example, the real object distance detection unit 35 has a distance analysis unit (not shown) that captures (detects) the foreground 200 of the vehicle and analyzes the captured (detected) data to generate distance information to the real object 300. In this case, the distance information of the real object 300 acquired by the distance information acquisition unit 34 is the distance from the host vehicle or the HUD device 1 to the real object 300.

The above-described analysis unit (identification unit, position analysis unit, distance analysis unit) may be provided in the HUD device 1. Specifically, the analysis unit may be provided in the display processing unit 70. In this case, the identification information acquired by the identification information acquisition unit 30, the position information acquired by the position information acquisition unit 32, the distance information acquisition The distance information acquired by the unit 34 is imaging (detection) data obtained by imaging (detecting) the foreground 200 by the real object identification unit 31 (the real object position detection unit 33 and the real object distance detection unit 35). Note that a part or all of the identification information acquisition unit 30, the position information acquisition unit 32, and the distance information acquisition unit 34 may be common to the camera, the sensor, and the analysis unit. In the following, the identification information, the position information, and the distance information of the real object 300 are also collectively referred to as real object information.

In another embodiment, instead of the identification information acquisition unit 30 (or / and the position information acquisition unit 32 and / or the distance information acquisition unit 34), or the identification information acquisition unit 30 (or / and / or the position information acquisition unit 32). Alternatively, the second interface 60 may function as the identification information acquisition unit 30 (the position information acquisition unit 32, the distance information acquisition unit 34) together with the / and the distance information acquisition unit 34). In other words, the second interface 60 includes the cloud server 500, the vehicle ECU 600, the navigation system 700, and other in-vehicle devices in the own vehicle (not shown), mobile devices in the own vehicle, other vehicles (inter-vehicle communication V2V), on the road. The real object information may be acquired from a communication infrastructure (road-to-vehicle communication V2I) or a portable device of a pedestrian (communication V2P between a vehicle and a pedestrian). The second interface 60 is, for example, from the cloud server 500, the vehicle ECU 600, and the navigation system 700, along with map information, position information, identification information, and coordinate information (distance information) of the real object 300 such as a road sign, a road, or a building. And other vehicles (not shown) (vehicle-to-vehicle communication V2V), road communication infrastructure (road-to-vehicle communication V2I), and pedestrian portable devices (vehicle-to-pedestrian communication V2P). (Other vehicles, persons, animals, objects), road signs, road and other positional information, identification information, coordinate information (distance information), and the like can be obtained. The display processing unit 70 outputs the position information of the own vehicle or the HUD device 1 (the direction information of the own vehicle may be added) to the external device via the second interface 60, and the external device Based on the input position information (additional direction information) of the own vehicle or the HUD device 1, the position information, the identification information, and the distance information of the real object 300 existing in the foreground 200 of the own vehicle are output to the second interface 60. You may. Note that the identification information may be individual identification information finer than the type of the real object 300.

The gaze position acquisition unit 40 is an input interface for acquiring gaze information on a range (gaze range GZ) that the viewer is gazing at. For example, the gaze position acquisition unit 40 receives a gaze range GZ from a gaze detection unit 41 including a camera mounted on a vehicle. And outputs it to the display processing unit 70. The gaze range GZ of the viewer is an area in which the viewer is watching (gazing), including the eye viewpoint (gazing point), and is a certain range in which a person can normally gaze. The method of setting the gaze range GZ of the viewer is not limited to this, and the line-of-sight detection unit 41 may use a predetermined time (for example, as described in JP-A-2015-126451). (1 second), a rectangular area including a plurality of eye viewpoints (gaze points) may be set in the gaze range GZ.

The vehicle state acquisition unit 50 acquires information on the state (position, direction, posture) of the vehicle. The vehicle state acquisition unit 50 acquires the position information of the vehicle or the HUD device 1 detected by the position detection unit 51 such as a GNSS (Global Navigation Satellite System), and is detected by the direction detection unit 52 including a direction sensor. Obtain direction information indicating the direction of the vehicle or the HUD device 1 and obtain posture information indicating the height of the vehicle from the road surface and / or the angle with respect to the road surface detected by the posture detection unit 53 including a height sensor and an acceleration sensor. , To the display processing unit 70.

The display processor 70 includes at least one processor (eg, a central processing unit (CPU)), at least one application specific integrated circuit (ASIC), and at least one application specific integrated circuit (ASIC), and / or It includes at least one semiconductor integrated circuit such as at least one field programmable gate array (FPGA). The at least one processor reads one or a plurality of instructions from at least one computer-readable and tangible recording medium to thereby provide a notification necessity determination unit 72, a first visibility adjustment unit 73, and a second visibility. All or a part of the functions of the display processing unit 70 including the adjustment unit 74 can be executed. Such recording media include any type of magnetic medium, such as a hard disk, any type of optical medium, such as CDs and DVDs, any type of semiconductor memory, such as volatile memory, and non-volatile memory. . Volatile memory includes DRAM and SRAM, and non-volatile memory includes ROM and NVROM. A semiconductor memory is also a semiconductor circuit that becomes part of a circuit with at least one processor. The ASIC is an integrated circuit that is customized to execute all or a part of the functional blocks shown in FIG. 1, and the FPGA is designed to be able to execute all or a part of the functional blocks shown in FIG. 1 after manufacturing. Integrated circuit. The display processing unit 70 may include an image generation unit 80 that generates image data for displaying a display image (an image that is a source of the virtual image V) on the image display unit 10. Some or all of the functions of the 80 may be provided separately from the display control device 20 or the HUD device 1.

The notification necessity determination unit 72 determines the notification necessity of the real object 300 based on information acquired from the identification information acquisition unit 30, the position information acquisition unit 32, the distance information acquisition unit 34, and notifies the viewer of the necessity. Determine whether or not. When the notification need becomes higher than a predetermined threshold value, it is determined that it is necessary to notify the viewer. The notification necessity is, for example, a danger derived from the degree of a serious situation of the real object 300 or an urgency derived from the length of a reaction time required to perform a reaction action. The information acquisition unit 30, the position information acquisition unit 32, and the distance information acquisition unit 34 may determine only the real object information related to the real object 300, and in addition to the real object information, various types input from another interface. It may be determined based on information. Note that the notification necessity determination unit 72 essentially has only to have a function of determining whether or not to notify the viewer of the existence of the real object 300 based on the notification necessity. May not be provided, and a part or all of the function of determining the notification necessity may be provided separately from the display control device 20. The display processing unit 70 instructs the image generation unit 80 to display at least the related information image V1 (in addition to the emphasized image V2) for the real object 300 for which the notification necessity determination unit 72 has determined that notification is necessary. I do. The necessity of notification is not limited to the method using the risk or the urgency. For example, if information on the real object 300 useful to the viewer is acquired from the cloud server 500, the vehicle ECU 600, and the navigation system 700, the notification necessity determination unit 72 may determine that notification is necessary. In addition, the real object 300 does not need to be seen by the viewer, and the display processing unit 70 needs to be notified even when the real object 300 is hidden behind other objects in the foreground 200 or is too far away to be visually recognized. Even if the image generation unit 80 is instructed to display the related information image V1 for the real object 300 at a predetermined position or to display the emphasized image V2 in the direction of the real object 300 that needs to be notified. Good.

The first visibility adjustment unit 73 determines the presence or absence of the relevant information of the real object 300 for which the notification necessity determination unit 72 has determined that the notification is necessary. The image generation unit 80 is instructed to display. The first visibility adjustment unit 73 acquires the identification information of the real object 300 from the identification information acquisition unit 30 or the second interface 60, and the related information corresponding to the identification information is stored in the storage unit 71 or the second interface 60. If the information is in the external device via the interface, it is determined that there is related information.

The first visibility adjuster 73 adjusts the visibility of the virtual image V (the related information image V1 and the emphasized image V2). As a method of changing the “visibility” here, for example, changing the brightness, lightness, color, or a combination thereof of the virtual image V is used. The first visibility adjustment unit 73 changes the visibility of the related information image V1 based on the distance information of the real object 300 acquired from the real object distance detection unit 35. Specifically, the first visibility adjustment unit 73 gradually reduces the visibility of the related information image V1 as the real object 300 approaches.

As a modification, the first visibility adjustment unit 73 adjusts the visibility of the related information image V1 by the position of the real object 300 in the display area 11 (or the position of the emphasized image V2 displayed on the real object 300). It may be changed. Specifically, the first visibility adjuster 73 specifies the position of the real object 300 in the display area 11 based on the position information of the real object 300 acquired from the real object position detector 33, and May gradually decrease the visibility of the related information image V <b> 1 as approaches the outer edge of the display area 11. As described above, the first visibility adjustment unit 73 performs the visibility based on the distance information (the depth direction viewed from the viewer) or the position information (the vertical direction and / or the left / right direction viewed from the viewer) of the real object 300. Is referred to as first visibility adjustment processing.

The second visibility adjustment unit 74 determines whether the viewer is visually recognizing the real object 300 (or the emphasized image V2 displayed on the real object 300) or the related information image V1. I do. The second visibility adjustment unit 74 acquires the position information of the currently displayed related information image V1 and the emphasized image V2 stored in the storage unit 71, and acquires the position information of the real object 300 from the position information acquisition unit 32. Acquired, the gaze range GZ is acquired from the gaze position acquisition unit 40, and the determination is made based on the position information of the related information image V1 or the emphasized image V2 and the gaze range GZ.

In addition, the second visibility adjuster 74 determines whether the viewer visually recognizes the real object 300 (or the emphasized image V2 displayed on the real object 300) or the related information image V1. After increasing the visibility, it is possible to execute a second visibility adjustment process for decreasing the visibility. The second visibility adjuster 74 determines that the viewer does not visually recognize the real object 300 (and the emphasized image V2 displayed on the real object 300) and does not visually recognize the related information image V1. When the first visibility adjustment process is executed and the viewer visually recognizes the real object 300 (or the emphasized image V2 displayed on the real object 300) or the related information image V1, 2 Execute visibility adjustment processing.

Next, the virtual image V displayed by the HUD device 1 will be described with reference to FIG. In the example of FIG. 2, the real object 300 is a parking lot, the emphasized image V2 is displayed so as to indicate the parking lot, and the related information image V1 displaying “P” indicating the parking lot is displayed in the display area 11. It is displayed on the upper end portion 12. The display processing unit 70 displays the emphasized image V2 in the vicinity of the real object 300 for which the notification necessity determination unit 72 has determined that notification is necessary or at a position where the real object 300 is not visually recognized but exists, and the related information image of the real object 300 is displayed. V1 is displayed at a predetermined specific position in the display area 11 (the upper end 12 in the example of FIG. 2). The specific position is preferably the upper end portion 12 or the lower end portion 13 when the vertical direction of the display area 11 in which the HUD 10 can display the virtual image V is divided into four equal parts, but the vertical direction of the display area 11 is divided into three equal parts. The upper end portion 12 or the lower end portion 13 may be used. In other words, the display region 11 has a vertical direction that is about twice as large as the vertical width of the upper end portion 12 displaying the related information image V1 (the length between the upper end of the display region 11 and the lower end of the related information image V1). Is preferably provided below the upper end portion 12, or the width of the lower end portion 13 for displaying the related information image V1 in the vertical direction (between the lower end of the display region 11 and the upper end of the related information image V1). It is preferable to provide a region having a width in the vertical direction which is about twice as large as the length between them (the length between them).

Next, FIG. 3 is referred to. FIG. 3 is a flowchart illustrating an example of a main procedure of a first visibility adjustment process related to display control of the HUD device 1 of the present embodiment. When the HUD device 1 is started, a first visibility adjustment process is started.

First, real object information (identification information, position information, distance information) is obtained (step S1), and it is determined whether the real object 300 is to be notified to the viewer (step S2). (Step S3), and in the case of NO, the first visibility adjustment processing ends.

Subsequently, it is determined whether the display condition of the related information image V1 is satisfied (step S4). If YES, the related information image V1 is displayed (step S5). If NO, the related information image V1 is not displayed. Or, if the related information image V1 is displayed, it is not displayed (step S6), and the first visibility adjustment processing is ended. The display condition of the related information image V1 in step S4 includes at least the presence of the related information for the real object 300, and in addition, it can be determined that the priority (necessity) of the related information image V1 is high according to the situation. May be included.

Thereafter, it is determined whether a condition for executing the first visibility adjustment processing is satisfied (step S7), and in the case of YES, the visibility of the related information image V1 is determined based on the distance information of the real object 300 acquired in step S1 or the like. It is lowered based on the position information (step S8, also referred to as first visibility adjustment processing), and in the case of NO, the first visibility adjustment processing ends. The condition for executing the first visibility adjustment process in step S7 is that the real object 300 is within a predetermined distance Dm (see FIG. 7), which will be described later, or that the real object 300 is close to the outer edge of the display area 11. It is within a defined area (not shown). In the first visibility adjustment processing in step S8, the visibility of the related information image V1 is gradually reduced as the real object 300 approaches or the real object 300 approaches the outer edge of the display area 11. Automatically lowering the visibility of the related information of the real object 300 having a lower notification priority (necessity) when approaching the vehicle, making it harder for a viewer to lose visual attention, and concentrating on vehicle operation Can be.

(4) Referring to FIG. FIG. 4 is a flowchart illustrating an example of a main procedure of a second visibility adjustment process related to display control of the HUD device 1 according to the present embodiment. When the first visibility adjustment process ends, the second visibility adjustment process starts. When the second visibility adjustment process ends, the process is restarted from the first visibility adjustment process.

First, the gaze range GZ of the viewer is obtained (step S11), and it is determined whether the related information image V1 is being watched (step S12). If YES, the process proceeds to step S13. If NO, the second visibility is determined. The adjustment process ends.

Subsequently, in step S13, it is determined whether the related information image V1 determined to be watched in step S12 is undergoing the first visibility adjustment process. In other words, it is determined in step S8 whether the condition of the first visibility adjustment processing is satisfied. In the case of YES in step S13, the current visibility of the related information image V1 is obtained (step S14), and the visibility is increased by a predetermined value based on the current visibility (step S15). The visibility is reduced (step S16), the second visibility adjustment process is terminated, and the routine is started again from the first visibility adjustment process. In the case of NO in step S13, the visibility is increased by a predetermined value based on the normal visibility of the related information image V1 (step S15), and thereafter, the visibility is reduced (step S16), and the second visibility is performed. The gender adjustment processing is terminated, and the routine is started again from the first visibility adjustment processing. The processing from step S13 to step S16 is referred to as second visibility adjustment processing. That is, when the viewer visually recognizes the related information image V1, the first visibility adjustment process in which the visibility gradually decreases is interrupted, and the visibility increases, so that the related information image V1 being viewed is gradually recognized. It is possible to avoid a situation where the related information image V1 becomes difficult, and it is possible to easily recognize the related information image V1.

Next, the first visibility adjustment process and the second visibility adjustment process will be described with reference to FIGS. FIG. 5 shows the foreground 200, which is visually recognized via the projection target portion 2 (the front windshield of the vehicle) when the viewer faces forward, and the case where the first visibility adjustment processing in the present embodiment is executed. 5 is a diagram showing a virtual image V of FIG.

In FIG. 5A, since the real object 300 is located far away (away from the predetermined distance Dm), the result of step S7 in FIG. 3 is NO, and the first visibility adjustment processing in step S8 is executed. The related information image V1 that is not present and the rectangular emphasized image V2 near the real object 300 are displayed. At this time, the visibility of the related information image V1 is an initial value Lm (see FIG. 7) described later.

In FIG. 5B, since the real object 300 approaches (becomes closer than the predetermined distance Dm) as the vehicle travels, the result of step S7 in FIG. 3 is YES, and the first visibility adjustment processing in step S8. Are displayed, and a rectangular emphasized image V2 is displayed near the real object 300. At this time, the visibility of the related information image V1 is set to be lower than the initial value Lm.

In FIG. 5C, since the real object 300 is located in the vicinity (approached from the predetermined distance Ds), the related information image V1 is subjected to the first visibility adjustment processing in step S8, and the visibility is reduced. Since the real object 300 is outside the display area 11 when viewed from the viewer, the visibility of the emphasized image V2 is set to zero (non-display). That is, in the example of the first visibility adjustment processing illustrated in FIG. 5, the visibility of the related information image V <b> 1 is reduced as the real object 300 approaches. In the example of FIG. 5, the visibility of the emphasized image V2 is also reduced as the distance of the real object 300 decreases (or as the real object 300 approaches the outer edge of the display area 11), but is not limited thereto. Not done.

FIG. 6 shows the foreground 200, which is visually recognized through the projection target (windshield of the vehicle) 2 when the viewer faces forward, and the case where the second visibility adjustment processing in this embodiment is executed. It is a figure which shows a virtual image V.

In FIG. 6A and FIG. 6B, since the real object 300 is located at a far distance (away from the predetermined distance Dm), NO is obtained in step S7 in FIG. 3 and the first visual recognition is performed in step S8. A related information image V1 for which the gender adjustment processing has not been performed and a rectangular emphasized image V2 near the real object 300 are displayed. In FIG. 6B, since the related information image V1 is visually recognized by the viewer (the related information image V1 is included in the gazing range GZ), the result is YES in step S12 in FIG. 4, and the visibility is increased in step S15. Is displayed, and a rectangular emphasized image V2 is displayed near the real object 300. At this time, the visibility of the related information image V1 is set to be increased by a predetermined value based on the visibility if the related information image V1 is undergoing the first visibility adjustment processing. Is not in the first visibility adjustment process, the value is increased by a predetermined value based on the normal visibility (initial value Lm). 6C, the related information image V1 in which the visibility has been reduced in step S16 in FIG. 4 and the rectangular emphasized image V2 near the real object 300 are displayed. That is, in the example of the second visibility adjustment processing illustrated in FIG. 6, the related information image V1 is highlighted when the related information image V1 is visually recognized, and thereafter the visibility is reduced.

FIG. 7 is a characteristic diagram showing the relationship between the distance to the real object 300 and the visibility of the related information image V1 in the first visibility adjustment processing of the present embodiment. If the distance to the real object 300 is equal to or greater than the predetermined first distance threshold Dm, the determination in step S7 of FIG. 3 is NO, and the related information image V1 is maintained at the initial visibility Lm, and the first distance If the distance is less than the threshold value Dm, the result of step S7 in FIG. 3 is YES, and the process proceeds to step S8. As the distance becomes shorter, the visibility is gradually reduced. Display) (first visibility adjustment processing is executed). The relationship between the distance and the visibility between the first distance threshold Dm and the second distance threshold Ds is linear in the example of FIG. 7, but may be non-linear, and the visibility is gradually increased according to the distance. May be changed.

FIG. 8 is a time chart illustrating a change in the visibility of the related information image V1 due to the second visibility adjustment processing. FIG. 8A is a diagram illustrating the distance D1 in FIG. 7 before the first visibility adjustment processing is started. 8B is a time chart of the visibility of the related information image V1 by the second visibility adjustment processing when the viewer gazes at the related information image V1 at the time (time t10), and FIG. 8B shows the first visibility adjustment. 8 is a time chart of the visibility of the related information image V1 by the second visibility adjustment process when the viewer gazes at the related information image V1 at the distance D2 (time t20) in FIG. 7 after the process is started. .

In the embodiment illustrated in FIG. 8A, when the related information image V1 is gazed, it is gradually applied over a certain first period Ta (for example, 1 second) based on the visibility at that time (initial visibility Lm). The visibility is increased by a predetermined value, and then the visibility is kept constant until the vehicle travels a predetermined first traveling distance La. Thereafter, the vehicle further travels a second traveling distance Lb. By the time, the visibility is gradually reduced so that the visibility becomes 0 (zero). By gradually increasing the visibility of the related information image V1, the visual attention of the viewer can be easily maintained or redirected to the related information image V1 whose visibility has been increased.

In the embodiment illustrated in FIG. 8B, when the related information image V1 is gazed at, the visibility at that time (the visibility L2 adjusted corresponding to the distance D2 by the first display adjustment processing illustrated in FIG. 7) is used as a reference. The visibility is gradually increased by a predetermined constant value over a constant first period Ta, and thereafter, the visibility is maintained constant until the vehicle travels a constant first traveling distance La. The visibility is gradually reduced so that the visibility becomes 0 (zero) before the vehicle further travels the third traveling distance Lc. Note that the rate of decrease in visibility with respect to an increase in travel distance is the visibility when the second visibility adjustment process is performed (Lm at time t10 in FIG. 8A, L2 at time t20 in FIG. 8B (<Lm). ), It is preferable to be constant, but the present invention is not limited to this. For example, the lower the visibility may be when the second visibility adjustment process is performed, the more rapidly the reduction rate may be increased (unit). The rate of decrease in visibility per mileage may be increased).

FIG. 9 is a time chart illustrating a change in the visibility of the related information image V1 due to the second visibility adjustment process. FIG. 9A is a diagram illustrating the distance D1 in FIG. 7 before the first visibility adjustment process is started. Is a time chart of the visibility of the related information image V1 by the second visibility adjustment process when the viewer gazes at the related information image V1 at the time (time t30), and FIG. 9B is the first visibility adjustment. 8 is a time chart of the visibility of the related information image V1 by the second visibility adjustment process when the viewer gazes at the related information image V1 at the time of the distance D2 (time t40) in FIG. 7 after the process is started. .

In the embodiment illustrated in FIG. 9A, when the related information image V1 is gazed, a predetermined constant value is gradually set over a predetermined first period Ta based on the visibility at that time (initial visibility Lm). , The visibility is kept constant until a certain second period Tb elapses, and thereafter, the visibility becomes 0 (zero) until the third period Tc elapses. So that the visibility is gradually reduced.

In the embodiment illustrated in FIG. 9B, when the related information image V1 is gazed, the visibility at that time (the visibility L2 adjusted according to the distance D2 by the first display adjustment processing illustrated in FIG. 7) is used as a reference. The visibility is gradually increased by a predetermined constant value over a certain first period Ta, and thereafter, the visibility is kept constant until a certain second period Tb elapses. The visibility is gradually reduced so that the visibility becomes 0 (zero) before the period Td (<Tc) of 4 elapses. Note that the rate of decrease in visibility with respect to the passage of time is determined by the visibility when the second visibility adjustment processing is performed (Lm at time t30 in FIG. 9A, and L2 (<Lm) at time t40 in FIG. 9B). Instead, it is preferable, but not limited to, that the constant be constant. For example, the lower the visibility when the second visibility adjustment process is performed, the more the reduction rate may be increased (per unit time). May be increased. In the second visibility adjustment process, the visibility is not increased by a certain value from the visibility adjusted in the first display adjustment process over a certain first period Ta, but is changed to a specific visibility. (For example, normal visibility Lm).

DESCRIPTION OF SYMBOLS 1 ... HUD apparatus (head-up display apparatus), 2 ... windshield, 10 ... image display part, 10a ... relay optical part, 11 ... display area, 12 ... upper end part, 13 ... lower end part, 20 ... display control apparatus, 30 ... identification information acquisition unit, 31 ... real object identification unit, 32 ... position information acquisition unit, 33 ... real object position detection unit, 34 ... distance information acquisition unit, 35 ... real object distance detection unit, 40 ... gaze position acquisition unit, 41: line-of-sight detection unit, 50: vehicle state acquisition unit, 51: position detection unit, 52: direction detection unit, 53: attitude detection unit, 60: second interface, 70: display processing unit, 71: storage unit, 72 ... Notification necessity determination unit 73 73 First visibility adjustment unit 74 74 Second visibility adjustment unit 80 Image generation unit 200 Foreground 300 Real object 500 Cloud server 60 ... vehicle ECU, 700 ... navigation system, GZ ... gaze range, V ... virtual image, V1 ... related information image, V2 ... enhanced image

Claims

A display control device that is mounted on a vehicle and includes an image generation unit that generates a related information image of a real object existing in the foreground in a display area overlapping a foreground of the vehicle,
One or more processors,
Memory and
One or more instructions stored in the memory and configured to be executed by the one or more processors;
The one or more processors include:
Obtain the position of the real object,
As the distance to the real object decreases, or as the real object approaches the outer edge of the display area, gradually reduces the visibility of the related information image,
Obtain the gaze range of the viewer,
When it is determined that the related information image has been watched, after increasing the visibility of the related information image, based on the traveling distance of the vehicle, or the elapsed time, gradually reduce the command,
Display control device.
The one or more processors include:
If it is determined that the related information image has been watched, after gradually increasing the visibility of the related information image over a predetermined period, based on the travel distance, or the elapsed time, gradually decrease, Execute instructions,
The display control device according to claim 1.
The one or more processors include:
The second visibility adjustment unit, when it is determined that the related information image is gazed, when the related information image is lower than normal visibility, takes a predetermined period of time of the related information image Executing the instruction, gradually increasing the visibility to the normal visibility,
The display control device according to claim 1.
The one or more processors include:
The travel distance, or a decrease rate of visibility based on the elapsed time, regardless of the visibility of the related information image when it is determined that the related information image has been watched, to execute a command,
The display control device according to claim 1.
The one or more processors include:
At a position overlapping with the real object, or in the vicinity, an enhanced image that emphasizes the real object is further displayed,
As the distance to the real object decreases, or as the real object approaches the outer edge of the display area, gradually lowers the visibility of the enhanced image,
Even if it is determined that the emphasized image has been watched, do not adjust the visibility of the emphasized image, execute a command,
The display control device according to claim 1.
A display control device according to any one of claims 1 to 5,
An image display unit that displays an image that is a source of the related information image based on control of the display control device,
A head-up display device, comprising: a relay optical unit configured to project the image displayed by the image display unit toward a projection target unit to visually recognize a virtual image of the image.