JP2022077138A - Display controller, head-up display device, and display control method - Google Patents

Abstract

To provide a display controller which displays an image that is easy-to-realize so as to be parallel to a face of a travel lane, a head-up display device, and a display control method and so on.SOLUTION: A head-up display device, which causes an observer to visually recognize a virtual image of an image while overlaying the same on a foreground, executes: first display processing for displaying as a perspective virtual image of a first display mode having a first vanishing point VP1 that is located below a vanishing point VP0 of the foreground when viewed from the observer; and second display processing for displaying as a perspective virtual image of a second display mode having a second vanishing point VP2 that is located above the first vanishing point VP1 when viewed from the observer.SELECTED DRAWING: Figure 6

Description

The present disclosure is used in a moving body such as a vehicle, and is a display control device, a head-up display device, and a head-up display device that superimposes an image on the foreground of the moving body (the actual view in the forward direction of the moving body as seen from the occupant of the vehicle). Regarding display control method, etc.

In Patent Document 1, the display light projected on the projected portion such as the front windshield of the vehicle is reflected toward the occupant (observer) of the vehicle inside the vehicle, so that the observer can see the vehicle. A head-up display device (an example of a virtual image display device) for visually recognizing a virtual image overlapping the foreground of the light is described. In particular, the head-up display device described in Patent Document 1 virtually displays an object (here, the position is referred to as a target position) at a predetermined position in the depth in the foreground space or in the vertical and horizontal directions. (Imaginary image) is placed, and even if there is a change in the posture of the vehicle or a change in the eye position of the observer, the head-up is as if the display object exists at the target position in the foreground. Controls the image displayed inside the display device. That is, such a head-up display device forms an augmented reality in which a virtual object is added to and displayed in a real landscape (foreground).

In the example of FIG. 4 of Patent Document 1, an arrow shape indicating the front (in other words, extending from the rear to the front) is provided at a position slightly floating from the road surface on which the own vehicle is traveling (a position away from the road surface in the upward direction). A virtual object (in the shape of an arrow) is displayed. In such a case, the observer gradually narrows the width on the rear end side of the arrow (typically, the width in the left-right direction) toward the tip, as in the example of FIG. 5 of Patent Document 1. So, a sense of perspective is expressed. That is, it is perceived that the rear end of the arrow is near and the tip of the arrow is far away. As an expression that causes such a sense of perspective, a method such as a line perspective method (perspective drawing method) is typically used.

Japanese Unexamined Patent Publication No. 2010-156608

By the way, in a head-up display device as described above, a region in which a virtual image can be displayed is typically a plane or an aspect region (here, the region is referred to as a virtual image display region), and the virtual image is described. The display area is arranged so as to face the line of sight of the observer. That is, when viewed from the left-right direction of the own vehicle, the angle between the virtual image display area where the virtual image is displayed and the road surface on which the own vehicle travels is set to approximately 90 degrees (the virtual image display area is approximately the same as the road surface). Not arranged to be parallel).

FIG. 10 shows a foreground that is visually recognized by the observer through the front windshield WS of the own vehicle while the own vehicle is traveling, and a perspective image 1100 displayed in the virtual image display area 1000 that overlaps with the foreground. It is a figure which shows. FIG. 10 shows an aspect of a perspective image expressed as if it is arranged parallel to the road surface using the linear perspective method. The left side line of the traveling lane 1010 of the own vehicle is indicated by reference numeral 1011 and the right side line is indicated by reference numeral 1012. The intersection of the extension line of the left side line 1011 and the extension line of the right side line 1012 is defined as a vanishing point 1020. The perspective image 1100 of FIG. 10 is a trapezoid, and is represented by the line perspective method (one-point perspective method) so that the intersection of the extension line of the left side 1101 and the extension line of the right side 1102 of the trapezoid coincides with the vanishing point 1020. To. As a result, the perspective image 1100 is pictorially represented as if it were arranged parallel to the plane of the traveling lane 1010.

However, as described above, the virtual image display area is not arranged so as to be substantially parallel to the road surface, that is, in the virtual image display area that rises to the observer side by approximately 90 degrees with respect to the road surface on which the own vehicle travels. Even if the perspective image 1100 as shown in FIG. 10 is displayed, the distance between the far end 1110 and the traveling lane 1010 of the perspective image 1100 is longer than the distance between the near end 1120 and the traveling lane 1010, and the vehicle travels. With respect to the parallel plane of the lane 1010, the far end 1110 of the perspective image 1100 is perceived as rising to the observer side, and it becomes difficult to perceive that the perspective image 1100 is parallel to the plane of the traveling lane 1010. There was room.

A summary of the particular embodiments disclosed herein is presented below. It should be understood that these embodiments are presented solely to provide the reader with an overview of these particular embodiments and do not limit the scope of this disclosure. In fact, the present disclosure may include various aspects not described below.

The outline of the present disclosure relates to a display control device, a head-up display device, a display control method, and the like for displaying an image that is easily perceived as being parallel to a plane of a traveling lane.

Therefore, the display control device, the head-up display device, the display control method, and the like described in the present specification employ the following means in order to solve the above-mentioned problems. The present embodiment includes a first display process for displaying as a perspective virtual image of the first display mode having a first vanishing point arranged below the vanishing point of the foreground when viewed from the observer, and a first display process when viewed from the observer. (1) The gist is to execute a second display process of displaying as a perspective virtual image of a second display mode having a second vanishing point arranged above the vanishing point.

Therefore, the display control device described in the present specification is a display control that executes display control in a head-up display device that causes an observer to visually recognize an imaginary image of an image superimposed on the foreground by projecting an image onto a projected member. A device comprising one or more processors, a memory, and one or more computer programs stored in the memory and configured to be executed by the one or more processors. Is a first display process for displaying as a perspective imaginary image of the first display mode having a first vanishing point arranged below the vanishing point of the foreground when viewed from the observer, and a first vanishing point when viewed from the observer. The second display process of displaying as a perspective imaginary image of the second display mode having the second vanishing point arranged above is executed.

FIG. 1 is a diagram showing an example of application of a virtual image display system for a vehicle to a vehicle. FIG. 2 is a diagram showing a configuration of a head-up display device. FIG. 3 is a diagram conceptually showing a virtual object arranged at the target position of the actual scene and an image displayed in the virtual image display area so that the virtual object can be visually recognized at the target position of the actual scene. FIG. 4 is a diagram showing an example of a foreground visually recognized by an observer and a virtual image of a first display mode superimposed on the foreground while the own vehicle is traveling. FIG. 5A is a diagram showing a modified example of the perspective image. FIG. 5B is a diagram showing a modified example of the perspective image. FIG. 5C is a diagram showing a modified example of the perspective image. FIG. 5D is a diagram showing a modified example of the perspective image. FIG. 6 is a diagram showing an example of a foreground to be visually recognized by an observer and a virtual image of a second display mode superimposed on the foreground while the own vehicle is traveling. FIG. 7A is a diagram showing a perspective virtual image V of a second display mode having a second vanishing point. FIG. 7B is a diagram showing a perspective virtual image V of a second display mode having a second vanishing point. FIG. 8 is a block diagram of a vehicle virtual image display system of some embodiments. FIG. 9 is a flow chart showing a display control method for executing a display control process for displaying a perspective virtual image in the first display mode or the second display mode. FIG. 10 is a diagram relating to a conventional example, and is a diagram showing an example of a foreground visually recognized by an observer and a virtual image superimposed on the foreground while the own vehicle is traveling.

Hereinafter, FIGS. 1 to 8 provide a description of the configuration and operation of an exemplary vehicle virtual image display system. The present invention is not limited to the following embodiments (including the contents of the drawings). Of course, changes (including deletion of components) can be made to the following embodiments. Further, in the following description, in order to facilitate understanding of the present invention, description of known technical matters will be omitted as appropriate.

See FIG. FIG. 1 is a diagram showing an example of application of a virtual image display system for a vehicle to a vehicle. In FIG. 1, the left-right direction (in other words, the width direction of the vehicle 1) of the vehicle (an example of a moving body; hereinafter also referred to as the own vehicle) 1 is defined as the X-axis (the positive direction of the X-axis is the vehicle 1). (Left direction when facing forward)), which is orthogonal to the left-right direction and vertically along the ground or a line line orthogonal to the surface corresponding to the ground (here, the road surface 6) (in other words, the height of the vehicle 1). The direction) is the Y-axis (the positive direction of the Y-axis is the upward direction), and the front-rear direction along the lines orthogonal to each of the left-right direction and the up-down direction is the Z-axis (the positive direction of the Z-axis is the straight direction of the vehicle 1). Direction.). This point is the same in other drawings.

The head-up display device (an example of a virtual image display device) 20 in the virtual image display system 10 for a vehicle is a head-up display (HUD: Head-Up Display) provided in a dashboard 5 of a vehicle (an example of a moving body) 1. ) It is a device. The HUD device 20 emits the display light K toward the front window shield 2 (an example of the projected portion), and the front windshield 2 transmits the display light K of the image displayed by the HUD device 20 to the eye box 200. reflect. The HUD device 20 integrates not only information about the vehicle 1 (hereinafter referred to as vehicle information) but also information other than the vehicle information as an occupant of the vehicle 1 (the occupant is typically a driver of the vehicle 1). Is notified.). The vehicle information includes not only the information of the vehicle 1 itself but also the external information of the vehicle 1 related to the operation of the vehicle 1.

As shown in the figure, the vehicle virtual image display system 10 provided in the vehicle (own vehicle) 1 is the position and line-of-sight direction of the left eye 700L and the right eye 700R of the observer (typically, the driver sitting in the driver's seat of the vehicle 1). An external sensor 411 composed of an eye position detection unit (line-of-sight detection unit) 409 for detecting pupils (or faces), a camera (for example, a stereo camera) that captures the front (in a broad sense, the surroundings) of the vehicle 1, and the like. It has a head-up display device (hereinafter, also referred to as a HUD device) 20 and a display control device 30 that controls the HUD device 20.

Further, the "eye box" used in the description of the present embodiment is (1) a region where the entire virtual image V of the image can be visually recognized within the region, and at least a part of the virtual image V of the image cannot be visually recognized outside the region, (2). At least a part of the virtual image V of the image can be visually recognized in the area, and a part of the virtual image V of the image cannot be visually recognized outside the area. Yes, outside the region, the entire virtual image V of the image is less than the predetermined brightness, or (4) when the HUD device 20 can display the virtual image V that can be viewed stereoscopically, at least a part of the virtual image V is stereoscopic. It is a region that can be seen and a part of the virtual image V is not stereoscopically viewed outside the region. That is, when the observer places the eyes (both eyes) outside the eye box 200, the observer cannot see the entire virtual image V of the image, the visibility of the entire virtual image V of the image is very low, or it is difficult to perceive. The virtual image V of the image cannot be viewed stereoscopically. The predetermined luminance is, for example, about 1/50 of the luminance of the virtual image of the image visually recognized at the center of the eye box. The "eye box" is the same as the area (also referred to as eye lip) where the observer's viewpoint position is expected to be arranged in the vehicle on which the HUD device 20 is mounted, or most of the eye lip (for example, 80% or more). Is set to include.

The virtual image display region VS is a region of a plane, a curved surface, or a partially curved surface in which an image generated inside the HUD device 20 forms an image as a virtual image V, and is also called an image plane. The virtual image display area VS is a position where the display surface (for example, the emission surface of the liquid crystal optical modulation element) 51 of the display 40 described later of the HUD device 20 is imaged as a virtual image, that is, the virtual image display area VS is the HUD. Corresponding to the display surface 51 described later of the apparatus 20 (in other words, the virtual image display area VS has a conjugate relationship with the display surface 51 of the display 40 described later), and the virtual image visually recognized in the virtual image display area VS. Can be said to correspond to the image displayed on the display surface 51 described later of the HUD device 20. It is preferable that the virtual image display area VS itself has low visibility to the extent that it is not actually visible to the observer's eyes or is difficult to see.

In the imaginary image display area VS, the angle (tilt angle θt in FIG. 1) formed by the horizontal direction (XZ plane) with respect to the left-right direction (X-axis direction) of the vehicle 1 and the center 205 of the eyebox 200 and the imaginary image display are displayed. The angle formed by the line segment connecting the upper end VS1 of the area VS and the line segment connecting the center of the eyebox and the lower end VS2 of the imaginary image display area VS is defined as the vertical angle of the imaginary image display area VS, and is the second grade of this vertical angle. The angle (vertical arrangement angle θd in FIG. 1) formed by the branch line and the horizontal direction (XZ plane) is set. The vertical arrangement angle θd is typically set downward from the horizontal direction (XZ plane), and is, for example, +2 [degree] (here, the positive direction of the vertical arrangement angle θd is two of the vertical image angles. The equidistant line is the depression angle downward from the center 205 of the eyebox 200, and the negative direction of the vertical arrangement angle θd is the elevation angle at which the bisector of the vertical angle is upward from the center 205 of the eyebox 200. It can also be said that.).

Further, the virtual image display region VS of the present embodiment has a tilt angle θt of approximately 90 [degree] so as to substantially face the front (Z-axis positive direction). However, the tilt angle θt is not limited to this, and can be changed in the range of −10 ≦ θt <90 [degree]. The tilt angle θt may be set to, for example, 60 [degree], and the virtual image display area VS may be arranged so that the upper area is farther than the lower area when viewed from the observer. However, the range of the tilt angle θt is not limited to this.

FIG. 2 is a diagram showing one aspect of the configuration of the head-up display device. The HUD device 20 is installed, for example, in a dashboard (reference numeral 5 in FIG. 1). The HUD device 20 houses the display 40, the relay optical system 80, and the display 40 and the relay optical system 80, and is capable of emitting the display light K from the display 40 from the inside to the outside. It has a housing 22 having 21.

The display 40 of FIG. 2 is typically mainly composed of a liquid crystal display panel (light modulation element) 50 and a light source unit 60. The display surface 51 is a surface on the visual side of the liquid crystal display panel 50, and emits the display light K of the image. A virtual image is set by setting the angle of the display surface 51 with respect to the optical axis Kp of the display light K from the center of the display surface 51 toward the eye box 200 (center 205 of the eye box 200) via the relay optical system 80 and the projected portion. The angle of the display area VS (including the tilt angle θt) can be set.

The liquid crystal display panel (light modulation element) 50 incidents light from the light source unit 60 and emits the display light K spatially light-modulated to the 2D image toward the relay optical system 80 (second mirror 82). The liquid crystal display panel 50 has, for example, a rectangular shape whose short side is the direction in which the pixels corresponding to the vertical direction (Y-axis direction) of the virtual image V seen from the observer are arranged. The observer visually recognizes the transmitted light of the liquid crystal display panel 50 via the virtual image optical system 90. The virtual image optical system 90 is a combination of the relay optical system 80 shown in FIG. 2 and the front windshield 2. The light modulation element 50 (display 40) may be a self-luminous display, or a projection type display (for example, a reflection type display panel using LCoS or DMD) that projects an image on a screen. However, it is not limited to these. In this case, the display surface 51 is a screen of a projection type display.

The light source unit 60 includes a light source (not shown) and an illumination optical system (not shown).

The light source (not shown) is, for example, a plurality of chip-type LEDs, and emits illumination light to a liquid crystal display panel (an example of a light modulation element) 22. The light source unit 60 is composed of, for example, four light sources, and is arranged in a row along the long side of the liquid crystal display panel 50. The light source unit 60 emits illumination light toward the liquid crystal display panel 50 under the control of the display control device 30. The configuration of the light source unit 60 and the arrangement of the light sources are not limited to this.

The illumination optical system (not shown) includes, for example, one or a plurality of lenses (not shown) arranged in the emission direction of the illumination light of the light source unit 60, and diffusion arranged in the emission direction of the one or a plurality of lenses. It is composed of a board (not shown).

The light source unit 60 is configured to be locally dimmable, and the degree of illumination for each area of the display surface 51 may be changed under the control of the display control device 30. As a result, the light source unit 60 can adjust the brightness of the image displayed on the display surface 51 for each area.

The relay optical system 80 is arranged on the optical path of the display light K (light toward the eyebox 200 from the display 40) emitted from the display 40, and the display light K from the display 40 is directed to the outside of the HUD device 20. It is composed of one or more optical members projected onto the front windshield 2 of the above. The relay optical system 80 of FIG. 2 includes one concave first mirror 81 and one flat second mirror 82.

The first mirror 81 is, for example, a free curved surface shape having positive optical power. In other words, the first mirror 81 may have a curved surface shape in which the optical power differs for each region, that is, the optical power added to the display light K according to the region (optical path) through which the display light K passes. It may be different. Specifically, the first display light K11, the second display light K12, and the third display light K13 (see FIG. 2) heading from each region of the display surface 51 toward the eye box 200 are added by the relay optical system 80. The optical power may be different.

The second mirror 82 is, for example, a planar mirror, but is not limited to this, and may be a curved surface having optical power. That is, the relay optical system 80 is added according to the region (optical path) through which the display light K passes by synthesizing a plurality of mirrors (for example, the first mirror 81 and the second mirror 82 of the present embodiment). The optical power may be different. The second mirror 82 may be omitted. That is, the display light K emitted from the display 40 may be reflected by the first mirror 81 on the projected portion (front windshield) 2.

Further, the relay optical system 80 includes, but is not limited to, two mirrors (reflection optical system) in the present embodiment. The relay optics 80 may include one or more refractive optics such as lenses, diffractive optics such as holograms, reflective optics different from those described above, or combinations thereof. , May be added to the above-mentioned reflective optics, and may replace some or all of the above-mentioned reflected optics.

Further, the relay optical system 80 has a function of setting the distance to the virtual image display area VS by the curved surface shape (an example of optical power) and a function of generating an enlarged virtual image of the image displayed on the display surface 51. , But in addition to this, it may have a function of suppressing (correcting) the distortion of the virtual image that may occur due to the curved shape of the front windshield 2, or a known optical function other than these.

Further, the relay optical system 80 may be rotatable with one or more actuators 86, 87 controlled by the display control device 30 attached. For example, the actuator 86 may rotate (and / or move) the first mirror 81 about the first rotation axis AX1. The actuator 87 may rotate (and / or move) the second mirror 82 about the first rotation axis AX1.

The head-up display device 20 is a road surface and a branch of a traveling lane existing in a foreground which is a real space (actual view) visually recognized through the front windshield 2 of the vehicle 1 based on the control of the display control device 30 described later. Near objects such as roads, road signs, obstacles (pedestrians, bicycles, motorcycles, other vehicles, etc.), and features (buildings, bridges, etc.), positions that overlap objects, or positions set relative to the object. By displaying an image on the vehicle, an observer (typically, an observer sitting in the driver's seat of the vehicle 1) can perceive a visual augmented reality (AR). In the description of the present embodiment, an image in which the displayed position can be changed according to the depth in a predetermined space or the target position in the vertical and horizontal directions in the real object existing in the real scene or the real scene defined by the display control device 30 is AR. It is defined as an image, and an image in which the displayed position is set regardless of the position of the actual object is defined as a non-AR image. Further, in the description of the present embodiment, an image composed of one or more represented by a perspective projection that causes a sense of perspective is defined as a perspective image, and an image not represented by the perspective projection is defined as a non-perspective image. .. The perspective image is typically an AR image.

The display control device 30, which will be described later, controls the mode of the image V displayed (perceived by the observer) by the HUD device 20 by executing, for example, image rendering processing (graphic processing), display drive processing, and the like. Can be done.

FIG. 3 is a diagram conceptually showing a virtual object arranged at the target position of the actual scene and an image displayed in the virtual image display area so that the virtual object can be visually recognized at the target position of the actual scene. As shown in FIG. 3, when viewed from the viewer 700, the depth direction is the Z-axis direction, the left-right direction (the width direction of the own vehicle 1) is the X-axis direction, and the vertical direction (the vertical direction of the own vehicle 1). Is the Y-axis direction. The direction away from the viewer is the positive direction of the Z axis, the left direction of the viewer is the positive direction of the X axis, and the upward direction of the viewer is the positive direction of the Y axis. The direction.

The viewer 700 perceives the virtual object FU at a predetermined target position PT in the actual scene by visually recognizing the virtual image V formed (imaged) in the virtual image display area VS through the projected portion 2. .. The viewer visually recognizes the virtual image V of the image of the display light K reflected by the projected portion 2. At this time, when the virtual image V is, for example, an arrow indicating a course, the arrow of the virtual image V is the virtual image display area VS so that the virtual object FU is arranged and visually recognized at a predetermined target position PT in the foreground of the own vehicle 1. Is displayed in.

(Perspective virtual image V10 of the first display mode)
FIG. 4 is a diagram showing an example of a foreground to be visually recognized by an observer and a virtual image of a first display mode superimposed on the foreground while the own vehicle 1 is traveling. The virtual image V10 of the first display mode (hereinafter, also referred to as a perspective virtual image V10) has a vanishing point VP1 (first vanishing point VP1). That is, the figure of the perspective imaginary image V10 has a shape having a vanishing point VP1 that allows the depth to be perceived. Cross at.

As shown in the figure, when the perspective virtual image V10 is arranged below (the position where the vertical arrangement angle θd is in the positive direction) when viewed from the viewer, the vanishing point VP1 is arranged above the position of the virtual image V10. Will be done. On the other hand, when the perspective virtual image V10 is arranged above (the position where the vertical arrangement angle θd is in the negative direction) when viewed from the viewer, the vanishing point VP1 is arranged below the position of the virtual image.

The viewer obtains a sense of depth of the perspective virtual image V10 when viewed from the viewer, based on the relationship between the vanishing point VP1 and the position of the perspective virtual image V10. As described above, since the perspective virtual image V10 has a vanishing point VP1, it becomes easy for the viewer to estimate the depth position of the perspective virtual image V10.

On the other hand, as shown in FIG. 4, the traveling lane 6 of the own vehicle 1 has a vanishing point VP0 (actual vanishing point VP0). In this specific example, there is a traveling lane 6 that goes straight ahead when viewed from the viewer, and the extension lines of the boundary lines 7 and 8 on both sides of the traveling lane 6 substantially intersect at the vanishing point VP0 (point). (Becomes). As described above, since the traveling lane (road surface) 6 has the vanishing point VP0, the viewer obtains a sense of depth in the actual view (traveling lane 6).

Then, in the vehicle virtual image display system 10 according to the present embodiment, as shown in FIG. 4, the vanishing point VP1 of the perspective virtual image V10 is on the front side (from the observer) of the vanishing point VP0 of the actual view (traveling lane 6). It is placed below) when viewed.

Generally, when creating an image including paintings using the linear perspective method (perspective projection), vanishing points are used when arranging various objects in the image at their respective depth positions. For example, a straight line is virtually drawn radially from a predetermined vanishing point, and by aligning the contour line of each object along the straight line, each object is fixed and perceived at a predetermined depth position. Although a plurality of vanishing points can be provided, here, in order to simplify the explanation, a case where one vanishing point is provided in one image will be described.

When the general line perspective method (perspective method) is used, the extension lines of each boundary line forming the outline of the virtual object to be perceived by the observer intersect at the vanishing point VP0 of the foreground (traveling lane 6). The imaginary image is displayed, that is, the imaginary image is generated so that the position of the vanishing point VP1 of the imaginary image coincides with the position of the vanishing point VP0 of the foreground (traveling lane 6). The perspective imaginary image V10 is generated so that the position of the vanishing point VP1 of V10 is closer to the position of the vanishing point VP0 of the foreground (traveling lane 6) (first display process).

This makes it easier for the observer to perceive the perceived placement angle of the virtual object FU10 (the angle with respect to the road surface with the X2 axis as the rotation axis in FIG. 3) substantially parallel to the traveling lane 6. can.

The virtual object FU10 is an "arrow", and normally, the target position PT on which the virtual object FU10 is placed (set) matches the height of the surface of the foreground (traveling lane 6) (that is, the set height). To 0m). However, this is an example and is not limited. In another example, the target position PT may be set at a position higher than the height l of the surface of the foreground (travel lane 6) (that is, the set height may be set to 0.5 m or 1 m). Further, in another example, the target position PT may be set at a position lower than the height of the surface of the foreground (traveling lane 6) (that is, the set height may be set to -1 m or -2 m). ..

5A, 5B, 5C, and 5D are diagrams showing a modified example of the perspective image. The perspective virtual image V10 of the above embodiment is composed of one straight line "arrow", but is not limited thereto. The perspective virtual image V20 shown in FIG. 5A may be bent from the middle of the image (“arrow”). In this case, the left perspective line V1A can be defined as an extension of the left side until the perspective virtual image V20 bends, and the right perspective line V1B can be defined as an extension of the right side until the perspective virtual image V20 bends. can do.

Further, the perspective virtual image V30 shown in FIG. 5B is arranged above the first perspective virtual image V31 and the first perspective virtual image V31, and is set to be smaller in size than the first perspective virtual image V31. Consists of. In this case, the left perspective line V1A can be defined as a straight line connecting the left end of the first perspective V31 and the left end of the second perspective V32, and the right perspective line V1B is the right end of the first perspective V31. Can be defined as a straight line connecting and the right end of the second perspective V32.

Further, the perspective virtual image V40 shown in FIG. 5C may be composed of a plurality of perspective virtual images V41 to V49 divided on the left and right. In this case, the left perspective line V1A can be defined as a straight line (approximate straight line) connecting the left ends of the left perspectives V41 to V44, and the right perspective line V1B can be defined as the right end of the right perspectives V45 to V48. It can be defined as a straight line connecting (approximate straight line).

Further, the vanishing point VP1 (first vanishing point VP1) of the perspective imaginary image V50 shown in FIG. 5D is displaced in the left-right direction of the vanishing point VP0 which is the intersection of the boundary lines 7 and 8 of the traveling lane 6 when viewed from the observer. It may be set to a position directed to the vanishing point VP0'(the vanishing point VP0'shown in FIG. 5D is located to the left of the vanishing point VP0). Even in such a case, the vanishing point VP1 (first vanishing point VP1) of the perspective virtual image V50 can be arranged closer to the vanishing point VP0'(real scene vanishing point VP0).

(Perspective virtual image V60 of the second display mode)
FIG. 6 is a diagram showing an example of a foreground to be visually recognized by an observer and a virtual image of a second display mode superimposed on the foreground while the own vehicle 1 is traveling. The virtual image V60 in the second display mode (hereinafter, also referred to as a perspective virtual image V60) is a vanishing point VP2 (second vanishing point VP2) farther from the first vanishing point VP1 of the perspective virtual image V10 in the first display mode. ). That is, the figure of the perspective virtual image V60 has a shape having a second vanishing point VP2 for perceiving the depth. For example, the extension line V1B on the left side and the extension line V2B on the right side of the trunk portion of the "arrow" are the second 2 Crosses at vanishing point VP2.

As shown in the figure, when the perspective virtual image V60 of the second display mode is arranged downward (the position where the vertical arrangement angle θd is in the positive direction) when viewed from the viewer, the second vanishing point VP2 is the second. It is located above the position of 1 vanishing point VP1. On the other hand, when the perspective virtual image V60 of the second display mode is arranged above (the position where the vertical arrangement angle θd is in the negative direction) when viewed from the viewer, the second vanishing point VP2 is the first vanishing point. It is located below the position of VP1.

The HUD device 20 (display control device 30) of some embodiments displays the perspective virtual image V in the first display mode having the first vanishing point VP1 depending on the type of the perspective virtual image V, or the first display mode. 1 The second display mode having the second vanishing point VP2 set on the far side from the vanishing point VP1 is changed. The HUD device 20 (display control device 30) of some embodiments is represented by a second display mode having a second vanishing point VP2 if (10) the perspective virtual image V moves or expands and contracts in the perspective direction. (An example of the second display process) If (15) the perspective virtual image V is a still image, a moving image that does not move in the perspective direction very much, or a moving image that does not expand / contract in the perspective direction too much, the first vanishing point VP1 is provided. Expressed in a display mode.

7A and 7B are diagrams showing the perspective virtual image V of the second display mode having the second vanishing point VP2. In some embodiments, the HUD device 20 (display control device 30) is a moving image in which the perspective virtual image V is perceived as (11) gradually expanding and contracting in the perspective direction (in the example of FIG. 7A, from the near side). A moving image expressed in perspective that gradually extends toward the second vanishing point VP2 on the distant side) or (12) a moving image that is perceived to gradually move in the perspective direction (in the example of FIG. 7B, from the near side). In the case of a moving image expressed by a perspective method that gradually moves toward the second vanishing point VP2 on the distant side, it is expressed in the second display mode having the second vanishing point VP2 (an example of the second display process). .. On the other hand, the HUD device 20 (display control device 30) perceives the perspective virtual image V as (16) a still image, (17) a moving image that is not perceived by expanding and contracting in a very perspective direction, or (18) moving in a very perspective direction. In the case of a moving image that is not performed, it is expressed in the first display mode having the first vanishing point VP1 (an example of the first display process). The "(17) video that does not expand and contract in the perspective direction or (18) video that does not move in the perspective direction" specifically includes, for example, a video that moves in the left-right direction (X-axis direction) and a video that moves in the vertical direction (Y-axis direction). There is a moving image that moves by a minute distance below a predetermined threshold in a direction that can be perceived in the depth direction (Z-axis direction) by human perception (more specifically, for a distance that moves in the left-right direction (X-axis direction). A moving image in which the moving distance in the vertical direction (Y-axis direction) is 1/10 or less), and the like.

Further, the HUD device 20 (display control device 30) of some embodiments has a first vanishing point VP1 for displaying the perspective virtual image V according to the moving speed (deformation speed) of the moving image of the perspective virtual image V. The display mode is changed to the first display mode or the second display mode having a second vanishing point VP2 set farther from the first vanishing point VP1. For example, the HUD device 20 (display control device 30) of some embodiments has (20) a second display mode having a second vanishing point VP2 if the moving speed (deformation speed) of the moving image of the perspective virtual image V is high. Expression (an example of a second display process) is performed, and (25) if the moving speed of the moving speed of the moving image of the perspective virtual image V is slow, it is expressed in the first display mode having the first vanishing point VP1 (an example of the first display process). ..

In the HUD device 20 (display control device 30) of the above embodiment, the speed of the image in which the perspective virtual image V gradually expands and contracts in the (21) perspective direction or gradually moves in the perspective direction is equal to or higher than a predetermined image speed threshold value. If it is determined to be fast, it is expressed in a second display mode having a second vanishing point VP2 (an example of a second display process). On the other hand, in the HUD device 20 (display control device 30), the speed of the image in which the perspective virtual image V gradually expands and contracts in the (26) perspective direction or gradually moves in the perspective direction is less than a predetermined image speed threshold value. If it is determined to be slow, it is expressed in the first display mode having the first vanishing point VP1 (an example of the first display process).

Further, the HUD device 20 (display control device 30) of some embodiments has the first vanishing point VP1 according to the brightness of the front (periphery) of the own vehicle 1 in the display mode of the perspective virtual image V. The display mode is changed to the first display mode or the second display mode having the second vanishing point VP2 set farther from the first vanishing point VP1 (an example of the second display process). For example, the HUD device 20 (display control device 30) of some embodiments has (30) a second display mode having a second vanishing point VP2 when it is determined that the front (periphery) of the own vehicle 1 is bright. (35) When it is determined that the front (periphery) of the own vehicle 1 is dark, it is expressed in the first display mode having the first vanishing point VP1 (an example of the first display process).

Further, the HUD device 20 (display control device 30) of some embodiments displays the perspective imaginary image V in the vertical direction (Y-axis direction) of the perspective imaginary image V, and is perceived by humans in the depth direction (Z-axis direction). The first display mode having the first vanishing point VP1 or the second display mode having the second vanishing point VP2 set farther from the first vanishing point VP1 depending on the length of the direction). Change to. For example, the HUD device 20 (display control device 30) of some embodiments is the length in the vertical direction (Y-axis direction) of the (40) perspective imaginary image V, and is the depth direction (Z-axis direction) in human perception. When the distance of is the same as or longer than a predetermined threshold value, it is expressed in the second display mode having the second vanishing point VP2 (an example of the second display process), and (45) the vertical direction (Y) of the perspective imaginary image V. It is the length in the axial direction, and in human perception, when the distance in the depth direction (Z-axis direction) is shorter than a predetermined threshold value, it is expressed in the first display mode having the first vanishing point VP1 (first display). An example of processing).

Further, the HUD device 20 (display control device 30) of some embodiments has the first vanishing point VP1 for displaying the perspective virtual image V according to the length of the period during which the display is continued. The display mode is changed to the above-mentioned second display mode having the second vanishing point VP2 set on the far side from the first vanishing point VP1. For example, in some embodiments, the HUD device 20 (display control device 30) has (50) a period from the start of displaying the perspective virtual image V of a moving image or a still image to the end of displaying the same or longer than a predetermined threshold value. , Expressed in the second display mode having the second vanishing point VP2 (an example of the second display process), and (55) the period from the start of displaying the perspective virtual image V of the moving image or the still image to the end of the display is predetermined. When it is shorter than the threshold value, it is expressed in the first display mode having the first vanishing point VP1 (an example of the first display process).

FIG. 8 is a block diagram of a virtual image display system for a vehicle according to some embodiments. The display control device 30 includes one or more I / O interfaces 31, one or more processors 33, one or more image processing circuits 35, and one or more memories 37. The various functional blocks shown in FIG. 8 may consist of hardware, software, or a combination of both. FIG. 8 is only one embodiment, and the illustrated components may be combined with a smaller number of components or may have additional components. For example, the image processing circuit 35 (eg, graphic processing unit) may be included in one or more processors 33.

As shown, the processor 33 and the image processing circuit 35 are operably connected to the memory 37. More specifically, the processor 33 and the image processing circuit 35 execute a program stored in the memory 37 to generate and / or transmit image data, for example, for a vehicle virtual image display system 10 (display). The device 40) can be controlled. The processor 33 and / or the image processing circuit 35 includes at least one general purpose microprocessor (eg, central processing unit (CPU)), at least one application specific integrated circuit (ASIC), and at least one field programmable gate array (FPGA). , Or any combination thereof. The memory 37 includes any type of magnetic medium such as a hard disk, any type of optical medium such as a CD and DVD, any type of semiconductor memory such as a volatile memory, and a non-volatile memory. Volatile memory may include DRAM and SRAM, and non-volatile memory may include ROM and NVRAM.

As shown, the processor 33 is operably linked to the I / O interface 31. The I / O interface 31 communicates (CAN) with, for example, the vehicle ECU 401 and / or other electronic devices (reference numerals 403 to 419 described later) provided in the vehicle according to the standard of CAN (Controller Area Network). Also called communication). The communication standard adopted by the I / O interface 31 is not limited to CAN, and is, for example, CANFD (CAN with Flexible Data Rate), LIN (Local Interconnect Network), Ethernet (registered trademark), MOST (Media Oriented Systems Transport). : MOST is a registered trademark), a wired communication interface such as UART, or USB, or a local such as a personal area network (PAN) such as a Bluetooth® network, or an 802.1x Wi-Fi® network. In-vehicle communication (internal communication) interface, which is a short-range wireless communication interface within several tens of meters such as an area network (LAN), is included. The I / O interface 31 is a wireless wide area network (WWAN0, IEEE 802.16-2004 (WiMAX: Worldwide Interoperability for Microwave Access)), an IEEE 802.16e base (Mobile WiMAX), 4G, 4G-LTE, LTE Advanced, It may include an out-of-vehicle communication (external communication) interface such as a wide area communication network (for example, an Internet communication network) according to a cellular communication standard such as 5G.

As shown in the figure, the processor 33 is interoperably connected to the I / O interface 31 to be connected to various other electronic devices and the like connected to the vehicle imaginary image display system 10 (I / O interface 31). Information can be exchanged. The I / O interface 31 includes, for example, a vehicle ECU 401, a road information database 403, a vehicle position detection unit 405, an operation detection unit 407, an eye position detection unit 409, an external sensor 411, a brightness detection unit 413, an IMU415, and mobile information. The terminal 417, the external communication device 419, and the like are operably connected. The I / O interface 31 may include a function of processing (converting, calculating, and analyzing) information received from another electronic device or the like connected to the vehicle virtual image display system 10.

The display 40 is operably connected to the processor 33 and the image processing circuit 35. Therefore, the image displayed by the light modulation element 50 may be based on the image data received from the processor 33 and / or the image processing circuit 35. The processor 33 and the image processing circuit 35 control the image displayed by the light modulation element 50 based on the information acquired from the I / O interface 31.

The vehicle ECU 401 uses sensors and switches provided on the vehicle 1 to indicate the state of the vehicle 1 (for example, mileage, vehicle speed, accelerator pedal opening, brake pedal opening, engine throttle opening, injector fuel injection amount, engine rotation speed). , Motor speed, steering angle, shift position, drive mode, various warning states, posture (including roll angle and / or pitching angle), vehicle vibration (including magnitude, frequency, and / or frequency of vibration) )) And the like, and the state of the vehicle 1 is collected and managed (may include control), and as a part of the function, the numerical value of the state of the vehicle 1 (for example, the vehicle speed of the vehicle 1). ) Can be output to the processor 33 of the display control device 30. In addition, the vehicle ECU 401 simply transmits the numerical value detected by the sensor or the like (for example, the pitching angle is 3 [brake] in the forward tilt direction) to the processor 33, or instead, the numerical value detected by the sensor is used. A determination result based on one or more states of the vehicle 1 including (for example, the vehicle 1 satisfies a predetermined condition of a forward leaning state) and / and an analysis result (for example, a brake pedal opening degree). Combined with the information, the brake has caused the vehicle to lean forward.) May be transmitted to the processor 33. For example, the vehicle ECU 401 may output a signal indicating a determination result indicating that the vehicle 1 satisfies a predetermined condition stored in advance in a memory (not shown) of the vehicle ECU 401 to the display control device 30. The I / O interface 31 may acquire the above-mentioned information from the sensors and switches provided in the vehicle 1 provided in the vehicle 1 without going through the vehicle ECU 401.

Further, the vehicle ECU 401 may output an instruction signal indicating an image displayed by the vehicle virtual image display system 10 to the display control device 30, and at this time, the coordinates, size, type, display mode, and image notification of the image may be output. Necessity-related information that is a source for determining the necessity and / or the necessity of notification may be added to the instruction signal and transmitted.

The road information database 403 is included in a navigation device (not shown) provided in the vehicle 1 or an external server connected to the vehicle 1 via an external communication interface (I / O interface 31), and the vehicle position detection described later. Based on the position of the vehicle 1 acquired from the unit 405, the road information (lane, white line, stop line, crosswalk, etc.) on which the vehicle 1 travels, which is the information around the vehicle 1 (information related to the actual object around the vehicle 1). Road width, number of lanes, intersections, curves, branch roads, traffic regulations, etc.), feature information (buildings, bridges, rivers, etc.), presence / absence, position (including distance to vehicle 1), direction, shape, type , Detailed information and the like may be read out and transmitted to the processor 33. Further, the road information database 403 may calculate an appropriate route (navigation information) from the departure point to the destination, and output a signal indicating the navigation information or image data indicating the route to the processor 33.

The own vehicle position detection unit 405 is a GNSS (Global Navigation Satellite System) or the like provided in the vehicle 1, detects the current position and direction of the vehicle 1, and transmits a signal indicating the detection result via the processor 33. , Or directly output to the road information database 403, the portable information terminal 417 described later, and / or the external communication device 419. The road information database 403, the mobile information terminal 417 described later, and / or the external communication device 419 obtains the position information of the vehicle 1 from the own vehicle position detection unit 405 continuously, intermittently, or at a predetermined event. , Information around the vehicle 1 may be selected and generated and output to the processor 33.

The operation detection unit 407 is, for example, a CID (Center Information Display) of the vehicle 1, a hardware switch provided on an instrument panel, or a software switch that combines an image and a touch sensor, and is a vehicle 1. The operation information based on the operation by the occupant (the user sitting in the driver's seat and / or the user sitting in the passenger seat) is output to the processor 33. For example, the operation detection unit 407 sets display area setting information based on the operation of moving the virtual image display area 100, eye box setting information based on the operation of moving the eye box 200, and the observer's eye position 700 by the user's operation. Information and the like based on the operation to be performed are output to the processor 33.

The eye position detection unit 409 includes a camera such as an infrared camera that detects the eye position 700 (see FIG. 1) of the observer sitting in the driver's seat of the vehicle 1, and even if the captured image is output to the processor 33. good. The processor 33 acquires an image taken from the eye position detection unit 409 (an example of information that can estimate the eye position 700), and analyzes the captured image by a method such as pattern matching to obtain the eye position of the observer. The coordinates of 700 may be detected, and a signal indicating the detected coordinates of the eye position 700 may be output to the processor 33.

Further, the eye position detection unit 409 determines where in the spatial region corresponding to the plurality of preset display parameters the eye position 700 of the observer belongs to the analysis result obtained by analyzing the image captured by the camera (for example, the eye position 700 of the observer belongs to). The indicated signal) may be output to the processor 33. The method for acquiring information capable of estimating the eye position 700 of the observer of the vehicle 1 or the eye position 700 of the observer is not limited to these, and a known eye position detection (estimation) technique is used. May be obtained.

Further, the eye position detection unit 409 detects the moving speed and / or moving direction of the observer's eye position 700, and outputs a signal indicating the moving speed and / or moving direction of the observer's eye position 700 to the processor 33. It may be output to.

Further, the eye position detection unit 409 is presumed to have (10) a signal indicating that the observer's eye position 700 is outside the eye box 200, and (20) the observer's eye position 700 is outside the eye box 200. When a signal or (30) a signal in which the observer's eye position 700 is predicted to be outside the eye box 200 is detected, it is determined that a predetermined condition is satisfied, and a signal indicating the state is output to the processor 33. You may.

(20) The signal presumed that the observer's eye position 700 is outside the eye box 200 is (21) a signal indicating that the observer's eye position 700 cannot be detected, and (22) the observer's eye position 700. After the movement is detected, a signal indicating that the observer's eye position 700 cannot be detected, and / or (23) either of the observer's eye positions 700R or 700L is near the boundary 200A of the eyebox 200 (near the above). Includes, for example, a signal indicating that it is within a predetermined coordinate from the boundary 200A).

(30) The signal predicted that the observer's eye position 700 is outside the eye box 200 is (31) stored in advance in the memory 37 with respect to the previously detected eye position 700 by the newly detected eye position 700. A signal indicating that the eye position movement distance is equal to or greater than the specified eye position movement distance threshold (the movement of the eye position within a predetermined unit time is larger than the specified range), and (32) the movement speed of the eye position are stored in advance in the memory 37. Includes a signal indicating that the eye position movement speed is equal to or higher than the threshold value, and the like.

Further, the eye position detection unit 409 may have a function as a line-of-sight direction detection unit 409. The line-of-sight direction detection unit 409 may include an infrared camera or a visible light camera that captures the face of an observer sitting in the driver's seat of the vehicle 1, and may output the captured image to the processor 33. The processor 33 acquires a captured image (an example of information capable of estimating the line-of-sight direction) from the line-of-sight direction detection unit 409, and analyzes the captured image to determine the line-of-sight direction (and / or the gaze position) of the observer. Can be identified. The line-of-sight direction detection unit 409 may analyze the captured image from the camera and output a signal indicating the line-of-sight direction (and / or the gaze position) of the observer, which is the analysis result, to the processor 33. The method for acquiring information that can estimate the line-of-sight direction of the observer of the vehicle 1 is not limited to these, and the EOG (Electro-oculogram) method, the corneal reflex method, the scleral reflex method, and the Purkinje image detection. It may be obtained using other known line-of-sight detection (estimation) techniques such as method, search coil method, infrared fundus camera method.

The out-of-vehicle sensor 411 detects real objects existing around the vehicle 1 (front, side, and rear). The actual objects detected by the out-of-vehicle sensor 411 are, for example, obstacles (pedestrians, bicycles, motorcycles, other vehicles, etc.), road surfaces, lane markings, roadside objects, and / or features (buildings, etc.) of the traveling lane described later. Etc. may be included. As the out-of-vehicle sensor, for example, a detection unit composed of a radar sensor such as a millimeter wave radar, an ultrasonic radar, a laser radar, a camera, or a combination thereof, and detection data from the one or a plurality of detection units are processed (. It consists of a processing device (data fusion) and a processing device. Conventional well-known methods are applied to object detection by these radar sensors and camera sensors. By object detection by these sensors, the presence or absence of a real object in the three-dimensional space, and if the real object exists, the position of the real object (relative distance from the vehicle 1 and the traveling direction of the vehicle 1 in the front-rear direction). (Horizontal position, vertical position, etc.), size (horizontal direction (horizontal direction), height direction (vertical direction), etc.), movement direction (horizontal direction (horizontal direction), depth) Direction (front-back direction)), movement speed (horizontal direction (left-right direction), depth direction (front-back direction)), and / or type may be detected. One or a plurality of out-of-vehicle sensors 411 detect a real object in front of the vehicle 1 in each detection cycle of each sensor, and the real object information (presence / absence of the real object, existence of the real object exists) which is an example of the real object information. In this case, information such as the position, size, and / or type of each real object can be output to the processor 33. It should be noted that these real object information may be transmitted to the processor 33 via another device (for example, vehicle ECU 401). Infrared cameras and near-infrared cameras are desirable when using a camera as a sensor so that a real object can be detected even when the surroundings are dark, such as at night. When using a camera as a sensor, a stereo camera that can acquire a distance or the like by parallax is desirable.

The brightness detection unit 413 determines the illuminance or brightness in a predetermined range of the foreground existing in front of the vehicle interior of the vehicle 1 as the outside brightness (an example of brightness information), or the illuminance or brightness in the vehicle interior as the brightness inside the vehicle (brightness). It is detected as an example of information). The brightness detection unit 413 is, for example, a phototransistor, a photodiode, or the like, and is mounted on an instrument panel, a rearview mirror, a HUD device 20, or the like of the vehicle 1 shown in FIG.

The IMU415 is one or more sensors (eg, accelerometers and gyroscopes) configured to detect the position, orientation, and changes (speed of change, acceleration of change) of vehicle 1 based on inertial acceleration. Can include combinations of. The IMU415 is a processor that analyzes the detected values (the detected values include the position and orientation of the vehicle 1, and signals indicating these changes (change speed, change acceleration), etc.) and the detected values. It may be output to 33. The analysis result is a signal or the like indicating a determination result of whether or not the detected value satisfies a predetermined condition, and is, for example, from a value related to a change (change speed, change acceleration) in the position or orientation of the vehicle 1. , It may be a signal indicating that the behavior (vibration) of the vehicle 1 is small.

The personal digital assistant 417 is a smartphone, a laptop computer, a smart watch, or other information device that can be carried by an observer (or another occupant of vehicle 1). The I / O interface 31 can communicate with the mobile information terminal 417 by pairing with the mobile information terminal 417, and the data recorded in the mobile information terminal 417 (or the server through the mobile information terminal). To get. The mobile information terminal 417 has, for example, the same functions as the above-mentioned road information database 403 and own vehicle position detection unit 405, acquires the road information (an example of real object-related information), and transmits it to the processor 33. You may. Further, the personal digital assistant 417 may acquire commercial information (an example of information related to a real object) related to a commercial facility in the vicinity of the vehicle 1 and transmit it to the processor 33. The mobile information terminal 417 transmits schedule information of the owner (for example, an observer) of the mobile information terminal 417, incoming information on the mobile information terminal 417, mail reception information, and the like to the processor 33, and the processor 33 and an image. The processing circuit 35 may generate and / or transmit image data relating to these.

The external communication device 419 is a communication device that exchanges information with the vehicle 1, for example, another vehicle connected to the vehicle 1 by vehicle-to-vehicle communication (V2V: Vehicle To Vehicle), and pedestrian-to-vehicle communication (V2P: Vehicle To Pestation). ), A network communication device connected by pedestrians (portable information terminals carried by pedestrians) and road-to-vehicle communication (V2I: Vehicle To Roadside Infrastructure), and in a broad sense, communication with vehicle 1 (V2X). : Vehicle To Everything) includes everything connected. The external communication device 419 acquires, for example, the positions of pedestrians, bicycles, motorcycles, other vehicles (preceding vehicles, etc.), road surfaces, lane markings, roadside objects, and / or features (buildings, etc.) and transfers them to the processor 33. It may be output. Further, the external communication device 419 has the same function as the own vehicle position detection unit 405 described above, and may acquire the position information of the vehicle 1 and transmit it to the processor 33, and further, the road information database 403 described above. It also has a function, and may acquire the road information (an example of information related to a real object) and transmit it to the processor 33. The information acquired from the external communication device 419 is not limited to the above.

The software components stored in the memory 37 include a display parameter setting module 502, a graphic module 504, a light source drive module 506, and an actuator drive module 508.

FIG. 9 is a flow chart showing a method S100 for executing a display control process for displaying a perspective virtual image in the first display mode or the second display mode. The method S100 is executed in the HUD device 20 including the display and the display control device 30 that controls the HUD device 20. Some actions in the method S100 are optionally combined, some steps are optionally modified, and some actions are optionally omitted.

The display control device 30 (graphic module 504 described later) is, for example, an appropriate route (navigation information) from the road information database 403 to the destination, and real object information (presence / absence of a real object, real object) from the outside sensor 411. If there is, information such as the position, size, and / or type of each real object) is acquired, and a virtual object to be displayed is selected (S110).

As shown in S131, the display parameter setting module 502 changes the display mode of the perspective virtual image V based on one or more information acquired from the I / O interface 31. The display parameter setting module 502 sets the (10) perspective virtual image V as a moving or expanding moving image in the perspective direction based on one or more information acquired from the I / O interface 31, a second vanishing point. Set to the second display mode having VP2 (an example of the second display process), and (15) set the perspective virtual image V to a still image, a moving image that does not move in a very perspective direction, or a moving image that does not expand / contract in a very perspective direction. In this case, the first display mode having the first vanishing point VP1 is set (an example of the first display process).

Further, as shown in S132, the display parameter setting module 502 sets the moving speed of the moving image of the (20) perspective virtual image V faster than a predetermined threshold value based on one or a plurality of information acquired from the I / O interface 31. In this case, the second display mode having the second vanishing point VP2 is set (an example of the second display process), and (25) the moving speed of the moving speed of the moving image of the perspective virtual image V is set to be faster than a predetermined threshold value. It is set to the first display mode having the vanishing point VP1 (an example of the first display process).

Further, as shown in S133, the display parameter setting module 502 determines that (30) the front (periphery) of the own vehicle 1 is bright based on the information acquired from the brightness detection unit 413 (but not limited to this). When it is determined, the perspective virtual image V is set in the second display mode having the second vanishing point VP2 (an example of the second display process), and (35) it is determined that the front (periphery) of the own vehicle 1 is dark. If so, the perspective virtual image V is set in the first display mode having the first vanishing point VP1 (an example of the first display process).

Further, as shown in S134, the display parameter setting module 502 is not limited to the road information acquired from the road information database 403 and the position information of the real object existing in the foreground acquired from the vehicle outside sensor 411 (but not limited to this). ), It is the length in the vertical direction (Y-axis direction) of the perspective virtual image V that indicates the real object existing in the guide path or the foreground, and the distance in the depth direction (Z-axis direction) in human perception. When the value is the same as or longer than the predetermined threshold value stored in the memory 37 in advance, the perspective virtual image V is set to the second display mode having the second vanishing point VP2 (an example of the second display process), (45). ) The length of the perspective imaginary image V in the vertical direction (Y-axis direction), and when the distance in the depth direction (Z-axis direction) is shorter than a predetermined threshold in human perception, the perspective imaginary image V is set to the first vanishing point VP1. (An example of the first display process).

Further, as shown in S135, the display parameter setting module 502 displays (50) the perspective virtual image V of the moving image or the still image after starting to display it based on one or a plurality of information acquired from the I / O interface 31. When the period until the end is the same as or longer than a predetermined threshold value predetermined in the memory 37, the perspective virtual image V is set to the second display mode having the second vanishing point VP2 (an example of the second display process). (55) When the period from the start of displaying the perspective virtual image V of the moving image or the still image to the end of displaying it is shorter than a predetermined threshold stored in advance in the memory 37, the perspective virtual image V has a first vanishing point VP1. It is set in the first display mode (an example of the first display process).

The graphic module 504 includes various known software components for performing image processing such as rendering to generate image data and driving the display 40. Further, the graphic module 504 has a type (moving image, still image, shape), arrangement (positional coordinates, angle), size, display distance (in the case of 3D), and visual effect (for example, brightness) of the displayed image. It may include various known software components for changing transparency, saturation, contrast, or other visual characteristics). The graphic module 504 determines the type of image (one of the examples of display parameters), the position coordinates of the image (one of the examples of display parameters), the angle of the image (the pitching angle about the X direction, and the Y direction). Yaw rate angle as an axis, rolling angle as an axis in the Z direction, etc., which is one of the examples of display parameters), image size (one of the examples of display parameters), image color (hue, saturation). , One of the display parameters set by brightness, etc.), the intensity of the perspective expression of the image (one of the display parameters set by the position of the vanishing point, etc.) so that the image can be visually recognized by the observer. It can generate data and drive the optical modulation element 50.

The light source drive module 506 includes various known software components for performing driving of the light source unit 24. The light source drive module 506 can drive the light source unit 24 based on the set display parameters.

The actuator drive module 508 contains various known software components for performing driving of the first actuator 28 and / or the second actuator 29. The actuator drive module 508 is based on set display parameters. 1 Actuator 28 and 2nd actuator 29 may be driven.

The eye position detection module 510 detects the observer's eye position 700. The eye position detection module 510 is a coordinate indicating the eye position 700 of the observer (a position in the X and Y axis directions, which is an example of a signal indicating the eye position 700) and a coordinate indicating the height of the observer's eyes (a position indicating the eye position 700). It is a position in the Y-axis direction, which is an example of a signal indicating the eye position 700), and coordinates (positions in the Y and Z-axis directions) indicating the height and depth of the observer's eyes. , An example of a signal indicating the eye position 700), and / or coordinates indicating the observer's eye position 700 (positions in the X, Y, Z axis directions, and of the signal indicating the eye position 700). An example.) Includes various software components for performing various actions related to detecting).

The positions of the first vanishing point VP1 and the second vanishing point VP2 are adjusted by the posture of the observer's eye position 700 detected by the eye position detection unit 409 or the own vehicle 1 detected by the IMU 415. May be good. For example, the display control device 30 (display parameter setting module 502) has a first vanishing point VP1 and a second vanishing point according to a change in the posture (pitching angle (angle about the X direction)) of the own vehicle 1. The position of VP2 can be adjusted. The display control device 30 (display parameter setting module 502) has a first vanishing point VP1 and a second vanishing point VP2 with respect to the virtual image display area VS as seen from the observer when the pitching angle is such that the front of the own vehicle 1 is lowered. On the other hand, when the pitching angle is such that the front of the own vehicle 1 rises, the first vanishing point VP1 and the second vanishing point with respect to the virtual image display area VS from the observer's point of view are adjusted. It can be adjusted so that the relative position of VP2 is on the lower side.

The operation of the above-mentioned processing process can be carried out by executing one or more functional modules of an information processing apparatus such as a general-purpose processor or a chip for a specific application. All of these modules, combinations of these modules, and / or combinations with known hardware capable of substituting their functions are within the scope of the protection of the present invention.

The functional blocks of the vehicle virtual image display system 10 are optionally performed by hardware, software, or a combination of hardware and software to implement the principles of the various embodiments described. The functional blocks described in FIG. 8 may be optionally combined or one functional block separated into two or more subblocks in order to implement the principles of the embodiments described. It will be understood by those skilled in the art. Accordingly, the description herein optionally supports any possible combination or division of functional blocks described herein.

1: Own vehicle 2: Projected part (front windshield)
5: Dashboard 6: Driving lane (road surface)
7: Boundary line 8: Boundary line 10: Virtual image display system for vehicles 20: HUD device (head-up display device)
21: Light emission window 22: Housing 24: Light source unit 28: First actuator 29: Second actuator 30: Display control device 31: I / O interface 33: Processor 35: Image processing circuit 37: Memory 40: Display 50 : Optical modulation element 51: Display surface 60: Light source unit 80: Relay optical system 81: First mirror 82: Second mirror 86: Actuator 87: Actuator 90: Virtual image optical system 100: Virtual image display area 200: Eyebox 200A: Boundary 205: Center 401: Vehicle ECU
403: Road information database 405: Own vehicle position detection unit 407: Operation detection unit 409: Eye position detection unit (line-of-sight direction detection unit)
411: External sensor 413: Brightness detection unit 417: Mobile information terminal 419: External communication device 502: Display parameter setting module 504: Graphic module 506: Light source drive module 508: Actuator drive module 510: Eye position detection module 700: Eye position (Viewer)
700L: Left eye (eye position)
700R: Right eye (eye position)
AX1: First rotation axis FU: Virtual object FU10: Virtual object K: Display light Kp: Optical axis PT: Target position V: Perspective virtual image V10: Perspective virtual image V1A: Left perspective line V20: Perspective virtual image V1B: Right perspective Line V30: Perspective virtual image V31: First perspective virtual image V32: Second perspective virtual image VP0: Real scene vanishing point VP0': Vanishing point VP1: First vanishing point VP2: Second vanishing point VS: Virtual image display area VS1: Upper end VS2 : Lower end θd: Vertical arrangement angle θt: Tilt angle

Claims (10)

A display control device (30) that executes display control in a head-up display device that allows an observer to visually recognize a virtual image of the image superimposed on the foreground by projecting an image onto a projected member.
With one or more processors (33),
Memory (37) and
It comprises one or more computer programs stored in the memory (37) and configured to be executed by the one or more processors (33).
The processor (33)
The first display process of displaying as a perspective virtual image of the first display mode having a first vanishing point (VP1) arranged below the vanishing point (VP0) of the foreground when viewed from the observer.
A second display process of displaying as a perspective virtual image of the second display mode having a second vanishing point (VP2) arranged above the first vanishing point (VP1) when viewed from the observer is executed. do,
Display control device (30). In the second display process,
The second vanishing point (VP2) is located below the vanishing point (VP0) in the foreground.
The display control device (30) according to claim 1. The processor (33)
If the perspective virtual image is a still image, the first display process is executed.
If the perspective virtual image is a moving image, the second display process is executed.
The display control device (30) according to claim 1. The processor (33)
If the perspective virtual image is a moving image that does not expand or contract in the perspective direction or a moving image that does not move in the perspective direction when viewed from the observer, the first display process is executed.
If the perspective virtual image is a moving image that is perceived as expanding or contracting in the perspective direction or a moving image that is perceived as moving in the perspective direction from the observer's point of view, the second display process is performed. Run,
The display control device (30) according to claim 1. The processor (33)
If the surroundings are dark, the first display process is executed.
When the surroundings are bright, the second display process is executed.
The display control device (30) according to claim 1. The processor (33)
If the moving speed or deformation speed of the perspective virtual image is slow, the first display process is executed.
When the moving speed or the deformation speed of the perspective virtual image is high, the second display process is executed.
The display control device (30) according to claim 1. The processor (33)
When the vertical length of the perspective virtual image is shorter than a predetermined threshold value, the first display process is executed.
When the vertical length of the perspective virtual image is longer than the predetermined threshold value, the second display process is executed.
The display control device (30) according to claim 1. The processor (33)
When the period from the start of displaying the perspective to the end of displaying the perspective is shorter than a predetermined threshold value, the first display process is executed.
When the period from the start of displaying the perspective to the end of displaying the perspective is longer than the predetermined threshold value, the second display process is executed.
The display control device (30) according to claim 1. The display control device (30) according to any one of claims 1 to 8.
A light modulation element (50) that emits display light and
A head-up display device (20) comprising a relay optical system (80) that directs the display light from the light modulation element (50) to a projected portion. It is an image display control method.
The first display process of displaying as a perspective virtual image of the first display mode having a first vanishing point (VP1) arranged below the vanishing point (VP0) of the foreground when viewed from the observer.
A second display process of displaying as a perspective virtual image of the second display mode having a second vanishing point (VP2) arranged above the first vanishing point (VP1) when viewed from the observer is executed. do,
Display control method.

Images