JP7396404B2 - Display control device and display control program - Google Patents

Description

The disclosure in this specification relates to a display control device and a display control program that display a virtual image superimposed on a superimposition target in the foreground.

BACKGROUND ART Conventionally, for example, Patent Document 1 discloses a head-up display (hereinafter referred to as "HUD") that displays a virtual image, in which a light projection position can be adjusted according to the viewpoint height of a viewer. In the HUD of Patent Document 1, the inclination angle of the concave mirror in the front-rear direction and the position of the casing are adjusted based on the viewer's viewpoint position information. As a result, even if the viewer's viewpoint position changes, the position of the virtual image viewed by the viewer can be maintained constant.

JP2015-60180A

In recent years, new attempts have been made to present information by setting a specific object in the foreground as a superimposition target and superimposing a virtual image on the superimposition target. In such a configuration in which a virtual image is superimposed on the superimposition target, conventional adjustment of the projection optical system may be inappropriate.

To explain in detail, in Patent Document 1 , an area in the foreground on which a virtual image can be superimposed when viewed from the viewer (hereinafter referred to as a "displayable area") is inevitably moved by adjustment according to changes in the viewpoint position. Resulting in. According to such movement of the displayable area, a superimposition target on which a virtual image is desired to be superimposed moves out of the displayable area, and a scene may occur in which a virtual image cannot be superimposed.

The present disclosure aims to provide a display control device and a display control program suitable for presenting information that superimposes a virtual image on a superimposition target.

In order to achieve the above object, one aspect disclosed is that a projection optical system (74) which is used in a vehicle (A) and whose state is adjusted and controlled projects light onto a projection member (WS) of the vehicle. A display control device that controls the display of a virtual image (Vi) superimposed on a superimposition target in the foreground by an image, and a display control device that acquires attitude change information in a low frequency band (LB) among attitude changes that occur in a vehicle. a second acquisition unit (52) that acquires attitude change information in a high frequency band (HB) having a higher frequency than the low frequency band among changes in the attitude of the vehicle; A first correction unit (39) that electronically corrects the positional deviation of the virtual image caused by the change, and a second correction unit that corrects the positional deviation of the virtual image caused by the posture change in the high frequency band by a process different from that of the first correction unit. The second correction unit determines the extraction range to be used for the output image (Po) in the electronically corrected input image (Pi) based on posture change information in a high frequency band. It is assumed that the display control device makes the decision .

Further, one aspect disclosed is that superimposition in the foreground is caused by imaging of light projected onto a projection member (WS) of the vehicle by a projection optical system (74) used in the vehicle (A) and whose state is adjusted and controlled. A display control program that controls the display of a virtual image (Vi) superimposed on an object, the program causing at least one processing unit to acquire posture change information in a low frequency band (LB) among posture changes occurring in a vehicle. A first acquisition unit (33), a second acquisition unit (52) that acquires attitude change information in a high frequency band (HB) whose frequency is higher than a low frequency band among changes in the attitude of the vehicle, and A first correction unit (39) that electronically corrects the positional deviation of the virtual image caused by the attitude change; and a second correction unit (39) that corrects the virtual image positional deviation caused by the attitude change in the high frequency band by a process different from that of the first correction unit. A correction unit (53) is configured to function so as to include a correction unit (53). This is a display control program that makes decisions based on posture change information .

In these aspects , the number of scenes in which a virtual image cannot be superimposed on a superimposition target on which a virtual image should be superimposed can be reduced. According to the above, it is suitable for presenting information in which a virtual image is superimposed on a superimposition target.

Note that the reference numbers in parentheses above merely indicate an example of the correspondence with specific configurations in the embodiments to be described later, and do not limit the technical scope in any way.

FIG. 1 is a block diagram showing an overall image of a virtual image display system according to a first embodiment of the present disclosure. FIG. 3 is a diagram showing vibration characteristics of posture changes occurring in a vehicle, and is a diagram showing an example of each range of a high frequency band and a low frequency band. FIG. 6 is a diagram showing details of a distortion correction table stored in a storage unit of the display control ECU and details of processing contents of distortion correction based on the distortion correction table. It is a figure showing the details of the adjustment mechanism provided in HUD. FIG. 6 is a diagram schematically showing optical correction for reducing changes in a displayable area together with a specific example of AR display. 5 is a flowchart showing details of virtual image drawing processing performed in each processing unit of the virtual image display system. FIG. 2 is a block diagram showing an overall image of a virtual image display system according to a second embodiment of the present disclosure.

Hereinafter, multiple embodiments of the present disclosure will be described based on the drawings. Note that duplicate explanations may be omitted by assigning the same reference numerals to corresponding components in each embodiment. When only a part of the configuration is described in each embodiment, the configuration of the other embodiments previously described can be applied to other parts of the configuration. Furthermore, in addition to the combinations of configurations specified in the description of each embodiment, configurations of a plurality of embodiments may be partially combined even if not explicitly specified, as long as no particular problem arises in the combination. It is assumed that combinations of structures described in the plurality of embodiments and modifications that are not explicitly described are also disclosed in the following description.

(First embodiment)
A virtual image display system 10 according to the first embodiment of the present disclosure shown in FIG. 1 is used in a vehicle A. The virtual image display system 10 uses a portion of the windshield WS as a projection area PA, and displays a virtual image Vi superimposed on the foreground of the vehicle A so that the driver can visually recognize it by projecting light onto the projection area PA. The virtual image display system 10 presents various information related to the vehicle A to the driver through augmented reality (hereinafter referred to as "AR") display using the virtual image Vi.

The display position of the virtual image Vi displayed in AR is associated with a specific object in the foreground that is visible through the projection area PA, such as a superimposed object such as a vehicle running in front, a pedestrian, and a road surface. The virtual image Vi moves according to the driver's appearance, following the superimposition object, as if it were fixed relative to the superimposition object. Therefore, even if the attitude of the vehicle A changes, the state in which the virtual image Vi is displayed superimposed on the superimposition target is maintained. As an example, a virtual image Vi indicating the range of lanes in which the vehicle should travel is superimposed and displayed as an AR display object on the road surface between the left and right lane markings in the foreground (see FIG. 5).

The virtual image display system 10 can display the superimposition target even if the relative positional relationship between the superimposition target, the projection area PA, and the eyepoint EP changes due to a change in the posture of the vehicle A or a change in the position of the driver's eye point EP. Control is performed to maintain the superimposed state of the virtual image Vi on the virtual image Vi. The virtual image display system 10 is capable of sequentially correcting the shape and projection position of the light formed as the virtual image Vi so that the deviation of the display position of the virtual image Vi with respect to the superimposition target is reduced and preferably eliminated.

The virtual image display system 10 includes a plurality of correction functions for correcting virtual image display. Each correction function exhibits the effect of correcting the display position shift with respect to vibrations in different frequency bands. To be more specific, the vibration (for example, vehicle vibration) that causes the display position shift mainly includes components in a frequency band up to about 2 Hz, as shown in FIG. In the virtual image display system 10, such a frequency band is divided into, for example, a low frequency band LB and a high frequency band HB, and the display shift is reduced using different methods for each band.

The low frequency band LB is defined as a frequency band from about 0 to 0.5 Hz. Of this low frequency band LB, a band particularly on the lower frequency side is defined as a first band LB1, and a band on the higher frequency side is defined as a second band LB2. The bandwidth of the first band LB1 is set narrower than the bandwidth of the second band LB2. The first band LB1 is a band whose frequency is generally close to zero. For example, changes in vehicle posture due to changes in the number of occupants of vehicle A and changes in loaded weight are included in the first band LB1. In addition, changes in the position of the eyepoint EP due to the driver's physique are also included in the first band LB1. On the other hand, the second band LB2 is a band that mainly includes vehicle vibrations of less than 0.5 Hz among vehicle vibrations that occur during driving. For example, when the vehicle A is slowly accelerated or decelerated, a posture change due to acceleration or deceleration based on normal driving operation (pedal operation) becomes vehicle vibration belonging to the second band LB2.

The high frequency band HB is a frequency band from 0.5 to 2 Hz, which is higher in frequency than the low frequency band LB. For example, while driving on a rough road, a change in posture in a driving scene such as when a brake operation is input multiple times or when the vehicle A suddenly starts or decelerates becomes vehicle vibration that belongs to the high frequency band HB.

The virtual image display system 10 shown in FIG. 1 includes a drawing ECU (Electronic Control Unit) 30, a display control ECU 40, a head-up display (hereinafter referred to as "HUD") 70, and the like. Among the above-mentioned frequency bands, the positional deviation of the virtual image Vi caused by a change in posture in the first band LB1 is reduced by electronic correction in the display control ECU 40 or optical correction in the HUD 70. Further, the positional shift of the virtual image Vi caused by the attitude change in the second band LB2 is reduced by electronic correction in the drawing ECU 30. Furthermore, the positional shift of the virtual image Vi caused by the attitude change in the high frequency band HB is reduced by electronic correction in the display control ECU 40. The details of the drawing ECU 30, display control ECU 40, and HUD 70 will be described below in order. Note that the above-mentioned electronic correction (hereinafter referred to as "electronic correction") is a correction performed on the video data provided to the HUD 70 through arithmetic processing, and is a non-optical correction.

The drawing ECU 30 is a control device that is directly or indirectly connected to a plurality of vehicle-mounted displays and integrally controls the display of each vehicle-mounted display. The drawing ECU 30 individually generates video data to be displayed on each in-vehicle display and sequentially outputs it to each in-vehicle display. The drawing ECU 30 is electrically connected to a communication bus 29 of the in-vehicle network, and can communicate with other in-vehicle components via the communication bus 29. The drawing ECU 30 acquires data necessary for generating video data from the in-vehicle network. The communication bus 29 is directly or indirectly electrically connected to the external sensor 21, the height sensor 22, the vehicle control unit 24, the driver status monitor 25, and the like.

The external sensor 21 detects moving objects such as pedestrians and other vehicles, as well as stationary objects such as curbs, road signs, road markings, and lane markings on the road. At least a portion of these moving objects and stationary objects are to be superimposed with the virtual image Vi. The vehicle A is equipped with, for example, a camera unit, a lidar, a millimeter wave radar, and the like as an external sensor 21. The external sensor 21 sequentially outputs object information indicating the relative positions, types, etc. of the detected moving objects and stationary objects to the communication bus 29.

The height sensor 22 is a sensor that detects displacement in the vertical direction that occurs in the vehicle A in order to measure the height from the road surface on which the vehicle A is placed to the body. The height sensor 22 is installed, for example, on either the left or right rear suspension. The height sensor 22 measures the relative distance between the body and the suspension arm and sequentially outputs it to the communication bus 29 as a vehicle height measurement result.

The vehicle control unit 24 is a control device mainly composed of a microcontroller. The vehicle control unit 24 controls the behavior of the vehicle A based on object information detected by the external sensor 21 and the driver's driving operation. Vehicle control unit 24 is electrically connected to pedal sensor 23. The pedal sensor 23 includes an accelerator position sensor and a brake pedal force sensor.

The vehicle control unit 24 controls the longitudinal acceleration generated in the vehicle A, that is, the axle torque and the braking force, based on the detection signal of the pedal sensor 23. In addition, the vehicle control unit 24 performs feedforward control of the axle torque and braking force so that vibrations of the vehicle A caused by acceleration/deceleration operations by the driver and disturbances such as road surface irregularities are suppressed. Vehicle control unit 24 sequentially outputs each target value of axle torque and brake force in feedforward control to communication bus 29 as control information.

The driver status monitor (hereinafter referred to as "DSM") 25 includes a near-infrared light source, a near-infrared camera, and an image analysis section. The DSM 25 is arranged, for example, on the top surface of the instrument panel, with the near-infrared camera facing the driver's seat side. The DSM 25 uses a near-infrared camera to photograph the driver's face periphery or upper body irradiated with near-infrared light by a near-infrared light source, and captures a facial image including the driver's face. The DSM 25 analyzes the captured facial image using an image analysis unit and detects the position of the driver's eye point EP. The DSM 25 sequentially outputs the position information of the eyepoint EP to the communication bus 29.

The drawing ECU 30 is mainly composed of a computer having a processing section 30a, a RAM, a storage section 30b, and an input/output interface. The processing unit 30a is hardware for arithmetic processing combined with RAM. The processing unit 30a has a configuration including at least one arithmetic core such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The processing unit 30a may further include an FPGA (Field-Programmable Gate Array), an IP core with other dedicated functions, and the like. The storage unit 30b stores various programs (display control programs, etc.) executed by the processing unit 30a.

The drawing ECU 30 executes a program stored in a storage section 30b using a processing section 30a, and includes a plurality of functional sections. Specifically, the drawing ECU 30 includes functional units such as a sensor value acquisition unit 31, a brightness setting unit 34, a content selection unit 35, a target setting unit 36, a projection adjustment unit 37, a correction amount calculation unit 38, and a drawing data generation unit 39. is constructed. The sensor value acquisition section 31 , brightness setting section 34 , content selection section 35 , and target setting section 36 can acquire information from the communication bus 29 .

The sensor value acquisition unit 31 acquires information indicating posture changes in the low frequency band LB from the communication bus 29. The sensor value acquisition section 31 is provided with auxiliary functional sections such as a DC information processing section 32 and an AC information processing section 33.

The DC information processing unit 32 acquires state information of a DC (Direct Current) component belonging to the first band LB1. Specifically, the DC information processing unit 32 calculates the vehicle height measurement result when the vehicle A is stopped (hereinafter referred to as "vehicle reference posture") and the position information of the eye point EP when the driver is substantially stationary. (hereinafter referred to as "eyepoint reference position") is acquired as state information of the DC component. The DC information processing unit 32 provides the vehicle reference attitude and the eyepoint reference position to the projection adjustment unit 37 and the correction amount calculation unit 38 as reference state values of specific parameters, details of which will be described later.

The AC information processing unit 33 uses the axle torque and brake force control information to obtain attitude change information of an AC (Alternate Current) component belonging to the second band LB2 among attitude changes occurring in the vehicle A. Specifically, the AC information processing unit 33 extracts vibration components of attitude changes (mainly pitch angle changes, etc.) belonging to the second band LB2 by passing the control information of the axle torque and brake force through a band pass filter. .

The brightness setting unit 34 grasps the required brightness required for the virtual image Vi. The brightness setting unit 34 acquires, for example, exposure information set by the camera unit of the external sensor 21 from the communication bus 29, and estimates the current brightness outside the vehicle (external light amount). The brightness setting unit 34 sets the required brightness of the virtual image Vi according to the estimated amount of external light. The brighter the outside of the vehicle, the higher the required brightness, and the darker the outside of the vehicle, the lower the required brightness. The brightness setting unit 34 sequentially notifies the HUD 70 of the required brightness that is suitable for the current amount of external light.

The content selection unit 35 selects and arbitrates content to be displayed as a virtual image based on various information obtained from the communication bus 29. Specifically, the content selection unit 35 comprehensively determines the display priority of each content, and selects information to be notified to the driver using the virtual image Vi. The content selection unit 35 sequentially provides the drawing data generation unit 39 with instructions for specifying the selected content and information necessary for drawing the selected content.

The target setting unit 36 acquires object information output by the external sensor 21 from the communication bus 29, and selects a superimposition target on which the virtual image Vi is to be superimposed from among moving objects and stationary objects. The object setting section 36 sequentially provides the relative positions of the selected superimposition objects to the drawing data generation section 39. As an example, when the road surface of the lane in which the driver is driving is set as the superimposition target and the virtual image Vi is superimposed on the road surface (see FIG. 5), the target setting unit 36 determines the relative positions of the left and right lane markings, and the drawing data generating unit 39 will be provided sequentially.

The projection adjustment unit 37 controls the position of the projection area PA defined on the windshield WS, and furthermore, the projection position of the light formed as the virtual image Vi, by controlling the attitude of the projection optical system 74 (described later) of the HUD 70. , to displace it in the vertical direction from the driver's perspective. The projection adjustment unit 37 acquires the vehicle reference attitude and eyepoint reference position provided by the DC information processing unit 32. The projection adjustment unit 37 determines the target attitude of the projection optical system 74 based on these reference state values. The correlation between the reference state value and the target posture is defined in advance at the time of design, and is stored in the storage unit 30b in a data format such as a correlation table or a correlation function. The projection adjustment unit 37 applies the reference state value to a correlation table or correlation function, and determines a target orientation corresponding to the reference state value. The projection adjustment unit 37 notifies the HUD 70 of the determined target posture as control information. In addition, the projection adjustment section 37 also notifies the drawing data generation section 39 and the display control ECU 40 of the determined target posture as information indicating the adjustment result of the projection optical system 74.

The correction amount calculation unit 38 specifies the current attitude of the vehicle A mainly using the attitude change information of the second band LB2 acquired by the AC information processing unit 33. The correction amount calculation unit 38 calculates a correction amount (hereinafter referred to as "low frequency correction amount") for correcting the display shift of the virtual image Vi from the specified current posture of the vehicle A. The correction amount calculation unit 38 sequentially provides the calculated low frequency correction amount value to the drawing data generation unit 39.

The correction amount calculation section 38 takes into consideration the adjustment result of the projection optical system 74 provided by the projection adjustment section 37 and the reference state value provided from the DC information processing section 32 when calculating the low frequency correction amount. For example, as will be described later, when the projection adjustment unit 37 interrupts adjustment of the projection optical system 74 to the target attitude corresponding to the reference state value, the correction amount calculation unit 38 adjusts the current attitude of the projection optical system 74 and , a low frequency correction amount is calculated in consideration of the vehicle reference attitude and the eyepoint reference position.

The drawing data generation section 39 is a 3D drawing section that generates video data. The drawing data generation section 39 is electrically connected to the display control ECU 40 via a video output line or the like. The drawing data generation unit 39 sequentially outputs video data for virtual image display, ie, input video data Vdi for the display control ECU 40, to the display control ECU 40 in a predefined video format.

The drawing data generation unit 39 draws each frame image (hereinafter referred to as "input image Pi") of the input video data Vdi. The drawing position and drawing shape of each drawn object within the frame of the input image Pi are controlled so that the formed virtual image Vi is visually recognized while correctly overlapping the superimposition target. Specifically, the drawing data generation unit 39 grasps the relative positions of the eye point EP, the projection area PA, and the superimposition target, and determines the projection position and projection of the light projected in the projection area PA based on these positional relationships. The shape etc. are calculated by geometric calculations.

In addition, the drawing data generation unit 39 electronically corrects the positional shift of the virtual image Vi caused by the posture change in the low frequency band LB. The correction amount calculation unit 38 reduces (cancels out) the positional deviation of the virtual image Vi caused by the posture change in the low frequency band LB (or second band LB2) based on the low frequency correction amount obtained from the correction amount calculation unit 38. The input image Pi is sequentially corrected so that The drawing data generation unit 39 continuously transmits the input image Pi, which has been corrected in advance for posture changes in the low frequency band LB, to the display control ECU 40 as input video data Vdi.

The display control ECU 40 is a control device that relays video data between the drawing ECU 30 and the HUD 70 and controls the display of the virtual image Vi by the HUD 70. The display control ECU 40 includes an inertial sensor 41, a filter circuit 42, an LCD (Liquid Crystal Display) control board 50, and the like.

The inertial sensor 41 is a measurement unit that measures changes in the attitude of the vehicle A. The inertial sensor 41 is a combination of a gyro sensor and an acceleration sensor. The inertial sensor 41 is mounted on the vehicle A as an attitude detection sensor separate from the height sensor 22, the vehicle control unit 24, and the like. The inertial sensor 41 measures each angular velocity in the pitch direction and the roll direction of the vehicle A, and the acceleration in the vertical direction along the yaw axis of the vehicle A, and transmits each measurement signal to the filter circuit 42. The inertial sensor 41 may be fixed to the LCD control board 50 or may be fixed to the metal casing of the display control ECU 40 or the like.

The filter circuit 42 is a signal processing circuit that processes measurement signals from the inertial sensor 41, and includes, for example, at least a high-pass filter, an integral processing section, and the like. The high-pass filter generally passes signals in the high frequency band HB and attenuates signals in the low frequency band LB and below (see FIG. 2). In addition, the high-pass filter removes drift components included in the measurement signal. The integral processing section has a configuration mainly composed of, for example, a low-pass filter, and generates a signal indicating the vehicle attitude (pitch angle, roll angle, etc.) by time-integrating a measurement signal indicating the angular velocity of attitude change. The filter circuit 42 acquires the signal that has passed through the high-pass filter and the integral processing section in order as attitude change information in the high frequency band HB, and sequentially provides the information to the LCD control board 50.

The LCD control board 50 is provided with a processing section 50a, a RAM, a storage section 50b, an input/output interface, and the like. A filter circuit 42, a RAM, a storage section 50b, a processing section 50a, and the like are connected to a data bus formed on the LCD control board 50 for high-speed access.

The storage unit 50b is mainly composed of a nonvolatile storage medium such as a ROM or a flash memory. The storage unit 50b holds data provided to the processing unit 50a. The storage unit 50b stores various programs executed by the processing unit 50a and a large number of parameters necessary for virtual image display. As one of these parameters, a distortion correction table TB (see FIG. 3), which will be described later, is stored in the storage unit 50b.

The processing unit 50a is hardware for image processing combined with RAM. The processing unit 50a corrects the input image Pi that constitutes each frame of the input video data Vdi, and draws the output image Po that constitutes each frame of the output video data Vdo. The processing unit 50a has a configuration that includes at least a memory controller that controls access to the RAM, and an arithmetic core such as a CPU and an FPGA. The processing unit 50a may further include a GPU and an IP core with other dedicated functions. The processing unit 50a includes functional units such as a drawing data acquisition unit 51, a posture information acquisition unit 52, and a correction execution unit 53.

The drawing data acquisition section 51 is electrically connected to the drawing data generation section 39. Each input image Pi of the input video data Vdi is sequentially transmitted to the drawing data acquisition section 51 from the drawing data generation section 39 . The drawing data acquisition unit 51 prepares the input image Pi in a correctable state by sequentially writing the acquired input image Pi into the RAM.

The attitude information acquisition unit 52 acquires attitude change information in the high frequency band HB (see FIG. 2) from the filter circuit 42 as information related to the attitude change of the vehicle A.

The correction execution unit 53 is a correction unit that corrects the input image Pi through 2D graphics processing that transforms the input image Pi two-dimensionally. The correction execution unit 53 is electrically connected to the HUD 70 via a video output line or the like. The correction execution unit 53 generates an output image Po by correcting the input image Pi, and continuously transmits it to the HUD 70 as output video data Vdo. The correction execution unit 53 electronically corrects the virtual image Vi using processing different from that of the drawing data generation unit 39. The correction execution unit 53 performs posture correction and distortion correction as corrections to the input image Pi.

The posture correction is a correction that reduces (cancels out) the positional deviation of the virtual image Vi from the superimposition target due to the posture change in the high frequency band HB. The correction execution unit 53 determines an extraction range to be used for the output image Po in the input image Pi. The correction execution unit 53 shifts the position of the extraction range mainly in the vertical direction of the input image Pi based on the posture change information acquired by the posture information acquisition unit 52.

Distortion correction takes into consideration the deformation of the virtual image Vi caused by reflection in the projection area PA and the projection optical system 74, and changes the shape of the output image Po so that the virtual image Vi is displayed with the distortion reduced (cancelled). This is a correction that deforms in advance. In the correction calculation for distortion correction, the distortion correction table TB shown in FIG. 3 is used as correction information.

The distortion correction table TB has a file configuration in which pre-correction coordinates and post-correction coordinates are set as a data set, and a large number of these data sets are arranged in succession. The corrected coordinates are information indicating the position of a pixel in the output image Po. The pre-correction coordinates are information that indicates the position of the pixel of the input image Pi that is assigned to the pixel of the post-correction coordinates, using the X and Y coordinates.

Specifically, the numerical values in the distortion correction table TB are arranged as follows: <X coordinate of post-correction coordinates>, <Y coordinate of post-correction coordinates>, <X coordinate of pre-correction coordinates>, <Y coordinate of pre-correction coordinates> It is said to be in order. For example, the group of numerical values in the first row of the distortion correction table indicates that the output image Po is an invalid pixel (black background) whose coordinates (0, 0) are (-1, -1). The group of numerical values in the second line indicates that pixel information (gradation value) of (0,1) of the input image Pi is assigned to (1,0) of the output image Po.

In addition, the distortion correction table TB shows the relationship between the attitude of the projection optical system 74 and the eyepoint EP. Therefore, the correction execution unit 53 shown in FIG. 1 acquires the adjustment result of the projection optical system 74 from the projection adjustment unit 37, and selects the distortion correction table TB whose contents correspond to the current posture of the projection optical system 74 from the storage unit 50b. can be read out.

As described above, the correction execution unit 53 acquires the distortion correction table TB whose contents correspond to the target posture set by the projection adjustment unit 37, and converts the two-dimensional array of pixels in the extraction range based on the posture correction into the distortion correction table TB. Sort based on table TB. Through such processing, the correction execution unit 53 generates an output image Po by transforming the extraction range of the input image Pi.

The HUD 70 shown in FIGS. 1 and 4 is an in-vehicle display device that is housed in a housing space provided in the instrument panel below the windshield WS. The light emitted from the HUD 70 toward the projection area PA of the windshield WS is reflected by the projection area PA toward the eyepoint EP and is perceived by the driver. The driver visually recognizes a display in which the virtual image Vi is superimposed on the foreground seen through the projection area PA. The HUD 70 forms a virtual image Vi in a space of about 10 to 20 meters in the front direction of the vehicle A from the eye point EP, for example.

In the HUD 70, an eye box EBX is set above the seat surface of the driver's seat and near the headrest. The eyebox EBX is a spatial area where the virtual image Vi can be visually recognized at a predetermined brightness or higher. The eye box EBX is a virtual area defined for each vehicle type, and is set to overlap with an eye range that statistically represents the distribution of the driver's eye positions.

The HUD 70 includes an LCD 71, a backlight 72, a backlight control section 73, a projection optical system 74, and an adjustment mechanism 75. Each optical element is housed in a housing 79 of the HUD 70, and the relative positional relationship is defined with high precision by the housing 79.

The LCD 71 has a display surface on which a large number of pixels are arranged. The LCD 71 displays various images in color on the display screen by increasing or decreasing the light transmittance of sub-pixels of red, green, blue, etc. provided in each pixel. The light transmittance in each sub-pixel is controlled by the LCD control board 50. On the display surface of the LCD 71, an output image Po based on the output video data Vdo is displayed.

The backlight 72 has a configuration including a large number of light emitting diodes. A large number of light emitting diodes are arranged on the back side of the LCD 71, and are two-dimensionally arranged at predetermined intervals along the display surface of the LCD 71. The backlight 72 is configured such that the luminance of the light emitted by each light emitting diode can be adjusted for each area illuminated by each light emitting diode. The backlight 72 transmits illumination of the LCD 71 from the back side, and causes the output image Po drawn on the display surface to be displayed by emitting light. As a result, the light of the output image Po is emitted from the display surface.

The backlight control section 73 mainly includes a drive circuit that drives each light emitting diode of the backlight 72. The backlight control unit 73 controls the luminance of the backlight 72 for each area by partially driving each light emitting diode to emit light individually (local dimming). The backlight control unit 73 acquires the output video data Vdo from the LCD control board 50, and determines the light emission brightness of each area in the backlight 72 according to the position and shape of the drawn object in each output image Po. In addition, the backlight control unit 73 adjusts the brightness of each area according to the amount of external light based on the required brightness notified by the brightness setting unit 34.

The projection optical system 74 includes, for example, a plane mirror 74a and a concave mirror 74b, and projects the light of the output image Po, which is formed as a virtual image Vi, onto the projection area PA. Each reflecting mirror is made by depositing a metal such as aluminum on the surface of a colorless and transparent base material made of synthetic resin, glass, or the like. The projection optical system 74 spreads the light of the virtual image Vi emitted from the LCD 71 by reflection and projects it onto the projection area PA, thereby displaying a virtual image Vi formed by enlarging the output image Po. The projection optical system 74 may include, for example, a diffractive optical element (DOE) that enlarges the output image Po by diffraction.

The adjustment mechanism 75 is a mechanism that controls the state of the projection optical system 74 in cooperation with the projection adjustment section 37. The adjustment mechanism 75 is configured integrally with the concave mirror 74b of the projection optical system 74, and mechanically adjusts the attitude of the concave mirror 74b. The adjustment mechanism 75 moves the position of the projection area PA on the windshield WS in the vertical direction by rotating the concave mirror 74b around the rotation axis Ax. The adjustment mechanism 75 includes a movable support portion 75a, a potentiometer 76, a stepper motor 77, a controller 78, and the like.

The movable support portion 75a supports the concave mirror 74b with respect to the housing 79 in a rotatable state around the rotation axis Ax. The potentiometer 76 is a rotation angle sensor mainly composed of a variable resistor. The potentiometer 76 detects the current rotation angle of the movable support portion 75a and the concave mirror 74b, and outputs an output to the controller 78 according to the rotation angle. The stepper motor 77 is a drive section that rotationally drives the movable support section 75a. The stepper motor 77 rotates the movable support portion 75a and the concave mirror 74b around the rotation axis Ax according to a drive signal from the controller 78.

The controller 78 is a drive control section that controls the operation of the stepper motor 77. The controller 78 sets the rotation angles of the movable support section 75a and the concave mirror 74b based on the target posture determined by the projection adjustment section 37. The controller 78 controls the drive of the stepper motor 77 while referring to the output of the potentiometer 76, so that the rotation angle of the concave mirror 74b matches the target posture.

Next, details of the optical correction by cooperation between the projection adjustment section 37 and the adjustment mechanism 75 will be explained. Optical correction is performed to reduce changes in the displayable area SA of the virtual image Vi. The displayable area SA is a foreground range that overlaps the projection area PA when viewed from the eyepoint EP. The virtual image Vi can be superimposed only on objects in the displayable area SA. The displayable area SA changes mainly due to changes in the attitude of the vehicle A in the pitch direction and changes in the position of the eyepoint EP. Therefore, the pitch attitude of the vehicle A and the eye point EP are the main "specific parameters" that change the displayable area SA. As described above, the vehicle reference attitude and eyepoint reference position acquired by the DC information processing unit 32 serve as the "reference state value" that serves as the reference for the specific parameter.

The projection adjustment unit 37 acquires the reference state value of the specific parameter, and sets the target attitude of the projection optical system 74 in accordance with the reference state value so as to reduce the change in the displayable area SA as seen by the driver. . The projection adjustment unit 37 adjusts the projection optical system 74 and moves the projection area PA by controlling the adjustment mechanism 75 based on the target orientation. By changing the attitude of the projection optical system 74, the vertical position of the projection area PA is adjusted.

As an example, if the eye point EP is higher than the standard position (EPh in Figure 5) or if the front side of the vehicle is in a sunken position, the displayable area SA will move in the direction closer to the vehicle A if no correction is performed. do. Therefore, the projection adjustment unit 37 shifts the projection area PA in the windshield WS upward in order to reduce the change in the displayable area SA. According to the above, since the imaging position of the virtual image Vi also moves upward, the road surface range overlapping with the virtual image Vi seen from the eye point EP is maintained.

On the other hand, if the eye point EP is lower than the standard position (Fig. 5 EPl) or if the rear of the vehicle is in a sunken position, the displayable area SA will move away from the vehicle A unless correction is performed. . Therefore, the projection adjustment unit 37 shifts the projection area PA in the windshield WS downward in order to reduce the change in the displayable area SA. According to the above, since the imaging position of the virtual image Vi also moves downward, the road surface range overlapping with the virtual image Vi seen from the eye point EP is maintained.

Here, the movement of the projection area PA to maintain the displayable area SA may move the center of the eyebox EBX (hereinafter referred to as the "eyebox center") away from the eyepoint EP. Therefore, when performing control to maintain the optically displayable area SA, when the eye point EP deviates from the initial reference position due to a change in the driver's posture, the virtual image Vi will noticeably decrease in brightness and be cut off. etc. are likely to occur.

Therefore, the projection adjustment section 37 adjusts the projection optical system 74 while taking into consideration the optical disadvantages as described above. Specifically, when it is difficult to fix the displayable area SA due to the optical performance of the HUD 70, the projection adjustment unit 37 prevents the important range SAi of the displayable area SA from deviating from the displayable area SA. Next, the attitude of the projection optical system 74 is controlled. The important range SAi is defined in advance as an essential range for the driver to understand the meaning when presenting information using the virtual image Vi. The projection adjustment section 37 can adjust the position of the important range SAi according to the content selected by the content selection section 35. Further, as an example, the projection adjustment unit 37 controls the position of the projection area PA so that the angle of depression from the eye point EP to the displayable area SA is constant, so that the important range SAi remains within the displayable area SA. It's okay.

In addition, the projection adjustment unit 37 interrupts control of the projection optical system 74 that maintains the displayable area SA according to the required brightness determined by the brightness setting unit 34. Specifically, the projection adjustment unit 37 determines whether the required brightness exceeds a threshold value. When the required brightness exceeds the threshold, the projection adjustment unit 37 interrupts the maintenance control of the displayable area SA, and adjusts the attitude of the projection optical system 74 so that the center of the eyebox coincides with the eyepoint reference position. Switch to brightness priority control.

In order to realize the virtual image display described above, details of a series of virtual image drawing processes performed by each component of the virtual image display system 10 will be explained based on FIG. 6 and with reference to FIGS. 1, 2, 5, etc. do. The virtual image drawing process shown in FIG. 6 is started, for example, when the vehicle power source is turned on, and is repeatedly executed until the vehicle power source is turned off.

The processes of S101 to S103 are executed by the projection adjustment section 37 of the drawing ECU 30. In S101, the required brightness set by the brightness setting unit 34 is compared with a threshold value. If it is determined in S101 that the required brightness is higher than the threshold value, the process advances to S102. In S102, the target orientation of the projection optical system 74 is set by brightness priority control so that the center of the eyebox coincides with the eyepoint reference position. Then, the adjustment mechanism 75 is notified of the set target posture, and the process proceeds to S104. Based on the target orientation notified in S102, the adjustment mechanism 75 adjusts the rotation angle of the projection optical system 74.

On the other hand, if it is determined in S101 that the required brightness is less than or equal to the threshold value, the process advances to S103. In S103, as maintenance control of the displayable area SA, the projection optical system 74 is adjusted so as to reduce changes in the displayable area SA. In S103, the projection optical system 74 is adjusted so as to reduce changes in the displayable area SA within a range that does not cause the virtual image Vi to be cut off, while taking into account the distance between the eyepoint reference position and the center of the eyebox. , proceed to S104.

Each process of S104 and S105 is mainly executed by the drawing data generation unit 39 of the drawing ECU 30. In S104, the eyepoint reference position and the target posture set by the projection adjustment unit 37 are set to be reflected in the drawing of the input image Pi, and the process proceeds to S105.

In S105, the posture change information indicating the posture change in the low frequency band LB is further reflected in the drawing of the input image Pi, and the process proceeds to S106. If it is determined in S101 that the required brightness is below the threshold, changes in the vehicle reference posture due to the number of occupants and loaded weight, and changes in the eyepoint reference position due to physical differences are controlled by maintenance control in S103. Optically corrected. On the other hand, if it is determined in S101 that the required brightness exceeds the threshold value, the positional deviation correction of the optical virtual image Vi is not performed. Therefore, in S105 in this case, variations in the vehicle reference posture and eyepoint reference position are electronically corrected in drawing the input image Pi.

Each process of S106 to S108 is mainly executed by the correction execution unit 53 of the display control ECU 40. In S106, the attitude change information indicating the attitude change in the high frequency band HB is reflected in the attitude correction of the output image Po, and the process proceeds to S107. In S107, the reference position of the eyepoint EP and the target orientation set by the projection adjustment unit 37 are reflected in the distortion correction of the output image Po, and the process proceeds to S108. In S108, the output image Po, which has undergone posture correction and distortion correction, is transmitted to the HUD 70 as output video data Vdo. As described above, the output image Po is displayed by emitting light on the screen of the LCD 71, and is formed as a virtual image Vi in the foreground.

In the first embodiment described so far, even if the reference state value for the specific parameter changes, the change in the displayable area SA is reduced by adjusting the projection optical system 74 by the projection adjustment section 37. If the movement of the displayable area SA is suppressed in this way, the number of scenes in which the virtual image Vi cannot be superimposed on the superimposition target on which the virtual image Vi should be superimposed can be reduced. According to the above, it becomes possible to adjust the projection optical system 74 suitable for presenting information to superimpose the virtual image Vi on the superimposition target.

In addition, in the first embodiment, the viewer's eye point EP and the pitch attitude of the vehicle A are employed as specific parameters. If the displayable area SA can be optically corrected when the eyepoint EP and pitch posture, which greatly affect the displayable area SA, change as described above, the scenes in which the virtual image Vi cannot be superimposed can be further reduced.

Further, in each of the ECUs 30 and 40 of the first embodiment, the AC information processing section 33 and the attitude information acquisition section 52 each function as an attitude acquisition section, and acquire attitude change information obtained by detecting an attitude change occurring in the vehicle A. Furthermore, in each ECU 30, 40, the drawing data generation section 39 and the correction execution section 53 each function as a display correction section, and electronically correct the positional shift of the virtual image Vi caused by each posture change.

As described above, in the first embodiment, in addition to optical correction for maintaining the displayable area SA, the positional shift of the virtual image Vi caused by vehicle vibration or the like is electronically corrected. Therefore, it is possible to accurately track the virtual image Vi to the superimposition target while making it possible to superimpose the virtual image Vi on the superimposition target.

Further, in the first embodiment, while the electronic correction for posture changes in the low frequency band LB is performed by the drawing data generation unit 39, the correction execution unit 53 performs high Electronically correct posture changes in frequency band HB. As described above, if the vibrations in the low frequency band LB, which has a large amplitude, are corrected during drawing, and the vibrations in the high frequency band HB are further electronically corrected separately, the virtual image Vi can be superimposed on the superimposition target with even more precision. become.

In addition, as in the first embodiment, when the projection optical system 74 that maintains the displayable area SA is adjusted, the center of the eyebox, where it is easy to ensure the brightness of the virtual image Vi, may move away from the eyepoint EP. In this case, visibility of the virtual image Vi may deteriorate due to insufficient brightness. Therefore, when the required brightness of the virtual image Vi is high, the projection adjustment unit 37 interrupts the maintenance control of the displayable area SA by adjusting the projection optical system 74. Then, the drawing data generation unit 39 electronically corrects the positional deviation of the virtual image Vi caused by changes in the vehicle reference attitude and the eyepoint reference position. According to the above, a situation in which information presentation using the virtual image Vi becomes difficult to understand due to insufficient brightness due to maintenance control of the displayable area SA can be avoided.

Further, in the first embodiment, when the required brightness of the virtual image Vi is high, the projection adjustment unit 37 adjusts the adjustment mechanism 75 so that the center of the eyebox EBX overlaps the eyepoint EP. Therefore, during the interruption period of the maintenance control of the displayable area SA, the brightness of the virtual image Vi can be ensured sufficiently.

Furthermore, in the LCD control board 50 of the first embodiment, a distortion correction table TB for electronic correction is stored in the storage unit 50b, and the distortion correction table TB whose contents correspond to the adjustment results of the projection optical system 74 is corrected. The information is provided to the execution unit 53. Therefore, the correction execution unit 53 can correct the distortion of the virtual image Vi in consideration of the angle of the concave mirror 74b, the eye point reference position, and the like.

In the first embodiment, the drawing ECU 30 corresponds to a "display control device", the DC information processing section 32 corresponds to a "state value acquisition section", and the AC information processing section 33 corresponds to a "first acquisition section" and " This corresponds to the “Attitude Acquisition Unit”. Further, the brightness setting unit 34 corresponds to a “brightness grasping unit”, the drawing data generating unit 39 corresponds to a “first correction unit” and a “display correction unit”, and the posture information acquisition unit 52 corresponds to a “second acquisition unit”. and a "posture acquisition section," and the correction execution section 53 corresponds to a "second correction section" and a "display correction section." Further, the distortion correction table TB corresponds to "correction information", and the windshield WS corresponds to a "projection member".

(Second embodiment)
The second embodiment shown in FIG. 7 is a modification of the first embodiment. The virtual image display system 210 of the second embodiment includes a drawing ECU 30, a HUD 270, and the like. In the virtual image display system 210, a configuration corresponding to the display control ECU 40 (see FIG. 1) of the first embodiment is omitted. In the virtual image display system 210, the drawing ECU 30 electronically corrects vehicle vibrations in a high frequency band HB (see FIG. 2) in addition to vehicle vibrations in a low frequency band LB (see FIG. 2).

A functional unit corresponding to the brightness setting unit 34 (see FIG. 1) is omitted from the drawing ECU 30 of the second embodiment. On the other hand, in the drawing ECU 30, the functions of the AC information processing section 233 and the drawing data generation section 239 are different from those in the first embodiment. The AC information processing unit 233 acquires attitude change information of vehicle vibration that is in the high frequency band HB in addition to attitude change information of vehicle vibration that is in the second band LB2 (see FIG. 2).

The drawing data generation unit 239 draws the output image Po to be transmitted to the HUD 270 while reflecting the adjustment result of the projection optical system 74 by the projection adjustment unit 37. The drawing data generation unit 239 generates both the low frequency band LB and the high frequency band HB in the process of generating the output image Po as 3D graphics based on the posture change information provided from the AC information processing unit 233. Electronically corrects display position deviations caused by vehicle vibration.

The HUD 270 further includes an external light sensor 171 and a brightness setting section 172. The outside light sensor 171 detects the amount of outside light that enters the vehicle interior through the windshield WS. The brightness setting unit 172 determines the required brightness required for the virtual image Vi based on the detection result of the external light sensor 171. The brightness setting section 172 sets the required brightness of the virtual image Vi, and notifies each of the LCD 71, the backlight control section 73, and the controller 78.

Next, details of control of the adjustment mechanism 75 in the virtual image display system 210 will be explained.

The projection adjustment unit 37 cooperates with the controller 78 and switches the content of attitude control of the projection optical system 74 based on the required brightness notified by the brightness setting unit 172. When the required brightness is less than or equal to the threshold (see S103 in FIG. 6), the projection adjustment unit 37 and the controller 78 match the rotation angle of the concave mirror 74b (see FIG. 4) to the target posture based on the reference state value. In this case, control is performed to reduce changes in the displayable area SA (see FIG. 5).

On the other hand, when the required brightness is higher than the threshold (see FIG.
(see S102), the maintenance control of the displayable area SA as described above is canceled and switched to brightness priority control. Specifically, the distance between the eyepoint EP and the center of the eyebox is controlled according to the required brightness. If the rotation angle of the concave mirror 74b where the eye point EP and the center of the eye box match is the maximum brightness attitude, the projection adjustment unit 37 and the controller 78 set the concave mirror 74b to a rotation angle between the target attitude and the maximum brightness attitude. Control. The projection adjustment unit 37 and the controller 78 move the rotation angle of the concave mirror 74b closer to the maximum brightness orientation as the required brightness notified from the brightness setting unit 172 becomes higher. As a result, the center of the eyebox comes closer to the eyepoint EP as the required brightness increases, and when the required brightness is higher than a specific required brightness, the center of the eyebox matches the eyepoint EP.

In the second embodiment described so far, the adjustment of the projection optical system 74 is performed, which produces the same effects as the first embodiment and is suitable for presenting information to superimpose the virtual image Vi on the superimposition target. Additionally, in the second embodiment, when the required brightness exceeds the threshold, the higher the required brightness, the closer the eyebox center is to the eyepoint EP. According to such control, it becomes possible to maintain the displayable area SA while ensuring the visibility of the virtual image Vi.

Further, in the second embodiment, electronic correction for correcting a display position shift due to vehicle vibration is entirely performed by the drawing data generation unit 239 without dividing the frequency bands. According to the above system configuration, the configuration corresponding to the display control ECU 40 (see FIG. 1) is omitted. As a result, the virtual image display system 210 can be simplified.

In the second embodiment, the AC information processing section 233 corresponds to an "attitude acquisition section," the drawing data generation section 239 corresponds to a "display correction section," and the projection adjustment section 37 and the controller 78 correspond to a "projection adjustment section." ”, and the brightness setting unit 172 corresponds to a “brightness understanding unit”.

(Other embodiments)
Although multiple embodiments of the present disclosure have been described above, the present disclosure is not to be construed as being limited to the above embodiments, and may be applied to various embodiments and combinations within the scope of the gist of the present disclosure. can do.

In the first modification of the second embodiment, the virtual image electronic correction function in the drawing ECU is omitted. In the virtual image display system of Modification 1, control for causing the virtual image to follow the superimposition target is not performed due to vehicle vibration correction. In modification example 1, among the correction processes related to the virtual image, only optical correction for adjusting the projection area in accordance with the vehicle reference attitude and the eyepoint reference position is performed.

In the second modification of the above embodiment, switching to brightness priority control based on required brightness is not performed. As in Modification 2, even if the required brightness of the virtual image is high, as long as the virtual image brightness at the eyepoint reference position and its vicinity can be ensured, maintenance control of the displayable area may be continued. Note that the functional unit that determines the required brightness of the virtual image may be provided in any configuration of the virtual image display system. Further, the specific configuration for detecting the amount of external light can be changed as appropriate. Furthermore, the details of the brightness priority control can also be changed as appropriate.

In the third modification of the above embodiment, only the pitch attitude of the vehicle is used as the specific parameter. The projection adjustment section controls the state of the projection optical system in cooperation with the adjustment mechanism so as to maintain the displayable area based on the vehicle reference attitude acquired as the reference state value by the DC information processing section. Further, in the fourth modification of the above embodiment, only the eye point is used as the specific parameter. The projection adjustment section controls the state of the projection optical system in cooperation with the adjustment mechanism so as to maintain the displayable area based on the eyepoint reference position acquired as the reference state value by the DC information processing section.

In the fifth modification of the first embodiment, the correction execution unit uses a correction function as correction information for distortion correction instead of the distortion correction table TB (see FIG. 3). The correction function is predefined as a polynomial or a monomial that takes into account the projection area and the curved shape of the optical reflection system. In the correction function, the post-correction coordinates (X, Y) are used as input parameters, and the pre-correction coordinates (X, Y) are used as output parameters. The correction execution unit specifies the pre-correction coordinates (X, Y) assigned to each post-correction coordinate by sequentially substituting the post-correction coordinates (X, Y) into the correction function.

The drawing ECU in the embodiment described above may be configured integrally with other vehicle-mounted configurations. Specifically, in the sixth modification of the above embodiment, the drawing ECU is integrally configured with the display control ECU. That is, in the sixth modification, a function of correcting vehicle vibrations in a low frequency band by 3D drawing and a function of correcting vehicle vibrations in a high frequency band in 2D are implemented in one display control device. Furthermore, in a seventh modification of the above embodiment, all functions of the drawing ECU, display control ECU, and HUD of the first embodiment are integrated into one virtual image display unit.

The specific projection configuration of the HUD that displays the virtual image in the air may be changed as appropriate. For example, the HUD of modification 8 is provided with an EL (Electro Luminescence) panel instead of the LCD and backlight. Furthermore, instead of the EL panel, a projector using a display device such as a plasma display panel, a cathode ray tube, or an LED can be used as the HUD.

Further, the HUD of Modification Example 9 is provided with a laser module (hereinafter referred to as "LSM") and a screen instead of the LCD and backlight. The LSM has a configuration including, for example, a laser light source and a MEMS (Micro Electro Mechanical Systems) scanner. The screen is, for example, a micromirror array or a microlens array. In the HUD of modification 9, an output image is drawn on the screen by scanning with laser light emitted from the LSM. The HUD projects an output image projected onto a screen onto a projection area using a reflective optical system, thereby displaying a virtual image in the air.

Further, the HUD of Modification 10 is provided with a DLP (Digital Light Processing, registered trademark) projector. A DLP projector includes a digital mirror device (hereinafter referred to as "DMD") provided with a large number of micromirrors, and a projection light source that projects light toward the DMD. A DLP projector draws an output image on a screen by controlling a DMD and a projection light source in coordination. Furthermore, the HUD of modification 11 employs a projector using LCOS (Liquid Crystal On Silicon).

In a twelfth variation of the above embodiment, the light of the output image is projected onto a vehicle-mounted arrangement other than the windshield. Specifically, in Modification 13, the light of the output image is projected onto a projection member (for example, a combiner, etc.) provided separately from the windshield.

In the thirteenth modification of the first embodiment, each of the low frequency band and the high frequency band is different from the first embodiment. Furthermore, in the modification 14 of the first embodiment, electronic correction for vehicle vibrations in the low frequency band and the high frequency band is performed by the correction execution unit of the display control ECU. Furthermore, the inertial sensor of Modification 15 of the first embodiment is configured to include only one of an acceleration sensor and a gyro sensor. Furthermore, in the sixteenth modification of the first embodiment, the posture information acquisition section of the display control ECU acquires posture change information from the communication bus, similarly to the sensor value acquisition section of the drawing ECU.

The specific configuration of each processing section in the display control ECU and LCD control board can be changed as appropriate. As described above, the processing unit may have a configuration in which arithmetic cores such as a CPU, a GPU, an FPGA, and other dedicated IPs are combined as appropriate. Furthermore, the processing unit may be in the form of an SoC (System-on-a-Chip) that includes a large number of computing cores, or may be in the form of a plurality of physically independent computing cores that are individually mounted on a circuit board. It may be.

Various programs (display control programs) that construct each functional section in the processing section may be ordinary software that instructs the CPU, GPU, etc. to process contents, or may be hardware that allows the FPGA to function as an arbitrary hardware logic circuit. It may also be a software program.

Various non-transitory tangible storage media, such as a flash memory and a hard disk, can be used as the storage units of the display control ECU and the LCD control board. In addition, the storage medium that stores programs and parameters related to virtual image display is not limited to a storage medium mounted on a circuit board, but may be in the form of a memory card, etc., and is inserted into a card slot section to store data. It may be configured to be electrically connected to a bus. Furthermore, the storage medium may be an optical disk, a hard disk drive of a general-purpose computer, etc. that serves as a copy source to a display control ECU, LCD control board, etc.

The control unit and techniques described in this disclosure may be implemented by a special purpose computer comprising a processor programmed to perform one or more functions embodied by a computer program. Alternatively, the apparatus and techniques described in this disclosure may be implemented with dedicated hardware logic circuits. Alternatively, the apparatus and techniques described in this disclosure may be implemented by one or more special purpose computers configured by a combination of a processor executing a computer program and one or more hardware logic circuits. The computer program may also be stored as instructions executed by a computer on a computer-readable non-transitory tangible storage medium.
The technical ideas understood from the embodiments and modified examples described so far will be described below as "additional notes".
(Appendix 1-1)
A virtual image display system that is used in a vehicle (A) and displays a virtual image (Vi) superimposed on a superimposition target in the foreground,
a projection optical system (74) that projects light to be imaged as the virtual image onto a projection member (WS) of the vehicle;
a state value acquisition unit (32) that acquires a reference state value serving as a reference for at least one specific parameter that changes a displayable area (SA) on which the virtual image can be superimposed in the foreground;
a projection adjustment unit (37) that adjusts the projection optical system so as to reduce changes in the displayable area caused by changes in the reference state value;
a first acquisition unit (33) that acquires posture change information in a low frequency band (LB) among posture changes occurring in the vehicle;
a second acquisition unit (52) that acquires attitude change information of a high frequency band (HB) having a higher frequency than the low frequency band among the attitude changes of the vehicle;
a first correction unit (39) that electronically corrects a positional shift of the virtual image caused by a posture change in the low frequency band while reflecting an adjustment result of the projection optical system;
a second correction unit (53) that corrects a positional shift of the virtual image caused by a posture change in the high frequency band by a process different from that of the first correction unit while reflecting an adjustment result of the projection optical system; A virtual image display system comprising:
(Appendix 1-2)
further comprising a luminance grasping unit (34, 172) that grasps the required luminance required for the virtual image,
According to supplementary note 1, the first correction unit electronically corrects a positional deviation of the virtual image caused by a change in the reference state value, instead of adjusting the projection optical system, when the required brightness exceeds a threshold value. virtual image display system.
(Appendix 1-3)
The projection adjustment unit is configured to adjust the projection adjustment unit so that, when the required brightness exceeds the threshold, the center of an eye box (EBX) that is a range in which the virtual image can be visually recognized overlaps with an eye point (EP) of a viewer of the virtual image. The virtual image display system according to appendix 2, wherein the projection optical system is adjusted.
(Appendix 1-4)
The projection adjustment unit adjusts the projection optical system so that the center of an eye box (EBX), which is a range in which the virtual image can be viewed, approaches an eye point (EP) of a viewer of the virtual image as the required brightness increases. The virtual image display system according to appendix 2, which adjusts the virtual image display system.
(Appendix 1-5)
Any of Supplementary Notes 1 to 4, further comprising a storage unit (50b) that stores correction information (TB) used for electronic correction of the virtual image and provides the correction information with content corresponding to the adjustment result of the projection optical system. The virtual image display system according to item 1.
(Appendix 1-6)
The virtual image display system according to any one of appendices 1 to 5, wherein the specific parameter includes at least one of an eye point (EP) of a viewer of the virtual image and a posture of the vehicle in a pitch direction.
(Appendix 1-7)
A display control device that is used in a vehicle (A) and controls the display of a virtual image (Vi) superimposed on a superimposition target in the foreground,
a state value acquisition unit (32) that acquires a reference state value serving as a reference for at least one specific parameter that changes a displayable area (SA) on which the virtual image can be superimposed in the foreground;
Controlling the state of a projection optical system (74) that projects light to be imaged as the virtual image onto the projection member (WS) of the vehicle, and reducing changes in the displayable area caused by fluctuations in the reference state value. a projection adjustment section (37) that adjusts the projection optical system;
a first acquisition unit (33) that acquires posture change information in a low frequency band (LB) among posture changes occurring in the vehicle;
a second acquisition unit (52) that acquires attitude change information of a high frequency band (HB) having a higher frequency than the low frequency band among the attitude changes of the vehicle;
a first correction unit (39) that electronically corrects a positional shift of the virtual image caused by a posture change in the low frequency band while reflecting an adjustment result of the projection optical system;
a second correction unit (53) that corrects a positional shift of the virtual image caused by a posture change in the high frequency band by a process different from that of the first correction unit while reflecting an adjustment result of the projection optical system; A display control device comprising:
(Appendix 1-8)
A display control program used in a vehicle (A) to control display of a virtual image (Vi) superimposed on a superimposition target in the foreground,
At least one processing unit (50a),
a state value acquisition unit (32) that acquires a reference state value serving as a reference for at least one specific parameter that changes a displayable area (SA) on which the virtual image can be superimposed in the foreground;
Controlling the state of a projection optical system (74) that projects light to be imaged as the virtual image onto the projection member (WS) of the vehicle, and reducing changes in the displayable area caused by fluctuations in the reference state value. a projection adjustment section (37) that adjusts the projection optical system;
a first acquisition unit (33) that acquires posture change information in a low frequency band (LB) among posture changes occurring in the vehicle;
a second acquisition unit (52) that acquires attitude change information of a high frequency band (HB) having a higher frequency than the low frequency band among the attitude changes of the vehicle;
a first correction unit (39) that electronically corrects a positional shift of the virtual image caused by a posture change in the low frequency band while reflecting an adjustment result of the projection optical system;
a second correction unit (53) that corrects a positional shift of the virtual image caused by a posture change in the high frequency band by a process different from that of the first correction unit while reflecting an adjustment result of the projection optical system; A display control program that functions to include.

A Vehicle, EP Eyepoint, EBX Eyebox, HB High frequency band, LB Low frequency band, SA Displayable area, TB Distortion correction table (correction information), Vi Virtual image, WS Windshield (projection member), 10,210 Virtual image Display system, 30 Drawing ECU (display control device), 32 DC information processing unit (state value acquisition unit), 33,233 AC information processing unit (attitude acquisition unit, first acquisition unit), 34,172 Brightness setting unit (brightness grasping unit), 37 projection adjustment unit, 39, 239 drawing data generation unit (display correction unit, first correction unit), 50a processing unit, 50b storage unit, 52 posture information acquisition unit (posture acquisition unit, second acquisition unit) , 53 correction execution unit (display correction unit, second correction unit), 74 projection optical system

Claims (3)

A virtual image (Vi ), a display control device that controls the display of
a first acquisition unit (33) that acquires posture change information in a low frequency band (LB) among posture changes occurring in the vehicle;
a second acquisition unit (52) that acquires attitude change information of a high frequency band (HB) having a higher frequency than the low frequency band among the attitude changes of the vehicle;
a first correction unit (39) that electronically corrects a positional shift of the virtual image caused by a posture change in the low frequency band;
a second correction unit (53) that corrects a positional shift of the virtual image caused by a posture change in the high frequency band by a process different from that of the first correction unit ;
The second correction unit is a display control device that determines an extraction range to be used for an output image (Po) in the electronically corrected input image (Pi) based on the posture change information in the high frequency band. . The display control device according to claim 1, wherein the second correction unit generates the output image subjected to distortion correction by rearranging a two-dimensional array of pixels in the extraction range. A virtual image (Vi ), a display control program that controls the display of
at least one processing unit,
a first acquisition unit (33) that acquires posture change information in a low frequency band (LB) among posture changes occurring in the vehicle;
a second acquisition unit (52) that acquires attitude change information of a high frequency band (HB) having a higher frequency than the low frequency band among the attitude changes of the vehicle;
a first correction unit (39) that electronically corrects a positional shift of the virtual image caused by a posture change in the low frequency band;
a second correction unit (53) that corrects a positional shift of the virtual image caused by a posture change in the high frequency band by a process different from that of the first correction unit;
The second correction unit is a display control program that determines an extraction range to be used for an output image (Po) in the electronically corrected input image (Pi) based on the attitude change information in the high frequency band. .

Images