JP2020050328A - Display control device and display control program - Google Patents

Abstract

To provide a display control device or the like capable of displaying a virtual image that suppresses distortion even though a projection position is changed so as to correspond to attitude change of a vehicle.SOLUTION: A HUD 100 is used in a vehicle A, and has a function of a display control device for controlling display of a virtual image Vi. The virtual image Vi is displayed by projecting light to a projection area PA. The HUD 100 prepares an AR drawing object DS1 of an AR display object 11 to be displayed as a virtual image in association with a superposition object existing in a foreground, and acquires attitude information related to the attitude of the vehicle A. Then, contents of attitude correction performed to the AR drawing object DS1 is set by using the attitude information so as to reduce display displacement to the superposition object of the AR display object 11 due to the attitude change of the vehicle A. Further, contents of distortion correction to be performed to the AR drawing object DS1 is set after setting the contents of the attitude correction so as to reduce distortion of the virtual image Vi due to the projection of light to the projection area PA.SELECTED DRAWING: Figure 1

Description

  The disclosure according to this specification relates to a display control device and a display control program for controlling display of a virtual image.

  BACKGROUND ART Conventionally, for example, Patent Document 1 discloses a virtual image generation system in which a virtual image of an arrow indicating a traveling direction of a vehicle is superimposed and displayed on a road surface in front of the vehicle. This virtual image generation system determines the projection position of the image on the screen, the image of the front camera, and the operation information of the suspension so that the superimposed display of the virtual image of the arrow on the road surface is maintained even if the pitch of the vehicle changes. Calculate using Information indicating the display position of the video calculated in this way is transmitted to a head-up display (hereinafter, “HUD”). As a result, the HUD can change the projection position of the image formed as the virtual image of the arrow according to the change in the pitch of the vehicle.

JP 2017-94882 A

  In Patent Literature 1, after data of a video image formed as a virtual image is generated by the CPU, control for changing the projection position of the video image to a position corresponding to the change in the pitch of the vehicle is performed by the HUD. According to such control, while the followability of the virtual image to the road surface is easily ensured, the video is projected to a position that is not assumed when data is generated. As a result, for example, distortion due to the curved shape of the screen or the like may easily occur in the virtual image. Such distortion may occur even when a virtual image indicating the vehicle speed is further displayed in addition to the virtual image of the arrow indicating the traveling direction.

  An object of the present disclosure is to provide a display control device capable of displaying a virtual image in which distortion is suppressed even when correction corresponding to a change in the attitude of a vehicle is performed.

  In order to achieve the above object, one disclosed aspect is a display control which is used in a vehicle (A) in which a virtual image (Vi) is displayed by projecting light onto a projection area (PA), and controls display of the virtual image. A drawing data preparation unit (51) for preparing specific drawing data (DS1) of a specific display object (11) that is displayed as a virtual image in association with a specific object in a foreground of a vehicle; A posture information acquisition unit (52) for acquiring posture information to be performed, and performing specific image data on specific drawing data using the posture information so as to reduce display displacement of a specific display object with respect to a specific object due to a change in vehicle posture. A posture correction setting unit (53) for setting the contents of the posture correction, and after the contents of the posture correction are set by the posture correction setting unit so as to reduce the distortion of the virtual image accompanying the projection of light onto the projection area. , Specific drawing data It is a display control device including a distortion correction setting unit (54) to set the contents of the distortion correction performed against.

  One embodiment disclosed is a display control program used in a vehicle (A) in which a virtual image (Vi) is displayed by projecting light onto a projection area (PA), and controlling display of the virtual image. A drawing data preparation unit (51) that prepares specific drawing data (DS1) of a specific display object (11) displayed as a virtual image by associating one processor (50a, 550a, 650a) with a specific object in a foreground of a vehicle; A posture information acquisition unit (52) for acquiring posture information related to the posture of the vehicle, and a posture information using the posture information so as to reduce a deviation of the specific display object from the specific object due to a change in the posture of the vehicle. A posture correction setting unit (53) for setting the contents of the posture correction to be performed on the drawing data, and a posture correction setting unit for reducing the distortion of the virtual image accompanying the projection of light onto the projection area. After positive content is set, the display control program to function as a structure including, a distortion correction setting unit (54) to set the contents of the distortion correction to be performed on a particular drawing data.

  In these aspects, after the content of the posture correction performed on the specific drawing data is set so that the display shift of the specific display object with respect to the specific object is reduced, the content of the distortion correction for reducing the distortion of the virtual image is changed. , Are further set. Therefore, even if the correction corresponding to the change in the attitude of the vehicle is performed after the preparation of the specific drawing data, the virtual image based on the specific drawing data can be displayed in a mode in which the distortion is suppressed.

  Further, one embodiment disclosed is a display control device that is used in a vehicle (A) in which a virtual image (Vi) is displayed by projecting light onto a projection area (PA) and controls display of the virtual image, Specific drawing data (DS1) of a specific display object (11) displayed as a virtual image in association with a specific object in the foreground of the vehicle, and separate drawing data (DS2, DS3) of another display object (12) different from the specific display object ), A posture information acquisition unit (52) for acquiring posture information related to the posture of the vehicle, and a display displacement of the specific display object with respect to the specific object due to the change in the posture of the vehicle is reduced. The posture correction setting unit (53) that sets the contents of the posture correction performed on the specific drawing data using the posture information, and the posture correction of the contents set by the posture correction setting unit are applied. Specific drawing An image synthesizing unit (55) for synthesizing the data and the separate drawing data; and a distortion applied to the specific drawing data and the separate drawing data so that the distortion of the virtual image accompanying the projection of light onto the projection area is reduced. And a distortion correction setting unit (54) for setting the content of the correction.

  One embodiment disclosed is a display control program used in a vehicle (A) in which a virtual image (Vi) is displayed by projecting light onto a projection area (PA), and controlling display of the virtual image. One processor (50a, 550a, 650a) is provided with specific drawing data (DS1) of a specific display object (11) displayed as a virtual image in association with a specific object in the foreground of the vehicle, and another display different from the specific display object A drawing data preparation unit (51) for preparing separate drawing data (DS2, DS3) for the object (12); a posture information acquisition unit (52) for acquiring posture information related to the posture of the vehicle; A posture correction setting unit (53) for setting the contents of a posture correction to be performed on the specific drawing data using the posture information so that a displacement of the specific display object with respect to the specific object is reduced; An image synthesizing unit (55) for synthesizing the specific drawing data to which the orientation correction of the content set by the main setting unit has been applied and another drawing data, and reducing distortion of a virtual image due to light projection onto a projection area Thus, the display control program functions as a configuration including a distortion correction setting unit (54) for setting the content of distortion correction applied to the specific drawing data and the different drawing data.

  The specific drawing data according to these aspects is combined with the other drawing data by applying the content of the posture correction for reducing the display shift of the specific display object with respect to the specific object. The content of the distortion correction for reducing the distortion of the virtual image is set on the assumption that the distortion correction is applied to both the specific drawing data to be combined and the different drawing data. According to the above, even if the correction corresponding to the change in the attitude of the vehicle and the synthesis with the different drawing data are performed after the preparation of the specific drawing data, the virtual image based on the specific drawing data is in a mode in which the distortion is suppressed. It can be displayed.

  It should be noted that the reference numbers in the parentheses merely show an example of a correspondence relationship with a specific configuration in the embodiment described later, and do not limit the technical scope at all.

1 is a block diagram illustrating an overall image of a vehicle-mounted system including a HUD according to a first embodiment of the present disclosure. FIG. 4 is a diagram illustrating an example of a virtual image display in a scene in which a change in the posture of a vehicle has not occurred. FIG. 9 is a diagram illustrating an example of a virtual image display in a scene in which a posture change has occurred in a vehicle, and is a diagram for describing an effect of correction according to a posture change. It is a figure which shows the vibration characteristic of the attitude | position change which arises in a vehicle, and is a figure which shows an example of each range of a high frequency band and a low frequency band. FIG. 4 is a diagram schematically illustrating a process of generating output data by cooperation of a drawing ECU and an LCD control board. FIG. 7 is a diagram for explaining details of a process of posture correction, main synthesis, and distortion correction performed on the LCD control board. 5 is a flowchart illustrating details of a drawing process performed by a drawing ECU. 9 is a flowchart illustrating details of a correction process performed by the LCD control board. It is a figure which shows the generation process of the output data in 2nd embodiment typically, and is a figure for demonstrating the process detail of each process implemented in an LCD control board. It is a figure which shows typically the detail of the code | cord | chord embedded in the drawing data produced | generated by the drawing ECU of 2nd embodiment. It is a flowchart which shows the detail of the correction process by 2nd embodiment. It is a figure which shows typically the generation process of the output data in 3rd embodiment, and is a figure for demonstrating the process of each process implemented by an LCD control board in detail. It is a figure which shows typically the production | generation process of the output data in 4th Embodiment, and is a figure for demonstrating the process detail of each process implemented in an LCD control board. It is a block diagram showing the whole picture of the in-vehicle system containing the display control device by a 5th embodiment. It is a figure which shows typically the detail of the code | cord | chord embedded in the drawing data produced | generated by the drawing ECU of 5th Embodiment. It is a block diagram showing the whole picture of the in-vehicle system containing the display control device by a 6th embodiment. FIG. 9 is a diagram schematically illustrating details of a main synthesis process according to a first modification. FIG. 14 is a diagram schematically illustrating a method of transmitting information in Modification Example 2. FIG. 13 is a diagram schematically illustrating a method of transmitting information in Modification Example 3.

  Hereinafter, a plurality of embodiments of the present disclosure will be described with reference to the drawings. In addition, in each embodiment, the corresponding components are denoted by the same reference numerals, and redundant description may be omitted. When only a part of the configuration is described in each embodiment, the configuration of another embodiment described earlier can be applied to the other part of the configuration. In addition, not only the combination of configurations explicitly described in the description of each embodiment, but also the configuration of a plurality of embodiments can be partially combined with each other even if it is not explicitly described, as long as the combination does not interfere. Unspecified combinations of configurations described in a plurality of embodiments and modifications are also disclosed by the following description.

(First embodiment)
The function of the display control device according to the first embodiment of the present disclosure illustrated in FIG. 1 is implemented in a head-up display (hereinafter, “HUD”) 100. The HUD 100 constitutes a virtual image display system 10 used in the vehicle A together with the drawing ECU (Electronic Control Unit) 30 and the like.

  The virtual image display system 10 presents various information related to the vehicle A to the driver by a display using the virtual image Vi superimposed on the foreground of the vehicle A. As shown in FIGS. 1 and 2, the virtual image display system 10 displays a virtual image Vi including a plurality of display objects so as to be visible from a driver by projecting light onto a projection area PA defined in the windshield WS. I do. The virtual image Vi may include an AR (Augmented Reality) display 11 and a non-AR display 12.

  The AR display object 11 is a virtual image Vi used for augmented reality display. The display position of the AR display object 11 in a state where the AR display object 11 is displayed as the virtual image Vi is associated with a specific object in the foreground visually recognized through the projection area PA, for example, a superimposed object such as a preceding vehicle, a pedestrian, and a road surface. I have. The AR display object 11 can be visually moved by the driver following the superimposition target so as to be relatively fixed to the superimposition target. As described above, since the positional relationship between the driver's eye point, the object to be superimposed in the foreground, and the projection position in the projection area PA is determined, even when the attitude of the vehicle A changes, the AR display object 11 is displayed. The state of being superimposed on the superimposition target is maintained (see FIG. 3). In other words, the display position of the AR display object 11 is constantly moving within the angle of view of the HUD 100. As an example, the virtual image Vi indicating the range of the lane to be driven is superimposed and displayed as the AR display object 11 on the road surface between the left and right lane markings in the foreground in a manner extending in the depth direction as viewed from the driver. .

  The non-AR display object 12 is not superimposed on a specific superimposition target, but is simply superimposed and displayed on the foreground. The non-AR display object 12 is displayed below the AR display object 11 as viewed from the driver. The non-AR display object 12 is a display object for mainly notifying the state of the vehicle A, and is displayed as being relatively fixed to a windshield WS or the like, which is a configuration of the vehicle A. That is, the display position of the non-AR display object 12 does not substantially move within the angle of view of the HUD 100. Therefore, when the attitude of the vehicle A changes, the positional relationship between the non-AR display object 12 and the foreground visually recognized by the driver changes (see FIG. 3).

  The display using the non-AR display object 12 includes an always-on display that is always displayed in a state where the vehicle A can travel, and an event display that is displayed only under a specific situation related to the vehicle A. As an example, the speed display indicating the traveling speed of the vehicle A corresponds to a constant display. On the other hand, an indicator display indicating an operating state of a driving support function such as ACC (Adaptive Cruise Control) and LTC (Lane Trace Control) corresponds to an event display.

  The virtual image display system 10 illustrated in FIG. 1 is configured to superimpose on the superimposition target even when the relative positional relationship between the superimposition target, the projection area PA, and the driver's eye point changes due to a change in the posture of the vehicle A. Control for maintaining the displayed state of the AR display object 11 is performed. More specifically, when a change in attitude such as a pitch, a roll, and a heave occurs in the vehicle A, the appearance of the light formed as the virtual image Vi (hereinafter, “virtual image light Lvi”) is not changed unless viewed from the driver. Above, the AR display object 11 is displayed shifted from the superimposition target. Therefore, the virtual image display system 10 adjusts the virtual image light Lvi according to the state of the superimposition target viewed through the projection area PA so that the display shift of the AR display object 11 with respect to the superposition target is reduced in accordance with the change in the attitude of the vehicle A. The shape is corrected sequentially.

  Here, as shown in FIG. 4, the vibration of the attitude change of the vehicle A does not occur at a constant frequency, but is a vibration mainly including a frequency band component up to about 2 Hz, for example. In the virtual image display system 10, a frequency band from about 0 to 0.5 Hz is a low frequency band LB, and a frequency band from 0.5 to 2 Hz, which is higher in frequency than the low frequency band LB, is a high frequency band. HB. As an example, when the vehicle A is slowly accelerated or decelerated, vibration in the low frequency band LB occurs. On the other hand, when the brake operation is input a plurality of times, when the vehicle A is suddenly started or decelerated, or when traveling on a rough road, vibration in the high frequency band HB occurs.

  In the virtual image display system 10 illustrated in FIG. 1, a function of reducing a display shift of the virtual image Vi due to a change in the posture of the low frequency band LB is realized by the drawing ECU 30. On the other hand, the function of reducing the display shift of the virtual image Vi due to the change in the posture of the high frequency band HB is realized by the HUD 100. Hereinafter, details of the drawing ECU 30 and the HUD 100 will be described with reference to FIGS.

  The drawing ECU 30 is connected to a plurality of in-vehicle displays including the HUD 100, and has a function of integrally controlling the display of each in-vehicle display. The drawing ECU 30 determines the display content of each vehicle-mounted display, and sequentially outputs a video signal to each vehicle-mounted display. The video signal output to the HUD 100 is also generated by the drawing ECU 30.

  The drawing ECU 30 can communicate with other on-vehicle components via the communication bus 29 of the on-vehicle network in order to acquire data necessary for generating a video signal. An external sensor 21, a locator 22, a three-dimensional map database 23, a height sensor 24, a vehicle control ECU 26, and the like are directly or indirectly electrically connected to the communication bus 29.

  The external sensor 21 detects moving objects such as pedestrians and other vehicles, as well as stationary objects such as curbs, road signs, road markings, and lane markings on the road. At least a part of the moving object and the stationary object is a superimposition target on which the AR display object 11 is superimposed. In the vehicle A, for example, a camera unit, a lidar, a millimeter wave radar, and the like are mounted as the external sensor 21. The external sensor 21 sequentially outputs, to the communication bus 29, object information indicating the detected relative positions and types of the moving object and the stationary object.

  The locator 22 can receive a positioning signal from each positioning satellite of at least one satellite positioning system among satellite positioning systems such as GPS, GLONASS, Galileo, IRNSS, QZSS, and Beidou. Locator 22 measures the position of vehicle A based on the received positioning signal. The locator 22 sequentially outputs the measured position information of the vehicle A to the communication bus 29. Note that locator 22 may include an inertial sensor for correcting position information.

  The three-dimensional map database (hereinafter, “three-dimensional map DB”) 23 has a configuration mainly including a large-capacity storage medium storing a large number of three-dimensional map data. The three-dimensional map data is, for example, high-precision map data that enables automatic driving, and includes information indicating the latitude, longitude, and altitude of a road. The three-dimensional map DB 23 can update the three-dimensional map data to the latest information through a network. The three-dimensional map DB 23 can provide the drawing ECU 30 with three-dimensional map data around the vehicle A and the traveling direction in response to a request from the drawing ECU 30.

  The height sensor 24 is a sensor that detects a vertical displacement generated in the vehicle A in order to measure a height from a road surface on which the vehicle A is placed to a body. The height sensor 24 is installed, for example, on one of the left and right rear suspensions. The height sensor 24 measures the sinking amount of the specific wheel, which is displaced in the vertical direction by the operation of the suspension arm suspended from the body, with respect to the body. Specifically, the height sensor 24 measures the relative distance between the body and the suspension arm, and sequentially outputs the measurement result to the communication bus 29.

  The vehicle control ECU 26 is an arithmetic device mainly composed of a microcontroller. The vehicle control ECU 26 controls the behavior of the vehicle A based on the object information detected by the external sensor 21 and the driving operation of the driver. The pedal sensor 25 is electrically connected to the vehicle control ECU 26. The pedal sensor 25 includes at least an accelerator position sensor and a brake depression force sensor.

  The vehicle control ECU 26 controls the longitudinal acceleration to be generated in the vehicle A, that is, the axle torque and the braking force, based on the detection signal of the pedal sensor 25. In addition, the vehicle control ECU 26 performs feed-forward control of the axle torque and the braking force so that the vibration of the vehicle A due to the driver's acceleration / deceleration operation and disturbance such as unevenness of the road surface is suppressed. The vehicle control ECU 26 sequentially outputs the target values of the axle torque and the braking force in the feedforward control to the communication bus 29 as control information.

  The drawing ECU 30 mainly includes a computer having a processing unit, a RAM, a storage device, and an input / output interface. The processing unit is configured to include at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field-Programmable Gate Array), and the like. Various programs executed by the processing unit are stored in the storage device. The plurality of programs stored in the storage device include a drawing program for generating the drawing data DS, a posture estimation program for estimating a posture change of the vehicle A, and the like.

  The drawing ECU 30 executes a drawing program and a posture estimation program by a processing unit, and functional units such as a sensor value obtaining unit 31, a content selecting unit 34, a target information obtaining unit 35, a drawing correction amount calculating unit 36, and a drawing data generating unit 37. Is provided. The sensor value acquisition unit 31, the content selection unit 34, and the target information acquisition unit 35 are functional units that can acquire information from the communication bus 29.

  The sensor value acquisition unit 31 is an output signal output from the height sensor 24 and the vehicle control ECU 26, and receives a measurement result of the vehicle height, and control information on the axle torque and the braking force. The sensor value acquisition unit 31 processes these output signals and acquires posture change information related to the posture change in the low frequency band LB.

  The content selection unit 34 selects and arbitrates the content for which a virtual image is to be displayed, based on various types of information obtained from the communication bus 29. Specifically, the content selection unit 34 comprehensively determines the necessity of the AR display, the constant display, the event display, and the like, and the respective priorities and the like, and informs the driver using the virtual image Vi. Select the content. The content selection unit 34 sequentially provides the drawing data generation unit 37 with a command for designating the selected content and information necessary for drawing the selected content.

  The target information obtaining unit 35 obtains the object information output by the external sensor 21 from the communication bus 29, and selects a superimposition target on which the virtual image Vi is superimposed from the moving object and the stationary object. The target information acquisition unit 35 sequentially provides the selected relative positions of the superimposition targets to the drawing data generation unit 37. As an example, when the road surface of the traveling lane is to be superimposed and the AR display object 11 is superimposed on the road surface (see FIG. 2), the target information acquisition unit 35 draws the relative positions of the left and right lane markings. It is sequentially provided to the data generation unit 37.

  The drawing correction amount calculation unit 36 specifies the current posture of the vehicle A using the posture change information of the low frequency band LB acquired by the sensor value acquisition unit 31. The drawing correction amount calculation unit 36 calculates a correction amount (hereinafter, “low frequency correction amount”) for correcting a display shift of the virtual image Vi due to a change in the posture of the low frequency band LB from the specified vehicle posture. The drawing correction amount calculation unit 36 sequentially provides the calculated value of the low frequency correction amount to the drawing data generation unit 37.

  The drawing data generator 37 generates drawing data DS. The drawing data DS is image data constituting individual frames in a video signal output to the HUD 100. The drawing data generation unit 37 is electrically connected to the HUD 100 via a video output line 38. The drawing data generator 37 sequentially outputs the generated drawing data DS to the HUD 100 in a predetermined video format.

  The drawing data DS is temporary combined data in which a plurality of drawn objects corresponding to the content selected by the content selection unit 34 are combined into a temporary state. The drawing data DS includes, as a plurality of drawing objects, an AR drawing object DS1 serving as an original image of the AR display object 11, and non-AR drawing objects DS2 and DS3 serving as an original image of each non-AR display object 12. I have.

  The drawing data generation unit 37 draws each of the drawing objects DS1 to DS3 corresponding to each content individually, and arranges them so as not to overlap each other. By such pre-synthesis processing, the drawing data generation unit 37 generates one drawing data DS. In the drawing data DS, the drawing positions P1 to P3, the horizontal resolution, and the vertical resolution of each drawing object DS1 to DS3 are fixed to predetermined values. The number of drawings DS1 to DS3 included in one drawing data DS corresponds to the number of contents selected by the content selection unit 34. The drawings DS1 to DS3 are again synthesized in a regular positional relationship by the main synthesis (described later) by the LCD control board 50.

  The drawing data generation unit 37 draws the AR drawing DS1 in a mode that takes into account the relative position and shape of the superimposition target and the change in posture in the low frequency band LB. Specifically, the drawing data generation unit 37 reads the preset eye point and each set position of the projection area PA from a storage device or the like. In addition, the drawing data generation unit 37 obtains the relative position of the superimposition target from the target information obtaining unit 35, and obtains the low frequency correction amount from the drawing correction amount calculation unit 36. After grasping the relative positional relationship between the eye point, the projection area PA, and the superimposition target, the drawing data generation unit 37 corrects these positional relationships with a low-frequency correction amount, and can overlap the superimposition target without display shift. The AR drawing object DS1 is drawn. The AR drawing DS1 is set to an image size in which a margin is secured at least in the vertical direction with respect to the image size required (cut out) in this synthesis in consideration of a posture change of a high frequency band HB described later. Is drawn.

  The HUD 100 is an in-vehicle display device housed in a housing space provided in the instrument panel below the windshield WS. The virtual image light Lvi emitted from the HUD 100 toward the projection area PA of the windshield WS is reflected toward the eye point side by the projection area PA and is perceived by the driver. The driver visually recognizes the display in which the virtual image Vi is superimposed on the foreground seen through the projection area PA. The HUD 100 forms a virtual image Vi in a space of about 10 to 20 m in a forward direction of the vehicle A from the eye point, for example.

  The HUD 100 includes a projection unit 70, an inertial sensor 41, a filter circuit 42, an LCD control board 50, and a BL control board 60 as a configuration for displaying a virtual image.

  The projection unit 70 has an optical configuration that projects the virtual image light Lvi onto the projection area PA. The projection unit 70 includes an LCD (Liquid Crystal Display) 71, a backlight 72, a reflection optical system 73, and the like. These optical elements are housed in the housing 40 of the HUD 100. The housing 40 defines the relative positional relationship between the optical elements with high accuracy.

  The LCD 71 has a display surface on which a number of pixels are arranged. The LCD 71 increases or decreases the light transmittance of sub-pixels such as red, green, and blue provided for each pixel, and displays various images in color on the display surface. The light transmittance of each sub-pixel is controlled by the LCD control board 50. On the display surface of the LCD 71, an original image Pi based on the output data DSc generated by the main synthesis is displayed.

  The backlight 72 has a configuration having many light emitting diodes. A large number of light emitting diodes are arranged on the back side of the LCD 71, and are two-dimensionally arranged at predetermined intervals along the display surface of the LCD 71. The backlight 72 is configured to be capable of adjusting light emission luminance for each area illuminated by each light emitting diode. The light emission luminance of each light emitting diode is individually controlled by the BL control board 60. The backlight 72 illuminates the LCD 71 by transmitting light from the rear side, and emits and displays the original image Pi drawn on the display surface. As a result, the light of the original image Pi is emitted from the display surface as virtual image light Lvi.

  The reflection optical system 73 is configured to include, for example, a convex mirror and a concave mirror. The convex mirror and the concave mirror are reflection mirrors in which a metal such as aluminum is vapor-deposited on a surface of a colorless and transparent base made of synthetic resin or glass. The convex mirror and the concave mirror project the virtual image light Lvi emitted from the LCD 71 onto the projection area PA while expanding the virtual image light Lvi by reflection, and display a virtual image Vi obtained by enlarging the original image Pi. In addition, a diffractive optical element (Diffractive Optical Element: DOE) that enlarges the original image Pi by diffraction, for example, may be used as the reflection optical system 73.

  The inertial sensor 41 is a measuring unit that measures a change in the attitude of the vehicle A, and has a configuration in which a gyro sensor and an acceleration sensor are combined. The inertia sensor 41 is mounted on the vehicle A separately from the height sensor 24 and the vehicle control ECU 26. The inertial sensor 41 measures the angular velocities of the vehicle A in the pitch direction and the roll direction, and the vertical acceleration of the vehicle A along the yaw axis.

  The inertial sensor 41 is provided with a low-pass filter and an AD converter. The low-pass filter removes high-frequency noise from each output of the gyro sensor and the acceleration sensor. The AD converter converts the analog signal that has passed through the low-pass filter into a digital signal. The inertial sensor 41 can transmit a signal to the filter circuit 42 using communication standards such as I2C (Inter-Integrated Circuit: registered trademark) and SPI (Serial Peripheral Interface). The inertial sensor 41 outputs a digital measurement signal according to the above communication standard to the filter circuit 42.

  The filter circuit 42 acquires a measurement signal output from the inertial sensor 41. The delay time until the measurement signal is acquired by the filter circuit 42 (hereinafter, “measurement delay time”) is shorter than the communication delay time generated in the communication bus 29. The filter circuit 42 has a configuration including at least a high-pass filter and an integration processing unit.

  The high-pass filter substantially passes signals in the high frequency band HB (see FIG. 4) and attenuates signals in the low frequency band LB (see FIG. 4). The cutoff frequency of the high-pass filter is set to be a boundary value between the high frequency band HB and the low frequency band LB. With such a setting, the high-pass filter attenuates a signal in a band overlapping with the low frequency band LB among the frequency bands included in the measurement signal. In addition, a drift component generated in the gyro sensor of the inertial sensor 41 due to the passage through the high-pass filter is removed from the measurement signal.

  The integration processing unit has a configuration mainly including, for example, a low-pass filter. The integration processing unit generates a signal indicating the vehicle attitude (pitch angle, roll angle, etc.) by performing signal processing for time-integrating a measurement signal indicating the angular velocity of the attitude change. The filter circuit 42 generates a signal that has passed through the high-pass filter and the integration processing unit in order as posture change information of the high frequency band HB, and sequentially provides the information to the LCD control board 50.

  The LCD control board 50 shown in FIGS. 1, 5, and 6 has an electrical configuration for controlling the virtual image display by the projection unit 70. The control circuit formed on the LCD control board 50 mainly includes a processing unit 50a, a RAM, a storage device, an input / output interface, and a computer having a data bus connecting these components.

  The processing unit 50a is hardware for arithmetic processing combined with the RAM. The processing unit 50a includes a memory controller that controls access to the RAM. The processing unit 50a is configured to include at least one of a CPU, a GPU, an FPGA, and an IP core having other dedicated functions.

  The RAM is combined with the processing unit 50a and functions as a working memory of the processing unit 50a. The LCD control board 50 includes a first memory 151 and a second memory 152 as RAMs. The first memory 151 and the second memory 152 are mainly configured by SDRAM memory modules conforming to a standard such as DDR (Dynamic Random Access Memory) 2 or DDR3. The first memory 151 and the second memory 152 are volatile memories having substantially the same configuration, and are arranged so that the capacities and the access speeds are substantially the same. The first memory 151 and the second memory 152 are individually connected to the data bus.

  The first memory 151 and the second memory 152 can function as working memories of the processing unit 50a independently of each other. Specifically, the first memory 151 and the second memory 152 are assembled in a memory slot on a circuit board connected to a data bus as a physically independent memory configuration. Further, the processing unit 50a can access the second memory 152 concurrently even during the access period to the first memory 151. Similarly, the processing unit 50a can access the first memory 151 at the same time even during the access period to the second memory 152. That is, the processing unit 50a can access the other memory without being limited to one of the first memory 151 and the second memory 152.

  The storage device is mainly configured by a non-volatile storage medium such as a ROM or a flash memory. The storage device stores a display control program executed by the processing unit 50a. The LCD control board 50 executes the display control program by the processing unit 50a, and controls the drawing data acquisition unit 51, the posture information acquisition unit 52, the posture correction setting unit 53, the distortion correction setting unit 54, and the functional units such as the output generation unit 55. Prepare.

  The drawing data acquisition unit 51 is electrically connected to the drawing data generation unit 37 via the video output line 38. To the drawing data acquisition unit 51, drawing data DS for displaying a virtual image is transmitted from the drawing data generation unit 37. The drawing data acquisition unit 51 prepares the drawing data DS by receiving the data through the video output line 38.

  The posture information acquisition unit 52 acquires posture change information of the high frequency band HB (see FIG. 4) from the filter circuit 42 as posture information related to the posture of the vehicle A.

  The posture correction setting unit 53 uses the posture change information of the high frequency band HB acquired by the posture information acquisition unit 52 to determine the displacement of the AR display object 11 due to the posture change of the high frequency band HB (see FIG. 4). The content of the posture correction to be performed on the AR drawing DS1 is set so as to be reduced. The content of the posture correction is set for each drawing data DS (for each frame) based on the latest posture change information. The posture correction is a pitch correction for reducing (cancelling) the positional deviation of the virtual image Vi from the superimposition target mainly due to pitching. The target of such pitch correction is the vibration in the frequency band of the high frequency band HB (see FIG. 4) among the vibrations in the pitch direction generated in the vehicle A.

  Specifically, in the posture correction, the posture correction setting unit 53 offsets the drawing position P1 in the AR drawing object DS1 to the side corresponding to the direction of the posture change. The posture correction setting unit 53 calculates the number of pixels to be offset (hereinafter, referred to as “the number of offset pixels”) by applying (multiplying) a predetermined conversion coefficient to the posture change amount indicated by the posture change information. The number of offset pixels increases as the amount of change in attitude of the vehicle A (vehicle pitch angle) increases. The posture correction setting unit 53 sets a position obtained by applying the number of offset pixels to the drawing position P1 as a cutout reference position P1c. In the AR drawing DS1, an image range based on the cut-out reference position P1c is used in the main synthesis process described later. As described above, the process of determining the number of offset pixels and the cutout reference position P1c corresponds to the process of setting the content of the posture correction.

  Note that the size (for example, 1280 × 480 pixels) of an image range cut out from the AR drawing DS1 is smaller than the size (for example, 1280 × 720 pixels) of the AR drawing DS1 and the same as the size used for the output data DSc. Degree. When there is no change in the posture of the high frequency band HB, the number of offset pixels is zero, and the cutout reference position P1c matches the drawing position P1.

  After the image range cut out from the AR drawing DS1 is set by the posture correction setting unit 53, the distortion correction setting unit 54 reduces the distortion of the virtual image Vi due to the projection of light onto the projection area PA. The content of distortion correction to be performed on each of the drawn objects DS1 to DS3 is set. The content of the distortion correction depends on the curved shape of the windshield WS in the projection area PA. The distortion correction takes into account the deformation of the virtual image Vi caused by the reflection in the projection area PA and the reflection optical system 73, and the shape of the output data DSc is displayed so that the virtual image Vi is displayed with the distortion reduced (cancelled). Is corrected in advance.

  The distortion correction setting unit 54 is used by the output generation unit 55 to generate the output data DSc from a plurality of types of distortion correction tables TBc prepared in advance (see “Synthesis & distortion correction table” in FIG. 6). Choose one. The distortion correction table TBc is a content that further includes not only distortion correction information necessary for distortion correction of each of the drawings DS1 to DS3 but also synthesis use information necessary for synthesis of each of the drawings DS1 to DS3. The distortion correction table TBc is information linking which pixel of the output data DSc corresponds to which pixel of the drawing data DS.

  As an example, the distortion correction table TBc has a file configuration in which a large number of data sets each including an output image coordinate and an original image coordinate are made continuous. The output image coordinates are values indicating the X coordinate and the Y coordinate of the pixel in the output data DSc. The original image coordinates are information indicating an X coordinate and a Y coordinate of a pixel in the drawing data DS (a pre-synthesized image DSi described later).

  Note that, as an example, the arrangement of pixels in the drawing data DS and the output data DSc is such that the upper left is the coordinates of the first pixel (0, 0), and the second pixel, the third pixel,. This is the nth pixel. Then, the leftmost pixel in the row immediately below is the (n + 1) th pixel.

  The plurality of types of distortion correction tables TBc are set in accordance with the layout of each of the drawn objects DS1 to DS3 after synthesis. Specifically, at least a distortion correction table TBc (for no combination) in which the AR drawing DS1 is not combined with the non-AR drawing DS2 and DS3 is prepared. In addition, the distortion correction setting unit 54 can prepare in advance a plurality of types in which the layouts (positions and sizes) of the combined drawn objects DS1 to DS3 are different from each other as the distortion correction table TBc for performing the combination.

  The distortion correction setting unit 54 selects the distortion correction table TBc based on the drawing data DS acquired by the drawing data acquisition unit 51. Note that a command to specify the distortion correction table TBc to be selected may be provided by the drawing data generation unit 37. The process of reading the content of the distortion correction table TBc selected in this manner so as to be applicable to the drawing data DS corresponds to the process of setting the content of the distortion correction. Specifically, in the first embodiment, reading of the distortion correction table TBc by the correction circuit 154 (described later) corresponds to setting of the content of the distortion correction. The content of the distortion correction is set for each drawing data DS (each frame), similarly to the content of the posture correction. The contents of the distortion correction are collectively applied by the correction circuit 154 to each of the drawn objects DS1 to DS3.

  The output generation unit 55 generates output data DSc from the drawing data DS by performing signal processing in the frame buffer. The output generation unit 55 applies each of the non-AR drawing objects DS2 to the image range cut out from the AR drawing DS1 based on the cutout reference position P1c by applying the posture correction of the content set by the posture correction setting unit 53. , DS3 are synthesized.

  The output generation unit 55 executes a process of reflecting the distortion correction of the content set by the distortion correction setting unit 54 on each of the drawing objects DS1 to DS3 in parallel with the processing of synthesizing the respective drawing objects DS1 to DS3. . The output generation unit 55 sorts the deformed drawing objects DS1 to DS3 into one by performing a process of rearranging the two-dimensional array of pixels in the cut out individual images in accordance with the arrangement of the drawing objects DS1 to DS3 after the synthesis. The output data DSc that is combined into an image is generated.

  The drive circuit provided on the BL (Backlight) control board 60 controls the light emission luminance of the backlight 72 for each of a plurality of areas by partial driving (local dimming) for individually controlling each light emitting diode of the backlight 72. The BL control board 60 determines the content of the area control of the backlight 72 after the correction content is set by the attitude correction setting unit 53.

  Specifically, the BL control board 60 acquires the output data DSc from the output generation unit 55. The BL control board 60 adjusts the shape of the drawing object DS1 to DS3 on which the distortion correction is reflected, that is, the position and the shape of the AR display object 11 and the non-AR display object 12 in the output data DSc. The light emission luminance of each area is determined. The BL control board 60 outputs a drive signal for causing each area to emit light at the determined light emission luminance to each light emitting diode of the backlight 72.

  As shown in FIG. 6, the LCD control board 50 is provided with an input circuit 153 and a correction circuit 154 as processing circuits for realizing the functions of the functional units described so far. The input circuit 153 and the correction circuit 154 are included in the processing unit 50a (see FIG. 1), and can access the first memory 151 and the second memory 152 at high speed through the data bus.

  The input circuit 153 is a circuit unit that includes at least a part of the function of the drawing data acquisition unit 51 (see FIG. 1). The input circuit 153 executes a process of writing data to each of the first memory 151 and the second memory 152. The input circuit 153 reads the plurality of types of distortion correction tables TBc stored in the storage device immediately after the start of the LCD control board 50 and before the start of the virtual image display, and writes the same into both the first memory 151 and the second memory 152. In the first memory 151 and the second memory 152, substantially the same information is stored as a distortion correction table TBc. As a result, a plurality of types of distortion correction tables TBc are prepared in the first memory 151 and the second memory 152 in advance. The first memory 151 and the second memory 152 continue to hold the data of the distortion correction table TBc without erasing until the power supply to the LCD control board 50 is stopped.

  When the output of the drawing data DS from the drawing ECU 30 to the LCD control board 50 is started, the input circuit 153 receives the drawing data DS transmitted through the video output line 38. The input circuit 153 performs a process of alternately writing the drawing data DS that is continuously obtained as the pre-synthesized image DSi in the first memory 151 and the second memory 152. Thus, the pre-combined image DSi is prepared.

  When the input circuit 153 writes the (n-1) th frame in the first memory 151 as the pre-synthesized image DSi, the input circuit 153 writes the n-th frame in the second memory 152 as the pre-synthesized image DSi. The pre-synthesized image DSi is written in each of the first memory 151 and the second memory 152 at an address different from the storage area of each distortion correction table TBc. By such switching, the first memory 151 and the second memory 152 are in a state where different pre-synthesized images DSi are stored so as to be readable by the correction circuit 154.

  The correction circuit 154 is a circuit unit including at least a part of each function of the posture correction setting unit 53, the distortion correction setting unit 54, and the output generation unit 55 (see FIG. 1). The correction circuit 154 applies the processes of posture correction, main synthesis, and distortion correction substantially simultaneously to the pre-synthesized image DSi to generate output data DSc. The correction circuit 154 sequentially outputs the generated output data DSc to the LCD 71 and the BL control board 60.

  When generating the output data DSc, the correction circuit 154 reads the pre-synthesized image DSi from one of the first memory 151 and the second memory 152 and reads the distortion correction table TBc from the other of the first memory 151 and the second memory 152. . The correction circuit 154 generates output data DSc by controlling reading of the pre-synthesized image DSi from one memory in accordance with the distortion correction table TBc read from the other memory.

  Here, the correction circuit 154 sets a memory in which the pre-synthesized image DSi is written by the input circuit 153 as a memory from which the distortion correction table TBc is read (corresponding to the second memory 152 in FIG. 6). As described above, a memory (corresponding to the first memory 151 in FIG. 6) dedicated to reading out the pre-synthesized image DSi in which random access easily occurs is secured.

  The correction circuit 154 selects a distortion correction table TBc to be read from a plurality of types. The correction circuit 154 sequentially reads out the value of the distortion correction table TBc while performing processing of adding or subtracting the number of offset pixels to or from the value of the Y coordinate in the original image coordinates of the distortion correction table TBc. By applying the number of offset pixels to the distortion correction table TBc, the AR drawing DS1 is cut into the output data DSc based on the cutout reference position P1c (see FIG. 5).

  The correction circuit 154 extracts the pixel data of the pixel indicated by the original image coordinate from the pre-synthesized image DSi based on the positional relationship between the output image coordinate indicated by the distortion correction table TBc and the original image coordinate, and calculates the pixel data indicated by the output image coordinate. It is sequentially output as pixel data. As described above, for the output data DSc output from the correction circuit 154, the attitude correction process for the AR drawing object DS1, the combining process of each of the drawing objects DS1 to DS3, and the distortion correction process for each of the drawing objects DS1 to DS3 are all applied. It becomes an aspect.

  Details of each processing performed by the drawing ECU 30 and the LCD control board 50 to realize the above-described virtual image display will be described based on FIGS. 7 and 8 with reference to FIGS. 1, 5 and 6. Each process shown in FIGS. 7 and 8 is started based on, for example, switching the vehicle power supply to the on state, and is repeated until the vehicle power supply is turned off.

  The drawing process shown in FIG. 7 is performed by the drawing ECU 30. In S101 of the drawing process, based on the information acquired from the communication bus 29, a content to be presented to the driver by the virtual image Vi is selected, and the process proceeds to S102. In S102, the drawing objects DS1 to DS3 corresponding to the content selected in S101 are drawn, and the process proceeds to S103. In S103, one piece of drawing data DS is generated by pre-synthesizing each of the drawn objects DS1 to DS3 drawn in S102, and the process proceeds to S104. In S104, the drawing data DS generated in S103 is output to the drawing data acquisition unit 51.

  The correction process shown in FIG. 8 is performed by the LCD control board 50. In S111 of the correction process, the drawing data DS drawn by the drawing ECU 30 is prepared by acquisition through communication, and the process proceeds to S112. In S112, the posture change information of the high frequency band HB (see FIG. 4) is obtained, and the process proceeds to S113. In S113, the contents of the posture correction are set for the AR drawing DS1. Specifically, in step S114, the number of offset pixels is calculated in consideration of the posture change information acquired in step S112, and a reference coordinate position serving as a cutout reference, and a cutout reference position P1c as an address at which to start reading an image are determined. .

  In S114, a distortion correction table TBc to be applied to each of the drawings DS1 to DS3 is selected. Then, while applying the contents of the posture correction set in S113, the selected distortion correction table TBc is read out, the contents of the distortion correction are set, and the process proceeds to S115. In S115, according to the contents of the distortion correction table TBc that has been read out in S114, the main synthesis process for generating the output data DSc from the pre-synthesized image DSi is started, and the process proceeds to S116. S115 is a process of rearranging the pixels of the drawing data DS as the pixels of the output data DSc. In S116, the output data DSc generated in S115 is output to the LCD 71 and the BL control board 60. In the first embodiment, the processes of S114, S115, and S116 are integrally performed on the drawing data DS and the output data DSc that constitute one frame. The BL control board 60 controls the area of the backlight 72 corresponding to the mode of each of the drawings DS1 to DS3 based on the output data DSc output in S116.

  In the first embodiment described so far, after the content of the posture correction performed on the AR drawing DS1 is set so that the display shift of the AR display object 11 with respect to the superimposition target is reduced, the virtual image Vi of the virtual image Vi is set. The content of distortion correction for reducing distortion is further set. Therefore, even if the correction for responding to the change in the attitude of the vehicle A is performed after the preparation of the AR drawing DS1, the virtual image Vi of the AR display 11 based on the AR drawing DS1 is suppressed in a manner in which distortion is suppressed. It can be displayed.

  Specifically, in the first embodiment, the cut-out reference position P1c is first set as the content of the posture correction for one pre-synthesized image DSi. After that, the distortion correction table TBc applied to the pre-synthesized image DSi is selected and set in the correction circuit 154. Then, the correction circuit 154 generates an AR drawing DS1 of the output data DSc by deforming the image range based on the extraction reference position P1c according to the contents of the distortion correction table TBc. According to the above-described processing, the virtual image Vi of the AR display object 11 can be in a mode in which distortion is appropriately suppressed even if the posture correction is performed after drawing by the drawing ECU 30 and immediately before display on the LCD 71.

  In addition, in the first embodiment, the process of actually synthesizing the AR drawing object DS1 and the non-AR drawing objects DS2 and DS3 is performed after the content of the posture correction is set. In this synthesis, the AR drawing DS1 is combined with the non-AR drawing DS2 and DS3 after applying the content of the posture correction to reduce the display shift of the AR display 11 with respect to the superimposition target. The content of the distortion correction for reducing the distortion of the virtual image Vi is set on the assumption that the distortion correction is applied to the AR drawing DS1 and the non-AR drawing DS2 and DS3.

  According to the above, even if the correction for coping with the change in the attitude of the vehicle and the synthesis with the non-AR drawing DS2 and DS3 are performed after the preparation of the AR drawing DS1, the virtual image Vi of the AR display 11 is Display can be performed in a mode in which distortion is suppressed. Further, not only the AR display object 11 based on the AR drawing object DS1 but also the non-AR display object 12 based on the non-AR drawing objects DS2 and DS3 can be displayed as a virtual image with distortion suppressed.

  In the first embodiment, the correction circuit 154 applies the contents of the distortion correction to each drawing object based on the correspondence of the pixel positions between the pre-synthesized image DSi and the output data DSc defined in the distortion correction table TBc. This synthesis of DS1 to DS3 is performed. As described above, if the distortion correction and the main synthesis are performed as a group of processes, the generation of the output data DSc is easily performed at a low load and at a high speed.

  Further, in the first embodiment, processing for applying the content of the posture correction to the AR drawing object DS1 data is performed in synchronization with the main synthesis. As described above, if the execution of the posture correction is performed in parallel with the main synthesis, the delay time from the execution of the posture correction to the output of the output data DSc can be reduced. Therefore, even if a virtual image display that combines the drawing objects DS1 to DS3 is performed, the virtual image Vi of the AR display object 11 can appropriately follow the superimposition target.

  Here, if the posture correction is performed on the drawing data DS provided by the drawing data generation unit 37 without performing the main synthesis, even if the display shift of the AR display object 11 can be reduced, the non-AR The display objects 12 and 13 may be cut off from the angle of view of the projection unit 70. Therefore, if the present synthesis is performed after the application of the posture correction as in the first embodiment, it is possible to reduce the display displacement of the AR display object 11 and also to avoid the non-AR display objects 12 and 13 from being cut off. . Therefore, both the AR display object 11 and the non-AR display object 12 that are formed as the virtual image Vi can be displayed in a manner that does not make the driver feel uncomfortable.

  In addition, in the first embodiment, a distortion correction table TBc created in a table format is prepared in advance with contents including combined use information. Therefore, based on the distortion correction table TBc, the output generation unit 55 can execute the main synthesis in parallel with the distortion correction. Therefore, according to the preparation of the combined use information in the table format, it is possible to reduce the calculation load accompanying the generation of the output data DSc.

  Further, in the first embodiment, a display shift of the AR display object 11 due to a change in the posture of the low frequency band LB can be corrected in advance in a drawing stage of the AR drawing object DS1 in the drawing data generation unit 37. Therefore, the LCD control board 50 only needs to set the contents of the posture correction so that the display shift of the AR display object 11 due to the posture change in the high frequency band HB is reduced. Such a posture change in the high frequency band HB is a movement having a smaller amplitude than a posture change in the low frequency band LB (see FIG. 4). Therefore, if the attitude correction on the LCD control board 50 is specialized to the high frequency band HB, the AR display object 11 can follow the superimposition target within the maximum range of the margin secured for the AR drawing DS1. it can.

  In addition, in the first embodiment, the BL control board 60 that acquires the output data DSc controls the emission luminance of each area of the backlight 72 in accordance with the mode of the AR drawing DS1 that reflects the distortion correction. Therefore, even if the distortion correction is performed at the substantially final stage of the generation of the output data DSc, a situation in which the illumination of the backlight 72 and the display objects 11 and 12 are shifted can be avoided.

  In the first embodiment, a plurality of distortion correction tables TBc including the combined use information are prepared in a table format. Therefore, by increasing the number of the distortion correction tables TBc, it is possible to increase the variation in the arrangement of the respective drawn objects DS1 to DS3 after the combination.

  In the first embodiment, the drawing data DS and the pre-synthesized image DSi correspond to “temporary synthesized data”, the output data DSc corresponds to “re-synthesized data”, and the AR drawing DS1 corresponds to “specific drawing data”. The non-AR drawing objects DS2 and DS3 correspond to “separate drawing data”. In addition, the distortion correction table TBc corresponds to “synthesis use information”, the AR display object 11 corresponds to “specific display object”, the non-AR display object 12 corresponds to “separate display object”, and the processing unit 50a sets “ Processor ". Furthermore, the drawing data acquisition unit 51 corresponds to a “drawing data preparation unit”, the output generation unit 55 corresponds to an “image synthesis unit”, the BL control board 60 corresponds to a “brightness control unit”, and the HUD 100 Controller ".

(Second embodiment)
The second embodiment shown in FIGS. 9 to 11 is a modification of the first embodiment. In the virtual image display system of the second embodiment, the drawing ECU 30 is also connected to the LCD control board 50 by a code output line 39 in addition to the video output line 38. In addition, the LCD control board 50 further includes a combined use information generation unit 237 as a function unit based on the drawing program, in addition to the content selection unit 34 and the drawing data generation unit 37.

  The combined use information generation unit 237 generates a code CD including combined use information used in the main combination on the LCD control board 50. The combined use information in the second embodiment is sequentially generated by the combined use information generation unit 237, and provided as message data to the LCD control board 50 through the code output line 39. The code CD is one-dimensional character string information in which the number of contents (drawings) included in one drawing data DS and detailed information indicating details of each of the drawings DS1 to DS3 are described.

  The detailed information is information used for the main synthesis of each of the drawings DS1 to DS3. In the code CD, detailed information on the AR drawing DS1 is described after the information indicating the number of contents, and thereafter, detailed information on the second non-AR drawing DS2, and the third non-AR drawing Detailed information on the object DS3 is described in order.

  The detailed information of each of the drawings DS1 to DS3 includes switch information, corresponding coordinate information, correction necessity information, and adjustment range information. The switch information is information for instructing whether or not each of the drawn objects DS1 to DS3 is to be drawn on the output data DSc. In other words, the switch information is information for instructing display and non-display of each of the displayed objects 11 to 13. When the switch information indicates the ON state, the corresponding drawing objects DS1 to DS3 are drawn on the output data DSc. On the other hand, when the switch information indicates the OFF state, the corresponding drawing objects DS1 to DS3 are not drawn on the output data DSc. The combined use information generation unit 237 can make the virtual images Vi (see FIG. 2) of the respective display objects 11 to 13 blink by switching the on and off values described as the switch information at regular time intervals.

  The corresponding coordinate information is information that defines the correspondence between each coordinate of each drawing object DS1 to DS3 in the drawing data DS and each coordinate of each drawing object DS1 to DS3 in the output data DSc. The correction necessity information is information indicating whether correction is required for each of the drawn objects DS1 to DS3 due to a change in posture. Correction necessity information indicating that correction is necessary is described in the detailed information of the AR drawing DS1. On the other hand, in the detailed information of the non-AR drawing objects DS2 and DS3, correction necessity information indicating that correction is unnecessary is described. The adjustment range information is information indicating the maximum range of the posture correction that can be performed on the AR drawing object DS1.

  A more detailed example of such a code CD is described in FIG. The detailed information of the code CD shown in FIG. 11 includes the above-described switch information, the drawing positions P1 to P3 at the time of pre-synthesis, the horizontal resolution at the time of pre-synthesis, the vertical resolution at the time of pre-synthesis, and the display position P1s to P3s are described in order. Furthermore, the horizontal resolution at the time of main synthesis, the vertical resolution at the time of main synthesis, the necessity of position correction for posture changes such as pitching, the resolution that can be corrected to follow the posture change, and the upper margin The resolution that can be used as a margin is continuously described. Then, a conversion coefficient that defines the correspondence between the amount of change in posture and the number of offset pixels is further described.

  Among these detailed information, the drawing positions P1 to P3 at the time of the pre-synthesis and the display positions P1s to P3s at the time of the main synthesis correspond to the corresponding coordinate information described above. The necessity of the position correction for the posture change corresponds to the above-described correction necessity information. Further, each of the upper and lower resolutions that can be corrected corresponds to the above-mentioned adjustment range information. In the detailed information of the non-AR drawing objects DS2 and DS3, a portion where each resolution serving as a margin is described is in a blank state.

  The LCD control board 50 is provided with substantially the same functional units as those of the first embodiment (see FIG. 1). The drawing data acquisition unit 51 acquires the code CD transmitted from the combined use information generation unit 237 through the code output line 39. The posture correction setting unit 53 calculates the number of offset pixels by performing a process of applying the posture change amount indicated by the posture change information to the conversion coefficient described in the detailed information of the AR drawing object DS1 in the code CD. The posture correction setting unit 53 sets the extraction reference position P1c by performing a process of shifting the drawing position P1 in the detailed information based on the number of offset pixels.

  The output generation unit 55 refers to the cut-out reference position P1c set by the posture correction setting unit 53, the drawing positions P2 and P3 described in the detailed information of the code CD, and the respective resolutions at the time of pre-synthesis. In addition, the output generation unit 55 refers to the display positions P1s to P3s of the drawing objects DS1 to DS3 described in the detailed information of the code CD and the respective resolutions at the time of the pre-synthesis. The output generation unit 55 cuts out each of the drawing objects DS1 to DS3 from the drawing data DS based on these correspondences, adjusts the respective resolutions, and combines them into one piece of image data (a combined image DSm described later). However, the output generation unit 55 does not include the drawn object whose OFF state is instructed by the switch information in the image data after the synthesis.

  In order to perform the above image processing, the LCD control board 50 according to the second embodiment is provided with a combination circuit 254 and a distortion correction circuit 255 in addition to the input circuit 153 substantially the same as the first embodiment. The combination circuit 254 and the distortion correction circuit 255 are included in the processing unit 50a (see FIG. 1), and can access the first memory 151 and the second memory 152 via the data bus at high speed.

  The synthesis circuit 254 is a circuit unit including at least a part of each function of the attitude correction setting unit 53 and the output generation unit 55 (see FIG. 1). The synthesizing circuit 254 applies the respective processes of the posture correction and the main synthesizing to the pre-synthesized image DSi substantially simultaneously to generate a synthesized image DSm. The combining circuit 254 refers to each detailed information of the code CD while reading the pre-combined image DSi from one of the first memory 151 and the second memory 152. Then, the combining circuit 254 rearranges the drawing objects DS1 to DS3 included in one pre-combined image DSi based on the display positions P1s to P3s to generate one combined image DSm. The synthesizing circuit 254 writes the generated synthesized image DSm to the memory from which the pre-synthesized image DSi is read out of the first memory 151 and the second memory 152.

  The distortion correction circuit 255 is a circuit unit including at least a part of each function of the distortion correction setting unit 54 and the output generation unit 55 (see FIG. 1). Also in the second embodiment, the distortion correction table TBc having the same contents is prepared in advance in the first memory 151 and the second memory 152 so as to be readable by the distortion correction circuit 255. Unlike the first embodiment, the distortion correction table TBc of the second embodiment does not include the combined use information. The distortion correction table TBc is information defining the positional relationship between pixels between the output data DSc and the pre-combined image DSi.

  The distortion correction circuit 255 generates output data DSc by applying distortion correction to the composite image DSm. The distortion correction circuit 255 sequentially outputs the generated output data DSc to the LCD 71 and the BL control board 60. The distortion correction circuit 255 sets a memory (corresponding to the first memory 151 in FIG. 9) in which the composite image DSm is written by the combination circuit 254 as a target to read the distortion correction table TBc. In addition, the distortion correction circuit 255 sets a memory (corresponding to the second memory 152 in FIG. 9) in which the pre-synthesized image DSi has been written by the input circuit 153 as a reading target of the synthesized image DSm. The distortion correction circuit 255 extracts pixel data from the composite image DSm based on the contents of the distortion correction table TBc, and sequentially outputs the pixel data to the LCD 71 and the BL control board 60 as pixel data of the output data DSc.

  In S211 of the correction process (see FIG. 11) performed by the LCD control board 50, the drawing data DS generated by the drawing data generation unit 37 and the code CD generated by the combined use information generation unit 237 Are prepared by acquisition through communication, and the process proceeds to S212. In S212, the posture change information is acquired, and the process proceeds to S213. In S213, the code CD acquired in S211 is referred to. Then, for each of the drawn objects DS1 to DS3, it is determined based on the switch information whether drawing to the output data DSc is necessary. Further, for each of the drawing objects DS1 to DS3, the necessity of the posture correction is determined based on the correction necessity information.

  In S214, the content of the posture correction is set for the AR drawing DS1 determined to require the posture correction. Specifically, in S214, the cut-out reference position P1c is set based on the posture change information acquired in S212. On the other hand, for the non-AR drawing objects DS2 and DS3, the address at which to start reading an image is determined by referring to the drawing positions P2 and P3 at the time of pre-combination described in the code CD. Then, in S215, a combined image DSm is generated, and the process proceeds to S216.

  In S216, a distortion correction table TBc to be applied to each of the drawn objects DS1 to DS3 is selected. Then, in order to set the content of the distortion correction, reading of the selected distortion correction table TBc is started, and the process proceeds to S217. In S217, the process of generating the output data DSc from the composite image DSm is started according to the contents of the distortion correction table TBc that has been read out in S216, and the process proceeds to S218. In S218, the process of outputting the output data DSc generated in S217 to the LCD 71 and the BL control board 60 is started.

  In the second embodiment described so far, the content setting and application of the distortion correction are performed after the content setting and application of the posture correction are performed on one drawing data DS. Also in such a second embodiment, the same effect as in the first embodiment is exerted, and the virtual image Vi of the AR display object 11 can be displayed in a mode in which distortion is suppressed. In addition, each virtual image Vi of each of the non-AR display objects 12 and 13 is in a mode in which distortion is suppressed, and can be displayed without being cut off from the angle of view of the HUD 100.

  In addition, in the second embodiment, distortion correction is performed after the main composition of each of the drawn objects DS1 to DS3. As described above, according to the setting for separating the distortion correction processing from the main synthesis processing, it is easy to secure a variation of the processing that can be performed in the main synthesis. As an example, the drawing ECU 30 can describe, in the code CD, a setting for arranging the non-AR drawn objects DS2 and DS3 at a plurality of locations of the composite image DSm. As another example, the drawing ECU 30 can rearrange a plurality of drawn objects on the composite image DSm in a state of being superimposed by setting the coordinates of the display positions P2s and P3s described as the corresponding coordinate information.

  Further, in the second embodiment, a code CD including the combined use information used for the main combination is generated by the combined use information generation unit 237 and provided to the LCD control board 50. As a result, the processing load on the LCD control board 50 due to the execution of the synthesis can be reduced as compared with the case where the code CD is not prepared.

  Furthermore, the combined use information generation unit 237 of the second embodiment can change the coordinates of the display positions P2s and P3s described as detailed coordinate information in the detailed information of the non-AR drawing objects DS2 and DS3 over time. According to the above, it is possible to display a dynamic animation such as moving the virtual image Vi of the non-AR display objects 12 and 13 on the display.

  In addition, in the second embodiment, the correction necessity information indicating the necessity of the posture correction is included in the code CD. Therefore, even if the AR display object 11 and the non-AR display object 12 are mixed, the output generation unit 55 can appropriately select the AR drawing object DS1 linked to the AR display object 11. As a result, the output generation unit 55 combines the non-AR drawing objects DS2 and DS3 while performing the posture correction on the selected AR drawing object DS1. According to the above, even if the AR data 11 and the non-AR data 12 and 13 are mixed in the output data DSc, each virtual image Vi can be displayed in an appropriate mode without being cut off.

  Further, in the second embodiment, the adjustment range information indicating the maximum range of the posture correction that can be performed for the AR drawing object DS1 is included in the code CD. Therefore, when a posture change having a large amplitude occurs, the output generation unit 55 can temporarily suspend the correction for following the superimposition target. According to the above, it is possible to avoid a situation in which a part of the AR display object 11 is cut off and a strange virtual image Vi is displayed in order to correct the posture change.

  In the second embodiment, the combined use information is described by a character string. With such a format, the drawing ECU 30 can easily give detailed instructions to the LCD control board 50 when generating the composite image DSm from the pre-composite image DSi. As a result, variations of the output data DSc generated by the LCD control board 50 can be easily secured, so that a virtual image display with high user convenience can be realized. In the second embodiment, the code CD corresponds to “combined use information”, the combined image DSm corresponds to “recombined data”, and the switch information corresponds to “display setting information”.

(Third embodiment)
The third embodiment shown in FIG. 12 is a modification of the second embodiment. In the third embodiment, the drawing data generation unit 37 includes the function of the combined use information generation unit 237 (see FIG. 9) of the second embodiment. That is, in the drawing ECU 30, the code CD including the combined use information is generated by the drawing data generating unit 37. The drawing data generation unit 37 sequentially generates a code CD linked to the drawing data DS in the pre-synthesis (see S103 in FIG. 7). The drawing data generating unit 37 generates the drawing data DS in a mode in which the code CD is included in each of the drawing objects DS1 to DS3, and transmits the drawing data DS to the LCD control board 50 through the video output line 38. The code CD is embedded in, for example, a pixel in the top row of one drawing data DS. By obtaining the drawing data DS, the drawing data obtaining unit 51 (see FIG. 1) can also obtain the code CD for the main composition.

  In addition, in the third embodiment, the method of reading and writing data by the combining circuit 254 and the distortion correction circuit 255 is different from the second embodiment. Specifically, the synthesizing circuit 254 stores the generated synthesized image DSm in a memory (corresponding to the second memory 152 in FIG. 12) of the first memory 151 and the second memory 152 from which the pre-synthesized image DSi is not read. Write in. As a result, the distortion correction circuit 255 sets a memory (corresponding to the first memory 151 in FIG. 12) in which the pre-synthesized image DSi has not been written by the synthesizing circuit 254 as a target for reading the immediately preceding synthesized image DSm. . In addition, the distortion correction circuit 255 sets a memory (corresponding to the second memory 152 in FIG. 12) in which the composite image DSm is written by the combination circuit 254 as a target from which the distortion correction table TBc is read. The distortion correction circuit 255 generates output data DSc obtained by deforming the composite image DSm according to the contents of the distortion correction table TBc.

  The third embodiment described so far has the same effect as the second embodiment, and the virtual images Vi of the AR display object 11 and the non-AR display objects 12 and 13 can be displayed in a mode in which distortion is suppressed. Becomes

(Fourth embodiment)
The fourth embodiment shown in FIG. 13 is another modification of the second embodiment. The LCD control board 50 of the fourth embodiment is provided with an input synthesizing circuit 453 together with a distortion correction circuit 255 substantially the same as that of the second embodiment. The input synthesis circuit 453 has a configuration included in the processing unit 50a (see FIG. 1), and can access the first memory 151 and the second memory 152 at high speed.

  The input synthesis circuit 453 is a circuit unit including at least a part of each function of the drawing data acquisition unit 51, the posture correction setting unit 53, and the output generation unit 55 (see FIG. 1). In the fourth embodiment, in the drawing data DS obtained by the input synthesis circuit 453, the non-AR drawing objects DS2 and DS3 are arranged below the AR drawing object DS1. The input synthesizing circuit 453 generates a synthetic image DSm from the drawing data DS input through the video output line 38 based on each detailed information of the code CD (see FIG. 10) obtained through the code output line 39. Specifically, the input synthesizing circuit 453 applies the processes of the posture correction and the main synthesis to the drawing data DS substantially simultaneously based on each detailed information of the code CD and the posture change information. As a result, similarly to the synthesis circuit 254 (see FIG. 9) of the second embodiment, the input synthesis circuit 453 sets each drawing object DS1 to DS3 included in one drawing data DS with respect to each display position P1s to P3s. And a composite image DSm is generated. The input combining circuit 453 alternately switches the memory in which the generated combined image DSm is to be written, between the first memory 151 and the second memory 152 for each frame.

  The distortion correction circuit 255 sets a memory (corresponding to the first memory 151 in FIG. 13) in which the input composite circuit 453 has not written the composite image DSm as a target to read the composite image DSm one frame before. In addition, the distortion correction circuit 255 sets a memory (corresponding to the second memory 152 in FIG. 13) in which the synthesized image DSm is written by the input synthesis circuit 453 as an object to read the distortion correction table TBc. The distortion correction circuit 255 deforms the composite image DSm based on the contents of the distortion correction table TBc, and sequentially outputs the resultant to the LCD 71 and the BL control board 60 as output data DSc.

  The fourth embodiment described so far has the same effect as the second embodiment, and the virtual images Vi of the AR display object 11 and the non-AR display objects 12, 13 can be displayed in a mode in which distortion is suppressed. Becomes

  In addition, also in the fourth embodiment, as in the second embodiment, the non-AR drawing objects DS2 and DS3 are arranged at a plurality of locations of the composite image DSm, and the composite image DS1 to DS3 are superimposed on each other. It is possible to relocate to DSm. Further, in the fourth embodiment, blinking display of each of the display objects 11 to 13 and dynamic animation display can be performed.

  Further, the input synthesis circuit 453 of the fourth embodiment does not temporarily store the pre-synthesized image DSi (see FIG. 9) in the RAM in the process of obtaining the drawing data DS. The input synthesis circuit 453 can store the synthesized image DSm to which the posture correction and the main synthesis have been applied in the RAM. According to the above, the delay of one frame can be reduced as compared with the case where the pre-synthesized image DSi is stored.

  Furthermore, in the drawing data DS of the fourth embodiment, since the non-AR drawing objects DS2 and DS3 are arranged below the AR drawing object DS1, the number of blank pixels in the drawing data DS is reduced. According to the above, the calculation resources used for generating the output data DSc in the drawing ECU 30 and the LCD control board 50 can be reduced.

(Fifth embodiment)
The fifth embodiment shown in FIGS. 14 and 15 is another modification of the first embodiment. The virtual image display system according to the fifth embodiment includes the drawing ECU 30, the HUD 570, the display control device 500, and the like.

  The drawing data generation unit 37 of the drawing ECU 30 is connected to the display control device 500 via a video output line 38 and a code output line 39. The drawing data generation unit 37 draws a plurality of drawing objects DS1 to DS3 corresponding to the content selected by the content selection unit 34, as in the first embodiment. On the other hand, the drawing data generation unit 37 does not perform the pre-synthesis processing of these drawing objects DS1 to DS3. The drawing data generation unit 37 transmits each of the drawn objects DS1 to DS3 individually to the display control device 500 via the video output line 38.

  In addition, as in the second embodiment, the drawing data generation unit 37 generates a code CD used in the main synthesis in parallel with the drawing of each of the drawn objects DS1 to DS3. On the other hand, the code CD of the fifth embodiment is generated in a two-dimensional table format. One line describes detailed information of one drawing. The drawing data generation unit 37 responds to an increase in the number of pre-synthesized drawing objects by increasing the number of rows in the table. The drawing data generation unit 37 transmits the generated code CD to the display control device 500 via the code output line 39. The switch information is omitted from the code CD of the fifth embodiment (see FIG. 15).

  The HUD 570 has a configuration corresponding to the projection unit 70 (see FIG. 1) of the first embodiment, and is a virtual image formed as an AR display object 11 and a non-AR display object 12 toward the projection area PA of the windshield WS. The light Lvi is projected. The HUD 570 includes a laser module (hereinafter, “LSM”) 571 and a screen 572 in addition to the reflection optical system 73.

  The LSM 571 has a configuration including, for example, a laser light source and a MEMS (Micro Electro Mechanical Systems) scanner. The LSM 571 is controlled by the LSM control board 550 of the display control device 500 to emit light from the laser light source and scan the mirror part of the MEMS scanner. The LSM 571 draws an original image Pi on the screen 572 by scanning with a laser beam applied to the screen 572.

  The screen 572 is formed in a horizontally long rectangular plate shape from a colorless and transparent material such as glass. The screen 572 is, for example, a micro mirror array. The screen 572 is provided with a screen reflection surface that reflects the laser light. On the screen reflection surface, a large number of minute reflection convex surfaces formed by vapor deposition of a metal such as aluminum are two-dimensionally arranged. The original image Pi based on the output data DSc is displayed on the screen reflection surface by scanning with the LSM571 laser.

  The display control device 500 includes an LSM control board 550 in addition to the inertial sensor 41 and the filter circuit 42. The LSM control board 550 has a configuration corresponding to the LCD control board 50 of the first embodiment (see FIG. 1), and is an electrical configuration for controlling the virtual image display by the HUD 570. The LSM control board 550 executes the display control program by the processing unit 550a, and outputs the drawing data acquisition unit 51, the posture information acquisition unit 52, the posture correction setting unit 53, the distortion correction setting unit 54, and the output which are substantially the same as those of the first embodiment. It has a functional unit such as the generation unit 55.

  The drawing data acquisition unit 51 is electrically connected to the drawing data generation unit 37 via the video output line 38 and the code output line 39. The drawing data acquisition unit 51 is provided with the drawing objects DS1 to DS3 and the code CD required for synthesizing them in a synchronizable state from the drawing data generation unit 37. The drawing data acquisition unit 51 prepares each of the drawn objects DS1 to DS3 and the code CD by receiving through the video output line 38 and the code output line 39.

  The output generation unit 55 generates output data DSc from each of the drawn objects DS1 to DS3. After the image range cut out from the AR drawing DS1 is set by the posture correction setting unit 53, the output generation unit 55 refers to the information such as the display position and the resolution at the time of the main synthesis described in the code CD, and A process for performing the main synthesis of the drawn objects DS1 to DS3 into one image data is performed. The output generation unit 55 reflects the content of the distortion correction set by the distortion correction setting unit 54 on the composite image in parallel with the rendering of each of the rendered objects DS1 to DS3 or after synthesizing each of the rendered objects DS1 to DS3. Then, output data DSc is generated. The output generation unit 55 controls the LSM 571 so that the original image Pi based on the output data DSc is drawn on the screen 572.

  In the fifth embodiment described so far, the same effect as in the first embodiment can be obtained. That is, even if the correction for responding to the change in the attitude of the vehicle A is performed after the preparation of the AR drawing DS1, the virtual image Vi of the AR display 11 can be displayed in a manner in which distortion is suppressed. In the fifth embodiment, the processing unit 550a corresponds to a “processor”.

(Sixth embodiment)
The sixth embodiment shown in FIG. 16 is still another modification of the first embodiment. The virtual image display system according to the sixth embodiment includes a HUD 670, a display control device 600, and the like.

  The HUD 670, similar to the HUD 570 of the fifth embodiment (see FIG. 14), displays the AR display object 11 and the non-AR display object 12 so as to be visible to the driver by projecting the virtual image light Lvi toward the projection area PA. . The HUD 670 includes a DLP (Digital Light Processing, registered trademark) projector 671 in addition to the reflection optical system 73 and the screen 572.

  The DLP projector 671 has a digital mirror device (hereinafter, “DMD”) provided with a large number of micromirrors, and a projection light source that projects light toward the DMD. The DMD and the projection light source are electrically connected to the DLP control board 650 of the display control device 600. The scanning of the light by the DMD and the emission of the projection light source are integrally controlled by the DLP control board 650. The DLP projector 671 draws an original image Pi based on the output data DSc on the screen 572 under the control of the DLP control board 650. The light of the original image Pi displayed on the screen 572 is projected by the reflection optical system 73 as virtual image light Lvi onto the projection area PA.

  The display control device 600 has an electric configuration integrally having the functions of the drawing ECU 30 (see FIG. 14) and the display control device 500 (see FIG. 14) of the fifth embodiment. The display control device 600 includes a drawing board 630 and a DLP control board 650 in addition to the inertial sensor 41 and the filter circuit 42.

  The drawing substrate 630 has substantially the same function as the drawing ECU 30 (see FIG. 15), and the drawing data generation unit 37 draws a plurality of drawn objects DS1 to DS3 in a predetermined format. The drawing data generation unit 37 transmits each drawing object DS1 to DS3 individually to the DLP control board 650 through the video output line 38. The drawing data generation unit 37 of the sixth embodiment does not generate and transmit the code CD as in the first embodiment.

  The DLP control board 650 has a configuration corresponding to the LCD control board 50 (see FIG. 1) of the first embodiment, and is an electrical configuration for controlling the virtual image display by the HUD 670. The DLP control board 650 executes the display control program by the processing unit 650a, and outputs the drawing data acquisition unit 51, the posture information acquisition unit 52, the posture correction setting unit 53, the distortion correction setting unit 54, and the output which are substantially the same as those in the first embodiment. It has a functional unit such as the generation unit 55.

  The drawing data acquisition unit 51 is electrically connected to the drawing data generation unit 37 via the video output line 38. The drawing data acquisition unit 51 is provided with the drawing objects DS1 to DS3 from the drawing data generation unit 37.

  The output generation unit 55 generates output data DSc from the drawings DS1 to DS3. The output generation unit 55 refers to the synthesis use information stored in the storage device in advance, and performs the main synthesis of each of the drawn objects DS1 to DS3 into one image data. Further, the output generation unit 55 performs the process of reflecting the content of the distortion correction set by the distortion correction setting unit 54 after the main synthesis or at the same time as the main synthesis to generate the output data DSc. The output generation unit 55 controls the DLP projector 671 so that the original image Pi based on the output data DSc is drawn on the screen 572.

  In the sixth embodiment described so far, the same effects as in the first embodiment can be obtained. That is, even if the correction for responding to the change in the attitude of the vehicle A is performed after the preparation of the AR drawing DS1, the virtual image Vi of the AR display 11 can be displayed in a manner in which distortion is suppressed. In the sixth embodiment, the processing unit 650a corresponds to a “processor”.

(Other embodiments)
As described above, a plurality of embodiments of the present disclosure have been described. However, the present disclosure is not construed as being limited to the above embodiments, and may be applied to various embodiments and combinations without departing from the gist of the present disclosure. can do.

  In the present composition of the above embodiment, a plurality of drawing objects are rearranged in one image data in a layout that does not overlap each other. However, in the output data generated by the synthesis, the arrangement of each drawing object may be changed as appropriate. For example, as in Modification Example 1 shown in FIG. 17, a plurality of drawing objects DS1 to DS3 are combined in the output data DSc such that two non-AR drawing objects DS2 and DS3 are superimposed on the AR drawing object DS1. Is also good. In this case, in the image cut out from the AR drawing object DS1, a range overlapping with each of the non-AR drawing objects DS2 and DS3 may skip reading of the pixel data. Alternatively, the pixel data of the image cut out from the AR drawing DS1 may be overwritten by the pixel data of the non-AR drawing DS2 and DS3.

  As in the above embodiments, the method of transmitting information from the drawing data generation unit to the drawing data acquisition unit may be appropriately changed. For example, in Modification 2 shown in FIG. 18, the drawing data generation unit 37 and the drawing data acquisition unit 51 are connected by the video output line 38. The code CD is embedded in one of the plurality of drawn objects DS1 to DS3 (for example, the AR drawn object DS1). Even with such a configuration, the main synthesizing process of generating the output data DSc in the output generating unit 55 can be smoothly performed by the code CD.

  19, the drawing data generation unit 37 and the drawing data acquisition unit 51 are connected by both the video output line 38 and the code output line 39. The drawing data generation unit 37 pre-synthesizes each of the drawing objects DS1 to DS3, generates drawing data DS as temporary synthesis data, and transmits the drawing data DS to the drawing data acquisition unit 51 via the video output line 38. . Further, the drawing data generation unit 37 generates a code CD in parallel with the pre-synthesis, and transmits the code CD to the drawing data acquisition unit 51 via the code output line 39.

  Further, in Modification 4, similarly to the first embodiment, the drawing data generation unit and the drawing data acquisition unit are connected by a video output line. Then, the drawing data generating unit pre-synthesizes a plurality of drawing objects at a resolution and a layout specified in advance to generate drawing data (pre-synthesized image). The drawing data is acquired by the output generation unit through the video output line and the drawing data acquisition unit. The output generation unit refers to the combined use information read from the storage device and prepared in each RAM, cuts out each drawn matter drawn at a specific position of the drawn data, and generates final combined output data.

  When the code is embedded in the drawing data or the pixel of each drawing, the form of the pixel used for storing the code does not have to be the one line at the top as in the above embodiment. For example, in the fifth modification, any one pixel of the drawing data or the four corners of each drawing is used as an area for storing a code. In the sixth modification, only some pixels in one row of the uppermost row are used as an area for storing a code.

  In the seventh modification of the embodiment, only one of the AR display object and the non-AR display object is displayed as a virtual image. In such a configuration, the main synthesis in the output generation unit is not performed. In Modification 7, the posture correction setting unit sets a cutout reference position, that is, a range cut out from the AR drawing to output data, based on the posture change information. A process of applying the content of the distortion correction set by the distortion correction setting unit while extracting the pixel data of the AR drawing in order from the address of the extraction reference position is performed by the output generation unit (distortion correction circuit). Will be implemented. Note that output data for displaying a plurality of AR display objects may be generated by the output generation unit. That is, the “different display object” may be an AR display object.

  In the eighth modification of the above embodiment, a correction mechanism capable of displacing the attitude of an optical element such as a reflection optical system or a screen is provided in the projection unit. The display control device changes the attitude of the optical element under the control of the correction mechanism, and corrects a display shift of the AR display object with respect to the superimposition target. In the modification 8 as well, the content of the posture correction of the optical element is determined, and then the content of the distortion correction corresponding to the posture correction is set.

  In the ninth modification of the third embodiment, the drawing board and the DLP control board are integrally formed. As a result, after the drawing data acquisition unit is omitted, a display generation unit that also functions as a drawing data generation unit and an output generation unit is provided. The display generation unit can execute both the pre-synthesis process and the main synthesis process, and prepares each drawing and code by generating it by itself.

  In the above embodiment, the correction for canceling the vibration in the low frequency band is performed by the drawing data generation unit. However, the LCD control board according to the tenth modification is provided with a processing unit having a high calculation capability. In the modification 10, the vibration in both the low frequency band and the high frequency band is corrected by the LCD control board. As described above, if the processing unit has a sufficient calculation capability, the correction for the vibration in substantially all the bands may be performed by the LCD control board.

  In the above embodiment, the posture change such as pitching, rolling, and heave is targeted for correction, but the posture change targeted for correction may be only pitching. Note that, with respect to the roll change, a correction is performed to incline the image range cut out from the AR drawing with respect to the AR drawing. Further, for the heave change, a correction is performed to vertically displace the image range cut out from the AR drawing as in the case of the pitch change.

  The configuration of the projector that emits virtual image light is not limited to the configuration of the above-described embodiment, and can be appropriately changed. For example, in the eleventh modification of the first embodiment, an EL (Electro Luminescence) panel is provided instead of the LCD and the backlight. Further, instead of the EL panel, a projector using a display such as a plasma display panel, a cathode ray tube, and an LED can be employed. Further, a projector using LCOS (Liquid Crystal On Silicon) or the like may be employed instead of LSM and DLP.

  In the above embodiment, the projection area on which the display light image is projected is defined by the windshield. However, a projection member (for example, a combiner) for projecting the display light image may be provided separately from the windshield.

  The LCD control board according to the twelfth modification of the first to fourth embodiments has only one memory configuration as a RAM. The memory configuration of Modification 12 corresponds to only one of the first memory 151 and the second memory 152 (see FIG. 6). As in the twelfth modification, if the capacity of the RAM and the read / write speed are sufficient, a plurality of memory modules need not be connected to the bus. In the modification 12, the operation of inverting each memory configuration by the input circuit and the correction circuit is not performed.

  Further, the LCD control board of Modification 13 of the first to fourth embodiments has three or more memory configurations of Modification 12. When the access speed of each data is insufficient, a large number of memory configurations may be connected to the data bus as in the twelfth modification. In the thirteenth modification, the input circuit and the correction circuit switch the plurality of memory configurations cyclically, and write and read data.

  The inertial sensor of the above embodiment has a configuration in which a gyro sensor and an acceleration sensor are combined. However, the configuration of the inertial sensor can be changed as appropriate. For example, the inertial sensor includes a three-axis gyro sensor that detects each angular velocity in the yaw direction, the pitch direction, and the roll direction, and a three-axis acceleration sensor that detects each acceleration in the front-rear direction, the up-down direction, and the left-right direction of the vehicle. Alternatively, a six-axis motion sensor may be used. Further, the inertial sensor may be configured to include only the acceleration sensor or may be configured to include only the gyro sensor among the gyro sensor and the acceleration sensor.

  In the fourteenth modification of the first embodiment, the process of applying the posture correction is performed not by the correction circuit 154 but by the input circuit 153. The input circuit 153 writes only the image range based on the cut-out reference position P1c in the AR drawing object DS1 into one of the first memory 151 and the second memory 152 based on the posture change information.

  Further, the inertial sensor does not need to have a sensor configuration built in the HUD. The inertial sensor may be an existing configuration mounted on a vehicle as long as it is directly connected to a filter circuit or an LSM control board without being connected to a communication bus.

  The combined use information may be included in the distortion correction table TBc as in the first embodiment, or may be transmitted as a code CD as in the second embodiment. Further, the distortion correction table TBc may be in a table format as in the first embodiment, or may be a one-dimensional character string. Further, the code CD may be in a table format as in the fifth embodiment, or may be a character string as in the second embodiment. A table in which information is two-dimensionally arranged according to rows and columns is a table, and a table in which information is arranged one-dimensionally is a character string.

  At least a part of each functional unit provided on the drawing ECU and each control board may be realized by hardware (electric circuit unit) configured by combining a plurality of electric elements and the like. Alternatively, each functional unit realized by a combination of software and hardware may be provided in the drawing ECU and each control board.

  Various non-transitory tangible storage media, such as a flash memory and a hard disk, can be used as the storage devices provided in the drawing ECU and each control board. In addition, the storage medium that stores the program related to the virtual image display is not limited to the storage medium of each configuration mounted on the vehicle, and may be an optical disk serving as a copy source to the storage medium, a hard disk drive of a general-purpose computer, or the like. Is also good.

  The control unit and the method thereof according to the present disclosure may be realized by a dedicated computer configuring a processor programmed to execute one or a plurality of functions embodied by a computer program. Alternatively, the devices and techniques described in this disclosure may be implemented by dedicated hardware logic. Alternatively, the device and the technique described in the present disclosure may be implemented by one or more dedicated computers configured by a combination of a processor executing a computer program and one or more hardware logic circuits. Further, the computer program may be stored in a computer-readable non-transitional tangible recording medium as instructions to be executed by a computer.

A vehicle, CD code (synthesis use information), DS drawing data (temporary synthesis data), DSi pre-synthesis image (temporary synthesis data), DS1 AR drawing (specific drawing data), DS2 non-AR drawing (other drawing data) , DSc output data (re-synthesized data), DSm synthesized image (re-synthesized data), HB high frequency band, PA projection area, TBc distortion correction table (synthesis utilization information), Vi virtual image, 11 AR display (specific display) , 12 Non-AR display object (separate display object), 50a, 550a, 650a, processing unit (processor), 51 drawing data acquisition unit (drawing data preparation unit), 52 posture information acquisition unit, 53 posture correction setting unit, 54 distortion Correction setting unit, 55 output generation unit (image synthesis unit), 60 BL control board (luminance control unit), 70 projection units, 570, 570 HUD , 72 backlight, 100 HUD (display control device), 500,600 display control device

Claims (17)

A display control device that is used in a vehicle (A) in which a virtual image (Vi) is displayed by projecting light onto a projection area (PA) and controls display of the virtual image,
A drawing data preparation unit (51) for preparing specific drawing data (DS1) of a specific display object (11) that is displayed as a virtual image in association with a specific object in the foreground of the vehicle;
A posture information acquisition unit (52) for acquiring posture information related to the posture of the vehicle,
Attitude correction setting for setting the content of the attitude correction to be performed on the specific drawing data using the attitude information so that a display shift of the specific display object with respect to the specific object due to a change in the attitude of the vehicle is reduced. Part (53),
After the contents of the posture correction are set by the posture correction setting unit so that distortion of the virtual image due to projection of light onto the projection area is reduced, distortion correction performed on the specific drawing data is performed. A display control device comprising: a distortion correction setting unit (54) for setting contents. The drawing data preparation unit further prepares another drawing data (DS2, DS3) of another display object (12) different from the specific display object,
The display control device according to claim 1, further comprising: an image combining unit (55) that combines the specific drawing data and the different drawing data after the content of the posture correction is set by the posture correction setting unit. . A display control device that is used in a vehicle (A) in which a virtual image (Vi) is displayed by projecting light onto a projection area (PA) and controls display of the virtual image,
Specific drawing data (DS1) of a specific display object (11) displayed as a virtual image in association with a specific object in the foreground of the vehicle, and separate drawing data (DS2) of another display object (12) different from the specific display object , DS3), a drawing data preparation unit (51),
A posture information acquisition unit (52) for acquiring posture information related to the posture of the vehicle,
Attitude correction setting for setting the content of attitude correction to be performed on the specific drawing data using the attitude information so that a display shift of the specific display object with respect to the specific object due to a change in the attitude of the vehicle is reduced. Part (53),
An image synthesizing unit (55) for synthesizing the specific drawing data to which the posture correction of the content set by the posture correction setting unit is applied, and the separate drawing data;
A distortion correction setting unit (54) configured to set details of distortion correction to be applied to the specific drawing data and the different drawing data so that distortion of the virtual image caused by projection of light onto the projection area is reduced; A display control device comprising:   4. The display control according to claim 2, wherein the image combining unit combines the specific drawing data and the different drawing data while applying the content of the distortion correction set by the distortion correction setting unit. 5. apparatus.   4. The image combining unit combines the specific drawing data and the different drawing data while applying the content of the posture correction set by the posture correction setting unit to the specific drawing data. 5. 3. The display control device according to 1.   The display control according to claim 5, wherein the image combining unit performs a process of applying the content of the distortion correction set by the distortion correction setting unit after combining the specific drawing data and the different rendering data. apparatus.   The display control according to any one of claims 2 to 6, wherein the drawing data preparation unit further prepares synthesis use information (TBc, CD) used for synthesizing the specific drawing data and the different drawing data. apparatus. The drawing data preparation unit prepares temporary combined data (DS, DSi) that combines the specific drawing data and the different drawing data into one,
The image synthesizing unit generates re-synthesized data (DSc, DSm) obtained by synthesizing the specific drawing data cut out from the temporary synthesized data and the different drawing data again in a positional relationship based on the content of the posture correction. And
8. The combined use information according to claim 7, wherein at least corresponding coordinate information that defines a correspondence relationship between the coordinates of the different drawing data in the temporary combined data and the coordinates of the different drawing data in the recombined data is included. 9. The display control device according to the above.   9. The display control device according to claim 8, wherein the coordinates of the different drawing data change with time in the corresponding coordinate information.   10. The composite use information according to claim 7, wherein the combined use information includes at least correction necessity information indicating whether correction is required for the specific drawing data and the separate drawing data due to a change in posture. 11. Display control device.   The display control device according to claim 7, wherein the combined use information includes at least adjustment range information indicating a maximum range of the posture correction that can be performed on the specific drawing data. The combined use information includes at least display setting information for setting display and non-display for at least one of the specific drawing data and the separate drawing data,
12. The image composition unit according to claim 7, wherein the image compositing unit does not include, among the specific rendering data and the different rendering data, rendering data designated to be non-display by the display setting information as a compositing target. A display control device according to the item.   The display control device according to any one of claims 7 to 12, wherein the combined use information is prepared in one of a character string format and a table format. The posture information acquisition unit acquires the posture information about a posture change belonging to a high frequency band (HB) exceeding a specific frequency,
The attitude correction setting unit sets the content of the attitude correction using the attitude information of the high frequency band (HB) such that a shift of the specific display object is reduced. The display control device according to claim 1. The virtual image light is projected by a projection unit (70) having a backlight (72) whose emission luminance can be adjusted for each area,
15. The brightness control unit (60), further comprising: a brightness control unit (60) that controls light emission brightness of each area in the backlight in accordance with a form of the specific drawing data in which the distortion correction is reflected. 3. The display control device according to 1. A display control program used in a vehicle (A) in which a virtual image (Vi) is displayed by projecting light onto a projection area (PA), and controlling display of the virtual image.
At least one processor (50a, 550a, 650a)
A drawing data preparation unit (51) for preparing specific drawing data (DS1) of a specific display object (11) that is displayed as a virtual image in association with a specific object in the foreground of the vehicle;
A posture information acquisition unit (52) for acquiring posture information related to the posture of the vehicle,
A posture correction setting unit configured to set contents of a posture correction to be performed on the specific drawing data using the posture information so that a shift of the specific display object with respect to the specific object due to a change in the posture of the vehicle is reduced. (53)
After the content of the posture correction is set by the posture correction setting unit, so that distortion of the virtual image accompanying the projection of light onto the projection area is reduced, distortion correction performed on the specific drawing data is performed. A display control program functioning as a configuration including a distortion correction setting unit (54) for setting contents. A display control program used in a vehicle (A) in which a virtual image (Vi) is displayed by projecting light onto a projection area (PA), and controlling display of the virtual image.
At least one processor (50a, 550a, 650a)
Specific drawing data (DS1) of a specific display object (11) displayed as a virtual image in association with a specific object in the foreground of the vehicle, and separate drawing data (DS2) of another display object (12) different from the specific display object , DS3), a drawing data preparation unit (51),
A posture information acquisition unit (52) for acquiring posture information related to the posture of the vehicle,
A posture correction setting unit configured to set contents of a posture correction to be performed on the specific drawing data using the posture information so that a shift of the specific display object with respect to the specific object due to a change in the posture of the vehicle is reduced. (53)
An image synthesizing unit (55) for synthesizing the specific drawing data to which the posture correction of the content set by the posture correction setting unit is applied, and the separate drawing data;
A distortion correction setting unit (54) configured to set details of distortion correction to be applied to the specific drawing data and the different drawing data, so that distortion of the virtual image caused by projection of light onto the projection area is reduced; A display control program functioning as a configuration including

Images