US20210109355A1 - Head-up display calibration - Google Patents
Head-up display calibration Download PDFInfo
- Publication number
- US20210109355A1 US20210109355A1 US16/599,777 US201916599777A US2021109355A1 US 20210109355 A1 US20210109355 A1 US 20210109355A1 US 201916599777 A US201916599777 A US 201916599777A US 2021109355 A1 US2021109355 A1 US 2021109355A1
- Authority
- US
- United States
- Prior art keywords
- test image
- fiducial
- image
- virtual image
- calibrated
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012360 testing method Methods 0.000 claims abstract description 149
- 238000000034 method Methods 0.000 claims description 18
- 238000004519 manufacturing process Methods 0.000 claims description 7
- 230000004438 eyesight Effects 0.000 claims description 4
- 230000003287 optical effect Effects 0.000 description 7
- 230000006870 function Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 230000004313 glare Effects 0.000 description 3
- 230000003190 augmentative effect Effects 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 239000000853 adhesive Substances 0.000 description 1
- 230000001070 adhesive effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000003750 conditioning effect Effects 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0179—Display position adjusting means not related to the information to be displayed
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
- B60K35/20—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor
- B60K35/21—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor using visual output, e.g. blinking lights or matrix displays
- B60K35/23—Head-up displays [HUD]
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
- B60K35/90—Calibration of instruments, e.g. setting initial or reference parameters; Testing of instruments, e.g. detecting malfunction
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/62—Optical apparatus specially adapted for adjusting optical elements during the assembly of optical systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
- H04N9/3179—Video signal processing therefor
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
- H04N9/3179—Video signal processing therefor
- H04N9/3185—Geometric adjustment, e.g. keystone or convergence
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
- H04N9/3191—Testing thereof
- H04N9/3194—Testing thereof including sensor feedback
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
- B60K2360/20—Optical features of instruments
- B60K2360/31—Virtual images
-
- B60K2370/1529—
-
- B60K2370/31—
-
- B60K2370/92—
-
- B60K2370/95—
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
- B60K35/40—Instruments specially adapted for improving the visibility thereof to the user, e.g. fogging prevention or anti-reflection arrangements
- B60K35/415—Glare prevention
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
- B60K35/50—Instruments characterised by their means of attachment to or integration in the vehicle
- B60K35/53—Movable instruments, e.g. slidable
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
Definitions
- This disclosure is related to head-up displays (HUD).
- a HUD is a display that presents data in a partially transparent manner and at a position allowing a user to see it without having to look away from his/her usual viewpoint (e.g., substantially forward).
- HUDs are now used in commercial aircraft, automobiles, computer gaming, and other applications.
- HUDs may be used to project virtual images or vehicle parameter data in front of the vehicle windshield or surface so that the image is in or immediately adjacent to the operator's line of sight.
- Vehicle HUD systems can project data based on information received from operating components (e.g., sensors) internal to the vehicle to, for example, notify users of lane markings, identify proximity of another vehicle, or provide nearby landmark information.
- HUDs may also receive and project information from information systems external to the vehicle, such as a navigational system on a smartphone.
- Navigational information presented by the HUD may include, for example, projecting distance to a next turn and current speed of the vehicle as compared to a speed limit, including an alert if the speed limit is exceeded.
- External system information advising what lane to be in for an upcoming maneuver or warning the user of potential traffic delays can also be presented on the HUD.
- HUDs may also be employed in augmented reality displays or enhanced vision systems that identify, index, overlay or otherwise reference objects and road features including infrastructure. Such advanced systems require precision alignment of HUD images relative to the observers within the vehicle and objects within their field of view.
- a method for calibrating a virtual image projected from a vehicle HUD system onto a windshield includes displaying a fiducial at a predetermined location upon the windshield, projecting a test image from a test image generator within the HUD system, aligning the test image with the fiducial, determining a positional deviation from a test image calibrated position required to align the test image with the fiducial, and based upon the positional deviation and a known positional relationship between a virtual image calibrated position and the test image calibrated position, aligning the virtual image with the fiducial.
- displaying the fiducial includes projecting the fiducial from an image projector affixed to a predetermined vehicle component.
- displaying the fiducial includes affixing the fiducial to the windshield.
- aligning the test image with the fiducial includes manually adjusting the test image.
- aligning the test image with the fiducial includes autonomously adjusting the test image.
- a method for calibrating a virtual image projected from a vehicle HUD system onto a windshield includes displaying a fiducial at a predetermined location upon the windshield, projecting a test image from a test image generator within the HUD system, aligning the test image with the fiducial by moving the test image from a predetermined test image calibrated position to a test image alignment position, determining a test image alignment offset based upon the predetermined test image calibrated position and the test image alignment position, determining a test image offset based upon the predetermined test image calibrated position and a predetermined virtual image calibrated position, determining a desired virtual image offset based upon the test image alignment offset and the test image offset, and aligning the virtual image with the fiducial by moving the virtual image from the predetermined virtual image calibrated position to the test image alignment position.
- displaying the fiducial includes projecting the fiducial from an image projector affixed to a predetermined vehicle component.
- displaying the fiducial includes affixing the fiducial to the windshield.
- aligning the test image with the fiducial includes manually adjusting the test image.
- aligning the test image with the fiducial includes autonomously adjusting the test image
- an apparatus for calibrating a virtual image projected from a vehicle HUD system onto a windshield includes a HUD having a virtual image generator for projecting a virtual image and a test image generator for projecting a test image, and a fiducial displayed upon the windshield at a predetermined location.
- the apparatus further includes a control module configured to project the test image upon the windshield from the test image generator at a calibrated test image position, align the test image with the fiducial based upon alignment inputs received by the control module, and determine a positional deviation of the test image from the calibrated test image position required to align the test image with the fiducial.
- the control module is further configured to determine a positional deviation of the virtual image from the calibrated virtual image position required to align the virtual image with the fiducial based upon the positional deviation of the test image and a known positional relationship between a calibrated virtual image position and the calibrated test image position, and set the calibrated virtual image position to align with the fiducial based upon the positional deviation of the virtual image from the calibrated virtual image position required to align the virtual image with the fiducial.
- the apparatus further includes at least one user interface for receiving inputs from an operator, wherein the control module is further configured to receive the inputs from the at least one user interface, and wherein the alignment inputs received by the control module include manual inputs from the operator guiding the movement of the test image into alignment with the fiducial.
- the apparatus further includes at least one user interface for receiving inputs from an operator and an image sensor, wherein the control module is further configured to receive the inputs from the at least one user interface, and wherein the alignment inputs received by the control module include a command to the control module to iteratively move the test image into alignment with the fiducial as determined with the image sensor.
- the apparatus further includes an image projector projecting the fiducial upon the windshield.
- the apparatus further includes an image projector projecting the fiducial upon the windshield.
- the apparatus further includes a fixture attached to a vehicle component wherein the image projector is attached to the fixture.
- the image projector includes an in-plant production tooling including a vision system.
- FIG. 1 illustrates a HUD system in accordance with the present disclosure
- FIG. 2 illustrates various positional relationships among various feature positions in accordance with the present disclosure
- FIG. 3 illustrates a forward-looking view of a portion of the interior of a passenger vehicle substantially along a longitudinal centerline in accordance with the present disclosure
- FIG. 4 illustrates an exemplary apparatus for calibrating the position of a virtual image to a vehicle reference frame in accordance with the present disclosure
- FIG. 5 illustrates an exemplary method for calibrating the position of a virtual image in accordance with the present disclosure.
- control module, module, control, controller, control unit, processor and similar terms mean any one or various combinations of one or more of Application Specific Integrated Circuit(s) (ASIC), electronic circuit(s), central processing unit(s) (preferably microprocessor(s)) and associated memory and storage (read only memory (ROM), random access memory (RAM), electrically programmable read only memory (EPROM), hard drive, etc.) or microcontrollers executing one or more software or firmware programs or routines, combinational logic circuit(s), input/output circuitry and devices (I/O) and appropriate signal conditioning and buffer circuitry, high speed clock, analog to digital (A/D) and digital to analog (D/A) circuitry and other components to provide the described functionality.
- ASIC Application Specific Integrated Circuit
- ROM read only memory
- RAM random access memory
- EPROM electrically programmable read only memory
- microcontrollers executing one or more software or firmware programs or routines, combinational logic circuit(s), input/output circuitry and devices (I/O) and appropriate signal conditioning and buffer circuit
- a control module may include a variety of communication interfaces including point-to-point or discrete lines and wired or wireless interfaces to networks including wide and local area networks, on vehicle controller area networks and in-plant and service related networks. Functions of a control module as set forth in this disclosure may be performed in a distributed control architecture among several networked control modules.
- Software, firmware, programs, instructions, routines, code, algorithms and similar terms mean any controller executable instruction sets including calibrations, data structures, and look-up tables.
- a control module has a set of control routines executed to provide described functions. Routines are executed, such as by a central processing unit, and are operable to monitor inputs from sensing devices and other networked control modules and execute control and diagnostic routines to control operation of actuators. Routines may be executed at regular intervals during ongoing engine and vehicle operation. Alternatively, routines may be executed in response to occurrence of an event.
- FIG. 1 shows a head-up display (HUD) 101 in accordance with the present disclosure.
- the HUD 101 includes a control module 102 , a virtual image generator 105 including a virtual image generating module 104 including a first light source 106 and a mirror actuator 108 , and a test image generator 103 including a second light source 110 and a test image optic assembly 112 .
- the control module 102 controls the operation of the virtual image generating module 104 and first light source 106 via control line 145 to generate a virtual image.
- the control module 102 also controls the operation of the second light source 110 via control line 147 to generate a test image on windshield 116 .
- the control module 102 controls the operation of the mirror actuator 108 via control line 143 to rotate or tilt a mirror 117 and adjust where on windshield the HUD virtual image is projected.
- the mirror actuator may include a motor, gears, shafts, and/or other components to change the position and/or orientation of the mirror 117 .
- the mirror 117 may magnify the image generated by the virtual image generating module 104 and/or correct for distortions associated with the windshield 116 .
- the HUD 101 includes the virtual image generating module 104 , the mirror actuator 108 , the mirror 117 and the one or more mirrors 122 .
- the HUD 101 also includes the second light source 110 and the test image optic assembly 112 .
- the virtual image generating module 104 , the mirror actuator 108 , the mirror 117 , and the one or more mirrors 122 are implemented in a separate housing than the second light source 110 and the test image optic assembly 112 .
- the virtual image generating module 104 , the mirror actuator 108 , the mirror 117 , the one or more mirrors 122 , the second light source 110 and the test image optic assembly 112 are implemented in a single housing.
- the virtual image generating module 104 may include the first light source 106 and a display and lens assembly 120 .
- the first light source 106 generates a virtual image light beam 121 including graphic images that are projected onto a display of the display and lens assembly 120 .
- the virtual image light beam 121 is then directed at a series of one or more mirrors 122 .
- the one or more mirrors 122 may include, for example, a fold mirror.
- the one or more mirrors 122 may be used for packaging reasons.
- the virtual image light beam 121 is reflected at the mirror 117 and then reflected through light and glare traps 124 to the windshield 116 .
- the virtual image light beam 121 is displayed on the windshield.
- the light and glare traps 124 may filter and thus prevent, for example, sun light (or ambient light) from being reflected from the windshield 116 towards the mirror 117 and minimize effects of glare.
- the second light source 110 may include a laser, one or more light emitting diodes (LEDs), or other test image generating light source.
- the second light source 110 generates a test image light beam 126 , which is directed through the test image optic assembly 112 , off adjustable mirror 160 , and reflected off of the windshield 116 .
- the control module 102 controls operation of the adjustable mirror 160 via control line 149 .
- Adjustable mirror 160 may include an x,y galvanic mirror allowing for positional adjustments of the test image.
- the HUD 101 may further include and/or be connected to a manual controller 136 including switches (buttons, paddles, sliders, rotaries, joysticks or the like) 138 .
- the HUD 101 may also include and/or be connected to a display, seat motors, or seat switches (not separately illustrated).
- the display may be, for example, a touchscreen, an infotainment display located in a center console of a vehicle, or other display.
- the seat motors are used to position one or more seats.
- the control module 102 may control operations of the seat motors based on user inputs via the seat switches and/or seat settings stored in memory.
- the manual controller 136 may be used by a user to manually adjust height of virtual images provided by the virtual image generating module 104 via the switches 138 .
- a display touchscreen may provide a user interface (UI) for manual adjustments of the HUD virtual image during end user application such as by a vehicle occupant.
- UI user interface
- Such a display, seat switches and switches 138 may be referred to as input devices and/or interfaces or more generally as a user interface.
- the input devices may provide a user interface for manual adjustments of the test image or to establish operator intent or control of an automated or partially automated alignment procedure.
- the virtual image light beam 121 may follow a main ray path of a HUD virtual image from a center of a HUD display, which is part of the display and lens assembly 120 to the windshield 116 .
- the main ray path is referred to as a gut ray.
- the test image light beam 126 for the test image may follow a different path than the virtual image light beam 121 .
- the path of the test image light beam 126 may or may not be parallel to a portion of or the entire path of the virtual image light beam 121 .
- the light beams 121 , 126 do not need to follow the same optical path.
- Certain HUD applications require precise alignment of the virtual images produced by the HUD. Placement of simple information presentation on the windshield, such as conventional engine gauge display, is not positionally critical.
- augmented reality systems intended to improve driver or occupant situational awareness by identifying, overlaying, or otherwise enhancing visibility of objects or features located on a road scene require virtual image placement taking into consideration the observer's eye position, the scene object position and the vehicle windshield position. In order to enable robust virtual image placement fidelity in such systems, the virtual image position must be calibrated relative to the vehicle reference frame.
- a temporary fiducial is displayed upon the windshield of the vehicle at a predetermined position.
- the fiducial may be physically affixed to the windshield or may be virtually displayed thereon such as by light projection.
- the term displayed includes physically affixed to or virtually displayed on such as by light projection.
- FIG. 3 illustrates a forward-looking view of a portion of the interior of a passenger vehicle 301 substantially along a longitudinal centerline.
- Dashboard assembly 309 extends laterally within the vehicle cabin substantially forward of the doors and beneath and between the A-pillars 307 .
- A-pillars 307 frame the windshield 116 .
- Steering wheel 303 is located forward of the driver seating 450 ( FIG.
- Fiducial 311 is illustrated affixed or projected upon windshield 116 substantially centered with respect to the steering wheel 303 though not limited to such placement.
- Fiducial 311 may be a simple geometric graphic such as a grid, concentric circles or, as illustrated, a simple cross-hair pattern formed by a pair of normally intersecting lines.
- a physically affixed fiducial such may be affixed by any practical means and take such exemplary forms as mild adhesive backed or static cling material which may be readily removed.
- any of a variety of simple fixturing or gauging may be employed.
- the A-pillars may provide fixture mounting locations for precise placement of an affixed fiducial relative to the A-pillars.
- a simple template aligned to features on the windshield may be employed for fiducial adhesion to the windshield.
- a light projector may be employed to project a virtual fiducial upon the windshield. The location of such a virtual fiducial at the predetermined location may be ensured by fixturing a projector, for example a so-called pico-projector, to the vehicle reference frame.
- One or both of the A-pillars may be employed for fixturing as can the steering wheel 303 as will be later described.
- the projector or other light source for producing and projecting the fiducial may be part of the in-plant production tooling including, for example, a vision system for locating the projected fiducial on the windshield relative to the vehicle reference frame.
- a vision system for locating the projected fiducial on the windshield relative to the vehicle reference frame.
- fiducial image projection projector, image projector, or fiducial projector are understood to mean any suitable light image projector (LED, LASER, lamp based, etc.).
- Assembly of HUD 101 into the vehicle may be accomplished by the installation of an entire dash assembly into which the HUD 101 has been assembled as part of a subassembly process or build-up of the dash assembly 309 .
- a smaller subassembly including an instrument cluster pod may contain the HUD 101 and may be assembled to the dash assembly already installed within the vehicle.
- the HUD 101 may be assembled into the instrument cluster, dash assembly or upper dash pad as a separate assembly component.
- a test image 305 is projected upon the windshield 116 subsequent to the HUD 101 installation within the vehicle.
- Test image 305 is produced by test image generator 103 of HUD 101 and may be the same simple geometric graphic as fiducial 311 or a complementary graphic such as an outline of the fiducial graphic. In the illustrated embodiment, test image 305 is also a simple cross-hair pattern formed by a pair of normally intersecting lines.
- the virtual image may have a calibrated alignment position specification corresponding to the predetermined fiducial position 211 ( FIG. 2 ). However, numerous tolerance stack ups may result in the projected virtual image being displaced from the predetermined fiducial position.
- the test image 305 may have a projected position displaced from the predetermined fiducial position 211 as illustrated by the separation of the test image 305 from the fiducial 311 displayed upon the windshield 116 .
- HUD 101 is illustrated including virtual image generator 105 and test image generator 103 .
- Virtual image generator 105 projects virtual image light beam 121 toward windshield 116 .
- Test image generator 103 also projects test image light beam 126 toward windshield 116 .
- the HUD 101 projections from virtual image generator 105 and test image generator 103 are positionally controllable within limits in a substantially planar, two-dimensional space illustrated in FIG. 2 by horizontal x-axis and vertical y-axis.
- the virtual image generator 105 projects the virtual image to a calibrated virtual image position 203 which is initially set during HUD manufacture and stored in a non-volatile memory location within HUD controller 102 .
- the test image generator 103 projects the test image 305 ( FIG. 3 ) to a calibrated test image position 205 which also is initially set during HUD manufacture and stored in a non-volatile memory location within HUD controller 102 .
- the x,y positional difference between the calibrated virtual image position 203 and the calibrated test image position 205 is known from these design parameters and may be referred to as a test image offset 230 shown as vector ⁇ right arrow over (A) ⁇ with tail at calibrated virtual image position 203 .
- the virtual image from the virtual image generator 105 is to be positioned at the fiducial position 211 .
- test image 305 ( FIG. 3 ) is projected upon the windshield 116 at calibrated test image position 205 .
- the difference between the calibrated test image position 205 and the fiducial position 211 may be referred to as test image alignment offset 225 shown as vector ⁇ right arrow over (B) ⁇ with tail at calibrated test image position 205 .
- Test image 305 position may be adjusted such that the test image 305 overlays and aligns with the fiducial 311 .
- test image alignment offset 225 shown as vector ⁇ right arrow over (B) ⁇ may be determined.
- the virtual image is projected upon the windshield 116 at calibrated virtual image position 203 .
- the difference between the calibrated virtual image position 203 and the fiducial position 211 may be referred to as desired virtual image offset 235 shown as vector ⁇ right arrow over (C) ⁇ with tail at calibrated virtual image position 205 .
- the difference between the calibrated virtual image position 203 and the fiducial position 211 may be determined therefore from the x,y positional deviation required to align test image 305 with fiducial 311 and the x,y positional difference between the calibrated virtual image position 203 and the calibrated test image position 205 .
- a method for calibrating the position of the virtual image 501 includes displaying a fiducial at a predetermined location upon the windshield 503 .
- Display of the fiducial may be in accordance with affixing a fiducial to the windshield or projecting a fiducial image upon the windshield.
- a test image is projected from a test image generator within the HUD system 505 onto the windshield at a test image calibrated position known in accordance with the HUD calibration.
- Test image projection may be initiated by an operator through any of a variety of user interfaces as previously set forth in the disclosure.
- the test image is aligned with the fiducial 507 .
- Alignment may be manually such as through any of a variety of user interfaces as previously set forth in the disclosure or as established in accordance with operator intent or control of an automated or partially automated alignment procedure which may be invoked through any of a variety of user interfaces as previously set forth in the disclosure.
- the positional deviation required to align the test image with the fiducial is determined from the calibrated position of the test image and the position of the test image when aligned with the fiducial 509 .
- the virtual image has a known virtual image calibrated position in accordance with the HUD calibration.
- the test image calibrated position is known, and the positional deviation required to align the test image with the fiducial is also known, the deviation in the location of the virtual image required to align the virtual image with the fiducial is readily determined. This deviation may be used to align the virtual image to the fiducial position 511 . Alignment of the virtual image may be implemented in various ways depending upon the particular hardware configuration of the HUD and more particularly of the virtual image generator 105 .
- alignment of the virtual image may be effected by rotation of an interior mirror in the case of a DLP HUD system, application of a lens function to the phase hologram in the case of a holographic HUD system, or image translation on an LCD display having reserve pixels or x,y translation of the entire LCD display in a LCD HUD system.
- an apparatus for calibrating the position of the virtual image to a vehicle reference frame in accordance with the present disclosure is illustrated.
- the embodiment illustrated is well suited to the controlled production environment of a final assembly plant though such apparatus may be employed in a field service environment.
- a vehicle interior includes a dash assembly 309 wherein HUD 101 is contained.
- Steering column assembly 405 is attached to structural brackets beneath the dash assembly 309 .
- Steering wheel 303 is attached to an end of steering column assembly 405 .
- Steering column assembly 405 may include a tilt/telescoping mechanism which allows for selectively adjusting the steering column/wheel between extreme raised and lowered positions and between extreme fore/aft positions.
- HUD 101 is configured to project a test image via test image light beam 126 toward windshield 116 and a virtual image via virtual image light beam 126 also toward windshield 116 .
- a calibration set-up 408 includes a calibration fixture 411 and optical calibration assembly 407 secured thereto.
- Optical calibration assembly may include one or both of a fiducial projector 417 and an image sensor 419 .
- a fiducial projector 417 for example a pico-projector, is useful in an arrangement wherein a projected fiducial is desired.
- An image sensor for example a CCD or CMOS camera, is useful in an arrangement wherein the alignment of the test image to the fiducial is automated.
- the calibration set-up 408 may also include a display or other user/control interface 415 which may be a part of or separate from the optical calibration assembly, for example an integrated display or a separate smart device for example a tablet, laptop computer or the like.
- the calibration set-up may also include one or more control modules adapted to provide a calibration application including UI and instructional prompts, and communication and networking functions. Such a control module is preferably associated with one or more of the calibration assembly or separate smart device. Communications among the HUD 101 , optical calibration assembly 407 , user/control interface 415 , and various in-plant production control and information systems may be carried out by any appropriate wired or wireless means including for example the vehicle controller area network.
- the steering column assembly 405 is placed into extreme tilt and telescoping positions to reduce the variability of the steering wheel position relative to the vehicle reference frame.
- the calibration fixture 411 and optical calibration assembly 407 is secured to the steering wheel such that the fiducial projector 417 will display the fiducial image upon the windshield at the desired position.
- calibration fixture 411 and optical calibration assembly 407 secured thereto may be secured to other structures within the vehicle providing a repeatable positional relationship to the vehicle reference frame, for example one or more of A-pillars, B-pillars, door opening frame, etc.
- first and second elements can be a direct relationship where no other intervening elements are present between the first and second elements, but can also be an indirect relationship where one or more intervening elements are present (either spatially or functionally) between the first and second elements.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Chemical & Material Sciences (AREA)
- Combustion & Propulsion (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
- General Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Geometry (AREA)
- Instrument Panels (AREA)
Abstract
Calibrating a virtual image projected from a vehicle HUD system onto a windshield includes displaying a fiducial at a predetermined location upon the windshield, projecting a test image from a test image generator within the HUD system, aligning the test image with the fiducial, determining a positional deviation from a test image calibrated position required to align the test image with the fiducial, and based upon the positional deviation and a known positional relationship between a virtual image calibrated position and the test image calibrated position, aligning the virtual image with the fiducial.
Description
- This disclosure is related to head-up displays (HUD).
- A HUD is a display that presents data in a partially transparent manner and at a position allowing a user to see it without having to look away from his/her usual viewpoint (e.g., substantially forward). Although developed for military use, HUDs are now used in commercial aircraft, automobiles, computer gaming, and other applications.
- Within vehicles, HUDs may be used to project virtual images or vehicle parameter data in front of the vehicle windshield or surface so that the image is in or immediately adjacent to the operator's line of sight. Vehicle HUD systems can project data based on information received from operating components (e.g., sensors) internal to the vehicle to, for example, notify users of lane markings, identify proximity of another vehicle, or provide nearby landmark information.
- HUDs may also receive and project information from information systems external to the vehicle, such as a navigational system on a smartphone. Navigational information presented by the HUD may include, for example, projecting distance to a next turn and current speed of the vehicle as compared to a speed limit, including an alert if the speed limit is exceeded. External system information advising what lane to be in for an upcoming maneuver or warning the user of potential traffic delays can also be presented on the HUD.
- HUDs may also be employed in augmented reality displays or enhanced vision systems that identify, index, overlay or otherwise reference objects and road features including infrastructure. Such advanced systems require precision alignment of HUD images relative to the observers within the vehicle and objects within their field of view.
- In one exemplary embodiment, a method for calibrating a virtual image projected from a vehicle HUD system onto a windshield includes displaying a fiducial at a predetermined location upon the windshield, projecting a test image from a test image generator within the HUD system, aligning the test image with the fiducial, determining a positional deviation from a test image calibrated position required to align the test image with the fiducial, and based upon the positional deviation and a known positional relationship between a virtual image calibrated position and the test image calibrated position, aligning the virtual image with the fiducial.
- In addition to one or more of the features described herein, displaying the fiducial includes projecting the fiducial from an image projector affixed to a predetermined vehicle component.
- In addition to one or more of the features described herein, displaying the fiducial includes affixing the fiducial to the windshield.
- In addition to one or more of the features described herein, aligning the test image with the fiducial includes manually adjusting the test image.
- In addition to one or more of the features described herein, aligning the test image with the fiducial includes autonomously adjusting the test image.
- In another exemplary embodiment, a method for calibrating a virtual image projected from a vehicle HUD system onto a windshield includes displaying a fiducial at a predetermined location upon the windshield, projecting a test image from a test image generator within the HUD system, aligning the test image with the fiducial by moving the test image from a predetermined test image calibrated position to a test image alignment position, determining a test image alignment offset based upon the predetermined test image calibrated position and the test image alignment position, determining a test image offset based upon the predetermined test image calibrated position and a predetermined virtual image calibrated position, determining a desired virtual image offset based upon the test image alignment offset and the test image offset, and aligning the virtual image with the fiducial by moving the virtual image from the predetermined virtual image calibrated position to the test image alignment position.
- In addition to one or more of the features described herein, displaying the fiducial includes projecting the fiducial from an image projector affixed to a predetermined vehicle component.
- In addition to one or more of the features described herein, displaying the fiducial includes affixing the fiducial to the windshield.
- In addition to one or more of the features described herein, aligning the test image with the fiducial includes manually adjusting the test image.
- In addition to one or more of the features described herein, aligning the test image with the fiducial includes autonomously adjusting the test image
- In yet another exemplary embodiment, an apparatus for calibrating a virtual image projected from a vehicle HUD system onto a windshield includes a HUD having a virtual image generator for projecting a virtual image and a test image generator for projecting a test image, and a fiducial displayed upon the windshield at a predetermined location. The apparatus further includes a control module configured to project the test image upon the windshield from the test image generator at a calibrated test image position, align the test image with the fiducial based upon alignment inputs received by the control module, and determine a positional deviation of the test image from the calibrated test image position required to align the test image with the fiducial. The control module is further configured to determine a positional deviation of the virtual image from the calibrated virtual image position required to align the virtual image with the fiducial based upon the positional deviation of the test image and a known positional relationship between a calibrated virtual image position and the calibrated test image position, and set the calibrated virtual image position to align with the fiducial based upon the positional deviation of the virtual image from the calibrated virtual image position required to align the virtual image with the fiducial.
- In addition to one or more of the features described herein the apparatus further includes at least one user interface for receiving inputs from an operator, wherein the control module is further configured to receive the inputs from the at least one user interface, and wherein the alignment inputs received by the control module include manual inputs from the operator guiding the movement of the test image into alignment with the fiducial.
- In addition to one or more of the features described herein the apparatus further includes at least one user interface for receiving inputs from an operator and an image sensor, wherein the control module is further configured to receive the inputs from the at least one user interface, and wherein the alignment inputs received by the control module include a command to the control module to iteratively move the test image into alignment with the fiducial as determined with the image sensor.
- In addition to one or more of the features described herein, the apparatus further includes an image projector projecting the fiducial upon the windshield.
- In addition to one or more of the features described herein, the apparatus further includes an image projector projecting the fiducial upon the windshield.
- In addition to one or more of the features described herein, the apparatus further includes a fixture attached to a vehicle component wherein the image projector is attached to the fixture.
- In addition to one or more of the features described herein, the image projector includes an in-plant production tooling including a vision system.
- The above features and advantages, and other features and advantages of the disclosure are readily apparent from the following detailed description when taken in connection with the accompanying drawings.
- Other features, advantages and details appear, by way of example only, in the following detailed description, the detailed description referring to the drawings in which:
-
FIG. 1 illustrates a HUD system in accordance with the present disclosure; -
FIG. 2 illustrates various positional relationships among various feature positions in accordance with the present disclosure; -
FIG. 3 illustrates a forward-looking view of a portion of the interior of a passenger vehicle substantially along a longitudinal centerline in accordance with the present disclosure; -
FIG. 4 illustrates an exemplary apparatus for calibrating the position of a virtual image to a vehicle reference frame in accordance with the present disclosure; and -
FIG. 5 illustrates an exemplary method for calibrating the position of a virtual image in accordance with the present disclosure. - The following description is merely exemplary in nature and is not intended to limit the present disclosure, its application or uses. Throughout the drawings, corresponding reference numerals indicate like or corresponding parts and features. As used herein, control module, module, control, controller, control unit, processor and similar terms mean any one or various combinations of one or more of Application Specific Integrated Circuit(s) (ASIC), electronic circuit(s), central processing unit(s) (preferably microprocessor(s)) and associated memory and storage (read only memory (ROM), random access memory (RAM), electrically programmable read only memory (EPROM), hard drive, etc.) or microcontrollers executing one or more software or firmware programs or routines, combinational logic circuit(s), input/output circuitry and devices (I/O) and appropriate signal conditioning and buffer circuitry, high speed clock, analog to digital (A/D) and digital to analog (D/A) circuitry and other components to provide the described functionality. A control module may include a variety of communication interfaces including point-to-point or discrete lines and wired or wireless interfaces to networks including wide and local area networks, on vehicle controller area networks and in-plant and service related networks. Functions of a control module as set forth in this disclosure may be performed in a distributed control architecture among several networked control modules. Software, firmware, programs, instructions, routines, code, algorithms and similar terms mean any controller executable instruction sets including calibrations, data structures, and look-up tables. A control module has a set of control routines executed to provide described functions. Routines are executed, such as by a central processing unit, and are operable to monitor inputs from sensing devices and other networked control modules and execute control and diagnostic routines to control operation of actuators. Routines may be executed at regular intervals during ongoing engine and vehicle operation. Alternatively, routines may be executed in response to occurrence of an event.
-
FIG. 1 shows a head-up display (HUD) 101 in accordance with the present disclosure. The HUD 101 includes acontrol module 102, avirtual image generator 105 including a virtualimage generating module 104 including afirst light source 106 and amirror actuator 108, and atest image generator 103 including asecond light source 110 and a test imageoptic assembly 112. Thecontrol module 102 controls the operation of the virtualimage generating module 104 andfirst light source 106 viacontrol line 145 to generate a virtual image. Thecontrol module 102 also controls the operation of thesecond light source 110 viacontrol line 147 to generate a test image onwindshield 116. Thecontrol module 102 controls the operation of themirror actuator 108 viacontrol line 143 to rotate or tilt amirror 117 and adjust where on windshield the HUD virtual image is projected. The mirror actuator may include a motor, gears, shafts, and/or other components to change the position and/or orientation of themirror 117. Themirror 117 may magnify the image generated by the virtualimage generating module 104 and/or correct for distortions associated with thewindshield 116. - In one embodiment, the
HUD 101 includes the virtualimage generating module 104, themirror actuator 108, themirror 117 and the one ormore mirrors 122. In another embodiment, theHUD 101 also includes thesecond light source 110 and the test imageoptic assembly 112. In one embodiment, the virtualimage generating module 104, themirror actuator 108, themirror 117, and the one ormore mirrors 122 are implemented in a separate housing than thesecond light source 110 and the test imageoptic assembly 112. In another, preferred, embodiment, the virtualimage generating module 104, themirror actuator 108, themirror 117, the one ormore mirrors 122, thesecond light source 110 and the test imageoptic assembly 112 are implemented in a single housing. - The virtual
image generating module 104 may include thefirst light source 106 and a display andlens assembly 120. Thefirst light source 106 generates a virtualimage light beam 121 including graphic images that are projected onto a display of the display andlens assembly 120. The virtualimage light beam 121 is then directed at a series of one ormore mirrors 122. The one ormore mirrors 122 may include, for example, a fold mirror. The one ormore mirrors 122 may be used for packaging reasons. The virtualimage light beam 121 is reflected at themirror 117 and then reflected through light andglare traps 124 to thewindshield 116. The virtualimage light beam 121 is displayed on the windshield. The light andglare traps 124 may filter and thus prevent, for example, sun light (or ambient light) from being reflected from thewindshield 116 towards themirror 117 and minimize effects of glare. - The second
light source 110 may include a laser, one or more light emitting diodes (LEDs), or other test image generating light source. The secondlight source 110 generates a testimage light beam 126, which is directed through the test imageoptic assembly 112, offadjustable mirror 160, and reflected off of thewindshield 116. Thecontrol module 102 controls operation of theadjustable mirror 160 viacontrol line 149.Adjustable mirror 160 may include an x,y galvanic mirror allowing for positional adjustments of the test image. - The
HUD 101 may further include and/or be connected to amanual controller 136 including switches (buttons, paddles, sliders, rotaries, joysticks or the like) 138. TheHUD 101 may also include and/or be connected to a display, seat motors, or seat switches (not separately illustrated). The display may be, for example, a touchscreen, an infotainment display located in a center console of a vehicle, or other display. The seat motors are used to position one or more seats. Thecontrol module 102 may control operations of the seat motors based on user inputs via the seat switches and/or seat settings stored in memory. Themanual controller 136 may be used by a user to manually adjust height of virtual images provided by the virtualimage generating module 104 via theswitches 138. Alternatively, a display touchscreen may provide a user interface (UI) for manual adjustments of the HUD virtual image during end user application such as by a vehicle occupant. Such a display, seat switches and switches 138 may be referred to as input devices and/or interfaces or more generally as a user interface. In limited circumstances in accordance with the present disclosure, the input devices may provide a user interface for manual adjustments of the test image or to establish operator intent or control of an automated or partially automated alignment procedure. - The virtual
image light beam 121 may follow a main ray path of a HUD virtual image from a center of a HUD display, which is part of the display andlens assembly 120 to thewindshield 116. The main ray path is referred to as a gut ray. The testimage light beam 126 for the test image may follow a different path than the virtualimage light beam 121. The path of the testimage light beam 126 may or may not be parallel to a portion of or the entire path of the virtualimage light beam 121. The light beams 121, 126 do not need to follow the same optical path. - Certain HUD applications require precise alignment of the virtual images produced by the HUD. Placement of simple information presentation on the windshield, such as conventional engine gauge display, is not positionally critical. However, augmented reality systems intended to improve driver or occupant situational awareness by identifying, overlaying, or otherwise enhancing visibility of objects or features located on a road scene require virtual image placement taking into consideration the observer's eye position, the scene object position and the vehicle windshield position. In order to enable robust virtual image placement fidelity in such systems, the virtual image position must be calibrated relative to the vehicle reference frame.
- In accordance with one embodiment, a temporary fiducial is displayed upon the windshield of the vehicle at a predetermined position. The fiducial may be physically affixed to the windshield or may be virtually displayed thereon such as by light projection. As used herein with respect to fiducial placement or location, the term displayed includes physically affixed to or virtually displayed on such as by light projection.
FIG. 3 illustrates a forward-looking view of a portion of the interior of apassenger vehicle 301 substantially along a longitudinal centerline.Dashboard assembly 309 extends laterally within the vehicle cabin substantially forward of the doors and beneath and between the A-pillars 307. A-pillars 307 frame thewindshield 116.Steering wheel 303 is located forward of the driver seating 450 (FIG. 4 ) and is coupled to a steering column assembly 405 (FIG. 4 ).Fiducial 311 is illustrated affixed or projected uponwindshield 116 substantially centered with respect to thesteering wheel 303 though not limited to such placement.Fiducial 311 may be a simple geometric graphic such as a grid, concentric circles or, as illustrated, a simple cross-hair pattern formed by a pair of normally intersecting lines. In the case of a physically affixed fiducial, such may be affixed by any practical means and take such exemplary forms as mild adhesive backed or static cling material which may be readily removed. In order to index placement of such affixed fiducials and ensure accurate placement at the predetermined position, any of a variety of simple fixturing or gauging may be employed. For example, the A-pillars may provide fixture mounting locations for precise placement of an affixed fiducial relative to the A-pillars. Alternatively, a simple template aligned to features on the windshield may be employed for fiducial adhesion to the windshield. In accordance with another embodiment for fiducial display, a light projector may be employed to project a virtual fiducial upon the windshield. The location of such a virtual fiducial at the predetermined location may be ensured by fixturing a projector, for example a so-called pico-projector, to the vehicle reference frame. One or both of the A-pillars may be employed for fixturing as can thesteering wheel 303 as will be later described. Alternatively, the projector or other light source for producing and projecting the fiducial may be part of the in-plant production tooling including, for example, a vision system for locating the projected fiducial on the windshield relative to the vehicle reference frame. Regarding fiducial image projection, projector, image projector, or fiducial projector are understood to mean any suitable light image projector (LED, LASER, lamp based, etc.). - Assembly of
HUD 101 into the vehicle may be accomplished by the installation of an entire dash assembly into which theHUD 101 has been assembled as part of a subassembly process or build-up of thedash assembly 309. Alternatively, a smaller subassembly including an instrument cluster pod may contain theHUD 101 and may be assembled to the dash assembly already installed within the vehicle. Alternatively, theHUD 101 may be assembled into the instrument cluster, dash assembly or upper dash pad as a separate assembly component. In accordance with one embodiment, atest image 305 is projected upon thewindshield 116 subsequent to theHUD 101 installation within the vehicle.Test image 305 is produced bytest image generator 103 ofHUD 101 and may be the same simple geometric graphic as fiducial 311 or a complementary graphic such as an outline of the fiducial graphic. In the illustrated embodiment,test image 305 is also a simple cross-hair pattern formed by a pair of normally intersecting lines. The virtual image may have a calibrated alignment position specification corresponding to the predetermined fiducial position 211 (FIG. 2 ). However, numerous tolerance stack ups may result in the projected virtual image being displaced from the predetermined fiducial position. Similarly, thetest image 305 may have a projected position displaced from the predeterminedfiducial position 211 as illustrated by the separation of thetest image 305 from the fiducial 311 displayed upon thewindshield 116. - With reference to
FIG. 2 ,HUD 101 is illustrated includingvirtual image generator 105 andtest image generator 103.Virtual image generator 105 projects virtualimage light beam 121 towardwindshield 116.Test image generator 103 also projects testimage light beam 126 towardwindshield 116. While the surface ofwindshield 116 is not likely to be planar, theHUD 101 projections fromvirtual image generator 105 andtest image generator 103 are positionally controllable within limits in a substantially planar, two-dimensional space illustrated inFIG. 2 by horizontal x-axis and vertical y-axis. Thevirtual image generator 105 projects the virtual image to a calibratedvirtual image position 203 which is initially set during HUD manufacture and stored in a non-volatile memory location withinHUD controller 102. Thetest image generator 103 projects the test image 305 (FIG. 3 ) to a calibratedtest image position 205 which also is initially set during HUD manufacture and stored in a non-volatile memory location withinHUD controller 102. The x,y positional difference between the calibratedvirtual image position 203 and the calibratedtest image position 205 is known from these design parameters and may be referred to as a test image offset 230 shown as vector {right arrow over (A)} with tail at calibratedvirtual image position 203. Desirably, the virtual image from thevirtual image generator 105 is to be positioned at thefiducial position 211. For example, the virtual image center is desirably aligned with the center of the fiducial thereby aligning the virtual image with the vehicle reference frame. Test image 305 (FIG. 3 ) is projected upon thewindshield 116 at calibratedtest image position 205. The difference between the calibratedtest image position 205 and thefiducial position 211 may be referred to as test image alignment offset 225 shown as vector {right arrow over (B)} with tail at calibratedtest image position 205.Test image 305 position may be adjusted such that thetest image 305 overlays and aligns with the fiducial 311. From the calibratedtest image position 205 and the x,y positional deviation required to aligntest image 305 with fiducial 311, test image alignment offset 225 shown as vector {right arrow over (B)} may be determined. The virtual image is projected upon thewindshield 116 at calibratedvirtual image position 203. The difference between the calibratedvirtual image position 203 and thefiducial position 211 may be referred to as desired virtual image offset 235 shown as vector {right arrow over (C)} with tail at calibratedvirtual image position 205. The difference between the calibratedvirtual image position 203 and thefiducial position 211 may be determined therefore from the x,y positional deviation required to aligntest image 305 with fiducial 311 and the x,y positional difference between the calibratedvirtual image position 203 and the calibratedtest image position 205. - In accordance with one embodiment and with reference to
FIG. 5 , a method for calibrating the position of thevirtual image 501 includes displaying a fiducial at a predetermined location upon thewindshield 503. Display of the fiducial may be in accordance with affixing a fiducial to the windshield or projecting a fiducial image upon the windshield. A test image is projected from a test image generator within theHUD system 505 onto the windshield at a test image calibrated position known in accordance with the HUD calibration. Test image projection may be initiated by an operator through any of a variety of user interfaces as previously set forth in the disclosure. The test image is aligned with the fiducial 507. Alignment may be manually such as through any of a variety of user interfaces as previously set forth in the disclosure or as established in accordance with operator intent or control of an automated or partially automated alignment procedure which may be invoked through any of a variety of user interfaces as previously set forth in the disclosure. The positional deviation required to align the test image with the fiducial is determined from the calibrated position of the test image and the position of the test image when aligned with the fiducial 509. The virtual image has a known virtual image calibrated position in accordance with the HUD calibration. Since the virtual image calibrated position is known, the test image calibrated position is known, and the positional deviation required to align the test image with the fiducial is also known, the deviation in the location of the virtual image required to align the virtual image with the fiducial is readily determined. This deviation may be used to align the virtual image to thefiducial position 511. Alignment of the virtual image may be implemented in various ways depending upon the particular hardware configuration of the HUD and more particularly of thevirtual image generator 105. By way of non-limiting examples, alignment of the virtual image may be effected by rotation of an interior mirror in the case of a DLP HUD system, application of a lens function to the phase hologram in the case of a holographic HUD system, or image translation on an LCD display having reserve pixels or x,y translation of the entire LCD display in a LCD HUD system. - In accordance with one embodiment and with reference to
FIG. 4 , an apparatus for calibrating the position of the virtual image to a vehicle reference frame in accordance with the present disclosure is illustrated. The embodiment illustrated is well suited to the controlled production environment of a final assembly plant though such apparatus may be employed in a field service environment. A vehicle interior includes adash assembly 309 whereinHUD 101 is contained.Steering column assembly 405 is attached to structural brackets beneath thedash assembly 309.Steering wheel 303 is attached to an end ofsteering column assembly 405.Steering column assembly 405 may include a tilt/telescoping mechanism which allows for selectively adjusting the steering column/wheel between extreme raised and lowered positions and between extreme fore/aft positions.HUD 101 is configured to project a test image via testimage light beam 126 towardwindshield 116 and a virtual image via virtualimage light beam 126 also towardwindshield 116. A calibration set-up 408 includes acalibration fixture 411 andoptical calibration assembly 407 secured thereto. Optical calibration assembly may include one or both of afiducial projector 417 and animage sensor 419. Afiducial projector 417, for example a pico-projector, is useful in an arrangement wherein a projected fiducial is desired. An image sensor, for example a CCD or CMOS camera, is useful in an arrangement wherein the alignment of the test image to the fiducial is automated. The calibration set-up 408 may also include a display or other user/control interface 415 which may be a part of or separate from the optical calibration assembly, for example an integrated display or a separate smart device for example a tablet, laptop computer or the like. The calibration set-up may also include one or more control modules adapted to provide a calibration application including UI and instructional prompts, and communication and networking functions. Such a control module is preferably associated with one or more of the calibration assembly or separate smart device. Communications among theHUD 101,optical calibration assembly 407, user/control interface 415, and various in-plant production control and information systems may be carried out by any appropriate wired or wireless means including for example the vehicle controller area network. In practice, thesteering column assembly 405 is placed into extreme tilt and telescoping positions to reduce the variability of the steering wheel position relative to the vehicle reference frame. Subsequently, thecalibration fixture 411 andoptical calibration assembly 407 is secured to the steering wheel such that thefiducial projector 417 will display the fiducial image upon the windshield at the desired position. Alternatively, as previously mentioned,calibration fixture 411 andoptical calibration assembly 407 secured thereto may be secured to other structures within the vehicle providing a repeatable positional relationship to the vehicle reference frame, for example one or more of A-pillars, B-pillars, door opening frame, etc. - Unless explicitly described as being “direct,” when a relationship between first and second elements is described in the above disclosure, that relationship can be a direct relationship where no other intervening elements are present between the first and second elements, but can also be an indirect relationship where one or more intervening elements are present (either spatially or functionally) between the first and second elements.
- It should be understood that one or more steps within a method may be executed in different order (or concurrently) without altering the principles of the present disclosure. Further, although each of the embodiments is described above as having certain features, any one or more of those features described with respect to any embodiment of the disclosure can be implemented in and/or combined with features of any of the other embodiments, even if that combination is not explicitly described. In other words, the described embodiments are not mutually exclusive, and permutations of one or more embodiments with one another remain within the scope of this disclosure.
- While the above disclosure has been described with reference to exemplary embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted for elements thereof without departing from its scope. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the disclosure without departing from the essential scope thereof. Therefore, it is intended that the present disclosure not be limited to the particular embodiments disclosed, but will include all embodiments falling within the scope thereof.
Claims (17)
1. A method for calibrating a virtual image projected from a vehicle head-up display (HUD) system onto a windshield, comprising:
displaying a fiducial at a predetermined fiducial position upon the windshield;
projecting a test image from a test image generator to a calibrated test image position stored within the HUD system;
aligning the projected test image with the fiducial;
determining a test image alignment offset comprising positional deviation of the projected test image from a the calibrated test image position required to align the projected test image with the fiducial;
projecting a virtual image from a virtual image generator independent from the test image generator; and
aligning the projected virtual image with the fiducial based upon the test image alignment offset and a known positional relationship between a calibrated virtual image position stored within the HUD system and the calibrated test image position stored within the HUD system.
2. The method of claim 1 , wherein displaying the fiducial comprises projecting the fiducial from an image projector that is separate from the HUD system and affixed to a predetermined vehicle component.
3. The method of claim 1 , wherein displaying the fiducial comprises affixing the fiducial to the windshield.
4. The method of claim 1 , wherein aligning the test image with the fiducial comprises manually adjusting the test image.
5. The method of claim 1 , wherein aligning the test image with the fiducial comprises autonomously adjusting the test image.
6. A method for calibrating a virtual image projected from a vehicle head-up display (HUD) system onto a windshield, comprising:
displaying a fiducial at a predetermined fiducial position upon the windshield;
projecting a test image from a test image generator to a calibrated test image position stored within the HUD system;
aligning the projected test image with the fiducial by moving the projected test image from the calibrated test image position to a test image alignment position;
determining a test image alignment offset as the difference between the calibrated test image position and the test image alignment position;
determining a test image offset as the difference between the calibrated test image position and a calibrated virtual image position stored within the HUD system;
determining a desired virtual image offset from the calibrated virtual image position based upon the test image alignment offset and the test image offset; and
projecting a virtual image from a virtual image generator independent from the test image generator based upon the calibrated virtual image position and the desired virtual image offset.
7. The method of claim 6 , wherein displaying the fiducial comprises projecting the fiducial from an image projector that is separate from the HUD system and affixed to a predetermined vehicle component.
8. The method of claim 6 , wherein displaying the fiducial comprises affixing the fiducial to the windshield.
9. The method of claim 6 , wherein aligning the test image with the fiducial comprises manually adjusting the test image.
10. The method of claim 6 , wherein aligning the test image with the fiducial comprises autonomously adjusting the test image.
11. An apparatus for calibrating a virtual image projected from a vehicle head-up display (HUD) system onto a windshield, comprising:
a HUD including a virtual image generator for projecting a virtual image and a test image generator independent from the virtual image generator for projecting a test image;
a fiducial displayed upon the windshield at a predetermined fiducial position;
a control module configured to:
project the test image upon the windshield from the test image generator at a calibrated test image position stored within a first non-volatile memory location of the control module;
align the projected test image with the fiducial based upon alignment inputs received by the control module;
determine a test image alignment offset comprising positional deviation of the projected test image from the calibrated test image position required to align the projected test image with the fiducial;
based upon the test image alignment offset and a known positional relationship between a calibrated virtual image position stored within a second non-volatile memory location of the control module and the calibrated test image position stored within the first non-volatile memory location of the control module, determine a positional deviation of the virtual image from the calibrated virtual image position required to align the virtual image with the fiducial; and
set the calibrated virtual image position to align with the fiducial based upon the positional deviation of the virtual image from the calibrated virtual image position required to align the virtual image with the fiducial.
12. The apparatus of claim 11 , further comprising:
at least one user interface for receiving inputs from an operator, wherein said control module is further configured to receive said inputs from said at least one user interface, and wherein said alignment inputs received by the control module comprise manual inputs from said operator guiding the movement of the test image into alignment with the fiducial.
13. The apparatus of claim 11 , further comprising:
at least one user interface for receiving inputs from an operator and an image sensor, wherein said control module is further configured to receive said inputs from said at least one user interface, and wherein said alignment inputs received by the control module comprise a command to the control module to iteratively move the test image into alignment with the fiducial as determined with said image sensor.
14. The apparatus of claim 11 , further comprising:
an image projector that is separate from the HUD system projecting said fiducial upon the windshield.
15. The apparatus of claim 13 , further comprising:
an image projector that is separate from the HUD system projecting said fiducial upon the windshield.
16. The apparatus of claim 14 , further comprising:
a fixture attached to a vehicle component wherein the image projector is attached to the fixture.
17. The apparatus of claim 14 , wherein the image projector comprises an in-plant production tooling including a vision system.
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/599,777 US10996480B1 (en) | 2019-10-11 | 2019-10-11 | Head-up display calibration |
DE102020124756.7A DE102020124756B4 (en) | 2019-10-11 | 2020-09-23 | Method and device for calibrating a virtual image |
CN202011082687.XA CN112649959B (en) | 2019-10-11 | 2020-10-12 | Head-up display calibration |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/599,777 US10996480B1 (en) | 2019-10-11 | 2019-10-11 | Head-up display calibration |
Publications (2)
Publication Number | Publication Date |
---|---|
US20210109355A1 true US20210109355A1 (en) | 2021-04-15 |
US10996480B1 US10996480B1 (en) | 2021-05-04 |
Family
ID=75155611
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/599,777 Active US10996480B1 (en) | 2019-10-11 | 2019-10-11 | Head-up display calibration |
Country Status (3)
Country | Link |
---|---|
US (1) | US10996480B1 (en) |
CN (1) | CN112649959B (en) |
DE (1) | DE102020124756B4 (en) |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11454813B2 (en) | 2019-11-07 | 2022-09-27 | GM Global Technology Operations LLC | Holographic display systems with polarization correction and distortion reduction providing enhanced image quality |
US11880036B2 (en) | 2021-07-19 | 2024-01-23 | GM Global Technology Operations LLC | Control of ambient light reflected from pupil replicator |
CN113923427B (en) * | 2021-11-23 | 2023-08-18 | 中国航空工业集团公司洛阳电光设备研究所 | Rapid calibration terminal and calibration method for assembly precision of head-up display combiner |
US11953697B1 (en) * | 2023-05-05 | 2024-04-09 | Ford Global Technologies, Llc | Position tracking sensor in a head up display |
CN117288095B (en) * | 2023-11-27 | 2024-02-13 | 浙江极氪汽车研究开发有限公司 | Detection device and method for threshold floor lamp |
Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080192312A1 (en) * | 2007-02-09 | 2008-08-14 | Gm Global Technology Operations, Inc. | Holographic information display |
US20150317838A1 (en) * | 2014-05-02 | 2015-11-05 | Thales Visionix, Inc. | Registration for vehicular augmented reality using auto-harmonization |
US20150331487A1 (en) * | 2012-12-21 | 2015-11-19 | Harman Becker Automotive Systems Gmbh | Infotainment system |
US20160349510A1 (en) * | 2015-05-28 | 2016-12-01 | Quentin Simon Charles Miller | Alignment based view matrix tuning |
US20170195647A1 (en) * | 2016-01-04 | 2017-07-06 | Microvision, Inc. | MEMS Scan Controlled Keystone and Distortion Correction |
US20170343805A1 (en) * | 2016-05-27 | 2017-11-30 | Sensedriver Technologies Llc | Method and apparatus for in-vehicular communications |
US20180143682A1 (en) * | 2016-11-22 | 2018-05-24 | Honeywell International Inc. | Nte display systems and methods with optical trackers |
US20200082570A1 (en) * | 2018-09-06 | 2020-03-12 | Volkswagen Aktiengesellschaft | Method, device and computer-readable storage medium with instructions for setting a head-up display in a transportation vehicle |
US20200108719A1 (en) * | 2017-01-31 | 2020-04-09 | Mitsubishi Electric Corporation | Display control device |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE102017001015A1 (en) | 2017-02-04 | 2017-11-23 | Daimler Ag | Low-latency correction of a projection of a head-up display |
DE102020004550A1 (en) | 2020-07-27 | 2020-10-01 | Daimler Ag | Method for calibrating an augmented reality head-up display of a vehicle, and calibration system set up to carry out such a method |
-
2019
- 2019-10-11 US US16/599,777 patent/US10996480B1/en active Active
-
2020
- 2020-09-23 DE DE102020124756.7A patent/DE102020124756B4/en active Active
- 2020-10-12 CN CN202011082687.XA patent/CN112649959B/en active Active
Patent Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080192312A1 (en) * | 2007-02-09 | 2008-08-14 | Gm Global Technology Operations, Inc. | Holographic information display |
US20150331487A1 (en) * | 2012-12-21 | 2015-11-19 | Harman Becker Automotive Systems Gmbh | Infotainment system |
US20150317838A1 (en) * | 2014-05-02 | 2015-11-05 | Thales Visionix, Inc. | Registration for vehicular augmented reality using auto-harmonization |
US20160349510A1 (en) * | 2015-05-28 | 2016-12-01 | Quentin Simon Charles Miller | Alignment based view matrix tuning |
US20170195647A1 (en) * | 2016-01-04 | 2017-07-06 | Microvision, Inc. | MEMS Scan Controlled Keystone and Distortion Correction |
US20170343805A1 (en) * | 2016-05-27 | 2017-11-30 | Sensedriver Technologies Llc | Method and apparatus for in-vehicular communications |
US20180143682A1 (en) * | 2016-11-22 | 2018-05-24 | Honeywell International Inc. | Nte display systems and methods with optical trackers |
US20200108719A1 (en) * | 2017-01-31 | 2020-04-09 | Mitsubishi Electric Corporation | Display control device |
US20200082570A1 (en) * | 2018-09-06 | 2020-03-12 | Volkswagen Aktiengesellschaft | Method, device and computer-readable storage medium with instructions for setting a head-up display in a transportation vehicle |
Also Published As
Publication number | Publication date |
---|---|
US10996480B1 (en) | 2021-05-04 |
DE102020124756A1 (en) | 2021-04-15 |
CN112649959B (en) | 2023-04-14 |
CN112649959A (en) | 2021-04-13 |
DE102020124756B4 (en) | 2023-09-21 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10996480B1 (en) | Head-up display calibration | |
US10996481B1 (en) | Head-up display calibration | |
CN110573369B (en) | Head-up display device and display control method thereof | |
US10302940B2 (en) | Head-up display | |
US10775621B2 (en) | Method, device and computer-readable storage medium with instructions for setting a head-up display in a transportation vehicle | |
US20180253907A1 (en) | Augmented reality alignment system and method | |
US20210157135A1 (en) | Head-up display image acquistion and correction | |
US10732412B2 (en) | Display device for vehicle | |
CN111433067A (en) | Head-up display device and display control method thereof | |
WO2017163292A1 (en) | Headup display device and vehicle | |
CN108225734B (en) | Error calibration system based on HUD system and error calibration method thereof | |
JP6512016B2 (en) | Vehicle display device | |
US11999231B2 (en) | Head-up display device | |
CN111954902A (en) | Video display system, video display method, program, and moving object provided with video display system | |
US9835861B2 (en) | System having an adjustment device and method for automatically adjusting/switching the adjustment device for a head-up display device | |
WO2016136573A1 (en) | Vehicle display device | |
WO2018088362A1 (en) | Head-up display | |
US11808946B2 (en) | Diaphragm for performing image position correction of a virtual image projectable on a windshield of a vehicle by means of a head-up display | |
US11482141B1 (en) | Reticle and target based calibration for augmented reality head-up displays | |
JP2019026198A (en) | Head-up display device, and driver viewpoint detection method therefor | |
WO2018030320A1 (en) | Vehicle display device | |
JP2019015936A (en) | Head-up display device | |
JP2024048865A (en) | Display control device and display device | |
WO2018037887A1 (en) | Vehicular display device | |
CN117294823A (en) | Display method, display device, storage medium and vehicle |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
AS | Assignment |
Owner name: GM GLOBAL TECHNOLOGY OPERATIONS LLC, MICHIGAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:CHANG, KAI-HAN;SEDER, THOMAS A.;REEL/FRAME:050696/0643 Effective date: 20191009 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |