WO2005055189A1 - Perceptual enhancement displays based on knowledge of head and/or eye and/or gaze position - Google Patents

Perceptual enhancement displays based on knowledge of head and/or eye and/or gaze position Download PDF

Info

Publication number
WO2005055189A1
WO2005055189A1 PCT/EP2003/013479 EP0313479W WO2005055189A1 WO 2005055189 A1 WO2005055189 A1 WO 2005055189A1 EP 0313479 W EP0313479 W EP 0313479W WO 2005055189 A1 WO2005055189 A1 WO 2005055189A1
Authority
WO
WIPO (PCT)
Prior art keywords
head
lane
knowledge
eye
display
Prior art date
Application number
PCT/EP2003/013479
Other languages
French (fr)
Inventor
Trent Victor
Original Assignee
Volvo Technology Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Volvo Technology Corporation filed Critical Volvo Technology Corporation
Priority to PCT/EP2003/013479 priority Critical patent/WO2005055189A1/en
Priority to AU2003300514A priority patent/AU2003300514A1/en
Priority to JP2006541881A priority patent/JP5162132B2/en
Priority to EP04803401A priority patent/EP1689607A1/en
Priority to BRPI0417094-6A priority patent/BRPI0417094A/en
Priority to CNB2004800356082A priority patent/CN100541141C/en
Priority to EP04803400.3A priority patent/EP1733190B1/en
Priority to BRPI0417095-4A priority patent/BRPI0417095A/en
Priority to PCT/EP2004/013632 priority patent/WO2005053991A1/en
Priority to PCT/EP2004/013631 priority patent/WO2005054786A1/en
Priority to JP2006541882A priority patent/JP4994040B2/en
Priority to CNB2004800356631A priority patent/CN100423964C/en
Publication of WO2005055189A1 publication Critical patent/WO2005055189A1/en
Priority to US11/421,494 priority patent/US7656313B2/en
Priority to US11/421,493 priority patent/US8497880B2/en
Priority to JP2011104454A priority patent/JP2011242769A/en

Links

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B62—LAND VEHICLES FOR TRAVELLING OTHERWISE THAN ON RAILS
    • B62D—MOTOR VEHICLES; TRAILERS
    • B62D15/00—Steering not otherwise provided for
    • B62D15/02—Steering position indicators ; Steering position determination; Steering aids
    • B62D15/029—Steering assistants using warnings or proposing actions to the driver without influencing the steering system
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
    • B60K35/10—Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
    • B60K35/20—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor
    • B60K35/29—Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
    • B60K35/60—Instruments characterised by their location or relative disposition in or on vehicles
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
    • B60Q9/00—Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00, e.g. haptic signalling
    • B60Q9/008—Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00, e.g. haptic signalling for anti-collision purposes
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
    • G01C21/34—Route searching; Route guidance
    • G01C21/36—Input/output arrangements for on-board computers
    • G01C21/3626—Details of the output of route guidance instructions
    • G01C21/365—Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
    • G01C21/34—Route searching; Route guidance
    • G01C21/36—Input/output arrangements for on-board computers
    • G01C21/3626—Details of the output of route guidance instructions
    • G01C21/3652—Guidance using non-audiovisual output, e.g. tactile, haptic or electric stimuli
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
    • G01C21/34—Route searching; Route guidance
    • G01C21/36—Input/output arrangements for on-board computers
    • G01C21/3626—Details of the output of route guidance instructions
    • G01C21/3658—Lane guidance
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
    • G01C21/34—Route searching; Route guidance
    • G01C21/36—Input/output arrangements for on-board computers
    • G01C21/3697—Output of additional, non-guidance related information, e.g. low fuel level
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/149—Instrument input by detecting viewing direction not otherwise provided for
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/18—Information management
    • B60K2360/184—Displaying the same information on different displays
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/20—Optical features of instruments
    • B60K2360/33—Illumination features
    • B60K2360/334—Projection means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/77—Instrument locations other than the dashboard
    • B60K2360/785—Instrument locations other than the dashboard on or in relation to the windshield or windows
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60T—VEHICLE BRAKE CONTROL SYSTEMS OR PARTS THEREOF; BRAKE CONTROL SYSTEMS OR PARTS THEREOF, IN GENERAL; ARRANGEMENT OF BRAKING ELEMENTS ON VEHICLES IN GENERAL; PORTABLE DEVICES FOR PREVENTING UNWANTED MOVEMENT OF VEHICLES; VEHICLE MODIFICATIONS TO FACILITATE COOLING OF BRAKES
    • B60T2201/00—Particular use of vehicle brake systems; Special systems using also the brakes; Special software modules within the brake system controller
    • B60T2201/08—Lane monitoring; Lane Keeping Systems
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60T—VEHICLE BRAKE CONTROL SYSTEMS OR PARTS THEREOF; BRAKE CONTROL SYSTEMS OR PARTS THEREOF, IN GENERAL; ARRANGEMENT OF BRAKING ELEMENTS ON VEHICLES IN GENERAL; PORTABLE DEVICES FOR PREVENTING UNWANTED MOVEMENT OF VEHICLES; VEHICLE MODIFICATIONS TO FACILITATE COOLING OF BRAKES
    • B60T2201/00—Particular use of vehicle brake systems; Special systems using also the brakes; Special software modules within the brake system controller
    • B60T2201/08—Lane monitoring; Lane Keeping Systems
    • B60T2201/082—Lane monitoring; Lane Keeping Systems using alarm actuation
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60T—VEHICLE BRAKE CONTROL SYSTEMS OR PARTS THEREOF; BRAKE CONTROL SYSTEMS OR PARTS THEREOF, IN GENERAL; ARRANGEMENT OF BRAKING ELEMENTS ON VEHICLES IN GENERAL; PORTABLE DEVICES FOR PREVENTING UNWANTED MOVEMENT OF VEHICLES; VEHICLE MODIFICATIONS TO FACILITATE COOLING OF BRAKES
    • B60T2201/00—Particular use of vehicle brake systems; Special systems using also the brakes; Special software modules within the brake system controller
    • B60T2201/08—Lane monitoring; Lane Keeping Systems
    • B60T2201/087—Lane monitoring; Lane Keeping Systems using active steering actuation
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0179—Display position adjusting means not related to the information to be displayed
    • G02B2027/0187—Display position adjusting means not related to the information to be displayed slaved to motion of at least a part of the body of the user, e.g. head, eye

Definitions

  • Perceptual Enhancement displays based on knowledge of head and/or eye and/or gaze position
  • This invention relates to methods, systems and displays for perceptual enhancement based on knowledge of head and/or eye position and/or gaze position especially of a vehicle driver.
  • the invention is not only applicable to road-vehicle driving, but could also be applied to military vehicles, trains, planes, ships, etc..
  • a common element in the proposed perceptual enhancement solutions presented below is the requirement to determine where the driver's head and/or eyes are positioned in space, where gaze is directed, and to be able to adapt displays for this purpose.
  • Glances away from the road into the vehicle interior are potentially unsafe because the driver's ability to detect changes in the on-road environment is reduced. Off-road glances lead to undesirable safety consequences such as increased variability of lane-keeping performance, lane exceedencies, increased brake reaction times, mis- sed events and more.
  • the solution has the advantage that it is able to inceases safety of driving considerably.
  • BESTATIGUNGSKOPIE The invention proposed here enables the driver to read information with peripheral vision when gaze is not directed at- or close to an in- vehicle display or head-up-dis- play.
  • This invention requires instantaneous gaze position to be sensed by the system as e.g. disclosed in the PCT/SE03/01600 filed on October 15, 2003, which by reference is made to a part of this disclosure. If the gaze position is continually known, then the following information presentation strategy can be employed.
  • state 1 Foveal- vision-state
  • state 2 Peripheral-vision-state (see Figure 3).
  • Foveal- vision-state In Foveal- vision-state the information in the display is presented in the standard format found in all displays today.
  • Peripheral-vision-state In Peripheral-vision-state the information in the dis- play is presented in such a way as to be readable with peripheral vision.
  • Figure 2 shows a block diagramm of the basic method steps which have to be conducted for realization of this gaze-contingent peripheral-vision information presentation.
  • text and line size have to be magnified to compensate for visual acuity loss as information is presented in increasing angles from the fovea.
  • the magnification factor could be directly tied to visual acuity curve as shown in Figure 1 so that the information presented changes its size dynamically with visual angle (see Figure 4), or the magnification factor could be set at a single (large) size when the system is in Peripheral- vision-state.
  • the information presented has to be very simple and large. To counteract double vision effects, text could be doubled and slightly offset. Examples of information-types that could be presented are current speed, navigation information, or current gear. Ultimately the choice of what information to display should be made by the driver.
  • Figure 3(a) to (d) show examples of gaze-contingent peripheral-vision information presentation.
  • Figure 4 shows such equal magnification of letters ten times stimuli threshold size. If the viewer's gaze is held on the central dot then the letters are equally readable. Continuous heading and lane-keeping control systems:
  • the basic principle here is to provide predictive information to the driver about the vehicle's future path so that the driver can directly see the difference between where the vehicle is heading and where the driver actually wants to go, i.e. to make the error term more visible.
  • Lane-keeping is a skill that novices are especially poor at and are over-represented in crash statistics.
  • This display ideally requires the system to have information of 1) head position and or eye position, 2) a path prediction estimate, and 3) the means with which to present information.
  • Figure 5 presents an example of how this may be presented in a head-up-display which is overlaid on the driving scene.
  • Figure 5 shows an example of a path prediction quickening display
  • (a) and (b) represent on-path situations when driving straight ahead and into a left turn
  • (c) and (d) represent off-path situations.
  • the dashed lines and arrows show what is displayed to the driver, the thicker solid lines illustrate the road markings (in reality the driver would not see the black lines but rather the real road markings).
  • the thin lines represent the current optic-flow as calculated from vehicle data (e.g. yaw-rate data).
  • Figure 6 shows a block diagramm of the basic method steps which have to be conducted for realization of this path prediction quickening display.
  • Lane departures and run-off-road incidents represent a large portion of accidents. Lane-keeping difficulties often are the consequence of distraction caused by use of in-vehicle devices (e.g. a radio). This invention assists the driver in perceiving the effects of lane deviation through peripheral vision stimulus.
  • in-vehicle devices e.g. a radio
  • Lane-trackers are commercially available products. Devices which provide information on head position (e.g. the ASCII Minds system, Seeing Machines FaceLAB) are soon commercially available.
  • LEDs Light emitting diodes
  • Figure 7 shows such an array of multicolour LEDs which are placed on or in the top of the dashboard.
  • Figure 8 shows a block diagramm of the basic method steps which have to be conducted for realization of such display supporting lane-keeping.
  • Figure 9 shows examples of the aiming device for lane keeping.
  • Figure 9 (a) shows the display output when the car is in the middle of the lane, (b) shows the display output when the car is in the left part of the lane, and (c) shows the display output when the car is in the right part of the lane.
  • the white markings represent the goal state for staying in the center of the lane
  • the light grey markings represent deviations from this goal state.
  • the dark grey markings in Figure 9(a) represent the goal state being achieved and the vehicle being centered in the lane.
  • the LED array can be used in a number of ways to support lane-keeping.
  • Figure 9 gives one example of a display strategy.
  • Figure 9 shows how deviation in lane is represented by an increase in the number of LEDs being displayed.
  • the amount and placement of LEDs being displayed corresponds to the amount of lane deviation registered by the lane-tracking device.
  • the goal-state markings (represented by either white or dark grey markings in Figure 9) are calculated from knowledge of head po- sition and/or eye position and the width of the vehicle.
  • the knowledge of head and eye position allows the system to position the goal-state LED markings to match a continuation of the lane or road markings. If no head or eye position were known this lane-matching would not be possible because of variations in seating position, height, and head movements.
  • the LED array can also be used to present to the driver a continuation of optic-flow, called synthetic optic-flow, inside the vehicle (see figure 10 for an example of optic- flow in a straight moving vehicle).
  • Added synthetic optic-flow within the vehicle acts as extra spatial orientation information especially when the drivers eyes are diverted from the road. For example, when the driver looks at the gear shift, very little of the outside environment is available on the drivers retina. By having this extra, synthetic spatial orientation information, the driver can easily detect lateral changes to the optic flow.
  • One example of how this can be achieved is to use the LED array. In the LED array, a randomly spread out subset of LEDs moves towards the driver in manner that mimics the outside optic array at a speed determined by vehicle speed.
  • the LED array could also be used to provide stimulus which would induce a corrective lane-keeping action (other displays could also be used to do this). For example, by exaggerating the synthetic optic-flow to simulate more curvature than what is actually the case, the driver is given the impression that the vehicle is turning more than it actually is. A correctly designed system would induce a change to the driver steering patterns wherein the driver compensates for lane deviations without being aware of it. Thus the unattentive or distracted driver is able to achieve a better lane- keeping performance. This is implementable without having data from a lane- tracker.
  • the LED array could also be used to provide warning stimuli as exemplified in patent application US03/04504.
  • Visual distraction warnings, cognitive distraction warnings, directed collision warnings, and lane-departure warnings, providing a flow of light in the appropriate manner, are all possible with the LED array.
  • the LED array could also be used to provide symbolic peripheral-vision informati- on of the kind outlined in invention 1 in this document.
  • the LED array could also be used to show foveal, readable text messages.
  • the LED array could also be used for entertainment purposes, preferably while standing still.
  • the stereo could use the LEDs for sound illustration.

Landscapes

  • Engineering & Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • General Physics & Mathematics (AREA)
  • Mechanical Engineering (AREA)
  • Transportation (AREA)
  • Combustion & Propulsion (AREA)
  • Chemical & Material Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Optics & Photonics (AREA)
  • Multimedia (AREA)
  • Instrument Panels (AREA)
  • Traffic Control Systems (AREA)
  • Controls And Circuits For Display Device (AREA)
  • Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)
  • Navigation (AREA)
  • Feedback Control In General (AREA)
  • Transforming Electric Information Into Light Information (AREA)
  • Control Of Indicators Other Than Cathode Ray Tubes (AREA)

Abstract

Methods, systems and displays for perceptual enhancement based on knowledge of head and/or eye position and/or gaze position are disclosed which are especially provided for a vehicle driver for increasing safety of driving. This is achieved mainly by a gaze-contingent peripheral-vision information presentation, a path-prediction quickening display and a display supporting lane-keeping.

Description

Perceptual Enhancement displays based on knowledge of head and/or eye and/or gaze position
This invention relates to methods, systems and displays for perceptual enhancement based on knowledge of head and/or eye position and/or gaze position especially of a vehicle driver. However, the invention is not only applicable to road-vehicle driving, but could also be applied to military vehicles, trains, planes, ships, etc..
A common element in the proposed perceptual enhancement solutions presented below is the requirement to determine where the driver's head and/or eyes are positioned in space, where gaze is directed, and to be able to adapt displays for this purpose.
1. Gaze-contingent peripheral- vision information presentation:
Drivers in current vehicles are often required to look away from the road into the vehicle to extract the information presented there. For example, the driver frequently needs to directly fixate (look straight at) the speedometer, the radio or navigation displays and he must be able to read and understand the information presented there. The reason for this need for fixation is that visual acuity is greatest in the central 3 degrees of the retina called the fovea and that the information presented is too detailed to be extracted with peripheral vision. Figure 1 shows such relative acuity of vision.
Glances away from the road into the vehicle interior are potentially unsafe because the driver's ability to detect changes in the on-road environment is reduced. Off-road glances lead to undesirable safety consequences such as increased variability of lane-keeping performance, lane exceedencies, increased brake reaction times, mis- sed events and more.
Consequently it is one object of the invention to provide a method, system and display by which the amount of in- vehicle glances is reduced.
The solution has the advantage that it is able to inceases safety of driving considerably.
BESTATIGUNGSKOPIE The invention proposed here enables the driver to read information with peripheral vision when gaze is not directed at- or close to an in- vehicle display or head-up-dis- play. This invention requires instantaneous gaze position to be sensed by the system as e.g. disclosed in the PCT/SE03/01600 filed on October 15, 2003, which by reference is made to a part of this disclosure. If the gaze position is continually known, then the following information presentation strategy can be employed.
The idea is to switch between two modes of information presentation based on knowledge of gaze position, either state 1: Foveal- vision-state, or state 2: Peripheral-vision-state (see Figure 3).
State 1: Foveal- vision-state: In Foveal- vision-state the information in the display is presented in the standard format found in all displays today.
State 2: Peripheral-vision-state: In Peripheral-vision-state the information in the dis- play is presented in such a way as to be readable with peripheral vision.
Figure 2 shows a block diagramm of the basic method steps which have to be conducted for realization of this gaze-contingent peripheral-vision information presentation.
Basically, text and line size have to be magnified to compensate for visual acuity loss as information is presented in increasing angles from the fovea. The magnification factor could be directly tied to visual acuity curve as shown in Figure 1 so that the information presented changes its size dynamically with visual angle (see Figure 4), or the magnification factor could be set at a single (large) size when the system is in Peripheral- vision-state. The information presented has to be very simple and large. To counteract double vision effects, text could be doubled and slightly offset. Examples of information-types that could be presented are current speed, navigation information, or current gear. Ultimately the choice of what information to display should be made by the driver.
Figure 3(a) to (d) show examples of gaze-contingent peripheral-vision information presentation. Figure 4 shows such equal magnification of letters ten times stimuli threshold size. If the viewer's gaze is held on the central dot then the letters are equally readable. Continuous heading and lane-keeping control systems:
2. Path-prediction quickening display:
Quickening displays exist in military vehicles and aircraft, e.g. fighter planes, air- traffic control, and in shipping. In air-traffic control the future path of an airplane is presented on screen as a forward extending line of varying curvature.
The basic principle here is to provide predictive information to the driver about the vehicle's future path so that the driver can directly see the difference between where the vehicle is heading and where the driver actually wants to go, i.e. to make the error term more visible. Lane-keeping is a skill that novices are especially poor at and are over-represented in crash statistics.
Racing schools regularly teach the importance of «reading the line» in their racing courses. Thus, seeing where your vehicle is going and where it will be in the future relative to the upcoming road is a skill that is very important and difficult to learn.
This display ideally requires the system to have information of 1) head position and or eye position, 2) a path prediction estimate, and 3) the means with which to present information. Figure 5 presents an example of how this may be presented in a head-up-display which is overlaid on the driving scene.
In detail, Figure 5 shows an example of a path prediction quickening display, (a) and (b) represent on-path situations when driving straight ahead and into a left turn, (c) and (d) represent off-path situations. The dashed lines and arrows show what is displayed to the driver, the thicker solid lines illustrate the road markings (in reality the driver would not see the black lines but rather the real road markings). The thin lines represent the current optic-flow as calculated from vehicle data (e.g. yaw-rate data).
Figure 6 shows a block diagramm of the basic method steps which have to be conducted for realization of this path prediction quickening display.
3. Display supporting lane-keeping: Lane departures and run-off-road incidents represent a large portion of accidents. Lane-keeping difficulties often are the consequence of distraction caused by use of in-vehicle devices (e.g. a radio). This invention assists the driver in perceiving the effects of lane deviation through peripheral vision stimulus.
This invention uses a combination of a lane tracker, a head/eye position sensor, and a display mechanism. Lane-trackers are commercially available products. Devices which provide information on head position (e.g. the ASCII Minds system, Seeing Machines FaceLAB) are soon commercially available.
One possible display mechanism for this embodiment of the invention is shown in Figure 7. Light emitting diodes (LEDs) are placed on or in a top portion of the dashboard according to Figure 7 which shows such an array of multicolour LEDs which are placed on or in the top of the dashboard.
Figure 8 shows a block diagramm of the basic method steps which have to be conducted for realization of such display supporting lane-keeping.
Figure 9 shows examples of the aiming device for lane keeping. Figure 9 (a) shows the display output when the car is in the middle of the lane, (b) shows the display output when the car is in the left part of the lane, and (c) shows the display output when the car is in the right part of the lane.
In Figure 9(b) and (c) the white markings represent the goal state for staying in the center of the lane, the light grey markings represent deviations from this goal state. The dark grey markings in Figure 9(a) represent the goal state being achieved and the vehicle being centered in the lane.
The LED array can be used in a number of ways to support lane-keeping. Figure 9 gives one example of a display strategy. Figure 9 shows how deviation in lane is represented by an increase in the number of LEDs being displayed. The amount and placement of LEDs being displayed corresponds to the amount of lane deviation registered by the lane-tracking device. The goal-state markings (represented by either white or dark grey markings in Figure 9) are calculated from knowledge of head po- sition and/or eye position and the width of the vehicle. The knowledge of head and eye position allows the system to position the goal-state LED markings to match a continuation of the lane or road markings. If no head or eye position were known this lane-matching would not be possible because of variations in seating position, height, and head movements.
The LED array can also be used to present to the driver a continuation of optic-flow, called synthetic optic-flow, inside the vehicle (see figure 10 for an example of optic- flow in a straight moving vehicle). Added synthetic optic-flow within the vehicle acts as extra spatial orientation information especially when the drivers eyes are diverted from the road. For example, when the driver looks at the gear shift, very little of the outside environment is available on the drivers retina. By having this extra, synthetic spatial orientation information, the driver can easily detect lateral changes to the optic flow. One example of how this can be achieved is to use the LED array. In the LED array, a randomly spread out subset of LEDs moves towards the driver in manner that mimics the outside optic array at a speed determined by vehicle speed. For a more realistic representation of curved optic-flow (see Figure 11), the system would have to make use of path prediction calculated from vehicle data (e.g. yaw rate and/or wheel angle) and present curved synthetic optic-flow. The presentation of synthetic optic-flow could also be used together with the display strategy outlined in figure 9.
The LED array could also be used to provide stimulus which would induce a corrective lane-keeping action (other displays could also be used to do this). For example, by exaggerating the synthetic optic-flow to simulate more curvature than what is actually the case, the driver is given the impression that the vehicle is turning more than it actually is. A correctly designed system would induce a change to the driver steering patterns wherein the driver compensates for lane deviations without being aware of it. Thus the unattentive or distracted driver is able to achieve a better lane- keeping performance. This is implementable without having data from a lane- tracker.
The LED array could also be used to provide warning stimuli as exemplified in patent application US03/04504. Visual distraction warnings, cognitive distraction warnings, directed collision warnings, and lane-departure warnings, providing a flow of light in the appropriate manner, are all possible with the LED array.
The LED array could also be used to provide symbolic peripheral-vision informati- on of the kind outlined in invention 1 in this document.
The LED array could also be used to show foveal, readable text messages.
The LED array could also be used for entertainment purposes, preferably while standing still. For example the stereo could use the LEDs for sound illustration.

Claims

Claims
1. Method for perceptual enhancement based on knowledge of head and/or eye and/or gaze position.
2. System and displays for perceptual enhancement based on knowledge of head and/or eye and/or gaze position.
3. Display for perceptual enhancement based on knowledge of head and/or eye and/or gaze position.
Figure imgf000010_0001
FIG. 1
Figure imgf000010_0002
Figure imgf000011_0001
FIG. 3
Figure imgf000012_0001
Figure imgf000012_0002
FIG. 5 Vehicle data,
Head position or e.g. yaw rate or Eye position data wheel angle
Calculation of Path prediction
Calculation of Display parameters
Display presentation
FIG. 6
58
Figure imgf000014_0001
FIG. 7
Figure imgf000014_0002
Figure imgf000015_0001
(c)
FIG. 9
Figure imgf000016_0001
FIG. 10
8/8
Figure imgf000017_0001
FIG. 11
PCT/EP2003/013479 2003-11-30 2003-12-01 Perceptual enhancement displays based on knowledge of head and/or eye and/or gaze position WO2005055189A1 (en)

Priority Applications (15)

Application Number Priority Date Filing Date Title
PCT/EP2003/013479 WO2005055189A1 (en) 2003-12-01 2003-12-01 Perceptual enhancement displays based on knowledge of head and/or eye and/or gaze position
AU2003300514A AU2003300514A1 (en) 2003-12-01 2003-12-01 Perceptual enhancement displays based on knowledge of head and/or eye and/or gaze position
BRPI0417095-4A BRPI0417095A (en) 2003-12-01 2004-12-01 method and system for presenting information
PCT/EP2004/013631 WO2005054786A1 (en) 2003-12-01 2004-12-01 Method and system for presenting information
BRPI0417094-6A BRPI0417094A (en) 2003-12-01 2004-12-01 method and system for path control support
CNB2004800356082A CN100541141C (en) 2003-12-01 2004-12-01 The method and system that is used for presenting information
EP04803400.3A EP1733190B1 (en) 2003-12-01 2004-12-01 Method and system for presenting information
JP2006541881A JP5162132B2 (en) 2003-12-01 2004-12-01 Method and system for displaying information
PCT/EP2004/013632 WO2005053991A1 (en) 2003-12-01 2004-12-01 Method and system for supporting path control
EP04803401A EP1689607A1 (en) 2003-12-01 2004-12-01 Method and system for supporting path control
JP2006541882A JP4994040B2 (en) 2003-12-01 2004-12-01 Method and system for supporting route control
CNB2004800356631A CN100423964C (en) 2003-12-01 2004-12-01 Method and system for supporting path control
US11/421,494 US7656313B2 (en) 2003-11-30 2006-06-01 Method and system for supporting path control
US11/421,493 US8497880B2 (en) 2003-12-01 2006-06-01 Method and system for presenting information
JP2011104454A JP2011242769A (en) 2003-12-01 2011-05-09 Method and system for presenting information

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/EP2003/013479 WO2005055189A1 (en) 2003-12-01 2003-12-01 Perceptual enhancement displays based on knowledge of head and/or eye and/or gaze position

Publications (1)

Publication Number Publication Date
WO2005055189A1 true WO2005055189A1 (en) 2005-06-16

Family

ID=34639226

Family Applications (3)

Application Number Title Priority Date Filing Date
PCT/EP2003/013479 WO2005055189A1 (en) 2003-11-30 2003-12-01 Perceptual enhancement displays based on knowledge of head and/or eye and/or gaze position
PCT/EP2004/013632 WO2005053991A1 (en) 2003-11-30 2004-12-01 Method and system for supporting path control
PCT/EP2004/013631 WO2005054786A1 (en) 2003-12-01 2004-12-01 Method and system for presenting information

Family Applications After (2)

Application Number Title Priority Date Filing Date
PCT/EP2004/013632 WO2005053991A1 (en) 2003-11-30 2004-12-01 Method and system for supporting path control
PCT/EP2004/013631 WO2005054786A1 (en) 2003-12-01 2004-12-01 Method and system for presenting information

Country Status (6)

Country Link
US (2) US7656313B2 (en)
JP (3) JP5162132B2 (en)
CN (2) CN100541141C (en)
AU (1) AU2003300514A1 (en)
BR (2) BRPI0417094A (en)
WO (3) WO2005055189A1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102006050017A1 (en) * 2006-10-24 2008-04-30 Volkswagen Ag Motor vehicle i.e. land vehicle, has operating device for operating function of vehicle or function, which is connected with vehicle, where operating device comprises controller for adapting arrangement of device to condition of driver
US7656313B2 (en) 2003-11-30 2010-02-02 Volvo Technology Corp. Method and system for supporting path control
DE102014207397A1 (en) * 2014-04-17 2015-10-22 Bayerische Motoren Werke Aktiengesellschaft Method, device, vehicle operating system, computer program for operating a vehicle and computer program product

Families Citing this family (117)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007230491A (en) * 2006-03-03 2007-09-13 Nissan Motor Co Ltd Visual sense information presentation device and visual sense information presentation method
DE102006026572A1 (en) * 2006-06-06 2007-12-13 Claas Selbstfahrende Erntemaschinen Gmbh Method and device for displaying vehicle movements
DE102007005028B4 (en) * 2007-02-01 2022-10-27 Volkswagen Ag Method and device for displaying information on a projection screen in a vehicle
JP4656070B2 (en) * 2007-02-19 2011-03-23 株式会社デンソー Visible light laser irradiation device
JP5016959B2 (en) * 2007-03-27 2012-09-05 パナソニック株式会社 Visibility determination device
DE102007016868A1 (en) * 2007-04-10 2008-10-16 Robert Bosch Gmbh Method for displaying a lane course and control device
JP4803114B2 (en) * 2007-05-29 2011-10-26 株式会社デンソー Vehicle display device
DE102007025530A1 (en) * 2007-05-31 2008-12-04 Volkswagen Ag Information exchange apparatus and method for communicating information
DE102007025352A1 (en) * 2007-05-31 2008-12-11 Siemens Ag Driver information e.g. speed, system for use in motor vehicle i.e. passenger car, has cockpit controller controlling adaptation of information representation of information-playback unit depending on determined contexts
CN101796376A (en) * 2007-09-04 2010-08-04 大陆-特韦斯贸易合伙股份公司及两合公司 Navigation system for a complex, menu-controlled, multifunctional vehicle system
US20110140873A1 (en) * 2007-09-04 2011-06-16 Continental Teves Ag & Co. Ohg Navigation system for a complex, menu-controlled, multifunctional vehicle system
US8218007B2 (en) * 2007-09-23 2012-07-10 Volkswagen Ag Camera system for a vehicle and method for controlling a camera system
US8089375B1 (en) 2008-06-09 2012-01-03 Rockwell Collins, Inc. Head-up display/synthetic vision system predicted flight path depiction
JP5196252B2 (en) * 2008-06-26 2013-05-15 株式会社アドヴィックス Vehicle control device
JP4735676B2 (en) * 2008-08-06 2011-07-27 株式会社デンソー Driving support device
JP2010164941A (en) * 2008-10-30 2010-07-29 Honda Motor Co Ltd Display device for vehicle
JP2010120501A (en) * 2008-11-19 2010-06-03 Yazaki Corp Visual guide supporting device and vehicular display system
US20100169792A1 (en) * 2008-12-29 2010-07-01 Seif Ascar Web and visual content interaction analytics
JP5369694B2 (en) * 2009-01-16 2013-12-18 トヨタ自動車株式会社 Maneuvering support device
JP5392470B2 (en) * 2009-01-30 2014-01-22 マツダ株式会社 Vehicle display device
JP5256075B2 (en) * 2009-02-23 2013-08-07 スタンレー電気株式会社 Speed sensor
US8564502B2 (en) * 2009-04-02 2013-10-22 GM Global Technology Operations LLC Distortion and perspective correction of vector projection display
US8629903B2 (en) * 2009-04-02 2014-01-14 GM Global Technology Operations LLC Enhanced vision system full-windshield HUD
US8629784B2 (en) 2009-04-02 2014-01-14 GM Global Technology Operations LLC Peripheral salient feature enhancement on full-windshield head-up display
US8384532B2 (en) 2009-04-02 2013-02-26 GM Global Technology Operations LLC Lane of travel on windshield head-up display
US8912978B2 (en) * 2009-04-02 2014-12-16 GM Global Technology Operations LLC Dynamic vehicle system information on full windshield head-up display
JP4957747B2 (en) * 2009-05-18 2012-06-20 トヨタ自動車株式会社 Vehicle environment estimation device
US8249806B1 (en) * 2009-08-12 2012-08-21 Rockwell Collins, Inc. System, module, and method for varying the intensity of a visual aid depicted on an aircraft display unit
JP5600256B2 (en) * 2010-01-21 2014-10-01 富士重工業株式会社 Information display device
US8417448B1 (en) 2010-04-14 2013-04-09 Jason Adam Denise Electronic direction technology
WO2011132388A1 (en) * 2010-04-19 2011-10-27 本田技研工業株式会社 Device for monitoring vicinity of vehicle
JP5477158B2 (en) * 2010-05-17 2014-04-23 日産自動車株式会社 Vehicle display device
JP4978721B2 (en) * 2010-08-23 2012-07-18 株式会社デンソー Driving assistance device
JP4978722B2 (en) * 2010-08-23 2012-07-18 株式会社デンソー Speed control device
US8779909B1 (en) * 2010-11-24 2014-07-15 Gus C. Richardson Vehicle based communication system
US9060378B1 (en) * 2010-11-24 2015-06-16 Gus C. Richardson Vehicle based communication system with a handheld device
TWI439785B (en) * 2010-12-15 2014-06-01 Univ Nat Taiwan Multi resolution display system
US8924150B2 (en) * 2010-12-29 2014-12-30 GM Global Technology Operations LLC Vehicle operation and control system for autonomous vehicles on full windshield display
US20120229641A1 (en) * 2011-03-10 2012-09-13 Nissan North America, Inc. Vehicle monitoring system
US9575946B2 (en) * 2011-05-23 2017-02-21 Nuance Communications, Inc. Text browsing, editing and correction methods for automotive applications
DE102011112717B4 (en) * 2011-09-07 2017-05-04 Audi Ag A method for providing a representation in a motor vehicle depending on a viewing direction of a vehicle driver and motor vehicle with a device for providing a representation in a motor vehicle
US9055509B2 (en) 2011-12-21 2015-06-09 Nuance Communications, Inc. Situation-aware message presentation for automotive messaging
US8903714B2 (en) 2011-12-21 2014-12-02 Nuance Communications, Inc. Concept search and semantic annotation for mobile messaging
EP2618108B1 (en) * 2012-01-19 2018-10-17 Volvo Car Corporation Driver assisting system
FR2994736B1 (en) * 2012-08-24 2017-02-10 Thales Sa VISUALIZATION SYSTEM, IN PARTICULAR FOR AIRCRAFT, TRANSPARENT SCREEN AND PERIPHERAL SYMBOLOGY
US9165535B2 (en) * 2012-09-27 2015-10-20 Google Inc. System and method for determining a zoom factor of content displayed on a display device
JP5783155B2 (en) * 2012-10-05 2015-09-24 株式会社デンソー Display device
DE102012218837A1 (en) * 2012-10-16 2014-06-12 Bayerische Motoren Werke Aktiengesellschaft Method for displaying a representation on head-up display or head-mounted display e.g. LCD in passenger car, involves determining representation that is to-be-superimposed from view of user in environment, and assigning representation
US9146124B2 (en) * 2012-12-18 2015-09-29 Nokia Technologies Oy Helmet-based navigation notifications
EP2778007B1 (en) * 2013-03-12 2022-09-07 INRIA - Institut National de Recherche en Informatique et en Automatique Method and system to assess abnormal driving behaviour of vehicles travelling on road
JP6115278B2 (en) * 2013-04-16 2017-04-19 株式会社デンソー Vehicle display device
US20160086305A1 (en) 2013-04-22 2016-03-24 Toyota Jidosha Kabushiki Kaisha Head-up display apparatus for vehicle
US20140362110A1 (en) * 2013-06-08 2014-12-11 Sony Computer Entertainment Inc. Systems and methods for customizing optical representation of views provided by a head mounted display based on optical prescription of a user
JP2014240228A (en) * 2013-06-11 2014-12-25 株式会社デンソー Information presentation system
JP2014240229A (en) * 2013-06-11 2014-12-25 株式会社デンソー Information presentation system
WO2015025350A1 (en) * 2013-08-19 2015-02-26 三菱電機株式会社 Vehicle-mounted display control device
US20150116197A1 (en) * 2013-10-24 2015-04-30 Johnson Controls Technology Company Systems and methods for displaying three-dimensional images on a vehicle instrument console
FR3012784B1 (en) * 2013-11-04 2016-12-30 Renault Sa DEVICE FOR DETECTING THE LATERAL POSITION OF A PIETON IN RELATION TO THE TRACK OF THE VEHICLE
KR101533581B1 (en) * 2013-12-06 2015-07-03 한국항공우주연구원 Apparatus for guiding an airplane to land
DE102014000803B4 (en) 2014-01-22 2019-05-09 Audi Ag Vehicle, display system and method for displaying traffic-related information
JP6233599B2 (en) * 2014-02-25 2017-11-22 マツダ株式会社 Display control device for vehicle
DE102014204800A1 (en) 2014-03-14 2015-09-17 Volkswagen Aktiengesellschaft Method and apparatus for providing a graphical user interface in a vehicle
AU2015238339B2 (en) * 2014-03-25 2018-02-22 Jaguar Land Rover Limited Navigation system
DE102014009608A1 (en) 2014-06-27 2015-12-31 Audi Ag Operation of AR glasses in the motor vehicle
JP6282567B2 (en) * 2014-09-29 2018-02-21 矢崎総業株式会社 Vehicle display device
JP2016071666A (en) * 2014-09-30 2016-05-09 富士重工業株式会社 Visual guidance device for vehicle
JP6518497B2 (en) * 2014-09-30 2019-05-22 株式会社Ｓｕｂａｒｕ Vehicle gaze guidance device
JP2016071665A (en) * 2014-09-30 2016-05-09 富士重工業株式会社 Visual guidance device for vehicle
WO2016056199A1 (en) * 2014-10-07 2016-04-14 株式会社デンソー Head-up display device, and display method for head-up display
JP2016074410A (en) * 2014-10-07 2016-05-12 株式会社デンソー Head-up display device and head-up display display method
KR102270577B1 (en) * 2014-10-21 2021-06-29 현대모비스 주식회사 Apparatus and method for controlling outputting external image of vehicle
DE102014015871B4 (en) 2014-10-25 2020-10-15 Audi Ag Display system for a motor vehicle, motor vehicle with a display system and method for operating a display system
US9811932B2 (en) * 2015-04-17 2017-11-07 Nxp Usa, Inc. Display controller, heads-up image display system and method thereof
JP6696119B2 (en) * 2015-05-01 2020-05-20 富士通株式会社 Conversion device, conversion method, and conversion program
CN104931043A (en) * 2015-05-28 2015-09-23 成都亿邻通科技有限公司 Route guiding method
DE102015111909B4 (en) * 2015-07-22 2019-10-02 Deutsches Zentrum für Luft- und Raumfahrt e.V. Method and device for informing a pilot of relevant flight information as a function of his eye activity
ITUB20152951A1 (en) * 2015-08-06 2017-02-06 Thomas Bleiner MULTIFUNCTIONAL VISUALIZATION SYSTEM FOR NAUTICAL USE
JP6432491B2 (en) * 2015-11-20 2018-12-05 トヨタ自動車株式会社 Driving assistance device
US10558043B2 (en) 2015-12-02 2020-02-11 Rockwell Collins, Inc. Worn display using a peripheral view
CN114612877A (en) 2016-01-05 2022-06-10 御眼视觉技术有限公司 System and method for estimating future path
US10643296B2 (en) 2016-01-12 2020-05-05 Qualcomm Incorporated Systems and methods for rendering multiple levels of detail
US10643381B2 (en) 2016-01-12 2020-05-05 Qualcomm Incorporated Systems and methods for rendering multiple levels of detail
JP6715659B2 (en) * 2016-04-11 2020-07-01 矢崎総業株式会社 Vehicle display device
CN105996975A (en) * 2016-05-31 2016-10-12 乐视控股（北京）有限公司 Method, device and terminal for testing vision
KR101816423B1 (en) * 2016-07-12 2018-01-08 현대자동차주식회사 Displaying apparatus replacing side mirror and method for controlling output brightness thereof
US20180165954A1 (en) * 2016-07-26 2018-06-14 Faraday&Future Inc. Dynamic traffic lane assignment
JP2019535089A (en) 2016-08-19 2019-12-05 マサチューセッツ インスティテュート オブ テクノロジー System and method for providing visual distribution management
EP3500466A2 (en) 2016-08-19 2019-06-26 Massachusetts Institute of Technology Systems and methods for using an attention buffer to improve resource allocation management
JP6865006B2 (en) * 2016-09-28 2021-04-28 矢崎総業株式会社 Vehicle display device
JP6607163B2 (en) * 2016-10-07 2019-11-20 株式会社デンソー Image processing device
DE102016224246A1 (en) * 2016-12-06 2018-06-07 Volkswagen Aktiengesellschaft Method and apparatus for interacting with a graphical user interface
JP6345225B1 (en) * 2016-12-21 2018-06-20 三菱電機株式会社 Vehicle steering system and lane keeping system
DE102017200895A1 (en) * 2017-01-20 2018-07-26 Zf Friedrichshafen Ag Method and device for checking a display device for a vehicle and display device for a vehicle
JP6801508B2 (en) * 2017-02-24 2020-12-16 日本精機株式会社 Head-up display device
US10628994B2 (en) * 2017-03-07 2020-04-21 Google Llc Reducing visually induced motion sickness in head mounted display systems
US11874659B2 (en) 2017-06-09 2024-01-16 Volvo Construction Equipment Ab Information system for a working machine
DE102017114450B4 (en) * 2017-06-29 2020-10-08 Grammer Aktiengesellschaft Apparatus and method for mapping areas
US10527849B2 (en) * 2017-07-18 2020-01-07 Toyota Jidosha Kabushiki Kaisha Augmented reality vehicular assistance for color blindness
DE102017212367B4 (en) 2017-07-19 2022-12-22 Volkswagen Aktiengesellschaft Device for displaying the course of a trajectory in front of a vehicle or an object with a display unit and motor vehicle
JP6637932B2 (en) * 2017-08-03 2020-01-29 株式会社Ｓｕｂａｒｕ Driving support device for vehicles
US10423844B2 (en) * 2017-09-27 2019-09-24 Toyota Jidosha Kabushiki Kaisha Personalized augmented reality vehicular assistance for color blindness condition
CN109696953B (en) * 2017-10-19 2020-10-16 华为技术有限公司 Virtual reality character display method and device and virtual reality equipment
US10717384B2 (en) 2017-10-25 2020-07-21 Pony Ai Inc. System and method for projecting trajectory path of an autonomous vehicle onto a road surface
DE102017221191B4 (en) 2017-11-27 2019-06-13 Volkswagen Aktiengesellschaft Method for displaying the course of a safety zone in front of a vehicle or an object with a display unit, device for carrying out the method and motor vehicle and computer program
DE102017221488A1 (en) 2017-11-30 2019-06-06 Volkswagen Aktiengesellschaft Method for displaying the course of a trajectory in front of a vehicle or an object with a display unit, device for carrying out the method and motor vehicle and computer program
CN108322719A (en) * 2018-02-12 2018-07-24 京东方科技集团股份有限公司 Head-up-display system and new line display methods, mobile devices
US10627621B2 (en) * 2018-05-30 2020-04-21 Nissan North America, Inc. Heads-up display system
WO2020012227A1 (en) 2018-07-10 2020-01-16 Bosch Car Multimedia Portugal, S.A. Optical flow for motion sickness reduction
WO2020022240A1 (en) * 2018-07-25 2020-01-30 日本精機株式会社 Vehicle display device, control method for vehicle display device, and control program for vehicle display device
CN112534490B (en) * 2018-08-08 2022-11-08 日本软通股份有限公司 Driving simulation device
JP1632766S (en) * 2018-09-10 2019-06-03
KR102528232B1 (en) * 2018-10-08 2023-05-03 현대자동차주식회사 Vehicle, and control method for the same
CN110780445A (en) * 2018-11-12 2020-02-11 芋头科技（杭州）有限公司 Method and system for active calibration for assembling optical imaging systems
US11487132B2 (en) * 2018-11-12 2022-11-01 Yutou Technology (Hangzhou) Co., Ltd. Active alignment for assembling optical devices
CN113460063A (en) * 2020-03-30 2021-10-01 本田技研工业株式会社 Information providing device, information providing method, and storage medium
DE102020212520B3 (en) * 2020-10-05 2021-10-21 Volkswagen Aktiengesellschaft Horizontal position definition for the display of virtual objects in a head-up display device
CN112255903A (en) * 2020-10-15 2021-01-22 启迪虚拟现实(南京)科技发展有限公司 Holographic three-dimensional space naked eye 3d stage presentation device

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5289185A (en) * 1990-09-05 1994-02-22 Aerospatiale Societe Nationale Industrielle Process for displaying flying aid symbols on a screen on board an aircraft
US20020032524A1 (en) * 2000-05-13 2002-03-14 Mannesmann Vdo Ag Routing display for navigation systems
US20020067366A1 (en) * 2000-12-01 2002-06-06 Nissan Motor Co., Ltd. Display apparatus for vehicle
EP1235052A1 (en) * 2001-02-26 2002-08-28 Renault Lane keeping assistance system for a vehicle
JP2003054334A (en) * 2001-08-13 2003-02-26 Nissan Motor Co Ltd Display for vehicle
JP2003341383A (en) * 2002-05-29 2003-12-03 Denso Corp Display unit for vehicle
US20040066376A1 (en) * 2000-07-18 2004-04-08 Max Donath Mobility assist device

Family Cites Families (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS6483424A (en) * 1987-09-25 1989-03-29 Honda Motor Co Ltd Indicator for vehicle
JPH06230132A (en) * 1993-02-03 1994-08-19 Nissan Motor Co Ltd Obstacle detector for vehicle
JP2896291B2 (en) * 1993-06-14 1999-05-31 株式会社エイ・ティ・アール通信システム研究所 Image display device
JPH07156685A (en) * 1993-12-02 1995-06-20 Honda Motor Co Ltd Information display device for vehicle
JPH07257228A (en) * 1994-03-18 1995-10-09 Nissan Motor Co Ltd Display device for vehicle
JP3214219B2 (en) 1994-04-08 2001-10-02 日産自動車株式会社 Vehicle meter device
JP3412285B2 (en) * 1994-09-12 2003-06-03 日産自動車株式会社 Route guidance device for vehicles
JP3592784B2 (en) 1995-03-02 2004-11-24 本田技研工業株式会社 Apparatus for calculating and displaying predicted trajectories of vehicles
JPH09178506A (en) * 1995-12-27 1997-07-11 Honda Motor Co Ltd Vehicle guide system
JPH09265054A (en) * 1996-03-28 1997-10-07 Honda Motor Co Ltd Head-up type display device for vehicle
US5731805A (en) * 1996-06-25 1998-03-24 Sun Microsystems, Inc. Method and apparatus for eyetrack-driven text enlargement
US5880734A (en) * 1996-10-03 1999-03-09 Intel Corporation Peripheral vision simulator for immersive 3D virtual environments
IT1289710B1 (en) * 1996-12-04 1998-10-16 Fiat Ricerche VEHICLE INFORMATION DISPLAY DEVICE
JPH10176928A (en) * 1996-12-18 1998-06-30 Hitachi Ltd Viewpoint position measuring method and device, head-up display, and mirror adjustment device
US5777715A (en) * 1997-01-21 1998-07-07 Allen Vision Systems, Inc. Low vision rehabilitation system
JP3884815B2 (en) * 1997-03-03 2007-02-21 本田技研工業株式会社 Vehicle information display device
JP3680243B2 (en) * 1999-01-20 2005-08-10 トヨタ自動車株式会社 Runway shape display device and map database recording medium
AUPQ181699A0 (en) * 1999-07-23 1999-08-19 Cmte Development Limited A system for relative vehicle navigation
EP1083076A3 (en) * 1999-09-07 2005-01-12 Mazda Motor Corporation Display apparatus for vehicle
DE19951001C2 (en) * 1999-10-22 2003-06-18 Bosch Gmbh Robert Device for displaying information in a vehicle
US6781585B2 (en) * 2000-01-11 2004-08-24 Sun Microsystems, Inc. Graphics system having a super-sampled sample buffer and having single sample per pixel support
JP2001270346A (en) * 2000-03-24 2001-10-02 Nissan Motor Co Ltd Display device for vehicle
JP4698002B2 (en) * 2000-07-11 2011-06-08 マツダ株式会社 Vehicle display device
US6711501B2 (en) 2000-12-08 2004-03-23 Satloc, Llc Vehicle navigation system and method for swathing applications
DE10103922A1 (en) * 2001-01-30 2002-08-01 Physoptics Opto Electronic Gmb Interactive data viewing and operating system
JP2002225592A (en) * 2001-02-01 2002-08-14 Nissan Motor Co Ltd Data display device for vehicle
US7081870B2 (en) * 2001-05-09 2006-07-25 Hewlett-Packard Development Company, L.P. Wearable display and method of displaying images using a wearable display
JP2003072495A (en) 2001-09-06 2003-03-12 Yazaki Corp Parking support device and parking support method
US20030067476A1 (en) * 2001-10-04 2003-04-10 Eastman Kodak Company Method and system for displaying an image
WO2003074307A1 (en) * 2002-03-07 2003-09-12 Yechezkal Evan Spero Enhanced vision for driving
JP2003291688A (en) * 2002-04-03 2003-10-15 Denso Corp Display method, driving support device and program
AU2003300514A1 (en) 2003-12-01 2005-06-24 Volvo Technology Corporation Perceptual enhancement displays based on knowledge of head and/or eye and/or gaze position

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5289185A (en) * 1990-09-05 1994-02-22 Aerospatiale Societe Nationale Industrielle Process for displaying flying aid symbols on a screen on board an aircraft
US20020032524A1 (en) * 2000-05-13 2002-03-14 Mannesmann Vdo Ag Routing display for navigation systems
US20040066376A1 (en) * 2000-07-18 2004-04-08 Max Donath Mobility assist device
US20020067366A1 (en) * 2000-12-01 2002-06-06 Nissan Motor Co., Ltd. Display apparatus for vehicle
EP1235052A1 (en) * 2001-02-26 2002-08-28 Renault Lane keeping assistance system for a vehicle
JP2003054334A (en) * 2001-08-13 2003-02-26 Nissan Motor Co Ltd Display for vehicle
JP2003341383A (en) * 2002-05-29 2003-12-03 Denso Corp Display unit for vehicle

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
PATENT ABSTRACTS OF JAPAN vol. 2003, no. 06 3 June 2003 (2003-06-03) *
PATENT ABSTRACTS OF JAPAN vol. 2003, no. 12 5 December 2003 (2003-12-05) *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7656313B2 (en) 2003-11-30 2010-02-02 Volvo Technology Corp. Method and system for supporting path control
US8497880B2 (en) 2003-12-01 2013-07-30 Volvo Technology Corporation Method and system for presenting information
DE102006050017A1 (en) * 2006-10-24 2008-04-30 Volkswagen Ag Motor vehicle i.e. land vehicle, has operating device for operating function of vehicle or function, which is connected with vehicle, where operating device comprises controller for adapting arrangement of device to condition of driver
DE102006050017B4 (en) 2006-10-24 2019-07-11 Volkswagen Ag Motor vehicle with a sensor arrangement for determining a condition of a driver of the motor vehicle
DE102014207397A1 (en) * 2014-04-17 2015-10-22 Bayerische Motoren Werke Aktiengesellschaft Method, device, vehicle operating system, computer program for operating a vehicle and computer program product

Also Published As

Publication number Publication date
CN1890128A (en) 2007-01-03
JP4994040B2 (en) 2012-08-08
BRPI0417095A (en) 2007-03-13
CN1886639A (en) 2006-12-27
WO2005054786A1 (en) 2005-06-16
WO2005053991A1 (en) 2005-06-16
JP5162132B2 (en) 2013-03-13
US8497880B2 (en) 2013-07-30
JP2007512636A (en) 2007-05-17
US7656313B2 (en) 2010-02-02
JP2011242769A (en) 2011-12-01
CN100541141C (en) 2009-09-16
JP2007512582A (en) 2007-05-17
US20070139176A1 (en) 2007-06-21
US20080079753A1 (en) 2008-04-03
BRPI0417094A (en) 2007-03-13
CN100423964C (en) 2008-10-08
AU2003300514A1 (en) 2005-06-24

Similar Documents

Publication Publication Date Title
WO2005055189A1 (en) Perceptual enhancement displays based on knowledge of head and/or eye and/or gaze position
JP5664784B2 (en) Vehicle information transmission device
Van Erp et al. Vibrotactile in-vehicle navigation system
EP3668742B1 (en) Method for operating a driver assistance system of a motor vehicle and motor vehicle
US20170249923A1 (en) Display system
US20170136878A1 (en) Method for operating a motor vehicle in different driving modes, and motor vehicle
US9222636B2 (en) Vehicle information transmission device
US10343694B2 (en) Vehicle drive assistance system
EP1772317A1 (en) Alarm device
JP4274111B2 (en) Proper inter-vehicle distance display control device
WO2012076954A2 (en) Vehicle information transmission device
WO2004018970A1 (en) Method and device for displaying navigational information for a vehicle
DE102009027026A1 (en) Generation and display of virtual road marking for driver following curve, combines data from navigational and optical interfaces for representation in head-up display
US20180284766A1 (en) Vehicle drive assistance system
Sharfi et al. Enhancement of road delineation can reduce safety
DE102015222844A1 (en) Method and control device for operating a field of view display device for a vehicle
DE102012220100A1 (en) Method for displaying driver formation on head-up display for vehicle, involves selecting display levels of head-up display, and outputting display levels as display icon to represent driver information on head-up display
CN116963926A (en) Improved visual display using augmented reality heads-up display
EP1733190B1 (en) Method and system for presenting information
WO2023054305A1 (en) Display control device, display device, and display control method
DE102020201519B4 (en) Method and apparatus for operating a visual interface between a vehicle and a vehicle occupant
Huang et al. Investigating the effectiveness of Peripheral Vision in reading digital speed limit information displayed in AR-HUD technology
EP1689607A1 (en) Method and system for supporting path control
Coughlin et al. New age philosophy

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ OM PH PL PT RO RU SC SD SE SG SK SL TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PT SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
NENP Non-entry into the national phase

Ref country code: DE

WWW Wipo information: withdrawn in national office

Country of ref document: DE

122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase

Ref country code: JP