WO2020035380A1 - Operating vr glasses in autonomously driving vehicle - Google Patents

Operating vr glasses in autonomously driving vehicle Download PDF

Info

Publication number
WO2020035380A1
WO2020035380A1 PCT/EP2019/071266 EP2019071266W WO2020035380A1 WO 2020035380 A1 WO2020035380 A1 WO 2020035380A1 EP 2019071266 W EP2019071266 W EP 2019071266W WO 2020035380 A1 WO2020035380 A1 WO 2020035380A1
Authority
WO
WIPO (PCT)
Prior art keywords
vehicle
glasses
driver
video stream
multimedia device
Prior art date
Application number
PCT/EP2019/071266
Other languages
German (de)
French (fr)
Inventor
Thomas Reichert
Original Assignee
Audi Ag
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Audi Ag filed Critical Audi Ag
Publication of WO2020035380A1 publication Critical patent/WO2020035380A1/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
    • B60K35/20—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor
    • B60K35/28—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
    • G01C21/34—Route searching; Route guidance
    • G01C21/36—Input/output arrangements for on-board computers
    • G01C21/3626—Details of the output of route guidance instructions
    • G01C21/365—Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
    • G01C21/34—Route searching; Route guidance
    • G01C21/36—Input/output arrangements for on-board computers
    • G01C21/3697—Output of additional, non-guidance related information, e.g. low fuel level
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/414—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
    • H04N21/41407—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/414—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
    • H04N21/41422—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance located in transportation means, e.g. personal vehicle
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/4223—Cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/816—Monomedia components thereof involving special video data, e.g 3D video
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/16—Type of output information
    • B60K2360/165—Videos and animations
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/16—Type of output information
    • B60K2360/175—Autonomous driving
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/16—Type of output information
    • B60K2360/178—Warnings

Definitions

  • the invention relates to a method for operating VR glasses during an autonomous drive of a vehicle, in which a vehicle drives autonomously, a driver of the vehicle wears VR glasses connected to a multimedia device and the VR glasses an entertainment program of the multimedia device for displays the driver.
  • the invention further relates to a system for a vehicle for operating VR glasses during an autonomous drive of a vehicle.
  • An autonomously driving vehicle releases a driver of the vehicle from controlling it, ie accelerating, braking, steering and the like. Accordingly, the driver can easily concentrate on an entertainment program of a multimedia device (Multi Media Interface, MMI) of the vehicle, for example a game or a feature film, during an autonomous drive.
  • MMI Multi Media Interface
  • DE 10 2017 116 713 A1 discloses a corresponding method for operating a head-up display (HUD) on a windshield of an autonomously driving vehicle.
  • HUD head-up display
  • This method takes advantage of the fact that the driver does not need to look forward through the windshield during the autonomous drive. However, other occupants of the vehicle of the vehicle may still wish to look forward through the windshield. Remedies are available, for example, in the form of VR (Virtual Reality) glasses, which are also referred to as 3D glasses. When the VR glasses display the entertainment program for the driver, the other passengers are not disturbed by looking through the windshield of the vehicle.
  • VR Virtual Reality
  • an information program can also be displayed in the VR glasses.
  • Such an information program can also relate to an environment of the vehicle.
  • EP 3 244 591 A1 discloses a method for enriching an image of the surroundings of a vehicle.
  • the vehicle transmits the image and position data belonging to the image to an external server which enriches the image with information relating to the depicted environment based on the position data.
  • the server then transfers the enriched image to the vehicle, where it is displayed in VR glasses, for example.
  • US 2018/0040163 A1 eliminates this disadvantage. It discloses an autonomously driving vehicle with a VR system.
  • the VR system comprises VR glasses and is configured to display a video stream from an environment of the vehicle and to modify the video stream by inserting additional content from databases, for example tourist information, in order for a person wearing the VR glasses To give drivers a more informative view of the surroundings, for example a city.
  • Removing and stowing the VR glasses as well as getting used to the real lighting conditions takes a certain amount of time in which the driver cannot decide or intervene. This can cause a reaction, e.g. H. a decision or intervention by the driver being late, which is why the risk of an accident for the vehicle is increased if the driver wears the VR glasses during an autonomous drive of the vehicle.
  • a reaction e.g. H. a decision or intervention by the driver being late, which is why the risk of an accident for the vehicle is increased if the driver wears the VR glasses during an autonomous drive of the vehicle.
  • the invention is therefore based on the object of proposing an improved method for operating VR glasses during an autonomous drive of a vehicle which avoids the disadvantages described.
  • the present invention relates to a method for operating VR glasses during an autonomous drive of a vehicle.
  • the vehicle drives autonomously, a driver of the vehicle wears the VR glasses, which is connected to a multimedia device, and the VR glasses show Entertainment program of the multimedia device for the driver.
  • the starting point of the proposed method is that the driver concentrates on an entertainment program displayed in the VR glasses during an autonomous drive.
  • An entertainment program is generally understood to mean any program which has no relation to the control of the vehicle, that is to say also an information program relating to the travel of the vehicle.
  • the entertainment program is automatically stopped and the VR glasses automatically display a video stream of an environment of the vehicle instead of the entertainment program. Due to the stopping of the entertainment program
  • the VR glasses are connected to a multimedia device of the vehicle as the multimedia device or to a mobile terminal arranged in the vehicle as the multimedia device.
  • the VR glasses can be wireless, for example via Bluetooth to the
  • Multimedia device of the vehicle or coupled to the mobile terminal If the VR glasses are connected to the mobile terminal, for example a smartphone, the mobile terminal is in turn connected to the vehicle.
  • the entertainment program is preferably stopped automatically and the video stream is displayed if on as a result of a situational event
  • the situational event requiring decision and / or intervention can occur unexpectedly in a driving situation of the vehicle or in a traffic situation, detected by a sensor system of the vehicle and by one with the Multimedia device connected control unit of the vehicle can be determined.
  • the control device can then cause the multimedia device to stop the entertainment program and to display the video stream in the VR glasses instead of the entertainment program.
  • the display of the VR glasses is switched, the driver immediately recognizes that he has to make a decision or manually intervene in the autonomous driving of the vehicle.
  • the driver receives relevant information regarding the surroundings of the vehicle, on which he can base his decision or his intervention in the autonomous driving of the vehicle. Accordingly, the driver is easily able to make a decision and / or intervene in the autonomous drive.
  • the driver wears the VR glasses during the decision and / or the intervention and makes a decision based on the displayed video stream or intervenes on the basis of the displayed video stream.
  • the video stream is generated by a camera of the vehicle which is connected to the multimedia device or is attached to the VR glasses and connected to the VR glasses and detects the surroundings.
  • a vehicle that is designed for autonomous driving generally includes extensive sensors for detecting a driving situation and / or a traffic situation.
  • the sensor system in turn also includes a camera that continuously records the surroundings of the vehicle.
  • the video stream generated by this camera can easily be displayed in the driver's VR glasses.
  • the driver can see the surroundings according to his natural field of view in the VR glasses without turning his head.
  • a video stream modified by overlays and / or highlights is advantageously displayed.
  • the video stream is enriched (Augmented Reality, AR).
  • the video stream contains additional information which makes it easier for the driver to decide and / or intervene.
  • a control element of the vehicle is faded into the video stream or a control element visible in the video stream is visually highlighted, in particular a steering wheel, preferably with a directional arrow, if an evasive maneuver by the driver is required.
  • the displayed and / or highlighted control element gives the driver a clear indication of the type of decision and / or intervention required, after which the driver's reaction time is further reduced.
  • the control can be highlighted if it is visible in the video stream.
  • the camera capturing the surroundings must be in an interior of the vehicle, i. H. in a passenger compartment of the vehicle, and the control element can be arranged in the field of view of the camera.
  • An evasive maneuver represents a particularly frequent case of manual intervention by the driver in the autonomous driving of the vehicle.
  • the situational event which requires the evasive maneuver is, for example, an obstacle present on the road ahead of the vehicle.
  • the driver is suggested by the vehicle by means of the video stream a favorable alternative direction.
  • a warning display of the vehicle can be superimposed on the video stream or a warning display visible in the video stream can be visually highlighted.
  • the warning display immediately informs the driver of the type of decision and / or intervention required, which further shortens the driver's reaction time.
  • the warning indicator can be highlighted if it is visible in the video stream.
  • the camera capturing the surroundings must be used be arranged in an interior of the vehicle, ie in a passenger compartment of the vehicle, and the warning display in the field of view of the camera.
  • a change recommendation for the driver is advantageously superimposed on the video stream.
  • the change recommendation can be displayed in a text form and / or a symbol form in order to be easily understood by the driver.
  • the fade-in recommendation is arranged in the video stream in such a way that no area of the surroundings relevant for the decision and / or the intervention is covered by the recommendation.
  • the entertainment program is preferably continued automatically after a decision and / or intervention by the driver.
  • the entertainment program for a decision and / or intervention by the driver is only interrupted automatically, so that the driver can easily concentrate on the entertainment program again after the decision and / or the intervention.
  • the invention also relates to a system for a vehicle for operating VR glasses during an autonomous travel of the vehicle, comprising a multimedia device, VR glasses that can be connected to the multimedia device, and a camera that is connected to the multimedia device and detects the surroundings of the vehicle is configured to display an entertainment program of the multimedia device in the VR glasses for the driver.
  • the VR glasses can be connected to the multimedia device wirelessly or by means of a cable.
  • Such systems can be used in vehicles with a self-driving function, i. H. autonomous vehicles are used to entertain the driver or other occupants of the vehicle during autonomous driving.
  • the system according to the invention is further configured to automatically stop the entertainment program and automatically display a video stream generated by the camera from the surroundings in the VR glasses, in particular in one, instead of the entertainment program method according to the invention.
  • the driver can be picked up from the virtual world and returned to the real world by means of the video stream from the surroundings of the vehicle displayed in the VR glasses.
  • the VR glasses are arranged continuously on the driver's head while he is making a necessary decision and / or is making a necessary intervention in the autonomous driving of the vehicle.
  • Figure 1 is a schematic representation of a block diagram of a
  • Figure 2 is a schematic representation of one in the VR glasses of the in
  • Figure 1 shown system displayed video stream.
  • FIG. 1 shows a schematic illustration of a block diagram of an embodiment of the system 1 according to the invention for a vehicle 10 for operating VR glasses 15 during an autonomous travel of the vehicle 10.
  • the vehicle 10 comprises an operating element 13, which is shown as an example as a steering wheel and a warning indicator 14, which is exemplified as a fuel gauge.
  • the system 1 comprises a multimedia device 11 designed as a multimedia device (MMI) of the vehicle 10, which is connected to the multimedia device 11
  • Multimedia device 11 connectable VR glasses 15 and one with the
  • Multimedia device 11 connected and a camera 30 capturing an environment 30 of the vehicle 10, which is designed, for example, as a forward-facing external camera.
  • the VR glasses 15 are connected to the multimedia device 11 wirelessly, for example by means of Bluetooth.
  • the VR glasses can also be connected to a mobile terminal, for example a smartphone, or the camera can be attached to the VR glasses 15 and connected to it.
  • a construction site with a plurality of warning beacons 31 for redirecting the vehicle 10 is arranged in the front environment 30 of the vehicle 10.
  • the system 1 is configured to display an entertainment program of the multimedia device 11 in the VR glasses 15 for the driver 40 and to automatically stop the entertainment program and automatically a video stream 20 generated by the camera 12 (see FIG. 2) from the environment 30 in instead of the entertainment program the VR glasses 15 display.
  • the system 1 comprises a control device (not shown) connected to the multimedia device 11, which is configured to cause the multimedia device 11 to stop the entertainment program and to display the video stream 20 in the VR glasses 15 instead of the entertainment program.
  • FIG. 2 shows a schematic representation of a video stream 20 displayed in the VR glasses 15 of the system 1 shown in FIG. 1 for the driver 40, in which the warning beacons 31 and the diversion are visible.
  • the video stream 20 is modified by fade-ins and / or highlighting (augmented reality).
  • the control element 13 visible in the video stream 20 is faded into the video stream 20 and supplemented with a directional arrow 22 if an evasive maneuver by the driver 40 is required.
  • the warning display 14 of the vehicle 10 visible in the video stream 20 is faded into the video stream 20 when the vehicle 10 needs to be refueled.
  • Another warning display can indicate, for example, a fault in the system for autonomous driving, for example in the form of a triangle with an exclamation mark shown therein.
  • a modification recommendation 21 in the form of a headlight symbol for the driver 40 is faded into the video stream 20 if a manual switching on of a lighting system of the vehicle 10 is required.
  • the headlight symbol is only to be understood as an example. diverse Further recommendations for action are of course possible, for example a money symbol if a road toll is payable.
  • control element 13 and / or the warning display 14 of the vehicle 10 can also be visually highlighted in the video stream 20 if the control element 13 and / or the warning display 14 is in the field of view of an alternatively arranged in a vehicle interior, for example attached to the VR glasses 15 is arranged.
  • Each overlay is arranged in the video stream 20 such that no relevant area of the environment 30 is covered by the overlay.
  • control element 13 with the directional arrow 22, the warning display 14 and the recommended action 21 in real driving situations of the vehicle are usually neither visually highlighted in this combination nor simultaneously in the video stream 20 or faded into the video stream 20.
  • Deviating control elements, warning displays and / or recommendations for action in any number and in any combination can be visually highlighted in the video stream 20 or faded into the video stream 20.
  • the selected representation is only an example and serves for illustration.
  • the vehicle 10 drives autonomously during the operation of the system 1.
  • the driver 40 of the vehicle 10 wears the VR glasses 15 connected to the multimedia device 11 of the vehicle 10 and concentrates on an entertainment program which the multimedia device 11 displays for the driver 40.
  • a situation and event requiring a decision and / or intervention can occur unexpectedly in a driving situation of the vehicle 10 or a traffic situation, detected by a sensor system of the vehicle 10, which in addition to the camera 12 can include further sensors, not shown, and by the same the control device of the vehicle 10 connected to the multimedia device 11. If a decision and / or a result of such a situational event If intervention by the driver 40 is required, the entertainment program is automatically stopped and the VR glasses 15 automatically display the video stream 20 from the surroundings 30 of the vehicle 10, which is generated by the camera 12 of the vehicle 10, instead of the entertainment program.
  • the driver 40 wears the VR glasses 15 and makes a decision based on the displayed video stream 20 and / or intervenes on the basis of the displayed video stream 20. After the driver 40 decides and / or intervenes, this becomes
  • a major advantage of the method and system 1 according to the invention is that the driver is efficiently picked up from the virtual world and returned to the real world due to the automatic stopping of the entertainment program and automatic start of the video stream 20.
  • the visual information of the video stream 20 relating to the environment 30 of the vehicle 10 is displayed in the same lighting conditions as the entertainment program previously, so that there is no physiological accustoming of the eyes to changed lighting conditions, as a result of which the driver immediately recognizes the environment 30 of the vehicle 10.

Landscapes

  • Engineering & Computer Science (AREA)
  • Remote Sensing (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Automation & Control Theory (AREA)
  • General Physics & Mathematics (AREA)
  • Chemical & Material Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • Mechanical Engineering (AREA)
  • Transportation (AREA)
  • Combustion & Propulsion (AREA)
  • Human Computer Interaction (AREA)
  • Traffic Control Systems (AREA)

Abstract

The invention relates to a method and a system for operating VR glasses during the autonomous driving of a vehicle, in which a vehicle drives autonomously, a driver of the vehicle wears the VR glasses which are connected to a multimedia interface, and the VR glasses display an entertainment program of the multimedia interface for the driver.

Description

Betrieb einer VR-Brille in einem autonom fahrenden Fahrzeug  Operation of VR glasses in an autonomously driving vehicle
BESCHREIBUNG: Die Erfindung betrifft ein Verfahren zum Betreiben einer VR-Brille während einer autonomen Fahrt eines Fahrzeugs, bei dem ein Fahrzeug autonom fährt, ein Fahrer des Fahrzeugs eine mit einem Multimediagerät verbundene VR-Brille trägt und die VR-Brille ein Unterhaltungsprogramm des Multimediageräts für den Fahrer anzeigt. Ferner betrifft die Erfindung ein System für ein Fahrzeug zum Betreiben einer VR-Brille während einer autonomen Fahrt eines Fahrzeugs. DESCRIPTION: The invention relates to a method for operating VR glasses during an autonomous drive of a vehicle, in which a vehicle drives autonomously, a driver of the vehicle wears VR glasses connected to a multimedia device and the VR glasses an entertainment program of the multimedia device for displays the driver. The invention further relates to a system for a vehicle for operating VR glasses during an autonomous drive of a vehicle.
Ein autonom fahrendes Fahrzeug entbindet einen Fahrer des Fahrzeugs von dessen Steuerung, d. h. einem Beschleunigen, Bremsen, Lenken und dergleichen. Entsprechend kann sich der Fahrer während einer autonomen Fahrt ohne Weiteres auf ein Unterhaltungsprogramm eines Multimediageräts (Multi Media Interface, MMI) des Fahrzeugs, beispielsweise ein Spiel oder einen Spielfilm, konzentrieren. Ein entsprechendes Verfahren zum Betreiben eines Head-Up-Displays (HUD) auf einer Windschutzscheibe eines autonom fahrenden Fahrzeugs offenbart die DE 10 2017 116 713 A1. Bei dem Verfahren wird während der autonomen Fahrt ein Unterhaltungsprogramm eines Multimediageräts des Fahrzeugs für einen Fahrer des Fahrzeugs vollflächig auf eine Windschutzscheibe des Fahrzeugs projiziert. Die Projektion wird verkleinert und in einen nicht benötigten Bereich der Windschutzscheibe verschoben, wenn eine Steuerung des Fahrzeugs an den Fahrer übergeben wird. Dieses Verfahren macht sich den Umstand zunutze, dass ein Blick nach vorn durch die Windschutzscheibe während der autonomen Fahrt von dem Fahrer nicht benötigt wird. Allerdings können weitere Insassen des Fahrzeugs des Fahrzeugs dennoch den Wunsch haben, durch die Windschutzscheibe nach vorn zu blicken. Abhilfe ist beispielsweise in Form einer VR (Virtual Reality)- Brille, die auch als 3D-Brille bezeichnet wird, verfügbar. Wenn die VR-Brille das Unterhaltungsprogramm für den Fahrer anzeigt, ist für die weiteren Insassen ein Blicken durch die Windschutzscheibe des Fahrzeugs nicht beeinträchtigt. An autonomously driving vehicle releases a driver of the vehicle from controlling it, ie accelerating, braking, steering and the like. Accordingly, the driver can easily concentrate on an entertainment program of a multimedia device (Multi Media Interface, MMI) of the vehicle, for example a game or a feature film, during an autonomous drive. DE 10 2017 116 713 A1 discloses a corresponding method for operating a head-up display (HUD) on a windshield of an autonomously driving vehicle. In the method, an entertainment program of a multimedia device of the vehicle for a driver of the vehicle is projected over the entire surface onto a windshield of the vehicle during the autonomous drive. The projection is reduced and shifted to an unnecessary area of the windshield when control of the vehicle is handed over to the driver. This method takes advantage of the fact that the driver does not need to look forward through the windshield during the autonomous drive. However, other occupants of the vehicle of the vehicle may still wish to look forward through the windshield. Remedies are available, for example, in the form of VR (Virtual Reality) glasses, which are also referred to as 3D glasses. When the VR glasses display the entertainment program for the driver, the other passengers are not disturbed by looking through the windshield of the vehicle.
Alternativ von einem Unterhaltungsprogramm kann in der VR-Brille aber auch ein Informationsprogramm angezeigt werden. Ein solches Informationsprogramm kann auch eine Umgebung des Fahrzeugs betreffen. So offenbart die EP 3 244 591 A1 ein Verfahren zum Anreichern eines Bildes von einer Umgebung eines Fahrzeugs. Bei dem Verfahren überträgt das Fahrzeug das Bild und zu dem Bild gehörende Positionsdaten zu einem externen Server, der das Bild anhand der Positionsdaten mit Informationen betreffend die abgebildete Umgebung anreichert. Dann überträgt der Server das angereicherte Bild in das Fahrzeug, wo es beispielsweise in einer VR- Brille angezeigt wird. As an alternative to an entertainment program, an information program can also be displayed in the VR glasses. Such an information program can also relate to an environment of the vehicle. For example, EP 3 244 591 A1 discloses a method for enriching an image of the surroundings of a vehicle. In the method, the vehicle transmits the image and position data belonging to the image to an external server which enriches the image with information relating to the depicted environment based on the position data. The server then transfers the enriched image to the vehicle, where it is displayed in VR glasses, for example.
Bei diesem Verfahren werden also einzelne mit zusätzlichen Informationen angereicherte Bilder in der VR-Brille angezeigt. Das Betrachten von Standbildern bleibt im Erleben jedoch deutlich hinter den meisten VR- Anwendungen zurück, welche auf bewegten Bildern beruhen. With this method, individual images enriched with additional information are therefore displayed in the VR glasses. Viewing still images, however, lags far behind most VR applications in the experience, which are based on moving images.
Diesen Nachteil beseitigt die US 2018/0040163 A1. Sie offenbart ein autonom fahrendes Fahrzeug mit einem VR-System. Das VR-System umfasst eine VR-Brille und ist konfiguriert, einen Videostrom von einer Umgebung des Fahrzeugs anzuzeigen und den Videostrom durch Einblenden zusätzlicher Inhalte aus Datenbanken, beispielsweise Touristeninformationen, zu modifizieren, um für einen die VR-Brille tragenden Fahrer eine informativere Ansicht der Umgebung, beispielsweise einer Stadt, zu ermöglichen. US 2018/0040163 A1 eliminates this disadvantage. It discloses an autonomously driving vehicle with a VR system. The VR system comprises VR glasses and is configured to display a video stream from an environment of the vehicle and to modify the video stream by inserting additional content from databases, for example tourist information, in order for a person wearing the VR glasses To give drivers a more informative view of the surroundings, for example a city.
Ein Problem von Unterhaltungs- oder Informationslösungen, welche auf einer VR-Brille beruhen, besteht allerdings darin, dass der Fahrer des Fahrzeugs meist schwierig aus der virtuellen Welt abholbar und in die reale Welt rückführbar ist, wenn ein Entscheiden oder Eingreifen des Fahrers während der autonomen Fahrt des Fahrzeugs erforderlich ist. Zudem muss der Fahrer die VR-Brille zunächst von seinem Kopf lösen, d. h. abnehmen, und sicher verstauen, bevor er in die Steuerung des Fahrzeugs manuell eingreifen kann. Abgesehen davon sind die Lichtverhältnisse der realen Welt in der Regel sehr verschieden von den Lichtverhältnissen der virtuellen Welt, weshalb sich die Augen des Fahrers zunächst an die Lichtverhältnisse der realen Welt gewöhnen, d. h. adaptieren, müssen. One problem with entertainment or information solutions that are based on VR glasses, however, is that the driver of the vehicle is usually difficult to pick up from the virtual world and can be returned to the real world if the driver decides or intervenes during the autonomous one Driving the vehicle is required. In addition, the driver must first remove the VR glasses from his head, i.e. H. remove and stow away safely before he can intervene in the control of the vehicle manually. Apart from that, the lighting conditions of the real world are usually very different from the lighting conditions of the virtual world, which is why the driver's eyes first get used to the lighting conditions of the real world, i. H. to adapt.
Sowohl das Abnehmen und Verstauen der VR-Brille als auch das physiologische Gewöhnen an die realen Lichtverhältnisse braucht eine gewisse Zeit, in der das Entscheiden oder Eingreifen des Fahrers noch nicht möglich ist. Dadurch kann eine Reaktion, d. h. ein Entscheiden oder Eingreifen, des Fahrers verspätet sein, weshalb eine Unfallgefahr für das Fahrzeug erhöht ist, wenn der Fahrer während einer autonomen Fahrt des Fahrzeugs die VR-Brille trägt. Removing and stowing the VR glasses as well as getting used to the real lighting conditions takes a certain amount of time in which the driver cannot decide or intervene. This can cause a reaction, e.g. H. a decision or intervention by the driver being late, which is why the risk of an accident for the vehicle is increased if the driver wears the VR glasses during an autonomous drive of the vehicle.
Der Erfindung liegt daher die Aufgabe zugrunde, ein verbessertes Verfahren zum Betreiben einer VR-Brille während einer autonomen Fahrt eines Fahrzeugs vorzuschlagen, welches die beschriebenen Nachteile vermeidet. Darüber hinaus ist es Aufgabe der Erfindung, ein System zum Betreiben einer VR-Brille während einer autonomen Fahrt eines Fahrzeugs bereitzustellen. The invention is therefore based on the object of proposing an improved method for operating VR glasses during an autonomous drive of a vehicle which avoids the disadvantages described. In addition, it is an object of the invention to provide a system for operating VR glasses during an autonomous drive of a vehicle.
Ein Gegenstand der vorliegenden Erfindung ist ein Verfahren zum Betreiben einer VR-Brille während einer autonomen Fahrt eines Fahrzeugs. Dabei fährt das Fahrzeug autonom, ein Fahrer des Fahrzeugs trägt die VR-Brille, die mit einem Multimediagerät verbunden ist, und die VR-Brille zeigt ein Unterhaltungsprogramm des Multimediageräts für den Fahrer an. Ausgangspunkt des vorgeschlagenen Verfahrens ist, dass der Fahrer sich während einer autonomen Fahrt auf ein in der VR-Brille angezeigtes Unterhaltungsprogramm konzentriert. Dabei wird unter einem Unterhaltungsprogramm allgemein jedes Programm verstanden, das keinen Bezug zum Steuern des Fahrzeugs aufweist, also auch ein die Fahrt des Fahrzeugs betreffendes Informationsprogramm. The present invention relates to a method for operating VR glasses during an autonomous drive of a vehicle. The vehicle drives autonomously, a driver of the vehicle wears the VR glasses, which is connected to a multimedia device, and the VR glasses show Entertainment program of the multimedia device for the driver. The starting point of the proposed method is that the driver concentrates on an entertainment program displayed in the VR glasses during an autonomous drive. An entertainment program is generally understood to mean any program which has no relation to the control of the vehicle, that is to say also an information program relating to the travel of the vehicle.
Bei dem erfindungsgemäßen Verfahren wird weiterhin das Unterhaltungsprogramm automatisch angehalten und zeigt die VR-Brille automatisch statt des Unterhaltungsprogramms einen Videostrom von einer Umgebung des Fahrzeugs an. Aufgrund des Anhaltens desIn the method according to the invention, the entertainment program is automatically stopped and the VR glasses automatically display a video stream of an environment of the vehicle instead of the entertainment program. Due to the stopping of the
Unterhaltungsprogramms und Starten des Videostroms wird der Fahrer effizient aus der virtuellen Welt abgeholt und in die reale Welt rückgeführt. Die visuellen Informationen werden in denselben Lichtverhältnissen angezeigt wie zuvor das Unterhaltungsprogramm, so dass ein physiologisches Gewöhnen seiner Augen an veränderte Lichtverhältnisse entfällt, wodurch der Fahrer die Umgebung des Fahrzeugs sofort erkennt. In einer Ausführungsform ist die VR-Brille mit einer Multimediaeinrichtung des Fahrzeugs als dem Multimediagerät oder mit einem in dem Fahrzeug angeordneten mobilen Endgerät als dem Multimediagerät verbunden. Die VR-Brille kann drahtlos, beispielsweise über Bluetooth an dieEntertainment program and starting the video stream, the driver is efficiently picked up from the virtual world and returned to the real world. The visual information is displayed in the same lighting conditions as the entertainment program before, so that there is no physiological acclimatization of the eyes to changed lighting conditions, so that the driver immediately recognizes the surroundings of the vehicle. In one embodiment, the VR glasses are connected to a multimedia device of the vehicle as the multimedia device or to a mobile terminal arranged in the vehicle as the multimedia device. The VR glasses can be wireless, for example via Bluetooth to the
Multimediaeinrichtung des Fahrzeugs oder an das mobile Endgerät gekoppelt sein. Wenn die VR-Brille mit dem mobilen Endgerät, beispielsweise einem Smartphone, verbunden ist, ist das mobile Endgerät seinerseits mit dem Fahrzeug verbunden. Multimedia device of the vehicle or coupled to the mobile terminal. If the VR glasses are connected to the mobile terminal, for example a smartphone, the mobile terminal is in turn connected to the vehicle.
Bevorzugt wird das Unterhaltungsprogramm automatisch angehalten und der Videostrom angezeigt, wenn infolge eines situativen Ereignisses einThe entertainment program is preferably stopped automatically and the video stream is displayed if on as a result of a situational event
Entscheiden und/oder ein Eingreifen des Fahrers erforderlich ist. Das entscheidungs- und/oder eingriffsbedürftige situative Ereignis kann in einer Fahrsituation des Fahrzeugs oder einer Verkehrssituation unvorhergesehen auftreten, von einer Sensorik des Fahrzeugs erfasst und von einem mit dem Multimediagerät verbundenen Steuergerät des Fahrzeugs festgestellt werden. Das Steuergerät kann dann das Multimediagerät veranlassen, das Unterhaltungsprogramm anzuhalten und statt des Unterhaltungsprogramms den Videostrom in der VR-Brille anzuzeigen. An dem Umschalten der Anzeige der VR-Brille erkennt der Fahrer unmittelbar, dass er eine Entscheidung treffen oder in die autonome Fahrt des Fahrzeugs manuell eingreifen muss. Gleichzeitig erhält der Fahrer relevante Informationen betreffend die Umgebung des Fahrzeugs, auf die er seine Entscheidung oder seinen Eingriff in die autonome Fahrt des Fahrzeugs stützen kann. Entsprechend ist der Fahrer ohne Weiteres in die Lage versetzt, eine Entscheidung zu treffen und/oder in die autonome Fahrt einzugreifen. Deciding and / or driver intervention is required. The situational event requiring decision and / or intervention can occur unexpectedly in a driving situation of the vehicle or in a traffic situation, detected by a sensor system of the vehicle and by one with the Multimedia device connected control unit of the vehicle can be determined. The control device can then cause the multimedia device to stop the entertainment program and to display the video stream in the VR glasses instead of the entertainment program. When the display of the VR glasses is switched, the driver immediately recognizes that he has to make a decision or manually intervene in the autonomous driving of the vehicle. At the same time, the driver receives relevant information regarding the surroundings of the vehicle, on which he can base his decision or his intervention in the autonomous driving of the vehicle. Accordingly, the driver is easily able to make a decision and / or intervene in the autonomous drive.
In einer Ausführungsform des Verfahrens trägt der Fahrer die VR-Brille während des Entscheidens und/oder des Eingreifens und entscheidet anhand des angezeigten Videostroms oder greift anhand des angezeigten Videostroms ein. Ein Abnehmen und Verstauen der VR-Brille entfällt, wodurch eine entsprechende Verzögerung entfällt und eine Entscheidung oder ein Eingriff des Fahrers früh erfolgt, d. h. sehr zeitnah zu dem Zeitpunkt des jeweiligen situativen Ereignisses, welches ein Entscheiden und/oder ein Eingreifen des Fahrers erfordert. In one embodiment of the method, the driver wears the VR glasses during the decision and / or the intervention and makes a decision based on the displayed video stream or intervenes on the basis of the displayed video stream. There is no need to remove and stow the VR glasses, which means that there is no corresponding delay and that the driver decides or intervenes early, i.e. H. very promptly at the time of the respective situational event, which requires the driver to decide and / or intervene.
In weiteren Ausführungsformen wird der Videostrom von einer mit dem Multimediagerät verbundenen oder einer an der VR-Brille angebrachten und mit der VR-Brille verbundenen und die Umgebung erfassenden Kamera des Fahrzeugs erzeugt. Ein Fahrzeug, das für autonomes Fahren ausgebildet ist, umfasst in der Regel eine umfangreiche Sensorik zum Erfassen einer Fahrsituation und/oder einer Verkehrssituation. Die Sensorik umfasst ihrerseits auch eine Kamera, welche fortlaufend die Umgebung des Fahrzeugs erfasst. Der von dieser Kamera erzeugte Videostrom kann ohne Weiteres in der VR-Brille des Fahrers angezeigt werden. Wenn die Kamera an der VR-Brille angebracht und mit dieser verbunden ist, kann der Fahrer die Umgebung entsprechend seinem natürlichen Blickfeld in der VR-Brille sehen, ohne seinen Kopf zu wenden. Vorteilhaft wird ein durch Einblendungen und/oder Hervorhebungen modifizierter Videostrom angezeigt. Mit anderen Worten wird der Videostrom angereichert (Augmented Reality, AR). Infolge der Anreicherung enthält der Videostrom zusätzliche Informationen, welche dem Fahrer ein Entscheiden und/oder Eingreifen erleichtern. In further embodiments, the video stream is generated by a camera of the vehicle which is connected to the multimedia device or is attached to the VR glasses and connected to the VR glasses and detects the surroundings. A vehicle that is designed for autonomous driving generally includes extensive sensors for detecting a driving situation and / or a traffic situation. The sensor system in turn also includes a camera that continuously records the surroundings of the vehicle. The video stream generated by this camera can easily be displayed in the driver's VR glasses. When the camera is attached to and connected to the VR glasses, the driver can see the surroundings according to his natural field of view in the VR glasses without turning his head. A video stream modified by overlays and / or highlights is advantageously displayed. In other words, the video stream is enriched (Augmented Reality, AR). As a result of the enrichment, the video stream contains additional information which makes it easier for the driver to decide and / or intervene.
In einer bevorzugten Ausführungsform wird ein Bedienelement des Fahrzeugs in den Videostrom eingeblendet oder ein in dem Videostrom sichtbares Bedienelement visuell hervorgehoben, insbesondere ein Lenkrad bevorzugt mit einem Richtungspfeil, wenn ein Ausweichmanöver des Fahrers erforderlich ist. Das eingeblendete und/oder hervorgehobene Bedienelement gibt dem Fahrer einen klaren Hinweis auf die Art der erforderlichen Entscheidung und/oder des erforderlichen Eingriffs, worauf eine Reaktionszeit des Fahrers weiter verkürzt ist. Das Bedienelement kann hervorgehoben werden, wenn es im Videostrom sichtbar ist. Dazu muss die die Umgebung erfassende Kamera in einem Innenraum des Fahrzeugs, d. h. in einer Fahrgastzelle des Fahrzeugs, und das Bedienelement im Blickfeld der Kamera angeordnet sein. Dies kann beispielsweise der Fall sein, wenn die Kamera an der VR-Brille angebracht ist und der Fahrer seinen Kopf wie üblich nach vorn gerichtet, d. h. mit dem Gesicht in einer Fahrtrichtung des Fahrzeugs, hält. Ein Ausweichmanöver stellt einen besonders häufigen Fall eines manuellen Eingriffs des Fahrers in die autonome Fahrt des Fahrzeugs dar. Das das Ausweichmanöver erfordernde situative Ereignis ist dabei bspw. ein auf der vorausliegenden Fahrbahn des Fahrzeugs vorhandenes Hindernis. Mittels des Richtungspfeils wird dem Fahrer von dem Fahrzeug mittels des Videostroms eine günstige Ausweichrichtung vorgeschlagen. In a preferred embodiment, a control element of the vehicle is faded into the video stream or a control element visible in the video stream is visually highlighted, in particular a steering wheel, preferably with a directional arrow, if an evasive maneuver by the driver is required. The displayed and / or highlighted control element gives the driver a clear indication of the type of decision and / or intervention required, after which the driver's reaction time is further reduced. The control can be highlighted if it is visible in the video stream. For this purpose, the camera capturing the surroundings must be in an interior of the vehicle, i. H. in a passenger compartment of the vehicle, and the control element can be arranged in the field of view of the camera. This can be the case, for example, when the camera is attached to the VR glasses and the driver's head is forward, as usual, i.e. H. with the face in a direction of travel of the vehicle. An evasive maneuver represents a particularly frequent case of manual intervention by the driver in the autonomous driving of the vehicle. The situational event which requires the evasive maneuver is, for example, an obstacle present on the road ahead of the vehicle. By means of the directional arrow, the driver is suggested by the vehicle by means of the video stream a favorable alternative direction.
In anderen Ausführungsformen kann eine Warnanzeige des Fahrzeugs in den Videostrom eingeblendet oder eine in dem Videostrom sichtbare Warnanzeige visuell hervorgehoben werden. Die Warnanzeige weist den Fahrer unmittelbar auf die Art der erforderlichen Entscheidung und/oder des erforderlichen Eingriffs hin, wodurch eine Reaktionszeit des Fahrers weiter verkürzt ist. Die Warnanzeige kann hervorgehoben werden, wenn sie im Videostrom sichtbar ist. Dazu müssen die die Umgebung erfassende Kamera in einem Innenraum des Fahrzeugs, d. h. in einer Fahrgastzelle des Fahrzeugs, und die Warnanzeige im Blickfeld der Kamera angeordnet sein. In other embodiments, a warning display of the vehicle can be superimposed on the video stream or a warning display visible in the video stream can be visually highlighted. The warning display immediately informs the driver of the type of decision and / or intervention required, which further shortens the driver's reaction time. The warning indicator can be highlighted if it is visible in the video stream. To do this, the camera capturing the surroundings must be used be arranged in an interior of the vehicle, ie in a passenger compartment of the vehicle, and the warning display in the field of view of the camera.
Alternativ oder zusätzlich wird vorteilhaft eine Flandlungsempfehlung für den Fahrer in den Videostrom eingeblendet. Die Flandlungsempfehlung kann in einer Textform und/oder einer Symbolform eingeblendet werden, um von dem Fahrer einfach verstanden werden zu können. Dabei ist die eingeblendete Flandlungsempfehlung in dem Videostrom derart angeordnet, dass kein für die Entscheidung und/oder den Eingriff relevanter Bereich der Umgebung von der Flandlungsempfehlung verdeckt ist. Alternatively or additionally, a change recommendation for the driver is advantageously superimposed on the video stream. The change recommendation can be displayed in a text form and / or a symbol form in order to be easily understood by the driver. The fade-in recommendation is arranged in the video stream in such a way that no area of the surroundings relevant for the decision and / or the intervention is covered by the recommendation.
Bevorzugt wird das Unterhaltungsprogramm nach einem Entscheiden und/oder Eingreifen des Fahrers automatisch fortgesetzt. Mit anderen Worten wird das Unterhaltungsprogramm für ein Entscheiden und/oder Eingreifen des Fahrers lediglich automatisch unterbrochen, so dass sich der Fahrer nach der Entscheidung und/oder dem Eingriff ohne Weiteres erneut auf das Unterhaltungsprogramm konzentrieren kann. The entertainment program is preferably continued automatically after a decision and / or intervention by the driver. In other words, the entertainment program for a decision and / or intervention by the driver is only interrupted automatically, so that the driver can easily concentrate on the entertainment program again after the decision and / or the intervention.
Gegenstand der Erfindung ist auch ein System für ein Fahrzeug zum Betreiben einer VR-Brille während einer autonomen Fahrt des Fahrzeugs, umfassend ein Multimediagerät, eine mit dem Multimediagerät verbindbare VR-Brille und eine mit dem Multimediagerät verbundene und eine Umgebung des Fahrzeugs erfassende Kamera, welches konfiguriert ist, für den Fahrer ein Unterhaltungsprogramm des Multimediageräts in der VR-Brille anzuzeigen. Die VR-Brille kann mit dem Multimediagerät drahtlos oder mittels eines Kabels verbindbar sein. Solche Systeme können in Fahrzeugen mit einer Selbstfahrfunktion, d. h. autonom fahrenden Fahrzeugen, verwendet werden, um den Fahrer oder weitere Insassen des Fahrzeugs während der autonomen Fahrt zu unterhalten. The invention also relates to a system for a vehicle for operating VR glasses during an autonomous travel of the vehicle, comprising a multimedia device, VR glasses that can be connected to the multimedia device, and a camera that is connected to the multimedia device and detects the surroundings of the vehicle is configured to display an entertainment program of the multimedia device in the VR glasses for the driver. The VR glasses can be connected to the multimedia device wirelessly or by means of a cable. Such systems can be used in vehicles with a self-driving function, i. H. autonomous vehicles are used to entertain the driver or other occupants of the vehicle during autonomous driving.
Das erfindungsgemäße System ist ferner konfiguriert, das Unterhaltungsprogramm automatisch anzuhalten und automatisch statt des Unterhaltungsprogramms einen von der Kamera erzeugten Videostrom von der Umgebung in der VR-Brille anzuzeigen, insbesondere in einem erfindungsgemäßen Verfahren. Mittels des in der VR-Brille angezeigten Videostroms von der Umgebung des Fahrzeugs wird der Fahrer aus der virtuellen Welt abgeholt und in die reale Welt rückgeführt. Infolge gleichbleibender Lichtverhältnisse entfällt eine Adaption der Augen des Fahrers, so dass er die Umgebung des Fahrzeugs sofort erkennt. Bei dem erfindungsgemäßen Verfahren ist die VR-Brille ununterbrochen am Kopf des Fahrers angeordnet, während er eine erforderliche Entscheidung trifft und/oder einen erforderlichen Eingriff in die autonome Fahrt des Fahrzeugs vornimmt. The system according to the invention is further configured to automatically stop the entertainment program and automatically display a video stream generated by the camera from the surroundings in the VR glasses, in particular in one, instead of the entertainment program method according to the invention. The driver can be picked up from the virtual world and returned to the real world by means of the video stream from the surroundings of the vehicle displayed in the VR glasses. As the lighting conditions remain the same, the driver's eyes do not have to be adapted so that he immediately recognizes the surroundings of the vehicle. In the method according to the invention, the VR glasses are arranged continuously on the driver's head while he is making a necessary decision and / or is making a necessary intervention in the autonomous driving of the vehicle.
Die Erfindung ist anhand einer Ausführungsform in der Zeichnung schematisch dargestellt und wird unter Bezugnahme auf die Zeichnung weiter beschrieben. Es zeigt: The invention is shown schematically in the drawing using an embodiment and is further described with reference to the drawing. It shows:
Figur 1 in einer schematischen Darstellung ein Blockdiagramm einer Figure 1 is a schematic representation of a block diagram of a
Ausführungsform des erfindungsgemäßen Systems;  Embodiment of the system according to the invention;
Figur 2 in einer schematischen Darstellung einen in der VR-Brille des in Figure 2 is a schematic representation of one in the VR glasses of the in
Figur 1 dargestellten Systems angezeigten Videostrom.  Figure 1 shown system displayed video stream.
Figur 1 zeigt in einer schematischen Darstellung ein Blockdiagramm einer Ausführungsform des erfindungsgemäßen Systems 1 für ein Fahrzeug 10 zum Betreiben einer VR-Brille 15 während einer autonomen Fahrt des Fahrzeugs 10. Das Fahrzeug 10 umfasst ein Bedienelement 13, welches beispielhaft als ein Lenkrad dargestellt ist, und eine Warnanzeige 14, welche beispielhaft als eine Tankanzeige dargestellt ist. FIG. 1 shows a schematic illustration of a block diagram of an embodiment of the system 1 according to the invention for a vehicle 10 for operating VR glasses 15 during an autonomous travel of the vehicle 10. The vehicle 10 comprises an operating element 13, which is shown as an example as a steering wheel and a warning indicator 14, which is exemplified as a fuel gauge.
Das System 1 umfasst ein als eine Multimediaeinrichtung (MMI) des Fahrzeugs 10 ausgebildetes Multimediagerät 11 , die mit demThe system 1 comprises a multimedia device 11 designed as a multimedia device (MMI) of the vehicle 10, which is connected to the
Multimediagerät 11 verbindbare VR-Brille 15 und eine mit demMultimedia device 11 connectable VR glasses 15 and one with the
Multimediagerät 11 verbundene und eine Umgebung 30 des Fahrzeugs 10 erfassende Kamera 12, die beispielhaft als eine nach vorn gerichtete Außenkamera ausgebildet ist. Die VR-Brille 15 ist mit dem Multimediagerät 11 beispielsweise mittels Bluetooth drahtlos verbunden. Alternativ kann die VR-Brille auch mit einem mobilen Endgerät, beispielsweise einem Smartphone, verbunden sein oder die Kamera an der VR-Brille 15 angebracht und mit dieser verbunden sein. In der vorderen Umgebung 30 des Fahrzeugs 10 ist eine Baustelle mit einer Mehrzahl von Warnbaken 31 zum Umleiten des Fahrzeugs 10 angeordnet. Multimedia device 11 connected and a camera 30 capturing an environment 30 of the vehicle 10, which is designed, for example, as a forward-facing external camera. The VR glasses 15 are connected to the multimedia device 11 wirelessly, for example by means of Bluetooth. Alternatively, the VR glasses can also be connected to a mobile terminal, for example a smartphone, or the camera can be attached to the VR glasses 15 and connected to it. A construction site with a plurality of warning beacons 31 for redirecting the vehicle 10 is arranged in the front environment 30 of the vehicle 10.
Das System 1 ist konfiguriert, für den Fahrer 40 ein Unterhaltungsprogramm des Multimediageräts 11 in der VR-Brille 15 anzuzeigen und das Unterhaltungsprogramm automatisch anzuhalten und automatisch statt des Unterhaltungsprogramms einen von der Kamera 12 erzeugten Videostrom 20 (siehe Figur 2) von der Umgebung 30 in der VR-Brille 15 anzuzeigen. Dazu umfasst das System 1 ein nicht dargestelltes mit dem Multimediagerät 11 verbundenes Steuergerät, welches konfiguriert ist, das Multimediagerät 11 zu veranlassen, das Unterhaltungsprogramm anzuhalten und statt des Unterhaltungsprogramms den Videostrom 20 in der VR-Brille 15 anzuzeigen. The system 1 is configured to display an entertainment program of the multimedia device 11 in the VR glasses 15 for the driver 40 and to automatically stop the entertainment program and automatically a video stream 20 generated by the camera 12 (see FIG. 2) from the environment 30 in instead of the entertainment program the VR glasses 15 display. For this purpose, the system 1 comprises a control device (not shown) connected to the multimedia device 11, which is configured to cause the multimedia device 11 to stop the entertainment program and to display the video stream 20 in the VR glasses 15 instead of the entertainment program.
Figur 2 zeigt in einer schematischen Darstellung einen in der VR-Brille 15 des in Figur 1 dargestellten Systems 1 für den Fahrer 40 angezeigten Videostrom 20, in dem die Warnbaken 31 und die Umleitung sichtbar sind. Der Videostrom 20 ist durch Einblendungen und/oder Flervorhebungen modifiziert (Augmented Reality). Das in dem Videostrom 20 sichtbare Bedienelement 13 ist in den Videostrom 20 eingeblendet und mit einem Richtungspfeil 22 ergänzt, wenn ein Ausweichmanöver des Fahrers 40 erforderlich ist. FIG. 2 shows a schematic representation of a video stream 20 displayed in the VR glasses 15 of the system 1 shown in FIG. 1 for the driver 40, in which the warning beacons 31 and the diversion are visible. The video stream 20 is modified by fade-ins and / or highlighting (augmented reality). The control element 13 visible in the video stream 20 is faded into the video stream 20 and supplemented with a directional arrow 22 if an evasive maneuver by the driver 40 is required.
Die in dem Videostrom 20 sichtbare Warnanzeige 14 des Fahrzeugs 10 ist in den Videostrom 20 eingeblendet, wenn ein Betanken des Fahrzeugs 10 erforderlich ist. Eine andere Warnanzeige kann beispielsweise auf eine Störung des Systems für autonomes Fahren, beispielsweise in Form eines Dreiecks mit darin dargestelltem Ausrufezeichen, hinweisen. Ferner ist eine Flandlungsempfehlung 21 in Form eines Scheinwerfersymbols für den Fahrer 40 in den Videostrom 20 eingeblendet, wenn ein manuelles Einschalten einer Beleuchtungsanlage des Fahrzeugs 10 erforderlich ist. Das Scheinwerfersymbol soll lediglich beispielhaft verstanden werden. Vielfältige weitere Handlungsempfehlungen sind selbstverständlich möglich, z.B. ein Geldsymbol, wenn eine Straßenmaut zu entrichten ist. The warning display 14 of the vehicle 10 visible in the video stream 20 is faded into the video stream 20 when the vehicle 10 needs to be refueled. Another warning display can indicate, for example, a fault in the system for autonomous driving, for example in the form of a triangle with an exclamation mark shown therein. Furthermore, a modification recommendation 21 in the form of a headlight symbol for the driver 40 is faded into the video stream 20 if a manual switching on of a lighting system of the vehicle 10 is required. The headlight symbol is only to be understood as an example. diverse Further recommendations for action are of course possible, for example a money symbol if a road toll is payable.
Das Bedienelement 13 und/oder die Warnanzeige 14 des Fahrzeugs 10 kann auch im Videostrom 20 visuell hervorgehoben sein, wenn das Bedienelement 13 und/oder die Warnanzeige 14 im Blickfeld einer alternativ in einem Fahrzeuginneren angeordneten Kamera, die beispielsweise an der VR-Brille 15 angebracht ist, angeordnet ist. Jede Einblendung ist in dem Videostrom 20 derart angeordnet, dass kein relevanter Bereich der Umgebung 30 durch die Einblendung verdeckt ist. The control element 13 and / or the warning display 14 of the vehicle 10 can also be visually highlighted in the video stream 20 if the control element 13 and / or the warning display 14 is in the field of view of an alternatively arranged in a vehicle interior, for example attached to the VR glasses 15 is arranged. Each overlay is arranged in the video stream 20 such that no relevant area of the environment 30 is covered by the overlay.
Es versteht sich, dass das Bedienelement 13 mit dem Richtungspfeil 22, die Warnanzeige 14 und die Handlungsempfehlung 21 in realen Fahrsituationen des Fahrzeugs zumeist weder in dieser Kombination noch gleichzeitig in dem Videostrom 20 visuell hervorgehoben bzw. in den Videostrom 20 eingeblendet sind. Auch können abweichende Bedienelemente, Warnanzeigen und/oder Handlungsempfehlungen in beliebiger Anzahl und in beliebigen Kombinationen in dem Videostrom 20 visuell hervorgehoben bzw. in den Videostrom 20 eingeblendet sein. Die gewählte Darstellung ist lediglich beispielhaft und dient der Veranschaulichung. It goes without saying that the control element 13 with the directional arrow 22, the warning display 14 and the recommended action 21 in real driving situations of the vehicle are usually neither visually highlighted in this combination nor simultaneously in the video stream 20 or faded into the video stream 20. Deviating control elements, warning displays and / or recommendations for action in any number and in any combination can be visually highlighted in the video stream 20 or faded into the video stream 20. The selected representation is only an example and serves for illustration.
Während des Betriebs des Systems 1 fährt das Fahrzeug 10 autonom. Der Fahrer 40 des Fahrzeugs 10 trägt die mit dem Multimediagerät 11 des Fahrzeugs 10 verbundene VR-Brille 15 und konzentriert sich auf ein Unterhaltungsprogramm, welches das Multimediagerät 11 für den Fahrer 40 anzeigt. The vehicle 10 drives autonomously during the operation of the system 1. The driver 40 of the vehicle 10 wears the VR glasses 15 connected to the multimedia device 11 of the vehicle 10 and concentrates on an entertainment program which the multimedia device 11 displays for the driver 40.
Während der autonomen Fahrt kann in einer Fahrsituation des Fahrzeugs 10 oder einer Verkehrssituation ein entscheidungs- und/oder eingriffsbedürftiges situatives Ereignis unvorhergesehen auftreten, von einer Sensorik des Fahrzeugs 10, welche neben der Kamera 12 weitere nicht dargestellte Sensoren umfassen kann, erfasst und von dem mit dem Multimediagerät 11 verbundenen Steuergerät des Fahrzeugs 10 festgestellt werden. Wenn infolge eines solchen situativen Ereignisses ein Entscheiden und/oder ein Eingreifen des Fahrers 40 erforderlich ist, wird das Unterhaltungsprogramm automatisch angehalten, und zeigt die VR-Brille 15 automatisch statt des Unterhaltungsprogramms den Videostrom 20 von der Umgebung 30 des Fahrzeugs 10 an, der von der Kamera 12 des Fahrzeugs 10 erzeugt wird. Während des Entscheidens und/oder des Eingreifens trägt der Fahrer 40 die VR-Brille 15 und entscheidet anhand des angezeigten Videostroms 20 und/oder greift anhand des angezeigten Videostroms 20 ein. Nach dem Entscheiden und/oder Eingreifen des Fahrers 40 wird dasDuring the autonomous drive, a situation and event requiring a decision and / or intervention can occur unexpectedly in a driving situation of the vehicle 10 or a traffic situation, detected by a sensor system of the vehicle 10, which in addition to the camera 12 can include further sensors, not shown, and by the same the control device of the vehicle 10 connected to the multimedia device 11. If a decision and / or a result of such a situational event If intervention by the driver 40 is required, the entertainment program is automatically stopped and the VR glasses 15 automatically display the video stream 20 from the surroundings 30 of the vehicle 10, which is generated by the camera 12 of the vehicle 10, instead of the entertainment program. During the decision and / or the intervention, the driver 40 wears the VR glasses 15 and makes a decision based on the displayed video stream 20 and / or intervenes on the basis of the displayed video stream 20. After the driver 40 decides and / or intervenes, this becomes
Unterhaltungsprogramm von dem Multimediagerät 11 automatisch fortgesetzt. Dabei wird auch das Entscheiden und/oder Eingreifen des Fahrers 40 mittels Sensorik, insbesondere mittelbar, d. h. in der Regel anhand aufgrund des Entscheidens und/oder Eingreifens erfolgter Vorgänge, wie bspw. Ausweichen, Tanken etc., erfasst und dem Steuergerät mitgeteilt, das daraufhin das Multimediagerät 11 veranlasst, das Unterhaltungsprogramm vorzugsweise an der Stelle fortzusetzen, an der es zuvor angehalten wurde. Entertainment program automatically continued from the multimedia device 11. The decision and / or intervention of the driver 40 by means of sensors, in particular indirectly, i. H. As a rule, based on the decisions and / or interventions that have taken place, such as evasive action, refueling, etc., detected and communicated to the control device, which then causes the multimedia device 11 to preferably continue the entertainment program at the point at which it was previously stopped ,
Ein wesentlicher Vorteil des erfindungsgemäßen Verfahrens und Systems 1 besteht darin, dass der Fahrer aufgrund des automatischen Anhaltens des Unterhaltungsprogramms und automatischen Startens des Videostroms 20 effizient aus der virtuellen Welt abgeholt und in die reale Welt rückgeführt wird. Die visuellen Informationen des Videostroms 20 betreffend die Umgebung 30 des Fahrzeugs 10 werden in denselben Lichtverhältnissen angezeigt wie zuvor das Unterhaltungsprogramm, so dass ein physiologisches Gewöhnen seiner Augen an veränderte Lichtverhältnisse entfällt, wodurch der Fahrer die Umgebung 30 des Fahrzeugs 10 sofort erkennt. BEZUGSZEICHENLISTE:A major advantage of the method and system 1 according to the invention is that the driver is efficiently picked up from the virtual world and returned to the real world due to the automatic stopping of the entertainment program and automatic start of the video stream 20. The visual information of the video stream 20 relating to the environment 30 of the vehicle 10 is displayed in the same lighting conditions as the entertainment program previously, so that there is no physiological accustoming of the eyes to changed lighting conditions, as a result of which the driver immediately recognizes the environment 30 of the vehicle 10. LIST OF REFERENCE NUMBERS:
I System I system
10 Fahrzeug  10 vehicle
I I Multimediagerät 12 Kamera  I I multimedia device 12 camera
13 Bedienelement 13 control element
14 Warnanzeige 14 Warning indicator
15 VR-Brille  15 VR glasses
20 Videostrom  20 video stream
21 Handlungsempfehlung21 Recommended action
22 Richtungspfeil22 Direction arrow
30 Umgebung 30 environment
31 Warnbake  31 warning beacon
40 Fahrer  40 drivers

Claims

PATENTANSPRÜCHE: CLAIMS:
1. Verfahren zum Betreiben einer VR-Brille (15) während einer autonomen Fahrt eines Fahrzeugs (10), bei dem 1. A method for operating a VR glasses (15) during an autonomous drive of a vehicle (10), in which
- das Fahrzeug (10) autonom fährt; - The vehicle (10) drives autonomously;
ein Fahrer (40) des Fahrzeugs (10) die VR-Brille (15) trägt, wobei die VR-Brille (15) mit einem Multimediagerät (11 ) verbunden ist;  a driver (40) of the vehicle (10) wears the VR glasses (15), the VR glasses (15) being connected to a multimedia device (11);
die VR-Brille (15) ein Unterhaltungsprogramm des Multimediageräts (11 ) für den Fahrer (40) anzeigt; und bei dem  the VR glasses (15) display an entertainment program of the multimedia device (11) for the driver (40); and with that
- das Unterhaltungsprogramm automatisch angehalten wird und die VR-- the entertainment program is automatically stopped and the VR
Brille (15) automatisch statt des Unterhaltungsprogramms einen Videostrom (20) von einer Umgebung (30) des Fahrzeugs (10) anzeigt. Glasses (15) automatically display a video stream (20) from an environment (30) of the vehicle (10) instead of the entertainment program.
2. Verfahren nach Anspruch 1 , bei dem die VR-Brille (15) mit einer Multimediaeinrichtung des Fahrzeugs (10) als dem Multimediagerät2. The method according to claim 1, wherein the VR glasses (15) with a multimedia device of the vehicle (10) as the multimedia device
(11 ) oder mit einem in dem Fahrzeug (10) angeordneten mobilen Endgerät als dem Multimediagerät (11 ) verbunden ist. (11) or with a mobile terminal arranged in the vehicle (10) as the multimedia device (11).
3. Verfahren nach einem der Ansprüche 1 oder 2, bei dem der Videostrom (20) angezeigt wird, wenn infolge eines situativen Ereignisses ein3. The method according to any one of claims 1 or 2, wherein the video stream (20) is displayed when a due to a situational event
Entscheiden und/oder ein Eingreifen des Fahrers (40) erforderlich ist. Deciding and / or intervention by the driver (40) is required.
4. Verfahren nach einem der Ansprüche 1 bis 3, bei dem der Videostrom (20) von einer mit dem Multimediagerät (11 ) verbundenen oder einer an der VR-Brille (15) angebrachten und mit der VR-Brille (15) verbundenen und die Umgebung (30) erfassenden Kamera (12) des Fahrzeugs (10) erzeugt wird. 4. The method according to any one of claims 1 to 3, wherein the video stream (20) from one connected to the multimedia device (11) or attached to the VR glasses (15) and connected to the VR glasses (15) and the Environment (30) capturing camera (12) of the vehicle (10) is generated.
5. Verfahren nach einem der Ansprüche 1 bis 4, bei dem ein durch Einblendungen und/oder Hervorhebungen modifizierter Videostrom (20) angezeigt wird. 5. The method according to any one of claims 1 to 4, wherein a video stream (20) modified by overlays and / or highlighting is displayed.
6. Verfahren nach Anspruch 5, bei dem ein Bedienelement (13) des Fahrzeugs (10) in den Videostrom (20) eingeblendet oder ein in dem Videostrom (20) sichtbares Bedienelement (13) visuell hervorgehoben wird, wenn ein Ausweichmanöver des Fahrers (40) erforderlich ist. 6. The method according to claim 5, wherein a control element (13) of the vehicle (10) in the video stream (20) or one in the Video stream (20) visible control element (13) is visually highlighted when an evasive maneuver by the driver (40) is required.
7. Verfahren nach einem der Ansprüche 5 oder 6, bei dem eine Warnanzeige (14) des Fahrzeugs (10) in den Videostrom (20) eingeblendet oder eine in dem Videostrom (20) sichtbare Warnanzeige (14) visuell hervorgehoben wird. 7. The method according to any one of claims 5 or 6, in which a warning display (14) of the vehicle (10) is faded into the video stream (20) or a warning indicator (14) visible in the video stream (20) is visually highlighted.
8. Verfahren nach einem der Ansprüche 5 bis 7, bei dem eine Flandlungsempfehlung (21 ) für den Fahrer (40) in den Videostrom (20) eingeblendet wird. 8. The method according to any one of claims 5 to 7, in which a change recommendation (21) for the driver (40) is faded into the video stream (20).
9. Verfahren nach einem der Ansprüche 1 bis 8, bei dem das9. The method according to any one of claims 1 to 8, in which the
Unterhaltungsprogramm nach einem Entscheiden und/oder Eingreifen des Fahrers (40) fortgesetzt wird. Entertainment program after a decision and / or intervention by the driver (40) is continued.
10. System (1 ) für ein Fahrzeug (10) zum Betreiben einer VR-Brille (15) während einer autonomen Fahrt des Fahrzeugs (10), umfassend ein Multimediagerät (11 ), eine mit dem Multimediagerät (11 ) verbindbare VR-Brille (15) und eine mit dem Multimediagerät (11 ) verbundene und eine Umgebung (30) des Fahrzeugs (10) erfassende Kamera (12), welches konfiguriert ist, für den Fahrer (40) ein Unterhaltungsprogramm des Multimediageräts (11 ) in der VR-Brille (15) anzuzeigen und das Unterhaltungsprogramm automatisch anzuhalten und automatisch statt des Unterhaltungsprogramms einen von der Kamera (12) erzeugten10. System (1) for a vehicle (10) for operating VR glasses (15) during an autonomous journey of the vehicle (10), comprising a multimedia device (11), a VR glasses (11) that can be connected to the multimedia device (11) 15) and a camera (12), which is connected to the multimedia device (11) and detects an environment (30) of the vehicle (10) and is configured, for the driver (40) an entertainment program of the multimedia device (11) in the VR glasses (15) and to automatically stop the entertainment program and automatically generate one from the camera (12) instead of the entertainment program
Videostrom (20) von der Umgebung (30) in der VR-Brille (15) anzuzeigen, insbesondere in einem Verfahren nach einem der Ansprüche 1 bis 9. Displaying the video stream (20) from the surroundings (30) in the VR glasses (15), in particular in a method according to one of claims 1 to 9.
PCT/EP2019/071266 2018-08-13 2019-08-07 Operating vr glasses in autonomously driving vehicle WO2020035380A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102018213590 2018-08-13
DE102018213590.8 2018-08-13

Publications (1)

Publication Number Publication Date
WO2020035380A1 true WO2020035380A1 (en) 2020-02-20

Family

ID=67660523

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2019/071266 WO2020035380A1 (en) 2018-08-13 2019-08-07 Operating vr glasses in autonomously driving vehicle

Country Status (1)

Country Link
WO (1) WO2020035380A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021219414A1 (en) * 2020-04-30 2021-11-04 Audi Ag Traffic situation-dependent control of a virtual reality device

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170103571A1 (en) * 2015-10-13 2017-04-13 Here Global B.V. Virtual Reality Environment Responsive to Predictive Route Navigation
EP3244591A1 (en) 2016-05-11 2017-11-15 Baidu USA LLC System and method for providing augmented virtual reality content in autonomous vehicles
DE102017116713A1 (en) 2016-07-25 2018-01-25 Ford Global Technologies, Llc SYSTEMS, METHODS AND DEVICES FOR REPRODUCING MEDIA CONTENT INTO THE VEHICLE BASED ON VEHICLE SENSATIONS
US20180040163A1 (en) 2016-08-05 2018-02-08 Uber Technologies, Inc. Virtual reality experience for a vehicle
WO2018056981A1 (en) * 2016-09-22 2018-03-29 Ford Global Technologies, Llc Solar-powered, virtual-reality windshield

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170103571A1 (en) * 2015-10-13 2017-04-13 Here Global B.V. Virtual Reality Environment Responsive to Predictive Route Navigation
EP3244591A1 (en) 2016-05-11 2017-11-15 Baidu USA LLC System and method for providing augmented virtual reality content in autonomous vehicles
DE102017116713A1 (en) 2016-07-25 2018-01-25 Ford Global Technologies, Llc SYSTEMS, METHODS AND DEVICES FOR REPRODUCING MEDIA CONTENT INTO THE VEHICLE BASED ON VEHICLE SENSATIONS
US20180040163A1 (en) 2016-08-05 2018-02-08 Uber Technologies, Inc. Virtual reality experience for a vehicle
WO2018056981A1 (en) * 2016-09-22 2018-03-29 Ford Global Technologies, Llc Solar-powered, virtual-reality windshield

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021219414A1 (en) * 2020-04-30 2021-11-04 Audi Ag Traffic situation-dependent control of a virtual reality device

Similar Documents

Publication Publication Date Title
EP3543059B1 (en) Method for calculating the insertion of additional information for display on a display unit, device for carrying out the method, motor vehicle and computer program
EP3668742B1 (en) Method for operating a driver assistance system of a motor vehicle and motor vehicle
EP3164311B1 (en) Method for operating a motor vehicle in different driving modes, and motor vehicle
EP3744595B1 (en) Method for a driver assistance system of a vehicle and driver assistance system of a vehicle
EP3762684A1 (en) Overlaying additional information on a display unit
DE102017221191B4 (en) Method for displaying the course of a safety zone in front of a vehicle or an object with a display unit, device for carrying out the method and motor vehicle and computer program
EP3803275A1 (en) Method for calculating an "augmented reality" overlay for displaying a navigation route on an ar display unit, device for carrying out the method, motor vehicle and computer program
EP2172801B1 (en) Method for displaying a visual warning sign
EP3570225A1 (en) Method for calculating augmented reality integration for displaying a navigation route on an ar display unit, device for implementing the method and motor vehicle and computer program
EP3820727B1 (en) Method for operating a display device worn on the head by a vehicle occupant of a motor vehicle, and display system for a motor vehicle
EP2956833B1 (en) Method and device for displaying information of a system
DE102017211244B3 (en) Method for enriching a field of view of a driver of a vehicle with additional information, device for use in an observer vehicle, device for use in an object and motor vehicle
DE102013005342A1 (en) Motor vehicle control device has viewing direction sensor such as position sensor that is arranged at augmented reality glasses, to detect movements of head
DE102013016242A1 (en) Method and device for supporting at least one driver assistance system
EP1527322A1 (en) Method and device for displaying navigational information for a vehicle
WO2019166222A1 (en) Method for calculating an ar-overlay of additional information for a display on a display unit, device for carrying out the method, as well as motor vehicle and computer program
DE102016225268A1 (en) Method and device for operating a display system with data glasses
WO2020035380A1 (en) Operating vr glasses in autonomously driving vehicle
DE102020200047A1 (en) Method and device for displaying virtual navigation elements
EP3847050B1 (en) Method and control device for operating a display device which can be worn on the head in a motor vehicle
DE102021100577A1 (en) Expanded reality
DE102020200902A1 (en) Method and device for displaying additional information in the field of vision of the driver of a vehicle and vehicle
DE102015212288B3 (en) A method of displaying camera image content in a vehicle
DE102018221718A1 (en) Method for operating a driver assistance system and driver assistance system
DE102021111214A1 (en) Display system for a vehicle

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19755561

Country of ref document: EP

Kind code of ref document: A1

DPE2 Request for preliminary examination filed before expiration of 19th month from priority date (pct application filed from 20040101)
122 Ep: pct application non-entry in european phase

Ref document number: 19755561

Country of ref document: EP

Kind code of ref document: A1