EP3903172A1 - Method and arrangement for outputting a head-up display on a head-mounted display - Google Patents

Method and arrangement for outputting a head-up display on a head-mounted display

Info

Publication number
EP3903172A1
EP3903172A1 EP20703942.1A EP20703942A EP3903172A1 EP 3903172 A1 EP3903172 A1 EP 3903172A1 EP 20703942 A EP20703942 A EP 20703942A EP 3903172 A1 EP3903172 A1 EP 3903172A1
Authority
EP
European Patent Office
Prior art keywords
hud
user
gesture
head
elements
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP20703942.1A
Other languages
German (de)
French (fr)
Inventor
Robert Wilde
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Siemens AG
Original Assignee
Siemens AG
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Siemens AG filed Critical Siemens AG
Publication of EP3903172A1 publication Critical patent/EP3903172A1/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16—Constructional details or arrangements
    • G06F1/1613—Constructional details or arrangements for portable computers
    • G06F1/163—Wearable computers, e.g. on a belt
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04815—Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/0138—Head-up displays characterised by optical features comprising image capture systems, e.g. camera
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G02B2027/0178—Eyeglass type
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0179—Display position adjusting means not related to the information to be displayed
    • G02B2027/0187—Display position adjusting means not related to the information to be displayed slaved to motion of at least a part of the body of the user, e.g. head, eye

Definitions

  • the invention relates to the output of a head-up display (HUD) on a head-mounted display (HMD).
  • HUD head-up display
  • HMD head-mounted display
  • head-up display (HUD) is used in the meaning used in computer games.
  • the HUD designates virtual control elements and / or displays which are at least predominantly positioned at the edges of a user's field of vision.
  • the field of vision of the user here designates the area within which the user can fix objects only by eye movements, that is to say without head movements, that is to say he can see clearly with his visual center.
  • head-mounted display denotes a visual output device worn on the head, for example VR glasses (for example HTC Vive) or AR glasses (for example Microsoft Hololens), which are used for the visual output of a virtual or augmented reality Representation of three-dimensional images is set up.
  • VR glasses for example HTC Vive
  • AR glasses for example Microsoft Hololens
  • HUD head-up display
  • the term “head-up display” is also intended in the following to refer to virtual control elements and / or displays which are at least predominantly positioned at the edges of the user's field of vision. This means that most of the elements of the HUD are positioned in the head coordinates at the side of the user's field of vision and are carried along when the head moves.
  • the HMD forms the origin of the head coordinate system. Since they follow the head movements, the elements of the HUD move completely independently of virtual or real ones Objects in world coordinates, which can also be in the user's field of vision.
  • the virtual control elements and / or displays are, for example, elements as are known in principle from graphical user interfaces. Suitable control elements are, for example, symbols, menus, buttons, selection boxes, option fields or list fields. Suitable displays are, for example, text fields, graphic status displays or pictures.
  • the virtual control elements and / or displays can be designed as two-dimensional (flat) or three-dimensional representations.
  • the present invention is intended to provide an alternative to the prior art.
  • a computing unit outputs a HUD with an HMD to a user, with at least some elements of the HUD also being displayed essentially stationary in head coordinates at the edge of a field of view of the user even when the head is moved.
  • the method is characterized in that a tracking device detects at least one hand and / or finger movement of the user as a gesture, and the computing unit recognizes and processes the gesture as an interaction with at least one element of the HUD.
  • the arrangement for outputting a HUD with an HMD includes an HMD and a computing unit which is programmed for outputting a HUD with the HMD to a user, the output of at least some elements of the HUD being essentially stationary in head coordinates at the edge even when the head is moved a field of view of the user is arranged.
  • the arrangement is characterized by a tracking device which is set up to record at least one hand and / or finger movement of the user as a gesture.
  • the computing unit is programmed to recognize and process the gesture as an interaction with at least one element of the HUD.
  • the gesture-based interaction of the user is an immediate interaction with the HUD.
  • An indirect interaction with controllers within an arm's length in front of the user's face would be uncomfortable, unergonomic and also imprecise. For this reason, no attempt has ever been made to control objects that are arranged in head coordinates using VR controllers.
  • the direct interaction with the bare hand enables the user to operate the HUD elements very intuitively, even close to his face.
  • Direct operation thanks to the precise tracking of the fingers of the bare hand has become possible thanks to tracking devices such as Leap Motion or the tracking system from Microsoft Hololens that have only recently appeared on the market.
  • the direct interaction with the HUD using gestures means a disruptive break with previous approaches to implementing a HUD for VR and AR applications. Instead of moving away from plateau in the prior art Information in head coordinates enables the increased ergonomics of gesture-based interaction to use the HUD to display information that should always be available. The user is enabled to operate such menus intuitively with his bare hand in front of his face.
  • the computing unit of the arrangement is, for example, a processor, a notebook or a PC.
  • the processor is, for example, a microprocessor or microcontroller, a system-on-chip or a programmable digital module, such as a "field programmable gate array” (FPGA).
  • FPGA field programmable gate array
  • the interaction is to change an arrangement of the elements of the HUD depending on the gesture.
  • This embodiment represents a disruptive departure from the prior art in which the HUD is designed and implemented as a static arrangement.
  • the embodiment makes it possible for the user to move and rearrange elements of the HUD through direct gesture-based interaction.
  • the elements of the HUD are shown at a distance from the user that does not exceed the reach of the user's hands.
  • the user touches an element of the HUD as a gesture, the interaction actuating or selecting the element in particular.
  • the element is, for example, a button that is touched by the gesture.
  • the user performs a swipe gesture in front of or on an element of the HUD as a gesture, the interaction shifting and / or enlarging or reducing the element.
  • This embodiment offers the advantage that the user can directly place elements of the HUD that are relevant to him prominently in his field of vision, while he can push unimportant elements of the HUD into the periphery.
  • the elements of the HUD are presented to the user at different distances.
  • the user performs a gesture of a beckoning gesture in front of or on an element of the HUD, where the interaction leads the element closer to the user and / or enlarges.
  • the user performs a driving gesture in front of or on an element of the HUD as a gesture, the interaction removing and / or reducing the element from the user.
  • the elements of the HUD head movements of the user follow with a time delay which is between 400 ms and 1200 ms. This is visualized by resetting the elements in head coordinates against the direction of movement for the duration of the time delay from the beginning of the head movement.
  • the time delay between 400 ms and 1200 ms results in a significant increase in ergonomics, since the visual impression of the HUD becomes more natural for the user.
  • the tracking device is mounted on the HMD.
  • a suitable tracking device is the product Leap Motion.
  • a computer program is stored on the computer-readable data carrier and executes the method when it is processed in a processor.
  • the computer program is processed in a processor and executes the method.
  • HUD 1 shows a HUD 1, which with an HMD 2 to a user
  • FIG. 2 shows a gesture 4 with which a user actuates a third element 13 of a HUD
  • FIG. 3 shows a gesture 4 with which a user moves a third element 13 of a HUD into the center of his field of vision 31,
  • FIG. 5 shows a gesture 4 with which a user beckons a third element 13 of a HUD from the background.
  • FIG. 1 shows a HUD 1 which is output to a user 3 using an HMD 2.
  • the HUD 1 is installed within a current display area 10 in front of the user 3 three-dimensionally. It consists of a number of flat fields or windows, which are arranged in spatially staggered levels in front of the face of the user 3 in head coordinates.
  • the HMD 2 has a tracking device 21 with which the hand and / or finger movement of the user 3 can be recognized and recognized as gestures.
  • a tracking device 21 with which the hand and / or finger movement of the user 3 can be recognized and recognized as gestures.
  • the product Leap Motion which is placed on the outside of VR glasses, or the tracking technology from Microsoft Hololens are suitable.
  • the tracking device 21 enables the user 3 to interact directly with the elements of the HUD 1 by means of his hand and / or finger movements in order to rearrange them, for example.
  • FIG. 2 shows a gesture 4 with which a user actuates or pushes a third element 13 of a HUD with his hand 32.
  • the third element 13 is displayed to the user in a plane with a first element 11, a second element 12, a fifth element 15, a sixth element 16 and a seventh element 17.
  • the level is so close to the user's face that he can easily reach the level with his fingertip.
  • gesture 4 shown in FIG. 2 the user touches third element 13 with his bare hand in front of his face.
  • the interaction associated with this gesture could be, for example, a selection of the third element 13 for future purposes, a drag-and-drop operation or the actuation of a button.
  • FIG. 3 shows a gesture 4, here a swiping gesture, with which a third element 13 of an HUD is moved into the center of the field of view 31 of the user as interaction 41.
  • a fourth element 14 and an eighth element 18 are also shown.
  • the third element 13 need not be located centrally in the field of vision 31 of the user. It can continue to edge of the field of view 31, but more central than the other elements, which are more in the periphery.
  • the user performs the gesture 4, for example, in order to be able to better focus and read text which is represented in the third element 13. In this way, he can keep his field of view 31 essentially free and only move the most important elements such as the third element 13 to the center.
  • the elements contain, for example, control elements or displays as they are known from graphical user interfaces.
  • Suitable control elements are, for example, symbols, menus, buttons, check boxes, radio buttons or list boxes.
  • Suitable displays are, for example, text fields, graphic status displays or pictures.
  • the elements can be designed as two-dimensional (flat) or three-dimensional representations.
  • FIG. 4 shows a gesture 4 with which the user at least partially removes a third element 13 of a HUD from his field of vision 31.
  • the correspondingly recognized and executed interaction 41 is shown as an arrow.
  • Figure 5 shows another gesture 4, with which the user beckons a third element 13 of a HUD from the background. After recognition of this gesture by the tracking device 21, a pick-up of the third element 13 is identified and carried out as the associated interaction 41. Conversely, the user can use an expelling gesture to move elements into the background. In this way, the user can also elements that are initially out of the reach of his hands. Furthermore, the user can organize stacks of elements in this way, whereby the HUD is additionally structured and the user's field of vision is better kept clear.

Abstract

The invention relates to a virtual head-up display (HUD) (1) which is displayed to a user (3) with a head-mounted display (HMD) (2), wherein at least some elements of the HUD are displayed substantially stationary in respect of the head's coordinates at the edge of the user's field of vision, even if the head is moved. A tracking device (21) captures at least one hand and/or finger movement by the user as a gesture. The gesture is detected and processed as an interaction with at least one element of the HUD. Direct interaction with the naked hand allows the user to operate, very intuitively, elements of the HUD, even close to his face. Direct interaction with the HUD by means of gestures represents a disruptive break with previous approaches to the implementation of a HUD for VR and AR applications. The enhanced ergonomics of gesture-based interaction allows the HUD to be used to display information which is intended to be constantly available, contrary to the shift away from the placement of information in the head's coordinates, as described in the prior art. The system makes it possible for the user to operate menus of this kind intuitively using the naked hand in front of the face.

Description

Beschreibung description
VERFAHREN UND ANORDNUNG ZUR AUSGABE EINES HEAD UP DISPLAY AUF EINEM HEAD MOUNTED DISPLAY METHOD AND ARRANGEMENT FOR OUTPUTING A HEAD UP DISPLAY ON A HEAD MOUNTED DISPLAY
Die Erfindung betrifft die Ausgabe eines Head-up-Display (HUD) auf einem Head-Mounted-Display (HMD) . The invention relates to the output of a head-up display (HUD) on a head-mounted display (HMD).
Im Folgenden wird der Begriff "Head-up Display" (HUD) in der bei Computerspielen gebräuchlichen Bedeutung verwendet. Das HUD bezeichnet in diesem Zusammenhang virtuelle Steuerelemen te und/oder Anzeigen, welche zumindest überwiegend an Rändern des Blickfelds eines Nutzers positioniert sind. In the following, the term "head-up display" (HUD) is used in the meaning used in computer games. In this context, the HUD designates virtual control elements and / or displays which are at least predominantly positioned at the edges of a user's field of vision.
Das Blickfeld des Nutzers bezeichnet hierbei den Bereich, in nerhalb dessen der Nutzer Objekte lediglich durch Augenbewe gungen, also ohne Kopfbewegungen, fixieren, also mit seinem Sehzentrum scharf sehen kann. The field of vision of the user here designates the area within which the user can fix objects only by eye movements, that is to say without head movements, that is to say he can see clearly with his visual center.
Der Begriff "Head-Mounted Display" (HMD) bezeichnet ein am Kopf getragenes visuelles Ausgabegerät, beispielsweise eine VR-Brille (etwa HTC Vive) oder eine AR-Brille (etwa Microsoft Hololens), welche zur visuellen Ausgabe einer virtuellen oder erweiterten Realität durch Darstellung dreidimensionaler Bil der eingerichtet ist. The term “head-mounted display” (HMD) denotes a visual output device worn on the head, for example VR glasses (for example HTC Vive) or AR glasses (for example Microsoft Hololens), which are used for the visual output of a virtual or augmented reality Representation of three-dimensional images is set up.
In klassischen Computerspielen wird das HUD ortsfest an Rand bereichen eines Bildschirms angezeigt. Bei der Ausgabe eines HUD mit einem HMD müssen gegebenenfalls neue Wege gegangen werden. Der Begriff "Head-up-Display" (HUD) soll im Folgenden auch in diesem Zusammenhang virtuelle Steuerelemente und/oder Anzeigen bezeichnen, welche zumindest überwiegend an Rändern des Blickfelds des Nutzers positioniert sind. Das bedeutet, dass die meisten Elemente des HUD im Wesentlichen ortsfest in Kopfkoordinaten seitlich im Blickfeld des Nutzers positio niert und bei Kopfbewegungen entsprechend mitgeführt werden. Das HMD bildet hierbei den Ursprung des Kopfkoordinatensys- tems . Da sie den Kopfbewegungen folgen, bewegen sich die Ele mente des HUD völlig unabhängig von virtuellen oder realen Objekten in Weltkoordinaten, welche sich ebenfalls im Blick feld des Nutzers befinden können. In classic computer games, the HUD is displayed at the edge of a screen. When issuing a HUD with an HMD, new paths may have to be taken. In this context, the term “head-up display” (HUD) is also intended in the following to refer to virtual control elements and / or displays which are at least predominantly positioned at the edges of the user's field of vision. This means that most of the elements of the HUD are positioned in the head coordinates at the side of the user's field of vision and are carried along when the head moves. The HMD forms the origin of the head coordinate system. Since they follow the head movements, the elements of the HUD move completely independently of virtual or real ones Objects in world coordinates, which can also be in the user's field of vision.
Die virtuellen Steuerelemente und/oder Anzeigen sind bei spielsweise Elemente, wie sie grundsätzlich aus grafischen Benutzeroberflächen bekannt sind. Geeignete Steuerelemente sind beispielsweise Symbole, Menüs, Schaltflächen, Auswahl kästchen, Optionsfelder oder Listenfelder. Geeignete Anzeigen sind beispielsweise Textfelder, grafische Statusanzeigen oder Bilder. Die virtuellen Steuerelemente und/oder Anzeigen kön nen als zweidimensionale (flache) oder dreidimensionale Dar stellungen ausgestaltet sein. The virtual control elements and / or displays are, for example, elements as are known in principle from graphical user interfaces. Suitable control elements are, for example, symbols, menus, buttons, selection boxes, option fields or list fields. Suitable displays are, for example, text fields, graphic status displays or pictures. The virtual control elements and / or displays can be designed as two-dimensional (flat) or three-dimensional representations.
Im Stand der Technik wird die Ausgabe eines HUD mit einem HMD an einen Nutzer, wobei zumindest einige Elemente des HUD auch bei Kopfbewegungen im Wesentlichen ortsfest in Kopfkoordina- ten am Rand eines Blickfelds des Nutzers angezeigt werden, im Virtual-Reality-Bereich als störend empfunden, da sie den Sichtbereich einengen können. Es wird daher dazu übergegan gen, Elemente des HUD statt der Positionierung in Kopfkoordi naten an Objekte in Weltkoordinaten zu heften. Solche Ansätze werden etwa in Joost Peters: "Designing a HUD for a Third- Person VR Game", Virtual Reality Developers Conference, No vember 2016, Vortrag verfügbar als Video am 09.01.2019 im In ternet unter https://youtu.be/f8an45s_-qs, erläutert. In the prior art, the output of a HUD with an HMD to a user, wherein at least some elements of the HUD are also displayed in a stationary manner in head coordinates at the edge of a field of vision of the user, even when the head is moving, is disruptive in the virtual reality area because they can narrow the field of view. It is therefore necessary to attach elements of the HUD to objects in world coordinates instead of positioning them in head coordinates. Such approaches are described in Joost Peters: "Designing a HUD for a Third-Person VR Game", Virtual Reality Developers Conference, No vember 2016, lecture available as video on January 9th, 2019 on the Internet at https://youtu.be/ f8an45s_-qs, explained.
Durch die vorliegende Erfindung soll eine Alternative zum Stand der Technik geschaffen werden. The present invention is intended to provide an alternative to the prior art.
Diese Aufgabe wird erfindungsgemäß dadurch gelöst, dass eine Recheneinheit ein HUD mit einem HMD an einen Nutzer ausgibt, wobei zumindest einige Elemente des HUD auch bei Kopfbewegun gen im Wesentlichen ortsfest in Kopfkoordinaten am Rand eines Blickfelds des Nutzers angezeigt werden. Das Verfahren ist dadurch gekennzeichnet, dass eine Trackingvorrichtung mindes tens eine Hand- und/oder Fingerbewegung des Nutzers als Geste erfasst, und die Recheneinheit die Geste als Interaktion mit mindestens einem Element des HUD erkennt und verarbeitet. Die Anordnung zur Ausgabe eines HUD mit einem HMD beinhaltet ein HMD und eine Recheneinheit, welche programmiert ist zur Ausgabe eines HUD mit dem HMD an einen Nutzer, wobei die Aus gabe zumindest einiger Elemente des HUD auch bei Kopfbewegun gen im Wesentlichen ortsfest in Kopfkoordinaten am Rand eines Blickfelds des Nutzers angeordnet ist. Die Anordnung ist ge kennzeichnet durch eine Trackingvorrichtung, welche zur Er fassung mindestens einer Hand- und/oder Fingerbewegung des Nutzers als Geste eingerichtet ist. Die Recheneinheit ist zur Erkennung und Verarbeitung der Geste als Interaktion mit min destens einem Element des HUD programmiert. This object is achieved according to the invention in that a computing unit outputs a HUD with an HMD to a user, with at least some elements of the HUD also being displayed essentially stationary in head coordinates at the edge of a field of view of the user even when the head is moved. The method is characterized in that a tracking device detects at least one hand and / or finger movement of the user as a gesture, and the computing unit recognizes and processes the gesture as an interaction with at least one element of the HUD. The arrangement for outputting a HUD with an HMD includes an HMD and a computing unit which is programmed for outputting a HUD with the HMD to a user, the output of at least some elements of the HUD being essentially stationary in head coordinates at the edge even when the head is moved a field of view of the user is arranged. The arrangement is characterized by a tracking device which is set up to record at least one hand and / or finger movement of the user as a gesture. The computing unit is programmed to recognize and process the gesture as an interaction with at least one element of the HUD.
Die im Folgenden genannten Vorteile müssen nicht notwendiger weise durch die Gegenstände der unabhängigen Patentansprüche erzielt werden. Vielmehr kann es sich hierbei auch um Vortei le handeln, welche lediglich durch einzelne Ausführungsfor men, Varianten oder Weiterbildungen erzielt werden. Gleiches gilt für die nachfolgenden Erläuterungen. The advantages mentioned below need not necessarily be achieved by the subject matter of the independent claims. Rather, these can also be advantages that can only be achieved by individual embodiments, variants or further developments. The same applies to the following explanations.
Die gestenbasierte Interaktion des Nutzers ist eine unmittel bare Interaktion mit dem HUD. Eine mittelbare Interaktion mit Controllern innerhalb einer Armlänge vor dem Gesicht des Nut zers wäre unangenehm, unergonomisch und auch ungenau. Daher wurde nie versucht, mittels VR-Controllern Objekte anzusteu ern, welche in Kopfkoordinaten angeordnet sind. Die unmittel bare Interaktion mit der bloßen Hand ermöglicht es dem Nut zer, Elemente des HUD auch nah vor seinem Gesicht sehr intui tiv zu bedienen. Die unmittelbare Bedienung durch das genaue Tracking der Finger der bloßen Hand ist durch erst kürzlich am Markt erschienene Trackingvorrichtungen wie Leap Motion oder das Trackingsystem der Microsoft Hololens möglich gewor den . The gesture-based interaction of the user is an immediate interaction with the HUD. An indirect interaction with controllers within an arm's length in front of the user's face would be uncomfortable, unergonomic and also imprecise. For this reason, no attempt has ever been made to control objects that are arranged in head coordinates using VR controllers. The direct interaction with the bare hand enables the user to operate the HUD elements very intuitively, even close to his face. Direct operation thanks to the precise tracking of the fingers of the bare hand has become possible thanks to tracking devices such as Leap Motion or the tracking system from Microsoft Hololens that have only recently appeared on the market.
Die unmittelbare Interaktion mit dem HUD mittels Gesten be deutet einen disruptiven Bruch mit bisherigen Ansätzen zur Implementierung eines HUD für VR- und AR-Anwendungen . Anstatt der im Stand der Technik beschrittenen Abkehr von der Plat- zierung von Informationen in Kopfkoordinaten ermöglicht es die gesteigerte Ergonomie der gestenbasierten Interaktion, das HUD zur Anzeige von Informationen zu nutzen, welche stets verfügbar sein sollen. Der Nutzer wird in die Lage versetzt, derartige Menüs intuitiv mit der bloßen Hand vor seinem Ge sicht zu bedienen. The direct interaction with the HUD using gestures means a disruptive break with previous approaches to implementing a HUD for VR and AR applications. Instead of moving away from plateau in the prior art Information in head coordinates enables the increased ergonomics of gesture-based interaction to use the HUD to display information that should always be available. The user is enabled to operate such menus intuitively with his bare hand in front of his face.
Die Recheneinheit der Anordnung ist beispielsweise ein Pro zessor, ein Notebook oder ein PC. Der Prozessor ist bei spielsweise ein Mikroprozessor oder Mikrocontroller, ein Sys- tem-on-Chip oder ein programmierbarer Digitalbaustein, etwa ein "Field Programmable Gate Array" (FPGA) . The computing unit of the arrangement is, for example, a processor, a notebook or a PC. The processor is, for example, a microprocessor or microcontroller, a system-on-chip or a programmable digital module, such as a "field programmable gate array" (FPGA).
Gemäß einer Ausführungsform besteht die Interaktion darin, eine Anordnung der Elemente des HUD in Abhängigkeit von der Geste abzuändern. According to one embodiment, the interaction is to change an arrangement of the elements of the HUD depending on the gesture.
Diese Ausführungsform bedeutet eine disruptive Abkehr vom Stand der Technik, in dem das HUD als statische Anordnung entworfen und implementiert wird. Durch die Ausführungsform wird es für den Nutzer möglich, Elemente des HUD durch unmit telbare gestenbasierte Interaktion zu bewegen und neu anzu ordnen . This embodiment represents a disruptive departure from the prior art in which the HUD is designed and implemented as a static arrangement. The embodiment makes it possible for the user to move and rearrange elements of the HUD through direct gesture-based interaction.
In einer Weiterbildung werden die Elemente des HUD in einer Entfernung zum Nutzer dargestellt, welche eine Reichweite der Hände des Nutzers nicht überschreitet. Der Nutzer berührt als Geste ein Element des HUD, wobei die Interaktion das Element insbesondere betätigt oder ausgewählt. In a further development, the elements of the HUD are shown at a distance from the user that does not exceed the reach of the user's hands. The user touches an element of the HUD as a gesture, the interaction actuating or selecting the element in particular.
Das Element ist beispielsweise eine Schaltfläche, welche durch die Geste berührt wird. The element is, for example, a button that is touched by the gesture.
Gemäß einer Ausführungsform führt der Nutzer als Geste eine Wischgeste vor oder auf einem Element des HUD aus, wobei die Interaktion das Element verschiebt und/oder vergrößert oder verkleinert . Dies Ausführungsform bietet den Vorteil, dass der Nutzer un mittelbar durch Gesten für ihn relevante Elemente des HUD prominent in seinem Blickfeld platzieren kann, während er für ihn unwichtige Elemente des HUD in die Peripherie wegschieben kann . According to one embodiment, the user performs a swipe gesture in front of or on an element of the HUD as a gesture, the interaction shifting and / or enlarging or reducing the element. This embodiment offers the advantage that the user can directly place elements of the HUD that are relevant to him prominently in his field of vision, while he can push unimportant elements of the HUD into the periphery.
In einer Weiterbildung werden die Elemente des HUD in unter schiedlichen Entfernungen vor dem Nutzer dargestellt. In a further development, the elements of the HUD are presented to the user at different distances.
Gemäß einer Ausführungsform führt der Nutzer als Geste eine Herbeiwink-Geste vor oder auf einem Element des HUD aus, wo bei die Interaktion das Element näher an den Nutzer heran führt und/oder vergrößert. According to one embodiment, the user performs a gesture of a beckoning gesture in front of or on an element of the HUD, where the interaction leads the element closer to the user and / or enlarges.
In einer Weiterbildung führt der Nutzer als Geste eine ver treibende Geste vor oder auf einem Element des HUD aus, wobei die Interaktion das Element vom Nutzer entfernt und/oder ver kleinert . In a further development, the user performs a driving gesture in front of or on an element of the HUD as a gesture, the interaction removing and / or reducing the element from the user.
Diese beiden Weiterbildungen und die Ausführungsform bieten den Vorteil, dass ein sehr komplexes HUD mit einer Vielzahl von Elementen für den Nutzer bereitgestellt werden kann, ohne dessen Blickfeld zu überhäufen. Die räumliche Staffelung ähn lich eines Stapels von Unterlagen auf einem Schreibtisch sorgt für zusätzliche Ordnung. These two further developments and the embodiment offer the advantage that a very complex HUD with a multitude of elements can be provided for the user without overwhelming his field of vision. The spatial staggering, similar to a stack of documents on a desk, creates additional order.
Gemäß einer Ausführungsform folgen die Elemente des HUD Kopf bewegungen des Nutzers mit einer zeitlichen Verzögerung, wel che zwischen 400 ms und 1200 ms liegt. Dies wird visuali- siert, indem die Elemente für die Dauer der zeitlichen Verzö gerung ab Beginn der Kopfbewegung entgegen der Bewegungsrich tung in Kopfkoordinaten zurückversetzt werden. According to one embodiment, the elements of the HUD head movements of the user follow with a time delay which is between 400 ms and 1200 ms. This is visualized by resetting the elements in head coordinates against the direction of movement for the duration of the time delay from the beginning of the head movement.
Die zeitliche Verzögerung zwischen 400 ms und 1200 ms bewirkt eine deutliche Steigerung in der Ergonomie, da der visuelle Eindruck des HUD für den Nutzer natürlicher wird. In einer Weiterbildung ist die Trackingvorrichtung an dem HMD montiert . The time delay between 400 ms and 1200 ms results in a significant increase in ergonomics, since the visual impression of the HUD becomes more natural for the user. In a further development, the tracking device is mounted on the HMD.
Eine hierfür geeignete Trackingvorrichtung ist das Produkt Leap Motion. A suitable tracking device is the product Leap Motion.
Auf dem computerlesbaren Datenträger ist ein Computerprogramm gespeichert, welches das Verfahren ausführt, wenn es in einem Prozessor abgearbeitet wird. A computer program is stored on the computer-readable data carrier and executes the method when it is processed in a processor.
Das Computerprogramm wird in einem Prozessor abgearbeitet und führt dabei das Verfahren aus. The computer program is processed in a processor and executes the method.
Im Folgenden werden Ausführungsbeispiele der Erfindung anhand von Figuren näher erläutert. In den Figuren sind gleiche oder funktionsgleiche Elemente mit denselben Bezugszeichen verse hen, sofern nichts anderes angegeben ist. Es zeigen: Exemplary embodiments of the invention are explained in more detail below with reference to figures. In the figures, identical or functionally identical elements are provided with the same reference numerals, unless stated otherwise. Show it:
Figur 1 ein HUD 1, welches mit einem HMD 2 an einen Nutzer 1 shows a HUD 1, which with an HMD 2 to a user
3 ausgegeben wird, 3 is issued
Figur 2 eine Geste 4, mit der ein Nutzer ein drittes Ele ment 13 eines HUD betätigt, FIG. 2 shows a gesture 4 with which a user actuates a third element 13 of a HUD,
Figur 3 eine Geste 4, mit der ein Nutzer ein drittes Ele ment 13 eines HUD in die Mitte seines Blickfeldes 31 bewegt, FIG. 3 shows a gesture 4 with which a user moves a third element 13 of a HUD into the center of his field of vision 31,
Figur 4 eine Geste 4, mit der ein Nutzer ein drittes Ele ment 13 eines HUD aus der Mitte seines Blickfeldes 31 entfernt, und 4 shows a gesture 4 with which a user removes a third element 13 of a HUD from the center of his field of vision 31, and
Figur 5 eine Geste 4, mit der ein Nutzer ein drittes Ele ment 13 eines HUD aus dem Hintergrund zu sich her anwinkt . FIG. 5 shows a gesture 4 with which a user beckons a third element 13 of a HUD from the background.
Figur 1 zeigt ein HUD 1, welches mit einem HMD 2 an einen Nutzer 3 ausgegeben wird. Das HUD 1 wird innerhalb eines vir- tuellen Anzeige-Bereichs 10 vor dem Nutzer 3 dreidimensional dargestellt. Es besteht aus einer Anzahl flacher Felder oder Fenster, welche in räumlich gestaffelten Ebenen vor dem Ge sicht des Nutzers 3 in Kopfkoordinaten angeordnet sind. FIG. 1 shows a HUD 1 which is output to a user 3 using an HMD 2. The HUD 1 is installed within a current display area 10 in front of the user 3 three-dimensionally. It consists of a number of flat fields or windows, which are arranged in spatially staggered levels in front of the face of the user 3 in head coordinates.
Das HMD 2 weist eine Trackingvorrichtung 21 auf, mit der Hand- und/oder Fingerbewegung des Nutzers 3 als Gesten er fasst und erkannt werden könnten. Hierzu eignet sich bei spielsweise das Produkt Leap Motion, welches außen an einer VR-Brille platziert wird, oder die Trackingtechnologie der Microsoft Hololens. The HMD 2 has a tracking device 21 with which the hand and / or finger movement of the user 3 can be recognized and recognized as gestures. For example, the product Leap Motion, which is placed on the outside of VR glasses, or the tracking technology from Microsoft Hololens are suitable.
Die Trackingvorrichtung 21 ermöglicht es dem Nutzer 3, mit tels seiner Hand- und/oder Fingerbewegungen unmittelbar mit den Elementen des HUD 1 zu interagieren, um diese beispiels weise neu anzuordnen. The tracking device 21 enables the user 3 to interact directly with the elements of the HUD 1 by means of his hand and / or finger movements in order to rearrange them, for example.
Figur 2 zeigt eine Geste 4, mit der ein Nutzer mit seiner Hand 32 ein drittes Element 13 eines HUD betätigt oder ver schiebt. Hierbei wird das dritte Element 13 in einer Ebene mit einem ersten Element 11, einem zweiten Element 12, einem fünften Element 15, einem sechsten Element 16 und einem sieb ten Element 17 vor dem Nutzer angezeigt. Die Ebene ist so nah vor dem Gesicht des Nutzers angeordnet, dass dieser die Ebene mit der Fingerspitze bequem erreichen kann. Mit der in Figur 2 gezeigten Geste 4 berührt der Nutzer das dritte Element 13 mit seiner bloßen Hand vor seinem Gesicht. Die dieser Geste zugeordnete Interaktion könnte beispielsweise eine Auswahl des dritten Elements 13 für zukünftige Zwecke, eine Drag-and- Drop-Operation oder die Betätigung einer Schaltfläche sein. FIG. 2 shows a gesture 4 with which a user actuates or pushes a third element 13 of a HUD with his hand 32. Here, the third element 13 is displayed to the user in a plane with a first element 11, a second element 12, a fifth element 15, a sixth element 16 and a seventh element 17. The level is so close to the user's face that he can easily reach the level with his fingertip. With gesture 4 shown in FIG. 2, the user touches third element 13 with his bare hand in front of his face. The interaction associated with this gesture could be, for example, a selection of the third element 13 for future purposes, a drag-and-drop operation or the actuation of a button.
Figur 3 zeigt eine Geste 4, hier eine Wischgeste, mit der als Interaktion 41 ein drittes Element 13 eines HUD in die Mitte des Blickfeldes 31 des Nutzers bewegt wird. Neben den Elemen ten aus Figur 2 ist noch ein viertes Element 14 und ein ach tes Element 18 eingezeichnet. Nach Ausführen der Interaktion 41 muss sich das dritte Element 13 nicht zentral im Blickfeld 31 des Nutzers befinden. Es kann sich auch weiterhin am Rand des Blickfeldes 31 befinden, jedoch zentraler als die anderen Elemente, welche sich stärker in der Peripherie befinden. Der Nutzer führt die Geste 4 beispielsweise durch, um Text, wel cher im dritten Element 13 dargestellt wird, besser fokussie ren und lesen zu können. Auf diese Weise kann er sein Blick feld 31 im Wesentlichen freihalten und lediglich die wich tigsten Elemente wie das dritte Element 13 in die Mitte rü cken. Nicht benötigte Elemente können auch nahezu in der Pe ripherie verschwinden, wo der Nutzer sie nicht mehr fixieren kann; wie in Figur 3 gezeigt, befinden sich die meisten Ele mente lediglich zur Hälfte im Blickfeld des Nutzers. Der Nut zer sieht jedoch auch die andere Hälfte unscharf im Augenwin kel, sofern der Sichtbereich des HMD dies zulässt. In jedem Fall kann er durch entsprechende Gesten die jeweiligen Ele mente in sein Blickfeld 31 zurückholen, um sie sich genauer anzusehen . FIG. 3 shows a gesture 4, here a swiping gesture, with which a third element 13 of an HUD is moved into the center of the field of view 31 of the user as interaction 41. In addition to the elements from FIG. 2, a fourth element 14 and an eighth element 18 are also shown. After the interaction 41 has been carried out, the third element 13 need not be located centrally in the field of vision 31 of the user. It can continue to edge of the field of view 31, but more central than the other elements, which are more in the periphery. The user performs the gesture 4, for example, in order to be able to better focus and read text which is represented in the third element 13. In this way, he can keep his field of view 31 essentially free and only move the most important elements such as the third element 13 to the center. Elements that are not required can almost disappear in the periphery, where the user can no longer fix them; As shown in Figure 3, most elements are only half in the user's field of vision. However, the user also sees the other half out of focus in the angle of the eye, provided the viewing area of the HMD permits this. In any case, he can bring the respective elements back into his field of view 31 by appropriate gestures in order to take a closer look at them.
Die Elemente enthalten beispielsweise Steuerelemente oder An zeigen, wie sie aus grafischen Benutzeroberflächen bekannt sind. Geeignete Steuerelemente sind beispielsweise Symbole, Menüs, Schaltflächen, Auswahlkästchen, Optionsfelder oder Listenfelder. Geeignete Anzeigen sind beispielsweise Textfel der, grafische Statusanzeigen oder Bilder. Die Elemente kön nen als zweidimensionale (flache) oder dreidimensionale Dar stellungen ausgestaltet sein. The elements contain, for example, control elements or displays as they are known from graphical user interfaces. Suitable control elements are, for example, symbols, menus, buttons, check boxes, radio buttons or list boxes. Suitable displays are, for example, text fields, graphic status displays or pictures. The elements can be designed as two-dimensional (flat) or three-dimensional representations.
Figur 4 zeigt eine Geste 4, mit der der Nutzer ein drittes Element 13 eines HUD aus seinem Blickfeldes 31 zumindest teilweise entfernt. Die entsprechend erkannte und ausgeführte Interaktion 41 ist als Pfeil dargestellt. FIG. 4 shows a gesture 4 with which the user at least partially removes a third element 13 of a HUD from his field of vision 31. The correspondingly recognized and executed interaction 41 is shown as an arrow.
Figur 5 eine weitere Geste 4, mit der der Nutzer ein drittes Element 13 eines HUD aus dem Hintergrund zu sich heranwinkt. Nach Erkennung dieser Geste durch die Trackingvorrichtung 21 wird als zugehörige Interaktion 41 ein Heranholen des dritten Elements 13 identifiziert und ausgeführt. Umgekehrt kann der Nutzer mit einer vertreibenden Geste Elemente in den Hinter grund rücken. Auf diese Weise kann der Nutzer auch mit Eie- menten interagieren, welche sich zunächst nicht in Reichweite seiner Hände befinden. Weiterhin kann der Nutzer auf diese Weise Stapel von Elementen organisieren, wodurch das HUD zu sätzlich strukturiert und das Blickfeld des Nutzers besser freigehalten wird. Figure 5 shows another gesture 4, with which the user beckons a third element 13 of a HUD from the background. After recognition of this gesture by the tracking device 21, a pick-up of the third element 13 is identified and carried out as the associated interaction 41. Conversely, the user can use an expelling gesture to move elements into the background. In this way, the user can also elements that are initially out of the reach of his hands. Furthermore, the user can organize stacks of elements in this way, whereby the HUD is additionally structured and the user's field of vision is better kept clear.
Obwohl die Erfindung durch die Ausführungsbeispiele im Detail illustriert und beschrieben wurde, ist sie nicht durch die offenbarten Beispiele eingeschränkt. Andere Variationen kön- nen vom Fachmann hieraus abgeleitet werden, ohne den Schutz umfang der Erfindung zu verlassen. Die beschriebenen Ausfüh rungsbeispiele, Varianten, Ausführungsformen und Weiterbil dungen können auch frei miteinander kombiniert werden. Although the invention has been illustrated and described in detail by the exemplary embodiments, it is not restricted by the disclosed examples. Other variations can be derived therefrom by those skilled in the art without departing from the scope of the invention. The described exemplary embodiments, variants, embodiments and further training can also be freely combined with one another.

Claims

Patentansprüche Claims
1. Verfahren zur Ausgabe eines HUD (1) mit einem HMD (2), 1. Method for issuing a HUD (1) with an HMD (2),
bei dem eine Recheneinheit ein HUD (1) mit einem HMD (2) an einen Nutzer (3) ausgibt, wobei zumindest einige Ele mente (11, ..., 18) des HUD auch bei Kopfbewegungen im We sentlichen ortsfest in Kopfkoordinaten am Rand eines in which a computing unit outputs a HUD (1) with an HMD (2) to a user (3), with at least some elements (11, ..., 18) of the HUD being essentially stationary in head coordinates at the edge even with head movements one
Blickfelds (31) des Nutzers angezeigt werden, Field of view (31) of the user are displayed,
dadurch gekennzeichnet, dass characterized in that
eine Trackingvorrichtung (21) mindestens eine Hand- und/oder Fingerbewegung des Nutzers als Geste (4) erfasst, und a tracking device (21) detects at least one hand and / or finger movement of the user as a gesture (4), and
die Recheneinheit die Geste als Interaktion (41) mit min destens einem Element des HUD erkennt und verarbeitet. the computing unit recognizes and processes the gesture as interaction (41) with at least one element of the HUD.
2. Verfahren nach Anspruch 1, 2. The method according to claim 1,
wobei die Interaktion darin besteht, eine Anordnung der Elemente des HUD in Abhängigkeit von der Geste abzuändern. the interaction is to change an arrangement of the elements of the HUD depending on the gesture.
3. Verfahren nach einem der vorhergehenden Ansprüche, 3. The method according to any one of the preceding claims,
bei dem die Elemente des HUD in einer Entfernung zum Nut zer dargestellt werden, welche eine Reichweite der Hände des Nutzers nicht überschreitet, und in which the elements of the HUD are shown at a distance from the user which does not exceed the reach of the user's hands, and
bei dem der Nutzer als Geste ein Element des HUD berührt, wobei die Interaktion das Element insbesondere betätigt oder ausgewählt. in which the user touches an element of the HUD as a gesture, the interaction actuating or selecting the element in particular.
4. Verfahren nach einem der vorhergehenden Ansprüche, 4. The method according to any one of the preceding claims,
bei dem der Nutzer als Geste eine Wischgeste vor oder auf einem Element des HUD ausführt, wobei die Interaktion das Element verschiebt und/oder vergrößert oder verkleinert. in which the user performs a swipe gesture in front of or on an element of the HUD as a gesture, the interaction shifting and / or enlarging or reducing the element.
5. Verfahren nach einem der vorhergehenden Ansprüche, 5. The method according to any one of the preceding claims,
bei dem die Elemente des HUD in unterschiedlichen Entfer nungen vor dem Nutzer dargestellt werden. where the elements of the HUD are presented to the user at different distances.
6. Verfahren nach Anspruch 5, bei dem der Nutzer als Geste eine Herbeiwink-Geste vor o- der auf einem Element des HUD ausführt, wobei die Interak tion das Element näher an den Nutzer heranführt und/oder vergrößert . 6. The method according to claim 5, in which the user executes a gesture of gesture before or on an element of the HUD, the interaction bringing the element closer to and / or enlarging the user.
7. Verfahren nach Anspruch 5 oder 6, 7. The method according to claim 5 or 6,
bei dem der Nutzer als Geste eine vertreibende Geste vor oder auf einem Element des HUD ausführt, wobei die Inter aktion das Element vom Nutzer entfernt und/oder verklei nert . in which the user performs a distributing gesture in front of or on an element of the HUD as a gesture, the interaction removing and / or reducing the element from the user.
8. Verfahren nach einem der vorhergehenden Ansprüche, 8. The method according to any one of the preceding claims,
bei dem die Elemente des HUD Kopfbewegungen des Nutzers mit einer zeitlichen Verzögerung folgen, welche zwischen 400 ms und 1200 ms liegt, und in which the elements of the HUD follow the user's head movements with a time delay which is between 400 ms and 1200 ms, and
bei dem dies visualisiert wird, indem die Elemente für die Dauer der zeitlichen Verzögerung ab Beginn der Kopfbewe gung entgegen der Bewegungsrichtung in Kopfkoordinaten zu rückversetzt werden. in which this is visualized by the elements being set back in head coordinates against the direction of movement for the duration of the time delay from the beginning of the head movement.
9. Anordnung zur Ausgabe eines HUD (1) mit einem HMD (2), mit einem HMD (2), 9. Arrangement for dispensing a HUD (1) with an HMD (2), with an HMD (2),
mit einer Recheneinheit, welche programmiert ist zur Aus gabe eines HUD (1) mit dem HMD (2) an einen Nutzer (3), wobei die Ausgabe zumindest einiger Elemente (11, ..., 18) des HUD auch bei Kopfbewegungen im Wesentlichen ortsfest in Kopfkoordinaten am Rand eines Blickfelds (31) des Nut zers angeordnet ist, with a computing unit which is programmed to output a HUD (1) with the HMD (2) to a user (3), the output of at least some elements (11, ..., 18) of the HUD essentially also during head movements is fixedly arranged in head coordinates at the edge of a field of view (31) of the user,
gekennzeichnet durch marked by
eine Trackingvorrichtung (21), eingerichtet zur Erfassung mindestens einer Hand- und/oder Fingerbewegung des Nutzers als Geste (4), und dadurch, dass a tracking device (21) set up to record at least one hand and / or finger movement of the user as a gesture (4), and thereby
die Recheneinheit programmiert ist zur Erkennung und Ver arbeitung der Geste als Interaktion (41) mit mindestens einem Element des HUD. the computing unit is programmed to recognize and process the gesture as an interaction (41) with at least one element of the HUD.
10. Anordnung nach Anspruch 9, 10. Arrangement according to claim 9,
bei der die Trackingvorrichtung an dem HMD montiert ist. where the tracking device is mounted on the HMD.
11. Computerlesbarer Datenträger, 11. Computer-readable data carrier,
auf dem ein Computerprogramm gespeichert ist, welches das Verfahren nach einem der Ansprüche 1 bis 8 ausführt, wenn es in einem Prozessor abgearbeitet wird. on which a computer program is stored which executes the method according to one of claims 1 to 8 when it is processed in a processor.
12. Computerprogramm, 12. Computer program,
welches in einem Prozessor abgearbeitet wird und dabei das Verfahren nach einem der Ansprüche 1 bis 8 ausführt. which is processed in a processor and thereby executes the method according to one of claims 1 to 8.
EP20703942.1A 2019-01-30 2020-01-20 Method and arrangement for outputting a head-up display on a head-mounted display Pending EP3903172A1 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
DE102019201175 2019-01-30
DE102019202512.9A DE102019202512A1 (en) 2019-01-30 2019-02-25 Method and arrangement for outputting a HUD on an HMD
PCT/EP2020/051273 WO2020156854A1 (en) 2019-01-30 2020-01-20 Method and arrangement for outputting a head-up display on a head-mounted display

Publications (1)

Publication Number Publication Date
EP3903172A1 true EP3903172A1 (en) 2021-11-03

Family

ID=71524683

Family Applications (1)

Application Number Title Priority Date Filing Date
EP20703942.1A Pending EP3903172A1 (en) 2019-01-30 2020-01-20 Method and arrangement for outputting a head-up display on a head-mounted display

Country Status (5)

Country Link
US (1) US20220091662A1 (en)
EP (1) EP3903172A1 (en)
CN (1) CN113330393A (en)
DE (1) DE102019202512A1 (en)
WO (1) WO2020156854A1 (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022088103A1 (en) * 2020-10-30 2022-05-05 华为技术有限公司 Image calibration method and apparatus
CN114839782B (en) * 2022-06-07 2023-08-18 上汽大众汽车有限公司 Vehicle-mounted enhanced display system for vehicle control and information display
WO2024061163A1 (en) * 2022-09-19 2024-03-28 北京字跳网络技术有限公司 Human-computer interaction method, display method, apparatus, and device

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10451874B2 (en) * 2013-09-25 2019-10-22 Seiko Epson Corporation Image display device, method of controlling image display device, computer program, and image display system
US10203762B2 (en) * 2014-03-11 2019-02-12 Magic Leap, Inc. Methods and systems for creating virtual and augmented reality
JP6776155B2 (en) * 2017-02-28 2020-10-28 株式会社コロプラ A method for providing virtual reality, a program for causing a computer to execute the method, and an information processing device for executing the program.
US10521020B2 (en) * 2017-07-12 2019-12-31 Unity IPR ApS Methods and systems for displaying UI elements in mixed reality environments

Also Published As

Publication number Publication date
WO2020156854A1 (en) 2020-08-06
US20220091662A1 (en) 2022-03-24
CN113330393A (en) 2021-08-31
DE102019202512A1 (en) 2020-07-30

Similar Documents

Publication Publication Date Title
EP3903172A1 (en) Method and arrangement for outputting a head-up display on a head-mounted display
EP2989524B1 (en) Method for interacting with an object displayed on data eyeglasses
EP2017756A1 (en) Method for displaying and/or processing or manipulating image data for medical purposes with gesture recognition
Piumsomboon et al. User-defined gestures for augmented reality
DE60025926T2 (en) CONTROL OF THE DEPTH MOTION IN A DISPLAY DEVICE WITH A MULTILAYER SCREEN
EP2467822B1 (en) Continuous determination of a perspective
DE102012020607B4 (en) A motor vehicle with a gesture control device and method for controlling a selection element
EP2795451B1 (en) Method for operating a multi-touch-capable display and device having a multi-touch-capable display
EP1374027B1 (en) Positioning of areas displayed on a user interface
WO2017144298A1 (en) User interface comprising a plurality of display units, and method for positioning contents on a plurality of display units
DE102017121342A1 (en) DISPLAY CONTROL DEVICE, DISPLAY CONTROL SYSTEM AND DISPLAY CONTROL METHOD
DE10084249T5 (en) Additional LCD panel with touch screen
WO2014016162A2 (en) Input device having a lowerable touch-sensitive surface
EP2943866B1 (en) Method and device for providing a user interface in a vehicle
Colley et al. Guided touch screen: Enhanced eyes-free interaction
DE102016204692A1 (en) Control of multiple selection on touch-sensitive surfaces
DE102016115479A1 (en) User interface for electronic device, input processing method and electronic device
DE102013014889A1 (en) Mouse pointer control for an operating device
Chen et al. Study on the interactive interface design of gaze input smart TV
DE102004027289A1 (en) User navigating method, involves processing sensor information to realize motion in area of head or line of user`s vision, evaluating motion information to derive navigation action and effecting modification of document section by action
EP3292549A1 (en) Electronic system and method for planning and accomplishing a mission to be accomplished by means of a vehicle
DE19720051A1 (en) Method for entering information into a computer
DE102020100073A1 (en) Method and system for selecting an entry from a list
DE102019206443A1 (en) Display device with a touch-sensitive user interface and an operating and gesture recognition device, motor vehicle with a display device, mobile terminal with a display device, and a method for operating the display device
EP3281098B1 (en) Gesture-sensitive screen and computer with gesture-sensitive screen

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: UNKNOWN

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20210714

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20230804