DE102017219468A1 - Method and device for using a virtual reality device - Google Patents

Abstract

A method and a device for using the virtual reality function of a virtual reality device by a user in a vehicle performs the following steps: determining the self-motion of the vehicle, determining the position and orientation of the user's head in the vehicle Vehicle, - visual optical features in the virtual space generated by the virtual reality device, wherein the optical visual features correspond to the self-motion of the vehicle and the position and orientation of the head of the user in the vehicle.

Description

The invention relates to a method for using a virtual reality device according to the preamble of claim 1 and to a corresponding device according to the preamble of claim 11.

In the course of automatic driving functions and the increasing use of virtual reality devices, such as a virtual reality glasses, abbreviated VR glasses, the use of such VR glasses in vehicles is also becoming increasingly important. Inmates and drivers can use such glasses while driving, the other contents than the actual environment of the vehicle interior maps.

It is known that head movements and associated acceleration and rotation, which are not reported back to the user by the visual presentation, in very short time, sometimes within a few seconds, can lead to very unpleasant physical sensations. These include dizziness and nausea through to disorientation, headache and circulatory symptoms. According to the current state of research, the cause is assumed to be the acceleration stimuli acting on the body, for which there is no visual feedback.

To complicate the situation in the vehicle is added that in addition to the own body movements on the movement of the vehicle further acceleration forces acting on the user.

To mitigate dizziness in still environments, numerous discoveries are available, including both the design of the presented virtual environments, the processing of user inputs, and additional content creators that provide the user with visual feedback matching the stimuli they perceive. concerns. Among them is a method that fills a thin network of coordinates in the virtual world, which can be used by the user at any time for orientation.

The publication DE 10 2014 009 301 A1 relates to a method for operating a virtual reality goggle, comprising the steps of displaying at least one virtual object from a first viewing angle emanating from a first virtual observation point, this being predefined as a function of a continuously determined orientation of the virtual reality goggles if a predetermined condition is met, performing a virtual position change from the first observation point to a second virtual observation point, after which the virtual object is displayed from a second viewpoint emanating from the second virtual observation point, this depending on the continuously determined orientation of the virtual reality glasses is specified, wherein during the virtual position change, a virtual shutter is closed and reopened so that a view of a wearer of the virtual reality glasses is directed to the virtual object.

The publication DE 10 2014 015 871 A1 relates to a display system for a motor vehicle, comprising at least one electronic data goggles; a detection device, which is designed to detect both a position and a gesture of at least one vehicle occupant and a position of the at least one electronic data glasses within the motor vehicle; a control device, which is designed to control the at least one electronic data goggles in dependence on the detected by the detection device position of the at least one electronic goggles.

The publication WO 2016/150541 A1 relates to a method for operating a virtual reality glasses arranged in a motor vehicle, comprising the steps of: detecting the movement of the motor vehicle; Displaying a first virtual environment using the virtual reality goggles in a first display area of the virtual reality goggles, which is displayed independently of the detected movement of the motor vehicle; simultaneously displaying a second virtual environment by means of the virtual reality glasses in a second display region of the virtual reality glasses, which is displayed corresponding to the detected movement of the motor vehicle.

The publication WO 2016/165799 A1 relates to a method for operating a virtual reality glasses, in which respective virtual observation positions are given corresponding to a continuously detected position of the virtual reality glasses, wherein a virtual object and a virtual environment surrounding the vicinity of a virtual environment of the currently given virtual observation position and a short range of the surrounding area of the virtual environment regardless of the detected position of the virtual reality glasses are displayed from a fixed virtual observation position.

The publication US 2017/0075117 A1 describes a method and apparatus for generating a localized virtual reality system. A user interface is created that provides intuitive navigation of the localized virtual reality system as a function of position of the user's head. The localized virtual reality system can create a desktop computing environment. The user may switch the virtual reality system to an augmented reality viewing mode or a real world viewing mode so that the user can control and influence the assembled stationary environment. It is also possible to switch between the modes in order to allow the user a more comprehensive detection of the situation and a longer operating time.

The publication WO 2016/205175 A1 discloses a computer-implemented method for displaying an object in a first virtual reality view on, for example, a display of an electronic device. The method may further include fading in an emphasis within the first virtual reality view of the object on the display, wherein the method further includes transitioning the presentation of the object from the first virtual reality view to the emphasis in a representation in one second virtual reality view. The second virtual reality view may be a detail view of a part of the object that lies within the emphasis of the first virtual reality view.

The invention is therefore an object of the invention to provide a method and apparatus for using the reality function of a virtual reality device, in particular a virtual reality glasses, which saves the user from the perception of physical inconvenience or inconvenience or this at least reduced.

This object is achieved by a method having the features of claim 1 and by an apparatus having the features of claim 11. Preferred embodiments of the invention are subject of the dependent claims.

• - Ermittlung der Eigenbewegung des Fahrzeugs,
• - Ermittlung der Position und Orientierung des Kopfes des Nutzers im Fahrzeug,
• - Einblenden optischer visueller Merkmale in den von der Virtual-Reality-Einrichtung erzeugten virtuellen Raum, wobei die optischen visuellen Merkmale mit der Eigenbewegung des Fahrzeugs und der Position sowie der Orientierung des Kopfes des Nutzers im Fahrzeug korrespondieren.

The inventive method for using the virtual reality function of a virtual reality device by a user in a vehicle performs the following steps:

• Determination of the proper motion of the vehicle,
• Determination of the position and orientation of the user's head in the vehicle,
• - Fading visual visual features in the generated by the virtual reality device virtual space, wherein the visual visual features correspond to the intrinsic motion of the vehicle and the position and orientation of the head of the user in the vehicle.

Through the visual feedback, the unpleasant physical sensations are at least alleviated, in the best case prevented. Here, the feedback refers both to the self-motion of the vehicle as well as the position and orientation of the user's head.

The self-motion of the vehicle preferably includes the speed including the direction of travel and the acceleration of the vehicle including measurable rolling movements, rolling movements and pitching movements. Therefore, the visual feedback is generated as a function of the variables mentioned, so that the user in his virtual environment receives signals from the real environment acting on him.

More preferably, the virtual reality device is realized by a user-worn virtual reality glasses to be used in an automotive environment.

More preferably, the viewing direction of the user is determined from the position and the orientation of the user's head so that the viewing direction is taken into account when the visual optical features are superimposed. Furthermore, the derivations of the orientation by time can also be taken into account. In other words, in the displayed optical features, the temporal change of the viewing direction is taken into account, since the temporal change of the viewing direction of the user can have an influence on the physical condition of the user, if he uses a virtual reality device.

More preferably, the optical visual features are formed by a particle field, which moves corresponding to the proper motion of the vehicle in the virtual space fluently. In this way, the user can be taught the proper motion of the vehicle.

More preferably, the measurable rolling motions, rolling motions and pitching motions are transmitted to a visual model in the virtual space, the visual model being formed by the visual visual features.

More preferably, the optical visual features can be formed by an ensemble or field of arrows, lines and / or points moving in virtual space. The optical visual features may be adapted in their type, number, line width, transparency and color to the proper motion of the vehicle as well as to the position and orientation of the user's head.

More preferably, the observer position in the virtual space is influenced by the proper movement of the vehicle. Furthermore, the Self-motion of the vehicle can be mapped in the virtual environment. In other words, the observer position is changed by the proper movement and / or the proper motion is reflected in the virtual environment, for example in the objects.

Further preferably, parts of the determined proper motion can be mapped onto the optical visual features, while further parts of the determined proper motion can be mapped into the virtual objects and into the virtual scene. This could be done so that global low-frequency movements of the vehicle are modulated onto the artificial features, while sudden vibrations are additionally mapped to the objects in the scene. Even a reverse application may prove advantageous depending on the situation.

Developers of virtual reality applications can identify such objects and contents of the artificial environment at an interface especially for being able to map such acceleration measurements. Thus, for example, for an application in which one sits in a virtual space and views the external environment through windows, the objects which depict the distant sky, the so-called skydome, could map corresponding acceleration measurements as movements. The moving sky would in turn create a visual impression for the user in the virtual environment that fits well with his perceptions.

• - eine Virtual-Reality-Einrichtung,
• - eine Einrichtung zur Ermittlung der Eigenbewegung des Fahrzeugs,
• - eine Einrichtung zur Bestimmung der Position und der Orientierung des Kopfes des Nutzers, und
• - eine Einrichtung zur Erzeugung optischer visueller Merkmale und deren Einblendung in den von der Virtual-Reality-Einrichtung erzeugten virtuellen Raum, wobei die optischen visuellen Merkmale eine Funktion der Eigenbewegung des Fahrzeugs sowie der Position und Orientierung des Kopfes des Nutzers sind.

The device according to the invention for utilizing the virtual reality function of a virtual reality device by a user in a vehicle, wherein the device for implementing the method explained above is set up and designed, comprises:

• - a virtual reality facility,
• a device for determining the proper motion of the vehicle,
• a device for determining the position and orientation of the user's head, and
• a device for generating optical visual features and their superimposition in the virtual space generated by the virtual reality device, the optical visual features being a function of the vehicle's own motion and of the position and orientation of the user's head.

The object of the invention in summary relates to a system that measures the proper motion and occurring accelerations of the vehicle very accurately and offers the user of the VR glasses combined with the available knowledge about the position and orientation of his head in the vehicle reference system suitable visual content, based on which User perceives appropriate stimuli that correspond appropriately with the perceived accelerations.

Various configurations are possible. Thus, the driver's life could be represented by a visual effect in the virtual environment such that the viewer seems to surround a particle field that moves in a manner corresponding to the vehicle movement and the viewing direction of the user fluently.

However, it could also be taken away from a flow movement and purely the measurable rolling, rolling and nodding of the vehicle cab be transferred to a static visual model in the virtual environment, such as a thin drawn coordinate grid, possibly with the rest of the contents of the virtual environment move accordingly can.

Further, the system could be designed to provide visual stimuli derived from sensory accelerations, but not to faithfully reproduce the accelerations, but in a modified form that has been shown to also mitigate the adverse effect, but not so much to the viewer visually calls.

The advantages of the solution are, in particular, that virtual environments can be used while driving without adversely affecting the user due to the externally supplied acceleration stimuli.

The user receives intuitively experienceable visual feedback on the state of motion of the vehicle, such as standstill, start or drive, without having to give up immersion in the virtual environment.

• 1a zwei in Fahrtrichtung sitzende Personen in einem Fahrzeug mit VR-Brillen,
• 1b die Personen der 1a mit nicht mit der Fahrtrichtung übereinstimmenden Sitzrichtungen,
• 1c die Personen der 1b mit visuellen Hinweisen im Virtual-Reality-Raum,
• 2a eine virtuelle Arbeitsumgebung ohne visuelle Merkmale im Virtual-Reality-Raum,
• 2b eine virtuelle Arbeitsumgebung mit visuellen Merkmalen im Virtual-Reality-Raum, und
• 3 eine schematische Darstellung der Vorrichtung zur Nutzung der Virtual-Reality-Funktion einer Virtual-Reality-Einrichtung.

A preferred embodiment of the invention will be explained below with reference to the drawings. It shows

• 1a two persons sitting in the direction of travel in a vehicle with VR glasses,
• 1b the persons of the 1a with seating directions that do not match the direction of travel,
• 1c the persons of the 1b with visual cues in virtual reality space,
• 2a a virtual work environment without visual features in virtual reality space,
• 2 B a virtual work environment with visual features in virtual reality space, and
• 3 a schematic representation of the device for using the virtual reality function of a virtual reality device.

1a shows an example of two in a vehicle 1 persons 2 . 3 where each person 2 . 3 each a VR glasses 4 . 5 carries a virtual environment for each person 2 . 3 generated, where the vehicle 1 in the direction of the arrow 8th emotional.

By using the VR glasses 4 . 5 while driving take the persons 2 . 3 thus an environment true that does not match the real sensorially sensed by human environmental movements. Even if the persons 2 . 3 like in the 1a in the direction of travel 8th sitting, what is indicated by the, in the direction of travel 8th pointing directions 6 . 7 is displayed, it can quickly lead to negative effects.

1b shows a similar situation as in 1a , where now in the vehicle 1 seated persons 2 . 3 with respective VR glasses 4 . 5 not in the direction of travel 8th sit. That is the direction of view 6 the one person 2 to the direction of travel 8th opposite, while the line of sight 7 the second person 3 obliquely sideways forward. The expected negative effects can be more pronounced and unpleasant in this constellation.

1c shows the situation of 1b with the two persons 2 . 3 in the vehicle 1 with the direction of travel 8th as well as the VR glasses 4 . 5 with the directions of view 6 . 7 , To illustrate the movement of the vehicle 1 in the direction 8th For example, in the virtual reality created by the VR goggles, optical features may be artificially added, which here as in the direction of movement 8th of the vehicle pointing arrows 9 are shown. Because the arrows 9 pointing in the direction of the movement is attempted in this way to bring the movement of the vehicle in the virtual environment. This can have negative effects on the persons 2 . 3 avoided or at least reduced.

In 2 is presented a virtual work environment as the in 1b and 1c inclined person 3 with the line of sight 7 could see diagonally to the top right.

It shows 2a the in 1b described case of the person 3 with the oblique line of sight 7 without artificially added additional visual cues. Shown in 2a is a virtual work environment with keyboard 20 , the hands 21 . 22 the person 3 the 1b and a variety of screen contents 23 to 30 ,

2 B shows the same environment as 2a ie a virtual working environment with keyboard 20 , Hands 21 . 22 the Indian 1 represented person 3 and screens 23 to 30 with screen contents. In addition, visual characteristics corresponding to the direction of travel and travel are superimposed on the virtual work environment, in the present case a bevy of arrows 31 , These optical features 31 can be adjusted in terms of their type, number, line width, transparency and color so that they fulfill the task of visual feedback and are not perceived as disturbing at the same time.

These optical features 31 can also be used as a function of the measured movements of the vehicle 1 the 1 be varied, so that, for example, when the vehicle is stationary 1 a point network is shown, which widens when approaching to a network of lines, which in turn slides when moving fast self-moving through the field of view.

In a further variant, the measurements of the vehicle's own motion 1 and the user's gaze, not just at the artificially added visual features 31 but also on the actual contents of the simulation itself, ie on the keyboard 20 and screens 23 to 30 with screen contents.

Further, such measurements of vehicle motion may affect both the observer position in the artificial world and the artificial virtual environment affecting the movements of the vehicle 1 be imaged. For example, the keyboard could 20 and a screen of the screens 23 to 30 in the virtual environment perform a precisely matching erratic motion when the vehicle 1 travels over a bump, leaving the user 2 . 3 also receives an accurate and appropriate feedback on the perceived acceleration stimuli via these visible movements of the objects of his virtual environment.

3 shows a schematic representation of the device 40 to use the virtual reality feature of a virtual reality device 44 like a VR glasses. An institution 41 Determines the proper motion of the vehicle including the speed, the direction of travel and the accelerations about the longitudinal axis, the transverse axis and the vertical axis of the vehicle, so the pitching and rolling movements, and carries this information of a device 43 for generating visual visual features in the virtual space. Another device 42 determines the position of the user, ie his perspective in relation to the vehicle. This information will also be the device 43 supplied to the production of optical visual features that the visual visual features of the virtual reality device 44 , usually a VR glasses, supplies, so that the optical virtual features can be displayed in the virtual space.

LIST OF REFERENCE NUMBERS

1

vehicle

2

person

3

person

4

VR goggles

5

VR goggles

6

line of sight

7

line of sight

8th

direction of travel

9

optical features

20

keyboard

21

hand

22

hand

23

Screen with screen content

24

Screen with screen content

25

Screen with screen content

26

Screen with screen content

27

Screen with screen content

28

Screen with screen content

29

Screen with screen content

30

Screen with screen content

31

superimposed arrows as optical features

40

Device for using the virtual reality device

41

Device for determining the proper motion

42

Device for determining the position of the user

43

Device for generating visual visual features

44

Virtual reality device

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• DE 102014009301 A1 [0006]
• DE 102014015871 A1 [0007]
• WO 2016/150541 A1 [0008]
• WO 2016/165799 A1 [0009]
• US 2017/0075117 A1 [0010]
• WO 2016/205175 A1 [0011]

Claims (11)

Method for using the virtual reality function of a virtual reality device (4, 5) by a user (2, 3) in a vehicle (1), characterized by the steps determining the self-motion of the vehicle (1), determining the Position and orientation of the head of the user (2, 3) in the vehicle (1), fading visual visual features (9, 31) in the virtual space generated by the virtual reality device, wherein the optical visual features (9, 31 ) with the proper motion of the vehicle (1) and the position and orientation of the head of the user (2, 3) in the vehicle (1) correspond. Method according to Claim 1 , characterized in that the self-motion of the vehicle (1) comprises the speed including the direction of travel (8) and the acceleration of the vehicle (1) including measurable rolling movements, rolling movements and pitching movements. Method according to one of the preceding claims, characterized in that the virtual reality device is a user-worn (2, 3) virtual reality glasses. Method according to Claim 3 , characterized in that from the position and the orientation of the head of the user (2, 3) the viewing direction (6, 7) of the user (2, 3) is determined. Method according to one of the preceding claims, characterized in that the optical visual features (9, 31) are formed by a particle field, which moves corresponding to the proper motion of the vehicle (1) in the virtual space flowing. Method according to one of Claims 1 to 4 , characterized in that the measurable rolling movements, rolling movements and pitching movements are transmitted to a visual model in the virtual space, wherein the visual model is formed by the optical visual features (9, 31). Method according to one of Claims 1 to 4 , characterized in that the optical visual features (9, 31) are formed by an ensemble of arrows or lines moving in virtual space. Method according to one of the preceding claims, characterized in that the visual visual features (9, 31) in their nature, number, line width, transparency and color to the proper movement of the vehicle (1) and the position and orientation of the head of the user (2 , 3). Method according to one of the preceding claims, characterized in that the observer position in the virtual space is influenced by the proper movement of the vehicle (1) and / or that the virtual environment depicts the proper motion. Method according to Claim 9 , characterized in that parts of the determined proper motion are imaged onto the optical visual features (9, 31) and that parts of the determined proper motion are mapped into the virtual objects and into the virtual scene. Device (40) for using the virtual reality function of a virtual reality device (4, 5, 44) by a user (2, 3) in a vehicle (1), wherein the device for carrying out the method according to one of previous claims is set up and designed with a virtual reality device (4, 5, 44), a device (41) for determining the proper motion of the vehicle (1), a device (42) for determining the position of the user (2, 3), and a device (43) for generating optical visual features and their insertion in the virtual space generated by the virtual reality device (4, 5), wherein the visual visual features are a function of the vehicle's own motion and the position of the user.

Images