DE102016225261A1 - Method and device for operating a display system with data glasses - Google Patents

Abstract

The invention relates to a method for determining a pose detail for a pose of data glasses in a motor vehicle, wherein the pose specification indicates at least one spatial position and / or at least one spatial orientation of the data glasses relative to a predefined vehicle coordinate system, comprising the following steps: detecting one or multiple accelerations in at least one lateral spatial direction and / or at least one rotational spatial direction, - for each detected acceleration, continuously determining a position change as a function of the detected acceleration in the at least one lateral spatial direction and / or an alignment change in the at least one rotational spatial direction; for each particular position change and / or orientation change, updating the spatial position and / or the spatial orientation of the pose indication of the data glasses by integrating the determined position change and / or setting the spatial position and / or the spatial orientation of the pose information to a respective predetermined reference position and / or a reference orientation, if no position change and alignment change are detected for a certain period of time.

Description

Technical area

The invention relates to display systems for data glasses, in particular for use in a motor vehicle.

Technical background

Data glasses, also known as head-mounted displays, are known which, with the aid of a display device, can display an image on one or two display surfaces in the field of vision of the wearer of the data glasses. The display surfaces correspond to reflection surfaces that direct images into the eye of the wearer of the data glasses. The viewing apertures of the data glasses are transparent, so that the data glasses can be used to perceive the real environment in the usual way. The display surfaces are located in the viewing apertures and may be semitransparent, so that information to be displayed, such as text, symbols, graphics, video displays and the like, can be superimposed on the perception of the environment. In addition, data glasses are also known which comprise a non-transparent display.

The information is usually presented to the wearer of the data glasses contact-analogous, i. represented such that the information is superimposed on or oriented towards a specific object in the real environment or that the information to be displayed is displayed in a specific orientation of the data glasses or their wearer. Furthermore, it is desirable to display the information so that it appears perspectively correct with respect to the object in the real environment, i. the illusion arises that the object of the real environment has actually been supplemented by the additional feature of visual information.

In order to display the information corresponding to the contact analog on the display surfaces of the data glasses, it is necessary to determine the position of the object in the environment and the pose of the data glasses, i. to know the 3D position as well as the 3D alignment of the data glasses in a vehicle coordinate system, with respect to the object.

To determine the pose of the data glasses, a pose detection unit can be provided in the data glasses. The pose detection unit typically includes a camera and computing device, e.g. in the form of a microprocessor. Using the camera recorded images of the environment of the wearer of the data glasses can be determined based on deposited images or structures of the vehicle interior, the pose of the data glasses in the vehicle interior. This process is also called tracking.

This is from the publication DE 10 2014 206 623 A1 a device for determining the pose of data goggles is known, comprising a display and a camera. The device is designed to take pictures of the surroundings of the data glasses with the aid of the camera, to detect the image of a stored and predefined area of the surroundings in the recordings of the camera, to detect a feature in the detected image of the area and to reflect the pose of the data glasses Consideration of the particular feature in the recordings.

Furthermore, the pose of data glasses can also be provided by an external pose detection unit, in which an interior camera detects the head of the wearer of the data glasses and detects the pose of the data glasses by evaluating the camera image.

The provision of a pose detection function with a data camera in a data glasses or externally the data glasses is complex. Not all data glasses therefore have a data camera with which an absolute tracking, i. the determination of the absolute pose in the interior of the motor vehicle with respect to the vehicle coordinate system, is possible. However, in data glasses that have no absolute tracking, no contact-analogous functions are possible because they always need an updated indication of their pose.

By relative tracking, e.g. can be performed with a multi-axis acceleration sensor, only a relative position and direction change is detected, but not with respect to a reference point in the vehicle.

It is an object of the invention to provide data glasses which do not contain a pose detection unit for absolute tracking, e.g. based on a data glasses camera, or whose pose can not be recognized by an external pose detection unit to provide with a simple pose detection, can be implemented by the functions for the execution of an indication of a pose of the data glasses in a reference system is needed.

Disclosure of the invention

This object is achieved by the method for operating a display system with a data glasses according to claim 1 and by the data glasses according to the independent claim.

Further embodiments are specified in the dependent claims.

• - Erfassen von einer oder mehreren Beschleunigungen in mindestens einer lateralen Raumrichtung und/oder mindestens einer rotatorischen Raumrichtung;
• - für jede erfasste Beschleunigung, Kontinuierliches Bestimmen einer Positionsänderung abhängig von der erfassten Beschleunigung in der mindestens einen lateralen Raumrichtung und/oder einer Ausrichtungsänderung in der mindestens einen rotatorischen Raumrichtung;
• - für jede bestimmte Positionsänderung und/oder Ausrichtungsänderung, Aktualisieren der räumlichen Position und/oder der räumlichen Ausrichtung der Posenangabe der Datenbrille durch Integrieren der bestimmten Positionsänderung und/oder der bestimmten Ausrichtungsänderung;
• - Einstellen der räumlichen Position und/oder der räumlichen Ausrichtung der Posenangabe auf eine vorgegebene Referenzposition und/oder eine Referenzausrichtung, wenn für eine bestimmte Zeitdauer keine Positionsänderung und Ausrichtungsänderung festgestellt wird.

According to a first aspect, a method is provided for determining pose information for a pose of data glasses in a motor vehicle, wherein the pose statement indicates at least one spatial position and / or at least one spatial orientation of the data glasses relative to a predefined vehicle coordinate system, with the following steps:

• Detecting one or more accelerations in at least one lateral spatial direction and / or at least one rotational spatial direction;
• for each detected acceleration, continuously determining a change in position as a function of the detected acceleration in the at least one lateral spatial direction and / or an alignment change in the at least one rotational spatial direction;
• for each particular position change and / or orientation change, updating the spatial position and / or spatial orientation of the pose indication of the data glasses by integrating the determined position change and / or the determined orientation change;
• - Setting the spatial position and / or the spatial orientation of the pose information to a predetermined reference position and / or a reference orientation when no position change and alignment change is detected for a certain period of time.

One idea of the above method is to determine a pose detail for at least one spatial position and / or at least one spatial orientation of the data goggles relative to a given vehicle coordinate system by integrating relative movements of the data goggles detected by means of an acceleration sensor in the data goggles , It is assumed that the wearer of the data glasses most of the time looking forward in the direction of travel, whereby a certain Referenzpose, i. a certain reference position and certain reference orientation defined by smart glasses. By recognizing that the wearer of the data glasses gazes for a long time in a certain direction, in particular when the motor vehicle is moving, it can be assumed that the data glasses are in the reference pose and the pose specification of the data glasses is initialized to the reference pose. Subsequent integration based on the movements of the data glasses may then be based on the pose calibrated by the reference pose.

The pose statement allows the data goggles to be interpreted as absolute keystrokes at low vehicle speeds and with clear movements of the data glasses, e.g. a head shake or a slap on the data glasses, e.g. to call a given function or to interact with a control function. Even a rudimentary function control by the head movement is possible by a head direction-dependent function activation is realized.

Overall, the pose detail can be used for a simple absolute tracking of the data glasses for functions that an inaccurate pose statement of the data glasses are sufficient and are not safety-critical.

Furthermore, it can be provided that no change in position and alignment change of the data glasses is detected if the change in position in the at least one lateral spatial direction and / or the alignment change in the at least one rotational spatial direction are smaller in magnitude than a predetermined tolerance amount.

According to one embodiment, the reference position and the reference orientation may indicate a normal position of the driver in the motor vehicle and a straight-ahead orientation in the direction of travel of the motor vehicle.

Furthermore, the resetting of the current pose information to the reference position and / or the reference orientation can be limited to a certain number of times within a predetermined time period.

It may be provided that the detected accelerations are corrected by means of driving state variables, wherein the driving state variables include vehicle accelerations detected by a vehicle acceleration sensor.

Thus, vehicle accelerations of the motor vehicle can be transmitted to the data glasses in a display system so that the poses determined in the data glasses in accordance with the above method can be corrected or made plausible by the acceleration of the motor vehicle.

In particular, the detected vehicle accelerations may be taken into account taking into account the offset of the position of the data glasses from an origin of a vehicle coordinate system. The correction is thus preferably carried out taking into account the different fulcrums of data glasses and motor vehicle, wherein lateral and rotational movements of the motor vehicle in each case both on the lateral As well as the rotational tracking of the data glasses can affect, since the origins of the two coordinate systems are offset.

According to one embodiment, the detected accelerations may be corrected using driving state quantities, wherein the driving state variables include a vehicle speed, a steering angle, a road gradient, and / or an accelerator pedal position.

• - einen oder mehrere Beschleunigungssensoren zum Erfassen von einer oder mehreren Beschleunigungen in mindestens einer lateralen Raumrichtung und/oder mindestens einer rotatorischen Raumrichtung;
• - eine Steuereinheit zum Bestimmen einer Posenangabe für eine Pose der Datenbrille, wobei die Posenangabe mindestens eine räumliche Position und/oder mindestens eine räumliche Ausrichtung der Datenbrille bezogen auf ein vorgegebenes Fahrzeugkoordinatensystem angibt, wobei die Steuereinheit ausgebildet ist, um
  • ○ für jede erfasste Beschleunigung, kontinuierlich eine Positionsänderung abhängig von der erfassten Beschleunigung in der mindestens einen lateralen Raumrichtung und/oder einer Ausrichtungsänderung in der mindestens einen rotatorischen Raumrichtung zu bestimmen;
  • ○ für jede bestimmte Positionsänderung und/oder Ausrichtungsänderung, die räumlichen Position und/oder der räumlichen Ausrichtung der Posenangabe der Datenbrille durch Integrieren der bestimmten Positionsänderung und/oder der bestimmten Ausrichtungsänderung zu aktualisieren;
  • ○ um die räumliche Position und/oder die räumliche Ausrichtung der Posenangabe auf eine jeweilige vorgegebene Referenzposition und/oder eine Referenzausrichtung zurückzusetzen, wenn für eine bestimmte Zeitdauer keine Positionsänderung und Ausrichtungsänderung festgestellt wird.

According to another aspect, there is provided data goggles for a display system in a motor vehicle, comprising:

• one or more acceleration sensors for detecting one or more accelerations in at least one lateral spatial direction and / or at least one rotational spatial direction;
• a control unit for determining a pose information for a pose of the data glasses, wherein the pose statement indicates at least one spatial position and / or at least one spatial orientation of the data glasses relative to a predefined vehicle coordinate system, wherein the control unit is designed to
  • For each detected acceleration, continuously determining a change in position as a function of the detected acceleration in the at least one lateral spatial direction and / or an alignment change in the at least one rotational spatial direction;
  • ○ for each particular position change and / or orientation change, to update the spatial position and / or the spatial orientation of the pose indication of the data glasses by integrating the determined position change and / or the determined orientation change;
  • ○ to reset the spatial position and / or the spatial orientation of the pose information to a respective predetermined reference position and / or a reference orientation, if no position change and alignment change is detected for a certain period of time.

According to a further aspect, a display system with the above data glasses and with a driver assistance system is provided, which is configured to transmit driving state variables to the data glasses, wherein the control device of the data glasses is designed to adapt the detected accelerations depending on the driving state variables.

list of figures

• 1 eine schematische Darstellung eines Anzeigesystems mit einer Datenbrille zum Einsatz in einem Kraftfahrzeug; und
• 2 ein Flussdiagramm zur Veranschaulichung eines Verfahrens zum Betreiben des Anzeigesystems.

Embodiments are explained below with reference to the accompanying drawings. Show it:

• 1 a schematic representation of a display system with a data glasses for use in a motor vehicle; and
• 2 a flowchart illustrating a method for operating the display system.

Description of embodiments

1 shows a schematic representation of a display system 1 in particular for use in a motor vehicle. The display system 1 includes a driver assistance system 2 that in communication connection 4 with a data glasses 3 stands. The communication connection 4 is as a communication channel 4 formed, for example in the form of a wireless communication connection or a wired communication connection. The communication connection 4 is capable of any kind of data and information between the driver assistance system 2 and the data glasses 3 to transmit, for example, based on a packet-based data transmission.

The data glasses 3 is with two transparent lenses 32 in a frame 31 are enclosed in a conventional manner. The frame 31 is with eyeglass temples 33 provided so that the data glasses 3 can be worn on the head of a user in a conventional manner.

At least one of the lenses 32 can with a transparent display area 35 be provided in by a suitable means, such as one on the frame 31 arranged display device 36 , a display image can be displayed. The display device 36 may include a projection device or the like configured to provide a display image on the display surfaces. Furthermore, a control unit 37 be provided to the display information to the display device 36 for display on the display surfaces 35 to convey. The control unit 37 may include a microprocessor or similar computation unit and via the communication link 4 with the driver assistance system 2 keep in touch.

Due to the transparent design of the display surface 35 The electronically generated image can pass through the display area 35 Superimposed perceptible real environment. Using the display device 36 can be visual information, such as a text, a symbol, a video information, a Graphics or the like, on one or both display surfaces 35 being represented. By the separate representation of the information on both display surfaces 35 This can be shown spatially, ie three-dimensionally.

The data glasses 3 can be worn like a typical visual aid on the user's head, the data glasses 3 with the frame 31 rests on the nose of the user and the hanger 33 abut laterally on the user's head. The viewing direction of the user in the straight-ahead direction then takes place through the viewing windows 32 essentially through the transparent display surfaces 35 , so that the user's line of sight of the orientation of the data glasses 3 corresponds or can be derived from it.

The control unit 37 may be suitably configured to data glasses functions and functions of the display system 2 perform. This can be done by the driver assistance system 2 with the data glasses 3 in order to display information regarding contact-analog or non-contact-analogue virtual objects to the data glasses 3 to convey. The display information defines the position and representation of the virtual objects with respect to the vehicle, ie in a vehicle coordinate system (reference system of the motor vehicle). The vehicle coordinate system is defined by the longitudinal axis and transverse axis of the vehicle in which the data glasses 3 operated, determined.

For example, the control unit 37 Perform a pose detection function in the form of a tracking process to detect a pose of the head or the data glasses in a vehicle interior. The pose of a data glasses 3 herein refers to the spatial position of the data glasses 3 in the vehicle coordinate system and their three-dimensional orientation.

This can be done by the control unit 37 the data glasses 3 with an accelerometer 38 be connected to a lateral and / or rotational movement of the head of the wearer of the data glasses 3 to capture in at least one spatial direction. The acceleration values generated by the acceleration sensor 37 be provided, serve a pose statement of the data glasses 3 to determine. The acceleration sensor 37 is preferably a 3-axis acceleration sensor that detects accelerations in three lateral spatial directions and three rotational spatial directions. The posibility of the data glasses 3 Herein, the spatial position of the data glasses in a vehicle-fixed coordinate system and their spatial orientation, which is determined by the method described below for pose detection.

The driver assistance system 2 stands with facilities 21 of the motor vehicle, such as an engine control unit or the like, in order to detect vehicle state variables and driving state quantities, such as vehicle speed, vehicle acceleration magnitudes in a plurality of spatial directions, and magnitudes indicative of movement of the motor vehicle, such as steering angle, accelerator pedal position, and the like.

By integrating the acceleration values for each of the axes of motion, a relative position and orientation change of the data glasses 3 be detected for each of the lateral and rotational spatial directions.

The use of acceleration sensors for pose detection is advantageous if the data glasses do not comprise a pose detection unit for detecting an absolute pose of the data glasses and also the driver assistance system is not provided with a pose detection function, so that even there no absolute pose, i. a pose with respect to a vehicle coordinate system can be provided.

A method of operating the data glasses 3 is based on the flowchart of 2 described in more detail.

In step S1, the lateral and rotational accelerations of the data glasses in three spatial directions in each case by means of the six-axis acceleration sensor 37 determined.

In step S2, for each of the detected acceleration values, a respective position change is determined as a function of the detected lateral accelerations and a respective change in orientation as a function of the detected rotational accelerations. Changes can be determined by subtraction between each of the detected acceleration values and the previously acquired acceleration value.

Based on the movement speeds thus determined, the pose information is updated in step S3 by adding up the position changes and the alignment changes, such as the current spatial position and the spatial orientation of the data glasses 3 to determine. It is initially assumed that an assumed reference pose or the last determined pose statement.

In step S4, it is checked whether the orientation and position of the data glasses 3 for a long time have not or only slightly, ie within predetermined tolerance ranges changed. This can be used to determine if the data glasses 3 For has not moved for a long time. A longer stay in a rest position can be interpreted as an indication that the wearer of the data glasses most of the time looks straight forward, since the direction of looking forward in the direction of travel is the viewing direction, which occupies the driver for longer periods of time.

Will be a longer rest phase of the position and orientation of the data glasses 3 recognized (alternative: Yes), the process proceeds to step S5. Otherwise, it returns to step S1.

In step S4, in step S5, the pose indication becomes an absolute position and an absolute alignment of the data glasses 3 set to corresponding predetermined reference values, ie the position and orientation of the data glasses 3 are each set to a predetermined reference value with respect to an origin of a vehicle coordinate system. Subsequently, it returns to step S1.

In this way, a pose statement with a position and orientation of the data glasses 3 Relative to a vehicle coordinate system can be provided with sufficient accuracy by the relative movement of the data glasses 3 is evaluated (integrated) and in a current pose of the data glasses 3 (based on vehicle coordinate system) is implemented. It is believed that with prolonged lingering the pose of the data glasses 3 at a certain position and in a specific orientation, the wearer of the data glasses looks forward in the longitudinal direction. This allows readjustment of tracking errors resulting from the double integration of acceleration values acquired by the accelerometer into an absolute pose of the data glasses 3 can result.

The determination of the positions and orientation of the data glasses by integration in step S3 can be corrected by vehicle data. For example, by the driver assistance system 2 determined by vehicle provided acceleration sensors vehicle acceleration values in a vehicle-fixed coordinate system and the communication link 4 to the data glasses 3 be transmitted. This can then use the vehicle acceleration values to correct the position and orientation determination in the data goggles 3 use. It should be noted that lateral movements of the vehicle affect both the lateral and rotational tracking of the data glasses 3 because the origins of the two coordinate systems are offset from each other. The same applies to rotational movements of the vehicle, which also affect both the lateral and rotational tracking of the data glasses 3 can affect.

If acceleration values of the motor vehicle are not available, alternatively or additionally vehicle data, such as vehicle speed and steering angle, which determines a yaw rate of the motor vehicle, and an accelerator pedal position which can indicate an acceleration of the motor vehicle can be taken into account.

Basically, the above method allows determination of the absolute pose of the data glasses 3 with a relatively low accuracy. However, simple functions of the data glasses 3 For their performance there is a low resolution of the pose specification of data glasses 3 is sufficient. In particular, feature calls may be enabled by headers, such as head-direction-dependent feature activation, such as a look to the right.

LIST OF REFERENCE NUMBERS

1

display system

2

Driver assistance system

21

Facilities of the motor vehicle

3

data glasses

31

frame

32

visors

33

Temples

35

transparent display area

36

display

37

control unit

38

accelerometer

4

Data transmission channel

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• DE 102014206623 A1 [0006]

Claims (10)

Method for determining a pose information for a pose of a data goggle (3) provided in a motor vehicle, wherein the pose specification indicates at least one spatial position and / or at least one spatial orientation of the data goggles (3) relative to a predefined vehicle coordinate system, with the following steps: Detecting (S1) one or more accelerations in at least one lateral spatial direction and / or at least one rotational spatial direction; for each detected acceleration, continuously determining (S2) a change in position as a function of the detected acceleration in the at least one lateral spatial direction and / or an alignment change in the at least one rotational spatial direction; for each particular position change and / or orientation change, updating (S3) the spatial position and / or the spatial orientation of the pose indication of the data glasses (3) by integrating the determined position change and / or the determined orientation change; - setting (S5) the spatial position and / or the spatial orientation of the pose information to a respective predetermined reference position and / or a reference orientation, if no position change and alignment change is detected for a certain period of time. Method according to Claim 1 , wherein no change in the pose of the data glasses (3) is detected if the change in position in the at least one lateral spatial direction and / or the orientation change in the at least one rotational spatial direction for which acceleration values are detected are smaller in magnitude than a predetermined tolerance amount. Method according to Claim 1 or 2 wherein the reference position and reference orientation indicates a normal position of the driver in the motor vehicle and a straight-ahead orientation in the direction of travel of the motor vehicle relative to the vehicle coordinate system. Method according to one of Claims 1 to 3 wherein the setting of the current pose indication to the predetermined reference position and reference orientation is limited to a certain number of times within a predetermined period of time. Method according to one of Claims 1 to 4 wherein the detected accelerations are corrected using driving state quantities, wherein the driving state quantities include vehicle accelerations detected by a vehicle acceleration sensor. Method according to Claim 5 wherein the detected vehicle accelerations are taken into account taking into account the offset of the position of the data glasses (3) indicated by current pose information from an origin of a vehicle coordinate system. Method according to one of Claims 1 to 6 wherein the detected accelerations are corrected by means of driving state variables, the driving state variables comprising a vehicle speed, a steering angle, a road gradient and / or an accelerator pedal position. Method according to one of Claims 1 to 4 , wherein for adjusting (S5) the spatial position of the pose information, a reference position is taken into account, which is predetermined depending on a seat occupancy. Data glasses (3) for a display system (1) in a motor vehicle comprising: - One or more acceleration sensors (38) for detecting one or more accelerations in at least one lateral spatial direction and / or at least one rotational spatial direction; a control unit (37) for determining a pose information for a pose of the data goggles (3), wherein the pose specification indicates at least one spatial position and / or at least one spatial orientation of the data goggles (3) relative to a predefined vehicle coordinate system, wherein the control unit (37 ) is designed to For each detected acceleration, continuously determining a change in position as a function of the detected acceleration in the at least one lateral spatial direction and / or an alignment change in the at least one rotational spatial direction; ○ for each particular position change and / or orientation change, to update the spatial position and / or the spatial orientation of the pose indication of the data glasses (3) by integrating the determined position change and / or the determined orientation change; ○ to reset the spatial position and / or the spatial orientation of the pose information to a respective predetermined reference position and / or a reference orientation, if no position change and alignment change is detected for a certain period of time. Display system (1) with a data glasses (3) according to Claim 9 and with a driver assistance system (2) which is designed to transmit driving state variables to the data goggles, wherein the control unit (37) of the data goggles (3) is designed to hold the adjusted accelerations depending on the driving state variables.

Images