DE102016120366A1 - Mobile device and method for displaying georeferenced data - Google Patents

Abstract

The invention relates to a mobile unit and a method for operating a similar unit. The mobile unit comprises a display (101) having a display surface, a first means (102) for determining the current position of the mobile unit, a second means (103) for determining the current orientation of the display surface, wherein a viewing direction is defined relative to the display surface, an interface (104) on which a plurality of positions are provided georeferenced image data in each of which an environment is mapped around the respective positions; and third means (105) which determines, in the image data, the environment based on the image data for a dependent position and for the gaze direction image data type Position is shown in the viewing direction, where it is given, and the determined image data is displayed on the display.

Description

The invention relates to a mobile unit having a display with a display surface for optically outputting information. The invention further relates to: a method of operating such a mobile unit, a vehicle, a computer system, a digital storage medium, a computer program product, and a computer program and an application software (APP).

The object of the invention is to provide a mobile unit with a display that is designed and set up for an intuitive and easy to handle representation of image data in which location-dependent environments are mapped.

It is another object of the invention to provide a method for operating such a mobile unit, which allows an intuitive and easy to handle representation of image data in which location-dependent environments are mapped.

It is another object of the invention to provide a computer system, a digital storage medium, a computer program product and a computer program, and an APP, which specify just such a method.

The invention results from the features of the independent claims. Advantageous developments and refinements are the subject of the dependent claims. Other features, applications and advantages of the invention will become apparent from the following description, as well as the explanation of embodiments of the invention, which are illustrated in the figures.

der mobilen Einheit, ein zweites Mittel zur Ermittlung einer aktuellen Orientierung $\overrightarrow{O}$

der Displayfläche, wobei relativ zur Displayfläche eine Blickrichtung $\overrightarrow{R}\left(\overrightarrow{O}\right)$

definiert ist, eine Schnittstelle, an der für eine Vielzahl von Positionen ${\overrightarrow{p}}_i$

georeferenzierte Bilddaten $\text{BD}0\left({\overrightarrow{p}}_i\right)$

bereitgestellt werden, in denen jeweils eine Umgebung $\text{U}\left({\overrightarrow{p}}_i\right)$

um die jeweiligen Positionen ${\overrightarrow{p}}_i$

abgebildet ist, ein drittes Mittel, das auf Basis der Bilddaten $\text{BD}0\left({\overrightarrow{p}}_i\right)$

für eine von $\overrightarrow{P}$

abhängige Position ${\overrightarrow{P}}_{soll}={\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right)$

und für die Blickrichtung $\overrightarrow{R}\left(\overrightarrow{O}\right)$

Bilddaten $\text{BD}1\left({\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right),\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

derart ermittelt, dass in den Bilddaten $\text{BD}1\left({\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right),\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

die Umgebung $\text{U}\left({\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right)\right)$

der Position ${\overrightarrow{P}}_{soll}$

von ${\overrightarrow{P}}_{soll}$

aus in Blickrichtung $\overrightarrow{R}\left(\overrightarrow{O}\right)$

abgebildet ist, wobei die Funktion ${\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right)$

vorgegeben ist und wobei die ermittelten Bilddaten $\text{BD}1\left({\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right),\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

auf dem Display dargestellt werden.A first aspect of the invention relates to a mobile unit, comprising: a display having a display surface for optically outputting information, a first means for determining the current position $\overrightarrow{P}$

the mobile unit, a second means of determining a current orientation $\overrightarrow{O}$

the display surface, wherein relative to the display surface a viewing direction $\overrightarrow{R}\left(\overrightarrow{O}\right)$

is defined, an interface on which for a variety of positions ${\overrightarrow{p}}_i$

georeferenced image data $\text{BD}0\left({\overrightarrow{p}}_i\right)$

be provided, each containing an environment $\text{U}\left({\overrightarrow{p}}_i\right)$

around the respective positions ${\overrightarrow{p}}_i$

pictured is a third remedy based on the image data $\text{BD}0\left({\overrightarrow{p}}_i\right)$

for one of $\overrightarrow{P}$

dependent position ${\overrightarrow{P}}_{sOll}={\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)$

and for the line of sight $\overrightarrow{R}\left(\overrightarrow{O}\right)$

image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

determined such that in the image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

the environment $\text{U}\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)\right)$

the position ${\overrightarrow{P}}_{sOll}$

from ${\overrightarrow{P}}_{sOll}$

out in the direction $\overrightarrow{R}\left(\overrightarrow{O}\right)$

is shown, where the function ${\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)$

is predetermined and wherein the determined image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

be shown on the display.

The mobile unit is advantageously a smartphone, a tablet PC, a navigation display of a vehicle, a display of an infotainment system of a vehicle or a data glasses or VR glasses (VR = English "virtual reality"). The display is advantageous a color display (LCD display, LED display, plasma display, etc.). The display can be designed as a pure display unit or as a display and input unit (touch screen). The display surface is advantageously flat, i. not bent. The display can also have two display surfaces, such as in the case of data glasses, with a display surface is available for each eye. Advantageously, each of the display surfaces of the data glasses is driven in accordance with the above features, which describe only one display surface.

der mobilen Einheit umfasst vorteilhaft eine GPS-, Galileo-, GLONASS-Einheit, eine EGNOS-Einheit, eine Einheit zur Indoor-Navigation (bspw. Auf Basis vom W-LAN, UWB) und/oder eine Einheit zur Zellenortung in mobilen Telekommunikationsnetzen. Das erste Mittel ist vorteilhaft zur Indoor-Navigation ausgeführt. Weiterhin vorteilhaft ist das erste Mittel zur Ermittlung der aktuellen Position $\overrightarrow{P}$

via eines Datenzugriffs auf einen Server, der die aktuelle Position eines Fahrzeugs bereithält (bspw. „Bahn-Radar“ oder „Flightradar24“), in dem sich die mobile Einheit befindet, ausgeführt und eingerichtet.The first means of determining the current position $\overrightarrow{P}$

the mobile unit advantageously comprises a GPS, Galileo, GLONASS unit, an EGNOS unit, an indoor navigation unit (for example based on the W-LAN, UWB) and / or a unit for cell location in mobile telecommunication networks. The first means is advantageously designed for indoor navigation. Further advantageous is the first means for determining the current position $\overrightarrow{P}$

via a data access to a server which holds the current position of a vehicle (eg "rail radar" or "flight radar 24") in which the mobile unit is located, executed and set up.

der Displayfläche ist vorteilhaft ein in der mobilen Einheit integrierter Lagesensor. Derartige Lagesensoren sind im Stand der Technik hinreichend bekannt. Vorteilhaft umfasst der Lagesensor einen Magnetkompasssensor. Weiterhin vorteilhaft wird der Lagesensor durch eine Bildkorrelationseinheit unterstützt, die eine Bildkorrelation eines von einer Kamera der mobilen Einheit aufgenommenen Bildes mit über die Schnittstelle bereitgestellten Bilddaten $\text{BD}0\left({\overrightarrow{p}}_i\right)$

unterstützt. Die Bilddaten $\text{BD}0\left({\overrightarrow{p}}_i\right)$

sind jeweils von einer bestimmten Position unter definierten Winkeln aufgenommen worden. Durch Korrelation des von der mobilen Einheit aufgenommenen aktuellen Bildes mit den entsprechenden Bilddaten $\text{BD}0\left({\overrightarrow{p}}_i\right)$

kann unter Berücksichtigung der jeweiligen Positionen und der Blickwinkel die Orientierung $\overrightarrow{O}$

der Displayfläche bestimmt werden.The second means of determining the current orientation $\overrightarrow{O}$

the display surface is advantageously a position sensor integrated in the mobile unit. Such position sensors are well known in the art. Advantageously, the position sensor comprises a magnetic compass sensor. Furthermore, the position sensor is advantageously supported by an image correlation unit that performs an image correlation of an image recorded by a camera of the mobile unit with image data provided via the interface $\text{BD}0\left({\overrightarrow{p}}_i\right)$

supported. The image data $\text{BD}0\left({\overrightarrow{p}}_i\right)$

have each been taken from a certain position at defined angles. By correlating the current image captured by the mobile unit with the corresponding image data $\text{BD}0\left({\overrightarrow{p}}_i\right)$

can orientation taking into account the respective positions and the viewing angles $\overrightarrow{O}$

the display surface can be determined.

der Displayfläche ist erfindungsgemäß eine Blickrichtung $\overrightarrow{R}\left(\overrightarrow{O}\right)$

definiert. Vorteilhaft steht die Blickrichtung $\overrightarrow{R}\left(\overrightarrow{O}\right)$

senkrecht auf der Displayfläche. Die Blickrichtung $\overrightarrow{R}\left(\overrightarrow{O}\right)$

gibt dabei eine Strahlrichtung an, die vorteilhaft die Displayfläche mittig „durchstößt“ und vorteilhaft in (senkrechter) Aufsichtsrichtung auf die Displayfläche verläuft. Für besondere Anwendungen kann die Blickrichtung $\overrightarrow{R}\left(\overrightarrow{O}\right)$

entgegengesetzt zur Aufsichtsrichtung auf die Displayfläche definiert sein/werden. Vorteilhaft sind die Blickrichtung $\overrightarrow{R}\left(\overrightarrow{O}\right)$

und die Richtung einer senkrechten Aufsicht auf die Displayfläche identisch. Alternativ sind die Blickrichtung $\overrightarrow{R}\left(\overrightarrow{O}\right)$

und die Richtung einer senkrechten Aufsicht auf die Displayfläche entgegengesetzt. Unter „Displayfläche“ wird vorliegend die Fläche des Displays verstanden, bei der ein Nutzer bei Aufsicht die dargestellten Informationen erkennt.Relative to orientation $\overrightarrow{O}$

The display surface is according to the invention a viewing direction $\overrightarrow{R}\left(\overrightarrow{O}\right)$

Are defined. The viewing direction is advantageous $\overrightarrow{R}\left(\overrightarrow{O}\right)$

perpendicular to the display surface. The direction of view $\overrightarrow{R}\left(\overrightarrow{O}\right)$

indicates a beam direction, which advantageously "pierces" the display surface in the middle and advantageously extends in the (vertical) supervisory direction onto the display surface. For special applications, the viewing direction $\overrightarrow{R}\left(\overrightarrow{O}\right)$

be defined on the display surface opposite to the direction of supervision. The viewing direction is advantageous $\overrightarrow{R}\left(\overrightarrow{O}\right)$

and the direction of a vertical view of the display surface is identical. Alternatively, the viewing direction $\overrightarrow{R}\left(\overrightarrow{O}\right)$

and the direction of a vertical view of the display surface opposite. In the present case, the term "display surface" is understood to mean the surface of the display in which a user recognizes the displayed information when supervised.

In an advantageous embodiment, a crosshair is displayed on the display surface at the point at which the beam direction "pierces" the display surface. This allows a user to visualize and intuitively capture the current line of sight $\overrightarrow{R}\left(\overrightarrow{O}\right)$

zu, die beispielsweise in einem Datenspeicher der mobilen Einheit oder einem externen Datenspeicher (beispielsweise einer Daten-Cloud, einem Daten-Server) gespeichert sind. Der Zugriff auf einen externen Datenspeicher erfolgt vorteilhaft drahtungebunden, d.h. als Funksignal, beispielsweise via mobiles Internet und/oder wahlweise mit Offline mit gespeicherten Bilddaten sowie deren Online-/Offline-Synchronisation, um in Lokalitäten mit eingeschränktem Netz sowie hohen Roaming-Gebühren trotzdem Bilddaten eigenständig nutzen zu können.The interface accesses the georeferenced image data $\text{BD}0\left({\overrightarrow{p}}_i\right)$

for example, stored in a mobile unit data storage or external data storage (eg, a data cloud, a data server). The access to an external data storage is advantageously wirelessly, ie as a radio signal, for example via mobile Internet and / or optionally with offline image data stored and their online / offline synchronization, in localities with limited network and high roaming fees still image data independently to be able to use.

sind beispielsweise Bilddaten von Google StreetView oder Microsoft Bing Street Side, die insbesondere 360°-Ansichten einer Umgebung für eine Position ${\overrightarrow{p}}_i$

zeigen. Die über die Schnittstelle bereitgestellten Bilddaten $\text{BD}0\left({\overrightarrow{p}}_i\right)$

können insbesondere mittels optischer Sensoren, mittels Radarsensoren, mittels Lasersensoren und/oder mittels Ultraschall-Sensoren erfasste Daten sein.The image data $\text{BD}0\left({\overrightarrow{p}}_i\right)$

are, for example, image data from Google StreetView or Microsoft Bing Street Side, in particular 360 ° views of an environment for a position ${\overrightarrow{p}}_i$

demonstrate. The image data provided via the interface $\text{BD}0\left({\overrightarrow{p}}_i\right)$

may be in particular by means of optical sensors, by means of radar sensors, by means of laser sensors and / or by means of ultrasonic sensors detected data.

für eine Position ${\overrightarrow{p}}_i$

mehrere mit unterschiedlichen Sensoren aufgenommene Bilddaten der Umgebung von ${\overrightarrow{p}}_i.$

Advantageously, the image data includes $\text{BD}0\left({\overrightarrow{p}}_i\right)$

for a position ${\overrightarrow{p}}_i$

several image data taken with different sensors of the environment of ${\overrightarrow{p}}_i,$

für eine Position ${\overrightarrow{p}}_i$

mehrere zu unterschiedlichen Zeiten aufgenommene Bilddaten der Umgebung von ${\overrightarrow{p}}_i.$

So können sowohl aktuelle als auch historische Umgebungsansichten auf der mobilen Einheit angezeigt werden.Further advantageously, the image data includes $BD0\left({\overrightarrow{p}}_i\right)$

for a position ${\overrightarrow{p}}_i$

several image data taken at different times of the environment of ${\overrightarrow{p}}_i,$

This allows both current and historical environment views to be displayed on the mobile device.

auszuwählen.The mobile unit is advantageously set up and executed by an appropriate user input from different types of displayable image data $\text{BD}0\left({\overrightarrow{p}}_i\right)$

select.

Die mobile Einheit zeigt somit Bilder von der Umgebung der aktuellen Position der mobilen Einheit mit der Blickrichtung $\overrightarrow{R}\left(\overrightarrow{O}\right)$

auf dem Display an.In a particularly advantageous embodiment of the proposed mobile unit, the following applies: ${\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)=\overrightarrow{P},$

The mobile unit thus displays images of the surroundings of the current position of the mobile unit with the viewing direction $\overrightarrow{R}\left(\overrightarrow{O}\right)$

on the display.

definierbar ist, wobei gilt: ${\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right)=$

$\overrightarrow{P}+\overrightarrow{D}.$

Durch die Eingabe des Vektors $\overrightarrow{D}$

kann somit die Position ${\overrightarrow{P}}_{soll}$

relativ zur Position $\overrightarrow{P}$

der mobilen Einheit beliebig verschoben werden. Da der Vektor $\overrightarrow{D}$

konstant ist, ändert sich ${\overrightarrow{P}}_{soll}$

durch eine Änderung der Position $\overrightarrow{P}.$

A particularly advantageous embodiment of the proposed mobile unit has an input interface, by means of which a vector $\overrightarrow{D}$

is definable, where: ${\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)=$

$\overrightarrow{P}+\overrightarrow{D},$

By entering the vector $\overrightarrow{D}$

can thus position ${\overrightarrow{P}}_{sOll}$

relative to the position $\overrightarrow{P}$

the mobile unit can be moved anywhere. As the vector $\overrightarrow{D}$

is constant, changes ${\overrightarrow{P}}_{sOll}$

by changing the position $\overrightarrow{P},$

der Displayfläche der mobilen Einheit wird nun auf dem Display eine Umgebung bzw. ein Umgebungsausschnitt für die Position ${\overrightarrow{P}}_{soll}$

aus der Blickrichtung $\overrightarrow{R}\left(\overrightarrow{O}\right)$

dargestellt. Beispielsweise kann als Position ${\overrightarrow{P}}_{soll}$

eine Position gewählt werden, die auf der gegenüberliegenden Seite der Erdkugel liegt. Zur Unterstützung dieser Funktion sind vorteilhaft Geoid-Informationen in der mobilen Einheit hinterlegt, die dazu dienen, den entsprechenden Vektor $\overrightarrow{D}$

sinnvoll bzw. richtig zu wählen.Depending on the current orientation $\overrightarrow{O}$

The display surface of the mobile unit is now on the display an environment or an environmental section for the position ${\overrightarrow{P}}_{sOll}$

from the perspective $\overrightarrow{R}\left(\overrightarrow{O}\right)$

shown. For example, as a position ${\overrightarrow{P}}_{sOll}$

be chosen a position that lies on the opposite side of the globe. In order to support this function, geoid information is advantageously stored in the mobile unit, which serves the corresponding vector $\overrightarrow{D}$

make sense or choose correctly.

keine Bilddaten in $\text{BD}0\left({\overrightarrow{p}}_i\right)$

vorliegen, d.h.: ${\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right)\notin \left\{{\overrightarrow{p}}_i\right\}$

ist, ermittelt das dritte Mittel Bilddaten $\text{BD}1({\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right)={\overrightarrow{p}}_i{}^{*},\overrightarrow{R}\left(\overrightarrow{O}\right))$

auf Basis von Bilddaten $\text{BD}0\left({\overrightarrow{p}}_i{}^{*}\right),$

wobei ${\overrightarrow{p}}_i{}^{*}$

die zu ${\overrightarrow{P}}_{soll}$

nächstliegende Position aus der Menge $\left\{{\overrightarrow{p}}_i\right\}$

ist. Somit kann sichergestellt werden, dass entweder Bilddaten für die Position ${\overrightarrow{P}}_{soll}$

oder zumindest Bilddaten für die dazu nächstliegende Position ${\overrightarrow{p}}_i{}^{*}$

Basis für die Darstellung auf dem Display sind. Vorteilhaft wird die Differenz ${\overrightarrow{p}}_i{}^{*}-{\overrightarrow{P}}_{soll}$

oder ${\overrightarrow{P}}_{soll}-{\overrightarrow{p}}_i{}^{*}$

auf dem Display ausgegeben, sodass ein Nutzer über die verschobene Darstellung informiert ist.Unless for the position ${\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)$

no image data in $\text{BD}0\left({\overrightarrow{p}}_i\right)$

exist, ie: ${\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)\notin \left\{{\overrightarrow{p}}_i\right\}$

the third means determines image data $\text{BD}1({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)={\overrightarrow{p}}_i{}^{*}.\overrightarrow{R}\left(\overrightarrow{O}\right))$

based on image data $\text{BD}0\left({\overrightarrow{p}}_i{}^{*}\right).$

in which ${\overrightarrow{p}}_i{}^{*}$

the too ${\overrightarrow{P}}_{sOll}$

nearest position from the crowd $\left\{{\overrightarrow{p}}_i\right\}$

is. Thus, it can be ensured that either image data for the position ${\overrightarrow{P}}_{sOll}$

or at least image data for the next closest position ${\overrightarrow{p}}_i{}^{*}$

Basis for the representation on the display are. The difference becomes advantageous ${\overrightarrow{p}}_i{}^{*}-{\overrightarrow{P}}_{sOll}$

or ${\overrightarrow{P}}_{sOll}-{\overrightarrow{p}}_i{}^{*}$

displayed on the display so that a user is informed of the shifted representation.

so dass auch bei lückenhaften Bilddaten $\text{BD}0\left({\overrightarrow{p}}_i\right)$

stets die Umgebung dargestellt werden kann, die der Wunschposition am nächsten liegt.Among other things, this function allows an automatic correction of inputs of the vector $\overrightarrow{D}.$

so that even with incomplete image data $\text{BD}0\left({\overrightarrow{p}}_i\right)$

always the environment can be represented that is closest to the desired position.

die Umgebung $\text{U}\left({\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right)\right)$

aus der Blickrichtung $\overrightarrow{R}\left(\overrightarrow{O}\right)$

mit einem vorgebbaren Blickwinkel zeigen. Vorteilhaft ist die mobile Einheit dazu ausgeführt und eingerichtet den Blickwinkel variabel vorzugeben. Eine Veränderung des Blickwinkels ist vorteilhaft identisch mit einer Zoom-Funktion. Advantageously, the third means is designed and set up such that the image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

the environment $\text{U}\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)\right)$

from the perspective $\overrightarrow{R}\left(\overrightarrow{O}\right)$

show with a predefinable angle. Advantageously, the mobile unit is designed and furnished to set the viewing angle variable. A change in the viewing angle is advantageously identical to a zoom function.

für in den Bilddaten $\text{BD}0\left({\overrightarrow{p}}_i\right)$

abgebildete Objekte zugeordnete Attribute enthalten und die Attribute für die in den Bilddaten $\text{BD}1\left({\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right),\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

dargestellten Objekte auf dem Display dann angezeigt oder anderweitig ausgegeben werden, wenn die Blickrichtung $\overrightarrow{R}\left(\overrightarrow{O}\right)$

auf das jeweilige Objekt ausgerichtet ist. Unter „anderweitig ausgegeben“ wird vorliegend insbesondere die Ausgabe von akustischen Informationen verstanden. Vorteilhaft verfügt die mobile Einheit hierzu über einen elektroakustischen Wandler.A preferred embodiment of the proposed mobile unit is characterized in that the image data provided at the interface $\text{BD}0\left({\overrightarrow{p}}_i\right)$

for in the image data $\text{BD}0\left({\overrightarrow{p}}_i\right)$

mapped objects contain attributes associated with them and the attributes for those in the image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

displayed objects on the display can then be displayed or otherwise output when the line of sight $\overrightarrow{R}\left(\overrightarrow{O}\right)$

is aligned to the respective object. In the present case, the term "otherwise spent" is understood in particular to mean the output of acoustic information. Advantageously, the mobile unit has this for an electro-acoustic transducer.

The attributes advantageously include: text information, image information, acoustic information (music / speech), live streaming information, and / or video information. For example, the attributes may describe the nature of the object (eg, restaurant, hairdresser, museum, cinema, etc.) and include images, videos, menus, advertisements, or customer reviews deposited with the objects. Advantageously, the mobile unit includes an input interface for activating and controlling an output of attributes. Persons skilled in the art will be aware of many possible embodiments and applications of the mobile unit associated with the output of attributes, all of which are within the spirit of the invention.

The mobile unit is advantageously a smartphone or a tablet computer or a navigation system of a vehicle or an infotainment system of a vehicle.

der mobilen Einheit. In einem zweiten Schritt erfolgt ein Ermitteln der aktuellen Orientierung $\overrightarrow{O}$

der Displayfläche, wobei relativ zur Displayfläche eine Blickrichtung $\overrightarrow{R}\left(\overrightarrow{O}\right)$

definiert ist. In einem dritten Schritt erfolgt für eine Vielzahl von Positionen ${\overrightarrow{p}}_i$

ein Bereitstellen von georeferenzierten Bilddaten $\text{BD}0\left({\overrightarrow{p}}_i\right)$

in denen jeweils eine Umgebung $\text{U}\left({\overrightarrow{p}}_i\right)$

um die jeweiligen Positionen ${\overrightarrow{p}}_i$

abgebildet ist. In einem vierten Schritt erfolgt auf Basis der Bilddaten $\text{BD}0\left({\overrightarrow{p}}_i\right)$

für eine von $\overrightarrow{P}$

abhängige Position ${\overrightarrow{P}}_{soll}={\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right)$

und für die Blickrichtung $\overrightarrow{R}\left(\overrightarrow{O}\right)$

ein Ermitteln von Bilddaten $\text{BD}1\left({\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right),\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

derart, dass in den Bilddaten $\text{BD}1\left({\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right),\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

die Umgebung $\text{U}\left({\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right)\right)$

der Position ${\overrightarrow{P}}_{soll}$

von ${\overrightarrow{P}}_{soll}$

aus in Blickrichtung $\overrightarrow{R}\left(\overrightarrow{O}\right)$

abgebildet ist, wobei ${\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right)$

vorgegeben ist. In einem fünften Schritt erfolgt ein Darstellen der Bilddaten $\text{BD}1\left({\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right),\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

auf dem Display.Another aspect of the invention relates to a method of operating a mobile unit having a display with a display surface for optically outputting information. The proposed method comprises the following steps. In a first step, the current position is determined $\overrightarrow{P}$

the mobile unit. In a second step, the current orientation is determined $\overrightarrow{O}$

the display surface, wherein relative to the display surface a viewing direction $\overrightarrow{R}\left(\overrightarrow{O}\right)$

is defined. In a third step takes place for a variety of positions ${\overrightarrow{p}}_i$

providing georeferenced image data $\text{BD}0\left({\overrightarrow{p}}_i\right)$

in each case an environment $\text{U}\left({\overrightarrow{p}}_i\right)$

around the respective positions ${\overrightarrow{p}}_i$

is shown. In a fourth step, based on the image data $\text{BD}0\left({\overrightarrow{p}}_i\right)$

for one of $\overrightarrow{P}$

dependent position ${\overrightarrow{P}}_{sOll}={\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)$

and for the line of sight $\overrightarrow{R}\left(\overrightarrow{O}\right)$

a determination of image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

such that in the image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

the environment $\text{U}\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)\right)$

the position ${\overrightarrow{P}}_{sOll}$

from ${\overrightarrow{P}}_{sOll}$

out in the direction $\overrightarrow{R}\left(\overrightarrow{O}\right)$

is shown, where ${\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)$

is predetermined. In a fifth step, the image data is displayed $\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

on the display.

als senkrecht auf der Displayfläche stehend definiert.The viewing direction is advantageous $\overrightarrow{R}\left(\overrightarrow{O}\right)$

defined as standing vertically on the display surface.

oder ${\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right)=\overrightarrow{P}+\overrightarrow{D},$

wobei der Vektor $\overrightarrow{D}$

vorgegeben ist.Advantageously: ${\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)=\overrightarrow{P}$

or ${\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)=\overrightarrow{P}+\overrightarrow{D}.$

being the vector $\overrightarrow{D}$

is predetermined.

keine Bilddaten in $\text{BD}0\left({\overrightarrow{p}}_i\right)$

vorliegen, d.h.: ${\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right)\notin \left\{{\overrightarrow{p}}_i\right\}$

ist, Bilddaten $\text{BD}1\left({\overrightarrow{P}}_{soll}^{*}\left(\overrightarrow{P}\right)={\overrightarrow{p}}_i^{*},\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

auf Basis von Bilddaten $\text{BD}0\left({\overrightarrow{p}}_i^{*}\right)$

ermittelt werden, wobei ${\overrightarrow{p}}_i^{*}$

die zu ${\overrightarrow{P}}_{soll}$

nächstliegende Position aus der Menge $\left\{{\overrightarrow{p}}_i\right\}$

ist. Vorteilhaft wird die Differenz ${\overrightarrow{p}}_i^{*}-{\overrightarrow{P}}_{soll}$

oder ${\overrightarrow{P}}_{soll}-{\overrightarrow{p}}_i^{*}$

auf dem Display ausgegeben.A development of the proposed method is characterized by the fact that provided for the position ${\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)$

no image data in $\text{BD}0\left({\overrightarrow{p}}_i\right)$

exist, ie: ${\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)\notin \left\{{\overrightarrow{p}}_i\right\}$

is, image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}^{*}\left(\overrightarrow{P}\right)={\overrightarrow{p}}_i^{*}.\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

based on image data $\text{BD}0\left({\overrightarrow{p}}_i^{*}\right)$

be determined, where ${\overrightarrow{p}}_i^{*}$

the too ${\overrightarrow{P}}_{sOll}$

nearest position from the crowd $\left\{{\overrightarrow{p}}_i\right\}$

is. The difference becomes advantageous ${\overrightarrow{p}}_i^{*}-{\overrightarrow{P}}_{sOll}$

or ${\overrightarrow{P}}_{sOll}-{\overrightarrow{p}}_i^{*}$

on the display.

eine Vielzahl positionsabhängiger 360°-Bilddaten. In einer vorteilhaften Verfahrensvariante sind die Blickrichtung $\overrightarrow{R}\left(\overrightarrow{O}\right)$

und die Richtung einer senkrechten Aufsicht auf die Displayfläche identisch. In einer alternativen vorteilhaften Verfahrensvariante sind die Blickrichtung $\overrightarrow{R}\left(\overrightarrow{O}\right)$

und die Richtung einer senkrechten Aufsicht auf die Displayfläche entgegengesetzt. Vorteilhaft zeigen die Bilddaten $\text{BD}1\left({\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right),\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

die Umgebung $\text{U}\left({\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right)\right)$

aus der Blickrichtung $\overrightarrow{R}\left(\overrightarrow{O}\right)$

mit einem variabel vorgebbaren Blickwinkel.Advantageously, the image data includes $\text{BD}0\left({\overrightarrow{p}}_i\right)$

a variety of position-dependent 360 ° image data. In an advantageous variant of the method, the viewing direction $\overrightarrow{R}\left(\overrightarrow{O}\right)$

and the direction of a vertical view of the display surface is identical. In an alternative advantageous variant of the method, the viewing direction $\overrightarrow{R}\left(\overrightarrow{O}\right)$

and the direction of a vertical view of the display surface opposite. Advantageously show the image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

the environment $\text{U}\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)\right)$

from the perspective $\overrightarrow{R}\left(\overrightarrow{O}\right)$

with a variably definable angle.

für in den Bilddaten $\text{BD0}\left({\overrightarrow{p}}_i\right)$

abgebildete Objekte zugeordnete Attribute, wobei die Attribute für die in den Bilddaten $\text{BD}1\left({\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right),\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

dargestellten Objekte auf dem Display dann angezeigt werden, wenn die Blickrichtung $\overrightarrow{R}\left(\overrightarrow{O}\right)$

auf das jeweilige Objekt ausgerichtet ist.In a particularly preferred variant of the proposed method, the image data provided contain $\text{BD}0\left({\overrightarrow{p}}_i\right)$

for in the image data $\text{BD0}\left({\overrightarrow{p}}_i\right)$

Attributes mapped to mapped objects, with the attributes for those in the image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

displayed objects on the display then appear when the line of sight $\overrightarrow{R}\left(\overrightarrow{O}\right)$

is aligned to the respective object.

(der auch = 0 sein kann) lediglich seine mobile Einheit in eine bestimmte Lage (gleich Position und Orientierung) bringen muss, wobei die dadurch bestimmten Bilddaten $\text{BD}1\left({\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right),\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

auf dem Display angezeigt werden.The proposed method advantageously runs completely automatically, so that a user after activation of the method and, where appropriate, after specification of the vector $\overrightarrow{D}$

(which can also be = 0) only has to bring its mobile unit into a certain position (same position and orientation), whereby the image data determined thereby $\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

on the display.

Another aspect of the present invention relates to a vehicle, in particular motor vehicle, rail vehicle, watercraft, aircraft with a mobile unit as described above. The display of the mobile unit is advantageously identical to the display of a vehicle navigation system or an infotainment system of the vehicle.

Another aspect of the present invention relates to a computer system having a data processing device, wherein the data processing device is configured such that a method according to one of the preceding claims is performed on the data processing device.

Another aspect of the present invention relates to a digital storage medium having electronically readable control signals, wherein the control signals may interact with a programmable computer system such that a method according to any one of the preceding claims is performed.

Another aspect of the present invention relates to a computer program product having program code stored on a machine-readable medium for carrying out the method according to one of the preceding claims, when the program code is executed on a data processing device.

A further aspect of the present invention relates to a computer program or APP with program codes for carrying out the method according to one of the preceding claims, when the program runs on a data processing device. For this, the data processing device can be configured as any known from the prior art computer system.

Some examples of the use of the present invention are given below. It is assumed that the mobile unit for the examples given is a smartphone.

With an APP or an application running online in a web browser, which implements the method according to the invention, the user only has to hold his smartphone in front of him in order to automatically display the appropriate image information on the screen. Thus, it is possible in dark or poor visibility, such as in fog, comfortable to explore their own environment. This supports your own orientation and helps to find a specific location or details, such as an unlit entry-level bag in one place. The image data provided via the interface are automatically displayed in the desired view (depending on the orientation of the display area). For this purpose, the mobile unit is held in the direction with a desired orientation in which an image view is desired. The mobile unit can be pivoted in a variety of directions elevation and azimuth similar to a use of binoculars to probe the environment. Further advantageous is a zoom factor (angle of view) and various other options, such as the insertion of labels of objects displayed on the display possible.

des überflogenen Gebietes angezeigt. Der Nutzer kann dabei den Zoom-Faktor wählen und ein Übersichtsbild, bspw. von einer Satellitenaufnahme, ein höher aufgelöstes Luftbild oder ein Street View Bild von einer Straße angezeigt bekommen. Die mobile Einheit arbeitet daher ähnlich wie ein Fernglas mit Fadenkreuz, wobei sie vorzugsweise auf die in einer Cloud verfügbaren Datenbanken zugreift.If, for example, the smartphone is held against an aircraft window when it is cloudy, or if it is held in the direction of the underside of the aircraft, a picture will be taken $\left(\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)\right)$

of the overflown area. The user can select the zoom factor and get an overview image, for example from a satellite image, a higher-resolution aerial image or a Street View image of a street. The mobile unit therefore operates in a manner similar to reticule-type binoculars, preferably accessing databases available in a cloud.

entsprechend vorgegeben oder automatisch entsprechend geändert, wie vorstehend beschrieben. Damit ist es möglich, auch hinter Gebäude zu blicken bzw. dahinterliegende Straßen zu betrachten. Richtet man das Smartphone mit seiner definierten Blickrichtung auf ein Restaurant oder ein Geschäft, können einem beispielsweise die vom Betreiber hinterlegten Bilder, Speisekarten, Werbung oder Kundenbewertungen angezeigt werden.If a user is standing on a street with a mobile unit, it is still possible to click through different levels of distance and in this way also display environment images from a somewhat more distant position, for example a parallel street etc. This is the vector $\overrightarrow{D}$

preset or automatically changed accordingly, as described above. This makes it possible to look behind buildings or to look behind the streets. If one directs the smartphone with its defined viewing direction to a restaurant or a shop, one can, for example, display the pictures, menus, advertisements or customer reviews deposited by the operator.

If a user of the smartphone is in a vehicle or a train and if the user wishes to be shown images of the current environment in a desired direction, this is likewise possible with a mobile unit according to the invention.

The proposed APP according to the invention is advantageously used whenever the view of the local environment is restricted (for example at night, fog or rain) or when there are objects between the viewer and the surrounding scene to be displayed which block the view.

For example, if the user of the proposed mobile unit is traveling by car and is looking for a particular building known to him in bad visibility, the APP may be used to automatically and continuously display, for example, StreetView images of the road taken under optimal viewing conditions ,

This function can also be advantageously activated for a navigation display of a vehicle. In this case, the mobile unit is installed in the vehicle and uses the navigation display as a display. The user can advantageously choose from which side of the road and at which elevation angle the display of the StreetView images (= image data) should appear on the navigation display of the vehicle. When passing by, for example, an image of the selected environmental section is displayed every 2 seconds. The image data for generating the displayed images advantageously come from an archive and reflect the view in optimal lighting conditions.

Advantageously, the display of the mobile unit is a data glasses, which has two display surfaces, one for the left eye, another for the right eye. Since the display surfaces are firmly connected to the data glasses, the orientations of the two display surfaces can be easily derived from a known orientation of the data glasses. The environmental data displayed on the display surfaces preferably show images of an environment under good to very good viewing conditions.

The proposed mobile unit further enables the representation of environments within buildings, tunnels, etc. It is assumed that corresponding image contents are provided via the interface over environments within enclosed spaces / buildings. Furthermore, it is assumed that the current position P of the mobile unit by the first means is possible by a corresponding indoor navigation. Corresponding indoor navigation methods are known in the art. In emergency operations, for example, the police or the fire brigade no longer have to preview the building plans in advance, but can see the building in all directions with the APP on site, provided appropriate footage is provided at the interface.

The proposed APP also allows for intuitive navigation in hardware stores, grocery stores, etc. by displaying the environment view dependent on the current position P of the mobile unit on the smartphone, by fading in the attributes or the presence of a search feature for particular items in the hardware store certain way can be shown.

For example, the proposed APP allows a user traveling with their smartphone in the subway to display environmental images and / or mezzanines and network data of various trades, e.g. Gas or district heating network.

in einen Zwischenspeicher der mobilen Einheit geladen. Dies reduziert die Latenzzeiten und ermöglicht ein Koppeln der auf dem Display gezeigten Szene (Bildansicht) mit der aktuellen Blickrichtung in Echtzeit.Advantageously, the mobile unit predicts a further movement of the mobile unit, that is to say it predicts a future location trajectory. In order to distribute or minimize the data traffic via the interface as far as possible, corresponding predefined image data are predicted on the basis of the predicted location trajectory $\text{BD}0*\left({\overrightarrow{p}}_i\right)$

loaded into a cache of the mobile unit. This reduces the latency and allows coupling the scene shown on the display (image view) with the current viewing direction in real time.

mit dem Kamerabild zur Deckung zu bringen. Da die den Bilddaten $\text{BD}1\left({\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right),\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

zugrunde liegenden Bilddaten $\text{BD}0\left({\overrightarrow{p}}_i\right)$

typischerweise von diskreten Positionen aufgenommen wurden, die nicht mit der aktuellen Position der mobilen Einheit übereinstimmen müssen, kann eine Umrechnung der Bilder auf die aktuelle perspektivische Ansicht nötig sein. Eine gute Passung zwischen Kamerabild und den ermittelten Bilddaten $\text{BD}1\left({\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right),\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

ermöglicht eine automatische Erkennung von Bildunterschieden (englisch „change detetion“). Mit dieser Funktion können Änderungen in der Bebauung, des Straßeninventars oder der Vegetation automatisch erkannt und bildlich hervorgehoben werden. In Räumen können so auch automatisch fehlende Gegenstände erkannt und bildlich hervorgehoben werden.If the mobile unit has its own camera, it may be advantageous for some applications to obtain the image data determined $\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

to bring to coincidence with the camera image. Because the image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

underlying image data $\text{BD}0\left({\overrightarrow{p}}_i\right)$

typically taken from discrete positions that do not have to match the current position of the mobile unit, it may be necessary to convert the images to the current perspective view. A good fit between camera image and the determined image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

allows automatic detection of image differences (English "change detetion"). With this function, changes in the development, the road inventory or the vegetation can be automatically detected and highlighted. In rooms so also missing missing objects can be recognized and highlighted.

und das Smartphone hebt diese Objekte in der Darstellung vorteilhaft auf dem Display optisch hervor. Vorteilhaft werden zusätzlich Attribut-Informationen (Bilder, Daten, Icons etc.) zu den jeweiligen Objekten je nach Einstellung ausgegeben. Durch ein auf dem Display eingeblendetes Fadenkreuz kann das Smartphone nun derart in seiner Lage verändert werden, dass die Mitte des Fadenkreuzes auf ein bestimmtes in den dargestellten Bilddaten $\text{BD}1\left({\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right),\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

abgebildetes Objekt zeigt. Dan können gezielt vorhandene Attribut-Informationen zu diesem Objekt ausgegeben werden. Optische Attribut-Informationen werden auf dem Display akustische Attribut-Informationen werden durch einen im Smartphone vorhandenen Lautsprecher (elektroakustischen Wandler) ausgegeben. Hält man das Smartphone beispielsweise auf ein Fußballstadion, so würde man bspw. eine aktuelle Fernsehübertragung oder auf Wunsch auch historische Aufnahmen angezeigt bekommen.A user of a smartphone often knows only vaguely, where in his environment interesting objects are. In order to get these automatically displayed, it makes sense to first make a pre-selection for the desired topic via a corresponding entry in the smartphone, for example restaurants, tourist attractions, historical sites, architectural monuments, movie locations, webcams, videos of sports broadcasts etc. The user then sets the smartphone to wide-angle and the smartphone automatically detects corresponding objects in the displayed image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

and the smartphone visually highlights these objects in the display. Advantageously, additional attribute information (images, data, icons, etc.) are output to the respective objects depending on the setting. By means of a crosshair displayed on the display, the smartphone can now be changed in its position in such a way that the center of the crosshair points to a specific one in the displayed image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

shown object shows. It is possible to output specific attribute information for this object. Optical attribute information is displayed on the display. Acoustic attribute information is output through a speaker (electroacoustic transducer) provided in the smartphone. If you hold the smartphone, for example, on a football stadium, so you would, for example, get a current television broadcast or on request also historical recordings.

Further advantages, features and details will become apparent from the following description in which - where appropriate, with reference to the drawings - at least one embodiment is described in detail. The same, similar and / or functionally identical parts are provided with the same reference numerals.

• 1 einen schematisierten Aufbau einer erfindungsgemäßen mobilen Einheit,
• 2 ein Ablaufschema eines erfindungsgemäßen Verfahrens.

Show it:

• 1 a schematic structure of a mobile unit according to the invention,
• 2 a flow chart of a method according to the invention.

der mobilen Einheit, vorliegend eine GPS-Einheit; und ein zweites Mittel 103 zur Ermittlung der aktuellen Orientierung $\overrightarrow{O}$

der Displayfläche, vorliegend einen drei Achsenlagesensor, wobei relativ zur Displayfläche eine Blickrichtung $\overrightarrow{R}\left(\overrightarrow{O}\right)$

definiert ist. 1 shows a schematic structure of a mobile unit according to the invention, in this case a smartphone, comprising: a display 101 with a display surface for optically outputting information; a first remedy 102 to determine the current position $\overrightarrow{P}$

the mobile unit, in this case a GPS unit; and a second means 103 to determine the current orientation $\overrightarrow{O}$

the display surface, in this case a three-axis position sensor, wherein a viewing direction relative to the display surface $\overrightarrow{R}\left(\overrightarrow{O}\right)$

is defined.

verläuft vorliegend in Richtung einer senkrechten Aufsicht auf die Displayfläche. Der die Blickrichtung $\overrightarrow{R}\left(\overrightarrow{O}\right)$

repräsentierende Strahl steht somit senkrecht auf der Displayfläche, „durchstößt“ die Displayfläche mittig und verlässt die Rückseite der Displayfläche.The direction of view $\overrightarrow{R}\left(\overrightarrow{O}\right)$

in this case runs in the direction of a vertical plan view of the display surface. The direction of view $\overrightarrow{R}\left(\overrightarrow{O}\right)$

thus representing beam is perpendicular to the display surface, "pierces" the display area in the middle and leaves the back of the display surface.

georeferenzierte Bilddaten $\text{BD}0\left({\overrightarrow{p}}_i\right)$

bereitgestellt werden, in denen jeweils eine Umgebung $\text{U}\left({\overrightarrow{p}}_i\right)$

um die jeweiligen Positionen ${\overrightarrow{p}}_i$

abgebildet ist. Vorliegend sind dies sogenannte StreetView Bilddaten, in denen eine 360° Umgebungsaufnahme für alle 2 m eines Straßenverkehrswegenetzes vorliegt. Weiterhin umfassen die an der Schnittstelle bereitgestellten Bilddaten $\text{BD}0\left({\overrightarrow{p}}_i\right)$

Bilddaten, in denen eine 360° Umgebungsaufnahme für alle 2 m oder 5 m oder 10 m voneinander beabstandeten Positionen innerhalb öffentlich zugänglicher Gebäude enthalten sind. Weiterhin umfassen die an der Schnittstelle bereit gestellten Bilddaten $\text{BD}0\left({\overrightarrow{p}}_i\right)$

Satellitenaufnahmen der Erdoberfläche sowie hochaufgelöste Luftbildaufnahmen der Erdoberfläche.The mobile unit further comprises a data interface 104 , for a variety of positions ${\overrightarrow{p}}_i$

georeferenced image data $\text{BD}0\left({\overrightarrow{p}}_i\right)$

be provided, each containing an environment $\text{U}\left({\overrightarrow{p}}_i\right)$

around the respective positions ${\overrightarrow{p}}_i$

is shown. In the present case, these are so-called StreetView image data in which a 360 ° environmental image is available for every 2 m of a road network. Furthermore, the image data provided at the interface include $\text{BD}0\left({\overrightarrow{p}}_i\right)$

Image data that contains a 360 ° environmental image for all 2 m or 5 m or 10 m spaced locations within publicly accessible buildings. Furthermore, the image data provided at the interface include $\text{BD}0\left({\overrightarrow{p}}_i\right)$

Satellite images of the earth's surface and high-resolution aerial photographs of the earth's surface.

Attribute für die in den jeweiligen Bilddaten dargestellten Objekte. Die Attribute geben vorliegend die Namen und Arten der Objekte an. Beispielsweise: „Chinesisches Restaurant „Goldene Ente““.Finally, the image data provided at the interface includes $\text{BD}0\left({\overrightarrow{p}}_i\right)$

Attributes for the objects displayed in the respective image data. In the present case, the attributes indicate the names and types of the objects. For example: "Chinese restaurant" Golden duck "".

für eine von $\overrightarrow{P}$

abhängige Position ${\overrightarrow{P}}_{soll}={\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right)$

und für die Blickrichtung $\overrightarrow{R}\left(\overrightarrow{O}\right)$

Bilddaten $\text{BD}1\left({\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right),\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

derart ermittelt, dass in den Bilddaten $\text{BD}1\left({\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right),\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

die Umgebung $\text{U}\left({\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right)\right)$

der Position ${\overrightarrow{P}}_{soll}$

von ${\overrightarrow{P}}_{soll}$

aus in Blickrichtung $\overrightarrow{R}\left(\overrightarrow{O}\right)$

abgebildet ist, wobei ${\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right)$

vorgegeben ist.The mobile unit also includes a third agent 105 based on the image data $\text{BD}0\left({\overrightarrow{p}}_i\right)$

for one of $\overrightarrow{P}$

dependent position ${\overrightarrow{P}}_{sOll}={\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)$

and for the line of sight $\overrightarrow{R}\left(\overrightarrow{O}\right)$

image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

determined such that in the image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

the environment $\text{U}\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)\right)$

the position ${\overrightarrow{P}}_{sOll}$

from ${\overrightarrow{P}}_{sOll}$

out in the direction $\overrightarrow{R}\left(\overrightarrow{O}\right)$

is shown, where ${\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)$

is predetermined.

definierbar ist, wobei gilt: ${\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right)=\overrightarrow{P}$

$+\overrightarrow{D}.$

Die Eingabeschnittstelle ist mit der Datenschnittstelle verbunden und prüft, ob für die Position ${\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right)$

Bilddaten $\text{BD}0\left({\overrightarrow{p}}_i={\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right)\right)$

verfügbar sind. Ist dies nicht der Fall, d.h.: ${\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right)\notin \left\{{\overrightarrow{p}}_i\right\},$

so wird an der Eingabeschnittstelle der Vektor $\overrightarrow{D}$

automatisch derart in einen Vektor ${\overrightarrow{D}}^{\ast }$

geändert, dass Bilddaten $\text{BD}0\left({\overrightarrow{p}}_i^{*}\right)$

ermittelt werden, wobei ${\overrightarrow{p}}_i^{*}$

die zu ${\overrightarrow{P}}_{soll}$

nächstliegende Position aus der Menge $\left\{{\overrightarrow{p}}_i\right\}$

ist. Schließlich werden die ermittelten Bilddaten $\text{BD}1\left({\overrightarrow{P}}_{soll}{}^{*}\left(\overrightarrow{P}\right)={\overrightarrow{p}}_i^{*},\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

auf dem Display dargestellt. An der Eingabeschnittstelle wird der geänderte Vektor ${\overrightarrow{D}}^{\ast }$

angezeigt.The mobile unit further comprises an input interface (in the present case the touch screen of the smartphone), by means of which a vector $\overrightarrow{D}$

is definable, where: ${\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)=\overrightarrow{P}$

$+\overrightarrow{D},$

The input interface is connected to the data interface and checks for position ${\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)$

image data $\text{BD}0\left({\overrightarrow{p}}_i={\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)\right)$

Are available. If this is not the case, ie: ${\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)\notin \left\{{\overrightarrow{p}}_i\right\}.$

so the vector will be at the input interface $\overrightarrow{D}$

automatically into a vector like this ${\overrightarrow{D}}^{*}$

changed that image data $\text{BD}0\left({\overrightarrow{p}}_i^{*}\right)$

be determined, where ${\overrightarrow{p}}_i^{*}$

the too ${\overrightarrow{P}}_{sOll}$

nearest position from the crowd $\left\{{\overrightarrow{p}}_i\right\}$

is. Finally, the determined image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}{}^{*}\left(\overrightarrow{P}\right)={\overrightarrow{p}}_i^{*}.\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

shown on the display. At the input interface becomes the changed vector ${\overrightarrow{D}}^{*}$

displayed.

der mobilen Einheit, Ermitteln 202 der aktuellen Orientierung $\overrightarrow{O}$

der Displayfläche, wobei relativ zur Displayfläche eine Blickrichtung $\overrightarrow{R}\left(\overrightarrow{O}\right)$

definiert ist, für eine Vielzahl von Positionen ${\overrightarrow{p}}_i,$

Bereitstellen 203 von georeferenzierte Bilddaten $\text{BD}0\left({\overrightarrow{p}}_i\right),$

in denen jeweils eine Umgebung $\text{U}\left({\overrightarrow{p}}_i\right)$

um die jeweiligen Positionen ${\overrightarrow{p}}_i$

abgebildet ist, auf Basis der Bilddaten $\text{BD}0\left({\overrightarrow{p}}_i\right)$

für eine von $\overrightarrow{P}$

abhängige Position ${\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right)=\overrightarrow{P}$

und für die Blickrichtung $\overrightarrow{R}\left(\overrightarrow{O}\right),$

Ermitteln 204 von Bilddaten $\text{BD}1\left({\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right),\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

derart, dass in den Bilddaten $\text{BD}1\left({\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right),\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

die Umgebung $\text{U}\left({\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right)\right)$

der Position ${\overrightarrow{P}}_{soll}$

von ${\overrightarrow{P}}_{soll}$

aus in Blickrichtung $\overrightarrow{R}\left(\overrightarrow{O}\right)$

abgebildet ist, wobei ${\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right)$

vorgegeben ist, und Darstellen 205 der Bilddaten $\text{BD}1\left({\overrightarrow{P}}_{soll}\left(\overrightarrow{P}\right),\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

auf dem Display. 2 1 shows a flowchart of a method according to the invention for operating a mobile unit, which has a display with a display surface for optically outputting information, with the following steps: determining 201 the current position $\overrightarrow{P}$

the mobile unit, determining 202 the current orientation $\overrightarrow{O}$

the display surface, wherein relative to the display surface a viewing direction $\overrightarrow{R}\left(\overrightarrow{O}\right)$

is defined for a variety of positions ${\overrightarrow{p}}_i.$

Provide 203 of georeferenced image data $\text{BD}0\left({\overrightarrow{p}}_i\right).$

in each case an environment $\text{U}\left({\overrightarrow{p}}_i\right)$

around the respective positions ${\overrightarrow{p}}_i$

is shown based on the image data $\text{BD}0\left({\overrightarrow{p}}_i\right)$

for one of $\overrightarrow{P}$

dependent position ${\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)=\overrightarrow{P}$

and for the line of sight $\overrightarrow{R}\left(\overrightarrow{O}\right).$

Determining 204 of image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

such that in the image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

the environment $\text{U}\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)\right)$

the position ${\overrightarrow{P}}_{sOll}$

from ${\overrightarrow{P}}_{sOll}$

out in the direction $\overrightarrow{R}\left(\overrightarrow{O}\right)$

is shown, where ${\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)$

and representing 205 the image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

on the display.

Although the invention has been further illustrated and explained in detail by way of preferred embodiments, the invention is not limited by the disclosed examples, and other variations can be derived therefrom by those skilled in the art without departing from the scope of the invention. It is therefore clear that a multitude of possible variations exists. It is also to be understood that exemplified embodiments are really only examples that are not to be construed in any way as limiting the scope, applicability, or configuration of the invention. Rather, the foregoing description and description of the figures enable one skilled in the art to practice the exemplary embodiments, and those of skill in the knowledge of the disclosed inventive concept may make various changes, for example, to the operation or arrangement of particular elements recited in an exemplary embodiment, without Protection area defined by the claims and their legal equivalents, such as further explanations in the description.

LIST OF REFERENCE NUMBERS

101

display

102

first means

103

second means

104

interface

105

third means

201-205

steps

Claims (10)

Mobile unit, comprising: - a display (101) having a display surface for optically outputting information, - a first means (102) for determining the current position $\overrightarrow{P}$

the mobile unit, - a second means (103) for determining the current orientation $\overrightarrow{O}$

the display surface, wherein relative to the display surface a viewing direction $\overrightarrow{R}\left(\overrightarrow{O}\right)$

is defined, - an interface (104), at the for a variety of positions ${\overrightarrow{p}}_i$

georeferenced image data $\text{BD}0\left({\overrightarrow{p}}_i\right)$

be provided, each containing an environment $\text{U}\left({\overrightarrow{p}}_i\right)$

around the respective positions ${\overrightarrow{p}}_i$

and a third means (105) based on the image data $\text{BD}0\left({\overrightarrow{p}}_i\right)$

for one of $\overrightarrow{P}$

dependent position ${\overrightarrow{P}}_{sOll}={\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)$

and for the line of sight $\overrightarrow{R}\left(\overrightarrow{O}\right)$

image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

determined such that in the image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

the environment $\text{U}\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)\right)$

the position ${\overrightarrow{P}}_{sOll}$

from ${\overrightarrow{P}}_{sOll}$

out in the direction $\overrightarrow{R}\left(\overrightarrow{O}\right)$

is shown, where ${\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)$

is predetermined and wherein the determined image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

be shown on the display. Mobile unit after Claim 1 in which: ${\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)=\overrightarrow{P},$

Mobile unit after Claim 1 or 2 , further comprising an input interface by means of a vector $\overrightarrow{D}$

is definable, where: ${\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)=\overrightarrow{P}+\overrightarrow{D},$

Mobile unit after one of the Claims 1 to 3 in which the third means (105), if for the position ${\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)$

no image data in $\text{BD}0\left({\overrightarrow{p}}_i\right)$

exist, ie: ${\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)\notin \left\{{\overrightarrow{p}}_i\right\}.$

the image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}^{*}\left(\overrightarrow{P}\right)={\overrightarrow{p}}_i^{*}.\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

based on image data $\text{BD}0\left({\overrightarrow{p}}_i^{*}\right)$

determined, where ${\overrightarrow{p}}_i^{*}$

the too ${\overrightarrow{P}}_{sOll}$

nearest position from the crowd $\left\{{\overrightarrow{p}}_i\right\}$

is. Mobile unit after one of the Claims 1 to 4 in which the image data provided at the interface $\text{BD}0\left({\overrightarrow{p}}_i\right)$

for in the image data $\text{BD}0\left({\overrightarrow{p}}_i\right)$

mapped objects contain attributes associated with them and the attributes for those in the image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

displayed objects are output when the line of sight $\overrightarrow{R}\left(\overrightarrow{O}\right)$

is aligned to the respective object. A method of operating a mobile unit having a display with a display surface for optically outputting information, comprising the steps of: - determining (201) the current position $\overrightarrow{P}$

the mobile unit, - determining (202) the current orientation $\overrightarrow{O}$

the display surface, wherein relative to the display surface a viewing direction $\overrightarrow{R}\left(\overrightarrow{O}\right)$

is defined - for a variety of positions ${\overrightarrow{p}}_i$

Providing (203) georeferenced image data $\text{BD}0\left({\overrightarrow{p}}_i\right).$

in each case an environment $\text{U}\left({\overrightarrow{p}}_i\right)$

around the respective positions ${\overrightarrow{p}}_i$

is shown, - based on the image data $\text{BD}0\left({\overrightarrow{p}}_i\right)$

for one of $\overrightarrow{P}$

dependent position ${\overrightarrow{P}}_{sOll}=$

${\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)$

and for the line of sight $\overrightarrow{R}\left(\overrightarrow{O}\right)$

Determining (204) image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

such that in the image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

the environment $\text{U}\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)\right)$

the position ${\overrightarrow{P}}_{sOll}$

from ${\overrightarrow{P}}_{sOll}$

out in the direction $\overrightarrow{R}\left(\overrightarrow{O}\right)$

is shown, where ${\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)$

is predetermined, and - representing (205) the image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

on the display. Method according to Claim 6 in which a vector $\overrightarrow{D}$

can be specified, where: ${\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)=\overrightarrow{P}+\overrightarrow{D}.$

and provided for the position ${\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)$

no image data in $\text{BD}0\left({\overrightarrow{p}}_i\right)$

exist, ie: ${\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right)\notin \left\{{\overrightarrow{p}}_i\right\}.$

the image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}^{*}\left(\overrightarrow{P}\right)={\overrightarrow{p}}_i^{*}.\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

based on image data $\text{BD}0\left({\overrightarrow{p}}_i^{*}\right)$

be determined, where ${\overrightarrow{p}}_i^{*}$

the too ${\overrightarrow{P}}_{sOll}$

nearest position from the crowd $\left\{{\overrightarrow{p}}_i\right\}$

is. Method according to one of Claims 6 or 7 in which the provided image data $\text{BD}0\left({\overrightarrow{p}}_i\right)$

for in the image data $\text{BD}0\left({\overrightarrow{p}}_i\right)$

mapped objects contain attributes associated with them and the attributes for those in the image data $\text{BD}1\left({\overrightarrow{P}}_{sOll}\left(\overrightarrow{P}\right).\overrightarrow{R}\left(\overrightarrow{O}\right)\right)$

displayed objects on the display then appear when the line of sight $\overrightarrow{R}\left(\overrightarrow{O}\right)$

is aligned to the respective object. Vehicle, in particular motor vehicle, with a mobile unit according to one of Claims 1 to 5 , Computer program or APP with program codes for carrying out the method according to one of the preceding claims, when the program runs on a data processing device.

Images