EP3717954B1 - Method for displaying the course of a trajectory in front of a vehicle or an object by means of a display unit, and device for carrying out the method - Google Patents

Description

The proposal relates to the technical field of driver information systems, which are also known under the term infotainment system. In particular, this involves a method for displaying a trajectory in front of a vehicle or an object with additional information. Such systems are mainly used in vehicles. However, there is also the possibility of using the invention for pedestrians, cyclists, etc. with data glasses. The proposal also relates to a correspondingly designed device for carrying out the method, as well as a motor vehicle and a computer program.

Intensive work is currently being done on technologies that will later enable autonomous driving. A first approach is not to completely relieve the driver of his tasks, but to enable the driver to take control of the vehicle at any time. The driver also performs monitoring functions. Newer technologies in the field of driver information systems such as Head-Up Display (HUD) make it possible to better inform the driver about what is happening in the area around his vehicle.

For the near future, it can therefore be assumed that the use of newer technologies (vehicle-to-vehicle communication, use of databases, vehicle sensors, etc.) will make comprehensive information about objects (especially vehicles) in the immediate vicinity of one's own vehicle available on the system side will be. In the field of vehicle sensor technology, the following components are mentioned in particular, which enable surroundings to be monitored: RADAR devices corresponding to radio detection and ranging, LIDAR devices corresponding to light detection and ranging, mainly for the area of distance detection / warning and cameras with appropriate image processing for the area the object recognition. This data about the environment can thus be used as a basis for system-side driving recommendations, warnings, etc. For example, displays/warnings are conceivable in which direction (possibly in your own trajectory) another surrounding vehicle wants to turn.

Vehicle-to-vehicle communication is now also possible using mobile communication with systems such as LTE in accordance with Long Term Evolution. A specification called LTE V2X was adopted here by the 3GPP organization. As an alternative, systems based on WLAN technology are available for direct vehicle communication, in particular the WLAN p system.

The term "autonomous driving" is sometimes used differently in the literature.

• Level 0: "Driver only", der Fahrer fährt selbst, lenkt, gibt Gas, bremst etc.
• Level 1: Bestimmte Assistenzsysteme helfen bei der Fahrzeugbedienung (u.a. ein Abstandsregelsystem - Automatic Cruise Control ACC).
• Level 2: Teilautomatisierung. U.a. automatisches Einparken, Spurhaltefunktion, allgemeine Längsführung, Beschleunigen, Abbremsen etc. werden von den Assistenzsystemen übernommen (u.a. Stauassistent).
• Level 3: Hochautomatisierung. Der Fahrer muss das System nicht dauernd überwachen. Das Fahrzeug führt selbstständig Funktionen wie das Auslösen des Blinkers, Spurwechsel und Spurhalten durch. Der Fahrer kann sich anderen Dingen zuwenden, wird aber bei Bedarf innerhalb einer Vorwarnzeit vom System aufgefordert, die Führung zu übernehmen. Diese Form der Autonomie ist auf Autobahnen technisch machbar. Der Gesetzgeber arbeitet darauf hin, Level 3-Fahrzeuge zuzulassen. Die gesetzlichen Rahmenbedingungen wurden dafür bereits geschaffen.
• Level 4: Vollautomatisierung. Die Führung des Fahrzeugs wird dauerhaft vom System übernommen. Werden die Fahraufgaben vom System nicht mehr bewältigt, kann der Fahrer aufgefordert werden, die Führung zu übernehmen.
• Level 5: Kein Fahrer erforderlich. Außer dem Festlegen des Ziels und dem Starten des Systems ist kein menschliches Eingreifen erforderlich.

The following insert is therefore presented here to clarify this term. Autonomous driving (sometimes also called automatic driving, automated driving or piloted driving) refers to the movement of vehicles, mobile robots and driverless transport systems that behave largely autonomously. There are different levels of the term autonomous driving. At certain levels, autonomous driving is also referred to when there is still a driver in the vehicle who may only be responsible for monitoring the automatic driving process. In Europe, the various transport ministries (in Germany the Federal Highway Research Institute was involved) have worked together and defined the following levels of autonomy.

• Level 0: "Driver only", the driver drives himself, steers, accelerates, brakes etc.
• Level 1: Certain assistance systems help with vehicle operation (including a distance control system - Automatic Cruise Control ACC).
• Level 2: partial automation. Among other things, automatic parking, lane keeping function, general longitudinal guidance, acceleration, braking, etc. are taken over by the assistance systems (including traffic jam assistant).
• Level 3: High automation. The driver does not have to constantly monitor the system. The vehicle independently carries out functions such as triggering the turn signal, changing lanes and staying in lane. The driver can attend to other things, but if necessary the system will prompt the system to take the lead within a warning period. This form of autonomy is technically feasible on motorways. Legislators are working towards allowing Level 3 vehicles. The legal framework for this has already been created.
• Level 4: full automation. The vehicle is permanently controlled by the system. If the system can no longer handle the driving tasks, the driver can be asked to take control.
• Level 5: No driver required. No human intervention is required other than setting the target and starting the system.

Automated driving functions from level 3 relieve the driver of the responsibility for controlling the vehicle.

Due to the current development towards higher levels of autonomy, where many vehicles are still controlled by the driver, it can be assumed that corresponding additional information can already be used for manually operated vehicles in the medium term and not only in the long term for highly automated systems.

For the driver-vehicle interaction, the question arises as to how this information can be presented in such a way that real added value is created for the human driver and he can also locate the information provided quickly and intuitively. The following solutions in this area are already known from the prior art.

A vision of the future in the automotive industry is to be able to display virtual elements on the windscreen of one's own vehicle in order to give the driver a number of advantages. The so-called "Augmented Reality" technology (AR) is used. The corresponding German-language term "extended reality" is less common. The real environment is enriched with virtual elements. This has several advantages: There is no need to look down at displays other than the windscreen, since a lot of relevant information is shown on the windscreen. The driver does not have to take his eyes off the road. In addition, due to the precise location of the virtual elements in the real environment, less cognitive effort on the part of the driver is likely, since there is no need to interpret a graphic on a separate display. Added value can also be generated with regard to automatic driving.

Since the technological resources are correspondingly limited today, it can be assumed that in the medium term there will be no fully recordable windscreens in vehicles. This is why head-up displays are used in vehicles today. These displays are actually projection units that project an image onto the windshield. From the driver's point of view, however, this image is located a few meters to 15 meters in front of the vehicle, depending on the module design.

The "image" is composed as follows: it is less a virtual display and more a kind of "keyhole" into the virtual world. The virtual environment is theoretically placed over the real world and contains the virtual objects that support and inform the driver while driving. The limited display area of the HUD means that only a section of the real world can be shown with HUD overlays. So you look through the display area of the HUD at the section of the virtual world. Since this virtual environment supplements the real environment, this is also referred to as "mixed reality".

From the DE 10 2007 016 868 A1 is a method for displaying the course of a road in front of a vehicle, with a current speed of the vehicle being determined. Points of a vehicle environment in front of the vehicle are determined in such a way that an optical flow of these points is determined taking into account the current vehicle speed and that symbols for representing the optical flow for a representation of the course of the road are shown in the head-up display.

From the WO 2005/053991 A1 a method and system for supporting a path control method is known. The method and system is used to support path control, in particular of a vehicle on a road or in an off-road environment or a ship or an airplane. The method consists in carrying out at least one of the following steps (a) and (b): (a) Estimating an actual future path of the vehicle based on vehicle movement data and visual and/or acoustic and/or tactile display of the estimated actual future path to the driver, (b) detecting the actual current path of the vehicle, estimating a current deviation of the detected actual current path from a desired current path, and displaying optically and/or acoustically and/or tactilely the estimated current deviation to the driver.

From the DE 10 2011 075 887 A1 a head-up display is known in which the ambient brightness is detected and the light intensity of the emitted light is adjusted depending on the ambient brightness. The contrast ratio is also improved in the head-up display presented.

From the DE 10 2012 222 380 A1 a stereoscopic head-up display is known in which a photometric parameter of the display unit, such as the brightness and background color is determined from the surroundings of the vehicle, in particular only as a function of the position of the vehicle, a determined precipitation at the position of the vehicle, a determined ambient temperature at the position of the vehicle, time information and/or at least one value from an external data source.

From the DE 10 2016 200 136 A1 a method for operating a stereoscopic head-up display is known, the method comprising a step of changing in which image information intended for the right eye and/or image information intended for the left eye depending on a ratio or a difference between a Crosstalk value representing crosstalk between the image information intended for the right eye and the image information intended for the left eye, and a threshold value is changed.

From the WO 2005/121707 it is known to display a navigation route object in the form of an overhead navigation cable on a head-up display. The overhead navigation cable is displayed at a distance above the roadway. The overhead navigation cable can also be indicated by symbols placed along the cable.

A major advantage of the previously known "augmented reality" displays (AR displays) is that the corresponding displays are displayed directly within or as part of the environment. Relatively obvious examples usually relate to the area of navigation. While classic navigation displays (in conventional HUD) usually show schematic representations, e.g. a right-angled arrow pointing to the right as a sign that a right should be turned at the next opportunity, AR displays offer much more effective options. Since the displays can be presented as "part of the environment", extremely fast and intuitive interpretations are possible for the user. Nevertheless, the previously known approaches also have various problems for which no solutions are known at the present time.

Out Anonymous: "WayRay Augment Reality Navigation System for Alibaba", 15 March 2017 (2017-03-15), XP055550373 , it is known that an AR projection using a head-up display shows the course of a lane as a trajectory in a vehicle, with the generated lane delimitation of the lane to be used, within which the vehicle is to be controlled, being indicated by small dots. These lane boundaries in the form of trajectories can be perceived by the user as lines by the displayed sequence of points.

The known solutions have various disadvantages. This was recognized within the scope of the invention. The problem with the known solutions is that, depending on the environmental conditions, the representation of the course of the direction of travel is either difficult to see or is just too conspicuous and distracts from what is actually happening on the road.

The basic idea of this invention is to do without covering surfaces and instead to fragment the display content. A number of advantages are seen in this approach, which, in addition to meeting the requirements of low masking of the environment and at the same time sufficient error tolerance, consist in the fact that the human perception apparatus is effortlessly able to recognize the individual display elements as a coherent indication due to the evolutionary biological prerequisites to understand. The means of choice for the visualization of the individual grid elements were simple white spheres or points.

The current implementations in the vehicle also revealed two other interesting aspects. On the one hand, only a limited degree of aesthetic value can be achieved with the use of the simple element shape (white spheres/dots). On the other hand, the use of full-area display elements leads to a constant brightness and thus perceptibility of the displayed elements. This effect is definitely desirable for particularly bright environments (sunny day). However, in darker environments (later in the day, heavily cloudy, tunnel, etc.), undesirably high levels of visibility can occur, which lead to an unintentionally high level of attention being paid.

It can also happen that there is sufficient brightness, but the "Augmented Reality" HUD elements are poorly visible due to heavy rain. The display elements should be adapted both in the case of difficult brightness conditions and in the case of poor visibility due to other influences (e.g. rain).

There is therefore a need for further improvements in the display of travel direction profiles (navigation trajectories), which can be variably adjusted so that the driver is not distracted depending on the environmental conditions.

The object of the invention is to find such an approach. A further object is that, conversely, it should be prevented that the direction of travel changes certain environmental conditions is displayed too weakly, so that the driver can hardly follow the course.

This object is achieved by a method for displaying a course of a trajectory in front of a vehicle or an object using a display unit according to claim 1, a device for carrying out the method according to claim 11 and a motor vehicle according to claim 13 and a computer program according to claim 14.

The dependent claims contain advantageous developments and improvements of the invention according to the following description of these measures.

The solution according to the invention is based on adapting the display to the environmental conditions. For this purpose, the environmental conditions are recorded with the appropriate sensors and the symbols used for the grid representation of the course of travel (navigation path) in the form of a lane or driving tube are displayed differently depending on the environmental conditions recorded. As before, the course of the direction of travel is displayed in the driver's field of vision with the aid of a display unit, with the course again being displayed in the form of a grid.

In one embodiment, the invention consists in the fact that initially symbols are used for representing the grid points, which consist only of the outline or border. This display option can be used, for example, when the environmental conditions are rather dark, since the displayed element parts allow sufficient visibility for the driver. The prerequisite for this is the connection to at least one sensory unit that determines the environmental conditions. If the environmental conditions are brighter, the display of the symbols is adjusted up to a full-surface display.

In addition, there should also be a link to sensory units or software modules that evaluate the visibility conditions with regard to the AR displays.

In a particularly preferred embodiment, the ambient brightness comes into consideration as a parameter for evaluating the environmental condition.

When the ambient light is low, the points of the grid are represented with symbols, of which only the border is represented. In an example, they can be simple graphic symbols such as circles, spheres, rectangles, cuboids, triangles, bars, and so on.

This has the advantage that the proportions of the symbols that are displayed are sufficient for the driver to perceive them appropriately.

If the surrounding brightness is high, the points of the grid are displayed with symbols that are displayed completely filled. This has the advantage that the symbols are emphasized more so that they can be better perceived by the driver under these environmental conditions. A stepless adjustment of the fill level of the symbols preferably takes place between the two extreme values of darkness and high levels of solar radiation.

In a further embodiment of the invention, weather conditions, in particular types of precipitation such as rain, heavy rain, snow, hail or fog, are evaluated as environmental factors. The prerequisite is the presence of a corresponding sensor system. Alternatively, a very precise weather report can also be used, which can be downloaded from the Internet, for example. Mention should also be made of other weather conditions such as wind speeds and temperature conditions that may be taken into account.

In another embodiment, when it rains and/or heavy rain, the points of the grid are represented with symbols that have a shape that can be better distinguished from rain, in particular raindrops. Here, for example, the use of a rectangular symbol shape comes into question.

In addition, specific operating modes of the vehicle, such as engine settings, chassis settings and driver assistance settings, can be evaluated as environmental factors. For example, a "Sport" setting could use triangle symbols as symbols, a "Normal" setting could use rectangle symbols, and an "Eco" setting could use circle symbols.

Furthermore, in a further embodiment, the size and/or the color of the symbols is also varied depending on the environmental conditions. For example, it would be advantageous to adjust the color as well as the shape of the symbols for the previously mentioned operating mode settings "Sport", "Normal" and "Eco". For example, the color red could be used for the "Sport" mode and the color blue or black for the "Normal" mode and the color green for the "Eco" mode.

In a further refinement, the condition of the roadway and/or the surroundings of the roadway and/or the traffic situation is recorded and evaluated as an environmental factor. There are different types of road surfaces to consider. At certain Pavements such as cobblestones, for example, it can be advantageous to use different forms of the grid symbols. Other road surfaces are concrete, various types of asphalt such as whisper asphalt, forest and field paths, etc. If there are many potholes on the road, an adjustment should also be made. The potholes should be clearly visible so that the driver can avoid them. Even if there are a large number of line markings (e.g. hatching in restricted areas) on the roadway, the symbols could be adjusted.

In general, it is about dynamically adapting the grid symbols to the environmental conditions. Adjustments could also be made to the roadway environment and the traffic situation. When the vehicle is moving in an urban environment where the environment is more of a shade of gray, accentuation can be achieved by changing the color to yellow, blue, green for the grid symbols. For a more colorful rural setting, the raster icons may be more likely to stay in shades of gray or other monochrome tones. The roadway environment would therefore be evaluated here. If there are many vehicles on the road, e.g. on a motorway with a large number of cars or trucks, and the brake lights come on when you approach the end of a traffic jam, you should definitely choose a color other than red for the grid symbols. Here, the traffic situation would be evaluated as an environmental factor.

The invention can also be used in an advantageous manner if the display unit corresponds to data glasses. The method according to the invention can then even be used for pedestrians, cyclists, motorcyclists, etc.

In the vehicle, the invention is preferably implemented in such a way that the display unit is permanently installed in the vehicle, e.g. in the form of a head-up display. Nevertheless, a possible form of implementation would also be possible with the help of data glasses if the use of data glasses by the driver were allowed in the future.

It is advantageous for a device for carrying out the method if it has a display unit with which additional virtual information can be displayed in the field of vision of the driver or the operator of the object. This is the case with a head-up display as well as with data glasses. Furthermore, this device should have an arithmetic unit and the arithmetic unit should calculate a grid in such a way that it calculates the grid points so that they lie on grid lines through which the trajectory course is displayed, the arithmetic unit for the points of the grid Selects symbols, of which only the border or the fully filled symbols are displayed, depending on the environmental conditions.

It is advantageous if the processing unit either converts the selected symbols in order to be able to display only the border up to completely filled symbols, depending on the environmental conditions, or to select correspondingly precalculated and stored symbols for the display, depending on the environmental conditions.

It is particularly advantageous if the device contains a display unit that is designed as a head-up display.

Instead of a head-up display, data glasses or a monitor can be used as the display unit in the device, on which a camera image is displayed, in which a trajectory profile is superimposed.

The device according to the invention can advantageously be used in a motor vehicle.

The corresponding advantages as described for the method according to the invention apply to a computer program that is processed in the computing unit of the device in order to carry out the method according to the invention.

Embodiments of the invention are shown in the drawings and are explained in more detail below with reference to the figures.

Fig. 1

das Prinzip der Einblendung von Informationen in das Sichtfeld des Fahrers eines Fahrzeuges während der Fahrt mit Hilfe eines Head-Up Displays;

Fig. 2

das typische Cockpit eines Fahrzeuges;

Fig. 3

das Blockschaltbild des Infotainment-Systems des Fahrzeuges;

Fig. 4

zwei Darstellungen von Fahrtrichtungsverläufen, einmal auf gerader Strecke und einmal bei Durchfahren einer Kurve;

Fig. 5

eine Darstellung eines Fahrtrichtungsverlaufs gemäß eines ersten Ausführungsbeispiels der Erfindung;

Fig. 6

eine Darstellung eines Fahrtrichtungsverlaufs gemäß eines zweiten Ausführungsbeispiels der Erfindung;

Fig. 7

eine Darstellung eines Fahrtrichtungsverlaufs gemäß eines dritten Ausführungsbeispiels der Erfindung; und

Fig. 8

ein Flussdiagramm für ein Programm zur Berechnung der Einblendung eines Rasters zur Anzeige des Fahrtrichtungsverlaufs gemäß eines Ausführungsbeispiels der Erfindung.

Show it:

1

the principle of displaying information in the field of vision of the driver of a vehicle while driving using a head-up display;

2

the typical cockpit of a vehicle;

3

the block diagram of the vehicle's infotainment system;

4

two representations of the direction of travel, one on a straight stretch and one when driving through a curve;

figure 5

a representation of a course of the direction of travel according to a first embodiment of the invention;

6

a representation of a direction of travel according to a second embodiment of the invention;

7

a representation of a course of the direction of travel according to a third embodiment of the invention; and

8

a flowchart for a program for calculating the insertion of a grid for displaying the course of the direction of travel according to an embodiment of the invention.

This description illustrates the principles of the inventive disclosure. It is thus understood that those skilled in the art will be able to devise various arrangements that, while not explicitly described herein, embody principles of the inventive disclosure.

1 illustrates the basic functionality of a head-up display. The head-up display 20 is mounted in the vehicle 10 behind the dash panel. Additional information is projected onto the windscreen in the driver's field of vision. This additional information appears as if it were projected onto a projection surface 21 at a distance of 7-15 m in front of the vehicle 10 . However, the real world remains visible through this projection surface 21 . A virtual environment is created with the additional information that is displayed. The virtual environment is theoretically placed over the real world and contains the virtual objects that support and inform the driver while driving. However, it is only projected onto part of the windshield, meaning that the additional information cannot be arranged arbitrarily in the driver's field of vision.

2 shows the cockpit of the vehicle 10. A passenger car is shown. However, any other vehicles could also be considered as the vehicle 10. Examples of other vehicles are: buses, commercial vehicles, in particular trucks, agricultural machinery, construction machinery, rail vehicles, etc. The use of the invention would generally be applicable to land vehicles, rail vehicles, water vehicles and aircraft.

Three display units of an infotainment system are shown in the cockpit. It is the head-up display 20 and a touch-sensitive screen 30 mounted in the center console. When driving, the center console is not in the driver's field of vision. For this reason, the additional information is not displayed on the display unit 30 while driving.

The touch-sensitive screen 30 is used in particular to operate functions of the vehicle 10. For example, a radio, a navigation system, playback of stored pieces of music and/or an air conditioning system, other electronic devices or other comfort functions or applications of the vehicle 10 can be controlled. In summary, it is often referred to as an "infotainment system". In motor vehicles, especially passenger cars, an infotainment system refers to the combination of car radio, navigation system, hands-free device, driver assistance systems and other functions in a central control unit. The term infotainment is a portmanteau composed of the words information and entertainment (entertainment). The touch-sensitive screen 30 (“touch screen”) is mainly used to operate the infotainment system, with this screen 30 being able to be viewed and operated easily by a driver of the vehicle 10 in particular, but also by a passenger of the vehicle 10 . In addition, mechanical operating elements, for example keys, rotary controls or combinations thereof, such as for example push-type rotary controls, can be arranged below the screen 30 .

3 shows a schematic block diagram of infotainment system 200 and some subsystems or applications of the infotainment system by way of example. The operating device includes the touch-sensitive display unit 30, a computing device 40, an input unit 50 and a memory 60. The display unit 30 includes both a display surface for displaying changeable graphical information and a user interface (touch-sensitive layer) arranged above the display surface for entering commands User.

The display unit 30 is connected to the computing device 40 via a data line 70 . The data line can be designed according to the LVDS standard, corresponding to low-voltage differential signaling. The display unit 30 receives control data for driving the display surface of the touchscreen 30 from the computing device 40 via the data line 70 . Reference numeral 50 designates an input unit. The control elements already mentioned, such as buttons, rotary controls, slide controls or rotary push controls, which the operator can use to make entries via the menu navigation, belong to it. Entry is generally understood to mean selecting a selected menu option, as well as changing a parameter, turning a function on and off, etc. is typical Steering wheel operation of parts of the infotainment system is also possible. This unit is not shown separately, but is considered part of the input unit 50.

The memory device 60 is connected to the computing device 40 via a data line 80 . A list of pictograms and/or a list of symbols is stored in the memory 60, with the pictograms and/or symbols for the possible insertions of additional information.

The other parts of the infotainment system, camera 150, radio 140, navigation device 130, telephone 120 and instrument cluster 110 are connected via data bus 100 to the device for operating the infotainment system. The high-speed variant of the CAN bus according to ISO Standard 11898-2 can be used as the data bus 100 . Alternatively, the use of a bus system based on Ethernet technology such as BroadR-Reach could also be considered. Bus systems in which data is transmitted via optical fibers can also be used. The MOST bus (Media Oriented System Transport) or the D2B bus (Domestic Digital Bus) are mentioned as examples. It is also mentioned here that the camera 150 can be designed as a conventional video camera. In this case, it records 25 frames/s, which corresponds to 50 fields/s in the interlace recording mode. Alternatively, a special camera can be used that takes more frames/s to increase the accuracy of object recognition for faster-moving objects. Several cameras can be used to monitor the surroundings. In addition, the already mentioned RADAR or LIDAR systems could also be used in addition or as an alternative to carry out or expand the area monitoring. Vehicle 10 is equipped with a communication module 160 for internal and external wireless communication. This module is often also referred to as an on-board unit. It can be designed for cellular communication, e.g. according to the LTE standard, corresponding to Long Term Evolution. It can also be designed for WLAN communication, corresponding to wireless LAN, be it for communication with devices belonging to the occupants in the vehicle or for vehicle-to-vehicle communication, etc.

The method according to the invention for displaying a trajectory profile in front of a vehicle or an object using a display unit is explained below using a number of exemplary embodiments. The direction of travel in front of a vehicle is used as an example of a trajectory profile.

For the other figures, the same reference numbers denote the same fields and symbols as in the description of the Figures 1 to 3 explained.

As previously described, the display of the course of travel direction according to the invention is based on a virtual grid that is displayed at a distance above the actual real environment. The real environment corresponds to the real course of the road. The grid is preferably projected in such a way that it appears to the driver that the grid is a few centimeters above the roadway. The distance can vary depending on the design.

In 4 shows a first variant of how the course of the direction of travel can be displayed according to the invention. The direction of travel is typically displayed as a navigation route as part of the navigation function of navigation system 130 of vehicle 10 . The path that leads the vehicle 10 to the entered destination is thus marked. In the dark, the direction of travel can also be displayed to support the driver, independently of a destination that has been entered. In this case, the course of the road would be displayed. This helps the driver because he can see earlier which curve or turn is coming next. In this example, the course of the travel direction is represented with the aid of a point grid 22. The point grid 22 preferably extends over the width of the lane on which the vehicle 10 is moving. The type of display in this example is chosen so that the points in the middle area 23 are more strongly emphasized than the points at the edges of the lane. in the in 4 In the example shown, the emphasized points are highlighted in bold. By emphasizing the points in the middle of the lane, the impression of a line is reinforced for the driver, as he is used to from navigation systems. This exploits the ability of human perception already described, according to which the brain automatically completes fragmented patterns.

In the figure 5 a first variant of the display of the course of the direction of travel is shown, which is advantageous for use in rather dark ambient conditions. Instead of the full-surface display of the points of the grid 22 with which the direction of travel is displayed, only the outline of the symbols 23 with which the grid points are marked is displayed. In these dark ambient conditions, the parts shown here in white are sufficient for the driver to perceive them appropriately. Circles are used as symbols in this variant. The display of the number "50" indicates the current vehicle speed.

As the ambient brightness increases, the symbols 23 of the grid 22 become progressively more prominent. This is done by padding the symbols from the edge. This can be done steplessly, but it doesn't have to be. A finely graduated filling would also be considered as a variant.

In the 6 the stage is shown where the symbols 23 have been completely filled. This representation is typically selected in full daylight with sunshine. If the symbols are filled steplessly, the degree of filling can be calculated as a function of the brightness conditions.

An example calculation of the degree of filling FG depending on the ambient brightness UH is presented as follows.

, wobei das Infotainmentsystem so kalibriert sein muss, dass $\frac{{\mathit{UH}}_{\mathit{max}}}{{\mathit{UH}}_{\mathit{min}}}=\frac{1}{{\mathit{FG}}_R}$

gilt.The minimum ambient light level at which the border is just as in figure 5 is represented, let UH _min . Accordingly, the ambient brightness at full daylight is UH _max . If the degree of filling of the symbols in the display with minimum ambient brightness FG is _R , then the degree of filling FG of the symbols at a measured brightness value UH is calculated according to the formula: $\mathit{FG}={\mathit{FG}}_R\ast \frac{\mathit{uh}}{{\mathit{uh}}_{\mathit{at\; least}}}$

, whereby the infotainment system must be calibrated so that $\frac{{\mathit{uh}}_{\mathit{Max}}}{{\mathit{uh}}_{\mathit{at\; least}}}=\frac{1}{{\mathit{FG}}_R}$

is applicable.

It is also conceivable that the degree of filling for the minimum setting FG _R can be set by the driver.

In the case of extended embodiments, additional adaptations are made. In a first expansion, the expansion relates to other difficult lighting conditions.

If there are other difficult visibility conditions, the adaptive procedure is also related to the shape of the symbols 23 . In the case of heavy rain, for example, round raster symbols 23 can be dispensed with and the shape can be changed to more angular symbols 23 . The advantage of adapting the raster symbols 23 as a function of environmental conditions also lies in an optimized perceptibility of the AR-HUD display elements. The representation of the course of travel with square grid symbols is in 7 shown. If there are environmental conditions that rule out the use of certain shapes or make them not advisable (e.g. heavy rain or a certain road surface), the shape of the elements should be continuously adjusted will. In this representation, rather rectangular elements are shown, which can emerge from the round elements. The prerequisite for this is that a correspondingly precise sensor system is available in the vehicle. Rain sensors are no longer special equipment in motor vehicles and can serve this purpose. Alternatively, indirect recording could also be considered. If, for example, the speed of the wiper motor is recorded and evaluated, information about the intensity of the rain can also be obtained. Image evaluation methods with cameras could also be considered as an alternative. The shape of the raster symbols would then have to be updated at short intervals because the precipitation situation can change quickly while driving.

In addition, a change could also be made if certain changes in the operating mode occur on the vehicle side (e.g. selection of a sports program). In modern vehicles, such settings affect the entire drive system, i.e. engine, transmission and chassis. The operating mode is often displayed to the driver on a display unit in the instrument cluster 110 . The extension here is that a display also takes place via the head-up display 20 . In one embodiment, different raster symbols 23 would be used depending on the driving mode.

For example, triangles pointing forward could be used as grid symbols 23 in the "Sport" driving program. Rectangular symbols could be used for the "Normal" driving program. Circle symbols could be used for the “Eco” driving program. In addition, a colored design of the grid symbols could make the meaning of the symbols easier to recognize. The "Sport" mode icons would be red in color. The "Normal" mode icons would be colored blue or white. The "Eco" mode icons would be colored green or blue.

In the 8 the sequence of a program which is processed in the arithmetic unit 40 in order to carry out the method according to the invention is now also shown.

The start of the program is denoted by the reference number 405 . In program step 410, ambient brightness UH for vehicle 10 is recorded. A dedicated brightness sensor can be provided in the cockpit for this purpose. However, brightness sensors are often installed in other components. The instrument cluster 110 and the touch-sensitive screen 30 are mentioned as an example. The measured brightness values would are transmitted to the arithmetic unit 40 via the bus systems 100 and 70 . Another way of detecting the ambient brightness can also be done by evaluating the image recorded by the camera 150 . In program step 415, the type or intensity of precipitation NA is recorded. This can be done as already described, using a rain sensor, the speed of the wiper motor, or also using an image evaluation method and camera 150. In the subsequent step 420, the set driving mode FM is recorded. The setting is stored anyway in the arithmetic unit 40 in a corresponding register location. The value at the register location is read out in this step. The raster symbols 23 are then selected as a function of the detected parameters UH, NA, FM. The pictograms for the symbols are entered in a table which is stored in memory 60. A large number of pictograms can be stored there, for example for the various degrees of filling in a finely graded gradation. In this way, finished symbols can be called up with the required degree of filling without having to convert them. Then, in program step 430, the navigation route is transferred from the navigation system 130 to the computing unit 40. Then, in step 435, the grid 22 is calculated for the overlay on the head-up display 20 with the selected symbols 23 for the planned navigation route. How this step is carried out is known from the corresponding calculation methods of the known navigation systems. Finally, the data for overlaying the grid 22 is transmitted to the head-up display 20 in step 440 . This then blends the calculated grid 22 into the driver's field of vision. The program ends in program step 445.

All examples mentioned herein, as well as conditional language, are intended to be understood as not being limited to such specifically cited examples. For example, it will be appreciated by those skilled in the art that the block diagram presented herein represents a conceptual view of exemplary circuitry. Similarly, it will be appreciated that an illustrated flowchart, state transition diagram, pseudo-code, and the like are various variations representing processes that may be stored substantially on computer-readable media and thus executable by a computer or processor. The object named in the patent claims can expressly also be a person.

Another environmental factor, which can also be the reason for an adjustment of the raster symbols, can be the road surface, for example. With certain types of pavement, such as cobblestones, it can be advantageous to use different forms of the grid symbols. But this also applies to other road surfaces (Concrete, various types of asphalt such as whisper asphalt, forest and field paths, etc.). If there are many potholes on the road, an adjustment should also be made. The potholes should be clearly visible so that the driver can avoid them. Even if there are a large number of line markings (e.g. hatching in restricted areas) on the roadway, the symbols could be adjusted.

In general, it is about dynamically adapting the grid symbols to the environmental conditions. When the vehicle is moving in an urban environment where the environment is more of a shade of gray, accentuation can be achieved by changing the color to yellow, blue, green for the grid symbols. For a more colorful rural setting, the raster icons may be more likely to stay in shades of gray or other monochrome tones. The roadway environment would therefore be evaluated here. If there are many vehicles on the road, e.g. on a motorway with a large number of cars or trucks, and the brake lights come on when you approach the end of a traffic jam, you should definitely choose a color other than red for the grid symbols. Here, the traffic situation would be evaluated as an environmental factor. The adjustment should be as dynamic as possible. In a simpler variant, the adjustment due to the roadway environment would be specified by the navigation route and would be rather static in this regard.

It should be understood that the proposed method and associated devices can be implemented in various forms of hardware, software, firmware, special purpose processors or a combination thereof. Specialty processors can include Application Specific Integrated Circuits (ASICs), Reduced Instruction Set Computers (RISC), and/or Field Programmable Gate Arrays (FPGAs). Preferably, the proposed method and device is implemented as a combination of hardware and software. The software is preferably installed as an application program on a program storage device. Typically, it is a computer platform based machine that includes hardware such as one or more central processing units (CPU), random access memory (RAM), and one or more input/output (I/O) interfaces. An operating system is typically also installed on the computer platform. The various processes and functions described here can be part of the application program or a part that runs through the operating system.

The disclosure is not limited to the exemplary embodiments described here. There is room for various adaptations and modifications that those skilled in the art would contemplate based on their skill in the art as well as belonging to the disclosure.

The invention is explained in more detail in the exemplary embodiments using the example of use in vehicles. The application can also be of particular interest for emergency vehicles used by the fire brigade, doctors, police, civil protection, etc., in order to support the emergency services in finding people in need of help particularly quickly or in order to avert danger. The possibility of use in aircraft and helicopters, for example in landing maneuvers or search operations, etc., is also pointed out here.

However, it is pointed out that the use is not limited to this. The invention can be used whenever the field of view of a driver, an operator or simply a person with data glasses can be enriched with AR overlays.

AR overlays can also make operation easier for remote-controlled devices such as robots, which are controlled remotely via a monitor on which a camera image is displayed. So there is also an opportunity here.

Reference List

10

vehicle

20

Head Up Display HUD

21

virtual projection surface

22

grid

23

raster icon

30

touch-sensitive display unit

40

unit of account

50

input unit

60

storage unit

70

Data line to the display unit

80

Data line to storage unit

90

Data line to the input unit

100

data bus

110

instrument cluster

120

phone

130

navigation device

140

radio

150

camera

160

communication module

200

Infotainment system

405-445

different program steps

Claims (14)

1. Method for displaying the course of a trajectory in front of a vehicle (10) or in front of a person, a pedestrian, a bicycle or a motorcycle with the aid of an augmented reality display unit (20) in the form of a head-up display (HUD), data glasses or a monitor, on which a camera image is displayed, wherein the trajectory is overlaid in the field of view of the driver, the person or the pedestrian on the augmented reality display unit (20), wherein the trajectory course is represented in grid form, characterized in that the points of the grid (22) are represented using symbols (23), of which only the border or the completely filled symbols (23) are represented depending on the environmental condition, wherein the ambient brightness is detected and analysed as a first environmental condition, wherein, in the case of low ambient brightness, the points of the grid (22) are represented using symbols (23), of which only the border is represented, and wherein, in the case of high ambient brightness, the points of the grid (22) are represented using symbols (23) which are represented filled up over the full surface.
2. Method according to Claim 1, wherein, if the ambient brightness increases, the symbols (23) of the grid (22) are highlighted increasingly more strongly by filling up the symbols from the edge.
3. Method according to Claim 2, wherein the filling-up process takes place continuously.
4. Method according to Claim 2, wherein the filling-up process takes place as a finely graded filling-up process.
5. Method according to any one of the preceding claims, wherein a driving direction course is represented with the aid of a point grid (22), the point grid (22) extends over the width of a lane on which the vehicle (10) is moving, wherein the points in the middle region (23) are emphasized more strongly than the points at the edges of the lane.
6. Method according to any one of the preceding claims, wherein types of precipitation such as rain, heavy rain, snow, hail, or fog are detected and analysed as a second environmental condition.
7. Method according to Claim 6, wherein, in the case of rain and/or heavy rain, the points of the grid (22) are represented using symbols (23) which have a shape which can be differentiated better from rain, in particular raindrops, wherein the shape of the symbols (23) corresponds to a rectangular shape.
8. Method according to any one of Claims 1 to 7, wherein operating modes of the vehicle, such as engine settings, chassis settings, and driver assistance settings, are detected and analysed as a third environmental condition.
9. Method according to any one of Claims 1 to 8, wherein the roadway composition and/or the roadway environment and/or the traffic situation is/are detected and analysed as a fourth environmental condition.
10. Method according to any one of the preceding claims, wherein the size and/or the colour of the symbols (23) is/are varied depending on the environmental condition.
11. Device which is configured to carry out the method according to any one of the preceding claims, comprising an augmented reality display unit (20), using which virtual items of additional information can be overlaid in the field of view of the driver, the person or the pedestrian, and a processing unit (40) and also one or more components (110, 130, 150, 160) for detecting environmental conditions which comprise at least the ambient brightness, characterized in that the processing unit (40) computes a grid (22) for the display of a trajectory course, wherein the processing unit (40) either selects correspondingly precomputed and stored symbols (23) for the points of the grid (22), which only show the border or the completely filled symbols (23), in dependence on the ambient brightness, or wherein the processing unit (40) converts the selected symbols (23) in order to be able to represent only the border or the completely filled symbols (23) depending on the ambient brightness.
12. Device according to Claim 11, wherein the display unit (20) is a head-up display (HUD) or data glasses.
13. Motor vehicle, characterized in that the motor vehicle (10) comprises a device according to any one of Claims 11 to 12.
14. Computer program, characterized in that the computer program is designed, upon execution in a processing unit (40) in a device according to Claim 11, to carry out the steps of the method for displaying the course of a trajectory in front of a vehicle (10) or in front of a person, a pedestrian, a bicycle or a motorcycle according to any one of Claims 1 to 10.

Images