EP3759694A1

EP3759694A1 - Method for calculating an ar-overlay of additional information for a display on a display unit, device for carrying out the method, as well as motor vehicle and computer program

Info

Publication number: EP3759694A1
Application number: EP19705167.5A
Authority: EP
Inventors: Andro Kleen; Robert Jan Wyszka; Vitalij SADOVITCH; Adrian HAAR; Johannes Tümler; Michael Wittkämper
Original assignee: Volkswagen AG
Current assignee: Volkswagen AG
Priority date: 2018-03-02
Filing date: 2019-02-12
Publication date: 2021-01-06
Also published as: US20210046822A1; US11904688B2; CN111937044A; WO2019166222A1; DE102018203121B4; DE102018203121A1

Abstract

The invention relates to a method for calculating an overlay of additional information for a display on a display unit (20), in particular a head-up display (HUD) of a vehicle (10) or data glasses. The overlay of additional information serves the purpose of supporting the driver in the longitudinal control of a vehicle (10). The overlaying of the additional information occurs in the form of augmented reality such that it is calculated in a contact analogue manner in relation to one or more objects in the environment of the vehicle (10). The position of a preceding vehicle (300) is detected. When approaching the oncoming or preceding vehicle (300), a spatially extended animation graphic is calculated, wherein the animation graphic has a grid shape consisting of a plurality of grid elements (305), which extends from the observer vehicle up to the oncoming or preceding vehicle (300). The spatial extension is calculated such that the driver of the observer vehicle (10) has the impression of a kinematic or dynamic movement of the spatial extension, such as translation and rotation.

Description

description

Method for calculating an AR display of additional information for a display on a display unit, device for carrying out the method and motor vehicle and computer program

The proposal concerns the technical field of driver information systems, also known as infotainment systems. In particular, this involves a method for displaying a safety zone in front of a vehicle or an object on a display unit. Such systems are used primarily in vehicles. But there is also the possibility of using the invention in pedestrians, cyclists, etc. with data glasses. The proposal continues to cover a correspondingly designed one

Device for carrying out the method and a motor vehicle and a

Computer program.

At present, intensive work is being done on technologies that will later enable autonomous driving. A first approach is not to completely relieve the driver of his duties, but to ensure that the driver can always take control of the vehicle. The driver also takes

Monitoring functions true. Due to newer technologies in the field of

Driver information systems such as Head-Up Display (HUD), it is possible to better inform the driver about the happenings in the environment of his vehicle.

For the near future, it can therefore be assumed that comprehensive information about objects (in particular vehicles) in the immediate vicinity of one's own vehicle will be available on the system side through the use of newer technologies (vehicle-to-vehicle communication, use of databases, vehicle sensors, etc.) , In the area

Vehicle sensors are in particular the following components are called, which allow an environment observation: RADAR devices according to Radio Detection and Ranging, LIDAR devices, according to Light Detection and Ranging, mainly for distance detection / warning, and cameras with appropriate image processing for the field of object recognition. This data on the environment can be used as a basis for system-side driving recommendations, warnings, etc. For example, it is conceivable to display / warn in which direction (possibly in the own trajectory) another, surrounding vehicle wants to turn. The vehicle-to-vehicle communication is now also possible by means of mobile communication with systems such as LTE according to Long Term Evolution. Here, a specification called LTE V2X was adopted by the 3GPP organization. Alternatively, systems based on WLAN technology for direct vehicle communication are available, in particular the system according to WLAN p.

The term "autonomous driving" is sometimes used differently in the literature.

To clarify this term, therefore, the following insert is presented here. Autonomous driving (sometimes also called automatic driving, automated driving or piloted driving) is the movement of vehicles, mobile robots and driverless transport systems to understand the largely autonomous behavior. There are different gradations of the term autonomous driving. Autonomous driving is also discussed at certain stages when there is still a driver in the vehicle who, if necessary, only takes over the monitoring of the automatic driving process. In Europe, the various ministries of transport (in Germany the Federal Highway Research Institute was involved) worked together and the following

Defined autonomy levels.

• Level 0: Driver only, the driver drives, steers, accelerates, brakes, etc.

• Level 1: Certain assistance systems help with vehicle operation (including a cruise control system - Automatic Cruise Control ACC).

• Level 2: partial automation. Et al automatic parking, lane keeping, general longitudinal guidance, acceleration, deceleration, etc. are of the

Assistance systems taken over (including congestion assistant).

• Level 3: high automation. The driver does not have to keep the system permanently

monitor. The vehicle autonomously performs functions such as triggering the turn signal, lane change and lane keeping. The driver can turn to other things, but if necessary within a warning time of the system

asked to take the lead. This form of autonomy is technically feasible on motorways. Legislators are working to allow Level 3 vehicles. The legal framework has already been created for this purpose.

• Level 4: full automation. The leadership of the vehicle is permanently from

System adopted. If the system no longer copes with the driving tasks, the driver can be asked to take the lead. • Level 5: No driver required. There is no human intervention beyond setting the goal and starting the system.

Automated driving functions as of level 3 relieve the driver of the responsibility for controlling the vehicle.

Due to the current trend towards higher levels of autonomy, but where many vehicles are still controlled by the driver, it can be assumed that corresponding additional information can already be used in the medium term for manually-operated vehicles and not for long-term highly automated systems.

For the driver-vehicle interaction, this raises the question of how this information can be presented in such a way that real added value for the human driver is created and he can also locate the information provided quickly or intuitively. The following solutions in this area are already known from the prior art.

A future vision in the automotive industry is to be able to record the windscreen of one's own vehicle with virtual elements in order to offer the driver some advantages. The so-called "augmented reality" technology (AR) is used. Less familiar is the corresponding German-language term of "augmented reality". The real environment is enriched with virtual elements. This has several advantages: The view down, on displays other than the windshield, is eliminated, since many relevant information is displayed on the windshield. So the driver does not have to avert his gaze from the road. In addition, due to the positionally exact sclerosing of the virtual elements in the real environment, a lesser cognitive effort on the part of the driver is likely because no interpretation of a graphic must be done on a separate display. With regard to automatic driving, an added value can also be generated.

Given that today's technological resources are limited, it can be assumed that in the medium term, fully windable windshields will not be found in vehicles. Currently Head-Up Displays (HUD) are used in the vehicles. These also have the advantage that the image of the HUD appears closer to the real environment. These displays are actually about

Projection units that project an image on the windshield. However, from the driver's point of view, this image is a few meters to 15 meters, depending on the design of the module in front of the vehicle. This has the advantage that the displayed information is presented in such a way that the eyes of the driver are even relieved of accommodation activity.

The "image" is composed as follows: It is less a virtual display, but rather a kind of "keyhole" in the virtual world. The virtual environment is theoretically placed over the real world and contains the virtual objects that support and inform the driver while driving. The limited

Display area of the HUD has the consequence that a section of it can be seen.

So you look through the display area of the HUD on the section of the virtual world. Since this virtual environment complements the real environment, one speaks in this case of a "mixed reality".

From DE 10 2015 116 160 A1 a head-up display unit for a vehicle for generating a virtual image in the field of view of the driver is known. This results in a situation-based adapted representation of information.

DE 10 2013 016 241 A1 discloses a method for augmented presentation of at least one additional piece of information in at least one recorded digital image

Environment, especially a vehicle environment known. Furthermore, can

be provided that the additional information, in particular event-driven and optionally alternately, as a 2D representation and / or output as a 3D representation. For example, an additional information output as an additional object, eg. B. a virtual road sign, in 3D presentation and / or issued as additional text additional information, in particular a caption of a real or virtual object in the output image, are displayed in 2D representation in the output image.

From DE 10 2014 008 152 A1 a method for augmented presentation of at least one additional information in at least one image of an environment, in particular a vehicle environment, is known. Compared to conventional augmented

Representation method with the most accurate contact-analogous placement of the virtual objects and thus the closest possible placement, for example GPS position, the virtual objects on each associated real object allows the invention by outputting the virtual additional information in a previously determined display region that the virtual additional information is optimally placed , in particular taking into account a

Avoiding overlays with other real or virtual objects in the image of the ad. From DE 10 2014 119 317 A1 is a method for displaying a

Image overlay element in an image of a surrounding area of a motor vehicle known. The image is displayed on a display surface of the motor vehicle. Furthermore, at least one object from the surrounding area is imaged in the image. Furthermore, it is provided that the image overlay element is motion-coupled to the object and is shown moving the object with this carried at the same location on the object, depending on a change of direction and / or resizing the direction and / or size of the image overlay element to the appropriate change of the object is adjusted. So it can be provided that the

Image superposition element is adapted to the surface of the roadway. If the roadway is now rising, for example, because it is uphill, then this can be done on the basis of

Three-dimensional information about the object can be detected, and the representation of the image overlay element can with respect to the orientation and design

be adjusted accordingly.

From DE 10 2015 117 381 A1 a head-up display (HUD) is known, are offered on the context-based information for the driver. Such can be one

Ambient condition and / or a condition affecting the physiology of the user.

A major advantage of the so-called "augmented reality" (AR) displays is that they display the corresponding ads directly within or as part of the environment. Relatively obvious examples mostly relate to the field of navigation. While classic navigation displays (in conventional HUDs) typically display schematic representations (e.g., a right-angled arrow indicating that you want to turn right at the next opportunity), AR displays offer much more effective options. Since the ads can be viewed as "part of the environment", extremely fast and intuitive interpretations are possible for the user. Nevertheless, the previously known approaches also different

Problems for which there are no known solutions at the moment.

The known solutions are associated with various disadvantages. This was in the

Scope of the invention recognized. In the known solutions there is the problem that the cognitive interpretation and understanding of AR-impressions in the HUD some

Milliseconds of the driver's attention. The processing of this information can change situationally and lead to misinterpretations or Misunderstandings. This can lead to dangerous situations. For example, the driver is displayed on a conventional display of the navigation route, although the route to be traveled, it is characterized but not clearly enough that a

Risk potential builds on an oncoming vehicle at a bottleneck. Also, the location of the bottleneck in the driver's field of vision is not accurate enough

highlighted.

There is therefore a need for further improvements in the longitudinal and transverse guidance of a vehicle and the related feedback to the driver via the infotainment system.

The invention sets itself the task of finding such an approach.

This object is achieved by a method for calculating a fade of

Additional information for a display on a display unit, in particular a head-up display (HUD) of a vehicle or a data glasses according to claim 1, an apparatus for performing the method according to claim 8 and a motor vehicle according to claim 11 and a computer program according to claim 12 solved. The insertion of additional information serves the purpose of assisting the driver in the longitudinal and transverse guidance of the vehicle.

The dependent claims contain advantageous developments and improvements of the invention according to the following description of these measures.

The solution is to animate the elements of AR inserts, such as lines, surfaces, and other geometric elements, through "physical" behaviors.

This makes it easier to communicate the urgency of an interaction

Physical behavior of the elements of AR impressions in the HUD supports an intuitive interpretation of the current situation. Well-known patterns of behavior, which humans have learned or even congenitally during their lifetime, are used to better and faster interpret AR-impressions in the HUD. Dynamics, frequency and rotation are used in such a way that two- and three-dimensional elements wander around in the digital space. Lines, surfaces, and geometric elements reflect physical behaviors and properties known from the real world.

The intuitive perception of the human is used to promote a fast processing of the displayed information in the HUD. In one variant, the invention relates to a method for calculating an AR display, corresponding to "augmented reality" insertion, of additional information for a display on a display unit, in particular a head-up display (HUD) of an observer vehicle or data glasses, wherein the Display of

Additional information serves the purpose of assisting the driver in the longitudinal and / or transverse guidance of an observer vehicle. In this case, the AR overlay is calculated in the manner of augmented reality in accordance with "augmented reality" in a contact-analogous manner to one or more objects in the environment of the observer vehicle. The position of an oncoming or preceding vehicle or object is detected. When approaching the oncoming or preceding vehicle, a spatially extended animation graphic is calculated, wherein the animation graphic has a raster form consisting of a plurality of raster elements, which differs from the

Observer vehicle extends to the oncoming or preceding vehicle. A special feature is that the spatial extent is calculated in such a way that the driver of the observer vehicle has the impression of a kinematic or dynamic movement of the spatial extent, such as translation and rotation.

In a particularly advantageous embodiment, the spatial extent is calculated in such a way that the driver of the observer vehicle is given the impression of a shaft moving or moving away from him. The animation with a waveform can be designed so that the wave can travel on the X, Y, or Z axis.

In another embodiment, the animation graphic is calculated to periodically repeat the spatial extent of the animation graphic, giving the driver of the observer vehicle the impression that a number of wave trains are moving toward or away from him.

In another variant, one or two spatially extended animation graphics are calculated to support the lateral guidance of the vehicle, which are displayed laterally of the route, the further animation graphics having a grid shape consisting of a plurality of raster elements, and the spatial extent is calculated so that the side where an obstacle or an oncoming vehicle has been detected, the grid is spatially set up to emphasize a narrowing of the route. The goal of towering elements is to better communicate warnings. The piling / extrusion of elements can be done in any axis to any proportions.

Other variants would be that individual elements move around an object and are physically attracted / repelled by it. This makes distances clearer and can be highlighted. From layers, elements are extruded one-to-one or two-dimensionally in order to show urgency or to prioritize certain areas in the HUD / the real world. Many functions can be conveyed to the driver through the physical behavior of the AR overlays.

For the method, it is also advantageous if an estimate of the width of the bottleneck is made and falls below a minimum width one

Animated graphics is calculated so that the at least one grid-shaped

Turns animation graphics to guide the side of the vehicle into a warning symbol, indicating the need for an evasive maneuver. If elements are concentrated at specific locations / coordinates in the real world, resulting in compression, the driver's focus of attention can be directed there.

Here a particularly intuitive form of conversion is possible, wherein the conversion of the animation graphics is calculated so that the raster elements of the animation graphics to support the lateral guidance during the conversion phase swarming, from which at the end of the conversion phase, the hint symbol arises. The swarming behavior of the lines, surfaces, and geometric elements that cluster at each coordinate in the real world creates a superposition and an automatic increase in visual intensity.

It is also advantageous for the further animation graphics to support the side guidance of the vehicle to be calculated in such a way that they reflect the path of the vehicle

marked oncoming vehicle or object. This "bundling" of elements can be e.g. be used for functions of the area and object marking, but also for the indication of a navigation path or an attention control.

For a device for carrying out the method with the correspondingly programmed arithmetic unit apply the same advantages as in the corresponding

Procedural steps mentioned. It is particularly advantageous if the display unit of the device is designed as a head-up display. Instead of a head-up display, a data goggle or a monitor can be used in the device as a display unit, on which a camera image is displayed, in which the grid is displayed.

Advantageously, the device according to the invention can be used in a motor vehicle. In the vehicle, the invention is preferably realized so that the display unit is permanently installed in the vehicle, e.g. in the form of a head-up display.

Nevertheless, a possible form of realization would also be possible with the aid of data glasses, if the use of data glasses by the driver would be permitted in the future.

As mentioned, the invention can also be advantageously used if the display unit corresponds to data glasses. Then, the invention can be

Use procedures even for pedestrians, cyclists, bikers, etc.

For a computer program that comes in the arithmetic unit of the device for processing to perform the inventive method, the corresponding advantages apply as described for the inventive method.

Embodiments of the invention are illustrated in the drawings and are explained in more detail with reference to FIGS.

Show it:

1 shows the principle of the superimposition of information in the field of vision of the driver of a vehicle while driving with the aid of a head-up display;

Fig. 2 shows the typical cockpit of a vehicle;

3 shows the block diagram of the infotainment system of the vehicle;

Fig. 4 is an illustration of a wave-shaped arched Rastereinblendung for

Signaling the urgency of a danger due to an oncoming vehicle;

5 shows an illustration of a grid insertion for highlighting a narrowing of the travel path;

6 is an illustration of a flock-like conversion of a grid-shaped AR fade to an instruction to the driver in the example of the lane narrowing;

FIG. 7 is an illustration of three basic levels of driver overlay information and; FIG. FIG. 8 is a flow chart for a program for calculating the AR fades for the three basic levels. FIG.

The present description illustrates the principles of the disclosure of the invention. It will thus be understood that those skilled in the art will be able to devise various arrangements which, while not explicitly described herein, are intended to embody principles of the invention and to be equally limited in scope.

Fig. 1 illustrates the basic operation of a head-up display. The head-up display 20 is in the vehicle 10 below / behind the instrument cluster in

Dashboard area attached. By projection onto the windshield, additional information is displayed in the driver's field of vision. This additional information appears as if it were projected onto a projection surface 21 at a distance of 7 - 15 m in front of the vehicle 10. Through this projection surface 21, however, the real world remains visible. The additional information displayed creates a kind of virtual environment. The virtual environment is theoretically placed over the real world and contains the virtual objects that support and inform the driver while driving. However, it is only projected onto a part of the windshield, so that the additional information can not be arbitrarily arranged in the field of vision of the driver.

Fig. 2 shows the cockpit of the vehicle 10. Shown is a passenger car. As vehicle 10, however, any other vehicles would also be considered. Examples of other vehicles are: buses, commercial vehicles, especially trucks trucks, agricultural machinery, construction machinery, rail vehicles, etc. The use of the invention would be generally in land vehicles, rail vehicles, watercraft and aircraft possible.

In the cockpit three display units of an infotainment system are shown. It is the head-up display 20 and a touch-sensitive screen 30 which is mounted in the center console. When driving, the center console is not in the driver's field of vision. Therefore, the additional information is not displayed on the display unit 30 while driving.

The touch-sensitive screen 30 serves in particular for the operation of functions of the vehicle 10. For example, about a radio, a

Navigation system, playback of stored music and / or air conditioning, other electronic devices or other comfort features or Applications of the vehicle 10 are controlled. In summary, there is often talk of an "infotainment system". An infotainment system called at

Motor vehicles, especially passenger cars, the merger of car radio, navigation system, speakerphone, driver assistance systems and other functions in a central control unit. The term infotainment is a boxword word composed of the words information and entertainment (entertainment). To operate the infotainment system mainly the touch-sensitive screen 30 ("touch screen") is used, this screen 30 can be well viewed and operated in particular by a driver of the vehicle 10, but also by a passenger of the vehicle 10. In addition, below the screen 30, mechanical operating elements, for example keys, rotary encoders or combinations thereof, such as, for example, a push-dial regulator, can be arranged in an input unit 50. Typically, steering wheel control of parts of the infotainment system is also possible. This unit is not shown separately, but is considered part of the input unit 50.

3 shows schematically a block diagram of the infotainment system 200 as well as by way of example some subsystems or applications of the infotainment system. The operation device includes the touch-sensitive display unit 30, a calculator 40, an input unit 50, and a memory 60. The display unit 30 includes both a display surface for displaying variable graphic information and a control surface (touch-sensitive layer) disposed above the display surface

Entering commands by a user.

The display unit 30 is connected to the computing device 40 via a data line 70

connected. The data line can be designed according to the LVDS standard, corresponding to Low Voltage Differential Signaling. Via the data line 70, the display unit 30 receives control data for driving the display surface of the touch screen 30 from the

Computing device 40. Via the data line 70 are also control data of

entered commands from the touch screen 30 to the computing device 40. Reference numeral 50 denotes the input unit. It is associated with the already mentioned control elements such as buttons, knobs, sliders, or rotary pushbuttons, with the help of which the operator can make inputs via the menu. Input is generally understood as selecting a selected menu item, as well as changing a parameter, turning a function on and off, and so on.

The memory device 60 is connected to the computing device 40 via a data line 80. In the memory 60 is a pictogram directory and / or symbol directory deposited with the pictograms and / or symbols for the possible overlays of additional information. Here also the points / symbols can be stored, which serve as the basis for the calculation of the raster overlay.

The other parts of the infotainment system camera 150, radio 140, navigation device 130, telephone 120 and instrument cluster 110 are connected via the data bus 100 with the device for operating the infotainment system. The data bus 100 is the high-speed variant of the CAN bus according to ISO standard 11898-2. Alternatively, for example, The use of an Ethernet-based bus system such as BroadR-Reach in question. Bus systems in which the data is transmitted via optical fibers can also be used. Examples are the MOST Bus (Media Oriented System Transport) or the D2B Bus (Domestic Digital Bus). Here it is mentioned that the camera 150 can be designed as a conventional video camera. In this case, it will record 25 frames / s, which is equivalent to 50 fields / s in the interlace recording mode.

Alternatively, a special camera can be used that captures more images / sec to increase the accuracy of object detection on faster moving objects. Several cameras can be used for monitoring the environment. In addition, the already mentioned RADAR or LIDAR systems could be used in addition or alternatively to carry out or expand the field observation. For inbound and outbound wireless communication, the vehicle 10 is with a

Communication module 160 equipped. This module is often referred to as an on-board unit. It can be used for mobile communication, e.g. according to LTE standard,

according to Long Term Evolution, be designed. Likewise it can be designed for WLAN communication, according to wireless LAN, be it for the communication to devices of the occupants in the vehicle or for the vehicle-to-vehicle communication etc.

The inventive method for calculating a display of

Additional information for a display on a display unit 20 will be explained below with reference to an embodiment.

For the other figures, the same reference numerals denote the same fields and symbols as explained in the description of FIGS. 1 to 3.

In the following description of the invention, it is assumed that the driver is driving the vehicle 10 but is assisted by a driver assistance system. A driver assistance system for longitudinal guidance of the vehicle 10 is used. Examples such assistance systems are an automatic distance control ACC, according to Adaptive Cruise Control, and a cruise control system GRA, according to cruise control system. The invention would also be used in the same way, if the vehicle 10 would be controlled fully automatically. The following describes what steps are taken when the vehicle 10 with the longitudinal guidance system activated, here an ACC system, approaches the preceding vehicle 300, detects it and adapts its speed to the preceding vehicle 300. This is done so that a previously entered safety distance is maintained.

For the information of the driver of a vehicle to assist in the longitudinal and transverse guidance of the vehicle, the use of a grid-shaped AR insertion has proven. In this case, a grid-shaped AR overlay is calculated for the path predicted by the navigation system. This displays the route to the driver without obscuring important information of the real scene. The basic idea and technique of the raster-shaped AR overlay is shown in applicant's co-pending patent application DE 10 2017 212 367. It is also expressly referred to the co-pending application with respect to the disclosure of the invention described herein.

The basis of the display according to the invention of the longitudinal and / or transverse guidance function of the vehicle 10 on the HUD 20 is the display of a virtual grid 24 along the route, which is displayed at a distance above the actual road or without distance to the road. The road is located as a real road course in the field of vision of the driver. The special feature of the new proposal is that not only the track is marked with the grid 24, but also an event that is connected to the track. The event, in the example shown in FIG. 4, is that a vehicle 300 is approaching on the roadway, by which, after estimation of

Track width and the relative approach speed between the

oncoming vehicle 300 and the observer vehicle 10 a risk potential is justified. The danger potential, which leads to the AR-insertion of the event, consists in the case shown in by the relative movement of both moving towards each other vehicles 10, 300, taking into account possible objects or

Obstacles that are on the roadside, resulting narrowing of the track. If it is estimated that this will be below a limit, it will be on

Danger potential made aware. This is done, as shown in Fig. 4, by the calculation of a spatially extended AR fade. On the urgency of the Hazard potential is indicated by the fact that the spatial extent starting from the oncoming vehicle 300 moves towards the observer vehicle 10. This creates for the driver of the observer vehicle 10, the impression of a tapering shaft on him. It can be a wave crest or several wave peaks are shown, which move towards the observer vehicle 10. The AR display is preferably calculated in perspective. As a result, a wave crest towers more and more in front of the observer vehicle 10, which increasingly points to the urgency of the imminent danger.

FIG. 5 shows an example of an AR overlay which indicates the imminent

Narrowing of the driveway is pointed. It is shown that along the

Lane course, a left grid 26a and a right grid 28a is displayed. These extend from the observer vehicle 10 to the oncoming or

preceding vehicle 300. The impending constriction is thereby

indicated that the grid 26a and 28a position themselves spatially laterally outward. The spatial arrangement is preferably such that it increases in front of the bottleneck and subsides after the bottleneck.

Fig. 6 shows the case that the impending constriction of the driveway of the

Longitudinal guidance system is estimated so that the risk of collision or contact with the oncoming vehicle 300 is estimated to be too large. In that case, an AR overlay is computed which gives the driver an action to what needs to be done to avoid a collision or touch. The action statement is designed as a dodge arrow 28b. So that the driver understands the handling instruction directly and intuitively, it is not simply superimposed on the position of the escape point but is also specially shaped by the sequence of its formation. This is done so that the symbol of the escape arrow 28b arises from the points of the right grid 28a. The points of the right-hand grid 28a are animated in such a way that they move in a swarming manner and finally accumulate in such a way that they create the escape arrow 28b.

In Fig. 6 is also shown that the points of the left grid 26 a for

Shift side guide so that they mark the alleged route of the oncoming vehicle 300.

Fig. 7 shows a variant of how the various AR overlays can be combined. It is shown that the grid 24 with the representation of

Hazard potential of an event together with the grids 26a and 28a to Side guide are displayed. This can be done to better distinguish in different colors. In the example shown, the grid 24 is shown in red and the grid 26a and 28a in yellow. When the side guide rasters 26a or 28a convert to a hint symbol, instead of the side guide raster 28a, the

according to the indication symbol combined with the grid 24.

A computer program for the calculation of the AR fades is explained with reference to FIG. 8. The program is processed in the arithmetic unit 40. The program start is designated by the reference numeral 405. In the program step 410, the detection of an oncoming vehicle 300 takes place. For this purpose, the images supplied by the camera 150 are provided with those intended for this purpose

Object detection algorithms evaluated. The distance to the oncoming vehicle 300 is estimated and also the relative speed between the

Observer vehicle 10 and the oncoming vehicle 300. The

Instantaneous speed of the oncoming vehicle can be estimated by continuous image evaluation of the images supplied by the camera 150. In another embodiment, the instantaneous speed may be transmitted via car-2-car communication from the oncoming vehicle 300 to the observer vehicle 10. After the oncoming vehicle 300 has been detected and the distance and relative speed have been estimated, the calculation of the grid 24 with the corresponding spatial extent takes place in the program step 415.

Preferably, the grid 24 is calculated in perspective. The calculation continues to be such that the grid expands to the oncoming vehicle 300.

In program step 420, the calculated data for the grid 24 is transmitted to the head-up display 20. This performs the insertion of the grid 24, as seen in Fig. 4. In program step 425, objects or obstacles are detected at the roadside. As shown in Figures 4, 5 and 6, parking vehicles are located on the right lane edge in parking bays. In program step 430, a dynamic calculation of bottlenecks takes place. This is done as follows: The parked vehicle 310 is still at a distance from the observer vehicle 10. The oncoming vehicle 300 moves so that it arrives at approximately the level of the parked vehicle 310, even though the observer vehicle 10 passes the parked vehicle 310 , Thus, a bottleneck only arises in the future through the coincidence of

Oncoming vehicle 300 and observer vehicle 10 at the location of the parked vehicle 310. In this sense, the dynamic calculation of bottlenecks to understand. In program step 435, the calculation of the raster 26a and 28a to mark a bottleneck occurs. The lateral positioning of the grid is done so that they are up to calculated bottleneck increases and decays after the precalculated bottleneck. These rasters 26a and 28a are again calculated in perspective. in the

Program step 440, the calculated data for the rasters 26a and 28a are transmitted to the head-up display 20.

In program step 445, a calculation of the risk potential of the identified bottleneck takes place. In the subsequent query 450, if the predicted bottleneck falls short of a certain width, e.g. the vehicle width of the observer vehicle 10, so in the program step 455 is a calculation of the animation for the conversion of

Side guide grid 28a to an alternate icon 28b instead. As described, the animation consists in that the dots of the grid 28a move in a swarming manner and at the end of their arrangement form the evasion symbol. If no hazard potential is detected, the program branches back to program step 410. In step 460, the data calculated for the AR overlay animation is transmitted to the HUD 20.

Via steps 410 to 460, a loop is formed in the program, which is run through until a state change takes place. The state change is given when the driver intervenes and leaves the comfort function or shuts down the vehicle. Then the program ends in program step 465.

All examples mentioned herein as well as conditional formulations are to be understood without limitation to such specifically mentioned examples. This is how it is done, for example

Those skilled in the art will recognize that the block diagram presented here represents a conceptual view of exemplary circuitry. Similarly, it will be appreciated that an illustrated flowchart, state transition diagram, pseudocode, and the like represent various variants for representing processes that are described in the

Essentially stored in computer-readable media and thus can be executed by a computer or processor. The object mentioned in the claims can expressly also be a person.

It should be understood that the proposed method and apparatus may be embodied in various forms of hardware, software, firmware,

Special processors or a combination thereof can be implemented.

Special purpose processors may include Application Specific Integrated Circuits (ASICs), Reduced Instruction Set Computer (RISC), and / or Field Programmable Gate Arrays (FPGAs). Preferably, the proposed method and apparatus is implemented as a combination of hardware and software. The software is preferably implemented as an application program on a program storage device Installed. Typically, it is a machine based on a computer platform that has hardware, such as one or more

Central processing units (CPU), random access memory (RAM) and one or more

Input / output (I / O) interface (s). The computer platform also typically installs an operating system. The various processes and functions described herein may be part of the application program or part that is executed via the operating system.

The disclosure is not limited to the embodiments described herein. There is room for various adjustments and modifications that would be considered by those skilled in the art, as well as to the disclosure.

The invention will be explained in more detail in the embodiments using the example of the use in vehicles. Here is also referred to the use of aircraft and helicopters, for example, in landing maneuvers or search missions, etc.

It should be noted, however, that the use is not limited to this. The invention can always be used when the field of view of a driver, an operator or even just a person with data glasses can be enriched with AR impressions.

Even with remote-controlled devices such as robots, in which the remote control via a monitor on which a camera image is played, AR displays can facilitate the operation. So there is also an application here.

List of reference numbers vehicle

Head-Up Display HUD

virtual projection screen

grid

a left side guide grid

b left side guide grid (converted) a right side guide grid

b Dodge arrow

touch-sensitive display unit

computer unit

input unit

storage unit

Data line to the display unit

Data line to the storage unit

Data line to the input unit

0 data bus

0 instrument cluster

0 phone

0 navigation device

0 radio

0 camera

0 communication module

0 infotainment system

0 oncoming vehicle

0 parking vehicle

5 - different

5 program steps

Claims

claims

1. A method for calculating an AR overlay, corresponding augmented reality, of additional information for a display on a display unit (20), in particular a head-up display (HUD) of an observer vehicle (10) or a data glasses, wherein the Display of additional information for the purpose of

Assistance of the driver in the longitudinal and / or transverse guidance of a

Observer vehicle (10) is used, wherein the superimposed augmented reality augmented reality is calculated contact-analogous to one or more objects in the environment of the observer vehicle (10), wherein detects the situation of an oncoming or preceding vehicle or object (300) is characterized in that when approaching the

oncoming or preceding vehicle (300) a spatially extended animation graphic is calculated, the animation graphic having a screen form consisting of a plurality of screen elements (305) extending from the observer vehicle (10) to the oncoming or preceding vehicle (300), and the spatial extent is calculated such that the driver of the observer vehicle (10) has the impression of a kinematic or dynamic movement of the spatial extent, such as translation and rotation.

2. The method of claim 1, wherein the animation graphics is calculated so that the driver of the observer vehicle (10) gives the impression of a moving or moving away from him shaft.

3. The method of claim 2, wherein the animation graphic is calculated such that the spatial extent of the animation graphic is periodically repeated so that the driver of the observer vehicle (10) has the impression that a number of wave trains are approaching or leaving him move away.

4. The method according to any one of the preceding claims, wherein one or two spatially extended to support the side guide of the vehicle

Animation graphics are calculated, which are displayed on the side of the route, the further animation graphics having a grid shape consisting of a plurality of raster elements (305), and the spatial extent is calculated so that to the side where an obstacle or an oncoming vehicle detected the grid was placed spatially to emphasize a narrowing of the route.

5. The method of claim 4, wherein an estimate of the width of the bottleneck

is made and falls below a minimum width, an animation graphic is calculated so that the at least one grid-shaped animation graphics for lateral guidance of the vehicle into a hint symbol (28b) is converted, by which the need for an evasive maneuver is indicated.

6. The method of claim 5, wherein the conversion of the animation graphic is calculated so that the raster elements of the animation graphic in support of

Side guide move during the transformation phase swarming, from which at the end of the conversion phase, the hint symbol (28b) is formed.

7. The method of claim 6, wherein the further animation graphics for supporting the lateral guidance of the vehicle is calculated so as to determine the path of the vehicle

oncoming vehicle (300) or object marked.

8. Apparatus for carrying out the method according to one of the preceding

Claims, comprising a display unit (20) with which additional virtual information can be inserted into the field of vision of the driver or the operator of the object, and a computing unit (40), characterized in that the vehicle (10) comprises detection means (1 10, 150) which detect an oncoming vehicle (300), wherein the arithmetic unit (40) is adapted to the position of the

to calculate the oncoming vehicle (300) and the movement of the vehicle

To calculate an observer vehicle (10) relative to the preceding vehicle (300) and to compute a spatially extended animation graphic upon detection of an oncoming vehicle (300), the animation graphic having a grid shape consisting of a plurality of raster elements (305) different from the one Observer vehicle (10) to the oncoming or

leading vehicle (300) extends, and the spatial extent is calculated so that for the driver of the observer vehicle (10) the impression of a kinematic or dynamic movement of the spatial extent, such as

Translation and rotation arises.

The apparatus of claim 8, wherein the computing unit (40) is further configured to perform the calculations of any of claims 2 to 7.

10. Apparatus according to claim 8 or 9, wherein the display unit (20) is a head-up display (HUD) or a data glasses.

1 1. Motor vehicle, characterized in that the motor vehicle (10) comprises a device according to one of claims 8 to 10.

12. Computer program, characterized in that the computer program

is designed, when executed in a computing unit (40) perform the steps of the method for calculating an AR-insertion of additional information for a display on a display unit (20) according to one of claims 1 to 7.