DE102016003060A1 - Method for high-precision marking of vehicles ahead in augmented reality - Google Patents

Abstract

The invention relates to a method for high-precision marking of vehicles ahead in augmented reality, in which a coordinate (z _BV , z _AH ) of the preceding vehicle is determined by image processing and calculated by a distance measurement to the vehicle in front in conjunction with a height profile of the busy road becomes. In a method by means of which a highly accurate z-coordinate of the vehicle in front can be determined, the z-coordinate (Z _AH ) of the preceding vehicle, which is determined by the distance measurement in conjunction with the height profile, and which, independently of this, are obtained with the image obtained from the image processing Coordinate (z _BV ) of the preceding vehicle is mathematically fused to an exact z-coordinate (z _FUS ).

Description

The invention relates to a method for high-precision marking of vehicles ahead in augmented reality, in which a coordinate of the preceding vehicle is determined by image processing and is calculated by a distance measurement to the vehicle in front in conjunction with a height profile of the traveled road.

In vehicles, head-up displays are often used in which augmented reality is used for display. This expanded reality is to be understood below to mean a computer-aided expansion of the perception of reality, in that a visual representation in the display area is generated by additional information generated by means of overlaying and shifting. For the targeted insertion of this additional information, it is necessary that a preceding vehicle must be marked with high precision in the x and in the z direction, since it must be known exactly where the preceding vehicle is relative to the own vehicle. In addition, the width and height of the vehicle must be determined.

From the DE 10 2011 121 847 A1 For example, there is known a road condition warning method on a full windshield head-up display. In this method, the distance to a vehicle in front is detected with a camera system via the image recognition. Alternatively, this distance can also be determined by distance sensors, including radar and lidar sensors. The estimated values acquired by the image acquisition unit and by the distance sensors are monitored in a data merging module. By means of image acquisition, the x-coordinate can be determined very precisely, since there is a clear contrast between the vehicle and the background. The z-coordinate can be determined less precisely, because it is often not easy to recognize a lower edge of the vehicle by shadow effect. In addition, the determination of the z-coordinate is made difficult by the fact that the painted vehicle parts do not go down to the ground. In addition, unpainted bumpers can make detection difficult. Therefore, for precise calculation of the z-coordinate on the distance measurement additionally an accurate height profile of the road in front of the vehicle is necessary. This height profile is usually taken from a digital map, the accuracy of the digital map varies widely depending on the map material.

The object of the invention is to provide a method for high-precision marking of vehicles ahead by augmented reality, in which the z-coordinate is determined with high accuracy.

The invention results from the features of the independent claims. Advantageous developments and refinements are the subject of the dependent claims. Other features, applications and advantages of the invention will become apparent from the following description, as well as the explanation of embodiments of the invention, which are illustrated in the figures.

The object is achieved with a method in that the z coordinate of the preceding vehicle, which is determined via the distance measurement in conjunction with the height profile, and which independently of this are mathematically fused to the z coordinate of the preceding vehicle obtained from the image processing to form an exact z coordinate become. This has the advantage that the z-coordinate is determined much more precisely by the combination of these known methods. As a result, a much higher quality augmentation of vehicles ahead is achieved, resulting in a very high quality overall impression for the vehicle user.

Advantageously, the z-coordinate obtained from image processing is used for the fusing of the z-coordinate of the preceding vehicle, which is made plausible by the z-coordinate obtained from the distance measurement and the height profile of the road being traveled. In this case, the z-coordinate of the preceding vehicle, which was obtained from the image processing, is only based on augmented reality if no implausible values were determined.

In one embodiment, if the z-coordinate obtained from the image processing deviates from the z-coordinate calculated from the distance measurement and the height profile of the traveled road by a predetermined amount, the z-coordinate of the augmented from the dimensional measurement and the height profile of the traveled road Reality. If the distance between the two z-coordinates is too large, then it can be assumed that an implausible value was determined with the image processing, so that the z-coordinate calculated by the distance measurement and the height profile is used.

In one embodiment, the predetermined amount in the positive direction of the z-coordinate has a first value that differs from a second value in the negative direction of the z-coordinate. Therefore, the amount spanned by the z-coordinate determined by the image acquisition does not have to be symmetrical, but may be differentiated depending on the application.

In a variant, the z-coordinate obtained from the distance measurement and the height profile of the traveled road is used for augmented reality and plausibilized with the z-coordinate obtained from the image processing. The selection of which of the z-coordinates is used for the augmentation and which for the plausibility check is made according to the application.

In a refinement, in the determination of the z coordinate by the image processing and in the calculation of the z coordinate from the distance measurement and the height profile, in each case a reliability value is determined by which the independently determined z coordinates are weighted in the mathematical fusion. The reliability value is then taken into account in the fusing of the two z-coordinates, which increases the accuracy of the fused z-coordinate.

Advantageously, the mathematical fusion of the independently determined z-coordinates dynamically varies depending on the vehicle's own and / or vehicle-external parameters. The resulting characteristic of the z-coordinate can be influenced depending on road classes (highway or residential area), the road condition, the lighting conditions, in particular the image processing is influenced, the current speed, the weather and other factors. Depending on these parameters, the fused z-coordinate can be dynamically adjusted as a general function depending on the z-coordinate determined by the image acquisition and the z-coordinate calculated by the distance measurement and the elevation profile.

To sort out implausible values, the fused z-coordinate is used for the augmented reality if it changes continuously over a time course. If jumps occur at this fused z-coordinate, this can not be allowed for augmented reality.

A further development of the invention relates to a method for high-precision marking of vehicles driving ahead in augmented reality, in which a coordinate of the vehicle ahead is determined by image processing and is calculated by a distance measurement to the vehicle in front in conjunction with a height profile of a busy road. In a method in which a particularly accurate z-coordinate in the case of augmented reality is determined, the z-coordinate resulting from the distance measurement and the height profile is used as a known component of the z-coordinate of the image processing to be determined. The z-coordinate, which would normally be calculated by the distance measurement and the height difference, in this case applies as a-priori knowledge in image processing and is therefore an implicit component of the z-coordinate, which is determined by the image processing. Subsequent merging of the two z-coordinates can be omitted, which reduces the computational effort.

• – eine Kamera zur Aufnahme eines vorausfahrenden Fahrzeuges und eine Bildauswerteeinheit zur Ermittlung einer z-Koordinate des vorausfahrenden Fahrzeuges,
• – einen Abstandssensor zur Messung eines Abstandes des vorausfahrenden Fahrzeuges und eine, ein Höhenprofil der zu befahrenden Straße enthaltenden Abstandsberechnungseinheit zur Berechnung der z-Koordinate des Fahrzeuges aus dem Abstand und dem Höhenprofil,
• – eine mit der Bildauswerteeinheit und der Abstandsberechnungseinheit verbundene Fusionsrecheneinheit zur Bestimmung einer fusionierten z-Koordinate aus den unabhängig voneinander berechneten z-Koordinaten der Bildauswerteeinheit und der Abstandsberechnungseinheit, wobei
• – die Fusionsrecheneinheit mit einer Recheneinheit eines Head-up-Displays zur Verarbeitung der fusionierten z-Koordinate bei der Augmented Reality gekoppelt ist.

A further development of the invention relates to a vehicle comprising

• A camera for recording a vehicle ahead and an image evaluation unit for determining a z coordinate of the vehicle in front,
• A distance sensor for measuring a distance of the vehicle in front and a distance calculation unit containing a height profile of the road to be traveled for calculating the z-coordinate of the vehicle from the distance and the height profile,
• A fusion calculation unit connected to the image evaluation unit and the distance calculation unit for determining a fused z coordinate from the independently calculated z coordinates of the image evaluation unit and the distance calculation unit, wherein
• - The fusion computing unit is coupled to a computing unit of a head-up display for processing the fused z-coordinate in augmented reality.

Further advantages, features and details will become apparent from the following description in which - where appropriate, with reference to the drawings - at least one embodiment is described in detail. Described and / or illustrated features may form the subject of the invention itself or in any meaningful combination, optionally also independent of the claims, and in particular may also be the subject of one or more separate application / s. The same, similar and / or functionally identical parts are provided with the same reference numerals.

Show it:

1 an embodiment of the vehicle according to the invention,

2 an embodiment for marking a vehicle ahead in augmented reality,

3 an embodiment of the method according to the invention,

4 Example characteristics for the fused z-coordinate of the preceding vehicle,

5 a further embodiment of the method according to the invention.

In 1 is a vehicle 1 shown, which is a head-up display 3 that includes in the windshield 5 in a display area 7 visualizing a virtual object in the driver's field of vision, giving the driver the impression that the virtual object is actually part of reality. Such a head-up display 3 is called augmented reality head-up display. Contact-analogous contents of virtual objects are shown to the driver in his current view in such a way that he has the feeling that they are an integral part of the environment.

The head-up display 3 includes a microcontroller 9 , which with an image acquisition unit 11 connected to a camera 13 for image acquisition and an image evaluation unit 15 for determining x and z coordinates of a preceding vehicle from the captured image. Furthermore, the microcontroller 9 with a distance detection unit 17 connected, which is a distance sensor 19 which, in turn, includes a distance calculation unit 21 connected is. Between the image capture unit 11 and the distance detection unit 17 as well as the microcontroller 9 is a fusion calculator 23 arranged, the image evaluation unit 15 and the distance calculation unit 21 directly with this merger unit 23 are connected.

For high-precision marking, the detection of the vehicle ahead in the x-direction and in the z-direction is necessary. This is done using the camera 13 via the image evaluation unit 15 and using a distance sensor 19 For example, a radar or lidar sensor. At the same time, the coordinates of the preceding vehicle are determined by the distance calculation unit 21 determined from the data supplied by the distance sensor, in which an accurate height profile of the road is stored, on which the preceding vehicle is moving. The image capture unit 11 and the distance detection unit 17 determine the z-coordinate of the vehicle in front independently of each other.

As in 2 shown, the z-coordinate is determined based on the vehicle lower edge. From the distance and the height profile of the road so the z-coordinate can be calculated back in a video image, if the parameters of the video image such as focal length, optical axis and the like are known.

In order to eliminate the inaccuracies of the two methods for determining the z-coordinate, a method according to the invention is used, as described in US Pat 3 is described. In this case, the distance from the distance sensor 19 was detected (block 100 ) and the height profile taken from a digital map (block 110 ) for determining the z-coordinate of distance and height profile in the distance calculation unit 21 to the block 120 passed. At the same time, this is done by the camera 13 generated video image (block 130 ) on the block 140 for determining the z-coordinate from the image processing in the image evaluation unit 15 passed. Both the z-coordinate z _{BV of} the image processing and the z-coordinate z _AH from distance and height profile are sent to the fusion computing unit 23 in the block 150 passed on and mathematically linked together by means of geometric functions. As the starting function of the fusion computing unit 23 is based on the fused z-coordinate z _FUS of augmented reality.

In 4 two characteristic curves are shown in which the fused z-coordinate z _{FUS is shown} above the z-coordinate z _BV determined from the image processing. The z-coordinate z _AH determined from the distance and the height profile serves to check the plausibility of the z-coordinate z _BV obtained from image processing _. In this case, a predetermined range z _{AH + Δmax} ; z _{AH-Δmax spanned} . The determined by the image processing z-coordinate z _BV is with the range for _{AH-.delta..sub.max;} z _{AH + Δmax} compared. If the z coordinate z _BV deviates by more than a predetermined value Δ _max from z _AH , the value of the z coordinate z _AH is used instead of the value of the z coordinate z _BV in the augmented reality. If, however, the determined z coordinate z _BV falls within this range, only the z coordinate z _BV for augmented reality determined from the image processing is used. This procedure results in the 4 illustrated characteristic curve A. In a further expression, which is shown in the characteristic B, the fusion of the z-coordinates z _BV and z _{AH takes place} to the extent that the influence of z _{AH is} greater, the greater the deviation from the z Coordinate z _BV is.

In order to further increase the accuracy of the fused z-coordinate z _FUS , in the case of the separate determination of the z coordinates z _BV and z _AH , in each case additionally a reliability value is determined for the respective current measurement. This reliability value is then used in the formation of the fused z-coordinate z _FUS to weight the two individual measurements of the z coordinate z _BV and z _AH, respectively.

The characteristic curve can be dynamically varied in a further additional step. For example, the characteristic may be dependent on the road class (highway or residential area), the road condition, the lighting conditions, the current speed of the vehicle ahead or the weather. It can the Calculation of the fused z-coordinate z _{FUS is based on} a very general function as a function of the z-coordinates z _BV and z _AH , which may be adapted dynamically to different vehicle parameters, vehicle states, terrain characteristics from the digital map, road classes or countries. With all these possibilities, however, it must be assumed that the fused z-coordinate z _{FUS must} not make leaps, but must change continuously. In this case, the time course of the fused z-coordinate z _FUS over past measurements is considered in order to detect implausible values, which are accordingly weighted less in the calculation.

In 5 shows a second embodiment of the method according to the invention, in which the video image (block 130 ), which is currently measured, to the image evaluation unit 11 to determine the z-coordinate z _BV from the image processing is passed (block 140 ). In this case, the value of the z coordinate z _AH from height profile and distance is used as a priori knowledge in the image processing for determining the z coordinate z _BV . Thus, the z-coordinate z _BV determined by the image processing always forms the fused z-coordinate z _FUS .

Although the invention has been further illustrated and explained in detail by way of preferred embodiments, the invention is not limited by the disclosed examples, and other variations can be derived therefrom by those skilled in the art without departing from the scope of the invention. It is therefore clear that a multitude of possible variations exists. It is also to be understood that exemplified embodiments are really only examples that are not to be construed in any way as limiting the scope, applicability, or configuration of the invention. Rather, the foregoing description and description of the figures enable one skilled in the art to practice the exemplary embodiments, and those of skill in the knowledge of the disclosed inventive concept may make various changes, for example as to the function or arrangement of individual elements recited in an exemplary embodiment, without Protection area defined by the claims and their legal equivalents, such as further explanations in the description.

LIST OF REFERENCE NUMBERS

1

vehicle

3

Head-Up Display

5

Windshield

7

display area

9

microcontroller

11

Image capture unit

13

camera

15

image evaluation

17

Distance detection unit

19

distance sensor

21

Distance calculating unit

23

Merger processing unit

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• DE 102011121847 A1 [0003]

Claims (10)

Method for high-precision marking of vehicles ahead in augmented reality, in which a coordinate (z _BV , z _AH ) of the preceding vehicle is determined by image processing and is calculated by a distance measurement to the vehicle in front in conjunction with a height profile of the traveled road, characterized in that the z-coordinate (z _AH ) of the preceding vehicle determined by means of the distance measurement in conjunction with the height profile and the z coordinate (z _BV ) of the preceding vehicle, independently obtained from the image processing, lead to an exact z coordinate (z _FUS ) are mathematically fused. A method according to claim 1, characterized in that for the fusing of the z-coordinate (z _FUS ) of the preceding vehicle, the z-coordinate (z _BV ) obtained from the image processing is used, which by the distance measurement and the height profile of the traveled road obtained z-coordinate (z _AH) is checked for plausibility. Method according to claim 1 or 2, characterized in that in the case of a deviation of the z-coordinate (z _BV ) obtained from the image processing, the z-coordinate (z _AH ) calculated from the distance measurement and the height profile of the traveled road is _reduced by a predetermined amount ( Δmax) is based on the z-coordinate (z _AH ) of the augmented reality calculated from the distance measurement and the height profile of the traveled road. A method according to claim 3, characterized in that the predetermined amount (Δmax) in the positive direction of the z-coordinate has a first value which differs from a second value in the negative direction of the z-coordinate. A method according to claim 1, characterized in that the z-coordinate (z _AH ) calculated from the distance measurement and the height profile of the traveled road is used for augmented reality and is made plausible with the z-coordinate (z _BV ) obtained from the image processing. Method according to at least one of the preceding claims, characterized in that in the determination of the z-coordinate (z _BV ) by the image processing and in the calculation of the z-coordinate (z _AH ) from the distance measurement and the height profile in each case a reliability value is determined, by means of which the independently determined z-coordinates (z _BV , z _AH ) are weighted in the mathematical fusion. A method according to claim 1, characterized in that the mathematical fusion of independently determined z-coordinates (z _{BV ,} z _AH ) varies dynamically in dependence on the vehicle's own and / or vehicle external parameters. Method according to at least one of the preceding claims, characterized in that the fused Z-coordinate (z _FUS ) is used for augmented reality, if it changes continuously over a time course. A method for high-precision marking of vehicles traveling ahead in Augmented Reality in which a coordinate (z _BV, z _AH) is the preceding vehicle detected by image processing and is calculated by a distance measurement to the vehicle in front in connection with a height profile of the major road and, characterized in that the z-coordinate (z _AH ) resulting from the distance measurement and the height profile is used as a known component of the z-coordinate (z _BV ) to be determined for the image processing. Vehicle comprising - a camera ( 13 ) for receiving a preceding vehicle which has an image evaluation unit ( 15 ) for determining a z-coordinate (z _BV ) of the preceding vehicle, - a distance sensor ( 19 ) for measuring a distance of the vehicle ahead and a, a height profile of the busy road containing distance calculation unit ( 21 ) for calculating the z-coordinate (z _AH ) of the vehicle from the distance and the height profile, - one with the image evaluation unit ( 15 ) and the distance calculation unit ( 21 ) connected fusion computing unit ( 23 ) for determining a fused z-coordinate (z _FUS ) from the independently calculated z-coordinates (z _BV , z _AH ) of the image evaluation unit ( 15 ) and the distance calculation unit ( 21 ), wherein - the fusion computing unit ( 23 ) with a computing unit ( 9 ) of a head-up display ( 3 ) for processing the fused z-coordinate (z _FUS ) in Augmented Reality.

Images