DE102013219556A1 - Method and device for controlling an image generation device of a head-up display - Google Patents

Abstract

The invention relates to a method and a device for controlling an image generation device (2) of a head-up display in a vehicle (1), wherein a two-dimensional reference pattern (RM) is generated by means of the image generation device (2) and by means of a projection element of the head-up Displays are produced as a projected reference pattern (RM), wherein at least one image acquisition device (6) generates a plurality of two-dimensional images from mutually different layers and / or with mutually different orientations, the projected image being formed into each image of the plurality of two-dimensional images Reference pattern (RM) is at least partially imaged, in each case coordinates of at least one imaged element of the reference pattern (RM) are determined, wherein for each image, the corresponding position and / or orientation of the at least one image capture device (6) in a reference coordinate system wherein coordinates of the at least one imaged element are estimated at least as a function of at least one imaging parameter of a first imaging function and in dependence on a second imaging function, wherein the first imaging function is an image of a coordinate system (DK) of a virtual representation plane (7) of the Describing head-up displays in the reference coordinate system, wherein the second mapping function describes an image of the reference coordinate system in a coordinate system (BK) of the image capture device (6) in dependence on the position and / or orientation of the at least one image capture device (6), wherein the at least one imaging parameter of the first imaging function is determined such that a deviation between the determined and estimated coordinates of the imaged element is minimized, wherein an L depends on the at least one imaging parameter age and / or orientation of the virtual display plane (7) of the head-up display in the reference coordinate system is determined, wherein depending on the position and / or orientation of the virtual display plane (7), the control of the image generating device (2).

Description

The invention relates to a method and a device for controlling an image generation device of a head-up display in a vehicle.

With so-called augmented reality technologies, computer-generated two-dimensional or three-dimensional graphics can be embedded in a visual coverage area of an observer in the correct position and orientation.

Thus, e.g. known as head-up displays in a vehicle, which are used to show a vehicle driver virtual three-dimensional graphics in his field of view. These can assist him in driving the vehicle. However, in order for the generated graphics for the driver to appear in his field of vision in the correct position and orientation, the display has to be adapted to current framework conditions, in particular a geometry of the virtual image and a position of the viewer. In particular, it is necessary to consider a distortion pattern resulting from mirroring the virtual image on a non-planar windshield of the vehicle.

The US Pat. No. 7,835,592 B2 describes a method for adjusting an image projected onto a curved surface. The method includes detecting calibration points in a calibration image projected onto the curved surface. Furthermore, the method comprises the analysis of the calibration points in order to determine relative positions of the calibration points relative to one another.

The US 2008/0088526 A1 discloses a method of rendering an image incident on a non-planar surface. The document also describes that a new calibration is required if a viewing point of a viewer is changed.

This raises the technical problem of providing a method and a device for controlling an image generation device of a head-up display in a vehicle, which enable the most optimal position and orientation-correct display of virtual objects in a field of vision of a vehicle driver. Next, the technical problem of allowing the position and orientation correct display in response to a changing viewer's position.

The solution of the technical problem results from the objects with the features of claims 1 and 10. Further advantageous embodiments of the invention will become apparent from the dependent claims.

It is a basic idea of the invention to determine a position and / or orientation of a virtual image plane in a reference coordinate system in a calibration process, virtual objects being displayed in this virtual image plane to the vehicle driver. Depending on the position and / or orientation in the reference coordinate system, a position and orientation-correct representation of objects can then take place, wherein object coordinates are likewise known or at least determinable in the reference coordinate system. In particular, the virtual object to be displayed can be a three-dimensional virtual object. Another basic idea is to present the vehicle driver with a virtual object in such a way that the driver can perceive the virtual object without distortion from any viewing position.

A method is proposed for controlling an image generation device of a head-up display in a vehicle. The image generation device can also be referred to as a projector.

In this case, the head-up display denotes a system or a device for displaying in particular virtual objects, in particular virtual three-dimensional objects, in a field of view of a vehicle driver. The head-up display can also comprise means for beam guidance or deflection in addition to the image generation device, by means of which beams generated by the image generation device are guided or directed in a desired manner. Furthermore, the head-up display may also comprise a, in particular semitransparent, projection element, which may be e.g. forms a projection surface on which the image generated by the image forming device is projected. In this case, this projection element can be arranged in such a way that rays generated by the image generation device are reflected by the projection element in such a way that they can be detected by a driver after the projection.

By means of the image generating device, a two-dimensional reference pattern is generated. This two-dimensional reference pattern is represented by means of the projection element of the head-up display as a projected reference pattern. The two-dimensional reference pattern can thus be generated in the form of a two-dimensional image by the image generating device.

By means of at least one image capture device, a plurality of two-dimensional images of mutually different layers and / or mutually different orientations of the at least one image capture device is generated. This means that at least two, preferably one, sequence of more than two, two-dimensional image (s) is generated, wherein the position and / or orientation of the at least one image capture device is changed such that each image is generated with an image-specific position and / or orientation, which of image-specific layers and / or orientations of the image capture device in the generation of the further images of the sequence differs.

The image capture device can be designed in particular as a camera, for example as a webcam. Of course, several image capture devices can be used to generate the images. Preferably, however, the images are generated by exactly one image capture device.

Furthermore, the corresponding position and / or orientation of the at least one image acquisition device in a reference coordinate system is detected or determined for each image. For the purposes of the invention, detection means that a position and / or orientation is detected by means of corresponding detection devices, for example sensors. Thus, e.g. known attitude and / or orientation detection systems, such as optical, magnetic or mechanical detection systems, used to detect the location and / or orientation of the image capture device.

For the purposes of the invention, determining means that the position and / or orientation of the image capture device is not detected directly, but rather determined in particular as a function of information that is at least partially dependent on a position and / or orientation of the image capture device. In particular, a position and / or orientation of the image capture device can be determined as a function of image information of the images that were generated by the image capture device from the respective position and / or orientation. This will be explained in more detail below.

In each image of the plurality of two-dimensional images, the projected reference pattern is at least partially, preferably completely, imaged. This means that at least a part of the reference pattern is detected and imaged from different layers and / or with different orientations of the image capture device. Of course, it is conceivable that in the plurality of two-dimensional images individual images exist in which the projected reference pattern is not shown. However, these can not be taken into account in the evaluation described below.

Coordinates of at least one imaged element of the reference pattern are determined in each image. Thus, the two-dimensional reference pattern elements, in particular as a picture object or pixel trained reference elements included, which are designed such that an element-specific reference coordinate is determined as clearly as possible. Thus, the at least one element can be assigned a single reference coordinate, for example a coordinate of a geometric center, wherein the coordinate of the element can be determined as a function of this reference coordinate. Preferably, the reference coordinate corresponds to the coordinate of the at least one element to be determined according to the invention. The element may be formed, for example, circular or cross-shaped. Preferably, the element is designed in such a way that it is as simple and clearly image-based detectable. Unambiguous in this context means that there is an unambiguous assignment of an imaged element to an actual element in the reference pattern. Such a unique association may e.g. also be determined depending on a predetermined arrangement and / or design of elements in the reference pattern. Thus, the reference pattern or elements of the reference pattern may be e.g. Have color or brightness properties or gradients that allow a unique assignment.

More preferably, the element is rotationally invariant. Also, the element may have a unique color coding. Furthermore, the element can be designed in such a way that it can be detected image-based as independent as possible from a focus or resolution of a display device. Another example of an element may be a blurred circular shape. Also, an element may be a potato-like, e.g. oval shape.

It is also conceivable that the elements are provided by a checkerboard pattern forming the reference pattern, in which e.g. Intersections of edges between white and black boxes of the checkerboard pattern or points on these edges which can provide elements. The use of such checkerboard patterns and the identification of elements or points of the checkerboard pattern is known to those skilled in the field of camera calibration, in particular for the determination of intrinsic parameters of a camera.

In order to enable the above-described unique assignment, at least one or two panels of the checkerboard pattern may be color-configured with a color other than black and white.

The determination of the coordinates is image-based. The coordinates of the imaged element are in this case determined in a (two-dimensional) coordinate system of the image capture device, which can also be referred to as an image coordinate system.

Furthermore, coordinates of the at least one imaged element are estimated at least as a function of at least one imaging parameter of a first imaging function and as a function of a second imaging function. The coordinates can also be estimated in the coordinate system of the image capture device. The first mapping function describes a mapping from a coordinate system of a virtual representation plane, which can also be described as a plane coordinate system, of the head-up display into the reference coordinate system. Image parameters of the first mapping function can in particular describe a rotation, a translation and / or a scaling, in particular a double scaling, in the image. A two-fold scaling can be a heterogeneous scaling with two scaling parameters which describe a scaling along two orthogonal axes, for example a scaling in the horizontal and vertical image direction.

In this case, the virtual representation plane denotes a plane in which the images generated by the image generation device appear to the viewer, in particular the vehicle driver. This virtual representation level does not necessarily have to correspond to the level of the previously explained projection element.

The second mapping function describes a mapping from the reference coordinate system into the coordinate system of the image capture device as a function of the position and / or orientation of the at least one image capture device. The second mapping function takes into account intrinsic and / or extrinsic imaging parameters of the image capture device. Extrinsic and intrinsic imaging parameters of the image capture device in this case describe parameters of the image capture device that influence the imaging of objects in the capture area into the two-dimensional image. In this case, the extrinsic imaging parameters describe the position and / or orientation of the image capture device. These are determined or captured for each image, as previously explained.

The intrinsic imaging parameters can be determined, for example, in a separately performed calibration step. For example, intrinsic imaging parameters include all characteristics that describe the optics of the camera, e.g. Focal length, center of image, image shear, nonlinear lens distortion, and more.

Thus, each of the multiple images can be generated with the same intrinsic parameters but with different extrinsic parameters.

Further, the at least one imaging parameter of the first imaging function will be determined such that a deviation between the determined and estimated coordinates of the imaged element is minimized. Thus, therefore, the determination of the at least one parameter of the first mapping function takes place in the context of a minimization, wherein a cost function is dependent on the difference between the determined and estimated coordinates. For example, the cost function may be described as a distance or squared difference between the determined and estimated coordinates.

Preferably, the coordinates of a plurality of, preferably all imaged, elements of the reference pattern are determined and estimated in each image or for each image. Accordingly, the at least one mapping parameter of the first mapping function may then be determined such that a sum of the deviation between the determined and estimated coordinates for all elements whose coordinates have been determined and estimated and minimized for all the images under consideration.

Depending on the at least one imaging parameter, a position and / or orientation of the virtual representation plane of the head-up display in the reference coordinate system is then determined. Figuratively speaking, the virtual presentation plane represents a window through which the viewer, in particular the driver, looks out of his viewing position into a virtual world. Objects that are present in the virtual world are thus captured by the viewer through the virtual display plane. The virtual world can hereby be referenced in the reference coordinate system. This means that coordinates of objects of the virtual world in the reference coordinate system are known or determinable. Thus, it can advantageously be determined as a function of the position and / or orientation of the virtual display plane which objects of the virtual world of the viewer can perceive from a certain viewing position. Furthermore, it can be determined how objects of the virtual world have to be displayed by the image generation device so that they appear to be perspectively correct from the viewpoint of the viewer.

For example, virtual world objects may be colored virtual markers of actual road markings. Thus, it may be desirable, for example, to additionally mark colored side or center lane markings for better visual capture by the vehicle driver. For example, these actual lane markings can be detected by a suitable detection system, for example an optical detection system, and a position and orientation of the actual lane marking in the reference coordinate system can be determined. Depending on this position and orientation, the image generation device of the head-up display can then be controlled in such a way that an additional colored marking of the actual road marking is displayed in the virtual representation plane in the correct position and orientation.

Depending on the position and / or orientation of the virtual display plane, the control of the image generation device then takes place. Thus, the method according to the invention forms a method for calibrating the head-up display, in particular for calibrating the image-generating device.

In particular, depending on the position and / or orientation of the virtual presentation plane, the activation of the image generation device can take place in such a way that a positionally correct and / or orientation-correct display of, in particular, virtual objects takes place in the virtual presentation plane. In particular, depending on the position and / or orientation of the virtual display plane in the reference coordinate system, it can be determined on which (two-dimensional) image coordinate (s) of the image generated by the image generation device a virtual object to be displayed with coordinates known in the reference coordinate system must be displayed explained to achieve position and / or orientation-correct representation in the virtual representation level. Thus, for example, the calibration serves to determine parameters for generating the image of the image generation device.

As explained above, the proposed method thus advantageously enables the best possible positional and / or orientation-correct representation of objects in the virtual display plane of a head-up display in the vehicle.

In a preferred embodiment, the reference coordinate system is a vehicle coordinate system. This advantageously makes it possible, in particular, to display virtual objects that are easy to implement, in particular computationally simple, since these are generally known or determined in the vehicle-specific vehicle coordinate system.

In a further embodiment, at least one part of the vehicle, in particular of the vehicle interior, is imaged in each image of the plurality of two-dimensional images. The position and / or orientation of the image acquisition device in the reference coordinate system is then determined as a function of the imaged part of the vehicle and a model of the vehicle referenced in the reference coordinate system. The model referenced in the reference coordinate system can be, for example, a CAD model of the vehicle, in particular of the vehicle interior.

Thus, for example, mutually corresponding regions or elements can be determined in an image generated by the image capture device and in the model. Since coordinates of regions or elements of the model in the reference coordinate system are known, a position and / or orientation of the image capture device can be determined as a function of these coordinates and the previously explained, in particular intrinsic, imaging parameters of the image capture device. This can e.g. depending on a known or determinable mapping rule of an image of objects arranged in the detection range of the image capture device into a two-dimensional image captured by the image capture device.

In particular, so-called landmarks that correspond to one another can be identified both in the images generated by the image capture device and in the model. Landmarks here designate elements or objects that are image-based simple and preferably uniquely identified. For example, landmarks can be formed by ventilation grilles. Landmarks can also be formed by edges and corners in the model, which are also visible in the image. These may e.g. of operating and / or ornamental elements, such as Buttons, ledges, levers are formed. Also, on a further display device of the vehicle, for example on a multifunction display, which is also used by a navigation system, a previously known pattern can be displayed, which provides at least one landmark. It is also possible to arrange artificial patterns or markers in the vehicle whose position and / or orientation is measured in advance, for example likewise camera-based, and then integrated into the model. Such patterns or markers may e.g. be mounted on or on a windshield. Thus, therefore, an image-based determination of the position and / or orientation of the image capture device is proposed.

In a further embodiment, a further mapping rule is additionally determined, the further mapping rule describing a mapping from the coordinate system of the virtual representation plane into the coordinate system of the image capture device. This further mapping rule can also be described as a so-called 2D-2D transformation.

The further mapping rule can be described for an image as a function of the previously explained at least one imaging parameter of the first mapping rule, which is now known, the previously explained, in particular intrinsic, imaging parameters of the image capture device and the position and / or orientation of the image capture device in the generation of the image ,

This advantageously allows an immediate transformation between the previously explained coordinate system, which, as explained in more detail below, can be used to determine a distortion error.

In a further embodiment, an aberration of at least one element, preferably of all elements, of the reference pattern is determined for a plurality, that is at least two, preferably for each, image (s). The aberration here denotes a deviation of a position of the actually detected element of the reference pattern from a theoretically correct position of the element of the reference pattern from the viewpoint of the viewer. In this case, the viewer can be in particular a motor vehicle driver or user. The aberration is based e.g. Distortion effects, in particular by a curved windshield of the vehicle, and / or the gain optics of the head-up display. The distortion here is usually contingent position-dependent and dependent on a position of the element in the virtual representation plane (and thus also a position of the element in the previously explained image plane of the projection element). In other words, this means that the aberration may be different for each point in the virtual presentation plane and for each viewing position.

For example, the aberration may be calculated as the difference or distance between image-based determined actual coordinates of at least one element of the reference pattern and theoretically correct coordinates of the element of the reference pattern. Theoretically correct coordinates can designate coordinates of the element in an image without aberrations, ie an unaltered image.

To determine the aberration, the actual and theoretically correct coordinates of the element of the reference pattern can preferably be determined in a coordinate system of the virtual representation plane. However, it is also possible to determine these coordinates in another coordinate system, for example in a coordinate system of the image capture device.

For example, a coordinate of an imaged element of the reference pattern can be determined in the image. This coordinate can then be transformed into the coordinate system of the virtual presentation plane by means of the previously explained further mapping rule. This results in an estimated coordinate of the imaged element in the coordinate system of the virtual presentation plane. For example, Depending on previously known (geometric) properties of the reference pattern, a theoretically correct coordinate of the imaged element can then be determined.

Furthermore, depending on the position of the element and a position and / or orientation of the image generating device in the production of the image, a relationship between the aberration, a viewing position and a position of an element to be displayed is determined. The position of the element or of the element to be displayed can be specified, for example, as the coordinate of the element in the coordinate system of the virtual representation plane.

The relationship determined in this way makes it possible, at run-time, depending on the position of an element to be displayed and a viewing position, to control the image-generating device in such a way that the aberration is minimized. Of course, the control of the image generating device can also take place as a function of a viewing direction such that the aberration is minimized. The viewing direction may, for example, be determined as the direction of a vector from the viewing position to a midpoint of the virtual display plane. Of course, an alternative determination of the viewing direction is possible.

For example, a coordinate of an element to be displayed in the image generated by the image generator can be changed in such a way that the aberration is minimized. In summary, therefore, a control of the image generating device is made possible such that depending on a viewing position and optionally a viewing direction Minimized aberrations and thus a possible distortion-free representation takes place.

Thus, a viewing position and observer orientation-dependent correction of the aberration is made possible at runtime without recalibration.

By controlling the image generating device such that the aberration is minimized, there is thus a viewing position and position-dependent equalization of the element to be displayed in the virtual representation plane.

For the proposed method, it may be necessary to detect or determine the viewing position and possibly also a viewing direction of the viewer, in particular of the vehicle driver. For example, a head position and / or head orientation of the observer can be detected by a suitable detection system, for example an optical detection system, wherein depending on the head position and / or head orientation a viewing position and optionally a line of sight of the observer in the reference coordinate system can be determined.

Alternatively, it is also possible to computationally determine an equalization model from design data of the head-up display and of the vehicle. For this purpose, for example, parameters of the head-up display and of the vehicle, such as e.g. a mirror curvature of the projection element of the head-up display and a disc curvature of the windshield of the vehicle, are taken into account. This thus enables a model-based control of the image generation device such that the imaging error is minimized depending on the viewing position and the coordinate of the at least one element. However, deviations due to manufacturing tolerances and installation errors can not be detected in such a mathematical determination. However, these can be determined advantageously by the proposed method.

Thus, advantageously, the number of images generated is used to enable image-based equalization in later use of the head-up display.

This in turn makes it possible to generate a distortion-free image at any viewing position, which is particularly advantageous when the viewer changes his viewing position, in particular changes it continuously.

In a further embodiment, a relationship between the viewing position, the position of the element to be displayed and the aberration present in the respective viewing position is determined as a parameterized model, wherein the parameters of the parametric model are determined such that the aberration for a plurality of layers of the image capture device and / or a plurality of elements of the reference pattern is minimal. Here, the position of the image capture device corresponds to a possible viewing position of a viewer.

For example, the parameters may be determined such that the sum of squared aberrations for a plurality of layers of the image capture device and / or for a plurality of elements of the reference pattern is minimal. Of course, however, other cost functions can be used for optimization.

The proposed method makes it possible in an advantageous manner for a calculation of the aberration that is as fast as possible in terms of time to be possible and thus also the fastest possible adaptation of the control of the image generation device to take place as quickly as possible. In other words, this means that an equalization pattern, which describes the aberrations for a particular viewing position for a plurality or all pixels of the image displayed in the virtual representation plane, can be generated as a function of the parameterized model. Depending on the equalization pattern, the image generation device can in turn be controlled in such a way that the aberration is minimized.

In a preferred embodiment, the parameterized model is given by a polynomial having a predetermined number of independent variables. In particular, the variables may be the coordinates of the viewing position (x-position, y-position, z-position of the head) as well as a horizontal coordinate and a vertical coordinate of the considered pixel in the virtual image plane.

This degree is then predetermined at runtime, but arbitrary at the time of calibration.

However, the degree of the polynomial can be arbitrary. The higher it is, the better the nonlinearity of the distortion can be mapped. However, then more measurements are required in order to determine all coefficients with sufficient accuracy.

The use of polynomials advantageously results in an approximation of the nonlinear distortion with a desired one Accuracy. Furthermore, polynomials can be evaluated with little computational effort and are therefore computationally efficient.

In a further embodiment, the aberration is determined as a coordinate difference between theoretically correct coordinates of an element of the reference pattern in the virtual representation plane in a coordinate system of the virtual representation plane and coordinates of a corresponding element transformed from the image of the image acquisition device into the virtual representation plane. This has been explained previously. The coordinate difference can also be determined as the distance between a theoretically correct position of the element and the position of a transformed element.

The transformation from the image acquisition device's image into the virtual representation plane hereby designates a calculation of coordinates of a transformed pixel in the coordinate system of the virtual representation plane as a function of coordinates of a pixel in the coordinate system of the image acquisition device. The pixel in the coordinate system of the image acquisition device and the pixel transformed into the coordinate system of the virtual representation plane correspond to one another here. The transformation can be described in particular by the previously explained further mapping rule.

This allows the simplest possible determination of the aberration.

In a further embodiment, the reference pattern has a plurality of blurred drops arranged in a matrix form. Blurred drops here denote circular elements which, however, do not have a clearly identifiable object edge. A uniquely identifiable edge is given, for example, if an amount of an intensity difference between pixels of the object and pixels outside the object is greater than a predetermined threshold, or if an amount of intensity gradient between two adjacent pixels or between two pixels spaced apart by a predetermined distance is greater than one is predetermined threshold.

In matrix form, this means that centers of the blurred drops are arranged in a horizontal and vertical image direction with each other or side by side.

This advantageously makes possible the simplest possible image-based determination of a reference coordinate of an element of the reference pattern, the reference coordinate in particular corresponding to a center point of a blurred drop.

Further proposed is a device for controlling an image generation device of a head-up display in a vehicle. The device comprises at least one control device.

By means of an image generating device, a two-dimensional reference pattern can be generated and represented by means of a projection element of the head-up display as a projected reference pattern. In this case, the image generating device and the projection element can be part of the proposed device.

Furthermore, a plurality of two-dimensional images with mutually different layers and / or orientations of the image acquisition device can be generated by means of at least one image acquisition device and the corresponding position and / or orientation of the at least one image acquisition device can be detected or determined for each image in a reference coordinate system. In this case, the at least one image capture device may be part of the proposed device.

Furthermore, in each image of the plurality of two-dimensional images, the projected reference pattern can be at least partially imaged.

Furthermore, coordinates of at least one imaged element of the reference pattern can be determined by means of the control device or a further control and evaluation device in an image. Furthermore, coordinates of the at least one imaged element can be estimated at least as a function of at least one imaging parameter of a first imaging function and as a function of a second imaging function, wherein the first imaging function describes an image of a coordinate system of a virtual representation plane of the head-up display in the reference coordinate system the second mapping function describes an image from the reference coordinate system into a coordinate system of the image capture device as a function of the position and / or orientation of the at least one image capture device.

The at least one imaging parameter of the first imaging function can be determined in such a way that a deviation, in particular a distance, between the determined and estimated coordinates of the imaged element is minimized. Furthermore, as a function of the at least one imaging parameter, a position and / or orientation of the virtual representation plane of the head-up display in the reference coordinate system can be determined.

The further control and evaluation device may also be part of the proposed device.

By means of the control device, the control of the image generation device takes place as a function of the position and / or orientation of the virtual presentation plane.

By means of the proposed device, one of the previously explained methods can be carried out in an advantageous manner. Thus, the proposed device may in particular be designed such that one of the previously explained methods by means of the device is feasible.

In a further embodiment, a relationship between an aberration, a viewing position and a position of an element to be displayed can be determined. This can be done, for example, as explained above, by determining an aberration of at least one element of the reference pattern for several, in particular each, image, the relationship between a dependence of a position of the element and a position and / or orientation of the image generation device the aberration, a viewing position and a position of an element to be displayed is determined.

Then, in particular at a later point in time, in particular at runtime, the aberration can be determined by means of the control device as a function of the position of an element to be displayed and the viewing position, the control of the image generating device taking place as a function of the aberration. The control is in particular such that an aberration is minimized.

This advantageously makes possible a distortion-free representation for a plurality of viewing positions.

The invention will be explained in more detail with reference to an embodiment. The figures show:

1 a schematic side view of a vehicle with a device for controlling an image forming device,

2 a perspective view of a vehicle and

3 an exemplary reference pattern.

Hereinafter, like reference numerals designate elements having the same or similar technical features.

In 1 is a schematic side view of a vehicle 1 with a device according to the invention for controlling an image generating device 2 a head-up display in the vehicle 1 shown. The device comprises a control device 3 for controlling the image forming device 2 , The image forming device 2 generated in response to control signals of the control device 3 an image pointing to a curved, semitransparent mirror 4 is projected. The mirror 4 reflects light rays 5 coming from the imaging device 2 are generated in a detection area of an image capture device 6 , In 1 is the mirror 4 shown as a separate component. Of course, the mirror 4 However, also be part of the windshield or be formed by this. The control device 3 controls the image generating device 2 such that an image object is im in the from the image forming device 2 generated image for a viewer, not shown, in particular a driver, in a virtual display plane 7 appears.

Also shown is a vehicle coordinate system FK which is stationary with respect to the vehicle 1 is arranged. An origin of the vehicle coordinate system FK may be, for example, a center of a front axle of the vehicle 1 be.

Also shown is a coordinate system BK of the image capture device 6 , In a two-dimensional image, which of the image capture device 6 is generated, mapped objects, for example, in this coordinate system BK of the image capture device 6 be referenced. Also shown is a coordinate system DK of the virtual representation level. Objects in the virtual presentation layer 7 can be represented, for example, in this coordinate system BK of the virtual representation level 7 be referenced.

It is an object of the invention, the location and orientation of the virtual presentation plane 7 in the vehicle coordinate system FK.

For this purpose, the image generating device generates 2 a two-dimensional reference pattern RM, which, for example, in 3 is shown. This appears in the virtual presentation layer 7 , Next, the image capture device generates 6 from different layers and with different orientations in the vehicle coordinate system FK images of the virtual representation plane 7 in which the projected reference pattern RM appears.

At the same time, in the images of the plurality of images, elements of the vehicle, in particular of the vehicle interior, for example a steering wheel 8th and a ventilation grille 9 , pictured. A position and possibly an orientation of the steering wheel 8th and the ventilation grille 9 in the vehicle coordinate system FK is already known, for example, from a referenced in the vehicle coordinate system FK CAD model.

Furthermore, an imaging function of the image capture device 6 be known, the an image of objects in the detection range of the image capture device 6 in a two-dimensional image of the image capture device 6 describes. This can, for example, in dependence on the previously explained imaging parameters of the image capture device 6 in particular intrinsic imaging parameters, for example by a corresponding calibration of the image capture device 6 be determined. In particular, the mapping rule of the image capture device 6 be described as a perspective projection.

Depending on the known in the vehicle coordinate system FK coordinates of the steering wheel 8th and the ventilation grille 9 as well as in dependence of the mapping regulation of the image capture device 6 can then be a location and / or orientation of the image capture device 6 be determined in the vehicle coordinate system FK, from / with the image capture device 6 has generated the corresponding image.

Is a location and orientation of the image capture device 6 As is known, depending on the previously explained mapping rule of the image capture device 6 also a mapping rule or transformation of the vehicle coordinate system FK in the coordinate system BK of the image capture device 6 or from the coordinate system BK of the image capture device 6 be determined in the vehicle coordinate system FK.

We are looking for a mapping rule from the coordinate system DK of the virtual representation plane 7 in the vehicle coordinate system FK.

For example, elements of the virtual representation layer can be used 7 appearing or in the virtual presentation level 7 mapped reference pattern RM normalized coordinates in the coordinate system DK of the virtual representation plane 7 be assigned. The sought-after mapping rule then describes the transformation of the coordinate system DK of the virtual representation plane 7 in the vehicle coordinate system FK. The sought-after mapping rule can be described, for example, in the form of a scaled Euclidean transformation which corresponds to each point in the reference pattern RM, that is to say in the reference image, in the virtual representation plane 7 assigns a point in the vehicle coordinate system FK.

Further, it can be assumed that also the points of the virtual representation plane 7 by the previously explained mapping rule of the image capture device 6 in a two-dimensional image of the image capture device 6 be imaged.

In the image of the image capture device 6 can image coordinates of elements of the reference pattern RM in the coordinate system BK of the image capture device 6 be determined image-based. These can also be referred to as detected coordinates. Furthermore, in dependence on the mapping rule known for a mapping from the vehicle coordinate system FK into the coordinate system BK of the image capture device, as explained above 6 and the desired mapping rule of the coordinate system DK of the virtual representation plane 7 in the vehicle coordinate system FK calculated image coordinates of elements of the reference pattern RM in the coordinate system BK of the image capture device 6 be determined.

The parameters of the desired mapping rule from the coordinate system DK of the virtual representation plane 7 in the vehicle coordinate system FK can be determined in particular as parameters that minimize the sum of the squared difference or the distance between the image-based determined and calculated image coordinates of the elements of the reference pattern RM.

Thus, in an advantageous manner, parameters of the sought mapping rule from the coordinate system DK of the virtual representation plane 7 be determined in the vehicle coordinate system FK. This then also results in a location and orientation of the virtual presentation level 7 in the vehicle coordinate system FK.

The coordinate system DK of the virtual representation plane 7 For example, it may be constructed such that the four outer vertices x _TL , x _TR , x _BL , x _{BR of} the reference pattern RM (see FIG 3 ) the (nominated) coordinates (-1, 1), (1, 1), (-1, -1), (1, -1) in this coordinate system DK are assigned. To simplify calculations in a transformation, a third coordinate of a point in the virtual representation plane may be taken as a predetermined value, for example 1. Thus, the coordinate system DK of the virtual representation level 7 a by the reference pattern RM, in particular a geometric structure of the reference pattern RM, uniquely determined coordinate system.

Further, for each location and orientation of the image capture device 6 an aberration can be determined. The aberration is dependent on the viewing position, the position of the image capture device 6 when generating an image corresponds. Furthermore, the aberration is dependent on a coordinate of a considered point in the virtual representation plane 7 ,

The aberration can thus as a coordinate difference or distance between the image coordinate of an image of the image capture device 6 imaged and in the coordinate system DK of the virtual representation level 7 transformed pixel and a theoretically correct image coordinate of this pixel in the coordinate system DK of the virtual representation plane 7 be determined.

For example, it may be assumed that a left upper point x _TL in the coordinate system DK of the virtual representation plane 7 has the theoretically correct image coordinates (-1). In the two-dimensional image of the image capture device 6 then the corresponding, depicted upper left vertex can be identified image-based, whereby also its image coordinates in the coordinate system BK of the image capture device 6 be determined. These coordinates can then be determined from the coordinate system BK of the image capture device by means of the corresponding mapping rule that was previously determined 6 in the coordinate system DK of the virtual representation level 7 be transformed.

In particular, this aberration can be described as a polynomial with five independent variables, where coefficients of these variables of the polynomial are determined such that, for example, a squared aberration or a distance for all elements of the reference pattern over all images generated by the imaging device 6 is minimized. The variables include the three spatial coordinates of the viewer's head position and the two image coordinates of the viewed point in the virtual presentation plane 7 in the coordinate system DK of the virtual representation plane 7 ,

The parameters determined in this way can then be used at runtime in order to calculate the aberration for a specific viewing position and a specific pixel in the virtual representation plane in a simple and timely manner 7 to enable. Then turn, the control device 3 the image forming device 2 such that the calculated aberration is minimized. 2 shows a perspective view of a vehicle 1 with the different coordinate systems. Thus, a vehicle coordinate system FK is shown. Next is a coordinate system BK of an image capture device 6 (please refer 1 ).

Also shown is a coordinate system DK of the virtual representation plane 7 , Also shown are corner points x _BL , x _TL , x _TR , x _BR , each corner points of a reference pattern RM (see 3 ). These corner points x _TL , x _TR , x _BL , x _BR have the previously explained coordinates in the coordinate system DK of the virtual representation plane 7 on.

In 3 exemplarily a reference pattern RM is shown, which in the virtual representation level 7 is projected. Likewise, a corresponding coordinate system DK of the virtual representation level 7 shown. Also shown are the corner points x _TL , x _TR , x _BL , x _BR . The reference pattern RM comprises or contains a number of blurred drops 10 , wherein for the sake of simplicity only one drop with a reference numeral 10 is provided. Blurred here means that an object boundary of the blurred drop 10 is not clearly identifiable. This is indicated by a dotted circle around the blurred drops 10 indicated around. A c forms an element of the reference pattern RM.

A coordinate of such an element corresponds to a coordinate of a geometric center of a blurred drop 10 , A coordinate of an imaged blurred drop 10 in the coordinate system BK of the image capture device 6 For example, by minimizing a difference between a coordinate-dependent actual intensity curve and an assumed Gaussian intensity curve, a position of a maximum value of the Gaussian distribution is varied until the difference explained above is minimal. The location of the center then corresponds to the location of the maximum value, which minimizes the difference explained above. This advantageously makes it possible to determine the positions of the centers with subpixel accuracy.

For this purpose, a rough detection of an element can be carried out in a first step. In a further step, the previously explained determination of the position can then take place only in the area determined in the coarse detection.

Are the coordinates blurred drops 10 , Thus, the elements of the reference pattern RM, so may, depending on the known geometric properties of the reference pattern RM, for example, the distances between the coordinates of the blurred drops 10 and their distances to the vertices x _TL , x _TR , x _BL , x _BR , the coordinates of the vertices x _TL , x _TR , x _BL , x _BR be determined. If these coordinates of the corner points _x.sub.TL , _x.sub.TR , _x.sub.BL , _{x.sub.BR are} determined, then a size of the virtual representation plane can also be determined 7 be determined.

Overall, this results in a method and a device for controlling an image forming device 2 a head-up display in a vehicle 1 , which allows a calibration of the head-up display without prior knowledge. In particular, it is thus possible to compensate for errors that arise due to manufacturing and production inaccuracies.

LIST OF REFERENCE NUMBERS

1

vehicle

2

Imaging device

3

control device

4

mirror

5

beam of light

6

Image capture device

7

virtual presentation layer

8th

steering wheel

9

ventilation grille

10

blurred drops

DK

Coordinate system of the virtual representation layer

BK

Coordinate system of the image capture device

FK

Vehicle coordinate system

x _TL

vertex

x _TR

vertex

x _BL

vertex

x _BR

vertex

RM

reference pattern

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• US 7835592 B2 [0004]
• US 2008/0088526 A1 [0005]

Claims (11)

Method for controlling an image forming device ( 2 ) of a head-up display in a vehicle ( 1 ), whereby by means of the image generating device ( 2 ) generates a two-dimensional reference pattern (RM) and is represented by means of a projection element of the head-up display as a projected reference pattern (RM), wherein by means of at least one image capture device ( 6 ) a plurality of two-dimensional images of mutually different layers and / or mutually different orientations is generated, wherein in each image of the plurality of two-dimensional images, the projected reference pattern (RM) is at least partially mapped, wherein in an image in each case coordinates of at least one imaged Elements of the reference pattern (RM) are determined, characterized in that for each image, the corresponding position and / or orientation of the at least one image capture device ( 6 ) is detected or determined in a reference coordinate system, where coordinates of the at least one imaged element are estimated at least as a function of at least one imaging parameter of a first imaging function and in dependence on a second imaging function, wherein the first imaging function is an image of a coordinate system (DK) of a virtual representation plane ( 7 ) of the head-up display into the reference coordinate system, wherein the second mapping function maps an image from the reference coordinate system into a coordinate system (BK) of the image capture device (FIG. 6 ) as a function of the position and / or orientation of the at least one image capture device ( 6 ), wherein the at least one imaging parameter of the first imaging function is determined in such a way that a deviation between the determined and estimated coordinates of the imaged element is minimized, a position and / or orientation of the virtual imaging plane depending on the at least one imaging parameter ( 7 ) of the head-up display is determined in the reference coordinate system, wherein, depending on the position and / or orientation of the virtual representation plane ( 7 ) the control of the image generating device ( 2 ) he follows. A method according to claim 1, characterized in that the reference coordinate system is a vehicle coordinate system (FK). Method according to one of claims 1 or 2, characterized in that in each image of the plurality of two-dimensional images at least a part of the vehicle ( 1 ), wherein the position and / or orientation of the image capture device ( 6 ) depending on the imaged part of the vehicle ( 1 ) and a model of the vehicle referenced in the reference coordinate system ( 1 ) is determined. Method according to one of claims 1 to 3, characterized in that in addition a further mapping rule is determined, wherein the further mapping rule is an image of the coordinate system (DK) of the virtual representation plane ( 7 ) in the coordinate system (BK) of the image capture device ( 6 ) describes. Method according to one of claims 1 to 4, characterized in that for a plurality of images an aberration of at least one element of the reference pattern (RM) is determined, wherein, depending on a position of the element and a position and / or orientation of the image generating device ( 6 ) a relationship between the aberration, a viewing position and a position of an element to be displayed is determined. Method according to Claim 5, characterized in that the relationship between the viewing position, the position of the element to be displayed and the aberration present in the respective viewing position is determined as a parameterized model, the parameters of the parametric model being determined such that the aberration for a plurality of layers of the image capture device ( 6 ) and / or a plurality of elements of the reference pattern (RM) is minimal. A method according to claim 6, characterized in that the parameterized model is given by a polynomial having a predetermined number of independent variables. Method according to one of claims 5 to 7, characterized in that the aberration as a coordinate difference between theoretically correct coordinates of an element of the reference pattern (RM) in the virtual representation plane ( 7 ) in a coordinate system of the virtual representation plane ( 7 ) and coordinates from the image of the image capture device ( 6 ) into the virtual presentation level ( 7 ) transformed corresponding element. Method according to one of claims 1 to 8, characterized in that the reference pattern (RM) a plurality of arranged in a matrix form blurred drops ( 10 ) having. Device for controlling an image forming device ( 2 ) of a head-up display in a vehicle ( 1 ), wherein the device has at least one control device ( 3 ), wherein by means of an image generating device ( 2 ) Can be generated and by means of a two-dimensional reference pattern (RM) a projection element of the head-up display can be represented as a projected reference pattern (RM), wherein by means of at least one image capture device ( 6 ) a plurality of two-dimensional images with mutually different layers and / or orientations of the image capture device ( 6 ) is producible, wherein in each image of the plurality of two-dimensional images, the projected reference pattern (RM) is at least partially imageable, wherein by means of the control device ( 3 ) or another control and evaluation device in an image coordinates of at least one imaged element of the reference pattern (RM) can be determined, characterized in that for each image, the corresponding position and / or orientation of the at least one image capture device ( 6 ) can be detected or determined in a reference coordinate system, wherein coordinates of the at least one imaged element can be estimated at least as a function of at least one imaging parameter of a first imaging function and as a function of a second imaging function, wherein the first imaging function is an image of a coordinate system (DK) of a virtual representation plane ( 7 ) of the head-up display into the reference coordinate system, wherein the second mapping function maps an image from the reference coordinate system into a coordinate system (BK) of the image capture device (FIG. 6 ) as a function of the position and / or orientation of the at least one image capture device ( 6 ), wherein the at least one imaging parameter of the first imaging function is determinable such that a deviation between the determined and estimated coordinates of the imaged element is minimized, wherein depending on the at least one imaging parameter a position and / or orientation of the virtual imaging plane ( 7 ) of the head-up display in the reference coordinate system can be determined, wherein by means of the control device ( 3 ) depending on the position and / or orientation of the virtual presentation plane ( 7 ) the control of the image generating device ( 2 ) he follows. Apparatus according to claim 10, characterized in that a relationship between an aberration, a viewing position and a position of an element to be displayed can be determined, wherein by means of the control device ( 3 ) can be determined depending on the position of an element to be displayed and the viewing position of the aberrations, wherein the control of the image generating device ( 2 ) takes place as a function of the aberration.

Images