WO2024032971A1

WO2024032971A1 - Device and method for measuring three-dimensional virtual images and objects on a head-up display

Info

Publication number: WO2024032971A1
Application number: PCT/EP2023/068086
Authority: WO
Inventors: Christoph Boesel; Martin BAUMGARTL
Original assignee: Bayerische Motoren Werke Aktiengesellschaft
Priority date: 2022-08-09
Filing date: 2023-06-30
Publication date: 2024-02-15
Also published as: DE102022120088A1

Abstract

The invention relates to a method for measuring virtual 3D images on a field of view display device, which is designed to superimpose 3D images into the field of view of a user via reflection on a partially transparent reflection pane. The method comprises the steps of: providing image-generation data relating to a virtual 3D test object, wherein the three-dimensional object surface thereof is overlaid with a predetermined visual surface pattern and/or wherein the 3D test object itself or individual non-continuous partial objects thereof represent a predetermined 3D pattern; generating the virtual 3D test object according to the provided image-generation data; capturing the generated virtual 3D test object from two different perspectives using an optical stereo camera system and identifying corresponding object surface points in the two camera images on the basis of the surface patterns and/or 3D patterns captured therein; and determining therefrom at least one actual representation parameter of the generated virtual 3D test object.

Description

22-1285 1 Description Device and method for measuring three-dimensional virtual images and objects of a head-up display The invention relates to a method for measuring virtual 3D images of a field of view display device, which is used to display 3D images in a user's field of vision via reflection on a partially transparent reflection disk arranged in its field of vision. The field of view display device can be designed in particular for use in a motor vehicle or another land, aircraft or water vehicle, with, for example, a front, rear or side window of the vehicle or a combiner window specifically provided for this purpose being in the field of vision of a user, the passenger of the vehicle , serves as a reflection disk. The invention is also directed to a corresponding control unit, an associated field of view display device and a vehicle equipped therewith. Field of view display devices for vehicles are known, among other things, as head-up displays (HUD). This allows, for example, speed information and other useful navigation and vehicle operating instructions or even entertainment content in the form of a virtual image to be superimposed on the real image of the surroundings in front of or in the vehicle observed by the driver or another occupant. For this purpose, a HUD in a classic design has a projection unit housed in the instrument panel. This includes an imaging unit, for example a display, for generating a light beam with the desired display content and, if necessary, suitable projection optics in order to further shape the light beam and to the 22-1285 2 called partially transparent reflection disk. By reflecting on the reflection disk, the bundle of light rays reaches a space in the vehicle interior that is intended for the user's eyes (also called an eyebox), from which he can perceive the display content as a virtual image floating behind the reflection disk. The evaluation of the virtual image and the improvement of the HUD performance requires an objective capture of the displayed virtual image content using measuring devices. For conventional HUDs in motor vehicles, photos of the virtual image are usually created using a mono camera in fixed positions relative to the windshield and some important performance parameters, such as. B. a double image distance or any distortions are evaluated. However, any existing depth information of the virtual image is lost. It is therefore not possible to evaluate HUD design parameters that are particularly important for 3D images, such as the virtual image distance or image curvature, or performance parameters such as local point disparity. The relative position of the virtual image to the environment, which is of utmost importance for augmentation (augmented reality) through contact-analog virtual representations, i.e. virtual representations based on real surrounding objects, cannot be objectively recorded and corrected either. Head-up display development is increasingly focusing on the generation of three-dimensional virtual image structures and image augmentation through contact-analog virtual representations. Therefore, an objective evaluation of such image content using suitable measuring devices is becoming increasingly important in order to be able to ensure targeted improvement and reliable calibration of corresponding HUD systems. Both a quantitative evaluation of the display performance and a calibration of a head-up display intended for 3D image generation require precise recording of the three-dimensional virtual structures generated through measurements. 22-1285 3 In the state of the art, stereoscopic measurement using a specially designed stereo camera is an established method for measuring and recording the spatial structure of physical three-dimensional objects, as described, for example, in US 1,871,281 or US 6,430,373. Here, a photo of the object to be examined is created using at least two calibrated cameras, whose fixed positioning in space relative to one another is known. By comparing identical object features in the photos from the left and right cameras, a pixel disparity of the identical features in the left and right images can be determined. This in turn allows the calculation of a 3D position of the respective object features relative to the stereo camera in space using triangulation. The camera calibration of the stereo camera is usually carried out by photographing various targets, ie real objects whose spatial positioning and extent are known, by the cameras that are fixedly positioned relative to one another. From this, extrinsic camera parameters such as rotation and translation of the coordinate systems of both cameras relative to one another and, if not already known, also intrinsic camera parameters such as focal length and optical center can be determined using known mathematical methods. A well-known difficulty when measuring a real 3D object with a stereo camera is quickly and precisely determining corresponding features in both images. In general, this can be done, for example, by using a suitable similarity measure and e.g. B. a brute force search algorithm that compares the left and right camera images. A frequently used step that is intended to further simplify the search is to rectify both stereo images. The images are transformed using the known extrinsic parameters (such as a rotation matrix and a translation vector of the two camera coordinate systems of the stereo camera relative to one another) as if both cameras were perfectly aligned parallel to one another. This means that corresponding features in the right and in the 22-1285 4 left camera image has the same vertical pixel coordinate, and corresponding features are therefore on a horizontal line. In order to enable a quick, precise and robust measurement of the surface of a physical 3D object, a proposal is also made in the literature, such as in DE 102006049695 A1, to expand the stereoscopic camera system by a projection unit (e.g. in the form of a slide projector). , which projects a visual stripe pattern and, if necessary, a gray code suitable for identifying individual line pairs onto the 3D object. The projected pattern is used to quickly and robustly locate corresponding features in the left and right camera images, for example on large monochrome areas of the 3D object, using a suitable similarity measure. In order to be able to measure transparent or highly reflective 3D objects that have, for example, smooth or painted surfaces, DE 102015211954 A1 also proposes, as a variation of this projector idea, to replace the optical cameras of the stereo camera with thermal imaging cameras and to replace the projection unit with one Equip infrared heaters. In particular, it is proposed to imprint an irregular, for example any (quasi-) statistical thermal pattern (e.g. a speckle pattern) on the object surface to be examined. These suggestions do not apply to virtual 3D objects. Compared to the 3D measurement of real objects, the following additional problems arise when capturing virtual 3D images of a HUD: While real objects remain stationary, the position and shape of a virtual 3D object depends on the viewing direction from which the virtual image is viewed is observed. Consequently, on the one hand, the position of the virtual image in space depends on the actual eye position of the user (such as the driver). On the other hand, the depth perception of the virtual 3D image also changes by overlaying the images in the left and right eyes with different eye distances for the respective user. It is generally not possible to calibrate a stereo camera for an exact desired eye relief due to manufacturing tolerances of the individual cameras 22-1285 5 can be implemented without much effort. In addition, the generation of a virtual ideal image in accordance with the performance specifications of the HUD is not easily possible, for example due to manufacturing tolerances of the windshield, and requires a precise assessment of the influence of any construction deviations on the virtual image. It is therefore an object of the present invention to provide a technical concept (method and device) for measuring three-dimensional virtual images and objects of a head-up display or other field of view display device, which make it possible to overcome the problems and difficulties described. In particular, this is intended to enable a quick, precise and robust evaluation of the performance of a field-of-view display device designed for 3D representation and thus also its targeted improvement. The field of view display device should be particularly suitable for use in a vehicle. This object is achieved by a method according to claim 1 for measuring virtual 3D images of a field-of-view display device and by a corresponding control unit, field-of-view display device and a vehicle equipped therewith according to the independent claims. Further refinements are specified in the dependent claims. All further features and effects mentioned in the claims and the following description for the method also apply with regard to the control unit, the field of view display device and the vehicle, and vice versa. According to a first aspect, a method is provided for measuring virtual 3D images of a field of view display device, for example a head-up display (HUD), which can be designed in particular for use in a vehicle. The field of view display device is designed to display three-dimensional images and objects into the field of vision of a user, such as a driver or another occupant of the vehicle, via reflection on a partially transparent reflection disk arranged in his field of vision, 22-1285 6, for example a windshield or another vehicle window or a combiner window designed specifically for this purpose. The vehicle can be a motor vehicle, but also any other land, air or water vehicle. The representation of a respective 3D object by the field of view display device can in particular also be contact-analogous, ie oriented towards real surrounding objects outside the vehicle. The method includes the following steps: First, an image-generating unit of the field of view display device or its control unit is provided with image generation data of a virtual 3D test object. In this case, a virtual object surface of the 3D test object to be displayed in three dimensions is overlaid on the software side with a predetermined visual surface pattern so that it has variations corresponding to the surface pattern, for example in brightness and/or color. Both the 3D test object and the surface pattern can vary in position, shape and extent over time in a predetermined manner. The surface pattern allows the recognition of the individual surface points or areas of the virtual 3D test object to be significantly more precise, simplified and/or accelerated during its later stereoscopic detection and evaluation. In principle, the entire object surface to be displayed of the virtual 3D test object can be so completely overlaid/overlaid/covered with the surface pattern when its image generation data is provided that robust, rapid and at the same time complete recognition of individual surface points is made possible. Examples of surface patterns suitable for this are given below. The virtual 3D test object used for the measurement can have any three-dimensional shape and extent and can consist of any number of non-connected 3D partial objects, each of which can be arranged anywhere in space. In particular, the non-contiguous virtual 3D partial objects themselves can be arranged in such a way that 22-1285 7 they represent or result in a predetermined, for example regular, irregular, statistical or quasi-statistical three-dimensional pattern (hereinafter referred to as 3D pattern) in space. A 3D pattern generated in this way (virtually) can be used in the present method as an alternative or in addition to the above-mentioned overlay of the three-dimensional virtual object surface with the predetermined surface pattern in order to achieve the effects described below when measuring the 3D test object. The overlay of the object surface with a surface pattern can, for example, be more favorable than generating a virtual 3D test object in the form of a predetermined (virtual) 3D pattern if the virtual 3D test object is to have a predetermined three-dimensional object or surface shape, for example occurs particularly often during normal operation of the field of view display device and is therefore preferably also used in the evaluation of its performance described herein. The virtual 3D test object is now generated by the field of view display device according to the image generation data provided and captured from at least two different perspectives by an optical stereo camera system. For this purpose, the stereo camera system has at least one movable camera and/or at least two cameras at a definable distance from one another in the beam path of the field of view display device after the reflection disk. By moving the cameras of the stereo camera system relative to each other and to the beam path, different eye positions and/or eye distances of a user can be taken into account or simulated. When capturing the generated virtual 3D test object, the respective camera can be arranged, for example, at a position in space intended for the corresponding eye of the user. However, this is not mandatory. In the resulting camera recordings of the virtual 3D test object from the two different perspectives, image points are identified based on the surface patterns and/or 3D patterns contained therein 22-1285 8 each come from one and the same object surface point. From this, at least one actual display parameter of the generated virtual 3D test object is determined, which can be used to evaluate the 3D display performance of the field of view display device. The at least one actual representation parameter can include, for example, a projection distance (also called projection depth or image distance) and/or a spatial position and/or a spatial orientation of the generated virtual 3D test object or its surface points. Alternatively or additionally, the at least one actual representation parameter of the generated virtual 3D test object can include a local vertical point disparity, which indicates a vertical offset of the pairwise corresponding object surface points in the camera images from the two different perspectives. The respective actual display parameter can be determined, for example, with respect to a coordinate system of the stereo camera system, the field of view display device or the vehicle in which it is mounted, or with respect to an eyebox intended for its user. As usual, an eyebox is understood here to be a two- or three-dimensional spatial area intended for the user's eyes or for the respective user's eye, from which the generated virtual 3D images are visible to the user in the intended quality. In order to evaluate the 3D display performance of the field of view display device, the respective actual display parameter can be compared with a predetermined, associated target display parameter of the virtual 3D test object. Depending on this, the 3D display performance of the field of view display device can be evaluated and, for example, specifically improved if a predetermined deviation tolerance is exceeded. An idea to overcome the difficulties described above when evaluating the 3D display performance of a 22-1285 9 field of view display device thus consists of a method and a corresponding measuring device in the form of a stereo camera system that can be positioned in a suitable manner in the beam path of the field of view display device, which is used to record the 3D structure of a virtual image with a software-side superposition of the virtual image to be measured predefined patterns are formed. This metrological recording can in particular also be designed to be eye position-dependent, ie correspond to selectable/adjustable or variable eye positions and/or eye distances of a user. The field of view display device, in particular in the form of a head-up display, can be designed for 3D image generation in basically any technically feasible manner. This can therefore not only involve the mere creation of a depth effect through the use of several two-dimensional virtual image planes and image surfaces that are inclined or curved in space, but in particular also “real” three-dimensional virtual 3D objects, which can be created, for example, using holographic techniques a specially designed image-generating unit of the field of view display device can be generated. With such a 3D HUD in a motor vehicle, objects such as an arrow can be displayed at a great distance and at the same time, for example. B. a sign close to the driver. To make measurement easier, these objects are displayed superimposed with a checkerboard pattern, for example, so that they can be measured with a pair of stereo cameras (depending on the position of the pair of stereo cameras). The deviations from the target can then be determined. As mentioned at the beginning, a central special feature of virtually represented 3D objects is that their position and shape (both when using two-dimensional image planes and three-dimensional curved image surfaces as well as “real” three-dimensional virtual objects) change with the eyes - or camera position can change. In particular, the individual eye distance also plays a role 22-1285 10 a role for the user, so that users with different eye distances can perceive a virtual 3D object differently. When used in a vehicle, the correct position and orientation of a virtual object in space can be of crucial importance, especially in the case of contact-analog representation (ie based on real surrounding objects). With the technical concept presented here for 3D measurement of the virtual objects of a field-of-view display device, these effects and any deviations from the desired result can be recorded and quantified quickly, robustly and precisely in order to be able to evaluate and specifically improve the field-of-view display device in this respect. For the purpose described herein, the surface pattern can, for example, have at least one of the following pattern types or pattern properties, which can also be combined with one another and/or alternately, ie next to one another, in the object surface: a regular or periodic two-dimensional pattern (the periodicity can be one- or two-dimensional be, ie the two-dimensional pattern can change periodically in the two-dimensional surface, for example only in one direction, while it remains constant in a direction orthogonal thereto, or it can change in two independent directions with the respective periodicity); an irregular or aperiodic two-dimensional pattern (e.g., statistical or quasi-statistical); a checkerboard pattern; a flat distribution of circles with one or more predetermined diameters; a speckle pattern; an area-wide gray value pattern, the gray values of which preferably vary in a predetermined manner in each area point; a black and white pattern; a color pattern that has one or more different colors. Basically, comprehensive and/or irregular surface patterns, such as: B. aperiodic sine patterns or speckle-like gray value distributions, if necessary more options for quick and/or 22-1285 11 time-resolved feature mapping between the left and right camera images of the created virtual 3D object. According to one embodiment, when capturing the generated virtual 3D test object, the respective camera of the stereo camera system is arranged at a spatial position intended for the user's respective eye. In particular, an entire eyebox volume of the field of view display device intended for the respective eye of the user can be scanned by the stereo camera system in that the respective camera of the stereo camera system successively captures the generated virtual 3D test object from several different positions within the eyebox volume in order to evaluate the 3D display performance of the field of view display device as fully as possible for the entire eyebox. According to a further or alternative embodiment, the definable relative distance of the at least two cameras can be varied to adapt to different eye distances of the user. In particular, it can also be varied when capturing the generated virtual 3D test object in order to evaluate the 3D display performance of the field of view display device for different user eye distances. According to one embodiment, the method presented here further comprises a calibration of at least one of the cameras of the stereo camera system, which can be carried out in particular before the measurement of the actual 3D test object. For this purpose, this camera can, for example, be positioned at one or more discrete calibration positions and a calibration can be carried out by determining a spatial position and/or orientation and/or, if necessary, at least one internal imaging property of this camera with respect to a coordinate system of the stereo camera system or the field of view display device become. In this embodiment, a parametric recalibration function is then determined from this, which, depending on at least one (in particular continuously) measurable camera position parameter, indicates the spatial position and/or orientation of this camera at an infinite number of camera positions lying in between and/or beyond. Spatial positions and/or orientations of this camera that can be obtained using this parametric recalibration function can be used as its calibration in the actual measurement of the virtual 3D test object, ie to determine its at least one actual representation parameter, such as. B. its projection distance using triangulation. In this way, on the one hand, a time-consuming and cost-intensive recalibration of each individual camera in the stereo camera system can be avoided with each camera movement. On the other hand, such a parameterization of the camera calibration opens up the possibility of quickly and precisely taking into account different eye positions and/or eye distances of the user when measuring the virtual 3D test object. Some exemplary embodiments of this are given below. In particular, said camera can be displaceable along at least one translational rail with linear position feedback, so that said measurable camera position parameter is a linear camera position or a linear camera distance to the second camera of the stereo camera system on this rail according to the currently received position feedback. The position feedback can, for example, be measured or obtained directly during the capture of the generated virtual 3D test object or immediately before or after, so that in particular recalibration in real time is made possible by such a parametric recalibration function. According to a further aspect, a control unit is provided which is designed and set up to automatically carry out the method presented here. For this purpose, for example, in the control unit 13 corresponding computer program must be installed and run during operation of the field of view display device. According to a further aspect, a field of view display device is provided, which can be designed in particular for use in a vehicle. The field of view display device comprises a projection unit which is designed to generate and output a light beam with a 3D image content transported therein to a partially transparent reflection disk to be arranged or arranged in the field of vision of a user. The design and mutual arrangement of the projection unit and the reflection disk are chosen such that the user is presented with a desired virtual 3D image behind it, provided his eyes are in a designated spatial area (eyebox) opposite the reflection disk. For this purpose, the projection unit can comprise an image-generating unit, for example in the form of a suitable display or projector, and possibly further optical elements in the beam path of the light beam emanating from the display/projector for further beam shaping and deflection. The field of view display device further comprises an optical stereo camera system which comprises at least one movable camera and/or at least two cameras at a definable distance from one another, which can be positioned in the beam path of the light beam after its reflection on the reflection disk and are designed to display the virtual 3D shown -Capture image from at least two different perspectives. Furthermore, the field of view display device includes the above control unit, which is designed and set up to control the projection unit and the stereo camera system when automatically carrying out the method presented herein. The reflection disk mentioned can also be manufactured and/or sold as part of the field of view display device or, alternatively, separately. When used in a vehicle, the reflection pane can in particular be used as a section of a windshield of a vehicle or another 22-1285 14 vehicle window can be formed. However, it can also be designed as a combiner disk designed specifically for the purpose mentioned here. According to a further aspect, a vehicle, in particular a motor vehicle or any other land, air or water vehicle, is provided. The spatial orientation terms used here such as “above”, “below”, “in front”, “side”, “horizontal”, “vertical” etc. can in particular refer to the usual vehicle-fixed Cartesian coordinate system with mutually perpendicular longitudinal, transverse and height axes of the vehicle. The vehicle includes a windshield and an instrument panel located underneath and is equipped with the above field of view display device. Their image-generating unit or, if appropriate, their entire projection unit (which, in addition to the image-generating unit, can additionally include suitable projection optics) can be arranged in particular inside the instrument panel or in/on its top side, for example installed directly on or below the top side of the instrument panel, such that the light beam bundle is thrown by the projection unit onto the windshield or a combiner disk positioned inside the vehicle in front of it in the field of vision of the driver or another occupant, which serves as the above-mentioned partially transparent reflection disk. Alternatively, the field of view display device can also be installed at any other suitable location in the vehicle. The above aspects of the invention and their specific design variants, embodiments and further features and effects are explained in more detail below using the examples shown in the accompanying drawings. The drawings are to be understood as purely schematic illustrations, ie not to scale. Show it: 22-1285 15 Figure 1 shows a schematic top view of a field of view display device in a motor vehicle, which is designed to display 3D images into an occupant's field of vision via reflection on a vehicle window arranged in his field of vision and to carry out a method according to an exemplary embodiment of the invention; Figure 2 shows a flowchart of an exemplary embodiment of a method presented herein for measuring virtual 3D images of a field of view display device; 3 shows an exemplary step of providing image generation data of a virtual 3D test object in the method of FIG. 4 shows a schematic representation of a further example of a virtual 3D test object from two different perspectives, which are captured by the stereo camera system in the method of FIG is overlaid by full circles; 5 shows a schematic representation of a further example of a virtual 3D test object from two different perspectives, which are captured by the stereo camera system in the method of FIG a speckle-like gray value pattern is superimposed. 22-1285 16 All of the various embodiments, variants and specific design features of the method, the field of view display device, the control unit and the vehicle according to the above aspects of the invention mentioned above in the description and in the following claims can be used in the ones shown in Figures 1 to 5 Examples to be implemented. They will therefore not all be repeated again below. The same applies to the definitions and effects already given above in relation to individual features shown in Figures 1-5. 1 shows, in a highly simplified schematic top view, an exemplary embodiment of a vehicle 1 with a field of view display device 2 according to the aspects of the invention specified above and in the claims. This example is a motor vehicle that is only indicated by its windshield 3. A projection unit 5 of the field of view display device 2 is arranged opposite the windshield 3, for example below it in an instrument panel 4 of the vehicle 1 (not shown). In this example, the field of view display device 2 is designed as a 3D head-up display (3D HUD). The projection unit 5 is designed to generate a light beam L with a desired 3D image content. The light beam L emanating from the projection unit 5 is thrown onto the windshield 3, which in this example serves as a partially transparent reflection disk of the field of view display device 2, so that after reflection on the windshield 3 it reaches an eyebox E of a user 6, which in this example is a driver of vehicle 1. The eyebox E is a two- or three-dimensionally defined spatial area in the vehicle 1 at a predetermined position relative to the windshield 3, which is intended for the eyes of the user 6, so that he can see a virtual 3D image O generated by the field of view display device 2 with both eyes able to see. As illustrated in FIG. 1, the user 6 sees a virtual eye with his left eye 22-1285 17 image OL and with his right eye a virtual image OR, the relative position and shape of which can vary with the eye distance and the eye position of the user 6 relative to the light beam L, so that the resulting perception and spatial position of the generated 3D object depending on the eye distance and the eye position of the user 6 can vary. Furthermore, an optical stereo camera system 7 is provided in the vehicle 1 or in the field of view display device 2, which can include at least one movable camera and / or at least two cameras at a definable distance from one another (not shown individually), which are in the beam path of the light beam L whose reflection can be positioned on the windshield 3 in order to capture the displayed virtual 3D image O from two different perspectives to evaluate the 3D HUD performance (of course without the user 6 being there). For example, the stereo camera system 7 can be positioned directly in the eyebox E of the user 6 or, as indicated in FIG. 1, immediately in front of it. The stereo camera system 7 and its position are indicated purely by way of example and schematically in FIG. To carry out a method according to the above first aspect of the invention, a correspondingly configured control unit 8 is also provided, which can communicate in an appropriate manner with the projection unit 5 and the stereo camera system 7 in terms of information and control technology. The control unit 8 can be arranged, for example, in the projection unit 5 or outside it in the vehicle 1, for example in the instrument panel 4. 2 shows a flowchart of an exemplary embodiment of a method according to the above first aspect of the invention for measuring virtual 3D images O of a field of view display device 2, as shown, for example, in FIG. 1. However, the reference to FIG. 1 should not be construed as limiting, but rather serves solely as an exemplary illustration of the possible 22-1285 18 steps. The method can in particular include the following basic steps, with further details, variants and optional further steps being specified below: In a step S1, the projection unit 5 of the field of view display device 2 or its control unit 8 is provided with image generation data of a virtual 3D test object O. In this case, a virtual object surface 9 of the 3D test object to be displayed in three dimensions is overlaid on the software side with a predetermined visual surface pattern so that it has variations corresponding to the surface pattern, for example in brightness and/or color. Fig. 3 illustrates this step using the example of a checkerboard surface pattern. In this example, the entire object surface 9 of the virtual 3D test object O to be displayed has been overlaid with the surface pattern when its image generation data is provided. In Fig.3 on the left, the virtual 3D object to be created is shown before the software overlay with the surface pattern. On the right in Fig. 3, the virtual 3D object O to be created is shown in its representation form used for measurement/calibration. In this example, its cuboid object surface 9 is superimposed on the software side with a known structure, such as e.g. B. a checkerboard pattern, the corners 10 of which are particularly easy to localize by gradient formation after detection by the stereo camera system 7. Alternatively, more general (quasi-) statistical patterns can also be used as area patterns. The overlay with a suitable surface pattern in step S1 serves purely to measure and, if necessary, calibrate the field of view display device 2 (see below). At a step S2, the virtual 3D test object O is displayed by the field of view display device 2 according to the provided image formation data 22-1285 19 generated and captured from at least two different perspectives by an optical stereo camera system 7. For this purpose, the stereo camera system 7 has at least one movable camera and/or at least two cameras at a definable distance from one another in the beam path of the field of view display device 2 after the reflection disk. By moving the cameras of the stereo camera system 7 relative to one another and to the beam path of the light beam L, in particular different eye positions and/or eye distances of a user 6 can be taken into account or simulated. In a further step S3, image points are identified in the resulting camera recordings of the virtual 3D test object O from the two different perspectives (cf. FIGS. 4 and 5) based on the surface patterns contained therein, each of which originates from one and the same object surface point or feature (so-called feature matching). From this, at least one actual display parameter of the generated virtual 3D test object O is determined, which can be used to evaluate the 3D display performance of the field of view display device 2. This method and this device make possible, in particular, a fast, robust and precise measurement of virtual 3D test objects O, which are generated, for example, by a HUD, which also enables, among other things, a quick calibration of the HUD. Important measured variables or output parameters in the 3D measurement of the generated virtual test objects O for HUD evaluation with a stereo camera system 7 (also called 3D camera) are, for example: B. the projection distance (to answer the question of whether the projection distance corresponds to the specifications) and the vertical local point disparity (Local Point Disparity Y, to answer the question of how strong the virtual image OL is for the left eye and the virtual image OR differ vertically for the right eye, see Fig.1). The projection distance can e.g. B. deviate from the specification due to construction tolerances of the field of view display device 2. The vertical local 22-1285 20 Point disparity corresponds to a vertical offset of the features between the left and right eyes and is caused by the driver's left and right eyes seeing a different virtual image in the case of HUD projection. The latter does not occur, for example, when measuring real objects, ie there the vertical local point disparity is naturally equal to zero. To determine the projection distance and the local point disparity, the camera images recorded in step S2 from the two different perspectives can first be rectified in a manner known per se, that is, converted to a parallel alignment of both cameras. The corresponding pixel coordinates ( ) and ( ) in the (rectified) left and right camera images can then be determined. As is well known, this would apply in the rectified camera images of a real object. In the case of a virtually represented 3D object, however, the right and left eyes perceive different virtual images OL and OR (see Fig. 1) and the following applies in the rectified camera images. The projection distance results, for example, from triangulation or from the horizontal pixel deviation of the pixel coordinates ( ) of the corresponding features in the (rectified) left and right camera image as well as the camera distance from the set of rays. In order to improve and speed up the measurement, the present method proposes that the 3D objects typically displayed in the HUD for the driver (such as an arrow on the road, a marking of signs or pedestrians, a map for navigation, etc ) before or during their generation by the projection unit 5 (such as an augmented reality HUD display) solely for the purpose of measurement and / or calibration on the software side with regular (such as the checkerboard pattern 3), irregular, statistical or quasi-statistical surface patterns. This in turn allows a quick and precise assignment of corresponding features in the left and right camera image and is therefore used to measure the image quality or calibrate the HUD. Using a checkerboard pattern, for example: B. determine the vertical local point disparity at the corner points 10 of the checkerboard pattern with high precision. A calibration of the HUD can also take place, for example, by measuring the virtual, patterned 3D test object in reference to a known real object in the driver's field of vision. As mentioned above, another problem with virtual objects on a HUD is the dependence of the position of the virtual object on the distance between the eyes and the eye position of the user 6. This plays a role both in the stereo camera calibration and in the measurement of the 3D test object O with the stereo -Camera system 7 a role. The method described here enables a solution to this problem, which is described below using the example of a stereo camera system with cameras that can be positioned flexibly relative to one another: The integration of the eye distance into the measurement proposed here requires, in the first step, the determination of a rotation matrix and a translation vector as in a conventional stereo camera calibration. On the one hand, the distortion of the two cameras can be determined (if they have optical distortion) and calculated out algorithmically. In addition, the intrinsic camera parameters (focal lengths, optical center) are determined if these are not known, as well as the extrinsic parameters (rotation matrix, translation vector of the two camera coordinate systems relative to each other or to the vehicle). In order to avoid recalibration for any shift in the relative position of the cameras of the stereo camera system 7, the two cameras can z. B. be attached to a translational rail or other movement device with position feedback. Based on a stereo camera calibration, which is carried out in a few fixed and known relative positions of the cameras to one another, the stereo camera calibration according to the embodiment of the invention described above is carried out using mathematical means (such as mathematical modeling or polynomial interpolation or extrapolation and much more) in between and/or positions beyond this are expanded. In other words, a parametric recalibration function can be specified based on actual calibration at some known position points of the cameras, which depends on one or more (for example continuously) measurable camera position parameters. For example, the parametric recalibration function can specify a precise relative 3D camera position depending on a measurable distance of the camera on a rail (which can be described by a simple linear model). To integrate the eye distance and eye position into the measurement of a virtual 3D object, the eyebox volume of the HUD can be scanned or scanned, for example, using the stereo camera system for variable distances of the left and right cameras. The eye position can be parameterized by the respective center of the stereo camera system 7. For each camera position and each eye distance, a 3D point model is created, for example, by taking a stereo photograph of the virtual 3D test object (provided with the surface pattern). By describing the 3D point model as a fit function of the eye position and eye distance, 3D point cloud models can be interpolated or extrapolated for unknown eye distances and eye positions. The necessary number of measuring points of the grid in the eyebox volume or the complexity of the fit function (in the simplest case this can be achieved by a linear interpolation function and in the extreme case 23 are described by a neural network) generally depends on the complexity of the HUD optics or the reflection pane (such as the windshield 3). In the simplest case, the eyebox volume can be “sampled”, for example by a single eye position with a predetermined constant eye distance. Furthermore, in the present method, a suitable similarity measure and, for example, a brute force search algorithm can be used to identify corresponding surface points in both camera images, which compares the left and right camera images. In the method presented, the measuring device can therefore include, for example, one or more variably positionable camera(s) (stereo camera system 7), whose relative positions can be either fixed (only for at least two cameras) or variable. The calibration of the measuring device is ideally carried out using the parametric description presented above with the help of camera position-dependent rotation matrices and translation vectors or projection matrices composed thereof

where λ is the measurable camera position parameter mentioned above. In the photos of the virtual object from two or more different perspectives, the pixel disparities of the identical features can be interpreted as a function of the camera positions determined in this way. This allows the 3D structure of the surface 9 of the virtual 3D test object O to be calculated depending on the position using a parametric representation of this functional relationship. This means that the virtual 3D test object O can be interpolated for different eye positions and eye distances. In this way, with the present measurement concept, the problem of exact positioning of the two cameras during stereo calibration, which is very difficult to achieve (e.g. due to construction tolerances of the cameras), can be eliminated 22-1285 24 needed to describe/evaluate 3D HUD performance for a desired eye relief can be avoided using rigorous mathematical means. This allows quick and reliable measurement of the virtual 3D structures generated by the HUD for variable eye distances and eye positions. 4 and 5 show further examples of the software overlay of the virtual 3D test object O to be measured in step S1 of FIG. 2 before it is generated with predefined surface patterns, before it is generated in steps S2 and S3 and photographed from different camera perspectives . If the virtual object is overlaid with full circles 11, for example as in FIG User 6 perceivable virtual images OL and OR of FIG. 1 can correspond). Another possible implementation according to FIG. 5 is to overlay the virtual 3D test object to be measured (here a pyramid with a rectangular base) with a speckle-like surface pattern in the software. The feature matching between the photos from the individual camera perspectives can be done here, for example, by rectifying the two photos and assigning the gray value values around the corresponding maxima of the speckles 12 in the photos from the individual camera perspectives. Otherwise, the same thing as in Fig. 4 can apply here. Another possible implementation (see FIG. 3) is to overlay the virtual 3D test object O to be measured with a checkerboard pattern in the software. The feature matching between the photos from the individual camera perspectives is done here, for example, by determining the corner points 10 of the checkerboard patterns and assigning the corresponding coordinates in the photos from the individual camera perspectives. In order to enable a clear assignment of the corner points 11 of the checkerboard pattern between the left and right camera images, the 22-1285 25 individual squares of the chessboard can be coded, for example, by colors or QR-like patterns. Otherwise, the same thing as in Fig. 4 or 5 can apply here. Another possible implementation is that the virtual 3D test object O consists of N virtual sub-objects or sub-objects distributed arbitrarily in 3D space (e.g. it can be a regular or grid-like three-dimensional arrangement of three-dimensional spheres), each with an area pattern are superimposed, the feature(s) of which are used for stereoscopic feature matching. Another possible implementation would be to overlay the virtual 3D object to be measured with an aperiodic surface pattern, for example a sine pattern (not shown), on the software side. The feature matching between the photos from the individual camera perspectives can be implemented here by rectifying the photos and assigning the gray value values around the corresponding maxima of the sine pattern in the photos from the individual camera perspectives.

22-1285 26 List of reference symbols 1 Vehicle 2 Field of view display device 3 Windshield 4 Instrument panel 5 Projection unit 6 User 7 Stereo camera system 8 Control unit 9 Virtual object surface to be displayed in three dimensions 10 Corners of the checkerboard pattern 11 Full circles, circles 12 Speckles L Light beam bundle O Virtual 3D (test) object or . virtual 3D image OL virtual image visible to the left eye OR virtual image visible to the right eye E Eyebox

Claims

22-1285 27 Claims 1. Method for measuring virtual 3D images (O) of a field of view display device (2), in particular for a vehicle (1), which is used to display 3D images (O) into the field of vision of a user (6) via reflection is formed on a partially transparent reflection disk arranged in its field of vision, comprising the steps: - providing image generation data of a virtual 3D test object (O), the object surface (9) of which is to be displayed in three dimensions being superimposed in the image generation data with a predetermined visual surface pattern and/or where the virtual 3D test object (O) itself or its individual non-connected sub-objects represents/represent a predetermined 3D pattern; - Generating the virtual 3D test object (O) by the field of view display device (2) according to the provided image generation data; - Capturing the generated virtual 3D test object (O) from two different perspectives by an optical stereo camera system (7), which comprises at least one movable camera and/or at least two cameras at a definable distance from one another in the beam path of the field of view display device (2); - Identifying pairs of corresponding object surface points in the camera recordings from the two different perspectives based on the surface patterns and/or 3D patterns recorded therein and determining from this at least one actual representation parameter of the generated virtual 3D test object (O). 2. The method according to claim 1, wherein - the at least one actual display parameter is a projection distance and/or a spatial position and/or a spatial orientation of the generated virtual 3D test object (O) and/or a local vertical point disparity, which represents a vertical offset of the object surface points corresponding in pairs in the camera - Includes shots from two different perspectives. 3. The method according to claim 1 or 2, wherein - the respective actual display parameter is compared with a predetermined associated target display parameter of the virtual 3D test object (O) and the 3D display performance of the field of view display device (2) is evaluated depending on this. 4. The method according to any one of the preceding claims, wherein - essentially the entire object surface (9) to be displayed of the virtual 3D test object (O) is overlaid with the surface pattern when its image generation data is provided. 5. The method according to any one of the preceding claims, wherein the surface pattern has at least one of the following pattern types or properties: - a regular or periodic 2D pattern; - an irregular or aperiodic 2D pattern; - a checkerboard pattern; - a flat distribution of circles (11) with one or more predetermined diameters; - a speckle pattern; - an area-wide gray value pattern, the gray values of which preferably vary in a predetermined manner in each area point; 29 - a black and white pattern; - a color pattern that has one or more different colors. 6. The method according to any one of the preceding claims, wherein - when detecting the generated virtual 3D test object (O), the respective camera of the stereo camera system (7) is arranged at a spatial position intended for the respective eye of the user (6); and/or - when detecting the generated virtual 3D test object (O), an entire eyebox volume of the field of view display device (2) intended for the respective eye of the user (6) is scanned by the stereo camera system (7) by the respective camera generated virtual 3D test object (O) sequentially captured from several different positions within the eyebox volume; and/or - the definable relative distance of the at least two cameras can be varied to adapt to different eye distances of the user (6) and is preferably also varied when capturing the generated virtual 3D test object (O). 7. The method according to any one of the preceding claims, further comprising a calibration of at least one of said cameras of the stereo camera system (7), in that - said camera is positioned at one or more discrete calibration positions and in each case a calibration is carried out by determining a spatial position and /or orientation of this camera is carried out with respect to a coordinate system of the stereo camera system (7) or the field of view display device (2) or the vehicle (1); and 22-1285 30 - a parametric recalibration function is determined from this, which, depending on at least one measurable camera position parameter, indicates the spatial position and / or orientation of this camera at an infinite number of camera positions lying in between and / or beyond; - wherein the said spatial position and/or orientation of this camera is used as its calibration when determining the at least one actual display parameter of the generated virtual 3D test object (O). 8. The method according to claim 7, wherein - said camera is displaceable along at least one translational rail with a linear position feedback, so that the associated measurable camera position parameter is a linear camera position on this rail according to the currently received position feedback. 9. Control unit (8), which is designed and set up to automatically carry out the method according to one of the preceding claims. 10. Field of view display device (2), in particular for use in a vehicle (1), comprising: - a projection unit (5) which is used to generate and output a light beam (L) with a 3D image content in the field of vision of a user (6). The partially transparent reflection disk to be arranged or arranged is designed in such a way that a virtual 3D image (O) is displayed to the user (6) behind it, provided that his eyes are in an eyebox (E) intended for them; - an optical stereo camera system (7) which has at least one movable camera and/or at least two cameras 22-1285 31 comprises a definable distance from one another, which can be positioned in the beam path of the light beam (L) after the reflection disk and is designed to capture the displayed virtual 3D image (O) from two different perspectives; - a control unit (8) according to claim 9, which is designed and set up to control the projection unit (5) and the stereo camera system (7) when automatically carrying out the method according to one of claims 1 to 8; and - preferably also said reflection disk, which is designed in particular as a section of a windshield (3) of a vehicle (1). 11. Vehicle (1), in particular a motor vehicle, with mutually perpendicular longitudinal, transverse and height axes of a vehicle's own Cartesian coordinate system, comprising: - a windshield (3) and an instrument panel (4) arranged underneath; and - a field of view display device (2) according to claim 10, the reflection disk of which is designed in particular as a section of the windshield (3) or as a combiner disk arranged in front of it on the inside of the vehicle.