DE102016009333A1 - Native control of an image projector - Google Patents

Abstract

The invention relates to a method for driving an image projector (20) by a graphics processor (10). The method has the following steps:
Providing (S1) image information having a plurality of color components to the graphics processor (10) through a data interface (5),
- based on the image information for each of the color components generating (S2) a color component image (50) each having a number N image lines (52) and each having a number M of image pixels (54) per image line (52) by the graphics processor (10),
wherein a distance between immediately adjacent image pixels (54) is identical for all image pixels (54) in the line direction,
- transmitting sequentially (S5) the color component images (50) from the graphics processor (10) to the image projector (20) by a transmission unit (15), and
- representing sequentially (S6) the color component images (50) by the image projector (20).

Description

The invention relates to a method for driving an image projector by a graphics processor and a graphics processor for carrying out some method steps of the method.

Depending on the purpose and model, display units for visualizing data can use very different technical effects for data reproduction. Equally diverse are the protocols with which the various display units can be controlled. The manner in which a data stream from a graphics processor is converted into a perceptual representation by a display unit may require data buffers, especially if the internal drive signal of the display unit has protocol or systemic differences with respect to the data stream of the graphics processor. Such differences may be due to the order of the data or in the protocol. In the latter case, conversions of the signals are necessary. Such differences make buffer memory necessary, in which the data of the data stream are stored from the graphics processor, in order to subsequently be read out and / or converted in particular in packets and in a special sequence. Buffers inevitably lead to dead times which increase the overall latency of the transmission. However, especially in the case of a head-up display ("HUD") for an augmented reality application, a main quality feature is that the displayed virtual objects are moved as far as possible synchronously with the movement of an environment around the HUD. In other words, as the HUD moves relative to the environment, a virtual object in the display of the HUD should be moved in the opposite direction so that its relative position to the environment does not change-this is also referred to as "contact analogy." In order for this tracking to the motion compensation as time synchronous as possible by latencies as low as possible, it is crucial that the latency from the detection of a movement to the display of the image with tracking objects is as small as possible.

The object of the invention is to minimize latencies in the control of an image projector by a graphics processor.

The invention results from the features of the independent claims. Advantageous developments and refinements are the subject of the dependent claims.

• – Bereitstellen einer Bildinformation mit mehreren Farbkomponenten an den Grafikprozessor durch eine Datenschnittstelle,
• – auf Basis der Bildinformation für jede der Farbkomponenten Erzeugen eines Farbkomponentenbildes mit jeweils einer Anzahl N Bildzeilen und mit jeweils einer Anzahl M von Bildpixeln pro Bildzeile durch den Grafikprozessor, wobei ein Abstand zwischen unmittelbar benachbarten Bildpixeln für alle Bildpixel in Zeilenrichtung identisch ist,
• – sequentielles Übertragen der Farbkomponentenbilder vom Grafikprozessor an den Bildprojektor durch eine Übertragungseinheit, und
• – sequentielles Darstellen der Farbkomponentenbilder durch den Bildprojektor.

A first aspect of the invention relates to a method of driving an image projector through a graphics processor. The method comprises the following steps:

• Providing image information with a plurality of color components to the graphics processor through a data interface,
• On the basis of the image information for each of the color components, generating a color component image each having a number N image lines and each having a number M of image pixels per image line by the graphics processor, a distance between immediately adjacent image pixels being identical for all image pixels in the line direction,
• Sequentially transferring the color component images from the graphics processor to the image projector by a transmission unit, and
• - Sequential representation of the color component images by the image projector.

The data interface is preferably an interface to a processor, also called a CPU ("Central Processing Unit"), which transfers image information in the form of raw data for an image to be displayed to the graphics processor. In particular, this raw data is "rendered" in the graphics processor, that is, the image is generated and calculated for an image projector. The resulting calculations are preferably carried out in software. In the process, a data stream is finally generated in the graphics processor which is transmitted to the image projector. The data stream from the graphics processor to the image projector has color component images that are used internally in the image projector in this form without further conversion and transmitted via a microcontroller directly to the display elements of the image projector for outputting optical information. A buffering of the data in the image projector for conversion purposes is thus eliminated. This is made possible by the equally sequential representation of the color component images. The color component images are transmitted in this sense in the data stream from the graphics processor to the image projector sequentially and output directly and in the same order from the image projector. The sequential transmission means that one color component image is transmitted one after the other. The transmission takes place via a transmission unit, also called Videolink. The color component images themselves arise from the fact that the image information containing a plurality of color components is output from the data interface sorted by individual color components. In particular, in each case a color component image with blue values, one with green values and one with red values is present.

In other words, the color component images are transmitted by the graphics processor in a format that is natively compatible with the image projector. This particularly concerns the sequential nature of the data stream in which the color component images are transmitted one after the other. The image projector preferably has a DLP unit ("Digital Light Processing "), in which the light of a lighting unit is modulated by means of a micromirror array. Alternatively, the image projector has a device similar to a DLP unit, but having the characteristics characteristic of a DLP unit, namely at least the sequential representation of color component images.

In other words, in this native time control of the display device, the color components are transmitted in temporal sorting in which they are also displayed by the image projector: For example, if there is a classic DLP unit which displays three color component images for each image information consisting of the color components the image information (eg, first the red image, then the green image and then the blue image), then first the red component of all image pixels of the image information is transmitted, then the green component of all image pixels and then the blue component of all image pixels. In this way, the image projector successively receives the color component images and can display them immediately at once, without having to wait for a previous or subsequent color component image.

It is an advantageous effect of the invention that latencies in the control of an image projector are minimized by a graphics processor, in particular in that the control signals of the graphics processor for driving the image projector correspond to the native output signals of the image projector and buffer storage accounts for conversion purposes.

According to an advantageous embodiment, a complete sequence of color component images for displaying exactly one image information is repeatedly displayed n times by the image projector.

In order to avoid so-called "rainbow effects", the sequence of the display of the individual color components of an image information item is repeatedly shown by the image projector several times per image information. Preferably, each color component image is repeated twice or four times ("2 × color wheel speed" or "4 × color wheel speed"). For example, in "4 × color wheel speed" each of the three color components red, green, blue (RGB) is displayed four times, for example in the sequence "RGB-RGB-RGB-RGB". In this case, twelve color component images are displayed one after the other for each image information. Alternatively, a color component image of a display unit, preferably a micromirror array, is additionally exposed to further colors in addition to the three primary colors RGB. This applies in particular to the exposure to white for a brighter presentation, in order to achieve a better color representation. Advantageously, rainbow effects are thus avoided.

Alternatively preferably, a sequence of color component images which is complete to display exactly one image information item is repeatedly transmitted n times by the transfer unit to the image projector by the graphics processor. In this alternative, unlike the previously described embodiment, the sequence of color component images is already repeatedly output from the graphics processor and transmitted to the image projector. Advantageously, the requirements for the image projector are further reduced because the graphics processor offers greater computing capacity and a repetition of the image sequences is easier to implement.

According to a further advantageous embodiment, the relative position of the N picture lines in the line direction in the respective color component image to each other by the graphics processor is arranged such that a beginning of every second image line of the color component image relative to an immediately adjacent image line by half the distance between two immediately adjacent image pixels of a picture line is offset from a reference line perpendicular to the image lines.

In DLP units, the lines are often arranged in a "diamond pattern", in which, compared to a "Manhattan pattern" every other image line is offset by half the distance between two adjacent directly in a picture line pixels. For example, in the Manhattan pattern, or "orthogonal pattern" used in CRTs and LCDs, the pixels are arranged in a rectangular array along the rows and columns of the image. Raster units of 3D graphics chips can typically only rasterize in the Manhattan pattern, which is why deviating from the Manhattan pattern in the image projector would require a resampling of the rendered image at the actual pixel positions, which would also require a higher resolution of the image in the data stream of the graphics processor, Aliasing effects when converting to the Diamond pattern cause losses that could be compensated by a higher resolution of the source image. Characterized in that a color component image is generated based on the image information for each of the color components, which already has an arrangement of the lines (namely the diamond pattern) per color component image for the image projector, conversions from the data stream to the output by the image projector are unnecessary. The already fitting arrangement of the lines per color component image is such that a beginning of every second image line of the color component image is offset relative to an immediately adjacent image line by half the distance between two immediately adjacent image pixels of a picture line opposite a reference line perpendicular to the picture lines, each color component image one Number of N picture lines and each having a number M of image pixels per image line and a distance between immediately adjacent image pixels for all image pixels in the line direction is identical.

In other words, the graphics processor sequentially transfers color component images to the image projector with a row arrangement of the pixels in the diamond pattern, whereby exactly these two features (sequential sequence of the color component images and line arrangement of the image pixels in the diamond pattern) correspond to the native format of the image projector, so that advantageous Further conversion steps and thus both calculation times and buffer times from the output data stream of the graphics processor are eliminated.

According to a further advantageous embodiment, a difference between two color component images of different color components is transmitted by the transmission unit from the graphics processor to the image projector from a common image information.

According to a further advantageous embodiment, a difference between two color component images of the same color component of different image information is transmitted by the transfer unit from the graphics processor to the image projector.

It may be the case that the information of certain color component images can be obtained from previous color component images. In this case, as the refinement of the pure native time drive, only the color component images may be transmitted with new image information; at the times when in the native version color component images that can be obtained from previous color component images would be transmitted, simply nothing is transmitted since there is no difference. This advantageously serves to reduce power consumption or reduce heat generation in the video link.

According to a further advantageous embodiment, the arrangement of the relative position of the N picture lines in the row direction in the respective color component picture to one another takes place by the graphics processor in a warping unit of the graphics processor.

A warping unit is designed to compensate for image distortion occurring in a HUD. The warping is used in HUD and similar display optics, which have a distortion, to compensate for this distortion. Typically, a finished rendered undistorted color component image is resampled with the inverse of the distortion. If the rendered color component image is in the Manhattan pattern for technical reasons, the warped color component image may also be in the Manhattan pattern. Here, however, the target pixel pattern can be simply specified by specifying the locations at which the rendered color component image is resampled: If the points are selected for resampling along a Manhattan pattern, the warped color component image is obtained in the Manhattan pattern. When the points for resampling are selected along a Diamond pattern, the warped color component image in the Diamond pattern is obtained. Thus, no separate component for pattern conversion is necessary, which is why an additional latency or a loss of quality due to further resampling omitted.

According to a further advantageous embodiment, the color component images each have the color components red, green and blue.

Alternatively, preferably, the color component images each have the color components red, green, blue, and white, wherein the color component white is a combination of the color components red, green and blue, and wherein the combination of the respective color component images of the color components red, green and blue takes place in the image projector ,

• – Schätzen einer Relativbewegung des Bildprojektors gegenüber einer Umgebung des Bildprojektors durch eine kinematische Messvorrichtung, und
• – Ermitteln einer Ausgleichsfunktion zum Ausgleichen der Relativbewegung für das Farbkomponentenbild auf Basis der geschätzten Relativbewegung durch den Grafikprozessor. Vorteilhaft wird somit bereits durch den Grafikprozessor eine Bewegungskompensation des jeweiligen Farbkomponentenbildes berechnet, um eine Relativbewegung insbesondere eines in einer Virtual-Reality Umgebung dargestellten Objekts derart auszugleichen, dass das dargestellte Objekt für einen Betrachter ortsfest gegenüber einer Umgebung erscheint. Dadurch, dass diese Aufgabe im Grafikprozessor erledigt wird, entfallen weitere Latenzen für eine derartige Berechnung im Bildprozessor, und das Farbkomponentenbild beinhaltet bereits diese Bewegungskompensation bei Erreichen des Bildprojektors.

According to a further advantageous embodiment, the method further comprises the following steps:

• Estimating a relative movement of the image projector with respect to an environment of the image projector by a kinematic measuring device, and
• Determining a compensation function to compensate for the relative motion for the color component image based on the estimated relative motion through the graphics processor. Advantageously, a motion compensation of the respective color component image is thus already calculated by the graphics processor in order to compensate for a relative movement, in particular of an object displayed in a virtual reality environment, such that the object shown appears stationary to an environment for a viewer. The fact that this task is done in the graphics processor eliminates further latencies for such computation in the image processor and the color component image already includes this motion compensation when the image projector is reached.

According to a further advantageous embodiment of the image projector is arranged in a vehicle, wherein a rotational component of a self-motion of the vehicle relative to an object to be displayed in front of image projector by Movement and / or rotation of the color component image generated by the graphics processor is compensated.

Since the rendering of a separate color component image per image information sets high performance requirements on the graphics processor and introduces certain latencies, the present method can advantageously be combined with "asynchronous time warping". In doing so, the rotational component of the movement is compensated by moving and rotating an already rendered image. With a DLP unit with a standard frame rate of 60 fps ("frames per second") and a "4 × color wheel speed", latencies from the detection of a movement to the end of the display of the motion-compensated image can be achieved down to less than 2 ms while conventional control of the DLP unit requires only at least 33 ms for transmission and display of an image.

Another aspect of the invention relates to a graphics processor configured to drive an image projector in accordance with the respective method steps as described above and below.

Another aspect of the invention relates to a vehicle having a graphics processor as described above and below.

The vehicle may be a car, truck, bus, rail vehicle, watercraft (eg, ship), underwater vehicle, or an aircraft.

Further advantages, features and details will become apparent from the following description in which - where appropriate, with reference to the drawings - at least one embodiment is described in detail. The same, similar and / or functionally identical parts are provided with the same reference numerals.

Show it:

1 a vehicle having a graphics processor and an image projector according to an embodiment of the invention,

2 a color component image with image pixels according to a further embodiment of the invention, and

3 a method for driving an image projector by a graphics processor according to another embodiment of the invention.

The illustrations in the figures are schematic and not to scale.

1 shows a vehicle 1 with an image projector 20 , the head-up display ("HUD") to a driver of the vehicle 1 records visual information. The vehicle 1 also has a data interface 5 which provides image information with multiple color components to a graphics processor 10 supplies. The graphics processor 10 generates a color component image from the image information for each color component based on the image information 50 each with a number N picture lines 52 and each with a number M of image pixels 54 per image line 52 , A transmission unit 15 that between the graphics processor 10 and the image projector 20 is arranged, sequentially transfers the color component images 50 from the graphics processor 10 to the image projector 20 , Furthermore provides a kinematic measuring device 30 kinematic data from inertial sensors.

2 shows a color component image 50 each with a number N picture lines 52 and each with a number M of image pixels 54 per image line 52 , Here, N = 7 and M = 11. Here is a distance between immediately adjacent image pixels 54 for all image pixels 54 identical in the row direction. That is, all pairwise immediately adjacent image pixels 54 in a picture line 52 have the same distance from each other. Furthermore, the relative position of the N picture lines 52 with respect to their position in the row direction to each other arranged such that a beginning of every second image line 52 of the color component image 50 relative to an immediately adjacent image line 52 by half the distance between two immediately adjacent image pixels 54 a picture line 52 opposite one to the picture lines 52 right-angled reference line 56 is offset. 3 shows a method for driving an image projector 20 through a graphics processor 10 , The method sees in a first step S1 the provision of image information with a plurality of color components to the graphics processor 10 through a data interface 5 in front. The image information is preferably a data record of a CPU (Central Processing Unit), which contains information about the colors of the image to be finally displayed. Based on the image information, in a second step S2, a respective color component image is obtained for each of the red-green-blue color components included in the image information 50 each with a number N picture lines 52 and each with a number M of image pixels 54 per image line 52 through the graphics processor 10 generated. In a third step S3, an estimate of a relative movement of the image projector is found 20 towards an environment of the image projector 20 by a kinematic measuring device 30 instead of. The kinematic measuring device 30 is preferably a device for inertial measurement, also called IMU ("inertial measurement unit"), with a gyroscope for determining an orientation of the measuring device 30 and with acceleration sensors. In a fourth step S4, furthermore, determining a compensation function for compensating the relative movement for the color component image on the basis of this estimated relative movement by the graphics processor 10 instead of. Here, the color component image is modified such that an overlaid object is displayed as if it were fixedly connected to the real environment around the image projector. In a fifth step S5, the color component images become 50 one by one and sequentially from the graphics processor 10 to the image projector 20 through a transmission unit 15 transfer. In the sense of native control of the image projector 20 become these sequentially transmitted color component images 50 equally sequentially in a sixth step S6 by the image projector 20 , preferably a HUD ("head-up display"), output as optical information. Each of the color component images 50 is repeated by the transmission unit 15 transmitted per image information. A complete sequence of color component images to represent exactly one image information 50 In this case, the color components red-green-blue (RGB) are repeatedly displayed in such a way that the image information "RGB" is transmitted as "RGB-RGB-RGB-RGB", where "R" is the color component image 50 with the color component "red" describes. In exactly this order gives the image projector 20 that is the HUD, the color component images 50 As a visual information, which gives a viewer the impression of a colorful image due to the rapid color change, without being able to distinguish the individual color components that make up the visual representation.

Although the invention has been further illustrated and explained in detail by way of preferred embodiments, the invention is not limited by the disclosed examples, and other variations can be derived therefrom by those skilled in the art without departing from the scope of the invention. It is therefore clear that a multitude of possible variations exists. It is also to be understood that exemplified embodiments are really only examples that are not to be construed in any way as limiting the scope, applicability, or configuration of the invention. Rather, the foregoing description and description enable the skilled artisan to practice the exemplary embodiments, and those of skill in the knowledge of the disclosed inventive concept may make various changes, for example, to the function or arrangement of particular elements recited in an exemplary embodiment. without departing from the scope defined by the claims and their legal equivalents, such as further explanation in the specification.

LIST OF REFERENCE NUMBERS

1

vehicle

5

Data Interface

10

graphics processor

15

transmission unit

20

image viewer

30

kinematic measuring device

50

Color component image

52

image line

54

image pixels

56

reference line

S1

Provide

S2

Produce

S3

Estimate

S4

Determine

S5

Sequential transfer

S6

Sequential presentation

Claims (10)

Method for driving an image projector ( 20 ) by a graphics processor ( 10 ), comprising the steps: - providing (S1) image information with a plurality of color components to the graphics processor ( 10 ) through a data interface ( 5 ), - based on the image information for each of the color components generating (S2) a color component image ( 50 ) each with a number N picture lines ( 52 ) and each with a number M of image pixels ( 54 ) per image line ( 52 ) through the graphics processor ( 10 ), where a distance between immediately adjacent image pixels ( 54 ) for all image pixels ( 54 ) is identical in the row direction, - sequential transfer (S5) of the color component images ( 50 ) from the graphics processor ( 10 ) to the image projector ( 20 ) by a transmission unit ( 15 ), and - sequentially displaying (S6) the color component images ( 50 ) through the image projector ( 20 ). Method according to claim 1, wherein a complete sequence of color component images ( 50 ) through the image projector ( 20 ) is repeatedly shown n times. Method according to one of the preceding claims, wherein the relative position of the N picture lines ( 52 ) in the row direction in the respective color component image ( 50 ) to each other through the graphics processor ( 10 ) is arranged such that a beginning of every second picture lines ( 52 ) of the color component image ( 50 ) relative to an immediately adjacent image line ( 52 ) by half the distance between two immediately adjacent image pixels ( 54 ) of an image line ( 52 ) opposite one to the picture lines ( 52 ) right-angled reference line ( 56 ) is offset. Method according to one of the preceding claims, wherein the transmission unit ( 15 ) from the graphics processor ( 10 ) to the image projector ( 20 ) a difference between two color component images ( 50 ) of different color components is transmitted from a common image information. Method according to one of the preceding claims, wherein the transmission unit ( 15 ) from the graphics processor ( 10 ) to the image projector ( 20 ) a difference between two color component images ( 50 ) is transmitted to the same color component of different image information. Method according to one of the preceding claims, wherein the arrangement of the relative position of the N picture lines ( 52 ) in the row direction in the respective color component image ( 50 ) to each other through the graphics processor ( 10 ) in a warping unit of the graphics processor ( 10 ) takes place. Method according to any preceding claim, wherein the color component images ( 50 ) each have the color components red, green and blue. Method according to one of the preceding claims, further comprising the steps of: estimating (S3) a relative movement of the image projector ( 20 ) to an environment of the image projector ( 20 ) by a kinematic measuring device ( 30 ), and - determining (S4) a compensation function for compensating the relative movement for the color component image on the basis of the estimated relative movement by the graphics processor ( 10 ). Method according to any preceding claim, wherein the image projector ( 20 ) in a vehicle ( 1 ), and wherein a rotational component of a self-movement of the vehicle ( 1 ) compared to a stationary from the image projector ( 20 ) to be displayed by moving and / or rotating the by the graphics processor ( 10 ) generated color component image ( 50 ) is compensated. Graphics processor ( 10 ) that is running, an image projector ( 20 ) in accordance with the corresponding method steps according to one of claims 1 to 9.

Images