DE102015116405A1 - Display device with pre-distortion - Google Patents

Display device with pre-distortion

Info

Publication number
DE102015116405A1
DE102015116405A1 DE102015116405.1A DE102015116405A DE102015116405A1 DE 102015116405 A1 DE102015116405 A1 DE 102015116405A1 DE 102015116405 A DE102015116405 A DE 102015116405A DE 102015116405 A1 DE102015116405 A1 DE 102015116405A1
Authority
DE
Germany
Prior art keywords
image
distortion
eye
display
displaying
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
DE102015116405.1A
Other languages
German (de)
Inventor
Dietmar Gängler
Norbert Kerwien
Christoph-Hilmar Graf vom Hagen
Philipp Jester
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Carl Zeiss AG
Original Assignee
Carl Zeiss AG
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Carl Zeiss AG filed Critical Carl Zeiss AG
Priority to DE102015116405.1A priority Critical patent/DE102015116405A1/en
Publication of DE102015116405A1 publication Critical patent/DE102015116405A1/en
Withdrawn legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013Eye tracking input arrangements
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS, OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/0025Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 for optical correction, e.g. distorsion, aberration
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS, OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/017Head mounted
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformation in the plane of the image
    • G06T3/0006Affine transformations
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS, OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/0101Head-up displays characterised by optical features
    • G02B2027/011Head-up displays characterised by optical features comprising device for correcting geometrical aberrations, distortion
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS, OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/0101Head-up displays characterised by optical features
    • G02B2027/014Head-up displays characterised by optical features comprising information/image processing systems

Abstract

There are provided methods and apparatus for displaying images in which a pre-distortion of an image to be displayed is performed. In this case, translational decentrations, rotational decentrations and / or color fringes can be taken into account.

Description

  • The present application relates to a display device with predistortion and corresponding methods. In particular, the present application relates to corresponding head-mounted display devices, also referred to as a "head-mounted display" (HMD).
  • In order to avoid distortion and color fringes, a digital pre-distortion can be used in generic display devices. In this case, essentially an image which is displayed on a screen of the head-mounted display is pre-recorded in such a way that a distortion is fully or partially compensated by an optical system of the display device. Conventional types of pre-distortion are usually based on an eye positioned on an optical axis of the optical system.
  • The US 2014/0078023 A1 In addition to conventional polychromatic predistortion, it also treats an empirical determination of an optimal, user-dependent predistortion. Here, image contents are shifted according to different eye distances.
  • The US 2013/0241 947 also discloses a conventional method for correcting polychromatic distortions for an RGB (red, green, blue) screen.
  • The US 2013/0169943 A1 discloses a calibration process for measuring and correcting polychromatic pre-distortion.
  • The US 2010/0091027 A1 discloses an apparatus and method for determining user-dependent polychromatic predistortion.
  • The DE 10 2006 046 367 A1 describes an imaging device that displays an electronically monochromatically pre-recorded image in one direction.
  • D. Pohl, G. Johnson and T. Bolkart, "Improved Pre-Warping for Wide Angle, Head Mounted Displays", VRST'13 Proceedings of the 19th ACM Symposium on Virtual Reality Software and Technology also describes possibilities for pre-distortion for head-mounted displays.
  • It is an object of the present application to provide improved apparatus and methods for correcting aberrations, particularly distortions and color fringing.
  • There is provided a display device according to claim 1 or 10 and a method according to claim 14 or 23. The subclaims define further embodiments.
  • According to a first aspect, there is provided a display device comprising:
    an imaging device for displaying an image,
    an optical system for viewing the displayed image with at least one eye of a viewer, and
    a computing device, which is set up to control the imaging device for displaying the image, wherein the computing device is set up to pre-record the image to be displayed in such a way that a distortion is at least partially compensated by the optical system,
    the pre-recording being dependent on a translational decentering of the at least one eye in which a position of the eye is offset to an optical axis of the optical system and / or in dependence on a rotatory decentering, in which a viewing direction of the eye is not in one direction the optical axis coincides takes place.
  • By a predistortion, which takes into account decentrations of the eye, a better display quality can be achieved.
  • The predistortion can be calculated by means of a coordinate transformation function, which depends on the translatory decentration and / or the rotatory decentration.
  • The coordinate transformation function may be approximated as a polynomial model, where the polynomial model does not have rotationally symmetric terms. Thus, translational and rotatory decentrations can be approximated. Looking along the optical axis and on the optical axis arranged, ie not decentered, eye, the pre-distortion would be rotationally symmetric. Decentration breaks this rotational symmetry. Therefore, with conventional systems that use only rotationally symmetric terms, decentrations can not be considered.
  • The calculating means may be arranged to subdivide a visual field of the at least one eye into a plurality of regions, to determine a pre-distortion for each of the regions for a viewing direction of the eye in the direction of the region, and an entire pre-distortion for the image to be displayed on the basis of the pre-distortions for the areas to be determined.
  • By means of such a division, a rotatory decentration of the eye can be relatively largely taken into account with relatively little computational effort, without the need for an "eye tracking" system or the like.
  • The display device may further comprise a device for determining a viewing direction of the at least one eye, wherein the computing device may be configured to determine the pre-distortion on the basis of the viewing direction.
  • Thus, the pre-distortion can be determined as a function of the rotational decentration time-dependent.
  • The computing device may be configured to determine the pre-distortion based on a model of the optical system.
  • Alternatively, the computing means may be arranged to determine the pre-distortion based on pre-emphasis functions stored in a memory.
  • The computing device can be set up to determine the pre-distortion in a wavelength-dependent manner. So color fringes can be avoided.
  • The computing device may be configured to determine the pre-distortion according to a color-dependent image quality criterion for color fringes.
  • Disturbing color fringes can be reduced by a color-dependent image quality criterion, whereby it can be taken into account in particular that some colors are perceived as more disturbing than others.
  • According to a second aspect, there is provided a display device comprising:
    an imaging device for displaying an image,
    an optical system for viewing the displayed image with at least one eye of the observer, and
    a computing device for controlling the imaging device for displaying the image, wherein the computing device is set up to pre-record the image to be displayed, characterized in that the wave-dependent pre-recording is performed by optimization as a function of a wavelength-dependent image quality criterion.
  • By a color-dependent image quality criterion disturbing color fringes can be reduced, in particular it can be taken into account that some colors are more disturbing than others.
  • For optimization, n CC -1-free parameters can be introduced, where n CC is a number of color channels which are taken into account in the predistortion.
  • The free parameters can determine magnifications of scales of coordinate transformations for the corresponding color channels.
  • The display device may be a device to be worn on the head.
  • In other aspects, appropriate methods are provided.
  • The invention will be explained in more detail below with reference to the accompanying drawings with reference to embodiments. Show it:
  • 1 a schematic representation of a head-mounted display,
  • 2 a representation of the head-mounted display of 1 to illustrate a visual field,
  • 3 a representation of the head-mounted display of 1 to illustrate a translational decentering,
  • 4 a representation of the head-mounted display of 1 to illustrate a rotatory decentration,
  • 5 a representation of a part of a head-mounted display according to an embodiment,
  • 6 - 8th Representations to illustrate a Vorverzeichnung,
  • 9 FIG. 2 is a schematic representation for illustrating the calculation of a predecessor according to some embodiments. FIG.
  • 10 4 is a representation to illustrate the division into different visual field areas according to an exemplary embodiment,
  • 11 a representation of a head-mounted display to illustrate a convergence angle, and
  • 12 and 13 schematic representations illustrating a Vorverzeichnung in some embodiments.
  • The invention will be explained in more detail below with reference to exemplary embodiments. These embodiments are not to be construed as limiting and are merely illustrative. A description of an embodiment having a plurality of features or elements is not to be construed as requiring all such features or elements to practice the invention. Rather, for example, some features or elements may be omitted and / or replaced by alternative features or elements. Also, additional features or elements may be provided for what is explicitly illustrated and described, for example, elements used in conventional head-mounted displays. Features or elements of various embodiments may be combined with each other unless otherwise specified. For a better understanding, an example of a structure of a head-mounted display as well as various decentrations will be explained in more detail.
  • The 1 shows a schematic representation of a head-mounted display (HMD). The head-mounted display has a screen 12 on to display pictures. The term "image" is to be understood broadly, and such images may contain characters, symbols, data, still images or moving images (videos). The screen 12 For example, it may be an organic light emitting diode (OLED) based screen, an LCD screen, a TFT screen, or any other conventional type of screen. The screen 12 can be curved or even. Instead of such a screen, another imaging element representing an image may also be used, for example a laser scanner in conjunction with a ground-glass screen or the like. An image presented in this way will have an appearance 11 to an eye 10 of a viewer projects (shown). The optics 11 is in the example of 1 however, the optics can also comprise a plurality of lenses or other optical elements, for example diffractive optical elements. For low-cost and / or lightweight head-mounted displays, however, often only a single lens is used.
  • With 13 is an optical axis of the system referred to, in the example shown by a central axis of the lens 11 and at the in 1 illustrated alignment of the eye 10 also through a central axis of a lens of the eye 10 runs and with the line of sight of the eye 10 matches.
  • In reality, the eye can 10 move. This is in 2 for the head-mounted display of the 1 shown. The same reference numerals in the figures below designate the same or corresponding elements and will therefore not be explained again in more detail.
  • In 2 is illustratively shown that the eye 10 yourself within an angle 21 can turn. The angle 21 (which is a solid angle, in the figure shown only two-dimensional) limits or defines a field of view, which, as in 2 is limited in the two-dimensional representation of the 2 through lines 20A . 20B ,
  • In conventional systems, a pre-distortion is one on the screen 12 represented image to compensate for distortions by the optical system 11 for the in 1 and 2 illustrated eye position designed, ie the eye 10 looks along the optical axis 13 and is on the optical axis 13 centered. In practical use, however, deviations from this optimum position may occur depending on the optical system 11 can also change the perceived distortion. Such deviations are in the 3 and 4 shown.
  • In 3 is the eye 10 opposite the optical axis 13 offset on a line 30 arranged and has a line of sight along the line 30 parallel to the optical axis 13 on. An offset of the eye 10 to the optical axis 13 as in 3 is hereinafter referred to as translational decentering of the eye.
  • Such translational decentrations as in 3 can arise, for example, that a particular head-mounted display is designed for a certain eye relief, and the eye relief of the user deviates from this predetermined eye distance.
  • One way to compensate for this, are adjustable head-mounted displays, in which the optical system can be adapted to the respective eye relief, for example by appropriate shifts of optical elements. In embodiments of the present invention, the translational decentering caused by such eye distances, which changes the distortion, is compensated by a corresponding predistortion.
  • In 4 are three different directions of the eye 10 through three different pupil positions 42A . 42B . 42C as an example. The pupil position 42A corresponds to the in 2 position shown. The position 42B looks along a line 40 and is to the position 42A at an angle 43 twisted. In the position 42C the eye looks along the line 41 , For every position, ie for every angle 43 , which may be due to the characteristics of the optical system 11 change perceived distortion. A twist of the eye 10 through which the eye 10 no longer parallel to the optical axis 13 looks, is in the following referred to as rotary decentration. Rotational decentration is generally time-dependent, as the eye usually moves.
  • It should be noted that the 3 and 4 two-dimensional representations are. In real systems, the decentrations shown may be two-dimensional, which, for example, in the case of the translational decentering of the 3 by two coordinates x, y, where the x and y axes are perpendicular to each other and perpendicular to the optical axis 13 can be described. In the case of rotatory decentration, the decentering can be described by two angles, for example, an angle to the angle 43 of the 4 corresponds and the other angle is perpendicular thereto. These angles are also referred to below as φ x , φ y .
  • Next, some mathematical fundamentals of distortion correction will be explained. First of all, a conventional correction of the distortion in the monochromatic case, i. H. without consideration of different colors explained.
  • As explained, since HMDs are typically projecting optical systems, that is to say include a screen or other imaging devices, the data to be displayed can be manipulated such that the resulting distortion and color fringes in the displayed image become minimal. This procedure is referred to below as digital pre-distortion (VVZ). For example, an in 6 illustrated test object with a barrel distortion, as in 8th to be pre-recorded, so that at a pillow-shaped distortion of the optical system 11 , as in 7 shown, lines in the picture are perceived as straight. For a position (x o , y o ) T in the coordinate system of the data to be displayed, the VVZ can be used as a coordinate transformation
    Figure DE102015116405A1_0002
    Figure DE102015116405A1_0003
    with a position (x i , y i ) describe T in the pre-recorded image. As a simple model, the coordinate transformation F often becomes a radially symmetric model
    Figure DE102015116405A1_0004
    with r 2 = x o 2 + y o 2 and coefficients k i approximated.
  • In the case of colored images, the wavelength dependence of the distortion can also be taken into account. Since light for each wavelength λ in the optical system (eg. 11 of the 1 - 4 ), the coordinate transformation F depends on λ. Screens typically display three colors, red (R), green (G), and blue (B). Each color represents a partial spectrum in the corresponding part of the color spectrum. Since only one coordinate transformation can be used for each color channel (partial spectrum), the channels are usually identified with one wavelength, in which case three coordinate transformations then result
    Figure DE102015116405A1_0005
    In general, then
    Figure DE102015116405A1_0006
  • Where λ denotes the wavelength. In the case of three wavelengths used or narrow-band wavelength ranges as in RGB screens, the red, green and blue portions can be treated separately as explained above. Usually, the peak wavelengths of the emission spectra of the screen are used to correct the polychromatic distortion. Conventionally, the digital VVZ is only generally used for the compensation of polychromatic distortion, for. B. to reduce color fringes.
  • With fixed left and right eye optical channels (eg, fixed optical systems), different eye distances act as horizontal decentering of the eye relative to the optical system, as in Figs 3 shown. In general, translational decentrations can be described as a displacement of the eye relative to the optical system by (x D , y D , z D ) T. Distortion and color fringing are dependent on this decentering. For the coordinate transformation of the polychromatic VVZ is thus general
    Figure DE102015116405A1_0007
    Manufacturing tolerances, such as height errors of optical elements, can also be described as decentrations and taken into account in the distortion correction and compensated. Generally, in a left and right eye binocular system, different coordinate transformations F are possible. The function F thus depends here (in polychromatic case) for each eye both on the wavelength λ and from the translational decentering (x D, y D, z D) T ab.
  • In embodiments, decentration may be determined or determined, for example, with an eye tracker or by entering an eye distance, and then applying a corresponding function F for pre-distortion. As will be explained later, the function F can be directly derived from the design of the optical system 11 , For example, from the data of the lenses used to be calculated. These data are available in commercially manufactured optical designs since they are determined by a corresponding optical designer prior to fabrication.
  • Additionally or alternatively, in embodiments, the above with reference to 4 be considered rotary de-centering described. Taking into account both the translational decentration and the rotatory decentration come to the already described degrees of freedom two rotations z. B. to the x and y-axis, so two angles (eg., Φ x and φ y ), added. In general, the eye moves around these angles, they are thus time-dependent, it follows:
    Figure DE102015116405A1_0008
  • In embodiments in which only the rotatory decentration is taken into account, the dependence on (x D , y D , z D ) T is omitted. In monochromatic embodiments in which different wavelengths are not taken into account or play no role, the dependence on λ is eliminated.
  • It should be noted that the rotationally symmetric model used in conventional pre-nomenclature systems, as mentioned above, is generally unsuited to account for translational decentrations and / or rotatory decentrations.
  • It should also be noted that in binocular vision, ie seeing with both eyes, for example in a head-mounted display, a rotatory decentering is the rule, since both eyes are usually focused on a common point in the image plane. This is schematically in 11 shown. Here are two eyes 10A . 10B a common screen 12 by respective optical systems 11A . 11B , Instead of a common screen 12 For example, which may have different areas for the right and left eyes, separate screens or other imaging devices may also be used. 13A . 13B denote the respective optical axes of the optical systems 11A . 11B , and 121 denotes a central axis. With 120 is called a virtual image plane. 122 denotes a so-called convergence angle α. The convergence angle α depends on the position of the virtual image. The human eye is accustomed to the convergence angle of the two eyes and their accommodation to the distance of the perceived object to each other. If this is not the case, it may cause discomfort.
  • When calculating a digital predistortion, the convergence angle is therefore taken into account in embodiments in the calculation of the image center point.
  • In embodiments of the present invention, an optimal pre-recorded image can thus be offered to a user, in which decentrations of a translational and / or rotational nature, in particular also varying viewing directions of the eye or eyes, and / or different eye distances can be taken into account. In the most general case, each optical channel (i.e., each eye) is pre-recorded wavelength dependent for three translational, time constant, and two rotational, time dependent decentrations. In a translational decentering it is sufficient to determine this once when the HMD (or other display device) z. B. is always used by the same person and is put on the same, so that the translational decentering is the same every time you use the display device. Otherwise, the translational decentering can also be determined each time the display device is used. This determination can be made by measuring the pupillary distance or the pupil position or can be determined by simple empirical application tests. For the compensation of the rotatory decentrations of the rolling eye, the eye position is detected time-resolved in some embodiments. For this example, an eye-tracking system can be used. In polychromatic predistortion, the wavelengths to be corrected may be chosen to account for the transmission of the optical elements of the system and the sensitivity of the eye. In other embodiments, as will be explained later, a time-independent pre-distortion is used in which it is exploited that an eye sees only sharply in a relatively small area.
  • In further exemplary embodiments, the polychromatic predistortion of the type is optimized in addition or as an alternative to the further reduction of the color fringes so that remaining fringes appear as narrow as possible and inconspicuous in color. For this purpose, color fringe calculations are considered in the optimization of the optical design and optimal polychromatic coordinate transformations are determined. For this purpose, in particular color-dependent image quality criteria can be used to optimize the pre-distortion.
  • The 5 shows an embodiment with which such a pre-distortion can be achieved. Here are in 5 next to the screen shown only additional components to those already discussed, and the remaining components such as the 1 , in particular a corresponding optical system 11 , may also be present.
  • In the embodiment of the 5 becomes the screen 12 via a computing device 50 driven, which one or more processors 51 includes. The computing device 50 also has a memory 52 on. In the store 52 For example, functions for digital pre-distortion F can be provided for different translational and rotational decentrations and / or for different colors. In addition, in the memory 52 on the screen 12 be displayed data stored. Additionally or alternatively, a communication interface 53 , For example, a wireless interface or a wired interface, be provided, via which the computing device 50 can receive data to be displayed. In addition, an optional eye-tracker device 54 for determining a viewing direction of the eye, for example, to determine the above-mentioned angle φ x and φ y , be provided.
  • In another embodiment, a suitable Vorverzeichnung, z. B. a function F of the optical system 11 through the processor 51 calculated online during presentation based on the optical design. For example, this calculation may be done based on data provided by the eye tracker 54 to be delivered. One possible principle of such a calculation is in 9 shown schematically.
  • In such a calculation is for a specific position of the eye 10 calculates a position P '(x, y) at which one of the eye 10 outgoing virtual light beam 90 on the screen 12 meets. This calculation is based on the design of the optical system 11 and may be carried out by conventional optically-used methods. By comparison with a position of the point P (x, y) in the unmarked case, the function F can then be determined. This calculation can be done online, ie during the presentation, or even in advance, and corresponding information can for example in the memory 52 of the 5 be filed.
  • In the following, possibilities for the correction of the translatory decentration and the rotatory decentration will be explained in more detail. For a translationally decentered eye, a corresponding non-rotationally symmetric distortion grating z. As with reference to 9 explained from the optical design of the optical system 11 be calculated. Data to display, e.g. B. an object to be displayed is pre-recorded according to this grid. This coordinate transformation, in some embodiments, may be approximated by a non-rotationally symmetric mathematical model, such as polynomials or Legendre polynomials, or, in other embodiments, an interpolation operator F ≡ Q may be used directly. For given points (x o ', y o ') T and (x i ', y i ') T (ie it is given that the predistortion the point (x o ', y o ') to the point (x i ', y i ') T ), the interpolation operator Q calculates for the given points (x o , y o ) T the pre-marked points (x i , y i ) T. In general, the points (x o ', y o ') T and / or (x i ', y i ') T can depend on translational and / or rotational decentrations as well as on the wavelength λ. The following applies:
    Figure DE102015116405A1_0009
  • For example, a bi-linear or spline interpolation operator may be used for Q. By using such an interpolation Q so z. For example, the pre-distortion for some points can be specified (eg stored) and determined for the remaining points by interpolation.
  • As a non-rotationally symmetric mathematical model, for example, a polynomial model for approximating the coordinate transformation F can be used for a point (x, y) T :
    Figure DE102015116405A1_0010
  • As can be seen, non-rotationally symmetric terms (for example x 2 y or xy 2 ) are also taken into account. In other embodiments, a z-dependence (displacement along the optical axis) may also be considered. The number of coefficients used can be selected according to a desired accuracy of the distortion correction. The function can be determined and applied separately for different color channels.
  • In particular, different eye distances can thus be compensated without a shift of the optical channels. In 12 and 13 Exemplary polychromatic pre-recorded test objects for a centered and translationally decentered eye are shown.
  • As can be seen, the pre-distortion differs in 13 from the pre-distortion in 12 , where the change can also be color-dependent. Thus, the translational decentering can be taken into account in the pre-distortion.
  • Next, the correction of the rotatory decentering of the eye will be explained. In this case, first of all a first solution according to an exemplary embodiment is shown in which the computational effort is comparatively low. Then, a second solution is shown according to another embodiment, in which the computational effort is higher, but also the quality of the representation can be increased.
  • The first solution makes use of the fact that the eye sees only the immediate center in the direction of view, while peripherally it can only be seen blurred. Distortion and fringing are therefore less noticeable in peripheral vision, as this area is perceived only fuzzy anyway. Therefore, in embodiments, the field of view (FoV) of the optical system is decomposed into areas, each having its own predistortion, in particular polychromatic predistortion, for the respective viewing angle to the respective area, for example for the viewpoint to a center (center or centroid) of the respective area. To illustrate, the 10 three such areas 101 . 102 and 103 for a visual field 100 , In the example shown, the area is located 101 in the center of the visual field 100 , The area 103 at the edge of the field of vision 100 and the area 102 between. The remaining field of vision 100 can be divided into similar areas. For each area, a predistortion, for example for a viewing angle φ x , φ y to the center of the respective area, is calculated. Where areas overlap, then a distortion in the overlap area between the distortions for the respective areas can be interpolated. In this way, cracks are avoided at the edges of the areas. Thus, an entire pre-distortion and thus also a pre-recorded image can be determined which has a good correction of distortion and in particular color fringe for each calculated viewing direction in the area of sharp vision. In such an embodiment, the time dependence of the angles φ x and φ y need not be taken into account. Therefore, the computational effort is less than in the embodiment shown below, which takes into account the time dependency.
  • If the entire field can be corrected for a current angle with a correct Vorverzeichnung, the time dependence may the angle φ x and φ y not be neglected. The viewing angle is, for example, by a conventional eye-tracking system as in 5 shown in real time and the entire image to be displayed according to the time-dependent coordinate transformation
    Figure DE102015116405A1_0011
    vorverzeichnet. This procedure also enables optimal digital pre-distortion of the object in the area of peripheral vision and thus avoids or reduces residual distortion and color fringing. For this, the computational effort is generally higher than the embodiment, which with reference to the 10 has been described, and which uses a decomposition of the visual field into areas.
  • Conventional color fringe correction simply attempts to minimize color fringing. However, this does not take into account that it also depends on the color of a fringing, whether this is perceived as disturbing. For example, a magenta color fringe is often perceived more strongly than a green fringe.
  • Therefore, in some embodiments, a color-dependent image quality criterion is defined for the optimization of the perceived color fringes, and for the optical system to be optimized (to be pre-recorded), image simulations are calculated, for example, As with reference to 9 explained. In the polychromatic predistortion are now introduced n CC - 1 other free parameters, where n CC is the number of color channels (eg, n CC = 3 for RGB representation). For example, to optimize the predistortion for a RGB image
    Figure DE102015116405A1_0012
  • R denotes the red channel, B the blue channel and G the green channel. M R is the mentioned additional parameter for the red channel, M B for the blue channel.
  • The parameters M R , M B may be, for example, additional magnifications of the scales of the coordinate transformations in the red and blue channels. The optimizer now seeks, according to the image quality criterion, optimal parameters for the suppression of disturbing color fringes. For such an optimizer, conventional optimization algorithms may be used which, for example, vary the parameters M R , M B to maximize or minimize the image quality criterion (depending on the formulation of the criterion). For example, the criterion may include a weighting of remaining color fringes such that, for example, magenta remaining color fringes are weighted higher than green remaining fringes. Optimization then minimizes the so-weighted color fringes.
  • The illustrated exemplary embodiments can provide optimized, individual, time-constant or time-dependent digital predistortion and thus avoid distortion errors and / or color fringes, in particular in off-axis field regions when the eye is rotating (rotatory decentration), translational decentration and / or peripheral vision.
  • The application of the present invention is not limited to head-mounted displays, but may be generally used when an imaging device is viewed through an optical system causing distortion and / or color bleeding. In a binocular system, left and right eye channels can be pre-recorded differently. The field of view may be more than 65 ° in such systems, for example more than 80 °, more than 90 ° or more than 105 °. The remaining distortion may be less than 30%, and the color error may be between 0 and 30 pixels on the object side.
  • QUOTES INCLUDE IN THE DESCRIPTION
  • This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.
  • Cited patent literature
    • US 2014/0078023 A1 [0003]
    • US 2013/0241947 [0004]
    • US 2013/0169943 A1 [0005]
    • US 2010/0091027 A1 [0006]
    • DE 102006046367 A1 [0007]
  • Cited non-patent literature
    • D. Pohl, G. Johnson and T. Bolkart, "Improved Pre-Warping for Wide Angle, Head Mounted Displays", VRST'13 Proceedings of the 19th ACM Symposium on Virtual Reality Software and Technology [0008]

Claims (25)

  1. A display device, comprising: an imaging device ( 12 ) for displaying an image, an optical system ( 11 ) for viewing the displayed image with at least one eye ( 10 ) of a viewer, and a computing device ( 50 ), which is set up, the imaging device ( 12 ) to display the image, wherein the computing device ( 50 ) is arranged to pre-record the image to be displayed in such a way that a distortion by the optical system ( 11 ) at least partially compensated, characterized in that the pre-recording in dependence on a translational decentering of the at least one eye, in which a position of the eye to an optical axis ( 13 ) of the optical system ( 11 ), and / or depending on a rotatory decentering, in which a viewing direction of the eye ( 10 ) not with a direction of the optical axis ( 13 ) matches.
  2. A display device according to claim 1, wherein the pre-distortion is calculated by means of a coordinate transformation function which depends on the translational decentering and / or the rotational decentration.
  3. A display device according to claim 2, wherein said translational decenter coordinate transformation function approximates a polynomial model, said polynomial model having non-rotationally symmetric terms.
  4. Display device according to one of claims 1-3, wherein the computing device ( 50 ), a visual field of the at least one eye ( 10 ) into a plurality of areas, a pre-distortion for each of the areas for a viewing direction of the eye ( 10 ) in the direction of the area, and to determine a total pre-distortion for the image to be displayed on the basis of the pre-distortions for the areas.
  5. Display device according to one of claims 1-3, further comprising means ( 54 ) for determining a viewing direction of the at least one eye, wherein the computing device ( 50 ) is arranged to determine the pre-distortion on the basis of the viewing direction.
  6. Display device according to one of claims 1-5, wherein the computing device ( 50 ), the pre-distortion based on a design of the optical system ( 11 ).
  7. Display device according to one of claims 1-5, wherein the computing device ( 50 ) is arranged to determine the pre-distortion based on pre-emphasis functions stored in a memory.
  8. Display device according to one of claims 1-7, wherein the computing device ( 50 ) is arranged to determine the pre-distortion wavelength dependent.
  9. Display device according to claim 8, wherein the computing device ( 50 ) is arranged to determine the pre-distortion according to a color-dependent image quality criterion for color fringes.
  10. A display device, comprising: an imaging device ( 12 ) for displaying an image, an optical system ( 11 ) for viewing the displayed image with at least one eye of the observer, and a computing device ( 50 ) for driving the imaging device ( 12 ) for displaying the image, wherein the computing device ( 50 ) is arranged to pre-record the image to be displayed on a wavelength-dependent basis, characterized in that the wave-dependent pre-recording is performed by optimization as a function of a wavelength-dependent image quality criterion.
  11. A display device according to claim 9 or 10, wherein for optimization n CC -1 free parameters are introduced, wherein n CC is a number of color channels which are taken into account in the pre-distortion.
  12. The display device of claim 11, wherein the free parameters determine magnifications of scales of coordinate transformations for the respective color channels.
  13. A display device according to any one of claims 1-12, wherein the display device is a head-worn device.
  14. A method of displaying an image with a display device, the display device comprising: an imaging device ( 12 ) for displaying an image, and an optical system ( 11 ) for viewing the displayed image with at least one eye of an observer, the method comprising: driving the imaging device ( 12 ) for displaying the image, wherein the image to be displayed is pre-recorded such that a distortion by the optical system ( 11 ) is at least partially compensated, characterized in that the pre-recording in dependence on a translational decentering of the at least one eye, in which a position of the eye to an optical axis ( 13 ) of the optical system ( 11 ), and / or depending on a rotatory decentering, in which a viewing direction of the eye ( 10 ) not with a direction of the optical axis ( 13 ) matches.
  15. A method of displaying an image according to claim 14, wherein the pre-distortion is calculated by means of a coordinate transformation function which depends on the translational decentering and / or the rotatory decentration.
  16. A method of displaying an image according to claim 15, wherein said translational decenter coordinate transformation function approximates a polynomial model, said polynomial model having non-rotationally symmetric terms.
  17. A method of displaying an image according to any of claims 14-16, the method comprising: dividing a field of view of the at least one eye ( 10 ) in a plurality of areas, determining a pre-distortion for each of the areas for a viewing direction of the eye ( 10 ) in the direction of the area, and determining an overall pre-distortion for the image to be displayed on the basis of the pre-distortions for the areas.
  18. A method of displaying an image according to any one of claims 14-16, further comprising: determining a viewing direction of the at least one eye, and determining the pre-distortion based on the viewing direction.
  19. A method of displaying an image according to any of claims 14-18, wherein the pre-distortion is based on a design of the optical system ( 11 ) is determined.
  20. A method of displaying an image according to any one of claims 14-18, wherein said pre-distortion is determined based on pre-emphasis functions stored in a memory.
  21. A method of displaying an image according to any one of claims 14-20, wherein the pre-distortion is determined wavelength dependent.
  22. The method of displaying an image according to claim 21, wherein said pre-distortion is determined according to a color-dependent image quality criterion for color fringes.
  23. A method of displaying an image with a display device, the display device comprising: an imaging device ( 12 ) for displaying an image, and an optical system ( 11 ) for viewing the displayed image with at least one eye of the observer, the method comprising: driving the imaging device ( 12 ) for displaying the image, wherein the image to be displayed is pre-recorded wavelength-dependent, characterized in that the wave-dependent pre-recording is carried out by optimization as a function of a color-dependent image quality criterion.
  24. A method of displaying an image according to claim 22 or 23, wherein for the optimization n CC -1 free parameters are introduced, where n CC is a number of color channels which are taken into account in the pre-distortion.
  25. A method of displaying an image according to claim 24, wherein the further parameters determine magnifications of scales of coordinate transformations for the respective color channels.
DE102015116405.1A 2015-09-28 2015-09-28 Display device with pre-distortion Withdrawn DE102015116405A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
DE102015116405.1A DE102015116405A1 (en) 2015-09-28 2015-09-28 Display device with pre-distortion

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
DE102015116405.1A DE102015116405A1 (en) 2015-09-28 2015-09-28 Display device with pre-distortion

Publications (1)

Publication Number Publication Date
DE102015116405A1 true DE102015116405A1 (en) 2017-03-30

Family

ID=58281662

Family Applications (1)

Application Number Title Priority Date Filing Date
DE102015116405.1A Withdrawn DE102015116405A1 (en) 2015-09-28 2015-09-28 Display device with pre-distortion

Country Status (1)

Country Link
DE (1) DE102015116405A1 (en)

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102006046367A1 (en) 2006-09-29 2008-04-17 Carl Zeiss Ag Imaging device and imaging method with electronic distortion correction
US20100091027A1 (en) 2008-10-14 2010-04-15 Canon Kabushiki Kaisha Image processing apparatus and image processing method
US20130169943A1 (en) 2012-01-02 2013-07-04 Shan-Chieh Wen Image projection device and associated image projection method and calibration method
US20130241947A1 (en) 2012-03-15 2013-09-19 Sony Corporation Display device, image processing device, image processing method, and computer program
DE102012205164A1 (en) 2012-03-29 2013-10-02 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Projection display and method for projecting a total image
US20140078023A1 (en) 2012-03-22 2014-03-20 Sony Corporation Display device, image processing device and image processing method, and computer program
WO2014163869A1 (en) 2013-03-13 2014-10-09 Sony Computer Entertainment Inc. Digital inter-pupillary distance adjustment
DE102013206614A1 (en) 2013-04-12 2014-10-16 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Projection display for reflecting an image to be displayed in a view of an occupant of a means of transport
DE102014113686A1 (en) 2014-09-22 2016-03-24 Carl Zeiss Ag Display device which can be placed on the head of a user, and methods for controlling such a display device
WO2016053735A1 (en) 2014-09-30 2016-04-07 Sony Computer Entertainment Inc. Realtime lens aberration correction from eye tracking

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102006046367A1 (en) 2006-09-29 2008-04-17 Carl Zeiss Ag Imaging device and imaging method with electronic distortion correction
US20100091027A1 (en) 2008-10-14 2010-04-15 Canon Kabushiki Kaisha Image processing apparatus and image processing method
US20130169943A1 (en) 2012-01-02 2013-07-04 Shan-Chieh Wen Image projection device and associated image projection method and calibration method
US20130241947A1 (en) 2012-03-15 2013-09-19 Sony Corporation Display device, image processing device, image processing method, and computer program
US20140078023A1 (en) 2012-03-22 2014-03-20 Sony Corporation Display device, image processing device and image processing method, and computer program
DE102012205164A1 (en) 2012-03-29 2013-10-02 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Projection display and method for projecting a total image
WO2014163869A1 (en) 2013-03-13 2014-10-09 Sony Computer Entertainment Inc. Digital inter-pupillary distance adjustment
DE102013206614A1 (en) 2013-04-12 2014-10-16 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Projection display for reflecting an image to be displayed in a view of an occupant of a means of transport
DE102014113686A1 (en) 2014-09-22 2016-03-24 Carl Zeiss Ag Display device which can be placed on the head of a user, and methods for controlling such a display device
WO2016053735A1 (en) 2014-09-30 2016-04-07 Sony Computer Entertainment Inc. Realtime lens aberration correction from eye tracking

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
D. Pohl, G. Johnson und T. Bolkart, "Improved Pre-Warping for Wide Angle, Head Mounted Displays", VRST'13 Proceedings of the 19th ACM Symposium an Virtual Reality Software and Technology

Similar Documents

Publication Publication Date Title
TWI648559B (en) Light metering system display field
US10345599B2 (en) Tile array for near-ocular display
US10111583B1 (en) System, method, and non-transitory computer-readable storage media related to correction of vision defects using a visual display
US10008043B2 (en) Near-eye parallax barrier displays
KR101844883B1 (en) Apparatuses, methods and systems coupling visual accommodation and visual convergence to the same plane at any depth of an object of interest
Dunn et al. Wide field of view varifocal near-eye display using see-through deformable membrane mirrors
US10162412B2 (en) Display, control method of display, and program
US9928629B2 (en) Combining video-based and optic-based augmented reality in a near eye display
CN105190427B (en) Digital interocular distance is adjusted
US10445860B2 (en) Autofocus virtual reality headset
EP3368170B1 (en) Adjusting image frames based on tracking motion of eyes
JP6502586B2 (en) Virtual reality headset to focus
US10520730B2 (en) Methods and systems for augmented reality
US20180067306A1 (en) Head mounted display
US8970495B1 (en) Image stabilization for color-sequential displays
CA2941306C (en) Eye projection system
KR101803137B1 (en) Projection display and method for projecting an overall image
KR20190130147A (en) Depth-based povided rendering for display systems
US9430055B2 (en) Depth of field control for see-thru display
US20160147067A1 (en) 3-dimensional electro-optical see-through displays
Konrad et al. Accommodation-invariant computational near-eye displays
US8817379B2 (en) Whole image scanning mirror display system
EP2502410B1 (en) A method for augmenting sight
KR20190029795A (en) Display system and method
US9916690B2 (en) Correction of displayed images for users with vision abnormalities

Legal Events

Date Code Title Description
R083 Amendment of/additions to inventor(s)
R163 Identified publications notified
R119 Application deemed withdrawn, or ip right lapsed, due to non-payment of renewal fee