CN110930935A - Extraction type residual image compensation using pixel shift - Google Patents
Extraction type residual image compensation using pixel shift Download PDFInfo
- Publication number
- CN110930935A CN110930935A CN201910822215.4A CN201910822215A CN110930935A CN 110930935 A CN110930935 A CN 110930935A CN 201910822215 A CN201910822215 A CN 201910822215A CN 110930935 A CN110930935 A CN 110930935A
- Authority
- CN
- China
- Prior art keywords
- image frame
- display
- compensated
- receive
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G3/00—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
- G09G3/001—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background
- G09G3/003—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background to produce spatial visual effects
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G3/00—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
- G09G3/20—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters
- G09G3/22—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters using controlled light sources
- G09G3/30—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters using controlled light sources using electroluminescent panels
- G09G3/32—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters using controlled light sources using electroluminescent panels semiconductive, e.g. using light-emitting diodes [LED]
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G3/00—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
- G09G3/006—Electronic inspection or testing of displays and display drivers, e.g. of LED or LCD displays
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G3/00—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
- G09G3/007—Use of pixel shift techniques, e.g. by mechanical shift of the physical pixels or by optical shift of the perceived pixels
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G3/00—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
- G09G3/20—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters
- G09G3/22—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters using controlled light sources
- G09G3/30—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters using controlled light sources using electroluminescent panels
- G09G3/32—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters using controlled light sources using electroluminescent panels semiconductive, e.g. using light-emitting diodes [LED]
- G09G3/3208—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters using controlled light sources using electroluminescent panels semiconductive, e.g. using light-emitting diodes [LED] organic, e.g. using organic light-emitting diodes [OLED]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
- G06F3/147—Digital output to display device ; Cooperation and interconnection of the display device with other functional units using display panels
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G2320/00—Control of display operating conditions
- G09G2320/02—Improving the quality of display appearance
- G09G2320/0233—Improving the luminance or brightness uniformity across the screen
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G2320/00—Control of display operating conditions
- G09G2320/02—Improving the quality of display appearance
- G09G2320/0285—Improving the quality of display appearance using tables for spatial correction of display data
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G2320/00—Control of display operating conditions
- G09G2320/04—Maintaining the quality of display appearance
- G09G2320/043—Preventing or counteracting the effects of ageing
- G09G2320/046—Dealing with screen burn-in prevention or compensation of the effects thereof
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G2330/00—Aspects of power supply; Aspects of display protection and defect management
- G09G2330/10—Dealing with defective pixels
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G2340/00—Aspects of display data processing
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G3/00—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
- G09G3/20—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters
- G09G3/34—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters by control of light from an independent source
- G09G3/36—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters by control of light from an independent source using liquid crystals
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Chemical & Material Sciences (AREA)
- Crystallography & Structural Chemistry (AREA)
- Control Of Indicators Other Than Cathode Ray Tubes (AREA)
- Controls And Circuits For Display Device (AREA)
Abstract
The present disclosure relates to decimated afterimage compensation using pixel shifting. A flat panel display apparatus and method for preventing display panel afterimages by a decimated lookup table and pixel shifting in a display or an augmented reality display is disclosed.
Description
Cross Reference to Related Applications
The present application claims us provisional application serial No. 62/726,876 entitled "DECIMATED BURN-IN compention with PIXEL SHIFTING" filed on 4.9.2018; and U.S. patent application Ser. No. 16/403,366 entitled "DECIMATED BURN-IN COMPENSATION WITH PIXEL SHIFTING", filed on 3.5.2019; the disclosure of which is incorporated herein by reference in its entirety.
Technical Field
Aspects of the present disclosure generally relate to displays. Aspects include a method and apparatus for preventing display panel burn-in by decimating lookup tables and pixel shifting in a flat panel display or an augmented reality display.
Background
Displays are electronic viewing technologies used to enable people to see content, such as still images, moving images, text, or other visual material.
A flat panel display includes a display panel including a plurality of pixels arranged in a matrix format. The display panel includes a plurality of scan lines formed in a row direction (y-axis) and a plurality of data lines formed in a column direction (x-axis). The plurality of scan lines and the plurality of data lines are arranged to cross each other. Each pixel is driven by a scan signal and a data signal supplied from its corresponding scan line and data line.
A flat panel display may be classified as a passive matrix type light emitting display device or an active matrix type light emitting display device. The active matrix panel selectively illuminates each unit pixel. Active matrix panels are used because of their resolution, contrast and operational speed characteristics.
One type of active matrix display is an Active Matrix Organic Light Emitting Diode (AMOLED) display. An active matrix organic light emitting display generates an image by generating light by flowing current to an organic light emitting diode. An organic light emitting diode is a light emitting element in a pixel. A driving Thin Film Transistor (TFT) of each pixel causes a current to flow according to a gradation of image data.
Flat panel displays are used in many portable devices such as laptop computers and mobile phones.
Screen afterimage, image afterimage or ghosting is a regional discoloration on a display due to cumulative non-uniform use of pixels.
In Organic Light Emitting Diode (OLED) displays, a wide variation in luminance decay will result in a significant color shift over time, with one of the red-green-blue (RGB) colors becoming more pronounced due to the luminance decay of the light emitting pixels.
In the case of a Liquid Crystal Display (LCD), the mechanism of the afterimage is different. For LCDs, in some cases, image retention can occur because a pixel permanently loses its ability to return to its relaxed state after a sustained static use profile. In most typical usage profiles, this image persistence in the LCD is only temporary.
In desktop computer applications, "screen saver" software actively attempts to avoid screen retention. Phosphor brightness is maintained by ensuring that a static image is displayed for a long period of time without leaving any pixels or groups of pixels. Modern screen savers can turn the screen off when it is not in use.
Disclosure of Invention
Embodiments include an electronic display designed to prevent display panel image retention by way of a decimated lookup table and pixel shifting in a flat panel display or an augmented reality display.
In one embodiment, an electronic display includes a display panel and a removable afterimage compensator. The decimated afterimage compensator is configured to receive an image frame and output a compensated output frame to a display panel. The decimation afterimage compensator further comprises a pixel shifter, a down sampler, an NxN bin compensation look-up table, an interpolator and a multiplier. The pixel shifter is configured to receive an image frame and shift the image frame by a predetermined number of pixels, resulting in a shifted image frame. The down sampler is configured to receive an image frame and down sample the image frame into a down sampled image frame. The nxn binning compensation look-up table is configured to receive a downsampled image frame and compensate the downsampled image frame into a compensated image frame. The interpolator is configured to receive the compensated image frame and interpolate the compensated image frame into an interpolated image frame. The multiplier is configured to combine the shifted image frame with the interpolated image frame, resulting in a compensated output frame. The display panel is further configured to display the compensated output frame.
In an augmented reality implementation, the image data comes from two sources: real-world image components captured by a digital camera, and virtual image components generated by a Graphics Processing Unit (GPU). The augmented reality display comprises a display panel, a graphic processing unit, a camera, a mixing unit and a removable afterimage compensator. The graphics processing unit is configured to generate a virtual image frame. The camera is configured to capture real world image frames. The blending unit is configured to receive the virtual image frame from the graphics processing unit, receive the real-world image frame from the camera, and combine the virtual image frame with the real-world image frame to produce a combined image frame. The decimated afterimage compensator is configured to receive the combined image frames and output compensated output frames to the display panel. The extraction type afterimage compensator also comprises a pixel shifter, a down sampler, an NxN binning compensation lookup table, an interpolator and a multiplier. The pixel shifter is configured to receive the combined image frame and shift the image frame by a predetermined number of pixels, resulting in a shifted image frame. The down sampler is configured to receive an image frame and down sample the image frame into a down sampled image frame. The nxn binning compensation look-up table is configured to receive a downsampled image frame and compensate the downsampled image frame into a compensated image frame. N is an integer greater than one. The interpolator is configured to receive the compensated image frame and interpolate the compensated image frame into an interpolated image frame. The multiplier is configured to combine the shifted image frame with the interpolated image frame, resulting in a compensated output frame. The display panel is further configured to display the compensated output frame.
In an alternative embodiment, an augmented reality display includes a display panel, a graphics processing unit, a first removable afterimage compensator, a camera, a second removable afterimage compensator, and a blending unit. The graphics processing unit is configured to generate a virtual image frame. The first decimated afterimage compensator is configured to receive the virtual image frame and output a compensated virtual output frame. The first decimation afterimage compensator further comprises a first pixel shifter, a first down sampler, a first NxN binning compensation lookup table, a first interpolator and a first multiplier. The first pixel shifter is configured to receive a virtual image frame and shift the virtual image frame by a first predetermined number of pixels, resulting in a first shifted image frame. The first downsampler is configured to receive a virtual image frame and downsample the image frame to a first downsampled image frame. The first nxn binning compensation lookup table is configured to receive a first downsampled image frame and compensate the first downsampled image frame into a first compensated image frame, where N is an integer greater than one. The first interpolator is configured to receive a first compensated image frame and interpolate the first compensated image frame into a first interpolated image frame. The first multiplier is configured to combine the first shifted image frame with the first interpolated image frame resulting in a compensated virtual output frame. The camera is configured to capture real world image frames. A second decimated afterimage compensator is configured to receive the real world image frames and output compensated real world output frames. The second decimating residual compensator further includes a second pixel shifter, a second downsampler, an M × M binning compensation lookup table, a second interpolator, and a second multiplier. The second pixel shifter is configured to receive a real world image frame and shift the real world image frame by a second predetermined number of pixels, resulting in a second shifted image frame. The second downsampler is configured to receive a real world image frame and downsample the image frame into a second downsampled image frame. The M × M compensation look-up table is configured to receive a second downsampled image frame and compensate the second downsampled image frame into a second compensated image frame, where M is an integer greater than one. The second interpolator is configured to receive a second compensated image frame and interpolate the second compensated image frame into a second interpolated image frame. The second multiplier is configured to combine the second shifted image frame with the second interpolated image frame resulting in a compensated real world image frame. The blending unit is configured to receive the compensated virtual image frame from the first decimated afterimage compensator, configured to receive the compensated real world image frame from the second decimated afterimage compensator, and configured to combine the compensated virtual image frame with the compensated real world image frame to produce a combined image frame. The display panel is further configured to display the combined image frames.
Drawings
For a better understanding of the nature and advantages of the present disclosure, reference should be made to the following description and accompanying drawings. It is to be understood, however, that each of the figures is provided for purposes of illustration only and is not intended as a definition of the limits of the present disclosure. Moreover, as a general rule, and unless clearly contrary to the description, elements in different figures use the same reference numeral, the elements are generally the same or at least similar in function or purpose.
FIG. 1 is a block diagram of a decimated image retention compensator designed to prevent display panel image retention by decimating lookup tables and pixel shifting in a flat panel display.
FIG. 2 illustrates a flat panel display having a single extracted afterimage compensator designed to compensate for a combined augmented reality image to prevent afterimages at the display panel.
FIG. 3 illustrates a flat panel display for an augmented reality embodiment, where two decimated afterimage compensators are designed to compensate for real world image components and virtual image components to prevent afterimages at the display panel.
Detailed Description
One aspect of the present disclosure is the recognition that various afterimage compensation techniques are sub-optimal. While the flat panel display image retention problem can be addressed by pixel-by-pixel tracking and compensation, the use of pixel-by-pixel tracking results in large memory and read look-up tables (LUTs). The large amount of dynamic read only memory (DRAM) or static read only memory (SRAM) required increases semiconductor die area and manufacturing costs. Furthermore, the use of a large amount of DRAM results in excessive power consumption. In turn, higher power consumption leads to reduced battery life in portable devices such as tablet computers, mobile phones, virtual reality headsets or augmented reality glasses and augmented reality display smartphones, digital "smart" watches, and other digital devices.
One aspect of the present disclosure includes the observation that using blocks of N x N pixels instead of pixel-by-pixel tracking reduces compensation memory bandwidth and power requirements and saves lookup table footprint. This solution reduces memory and chip size requirements, thereby reducing manufacturing costs.
For a better understanding of the features and aspects of the present disclosure, in the following sections, more context of the present disclosure is provided by implementations of flat panel displays that avoid display panel afterimages through a decimated lookup table and pixel shifting in the flat panel display according to embodiments of the present disclosure. An alternative embodiment shows how such a flat panel display can be implemented for use in an augmented reality display. These embodiments are for illustrative purposes only, and other embodiments may be employed in other display devices. For example, embodiments of the present disclosure may be used with any display device that compensates and prevents display panel afterimages through a decimated lookup table and pixel shifts in a flat panel display.
FIG. 1 is a block diagram of an embodiment of a flat panel display 100 with a decimated afterimage compensator 1000 designed to prevent display panel afterimages by decimating compensation look-up tables and pixel shifts in the display panel 200, according to an embodiment of the present disclosure. The flat panel display 100 may be a stand-alone display, or part of: a computer display, a television, a laptop, a tablet, a mobile phone, a smart phone, an augmented reality display, a digital "smart" watch, or other digital device. The decimated afterimage compensator 1000 is configured to receive an image frame and output a compensation frame preventing afterimage of a display panel. Essentially, the flat panel display 100 receives inputs from two data streams combined by a multiplier: pixel shifted images from the frame buffer, and images through the N × N compensation look-up table.
In this embodiment, the flat panel display 100 has a removable afterimage compensator 1000 and a display panel 200.
The display panel 200 may be an Organic Light Emitting Diode (OLED) display, such as a Passive Matrix (PMOLED) or an Active Matrix (AMOLED). In other embodiments, the display panel 200 may be a Liquid Crystal Display (LCD) or a micro light emitting diode (micro LED) display. The display panel 200 displays an image received from the extraction type afterimage compensator 1000.
The decimating afterimage compensator 1000 uses a combination of pixel shifting and compensating an nxn block of pixels, where N is a decimation factor represented by an integer greater than one. For example, in some embodiments, N is 2. Therefore, the decimated image sticking compensator 1000 includes a pixel shifter 1200 and an nxn binning compensation lookup table 1600. An implementation of the decimated image residual compensator 1000 may also include a frame buffer 1100, a pixel history accumulation 1400, a downsampler 1500, an interpolator 1700, and a multiplier 1300.
The pixel shifter 1200 receives an image frame from the frame buffer 1100 and periodically (vertically and/or horizontally) shifts the image by a predetermined number of pixels. In some embodiments, the image frames are shifted in a circle at a defined cadence and pixel spacing. The pixel shifter 1200 may shift the image frame latently silently onto the viewer of the display panel 200.
When pixels within an nxn block of pixels have significantly different content histories, there is a risk of overcompensating for pixels with less residual pixels and undercompensating for other pixels. By periodically shifting the content, pixel shifter 1200 eliminates stress level differences within the N × N pixel blocks.
The image may be shifted by a maximum of 1 pixel, 2 pixels, 4 pixels, 8 pixels, 12 pixels, or 16 pixels. Image shifting may occur with 1x1 or 2x2 pixel shifts per step. In other embodiments, the image may be shifted by 1-16 pixels. The number of pixels shifted may be related to the N × N pixel block decimation compensation. For example, in one implementation, a 1 pixel shift may be used with an 8x8 pixel block. In practice, pixel shifting assigns an afterimage content stress and is equivalent to applying low pass filtering to the content before afterimage stress (BIS). The larger combination results in smoother gradient of the low pass filter and less error at sharp image edges. A longer image shift range achieves a good smoothing effect.
The pixel history accumulation 1400 is a computer memory that stores a history of previously displayed image frames. The pixel history accumulation 1400 may be a Random Access Memory (RAM), flash memory, or the like. In some implementations, pixel history accumulation 1400 includes images from frame buffer 1100.
The downsampler 1500 receives the previously displayed image frames from the pixel history accumulation 1400 and decimates the images to reduce their size. The decimation process reduces the image to nxn pixel binning, where N is a decimation factor represented by an integer greater than or equal to two (2). Size reduction of N2The size of the lookup table used by the removable afterimage compensator 1000 is significantly saved and the amount of memory used is also reduced.
The nxn binning compensation lookup table 1600 allows the decimated afterimage compensator 1000 to apply a characteristic curve to an image frame received from the downsampler 1500. In this application, the nxn binning compensation lookup table 1600 assigns an output value to each possible nxn input value, which allows the correction color space calculation to be performed quickly, thereby preventing afterimages. In the nxn binning compensation lookup table 1600, N is a decimation factor represented by an integer greater than one. It will be appreciated by those skilled in the art that the nxn binning compensation lookup table 1600 will facilitate image contrast, brightness variation, gray value extension, individual gradient tables, or enhanced image gamma. Once the image frame is compensated by the nxn compensation lookup table 1600, a compensated image is generated and sent to the interpolator 1700.
The interpolator 1700 resizes the compensated image received from the N × N compensation lookup table 1600 to an enlarged image matching the original resolution of the display panel 200. The enlarged image is forwarded to multiplier 1300, which serves as a synthesis block to combine the enlarged image with the shifted image from pixel shifter 1200. The resulting image is output to the display panel 200 by the removable afterimage compensator 1000.
Turning now to fig. 2-3, each shows an alternative embodiment of a flat panel display for augmented reality applications. Each alternative embodiment includes a removable afterimage compensator 1000 designed to prevent display panel afterimages. In an augmented reality implementation, the image data comes from two sources: real-world image components captured by a digital camera, and virtual image components generated by a Graphics Processing Unit (GPU). These two sources are combined by a mixing unit and finally projected on a display panel. Depending on the particular implementation of a particular implementation, a removable afterimage compensator may be used to compensate each of the two image data sources or the combined image data.
It will be appreciated by those skilled in the art that the embodiments shown in fig. 2-3 may include one or more image frame buffers, not shown.
Fig. 2 illustrates a flat panel display 2000 having a single extracted afterimage compensator 1000 designed to compensate a combined augmented reality image to prevent afterimages at a display panel 200 according to an embodiment of the present disclosure.
The virtual image component is composed of a plurality of virtual image frames generated by the graphics processing unit 2100. Graphics processing unit 2100 may be an Application Specific Integrated Circuit (ASIC) designed to quickly manipulate and change memory to speed up the creation of images in a frame buffer. In other embodiments, graphics processing unit 2100 may be embedded on a Central Processing Unit (CPU) chip or motherboard of flat panel display 2000. Once generated by the graphics processing unit 2100, the virtual image frames are forwarded to the blending unit 2400 for combination with the real world image frames.
The real world image component is made up of a plurality of real world image frames captured by the camera 2200. The camera 2200 may be any optical capture system with a digital image sensor such as a Charge Coupled Device (CCD) or a Complementary Metal Oxide Semiconductor (CMOS). Once captured by the camera 2200, the real world image frames are forwarded to an Image Signal Processor (ISP) 2300.
The image signal processor 2300 is a Digital Signal Processor (DSP) integrated circuit dedicated to image processing data from the camera 2200. In particular, image signal processor 2300 may perform bayer transformation or demosaicing to achieve color accuracy, noise reduction, image sharpening, or interpolation to adjust the size of a captured image received from camera 2200. Image signal processor 2300 may implement single instruction, multiple data (SIMD) or multiple instruction, multiple data (MIMD) techniques to allow parallel processing of image data. The resulting imaged real world image frames are forwarded to the mixing unit 2400 for combination with the virtual image frames.
The mixing unit 2400 is a dedicated image processing unit for combining the virtual image frame with the real-world image frame. The mixing unit 2400 may treat the virtual image frame and the real-world image frame as two layers to be mixed together. In such embodiments, the virtual image frames are considered to be a "top layer" or "active layer" image superimposed on top of a real-world "bottom layer" image. The resulting blended image is an augmented reality image that is forwarded to a lens distortion corrector 2500.
The lens distortion corrector 2500 is an image processor for correcting radial (optical) distortion caused by a lens in the camera 2200. In some embodiments, lens distortion corrector 2500 is implemented as a software or firmware correction that can correct for sagittal distortion and tangential image distortion using a brownian distortion model. Once the distortion is corrected, the corrected image is sent to the decimated afterimage compensator 1000. The decimating image retention compensator 1000 prevents display panel image retention through decimating compensation lookup tables and pixel shifting, as described in FIG. 1. The resulting image is forwarded to pixel pipeline 2600.
In the art, pixel pipeline 2600 is also sometimes referred to as a computer graphics pipeline, a rendering pipeline, or a graphics pipeline. The pixel pipeline 2600 renders a three-dimensional scene onto a two-dimensional display screen. Those skilled in the art will appreciate that the rendering of this operation depends on the software and hardware used and the characteristics of the display panel 200. In general, pixel pipeline 2600 performs real-time rendering implemented in hardware. The resulting image may then be displayed at the display panel 200.
In an alternative implementation, FIG. 3 shows a flat panel display 3000 for an augmented reality embodiment, where two removable afterimage compensators 1000a-b are designed to separately compensate for real world image components and virtual image components before the images are combined and displayed on the display panel 200.
The virtual image component is constituted by a plurality of virtual image frames generated by the graphics processing unit 3100. Graphics processing unit 3100 may be an application specific integrated circuit designed for fast manipulation and changing of memory to speed up the creation of images in a frame buffer. In other embodiments, the graphics processing unit 3100 may be embedded on a central processing unit chip or motherboard of the flat panel display 200. Once generated by the graphics processing unit 3100, the virtual image frames are forwarded to a first decimated image retention compensator 1000 a. The first decimating image retention compensator 1000a prevents display panel image retention through decimating compensation lookup tables and pixel shifting, as described in fig. 1. The resulting image is forwarded to the blending unit 3400 to be combined with the real world image frame.
The real world image component is made up of a plurality of real world image frames captured by the camera 3200. The camera 3200 may be any optical capture system having a digital image sensor such as a charge coupled device or a complementary metal oxide semiconductor. Once captured by the camera 3200, the real world image frames are forwarded to the image signal processor 3300.
The image signal processor 3300 is a digital signal processor integrated circuit dedicated to image processing data from the camera 3200. In particular, the image signal processor 3300 may perform demosaicing or bayer transformation to achieve color accuracy, noise reduction, image sharpening, or interpolation to adjust the size of a captured image received from the camera 3200. The image signal processor 3300 may implement parallel processing of image data by a single instruction, multiple data (SIMD) or multiple instruction, multiple data (MIMD) technique. The resulting imaged processed real world image frames are forwarded to a lens distortion corrector 3500.
The lens distortion corrector 3500 is an image processor for correcting radial (optical) distortion caused by a lens in the camera 3200. In some embodiments, lens distortion corrector 2500 is implemented as a software or firmware correction that can correct for sagittal distortion and tangential image distortion using a brownian distortion model. Once the distortion is corrected, the corrected image is sent to the second decimated afterimage compensator 1000 b.
The second decimating image retention compensator 1000b prevents display panel image retention by decimating the compensation look-up table and pixel shifting, as described in FIG. 1. It will be understood by those skilled in the art that the second decimated residual image compensator 1000b need not shift the image by the same number of pixels as the first decimated residual image compensator 1000a, and that image compensation may be performed using different N × N binning (designated as "M × M", where M is greater than one). In other embodiments, the first and second decimated image residue compensators 1000a and 1000b may shift the image by the same number of pixels and may use the same nxn binning compensation lookup table 1600. The resulting image is forwarded to the blending unit 3400 to be combined with the virtual image frame.
The dedicated image processor, mixing unit 3400 combines the virtual image frame with the real world image frame. The mixing unit 3400 may treat the virtual image frame and the real-world image frame as two layers to be mixed together. In such embodiments, the virtual image frames are considered to be a "top layer" or "active layer" image superimposed on top of a real-world "bottom layer" image. The resulting blended image is an augmented reality image that is forwarded to the pixel pipeline 3600.
In the art, pixel pipeline 3600 is sometimes also referred to as a computer graphics pipeline, a rendering pipeline, or a graphics pipeline. The pixel pipeline 3600 renders a three-dimensional scene onto a two-dimensional display screen. Those skilled in the art will appreciate that the rendering of this operation depends on the software and hardware used and the characteristics of the display panel 200. In general, the pixel pipeline 3600 performs real-time rendering implemented in hardware. The resulting image may then be displayed at the display panel 200.
Those skilled in the art will appreciate that the system described herein may be implemented in a variety of hardware or firmware solutions.
The previous description of the embodiments is provided to enable any person skilled in the art to practice the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without the use of the inventive faculty. Thus, the present disclosure is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (20)
1. An electronic display, comprising:
a display panel;
a removable afterimage compensator configured to receive an image frame and output a compensated output frame to the display panel, the removable afterimage compensator further comprising:
a pixel shifter configured to receive the image frame and shift the image frame by a predetermined number of pixels, resulting in a shifted image frame;
a down sampler configured to receive the image frames and down sample the image frames into down-sampled image frames;
an NxN binning compensation lookup table configured to receive the downsampled image frame and compensate the downsampled image frame into a compensated image frame, wherein N is an integer greater than one;
an interpolator configured to receive the compensated image frame and interpolate the compensated image frame into an interpolated image frame; and
a multiplier configured to combine the shifted image frame with the interpolated image frame resulting in the compensated output frame;
the display panel is further configured to display the compensated output frame.
2. The electronic display of claim 1, wherein N is 2.
3. The electronic display of claim 2, wherein the predetermined number of pixels of the pixel shifter is between 1-16.
4. The electronic display of any of claims 1-3, wherein the display panel is a light emitting diode or a liquid crystal display.
5. The electronic display of claim 4 integrated in a tablet, digital watch, mobile phone, virtual reality headset, or augmented reality glasses.
6. The electronic display of any of claims 1-3, wherein the display panel is an organic light emitting diode display.
7. The electronic display of claim 6 integrated in a tablet, digital watch, or mobile phone.
8. An augmented reality display comprising:
a display panel;
a graphics processing unit configured to generate a virtual image frame;
a camera configured to capture real world image frames;
a blending unit configured to receive the virtual image frame from the graphics processing unit, configured to receive the real-world image frame from the camera, and configured to combine the virtual image frame with the real-world image frame to produce a combined image frame;
a decimating afterimage compensator configured to receive the combined image frame and output a compensated output frame to the display panel, the decimating afterimage compensator further comprising:
a pixel shifter configured to receive the combined image frame and shift the image frame by a predetermined number of pixels resulting in a shifted image frame;
a down sampler configured to receive the image frames and down sample the image frames into down-sampled image frames;
an NxN binning compensation lookup table configured to receive the downsampled image frame and compensate the downsampled image frame into a compensated image frame, wherein N is an integer greater than one;
an interpolator configured to receive the compensated image frame and interpolate the compensated image frame into an interpolated image frame; and
a multiplier configured to combine the shifted image frame with the interpolated image frame resulting in the compensated output frame;
the display panel is further configured to display the compensated output frame.
9. The augmented reality display of claim 8, wherein N is 2.
10. The augmented reality display of claim 9, wherein the predetermined number of pixels of the pixel shifter is between 1-16.
11. The augmented reality display of any one of claims 8-10, wherein the display panel is a light emitting diode or a liquid crystal display.
12. The augmented reality display of claim 11, integrated in a tablet, digital watch, or mobile phone.
13. The augmented reality display of any one of claims 8-10, wherein the display panel is an organic light emitting diode display.
14. The augmented reality display of claim 13, integrated in a tablet, digital watch, mobile phone, virtual reality headset, or augmented reality glasses.
15. An augmented reality display comprising:
a display panel;
a graphics processing unit configured to generate a virtual image frame;
a first decimated afterimage compensator configured to receive the virtual image frame and output a compensated virtual output frame, the first decimated afterimage compensator further comprising:
a first pixel shifter configured to receive the virtual image frame and shift the virtual image frame by a first predetermined number of pixels resulting in a first shifted image frame;
a first downsampler configured to receive the virtual image frame and downsample the image frame to a first downsampled image frame;
a first NxN binning compensation lookup table configured to receive the first downsampled image frame and compensate the first downsampled image frame into a first compensated image frame, where N is an integer greater than one;
a first interpolator configured to receive the first compensated image frame and interpolate the first compensated image frame into a first interpolated image frame; and
a first multiplier configured to combine the first shifted image frame with the first interpolated image frame resulting in the compensated virtual output frame;
a camera configured to capture real world image frames;
a second decimated afterimage compensator configured to receive the real world image frames and output compensated real world output frames, the second decimated afterimage compensator further comprising:
a second pixel shifter configured to receive the real world image frame and to shift the real world image frame by a second predetermined number of pixels resulting in a second shifted image frame;
a second downsampler configured to receive the real world image frame and downsample the image frame to a second downsampled image frame;
an MxM compensation lookup table configured to receive the second downsampled image frame and compensate the second downsampled image frame into a second compensated image frame, wherein M is an integer greater than one;
a second interpolator configured to receive the second compensated image frame and interpolate the second compensated image frame into a second interpolated image frame; and
a second multiplier configured to combine the second shifted image frame with the second interpolated image frame resulting in the compensated real world image frame;
a mixing unit configured to receive the compensated virtual image frame from the first decimated afterimage compensator, configured to receive the compensated real world image frame from the second decimated afterimage compensator, and configured to combine the compensated virtual image frame with the compensated real world image frame to produce a combined image frame;
the display panel is further configured to display the combined image frame.
16. The augmented reality display of claim 15, wherein N is 2.
17. The augmented reality display of claim 16, wherein the first predetermined number of pixels of the first pixel shifter is between 1-16.
18. An augmented reality display according to any one of claims 15 to 17, wherein M is 2.
19. The augmented reality display of claim 18, wherein the second predetermined number of pixels of the second pixel shifter is between 1-16.
20. The augmented reality display of claim 19, wherein the display panel is a light emitting diode or a liquid crystal display.
Applications Claiming Priority (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US201862726876P | 2018-09-04 | 2018-09-04 | |
| US62/726,876 | 2018-09-04 | ||
| US16/403,366 | 2019-05-03 | ||
| US16/403,366 US10991283B2 (en) | 2018-09-04 | 2019-05-03 | Decimated burn-in compensation with pixel shifting |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110930935A true CN110930935A (en) | 2020-03-27 |
| CN110930935B CN110930935B (en) | 2022-09-23 |
Family
ID=69640674
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910822215.4A Active CN110930935B (en) | 2018-09-04 | 2019-09-02 | Extraction type residual image compensation using pixel shift |
Country Status (2)
| Country | Link |
|---|---|
| US (1) | US10991283B2 (en) |
| CN (1) | CN110930935B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20220085082A (en) * | 2020-12-14 | 2022-06-22 | 삼성디스플레이 주식회사 | Afterimage detection device and display device including the same |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1125269A1 (en) * | 1998-10-07 | 2001-08-22 | Microsoft Corporation | Methods and apparatus for detecting and reducing color artifacts in images |
| US20070115384A1 (en) * | 2005-11-24 | 2007-05-24 | Olympus Corporation | Image processing apparatus and method for preferably correcting distortion aberration |
| US20110254976A1 (en) * | 2009-04-23 | 2011-10-20 | Haim Garten | Multiple exposure high dynamic range image capture |
| CN102540463A (en) * | 2010-09-21 | 2012-07-04 | 微软公司 | Opacity filter for see-through head mounted display |
| CN104115491A (en) * | 2012-02-22 | 2014-10-22 | 索尼公司 | Display apparatus, image processing apparatus, image processing method, and computer program |
| CN105074807A (en) * | 2013-03-13 | 2015-11-18 | 苹果公司 | Compensation methods for display brightness change associated with reduced refresh rate |
| CN106095157A (en) * | 2015-04-30 | 2016-11-09 | 三星显示有限公司 | Touch screen display device |
| WO2017031246A1 (en) * | 2015-08-18 | 2017-02-23 | Magic Leap, Inc. | Virtual and augmented reality systems and methods |
| US20170206689A1 (en) * | 2016-01-14 | 2017-07-20 | Raontech, Inc. | Image distortion compensation display device and image distortion compensation method using the same |
| CN107111988A (en) * | 2014-12-29 | 2017-08-29 | 亚马逊技术股份有限公司 | Reduce visual artifact and reduce the power consumption in electric moistening display |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7663663B2 (en) | 2006-04-28 | 2010-02-16 | Nvidia Corporation | Burn-in control |
| US20090046089A1 (en) | 2007-08-16 | 2009-02-19 | Motorola, Inc. | Burn-in compensation for display |
| US11361729B2 (en) | 2017-09-08 | 2022-06-14 | Apple Inc. | Burn-in statistics and burn-in compensation |
| US10559276B2 (en) * | 2018-02-03 | 2020-02-11 | Facebook Technologies, Llc | Apparatus, system, and method for mitigating motion-to-photon latency in head-mounted displays |
-
2019
- 2019-05-03 US US16/403,366 patent/US10991283B2/en active Active
- 2019-09-02 CN CN201910822215.4A patent/CN110930935B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1125269A1 (en) * | 1998-10-07 | 2001-08-22 | Microsoft Corporation | Methods and apparatus for detecting and reducing color artifacts in images |
| US20070115384A1 (en) * | 2005-11-24 | 2007-05-24 | Olympus Corporation | Image processing apparatus and method for preferably correcting distortion aberration |
| US20110254976A1 (en) * | 2009-04-23 | 2011-10-20 | Haim Garten | Multiple exposure high dynamic range image capture |
| CN102540463A (en) * | 2010-09-21 | 2012-07-04 | 微软公司 | Opacity filter for see-through head mounted display |
| CN104115491A (en) * | 2012-02-22 | 2014-10-22 | 索尼公司 | Display apparatus, image processing apparatus, image processing method, and computer program |
| CN105074807A (en) * | 2013-03-13 | 2015-11-18 | 苹果公司 | Compensation methods for display brightness change associated with reduced refresh rate |
| CN107111988A (en) * | 2014-12-29 | 2017-08-29 | 亚马逊技术股份有限公司 | Reduce visual artifact and reduce the power consumption in electric moistening display |
| CN106095157A (en) * | 2015-04-30 | 2016-11-09 | 三星显示有限公司 | Touch screen display device |
| WO2017031246A1 (en) * | 2015-08-18 | 2017-02-23 | Magic Leap, Inc. | Virtual and augmented reality systems and methods |
| CN108140259A (en) * | 2015-08-18 | 2018-06-08 | 奇跃公司 | virtual and augmented reality system and method |
| US20170206689A1 (en) * | 2016-01-14 | 2017-07-20 | Raontech, Inc. | Image distortion compensation display device and image distortion compensation method using the same |
Also Published As
| Publication number | Publication date |
|---|---|
| US20200074900A1 (en) | 2020-03-05 |
| US10991283B2 (en) | 2021-04-27 |
| CN110930935B (en) | 2022-09-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107203056B (en) | Mura processing method for ultrahigh-resolution panel | |
| CN108766372B (en) | Method for improving mura phenomenon of display panel | |
| CN111816136B (en) | Liquid crystal display, driving compensation method and driving compensation device thereof | |
| US9202406B2 (en) | Liquid crystal display, display method, program, and recording medium | |
| JP7184788B2 (en) | Integrated circuit display driving method, integrated circuit, display screen and display device | |
| US10559244B2 (en) | Electronic apparatus, display driver and method for generating display data of display panel | |
| CN108492769B (en) | Brightness adjusting method and device and display device | |
| US11232745B2 (en) | Multi-frame buffer for pixel drive compensation | |
| US20160321976A1 (en) | Optical compensation system and optical compensation method thereof | |
| US20160329014A1 (en) | Display device | |
| US10467947B2 (en) | Display device | |
| US8531372B2 (en) | Method, device and system of response time compensation utilizing an overdrive signal | |
| CN111816121A (en) | Display panel brightness compensation method and system and display panel | |
| CN112053658B (en) | Apparatus and method for two-dimensional temperature compensation for pixel drive compensation | |
| US20180048914A1 (en) | Image processing method and related apparatus | |
| US20140184615A1 (en) | Sequential Rendering For Field-Sequential Color Displays | |
| CN105845077B (en) | Data compensator and display device including the same | |
| US20160232827A1 (en) | Pixel driving method and associated display device | |
| CN110930935B (en) | Extraction type residual image compensation using pixel shift | |
| KR20160048312A (en) | Method and data converter for upscailing of input display data | |
| US20090161015A1 (en) | Display device, video signal correction device, and video signal correction method | |
| JP2010048958A (en) | Image processing device, processing method therefor and image display system | |
| KR20170038989A (en) | Timing controller and display apparatus having the same | |
| US11386869B2 (en) | Display device and driving method thereof according to capturing conditions of an image | |
| CN113707065B (en) | Display panel, driving method of display panel and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |