WO2021243562A1

WO2021243562A1 - Compensating for pixel decay for a display

Info

Publication number: WO2021243562A1
Application number: PCT/CN2020/093928
Authority: WO
Inventors: Nan Zhang; Yongjun XU; Xinchao YANG
Original assignee: Qualcomm Incorporated
Priority date: 2020-06-02
Filing date: 2020-06-02
Publication date: 2021-12-09

Abstract

An example operation of this disclosure monitors, measures, and/or determines a decay level of a light-pass region of a display and apply compensation to the light-pass region to compensate for the decay. The light-pass region of a display allows ambient light to be transmitted through the display in one direction while the display emits light in the opposite direction. For example, the light-pass region may have reduced or removed reflective layers that enables more light to pass through, or may have pixels of a different density, shape, or size than the rest of the pixels for enabling light passing through. Such structural differences often cause the light-pass region to deteriorate at a higher rate than normal pixels in the display, and darken the light-pass region over time. To address such problems, the present disclosure provides techniques to accurately brighten or otherwise restore the visual appearances of the darkened pixels.

Description

COMPENSATING FOR PIXEL DECAY FOR A DISPLAY

TECHNICAL FIELD

The present disclosure relates generally to display panels and, more particularly, to one or more techniques for display or graphics processing for display panels.

BACKGROUND

Computing devices often utilize a graphics processing unit (GPU) to accelerate the rendering of graphical data for display. Such computing devices may include, for example, computer workstations, mobile phones such as so-called smartphones, embedded systems, personal computers, tablet computers, and video game consoles. GPUs execute a graphics processing pipeline that includes one or more processing stages that operate together to execute graphics processing commands and output a frame. A central processing unit (CPU) may control the operation of the GPU by issuing one or more graphics processing commands to the GPU. Modern day CPUs are typically capable of concurrently executing multiple applications, each of which may need to utilize the GPU during execution.

Portable computing devices, including smartphones, tablets, and wearable devices, may present graphical content on a display. Very often, these devices include a front camera (e.g., a camera facing toward the user) integrated with the display for video conferencing, video chat, taking photographs of one’s self, and other activities that involve the user. Some front cameras are hidden behind the display for aesthetics reasons. Such cameras are known as under-screen cameras. To allow light to pass through the display or screen, a light-pass region of the display above the camera has different physical structures than the remaining portions of the display. For example, in the light-pass region, pixels of the display may have a lower density, smaller size, or a different shape than other normal pixels in the display. The pixels in the light-pass region may thus age at a faster rate than the normal pixels, appearing darker than the normal pixels over time. Further, one or more pixels in a display may decay at a different rate than one or more other pixels of a display for other reasons, and appear different (e.g., darker) than the other pixels over time.

SUMMARY

The following presents a simplified summary of one or more aspects in order to provide a basic understanding of such aspects. This summary is not an extensive overview of all contemplated aspects, and is intended to neither identify key elements of all aspects nor delineate the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description that is presented later.

Certain aspects of the present disclosure provide a method for compensating for pixel decay of a display as discussed herein. The method includes recording a start pixel value time for a pixel of the display at a frame. The method further includes determining a first pixel value of the pixel at a first frame. The method further includes determining a second pixel value of the pixel at a second frame. The method further includes determining that a difference between the second pixel value and the first pixel value satisfies a threshold. Based on the difference satisfying the threshold, the method further includes recording a stop pixel value time for the pixel; determining a pixel decay parameter associated with the pixel based on the start pixel value time, the stop pixel value time, and the first pixel value; and applying a decay compensation to the pixel based on the pixel decay parameter.

Certain aspects of the present disclosure provide a computing device. The computing device further includes a memory and a processor. The memory and the processor are configured to record a start pixel value time for a pixel of the display at a frame; determine a first pixel value of the pixel at a first frame; and determine a second pixel value of the pixel at a second frame. When the processor determines that a difference between the second pixel value and the first pixel value satisfies a threshold, the processor is further configured to record a stop pixel value time for the pixel; determine a pixel decay parameter associated with the pixel based on the start pixel value time, the stop pixel value time, and the first pixel value; and apply a decay compensation to the pixel based on the pixel decay parameter.

Certain aspects of the present disclosure provide a non-transitory computer readable medium storing instructions that when executed by a computing device as discussed herein cause the computing device to operate as discussed herein. For example, the non- transitory computer readable medium storing instructions that, when executed by a computing device, cause the computing device to record a start pixel value time for a pixel of the display at a frame, determine a first pixel value of the pixel at a first frame, determine a second pixel value of the pixel at a second frame, and determine that a difference between the second pixel value and the first pixel value satisfies a threshold. Based on the difference satisfying the threshold, the non-transitory computer readable medium storing instructions that further cause the computing device to record a stop pixel value time for the pixel, determine a pixel decay parameter associated with the pixel based on the start pixel value time, the stop pixel value time, and the first pixel value; and apply a decay compensation to the pixel based on the pixel decay parameter.

Certain aspects of the present disclosure provide a computing device comprising means for recording a start pixel value time for a pixel of the display at a frame. The computing device also includes means for determining a first pixel value of the pixel at a first frame. The computing device further includes means for determining a second pixel value of the pixel at a second frame. The computing device includes means for determining that a difference between the second pixel value and the first pixel value satisfies a threshold. Based on the difference satisfying the threshold, the computer device may include means for recording a stop pixel value time for the pixel. The computer device may include means for determining a pixel decay parameter associated with the pixel based on the start pixel value time, the stop pixel value time, and the first pixel value. The computer device may further include means for applying a decay compensation to the pixel based on the pixel decay parameter.

The details of one or more examples of the disclosure are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the disclosure will be apparent from the description and drawings, and from the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

So that the manner in which the above-recited features of the present disclosure can be understood in detail, a more particular description, briefly summarized above, may be had by reference to aspects, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only certain typical aspects of this disclosure and are therefore not to be considered limiting of its scope, for the description may admit to other equally effective aspects.

FIG. 1 is a block diagram that illustrates an example computing device configured to compensate for decay in one or more pixels of a display, in accordance with one or more aspects of this disclosure.

FIG. 2 illustrates an example display panel implementing a technique for compensating for decay of one or more pixels in a region of a display, in accordance with one or more aspects of this disclosure.

FIG. 3 illustrates a local cross section of the region of FIG. 2, in accordance with one or more aspects of this disclosure.

FIG. 4 illustrates example operations for a computing device operable to compensate for decay of one or more pixels, according to certain aspects of the present disclosure.

Like numerals indicate like elements.

DETAILED DESCRIPTION

In general, aspects disclosed herein provide techniques for determining and compensating for different decay rates of pixels in a display. For example, such compensating may be used to maintain appearance consistency across pixels of a display. In one example, a display includes a region (e.g., a region where light can pass through) that decays faster than the rest of the display. In certain aspects, the techniques may be applicable in scenarios related to an under-screen front camera, or other situations that use a transparent region of a display for light transmission in both directions.

An example technique according to this disclosure monitors, measures, and/or determines a decay level of one or more pixels of a display and applies compensation to the one or more pixels to compensate for the decay.

In certain aspects, the one or more pixels for which compensation is applied are part of a light-pass region of a display that allows ambient light to be transmitted through the display in one direction while the display emits light in the opposite direction. For example, the light-pass region may have reduced or removed reflective layers that enables more light to pass through, or may have pixels of a different density, shape, or size than the rest of the pixels for enabling light passing through. Such structural differences often cause the light-pass region to deteriorate at a higher rate than normal pixels in the display, and darken the light-pass region over time. To address such problems, the present disclosure, in certain aspects, provides techniques to accurately brighten or otherwise restore the visual appearances of the darkened pixels.

In some cases, an under-screen camera configuration allows for a visual appearance of a full panel display while enabling the normal operation of a front facing camera under the display. Light transmission through the display is achieved using a light-passing region of the display above the front facing camera. The light-pass region may be considered as a transparent sub region of the display that emits light like the rest of the display while also allows ambient light to pass through to reach the under-screen camera.

In some examples, the display may be an OLED display. To allow for ambient light transmission through the display, the light-pass region may have pixels of lower/smaller physical density, shape, or size than the rest of the pixels of the display. In this way, light can pass through the spaces or gaps between the pixels. The pixels in the light-pass region, due to the relatively smaller footprint than and similar light emission requirement as the rest of the pixels, may decay faster. In another example, the light-pass region may remove reflective layers in the anode and/or cathode for allowing light transmission, which would also cause faster decay in the light-pass region. As a pixel decays or ages, it may appear darker when driven to the same output (e.g., brightness, color, etc. ) . As different pixels may decay at different rates, over time, the output of the different pixels may differ from one another when driven with the same output, leading to issues with display uniformity.

To overcome such differences in decaying rates, the present disclosure provides post processing to one or more pixels, such as to maintain visual consistency with the rest of the display. In an aspect, the disclosed post processing techniques maintains the one or more pixels consistency by boosting the code values of the one or more pixels, thus boosting the applied electrical current and lighting intensity, to counter the darkening effect of aging.

For example, when a light-pass region includes pixels at a lower density as compared to other regions of the display, higher emission intensity of radiant energy per pixel in the light-pass region than the rest of the display may be required. Such higher emission intensity causes the pixels in the light-pass region to burn-in at a higher rate than normal pixels of the display. In certain aspects, the present disclosure provides a separate or split user interface (UI) layer to control the light-pass region, by monitoring the region’s decay and determining a compensation value in response. The light-pass region is often small relative to the overall display, for example, the light-pass region may occupy a circle of no greater than 5 mm diameter of no more than 5000 valid pixels in some implementations. However, aspects of the present disclosure applies to light-pass regions of any size, both in terms of physical dimensions and in terms of number of pixels, even if the light-pass region is the full display.

In some aspects, the disclosed techniques first ascertain or allocate information of one or more pixels (e.g., the light-pass region) , including position, shape, size, etc. A separate UI layer is then created and assigned to the one or more pixels. For example, the one or more pixels may be positioned in an area that has an existing UI layer, such as where a status bar would be displayed (e.g., the top portion of a display that shows signal strength and battery levels) . The separate UI layer is created in addition to the existing UI layer for individual control. The disclosed techniques apply to the separate UI layer so that the GPU and/or processor can apply separate composition and post processing to the one or more pixels.

In some cases, one cache buffer layer of same size as the one or more pixels is added to a frame layer lists. The disclosed techniques assign the added one cache buffer layer a highest z-order in the layer lists, in order to have this cache buffer layer always on the top of frame layer lists. In some embodiments, a shape mask is applied or allocated to this cache buffer layer of the one or more pixels. The separate UI layer is then associated to the cache buffer layer for reducing unnecessary refreshing, composing, or compensation cycle. In many use cases, the content in the one or more pixels does not change even when the status bar layer (i.e., the existing UI layer) is refreshing. Thereafter, regional GPU composition of regular layers is then performed over the one or more pixels. Composition pixels values are input to this added cache buffer layer. In some cases, down-scaling, pixel remapping, brightness adjustment, and/or other post processing algorithms (including shape mask process if any) are performed to this added cache buffer layer.

In certain aspects, to determine when to apply compensation to a pixel, a timer is started for the pixel (e.g., after another compensation has been applied to the pixel, when the display is turned on, etc. ) , for example, a start time is recorded. While the timer is running, in certain aspects, the disclosed techniques may compare the value of the pixel in one frame with the value of the pixel in another frame (e.g., the next frame) to determine a change amount (e.g., emission light intensity) between frames (e.g., adjacent frames) . If the change amount is greater than a threshold, then the technique stops the timer, such as recording a stop time for the pixel. Based on the change amount and the timer value (e.g., difference between the stop time and the start time) , the burn-in accumulation can be recorded and a corresponding decay level can be determined for the pixel, as discussed in details below. The decay level may be represented by a pixels decay ratio (e.g., indicating an amount of decay over a life span of the pixel, such as a percentage of an expected lifespan or a percentage of decay, aging, or wear) . If the decay ratio reaches or exceeds a threshold, a compensation factor may be applied to the pixel, such as to increase its emission intensity to offset the darkening effect of the decay. The disclosed techniques apply in the composition cycles for all layers, including newly added layers.

Certain aspects herein, including certain devices, and systems therefore enjoy several technical advantages over the conventional methods for handling display decay. First, the disclosed techniques in certain aspects enable a display to have a consistent appearance over the lifespan of the display, avoiding noticeable darkened effect due to faster decay of one or more pixels. Second, the disclosed techniques in certain aspects tailor to pixel performance to be configurable for different sizes, shapes, and positions of one or more pixels of the display (e.g., a light-pass region) . Third, the disclosed techniques in certain aspects provide reduced computation burden over the one or more pixels by using thresholds to trigger qualified changes for determining the decay ratio.

Various aspects of systems, apparatuses, computer program products, and methods are described more fully hereinafter with reference to the accompanying drawings. This disclosure may, however, be embodied in many different forms and should not be construed as limited to any specific structure or function presented throughout this disclosure. Rather, these aspects are provided so that this disclosure will be thorough and complete, and will fully convey the scope of this disclosure to those skilled in the art. Based on the teachings herein one skilled in the art should appreciate that the scope of this disclosure is intended to cover any aspect of the systems, apparatuses, computer program products, and methods disclosed herein, whether implemented independently of, or combined with, other aspects of the disclosure. For example, an apparatus may be implemented or a method may be practiced using any number of the aspects set forth herein. In addition, the scope of the disclosure is intended to cover such an apparatus or method that is practiced using other structure, functionality, or structure and functionality in addition to or other than the various aspects of the disclosure set forth herein. Any aspect disclosed herein may be embodied by one or more elements of a claim.

Although various aspects are described herein, many variations and permutations of these aspects fall within the scope of this disclosure. Although some potential benefits and advantages of aspects of this disclosure are mentioned, the scope of this disclosure is not intended to be limited to particular benefits, uses, or objectives. Rather, aspects of this disclosure are intended to be broadly applicable to different wireless technologies, system configurations, networks, and transmission protocols, some of which are illustrated by way of example in the figures and in the following description. The detailed description and drawings are merely illustrative of this disclosure rather than limiting, the scope of this disclosure being defined by the appended claims and equivalents thereof.

Several aspects are presented with reference to various apparatus and methods. These apparatus and methods are described in the following detailed description and illustrated in the accompanying drawings by various blocks, components, circuits, processes, algorithms, and the like (collectively referred to as “elements” ) . These elements may be implemented using electronic hardware, computer software, or any combination thereof. Whether such elements are implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system.

By way of example, an element, or any portion of an element, or any combination of elements may be implemented as a “processing system” that includes one or more processors (which may also be referred to as processing units) . Examples of processors include microprocessors, microcontrollers, graphics processing units (GPUs) , general purpose GPUs (GPGPUs) , central processing units (CPUs) , application processors, digital signal processors (DSPs) , reduced instruction set computing (RISC) processors, systems-on-chip (SOC) , baseband processors, application specific integrated circuits (ASICs) , field programmable gate arrays (FPGAs) , programmable logic devices (PLDs) , state machines, gated logic, discrete hardware circuits, and other suitable hardware configured to perform the various functionality described throughout this disclosure.

One or more processors in the processing system may execute software. Software can be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software components, applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise. The term application may refer to software. As described herein, one or more techniques may refer to an application, i.e., software, being configured to perform one or more functions. In such examples, the application may be stored on a memory, e.g., on-chip memory of a processor, system memory, or any other memory.

Hardware described herein, such as a processor may be configured to execute the application. For example, the application may be described as including code that, when executed by the hardware, causes the hardware to perform one or more techniques described herein. As an example, the hardware may access the code from a memory and execute the code accessed from the memory to perform one or more techniques described herein. In some examples, components are identified in this disclosure. In such examples, the components may be hardware, software, or a combination thereof. The components may be separate components or sub-components of a single component.

Accordingly, in one or more examples described herein, the functions described may be implemented in hardware, software, or any combination thereof. If implemented in software, the functions may be stored on or encoded as one or more instructions or code on a computer-readable medium. Computer-readable media includes computer storage media. Storage media may be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise a random access memory (RAM) , a read-only memory (ROM) , an electrically erasable programmable ROM (EEPROM) , optical disk storage, magnetic disk storage, other magnetic storage devices, combinations of the aforementioned types of computer-readable media, or any other medium that can be used to store computer executable code in the form of instructions or data structures that can be accessed by a computer.

In general, examples disclosed herein provide techniques for maintaining consistent appearance of a display that has a region decaying faster than the rest. For example, the region may be monitored for its decay level and have a separate UI layer for controlling or providing compensation to achieve a consistent appearance for the overall display. Other example benefits are described throughout this disclosure.

As used herein, instances of the term “content” may refer to “graphical content, ” “image, ” and vice versa. This is true regardless of whether the terms are being used as an adjective, noun, or other parts of speech. In some examples, as used herein, the term “graphical content” may refer to content produced by one or more processes of a graphics processing pipeline. In some examples, as used herein, the term “graphical content” may refer to content produced by a processing unit configured to perform graphics processing. In some examples, as used herein, the term “graphical content” may refer to content produced by a graphics processing unit.

In some examples, as used herein, the term “display content” may refer to content generated by a processing unit configured to perform display processing. In some examples, as used herein, the term “display content” may refer to content generated by a display processing unit. Graphical content may be processed to become display content. For example, a graphics processing unit may output graphical content, such as a frame, to a buffer (which may be referred to as a framebuffer) . A display processing unit may read the graphical content, such as one or more frames from the buffer, and perform one or more display processing techniques thereon to generate display content. For example, a display processing unit may be configured to perform composition on one or more rendered layers to generate a frame.

As another example, a display processing unit may be configured to compose, blend, or otherwise combine two or more layers together into a single frame. A display processing unit may be configured to perform scaling, e.g., upscaling or downscaling, on a frame. In some examples, a frame may refer to a layer. In other examples, a frame may refer to two or more layers that have already been blended together to form the frame, i.e., the frame includes two or more layers, and the frame that includes two or more layers may subsequently be blended.

FIG. 1 is a block diagram that illustrates an example system 100 that includes an example computing device 104 configured to compensate for decay of one or more pixels of a display, in accordance with one or more aspects of this disclosure. The computing device 104 may include one or more components or circuits for performing various functions described herein. In some examples, one or more components of the computing device 104 may be components of an SOC. The computing device 104 may include one or more components configured to perform one or more techniques of this disclosure. In the example shown, the computing device 104 may include a processor 120 and a system memory 124.

In some examples, the computing device 104 can include a number of additional or alternative components, e.g., a communication interface 126, a transceiver 132, a receiver 128, a transmitter 130, a display processor 127, and a display client 131. Reference to the display client 131 may refer to one or more displays. For example, the display client 131 may include a single display or multiple displays. The display client 131 may include a first display and a second display, or a foldable or separable display. In other examples, the results of the graphics processing may not be displayed on the device, e.g., the first and second displays may not receive any frames for presentment thereon. Instead, the frames or graphics processing results may be transferred to another device. In some aspects, this can be referred to as split-rendering.

The processor 120 may include an internal memory 121. The processor 120 may be configured to perform graphics processing, such as in a graphics processing pipeline 107. In some examples, the computing device 104 may include a display processor or display processing unit, such as the display processor 127, to perform one or more display processing techniques on one or more frames generated by the processor 120 before presentment by the display client 131. The display processor 127 may be configured to perform display processing. For example, the display processor 127 may be configured to perform one or more display processing techniques on one or more frames generated by the processor 120. The display processor 127 may output image data to the display client 131 according to an interface protocol, such as, for example, the MIPI DSI (Mobile Industry Processor Interface, Display Serial Interface) .

The display client 131 may be configured to display or otherwise present frames processed by the display processor 127. In some examples, the display client 131 may include an organic light emitting diode (OLED) display, or a display that includes two or more regions of different decaying speeds. Although the examples herein refer to a display having a light-pass region that decays faster than the remaining of the display, other examples may include two or more regions of different decaying rates, regardless if such regions are configured to enable transmission of ambient light.

In the illustrated example of FIG. 1, the display client 131 includes a display controller 132, a buffer 134, and a display 136. The example display 136 includes a plurality of pixel elements for displaying image data. In certain aspects, the display 136 includes at least two sets of pixels having different rates of decay. For example, the pixels in a light-pass region may decay faster than the pixels outside of the light-pass region. In other cases, the two sets of pixels need not provide ambient light transmittance and/or may be scattered in three or more regions. The display controller 132 may receive image data from the display processor 127 and store the received image data in the buffer 134. In some examples, the display controller 132 may output the image data stored in the buffer 134 to the display 136. Thus, the buffer 134 may represent a local memory to the display client 131. In some examples, the display controller 132 may output the image data received from the display processor 127 to the display 136.

As explained in various aspects of this disclosure, the display 136 includes a region subject to faster decay, such as a light-pass region. One such example is shown in FIG. 2, where the display panel 240 includes a light-pass region 205 that allows light to reach the front camera 210. The display panel 240 belongs to an example device 215 of a computing system 200. In some situations, such as shown in FIG. 2, the light-pass region 205 provides a sufficient window for the front camera 210 to receive ambient light, while is able to perform consistent displaying functions as the remaining portions of the display panel 240.

Furthermore, as disclosed above, the display client 131 may be configured in accordance with MIPI DSI standards. The MIPI DSI standard supports a video mode and a command mode. In examples where the display client 131 is operating in video mode, the display processor 127 may continuously refresh the graphical content of the display client 131. For example, the entire graphical content may be refreshed per refresh cycle (e.g., line-by-line) .

In examples where the display client 131 is operating in command mode, the display processor 127 may write the graphical content of a frame to the buffer 134. In some such examples, the display processor 127 may not continuously refresh the graphical content of the display client 131. Instead, the display processor 127 may use a vertical synchronization (Vsync) pulse to coordinate rendering and consuming of graphical content at the buffer 134. For example, when a Vsync pulse is generated, the display processor 127 may output new graphical content to the buffer 134. Thus, the generating of the Vsync pulse may indicate when current graphical content at the buffer 134 has been rendered.

Memory external to the processor 120, such as system memory 124, may be accessible to the processor 120. For example, the processor 120 may be configured to read from and/or write to external memory, such as the system memory 124. The processor 120 may be communicatively coupled to the system memory 124 over a bus. In some examples, the processor 120 and the system memory 124 may be communicatively coupled to each other over the bus or a different connection.

It should be appreciated that in some examples, the computing device 104 may include a content encoder/decoder configured to receive graphical and/or display content from any source, such as the system memory 124 and/or the communication interface 126. The system memory 124 may be configured to store received encoded or decoded content. In some examples, the content encoder/decoder may be configured to receive encoded or decoded content, e.g., from the system memory 124 and/or the communication interface 126, in the form of encoded pixel data. In some examples, the content encoder/decoder may be configured to encode or decode any content.

The internal memory 121 or the system memory 124 may include one or more volatile or non-volatile memories or storage devices. In some examples, internal memory 121 or the system memory 124 may include RAM, SRAM, DRAM, erasable programmable ROM (EPROM) , electrically erasable programmable ROM (EEPROM) , flash memory, a magnetic data media or an optical storage media, or any other type of memory. In certain aspects, the internal memory 121 may further include a decay parameter 155 indicating one or more of: a decay ratio or a decay compensation for the processor 120 to adjust emission output of one or more pixels to maintain a consistent appearance in the display 136.

The internal memory 121 or the system memory 124 may be a non-transitory storage medium according to some examples. The term “non-transitory” may indicate that the storage medium is not embodied in a carrier wave or a propagated signal. However, the term “non-transitory” should not be interpreted to mean that internal memory 121 or the system memory 124 is non-movable or that its contents are static. As one example, the system memory 124 may be removed from the computing device 104 and moved to another device. As another example, the system memory 124 may not be removable from the computing device 104.

The processor 120 may be a central processing unit (CPU) , a graphics processing unit (GPU) , a general purpose GPU (GPGPU) , or any other processing unit that may be configured to perform graphics processing. In some examples, the processor 120 may be integrated into a motherboard of the computing device 104. In some examples, the processor 120 may be present on a graphics card that is installed in a port in a motherboard of the computing device 104, or may be otherwise incorporated within a peripheral device configured to interoperate with the computing device 104. The processor 120 may include one or more processors, such as one or more microprocessors, GPUs, application specific integrated circuits (ASICs) , field programmable gate arrays (FPGAs) , arithmetic logic units (ALUs) , digital signal processors (DSPs) , discrete logic, software, hardware, firmware, other equivalent integrated or discrete logic circuitry, or any combinations thereof. If the techniques are implemented partially in software, the processor 120 may store instructions for the software in a suitable, non-transitory computer-readable storage medium, e.g., internal memory 121, and may execute the instructions in hardware using one or more processors to perform the techniques of this disclosure. Any of the foregoing, including hardware, software, a combination of hardware and software, etc., may be considered to be one or more processors. In some aspects, the processor 120 may include or may integrate with the display processor 127.

In some aspects, the content generation system 100 can include a communication interface 126. The communication interface 126 may include a receiver 128 and a transmitter 130. The receiver 128 may be configured to perform any receiving function described herein with respect to the computing device 104. Additionally, the receiver 128 may be configured to receive information, e.g., eye or head position information, rendering commands, or location information, from another device. The transmitter 130 may be configured to perform any transmitting function described herein with respect to the computing device 104. For example, the transmitter 130 may be configured to transmit information to another device, which may include a request for content. The receiver 128 and the transmitter 130 may be combined into a transceiver 132. In such examples, the transceiver 132 may be configured to perform any receiving function and/or transmitting function described herein with respect to the computing device 104.

In some examples, the graphical content from the processor 120 for display via the display client 131 is not static and may be changing. Accordingly, the display processor 127 may periodically refresh the graphical content displayed via the display client 131. For example, the display processor 127 may periodically retrieve graphical content from the system memory 124, where the graphical content may have been updated by the execution of an application (and/or the processor 120) that outputs the graphical content to the system memory 124.

It should be appreciated that while shown as separate components in FIG. 1, in some examples, the display client 131 (sometimes referred to as a “display panel” ) may include the display processor 127.

As shown in FIG. 1, in certain aspects, the display processor 127 (also referred to as a display processing unit (DPU) ) may be configured to operate functions of the display client 131. For example, in certain aspects, the display processor 127 is configured to output a plurality of code words, such as corresponding to a frame, to the display controller 132. Each code word may be represented by a binary number in the digital domain. Each code word may correspond to a pixel (e.g., red, green, blue, white, etc. ) of the display 136.

The display controller 132 may be configured to convert the code words received from the display processor 127 to analog signals used to drive the pixels of the display 136. In certain aspects, for each code word corresponding to a pixel, the display controller 132 is configured to convert the code word to an analog signal (s) that drives the pixel to a particular brightness level. Accordingly, in certain aspects, the code word and/or analog signal (s) correspond to a brightness level for the pixel.

In certain aspects, the processor 120, the display processor 127, or both, may be configured to produce a UI layer on the display 136. The following description uses the processor 120 as an example, and similar techniques, methods, or processes performed by the processor 120 may also be performed in the display processor 127 or in combination therewith.

In certain aspects, the processor 120 is configured to retrieve, from a memory, such as internal memory 121 or system memory 124, the decay parameter 155. The decay parameter 155 may be indicative of the decay ratio or decay compensation of one or more pixels in the display 136, such as, for example, the light-pass region 205 of the display panel 240 in FIG. 2. In certain aspects, the computing device 104 may further include a camera positioned under the region in the display 136, such as the camera 210 of FIG. 2. In certain aspects, the processor 120 is configured to receive the decay parameter 155 from a second computing device and store the decay parameter 155 in the internal memory 121. In certain additional aspects, the decay parameter 155 may be specific to the computing device 104. For example, the decay parameter 155 may account for both the physical removal of materials in the display 136 and/or the positioning of the front camera (not shown in FIG. 1) behind the region.

In certain aspects, the processor 120 may compose an image, such as for the UI layer and one or more layers for the one or more pixels. In certain aspects, the UI layer is a separate layer from a base layer that displays content for the display 136. For example, the base layer may correspond to the full size of the display 136, while the separate UI layer corresponds to the one or more pixels. The processor 120 may compose the image for the UI layer based on the content of the base layer. There can be multiple content layers stacked in the z-axis while the UI layer is kept at the top of the stacked layers. The stack layers may be rendered to generate a frame by a graphics processing unit, such as the graphics processing pipeline 107. In some cases, the image for the UI layer may have modified emission intensity values for the one or more pixels to compensate for its faster decay.

As shown in FIG. 3, in certain aspects, the light-pass region 205 of the display panel 240 allows ambient light 310 to be transmitted through the display panel 240. As shown in FIG. 3, the light-pass region 205 may have reduced or removed reflective layers in the cathode layer 302, the organic luminescent material layer 304, and the anode layer 306. Alternative or in addition to the reduced or removed reflective layers, the organic luminescent material layer 304 may have pixels of a different density, shape, or size than the rest of the pixels of the display to facilitate light passing through the region. The ambient light 310 can be transmitted through the cathode layer 302, the organic luminescent material layer 304, and the anode layer 306, as indicated by the transmitted light 315. In some cases, the efficiency of the transmitted light 315 passing through the region 205 may be correlated with the rate of decay of the organic luminescent material layer 304.

The processor 120, or another suitable device, may be configured to generate the UI layer. For example, the UI layer may be required to have a smallest possible area to completely cover the region 205, to achieve a minimal overall size on the display panel 240. In other instances, the UI layer may be required to be concentric with the camera 210, whose position may be measured or presumed to be at a constant position relative to the edges of the display panel. In some other instances, the UI layer may be required to have a shape symmetric around the center of the camera 210. In still some other instances, the UI layer may be required to have a smooth edge as defined by a rounding radius threshold. In yet some other instances, the UI layer may be required to adjust its size, shape, and position based on user or application specific criteria and based on the decay parameter 155.

As described herein, a device, such as the computing device 104, may refer to any device, apparatus, or system configured to perform one or more techniques described herein. For example, a device may be a server, a base station, user equipment, a client device, a station, an access point, a computer, e.g., a personal computer, a desktop computer, a laptop computer, a tablet computer, a computer workstation, or a mainframe computer, an end product, an apparatus, a phone, a smart phone, a server, a video game platform or console, a handheld device, e.g., a portable video game device or a personal digital assistant (PDA) , a wearable computing device, e.g., a smart watch, an augmented reality device, or a virtual reality device, a non-wearable device, a display or display device, a television, a television set-top box, an intermediate network device, a digital media player, a video streaming device, a content streaming device, an in-car computer, any mobile device, any device configured to generate graphical content, or any device configured to perform one or more techniques described herein. Processes herein may be described as performed by a particular component (e.g., a GPU) , but, in further embodiments, can be performed using other components (e.g., a CPU) , consistent with disclosed embodiments.

FIG. 4 illustrates example operations 400 for a computing device operable to compensate decay of one or more pixels in a display. The example operations 400 may be performed on a computing device or a computing system. In some embodiments, the computing device includes an integral display and/or an external display. The example operations 400 may be performed by various components forming the computing device or computing system, even if such various components may be far apart. The operations 400 begins, at 405, by recoding a start pixel value time for a pixel of the display at a frame. In certain aspects, a timer is started at the start pixel value time. In certain aspects, the recording of the start pixel value time may be triggered when some criteria further described below are met. In some cases, the pixel may be controlled using a UI layer separated from a general display layer (such as one that displays a status bar) to reduce redundant composition, post processing, compensation calculation, or similarity check cycles.

At 410, a first pixel value of the pixel at a first frame is determined. The first frame is relative to the start pixel value time at 405. For example, the first frame may be any frame after the start pixel value time begins to record. At 415, a second pixel value of the pixel at the next frame, or a second frame subsequent to the first frame, is determined. For example, in certain aspects,

steps

410 and 415 compare the changing values of the pixel between the n ^th frame (e.g., second frame) and the (n-1) ^th frame (e.g., first frame) (or (n-m) ^th frame) . In certain aspects, the changing values are compared for each such pair of first and second frames after the start pixel value time. In certain aspects, the pixel value (either the first or the second) may be a value for red (R) , a value for green (G) , a value for blue (B) , a weighted average of RGB, or an intensity value calculated from these values. In certain aspects, the intensity value may be calculated using summation or selecting the greatest value of the three.

At 420, a difference between the second pixel value and the first pixel value is determined to satisfy a threshold value. For example, when the difference is less than the threshold value, the decay rate of the pixel due to burn-in may be neglected. In other words, one of the criteria to trigger the decay accumulation and compensation is how much drastic changes have applied to the pixel. A small difference may indicate a gradual change that needs not trigger the decay recording. For example, if the pixel changes from black to dark grey, the change in emission intensity would not significantly contribute to the decay of the pixel. Including such minor changes may unreasonably burden monitoring and recording. On the other hand, specific settings, configured specifically to the computing device, display panel, or by the user, may adjust the threshold value based on preferences or requirements in specific circumstances, to properly record and monitor the level of decay. For example, if the pixel is near its end of life span, the device may be configured to apply a smaller threshold value to increase the accuracy in start the decay monitoring and compensation process.

At 425, based on the difference satisfying the threshold, a stop pixel value time is recorded for the pixel. In certain aspects, a new start pixel value time is also recorded for the pixel when the stop pixel value time is recorded. Therefore, the time between the start pixel value time and the stop pixel value time is indicative of a time that the pixel had approximately a particular value for and that value did not change by more than a threshold amount between subsequent frames for the time.

At 430, a pixel decay parameter associated with the pixel is determined based on the start pixel value time, the stop pixel value time, and the first pixel value. For example, the pixel decay parameter can be a pixel decay ratio that reflects an accumulation of decay. The accumulation can be calculated as a sum of a previous decay ratio and a newly calculated amount of decay equal to the multiplication of a time duration and a rate of decay. As determined at 425, the time duration is determined using the start pixel value time and the stop pixel value time. The rate of decay may be specific to the first pixel value, for example, a value corresponding to a high number (i.e., very bright) is associated with a fast rate of decay, while a value of a low number is associated with a slow rate of decay. For example, a white color output of (R, G, B) = (255, 255, 255) has the highest rate of decay among other possible values; while a black color output of (R, G, B) = (0, 0, 0) has negligible impact to the lifespan of the pixel. The specific rate of decay for all the R, G, B value combinations may be determined in a laboratory or testing environment, or based on specification sheet of the display panel.

In certain aspects, the decay ratio may further include consideration of the average decay of the rest of the display. For example, the decay ratio of the pixel may be a difference between an actual accumulation of the decay ratio of that pixel and an overall average decay ratio of the display. This may enable accurate compensation to maintain a consistent appearance, as the rest of the display also darkens over time.

At 435, a decay compensation is applied to the pixel based on the pixel decay parameter. For example, when the pixel decay parameter is greater than a threshold, the GPU or CPU of the computing device may request the pixel to emit light in a greater intensity to overcome the decay. The threshold may be a level corresponding to a visual decay that is visually observable. The decay compensation may be processed along with continuous post processing and composition as one single batch command. An example code or program is provided below:

In certain aspects, to reduce unnecessary resource use, the operations 400 may be limited to or triggered by events updating content in the UI layer in a region corresponding to the one or more pixels. That is, because the UI layer is separate from the overall layer (such as the layer showing the status bar at the same position as the light-pass layer) , the computation on monitoring the decay ratio and applying the decay compensation is minimized. For example, the pixel time statistics may be included in a GPU composition layer. The pixel compensation may be set as mandatory for the region to compensate any burn-in effects. In certain aspects, contrary to some techniques that focus on bright icons displayed in some cases, the operations 400 is applicable to every pixel in the physical layer of the region, as a result of the separate UI layer discussed above.

In certain aspects, the pixel decay monitoring and compensation may be applied to all frames or only some frames, such as at a sampling rate. For example, the pixel values may be sampled at 30 Hz, or 60 Hz, among others. As discussed, operations 400 in certain aspects, are performed base on satisfaction of conditions or criteria to avoid wasting resources. For example, events updating visible layers may trigger the operations 400. In one example, the rate for performing the operations 400 may be about 10 Hz, while the display overall may have a refreshing frequency of 60 Hz. The frequency of updating the UI layer of the region may be about 0.1 Hz in this example. Other refreshing rates or sampling rates are proper depending on specific applications, though not enumerated herein. In certain aspects, the operations 400 are conducted or performed in the separate or split UI layer.

In one configuration, a method or apparatus for display processing is provided. The apparatus may be a processing unit, a display processor, a display processing unit (DPU) , a graphics processing unit (GPU) , a video processor, or some other processor that can perform display processing. In some examples, the apparatus may be the processor 120 within the computing device 104, or may be some other hardware within the computing device 104, or another device.

In accordance with this disclosure, the term “or” may be interrupted as “and/or” where context does not dictate otherwise. Additionally, while phrases such as “one or more” or “at least one” or the like may have been used for some features disclosed herein but not others, the features for which such language was not used may be interpreted to have such a meaning implied where context does not dictate otherwise.

In one or more examples, the functions described herein may be implemented in hardware, software, firmware, or any combination thereof. For example, although the term “processing unit” has been used throughout this disclosure, such processing units may be implemented in hardware, software, firmware, or any combination thereof. If any function, processing unit, technique described herein, or other module is implemented in software, the function, processing unit, technique described herein, or other module may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media may include computer data storage media or communication media including any medium that facilitates transfer of a computer program from one place to another. In this manner, computer-readable media generally may correspond to (1) tangible computer-readable storage media, which is non-transitory or (2) a communication medium such as a signal or carrier wave.

Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and/or data structures for implementation of the techniques described in this disclosure. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, . Disk and disc, as used herein, includes compact disc (CD) , laser disc, optical disc, digital versatile disc (DVD) , floppy disk, and Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media. A computer program product may include a computer-readable medium.

The code may be executed by one or more processors, such as one or more digital signal processors (DSPs) , general purpose microprocessors, application specific integrated circuits (ASICs) , arithmetic logic units (ALUs) , field programmable logic arrays (FPGAs) , or other equivalent integrated or discrete logic circuitry. Accordingly, the term “processor, ” as used herein may refer to any of the foregoing structure or any other structure suitable for implementation of the techniques described herein. Also, the techniques could be fully implemented in one or more circuits or logic elements.

The techniques of this disclosure may be implemented in a wide variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC) or a set of ICs, e.g., a chip set. Various components, modules or units are described in this disclosure to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily need realization by different hardware units. Rather, as described above, various units may be combined in any hardware unit or provided by a collection of interoperative hardware units, including one or more processors as described above, in conjunction with suitable software and/or firmware.

Various examples have been described. These and other examples are within the scope of the following claims.

Claims

A method for compensating for pixel decay of a display, the method comprising:

recording a start pixel value time for a pixel of the display at a frame;

determining a first pixel value of the pixel at a first frame;

determining a second pixel value of the pixel at a second frame;

determining that a difference between the second pixel value and the first pixel value satisfies a threshold; and

based on the difference satisfying the threshold:

recording a stop pixel value time for the pixel;

determining a pixel decay parameter associated with the pixel based on the start pixel value time, the stop pixel value time, and the first pixel value; and

applying a decay compensation to the pixel based on the pixel decay parameter.
The method of claim 1, wherein applying the decay compensation to the pixel comprises adjusting pixel value or luminance of the pixel for one or more frames based on the pixel decay parameter.
The method of claim 1, wherein applying the decay compensation to the pixel based on the pixel decay parameter comprises applying the decay compensation to the pixel based on the pixel decay parameter based on whether the pixel decay parameter satisfies a second threshold.
The method of claim 1, wherein determining the pixel decay parameter is further based on an average decay parameter for the display.
The method of claim 1, wherein determining the pixel decay parameter is further based on one or more additional pixel decay parameters associated with the pixel corresponding to one or more previous time periods of output of pixel values by the pixel.
The method of claim 1, wherein the pixel is one of a subset of pixels of a plurality of pixels of the display, wherein the applying the decay compensation to the pixel based on the pixel decay parameter is based on the pixel being part of the subset of pixels.
The method of claim 1, further comprising:

based on the difference satisfying the threshold, recording a second start pixel value time for the pixel.
The method of claim 1, wherein determining the pixel decay parameter is further based on an aging model for the first pixel value.
The method of claim 1, wherein determining the pixel decay parameter is further based on an average aging model for the display.
A computing device comprising:

a memory; and

a processor coupled to the memory wherein the memory and the processor are configured to perform the method of one or more of claims 1-9.
A computing device comprising:

various means for performing the method of one or more of claims 1-9.
A non-transitory computer-readable medium including instructions that when executed by a computing device, cause the computing device to perform the method of one or more of claims 1-9.