EP4584776A1

EP4584776A1 - Adjusting luminance responsive to changing refresh rate

Info

Publication number: EP4584776A1
Application number: EP23829241.1A
Authority: EP
Inventors: designation of the inventor has not yet been filed The
Original assignee: Google Technology Holdings LLC
Current assignee: Google LLC
Priority date: 2023-11-27
Filing date: 2023-11-27
Publication date: 2025-07-16
Also published as: WO2025116885A1

Abstract

This document describes systems and techniques for compensating for changes in luminance of frames of video data as a result of changing refresh rates at a display system. For example, a method includes selecting a luminance compensation factor to compensate for a change in luminance level of a display system resulting from a change in a current refresh rate of a current frame of video data to a next refresh rate of a next frame of the video data, the luminance compensation factor selected based on the current refresh rate associated with the current frame of the video data and the next refresh rate associated with the next frame of the video data. The luminance compensation factor is applied to the next frame of the video data to provide adjusted video data to adjust a luminance level of the next frame.

Description

ADJUSTING LUMINANCE RESPONSIVE TO CHANGING REFRESH RATE

BACKGROUND

[0001] Individuals increasingly use their mobile telephones and other mobile devices for watching video. Mobile devices are readily available to access video data wherever the user may be. Further, improving wireless communications speeds, increasing battery life, and improving display resolution all serve to make watching video on mobile devices increasingly more inviting.

[0002] However, there remain some shortcomings in watching video on a mobile device. For example, some video display technologies may be negatively affected when video refresh rates change. When refresh rates change, one or more frames displayed after the change in refresh rate may be dimmer than preceding frames. Although luminance of the video stabilizes over time, an abrupt shift in luminance may disrupt the viewer’s engagement with the video.

[0003] Some techniques have been devised to overcome such luminance changes. For example, some devices may insert intermediate frames at refresh rates between the original refresh rate and the changed refresh rate to avoid the appearance of dimmed frames. Although the use of intermediate frames may alleviate dimming of the video data, insertion of intermediate frames consumes a significant amount of power, which may be problematic when relying on limited battery power. Mobile display technologies would benefit from being able to avoid abrupt changes in luminance as a result of changing refresh rates without the increased power consumption resulting from current techniques.

SUMMARY

[0004] This document describes systems and techniques for compensating for changes in luminance of frames of video data caused by changing refresh rates. Luminance compensation, which may be based on a current refresh rate of a current frame, on a next refresh rate associated with a next frame of video data, and on a luminance level of the current frame of video data may be applied to the next frame to lessen or prevent changes in luminance levels without the power compensation involved in inserting intermediate video frames.

[0005] For example, a method includes selecting a luminance compensation factor to compensate for a change in luminance level of a display sy stem resulting from a change in a current refresh rate of a current frame of video data to a next refresh rate of a next frame of the video data, the luminance compensation factor selected based on the cunent refresh rate associated with the current frame of the video data and the next refresh rate associated w ith the next frame of the video data. The luminance compensation factor is applied to the next frame of the video data to provide adjusted video data to the display system to adjust a next luminance level of the next frame of the video data to compensate for the change in the luminance level resulting from the change in the refresh rate.

[0006] This Summary is provided to introduce systems and techniques for adjusting luminance of frames of video data to compensate for luminance changes caused by changing refresh rates at a display system, as further described below in the Detailed Description and Drawings. This Summary' is not intended to identify essential features of the claimed subject matter, nor is it intended for use in determining the scope of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The details of one or more aspects of systems and techniques for adjusting luminance of frames of video data to compensate for luminance changes caused by changing refresh rates are described in this document with reference to the following drawings. The same numbers are used throughout the drawings to reference like features and components:

[0008] FIG. 1 is a block diagram of a system for compensating for luminance changes of frames of video data caused by changing refresh rates:

[0009] FIG. 2 is a schematic diagram of frames of video data in w hich luminance of a frame is dimmed following a reduction in refresh rate;

[0010] FIG. 3 is a graph of luminance over time of the video data of FIG. 2;

[0011] FIG. 4 is a segment of the luminance graph of FIG. 2 representing a dimmed frame enlarged to depict application of a luminance compensation factor;

[0012] FIG. 5 is the graph of FIG. 3 including the application of the luminance compensation of FIG. 4;

[0013] FIG. 6 is the schematic diagram of FIG. 2 in which the luminance compensation counteracts the dimming of the previously' dimmed frame:

[0014] FIG. 7 is a schematic diagram of techniques of determining luminance of a frame of video data;

[0015] FIG. 8 is a schematic diagram of generating a luminance compensation table from preexisting or simulated video data usable as a store of compensation data;

[0016] FIG. 9 is a schematic diagram of alternative forms of luminance compensation tables;

[0017] FIG. 10 is a schematic diagram of a store of compensation data to account for different cunent luminance values, current refresh rates, and next refresh rates:

[0018] FIG. 11 is a block diagram of the system of FIG. 1 including additional components for determining and buffering values for compensating for luminance changes; and

[0019] FIG. 12 is a flow diagram of an example method for adjusting luminance of frames of video data to compensate for luminance changes caused by changing refresh rates. DETAILED DESCRIPTION

OVERVIEW

[0020] Some display technologies, such as organic light emitting diode (OLED) displays, may suffer from abrupt dimming of video data when a screen refresh rate changes. In particular, after a reduction in refresh rate, a next frame tends to significantly dim in luminance as a result of hysteresis of the OLED display’s drive transistor. Insertion of intermediate image frames by display driver circuitry' (e.g., one or more frames displayed at intermediate refresh rates between a previous refresh rate and a next refresh rate) helps to alleviate the drop in luminance and, thus, helps to avoid the visual disruption resulting from the drop in luminance. However, a significant amount of power is consumed each time the display driver circuitry generates such intermediate video frames.

[0021] Systems and techniques described herein overcome the dimming of video frames in response to changing refresh rates without inserting intermediate video frames. Instead, in response to detecting a change in refresh rate between a next frame of video data and a current frame of video data, a luminance compensation factor is applied to adjust the luminance of the next frame of video data. The luminance compensation factor is derived from changes in luminance level as a result of a change from a current refresh rate at which a current frame of video data is displayed and a next refresh rate at which a next frame of video data is displayed. In implementations, the luminance compensation factor is also based on a current luminance level at which the current frame is displayed and the change from the current refresh rate to the next refresh rate.

[0022] In implementations, the luminance compensation factor is empirically detennined from preexisting or simulated video data representing a change in luminance resulting from the change from the current refresh rate to the next refresh rate and/or approximating a difference between the current luminance level and a next luminance level. The luminance compensation factor may be retrieved from a store of compensation data, which may include luminance compensation tables for different current refresh rates, next refresh rates, and current luminance levels derived from the preexisting or simulated video data.

[0023] This document describes systems and techniques for compensating for changes in video luminance (e.g., perceived by a person as brightness) of frames caused by changing refresh rates. By detecting refresh rates and luminance of current and next frames of video data, luminance compensation can be applied to the next frame of video data to compensate for luminance changes without the power compensation involved in inserting intermediate video frames. SYSTEM FOR ADJUSTING LUMINANCE RESPONSIVE TO CHANGING REFRESH RATE

[0024] FIG. 1 illustrates a system 100 for adjusting for changes in luminance, such as dimming, as a result of changes in refresh rate. The system 100 includes a luminance compensator 102. In implementations, the luminance compensator 102 uses a store of compensation data 104, which may include one or more look-up tables as further described below. The luminance compensator 102 is responsive to attributes of a current frame 106 and a next frame 108 of video data 110 generated by a graphics processing unit 112. Specifically, the luminance compensator 102 includes a current refresh rate input 114, a next refresh rate input 116, and optionally a current luminance input 118. Based on the inputs 114, 116, and optionally 118. the luminance compensator 102 produces a luminance compensation factor 120 that is presented to a display interface 122.

[0025] In implementations as further described below, the luminance compensator 102 uses the inputs 114, 116, and 118 to select or retrieve an applicable luminance compensation factor 120 from the store of compensation data 104. The store of compensation data 104 maintains a plurality of luminance compensation factors derived from a change in luminance of representative sets of video data that may be displayed using a display driver 124 on a display 126 as a result of a difference in a current luminance level of the current frame 106 of the video data 110 display able at a current refresh rate and a next luminance level of the next frame 108 of the video data 110 displayable at a next refresh rate via the display driver 124 and the display 126.

[0026] In implementations, after the luminance compensator 102 provides the luminance compensation factor 120 to the display interface 122, the display interface 122 applies the luminance compensation factor 120 to the next frame of video data 108. For example, without luminance compensation, a reduction in refresh rate between the current refresh rate received at the current refresh rate input 114 and the next refresh rate received at the next refresh rate input 116 may result in the next frame 108 of the video data 110 displayed by the display driver 124 and the display 126 being noticeably dimmer than the previously current frame 106 of the video data 110 displayed at the former current refresh rate. However, the selected luminance compensation factor 120 applied at the display interface 122 generates adjusted video data 128 that is presented to the display driver 124 in which the luminance level of the next frame 108 of the video data 110 is boosted or otherwise changed. As a result, the next frame 108 is presented via the display driver 124 and the display 126 without noticeable dimming as compared to the current frame 106 of the video data 110.

[0027] The luminance compensation factor 120 may include a luminance reduction factor representative of a degree of dimming resulting from a change in the refresh rate. In other words, the luminance reduction factor may be a ratio of the next luminance level of the next frame 108 of the video data 110 presented at the next refresh rate to the cunent luminance level of the current frame 106 of the video data 110 presented at the current refresh rate. By applying the luminance compensation factor 120 that represents an inverse of the ratio of the next luminance level to the current luminance level, the luminance compensation factor 120 alleviates the dimming of the next frame 108 of the video data 110 when displayed by the display driver 124 and the display 126. As described below, the luminance compensation factor 120 representing the ratio of the next luminance level to the current luminance level resulting from changing from the current refresh rate to the next refresh rate - or the inverse of this ratio - may be determined from representative video data and stored in the compensation data 104 for retrieval and application to the next frame 108 of the video data 110.

[0028] Therefore, the adjusted video data 128 prevents, or at least reduces, dimming or other changes in the luminance level presented via the display driver 124 and the display 126 as a result of changing refresh rates. In other words, the system 100 thus provides seamless frame transition without noticeably significant changes in dimming between frames of video data as a result of changes in refresh rate.

[0029] FIG. 2 is a set of video data 200 including a first frame of video data 202, a second frame of video data 204, and a third frame of video data 206. The set of video data 200 includes slightly varied images to represent how the set of video data 200 represents a stream of video data. For purposes of this example, it is assumed that the first frame 202 of the video data 200 is presented at a current refresh rate (not shown), while the second frame 204 and the third frame 206 of the video data 200 are presented at a reduced refresh rate (also not shown). For further purposes of this example, it is assumed that the frames of video data 202, 204, and 206 are presented using an organic light-emitting diode (OLED) display system 208, including an OLED display and associated display driver circuitry. As previously described, using an OLED display, first-frame dimming may result for a next frame of video data subject after a refresh rate is reduced, causing the next frame to be presented at a reduced luminance. The second frame 204 of the video data 200 reflects this reduction in luminance, as depicted by shading 210 of the second frame 204 of the video data 200. The luminance of the third frame 206 of the video data 200 is not shaded because, after a next frame of video data is presented after a change in refresh rate, luminance normalizes or at least begins to normalize.

[0030] FIG. 3 is a graph 300 of luminance over time representing the luminance of the frames of video data 202, 204, and 206 in the set of video data 200 (see FIG. 2). Specifically, the graph 300 represents first luminance data 302 of the first frame 202 of the set of video data 200, second luminance data 304 of the second frame 204 of the set of video data 200, and third luminance data 306 of the third frame 206 of the set of video data 200. The set of video data 200 generally represents a same subject matter, and thus the luminance data 302, 304, and 306 of the frames of video data 202, 204, and 206 in the set of video data 200 depicted in the graph 300 may be anticipated to reach generally equivalent luminance levels.

[0031] As might be expected, a first luminance level 308 of the first luminance data 302 for the first frame 202 of the video data 200 is approximately the same as a third luminance level 310 of the third luminance data 306 for the third frame 206 of the video data 200. However, because of the dimming of the second frame 204 of the video data 200 (represented by the shading 210 as a result of first-frame dimming), a second luminance level 312 does not approach the first luminance level 308 or the third luminance level 310. This first-frame dimming may be noticeable and, thus, may detract from a user’s engagement with or enjoyment of the set of video data 200.

[0032] FIG. 4 graphically represents application of a luminance compensation factor 400 to the second luminance data 304 representing the second frame 204 of the video data 200 (see FIGS. 3 and 2, respectively) to alleviate first-frame dimming. Application of the luminance compensation factor 400, as represented by a vector 402, boosts or amplifies the second luminance data 304 to generate adjusted second luminance data 404 (represented in dotted lines in FIG. 4). As previously described, the luminance compensation factor 400 may represent an inverse of the ratio of a next luminance level of a next frame of video data, in this case the second luminance level 312, to a cunent luminance level of a current frame of video data, which in this case is the first luminance level 308 of the first frame 202 of the set of video data 200. The adjusted second luminance level data 404 reaches an adjusted second luminance level 406, which is approximately equivalent to the first luminance level 308 of the first frame 202 of the video data 200 and the third luminance level 310 of the third frame 206 of the video data 200.

[0033] FIG. 5 shows a graph 500 of luminance over time in which the second luminance data 304 (not shown in FIG. 5) is replaced with the adjusted second luminance data 404 (shown in dotted lines in FIG. 5). As a result of application of the luminance compensation factor 400 (see FIG. 4), the adjusted second luminance data 404 has the adjusted second luminance level 406, which is approximately equivalent to the first luminance level 308 of the first luminance data 302 and the third luminance level 310 of the third luminance data 306. Accordingly, because the adjusted second luminance level 406 of the second frame 204 is now approximately the same as the first luminance level 308 of the first frame 202 and the third luminance level 310 of the third frame 206, dimming of the second frame 204 is averted or alleviated.

[0034] As a result, referring to FIG. 6, adjusted video data 600 includes an adjusted second frame 602 that is not dimmed and thus does not include the shading 210 of the second frame 204 of the set of video data 200 that would have been presented w ithout application of the luminance compensation factor 400 to the second luminance data 304. Thus, determination and application of an appropriate luminance compensation factor 400 (see FIG. 4) may alleviate or prevent changes in luminance level as a result of changes in refresh rate.

DETERMINATION OF LUMINANCE COMPENSATION FACTORS

[0035] In implementations, values of the luminance compensation factor 400 (see FIG. 4) are determined or derived empirically from frames of video data as presented by a display driver before and after a change in refresh rate. Frame dimming is a function of a current refresh rate and a next refresh rate and may also be dependent upon a luminance level of a current frame of video data before the change in refresh rate. The extent of the frame dimming also may be a function of a particular display technology being used. As previously described, an OLED display is subject to first-frame dimming after a change in refresh rate because of hysteresis of a drive transistor. Thus, the derivation of appropriate values of the luminance compensation factor 400 may be a function of changing luminance levels as a result of changing refresh rates for a particular display technology.

[0036] As described with reference to FIG. 1, the luminance compensator 102 receives the current refresh rate input 114 and the next refresh rate input 116 and uses the inputs 114 and 116 to retrieve an appropriate luminance compensation factor from the store of compensation data 104. The compensation data 104 may include empirically derived luminance compensation factors for different luminance levels as the refresh rate shifts from one refresh rate to another. Derivation of the luminance compensation factors and the store of compensation data 104 is described with reference to FIGS. 7-11.

[0037] Suitable luminance compensation factors for a particular display driver and/or display are determined using a representative display driver and/or representative display using operationally equivalent or similar display technology. A representative display driver may be referred to as an evaluation display driver. Referring to FIG. 7, a luminance level of each of the frames of image data is generated by a representative display driver 700 and associated representative display 702. To determine appropriate luminance compensator factors, the representative display driver 700 and representative display 702 should be of the same type as the display driver and display that for which the luminance compensation factors are derived. As previously described, for example, an OLED display is subject to a certain degree of first-frame dimming, which may be different than what would be displayed using a different display technology. Thus, when determining luminance compensation factors for the compensation data 104 of the system 100 of FIG. 1, the representative display driver 700 and/or the representative display 702 should employ operationally equivalent (or functionally similar) display technology⁷ to the display technology used by the display driver 124 and the display 126 of the system 100. A representative display driver and representative display may be referred to collectively as a representative display system. The representative display driver, representative display, and representative display system may alternatively be referred to as an evaluation display driver, an evaluation display, and an evaluation display system, respectively. The evaluation and/or representative displays, display drivers and display systems may be considered to be operationally equivalent to the display driver, display, and/or display system upon which the compensated video data is to be displayed.

[0038] FIG. 7 depicts determination of a luminance level that may be generated by a representative display driver 700 and associated representative display 702. As previously described, to determine appropriate luminance compensator factors, the representative display driver 700 and representative display 702 should be of the same (or similar) type as the display driver and display for which the luminance compensation factors are derived. Taking a representative frame of video data 704 (which may also be referred to as “‘evaluation video data”) as an example, a luminance determination may be performed by determining an on-pixel ratio (OPR) for the frame of video data 704. A histogram 706 represents the OPR for the frame of video data 704. OPR is a ratio of the sum of red-green-blue (RGB) values of each of the pixels included in the representative frame of video data 704 as compared to what the RGB values would be if all the pixels were on so that the combined output of the pixels would be seen as white light. Thus, OPR is higher when more of the pixels on the display 702 generate the most illumination as opposed to when fewer of the pixels generate their highest possible illumination. In other words, OPR represents the percentage of the RGB (or other color pixels, such as magenta, cyan, green, etc.) values for the representative frame of video data 704 relative to a maximum luminance of the display 702.

[0039] Alternately, the luminance level of the representative frame of video data 704 may be made by generating a histogram 708 of luminance of the RGB pixels in the representative frame of video data 704. The histogram 708 includes curves 710, 712, and 714 that represent a red output 710. a green output 712, and a blue output 714 of the RGB pixels in the frame of video data 704. An aggregation of the red output 710, the green output 712, and the blue output 714 also may be used to determine luminance of the representative frame of video data 704. By determining OPR or generating histograms of frames of image data, such as the representative frame of image data 704, a relative luminance level of images may be determined relative to a maximum for the representative display driver 700 and the representative display 702.

[0040] In addition to determining OPR or generating histograms for determining luminance of representative frames of image data, such as the representative frame of image data 704, these methods may be used to determine a luminance level of the current frame 106 of the video data 110 in the system 100 to provide the current luminance level input 118 to the luminance compensator 102 of the system of FIG. 1. Thus, these methods may be used in generating the compensation data 104 and in analyzing the luminance level of the current frame 106 of the video data 110.

[0041] Referring to FIG. 8, representative video data 800 is used as a basis for determining a plurality of luminance compensation factors to create a store of compensation data 104 (see FIG. 1). The representative video data 800 may include preexisting or simulated frames of video data of particular luminance levels (which may be determined using one of the methods described with reference to FIG. 7) at a succession of current refresh rates and next refresh rates, which may be used to determine appropriate luminance compensation factors 400 (see FIG. 4). The luminance compensation factors 400 are then collected to develop the store of compensation data 104 for use by the luminance compensator 102 (see FIG. 1). In the foregoing example, the refresh rates are expressed in Hz and the luminance levels are expressed in OPR, as previously described with reference to FIG. 7. In other words, the store of compensation data 104 is created by accessing representative sets of video data in which each of the sets of video data includes a first frame of video data display able at a current or first refresh rate and a second frame of video data display able at a next or second refresh rate. Based on a difference between a next or second luminance level of the second frame and a current or first luminance level of the first frame, a plurality of luminance compensation factors is determined. The store of compensation factors 104 include a plurality of luminance compensation factors, with each of the luminance compensation factors being associated with a combination of the first refresh rate and the second refresh rate, as described below.

[0042] In the foregoing example, the representative video data 800 includes a first frame of preexisting video data (frame) 802 that is presented at a first refresh rate of 96 Hz 804 with a first luminance level of 50% OPR 806. The representative video data 800 also includes a number of second frames 808, 810, and 812 of preexisting video data presented at different, second refresh rates. The first frame 802 with each of the second frames 808, 810, and 812 form representative sets of video data 814, 816, and 818 that are used to determine a change in luminance level following a change in refresh rate to determine a level of first-frame dimming to be adjusted by the luminance compensator 102 (see FIG. 1).

[0043] For example, a first representative set of video data 814 includes the first frame 802 at the first refresh rate of 96 Hz 804 and the second frame 808 at a second refresh rate of 16 Hz 820. A second representative set of video data 816 includes the first frame 802 at the first refresh rate of 96 Hz 804 and the second frame 810 at a second refresh rate of 32 Hz 822. A third representative set of video data 818 includes the first frame 802 at the first refresh rate of 96 Hz 804 and the second frame 812 at a second refresh rate of 64 Hz 824. From the representative sets of video data 814, 816, and 818 and other sets of video data, luminance compensation factors are determined to generate the store of compensation data 104 (see FIG. 1). In the example of FIG. 8, the compensation data 104 includes a luminance compensation table 826 in which the luminance compensation factors derived may be stored. The luminance compensation table 826 includes rows representing current refresh rates 828 and columns representing next refresh rates 830. The luminance compensation factors derived are stored in cells associated with a combination of a current refresh rate at which the first frame 802 is presented and a next refresh rate at which one of the second frames 808, 810. and 812 is displayed, as further described below.

[0044] Starting with the first representative set of video data 814, the second frame 808 is presented at the second refresh rate 16 Hz 820 and is determined to have a second luminance level of 42.3% OPR 832. A luminance compensation factor 834 may be a ratio of the luminance of the next luminance level 832 of the second frame 808 to the first luminance level 806 of the first frame 802, yielding the luminance compensation factor 834 of .845. The luminance compensation factor 834 of .845 signifies that the second frame 808, presented as a result of the change from the first refresh rate of 96 Hz 804 of the first frame 802 to the second frame 808 at the next refresh rate of 16 Hz 820, has 85.2% of the luminance of the first frame 802 or is nearly 15% dimmer than the first frame 802. This empirically derived luminance compensation factor 834 thus indicates a degree to which the luminance compensator 102 (see FIG. 1) should boost the luminance of the next frame of video data, such as to avoid the appearance of dimming in the next frame 808. For example, the luminance compensator 102 may boost the luminance level by the inverse of the empirically derived luminance compensation factor 834 to compensate for the first- frame dimming caused by the reduction in refresh rate to alleviate noticeable first-frame dimming.

[0045] For the luminance compensator 102 to be able to access and retrieve the luminance compensation factor 834, the luminance compensation factor 834 is stored in a cell 836 of the luminance compensation table 826 associated with a row 838 for the current refresh rate of 96 Hz 804 and a column 840 for the next refresh rate of 16 Hz 820. Thus, the cell 836 in which the luminance compensation factor 834 is stored is associated with a combination of a first refresh rate representing the current refresh rate 806 of 96 Hz 838 and a second refresh rate representing the next refresh rate 820 of 16 Hz 840.

[0046] The ratio 834 represents a degree of first-frame dimming for a first frame having a luminance level of 50.0% OPR, as in the example first frame 802, when the refresh rate changes from the first refresh rate of 96 Hz 804 to the second refresh rate of 16 Hz 820. Thus, in the system 100 of FIG. 1, when the luminance compensator 102 has a current frame 106 at a current luminance level of 50% OPR at a current refresh level of 96 Hz and is transitioning to a next frame 108 at a next refresh rate of 16 Hz, the luminance compensator 102 may access the rows of current refresh rates 828 of the luminance compensation table 826 at the row 838 having the first refresh rate 804 corresponding to the current refresh rate of 96 Hz. Then, the luminance compensator 102 may access the columns of next refresh rates 830 at the column 840 having the second refresh rate 820 corresponding to the next refresh rate of 16 Hz. The luminance compensator 102 then applies the luminance compensation factor834 to boost the luminance level of the next frame 108 of video data to overcome the first-frame luminance reduction of .845 834. Thus, the luminance compensator 102 may boost the luminance of the next frame 808 by an inverse of the luminance compensation factor .845 834 to avoid the appearance of dimming of the next frame 808.

[0047] Although the luminance compensation factor .845 834 is derived from the first representative set of video data 814, which represents specific pre-existing or simulated images, the change in luminance is based on the change in the refresh rate from the current refresh rate 96 Hz 806 to the next refresh rate 16 Hz 820. Thus, the luminance compensation factor .845 834 is applicable to other frames of video data when the refresh rate from a cunent frame of video data at the current refresh rate of 96 Hz 806 changes to the next refresh rate of 16 Hz 820.

[0048] The luminance compensation table 826 is derived using a first luminance level of 50.0% OPR 842, representing the first luminance level 50.0% 806 of the first frame 802. The luminance compensation table 826 may be usable as a source for other luminance compensation factors when the current frame 802 has a current luminance level other than 50.0% to estimate the dimming from the current refresh rates 828 to the next refresh rates 830. The luminance compensation factors stored in the luminance compensation table 826 may be applicable for a range of current luminance values. The luminance compensation table 826 may be particularly well-suited for use with current lamination factors that are close to the current luminance level of 50% OPR 842. However, it should be noted that the change in luminance from a current frame displayed at a current refresh rate to a next frame displayed at a next refresh rate also may depend on the current luminance level, depending on the display technology used. Thus, it may be desired to generate additional luminance compensation tables for different values of current luminance levels, as described with reference to FIG. 10.

[0049] Similar to the luminance compensation factor .845 834 selected for the change in refresh rate from the first refresh rate of 96 Hz 804 to the second refresh rate of 16 Hz 820 for the first representative set of video data 814, luminance compensation factors may be determined for other representative sets of video data to complete the luminance compensation table 826. In the second set of video data 816, the second frame 810 generated at the second refresh rate 32 Hz 822 is determined to have a second luminance level of 44.0% OPR 844. A ratio of the second luminance level 44.0% OPR 844 to the first luminance level 50.0% 806 is .880 846, which is stored in a cell 848 of the luminance compensation table 826 in the row for which the first refresh rate is 96 Hz 838 and a column for which the second refresh rate is 32Hz 850. Similarly, the second frame 812 generated at the second refresh rate 64 Hz 824 is determined to have a second luminance level of 46.9% OPR 852. A ratio of the second luminance level 46.9% OPR 852 to the first luminance level 50.0% 806 is .938 854, which is stored in a cell 856 of the luminance compensation table 826 in the row for which the first refresh rate is 96 Hz 838 and a column for which the second refresh rate is 64 Hz 858.

[0050] In this manner, the rows and columns of the luminance compensation table 826 are populated for images at the first luminance level of 50% 806. The luminance compensator 102 (see FIG. 1) then draws from the luminance compensation table 826 from the store of compensation data 104 for the applicable luminance compensation factor 120 based on the current refresh rate 828 and the next refresh rate 830.

[0051] Referring to FIG. 9, as previously described, the data stored in the luminance compensation table 826 may include a ratio of a next refresh rate to a current refresh rate for a particular current luminance level to represent the first-frame dimming of a next frame of video data. Accordingly, the luminance compensation table 826 may be regarded as storing luminance reduction factors 900 representative of the decreased luminance resulting from decreased refresh rates. As also previously described, the luminance compensator 102 may boost the luminance level by the inverse of the relevant luminance reduction factor retrievable from the luminance compensation table 826 as described with reference to FIG. 8.

[0052] On the other hand, instead of maintaining the luminance compensation table 826 that maintains the luminance reduction factors 900, a luminance compensation table 902 may store luminance boost factors 904 that represent inverses of each of the corresponding luminance reduction factors 900. As previously described, the luminance reduction factors 900 would include the luminance compensation factor 834 representing the luminance reduction factor in the cell 836 of the luminance compensation table 826 for the current luminance level 806 in the row for which the current refresh rate is 96 Hz 838 and the column for which the next refresh rate is 16 Hz 840. By contrast, the luminance compensation table 902 stores inverses of the luminance reduction factors 900 as the luminance boost factors 904. Thus, the luminance compensation table 902 stores 1.18 906, an inverse of the luminance compensation factor 834, in the luminance compensation table 902 in a cell 908 in one of the rows of current refresh rates 910 for which a current refresh rate is 96 Hz 912 and in one of the columns of next refresh rates 914 for which a next refresh rate is 16 Hz 916. The remainder of the luminance compensation table 902 may be similarly populated to include luminance boost factors 904 for the current luminance level 842. [0053] The luminance compensation tables 826 and 902 of FIG. 9, as described, represent the luminance reduction factors 900 or luminance boost factors 904 for frames having the current luminance level of 50% OPR 842. As previously described, first-frame dimming or other luminance changes as a result of changes in refresh rate from a current refresh rate to a next refresh rate also may be a function of the current luminance level. Thus, luminance reduction factors 900 or luminance boost factors 904 may be generated for a number of different current luminance levels to more accurately render luminance compensation factors. In other words, it may be desirable to generate multiple luminance compensation tables to store luminance compensation factors for changes in refresh rate for each of a plurality of current luminance levels.

[0054] Referring to FIG 10, compensation data 1000 for luminance changes as a result of changes in refresh rates may include a three-dimensional luminance compensation table (LCT) or, equivalently, a series of luminance compensation tables 1002, 1004, 1006, 1008, 1010, 1012, 1014, 1016, and 1018. In the example of FIG. 10, the compensation data 1000 includes the luminance compensation tables 1002, 1004, 1006, 1008, 1010, 1012, 1014, 1016. and 1018 for current luminance levels of 10% OPR 1020, 20% OPR 1022, 30% OPR 1024, 40% OPR 1026, 50% OPR 1028, 60% OPR 1030, 70% OPR 1032, 80% OPR 1034, and 90% OPR 1036. Thus, in use, the luminance compensator 102 may access the luminance compensation table 1002, 1004, 1006, 1008, 1010, 1012, 1014. 1016. or 1018 corresponding to a current luminance level and then access a pertinent row and column of that table to retrieve an applicable luminance reduction factor or luminance boost factor. However, it will be appreciated that the dimensionality of the compensation data 1000 may be allocated and/or accessed according to any of the represented values, whether current luminance level, current refresh level, or next refresh level as described with reference to FIGS. 8 and 9. Thus, for example, the compensation data 1000 may first be accessed according to a current refresh rate or a next refresh rate. The compensation data 1000 may be used as the store of compensation data 104 accessed by the luminance compensator 102 of FIG. 1.

[0055] It will be appreciated that the represented values of the luminance compensation tables 826 and 902 (FIG. 9) and the compensation data 1000 (FIG. 10) use representative values of refresh rates and luminance levels for the sake of example. Greater or lesser levels of granularity of which refresh rates and luminance levels may be used without departing from the nature of the described systems and techniques. Thus, just to name one example, instead of the compensation data 1000 including data for current luminance levels at 10% OPR intervals (i.e., of 10% OPR 1020, 20% OPR 1022, 30% OPR 1024, etc.), the compensation data 1000 may include data for current luminance levels at 20% OPR intervals, 5% OPR intervals, 1% OPR intervals, etc. Further, where a current luminance level is not represented by a particular luminance compensation table a table with a closest luminance level may be used, or two luminance compensation tables accessed, and a value interpolated based on the current luminance level. Similarly, the current and next refresh rates may include any rates at which the display 126 (FIG. 1) is capable of presenting image data. Systems and techniques described herein are not restricted to using any particular set or range of values.

ADDITIONAL SYSTEM FOR ADJUSTING LUMINANCE

[0056] Considering the previous description of systems and techniques for luminance compensation, FIG. 11 shows a block diagram of another system 1100 that may be used to compensate for dimming of video data as a result of changing refresh rates. The system 1 100 is somewhat similar to the system 100 of FIG. 1 but includes additional elements to support luminance compensation. The system 1100 includes the luminance compensator 102 that uses the compensation data 104, as described with reference to FIGS. 8-11, to adjust for luminance changes resulting from changes in refresh rate, such as first-frame dimming.

[0057] As previously described, the luminance compensator 102 includes the current refresh rate input 114, the next refresh rate input 116, and the current luminance input 118, which are used by the luminance compensator 102 to access the compensation data 104 to determine a luminance reduction factor or luminance boost factor suitable to compensate for luminance reduction in the next frame of video data 108.

[0058] To provide the desired inputs 114, 116, and 118, in implementations, the system 1100 may include a refresh buffer 1102. The graphics processing unit 112 may be configured to generate a frame of video data, such as the current frame of video data 106 or the next frame of video data 108. The refresh rate may be part of the data included in the current frame of video data 106 or the next frame of video data 108. However, the graphics processing unit 112 may not be configured to present or maintain a refresh rate of a previous frame of video data. Thus, as the graphics processing unit 112 presents the next frame of video data 108, the refresh buffer 1102 may maintain a refresh rate associated with the previously presented current frame of video data 106 so that a current refresh rate of the current frame of video data 106 is available to the current refresh rate input 114 as the next frame of video data 108 and an associated next refresh rate associated with the next frame of video data 108 is presented to the next refresh input 116.

[0059] In addition, the graphics processing unit 112 may not be configured to identify a current luminance level of the current frame of video data 106. Accordingly, a luminance calculation module 1104 may receive the current frame of video data 106 from the graphics processing unit 112 and determine a luminance level of the current frame of video data 106 by, for example, generating a histogram of luminance of red, green, and blue pixels in the current frame of video data 106 or perform an OPR determination, as described with reference to FIG. 7. An output of the luminance calculation module 1104 may be presented to and maintained by a luminance buffer 1106 so that, when the luminance compensator 102 receives data about the next frame of video data 108, such as the next refresh rate at the next refresh rate input 116, the luminance level data for the current frame of video data 106.

[0060] In addition, the system 1100 may include other components used in generating video data at the display 126. For example, in addition to the graphics processing unit 112, the display interface 122, and the display driver 124, a display processing unit 1108 may receive the video data 110 from the graphics processing unit 112 and process the video data 110 for presentation to the display interface 122. The diagrams of the systems 100 and 1 100 of FIGS. 1 and 11, respectively, for adjusting for changes in luminance as a result of changes in refresh rate are not intended to exclude the incorporation and use of other components used or usable in processing video data for presentation on the display 126.

EXAMPLE METHOD OF ADJUSTING LUMINANCE

[0061] FIG. 12 illustrates an example method 1200 of compensating for changes in luminance of frames of video data as a result of changing refresh rates at a display system. At a block 1202, a luminance compensation factor is selected to compensate for a change in luminance level of a display system resulting from a change from a current refresh rate of a current frame of video data to a next refresh rate of a next frame of the video data, the luminance compensation factor selected based on a current luminance level of the current frame of video data, the current refresh rate associated with the current frame of the video data, and the next refresh rate associated with the next frame of the video data. The luminance compensation factor 120 (FIG. 1) may include a luminance reduction factor 900 or a luminance boost factor 904 (see FIG. 9), which are derived as explained with reference to FIGS. 8 and 9. The current luminance level 806 of the current frame 802 may be provided or determined by the luminance calculation module 1104 (see FIG. 11). The current luminance level 806 may be stored in the luminance buffer 1106 or otherwise provided to the current luminance level input 118. The current refresh rate 804 may be stored in the refresh buffer 1102 or otherwise provided to the current refresh rate input 114 of the luminance compensator 102. The next refresh rate 820, 822, or 824 may be provided directly to the next refresh rate input 116. Based on the inputs 114, 116, and 118, the luminance compensator 102 retrieves the applicable luminance compensation factor 120 from the compensation data 104.

[0062] At a block 1204, a luminance compensation factor is applied to the next frame of the video data to provide adjusted video data to the display system to adjust a next luminance level of the next frame of the video data to compensate for the change in the luminance level resulting from the change in the refresh rate. The luminance compensation factor 120, which again may include a luminance reduction factor 900 or a luminance boost factor 904, is presented to the display interface 122 that receives the video data 110. The display interface 122 applies the luminance compensation factor 120 to adjust a luminance level of the next frame 108 in the video data 110 in the adjusted video data 128 that is presented to the display system, which may include the display driver 124 and the display 126. As explained, application of the luminance compensation factor 120 may adjust for first-frame dimming upon a change in refresh rate and, thus, enables seamless frame transition without noticeably significant changes in dimming between frames of video data as a result of changes in refresh rate.

[0063] Unless context dictates otherwise, use herein of the word “or” may be considered use of an “inclusive or,” or a term that permits inclusion or application of one or more items that are linked by the word “or” (e.g., a phrase “A or B” may be interpreted as permitting just “A,” as permitting j ust “B,” or as permitting both “A" and “B”). Also, as used herein, a phrase referring to “at least one of’ a list of items refers to any combination of those items, including single members. For instance, “at least one of a, b, or c” can cover a, b, c, a-b, a-c, b-c, and a-b-c, as well as any combination with multiples of the same element (e.g., a-a, a- a- a, a-a-b, a-a-c, a-b-b, a-c-c, b-b, b-b-b, b-b-c, c-c, and c-c-c, or any other ordering of a, b, and c). Further, items represented in the accompanying figures and terms discussed herein may be indicative of one or more items or terms, and thus reference may be made interchangeably to single or plural forms of the items and terms in this written description.

ADDITIONAL EXAMPLES

[0064] In the following section, additional examples are provided.

[0065] Example 1 : A method comprising selecting a luminance compensation factor to compensate for a change in luminance level of a display system resulting from a change from a current refresh rate of a current frame of video data to a next refresh rate of a next frame of the video data, the luminance compensation factor selected based on: a current luminance level of the current frame of video data; the current refresh rate associated with the current frame of the video data; and the next refresh rate associated with the next frame of the video data; and applying the luminance compensation factor to the next frame of the video data to provide adjusted video data to the display system to adjust a next luminance level of the next frame of the video data to compensate for the change in the luminance level resulting from the change in the refresh rate.

[0066] Example 2: The method of example 1, w erein selecting the luminance compensation factor includes retrieving one of a plurality⁷ of previously determined luminance compensation factors from a store of compensation data. [0067] Example 3: The method of example 2, wherein each of the plurality of previously determined compensation factors is empirically determined by determining a change from an initial luminance level of a frame of evaluation video data displayable at an evaluation display system as the current refresh rate is changed to the next refresh rate.

[0068] Example 4: The method of example 3, wherein the evaluation display system is operationally equivalent to the display system used to display the current frame of the video data and the next frame of the video data.

[0069] Example 5: The method of example 4, wherein the display system includes an organic light-emitting diode (OLED) display subject to first-frame dimming responsive to a reduction in refresh rate between displaying the current frame of the video data and the next frame of the video data.

[0070] Example 6: The method of any one of examples 2 to 5, wherein the store of compensation data includes a plurality of luminance compensation factors, each of the plurality of luminance compensation factors selected to adjust the next luminance level of the next frame of the video data to compensate for a change from one of a plurality of current refresh rates to one of a plurality of next refresh rates.

[0071] Example 7: The method of example 6, wherein each of the plurality of luminance compensation factors is determined to adjust the next luminance of the next frame of the video data to compensate for the change from one of the plurality⁷ of current refresh rates to one of the plurality of next refresh rates for a particular current refresh rate.

[0072] Example 8: The method of example 6 or 7, wherein the store of compensation data includes one or more luminance compensation tables associating each of the plurality of luminance compensation factors with a combination of one of the plurality of cunent refresh rates and one of the plurality' of next refresh rates.

[0073] Example 9: The method of any one of examples 6 to 8, wherein each of the plurality' of luminance compensation factors includes one of: a luminance reduction factor representing a ratio of the next luminance level to the current luminance level resulting from the change from the current refresh rate of the current frame of the video data to the next refresh rate of the next frame of the video data; and a luminance boost factor representing an inverse of the luminance reduction factor.

[0074] Example 10: The method of any one of examples 6 to 9, further comprising creating the store of compensation data by: accessing representative sets of video data, each of the sets of video data including a first frame display able at a first refresh rate and a second frame displayable at a second refresh rate; determining each of the plurality of luminance compensation factors for each of the representative sets of video data based on a difference between a second luminance level of the second frame and a first luminance level of the first frame; and storing each of the plurality of luminance compensation factors in the store of luminance data associated with a combination of the first refresh rate and the second refresh rate.

[0075] Example 11 : The method of any preceding example, wherein the first luminance level and the second luminance level are empirically determined from the preexisting frames by: determining an on-pixel ratio for each of the representative sets of video data; or generating a histogram of luminance values for each of the representative sets of video data.

[0076] Example 12: The method of any preceding example, further comprising further comprising maintaining in one or more buffers the current refresh rate and the current luminance level to facilitate being able to use the current refresh rate and the current luminance level in selecting the luminance compensation factor to be applied to the next frame of the video data.

[0077] Example 13: The method of example 1, further comprising adjusting the next luminance level of the next frame of the video data by applying the luminance compensation factor to the next luminance level at a display driver of the display system.

[0078] Example 14: A system comprising means for performing a method of any one of examples 1 through 13.

[0079] Example 15: A program for causing a computer to execute the method recited in any one of examples 1 through 13.

CONCLUSION

[0080] Although implementations of systems and techniques for adjusting luminance responsive to a changing refresh rate have been described in language specific to certain features and/or methods, the subject of the appended claims is not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed as example implementations of the described systems and techniques.

Claims

CLAIMS What is claimed is:

1. A method comprising: selecting a luminance compensation factor to compensate for a change in a luminance level of a display system, the change in the luminance level of the display system resulting from a change from a current refresh rate of a current frame of video data to a next refresh rate of a next frame of the video data, the luminance compensation factor selected based on: the current refresh rate associated with the current frame of the video data; and the next refresh rate associated wi th the next frame of the video data: and applying the luminance compensation factor to the next frame of the video data to provide adjusted video data to the display system to adjust a next luminance level of the next frame of the video data effective to compensate for the change in the luminance level resulting from the change in the refresh rate.

2. The method of claim 1, wherein selecting the luminance compensation factor includes retrieving one of a plurality of previously determined luminance compensation factors from a store of compensation data.

3. The method of claim 2, wherein each of the plurality of previously determined luminance compensation factors is determined based on a change from an initial luminance level of a frame of evaluation video data displayable at an evaluation display system as the current refresh rate is changed to the next refresh rate.

4. The method of claim 3, wherein the evaluation display system is operationally equivalent to the display system used to display the current frame of the video data and the next frame of the video data.

5. The method of claim 4, wherein the display system includes an organic lightemitting diode (OLED) display subject to first-frame dimming responsive to a reduction in refresh rate between displaying the current frame of the video data and the next frame of the video data.

6. The method of any one of claims 2 to 5, wherein the store of compensation data includes a plurality of luminance compensation factors, each of the plurality of luminance compensation factors selected to adjust the next luminance level of the next frame of the video data to compensate for a change from one of a plurality of current refresh rates to one of a plurality’ of next refresh rates.

7. The method of claim 6, wherein each of the plurality of luminance compensation factors is determined to adjust the next luminance level of the next frame of the video data to compensate for the change from the one of the plurality of current refresh rates to one of the plurality of next refresh rates for a particular current luminance level.

8. The method of claim 6 or 7, wherein the store of compensation data includes one or more luminance compensation tables associating each of the plurality of luminance compensation factors with a combination of one of the plurality of current refresh rates and one of the plurality of next refresh rates.

9. The method of any one of claims 6 to 8, wherein each of the plurality of luminance compensation factors includes one of: a luminance reduction factor representing a ratio of a next luminance level to a current luminance level; and a luminance boost factor representing an inverse of the luminance reduction factor.

10. The method of any one of claims 6 to 9, further comprising creating the store of compensation data by: accessing representative sets of video data, each of the sets of video data including a first frame display able at a first refresh rate and a second frame displayable at a second refresh rate; determining each of the plurality of luminance compensation factors for each of the representative sets of video data based on a difference between a second luminance level of the second frame and a first luminance level of the first frame; and storing each of the plurality of luminance compensation factors in the store of compensation data associated with a combination of the first refresh rate and the second refresh rate.

11. The method of claim 10, wherein the first luminance level and the second luminance level are determined from the first frame and the second frame by: determining an on-pixel ratio for each of the representative sets of video data; or generating a histogram of luminance values for each of the representative sets of video data.

12. The method of any preceding claim, further comprising maintaining in one or more buffers the current refresh rate and a current luminance level to use the current refresh rate and the current luminance level in selecting the luminance compensation factor to be applied to the next frame of the video data.

13. The method of any preceding claim, further comprising adjusting the next luminance level of the next frame of the video data by applying the luminance compensation factor to the next luminance level at a display driver of the display system.

14. A system comprising means for performing a method of any one of claims 1 through 13.

15. A program for causing a computer to execute the method recited in any one of claims 1 through 13.