CN112581914B

CN112581914B - Image processing method and device

Info

Publication number: CN112581914B
Application number: CN201910937215.9A
Authority: CN
Inventors: 刘军; 杨梅; 许景翔; 刘洋; 牛泽宇
Original assignee: Shanghai Haisi Technology Co ltd
Current assignee: Haisi Technology Co ltd
Priority date: 2019-09-29
Filing date: 2019-09-29
Publication date: 2022-03-29
Anticipated expiration: 2039-09-29
Also published as: CN112581914A

Abstract

The application provides an image processing method and device. The method comprises the following steps: converting an image in a first format into an image in a second format, wherein the image in the first format comprises a plurality of first pixels, the image in the second format comprises a plurality of second pixels, the first pixels comprise three sub-pixels with different colors, and the second pixels comprise four sub-pixels with different colors; determining a frame level gain for an image in a first format; respectively determining the pixel level gain corresponding to each first pixel in the image in the first format; and determining the target gain corresponding to each second pixel according to the frame level gain and the pixel level gain corresponding to each first pixel, wherein the target gain is used for controlling the backlight brightness of the backlight module, so that the power consumption of the backlight module can be reduced to a greater extent.

Description

Image processing method and device

Technical Field

The present application relates to the field of image processing, and in particular, to an image processing method and apparatus.

Background

At present, Red Green Blue White (RGBW) panels can be adopted on devices such as televisions, mobile phones and the like. The RGBW panel has an advantage in that light transmittance of the display panel is increased due to an increase of white (W) sub-pixels, compared to a conventional Red Green Blue (RGB) panel. Therefore, the display luminance of the RGBW panel is higher than that of the RGB panel at the same backlight luminance. Under the condition of the same display brightness, the backlight brightness can be reduced by reducing the duty ratio of the backlight Pulse Width Modulation (PWM) of the backlight module, so as to achieve the purpose of reducing power consumption.

In the prior art, a frame-level gain is usually obtained according to input RGB data, and the frame-level gain is adopted to adjust the duty ratio of the backlight PWM, thereby reducing the power consumption of the backlight module. However, the effect of reducing the power consumption of the backlight module by using the frame-level gain is not ideal.

Disclosure of Invention

The application discloses an image processing method and device, which can provide a target gain to adjust the duty ratio of backlight PWM and can reduce the power consumption of a backlight module to a greater extent.

In a first aspect, an embodiment of the present application provides an image processing method, where the method includes: converting an image in a first format into an image in a second format, wherein the image in the first format comprises a plurality of first pixels, the image in the second format comprises a plurality of second pixels, the first pixels comprise three sub-pixels with different colors, and the second pixels comprise four sub-pixels with different colors; determining a frame level gain for the image in the first format; respectively determining the pixel level gain corresponding to each first pixel in the image with the first format; and determining a target gain corresponding to each second pixel according to the frame level gain and the pixel level gain corresponding to each first pixel, wherein the target gain is used for controlling the backlight brightness of the backlight module.

The embodiment of the application can fuse the frame level gain and the pixel level gain to obtain a target gain. The target gain can be used for controlling the backlight brightness of the backlight module, the target gain is combined with the frame level gain and the pixel level gain, all factors after RGB data is converted into RGBW data can be comprehensively considered by controlling the backlight brightness of the backlight module through the target gain, the backlight brightness of the backlight module is comprehensively calculated based on the display principle of the liquid crystal panel, and compared with the backlight brightness of the backlight module which is only controlled through the frame level gain, the target gain control method and the target gain control device are more consistent with the display principle of the liquid crystal panel, the brightness control is more accurate, the power consumption of the backlight module is lower, and the power consumption performance of the image processing device is better.

In a possible implementation manner, the target gain is further used to amplify a luminance value of each sub-pixel in the second pixel corresponding to the target gain, and after the determining the target gain corresponding to each second pixel according to the frame-level gain and the pixel-level gain corresponding to each first pixel, the method further includes: and amplifying the brightness value of each sub-pixel in the second pixel corresponding to the target gain according to the target gain.

In the embodiment of the present application, the target gain may also be used to amplify the luminance value of each sub-pixel in the second pixel. Since the frame-level gain can increase the brightness of saturated colors in an image, the pixel-level gain can increase the brightness of unsaturated colors in the image, and the target gain combines the frame-level gain and the pixel-level gain, the target gain can increase the brightness of both saturated colors and unsaturated colors, thereby increasing the contrast. Therefore, the brightness of the saturated color can be increased by fusing two gains, the brightness of the unsaturated color and the contrast of the whole picture are increased, and the display performance of the image processing device is improved. .

In a possible implementation manner, the determining a target gain corresponding to each second pixel according to the frame-level gain and the pixel-level gain corresponding to each first pixel includes: determining a second mapping relation table according to a first mapping relation table and the frame level gain, wherein the first mapping relation table is used for representing an initial corresponding relation between the saturation of a plurality of first pixels and the pixel level gain, and the second mapping relation table is a second corresponding relation between the saturation adjusted according to the frame level gain and the pixel level gain; and determining a target gain corresponding to the second pixel according to the saturation of the first pixel, the second mapping relation table and the frame level gain.

In a possible implementation manner, the determining the second mapping table according to the first mapping table and the frame level gain includes:

PixelGainLUT＝(2-FrameGain)×(PixelGainLUT0-1)+1

wherein, FrameGain is the frame level gain, pixeganllut is the second mapping table, and pixeganllut is the first mapping table.

In the embodiment of the present application, the second mapping relation table may be changed along with the change of the frame level gain. Specifically, when the frame level gain becomes larger, the pixel level gain corresponding to the second pixel with the same saturation level is reduced, so that the value of the target gain can be controlled, and the second pixel amplified by the target gain is prevented from overflowing the RGBW color space.

In one possible implementation manner, the determining the target gain corresponding to the second pixel according to the saturation of the first pixel, the second mapping table, and the frame-level gain includes:

FinalGain＝FrameGain+α×(PixelGainLUT(Saturation)-1)

wherein α is a control coefficient, α ∈ [0,1], Saturation is the Saturation of the first pixel, FinalGain is the target gain, and pixeganllut is the second mapping table.

In one possible implementation manner, the image processing apparatus includes a plurality of display modes, the control coefficients corresponding to different display modes are different, and the requirements of the image processing apparatus on power consumption in different display modes are different.

Specifically, the display mode may include a conservative mode, a medium mode, and an aggressive mode. In the conservative mode, the image processing apparatus focuses more on the quality of the displayed image without pursuing power consumption benefits, and the control coefficient is low at this time. In the aggressive mode, the image processing apparatus pays more attention to the power consumption yield without pursuing the quality of the displayed image, and the control coefficient is high at this time. The medium mode is between the conservative and aggressive modes.

According to the embodiment of the application, the control coefficient can be adjusted through different display modes, so that the image processing device can achieve the best power saving effect in various modes.

In a possible implementation manner, after the amplifying the luminance value of each sub-pixel in the second pixel corresponding to the target gain according to the target gain, the method further includes: determining the duty ratio of the backlight Pulse Width Modulation (PWM) of the backlight module according to the target gain; and controlling the backlight brightness of the backlight module according to the duty ratio.

The embodiment of the application can determine the duty ratio of the backlight PWM of the backlight module through the target gain, control the backlight brightness of the backlight module according to the duty ratio, comprehensively consider all factors after RGB data is converted into RGBW data, comprehensively calculate the power saving proportion of the RGBW panel based on the display principle of the liquid crystal panel, and compare with the mode that only frame level gain is considered, so that the brightness control is more accurate, the power consumption of the backlight module is reduced, and the power consumption performance of the image processing device is improved.

In a possible implementation manner, the determining the duty ratio of the backlight PWM of the backlight module according to the target gain includes: determining a power saving coefficient gamma of the second pixel according to the target gain; determining a third mapping relation table according to the gamma, wherein the third mapping relation table is used for representing the corresponding relation between different brightness level indication values and the duty ratio of the backlight PWM; and searching the third mapping relation table, and determining the duty ratio of the backlight PWM corresponding to the current brightness level indication value.

In the case where the RGBW panel and the RGB panel display the same luminance, the ratio of the backlight value of the RGBW panel to the backlight value of the RGB panel may be referred to as a power saving coefficient. The larger the power saving coefficient is, the more power consumption is saved indicating that the backlight luminance of the RGBW panel is more than that of the RGB panel. Since the image processing apparatus requires different power consumption in different display modes, the power saving coefficients in different display modes are different for the same image. The larger the power saving coefficient is, the lower the power consumption of the backlight module of the image processing device is, and the smaller the corresponding backlight PWM duty ratio is; the smaller the power saving coefficient is, the higher the power consumption of the backlight module of the image processing device is, and the larger the corresponding backlight PWM duty ratio is. Therefore, different power saving coefficients can correspond to different third mapping tables, and the PWM duty ratio determined according to the third mapping tables is most suitable for the current display mode and the power consumption of the backlight module is the lowest.

In one possible implementation manner, the first pixel includes three color-first sub-pixels, and the second pixel includes the three color-first sub-pixels and a color-second sub-pixel.

The determining the power saving coefficient γ of the second pixel according to the target gain includes:

wherein, X_rgbWo is the brightness value of any one of the three sub-pixels of the first pixel, Wo is the brightness value of the sub-pixel of the second pixel, and β is the characteristic parameter of the display panel.

Specifically, the display panel may be an RGBW panel. The three sub-pixels of one color may be a red sub-pixel, a green sub-pixel, and a blue sub-pixel. The sub-pixels of the above two types of colors may be white sub-pixels. The β may be a ratio of the luminance of the white sub-pixel to the mixed color luminance of the other sub-pixels on the RGBW panel.

In one possible implementation, the above-mentioned β ═ 1,

in a possible implementation manner, after the amplifying the luminance value of each sub-pixel in the second pixel corresponding to the target gain according to the target gain, the method further includes: and performing soft peak clipping processing on the second pixel after the brightness value of the sub-pixel is amplified.

In the embodiment of the application, the amplified pixels are subjected to soft peak clipping, so that highlight details caused by hard peak clipping can be prevented from being lost, and the color cast problem caused by saturation of a certain sub-pixel after the second pixel is amplified can be avoided.

In one possible implementation manner, the length of the second pixel luminance value is a first bit width, the length of the amplified second pixel luminance value of the sub-pixel is a second bit width, the length of the soft-peak-clipping second pixel luminance value is the first bit width, and the first bit width is smaller than the second bit width.

The embodiment of the application can reserve the details of brightness value overflow of partial sub-pixels by expanding the bit width of the brightness value of each sub-pixel of the amplified second pixel.

In one possible implementation form of the method,

wherein, X_rgbThe luminance value of any one of the three sub-pixels of the first color of the first pixel, Xsoft is the luminance value of any one of the three sub-pixels of the first color of the second pixel after soft peak clipping, Wsoft is the luminance value of the two sub-pixels of the second color of the second pixel after soft peak clipping, and β is the characteristic parameter of the display panel.

The embodiment of the present application provides another way to calculate the power saving coefficient, and specifically, the power saving coefficient may be calculated by using data after soft peak clipping. After the luminance value of the white sub-pixel is equivalent to the luminance value of the sub-pixel of any one color, the ratio of the luminance value of the sub-pixel of the color in the first pixel to the equivalent luminance value of the color in the second pixel is calculated, and the ratio is the power saving coefficient.

In a second aspect, an embodiment of the present application provides an image processing apparatus, including: an image format conversion module, configured to convert an image in a first format into an image in a second format, where the image in the first format includes a plurality of first pixels, the image in the second format includes a plurality of second pixels, the first pixels include three sub-pixels with different colors, and the second pixels include four sub-pixels with different colors; a frame level gain determining module for determining a frame level gain of the image in the first format; a pixel level gain determining module, configured to determine a pixel level gain corresponding to each first pixel in the image in the first format; and the fusion module is used for determining the target gain corresponding to each second pixel according to the frame level gain and the pixel level gain corresponding to each first pixel, and the target gain is used for controlling the backlight brightness of the backlight module.

In a possible implementation manner, the target gain is further used to amplify a luminance value of each sub-pixel in the second pixel corresponding to the target gain, and the apparatus further includes: and the amplifying module is used for amplifying the brightness value of each sub-pixel in the second pixel corresponding to the target gain according to the target gain.

In a possible implementation manner, the fusion module is specifically configured to: determining a second mapping relation table according to a first mapping relation table and the frame level gain, wherein the first mapping relation table is used for representing an initial corresponding relation between the saturation of a plurality of first pixels and the pixel level gain, and the second mapping relation table is a second corresponding relation between the saturation adjusted according to the frame level gain and the pixel level gain; and determining a target gain corresponding to the second pixel according to the saturation of the first pixel, the second mapping relation table and the frame level gain.

In a possible implementation manner, the second mapping relationship table is:

PixelGainLUT＝(2-FrameGain)×(PixelGainLUT0-1)+1

In one possible implementation, the target gain is:

FinalGain＝FrameGain+α×(PixelGainLUT(saturation)-1)

In a possible implementation manner, the apparatus includes a plurality of display modes, the control coefficients corresponding to different display modes are different, and the apparatus has different requirements on power consumption in different display modes.

In a possible implementation manner, the apparatus further includes: the duty ratio calculation module is used for determining the duty ratio of the backlight pulse width modulation PWM of the backlight module according to the target gain; and the backlight brightness control module is used for controlling the backlight brightness of the backlight module according to the duty ratio.

In a possible implementation manner, the duty ratio calculating module is specifically configured to: determining a power saving coefficient gamma of the second pixel according to the target gain; determining a third mapping relation table according to the gamma, wherein the third mapping relation table is used for representing the corresponding relation between different brightness level indication values and the duty ratio of the backlight PWM; and searching the third mapping relation table, and determining the duty ratio of the backlight PWM corresponding to the current brightness level indication value.

In one possible implementation manner, the first pixel includes three first-color sub-pixels, the second pixel includes three first-color sub-pixels and a second-color sub-pixel, and the power saving coefficient is:

In one possible implementation, the above-mentioned β ═ 1,

in a possible implementation manner, the apparatus further includes: and a soft peak clipping module, configured to perform soft peak clipping on the second pixel after the amplification module amplifies the luminance value of each sub-pixel in the second pixel corresponding to the target gain according to the target gain.

In one possible implementation form of the method,

In a third aspect, an embodiment of the present application provides an image processing apparatus, including a processor and a transmission interface; the transmission interface is used for receiving or transmitting image data; the processor is configured to invoke software instructions stored in the memory, so that the processor executes a method provided by the first aspect of the embodiments of the present application or any implementation manner of the first aspect.

In a fourth aspect, an embodiment of the present application provides a terminal, including a processor, a memory, a display panel, and a backlight module; wherein: the memory is configured to store a computer program, the computer program comprising program instructions; the processor is configured to invoke the program instruction, so that the terminal executes the method provided by the first aspect of the present application or any implementation manner of the first aspect.

In a fifth aspect, the present application provides a computer-readable storage medium, where instructions are stored, and when the instructions are executed on a computer, the computer is caused to execute the method provided by the first aspect or any one implementation manner of the first aspect.

In a sixth aspect, the present application provides a computer program product, which when run on a computer, causes the computer to execute the method provided in the first aspect or any one of the implementation manners of the first aspect.

It is to be understood that the image processing apparatus provided by the second aspect, the image processing apparatus provided by the third aspect, the terminal provided by the fourth aspect, the computer storage medium provided by the fifth aspect, and the computer program product provided by the sixth aspect are all configured to execute the image processing method provided by the first aspect. Therefore, the beneficial effects achieved by the method can refer to the beneficial effects in the image processing method provided by the first aspect, and are not described herein again.

Drawings

The drawings used in the embodiments of the present application are described below.

FIG. 1A is a schematic view of the light-emitting principle of an image processing apparatus;

FIG. 1B is a schematic diagram of a display system of a mobile phone;

FIG. 1C is a diagram illustrating a brightness level indication on a mobile phone;

FIG. 1D is a schematic diagram of a television display system;

FIG. 2 is a schematic flowchart of an image processing method provided in an embodiment of the present application;

FIG. 3 is a diagram illustrating a comparison between an RGB color space and an RGBW color space after format conversion according to an embodiment of the present application;

FIG. 4 is a cumulative histogram of 32 th order luminance values;

FIG. 5 is a representation of frame level gain lookup provided by an embodiment of the present application;

FIG. 6 is a comparison of RGB color space and RGBW color space after frame level gain amplification;

FIG. 7 is a diagram illustrating an initial correspondence between pixel level gain and saturation;

fig. 8 is a diagram illustrating a second corresponding relationship between gain and saturation of a pixel stage according to an embodiment of the present disclosure;

FIG. 9 is a comparison of RGB color space and RGBW color space after target gain amplification;

fig. 10 is a schematic diagram of a soft-clipping curve provided in an embodiment of the present application;

fig. 11 is a schematic diagram illustrating a mapping relationship between a luminance level indication value and a power saving coefficient mean value of a display panel and a backlight PWM duty ratio according to an embodiment of the present disclosure;

fig. 12 is a schematic structural diagram of an image apparatus according to an embodiment of the present disclosure;

fig. 13 is a schematic structural diagram of another image device according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.

The terms "first," "second," "third," and the like in the description and claims of this application and in the above-described drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.

The light emission principle of the image processing apparatus will be described with reference to fig. 1A.

As shown in fig. 1A, the image processing apparatus 10 may include a backlight module 110 and a display panel 120. The backlight module 110 may be used to provide the display panel 120 with sufficient brightness and uniformly distributed light sources, so that the display panel 120 can normally display images. The backlight module 110 may be controlled by a pulse signal, and for example, the backlight Brightness (BL) of the backlight module 110 may be adjusted by Pulse Width Modulation (PWM). Specifically, the backlight brightness is adjusted by adjusting the duty ratio of the PWM, and it should be understood that the duty ratio of the PWM is the proportion of the high level duration in one pulse period to one level period. Alternatively, the duty cycle of the PWM may also be referred to as a PWM value. The larger the duty ratio, the higher the backlight luminance.

The display panel 120 according to the embodiment of the present application may be a Liquid Crystal Display (LCD) panel. The LCD is a passive light emitting device, which does not emit light, and the display function can be achieved by the backlight module 110. Specifically, the light transmittance of each pixel can be adjusted to allow the light source emitted from the backlight module 110 to pass through each pixel, so that each pixel presents different colors, and finally presents an image. The display panel referred to in the embodiments of the present application is an RGBW panel.

RGBW panel the transmittance of the panel increases due to the addition of W sub-pixels. Therefore, under the condition of displaying the same display brightness as the RGB panel, the purpose of reducing the power consumption can be achieved by reducing the backlight PWM of the RGBW panel.

Currently, RGBW panels are used in image processing apparatuses such as televisions and mobile phones. For an RGBW panel, an RGBW algorithm is required to convert RGB format data into RGBW format data.

Fig. 1B schematically shows a structure of a display system of a mobile phone. As shown in fig. 1B, the display system of the mobile phone may include: an Application Processor (AP), a Display Driver Integrated Circuit (DDIC), a display panel, and a backlight module. For example, the backlight module may include a backlight driving circuit (IC) and a backlight. The backlight driving IC and the backlight form a loop, and the backlight driving IC is used for controlling the backlight to provide light sources for the display panel. The AP or DDIC may be used to place some display related algorithms of the display device, such as brightness adjustment algorithms, image format conversion algorithms, backlight brightness adjustment algorithms, contrast adjustment, color gamut adjustment, saturation adjustment, and the like. In combination with the luminance level indication value (luminance) and the display mode (such as conservative, medium, and aggressive) obtained by the Application Processor (AP), the DDIC or AP may complete the conversion from the RGB format to the RGBW format, and simultaneously control the duty ratio of the backlight PWM of the RGBW panel. Optionally, the brightness level indication value may be set by a user through a display interface and transmitted to the AP, and the display mode may be set by the user, or may be pre-stored in a memory or a storage unit of the AP.

The brightness level indicated value is the value of the brightness bar on the mobile phone. As shown in FIG. 1C, FIG. 1C illustrates an exemplary phone's page of a drop-down notification bar. Various switch controls (including but not limited to controls for turning WLAN on or off, controls for turning bluetooth on or off, controls for turning flashlight on or off, controls for turning ring mode off, controls for turning screen auto-rotation on or off, controls for turning huawei share function on or off, controls for turning flight mode on or off, controls for turning movement data on or off, controls for turning position information on or off, controls for turning screen shot on, etc.), intensity bars, and notification message lists may be included in the page. The notification message list 310 may include the latest and unread notification messages sent by each APP. The luminance bar may be used to adjust the luminance level indication value. The user can adjust the brightness level indicated value by sliding or clicking the brightness bar, so that the mobile phone displays pictures with different brightness. For the same picture, the larger the brightness level indication value is, the brighter the whole picture displayed by the mobile phone is; the smaller the brightness level indication value is, the darker the whole picture displayed by the mobile phone is. The AP may obtain the current brightness level indication value of the mobile phone.

Fig. 1D schematically shows a structure of a television display system. As shown in fig. 1D, the television display system may include: a system on a chip (SOC), a Timing Control (TCON) chip, a display panel, and a backlight module. Illustratively, the backlight module may include a backlight driving IC and a backlight. The backlight driving IC and the backlight form a loop, and the backlight driving IC is used for controlling the backlight to provide light sources for the display panel. The SOC or TCON chip may be used to place some display related algorithms of the display device, such as brightness adjustment algorithm, image format conversion algorithm, backlight brightness adjustment algorithm, contrast adjustment, color gamut adjustment, saturation adjustment, and the like. The television can input corresponding luminance through the ambient light sensor to complete control similar to the mobile phone.

In the following embodiments of the present application, a mobile phone is taken as an example for explanation. And the image of the first format mentioned in the following embodiments may be an image of RGB format, and the image of the second format may be an image of RGBW format. The first pixel included in the first format may be an RGB pixel, and the three different color sub-pixels included in the first pixel may be a red (R) sub-pixel, a green (G) sub-pixel, and a blue (B) sub-pixel. The second pixel included in the second format may be an RGBW pixel, and the four different color sub-pixels included in the second pixel may be an R sub-pixel, a G sub-pixel, a B sub-pixel, and a W sub-pixel. The R sub-pixel, the G sub-pixel and the B sub-pixel are first-class sub-pixels, and the W sub-pixel is a second-class sub-pixel.

An image processing method provided by the embodiment of the present application is described next with reference to fig. 2.

Fig. 2 schematically shows a flow chart of the image processing method. As shown in fig. 2, the image processing method may include at least the following steps:

s201: the image in the first format is converted to an image in a second format.

Specifically, the image in the first format is an image in an RGB format, and the image in the second format is an image in an RGBW format.

Specifically, the input data of the image processing apparatus may be RGB nonlinear data. The RGB non-linear data may also be converted to RGB linear data before converting the image in the first format to the image in the second format. In particular, a lookup table (LUT) may be used for implementation. The typical mapping curve of the LUT is Gamma 2.2. The input nonlinear data bit width may be, for example, 10 bits (bits), and the output linear data bit width may be, for example, 14 bits.

If the input nonlinear data is R'_rgbG′_rgbB′_rgbLinear data being R_rgbG_rgbB_rgb. Then R is_rgb＝R′_rgb ^2.2，G_rgb＝G′_rgb ^2.2，B_rgb＝B′_rgb ^2.2。

The input RGB data before format conversion is linear data R_rgbG_rgbB_rgb. The format-converted output RGBW data is RoGoBoWo. The calculation process of format conversion may be as follows:

wherein R is_rgb，G_rgb，B_rgbFor inputting the RGB data, Ro, Go, Bo, Wo are outputted RGBW data, and w _ a, r _ a, g _ a, b _ a, r _ a _ all, g _ a _ all, b _ a _ all are control coefficients. These control coefficients may be stored, for example, in control registers, with typical values of 1, Min (R)_rgb,G_rgb,B_rgb) Is the smallest sub-pixel luminance value among the luminance values of the three sub-pixels of the first pixel.

Fig. 3 exemplarily shows a comparison diagram of the a point after format conversion between the RGB color space and the RGBW color space. Fig. 3 shows the projection of the RGB color space on the RG plane and the projection of the RGBW color space on the RG plane. The point a is any point in the RGB color space. The point of the a point in the RGBW color space after the format conversion becomes a'. As can be seen from fig. 3, a coincides with a'. That is, the RGBW color space after format conversion has unchanged luminance and unchanged chromaticity compared to the RGB color space.

S202: a frame level gain for the image in the first format is determined.

Specifically, the frame level gain is calculated as follows:

first, the maximum luminance values Max (Ro, Go, Bo, Wo) and Max (Ro, Go, Bo, Wo) of the respective second pixels of the format-converted image are counted as the maximum luminance value of the sub-pixels among the luminance values of the 4 sub-pixels in the second pixels, and the number of the second pixels whose maximum luminance values are the luminances of each step is counted. Assuming that the luminance values of the second pixels are of 32 levels, the number of the second pixels with the maximum luminance values of 1-32 is counted respectively. For example, the number of second pixels having a maximum luminance value of 1 is 145, the number of second pixels having a maximum luminance value of 2 is 98, the number of second pixels having a maximum luminance value of 3 is 72, the number of second pixels having a maximum luminance value of N is M, the number of second pixels having a maximum luminance value of 31 is 11, and the number of second pixels having a maximum luminance value of 32 is 5.

Next, a cumulative histogram of the luminances of each step is calculated.

Specifically, a cumulative histogram of the luminance of each order is generated in order of the luminance values from high to low. The cumulative histogram may show the number of second pixels having a maximum luminance value greater than or equal to a luminance value.

Fig. 4 exemplarily shows a cumulative histogram of 32-step luminance. The abscissa of the cumulative histogram is the maximum luminance value, and the ordinate is the cumulative pixel number of the second pixel corresponding to the maximum luminance value. As shown in fig. 4, the cumulative pixel number of the second pixels having the maximum luminance value of 32 or more is 5, the cumulative pixel number of the second pixels having the maximum luminance value of 31 or more (i.e., the maximum luminance value of 31 or 32) is 16, the cumulative pixel number of the second pixels having the maximum luminance value of 30 or more (i.e., the maximum luminance value of 30 or 31 or 32) is 29, the cumulative pixel number of the second pixels having the maximum luminance value of 29 or more is 42, the cumulative pixel number of the second pixels having the maximum luminance value of 1 or more is 1016, and the like.

And thirdly, calculating an overflow value, searching the cumulative histogram according to the overflow value, and determining the corresponding maximum brightness value.

Specifically, the overflow value is the number of pixels that allow the luminance value to overflow. For a display panel with a resolution of X Y, the overflow percentage is assumed to be z%, i.e. the overflow value is z% xy. After the overflow value is determined, the overflow value can be used to search the cumulative histogram, and the maximum brightness value corresponding to the overflow value is determined.

For example, if the overflow value is 60, after looking up the cumulative histogram shown in fig. 4, the cumulative pixel number closest to the overflow value 60 is 68 or 55, and the maximum luminance value 28 corresponding to the cumulative pixel number 55 smaller than the overflow value 60 is taken as the maximum luminance value corresponding to the overflow value. That is, the maximum brightness value corresponding to the accumulated number of pixels that is less than the overflow value and closest to the overflow value is the maximum brightness value corresponding to the overflow value.

And finally, searching a frame level gain lookup table according to the determined maximum brightness value, and determining the frame level gain corresponding to the brightness value.

Fig. 5 illustrates a frame level gain look-up table. As shown in fig. 5, the frame level gain lookup table may be in the form of a curve with luminance values on the abscissa and frame level gains on the ordinate. And after the maximum brightness value is determined, searching a frame level gain lookup table according to the maximum brightness value, and determining that the frame level gain corresponding to the brightness value is the frame level gain corresponding to the picture in the second format.

The expression of the frame level gain lookup table may be, but is not limited to, as follows:

FrameGainLUT(index)＝MAX(ROUND(2048×255/index)，4095) (2)

wherein, framegainflut (index) is a frame level gain lookup table, index is a luminance value, ROUND represents rounding, MAX represents a maximum value, the normalization coefficient is 2048, that is, 2048 represents 1, and 4095 represents 2.

Without being limited to the curves illustrated in fig. 5, the frame-level gain lookup table may also be in the form of a table in a specific implementation, and the embodiment of the present application is not limited thereto.

It is understood that the overflow percentage z% may be determined by the display mode. The application processor may determine a display mode according to the usage scenario. The display mode may include a conservative mode, a medium mode and an aggressive mode.

For example, the display mode may be determined to be conservative in the context of viewing a picture. In the conservative mode, the image processing apparatus focuses more on the quality of the displayed image without pursuing power consumption benefits, and the overflow percentage is lower. The display mode may be determined to be an aggressive mode in a game scene. In the aggressive mode, the image processing apparatus pays more attention to the power consumption yield without pursuing the quality of the displayed image, and the overflow percentage is higher at this time. The medium mode may be between the conservative and aggressive modes.

If the frame-level gain is used to amplify the points a and B in the RGB color space, the performance in the RGBW color space is shown in fig. 6. In the RGBW color space, point A is stretched to point A 'and point B is stretched to point B', and the stretching ratios of the two are the same.

S203: and respectively determining the pixel-level gain corresponding to each first pixel in the image in the first format.

Specifically, the calculation process of the pixel level gain may be as follows:

first, the Saturation (Saturation) of the first pixel is calculated. The calculation formula of the saturation is as follows:

secondly, a pixel level gain lookup table is looked up according to the saturation of the first pixel, and the pixel level gain corresponding to the saturation is determined.

Fig. 7 illustrates an example of a pixel level gain look-up table. As shown in fig. 7, the pixel level gain lookup table may include different saturation to pixel level gain (pixegan) correspondence. Pixegan, pixeganllut 0(Saturation), wherein pixeganllut 0 is the curve shown in fig. 7.

In the embodiment of the present application, the pixel level gain lookup table may be referred to as a first mapping relation table. The first mapping table may be used to characterize an initial correspondence of saturation to pixel level gain for a plurality of first pixels. The pixel level gain lookup table is not limited to be presented in a curve form in fig. 7, and in a specific implementation, the pixel level gain lookup table may also be presented in other forms such as a table, which is not limited in this embodiment of the present application.

S204: and determining the target gain corresponding to each second pixel according to the frame level gain and the pixel level gain corresponding to each first pixel.

Specifically, the calculation formula of the target gain is as follows:

FinalGain＝FrameGain+α×(PixelGainLUT(Saturation)-1) (4)

wherein FinalGain is the target gain, FrameGain is the frame-level gain, and α is the control coefficient, and the value range of α may be [0,1] as an example. Wherein the pixegainlut changes with the dynamic change of FrameGain of the current frame image. The dynamically changing relationship is as follows:

therefore, pixegan LUT ═ 2-FrameGain) × (pixegan LUT0-1) + 1.

The pixeganllut 0 is an initial correspondence between the saturation of the first pixels and the gain of the pixel level. The pixegan LUT is a second corresponding relationship between the saturation adjusted according to the frame-level gain FrameGain and the pixel-level gain pixegan, and in this embodiment, the pixegan LUT may be referred to as a second mapping relationship table. A schematic diagram of the second mapping table may be as shown in fig. 8. The second mapping relation table presented in the form of a curve in fig. 8 is not limited, and in a specific implementation, the second mapping relation table may also be presented in other forms such as a table, which is not limited in this embodiment of the present application.

In the present application, the target gain may be used to control the backlight brightness of the backlight module. How to control the backlight brightness of the backlight module by using the target gain will be described in detail in the following embodiments, and will not be described in detail here.

In the present application, the target gain may also be used to amplify the luminance value of each sub-pixel in the second pixel corresponding to the target gain. After S204, the image processing method further includes: and amplifying the brightness value of each sub-pixel in the second pixel corresponding to the target gain according to the target gain.

Specifically, if the second pixel before enlargement is RoGoBoWo, the second pixel after enlargement is Ro 'Go' Bo 'Wo'. Wherein:

if the target gain is used to amplify the points a and B in the RGB color space, the performance in the RGBW color space is shown in fig. 9. As the target gain is fused with the frame-level gain and the pixel-level gain, the target gains of different pixels are different, the stretching ratios of the target gains to different pixel points are also different, the point A in the RGBW color space is stretched to the point A ', the point B is stretched to the point B', and the stretching ratio of the point B after the amplification of the target gain is smaller than that of the point A because the color of the point B is more saturated.

Since the frame-level gain can increase the brightness of saturated colors in an image, the pixel-level gain can increase the brightness of unsaturated colors in the image, and the target gain combines the frame-level gain and the pixel-level gain, the target gain can increase the brightness of both saturated colors and unsaturated colors, thereby increasing the contrast. Therefore, the method and the device have the advantages that the brightness of the saturated color can be increased by combining two gains, the brightness of the unsaturated color and the contrast of the whole picture are increased, and the advantages of the RGBW panel can be fully embodied.

Possibly, the length of the luminance value of the second pixel before the amplification may be a first bit width, and the length of the luminance value of the second pixel after the amplification may be a second bit width. Wherein the first bit width is less than the second bit width. The first bit width may be, for example, 14 bits and the second bit width may be, for example, 16 bits. This prevents loss of image detail due to overflow of the amplified luminance values.

In some possible embodiments, the image processing apparatus may further perform soft peak clipping on the amplified second pixel. The specific soft peak clipping process may be as follows:

1) for the input amplified RGBW data, a maximum value MaxIn of each second pixel is determined.

Specifically, the bit width of the input RGBW data may be the second bit width described above, for example, 16 bits. MaxIn is the maximum sub-pixel brightness value of the second pixel.

2) And searching a soft peak clipping mapping curve according to the MaxIn to obtain a corresponding MaxOut.

Fig. 10 illustrates an example of a soft-clipping mapping curve with MaxIn on the abscissa and MaxOut on the ordinate. After determining MaxIn, the soft clipping mapping curve may be looked up to determine MaxOut corresponding to the MaxIn. The data bit width of MaxOut is the first bit width, e.g., 14 bits.

3) And correcting the RGBW data.

The brightness value of each sub-pixel of the second pixel after soft peak clipping is calculated as follows:

wherein Rsoft is the brightness value of the red sub-pixel of the second pixel after soft peak clipping, Gsoft is the brightness value of the green sub-pixel of the second pixel after soft peak clipping, Bsoft is the brightness value of the blue sub-pixel of the second pixel after soft peak clipping, and Wsoft is the brightness value of the white sub-pixel of the second pixel after soft peak clipping. The luminance value of the white sub-pixel is not processed because the luminance value of the white sub-pixel does not overflow.

By adopting the soft peak clipping processing, the color cast problem caused by that a certain component is saturated after the RGBW data is amplified and stretched can be avoided.

In some possible embodiments, after the soft peak clipping, the image processing apparatus may further perform sub-pixel rendering (SPR) on the RGBW data, and map the RGBW data to the RGBW arrangement of the actual panel. Specifically, the high-resolution image may be displayed on a low-resolution display panel.

After the sub-pixel rendering, the image processing apparatus may further perform a non-linear conversion on the RGBW data, and specifically, may convert the linear RGBW data into the non-linear RGBW data. In particular, LUT may be used. The typical mapping curve of LUT is Gamma 1/2.2. The input linear data bit width may be, for example, 14 bits, and the output nonlinear data bit width may be, for example, 10 bits.

In some possible embodiments, after the soft peak clipping, the image processing method may further include: determining the duty ratio of backlight PWM of the backlight module according to the target gain; and controlling the backlight brightness of the backlight module according to the duty ratio.

How to control the backlight brightness of the backlight module using the target gain is described in detail below. The calculation process of the backlight PWM duty ratio is mainly described. The calculation process of the backlight PWM duty ratio is mainly divided into the following parts:

1. the power saving coefficient gamma of the second pixel is calculated.

2. And calculating the average value of the power saving coefficients of the image in the second format.

3. And determining a third mapping relation table according to the average value of the power saving coefficients, searching the third mapping relation table according to the brightness level indicating value, and determining the backlight PWM duty ratio corresponding to the brightness level indicating value.

Illustratively, the power saving coefficient γ of the second pixel is calculated as follows:

specifically, the image processing apparatus may calculate a ratio of backlight adjustment according to a ratio of equivalent luminances of the output RGBW pixel and the input RGB pixel, where the ratio of backlight adjustment is a power saving coefficient, or in a case that luminances displayed by the RGBW panel and the RGB panel are the same, the ratio of the backlight luminance of the RGBW panel and the backlight luminance of the RGB panel may be referred to as the power saving coefficient.

If the W sub-pixel in the RGBW panel is β times the color-mixed luminance of the R, G, B sub-pixels, assuming that the unit luminance of the W sub-pixel is | W |, the unit luminance of the R sub-pixel is | R |, and the unit luminance is nit (nite), taking the R sub-pixel as an example, the equivalent luminance of the R sub-pixel after the RGB pixel is converted into the RGBW pixel is:

|R_rgbw|＝(Rsoft+β×Wsoft)×|R| (8)

wherein, Rsoft and Wsoft are linear data output after soft peak clipping. | R_rgbwAnd | is the sum of the equivalent R component luminance of the W sub-pixel and the original R sub-pixel luminance.

Wherein β is a characteristic parameter of the display panel, and can be obtained by testing. Exemplary, specific test procedures are as follows:

the W sub-pixel is lit at a particular brightness on the RGBW panel, the other three color sub-pixels (R, G, B) do not emit light, and the brightness of the light emitted by the W sub-pixel is tested to be W1. And then, respectively lighting R, G, B sub-pixels on the RGBW panel with the same brightness, and testing that the brightness of the emitted light is R1, G1 and B1 respectively, if the other sub-pixels do not emit light:

if the RGBW panel is required to display the same luminance as the RGB panel, the following description will be given by taking the R sub-pixel as an example:

R′_rgb ^2.2×BL_rgb＝R′_rgbw ^2.2×BL_rgbw (10)

wherein BL_rgbwIs the backlight brightness, BL, of the RGBW panel_rgbIs the backlight brightness of the RGB panel. R'_rgbIs the luminance value, R ', of the R sub-pixel in the input non-linear data'_rgbwIs the equivalent luminance value of the R sub-pixel in the output nonlinear data.

In the present application, after converting input nonlinear data into linear data, R_rgb＝R′_rgb ^2.2. Linear data R output after soft peak clipping processing_rgbw＝(Rsoft+β×Wsoft)＝R′_rgbw ^2.2. Thus, the following results can be obtained in combination with formula (10):

the power saving coefficient γ is calculated as follows:

possibly, if the effect of soft peak clipping processing on the RGBW data is neglected, combining the result of format conversion represented by equation (1) and the amplified result represented by equation (6) is known:

Rsoft＝Ro′＝FinalGain×Ro＝FinalGain×(R_rgb-W_o) (13)

formula (13) can be substituted for formula (12):

the same calculation process can be applied to the G sub-pixel and the B sub-pixel, and is not described herein.

Specifically, if β is 1, then:

after determining the power saving coefficient γ of the second pixel, the average of the power saving coefficients of the image of the second format may be further determined

That is, the sum of the power saving coefficients of each second pixel in the image of the second format is calculated, and the average value is calculated to obtain the average value

The larger the power consumption is, indicating that the backlight luminance of the RGBW panel is more saved than the backlight luminance of the RGB panel.

The smaller, it indicates that the backlight luminance of the RGBW panel saves less power consumption compared to the backlight luminance of the RGB panel.

For different input images, final calculated

And also different. Of grey-scale images

Usually larger than a solid image

That is, the RGBW panel saves more power when displaying a gray-scale image than when displaying a pure-color image.

Determining a power saving coefficient mean for an image in a second format

Then, the image processing apparatus may be based on

And determining a third mapping relation table. FIG. 11 is a diagram illustrating a plurality of third mapping relationship tables, and it can be seen from FIG. 11 that the difference is

May correspond to a different third mapping table. With following

The smoother the third mapping relation table. The third mapping table may be used to represent the correspondence between different brightness level indication values and backlight PWM duty ratios. The third mapping relation table is not limited to be presented in a curve form in fig. 11, and in a specific implementation, the third mapping relation table may also be presented in other forms such as a table, which is not limited in this embodiment of the present application.

It can be seen that the same image is displayed in different display modes

Different. Calculated in aggressive mode

Larger, high power consumption benefit and poorer image quality. Calculated in conservative mode

Smaller, low power consumption benefit and better image quality.

In particular, the application processor may determine a current brightness level indication value. The image processing device may search the third mapping relation table according to the brightness level indicating value, and determine the backlight PWM duty ratio corresponding to the current brightness level indicating value.

After the duty ratio of the backlight PWM is determined, the image processing device can control the duration of the PWM high level according to the duty ratio to light the backlight module, so that the backlight module is driven to emit light, and the display panel presents corresponding pictures.

The embodiment of the application provides a method for calculating a backlight power saving coefficient, and the backlight PWM duty ratio is controlled according to the power saving coefficient. According to the embodiment of the application, all factors after RGB data are converted into RGBW data can be comprehensively considered, the power saving proportion of the RGBW panel is comprehensively calculated based on the display principle of the liquid crystal panel, and compared with the mode that only frame level gain is considered, the brightness control is more accurate, and the power saving effect can be better achieved.

An embodiment of the present application further provides an image processing apparatus, as shown in fig. 12, the image processing apparatus 10 may include at least: an image format conversion module 102, a frame level gain determination module 103, a pixel level gain determination module 104, and a fusion module 105. Wherein:

the image format conversion module 102 is configured to convert an image in a first format into an image in a second format, where the image in the first format includes a plurality of first pixels, the image in the second format includes a plurality of second pixels, the first pixels include three sub-pixels with different colors, and the second pixels include four sub-pixels with different colors. For a detailed description, please refer to the description of S201, which is not repeated herein.

A frame level gain determination module 103 for determining a frame level gain for the image in the first format. For a detailed description, please refer to the description of S202, which is not repeated herein.

And a pixel-level gain determining module 104, configured to determine a pixel-level gain corresponding to each first pixel in the image in the first format. For a detailed description, please refer to the description of S203, which is not repeated herein.

And the fusion module 105 is configured to determine a target gain corresponding to each second pixel according to the frame-level gain and the pixel-level gain corresponding to each first pixel, where the target gain is used to control the backlight brightness of the backlight module. For a detailed description, please refer to the description of S204, which is not repeated herein.

Possibly, the image processing apparatus 10 may further comprise a non-linear to linear module 101 for converting the RGB non-linear data into RGB linear data before the image format conversion module 102 converts the image of the first format into the image of the second format.

In a possible embodiment, the target gain may be further used to amplify the brightness value of each sub-pixel in the second pixel corresponding to the target gain, and the image processing apparatus 10 may further include: and the amplifying module 106 is configured to amplify the brightness value of each sub-pixel in the second pixel corresponding to the target gain according to the target gain. Specifically, the calculation process may refer to equation (6), which is not described herein again.

In a possible embodiment, the fusion module 105 may be specifically configured to determine a second mapping table according to the first mapping table and the frame-level gain, where the first mapping table is used to represent an initial corresponding relationship between the saturation of the plurality of first pixels and the pixel-level gain, and the second mapping table is a second corresponding relationship between the saturation adjusted according to the frame-level gain and the pixel-level gain; and determining the target gain corresponding to the second pixel according to the saturation of the first pixel, the second mapping relation table and the frame-level gain. The first mapping table may refer to fig. 7, and the second mapping table may refer to fig. 8. The calculation process of the target gain may refer to equation (4).

In one possible embodiment, the second mapping relationship table is:

PixelGainLUT＝(2-FrameGain)×(PixelGainLUT0-1)+1

wherein, FrameGain is a frame level gain, pixeganllut is a second mapping relation table, and pixeganllut is a first mapping relation table.

In one possible embodiment, the target gain is:

FinalGain＝FrameGain+α×(PixelGainLUT(Saturation)-1)

wherein, α is a control coefficient, α belongs to [0,1], Saturation is the Saturation of the first pixel, FinalGain is a target gain, and pixeganllut is a second mapping relation table.

In one possible embodiment, the image processing apparatus 10 may include a plurality of display modes, different display modes have different control coefficients, and the image processing apparatus 10 has different requirements for power consumption in the different display modes.

In a possible embodiment, the image processing apparatus 10 further includes a soft peak clipping module 107, which is configured to perform soft peak clipping on the data output by the amplifying module 106. For a specific soft peak reduction process, reference may be made to the description of the soft peak reduction process in the foregoing method embodiments, which is not repeated herein.

In a possible embodiment, the length of the luminance value of the second pixel is a first bit width, the length of the amplified luminance value of the sub-pixel is a second bit width, the length of the luminance value of the second pixel after the soft-clipping processing is the first bit width, and the first bit width is smaller than the second bit width.

In one possible embodiment, the image processing apparatus 10 may further include a sub-pixel rendering module 108 and a linear-to-nonlinear module 109. Wherein:

the subpixel rendering module 108 may be used to map the RGBW data to the RGBW arrangement of the actual panel. Specifically, the high-resolution image may be displayed on a low-resolution display panel.

The linear-to-nonlinear module 109 may be configured to perform nonlinear conversion on the RGBW data, and specifically, may convert the linear RGBW data into the nonlinear RGBW data.

In a possible embodiment, the image processing apparatus 10 may further include a duty ratio calculating module 110 and a backlight brightness control module 111, wherein:

the duty ratio calculating module 110 may be configured to determine a duty ratio of the backlight pulse width modulation PWM of the backlight module according to the target gain. For the specific calculation process, reference may be made to the description related to the equations (8) to (15) in the foregoing embodiments, which are not repeated herein.

The backlight brightness control module 111 may be configured to control the backlight brightness of the backlight module according to the duty ratio.

In a possible embodiment, the duty cycle calculation module 110 is specifically configured to: determining a power saving coefficient gamma of the second pixel according to a target gain; determining a third mapping relation table according to the gamma, wherein the third mapping relation table is used for representing the corresponding relation between different brightness level indication values and the duty ratio of backlight PWM; and searching a third mapping relation table, and determining the duty ratio of the backlight PWM corresponding to the current brightness level indication value. For the specific calculation process, reference may be made to the part of the calculation process of the backlight PWM duty ratio described in the foregoing embodiments, which is not described herein again.

In one possible embodiment, the first pixel includes three first-color sub-pixels, the second pixel includes three first-color sub-pixels and one second-color sub-pixel, and the power saving factor is:

It is to be understood that the description of the respective units or modules may also refer to the embodiments of the aforementioned image processing method, which are not described in detail here.

An embodiment of the present application provides another image processing apparatus, and as shown in fig. 13, the image processing apparatus 20 may include at least: a transmission interface 210, at least one processor 220, a memory 230, at least one communication bus 260, a display panel 250 and a backlight module 240. Where the communication bus 260 is used for implementing connection communication among these components, it should be understood that each component in the image processing apparatus 20 may also be coupled through other connectors, which may include various interfaces, transmission lines or buses, etc., and in various embodiments of the present application, coupling refers to mutual association in a specific manner, including direct connection or indirect connection through other devices.

Among other things, the processor 220 may include at least one of the following types: a general Central Processing Unit (CPU), a Digital Signal Processor (DSP), a microprocessor, an Application Specific Integrated Circuit (ASIC), a Microcontroller (MCU), a Field Programmable Gate Array (FPGA), or an Integrated Circuit for implementing logical operations. For example, the processor 220 may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. The multiple processors or units included within processor 220 may be integrated in one chip or located on multiple different chips.

The transmission interface 210 may be used to receive or transmit image data. The received image data may be RGB format data, and the transmitted data may be RGBW format data. The transmission interface 210 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface).

The Memory 230 may be a nonvolatile Memory, such as an EMMC (Embedded multimedia Card), an UFS (Universal Flash Storage) or a Read-Only Memory (ROM), and optionally the Memory 230 includes a Flash Memory in the embodiments of the present application, or other types of static Storage devices capable of storing static information and instructions, or a nonvolatile Memory (volatile Memory), such as a Random Access Memory (RAM) or other types of dynamic Storage devices capable of storing information and instructions, or an Electrically Erasable Programmable Read-Only Memory (EEPROM), a Compact Disc Read-Only Memory (CD-ROM), or other optical Disc Storage, optical Disc Storage (including Compact Disc, and optical Disc), or other optical Disc Storage, Optical disks, digital versatile disks, blu-ray disks, etc.), magnetic disk storage media or other magnetic storage devices, or any other computer-readable storage medium that can be used to carry or store program code in the form of instructions or data structures and that can be accessed by a computer, but are not limited to such. Optionally, the memory 230 may also be at least one memory system located remotely from the processor 230. Memory 230, which is a type of computer storage medium, may include an operating system, a network communication module, a user interface module, and program instructions.

Memory 230 may be self-contained and coupled to processor 220 via a connector. Memory 230 may also be integrated with processor 220. The memory 230 can store various computer program instructions for executing the program instructions of the present disclosure, and is controlled by the processor 220 to execute, and the executed computer program instructions can also be regarded as a driver of the processor 220. For example, processor 220 is operative to execute computer program instructions stored in memory 230 to implement the methods in the various method embodiments of the present application. The computer program instructions may be provided in large numbers to form computer-executable instructions that can be executed by at least one of the processors 220 to drive the associated processor to perform various types of processing, such as communication signal processing algorithms, operating system operations, or application program operations that support the various types of wireless communication protocols described above.

The backlight module 240 may be used to provide sufficient light sources with uniform brightness and distribution for the display panel 250, so that the display panel 250 can normally display images.

The display panel 250 may be an LCD. The LCD is a passive light emitting device, which does not emit light, and the display function can be achieved by the backlight module 240. Specifically, the light transmittance of each pixel can be adjusted to allow the light source emitted from the backlight module 240 to pass through each pixel, so that each pixel presents different colors, and finally presents an image. The display panel referred to in the embodiments of the present application is an RGBW panel.

By implementing the embodiment of the application, the frame-level gain and the pixel-level gain can be fused to amplify the image, wherein the frame-level gain can increase the brightness of saturated colors in the image, and the pixel-level gain can increase the brightness of unsaturated colors in the image, so that the contrast is increased. Therefore, the method and the device have the advantages that the brightness of the saturated color can be increased by combining two gains, the brightness of the unsaturated color and the contrast of the whole picture are increased, and the advantages of the RGBW panel can be fully embodied. According to the embodiment of the application, the bit width of the data can be expanded after the RGB data are converted into the RGBW data, and then soft peak clipping processing is carried out, so that the phenomenon that highlight and high-saturation color details are lost after the RGB is converted into the RGBW is avoided, color cast is avoided, and the quality of a picture displayed on the RGBW panel is further improved. In addition, the embodiment of the application provides a method for calculating a backlight power saving coefficient, and the backlight PWM duty ratio is controlled according to the power saving coefficient. According to the embodiment of the application, all factors after RGB data are converted into RGBW data can be comprehensively considered, the power saving proportion of the RGBW panel is comprehensively calculated based on the display principle of the liquid crystal panel, and compared with the mode that only frame level gain is considered, the brightness control is more accurate, and the power saving effect can be better achieved.

Embodiments of the present application also provide a computer-readable storage medium having stored therein instructions, which when executed on a computer or processor, cause the computer or processor to perform one or more steps of any one of the methods described above. The respective constituent modules of the signal processing apparatus may be stored in the computer-readable storage medium if they are implemented in the form of software functional units and sold or used as independent products.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in or transmitted over a computer-readable storage medium. The computer instructions may be transmitted from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.

One of ordinary skill in the art will appreciate that all or part of the processes in the methods of the above embodiments may be implemented by hardware related to instructions of a computer program, which may be stored in a computer-readable storage medium, and when executed, may include the processes of the above method embodiments. And the aforementioned storage medium includes: various media capable of storing program codes, such as ROM or RAM, magnetic or optical disks, etc.

Claims

1. An image processing method, characterized in that the method comprises:

converting an image in a first format to an image in a second format, the image in the first format comprising a plurality of first pixels, the image in the second format comprising a plurality of second pixels, the first pixels comprising three different color sub-pixels, the second pixels comprising four different color sub-pixels;

determining a frame level gain for the image in the first format;

respectively determining the pixel level gain corresponding to each first pixel in the image in the first format;

determining a second mapping relation table according to a first mapping relation table and the frame level gain, wherein the first mapping relation table is used for representing an initial corresponding relation between the saturation of a plurality of first pixels and the pixel level gain, and the second mapping relation table is a second corresponding relation between the saturation adjusted according to the frame level gain and the pixel level gain;

and determining a target gain corresponding to the second pixel according to the saturation of the first pixel, the second mapping relation table and the frame level gain, wherein the target gain is used for controlling the backlight brightness of the backlight module.

2. The method of claim 1, wherein the target gain is further used for amplifying a luminance value of each sub-pixel in the second pixel corresponding to the target gain, and wherein the method further comprises, after determining the target gain for each second pixel according to the frame-level gain and the pixel-level gain for each first pixel, the method further comprises:

and amplifying the brightness value of each sub-pixel in the second pixel corresponding to the target gain according to the target gain.

3. The method of claim 1, wherein determining a second mapping table based on the first mapping table and the frame-level gain comprises:

PixelGainLUT＝(2-FrameGain)×(PixelGainLUT0-1)+1

wherein FrameGain is the frame level gain, pixeganllut is the second mapping table, and pixeganllut 0 is the first mapping table.

4. The method of claim 1, wherein the determining the target gain for the second pixel according to the saturation of the first pixel, the second mapping table, and the frame-level gain comprises:

FinalGain＝FrameGain+α×(PixelGainLUT(Saturation)-1)

wherein α is a control coefficient, α ∈ [0,1], Saturation is the Saturation of the first pixel, FinalGain is the target gain, and pixeganllut is the second mapping relationship table.

5. The method of claim 4, wherein the image processing apparatus includes a plurality of display modes, the control coefficients are different for different display modes, and the image processing apparatus has different requirements for power consumption in the different display modes.

6. The method of claim 2, wherein after the amplifying the luminance values of the respective sub-pixels in the second pixel corresponding to the target gain according to the target gain, the method further comprises:

determining the duty ratio of backlight Pulse Width Modulation (PWM) of the backlight module according to the target gain;

and controlling the backlight brightness of the backlight module according to the duty ratio.

7. The method of claim 6, wherein the determining the duty cycle of the backlight PWM for the backlight module according to the target gain comprises:

determining a power saving coefficient gamma of the second pixel according to the target gain;

determining a third mapping relation table according to the gamma, wherein the third mapping relation table is used for representing the corresponding relation between different brightness level indication values and the duty ratio of backlight PWM;

and searching the third mapping relation table, and determining the duty ratio of the backlight PWM corresponding to the current brightness level indication value.

8. The method of claim 7, wherein the first pixel comprises three color-first sub-pixels, and the second pixel comprises the three color-first sub-pixels and a color-second sub-pixel;

the determining a power saving coefficient γ of the second pixel according to the target gain includes:

wherein, X_rgbThe gain control method includes that a brightness value of any one of three sub-pixels of first pixels with first colors is adopted, Wo is a brightness value of a sub-pixel of second pixels with second colors, beta is a characteristic parameter of a display panel, and FinalGain is a target gain corresponding to the second pixels.

9. The method according to claim 7 or 8, wherein after the amplifying the luminance value of each sub-pixel in the second pixel corresponding to the target gain according to the target gain, the method further comprises:

and performing soft peak clipping processing on the second pixel after the brightness value of the sub-pixel is amplified.

10. The method of claim 9 wherein said second pixel luminance value has a length of a first bit width, said amplified second pixel luminance value of said sub-pixel luminance value has a length of a second bit width, said soft-clipped second pixel luminance value has a length of said first bit width, and said first bit width is less than said second bit width.

11. An image processing apparatus, characterized in that the apparatus comprises:

an image format conversion module, configured to convert an image in a first format into an image in a second format, where the image in the first format includes a plurality of first pixels, the image in the second format includes a plurality of second pixels, the first pixels include three sub-pixels with different colors, and the second pixels include four sub-pixels with different colors;

a frame level gain determination module for determining a frame level gain for the image in the first format;

a pixel level gain determining module, configured to determine a pixel level gain corresponding to each first pixel in the image in the first format;

a fusion module, configured to determine a second mapping table according to a first mapping table and the frame-level gain, where the first mapping table is used to represent an initial correspondence between saturation of a plurality of first pixels and pixel-level gain, and the second mapping table is a second correspondence between saturation adjusted according to the frame-level gain and the pixel-level gain;

12. The apparatus of claim 11, wherein the target gain is further used for amplifying a luminance value of each sub-pixel in the second pixel corresponding to the target gain, the apparatus further comprising:

and the amplifying module is used for amplifying the brightness value of each sub-pixel in the second pixel corresponding to the target gain according to the target gain.

13. The apparatus of claim 11, wherein the second mapping table is:

PixelGainLUT＝(2-FrameGain)×(PixelGainLUT0-1)+1

14. The apparatus of claim 11, wherein the target gain is:

FinalGain＝FrameGain+α×(PixelGainLUT(Saturation)-1)

15. The apparatus of claim 14, wherein the apparatus comprises a plurality of display modes, different display modes having different control coefficients, and wherein the apparatus has different power consumption requirements in the different display modes.

16. The apparatus of claim 12, wherein the apparatus further comprises:

the duty ratio calculation module is used for determining the duty ratio of backlight Pulse Width Modulation (PWM) of the backlight module according to the target gain;

and the backlight brightness control module is used for controlling the backlight brightness of the backlight module according to the duty ratio.

17. The apparatus of claim 16, wherein the duty cycle calculation module is specifically configured to:

18. The apparatus of claim 17, wherein the first pixel comprises three first-color sub-pixels, the second pixel comprises the three first-color sub-pixels and a second-color sub-pixel, and the power saving factor is:

19. The apparatus of claim 17 or 18, wherein the apparatus further comprises:

and the soft peak clipping module is used for performing soft peak clipping processing on the second pixel after the brightness value of each sub-pixel in the second pixel corresponding to the target gain is amplified by the amplification module according to the target gain.

20. The apparatus according to claim 19 wherein said second pixel luminance value has a length of a first bit width, said amplified second pixel luminance value of said sub-pixel luminance value has a length of a second bit width, said soft-clipped second pixel luminance value has a length of said first bit width, and said first bit width is less than said second bit width.

21. An image processing apparatus comprising a processor and a transmission interface;

the transmission interface is used for receiving or transmitting image data;

the processor is configured to invoke software instructions stored in the memory to cause the processor to perform the method of any one of claims 1 to 11.

22. The apparatus of claim 19, wherein the apparatus further comprises: backlight unit and display panel.

23. A computer-readable storage medium having stored therein instructions, which when run on a computer or processor, cause the computer or processor to perform the method of any one of claims 1 to 11.