WO2018072270A1 - Method and device for enhancing image display - Google Patents

Abstract

Provided are a method and device for enhancing image display and a terminal capable of solving a problem in the prior art in which image display performance is poor. The method comprises: acquiring scene information of an image to be displayed, the scene information of the image to be displayed being obtained by performing scene analysis on the image to be displayed; executing, on the basis of the scene information, at least one of the following processing operations on the image to be displayed: color gamut mapping, color management, contrast enhancement, sharpening, or zooming; and displaying the processed image.

Description

Image display enhancement method and device

The present application claims priority to Chinese Patent Application No. 201610903016.2, entitled "A Method and Terminal for Enhancing Picture Display Effect", filed on October 17, 2016, the entire contents of which are incorporated by reference. In this application.

Technical field

The present application relates to the field of image processing technologies, and in particular, to an image display enhancement method and apparatus.

Background technique

Smartphones have become an increasingly important electronic product in people's lives. It not only has the functions of traditional mobile phone call communication, but also undertakes the mission of the entertainment terminal in the Internet era. Browsing images is one of the most basic features of a smartphone. When the user browses the image using the library, WeChat, etc. (English: application, abbreviation: APP), the application sends the image file information to the frame of the smartphone, so that the frame layer parses the image file information and calls The decoder decodes the image file information, and then sends it to the hardware layer of the smartphone for rendering and the like, and then displays it through a liquid crystal display (English: Liquid Crystal Display, referred to as LCD).

In the prior art, the image is not processed before the image is displayed, resulting in a poor image display effect.

Summary of the invention

The embodiment of the present application provides an image display enhancement method, device, and terminal, which are used to solve the problem that the image display effect existing in the prior art is poor.

In a first aspect, an embodiment of the present application provides an image display enhancement method, where the method includes:

Acquiring the scene information of the image to be displayed, the scene information of the image to be displayed is obtained by performing scene analysis on the image to be displayed; performing at least one of the following processing on the image to be displayed based on the scene information: color gamut Mapping, color management, contrast enhancement, sharpening or scaling; The processed image to be displayed after processing.

Scene information may include, but is not limited to, a blue sky scene, a green plant scene, a backlight scene, a night scene, and the like. In addition, the scene information may be written to the image to be displayed when being photographed by the camera, so that the scene information is obtained by analyzing the image information included in the image to be displayed.

In the embodiment of the present application, before the image is displayed, the image is subjected to gamut mapping, color management, contrast enhancement, sharpening, and scaling processing based on the scene information of the image to be displayed, and the android color difference is generated when the Android native system uses the application to browse the image. , aliasing, unclear, low contrast and other display effects. In addition, in the prior art, the processing of the image data stream depends on a hardware device such as a chip or a screen driving circuit to enhance the display effect of the image, but is limited by the image enhancement algorithm that can be executed by the hardware, and the display effect is poor, and the present application implements The example is not limited to hardware devices, thereby solving the problem in the prior art that the hardware-based display effect enhancement method is limited by the algorithm flexibility and the display effect is poor, thereby improving the display effect of the image.

In one possible design, the scene information is included in the metadata Metadata of the image to be displayed, or in the exchangeable image file EXIF data area, or in the vendor comment makernotes field.

When the scene information is written into the acquired image, the scene information may be written into a MakerNote field of an EXIF data area of an exchangeable image file (English: Exchangeable Image File, EXIF) of the image. in. The scene information may be included in the metadata Metadata of the image to be displayed, or in the exchangeable image file EXIF data area, or in the vendor comment makernotes field. The scene information includes at least one of the following: a sensitivity ISO value, an aperture value, a shutter time, or an exposure EV value. Specifically, the scene information may be included in a vendor note makernotes field of the exchangeable image file EXIF data area of the image to be displayed, or may be included in other fields of the exchangeable image file EXIF data area.

In one possible design, the scene information includes at least one of the following: sensitivity ISO value, aperture value, shutter time, or exposure EV value.

In a possible design, the performing gamut mapping on the image to be displayed based on the scene information includes:

Determining a first three-dimensional lookup table corresponding to the scene information, and a second three-dimensional lookup table, where the first three-dimensional lookup table is configured to maintain color gamut mapping and color gamut mapping before the color to be protected in the image to be displayed Similarly, the second three-dimensional lookup table is configured to adjust a color that is not protected in the image to be displayed;

Determining, that the color of the i-th pixel point among the N pixel points included in the image to be displayed corresponds to a first color value in the first three-dimensional lookup table, and determining that the color of the i-th pixel point corresponds to Describe a second color value in the second three-dimensional lookup table;

The first color value and the second color value are fused based on the predetermined fusion weight to obtain a color value of the ith pixel point after the gamut mapping, and the i takes a positive integer not greater than N.

Among them, the color to be protected includes at least one of the following: skin color, gray color, and memory color.

The grayscale color is black based on the reference color, and the black of different saturations is used to display the image. Each gray object has a brightness value from 0% (white) to 100% (black) of the gray bar. People have a deep memory of the understanding of certain colors in long-term practice, so the understanding of these colors has a certain regularity and forms an inherent habit. These colors are called memory colors.

The gamut mapping algorithm of the present application realizes the distinguishing process of the adjustment color and the protection color based on the double three-dimensional lookup table, and can realize the gamut mapping and the color chromaticity correction while protecting the skin color, the gray color, the memory color, and ensuring the transition from the adjustment color to the protection color. No false contours appear.

In one possible design, the method further includes:

When the color corresponding color value of the i-th pixel point is not included in the first three-dimensional lookup table, the i-th pixel point is obtained by interpolating colors included in the first three-dimensional lookup table by using a three-dimensional interpolation algorithm. The color corresponds to the color value; or

When the color corresponding color value of the i-th pixel point is not included in the second three-dimensional lookup table, the i-th pixel point is obtained by interpolating colors included in the second three-dimensional lookup table by using a three-dimensional interpolation algorithm The color corresponds to the color value.

The three-dimensional interpolation algorithm may be a trilinear interpolation method, a tetrahedral interpolation method, a pyramid method or a prism interpolation method, or the like.

In one possible design, the fusion weight is determined by the following formula:

Wherein, W _blending represents the fusion weight, T _blending represents a threshold for controlling smooth transition from the adjusted color to the protective color, and Δd represents that the color of the i-th pixel corresponds to the color node in the three-dimensional color space and is determined by the first three-dimensional Find the shortest distance between the convex hups formed by the color nodes in the three-dimensional color space; Δd<0 indicates that the color node of the ith pixel is located inside the convex hull; Δd=0 indicates the ith pixel The color node is located on the convex hull; Δd>0 indicates that the color node of the ith pixel is located outside the convex hull.

In a possible design, the first color value and the second color value are fused based on the predetermined fusion weight to obtain the color value of the ith pixel point after the gamut mapping, including:

The color value of the ith pixel point after the gamut mapping is determined by:

A _out =A ₁ *W _blending +A ₂ *(1-W _blending );

Wherein A _out represents a color value of the i-th pixel point after gamut mapping, A ₁ represents the first color value, A ₂ represents the second color value, and W _blending represents the fusion weight.

In a possible design, the performing color management on the image to be displayed based on the scene information includes:

Acquiring an adjustment strategy for adjusting pixel saturation of the image to be displayed corresponding to the scene information;

The brightness saturation hue YSH values of the jth pixel points in the image to be displayed are respectively adjusted as follows:

Keeping the jth pixel dot hue H value unchanged, adjusting the saturation S value of the jth pixel point based on the adjustment strategy;

Obtaining a quadratic function function corresponding to the S value of the jth pixel point before adjustment, the adjusted S value, and the brightness Y of the jth pixel point corresponding to the H value of the predetermined jth pixel point The compensated luminance Y value of the jth pixel; the j takes a positive integer not greater than N.

Different scene information corresponding to different adjustment strategies, such as the existence of blue sky, natural plants and other scenes, can improve the saturation to optimize the display effect; for example, scenes with skin color, etc., may reduce the saturation and make the skin smoother. The adjustment policy corresponding to different scenario information may be pre-configured in the electronic device.

The color management method provided in the embodiment of the present application can solve the problem of brightness change and brightness display change caused by brightness change based on the YSH color space, and the method is simple.

In a possible design, the quadratic function corresponding to the H value of the jth pixel point satisfies the condition described in the following formula:

Wherein, Y _out represents the j-th output luminance values of pixels, Y _in represents the input luminance value of the j-th pixel point, ΔY represents the luminance compensation value, a1, a2, a3, a4 , a5, A6 represents a quadratic function function coefficient, S _in represents a saturation value of the j-th pixel point before adjustment, and S _out represents a saturation value of the adjusted j-th pixel point.

In one possible design, the quadratic function coefficients are determined as follows:

In the case where there is a quadratic function corresponding to the H value of the jth pixel, the a1, a2, a3, a4, a5, a6 are respectively used in advance using the jth pixel before adjustment a saturation value, an adjusted saturation value of the jth pixel point, and the brightness compensation value, which are obtained based on a least squares fitting algorithm; or

In the case where there is no quadratic function corresponding to the H value of the jth pixel, the a1, a2, a3, a4, a5, a6 are closest to the H value of the jth pixel The two tonal values are respectively obtained by interpolating the coefficients of the quadratic function corresponding to each.

In one possible design, in the case where there is no quadratic function corresponding to the H value of the jth pixel, the a1, a2, a3, a4, a5, a6 are obtained by the following formula:

a _k represents the kth coefficient in a1, a2, a3, a4, a5, a6; H represents the saturation value of the jth pixel point, and H _p1 and H _p2 represent the nearest neighbor of the H value of the jth pixel point The two tonal values, H _p1 <H<H _p2 , a _{(p1, k)} represents the kth coefficient of the quadratic function corresponding to H _p1 , and a _{(p2, k)} represents the quadratic function corresponding to H _p2 The kth coefficient.

In a possible design, the performing zooming on the image to be displayed based on the scene information includes:

Determining a zoom factor of the image to be displayed, and determining a highest frequency of the periodic pattern in the image to be displayed according to the scene information;

Determining, according to a preset lookup table, the zoom factor and the number of taps of the bicubic filter corresponding to the highest frequency of the periodic pattern in the image to be displayed; and determining the double according to the highest frequency of the periodic pattern in the image to be displayed The coefficient of the cubic equation function corresponding to the cubic filter;

The image to be displayed is scaled according to the number of taps and the determined cubic function.

The number of taps is also the number of sampled pixels sampled.

Specifically, searching for a corresponding pixel position of the target image dst pixel point on the source image src, adaptively selecting the determined number of sampling pixel points (reference points) around the corresponding pixel point, and determining each according to the bicubic interpolation function. The weight value corresponding to each sampling pixel point is weighted and summed according to the weight value of the sampling number of sampling pixels and the pixel value of the sampling number of sampling pixels, and the pixel value of the scaled image is obtained by weighting, and finally the scaled target image is obtained.

The scaling method provided by the embodiment of the present application can perform adaptive anti-aliasing based on image scene information and the highest frequency of the image, thereby improving the image optimization effect.

In one possible design, the cubic function corresponding to the bicubic filter satisfies the conditions shown in the following formula:

Where x represents the position of the sampled pixel selected based on the number of taps, k(x) represents the weight value corresponding to the sampled pixel point, and dB and dC represent the coefficients of the cubic function, respectively; B and C are constants.

In a second aspect, an embodiment of the present application provides an image display enhancement device, where the device includes:

a display enhancement module, configured to acquire scene information of an image to be displayed, where the scene information of the image to be displayed is obtained by performing scene analysis on the image to be displayed; and performing, according to the scene information, the image to be displayed At least one type of processing: gamut mapping, color management, contrast enhancement, sharpening, or scaling;

And a display module, configured to display the image to be displayed after being processed by the display enhancement module.

In one possible design, the scene information is included in the metadata Metadata of the image to be displayed, or in the exchangeable image file EXIF data area, or in the vendor comment makernotes field.

In one possible design, the scene information includes at least one of the following: sensitivity ISO value, aperture value, shutter time, or exposure EV value.

In a possible design, the display enhancement module performs gamut mapping on the image to be displayed based on the scene information, specifically for:

Determining a first three-dimensional lookup table corresponding to the scene information, and a second three-dimensional lookup table, where the first three-dimensional lookup table is configured to maintain color gamut mapping and color gamut mapping before the color to be protected in the image to be displayed Similarly, the second three-dimensional lookup table is configured to adjust a color that is not protected in the image to be displayed;

Determining, that the color of the i-th pixel point among the N pixel points included in the image to be displayed corresponds to a first color value in the first three-dimensional lookup table, and determining that the color of the i-th pixel point corresponds to Describe a second color value in the second three-dimensional lookup table;

The first color value and the second color value are fused based on the predetermined fusion weight to obtain a color value of the ith pixel point after the gamut mapping, and the i takes a positive integer not greater than N.

In one possible design, the display enhancement module is further configured to:

When the color corresponding color value of the i-th pixel point is not included in the first three-dimensional lookup table, the i-th pixel point is obtained by interpolating colors included in the first three-dimensional lookup table by using a three-dimensional interpolation algorithm. The color corresponds to the color value; or

When the color corresponding color value of the i-th pixel point is not included in the second three-dimensional lookup table, the i-th pixel point is obtained by interpolating colors included in the second three-dimensional lookup table by using a three-dimensional interpolation algorithm The color corresponds to the color value.

In one possible design, the display enhancement module is further configured to determine the fusion weight by the following formula:

Wherein, W _blending represents the fusion weight, T _blending represents a threshold for controlling smooth transition from the adjusted color to the protective color, and Δd represents that the color of the i-th pixel corresponds to the color node in the three-dimensional color space and is determined by the first three-dimensional Find the shortest distance between the convex hups formed by the color nodes in the three-dimensional color space; Δd<0 indicates that the color node of the ith pixel is located inside the convex hull; Δd=0 indicates the ith pixel The color node is located on the convex hull; Δd>0 indicates that the color node of the ith pixel is located outside the convex hull.

In a possible design, the display enhancement module fuses the first color value and the second color value to obtain a color of the ith pixel point after gamut mapping based on a predetermined fusion weight. When the value is used, it is specifically used to:

The color value of the ith pixel point after the gamut mapping is determined by:

A _out =A ₁ *W _blending +A ₂ *(1-W _blending );

Wherein A _out represents a color value of the i-th pixel point after gamut mapping, A ₁ represents the first color value, A ₂ represents the second color value, and W _blending represents the fusion weight.

In a possible design, when the color management is performed on the image to be displayed based on the scene information, the display enhancement module is specifically configured to:

Acquiring an adjustment strategy for adjusting pixel saturation of the image to be displayed corresponding to the scene information;

The brightness saturation hue YSH values of the jth pixel points in the image to be displayed are respectively adjusted as follows:

Keeping the jth pixel dot hue H value unchanged, adjusting the saturation S value of the jth pixel point based on the adjustment strategy;

Obtaining a quadratic function function corresponding to the S value of the jth pixel point before adjustment, the adjusted S value, and the brightness Y of the jth pixel point corresponding to the H value of the predetermined jth pixel point The compensated luminance Y value of the jth pixel; the j takes a positive integer not greater than N.

In a possible design, the quadratic function corresponding to the H value of the jth pixel point satisfies the condition described in the following formula:

Wherein, Y _out represents the j-th output luminance values of pixels, Y _in represents the input luminance value of the j-th pixel point, ΔY represents the luminance compensation value, a1, a2, a3, a4 , a5, A6 represents a quadratic function function coefficient, S _in represents a saturation value of the j-th pixel point before adjustment, and S _out represents a saturation value of the adjusted j-th pixel point.

In one possible design, the display enhancement module is further configured to determine the quadratic function function coefficients by:

In the case where there is a quadratic function corresponding to the H value of the jth pixel, the a1, a2 A3, a4, a5, and a6 are the saturation values of the j-th pixel before the adjustment, the saturation value of the j-th pixel after the adjustment, and the brightness compensation value, respectively, based on the minimum Solved by a two-square fitting algorithm; or,

In the case where there is no quadratic function corresponding to the H value of the jth pixel, the a1, a2, a3, a4, a5, a6 are closest to the H value of the jth pixel The two tonal values are respectively obtained by interpolating the coefficients of the quadratic function corresponding to each.

In a possible design, in the case where there is no quadratic function corresponding to the H value of the jth pixel, the display enhancement module is specifically configured to determine a1, a2, a3, a4 by the following formula: , a5, a6:

a _k represents the kth coefficient in a1, a2, a3, a4, a5, a6; H represents the saturation value of the jth pixel point, and H _p1 and H _p2 represent the nearest neighbor of the H value of the jth pixel point The two tonal values, H _p1 <H<H _p2 , a _{(p1, k)} represents the kth coefficient of the quadratic function corresponding to H _p1 , and a _{(p2, k)} represents the quadratic function corresponding to H _p2 The kth coefficient.

In a possible design, the display enhancement module is specifically configured to: when performing scaling on the image to be displayed based on the scene information,

Determining a zoom factor of the image to be displayed, and determining a highest frequency of the periodic pattern in the image to be displayed according to the scene information;

Determining, according to a preset lookup table, the zoom factor and the number of taps of the bicubic filter corresponding to the highest frequency of the periodic pattern in the image to be displayed; and determining the double according to the highest frequency of the periodic pattern in the image to be displayed The coefficient of the cubic equation function corresponding to the cubic filter;

The image to be displayed is scaled according to the number of taps and the determined cubic function.

In one possible design, the cubic function corresponding to the bicubic filter satisfies the conditions shown in the following formula:

Where x represents the position of the sampled pixel selected based on the number of taps, k(x) represents the weight value corresponding to the sampled pixel point, and dB and dC represent the coefficients of the cubic function, respectively; B and C are constants.

In a third aspect, the embodiment of the present application further provides a terminal, including:

a processor, configured to acquire scene information of an image to be displayed, where the scene information of the image to be displayed is obtained by performing scene analysis on the image to be displayed; and performing at least one of the following on the image to be displayed based on the scene information Processing: gamut mapping, color management, contrast enhancement, sharpening or scaling;

And a display for displaying the image to be displayed after the processing.

In one possible design, the scene information is included in the metadata Metadata of the image to be displayed, or in the exchangeable image file EXIF data area, or in the vendor comment makernotes field.

In one possible design, the scene information includes at least one of the following: sensitivity ISO value, aperture value, shutter time, or exposure EV value.

In a possible design, the processor performs gamut mapping on the image to be displayed based on the scenario information, specifically for:

Determining a first three-dimensional lookup table corresponding to the scene information, and a second three-dimensional lookup table, where the first three-dimensional lookup table is configured to maintain color gamut mapping and color gamut mapping before the color to be protected in the image to be displayed Similarly, the second three-dimensional lookup table is configured to adjust a color that is not protected in the image to be displayed;

Determining, that the color of the i-th pixel point among the N pixel points included in the image to be displayed corresponds to a first color value in the first three-dimensional lookup table, and determining that the color of the i-th pixel point corresponds to Describe a second color value in the second three-dimensional lookup table;

The first color value and the second color value are fused based on the predetermined fusion weight to obtain a color value of the ith pixel point after the gamut mapping, and the i takes a positive integer not greater than N.

In a possible design, the processor is further configured to:

When the color corresponding color value of the i-th pixel point is not included in the first three-dimensional lookup table, the i-th pixel point is obtained by interpolating colors included in the first three-dimensional lookup table by using a three-dimensional interpolation algorithm. The color corresponds to the color value; or

When the color corresponding color value of the i-th pixel point is not included in the second three-dimensional lookup table, the i-th pixel point is obtained by interpolating colors included in the second three-dimensional lookup table by using a three-dimensional interpolation algorithm The color corresponds to the color value.

In one possible design, the fusion weight is determined by the following formula:

Wherein, W _blending represents the fusion weight, T _blending represents a threshold for controlling smooth transition from the adjusted color to the protective color, and Δd represents that the color of the i-th pixel corresponds to the color node in the three-dimensional color space and is determined by the first three-dimensional Find the shortest distance between the convex hups formed by the color nodes in the three-dimensional color space; Δd<0 indicates that the color node of the ith pixel is located inside the convex hull; Δd=0 indicates the ith pixel The color node is located on the convex hull; Δd>0 indicates that the color node of the ith pixel is located outside the convex hull.

In a possible design, the processor combines the first color value and the second color value to obtain a color value of the ith pixel point after gamut mapping based on a predetermined fusion weight When specifically used to:

The color value of the ith pixel point after the gamut mapping is determined by:

A _out =A ₁ *W _blending +A ₂ *(1-W _blending );

Wherein A _out represents a color value of the i-th pixel point after gamut mapping, A ₁ represents the first color value, A ₂ represents the second color value, and W _blending represents the fusion weight.

In a possible design, the processor performs color management on the image to be displayed based on the scene information, specifically for:

Acquiring an adjustment strategy for adjusting pixel saturation of the image to be displayed corresponding to the scene information;

The brightness saturation hue YSH values of the jth pixel points in the image to be displayed are respectively adjusted as follows:

Keeping the jth pixel dot hue H value unchanged, adjusting the saturation S value of the jth pixel point based on the adjustment strategy;

Obtaining a quadratic function function corresponding to the S value of the jth pixel point before adjustment, the adjusted S value, and the brightness Y of the jth pixel point corresponding to the H value of the predetermined jth pixel point The compensated luminance Y value of the jth pixel; the j takes a positive integer not greater than N.

In a possible design, the quadratic function corresponding to the H value of the jth pixel point satisfies the condition described in the following formula:

Wherein, Y _out represents the j-th output luminance values of pixels, Y _in represents the input luminance value of the j-th pixel point, ΔY represents the luminance compensation value, a1, a2, a3, a4 , a5, A6 represents a quadratic function function coefficient, S _in represents a saturation value of the j-th pixel point before adjustment, and S _out represents a saturation value of the adjusted j-th pixel point.

In one possible design, the processor is further configured to determine the quadratic function coefficients by:

In the case where there is a quadratic function corresponding to the H value of the jth pixel, the a1, a2, a3, a4, a5, a6 are respectively used in advance using the jth pixel before adjustment Saturation value, after The adjusted saturation value of the jth pixel point and the brightness compensation value are obtained based on a least squares fitting algorithm; or

In the case where there is no quadratic function corresponding to the H value of the jth pixel, the a1, a2, a3, a4, a5, a6 are closest to the H value of the jth pixel The two tonal values are respectively obtained by interpolating the coefficients of the quadratic function corresponding to each.

In a possible design, in the case where there is no quadratic function corresponding to the H value of the jth pixel, the processor is further configured to determine the a1, a2, a3, by the following formula: A4, a5, a6:

a _k represents the kth coefficient in a1, a2, a3, a4, a5, a6; H represents the saturation value of the jth pixel point, and H _p1 and H _p2 represent the nearest neighbor of the H value of the jth pixel point The two tonal values, H _p1 <H<H _p2 , a _{(p1, k)} represents the kth coefficient of the quadratic function corresponding to H _p1 , and a _{(p2, k)} represents the quadratic function corresponding to H _p2 The kth coefficient.

In a possible design, the processor further performs scaling on the image to be displayed based on the scene information, specifically for:

Determining a zoom factor of the image to be displayed, and determining a highest frequency of the periodic pattern in the image to be displayed according to the scene information;

Determining, according to a preset lookup table, the zoom factor and the number of taps of the bicubic filter corresponding to the highest frequency of the periodic pattern in the image to be displayed; and determining the double according to the highest frequency of the periodic pattern in the image to be displayed The coefficient of the cubic equation function corresponding to the cubic filter;

The image to be displayed is scaled according to the number of taps and the determined cubic function.

In one possible design, the cubic function corresponding to the bicubic filter satisfies the conditions shown in the following formula:

Where x represents the position of the sampled pixel selected based on the number of taps, k(x) represents the weight value corresponding to the sampled pixel point, and dB and dC represent the coefficients of the cubic function, respectively; B and C are constants.

In a fourth aspect, the embodiment of the present application further provides an image gamut mapping method, where the method includes:

Get the image to be processed;

Determining, that the color of the i-th pixel point of the N pixels included in the image to be processed corresponds to a first color value in the first three-dimensional lookup table, and determining that the color of the i-th pixel point corresponds to a second color value in the second three-dimensional lookup table; the first three-dimensional lookup table is configured to keep the gamut mapping of the color to be protected in the image to be processed, and the same as before the gamut mapping, the second The three-dimensional lookup table is configured to adjust a color that is not protected in the image to be processed;

The first color value and the second color value are fused based on the predetermined fusion weight to obtain a color value of the ith pixel point after the gamut mapping, and the i takes a positive integer not greater than N.

The gamut mapping method of the present application is based on a dual three-dimensional lookup table to realize the distinguishing process of the adjustment color and the protection color, and can realize the gamut mapping and the color chromaticity correction while protecting the skin color, the gray color, the memory color, and ensuring the transition from the adjustment color to the protection color. No false contours appear.

In one possible design, the method further includes:

When the color corresponding color value of the i-th pixel point is not included in the first three-dimensional lookup table, the i-th pixel point is obtained by interpolating colors included in the first three-dimensional lookup table by using a three-dimensional interpolation algorithm. The color corresponds to the color value; or

When the color corresponding to the color of the ith pixel point is not included in the second three-dimensional lookup table, The color corresponding color value of the ith pixel point is obtained by interpolating colors included in the second three-dimensional lookup table by a three-dimensional interpolation algorithm.

In one possible design, the fusion weight is determined by the following formula:

Wherein, W _blending represents the fusion weight, T _blending represents a threshold for controlling smooth transition from the adjusted color to the protective color, and Δd represents that the color of the i-th pixel corresponds to the color node in the three-dimensional color space and is determined by the first three-dimensional Find the shortest distance between the convex hups formed by the color nodes in the three-dimensional color space; Δd<0 indicates that the color node of the ith pixel is located inside the convex hull; Δd=0 indicates the ith pixel The color node is located on the convex hull; Δd>0 indicates that the color node of the ith pixel is located outside the convex hull.

In a possible design, the first color value and the second color value are fused based on the predetermined fusion weight to obtain the color value of the ith pixel point after the gamut mapping, including:

The color value of the ith pixel point after the gamut mapping is determined by:

A _out =A ₁ *W _blending +A ₂ *(1-W _blending );

Wherein A _out represents a color value of the i-th pixel point after gamut mapping, A ₁ represents the first color value, A ₂ represents the second color value, and W _blending represents the fusion weight.

In a fifth aspect, the embodiment of the present application further provides a color management method, including:

Obtaining an image to be processed, and determining an adjustment strategy for adjusting pixel saturation of the image to be processed by the processor;

The brightness saturation hue YSH values of the jth pixel in the image to be processed are respectively adjusted as follows:

Keeping the jth pixel dot hue H value unchanged, adjusting the saturation S value of the jth pixel point based on the adjustment strategy;

S value of the jth pixel before adjustment, adjusted S value, and the jth pixel The brightness Y of the point is input to the quadratic function corresponding to the H value of the predetermined jth pixel point to obtain the compensated brightness Y value of the jth pixel point; the j takes a positive integer not greater than N .

The color management method provided in the embodiment of the present application can solve the problem of brightness change and brightness display change caused by brightness change based on the YSH color space, and the method is simple.

In a possible design, the quadratic function corresponding to the H value of the jth pixel point satisfies the condition described in the following formula:

Wherein, Y _out represents the j-th output luminance values of pixels, Y _in represents the input luminance value of the j-th pixel point, ΔY represents the luminance compensation value, a1, a2, a3, a4 , a5, A6 represents a quadratic function function coefficient, S _in represents a saturation value of the j-th pixel point before adjustment, and S _out represents a saturation value of the adjusted j-th pixel point.

In one possible design, the quadratic function coefficients are determined as follows:

In the case where there is a quadratic function corresponding to the H value of the jth pixel, the a1, a2, a3, a4, a5, a6 are respectively used in advance using the jth pixel before adjustment a saturation value, an adjusted saturation value of the jth pixel point, and the brightness compensation value, which are obtained based on a least squares fitting algorithm; or

In the case where there is no quadratic function corresponding to the H value of the jth pixel, the a1, a2, a3, a4, a5, a6 are closest to the H value of the jth pixel The two tonal values are respectively obtained by interpolating the coefficients of the quadratic function corresponding to each.

In one possible design, in the case where there is no quadratic function corresponding to the H value of the jth pixel, the a1, a2, a3, a4, a5, a6 are obtained by the following formula:

a _k represents the kth coefficient in a1, a2, a3, a4, a5, a6; H represents the saturation value of the jth pixel point, and H _p1 and H _p2 represent the nearest neighbor of the H value of the jth pixel point The two tonal values, H _p1 <H<H _p2 , a _(p1,k) represents the kth coefficient of the quadratic function corresponding to H _p1 , and a _(p2,k) represents the quadratic function corresponding to H _p2 The kth coefficient.

In a sixth aspect, the embodiment of the present application further provides an image scaling method, including:

Determining a zoom factor of the image to be processed and a highest frequency of the image to be processed;

Determining, according to a preset lookup table, the zoom factor and the number of taps of the bicubic filter corresponding to the highest frequency of the image to be processed; and determining a cubic function corresponding to the bicubic filter according to the highest frequency of the image to be processed Coefficient of

The image to be processed is scaled according to the number of taps and the determined cubic function.

The scaling method provided by the embodiment of the present application can perform adaptive anti-aliasing based on image scene information and the highest frequency of the image, thereby improving the image optimization effect.

In one possible design, the cubic function corresponding to the bicubic filter satisfies the conditions shown in the following formula:

Where x represents the position of the sampled pixel selected based on the number of taps, k(x) represents the weight value corresponding to the sampled pixel point, and dB and dC represent the coefficients of the cubic function, respectively; B and C are constants.

DRAWINGS

FIG. 1 is a schematic diagram of a terminal device according to an embodiment of the present application;

2 is a flowchart of an image display enhancement method according to an embodiment of the present application;

FIG. 3 is a flowchart of an image gamut mapping method according to an embodiment of the present application;

4 is a schematic diagram of a correspondence relationship between a fusion weight W _blending and a Δd according to an embodiment of the present application;

FIG. 5 is a flowchart of an image color management method according to an embodiment of the present application;

FIG. 6 is a flowchart of an image scaling method according to an embodiment of the present application;

FIG. 7A is a schematic diagram of an image display enhancement apparatus according to an embodiment of the present application;

FIG. 7B is a schematic diagram of an image display enhancement method according to an embodiment of the present application.

detailed description

The present invention will be further described in detail with reference to the accompanying drawings, in which FIG. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without departing from the inventive scope are the scope of the present application.

The embodiment of the present invention provides an image display enhancement method and device, which are used to solve the problem that the image display effect existing in the prior art is poor. The method and the device are based on the same inventive concept. Since the principles of the method and the device for solving the problem are similar, the implementation of the device and the method can be referred to each other, and the repeated description is not repeated.

The image display enhancement scheme of the embodiments of the present application may be implemented using an electronic device that can be used for display, including but not limited to a personal computer, a server computer, a handheld or laptop device, a mobile device (such as a mobile phone, a tablet). Computers, personal digital assistants, media players, etc.), consumer electronics, small computers, mainframe computers, and more. However, the electronic device is preferably an intelligent mobile terminal. The solution provided by the embodiment of the present application is specifically described below by taking an intelligent mobile terminal as an example.

Referring to FIG. 1 , it is a schematic diagram of a hardware structure of a terminal applied to an embodiment of the present application. As shown in FIG. 1, the terminal 100 includes a display device 110, a processor 120, and a memory 130. The memory 130 can be used to store software programs and data, and the processor 120 runs the software stored in the memory 130. Programs and data, thereby performing various functional applications and data processing of the terminal 100. The memory 130 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application required for at least one function (such as an image display enhancement function, etc.), and the like; the storage data area may be stored according to the terminal 100. Use the created data (such as audio data, phone book, 3D lookup table, etc.). Moreover, memory 130 can include high speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. The processor 120 is a control center of the terminal 100, and connects various parts of the entire terminal by various interfaces and lines, and executes various functions and processing data of the terminal 100 by running or executing software programs and/or data stored in the memory 130. Therefore, the terminal is monitored as a whole. The processor 120 may include one or more general-purpose processors, and may also include one or more digital signal processors (English: Digital Signal Processor, DSP for short) for performing related operations to implement the embodiments of the present application. Technical solution.

The terminal 100 may further include an input device 140 for receiving input digital information, character information or contact touch/contactless gestures, and generating signal inputs related to user settings and function control of the terminal 100, and the like. Specifically, in the embodiment of the present application, the input device 140 may include a touch panel 141. The touch panel 141, also referred to as a touch screen, can collect touch operations on or near the user (such as the user's operation on the touch panel 141 or on the touch panel 141 using any suitable object or accessory such as a finger, a stylus, or the like. ), and drive the corresponding connection device according to a preset program. Optionally, the touch panel 141 may include two parts: a touch detection device and a touch controller. Wherein, the touch detection device detects the touch orientation of the user, and detects a signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts the touch information into contact coordinates, and sends the touch information. The processor 120 is provided and can receive commands from the processor 120 and execute them. For example, the user clicks an image thumbnail on the touch panel 141 with a finger, and the touch detection device detects the signal brought by the click, and then transmits the signal to the touch controller, and the touch controller then applies the signal. The signal is converted into coordinates and sent to the processor 120. The processor 120 determines an operation performed on the image (such as image enlargement, full-screen display of the image) according to the coordinates and the type of the signal (click or double-click), and then determines to execute the The memory space required for the operation, if it needs to occupy the memory If the space is smaller than the free memory, the enlarged image is displayed on the display panel 111 included in the display device in full screen, thereby realizing image display.

The touch panel 141 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 141, the input device 140 may further include other input devices 142, which may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, switch buttons, etc.), trackballs, mice, joysticks, and the like. One or more of them.

The display device 110 includes a display panel 111 for displaying information input by the user or information provided to the user, and various menu interfaces of the terminal device 100, etc., which are mainly used for displaying images in the terminal 100 in the embodiment of the present application. Optionally, the display panel can be configured by using a liquid crystal display (English: Liquid Crystal Display, LCD for short) or OLED (Organic Light-Emitting Diode). In some other embodiments, the touch panel 141 can cover the display panel 111 to form a touch display screen.

In addition to the above, the terminal 100 may further include a power source 150 for powering other modules and a camera 160 for taking photos or videos. Terminal 100 may also include one or more sensors 170, such as acceleration sensors, light sensors, and the like. The terminal 100 may further include a radio frequency (RF) circuit 180 for performing network communication with the wireless network device, and may further include a WiFi module 190 for performing WiFi communication with other devices.

The embodiment of the present application is applied to an Android system architecture. The following is a brief description of the Android (android) system architecture.

The Android system architecture includes: an application layer (APP), a framework layer, a core library (lib/HAL) layer, a driver layer, and a hardware layer. The driver layer includes a CPU driver, a GPU driver, a display controller driver, and the like. The core library layer is the core part of the Android system, including input/output services, core services, graphics device interfaces, and graphics engine (Graphics Engine) for CPU or GPU graphics processing. The graphics engine may include a 2D engine (such as Skia), a 3D engine, a composition, a Frame Buffer, an EGL (Embedded-System Graphics Library), etc., where EGL is a rendering API and an underlying native platform window. The interface between the systems, the API refers to the Application Programming Interface. Frame layer Including graphics services (Graphic Service), system services (System service), rendering services (SurfaceFlinger), etc.; graphics services, such as widgets, canvases (Canvas) and views (Views). The application layer may include a gallery, a media player (Media Player), a browser, and the like. The hardware layer may include a central processing unit (CPU) and a graphics processing unit (GPU) (corresponding to a specific implementation of the processor 150 in FIG. 1), and may also include a memory (equivalent to a figure). The memory 130 in 1 includes memory and external memory, and may further include an input device (corresponding to the input device 140 in FIG. 1), a display device (corresponding to the display device 110 in FIG. 1), and may also include one or more Sensor 456 (corresponding to sensor 170 in Figure 1). Of course, in addition, the hardware layer 450 may also include the power source, camera, RF circuit, and WiFi module shown in FIG. 1, and may also include other hardware modules not shown in FIG. 1, such as a memory controller and display control. And so on.

In the prior art, when a user browses an image using an application (gallery or WeChat, etc.), the file stream is sent to the frame layer, and the frame layer parses the image information and then calls the core library layer to decode the image data to create a bitmap file ( The bitmap object is returned to the application, so that the application sends the bitmap object to the hardware layer for rendering. As can be seen from the above, before the image is displayed by the hardware layer, the image is not processed, resulting in poor display of the image.

Based on this, the image display enhancement method provided by the embodiment of the present application may be implemented in the storage software program shown in FIG. 1 , and may be specifically executed by the processor 120 . Specifically, as shown in FIG. 2, the image display enhancement method provided by the embodiment of the present application includes:

S210: Obtain scene information of an image to be displayed, where the scene information of the image to be displayed is obtained by performing scene analysis on the image to be displayed.

The scene information can be obtained by performing scene analysis on the image to be displayed by using an existing scene recognition algorithm. Scene information may include, but is not limited to, a blue sky scene, a green plant scene, a backlight scene, a night scene, and the like. In addition, the scene information may be written to the image to be displayed when being photographed by the camera, so that the scene information is obtained by analyzing the image information included in the image to be displayed. When the scene information is written into the acquired image, the scene information may be written into the EXIF data area of the exchangeable image file (English: Exchangeable Image File, EXIF). The vendor's comment (MakerNote) field in the field. The scene information may be included in the metadata Metadata of the image to be displayed, or in the exchangeable image file EXIF data area, or in the vendor comment makernotes field. The scene information includes at least one of the following: a sensitivity ISO value, an aperture value, a shutter time, or an exposure EV value. Specifically, the scene information may be included in a vendor note makernotes field of the exchangeable image file EXIF data area of the image to be displayed, or may be included in other fields of the exchangeable image file EXIF data area.

S220: Perform at least one of the following processes on the image to be displayed based on the scenario information: gamut mapping (English: Gamut Mapping, GMP for short), color management (English: Adaptive Color Management, ACM), contrast enhancement (English: Adaptive Contrast Enhancement, referred to as: ACE), sharpening, scaling.

S230. Display the processed image to be displayed.

In the embodiment of the present application, before the image is displayed, the image is subjected to gamut mapping, color management, contrast enhancement, sharpening, and scaling processing based on the scene information of the image to be displayed, and the android color difference is generated when the Android native system uses the application to browse the image. , aliasing, unclear, low contrast and other display effects. In addition, in the prior art, the processing of the image data stream depends on a hardware device such as a chip or a screen driving circuit to enhance the display effect of the image, but is limited by the image enhancement algorithm that can be executed by the hardware, and the display effect is poor, and the present application implements The example is not limited to hardware devices, thereby solving the problem in the prior art that the hardware-based display effect enhancement method is limited by the algorithm flexibility and the display effect is poor, thereby improving the display effect of the image.

Optionally, when performing the gamut mapping on the to-be-displayed image based on the scenario information, the method may be implemented as follows:

S310, determining a first three-dimensional lookup table corresponding to the scene information, and a second three-dimensional lookup table, where the first three-dimensional lookup table is configured to maintain color gamut mapping and color gamut on the color to be protected in the image to be displayed. The second three-dimensional lookup table is used to adjust the color that is not required to be protected in the image to be displayed.

Among them, the color to be protected includes at least one of the following: skin color, gray color, and memory color.

The gray color is the corresponding color when the red (R) value, the green (G) value, and the blue (B) value are equal. Color, reflecting the different brightness conditions from black to white. The gray is protected during the adjustment process to avoid the relationship of R=G=B in the adjustment process, thereby avoiding the appearance of color in the area where the image is originally grayscale, and forming a wrong color expression.

In addition, people have a deep memory of the understanding of certain colors in long-term practice, so the understanding of these colors has a certain regularity and forms an inherent habit. Such colors are called memory colors. The memory color is protected during the adjustment process, avoiding the destruction of people's understanding of the inherent color of certain objects. For example, people's understanding of grassland is green.

S320, determining that a color of an i-th pixel point among the N pixel points included in the image to be displayed corresponds to a first color value in the first three-dimensional lookup table, and determining a color corresponding to the i-th pixel point a second color value in the second three-dimensional lookup table.

S330. The first color value and the second color value are fused according to the predetermined fusion weight to obtain a color value of the ith pixel point after the gamut mapping.

The i takes a positive integer not greater than N.

The gamut mapping algorithm of the present application realizes the distinguishing process of the adjustment color and the protection color based on the double three-dimensional lookup table, and can realize the gamut mapping and the color chromaticity correction while protecting the skin color, the gray color, the memory color, and ensuring the transition from the adjustment color to the protection color. No false contours appear.

The first three-dimensional lookup table may include color values corresponding to any input color. The first three-dimensional lookup table (3dlut1) may also be calculated after performing M x M×M node partitioning on the color space. In the three dimensions, each of the two dimensions is divided into M nodes, and the partitioning may be a uniform or a non-uniform partitioning, which is not specifically limited in this embodiment. The color space can be red, green and blue (RGB) space, or hue saturation brightness (HSV) space, or brightness blue red (YCbCr) space, or Lab space, L for Luminosity, and a for magenta to green. The range, b represents the range from yellow to blue, and the like. At this time, the first three-dimensional lookup table may include only the input/output correspondence of the colors at the nodes. The input-output correspondence of colors at non-nodes can be generated using a three-dimensional interpolation algorithm based on the color of the nodes. The three-dimensional interpolation algorithm may be a trilinear interpolation method, a tetrahedral interpolation method, a pyramid method or a prism interpolation method, or the like.

The first three-dimensional lookup table may include color values corresponding to any of the input colors. Second three-dimensional investigation The lookup table (3dlut2) can also be calculated after dividing the color space by N×N×N nodes. In the three dimensions, the N-th node is divided into a plurality of nodes, and the partitioning may be a uniform partitioning or a non-uniform partitioning, which is not specifically limited in this embodiment. The color space may be an RGB space, or an HSV space, or a YCbCr space, or a Lab space. At this time, the second three-dimensional lookup table may include only the input/output correspondence of the colors at the nodes. The input-output correspondence of colors at non-nodes can be generated using a three-dimensional interpolation algorithm based on the color of the nodes. The three-dimensional interpolation algorithm may be a trilinear interpolation method, a tetrahedral interpolation method, a pyramid method or a prism interpolation method, or the like.

On the basis of the implementation of performing the gamut mapping on the image to be displayed based on the scene information, when the color corresponding to the color of the ith pixel is not included in the first three-dimensional lookup table, Obtaining a color corresponding color value of the i-th pixel point based on interpolating a color included in the first three-dimensional lookup table by a three-dimensional interpolation algorithm; or

When the color corresponding color value of the i-th pixel point is not included in the second three-dimensional lookup table, the i-th pixel point is obtained by interpolating colors included in the second three-dimensional lookup table by using a three-dimensional interpolation algorithm The color corresponds to the color value.

When the first color value and the second color value are fused to obtain the color value of the ith pixel point after the gamut mapping based on the predetermined fusion weight, the fusion weight may be determined by the following formula:

Wherein, W _blending represents the fusion weight, T _blending represents a threshold for controlling smooth transition from the adjusted color to the protective color, and Δd represents that the color of the i-th pixel corresponds to the color node in the three-dimensional color space and is determined by the first three-dimensional Find the shortest distance between the convex hups formed by the color nodes in the three-dimensional color space; Δd<0 indicates that the color node of the ith pixel is located inside the convex hull; Δd=0 indicates the ith pixel The color node is located on the convex hull; Δd>0 indicates that the color node of the ith pixel is located outside the convex hull. Among them, the fusion weights W _blending and Δd satisfy the relationship shown in FIG. 4 .

In a possible implementation manner, when the first color value and the second color value are fused based on the predetermined fusion weight to obtain a color value of the ith pixel point after the gamut mapping, the method is determined as follows The color value of the i-th pixel after the gamut mapping:

A _out =A ₁ *W _blending +A ₂ *(1-W _blending );

Wherein A _out represents a color value of the i-th pixel point after gamut mapping, A ₁ represents the first color value, A ₂ represents the second color value, and W _blending represents the fusion weight.

Take the RGB color space as an example. Take the RGB color space as an example for illustration. Rin, Gin, and Bin represent the color of the i-th pixel. R1, G1, and B1 are the output results of Rin, Gin, and Bin through the first three-dimensional lookup table 3dlut1, and R2, G2, and B2 are the output results of Rin, Gin, and Bin through the second three-dimensional lookup table 3dlut2, and Δd is Rin, Gin, The Bin node distance is the shortest distance from the convex hull composed of multiple nodes included in 3lut1, which can be calculated by the spatial geometric relationship. When Δd<0, (Rin, Gin, Bin) is located inside the convex hull; when Δd=0, (Rin, Gin, Bin) is located on the convex hull; when Δd>0, (Rin, Gin, Bin) is located outside the convex hull .

Specifically, the three-channel values Rout, Gout, and Bout output after the gamut mapping are determined by the fusion weight W _blending and R1, G1, B1, R2, G2, and B2:

In the embodiment of the present application, when the color management is performed on the image to be displayed based on the scene information, the method may be implemented as follows:

S510. Acquire an adjustment strategy for adjusting pixel saturation of the image to be displayed corresponding to the scene information.

Different scene information corresponding to different adjustment strategies, such as the existence of blue sky, natural plants and other scenes, can improve the saturation to optimize the display effect; for example, scenes with skin color, etc., may reduce the saturation and make the skin smoother. Among them, the adjustment strategy corresponding to different scene information It can be pre-configured in an electronic device.

The brightness saturation hue YSH value of the j-th pixel in the image to be displayed is adjusted in the manner described in S520 to S530, respectively, and the j is taken over a positive integer not greater than N. The other pixel point YSH values among the N pixel points in the image to be displayed are adjusted according to the adjustment method of the jth pixel point.

S520: Keep the h-th value of the jth pixel point unchanged, and adjust the saturation S value of the j-th pixel point based on the adjustment strategy.

S530. Input the S value of the jth pixel point before adjustment, the adjusted S value, and the brightness Y of the jth pixel point into a quadratic equation corresponding to the H value of the predetermined jth pixel point. The function obtains the compensated luminance Y value of the jth pixel.

The color management method provided in the embodiment of the present application can solve the problem of brightness change and brightness display change caused by brightness change based on the YSH color space, and the method is simple.

Optionally, the quadratic function corresponding to the H value of the jth pixel point satisfies the condition described in the following formula:

Wherein, Y _out represents the j-th output luminance values of pixels, Y _in represents the input luminance value of the j-th pixel point, ΔY represents the luminance compensation value, a1, a2, a3, a4 , a5, A6 represents a quadratic function function coefficient, S _in represents a saturation value of the j-th pixel point before adjustment, and S _out represents a saturation value of the adjusted j-th pixel point.

In the embodiment of the present application, each color tone in the YSH space may be configured to correspond to a quadratic function. The YSH color space can also be divided to obtain p primary colors (primary hue), each primary color corresponding to a set of equation parameters, the set of equation parameters constitute a quadratic function, the quadratic function of the hue between the main tones The parameters are generated using parameter interpolation of the quadratic function corresponding to the nearest two main tones.

When a quadratic function corresponding to each dominant color is determined in advance, for a primary color A of p primary tones, for a partial saturation adjustment range, such as when adjusting saturation from S _in to S _out , where Δ Y luminance compensation The value can be obtained by calibrating the brightness experiment. Therefore, according to the ΔY and the input saturation S _in and the output saturation S _out obtained by the calibration experiment, the system parameters a1, a2, a3, a4, a5, and a6 obtained by the least square method are fitted to obtain the corresponding main tone A. Quadratic function.

Based on this, in the case where there is a quadratic function corresponding to the H value of the jth pixel, the a1, a2, a3, a4, a5, a6 are respectively used in advance before the jth before adjustment The saturation value of the pixel, the adjusted saturation value of the jth pixel point, and the brightness compensation value are obtained based on a least squares fitting algorithm.

In the case where there is no quadratic function corresponding to the H value of the jth pixel, the a1, a2, a3, a4, a5, a6 are closest to the H value of the jth pixel The two tonal values are respectively obtained by interpolating the coefficients of the quadratic function corresponding to each.

Specifically, in the case where there is no quadratic function corresponding to the H value of the jth pixel point, the a1, a2, a3, a4, a5, and a6 are obtained by the following formula:

a _k represents the kth coefficient in a1, a2, a3, a4, a5, a6; H represents the saturation value of the jth pixel point, and H _p1 and H _p2 represent the nearest neighbor of the H value of the jth pixel point The two tonal values, H _p1 <H<H _p2 , a _{(p1, k)} represents the kth coefficient of the quadratic function corresponding to H _p1 , and a _{(p2, k)} represents the quadratic function corresponding to H _p2 The kth coefficient.

Optionally, in the embodiment of the present application, performing scaling on the image to be displayed based on the scenario information may be implemented in the following manner, as shown in FIG. 6 :

S610. Determine a zoom factor of the image to be displayed, and determine a highest frequency of the periodic pattern in the image to be displayed according to the scene information. The highest frequency of the periodic pattern in the image corresponding to different scene information is different.

The periodic pattern in the image to be displayed refers to a pattern having a preset regular arrangement in the image, such as an arrangement pattern of railings in daily life, an arrangement pattern of tiles on the roof, and the like.

The highest frequency of the periodic pattern refers to transforming the image to be displayed containing the periodic pattern into the frequency domain After that, the periodic pattern content corresponds to the highest frequency in the frequency.

The determining, according to the scenario information, a highest frequency of the periodic pattern in the image to be displayed, including:

Determining, according to the scene information, that the image to be displayed includes a periodic pattern;

Performing frequency domain transformation on the image to be displayed, and acquiring a highest frequency among frequencies corresponding to the periodic pattern after frequency domain transformation.

S620, determining, according to a preset lookup table, the zoom factor and a number of taps of a bicubic filter corresponding to a highest frequency of a periodic pattern in the image to be displayed; and, according to the zoom factor, a period in the image to be displayed The highest frequency of the pattern determines the coefficient of the cubic function corresponding to the bicubic filter.

The preset lookup table can be generated as follows:

First, the correspondence between the scaling factor and the number of taps of the first filter is determined, which can be specifically expressed by the following formula:

Tap1=1+(SizeFactor*srcLen+dstLen-1)/dstLen;

Where tap1 represents the number of first filter taps, srcLen represents the resolution of the original image, and dstLen represents the resolution of the target image.

Indicates the zoom factor, and SizeFactor represents the preset zoom factor, which can be an empirical value. Then, the first filter tap number tap1 is finely adjusted based on the highest frequency f _{max of the} image to obtain the second filter tap number tap2. Specifically, tap2=tap1/(FreqFactor*f _max ), and FreqFactor represents a preset frequency coefficient.

S630. Scale the image to be displayed according to the number of taps and the determined cubic function.

It should be noted that the number of taps is also the number of sampled pixels. Specifically, searching for a corresponding pixel position of the target image dst pixel point on the source image src, adaptively selecting the determined number of taps around the corresponding pixel point tap2 sampling pixel points (reference points), and calculating according to the bicubic interpolation function The weight value corresponding to each sampled pixel point is weighted and summed according to the weight value of tap2 sampled pixel points and the pixel value of tap2 sampled pixel points, and the pixel value of the scaled image is obtained by weighting, and finally obtained after scaling Target image.

If the scaling is performed only according to the image scaling factor without considering the highest frequency in the image content, the cutoff frequency of the filter and the highest frequency of the image content do not necessarily match, and the image content corresponding to the highest frequency may be aliased or blurred. . Therefore, according to the scaling method provided by the embodiment of the present application, adaptive anti-aliasing can be performed based on the image scene information and the highest frequency of the image, thereby improving the image optimization effect.

Optionally, the cubic function corresponding to the bicubic filter satisfies the following formula:

Where x is an input variable representing the distance between the reference point and the center, k(x) represents the weight value corresponding to the reference point, and dB and dC respectively represent the coefficients of the cubic equation function; B and C are constants. The center described here is the corresponding pixel point position of the target image dst pixel on the source image src.

Optionally, determining a coefficient of a cubic function corresponding to the bicubic filter according to a highest frequency of the periodic pattern in the image to be displayed, which may be specifically implemented as follows:

dB=g*f _max *dC;

Where g is a constant and f _max represents the highest frequency of the image.

Based on the same inventive concept as the method embodiment, the embodiment of the present application further provides an image display enhancement device, which is applied to an electronic device, and is specifically applied to the terminal 100, and is implemented by the processor 120 in the terminal 100. As shown in 7A, the device includes:

a display enhancement module 710, configured to acquire scene information of an image to be displayed, where scene information of the image to be displayed is obtained by performing scene analysis on the image to be displayed; and based on the scene information Performing at least one of the following processes on the image to be displayed: gamut mapping, color management, contrast enhancement, sharpening, or scaling.

The display module 720 is configured to display the image to be displayed processed by the display enhancement module 710.

Optionally, the display enhancement module 710 may further include a gamut mapping module 711, a color management module 712, and a scaling module 713.

Optionally, when the gamut mapping is performed on the to-be-displayed image based on the scenario information, the gamut mapping module 711 included by the display enhancement module 710 is configured to perform the gamut mapping module 711 according to the scenario information. When the display image performs color management, it may be specifically executed by the color management module 712; when performing scaling on the image to be displayed based on the scene information, the zooming module 713 may be specifically executed.

When the image display enhancement device is applied to the scene in which the image is viewed by the application, the Android system is used as an example. In the case that the display enhancement module 710 and the display module 720 are included in the Android system framework of the scene, the method further includes, as shown in FIG. 7B. Show: application module 730. When the image signal of the image to be displayed is sent to the application module 730 for browsing, the application module 730 first sends the image data stream of the image to be displayed to the scene information analysis module 740; the scene information analysis module 740 analyzes the image data stream. Processing the scene information of the image to be displayed. The scene information included in the image to be displayed may be written in the image to be displayed when the image is captured by the camera. When the scene information is written into the acquired image, the scene information may be written into a MakerNote field of an EXIF data area of an exchangeable image file (English: Exchangeable Image File, EXIF) of the image. in. In addition, the scene information parsing module 740 may extract the scene information of the image based on the scene classification algorithm, and the scene classification algorithm may adopt an algorithm disclosed in the prior art, which is not specifically limited in the embodiment of the present application.

The scene information parsing module 740, after the scene information is obtained from the image to be displayed, sends the scene information to the display enhancement module 710. The scene information parsing module 740 can also send the scene information to the application module 730, so that the application module 730 receives the scene information. The image data stream is then sent to the display enhancement module 710, so that the display enhancement module 710 receives the from the scene information parsing module 740. After the scene information and the image data stream sent by the application module 730, the image data stream is subjected to display enhancement processing based on the scene information, including but not limited to gamut mapping, color management, contrast enhancement, sharpening, zooming, etc., display enhancement Module 710 sends the processed image data stream to display module 720 for display. After the scene information analysis module 740 obtains the scene information from the image to be displayed, the scene information and the image data stream of the image to be displayed are sent to the display enhancement module 710, so that the display enhancement module 710 displays the image data stream based on the scene information. Enhanced processing.

The application module 730 can be, but is not limited to, an image browsing application, a social application, an online shopping application, and the like.

The apparatus may further include: an image data decoding module. Specifically, the application module 730 sends the image data stream of the image to be displayed to the scene information parsing module 740, and then sends the image data stream to the image data decoding module to decode the image data stream, and sends the decoded image data to the scene information. Parsing module 740. When decoding the image data, the image data decoding module may specifically call the skia library to decode the image data. The Skia library is a 2D (2-dimensional) vector image processing function library. The fonts, coordinate transformations, and bitmaps contained in the image are all efficiently and concisely expressed by calling the Skia library.

Optionally, when the gamut mapping is performed on the image to be displayed based on the scenario information, the display enhancement module 710 is specifically configured to:

Determining a first three-dimensional lookup table corresponding to the scene information, and a second three-dimensional lookup table, where the first three-dimensional lookup table is configured to maintain color gamut mapping and color gamut mapping before the color to be protected in the image to be displayed Similarly, the second three-dimensional lookup table is configured to adjust a color that is not required to be protected in the image to be displayed; and determine that a color of an i-th pixel point among the N pixels included in the image to be displayed corresponds to the first a first color value in a three-dimensional lookup table, and determining that a color of the i-th pixel point corresponds to a second color value in the second three-dimensional lookup table; the first color is based on a predetermined fusion weight The value and the second color value are fused to obtain a color value of the ith pixel point after the gamut mapping, and the i takes a positive integer not greater than N.

Optionally, the display enhancement module 710 is further configured to:

When the color corresponding to the color of the ith pixel point is not included in the first three-dimensional lookup table, Obtaining a color corresponding color value of the i-th pixel point based on interpolating a color included in the first three-dimensional lookup table by a three-dimensional interpolation algorithm; or not including the i-th pixel point in the second three-dimensional lookup table When the color corresponds to the color value, the color corresponding color value of the i-th pixel point is obtained based on interpolating the color included in the second three-dimensional lookup table by the three-dimensional interpolation algorithm.

The display enhancement module 710 is further configured to determine the fusion weight by using the following formula:

Wherein, W _blending represents the fusion weight, T _blending represents a threshold for controlling smooth transition from the adjusted color to the protective color, and Δd represents that the color of the i-th pixel corresponds to the color node in the three-dimensional color space and is determined by the first three-dimensional Find the shortest distance between the convex hups formed by the color nodes in the three-dimensional color space; Δd<0 indicates that the color node of the ith pixel is located inside the convex hull; Δd=0 indicates the ith pixel The color node is located on the convex hull; Δd>0 indicates that the color node of the ith pixel is located outside the convex hull.

The display enhancement module 710 is configured to: when the first color value and the second color value are fused to obtain the color value of the ith pixel point after the gamut mapping based on the predetermined fusion weight, specifically:

The color value of the ith pixel point after the gamut mapping is determined by:

A _out =A ₁ *W _blending +A ₂ *(1-W _blending );

Wherein A _out represents a color value of the i-th pixel point after gamut mapping, A ₁ represents the first color value, A ₂ represents the second color value, and W _blending represents the fusion weight.

The display enhancement module 710 is specifically configured to: when performing color management on the image to be displayed based on the scenario information:

Acquiring an adjustment strategy for adjusting the saturation of the pixel point of the image to be displayed corresponding to the scene information; and adjusting the brightness saturation tone YSH value of the jth pixel in the image to be displayed as follows: maintaining the jth The pixel H-value of the pixel is unchanged, and the j-th adjustment is adjusted based on the adjustment strategy a saturation S value of the pixel; the S value of the jth pixel before the adjustment, the adjusted S value, and the brightness Y of the jth pixel are input to the predetermined jth pixel The quadratic function corresponding to the H value obtains the compensated luminance Y value of the jth pixel; the j takes a positive integer not greater than N.

Wherein, the quadratic function corresponding to the H value of the jth pixel point satisfies the condition described in the following formula:

Wherein, Y _out represents the j-th output luminance values of pixels, Y _in represents the input luminance value of the j-th pixel point, ΔY represents the luminance compensation value, a1, a2, a3, a4 , a5, A6 represents a quadratic function function coefficient, S _in represents a saturation value of the j-th pixel point before adjustment, and S _out represents a saturation value of the adjusted j-th pixel point.

The display enhancement module 710 is further configured to determine the quadratic function function coefficients by:

In the case where there is a quadratic function corresponding to the H value of the jth pixel, the a1, a2, a3, a4, a5, a6 are respectively used in advance using the jth pixel before adjustment a saturation value, an adjusted saturation value of the jth pixel point, and the brightness compensation value, which are obtained based on a least squares fitting algorithm; or

In the case where there is no quadratic function corresponding to the H value of the jth pixel, the a1, a2, a3, a4, a5, a6 are closest to the H value of the jth pixel The two tonal values are respectively obtained by interpolating the coefficients of the quadratic function corresponding to each.

In the case where there is no quadratic function corresponding to the H value of the jth pixel, the display enhancement module 710 is specifically configured to determine a1, a2, a3, a4, a5, a6 by the following formula:

a _k denotes the kth coefficient in a1, a2, a3, a4, a5, a6; H denotes the saturation value of the jth pixel point, and H _p1 and H _p2 denote the nearest neighbor of the H value of the jth pixel point The two tonal values, H _p1 <H<H _p2 , a _{(p1, k)} represents the kth coefficient of the quadratic function corresponding to H _p1 , and a _{(p2, k)} represents the quadratic function corresponding to H _p2 The kth coefficient.

Optionally, the display enhancement module 710 is specifically configured to: when performing scaling on the image to be displayed based on the scene information,

Determining a zoom factor of the image to be displayed, and determining a highest frequency of the periodic pattern in the image to be displayed according to the scene information; determining the zoom factor and periodicity in the image to be displayed according to a preset lookup table a number of taps of the bicubic filter corresponding to the highest frequency of the pattern; and determining a coefficient of a cubic function corresponding to the bicubic filter according to the scaling factor and a highest frequency of the periodic pattern in the image to be displayed; according to the tap The number and the determined cubic function function scale the image to be displayed.

Wherein, the cubic function corresponding to the bicubic filter satisfies the condition shown by the following formula:

Where x represents the input variable, k(x) represents the weight value corresponding to the input variable x, and dB and dC represent the coefficients of the cubic function, respectively; B and C are constants.

It should be noted that the division of the module in the embodiment of the present application is schematic, and is only a logical function division, and the actual implementation may have another division manner. The functional modules in the embodiments of the present application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules.

The integrated module is implemented in the form of a software functional module and sold as a standalone product Or when used, it can be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application, in essence or the contribution to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium. Including a number of instructions for causing a computer device (which may be a personal computer, a terminal device, etc.) or a processor (such as processor 120 shown in FIG. 1) to perform all or part of the methods described in various embodiments of the present application. step. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like. .

An embodiment of the present application further provides an image gamut mapping method, where the method includes:

Obtaining a to-be-processed image; determining, by the image of the to-be-processed image, a color of an i-th pixel point corresponding to a first color value in the first three-dimensional lookup table, and determining the i-th pixel point a color corresponding to the second color value in the second three-dimensional lookup table; the first three-dimensional lookup table is configured to keep the color gamut mapping of the color to be protected in the image to be processed after being the same as before the gamut mapping The second three-dimensional lookup table is configured to adjust a color that is not protected in the image to be processed; and the first color value and the second color value are merged according to the predetermined fusion weight to obtain a gamut-mapped a color value of the i-th pixel, the i taking a positive integer not greater than N.

The gamut mapping method of the present application is based on a dual three-dimensional lookup table to realize the distinguishing process of the adjustment color and the protection color, and can realize the gamut mapping and the color chromaticity correction while protecting the skin color, the gray color, the memory color, and ensuring the transition from the adjustment color to the protection color. No false contours appear.

In one possible design, the method further includes:

When the color corresponding color value of the i-th pixel point is not included in the first three-dimensional lookup table, the i-th pixel point is obtained by interpolating colors included in the first three-dimensional lookup table by using a three-dimensional interpolation algorithm. The color corresponds to the color value; or

When the color corresponding color value of the i-th pixel point is not included in the second three-dimensional lookup table, the i-th pixel point is obtained by interpolating colors included in the second three-dimensional lookup table by using a three-dimensional interpolation algorithm The color corresponds to the color value.

In one possible design, the fusion weight is determined by the following formula:

Wherein, W _blending represents the fusion weight, T _blending represents a threshold for controlling smooth transition from the adjusted color to the protective color, and Δd represents that the color of the i-th pixel corresponds to the color node in the three-dimensional color space and is determined by the first three-dimensional Find the shortest distance between the convex hups formed by the color nodes in the three-dimensional color space; Δd<0 indicates that the color node of the ith pixel is located inside the convex hull; Δd=0 indicates the ith pixel The color node is located on the convex hull; Δd>0 indicates that the color node of the ith pixel is located outside the convex hull.

In a possible design, the first color value and the second color value are fused based on the predetermined fusion weight to obtain the color value of the ith pixel point after the gamut mapping, including:

The color value of the ith pixel point after the gamut mapping is determined by:

A _out =A ₁ *W _blending +A ₂ *(1-W _blending );

Wherein A _out represents a color value of the i-th pixel point after gamut mapping, A ₁ represents the first color value, A ₂ represents the second color value, and W _blending represents the fusion weight.

The embodiment of the present application further provides a color management method, which may be implemented by an electronic device. The method includes:

Obtaining an image to be processed, and determining an adjustment strategy for adjusting pixel saturation of the image to be processed by the processor;

The brightness saturation hue YSH values of the jth pixel in the image to be processed are respectively adjusted as follows:

Keeping the jth pixel dot hue H value unchanged, adjusting the saturation S value of the jth pixel point based on the adjustment strategy;

Obtaining a quadratic function function corresponding to the S value of the jth pixel point before adjustment, the adjusted S value, and the brightness Y of the jth pixel point corresponding to the H value of the predetermined jth pixel point The compensated luminance Y value of the jth pixel; the j takes a positive integer not greater than N.

The color management method provided in the embodiment of the present application can solve the problem of brightness change and brightness display change caused by brightness change based on the YSH color space, and the method is simple.

In a possible design, the quadratic function corresponding to the H value of the jth pixel point satisfies the condition described in the following formula:

Wherein, Y _out represents the j-th output luminance values of pixels, Y _in represents the input luminance value of the j-th pixel point, ΔY represents the luminance compensation value, a1, a2, a3, a4 , a5, A6 represents a quadratic function function coefficient, S _in represents a saturation value of the j-th pixel point before adjustment, and S _out represents a saturation value of the adjusted j-th pixel point.

In one possible design, the quadratic function coefficients are determined as follows:

In the case where there is a quadratic function corresponding to the H value of the jth pixel, the a1, a2, a3, a4, a5, a6 are respectively used in advance using the jth pixel before adjustment a saturation value, an adjusted saturation value of the jth pixel point, and the brightness compensation value, which are obtained based on a least squares fitting algorithm; or

In the case where there is no quadratic function corresponding to the H value of the jth pixel, the a1, a2, a3, a4, a5, a6 are closest to the H value of the jth pixel The two tonal values are respectively obtained by interpolating the coefficients of the quadratic function corresponding to each.

In one possible design, in the case where there is no quadratic function corresponding to the H value of the jth pixel, the a1, a2, a3, a4, a5, a6 are obtained by the following formula:

a _k represents the kth coefficient in a1, a2, a3, a4, a5, a6; H represents the saturation value of the jth pixel point, and H _p1 and H _p2 represent the nearest neighbor of the H value of the jth pixel point The two tonal values, H _p1 <H<H _p2 , a _{(p1, k)} represents the kth coefficient of the quadratic function corresponding to H _p1 , and a _{(p2, k)} represents the quadratic function corresponding to H _p2 The kth coefficient.

The embodiment of the present application further provides an image scaling method, which may be implemented by an electronic device, and the method includes:

Determining a zoom factor of the image to be processed and a highest frequency of the image to be processed;

Determining, according to a preset lookup table, the zoom factor and the number of taps of the bicubic filter corresponding to the highest frequency of the image to be processed; and determining a cubic function corresponding to the bicubic filter according to the highest frequency of the image to be processed Coefficient of

The image to be processed is scaled according to the number of taps and the determined cubic function.

The scaling method provided by the embodiment of the present application can perform adaptive anti-aliasing based on image scene information and the highest frequency of the image, thereby improving the image optimization effect.

In one possible design, the cubic function corresponding to the bicubic filter satisfies the conditions shown in the following formula:

Where x represents the position of the sampled pixel selected based on the number of taps, k(x) represents the weight value corresponding to the sampled pixel point, and dB and dC represent the coefficients of the cubic function, respectively; B and C are constants.

Those skilled in the art will appreciate that embodiments of the present application can be provided as a method, system, or computer program product. Thus, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment in combination of software and hardware. Moreover, the application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (system), and computer program products according to embodiments of the present application. It will be understood that each flow and/or block of the flowchart illustrations and/or FIG. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine for the execution of instructions for execution by a processor of a computer or other programmable data processing device. Means for implementing the functions specified in one or more of the flow or in a block or blocks of the flow chart.

The computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device. The apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.

These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device. The instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

While the preferred embodiment of the present application has been described, those skilled in the art can make further changes and modifications to these embodiments once they are aware of the basic inventive concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and the modifications and

It is apparent that those skilled in the art can make various changes and modifications to the embodiments of the present application without departing from the spirit and scope of the embodiments of the present application. Thus, it is intended that the present invention cover the modifications and variations of the embodiments of the present invention.

Claims (39)

1. An image display enhancement method, comprising:

   Obtaining scene information of the image to be displayed, where the scene information of the image to be displayed is obtained by performing scene analysis on the image to be displayed;

   Performing at least one of the following processes on the image to be displayed based on the scene information: gamut mapping, color management, contrast enhancement, sharpening, or scaling;

   The processed image to be displayed is displayed.
2. The method of claim 1, wherein the scene information is included in metadata Metadata of the image to be displayed, or in an exchangeable image file EXIF data area, or in a vendor note makernotes field.
3. The method according to claim 1 or 2, wherein the scene information comprises at least one of the following: sensitivity ISO value, aperture value, shutter time or exposure EV value.
4. The method according to any one of claims 1 to 3, wherein the performing gamut mapping on the image to be displayed based on the scene information comprises:

   Determining a first three-dimensional lookup table corresponding to the scene information, and a second three-dimensional lookup table, where the first three-dimensional lookup table is configured to maintain color gamut mapping and color gamut mapping before the color to be protected in the image to be displayed Similarly, the second three-dimensional lookup table is configured to adjust a color that is not protected in the image to be displayed;

   Determining, that the color of the i-th pixel point among the N pixel points included in the image to be displayed corresponds to a first color value in the first three-dimensional lookup table, and determining that the color of the i-th pixel point corresponds to Describe a second color value in the second three-dimensional lookup table;

   The first color value and the second color value are fused based on the predetermined fusion weight to obtain a color value of the ith pixel point after the gamut mapping, and the i takes a positive integer not greater than N.
5. The method of claim 3, wherein the method further comprises:

   When the color corresponding to the color of the i-th pixel point is not included in the first three-dimensional lookup table, the i-th is obtained by interpolating colors included in the first three-dimensional lookup table by a three-dimensional interpolation algorithm The color of the pixel corresponds to the color value; or

   When the color corresponding color value of the i-th pixel point is not included in the second three-dimensional lookup table, the i-th pixel point is obtained by interpolating colors included in the second three-dimensional lookup table by using a three-dimensional interpolation algorithm The color corresponds to the color value.
6. The method according to claim 4 or 5, wherein the fusion weight is determined by the following formula:

   

   Wherein, W _blending represents the fusion weight, T _blending represents a threshold for controlling smooth transition from the adjusted color to the protective color, and Δd represents that the color of the i-th pixel corresponds to the color node in the three-dimensional color space and is determined by the first three-dimensional Find the shortest distance between the convex hups formed by the color nodes in the three-dimensional color space; Δd<0 indicates that the color node of the ith pixel is located inside the convex hull; Δd=0 indicates the ith pixel The color node is located on the convex hull; Δd>0 indicates that the color node of the ith pixel is located outside the convex hull.
7. The method according to any one of claims 4 to 6, wherein the first color value and the second color value are fused based on the predetermined fusion weight to obtain the ith pixel after gamut mapping The color value of the point, including:

   The color value of the ith pixel point after the gamut mapping is determined by:

   A _out =A ₁ *W _blending +A ₂ *(1-W _blending );

   Wherein A _out represents a color value of the i-th pixel point after gamut mapping, A ₁ represents the first color value, A ₂ represents the second color value, and W _blending represents the fusion weight.
8. The method according to any one of claims 1 to 7, wherein the performing color management on the image to be displayed based on the scene information comprises:

   Acquiring an adjustment strategy for adjusting pixel saturation of the image to be displayed corresponding to the scene information;

   The brightness saturation hue YSH values of the jth pixel points in the image to be displayed are respectively adjusted as follows:

   Keeping the jth pixel dot hue H value unchanged, adjusting the saturation S value of the jth pixel point based on the adjustment strategy;

   Obtaining a quadratic function function corresponding to the S value of the jth pixel point before adjustment, the adjusted S value, and the brightness Y of the jth pixel point corresponding to the H value of the predetermined jth pixel point The compensated luminance Y value of the jth pixel; the j takes a positive integer not greater than N.
9. The method according to claim 8, wherein the quadratic function corresponding to the H value of the jth pixel point satisfies the condition described in the following formula:

   

   Wherein, Y _out represents the j-th output luminance values of pixels, Y _in represents the input luminance value of the j-th pixel point, ΔY represents the luminance compensation value, a1, a2, a3, a4 , a5, A6 represents a quadratic function function coefficient, S _in represents a saturation value of the j-th pixel point before adjustment, and S _out represents a saturation value of the adjusted j-th pixel point.
10. The method of claim 9 wherein said quadratic function coefficients are determined as follows:

    In the case where there is a quadratic function corresponding to the H value of the jth pixel, the a1, a2, a3, a4, a5, a6 are respectively used in advance using the jth pixel before adjustment a saturation value, an adjusted saturation value of the jth pixel point, and the brightness compensation value, which are obtained based on a least squares fitting algorithm; or

    In the case where there is no quadratic function corresponding to the H value of the jth pixel, the a1, a2, a3, a4, a5, a6 are closest to the H value of the jth pixel The two tonal values are respectively obtained by interpolating the coefficients of the quadratic function corresponding to each.
11. The method according to claim 10, wherein in the case where there is no quadratic function corresponding to the H value of the jth pixel, the a1, a2, a3, a4, a5, a6 are as follows Public Get:

    

    a _k represents the kth coefficient in a1, a2, a3, a4, a5, a6; H represents the saturation value of the jth pixel point, and H _p1 and H _p2 represent the nearest neighbor of the H value of the jth pixel point The two tonal values, H _p1 <H<H _p2 , a _{(p1, k)} represents the kth coefficient of the quadratic function corresponding to H _p1 , and a _{(p2, k)} represents the quadratic function corresponding to H _p2 The kth coefficient.
12. The method according to any one of claims 1 to 11, wherein the performing zooming on the image to be displayed based on the scene information comprises:

    Determining a zoom factor of the image to be displayed, and determining a highest frequency of the periodic pattern in the image to be displayed according to the scene information;

    Determining, according to a preset lookup table, the zoom factor and the number of taps of the bicubic filter corresponding to the highest frequency of the periodic pattern in the image to be displayed; and determining the double according to the highest frequency of the periodic pattern in the image to be displayed The coefficient of the cubic equation function corresponding to the cubic filter;

    The image to be displayed is scaled according to the number of taps and the determined cubic function.
13. The method according to claim 12, wherein the cubic function corresponding to the bicubic filter satisfies the condition shown by the following formula:

    

    

    Where x represents the position of the sampled pixel selected based on the number of taps, k(x) represents the weight value corresponding to the sampled pixel point, and dB and dC represent the coefficients of the cubic equation function; B and C are constants. number.
14. An image display enhancement device, comprising:

    a display enhancement module, configured to acquire scene information of an image to be displayed, where the scene information of the image to be displayed is obtained by performing scene analysis on the image to be displayed; and performing, according to the scene information, the image to be displayed At least one type of processing: gamut mapping, color management, contrast enhancement, sharpening, or scaling;

    And a display module, configured to display the image to be displayed after being processed by the display enhancement module.
15. The apparatus according to claim 14, wherein said scene information is included in metadata Metadata of said image to be displayed, or in an exchangeable image file EXIF data area, or in a vendor comment makernotes field.
16. The apparatus according to claim 14 or 15, wherein the scene information comprises at least one of the following: sensitivity ISO value, aperture value, shutter time or exposure EV value.
17. The device according to claim 16, wherein the display enhancement module performs gamut mapping on the image to be displayed based on the scene information, specifically for:

    Determining a first three-dimensional lookup table corresponding to the scene information, and a second three-dimensional lookup table, where the first three-dimensional lookup table is configured to maintain color gamut mapping and color gamut mapping before the color to be protected in the image to be displayed Similarly, the second three-dimensional lookup table is configured to adjust a color that is not protected in the image to be displayed;

    Determining, that the color of the i-th pixel point among the N pixel points included in the image to be displayed corresponds to a first color value in the first three-dimensional lookup table, and determining that the color of the i-th pixel point corresponds to Describe a second color value in the second three-dimensional lookup table;

    The first color value and the second color value are fused based on the predetermined fusion weight to obtain a color value of the ith pixel point after the gamut mapping, and the i takes a positive integer not greater than N.
18. The device according to claim 17, wherein the display enhancement module is further configured to:

    When the color corresponding to the color of the i-th pixel point is not included in the first three-dimensional lookup table, the i-th is obtained by interpolating colors included in the first three-dimensional lookup table by a three-dimensional interpolation algorithm The color of the pixel corresponds to the color value; or

    When the color corresponding color value of the i-th pixel point is not included in the second three-dimensional lookup table, the i-th pixel point is obtained by interpolating colors included in the second three-dimensional lookup table by using a three-dimensional interpolation algorithm The color corresponds to the color value.
19. The device according to claim 17 or 18, wherein the display enhancement module is further configured to determine the fusion weight by the following formula:

    

    Wherein, W _blending represents the fusion weight, T _blending represents a threshold for controlling smooth transition from the adjusted color to the protective color, and Δd represents that the color of the i-th pixel corresponds to the color node in the three-dimensional color space and is determined by the first three-dimensional Find the shortest distance between the convex hups formed by the color nodes in the three-dimensional color space; Δd<0 indicates that the color node of the ith pixel is located inside the convex hull; Δd=0 indicates the ith pixel The color node is located on the convex hull; Δd>0 indicates that the color node of the ith pixel is located outside the convex hull.
20. The apparatus according to any one of claims 17 to 19, wherein the display enhancement module merges the first color value and the second color value to obtain a gamut mapping based on a predetermined fusion weight When the color value of the i-th pixel is specifically used for:

    The color value of the ith pixel point after the gamut mapping is determined by:

    A _out =A ₁ *W _blending +A ₂ *(1-W _blending );

    Wherein A _out represents a color value of the i-th pixel point after gamut mapping, A ₁ represents the first color value, A ₂ represents the second color value, and W _blending represents the fusion weight.
21. The device according to any one of claims 16 to 20, wherein the display enhancement module is configured to: when performing color management on the image to be displayed based on the scene information, specifically:

    Acquiring an adjustment strategy for adjusting pixel saturation of the image to be displayed corresponding to the scene information;

    The brightness saturation hue YSH values of the jth pixel points in the image to be displayed are respectively adjusted as follows:

    Keeping the jth pixel dot hue H value unchanged, adjusting the saturation S value of the jth pixel point based on the adjustment strategy;

    Obtaining a quadratic function function corresponding to the S value of the jth pixel point before adjustment, the adjusted S value, and the brightness Y of the jth pixel point corresponding to the H value of the predetermined jth pixel point The compensated luminance Y value of the jth pixel; the j takes a positive integer not greater than N.
22. The apparatus according to claim 21, wherein the quadratic function corresponding to the H value of the jth pixel point satisfies the condition described in the following formula:

    

    Wherein, Y _out represents the j-th output luminance values of pixels, Y _in represents the input luminance value of the j-th pixel point, ΔY represents the luminance compensation value, a1, a2, a3, a4 , a5, A6 represents a quadratic function function coefficient, S _in represents a saturation value of the j-th pixel point before adjustment, and S _out represents a saturation value of the adjusted j-th pixel point.
23. The apparatus according to claim 22, wherein said display enhancement module is further configured to determine said quadratic function coefficients by:

    In the case where there is a quadratic function corresponding to the H value of the jth pixel, the a1, a2, a3, a4, a5, a6 are respectively used in advance using the jth pixel before adjustment a saturation value, an adjusted saturation value of the jth pixel point, and the brightness compensation value, which are obtained based on a least squares fitting algorithm; or

    In the case where there is no quadratic function corresponding to the H value of the jth pixel, the a1, a2, a3, a4, a5, a6 are closest to the H value of the jth pixel The two tonal values are respectively obtained by interpolating the coefficients of the quadratic function corresponding to each.
24. The apparatus according to claim 23, wherein in the case where there is no quadratic function corresponding to the H value of the jth pixel point, the display enhancement module is specifically used for the following Determine a1, a2, a3, a4, a5, a6:

    

    a _k represents the kth coefficient in a1, a2, a3, a4, a5, a6; H represents the saturation value of the jth pixel point, and H _p1 and H _p2 represent the nearest neighbor of the H value of the jth pixel point The two tonal values, H _p1 <H<H _p2 , a _{(p1, k)} represents the kth coefficient of the quadratic function corresponding to H _p1 , and a _{(p2, k)} represents the quadratic function corresponding to H _p2 The kth coefficient.
25. The device according to any one of claims 16 to 24, wherein the display enhancement module is configured to: when performing scaling on the image to be displayed based on the scene information, specifically:

    Determining a zoom factor of the image to be displayed, and determining a highest frequency of the periodic pattern in the image to be displayed according to the scene information;

    Determining, according to a preset lookup table, the zoom factor and the number of taps of the bicubic filter corresponding to the highest frequency of the periodic pattern in the image to be displayed; and determining the double according to the highest frequency of the periodic pattern in the image to be displayed The coefficient of the cubic equation function corresponding to the cubic filter;

    The image to be displayed is scaled according to the number of taps and the determined cubic function.
26. The apparatus according to claim 25, wherein the cubic function corresponding to said bicubic filter satisfies the condition shown by the following formula:

    

    

    Where x represents the position of the sampled pixel selected based on the number of taps, k(x) represents the weight value corresponding to the sampled pixel point, and dB and dC represent the coefficients of the cubic equation function; B and C are constants. number.
27. A terminal, comprising:

    a processor, configured to acquire scene information of an image to be displayed, where the scene information of the image to be displayed is obtained by performing scene analysis on the image to be displayed; and performing at least one of the following on the image to be displayed based on the scene information Processing: gamut mapping, color management, contrast enhancement, sharpening or scaling;

    And a display for displaying the image to be displayed after the processing.
28. The terminal according to claim 27, wherein the scene information is included in metadata Metadata of the image to be displayed, or in an exchangeable image file EXIF data area, or in a vendor comment makernotes field.
29. The terminal according to claim 27 or 28, wherein the scene information comprises at least one of the following: sensitivity ISO value, aperture value, shutter time or exposure EV value.
30. The terminal according to any one of claims 27 to 29, wherein the processor performs gamut mapping on the image to be displayed based on the scene information, specifically for:

    Determining a first three-dimensional lookup table corresponding to the scene information, and a second three-dimensional lookup table, where the first three-dimensional lookup table is configured to maintain color gamut mapping and color gamut mapping before the color to be protected in the image to be displayed Similarly, the second three-dimensional lookup table is configured to adjust a color that is not protected in the image to be displayed;

    Determining, that the color of the i-th pixel point among the N pixel points included in the image to be displayed corresponds to a first color value in the first three-dimensional lookup table, and determining that the color of the i-th pixel point corresponds to Describe a second color value in the second three-dimensional lookup table;

    The first color value and the second color value are fused based on the predetermined fusion weight to obtain a color value of the ith pixel point after the gamut mapping, and the i takes a positive integer not greater than N.
31. The terminal according to claim 30, wherein the processor is further configured to:

    When the color corresponding color value of the i-th pixel point is not included in the first three-dimensional lookup table, the i-th pixel point is obtained by interpolating colors included in the first three-dimensional lookup table by using a three-dimensional interpolation algorithm. The color corresponds to the color value; or

    When the color corresponding color value of the i-th pixel point is not included in the second three-dimensional lookup table, the i-th pixel point is obtained by interpolating colors included in the second three-dimensional lookup table by using a three-dimensional interpolation algorithm The color corresponds to the color value.
32. The terminal according to claim 30 or 31, wherein the fusion weight is determined by the following formula:

    

    Wherein, W _blending represents the fusion weight, T _blending represents a threshold for controlling smooth transition from the adjusted color to the protective color, and Δd represents that the color of the i-th pixel corresponds to the color node in the three-dimensional color space and is determined by the first three-dimensional Find the shortest distance between the convex hups formed by the color nodes in the three-dimensional color space; Δd<0 indicates that the color node of the ith pixel is located inside the convex hull; Δd=0 indicates the ith pixel The color node is located on the convex hull; Δd>0 indicates that the color node of the ith pixel is located outside the convex hull.
33. The terminal according to any one of claims 30 to 32, wherein the processor fuses the first color value and the second color value to obtain a gamut-map based on a predetermined fusion weight The color value of the i-th pixel point is specifically used for:

    The color value of the ith pixel point after the gamut mapping is determined by:

    A _out =A ₁ *W _blending +A ₂ *(1-W _blending );

    Wherein A _out represents a color value of the i-th pixel point after gamut mapping, A ₁ represents the first color value, A ₂ represents the second color value, and W _blending represents the fusion weight.
34. The terminal according to any one of claims 27 to 33, wherein the processor performs color management on the image to be displayed based on the scene information, specifically for:

    Acquiring an adjustment strategy for adjusting pixel saturation of the image to be displayed corresponding to the scene information;

    The brightness saturation tone YSH value of the jth pixel point in the image to be displayed is respectively The line is adjusted as follows:

    Keeping the jth pixel dot hue H value unchanged, adjusting the saturation S value of the jth pixel point based on the adjustment strategy;

    Obtaining a quadratic function function corresponding to the S value of the jth pixel point before adjustment, the adjusted S value, and the brightness Y of the jth pixel point corresponding to the H value of the predetermined jth pixel point The compensated luminance Y value of the jth pixel; the j takes a positive integer not greater than N.
35. The terminal according to claim 34, wherein the quadratic function corresponding to the H value of the jth pixel point satisfies the condition described in the following formula:

    

    Wherein, Y _out represents the j-th output luminance values of pixels, Y _in represents the input luminance value of the j-th pixel point, ΔY represents the luminance compensation value, a1, a2, a3, a4 , a5, A6 represents a quadratic function function coefficient, S _in represents a saturation value of the j-th pixel point before adjustment, and S _out represents a saturation value of the adjusted j-th pixel point.
36. The terminal according to claim 35, wherein said processor is further configured to determine said quadratic function coefficients by:

    In the case where there is a quadratic function corresponding to the H value of the jth pixel, the a1, a2, a3, a4, a5, a6 are respectively used in advance using the jth pixel before adjustment a saturation value, an adjusted saturation value of the jth pixel point, and the brightness compensation value, which are obtained based on a least squares fitting algorithm; or

    In the case where there is no quadratic function corresponding to the H value of the jth pixel, the a1, a2, a3, a4, a5, a6 are closest to the H value of the jth pixel The two tonal values are respectively obtained by interpolating the coefficients of the quadratic function corresponding to each.
37. The terminal according to claim 36, wherein in the case where there is no quadratic function corresponding to the H value of the jth pixel, the processor is further configured to determine the a1 by the following formula , a2, a3, a4, a5, a6:

    

    a _k represents the kth coefficient in a1, a2, a3, a4, a5, a6; H represents the saturation value of the jth pixel point, and H _p1 and H _p2 represent the nearest neighbor of the H value of the jth pixel point The two tonal values, H _p1 <H<H _p2 , a _{(p1, k)} represents the kth coefficient of the quadratic function corresponding to H _p1 , and a _{(p2, k)} represents the quadratic function corresponding to H _p2 The kth coefficient.
38. The terminal according to any one of claims 27 to 37, wherein the processor further performs scaling on the image to be displayed based on the scene information, specifically for:

    Determining a zoom factor of the image to be displayed, and determining a highest frequency of the periodic pattern in the image to be displayed according to the scene information;

    Determining, according to a preset lookup table, the zoom factor and the number of taps of the bicubic filter corresponding to the highest frequency of the periodic pattern in the image to be displayed; and determining the double according to the highest frequency of the periodic pattern in the image to be displayed The coefficient of the cubic equation function corresponding to the cubic filter;

    The image to be displayed is scaled according to the number of taps and the determined cubic function.
39. The terminal according to claim 38, wherein the cubic function corresponding to the bicubic filter satisfies the condition shown by the following formula:

    

    

    Where x represents the position of the sampled pixel selected based on the number of taps, k(x) represents the weight value corresponding to the sampled pixel point, and dB and dC represent the coefficients of the cubic function, respectively; B and C are constants.

Images