CN115330621A

CN115330621A - Image processing method, apparatus, device, storage medium, and program product

Info

Publication number: CN115330621A
Application number: CN202210970241.3A
Authority: CN
Inventors: 汪雷; 汪涛; 霍星
Original assignee: Spreadtrum Communications Shanghai Co Ltd
Current assignee: Spreadtrum Communications Shanghai Co Ltd
Priority date: 2022-08-12
Filing date: 2022-08-12
Publication date: 2022-11-11

Abstract

The embodiment of the application provides an image processing method, an image processing device, image processing equipment, a storage medium and a program product. The method comprises the steps of obtaining an RGB image, converting the RGB image into a brightness image; determining the brightness gain value of each pixel point in the RGB image according to the brightness image; and processing the RGB image according to the brightness gain value to obtain the processed RGB image. The local contrast of the image is improved.

Description

Image processing method, apparatus, device, storage medium, and program product

Technical Field

The present application relates to the field of video display technologies, and in particular, to an image processing method, an image processing apparatus, an image processing device, a storage medium, and a program product.

Background

Contrast is a measure of the different brightness levels between the brightest white and darkest black of the bright and dark regions in an image, and a larger difference range represents a larger contrast, and a smaller difference range represents a smaller contrast. The magnitude of the contrast may determine the detail and sharpness quality of the image. Therefore, the image quality can be improved by adjusting the contrast.

In the related art, the contrast of the whole image is mainly improved through a global histogram equalization algorithm. But the global histogram equalization algorithm may result in a low contrast locally in the image.

Disclosure of Invention

The present application relates to an image processing method, apparatus, device, storage medium, and program product, which improve local contrast of an image.

In a first aspect, an embodiment of the present application provides an image processing method, including:

acquiring an RGB image;

converting the RGB image into a brightness image;

determining the brightness gain value of each pixel point in the RGB image according to the brightness image;

and processing the RGB image according to the brightness gain value to obtain a processed RGB image.

In a possible implementation manner, determining a luminance gain value of each pixel point in the RGB image according to the luminance image includes:

dividing the brightness image into N brightness sub-images, wherein N is a positive integer;

at least one of statistical processing, smoothing processing and accumulation processing is carried out on the N luminance sub-images respectively to obtain N histograms;

respectively converting the N histograms into N first sub-mapping curves;

and determining the brightness gain value of each pixel point in the RGB image according to the N first sub-mapping curves.

In a possible implementation manner, determining a luminance gain value of each pixel point in the RGB image according to the N first mapping curves includes:

determining M second mapping curves corresponding to the M frames of reference images, wherein each second mapping curve comprises N second sub-mapping curves, and M is a positive integer;

determining a third mapping curve according to the M second mapping curves and the N first sub-mapping curves;

and determining the brightness gain value of each pixel point in the RGB image according to the third mapping curve.

In a possible embodiment, determining a third mapping curve according to the M second mapping curves and the N first sub-mapping curves includes:

determining K unstable histograms in the N histograms, wherein K is a positive integer;

if the K is smaller than or equal to a first threshold value, determining a mapping curve corresponding to a previous frame of image of the RGB image as the third mapping curve, wherein the mapping curve corresponding to the previous frame of image is one of the M second mapping curves;

and if the K is larger than or equal to a first threshold value, carrying out fusion processing on the M second mapping curves and the N first sub-mapping curves to obtain a third mapping curve.

In a possible implementation manner, the fusing the M second mapping curves and the N first sub-mapping curves to obtain the third mapping curve includes:

determining H stable histograms in the N direct images, wherein H is a positive integer;

aiming at any stable histogram, determining a second sub-mapping curve corresponding to the stable histogram as a third sub-mapping curve, wherein an image corresponding to the second sub-mapping curve corresponding to the stable histogram is a previous frame image of the RGB image;

for any unstable histogram, performing weighting processing on M second sub-mapping curves and M first sub-mapping curves corresponding to the unstable histogram to obtain a third sub-mapping curve;

wherein K + H is equal to N, and the third mapping curve comprises N third sub-mapping curves.

In one possible embodiment, the histogram includes a plurality of luminance bins, and determining K unstable histograms in the N histograms includes:

determining a plurality of unstable luminance bins among the plurality of luminance bins for any one of the histograms;

and if the number of the unstable brightness intervals is larger than or equal to a second threshold value, determining the histogram as an unstable histogram.

In one possible embodiment, determining a plurality of unstable luminance intervals among the plurality of luminance intervals includes:

for any brightness interval, if the number of pixels corresponding to the brightness interval is greater than or equal to a third threshold value, determining the brightness interval as an unstable brightness interval;

the third threshold is determined according to an average value of the number of M pixels corresponding to M reference luminance sections, where the M reference luminance sections are luminance sections corresponding to the M reference images.

In a possible implementation manner, determining a luminance gain value of each pixel point in the RGB image according to the third mapping curve includes:

acquiring an initial brightness value corresponding to each pixel point;

determining i mapping brightness values according to the initial brightness value and i third sub-mapping curves, wherein i is a positive integer;

determining a brightness gain value of each pixel point in the RGB image according to the initial brightness value and the i mapping brightness values;

the i third sub-mapping curves include a mapping curve of a current luminance sub-image corresponding to the initial luminance value and a mapping curve of a luminance sub-image adjacent to the current luminance sub-image.

In a possible implementation, the statistical processing and the smoothing processing are respectively performed on the N luminance sub-images, and the statistical processing and the smoothing processing include:

performing statistical processing on the N luminance sub-images to obtain N statistical histograms;

aiming at any one statistical histogram, determining a cutting threshold value corresponding to the statistical histogram;

and carrying out smoothing treatment on the statistical histogram according to the cutting threshold value.

In a possible implementation, determining the clipping threshold corresponding to the statistical histogram includes:

the statistical histogram comprises a plurality of brightness intervals, and the brightness gradient of all pixel points in the brightness intervals is determined aiming at any one brightness interval;

determining a flat confidence corresponding to a brightness interval according to the brightness gradient and the gradient threshold of all the pixel points and the number of all the pixel points;

and determining a clipping threshold according to the flat confidence.

In a possible implementation, smoothing the statistical histogram according to the clipping threshold includes:

and distributing the part of the plurality of brightness intervals, the number of which is greater than the clipping threshold value, to the brightness interval, the number of which is less than the clipping threshold value.

In one possible embodiment, converting the RGB image into a luminance image comprises:

and aiming at any pixel point in the RGB image, taking the maximum value in the RGB numerical value as the initial brightness value of the pixel point in the brightness image.

In a possible implementation manner, processing the RGB image according to the brightness gain value to obtain a processed RGB image, includes:

and processing the RGB values of all pixel points in the RGB image according to the brightness gain value to obtain the processed RGB image.

In a second aspect, an embodiment of the present application provides an image processing apparatus, including an obtaining module, a converting module, a determining module, and a processing module,

the acquisition module is used for acquiring an RGB image;

the conversion module is used for converting the RGB image into a brightness image;

the determining module is used for determining the brightness gain value of each pixel point in the RGB image according to the brightness image;

and the processing module is used for processing the RGB image according to the brightness gain value to obtain a processed RGB image.

In a possible implementation, the determining module is specifically configured to:

respectively converting the N histograms into N first sub-mapping curves;

aiming at any stable histogram, determining a second sub-mapping curve corresponding to the stable histogram as a third sub-mapping curve, wherein an image corresponding to the second sub-mapping curve corresponding to the stable histogram is a previous frame image of the first image;

wherein K + H is equal to N, and the third mapping curve includes N third sub-mapping curves.

acquiring an initial brightness value corresponding to each pixel point;

the i third sub-mapping curves include a mapping curve of a current luminance sub-image corresponding to an initial luminance value and a mapping curve of a luminance sub-image adjacent to the current luminance sub-image.

performing statistical processing on the N brightness sub-images to obtain N statistical histograms;

aiming at any one statistical histogram, determining a cutting threshold corresponding to the statistical histogram;

the statistical histogram comprises a plurality of brightness intervals, and the brightness gradient of all pixel points in the brightness intervals is determined for any one brightness interval;

and determining a clipping threshold value according to the flat confidence coefficient.

In a possible implementation, the conversion module is specifically configured to:

In a possible implementation, the processing module is specifically configured to:

In a third aspect, an embodiment of the present application provides an image processing apparatus, including a processor, a memory;

the memory stores computer-executable instructions;

the processor executes computer-executable instructions stored by the memory, causing the processor to perform the image processing method of any of the first aspects.

In a fourth aspect, the present application provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, the computer-readable storage medium is configured to implement the image processing method according to any one of the first aspect.

In a fifth aspect, the present application provides a computer program product, which includes a computer program that, when executed by a processor, implements the image processing method according to any one of the first aspect.

In a sixth aspect, an embodiment of the present application provides a chip, where a computer program is stored on the chip, and when the computer program is executed by the chip, the communication method according to any one of the first aspect is implemented.

In one possible embodiment, the chip is a chip in a chip module.

The embodiment of the application provides an image processing method, an image processing device, image processing equipment, a storage medium and a program product. The method comprises the steps of obtaining an RGB image, converting the RGB image into a brightness image; determining the brightness gain value of each pixel point in the RGB image according to the brightness image; and processing the RGB image according to the brightness gain value to obtain the processed RGB image. The brightness gain value of each pixel point in the RGB image is determined firstly, the RGB value of each pixel point in the image is adjusted according to the brightness gain value of each pixel point, the local contrast of the image is improved, and then the local detail and the definition of the image are improved.

Drawings

Fig. 1 is a schematic view of an application scenario provided in an embodiment of the present application;

fig. 2 is a first flowchart illustrating an image processing method according to an embodiment of the present disclosure;

fig. 3 is a schematic flowchart of acquiring a first image according to an embodiment of the present disclosure;

FIG. 4 is a schematic flow chart illustrating post-processing operations provided by embodiments of the present application;

fig. 5 is a schematic flowchart of a second image processing method according to an embodiment of the present application;

fig. 6 is a schematic diagram of image partitioning according to an embodiment of the present application;

fig. 7 is a schematic diagram illustrating a process of determining a mapping curve of a multi-frame image according to an embodiment of the present application;

fig. 8 is a schematic flowchart of a third image processing method according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the embodiments of the present application, and it is obvious that the described embodiments are some but not all of the embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another. Alternatively, the first information may also be referred to as second information, and similarly, the second information may also be referred to as first information, without departing from the scope of the present application.

It will be further understood that the terms "comprises," "comprising," "includes" and/or "including," when used in this specification, specify the presence of stated features, steps, operations, elements, components, species, and/or groups, but do not preclude the presence, or addition of one or more other features, steps, operations, elements, components, species, and/or groups thereof. The terms "or," "and/or," "including at least one of the following," and the like, as used herein, are to be construed as inclusive or mean any one or any combination. Optionally, "includes at least one of: A. b, C "means" any of the following: a; b; c; a and B; a and C; b and C; a and B and C ", further for example," A, B or C "or" A, B and/or C "means" any of the following: a; b; c; a and B; a and C; b and C; a and B and C'. An exception to this definition will occur only when a combination of elements, functions, steps or operations are inherently mutually exclusive in some way.

For ease of understanding, an application scenario to which the embodiment of the present application is applied is described below with reference to fig. 1.

Fig. 1 is a schematic view of an application scenario provided in an embodiment of the present application. Referring to fig. 1, an image processing module may be disposed in the terminal device. When the terminal device needs to improve the image quality, the terminal device may first acquire a Red Green Blue (RGB) image, and then process the RGB image through the image processing module to obtain a processed RGB image.

In the related art, a global histogram equalization algorithm is mainly used for enhancing the global contrast of an image, and although the global contrast of the image is improved, the local contrast is reduced in a part of scenes. Meanwhile, on some video display devices, the time domain transformation of the image is not considered while the image contrast is improved, so that the problem of inter-frame flicker is introduced.

In the embodiment of the application, in order to solve the technical problem, the brightness gain value of each pixel point in the RGB image is determined first, and the RGB value of each pixel point in the image is adjusted according to the brightness gain value of each pixel point, so that the local contrast of the image is improved, the local details and the definition of the image are further improved, and inter-frame flicker among multiple frames of images is avoided.

The technical means shown in the present application will be described in detail below with reference to specific examples. It should be noted that the following embodiments may exist independently or may be combined with each other, and the description of the same or displayed contents is not repeated in different embodiments.

Fig. 2 is a first flowchart illustrating an image processing method according to an embodiment of the present application. Referring to fig. 2, the method may include:

s201, acquiring an RGB image.

The execution main body of the embodiment of the application may be a terminal device, or may also be an image processing apparatus provided in the terminal device, and the image processing apparatus may be implemented by software, or may also be implemented by a combination of software and hardware.

The terminal equipment can obtain an original image first and then process the original image to obtain an RGB image; the terminal equipment can also receive RGB images sent by other equipment.

For ease of understanding, the process of acquiring RGB images is described in detail below with reference to fig. 3.

Fig. 3 is a schematic flowchart of acquiring a first image according to an embodiment of the present disclosure. Referring to fig. 3, the terminal device may project optical information to a photosensitive area of the photosensitive element through a lens of the image capturing device, the photosensitive element converts an optical signal into an electrical signal through the photoelectric conversion module, so as to obtain a high-bit bayer (bayer) format raw image, and then preprocesses the bayer image in denoising, dead pixel correction, demosaicing interpolation mode, and the like, so as to obtain a high-bit RGB image.

S202, converting the RGB image into a brightness image.

The luminance image may also be referred to as a grayscale image.

The RGB image can be converted into a luminance image by:

aiming at any pixel point in the RGB image, the maximum value in the RGB numerical value is used as the initial brightness value of the pixel point in the brightness image.

For example, for pixel a in the RGB image, if the RGB value is (255,2,0), 255 is taken as the luminance value of pixel a.

By adopting the mode of the application to convert the image, the problem of single-channel overflow of a high-saturation area can be suppressed.

S203, determining the brightness gain value of each pixel point in the RGB image according to the brightness image.

The luminance gain (gain) value may be greater than 1 or less than 1.

And S204, processing the RGB image according to the brightness gain value to obtain the processed RGB image.

The RGB image may be processed according to the luminance gain value in the following manner to obtain a processed RGB image:

For example, for pixel a in the RGB image, the RGB value is (144,125,74), the corresponding luminance gain value is 1.2, and after 1.2 pairs of (145,125,75), the RGB value of pixel a is (174,150,90).

After the processed RGB image is obtained, a series of post-processing operations may also be performed on the image.

The post-processing operation includes at least one of: denoising, distortion removal, saturation adjustment and color gamut conversion, which are not limited in the present application.

Illustratively, a series of post-processing operations may be performed on the processed RGB image, as shown in fig. 4.

In the embodiment shown in fig. 2, the terminal device may acquire an RGB image, convert the RGB image into a luminance image; determining the brightness gain value of each pixel point in the RGB image according to the brightness image; and processing the RGB image according to the brightness gain value to obtain the processed RGB image. The brightness gain value of each pixel point in the image is determined, the RGB value of each pixel point in the image is adjusted according to the brightness gain value of each pixel point, the local contrast of the image is improved, and then the local detail and the definition of the image are improved.

In addition to any of the above embodiments, the following describes the image processing method further with reference to the embodiment shown in fig. 5.

Fig. 5 is a flowchart illustrating a second image processing method according to an embodiment of the present application. Referring to fig. 5, the method may include:

s501, acquiring an RGB image.

It should be noted that the execution process of S501 may refer to the execution process of S201, and is not described herein again.

And S502, converting the RGB image into a brightness image.

It should be noted that the execution process of S502 may refer to the execution process of S202, and is not described herein again.

And S503, dividing the brightness image into N brightness sub-images.

The size of the N luminance sub-images may be the same, N being a positive integer.

In one possible implementation, the image may be subjected to a blocking process according to the size of the luminance image.

As shown in fig. 6, the a-picture is 180 × 210 pixels in size and can be divided into 35 sub-pictures of 30 × 30 pixels.

In a possible implementation, the RGB image may be divided into N sub-images according to the size of the image, and then the N sub-images are converted into N luminance sub-images.

S504, at least one of statistical processing, smoothing processing and accumulation processing is carried out on the N luminance sub-images respectively to obtain N histograms.

One luminance sub-image corresponds to one histogram, and one histogram may include a plurality of luminance bins.

In one possible implementation, the N histograms may be obtained by:

(1) And carrying out statistical processing on the N luminance sub-images to obtain N histograms.

(2) And respectively carrying out statistical processing and smoothing processing on the N luminance sub-images to obtain N histograms.

(3) And respectively carrying out statistical processing and accumulation processing on the N brightness sub-images to obtain N histograms.

(4) And respectively carrying out statistical processing, smoothing processing and accumulation processing on the N luminance sub-images to obtain N histograms.

In one possible implementation, the statistical processing and the smoothing processing may be performed on the N luminance sub-images respectively by:

performing statistical processing on the N brightness sub-images to obtain N statistical histograms; aiming at any one statistical histogram, determining a cutting threshold corresponding to the statistical histogram; and smoothing the statistical histogram according to the cutting threshold value.

The purpose of the smoothing process is not only to perform different degrees of contrast stretching for flat regions and non-flat regions, but also to prevent the problem of excessive noise amplification due to adaptive histogram equalization.

In converting the luminance sub-image to a statistical histogram, a contrast control intensity may be configured to adjust the clipping threshold range for the flattest and least flattest regions. If the contrast control strength is weaker, the smaller the cutting threshold range is, and the contrast improvement degree is weakened; if the contrast control strength is strong, the clipping threshold range is larger, and the contrast improvement degree is enhanced.

One statistical histogram may correspond to one clipping threshold, or may correspond to a plurality of clipping thresholds.

In one possible implementation, the clipping threshold corresponding to the statistical histogram may be determined by:

the statistical histogram comprises a plurality of brightness intervals, and the brightness gradient of all pixel points in the brightness intervals is determined for any one brightness interval; determining a flat confidence corresponding to a brightness interval according to the brightness gradient and the gradient threshold of all the pixel points and the number of all the pixel points; and determining a clipping threshold according to the flat confidence, wherein the clipping threshold comprises a plurality of clipping thresholds.

A statistical histogram may include a plurality of luminance bins, each luminance bin may correspond to a gradient threshold, and the gradient threshold corresponding to each luminance bin may be the same or different.

The flat confidence corresponding to the luminance interval may be determined by:

determining a first number of pixel points with the brightness gradient smaller than the gradient threshold value in the brightness interval, and determining the flat confidence coefficient according to the ratio of the first number to the number of all the pixel points in the brightness interval.

For example, one luminance interval includes 100 pixel points, where the number of pixel points whose luminance gradient is smaller than the gradient threshold is 60, it may be determined that the flat confidence corresponding to the luminance interval is 60/100=0.6.

Each flat confidence may determine a clipping threshold. In a statistical histogram, how many brightness intervals there are can determine how many clipping thresholds. The clipping threshold determined according to the flat confidence is a value in the clipping threshold range.

In one possible implementation, the statistical histogram may be smoothed according to a clipping threshold by:

and distributing the part of the plurality of brightness intervals, the number of which is greater than the cutting threshold value, to the brightness interval, the number of which is less than the cutting threshold value.

For example, a statistical histogram includes 3 luminance bins, which are A, B, C respectively. Wherein, the number of pixels corresponding to the brightness interval A is 10, and the corresponding cutting threshold value is 7; the number of pixels corresponding to the B brightness interval is 5, and the corresponding cutting threshold value is 10; the number of pixels corresponding to the C luminance section is 3, and the corresponding clipping threshold is 10. The step of smoothing the statistical histogram is to divide the number of 3 pixels in the A brightness interval into 1B brightness interval and 2C brightness intervals to obtain 5 pixels in the 6,C brightness interval corresponding to the 7,B pixels in the A brightness interval.

In one possible implementation, the accumulation process may be to use the sum of the numbers of pixels of all luminance sections before the current luminance section as the accumulated value of the current luminance section.

For example, the smoothed histogram includes 3 luminance sections, the number of pixels corresponding to the a luminance section is 7,B, the number of pixels corresponding to the luminance section is 6,C, the number of pixels corresponding to the a luminance section is 7,B, the number of pixels corresponding to the c luminance section is 13 after the accumulation processing, and the number of pixels corresponding to the c luminance section is 18.

And S505, respectively converting the N histograms into N first sub-mapping curves.

The histogram may be a histogram obtained by performing statistical processing, smoothing processing, and accumulation processing on the luminance sub-image.

In a possible implementation manner, for any one histogram, which includes i luminance bins, the first sub-mapping curve is obtained by conversion according to the number of pixels corresponding to each luminance bin and the total number of pixels corresponding to the histogram.

Specifically, the first sub-mapping curve is determined according to the ratio of the number of pixels corresponding to each brightness interval to the total number of pixels corresponding to the histogram, and the bit width of the RGB image.

S506, determining the brightness gain value of each pixel point in the RGB image according to the N first sub-mapping curves.

In a possible implementation manner, the luminance gain value of each pixel point in the RGB image may be determined by:

determining M second mapping curves corresponding to the M frames of reference images, wherein each second mapping curve comprises N second sub-mapping curves; determining a third mapping curve according to the M second mapping curves and the N first sub-mapping curves; and determining the brightness gain value of each pixel point in the RGB image according to the third mapping curve.

Each frame of reference image corresponds to one second mapping curve, each second mapping curve includes N second sub-mapping curves, and the determination manner of the N second sub-mapping curves may refer to the determination manner of the N first sub-mapping curves, which is not described herein again.

And determining a third mapping curve of the RGB image according to the current mapping curve of the RGB image and the mapping curves of the previous frames of images, so that the problem of inter-frame flicker among multiple frames of images can be relieved.

Next, with reference to fig. 7, taking M as 3 as an example, a process of determining a mapping curve of a plurality of frames of images in a video will be described.

Fig. 7 is a schematic diagram of a process for determining a mapping curve of a plurality of frames of images according to an embodiment of the present application. Referring to fig. 7, a video segment includes N +3 frames of images, wherein the third mapping curve of the first 3 frames of images is the first mapping curve; the third mapping curve of the 4 th frame image is determined according to the third mapping curve of the previous 3 frames of images and the first mapping curve of the 4 th frame of images; the third mapping curve of the 5 th frame image is determined according to the third mapping curves of the 2 nd, 3 rd and 4 th frame images and the first mapping curve of the 5 th frame image; and the like until determining a third mapping curve of the last frame image.

In one possible embodiment, the third mapping curve may be determined by:

determining K unstable histograms in the N histograms, wherein K is a positive integer; if K is smaller than or equal to the first threshold, determining a mapping curve corresponding to a previous frame of image of the RGB image as a third mapping curve, wherein the mapping curve corresponding to the previous frame of image is one of the M second mapping curves; and if K is larger than or equal to the first threshold, performing fusion processing on the M second mapping curves and the N first sub-mapping curves to obtain a third mapping curve.

The fusion process may mitigate the magnitude of the change in the mapping curve.

In a possible implementation manner, the M second mapping curves and the N first sub-mapping curves may be fused to obtain a third mapping curve by:

h stable histograms are determined in the N direct images, wherein H is a positive integer; aiming at any stable histogram, determining a second sub-mapping curve corresponding to the stable histogram as a third sub-mapping curve, wherein an image corresponding to the second sub-mapping curve corresponding to the stable histogram is a previous frame image of the RGB image; for any unstable histogram, performing weighting processing on M second sub-mapping curves and the first sub-mapping curve corresponding to the unstable histogram to obtain a third sub-mapping curve; wherein K + H = N, the third mapping curve comprises N third sub-mapping curves.

The second sub-mapping curve corresponding to the stable histogram may be a second sub-mapping curve obtained by converting a sub-image of a previous frame image at a corresponding position.

For example, the RGB image corresponds to 3 histograms, i.e., A0, B0, and C0, and each histogram corresponds to a sub-mapping curve A0, B0, and C0. The first three frames of images (P1, P2, P3) of the RGB image respectively correspond to 3 histograms, wherein the three histograms of the P1 image are A1, B1, C1 respectively, and the sub-mapping curves corresponding to each histogram are A1, B1, C1 respectively; three histograms of the P2 image are respectively A2, B2 and C2, and the sub-mapping curves corresponding to each histogram are respectively A2, B2 and C2; three histograms of the P3 image are A3, B3, and C3, and the sub-mapping curves corresponding to each histogram are A3, B3, and C3. The luminance section of A0 is the same as the luminance sections of A1, A2, and A3, the luminance section of B0 is the same as the luminance sections of B1, B2, and B3, and the luminance section of C0 is the same as the luminance sections of C1, C2, and C3. The image block corresponding to the A0 histogram in the RGB image is an unstable image block. If the first threshold is 2, determining three sub-mapping curves in the P1 image as a1, b1 and c1 as sub-mapping curves of the RGB image; if the first threshold is 1, the a0 sub-mapping curve and the a1, a2 and a3 sub-mapping curves may be fused to obtain an a 'sub-mapping curve, and then a', b1 and c1 may be determined as sub-mapping curves of the first image.

In one possible implementation, K unstable histograms may be determined among the N histograms by:

determining a plurality of unstable luminance bins among a plurality of luminance bins for any one of the histograms; and if the number of the unstable brightness intervals is larger than or equal to the second threshold value, determining the histogram as an unstable histogram.

For example, if a histogram includes 3 luminance bins, where the number of unstable luminance bins is 2 and the second threshold value is 2, the histogram may be determined as an unstable histogram.

In one possible embodiment, the plurality of unstable luminance intervals may be determined in the plurality of luminance intervals by:

for any brightness interval, if the number of pixels corresponding to the brightness interval is greater than or equal to a third threshold value, determining the brightness interval as an unstable brightness interval; if the brightness interval is smaller than the third threshold, determining the brightness interval as a stable brightness interval; the third threshold is determined according to an average value of the number of M pixels corresponding to the M reference brightness intervals, and the M unstable brightness intervals are brightness intervals corresponding to the M frames of reference images.

The third threshold may be 80% to 120% of the average value of the number of M pixels, and for example, if the average value of the number of M pixels is X, the third threshold may be (80% to 120%) X.

The M frame reference image may be the first M frame image of the RGB image, M being a positive integer, for example, M may be 3.

The blocking mode, statistical mode, smoothing mode and accumulation mode of the M frame reference image can be the same as that of the RGB image. That is, the position and size of the image block after the M-frame reference image is partitioned may be the same as the position and size of the sub-image after the RGB image is partitioned, and the way of partitioning the M-frame reference image may be as shown in fig. 6.

The luminance ranges of the M reference luminance sections and the luminance section of the RGB image may be the same.

For example, an RGB image has 3 histograms, A0, B0, and C0. The first three frames of images (P1, P2, P3) of the RGB image respectively correspond to 3 histograms, wherein the three histograms of the P1 image are A1, B1, C1; three histograms of the P2 image are A2, B2 and C2 respectively; the three histograms of the P3 image are A3, B3, C3, respectively. The luminance section of A0 is the same as the luminance sections of A1, A2, and A3, the luminance section of B0 is the same as the luminance sections of B1, B2, and B3, and the luminance section of C0 is the same as the luminance sections of C1, C2, and C3. For A0, A1, A2 and A3, 3 luminance intervals, L1, L2 and L3, are included. The number of pixels corresponding to the L1 luminance section in A1 is 5, the number of pixels corresponding to the L1 luminance section in A2 is 9, and the number of pixels corresponding to the L1 luminance section in A3 is 10. The average number of pixels of A1, A2 and A3 in the L1 luminance interval is (5 +9+ 10) ÷ 3=8, and when the third threshold is 87.5% of the average number of pixels, that is, the third threshold is 7, if the number of pixels corresponding to the L1 luminance interval in A0 is 6, A0 is a stable luminance interval, and if the number of pixels corresponding to the L1 luminance interval in A0 is 10, A0 is an unstable luminance interval.

In a possible embodiment, the luminance gain value of each pixel point in the RGB image may be determined according to the third mapping curve by:

acquiring an initial brightness value corresponding to each pixel point; determining i mapping brightness values according to the initial brightness value and i third sub-mapping curves, wherein i is a positive integer; and determining the brightness gain value of each pixel point in the RGB image according to the initial brightness value and the i mapping brightness values, wherein the i third sub-mapping curves comprise the mapping curve of the current brightness sub-image corresponding to the initial brightness value and the mapping curve of the brightness sub-image adjacent to the current brightness sub-image.

The current luminance sub-image may be the luminance sub-image where the pixel is located.

The luminance sub-image adjacent to the current luminance sub-image may be a luminance sub-image positioned adjacent to the current luminance sub-image, and as shown in fig. 6, the adjacent luminance sub-images of the luminance sub-image 7 are

luminance sub-images

1, 2, 3, 6, 8, 11, 12, and 13.

And determining a final mapping brightness value according to the i mapping brightness values and the distance from the pixel point to the surrounding adjacent brightness sub-images, and determining the brightness gain value of the pixel point according to the ratio of the final mapping brightness value to the initial brightness value.

If the brightness gain value of a certain pixel point a in the subimage 7 needs to be calculated, the initial brightness value of the pixel point a needs to be substituted into the corresponding sub-mapping curve of the image 7 to obtain a mapping brightness value, the initial brightness value of the pixel point a is substituted into the corresponding sub-mapping curves of the

subimages

1, 2, 3, 6, 8, 11, 12 and 13 respectively to obtain 8 mapping brightness values, the 9 mapping brightness values are weighted to obtain a final mapping brightness value, the weighting of the weighting processing is determined according to the distance from the pixel point a to the adjacent subimage, and the closer the distance is, the larger the weighting is.

By the weighting mapping process with the peripheral sub-images, the luminance difference between the sub-images can be weakened.

And S507, processing the RGB image according to the brightness gain value to obtain the processed RGB image.

It should be noted that the execution process of S507 may refer to the execution process of S204, and details are not described here.

In the embodiment shown in fig. 5, the terminal device may acquire an RGB image, convert the RGB image into a luminance image; dividing the luminance image into N luminance sub-images; at least one of statistical processing, smoothing processing and accumulation processing is carried out on the N luminance sub-images respectively to obtain N histograms; respectively converting the N histograms into N first sub-mapping curves; determining the brightness gain value of each pixel point in the RGB image according to the N first sub-mapping curves; and processing the RGB image according to the brightness gain value to obtain the processed RGB image. The brightness gain value of each pixel point in the image is determined firstly, and the RGB value of each pixel point in the image is adjusted according to the brightness gain value of each pixel point, so that the local contrast of the image is improved, and further the local detail and definition of the image are improved.

On the basis of any of the embodiments described above, a specific example is given below to explain the image processing method of the present application.

As shown in fig. 8, optical information is projected to a photosensitive area of a photosensitive element through a lens of an image capturing device, the photosensitive element converts an optical signal into an electrical signal through a photoelectric conversion module, so as to obtain a high-bit bayer (bayer) format raw image, and then the bayer image is subjected to preprocessing such as denoising, dead pixel correction, and a demosaicing interpolation method, so as to obtain a high-bit RGB image.

And according to the size of the RGB image, carrying out block processing on the RGB image to obtain N RGB sub-images. The N RGB sub-images are converted into N luminance sub-images, and specifically, for each pixel point, the maximum value of the RGB values is used as the initial luminance value of the pixel point in the luminance image.

And respectively carrying out statistical processing on the N luminance sub-images to obtain N statistical histograms. And respectively carrying out smoothing treatment on the N histograms to obtain N smooth histograms. And respectively carrying out accumulation processing on the N smooth histograms to obtain N accumulated histograms. And respectively converting the N accumulated histograms into N first sub-mapping curves.

And acquiring a second sub-mapping curve corresponding to each image block in the first three frames of images of the RGB image in the same way.

Inputting a second sub-mapping curve corresponding to each image block in the first three frames of images and N first sub-mapping curves into a mapping curve control unit to obtain third sub-mapping curves corresponding to N RGB sub-images; and inputting the initial brightness value corresponding to each pixel point into the mapping curve control unit to obtain the brightness gain value of each pixel point, and processing the RGB values of all the pixel points in the RGB image according to the brightness gain value to obtain the processed RGB image.

After a series of post-processing operations are performed on the processed RGB image, the processed RGB image may be sent to a display device for display.

The brightness gain value of each pixel point in the image is determined firstly, and the RGB value of each pixel point in the image is adjusted according to the brightness gain value of each pixel point, so that the local contrast of the image is improved, and further the local detail and definition of the image are improved. Meanwhile, the mapping curve of the current frame image is processed by combining the previous three frame images, so that the problem of inter-frame flicker among the multiple frame images can be avoided.

Fig. 9 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application. Referring to fig. 9, the image processing apparatus 10 includes an obtaining module 11, a converting module 12, a determining module 13 and a processing module 14, wherein,

the acquisition module 11 is configured to acquire an RGB image;

the conversion module 12 is configured to convert the RGB image into a luminance image;

the determining module 13 is configured to determine a luminance gain value of each pixel in the RGB image according to the luminance image;

the processing module 14 is configured to process the RGB image according to the brightness gain value to obtain a processed RGB image.

In a possible implementation, the determining module 13 is specifically configured to:

dividing the luminance image into N luminance sub-images, wherein N is a positive integer;

respectively converting the N histograms into N first sub-mapping curves;

determining a third mapping curve according to the M second mapping curves and the N first sub-mapping curves, wherein the third mapping curve comprises N third sub-mapping curves;

acquiring an initial brightness value corresponding to each pixel point;

determining i mapping brightness values according to the initial brightness value and the i third sub-mapping curves, wherein i is a positive integer; (ii) a

And determining the brightness gain value of each pixel point in the RGB image according to the initial brightness value and the i mapping brightness values.

and determining a clipping threshold according to the flat confidence.

In a possible implementation, the conversion module 12 is specifically configured to:

In a possible implementation, the processing module 14 is specifically configured to:

The image processing apparatus 10 provided in the present application can execute the technical solution shown in the above-mentioned embodiment of the image processing method, and the implementation principle and the beneficial effect thereof are similar, which are not described again here.

Fig. 10 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application. Referring to fig. 10, the image processing apparatus 20 includes: memory 21, processor 22. Illustratively, the memory 21, the processor 22, and the various parts are interconnected by a bus 23.

Memory 21 stores computer-executable instructions;

processor 22 executes computer-executable instructions stored by memory 21 to cause processor 22 to perform any of the image processing methods described above.

The image processing apparatus shown in the embodiment shown in fig. 10 can execute the technical solution shown in the above-mentioned image processing method embodiment, and the implementation principle and the beneficial effect thereof are similar, and are not described herein again.

The image processing apparatus 20 may be a chip, a module, an Integrated Development Environment (IDE), or the like.

The embodiment of the application provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, the computer-readable storage medium is used for implementing the image processing method of any one of the above items.

The present application provides a computer program product, including a computer program, which when executed by a processor implements the image processing method of any one of the above.

The foregoing description is only exemplary of the preferred embodiments of the application and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other arrangements formed by any combination of the above features or their equivalents without departing from the spirit of the disclosure. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.

Further, while operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the application. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. An image processing method, comprising:

acquiring a three-primary-color RGB image;

converting the RGB image into a brightness image;

2. The method of claim 1, wherein determining a luminance gain value for each pixel in the RGB image from the luminance image comprises:

respectively converting the N histograms into N first sub-mapping curves;

3. The method as claimed in claim 2, wherein determining the luminance gain value of each pixel point in the RGB image according to the N first mapping curves comprises:

4. The method of claim 3, wherein determining a third mapping curve from the M second mapping curves and the N first sub-mapping curves comprises:

5. The method according to claim 4, wherein the fusing the M second mapping curves and the N first sub-mapping curves to obtain the third mapping curve comprises:

6. The method of claim 4 or 5, wherein the histogram includes a plurality of luminance bins, and wherein determining K first histograms in the N histograms includes:

7. The method of claim 6, wherein determining a plurality of unstable luminance intervals among the plurality of luminance intervals comprises:

the third threshold is determined according to an average value of M pixel numbers corresponding to M reference luminance intervals, where the M reference luminance intervals are luminance intervals corresponding to the M reference images.

8. The method as claimed in any one of claims 3 to 7, wherein determining a luminance gain value for each pixel point in the RGB image according to the third mapping curve comprises:

acquiring an initial brightness value corresponding to each pixel point;

9. The method according to any of claims 2-8, wherein performing the statistical processing and the smoothing processing on the N luminance sub-images, respectively, comprises:

10. The method of claim 9, wherein determining the clipping threshold corresponding to the statistical histogram comprises:

11. The method of claim 10, wherein smoothing the statistical histogram according to the clipping threshold comprises:

12. The method of any one of claims 1-11, wherein converting the RGB image into a luminance image comprises:

13. The method of claim 12, wherein processing the RGB image according to the luminance gain value to obtain a processed RGB image comprises:

and processing the RGB values of all the pixel points in the RGB image according to the brightness gain value to obtain the processed RGB image.

14. An image processing apparatus comprising an acquisition module, a conversion module, a determination module, and a processing module, wherein,

the acquisition module is used for acquiring an RGB image;

15. An image processing apparatus comprising a processor, a memory;

the memory stores computer-executable instructions;

the processor executes computer-executable instructions stored by the memory, causing the processor to perform the image processing method of any of claims 1 to 13.

16. A computer-readable storage medium having stored therein computer-executable instructions for implementing the image processing method of any one of claims 1 to 13 when executed by a processor.

17. A computer program product, characterized in that it comprises a computer program which, when being executed by a processor, implements the image processing method of any one of claims 1 to 13.