CN111145128B - Color enhancement method and related device - Google Patents
Color enhancement method and related device Download PDFInfo
- Publication number
- CN111145128B CN111145128B CN202010137897.8A CN202010137897A CN111145128B CN 111145128 B CN111145128 B CN 111145128B CN 202010137897 A CN202010137897 A CN 202010137897A CN 111145128 B CN111145128 B CN 111145128B
- Authority
- CN
- China
- Prior art keywords
- image
- color
- color enhancement
- resolution
- transformation matrix
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 87
- 239000011159 matrix material Substances 0.000 claims abstract description 65
- 230000009466 transformation Effects 0.000 claims abstract description 53
- 238000007781 pre-processing Methods 0.000 claims abstract description 20
- 238000012545 processing Methods 0.000 claims description 24
- 238000004590 computer program Methods 0.000 claims description 16
- 238000004891 communication Methods 0.000 claims description 11
- 238000012549 training Methods 0.000 claims description 11
- 230000002708 enhancing effect Effects 0.000 claims description 8
- 238000001514 detection method Methods 0.000 claims description 6
- 230000009286 beneficial effect Effects 0.000 abstract description 19
- 230000000694 effects Effects 0.000 abstract description 14
- 238000004422 calculation algorithm Methods 0.000 abstract description 5
- 238000004364 calculation method Methods 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 14
- 238000013135 deep learning Methods 0.000 description 10
- 238000006243 chemical reaction Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 230000002349 favourable effect Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000013138 pruning Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
Images
Classifications
-
- G06T5/90—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Abstract
The embodiment of the application discloses a color enhancement method and a related device, wherein the method comprises the following steps: preprocessing a first image to be processed to obtain a second image and a third image, wherein the resolutions of the second image and the third image are the same and smaller than a first resolution threshold, the resolution of the first image is larger than a second resolution threshold, and the third image is a second image after color enhancement; fitting the second image and the third image by a least square method to obtain a color transformation matrix; and carrying out color enhancement on the first image according to the color transformation matrix to obtain a fourth image, wherein the fourth image is the first image after color enhancement. The embodiment of the application is beneficial to improving the calculation speed and reducing the algorithm complexity while guaranteeing the color enhancement effect.
Description
Technical Field
The present disclosure relates to the field of electronic devices, and in particular, to a color enhancement method and a related device.
Background
Currently, color enhancement techniques for pictures mainly use generation countermeasure networks (Generative Adversarial Networks, GAN), but existing GAN is mainly for low resolution pictures, which are laborious when they handle high resolution pictures, so most of the input resolution is not very large.
Whereas for high resolution pictures, most of the existing neural network techniques use convolutional neural networks, since this type of computation is suitable for parallel acceleration. However, on terminal devices such as electronic devices (e.g. smart phones), deployment still appears to be relatively laborious, and there are two optimization schemes: firstly, a network pruning algorithm is adopted to cancel unimportant parts in the network, so that the weight of the unimportant parts is sparse, the size of the model is reduced, and the running memory of the algorithm model is also reduced; secondly, a quantization network model is adopted, so that all data of the model are represented by integer 8-bit values, the model can be optimized to one fourth of the original floating point type, the running memory is greatly reduced, and the problem of precision loss is also caused.
Disclosure of Invention
The embodiment of the application provides a color enhancement method and a related device, so as to improve the calculation speed and reduce the algorithm complexity while guaranteeing the color enhancement effect.
In a first aspect, embodiments of the present application provide a color enhancement method, the method including:
Preprocessing a first image to be processed to obtain a second image and a third image, wherein the resolutions of the second image and the third image are the same and smaller than a first resolution threshold, the resolution of the first image is larger than a second resolution threshold, and the third image is a second image after color enhancement;
fitting the second image and the third image by a least square method to obtain a color transformation matrix;
and carrying out color enhancement on the first image according to the color transformation matrix to obtain a fourth image, wherein the fourth image is the first image after color enhancement.
In a second aspect, embodiments of the present application provide a color enhancement device, comprising a processing unit, wherein:
the processing unit is used for preprocessing a first image to be processed to obtain a second image and a third image, the resolutions of the second image and the third image are the same and smaller than a first resolution threshold, the resolution of the first image is larger than a second resolution threshold, and the third image is a second image after color enhancement; the second image and the third image are fitted through a least square method to obtain a color transformation matrix; and the fourth image is the first image after color enhancement.
In a third aspect, an embodiment of the present application provides an electronic device, including a processor, a memory, a communication interface, and one or more programs, where the one or more programs are stored in the memory and configured to be executed by the processor, the programs including instructions for performing steps in any of the methods of the first aspect of the embodiments of the present application.
In a fourth aspect, embodiments of the present application provide a chip, including: and a processor for calling and running a computer program from the memory, so that the device on which the chip is mounted performs some or all of the steps as described in any of the methods of the first aspect of the embodiments of the present application.
In a fifth aspect, embodiments of the present application provide a computer-readable storage medium, where the computer-readable storage medium stores a computer program for electronic data exchange, where the computer program causes a computer to perform some or all of the steps as described in any of the methods of the first aspect of the embodiments of the present application.
In a sixth aspect, embodiments of the present application provide a computer program product, wherein the computer program product comprises a non-transitory computer readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps described in any of the methods of the first aspect of embodiments of the present application. The computer program product may be a software installation package.
It can be seen that, in the embodiment of the present application, the electronic device obtains, by preprocessing, a first image to be processed, to obtain a second image and a third image, where the resolutions of the second image and the third image are the same and smaller than a first resolution threshold, the resolution of the first image is greater than a second resolution threshold, and the third image is a second image after color enhancement; fitting the second image and the third image by a least square method to obtain a color transformation matrix; and carrying out color enhancement on the first image according to the color transformation matrix to obtain a fourth image, wherein the fourth image is the first image after color enhancement. Therefore, the electronic device obtains the fourth image by fitting the two second images with lower resolution and the third image corresponding to the first image to be processed, performs color enhancement on the first image, does not need to perform operations such as clipping on the first image with higher resolution, and meanwhile, the third image is the second image after color enhancement, which is beneficial to guaranteeing the color enhancement effect of the first image, and obtains the color conversion matrix only by a least square method without performing deep learning on the first image with higher resolution, thereby being beneficial to reducing the operation complexity and improving the operation efficiency.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following description will briefly introduce the drawings that are needed in the embodiments or the description of the prior art, it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic structural diagram of an electronic device according to an embodiment of the present application;
FIG. 2 is a schematic flow chart of a color enhancement method according to an embodiment of the present disclosure;
FIG. 3 is a flow chart of another color enhancement method according to an embodiment of the present disclosure;
FIG. 4 is a flow chart of a further color enhancement method according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application;
fig. 6 is a block diagram of functional units of a color enhancement device according to an embodiment of the present application.
Detailed Description
In order to make the present application solution better understood by those skilled in the art, the following description will clearly and completely describe the technical solution in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.
The terms first, second and the like in the description and in the claims of the present application and in the above-described figures, are used for distinguishing between different objects and not for describing a particular sequential order. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the present application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.
The electronic device according to the embodiment of the present application includes an electronic device, which may be an electronic device having communication capability, and the electronic device may include various handheld devices, vehicle-mounted devices, wearable devices, computing devices, or other processing devices connected to a wireless modem, and various types of User Equipment (UE), mobile Station (MS), terminal devices (terminal devices), and so on.
As shown in fig. 1, an electronic device 100 according to an embodiment of the present application includes a housing 110, a display 120, and a motherboard 130, where the motherboard 130 is provided with a front camera 131, a processor 132, a memory 133, a power management chip 134, and the like.
The processor 132 is a control center of the electronic device, connects various parts of the entire electronic device using various interfaces and lines, and performs various functions of the electronic device and processes data by running or executing software programs and/or modules stored in the memory 133 and calling data stored in the memory 133, thereby performing overall monitoring of the electronic device. Optionally, the processor 132 may include one or more processing units; preferably, the processor 132 may integrate an application processor that primarily handles operating systems, user interfaces, applications, etc., with a modem processor that primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 132. The processor 132 may be, for example, a central processing unit (Central Processing Unit, CPU), a general purpose processor, a digital signal processor (Digital Signal Processor, DSP), an Application-specific integrated circuit (ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic device, transistor logic device, hardware component, or any combination thereof. Which may implement or perform the various exemplary logic blocks, modules, and circuits described in connection with this disclosure. The processor described above may also be a combination that performs the function of a computation, e.g., a combination comprising one or more microprocessors, a combination of a DSP and a microprocessor, and so on.
The memory 133 may be used to store software programs and modules, and the processor 132 executes various functional applications and data processing of the electronic device by executing the software programs and modules stored in the memory 133. The memory 133 may mainly include a storage program area that may store an operating system, application programs required for at least one function, and the like, and a storage data area; the storage data area may store data created according to the use of the electronic device, etc. In addition, the memory 133 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. The Memory 133 may be, for example, random access Memory (Random Access Memory, RAM), flash Memory, read Only Memory (ROM), erasable programmable Read Only Memory (Erasable Programmable ROM), electrically Erasable Programmable Read Only Memory (EEPROM), registers, hard disk, a removable disk, a compact disc Read Only Memory (CD-ROM), or any other form of storage medium known in the art.
Referring to fig. 2, fig. 2 is a schematic flow chart of a color enhancement method according to an embodiment of the present application, and the color enhancement method may be applied to the electronic device shown in fig. 1. As shown, the intrinsic color enhancement method includes the following operations.
S201, the electronic device obtains a second image and a third image from the first image to be processed through preprocessing, the resolutions of the second image and the third image are the same and smaller than a first resolution threshold, the resolution of the first image is larger than a second resolution threshold, and the third image is a second image after color enhancement;
the second image and the third image are images obtained by different preprocessing, and the preprocessing may be, for example, compression processing, clipping processing, deep learning processing, and the like, which is not limited herein.
Wherein the first resolution threshold and the second resolution threshold may be the same or different, and when the first resolution threshold and the second resolution threshold are different, the first resolution threshold is smaller than the second resolution threshold.
For example, the resolution of the second image and the third image may be 100×100, and the resolution of the first image may be 1920×1080.
S202, the electronic equipment fits the second image and the third image through a least square method to obtain a color transformation matrix;
the specific implementation manner of the color transformation matrix obtained by fitting the second image and the third image through the least square method by the electronic device is to fit the pixel value of each pixel point of the second image and the pixel value of the corresponding pixel point in the third image through a least square method to obtain an overdetermined equation set, wherein the overdetermined equation set corresponds to the color transformation matrix, namely, the color transformation matrix is an optimal matching function of a minimized error between the non-color-enhanced second image and the color-enhanced third image.
And S203, the electronic equipment performs color enhancement on the first image according to the color transformation matrix to obtain a fourth image, wherein the fourth image is the first image after color enhancement.
In one possible example, the performing color enhancement on the first image according to the color transformation matrix to obtain a fourth image includes:
multiplying a matrix formed by the first pixel values of each pixel point in the first image by the color transformation matrix to obtain a second pixel value of each pixel point, wherein an image formed by the second pixel values of each pixel point is the fourth image.
It can be seen that in this example, the electronic device multiplies the first image before color enhancement by the color transformation matrix (i.e., the best matching function between the two that minimizes the error before color enhancement and after color enhancement), and can obtain the fourth image after color enhancement, which is beneficial for reducing the error of color enhancement.
The scheme can be applied to aspects of night scene picture enhancement, game image beautification and the like, and is not limited herein.
It can be seen that, in the embodiment of the present application, the electronic device obtains, by preprocessing, a first image to be processed, to obtain a second image and a third image, where the resolutions of the second image and the third image are the same and smaller than a first resolution threshold, the resolution of the first image is greater than a second resolution threshold, and the third image is a second image after color enhancement; fitting the second image and the third image by a least square method to obtain a color transformation matrix; and carrying out color enhancement on the first image according to the color transformation matrix to obtain a fourth image, wherein the fourth image is the first image after color enhancement. Therefore, the electronic device obtains the fourth image by fitting the two second images with lower resolution and the third image corresponding to the first image to be processed, performs color enhancement on the first image, does not need to perform operations such as clipping on the first image with higher resolution, and meanwhile, the third image is the second image after color enhancement, which is beneficial to guaranteeing the color enhancement effect of the first image, and obtains the color conversion matrix only by a least square method without performing deep learning on the first image with higher resolution, thereby being beneficial to reducing the operation complexity and improving the operation efficiency.
In one possible example, the preprocessing the first image to be processed to obtain a second image and a third image includes:
obtaining the second image by reducing the first image;
and inputting the second image into a preset network model to output the third image, wherein the third image is the second image after color enhancement.
The preset model is a network model for color enhancement trained through deep learning, for example, may be a generation model in a generation countermeasure network, which is not limited herein.
The preset network model is preset in the electronic equipment before the electronic equipment leaves a factory by a technical developer of the electronic equipment.
In this example, the electronic device obtains the third image after color enhancement by inputting the second image into the deep learning preset model, and inputs the picture with lower resolution into the preset model to enhance the color, thereby ensuring the operation speed and providing an effect guarantee for enhancing the color of the subsequent high-resolution picture.
In one possible example, the preset model is an inference network obtained by training a generation countermeasure network GAN by a first reference image and a second reference image, and separating training results to generate a model, wherein the first reference image and the second reference image include the same image content, and the first reference image and the second reference image have the same resolution, and the first reference image and the second reference image each have a resolution smaller than the first resolution threshold, and the first reference image is obtained by clipping a third reference image, the second reference image is obtained by clipping a fourth reference image, and the third reference image and the fourth reference image are obtained by different electronic devices.
The third reference image and the fourth reference image are images acquired through different devices, one of the two reference images is an image with higher resolution, and the other reference image is an image with lower resolution, for example, the third reference image is an image with lower resolution acquired through a smart phone, the fourth reference image is an image with higher resolution acquired through a single phase inverter, or the third reference image is an image with higher resolution acquired through a single phase inverter, and the fourth reference image is an image with lower resolution acquired through a smart phone, which is not limited herein.
The resolutions of the first reference image and the second reference image may be 100×100, 150×150, etc., which are not limited herein.
The preset model is generated by training the generation countermeasure network GAN through the first reference image and the second reference image, and after training to a convergence state, the generated model is separated to be used as an inference network, and then the inference network is deployed in the electronic equipment by a technical developer.
In this example, two images with lower resolution and the same content are obtained by clipping one image with higher resolution and one image with lower resolution, and training is performed on the GAN network, so as to obtain a converged generation model as a preset model, which is beneficial to improving the effect of the color enhancement model.
In one possible example, after the color enhancing the first image according to the color transformation matrix to obtain a fourth image, the method further includes:
determining a peak signal-to-noise ratio, PSNR, of the fourth image;
when the peak signal-to-noise ratio (Peak Signal to Noise Ratio, PSNR) is detected to be greater than a preset threshold, the fourth image is determined to be the first image after color enhancement.
The greater the peak signal-to-noise ratio, the brighter the color of the image, indicating that the color enhancement is successful, and thus is used to detect whether the fourth image is the color enhanced first image.
In this example, the electronic device detects the fourth image through the peak signal-to-noise ratio after obtaining the fourth image, determines whether the color enhancement is successful, and detects the color enhancement effect, thereby being beneficial to guaranteeing the accuracy of the color enhancement.
In one possible example, after the color enhancing the first image according to the color transformation matrix to obtain a fourth image, the method further includes:
-detecting the first image and the fourth image by means of structural similarity (Structural Similarity Index, SSIM);
and when the detection is successful, determining the fourth image as the image after the color enhancement of the first image.
When the structural similarity detection is successful, the fourth image obtained by the method is free from distortion, and the fourth image is used for guaranteeing that the fourth image is the image obtained by performing color enhancement on the first image.
In this example, the electronic device determines the structural similarity between the first image and the fourth image through the structural similarity, which is favorable for ensuring that the fourth image obtained by the first image through the color transformation matrix fitted by the least square method is not distorted, and is favorable for ensuring the accuracy of color enhancement.
Referring to fig. 3, fig. 3 is a flowchart of another color enhancement method according to an embodiment of the present application, where the color enhancement method may be applied to the electronic device shown in fig. 1. As shown, the intrinsic color enhancement method includes the following operations:
s301, the electronic device obtains a second image and a third image from the first image to be processed through preprocessing, the resolutions of the second image and the third image are the same and smaller than a first resolution threshold, the resolution of the first image is larger than a second resolution threshold, and the third image is a second image after color enhancement.
S302, the electronic equipment fits the second image and the third image through a least square method to obtain a color transformation matrix.
S303, the electronic device multiplies a matrix formed by the first pixel values of each pixel point in the first image by the color transformation matrix to obtain a second pixel value of each pixel point, and an image formed by the second pixel values of each pixel point is a fourth image.
S304, the electronic device detects the first image and the fourth image through structural similarity SSIM.
And S305, when the detection is successful, the electronic equipment determines that the fourth image is the image after the color enhancement of the first image.
It can be seen that, in the embodiment of the present application, the electronic device obtains, by preprocessing, a first image to be processed, to obtain a second image and a third image, where the resolutions of the second image and the third image are the same and smaller than a first resolution threshold, the resolution of the first image is greater than a second resolution threshold, and the third image is a second image after color enhancement; fitting the second image and the third image by a least square method to obtain a color transformation matrix; and carrying out color enhancement on the first image according to the color transformation matrix to obtain a fourth image, wherein the fourth image is the first image after color enhancement. Therefore, the electronic device obtains the fourth image by fitting the two second images with lower resolution and the third image corresponding to the first image to be processed, performs color enhancement on the first image, does not need to perform operations such as clipping on the first image with higher resolution, and meanwhile, the third image is the second image after color enhancement, which is beneficial to guaranteeing the color enhancement effect of the first image, and obtains the color conversion matrix only by a least square method without performing deep learning on the first image with higher resolution, thereby being beneficial to reducing the operation complexity and improving the operation efficiency.
In addition, the electronic device multiplies the first image before color enhancement by the color transformation matrix (i.e., the best matching function between the two to minimize the error before color enhancement and after color enhancement), and can obtain the fourth image after color enhancement, which is beneficial to reducing the error of color enhancement.
In addition, the electronic equipment determines the structural similarity between the first image and the fourth image through the structural similarity, so that the fourth image obtained by the color transformation matrix fitted by the least square method of the first image is free from distortion, and the accuracy of color enhancement is guaranteed.
Referring to fig. 4, fig. 4 is a flowchart of another color enhancement method according to an embodiment of the present application, where the color enhancement method may be applied to the electronic device shown in fig. 1. As shown, the intrinsic color enhancement method includes the following operations:
s401, the electronic device obtains a second image by shrinking the first image.
S402, the electronic device inputs the second image into a preset network model to output a third image, wherein the third image is a second image with enhanced color, the resolutions of the second image and the third image are the same and smaller than a first resolution threshold, and the resolution of the first image is larger than a second resolution threshold.
S403, the electronic device fits the second image and the third image through a least square method to obtain a color transformation matrix.
S404, the electronic device multiplies a matrix formed by the first pixel values of each pixel point in the first image by the color transformation matrix to obtain a second pixel value of each pixel point, and an image formed by the second pixel values of each pixel point is a fourth image.
S405, the electronic device determines a peak signal-to-noise ratio PSNR of the fourth image.
And S406, when the electronic equipment detects that the peak signal-to-noise ratio PSNR is greater than a preset threshold value, determining that the fourth image is the first image after color enhancement.
It can be seen that, in the embodiment of the present application, the electronic device obtains, by preprocessing, a first image to be processed, to obtain a second image and a third image, where the resolutions of the second image and the third image are the same and smaller than a first resolution threshold, the resolution of the first image is greater than a second resolution threshold, and the third image is a second image after color enhancement; fitting the second image and the third image by a least square method to obtain a color transformation matrix; and carrying out color enhancement on the first image according to the color transformation matrix to obtain a fourth image, wherein the fourth image is the first image after color enhancement. Therefore, the electronic device obtains the fourth image by fitting the two second images with lower resolution and the third image corresponding to the first image to be processed, performs color enhancement on the first image, does not need to perform operations such as clipping on the first image with higher resolution, and meanwhile, the third image is the second image after color enhancement, which is beneficial to guaranteeing the color enhancement effect of the first image, and obtains the color conversion matrix only by a least square method without performing deep learning on the first image with higher resolution, thereby being beneficial to reducing the operation complexity and improving the operation efficiency.
In addition, the electronic equipment obtains a third image after color enhancement by inputting the second image into the deep learning preset model, and inputs the picture with lower resolution into the preset model to enhance the color, so that the operation speed is ensured while the color enhancement effect is improved, and the effect guarantee is provided for the subsequent high-resolution picture to enhance the color.
In addition, the electronic device multiplies the first image before color enhancement by the color transformation matrix (i.e., the best matching function between the two to minimize the error before color enhancement and after color enhancement), and can obtain the fourth image after color enhancement, which is beneficial to reducing the error of color enhancement.
In addition, the electronic equipment detects the fourth image through the peak signal to noise ratio after obtaining the fourth image, determines whether the color enhancement is successful, detects the color enhancement effect, and is beneficial to guaranteeing the accuracy of the color enhancement.
Referring to fig. 5, in accordance with the embodiments shown in fig. 2, 3, and 4, fig. 5 is a schematic structural diagram of an electronic device 500 provided in an embodiment of the present application, where the electronic device 500 further includes an application processor 510, a memory 520, a communication interface 530, and one or more programs 521, where the one or more programs 521 are stored in the memory 520 and configured to be executed by the application processor 510, and the one or more programs 521 include instructions for performing the following steps:
Preprocessing a first image to be processed to obtain a second image and a third image, wherein the resolutions of the second image and the third image are the same and smaller than a first resolution threshold, the resolution of the first image is larger than a second resolution threshold, and the third image is a second image after color enhancement;
fitting the second image and the third image by a least square method to obtain a color transformation matrix;
and carrying out color enhancement on the first image according to the color transformation matrix to obtain a fourth image, wherein the fourth image is the first image after color enhancement.
It can be seen that, in the embodiment of the present application, the electronic device obtains, by preprocessing, a first image to be processed, to obtain a second image and a third image, where the resolutions of the second image and the third image are the same and smaller than a first resolution threshold, the resolution of the first image is greater than a second resolution threshold, and the third image is a second image after color enhancement; fitting the second image and the third image by a least square method to obtain a color transformation matrix; and carrying out color enhancement on the first image according to the color transformation matrix to obtain a fourth image, wherein the fourth image is the first image after color enhancement. Therefore, the electronic device obtains the fourth image by fitting the two second images with lower resolution and the third image corresponding to the first image to be processed, performs color enhancement on the first image, does not need to perform operations such as clipping on the first image with higher resolution, and meanwhile, the third image is the second image after color enhancement, which is beneficial to guaranteeing the color enhancement effect of the first image, and obtains the color conversion matrix only by a least square method without performing deep learning on the first image with higher resolution, thereby being beneficial to reducing the operation complexity and improving the operation efficiency.
In one possible example, in terms of the second image and the third image obtained by preprocessing the first image to be processed, the instructions in the program 521 are specifically configured to perform the following operations: obtaining the second image by reducing the first image; and the third image is used for inputting the second image into a preset network model and outputting the third image, and the third image is the second image after color enhancement.
In one possible example, the preset model is an inference network obtained by training a generation countermeasure network GAN by a first reference image and a second reference image, and separating training results to generate a model, wherein the first reference image and the second reference image include the same image content, and the first reference image and the second reference image have the same resolution, and the first reference image and the second reference image each have a resolution smaller than the first resolution threshold, and the first reference image is obtained by clipping a third reference image, the second reference image is obtained by clipping a fourth reference image, and the third reference image and the fourth reference image are obtained by different electronic devices.
In one possible example, in terms of the color enhancing the first image according to the color transformation matrix to obtain a fourth image, the instructions in the program 521 are specifically configured to: multiplying a matrix formed by the first pixel values of each pixel point in the first image by the color transformation matrix to obtain a second pixel value of each pixel point, wherein an image formed by the second pixel values of each pixel point is the fourth image.
In one possible example, the program 521 further includes instructions for: after the first image is subjected to color enhancement according to the color transformation matrix to obtain a fourth image, determining a peak signal-to-noise ratio PSNR of the fourth image; and when the peak signal-to-noise ratio PSNR is detected to be larger than a preset threshold value, determining the fourth image as the first image after the color enhancement.
In one possible example, the program 521 further includes instructions for: after the first image is subjected to color enhancement according to the color transformation matrix to obtain a fourth image, detecting the first image and the fourth image through structural similarity SSIM; and when the detection is successful, determining the fourth image as the image after the color enhancement of the first image.
The foregoing description of the embodiments of the present application has been presented primarily in terms of a method-side implementation. It will be appreciated that the electronic device, in order to achieve the above-described functions, includes corresponding hardware structures and/or software modules that perform the respective functions. Those of skill in the art will readily appreciate that the elements and algorithm steps described in connection with the embodiments disclosed herein may be embodied as hardware or a combination of hardware and computer software. Whether a function is implemented as hardware or computer software driven hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The embodiment of the application may divide the functional units of the electronic device according to the above method example, for example, each functional unit may be divided corresponding to each function, or two or more functions may be integrated in one processing unit. The integrated units may be implemented in hardware or in software functional units. It should be noted that, in the embodiment of the present application, the division of the units is schematic, which is merely a logic function division, and other division manners may be implemented in actual practice.
Fig. 6 is a functional unit block diagram of a color enhancement device 600 according to an embodiment of the present application. The color enhancement device 600 is applied to an electronic apparatus, and the color enhancement device includes a processing unit 601, where the processing unit 601 is configured to perform any step of the foregoing method embodiments, and the color enhancement device further includes a communication unit 602, where when the processing unit 601 performs data transmission such as sending, the communication unit 602 is selectively invoked to complete a corresponding operation. The following is a detailed description.
The processing unit 601 is configured to pre-process a first image to be processed to obtain a second image and a third image, where the resolutions of the second image and the third image are the same and smaller than a first resolution threshold, the resolution of the first image is greater than a second resolution threshold, and the third image is a second image after color enhancement; the second image and the third image are fitted through a least square method to obtain a color transformation matrix; and the fourth image is the first image after color enhancement.
It can be seen that, in the embodiment of the present application, the electronic device obtains, by preprocessing, a first image to be processed, to obtain a second image and a third image, where the resolutions of the second image and the third image are the same and smaller than a first resolution threshold, the resolution of the first image is greater than a second resolution threshold, and the third image is a second image after color enhancement; fitting the second image and the third image by a least square method to obtain a color transformation matrix; and carrying out color enhancement on the first image according to the color transformation matrix to obtain a fourth image, wherein the fourth image is the first image after color enhancement. Therefore, the electronic device obtains the fourth image by fitting the two second images with lower resolution and the third image corresponding to the first image to be processed, performs color enhancement on the first image, does not need to perform operations such as clipping on the first image with higher resolution, and meanwhile, the third image is the second image after color enhancement, which is beneficial to guaranteeing the color enhancement effect of the first image, and obtains the color conversion matrix only by a least square method without performing deep learning on the first image with higher resolution, thereby being beneficial to reducing the operation complexity and improving the operation efficiency.
In one possible example, in terms of the second image and the third image obtained by preprocessing the first image to be processed, the processing unit 601 is specifically configured to: obtaining the second image by reducing the first image; and the communication unit 602 is configured to input the second image into a preset network model to output the third image, where the third image is a second image after color enhancement.
In one possible example, the preset model is an inference network obtained by training a generation countermeasure network GAN by a first reference image and a second reference image, and separating training results to generate a model, wherein the first reference image and the second reference image include the same image content, and the first reference image and the second reference image have the same resolution, and the first reference image and the second reference image each have a resolution smaller than the first resolution threshold, and the first reference image is obtained by clipping a third reference image, the second reference image is obtained by clipping a fourth reference image, and the third reference image and the fourth reference image are obtained by different electronic devices.
In one possible example, in terms of said color enhancing said first image according to said color transformation matrix to obtain a fourth image, said processing unit 601 is specifically configured to: multiplying a matrix formed by the first pixel values of each pixel point in the first image by the color transformation matrix to obtain a second pixel value of each pixel point, wherein an image formed by the second pixel values of each pixel point is the fourth image.
In a possible example, the processing unit 601 is further configured to, after performing color enhancement on the first image according to the color transformation matrix to obtain a fourth image: determining a peak signal-to-noise ratio, PSNR, of the fourth image; and when the peak signal-to-noise ratio PSNR is detected to be larger than a preset threshold value, determining the fourth image as the first image after the color enhancement.
In one possible example, after the processing unit 601 performs color enhancement on the first image according to the color transformation matrix to obtain a fourth image, the processing unit is further configured to: detecting the first image and the fourth image through structural similarity SSIM; and when the detection is successful, determining the fourth image as the image after the color enhancement of the first image.
The color enhancement device 600 may further comprise a storage unit 603 for storing program code and data of the electronic device. The processing unit 601 may be a processor, the communication unit 602 may be a touch display screen or a transceiver, and the storage unit 603 may be a memory.
It can be understood that, since the method embodiment and the apparatus embodiment are in different presentation forms of the same technical concept, the content of the method embodiment portion in the present application should be adapted to the apparatus embodiment portion synchronously, which is not described herein.
The embodiment of the application also provides a chip, wherein the chip comprises a processor, and the processor is used for calling and running the computer program from the memory, so that the device provided with the chip executes part or all of the steps described in the electronic device in the embodiment of the method.
The embodiment of the application also provides a computer storage medium, where the computer storage medium stores a computer program for electronic data exchange, where the computer program causes a computer to execute part or all of the steps of any one of the methods described in the embodiments of the method, where the computer includes an electronic device.
Embodiments of the present application also provide a computer program product comprising a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any one of the methods described in the method embodiments above. The computer program product may be a software installation package, said computer comprising an electronic device.
It should be noted that, for simplicity of description, the foregoing method embodiments are all expressed as a series of action combinations, but it should be understood by those skilled in the art that the present application is not limited by the order of actions described, as some steps may be performed in other order or simultaneously in accordance with the present application. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily required in the present application.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments.
In the several embodiments provided in this application, it should be understood that the disclosed apparatus may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, such as the above-described division of units, merely a division of logic functions, and there may be additional manners of dividing in actual implementation, such as multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, or may be in electrical or other forms.
The units described above as separate components may or may not be physically separate, and components shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units described above, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable memory. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a memory, including several instructions for causing a computer device (which may be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in the embodiments of the present application. And the aforementioned memory includes: a U-disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Those of ordinary skill in the art will appreciate that all or a portion of the steps in the various methods of the above embodiments may be implemented by a program that instructs associated hardware, and that the program may be stored in a computer readable memory, which may include: flash disk, read-Only Memory (ROM), random-access Memory (Random Access Memory, RAM), magnetic or optical disk, and the like.
The foregoing has outlined rather broadly the more detailed description of embodiments of the present application, wherein specific examples are provided herein to illustrate the principles and embodiments of the present application, the above examples being provided solely to assist in the understanding of the methods of the present application and the core ideas thereof; meanwhile, as those skilled in the art will have modifications in the specific embodiments and application scope in accordance with the ideas of the present application, the present description should not be construed as limiting the present application in view of the above.
Claims (10)
1. A method of color enhancement, the method comprising:
the method comprises the steps that a second image and a third image are obtained through preprocessing a first image to be processed, the resolution of the second image is the same as that of the third image and is smaller than a first resolution threshold, the resolution of the first image is larger than a second resolution threshold, the third image is a second image after color enhancement, and the first resolution threshold is smaller than or equal to the second resolution threshold;
Fitting the second image and the third image by a least square method to obtain a color transformation matrix;
and carrying out color enhancement on the first image according to the color transformation matrix to obtain a fourth image, wherein the fourth image is the first image after color enhancement.
2. The method according to claim 1, wherein preprocessing the first image to be processed to obtain a second image and a third image comprises:
obtaining the second image by reducing the first image;
and inputting the second image into a preset model to output the third image, wherein the third image is the second image after color enhancement.
3. The method according to claim 2, wherein the preset model is an inference network obtained by training a generation countermeasure network GAN by a first reference image and a second reference image, and separating training results from the generation model, wherein the first reference image and the second reference image include the same image content and the first reference image and the second reference image have the same resolution, and the first reference image and the second reference image each have a resolution smaller than the first resolution threshold, and the first reference image is obtained by clipping a third reference image, the second reference image is obtained by clipping a fourth reference image, and the third reference image and the fourth reference image are obtained by different electronic devices.
4. A method according to any of claims 1-3, wherein said color enhancing said first image according to said color transformation matrix to obtain a fourth image comprises:
multiplying a matrix formed by the first pixel values of each pixel point in the first image by the color transformation matrix to obtain a second pixel value of each pixel point, wherein an image formed by the second pixel values of each pixel point is the fourth image.
5. A method according to any of claims 1-3, wherein after color enhancing the first image according to the color transformation matrix to obtain a fourth image, the method further comprises:
determining a peak signal-to-noise ratio, PSNR, of the fourth image;
and when the peak signal-to-noise ratio PSNR is detected to be larger than a preset threshold value, determining the fourth image as the first image after the color enhancement.
6. A method according to any of claims 1-3, wherein after color enhancing the first image according to the color transformation matrix to obtain a fourth image, the method further comprises:
detecting the first image and the fourth image through structural similarity SSIM;
And when the detection is successful, determining the fourth image as the image after the color enhancement of the first image.
7. A color enhancement device, comprising a processing unit, wherein:
the processing unit is used for preprocessing a first image to be processed to obtain a second image and a third image, the resolutions of the second image and the third image are the same and smaller than a first resolution threshold, the resolution of the first image is larger than a second resolution threshold, the third image is a second image after color enhancement, and the first resolution threshold is smaller than or equal to the second resolution threshold; the second image and the third image are fitted through a least square method to obtain a color transformation matrix; and the fourth image is the first image after color enhancement.
8. A chip, comprising: a processor for calling and running a computer program from a memory, causing a device on which the chip is mounted to perform the method of any of claims 1-6.
9. An electronic device comprising a processor, a memory, a communication interface, and one or more programs stored in the memory and configured to be executed by the processor, the programs comprising instructions for performing the steps in the method of any of claims 1-6.
10. A computer-readable storage medium, characterized in that a computer program for electronic data exchange is stored, wherein the computer program causes a computer to perform the method according to any one of claims 1-6.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010137897.8A CN111145128B (en) | 2020-03-02 | 2020-03-02 | Color enhancement method and related device |
| PCT/CN2021/073598 WO2021175045A1 (en) | 2020-03-02 | 2021-01-25 | Color enhancement method and related apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010137897.8A CN111145128B (en) | 2020-03-02 | 2020-03-02 | Color enhancement method and related device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111145128A CN111145128A (en) | 2020-05-12 |
| CN111145128B true CN111145128B (en) | 2023-05-26 |
Family
ID=70528252
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010137897.8A Active CN111145128B (en) | 2020-03-02 | 2020-03-02 | Color enhancement method and related device |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN111145128B (en) |
| WO (1) | WO2021175045A1 (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111145128B (en) * | 2020-03-02 | 2023-05-26 | Oppo广东移动通信有限公司 | Color enhancement method and related device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104935902A (en) * | 2015-06-02 | 2015-09-23 | 三星电子（中国）研发中心 | Image color enhancement method and device, and electronic equipment |
| CN109087269A (en) * | 2018-08-21 | 2018-12-25 | 厦门美图之家科技有限公司 | Low light image Enhancement Method and device |
Family Cites Families (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP3409541B2 (en) * | 1995-11-14 | 2003-05-26 | 三菱電機株式会社 | Color correction method, color correction device, color correction application device, and color image system |
| JP2005079834A (en) * | 2003-08-29 | 2005-03-24 | Pentax Corp | Method for calculating color conversion matrix and image signal processing unit |
| CN100515024C (en) * | 2005-12-27 | 2009-07-15 | 华晶科技股份有限公司 | Color information reconstruction method for producing image with low resolution |
| JP5552795B2 (en) * | 2009-10-26 | 2014-07-16 | 株式会社ニコン | Imaging apparatus, image processing apparatus, and program |
| CN102110289B (en) * | 2011-03-29 | 2012-09-19 | 东南大学 | Method for enhancing color image contrast ratio on basis of variation frame |
| US11222415B2 (en) * | 2018-04-26 | 2022-01-11 | The Regents Of The University Of California | Systems and methods for deep learning microscopy |
| CN109559276B (en) * | 2018-11-14 | 2020-09-08 | 武汉大学 | Image super-resolution reconstruction method based on quality evaluation and feature statistics |
| CN109903250B (en) * | 2019-02-25 | 2022-10-04 | 大连海事大学 | Underwater image sharpening processing method based on multi-scale gradient domain contrast stretching |
| CN110197471B (en) * | 2019-03-01 | 2023-03-14 | 合肥工业大学 | Image contrast enhancement method |
| CN110120024B (en) * | 2019-05-20 | 2021-08-17 | 百度在线网络技术（北京）有限公司 | Image processing method, device, equipment and storage medium |
| CN110363720A (en) * | 2019-07-04 | 2019-10-22 | 北京奇艺世纪科技有限公司 | A kind of color enhancement method, apparatus, equipment and the storage medium of image |
| CN110634170B (en) * | 2019-08-30 | 2022-09-13 | 福建帝视信息科技有限公司 | Photo-level image generation method based on semantic content and rapid image retrieval |
| CN111145128B (en) * | 2020-03-02 | 2023-05-26 | Oppo广东移动通信有限公司 | Color enhancement method and related device |
-
2020
- 2020-03-02 CN CN202010137897.8A patent/CN111145128B/en active Active
-
2021
- 2021-01-25 WO PCT/CN2021/073598 patent/WO2021175045A1/en active Application Filing
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104935902A (en) * | 2015-06-02 | 2015-09-23 | 三星电子（中国）研发中心 | Image color enhancement method and device, and electronic equipment |
| CN109087269A (en) * | 2018-08-21 | 2018-12-25 | 厦门美图之家科技有限公司 | Low light image Enhancement Method and device |
Non-Patent Citations (1)
| Title |
|---|
| 轮廓波域内局部对比度增强的彩色图像灰度化算法;王冰雪;刘广文;刘美;陈广秋;;液晶与显示(02);第151-160页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111145128A (en) | 2020-05-12 |
| WO2021175045A1 (en) | 2021-09-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110263909B (en) | Image recognition method and device | |
| AU2018290360B2 (en) | Picture data encoding method and apparatus and picture data decoding method and apparatus | |
| CN107909583B (en) | Image processing method and device and terminal | |
| KR20130115341A (en) | Method and apparatus for providing a mechanism for gesture recognition | |
| CN109151477B (en) | Image data encoding and decoding methods and devices | |
| CN111770340A (en) | Video encoding method, device, equipment and storage medium | |
| CN112767294B (en) | Depth image enhancement method and device, electronic equipment and storage medium | |
| CN111145128B (en) | Color enhancement method and related device | |
| US10467737B2 (en) | Method and device for adjusting grayscale values of image | |
| CN110717864A (en) | Image enhancement method and device, terminal equipment and computer readable medium | |
| CN113837980A (en) | Resolution adjusting method and device, electronic equipment and storage medium | |
| CN113225554A (en) | Image coding and decoding method and device based on neural network, storage medium and terminal | |
| CN110992387A (en) | Image processing method and device, electronic equipment and storage medium | |
| CN108427671B (en) | Information conversion method and apparatus, storage medium, and electronic apparatus | |
| CN111583111B (en) | Dynamic range image compression method, computer equipment and storage device | |
| CN109784226B (en) | Face snapshot method and related device | |
| CN114339252A (en) | Data compression method and device | |
| CN111405293A (en) | Video transmission method and device | |
| CN111489289A (en) | Image processing method, image processing device and terminal equipment | |
| CN111385491A (en) | Image superposition method and device and electronic equipment | |
| CN111161135B (en) | Picture conversion method and related product | |
| CN110879733A (en) | Detection method and device of electronic red packet, terminal and computer readable storage medium | |
| CN116149597A (en) | Data processing method, storage medium and electronic device | |
| CN115883839B (en) | Image verification method, device, equipment and computer readable storage medium | |
| CN116996503B (en) | Desktop image transmission method, system, electronic equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |