CN116703732A - Image processing method, device, terminal equipment and computer readable storage medium - Google Patents
Image processing method, device, terminal equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN116703732A CN116703732A CN202210168360.7A CN202210168360A CN116703732A CN 116703732 A CN116703732 A CN 116703732A CN 202210168360 A CN202210168360 A CN 202210168360A CN 116703732 A CN116703732 A CN 116703732A
- Authority
- CN
- China
- Prior art keywords
- image
- processed
- dimensional
- lookup table
- color lookup
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 32
- 238000012545 processing Methods 0.000 claims abstract description 45
- 238000012937 correction Methods 0.000 claims abstract description 27
- 238000004364 calculation method Methods 0.000 claims description 28
- 238000000034 method Methods 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 11
- 230000006870 function Effects 0.000 description 26
- 230000004913 activation Effects 0.000 description 15
- 239000003086 colorant Substances 0.000 description 15
- 238000013528 artificial neural network Methods 0.000 description 13
- 230000008569 process Effects 0.000 description 6
- 238000004891 communication Methods 0.000 description 4
- 230000003044 adaptive effect Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000007704 transition Effects 0.000 description 3
- 230000007774 longterm Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000007599 discharging Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000005312 nonlinear dynamic Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4007—Interpolation-based scaling, e.g. bilinear interpolation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The application provides an image processing method, an image processing device, a terminal device and a computer readable storage medium, wherein the image processing method comprises the following steps: calculating a three-dimensional grid of the image to be processed; calculating a three-dimensional color lookup table of the image to be processed; and carrying out color correction on the image to be processed based on the three-dimensional grid and the three-dimensional color lookup table. The image processing method provided by the application can improve the color contrast in the area with lower brightness in the image through the three-dimensional color lookup table of the image, further eliminate the shadow in the image, improve the contrast of the image and reduce the brightness of the high exposure area in the image.
Description
Technical Field
The present application relates to the field of intelligent devices, and in particular, to an image processing method, an image processing device, a terminal device, and a computer readable storage medium.
Background
With the development of mobile smart devices, such as mobile phones, the photographing capability of the mobile smart devices is increasing, and the frequency of photographing by using the mobile smart devices is increasing in daily life. However, the photographing capability of the mobile smart device is not yet achieved for a while by the prior art of mobile smart devices. The method is characterized in that the capability of the mobile intelligent device is lower than that of a professional camera in exposure degree, so that when a user takes a picture through the mobile intelligent device, the obtained image has obvious shadow parts or the contrast of the image in a low-brightness area is poor due to the insufficient exposure capability. Thus, how to resolve shadows in images acquired by mobile smart devices is a problem that needs to be resolved in the art.
Disclosure of Invention
The application provides an image processing method, which can reduce the shadow part in an image and the brightness of a high exposure area in the image by providing the contrast of darker colors in the image.
In a first aspect, the present application provides an image processing method, the method comprising:
calculating a three-dimensional grid of the image to be processed;
calculating a three-dimensional color lookup table of the image to be processed for color calibration;
and carrying out color correction on the image to be processed based on the three-dimensional grid and the three-dimensional color lookup table.
In some embodiments of the application, the computing a three-dimensional grid of the image to be processed includes:
performing downsampling processing on the image to be processed to obtain a processed image;
a three-dimensional grid of the processed image is calculated.
In some embodiments of the application, the computing the three-dimensional grid of the processed image includes:
sending the processed image into a preset grid prediction network;
and outputting the three-dimensional grid of the processed image according to the grid prediction network.
In some embodiments of the application, the calculating the three-dimensional color lookup table of the image to be processed includes:
performing downsampling processing on the image to be processed to obtain a processed image;
And calculating a three-dimensional color lookup table of the image to be processed according to the processed image.
In some embodiments of the present application, the calculating a three-dimensional color lookup table of the image to be processed according to the processed image includes:
inputting the processed image into a preset weight prediction network to obtain the linear weight of the processed image;
and calculating a three-dimensional color lookup table of the image to be processed according to the linear weight.
In some embodiments of the present application, the calculating a three-dimensional color lookup table of the image to be processed according to the linear weight includes:
and linearly combining the basic color lookup table with the linear weight to obtain the three-dimensional color lookup table.
In some embodiments of the present application, the performing color correction on an image to be processed based on the three-dimensional grid and the three-dimensional color lookup table performs color correction on the image to be processed based on the three-dimensional grid and the three-dimensional color lookup table, including:
discretizing a pixel grid in the three-dimensional grid, and determining pixel points for the tri-linear difference in the three-dimensional grid;
carrying out tri-linear difference calculation on the pixel points for tri-linear difference to obtain pixel points for correction;
Correcting the pixel points used for correction according to a three-dimensional color lookup table to obtain a first processed image;
performing tri-linear difference calculation on the pixel points for correction to obtain a first processed image;
and carrying out loss reconstruction processing on the first processed image.
In a second aspect, the present application also provides an image processing apparatus, the apparatus comprising:
the first calculation module is used for calculating a three-dimensional grid of the image to be processed;
the second calculation module is used for calculating a three-dimensional color lookup table of the image to be processed for color calibration;
and the processing module is used for carrying out color correction on the image to be processed based on the three-dimensional grid and the three-dimensional color lookup table.
In some embodiments of the present application, the first computing module is specifically configured to:
performing downsampling processing on the image to be processed to obtain a processed image;
a three-dimensional grid of the processed image is calculated.
In some embodiments of the present application, the first computing module is specifically further configured to:
sending the processed image into a preset grid prediction network;
and outputting the three-dimensional grid of the processed image according to the grid prediction network.
In some embodiments of the present application, the second computing module is specifically configured to:
Performing downsampling processing on the image to be processed to obtain a processed image;
and calculating a three-dimensional color lookup table of the image to be processed according to the processed image.
In some embodiments of the present application, the second computing module is specifically further configured to:
inputting the processed image into a preset weight prediction network to obtain the linear weight of the processed image;
and calculating a three-dimensional color lookup table of the image to be processed according to the linear weight.
In some embodiments of the present application, the second computing module is specifically further configured to:
and linearly combining the basic color lookup table with the linear weight to obtain the three-dimensional color lookup table.
In some embodiments of the present application, the processing module is specifically configured to:
discretizing a pixel grid in the three-dimensional grid, and determining pixel points for correction in the three-dimensional grid;
performing tri-linear difference calculation on the pixel points for correction to obtain a first processed image;
and carrying out loss reconstruction processing on the first processed image.
In a third aspect, the present application also provides a terminal device comprising a processor, a memory and a computer program stored in the memory and executable on the processor, the processor executing the computer program to carry out the steps of any of the image processing methods.
In a fourth aspect, the present application also provides a computer-readable storage medium having stored thereon a computer program for execution by a processor to perform the steps in the image processing method of any one of the above.
The image processing method provided by the application can improve the color contrast in the area with lower brightness in the image through the three-dimensional color lookup table of the image, further eliminate the shadow in the image, improve the contrast of the image and reduce the brightness of the high exposure area in the image.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic view of a scene of an image processing system provided in an embodiment of the application;
FIG. 2 is a flow chart of an embodiment of an image processing method according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a functional block of an image processing apparatus according to an embodiment of the present application;
Fig. 4 is a schematic structural diagram of a terminal device in an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to fall within the scope of the application.
In the description of the present application, it should be understood that the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the present application, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.
In the present application, the term "exemplary" is used to mean "serving as an example, instance, or illustration. Any embodiment described as "exemplary" in this disclosure is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is presented to enable any person skilled in the art to make and use the application. In the following description, details are set forth for purposes of explanation. It will be apparent to one of ordinary skill in the art that the present application may be practiced without these specific details. In other instances, well-known structures and processes have not been described in detail so as not to obscure the description of the application with unnecessary detail. Thus, the present application is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.
The application provides an image processing method, an image processing device, a terminal device and a computer readable storage medium, and the detailed description is given below.
First, some concepts presented in the present application are described:
neural network: also called artificial neural networks (Artificial Neural Networks, abbreviated as ANNs) also called Neural Networks (NNs) or Connection models (Connection models) for short, which are arithmetic mathematical models that imitate the behavior characteristics of animal neural networks and perform distributed parallel information processing. The network relies on the complexity of the system and achieves the purpose of processing information by adjusting the relationship of the interconnection among a large number of nodes. Artificial neural networks can be broadly divided into two major classes, namely feedforward-type networks (also called multi-layer perceptron networks) and feedback-type networks (also called Hopfield networks), according to their model structures, the former can be mathematically seen as a class of large-scale nonlinear mapping systems, and the latter is a class of large-scale nonlinear dynamics systems. According to the learning mode, the artificial neural network can be divided into three types of supervised learning, unsupervised learning and semi-supervised learning; the working modes can be divided into deterministic and random types; the time characteristics can also be classified into continuous type or discrete type, and so on.
Linear interpolation: linear interpolation refers to an interpolation mode in which the interpolation function is a polynomial of degree, and the interpolation error at the interpolation node is zero. Compared with other interpolation modes, such as parabolic interpolation, the linear interpolation has the characteristics of simplicity and convenience. The geometric meaning of the linear interpolation is to approximate the original function by using a straight line passing through the point A and the point B in the outline map. The linear interpolation can be used to replace the original function approximately, and can also be used to calculate the values that are not in the table look-up process. Wherein, the tri-linear interpolation and the di-linear interpolation belong to linear interpolation.
Referring to fig. 1, fig. 1 is a schematic view of a scenario of an image processing system according to an embodiment of the present application, where the image processing system may include a terminal device 100 and a storage device 200, and the storage device 200 may transmit data to the terminal device 100. Like the terminal device 100 in fig. 1, the image data stored in the storage device 200 may be acquired to perform the image processing method in the present application.
In the embodiment of the present application, the terminal device 100 may include, but is not limited to, a desktop computer, a portable computer, a web server, a palm computer (Personal Digital Assistant, PDA), a tablet computer, a wireless terminal device, an embedded device, and the like.
In embodiments of the present application, communication between the terminal device 100 and the storage device 200 may be implemented by any communication means, including, but not limited to, mobile communication based on the third generation partnership project (3rd Generation Partnership Project,3GPP), long term evolution (Long Term Evolution, LTE), worldwide interoperability for microwave access (Worldwide Interoperability for Microwave Access, wiMAX), or computer network communication based on the TCP/IP protocol family (TCP/IP Protocol Suite, TCP/IP), user datagram protocol (User Datagram Protocol, UDP), etc.
It should be noted that, the schematic view of the image processing system shown in fig. 1 is only an example, and the image processing system and the scene described in the embodiment of the present application are for more clearly describing the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided by the embodiment of the present application, and those skilled in the art can know that, with the evolution of the image processing system and the appearance of a new service scene, the technical solution provided by the embodiment of the present application is equally applicable to similar technical problems.
As shown in fig. 2, fig. 2 is a flowchart of an embodiment of an image processing method according to an embodiment of the present application, where the image processing method may include the following steps 201 to 203:
201. A three-dimensional grid of the image to be processed is calculated.
In general, the three-dimensional grid or the three-dimensional grid can be used for three-dimensionally spatialization of data, so that the data can be presented in the three-dimensional space or in a virtual three-dimensional space, and the data can be presented in the three-dimensional grid through the three-dimensional operation of the data, thereby facilitating the operations such as data modification and the like. For example: the image is three-dimensionally rasterized, and the grid points are captured to accurately draw, so that the control and presentation of the image data do not need to be performed by inputting coordinate values. The three-dimensional grid has the advantages that on the data recording structure, for example, whether the vertexes between the grids record neighborhood points, sides, surface information, whether the sides record neighborhood surface information and the like, the more the recorded information is, the more convenient the query is, and the efficiency of data modification can be improved when the query is convenient. It is thus possible to derive a rasterized image, which refers to an image that has been discretized both spatially and in terms of brightness, comprising a matrix of points with grey values. Wherein each dot has a gray value (gray value between 0 and 255, see rasterImage attribute information map), which correspond to pels in the matrix, commonly referred to as pixels. The method for obtaining the three-dimensional grid may be: by acquiring RGB information of each pixel point in the original image, for example: RGB information of pixel point is Discretizing the RGB information of the pixel point according to the RGB information of the color space to obtain RGB information processed by the pixel point>After the RGB information of these pixels is processed, a three-dimensional grid of the original image can be constructed. Wherein the concept of color space is: in Color science, a plurality of Color models are established, and a certain Color is represented by one-dimensional, two-dimensional, three-dimensional or even four-dimensional Space coordinates, and a Color range or a Color Space can be defined by the coordinate system. The color space we often use is mainly RGB, CMYK, lab, etc. This is of course only one way to obtain a three-dimensional grid, and is not limited in particular here.
It is known that an image gives a visual sense of space, and that the colors in an image have three-dimensional properties, so that the information expressed in an image that can be given to people is not planar information but three-dimensional space information. The acquired image to be processed in the embodiment of the present application may be an image obtained by photographing through a corresponding image capturing device, or may be an image stored in a corresponding storage device, which is not limited herein.
In order to better implement the embodiment of the present application, in an embodiment of the present application, calculating a three-dimensional grid of an image to be processed includes:
downsampling the image to be processed to obtain a processed image; a three-dimensional grid of the processed image is calculated.
In the above embodiment, in order to avoid errors in the processing process when processing the image later, when generating the three-dimensional raster data of the image, the image needs to be subjected to a previous processing operation, the resolution of the image is adjusted, and when the resolution of the image is adjusted to the required resolution, the three-dimensional raster is acquired, so that the deviation of the image information in the three-dimensional raster can be reduced. For example: the image size of the image to be processed is changed into the preset image size by a bilinear interpolation downsampling mode, then the resolution of the image processed by the image size is adjusted, the resolution of the image is adjusted to 256 multiplied by 256 resolution, and the image with the adjusted resolution is the processed image in the embodiment of the application.
In order to better implement the embodiment of the present application, in one embodiment of the present application, calculating a three-dimensional grid of a processed image includes:
Sending the processed image into a preset grid prediction network; and outputting a three-dimensional grid of the processed image according to the grid prediction network.
In the above embodiment, a three-dimensional grid computing manner is provided, and the three-dimensional grid computing manner of the image may be a manual computing manner, and in order to reduce the labor cost, the embodiment of the present application provides a three-dimensional grid computing manner of the image through a neural network. The concept of the neural network is described above, and will not be described herein. And the grid prediction network model may be composed of a plurality of convolution layers, output layers. Specifically, the plurality of convolution layers may include:
a first layer: 2D Convolution,16filters,kernel_size 3,stride 2,zero padding 1, an activation function LeakyReLU with a coefficient a=0.2;
a second layer: 2D Convolution,32filters,kernel_size 3,stride 2,zero padding 1, an activation function LeakyReLU with a coefficient a=0.2;
third layer: 2D Convolution,64filters,kernel_size 3,stride 2,zero padding 1, an activation function LeakyReLU with a coefficient a=0.2;
fourth layer: 2D Convolution,128filters,kernel_size 3,stride 2,zero padding 1, an activation function LeakyReLU with a coefficient a=0.2;
Fifth layer: 2D Convolution,128filters,kernel_size 3,stride 2,zero padding 1, an activation function LeakyReLU with a coefficient a=0.2;
sixth layer: 2D confirmation, # bin 3filters,kernel_size 8,no zero padding;
output layer: softmax activation function over first dimension (Softmax activation function in one dimension).
Wherein 2D Convolution is a two-dimensional Convolution, filters is a Convolution kernel, kernel_size is a kernel step size, zero padding is a padding distance, no zero padding is no padding, and # bin is the number of channels of the corresponding color. In addition, the activation function, leakyReLU, is:
x,if x>0；-a*xif≤0......①
the processed image is sent to the grid prediction network provided in this embodiment, and a three-dimensional grid of the image can be obtained. The manner of training the grid prediction network is not within the scope of the present application, and therefore will not be described herein. In addition, the number of the convolution layers of the grid prediction network is 6, and the number of the convolution layers is small, so that the calculation load is smaller when the convolution calculation is performed than that of the neural network with more convolution layers, and the grid prediction network is also suitable for the convolution calculation of the mobile terminal with poor calculation performance.
202. A three-dimensional color look-up table of the image to be processed is calculated.
The three-dimensional color look-up table (3D Look Up Table,3DLUT), which is a color calibration tool, can also be understood as a color calibration function, acts to calibrate the colors in the image. When different images are acquired, the color representation modes of the different images are different due to different factors such as exposure degree, shooting equipment and the like, for example, part of the images are darker, and the other part of the images are brighter. Thus, for different images, a three-dimensional color lookup table per image itself needs to be calculated. For example: when a processed image is acquired, the color of the image to be processed can be analyzed to obtain the color gamut range and the gray scale range of the color image, and the self-adaptive three-dimensional color lookup ratio belonging to the image to be processed is generated according to the color gamut range and the gray scale range. After the adaptive three-dimensional color lookup table of the image to be processed is generated, after the three-dimensional rasterization of the color data in the image, comparison may be performed according to the three-dimensional color lookup table, for example: the red color data in a certain grid is different from the corresponding red color data in the three-dimensional color lookup table, and at this time, the red in the three-dimensional color lookup table is replaced with the red in the grid.
In order to better implement the embodiment of the present application, in an embodiment of the present application, calculating a three-dimensional color lookup table of an image to be processed includes:
downsampling the image to be processed to obtain a processed image; and calculating a three-dimensional color lookup table of the image to be processed according to the processed image.
The purpose of downsampling the image to be processed is described in the above embodiments, and will not be described here. In order to avoid the influence of uncertain factors when the images are calculated, different image data derived from the same image to be processed can be processed, and the specific processing mode is the same as that of the embodiment, and the image to be processed is firstly subjected to downsampling related processing, so that the image size and the resolution of the image to be processed are unified, and unnecessary errors in the subsequent calculation process are avoided.
After the corresponding image preprocessing is completed on the image to be processed, the self-adaptive three-dimensional color lookup table of the image to be processed can be calculated. Specifically, the manner of calculating the three-dimensional color lookup table is the same as that of the above embodiment, and details thereof are not repeated here.
In order to better implement the embodiment of the present application, in an embodiment of the present application, calculating a three-dimensional color lookup table of an image to be processed according to the processed image includes:
Inputting the processed image into a preset weight prediction network to obtain the linear weight of the processed image; and calculating a three-dimensional color lookup table of the image to be processed according to the linear weight.
According to the above embodiment, the manner of calculating the three-dimensional color lookup table may be that the self-adaptive three-dimensional color lookup table of the image to be processed is generated through the color gamut range and the gray scale range of the image to be processed, which may involve a step of manual operation, so that in order to reduce the corresponding manual steps, the self-adaptive three-dimensional color lookup table of the image to be processed may be replaced by a corresponding model. For example, in the present embodiment, the weight prediction network is mentioned, and it should be noted that the purpose of the weight prediction network is to filter out the weight of the overall color in the image to be processed. Assuming that red in all colors in an image to be processed is taken as an example, if the image to be processed includes multiple types of red, such as highlight type red and dim type red, in order to reduce shadows of the image, contrast of the dim type red needs to be improved, so that weight of the highlight type red is greater than that of the dim type red, and therefore weight prediction can be performed on different colors and types of colors in the image to be processed according to the weight prediction network, so that a weight vector, namely linear weight, of the color with important weight of the image to be detected is obtained, and then a three-dimensional color lookup table of the image to be processed is obtained according to the linear weight. In addition, specific weights of different colors can be obtained according to the linear weights of the different colors, when a certain color is larger than a certain weight threshold value, the color can be determined to be an important color, and after the colors with a plurality of weights are obtained, the important color can be formed into a three-dimensional color weight table. However, in practice, the three-dimensional color look-up table is calculated for the entire color in the image, and for ease of understanding, only red is exemplified, and in practice, a comprehensive calculation is performed for the three-dimensional color look-up table for all colors in the image. When the comprehensive calculation is performed, the linear weight of each color can be understood as a comprehensive weight or global weight, so that the three-dimensional color lookup table corresponding to the specific image can be calculated.
The structure of the weight prediction network is not described herein, and specific weight prediction networks are described in detail below.
In order to better implement the embodiment of the present application, in an embodiment of the present application, calculating a three-dimensional color lookup table of an image to be processed according to a linear weight includes:
and linearly combining the basic color lookup table with the linear weight to obtain a three-dimensional color lookup table.
According to the above embodiments, it is possible to construct an adaptive three-dimensional color lookup table of an image to be processed according to different color weights, but a problem sometimes occurs, here, taking red in the image to be processed as an example, it is assumed that the image to be processed includes red, but the red may be all of a dull type red, so that when predicting the red with the greatest weight according to the weight prediction network, only one kind of dull type red with the greatest weight can be obtained, and the required bright type red cannot be obtained, so in order to solve the problem, it is necessary to provide a color sample that can be referred to, i.e., provide a basic color lookup table, and the basic color lookup table includes bright types of various colors, so that the problem can be avoided.
Wherein the structure of the weight prediction network may include a plurality of convolution layers, for example:
A first layer: 2D Convolution,16filters,kernel_size 3,stride 2,zero padding 1, an activation function LeakyReLU with a coefficient a=0.2;
a second layer: 2D Convolution,32filters,kernel_size 3,stride 2,zero padding 1, an activation function LeakyReLU with a coefficient a=0.2;
third layer: 2D Convolution,64filters,kernel_size 3,stride 2,zero padding 1, an activation function LeakyReLU with a coefficient a=0.2;
fourth layer: 2D Convolution,128filters,kernel_size 3,stride 2,zero padding 1, an activation function LeakyReLU with a coefficient a=0.2;
fifth layer: 2D Convolution,128filters,kernel_size 3,stride 2,zero padding 1, an activation function LeakyReLU with a coefficient a=0.2;
sixth layer: 2D Convolution,N filters,kernel_size 8,no zero padding.
Wherein, 2D Convolution is two-dimensional Convolution, filters is Convolution kernel, kernel_size is kernel step size, zero padding is padding distance, and no zero padding is no padding. The basic color lookup table may be plural, such as a basic color lookup table of red, a basic color lookup table of orange, etc., and N in N filters in the sixth layer is the number of basic color lookup tables. The activation function LeakyReLU is the same as the activation function described above, and will not be described here. After the sixth layer of the weight prediction network, since a plurality of linear weights can be output due to the existence of N, the number of linear weights is also the same as the number of basic color lookup tables. The basic color look-up table is linearly combined with the linear weights in the following manner:
L＝w 1 *L 1 +...+w n *L n ......②
Wherein L is an adaptive three-dimensional color lookup table, L n As a base color lookup table, w n For linear weights, n is the number of corresponding base color look-up tables and the number of linear weights. In addition, the number of convolution layers of the weight prediction network is 6, and the number of convolution layers is still small. Similarly, the computational burden is smaller than that of a neural network with a larger number of convolution layers when the convolution operation of the weight prediction is performed, so that the method is also suitable for the convolution operation of the weight of the mobile terminal with poor calculation performance.
203. And carrying out color correction on the image to be processed based on the three-dimensional grid and the three-dimensional color lookup table.
A three-dimensional color look-up table is understood to be a corresponding table in which there is a conversion rule for the corresponding pixel value, e.g. a mapping of pixel value a to pixel value B. The pixel value a may be a pixel value with a heavy shadow detected according to an actual image, or a pixel value with a high exposure; the pixel value B may be a pixel value with a normal color corresponding to the pixel value a. If the pixel value A appears in the three-dimensional grid of the image to be processed, the pixel value A in the three-dimensional grid can be replaced by the pixel value B according to the corresponding relation of the three-dimensional color lookup table, and thus the calibration work of the pixel value A of the three-dimensional grid is completed. Similarly, the three-dimensional color lookup table may include a plurality of pixel correspondence relationships, and according to the pixel correspondence relationships, the pixels in the three-dimensional grid are calibrated.
In order to better implement the embodiment of the present application, in an embodiment of the present application, performing color correction on an image to be processed based on a three-dimensional grid and a three-dimensional color lookup table includes:
discretizing a pixel grid in the three-dimensional grid to determine pixel points for the tri-linear difference in the three-dimensional grid; carrying out tri-linear difference calculation on the pixel points used for the tri-linear difference to obtain pixel points used for correction; correcting the pixel points used for correction according to the three-dimensional color lookup table to obtain a first processed image; and carrying out loss reconstruction processing on the first processed image.
After the three-dimensional grid and the three-dimensional color lookup table of the image to be processed are acquired, the mode of processing the image to be processed can be processed through a tri-linear interpolation mode, and the purpose of the tri-linear interpolation is to output the color data of the image to be finally generated according to the three-dimensional grid.
Before the tri-linear difference calculation is used, pixel values for the tri-linear difference calculation need to be acquired in a three-dimensional grid, and the acquisition mode can be shown as the following formula:
wherein, C (i, j, k) is the acquired pixel position from the three-dimensional grid, i.e. C (i, j, k) is the actual coordinates of the corresponding pixel to be adjusted in the three-dimensional grid. i, j, k represent coordinates on three coordinate axes in the three-dimensional coordinate system, respectively. Where #bin is the number of channels of a color, i.e., the number of i, j, k cannot exceed the number of channels of a color. The S is obtained from the grid preset network model, and it is to be noted that the three-dimensional grid is formed by a plurality of three-dimensional unit grids, and each side formed by each three-dimensional unit grid has a length of 1, and each three-dimensional grid in each channel (or dimension) has the same number of nodes. For example a three-dimensional grid 1 having 4 x 4 nodes, then the three-dimensional grid 2 also has 4 x 4 nodes. Between each successive node is a three-dimensional grid of cells, and the distance between each successive node is defined by S in this formula. For example: let the distance between the P node and the p+1th node in the first channel (in the first dimension) be S (P, 0), and similarly, when in the second channel, the distance between the P node and the p+1th node be S (P, 1), and S (P, 2) are similarly, and will not be described here. At this time, the position of the pixel value where the tri-linear difference is required can be determined, that is, after C (i, j, k) is obtained, C (i, j, k) is further subjected to tri-linear difference calculation to obtain C '(i, j, k) after the tri-linear difference calculation, and the C' (i, j, k) is subjected to corresponding color processing according to the three-dimensional color lookup table, for example: correcting darker colors in the original image, and enabling the darker colors to be the same colors with brighter colors; or correcting the color with higher exposure degree in the original image, and reducing the exposure degree of the corresponding color. The color with higher exposure is corrected in the same way as the shadow color is corrected by the three-dimensional grid and the three-dimensional color lookup table, and the description is omitted here.
According to the above embodiment, it is possible to obtain the processed image according to the three-line interpolation method, but when the processed image is directly replaced with the bright color after the dark color is removed, the phenomenon of uneven color replacement may occur, for example, after the dark color is replaced with the bright color, the replacing position of the color is a transition position, and the problem of uneven transition may occur. To solve this problem, the image obtained after the tri-linear interpolation is processed may be subjected to loss reconstruction so that the position of the color transition is smoother. The manner of performing the loss reconstruction may be as follows:
wherein L is 2 For the output second processed image, O (I, j) is the first processed image, I (I, j) is the image to be processed; i and j represent pixel positions.
The image processing method provided by the application can improve the color contrast in the area with lower brightness in the image through the three-dimensional color lookup table of the image, further eliminate the shadow in the image, improve the contrast of the image and reduce the brightness of the high exposure area in the image.
In order to better implement the image processing method in the embodiment of the present application, above the image processing method, an image processing apparatus is further provided in the embodiment of the present application, as shown in fig. 3, an image processing apparatus 300 includes:
A first calculation module 301, configured to calculate a three-dimensional grid of an image to be processed;
a second calculation module 302, configured to calculate a three-dimensional color lookup table of the image to be processed;
the processing module 303 is configured to perform color correction on the image to be processed based on the three-dimensional grid and the three-dimensional color lookup table.
According to the image processing method provided by the application, the three-dimensional grid of the image can be calculated through the first calculation module 301, the three-dimensional color lookup table of the image can be calculated through the second calculation module 302, and the color contrast in the area with lower brightness in the image can be improved through the processing module 303, so that the shadow in the image can be eliminated, the contrast of the image can be improved, and meanwhile, the brightness of the high exposure area in the image can be reduced.
In some embodiments of the present application, the first computing module 301 is specifically configured to:
downsampling the image to be processed to obtain a processed image;
a three-dimensional grid of the processed image is calculated.
In some embodiments of the present application, the first computing module 301 is specifically further configured to:
sending the processed image into a preset grid prediction network;
and outputting a three-dimensional grid of the processed image according to the grid prediction network.
In some embodiments of the present application, the second computing module 302 is specifically configured to:
Downsampling the image to be processed to obtain a processed image;
and calculating a three-dimensional color lookup table of the image to be processed according to the processed image.
In some embodiments of the present application, the second computing module 302 is specifically further configured to:
inputting the processed image into a preset weight prediction network to obtain the linear weight of the processed image;
and calculating a three-dimensional color lookup table of the image to be processed according to the linear weight.
In some embodiments of the present application, the second computing module 302 is specifically further configured to:
and linearly combining the basic color lookup table with the linear weight to obtain a three-dimensional color lookup table.
In some embodiments of the present application, the processing module 303 is specifically configured to:
discretizing a pixel grid in the three-dimensional grid to determine pixel points for the tri-linear difference in the three-dimensional grid;
carrying out tri-linear difference calculation on the pixel points used for the tri-linear difference to obtain pixel points used for correction;
correcting the pixel points used for correction according to the three-dimensional color lookup table to obtain a first processed image;
carrying out tri-linear difference calculation on the pixel points for correction to obtain a first processed image;
and carrying out loss reconstruction processing on the first processed image.
The embodiment of the application also provides a terminal device, which comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to realize the steps in the image processing method of any one of the embodiments of the application. The terminal device integrates any one of the image processing methods provided by the embodiments of the present application, as shown in fig. 4, which shows a schematic structural diagram of the terminal device according to the embodiments of the present application, specifically:
the terminal device may include one or more processing cores 'processors 401, one or more computer-readable storage media's memory 402, power supply 403, and input unit 404, among other components. It will be appreciated by those skilled in the art that the terminal device structure shown in fig. 4 is not limiting of the terminal device and may include more or fewer components than shown, or may combine certain components, or a different arrangement of components. Wherein:
the processor 401 is a control center of the terminal device, connects respective parts of the entire terminal device using various interfaces and lines, and performs various functions of the terminal device and processes data by running or executing software programs and/or modules stored in the memory 402 and calling data stored in the memory 402, thereby performing overall monitoring of the terminal device. Optionally, processor 401 may include one or more processing cores; the processor 401 may be a central processing unit (Central Processing Unit, CPU), but may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like, preferably, the processor 401 may integrate an application processor, which primarily handles operating systems, user interfaces, applications, etc., with a modem processor, which primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 401.
The memory 402 may be used to store software programs and modules, and the processor 401 executes various functional applications and data processing by executing the software programs and modules stored in the memory 402. The memory 402 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like; the storage data area may store data created according to the use of the terminal device, etc. In addition, memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.
The terminal device further comprises a power supply 403 for supplying power to the various components, preferably the power supply 403 may be logically connected to the processor 401 by a power management system, so that functions of managing charging, discharging, power consumption management, etc. are achieved by the power management system. The power supply 403 may also include one or more of any of a direct current or alternating current power supply, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like.
The terminal device may further comprise an input unit 404, which input unit 404 may be used for receiving input digital or character information and generating keyboard, mouse, joystick, optical or trackball signal inputs in connection with user settings and function control.
Although not shown, the terminal device may further include a display unit or the like, which is not described herein. In this embodiment, the processor 401 in the terminal device loads executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 executes the application programs stored in the memory 402, so as to implement various functions, for example:
calculating a three-dimensional grid of the image to be processed;
calculating a three-dimensional color lookup table of the image to be processed;
and carrying out color correction on the image to be processed based on the three-dimensional grid and the three-dimensional color lookup table.
Those of ordinary skill in the art will appreciate that all or a portion of the steps of the various methods of the above embodiments may be performed by instructions, or by instructions controlling associated hardware, which may be stored in a computer-readable storage medium and loaded and executed by a processor.
To this end, an embodiment of the present application provides a computer-readable storage medium, which may include: read Only Memory (ROM), random access Memory (RAM, random Access Memory), magnetic or optical disk, and the like. On which a computer program is stored, the computer program being loaded by a processor for performing the steps of any of the image processing methods provided by the embodiments of the present application. For example, the loading of the computer program by the processor may perform the steps of:
calculating a three-dimensional grid of the image to be processed;
calculating a three-dimensional color lookup table of the image to be processed;
and carrying out color correction on the image to be processed based on the three-dimensional grid and the three-dimensional color lookup table.
In the foregoing embodiments, the descriptions of the embodiments are focused on, and the portions of one embodiment that are not described in detail in the foregoing embodiments may be referred to in the foregoing detailed description of other embodiments, which are not described herein again.
In the implementation, each unit or structure may be implemented as an independent entity, or may be implemented as the same entity or several entities in any combination, and the implementation of each unit or structure may be referred to the foregoing method embodiments and will not be repeated herein.
The specific implementation of each operation above may be referred to the previous embodiments, and will not be described herein.
The foregoing has described in detail a method and apparatus for image processing according to embodiments of the present application, and specific examples have been applied to illustrate the principles and embodiments of the present application, where the foregoing examples are provided to assist in understanding the method and core idea of the present application; meanwhile, as those skilled in the art will vary in the specific embodiments and application scope according to the ideas of the present application, the present description should not be construed as limiting the present application in summary.
Claims (10)
1. An image processing method, the method comprising:
calculating a three-dimensional grid of the image to be processed;
calculating a three-dimensional color lookup table of the image to be processed;
and performing color correction on the image to be processed based on the three-dimensional grid and the three-dimensional color lookup table.
2. The image processing method according to claim 1, wherein the calculating a three-dimensional grid of the image to be processed includes:
performing downsampling processing on the image to be processed to obtain a processed image;
a three-dimensional grid of the processed image is calculated.
3. The image processing method according to claim 2, wherein the calculating the three-dimensional grid of the processed image includes:
inputting the processed image into a preset grid prediction network;
and outputting the three-dimensional grid of the processed image according to the grid prediction network.
4. The image processing method according to claim 1, wherein the calculating the three-dimensional color lookup table of the image to be processed includes:
performing downsampling processing on the image to be processed to obtain a processed image;
and calculating a three-dimensional color lookup table of the image to be processed according to the processed image.
5. The image processing method according to claim 4, wherein the calculating a three-dimensional color lookup table of the image to be processed from the processed image includes:
inputting the processed image into a preset weight prediction network to obtain the linear weight of the processed image;
and calculating a three-dimensional color lookup table of the image to be processed according to the linear weight.
6. The image processing method according to claim 5, wherein the calculating the three-dimensional color lookup table of the image to be processed based on the linear weights includes:
And linearly combining the basic color lookup table with the linear weight to obtain the three-dimensional color lookup table.
7. The image processing method according to claim 1, wherein the performing color correction on the image to be processed based on the three-dimensional grid and the three-dimensional color lookup table performs color correction on the image to be processed based on the three-dimensional grid and the three-dimensional color lookup table, comprising:
discretizing a pixel grid in the three-dimensional grid, and determining pixel points for the tri-linear difference in the three-dimensional grid;
carrying out tri-linear difference calculation on the pixel points for tri-linear difference to obtain pixel points for correction;
correcting the pixel points used for correction according to a three-dimensional color lookup table to obtain a first processed image;
performing tri-linear difference calculation on the pixel points for correction to obtain a first processed image;
and carrying out loss reconstruction processing on the first processed image.
8. An image processing apparatus, characterized in that the apparatus comprises:
the first calculation module is used for calculating a three-dimensional grid of the image to be processed;
the second calculation module is used for calculating a three-dimensional color lookup table of the image to be processed for color calibration;
And the processing module is used for carrying out color correction on the image to be processed based on the three-dimensional grid and the three-dimensional color lookup table.
9. A terminal device, characterized in that it comprises a processor, a memory and a computer program stored in the memory and executable on the processor, the processor executing the computer program to carry out the steps in the image processing method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a computer program that is executed by a processor to implement the steps in the image processing method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210168360.7A CN116703732A (en) | 2022-02-23 | 2022-02-23 | Image processing method, device, terminal equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210168360.7A CN116703732A (en) | 2022-02-23 | 2022-02-23 | Image processing method, device, terminal equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116703732A true CN116703732A (en) | 2023-09-05 |
Family
ID=87843758
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210168360.7A Pending CN116703732A (en) | 2022-02-23 | 2022-02-23 | Image processing method, device, terminal equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116703732A (en) |
-
2022
- 2022-02-23 CN CN202210168360.7A patent/CN116703732A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Jiang et al. | Learning the image processing pipeline | |
| CN100371950C (en) | Method and system for calculating transformed image from digital image | |
| JP2004112694A (en) | Color control method, color control apparatus, color conversion definition editing apparatus, image processing apparatus, program, and storage medium | |
| CN111489322B (en) | Method and device for adding sky filter to static picture | |
| US20060164662A1 (en) | Color conversion using barycentric projections | |
| CN113838134B (en) | Image key point detection method, device, terminal and storage medium | |
| CN113255664B (en) | Image processing method, related device and computer program product | |
| CN113034412B (en) | Video processing method and device | |
| JP4862554B2 (en) | Image processing program and image processing apparatus | |
| CN113766203B (en) | Image white balance processing method | |
| CN116703732A (en) | Image processing method, device, terminal equipment and computer readable storage medium | |
| Tan et al. | A simple gray-edge automatic white balance method with FPGA implementation | |
| CN109934783B (en) | Image processing method, image processing device, computer equipment and storage medium | |
| CN116843566A (en) | Tone mapping method, tone mapping device, display device and storage medium | |
| CN108876870B (en) | Domain mapping GANs image coloring method considering texture complexity | |
| CN115937395A (en) | Electrical equipment model rendering method and device, computer equipment and storage medium | |
| GB2534018A (en) | Method and system for determining parameters of an image processing pipeline of a digital camera | |
| WO2022121893A1 (en) | Image processing method and apparatus, and computer device and storage medium | |
| CN116152586A (en) | Model training method and device, electronic equipment and storage medium | |
| Zhang et al. | Learning a Single Convolutional Layer Model for Low Light Image Enhancement | |
| JP4661754B2 (en) | Image processing apparatus and image processing program | |
| CN112541868B (en) | Image processing method, device, computer equipment and storage medium | |
| US20210383515A1 (en) | Evenly Spaced Curve Sampling Technique for Digital Visual Content Transformation | |
| CN117474820A (en) | Image processing method, device, electronic equipment and storage medium | |
| CN109005411B (en) | Image compression method and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication |