CN116721038A - Color correction methods, electronic equipment and storage media - Google Patents

Abstract

This application discloses a color correction method, electronic device and storage medium, which relate to the technical field of image processing and are used to implement color correction of images, thereby improving the effect of image processing. The method includes: obtaining an image to be processed; obtaining a color correction coefficient of the image to be processed, which is used to correct the color of each pixel point included in the image to be processed on different color channels; based on the color correction coefficient, in the electronic device The first image is displayed on the display.

Description

Color correction methods, electronic equipment and storage media

Technical field

The present application relates to the field of image processing technology, and in particular to a color correction method, electronic equipment and storage media.

Background technique

In the field of image processing technology, high-quality images are an important prerequisite for image processing work. However, due to the influence of environmental factors, camera parameters, etc., the collected images often have problems such as image distortion, image blur, and image defects, resulting in low quality of the collected images and making subsequent image processing more difficult.

At present, when low-quality images are collected, one or more technologies such as image enhancement, image deblurring, and image restoration are usually used to process the collected images to improve the quality of the images.

However, the above-mentioned technologies only rely on image enhancement, image deblurring, image restoration and other technologies, and the effect of image processing is poor.

Contents of the invention

This application provides a color correction method, electronic device and storage medium for realizing color correction of images, thereby improving the effect of image processing.

In order to achieve the above purpose, this application adopts the following technical solutions:

The first aspect provides a color correction method applied to electronic devices.

The method may include: obtaining an image to be processed; obtaining a color correction coefficient of the image to be processed; and displaying the first image on the display screen of the electronic device based on the color correction coefficient.

The color correction coefficient is used to correct the color of each pixel included in the image to be processed on different color channels. In this way, the color correction on different color channels can be realized from the pixel dimension, thereby achieving color correction for the image to improve the effect of image processing.

The first image is an image obtained by performing color correction on the image to be processed based on the color correction coefficient. In this way, by displaying the color-corrected image on the display screen of the electronic device, an image with higher color quality and closer to the true color effect can be displayed to the user, thereby improving the display effect of the image.

In a possible implementation of the first aspect, the image to be processed may be a second image, or the image to be processed may also be an image obtained by image processing the second image. That is to say, the color correction method provided by this application can not only perform color correction on the original second image, but also can perform color correction on the image processed based on the second image. In this way, the types of images to be processed are enriched and the flexibility of color correction is improved.

Wherein, the second image is an image of the first application to be displayed on the display screen. For example, the first application may be a camera application, a gallery application, a video application, a game application, or an editing application. Correspondingly, the above-mentioned second image may be a preview image or a captured image in a camera application, an image saved in a gallery application, a video image to be played in a video application, a game image in a game application, or an image to be uploaded by the user in an editing application. Clip the image.

Correspondingly, when the image to be processed is an image obtained by performing image processing on the second image, obtaining the image to be processed includes: receiving an operation by the user to trigger the first application to display the image; and responding to the triggering of the image display. The first application displays an image to obtain the second image to be displayed; and performs image processing on the second image to obtain the image to be processed. That is to say, the user can trigger the first application to display the second image by performing a trigger operation in the first application. At this time, the user can directly obtain the second image to be displayed, and further, the image-processed image can be obtained through image processing. of images to be processed. In this way, a method of obtaining an image to be processed based on the first application is provided, which can quickly and efficiently obtain the second image to be displayed, and furthermore, can also quickly and efficiently obtain the image-processed image to be processed. image.

Furthermore, displaying the first image on the display screen of the electronic device includes: displaying the first image in the interface of the first application. At this time, the first image displayed in the interface of the first application is an image obtained after color correction of the image to be processed.

The following takes the first application shown above as an example, and takes the image to be processed as an image obtained by performing image processing on the second image as an example to illustrate the process of obtaining the image to be processed.

In another possible implementation of the first aspect, the first application is a camera application, and the second image is a preview image collected by the electronic device in response to the user's operation of opening the camera application, or in response to the user's operation of opening the camera application. The photographing operation in the camera application captures the photographed image captured by the electronic device.

In the case where the image to be processed is an image obtained after image processing of the second image, obtaining the image to be processed includes: receiving an operation of the user to open the camera application; in response to the operation of opening the camera application, obtaining The second image collected by the electronic device; perform image processing on the second image to obtain the image to be processed; or receive the user's shooting operation in the camera application; in response to the shooting operation, obtain the electronic device The captured second image is image-processed to obtain the image to be processed. That is to say, by performing a trigger operation in the camera application (such as the opening operation of the camera application or the shooting operation of the camera application), the user can trigger the camera application to display the preview image or capture the image. At this time, the user can directly obtain the preview image or capture the image. image, and then, through image processing, a preview image or a captured image can be obtained after image processing.

Furthermore, displaying the first image on the display screen of the electronic device includes: displaying the first image in a preview interface of the camera application; or displaying the first image in an image viewing interface of the camera application. At this time, the preview image or captured image displayed in the interface of the camera application is the preview image or captured image obtained after color correction.

In another possible implementation of the first aspect, the first application is a gallery application, and the second image is an image saved in the gallery application.

In the case where the image to be processed is an image obtained by performing image processing on the second image, obtaining the image to be processed includes: receiving an operation of the user to open the gallery application; in response to the operation of opening the gallery application, from The local gallery obtains the second image; performs image processing on the second image to obtain the image to be processed. That is to say, by performing a triggering operation in the gallery application (such as the opening operation of the gallery application), the user can trigger the gallery application to display the images saved in the gallery application. At this time, the user can directly obtain the images saved in the gallery application, and then, Through image processing, the image saved in the gallery application can be obtained after image processing.

Furthermore, displaying the first image on the display screen of the electronic device includes: displaying a thumbnail of the first image in an image list interface of the gallery application. At this time, the image saved in the gallery application displayed in the interface of the gallery application is the image saved in the gallery application obtained after color correction.

In another possible implementation of the first aspect, the first application is a video application, and the second image is a video image to be played in the video application.

In the case where the image to be processed is an image obtained by performing image processing on the second image, obtaining the image to be processed includes: receiving an operation of the user to play a video in the video application; in response to the operation of playing the video, Obtain the second image to be played from the server; perform image processing on the second image to obtain the image to be processed. That is to say, the user can trigger the video application to display the video image to be played by performing a trigger operation in the video application (such as the playback operation of the video application). At this time, the user can directly obtain the video image to be played, and then, through the image After processing, the video image to be played can be obtained after image processing.

Furthermore, displaying the first image on the display screen of the electronic device includes: displaying the first image in a playback interface of the video application. At this time, the video image to be played displayed in the interface of the video application is the video image to be played after color correction.

In another possible implementation of the first aspect, the first application is a game application, and the second image is a game image.

In the case where the image to be processed is an image obtained by performing image processing on the second image, obtaining the image to be processed includes: receiving an operation of the user to start the game application; in response to the operation of starting the game application, from The server obtains the second image to be displayed and performs image processing on the second image to obtain the image to be processed. That is to say, the user can trigger the game application to display the game image by performing a trigger operation in the game application (such as the opening operation of the game application). At this time, the game image can be obtained directly, and then the image can be obtained through image processing. Processed game image.

Furthermore, displaying the first image on the display screen of the electronic device includes: displaying the first image in a game interface of the game application. At this time, the game image displayed in the interface of the game application is the game image obtained after color correction.

In another possible implementation of the first aspect, the first application is an editing application, and the second image is an image to be edited uploaded by the user in the editing application.

In the case where the image to be processed is an image obtained by performing image processing on the second image, obtaining the image to be processed includes: receiving an operation of the user to upload an image in the editing application; in response to the operation of uploading the image, obtaining The uploaded second image; perform image processing on the second image to obtain the image to be processed. That is to say, by performing a trigger operation (such as uploading an image) in the editing application, the user can trigger the editing application to display the uploaded image to be edited. At this time, the user can directly obtain the uploaded image to be edited, and then, through Image processing can obtain the image to be edited after image processing.

Furthermore, displaying the first image on the display screen of the electronic device includes: displaying the first image in an editing interface of the editing application. At this time, the image to be edited displayed in the interface of the editing application is the image to be edited after color correction.

In summary, multiple types of first applications and processes for obtaining images to be processed based on corresponding types of applications are provided. In this way, the second image to be displayed can be obtained quickly and efficiently, and further, the image-processed image can be obtained through image processing.

In another possible implementation of the first aspect, the first color correction model can be used to execute the color correction method provided by the present application. in:

Obtaining the color correction coefficient of the image to be processed includes: inputting the image to be processed into a first color correction model, and predicting the color correction coefficient of the image to be processed through the prediction sub-model of the first color correction model. In this way, by setting the prediction sub-model for predicting the color correction coefficient in the first color correction model, prediction of the color correction coefficient of the image to be processed is achieved.

Based on the color correction coefficient, displaying the first image on the display screen of the electronic device includes: performing color correction on the image to be processed based on the color correction coefficient through the correction sub-model of the first color correction model to obtain the third An image; displaying the first image on the display screen of the electronic device. In this way, by setting the correction sub-model for color correction in the first color correction model, color correction of the image to be processed is achieved.

In this possible implementation, a color correction solution based on the first color correction model is provided. Through the first color correction model, the color correction on different color channels can be implemented from the pixel dimension, thereby achieving color correction for the image to improve the effect of image processing. Furthermore, by displaying the color-corrected image on the display screen of the electronic device, an image with higher color quality and closer to the real color effect can be displayed to the user, thereby improving the display effect of the image.

Based on the two types of images to be processed, the above-mentioned color correction scheme based on the first color correction model will be exemplified below.

For example, when the image to be processed is the second image, obtaining the color correction coefficient of the image to be processed includes: inputting the second image into a first color correction model, and using the predictor of the first color correction model. model to predict the color correction coefficient of the second image. Based on the color correction coefficient, displaying the first image on the display screen of the electronic device includes: performing color correction on the second image based on the color correction coefficient through the correction sub-model of the first color correction model to obtain the third An image; displaying the first image on the display screen of the electronic device.

For another example, when the image to be processed is an image obtained by performing image processing on the second image, obtaining the color correction coefficient of the image to be processed includes: inputting the image processed image into a first color correction model , predict the color correction coefficient of the image after image processing through the prediction sub-model of the first color correction model. Based on the color correction coefficient, displaying the first image on the display screen of the electronic device includes: performing color correction on the processed image based on the color correction coefficient through the correction sub-model of the first color correction model, to obtain The first image; display the first image on the display screen of the electronic device.

In another possible implementation of the first aspect, predicting the color correction coefficient of the image to be processed through the prediction sub-model of the first color correction model includes: using the feature extraction layer of the prediction sub-model, Feature extraction is performed on the image to be processed to obtain the color features of the image to be processed on different color channels; through the convolution layer of the prediction sub-model, the color features of the image to be processed on different color channels are convolved to obtain Color correction coefficients on different color channels of the image to be processed.

In this possible implementation, a feature extraction layer and a convolution layer are set up in the prediction sub-model, the feature extraction layer is used to extract the color features of the image to be processed on different color channels, and the convolution layer is used to extract the extracted color The features are convolved to obtain the color correction coefficient. In this way, a prediction sub-model based on the feature extraction layer and the convolution layer is provided, which can predict the color correction coefficients of the image to be processed on different color channels.

In another possible implementation of the first aspect, when the image to be processed is an image obtained by image processing the second image, a second color correction model can be used to perform the method provided by this application. Color correction methods. in:

Obtaining the image to be processed includes: inputting the second image into a second color correction model, performing image processing on the second image through the processing sub-model of the second color correction model, and obtaining the image to be processed. In this way, by setting the processing sub-model for image processing in the second color correction model, image processing of the image to be processed is achieved.

Obtaining the color correction coefficient of the image to be processed includes: predicting the color correction coefficient of the second image through the prediction sub-model of the second color correction model as the color correction coefficient of the image to be processed. In this way, by setting the prediction sub-model for predicting the color correction coefficient in the second color correction model, prediction of the color correction coefficient is achieved.

Based on the color correction coefficient, displaying the first image on the display screen of the electronic device includes: performing color correction on the image to be processed based on the color correction coefficient through the correction sub-model of the second color correction model to obtain the third An image; displaying the first image on the display screen of the electronic device. In this way, by setting the correction sub-model for color correction in the first color correction model, color correction of the image after image processing is achieved.

In this possible implementation, a color correction solution based on the second color correction model is provided. Among them, a model with both color correction function and image processing function is provided, which can not only perform image processing on the image to be processed, but also effectively improve the color quality of the image, thereby improving the effect of image processing. Furthermore, by displaying the color-corrected image on the display screen of the electronic device, an image with higher color quality and closer to the real color effect can be displayed to the user, thereby improving the display effect of the image.

In another possible implementation of the first aspect, predicting the color correction coefficient of the second image through a prediction sub-model of the second color correction model includes: using a feature extraction layer of the prediction sub-model to predict the color correction coefficient of the second image. Feature extraction is performed on the second image to obtain the color features of the second image on different color channels; through the convolution layer of the prediction sub-model, the color features of the second image on different color channels are convolved to obtain Color correction coefficients for the second image on different color channels.

In this possible implementation, a feature extraction layer and a convolution layer are set up in the prediction sub-model, the feature extraction layer is used to extract the color features of the second image on different color channels, and the convolution layer is used to extract the extracted color The features are convolved to obtain the color correction coefficient. In this way, a prediction sub-model based on the feature extraction layer and the convolution layer is provided, which can predict the color correction coefficients of the second image on different color channels.

In another possible implementation of the first aspect, the feature extraction layer includes at least two layers of target model structures, and the target model structure includes a convolution layer, a batch normalization layer, and an activation function layer.

In this possible implementation, a feature extraction layer based on at least two layers of target model structure is provided. Among them, the target model structure includes a convolution layer, a batch normalization layer and an activation function layer. In this way, the feature extraction layer formed has good feature extraction capabilities and can extract richer and more effective color features, thereby predicting the color correction coefficient of the image to be processed based on the extracted color features, which can improve the accuracy of prediction of the color correction coefficient. sex.

In another possible implementation of the first aspect, the color correction coefficient includes a first correction coefficient on different color channels for each pixel included in the image to be processed. Correspondingly, color correction can be performed based on the first correction coefficient on different color channels of each pixel included in the color correction coefficient.

The corresponding process may include: based on the first correction coefficient of each pixel point included in the image to be processed on different color channels, performing a first adjustment on the color value of each pixel point included in the image to be processed on the corresponding color channel. Process to obtain the first image.

That is to say, by combining the first correction coefficients on different color channels and the color values on the corresponding color channels for the first adjustment process, the color values on the color channels can be corrected from the pixel point dimension, thus achieving the treatment Process the color correction of the image, resulting in a first image with higher color quality.

In another possible implementation of the first aspect, the color correction coefficient further includes a first correction coefficient and a second correction coefficient on different color channels for each pixel included in the image to be processed. Correspondingly, color correction can be performed in combination with the first correction coefficient and the second correction coefficient on different color channels of each pixel included in the color correction coefficient.

The corresponding process may include: based on the second correction coefficient of each pixel point included in the image to be processed on different color channels, performing a second adjustment on the color value of each pixel point included in the image to be processed on the corresponding color channel. Process to obtain the image after the second adjustment process. That is to say, by first combining the second correction coefficients on different color channels and the color values on the corresponding color channels for the second adjustment process, the preliminary correction of the color values on the color channels can be quickly and efficiently implemented from the pixel dimension. .

Furthermore, based on the first correction coefficients of each pixel point included in the image to be processed on different color channels, a first adjustment is performed on the color value of each pixel point included in the image after the second adjustment process on the corresponding color channel. Process to obtain the first image. That is to say, by combining the first correction coefficients on different color channels and the color values on the corresponding color channels after the second adjustment process to perform the first adjustment process, the color values on the color channels can be further adjusted from the pixel dimension. Secondary correction to achieve color correction of the image to be processed, thereby obtaining the first image with higher color quality.

In this possible implementation, a method of performing color correction is provided by combining the first correction coefficient and the second correction coefficient of each pixel on different color channels included in the color correction coefficient.

In another possible implementation of the first aspect, based on the second correction coefficient of each pixel included in the image to be processed on different color channels, the corresponding color of each pixel included in the image to be processed is Before performing the second adjustment process on the color value on the channel, the method further includes: normalizing the second correction coefficient.

In this possible implementation, the second correction coefficient is normalized to eliminate the dimensional influence between different eigenvalues in the second correction coefficient, so that based on the normalized second correction coefficient , to perform the subsequent preliminary correction of the color value to ensure the smooth progress of the color correction.

In another possible implementation of the first aspect, the method further includes: performing image processing on the first image to obtain a processed first image. That is to say, after performing color correction on the image to be processed to obtain the first image, image processing can also be performed on the first image to obtain an image with higher image quality.

Displaying the first image on the display screen of the electronic device includes: displaying the image-processed first image on the display screen of the electronic device. In this way, images with higher image quality can be displayed to the user.

In another possible implementation of the first aspect, the image processing is image enhancement processing, image super-resolution processing, image restoration processing, image repair processing, image deblurring processing, image denoising processing, image rain removal processing, At least one of image defogging, quality improvement, or high dynamic range processing.

In this possible implementation, various types of image processing functions are shown. In this way, the image enhancement scene, image super-resolution scene, image restoration scene, image repair scene, image deblurring scene, image denoising scene, image rain removal scene, image dehazing scene, quality improvement scene or high dynamic range scene can be used. In at least one of the scenarios, color correction is performed on the image to be processed, which broadens the applicable scope of color correction.

In the second aspect, a color correction model training method is provided, which is applied to electronic devices.

The method may include iteratively training an initial model based on image training data to obtain the color correction model.

Among them, during any iterative training process, the image training data is input into the model obtained after the previous iterative training, the color correction coefficient of the image training data is obtained through the model, and the image training data is processed based on the color correction coefficient. Color correction to obtain a color-corrected output image. The color correction coefficient is used to correct the color of each pixel included in the image training data on different color channels.

Furthermore, the model parameters are adjusted based on the output image and the sample image corresponding to the image training data. Among them, the sample image is an image whose color quality meets the preset requirements. That is to say, in the process of model training, the model parameters are adjusted based on the output image of the model and the sample image of the image training data in order to optimize the training of the model and train a model with better color correction capabilities.

In the above technical solution, a method for training a color correction model is provided. Image training data is used to iteratively train the initial model, which can train a model with better color correction capabilities.

In a possible implementation of the second aspect, the color correction coefficient of the image training data is obtained through the model, the color correction is performed on the image training data based on the color correction coefficient, and a color-corrected output image is obtained, including: Through the prediction sub-model of the model, the color correction coefficients of the image training data are predicted. That is to say, by setting the prediction sub-model in the model, the prediction sub-model can be used to predict the color correction coefficient of the image training data during any iterative training process.

And, through the correction sub-model of the model, the image training data is color corrected based on the color correction coefficient to obtain the output image. That is to say, by setting the correction sub-model in the model, during any iterative training process, the correction sub-model is used to perform color correction on the image training data to obtain the output image.

Furthermore, based on the output image and the sample image, the model parameters are adjusted to achieve training optimization of the model, thereby training a model with a color correction function.

In another possible implementation of the second aspect, the color correction coefficient of the image training data is obtained through the model, the color correction coefficient is performed on the image training data based on the color correction coefficient, and a color-corrected output image is obtained, including : Perform image processing on the image training data through the processing sub-model of the model to obtain the image after image processing. That is to say, by setting the processing sub-model in the model, during any iterative training process, the processing sub-model is used to perform image processing on the image training data to obtain the image-processed image.

And, through the prediction sub-model of the model, the color correction coefficient of the image training data is predicted. That is to say, by setting the prediction sub-model in the model, the prediction sub-model can be used to predict the color correction coefficient of the image training data during any iterative training process.

And, through the correction sub-model of the model, the color correction is performed on the image after image processing based on the color correction coefficient to obtain the output image. That is to say, by setting the correction sub-model in the model, during any iterative training process, the correction sub-model is used to perform color correction on the image training data to obtain the output image.

Furthermore, based on the output image and the sample image, the model parameters are adjusted to achieve training optimization of the model, thereby training a model that has both color correction function and image processing function.

In another possible implementation of the second aspect, the image processing is image enhancement processing, image super-resolution processing, image restoration processing, image repair processing, image deblurring processing, image denoising processing, image rain removal processing, At least one of image defogging, quality improvement, or high dynamic range processing.

In this possible implementation, various types of image processing functions are shown. In this way, at least one of the functions of image enhancement, image super-resolution, image restoration, image repair, image deblurring, image denoising, image rain removal, image dehazing, quality improvement or high dynamic range can be combined to train to obtain at least one of them. A model of image processing functions.

In another possible implementation of the second aspect, predicting the color correction coefficient of the image training data through the prediction sub-model of the model includes: performing the following steps on the image training data through the feature extraction layer of the prediction sub-model. Feature extraction is used to obtain the color features of the image training data on different color channels; through the convolution layer of the prediction sub-model, the color features of the image training data on different color channels are convolved to obtain the image training data. Color correction coefficients on different color channels.

In this possible implementation, a feature extraction layer and a convolution layer are set up in the prediction sub-model, the feature extraction layer is used to extract the color features of the image training data on different color channels, and the convolution layer is used to extract the extracted color The features are convolved to obtain the color correction coefficient. In this way, a prediction sub-model based on the feature extraction layer and the convolution layer is provided, which can predict the color correction coefficient of the image training data.

In another possible implementation of the second aspect, the feature extraction layer includes at least two layers of target model structures, and the target model structure includes a convolution layer, a batch normalization layer, and an activation function layer.

In this possible implementation, a feature extraction layer based on at least two layers of target model structure is provided. Among them, the target model structure includes a convolution layer, a batch normalization layer and an activation function layer. In this way, the feature extraction layer formed has good feature extraction capabilities and can extract richer and more effective color features, thereby predicting the color correction coefficients of image training data based on the extracted color features, which can improve the accuracy of prediction of color correction coefficients. sex.

In another possible implementation of the second aspect, the color correction coefficient includes a first correction coefficient on different color channels of each pixel included in the image training data. Correspondingly, color correction can be performed based on the first correction coefficient on different color channels of each pixel included in the color correction coefficient.

The corresponding process may include: based on the first correction coefficient of each pixel point included in the image training data on different color channels, performing a first adjustment on the color value of each pixel point included in the image training data on the corresponding color channel. Process to get the output image.

That is to say, by combining the first correction coefficients on different color channels and the color values on the corresponding color channels for the first adjustment process, the color values on the color channels can be corrected from the pixel point dimension, which also realizes the correction of the color values on the color channels. Color correction of image training data, resulting in output images with higher color quality.

In another possible implementation of the second aspect, the color correction coefficient includes a first correction coefficient and a second correction coefficient on different color channels for each pixel included in the image training data. Correspondingly, color correction can be performed in combination with the first correction coefficient and the second correction coefficient on different color channels of each pixel included in the color correction coefficient.

The corresponding process may include: based on the second correction coefficient of each pixel point included in the image training data on different color channels, performing a second adjustment on the color value of each pixel point included in the image training data on the corresponding color channel. Process to obtain the image after the second adjustment process. That is to say, by first combining the second correction coefficients on different color channels and the color values on the corresponding color channels for the second adjustment process, the preliminary correction of the color values on the color channels can be quickly and efficiently implemented from the pixel dimension. .

Furthermore, based on the first correction coefficients of each pixel point included in the image training data on different color channels, a first adjustment is performed on the color value of each pixel point included in the second adjustment processed image on the corresponding color channel. Process to get the output image. That is to say, by combining the first correction coefficients on different color channels and the color values on the corresponding color channels after the second adjustment process to perform the first adjustment process, the color values on the color channels can be further adjusted from the pixel dimension. Secondary correction, thereby achieving color correction of the image training data, thereby obtaining an output image with higher color quality.

In this possible implementation, a method of performing color correction is provided by combining the first correction coefficient and the second correction coefficient of each pixel on different color channels included in the color correction coefficient.

In another possible implementation of the second aspect, based on the second correction coefficient of each pixel included in the image training data on different color channels, the corresponding color of each pixel included in the image training data is Before performing the second adjustment process on the color value on the channel, the method further includes: normalizing the second correction coefficient.

In this possible implementation, the second correction coefficient is normalized to eliminate the dimensional influence between different eigenvalues in the second correction coefficient, so that based on the normalized second correction coefficient , to perform the subsequent preliminary correction of the color value to ensure the smooth progress of the color correction.

In a third aspect, this application provides an electronic device, including: a display screen, a processor, and a memory. The display screen provides a display function. The memory is used to store the program code, and the processor is used to call the program code stored in the memory, thereby implementing any method provided in the first aspect or the second aspect.

A fourth aspect provides a computer-readable storage medium, including program code. When the program code is run on an electronic device, it causes the electronic device to execute any method provided in the first aspect or the second aspect.

A fifth aspect provides a computer program product, including program code. When the program code is run on an electronic device, it causes the electronic device to execute any method provided in the first aspect or the second aspect.

It should be noted that the technical effects brought by any implementation method in the third to fifth aspects can be referred to the technical effects brought by the corresponding implementation method in the first aspect, and will not be described again here.

Description of the drawings

Figure 1 is a schematic diagram of an electronic device provided by an embodiment of the present application;

Figure 2 is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present application;

Figure 3 is a schematic diagram of the software structure of an electronic device provided by an embodiment of the present application;

Figure 4 is a schematic flow chart of a color correction method provided by an embodiment of the present application;

Figure 5 is a schematic flow chart of a color correction method provided by an embodiment of the present application;

Figure 6 is a schematic diagram of a before and after comparison of color correction provided by an embodiment of the present application;

Figure 7 is a schematic flowchart of a training method for a first color correction model provided by an embodiment of the present application;

Figure 8 is a schematic flowchart of a training method for a second color correction model provided by an embodiment of the present application;

Figure 9 is a schematic framework diagram of a color correction model training device provided by an embodiment of the present application;

Figure 10 is a schematic framework diagram of a color correction model training device provided by an embodiment of the present application.

Detailed ways

In the description of this application, unless otherwise stated, "/" means "or". For example, A/B can mean A or B. "And/or" in this article is just an association relationship that describes related objects, indicating that there can be three relationships, for example, A and/or B, which can mean: A exists alone, A and B exist simultaneously, and B exists alone these three situations. In addition, "at least one" means one or more, and "plurality" means two or more. Words such as "first" and "second" do not limit the quantity and order of execution, and words such as "first" and "second" do not limit the number or order of execution.

It should be noted that in the embodiments of this application, words such as "exemplary" or "for example" are used to represent examples, illustrations or explanations. Any embodiment or design described herein as "exemplary" or "such as" is not intended to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the words "exemplary" or "such as" is intended to present the concept in a concrete manner.

The color correction method provided by the embodiments of the present application can be applied in the field of image processing technology, and specifically can be applied in color correction scenarios for images to be processed.

Furthermore, in some embodiments, the color correction method provided by the embodiments of the present application can be applied to image enhancement (image enhancement), image super-resolution (super-resolution), image restoration (imagerestoration), and image inpainting (inpainting). , image deblurring, image denoising, image deraining, image dehazing, quality improvement algorithm (low-level algorithm), high-dynamic range, HDR) and other image processing scenarios.

Among them, image enhancement refers to enhancing the contrast of an image to make an originally unclear image clear or to emphasize certain features of interest. Image super-resolution refers to increasing the resolution of the image to enrich the texture details of the image. Image restoration refers to restoring the real image with maximum fidelity from the obtained degraded image. Image restoration refers to repairing the lost parts of the image and reconstructing them based on background information. Image deblurring refers to removing motion blur from a blurred image to restore the blurred image to a clear image. Image denoising refers to reducing noise in digital images. Image rain removal refers to removing raindrops from images. Image defogging refers to processing haze images to eliminate or weaken the impact of haze on the image. Quality improvement algorithm refers to an algorithm that restores low-quality images to high-quality images. High dynamic range is used to provide more color range and image details to improve image contrast between light and dark.

In the above image processing scenario, the color correction method provided by the embodiment of the present application can also be applied. In this way, not only the above image processing can be performed on the image to be processed, but also the color correction of the image to be processed can be achieved.

The above image processing scene may be a scene requiring image processing in an electronic device. For example, the image processing scene may be in the shooting scene of the electronic device. Correspondingly, for the preview image or the photographed image obtained during the photographing process, the color correction method provided by the embodiment of the present application can be used to perform color correction on the preview image or the photographed image. Alternatively, the image processing scene may be in the video playback scene of the electronic device. Correspondingly, for the video image to be played, the color correction method provided by the embodiment of the present application can be used to perform color correction on the video image to be played. Of course, the above image processing scenes can also be in other scenes of electronic devices, such as image editing scenes, game screen display scenes, etc., which are not limited in the embodiments of the present application.

In related technologies, when low-quality images are collected, one or more of the above-mentioned image enhancement, image deblurring, image restoration, etc. technologies are usually used to process the collected images to improve the quality of the images. However, in related technologies, only relying on image enhancement, image deblurring, image restoration and other technologies, the effect of image processing is poor.

In view of this, embodiments of the present application provide a color correction method. By obtaining the color correction coefficient of the image to be processed, the color correction on different color channels can be implemented from the pixel point dimension, thereby achieving color correction for the image. to improve the effect of image processing. Furthermore, by displaying the color-corrected image on the display screen of the electronic device, an image with higher color quality and closer to the real color effect can be displayed to the user, thereby improving the display effect of the image.

In a possible implementation, the color correction method provided by the embodiment of the present application can be applied to the electronic device 100 as shown in FIG. 1 . Illustratively, FIG. 1 is a schematic diagram of an electronic device provided by an embodiment of the present application.

The electronic device 100 may be at least one of a smart phone, a smart watch, a desktop computer, a laptop computer, a virtual reality terminal, an augmented reality terminal, a wireless terminal, a laptop computer, and other devices.

The electronic device 100 provides an image processing function. In some embodiments, the electronic device 100 may run applications such as cameras, galleries, maps, navigation, videos, games, etc. that support image processing. For example, when the camera application running on the electronic device 100 is used to capture an image, the color correction method provided by the embodiment of the present application can be applied to perform color correction on the preview image or the captured image. Alternatively, when a video application running on the electronic device 100 is used to play a video, the color correction method provided by the embodiment of the present application can be applied to perform color correction on the video image to be played. Of course, the electronic device 100 can also run other types of applications that support image processing, such as editing, photo editing, and other applications.

In the embodiment of the present application, the electronic device 100 is used to obtain an image to be processed; obtain a color correction coefficient of the image to be processed, and perform color correction on different color channels of each pixel included in the image to be processed based on the color correction coefficient. Correction is performed to obtain a color-corrected first image, and the first image is displayed on the display screen of the electronic device 100 .

For example, a schematic structural diagram of the electronic device 100 in FIG. 1 is shown in FIG. 2 . FIG. 2 is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present application.

Referring to FIG. 2 , the electronic device 100 may include a processor 210 , an external memory interface 220 , an internal memory 221 , a universal serial bus (USB) interface 230 , a charging management module 240 , an antenna 1 , an antenna 2 , and a mobile communication module. 250. Wireless communication module 260, audio module 270, sensor module 280, button 290, motor 291, indicator 292, camera 293, display screen 294, subscriber identification module (subscriber identification module, SIM) card interface 295, etc. The sensor module 280 may include a pressure sensor 280A, a touch sensor 280B, and the like.

It can be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100 . In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figures, or some components may be combined, some components may be separated, or some components may be arranged differently. The components illustrated may be implemented in hardware, software, or a combination of software and hardware.

The processor 210 may include one or more processing units. For example, the processor 210 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (GPU), an image signal processor ( image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and/or neural-network processing unit (NPU), etc. . Among them, different processing units can be independent devices or integrated in one or more processors.

The controller may be the nerve center and command center of the electronic device 100 . The controller can generate operation control signals based on the instruction operation code and timing signals to complete the control of fetching and executing instructions.

Memory is used to store instructions (program code) and data. In some embodiments, the memory in processor 210 is cache memory. This memory may hold instructions or data that have been recently used or recycled by processor 210 . If the processor 210 needs to use the instruction or data again, it can be called directly from the memory. Repeated access is avoided and the waiting time of the processor 210 is reduced, thus improving the efficiency of the system.

NPU is a neural-network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can continuously learn by itself. Intelligent cognitive applications of the electronic device 100 can be implemented through the NPU, such as image recognition, face recognition, speech recognition, text understanding, etc. For example, in this embodiment of the present application, the electronic device 100 can provide a first color correction model or a second color correction model through the NPU to implement the color correction method provided by the embodiment of the present application, thereby achieving color correction of the image to be processed.

In some embodiments, processor 210 may include one or more interfaces. Interfaces may include integrated circuit (inter-integrated circuit, I2C) interface, integrated circuit built-in audio (inter-integrated circuitsound, I2S) interface, pulse code modulation (PCM) interface, universal asynchronous receiver (universal asynchronous receiver) /transmitter, UART) interface, mobile industry processor interface (MIPI), general-purpose input/output (GPIO) interface, subscriber identity module (subscriber identity module, SIM) interface, and/or Universal serial bus (USB) interface, etc.

In the embodiment of the present application, the processor 210 is used to call the program code stored in the memory. When the program code is run on the electronic device 100, the electronic device 100 executes the color correction method in the embodiment of the present application.

The charge management module 240 is used to receive charging input from the charger.

The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 250, the wireless communication module 260, the modem processor and the baseband processor, etc.

Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 may be used to cover a single or multiple communication frequency bands.

The mobile communication module 250 can provide solutions for wireless communication including 2G/3G/4G/5G applied on the electronic device 100 . The mobile communication module 250 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 250 can receive electromagnetic waves through the antenna 1, perform filtering, amplification and other processing on the received electromagnetic waves, and transmit them to the modem processor for demodulation. The mobile communication module 250 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves through the antenna 1 for radiation. In some embodiments, at least part of the functional modules of the mobile communication module 250 may be disposed in the processor 210 . In some embodiments, at least part of the functional modules of the mobile communication module 250 and at least part of the modules of the processor 210 may be provided in the same device.

The wireless communication module 260 can provide applications on the electronic device 100 including wireless local area networks (WLAN) (such as wireless fidelity (wireless fidelity, Wi-Fi) network), Bluetooth (bluetoo2, BT), global navigation satellite system Wireless communication solutions such as global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (infrared, IR), etc. The wireless communication module 260 may be one or more devices integrating at least one communication processing module. The wireless communication module 260 receives electromagnetic waves via the antenna 2 , frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 210 . The wireless communication module 260 can also receive the signal to be sent from the processor 210, perform frequency modulation and amplification on it, and convert it into electromagnetic waves through the antenna 2 for radiation.

In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 250, and the antenna 2 is coupled to the wireless communication module 260, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (codedivision multiple access, CDMA), broadband code division Multiple access (wideband code division multiple access, WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and/or IR technology, etc.

The electronic device 100 implements display functions through a GPU, a display screen 294, an application processor, and the like. The GPU is an image processing microprocessor and is connected to the display screen 294 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. Processor 210 may include one or more GPUs that execute program instructions to generate or alter display information.

The display screen 294 is used to display images, videos, etc. Display 294 includes a display panel. The display panel can use a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light emitting diode or an active matrix organic light emitting diode (active-matrix organic light emitting diode). (AMOLED), flexible light-emitting diodes (FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diodes (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 294, where N is a positive integer greater than 1. For example, in the embodiment of the present application, the display screen 294 is used to display the first image obtained through color correction.

The electronic device 100 can implement the shooting function through an ISP, a camera 293, a video codec, a GPU, a display screen 294, an application processor, and the like. For example, in the embodiment of the present application, when the electronic device 100 implements the shooting function through the ISP, camera 293, video codec, GPU, display screen 294, application processor, etc., the color correction method provided by the embodiment of the present application can be applied. Color correct the preview image during shooting or the captured image after shooting.

Camera 293 is used to capture still images or video. The object passes through the lens to produce an optical image that is projected onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to convert it into a digital image signal. ISP outputs digital image signals to DSP for processing. DSP converts digital image signals into standard RGB, YUV and other format image signals. In some embodiments, the electronic device 100 may include 1 or N cameras 293, where N is a positive integer greater than 1.

The external memory interface 220 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100 . The external memory card communicates with the processor 210 through the external memory interface 220 to implement the data storage function. For example, save music, video and other files on an external memory card.

Internal memory 221 may be used to store computer executable program code, which includes instructions. The processor 210 executes instructions stored in the internal memory 221 to execute various functional applications and data processing of the electronic device 100 . The internal memory 221 may include a program storage area and a data storage area. Among them, the stored program area can store the operating system, at least one application program required for a function (such as a sound playback function, an image playback function, etc.), etc. The storage data area may store data created during use of the electronic device 100 (such as audio data, phone book, etc.). In addition, the internal memory 221 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

The electronic device 100 may implement audio functions through the audio module 270 . For example, music playback, recording, etc.

The pressure sensor 280A is used to sense pressure signals and can convert the pressure signals into electrical signals. In some embodiments, pressure sensor 280A may be disposed on display screen 294.

Touch sensor 280B is also called a "touch panel". The touch sensor 280B can be disposed on the display screen 294. The touch sensor 280B and the display screen 294 form a touch screen, which is also called a "touch screen". Touch sensor 280B is used to detect touch operations on or near it.

The buttons 290 include a power button, a volume button, etc. Key 290 may be a mechanical key. It can also be a touch button. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100 .

The motor 291 can generate vibration prompts. The motor 291 can be used for vibration prompts for incoming calls and can also be used for touch vibration feedback.

The indicator 292 may be an indicator light, which may be used to indicate charging status, power changes, or may be used to indicate messages, missed calls, notifications, etc.

The SIM card interface 295 is used to connect a SIM card. The SIM card can be connected to or separated from the electronic device 100 by inserting it into the SIM card interface 295 or pulling it out from the SIM card interface 295 . The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1.

It should be noted that the structure shown in Figure 2 does not constitute a limitation of the electronic device. In addition to the components shown in Figure 2, the electronic device may also include more or less components than those shown in the figure, or a combination thereof. Certain parts, or different arrangements of parts.

Software systems of electronic devices can adopt layered architecture, event-driven architecture, microkernel architecture, microservice architecture or cloud architecture. The embodiment of this application takes the Android system with a layered architecture as an example to illustrate the software structure of the electronic device. Exemplarily, FIG. 3 is a schematic diagram of the software structure of an electronic device provided by an embodiment of the present application.

The layered architecture divides the software into several layers, and each layer has clear roles and division of labor. The layers communicate through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: application layer, application framework layer, Android runtime (Android runtime), system library and kernel layer.

The application layer can include a series of application packages. As shown in Figure 3, the application package can include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, game, short message, etc.

Among them, for applications that support image processing, such as galleries, maps, navigation, videos, games, etc., the color correction method provided by the embodiments of the present application can be used to perform color correction on images to be displayed or played.

The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

As shown in Figure 3, the application framework layer can include window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

System libraries can include multiple functional modules. For example: surface manager, media libraries, three-dimensional graphics processing library (for example: OpenGL ES), two-dimensional (2D) graphics engine (for example: SGL), etc.

The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

FIG. 4 is a schematic flowchart of a color correction method provided by an embodiment of the present application. Referring to Figure 4, the solution is explained by taking color correction based on the first color correction model as an example. The method includes the following S401-S404:

S401. The electronic device obtains the image to be processed.

In this embodiment of the present application, the image to be processed may be the second image. Alternatively, the image to be processed may be an image obtained by performing image processing on the second image. In this way, two types of images to be processed are provided, enriching the types of images to be processed. Furthermore, color correction can be performed not only on the original image to be displayed, but also on the image after image processing, which improves the flexibility of color correction.

The second image is an image of the first application to be displayed on the display screen. In some embodiments, when the image to be processed is a second image, the process of obtaining the image to be processed may be: receiving an operation by the user to trigger the first application to display the image, and responding to the operation of triggering the first application to display the image. Operation to obtain the second image to be displayed.

In the above embodiment, a process of obtaining an image to be processed based on the first application is provided. Wherein, by the user triggering the operation of the first application to display an image, the second image to be displayed can be obtained quickly and efficiently.

For example, taking the first application as a camera application as an example, the second image may be a preview image collected by the electronic device in response to the user's operation of opening the camera application, or in response to the user's shooting operation in the camera application. Captured images captured by the device. Correspondingly, taking the preview image as an example, the process of obtaining the image to be processed may be: receiving the user's operation of opening the camera application, and in response to the operation of opening the camera application, obtaining the second image collected by the electronic device (camera). image. It should be understood that the preview image refers to an image collected when the shooting operation is not performed. Taking photographing an image as an example, the process of obtaining an image to be processed may be: receiving a user's photographing operation in the camera application, and in response to the photographing operation, obtaining the second image photographed by the electronic device. It should be understood that the captured image refers to the image collected after performing the capturing operation.

For another example, assuming that the first application is a gallery application, the second image may be an image saved in the gallery application. Correspondingly, taking the image saved in the gallery application as an example, the process of obtaining the image to be processed may be: receiving the user's operation of opening the gallery application, and in response to the operation of opening the gallery application, obtaining the second image from the local gallery.

For another example, assuming that the first application is a video application, the second image may be a video image to be played in the video application. Correspondingly, taking the video image to be played as an example, the process of obtaining the image to be processed may be: receiving the user's operation of playing the video in the video application, and in response to the operation of playing the video, obtaining the second image to be played from the server. image. Among them, the server can store the video stream (including multiple frames of images) of the video to be played.

For another example, assuming that the first application is a game application, the second image may be a game image in the game application. Correspondingly, taking game images as an example, the process of obtaining the image to be processed may be: receiving the user's operation to start the game application, and in response to the operation of starting the game application, obtaining the second image to be displayed from the server. Among them, the server can also store game images to be displayed.

For another example, assuming that the first application is an editing application, the second image may be an image to be edited uploaded by the user in the editing application. Correspondingly, taking the image to be edited as an example, the process of obtaining the image to be processed may be: receiving an operation of the user to upload an image in the editing application, and in response to the operation of uploading the image, obtaining the uploaded second image.

In the embodiment of the present application, image processing may be image enhancement processing, image super-resolution processing, image restoration processing, image repair processing, image deblurring processing, image denoising processing, image rain removal processing, image dehazing processing, and quality improvement processing. or at least one of high dynamic range processing. In this embodiment, various types of image processing functions are shown. In this way, the image enhancement scene, image super-resolution scene, image restoration scene, image repair scene, image deblurring scene, image denoising scene, image rain removal scene, image dehazing scene, quality improvement scene or high dynamic range scene can be used. In at least one of the scenarios, color correction is performed on the image to be processed, which broadens the applicable scope of color correction.

In some embodiments, when the image to be processed is an image obtained by performing image processing on the second image, the above process of obtaining the image to be processed may be: receiving an operation from the user to trigger the first application to display the image, In response to the operation of triggering the first application to display an image, the second image to be displayed is obtained. Perform image processing on the second image to obtain the image to be processed.

In the above embodiment, a process of obtaining an image to be processed based on the first application is provided. Among them, by the user triggering the operation of the first application to display the image, the second image to be displayed can be quickly and efficiently obtained, and further, the image-processed image to be processed can be obtained through image processing.

For example, taking the preview image as an example, the process of obtaining the image to be processed may be: receiving the user's operation of opening the camera application, and in response to the operation of opening the camera application, obtaining the second image collected by the electronic device. Perform image processing on the second image to obtain the image to be processed. Taking photographing an image as an example, the process of obtaining an image to be processed may be: receiving a user's photographing operation in the camera application, and in response to the photographing operation, obtaining the second image photographed by the electronic device. Perform image processing on the second image to obtain the image to be processed.

For another example, taking the image saved in the gallery application as an example, the process of obtaining the image to be processed may be: receiving the user's operation to open the gallery application, and in response to the operation of opening the gallery application, obtaining the second image from the local gallery. Perform image processing on the second image to obtain the image to be processed.

For another example, taking the video image to be played as an example, the process of obtaining the image to be processed may be: receiving the user's operation of playing the video in the video application, and in response to the operation of playing the video, obtaining the second image to be played from the server. image. Perform image processing on the second image to obtain the image to be processed.

For another example, taking a game image as an example, the process of obtaining the image to be processed may be: receiving the user's operation of opening the game application, and in response to the operation of opening the game application, obtaining the second image to be displayed from the server. Perform image processing on the second image to obtain the image to be processed.

For another example, taking the image to be edited as an example, the process of obtaining the image to be processed may be: receiving the user's operation of uploading an image in the editing application, and in response to the operation of uploading the image, obtaining the uploaded second image. Perform image processing on the second image to obtain the image to be processed.

In the above example, multiple types of first applications and a process of obtaining an image to be processed based on the corresponding types of applications are provided. In this way, the second image to be displayed can be obtained quickly and efficiently, and further, the image-processed image can be obtained through image processing.

It should be noted that the various images to be processed shown above are only used as examples to illustrate the process of the electronic device acquiring the images to be processed. In other embodiments, the image to be processed can also be other types of images, and accordingly, the electronic device can also use other methods to obtain the image to be processed. The embodiments of this application do not limit the process of obtaining the image to be processed.

S402. The electronic device inputs the image to be processed into the first color correction model, and predicts the color correction coefficient of the image to be processed through the prediction sub-model of the first color correction model.

In the embodiment of the present application, the prediction sub-model is used to predict the color correction coefficient of the image to be processed. In some embodiments, the prediction sub-model may be a color coefficient prediction module in the first color correction model.

The color correction coefficient is used to correct the color of each pixel included in the image to be processed on different color channels. It should be understood that the image to be processed may include color values of each pixel on different color channels.

The color correction coefficient may be in the form of an n*3 feature vector. Among them, n represents the total number of pixels in the image to be processed, and n is a positive integer greater than 1. 3 represents the three color channels of RGB (i.e. red-R, green-G, blue-B). Correspondingly, the image to be processed can also be in the form of n*3 feature vectors. It should be understood that in the n*3 feature vector, one pixel corresponds to the correction coefficient values on three color channels.

In some embodiments, the color correction coefficient may include a first correction coefficient on different color channels for each pixel included in the image to be processed. In the embodiment of the present application, the first correction coefficient is used to perform color correction through a first adjustment process (such as a summation operation).

Further, the color correction coefficient may also include a second correction coefficient on different color channels for each pixel included in the image to be processed. In the embodiment of the present application, the second correction coefficient is used to perform color correction through a second adjustment process (such as a product operation). Wherein, the adjustment intensity of the first adjustment process is smaller than the adjustment intensity of the second adjustment process.

Referring to Figure 4, the first correction coefficient may be an offset coefficient tensor, that is, the n*3 feature vector (r ₄ , r ₅ , r ₆ ) shown in Figure 4. The second correction coefficient may be an adjustment coefficient tensor, that is, the n*3 feature vector (r ₁ , r ₂ , r ₃ ) shown in Figure 4 .

In some embodiments, the above-mentioned process of predicting the color correction coefficient of the image to be processed through the prediction sub-model may be: performing feature extraction on the image to be processed through the feature extraction layer of the prediction sub-model to obtain the image to be processed. Color features on different color channels. Through the convolution layer of the prediction sub-model, the color features of the image to be processed on different color channels are convolved to obtain the color correction coefficients of the image to be processed on different color channels.

In the process shown in S402 above, the process of the electronic device obtaining the color correction coefficient of the image to be processed is explained. Among them, by setting up a feature extraction layer and a convolution layer in the prediction sub-model, the feature extraction layer is used to extract the color features of the image to be processed on different color channels, and the convolution layer is used to perform convolution processing on the extracted color features. Get the color correction coefficient. In this way, a prediction sub-model based on the feature extraction layer and the convolution layer is provided, which can predict the color correction coefficients of the image to be processed on different color channels.

Wherein, the feature extraction layer may include at least two layers of target model structure, and the target model structure includes a convolution layer, a batch normalization layer and an activation function layer. For example, the target model structure may be a conv+bn+relu model structure. Referring to Figure 4, take the feature extraction layer including a two-layer target model structure as an example. The two-layer target model structure can be the two-layer conv+bn+relu model structure shown in Figure 4. The convolutional layer of the prediction sub-model can be the conv layer shown in Figure 4.

It should be understood that in the conv+bn+relu model structure, conv represents the convolution layer, which is used to perform convolution operations. bn (batch normalization) represents the batch normalization layer, which is used to adjust the distribution of the output data of the convolution layer so that it enters the action area of the activation function layer to speed up the convergence speed of the model, improve the generalization ability of the model and prevent the gradient from disappearing. relu represents the activation function layer, which is used to increase the nonlinear relationship between each layer to define the output of the model.

In the above embodiment, a feature extraction layer based on at least two layers of target model structure is provided. Among them, the target model structure includes a convolution layer, a batch normalization layer and an activation function layer. In this way, the feature extraction layer formed has good feature extraction capabilities and can extract richer and more effective color features, thereby predicting the color correction coefficient of the image to be processed based on the extracted color features, which can improve the accuracy of prediction of the color correction coefficient. sex.

S403. The electronic device uses the correction sub-model of the first color correction model to perform color correction on the image to be processed based on the color correction coefficient to obtain the first image.

In the embodiment of the present application, the first image is an image obtained by performing color correction on the image to be processed based on the color correction coefficient. For example, assuming that the image to be processed is in the form of n*3 feature vectors, correspondingly, the first image is also in the form of n*3 feature vectors.

In some embodiments, for example, the color correction coefficient includes the first correction coefficient on different color channels of each pixel included in the image to be processed. Correspondingly, the above color correction process may include: based on the image to be processed The first correction coefficient of each pixel included in the image on different color channels is used to perform a first adjustment process on the color value of each pixel included in the image to be processed on the corresponding color channel to obtain the first image.

Wherein, the above-mentioned first adjustment process may be: based on the first correction coefficient of each pixel point included in the image to be processed on different color channels, and the first correction coefficient of each pixel point included in the image to be processed on the corresponding color channel. The color values are summed to obtain the first image.

The above embodiment describes the color correction process by taking the first correction coefficient included in the color correction coefficient as an example. In this way, a method of performing color correction based on the first correction coefficient on different color channels of each pixel included in the color correction coefficient is provided. Among them, by combining the first correction coefficients on different color channels and the color values on the corresponding color channels for the first adjustment process, the color values on the color channels can be corrected from the pixel point dimension, which also realizes the image to be processed. of color correction, resulting in a first image with higher color quality.

In other embodiments, for example, the color correction coefficient includes the first correction coefficient and the second correction coefficient on different color channels of each pixel included in the image to be processed. Correspondingly, the above color correction process may include : Based on the second correction coefficient of each pixel point included in the image to be processed on different color channels, perform a second adjustment process on the color value of each pixel point included in the image to be processed on the corresponding color channel to obtain the second correction coefficient 2. Adjust the processed image. Based on the first correction coefficients of each pixel point included in the image to be processed on different color channels, perform a first adjustment process on the color value of each pixel point included in the image after the second adjustment process on the corresponding color channel, Get the first image.

Wherein, the process of the above-mentioned second adjustment process may be: based on the second correction coefficient of each pixel included in the image to be processed on different color channels, and the second correction coefficient of each pixel included in the image to be processed on the corresponding color channel. The color values are multiplied to obtain the image after the second adjustment process. Furthermore, based on the first correction coefficient of each pixel point included in the image to be processed on different color channels, a summation operation is performed with the color value of each pixel point included in the second adjusted image on the corresponding color channel. , get the first image.

The above embodiment describes the color correction process by taking the combination of the first correction coefficient and the second correction coefficient included in the color correction coefficient as an example. In this way, a method of performing color correction is provided by combining the first correction coefficient and the second correction coefficient of each pixel included in the color correction coefficient on different color channels. Among them, the second adjustment process is first performed by combining the second correction coefficients on different color channels and the color values on the corresponding color channels, so that the preliminary correction of the color values on the color channels can be quickly and efficiently implemented from the pixel point dimension. Furthermore, by performing the first adjustment process in combination with the first correction coefficients on different color channels and the color values on the corresponding color channels after the second adjustment process, it is possible to further realize the second adjustment of the color values on the color channels from the pixel point dimension. correction, thereby achieving color correction of the image to be processed, thereby obtaining a first image with higher color quality.

In some embodiments, before performing the above-mentioned second adjustment process, the second correction coefficient is also normalized. Then, based on the normalized second correction coefficient, a subsequent second adjustment process is performed.

Among them, the normalization process refers to scaling the characteristic value of the color correction coefficient to the [0,1] interval.

In some embodiments, the second correction coefficients on different color channels of each pixel included in the color correction coefficient are normalized through a sigmoid function to obtain the normalized second correction coefficient.

For example, referring to Figure 4, the above-mentioned adjustment coefficient tensor, that is, the n*3 feature vector (r ₁ , r ₂ , r ₃ ) shown in Figure 4, is normalized through the sigmoid function, and then, using The normalized adjustment coefficient tensor performs the subsequent second adjustment process.

In this way, the second correction coefficient is normalized to eliminate the dimensional influence between different eigenvalues in the second correction coefficient, so that subsequent color values can be performed based on the normalized second correction coefficient. preliminary correction to ensure the smooth progress of color correction.

Based on the above S402 to S403, a color correction method is provided, which can realize color correction for the image to be processed. Among them, by obtaining the color correction coefficient of the image to be processed, the color correction on different color channels can be implemented from the pixel point dimension, thereby achieving color correction for the image to improve the effect of image processing. Furthermore, by displaying the color-corrected image on the display screen of the electronic device, an image with higher color quality and closer to the real color effect can be displayed to the user, thereby improving the display effect of the image.

In some embodiments, the electronic device may be provided with a first control control for triggering turning on or off the color correction function. For example, the user can turn on or off the color correction function by performing a triggering operation on the first control control. For example, the trigger operation may be a click operation, a sliding operation, or other operations.

Accordingly, in some embodiments, in response to the user's opening operation of the first control control, after acquiring the image to be processed, the electronic device performs subsequent S402 to S403 to display the color-corrected image on the display screen of the electronic device. First image. Or, in other embodiments, in response to the user's closing operation of the first control control, the electronic device displays the image to be processed on the display screen of the electronic device without performing subsequent S402 to S403 after acquiring the image to be processed. .

S404. The electronic device displays the first image on the display screen of the electronic device.

In some embodiments, the electronic device displays the first image in an interface of the first application.

For example, assuming that the first application is a camera application, the first image is displayed in the preview interface of the camera application. The preview interface may be an interface including shooting controls in a camera application. Or, display the first image in the image viewing interface of the camera application. The image viewing interface may be an interface in a camera application used to trigger viewing of captured images.

For another example, assuming that the first application is a gallery application, the thumbnail of the first image is displayed in the image list interface of the gallery application. For another example, taking the first application as a video application, the first image is displayed in the playback interface of the video application. For another example, taking the first application as a game application, the first image is displayed in the game interface of the game application. For another example, taking the first application as an editing application, the first image is displayed in the editing interface of the editing application.

It should be noted that the various first applications shown above are only examples to illustrate the process of the electronic device displaying the first image. In other embodiments, the first application can also be other types of applications, and accordingly, the electronic device can also use other methods to display the first image. The embodiment of the present application does not limit the process of how to display the first image.

The above S403 to S404 describe the process of the electronic device displaying the first image on the display screen of the electronic device based on the color correction coefficient. In this way, images with higher color quality and closer to real color effects can be displayed to the user, thereby improving the display effect of the image.

In some embodiments, after the electronic device obtains the first image based on S403, it may also perform image processing on the first image to obtain a processed first image. Then, the image-processed first image is displayed on the display screen of the electronic device. In this way, after performing color correction on the image to be processed to obtain the first image, image processing can also be performed on the first image to obtain an image with higher image quality.

The technical solution provided by the embodiment shown in FIG. 4 above provides a solution for color correction based on the first color correction model. Prediction of the color correction coefficient of the image to be processed is achieved by setting a prediction sub-model for predicting the color correction coefficient in the first color correction model. Color correction of the image to be processed is achieved by setting a correction sub-model for color correction in the first color correction model. In this way, through the first color correction model, the color correction on different color channels can be implemented from the pixel dimension, thereby achieving color correction for the image to improve the effect of image processing. Furthermore, by displaying the color-corrected image on the display screen of the electronic device, an image with higher color quality and closer to the real color effect can be displayed to the user, thereby improving the display effect of the image.

For example, when the image to be processed is the second image, the process shown in Figure 4 can be replaced by: inputting the second image into the first color correction model, and using the predictor of the first color correction model to model to predict the color correction coefficient of the second image. Through the correction sub-model of the first color correction model, the second image is color corrected based on the color correction coefficient to obtain the first image. The first image is displayed on the display screen of the electronic device.

As another example, in the case where the image to be processed is an image obtained by image processing the second image, the process shown in Figure 4 above can be replaced by: inputting the image processed image into the first color correction model, The color correction coefficient of the image after image processing is predicted through the prediction sub-model of the first color correction model. Through the correction sub-model of the first color correction model, color correction is performed on the image processed image based on the color correction coefficient to obtain the first image. The first image is displayed on the display screen of the electronic device.

It should be noted that, when the image to be processed is an image obtained by performing image processing on the second image, other implementations are possible in addition to the process shown in the above example. For example, FIG. 5 is a schematic flowchart of a color correction method provided by an embodiment of the present application. Referring to Figure 5, the solution is explained by taking color correction based on the second color correction model as an example. The method includes the following S501-S504:

S501. The electronic device inputs the second image into the second color correction model, and performs image processing on the second image through the processing sub-model of the second color correction model to obtain the image to be processed.

In the embodiment of the present application, the processing sub-model is used to process the image to be processed to obtain a processed image. Referring to Figure 5, the image after image processing is also the output image of the processing sub-model.

In some embodiments, the image processed image may be in the form of n*3 feature vectors. It should be understood that the image processed may include the color values of each pixel on different color channels.

In some embodiments, the processing sub-model is used to provide an image enhancement function, an image super-resolution function, an image restoration function, an image repair function, an image deblurring function, an image denoising function, an image rain removal function, and an image dehazing function. , quality improvement function or high dynamic range function.

For example, taking the processing sub-model as a model used to provide an image enhancement function, the above-mentioned image processed image is also an image obtained after image enhancement processing. Taking the processing sub-model as a model used to provide image super-resolution function as an example, the above-mentioned image processed image is also an image obtained after image super-resolution processing. Other image processing functions are similar and will not be described again.

In the above embodiments, multiple types of image processing functions are shown, and the processing sub-model may be a model provided with at least one function of the multiple types of image processing functions. In this way, the image enhancement scene, image super-resolution scene, image restoration scene, image repair scene, image deblurring scene, image denoising scene, image rain removal scene, image dehazing scene, quality improvement scene or high dynamic range scene can be used. In at least one scene, color correction is performed on the image after image processing, which broadens the applicable scope of color correction.

In some embodiments, the processing sub-model may be any one of a Unet network model, a transformer network model, and a generative adversarial network (vector quantized generative adversarial network, VQ-GAN) network model.

Among them, the Unet network model is a commonly used segmentation network in the field of image segmentation. In some embodiments, the Unet network model may be an encoder-decoder architecture based on a convolutional neural network that segments the input image into multiple pixel-level categories. The transformer network model is a model that uses the attention mechanism to improve the speed of model training. The VQ-GAN network model is a GAN-based image generation model that can convert low-quality images into high-quality images.

For example, taking the Unet network model as an example, the processing sub-model may include a feature extraction layer (or called a downsampling module) and a deconvolution layer (or called an upsampling module). Correspondingly, the process of performing the above processing on the image to be processed through the processing sub-model may be: performing feature extraction on the image to be processed through the feature extraction layer of the processing sub-model to obtain the image features of the image to be processed. Furthermore, the image features are deconvolved through the deconvolution layer of the processing sub-model to obtain the image after image processing.

In the process shown in S501 above, the process of the electronic device acquiring the image to be processed is explained. Among them, by setting up a feature extraction layer and a deconvolution layer in the processing sub-model, the feature extraction layer is used to extract the image features of the image to be processed, and the deconvolution layer is used to deconvolve the extracted image features to obtain the image Processed image. In this way, a processing sub-model based on the feature extraction layer and the deconvolution layer is provided, which can predict the image after image processing.

S502. The electronic device uses the prediction sub-model of the second color correction model to predict the color correction coefficient of the second image as the color correction coefficient of the image to be processed.

In some embodiments, the above-mentioned process of predicting the color correction coefficient of the second image through the prediction sub-model may be: performing feature extraction on the second image through the feature extraction layer of the prediction sub-model to obtain the second image Color features on different color channels. Through the convolution layer of the prediction sub-model, the color features of the second image on different color channels are convolved to obtain the color correction coefficients of the second image on different color channels.

In the process shown in S502 above, the process of the electronic device obtaining the color correction coefficient of the image to be processed is explained. Among them, by setting up a feature extraction layer and a convolution layer in the prediction sub-model, the feature extraction layer is used to extract the color features of the second image on different color channels, and the convolution layer is used to perform convolution processing on the extracted color features. Get the color correction coefficient. In this way, a prediction sub-model based on the feature extraction layer and the convolution layer is provided, which can predict the color correction coefficients of the second image on different color channels.

It should be noted that the prediction sub-model of the second color correction model has the same structure as the prediction sub-model of the first color correction model shown in S402 in FIG. 4 . The process of predicting the color correction coefficient of the second image through the prediction sub-model of the second color correction model can be referred to S402 in Figure 4 and will not be described again.

S503. The electronic device uses the correction sub-model of the second color correction model to perform color correction on the image to be processed based on the color correction coefficient to obtain the first image.

It should be noted that the correction sub-model of the second color correction model has the same structure as the correction sub-model of the first color correction model shown in S403 in FIG. 4 . The process of performing color correction through the correction sub-model of the second color correction model can be referred to S403 in Figure 4 and will not be described again.

Based on the above-mentioned S502 to S503, a color correction method is provided, which can not only perform image processing (such as image enhancement, etc.) on the image to be processed, but also achieve color correction for the image after image processing.

In some embodiments, the electronic device may be provided with a first control control. Accordingly, in some embodiments, in response to the user's opening operation of the first control control, after acquiring the image to be processed, the electronic device performs subsequent S502 to S503 to display the color-corrected image on the display screen of the electronic device. First image. Or, in other embodiments, in response to the user's closing operation of the first control control, the electronic device does not need to perform subsequent S502 to S503 after acquiring the image to be processed, and displays the processed image on the display screen of the electronic device. of images to be processed.

In some other embodiments, the electronic device may be provided with a second control control, and the second control control is used to trigger turning on or off the image processing function. Among them, the image processing function includes the above-mentioned image processing and color correction functions. For example, the user can turn on or off the image processing function by performing a triggering operation on the second control control.

Correspondingly, in some embodiments, in response to the user's opening operation of the second control control, after acquiring the second image, the electronic device performs subsequent S501 to S503 to display the image-processed and image-processed image on the display screen of the electronic device. The first image after color correction. Or, in other embodiments, in response to the user's closing operation of the second control, the electronic device displays the second image on the display screen of the electronic device without performing subsequent S501 to S503 after acquiring the second image.

In some embodiments, the electronic device may be provided with the above-mentioned first control control and the above-mentioned second control control. Furthermore, users can select corresponding control controls to flexibly control the turning on or off of the color correction function or image processing function.

S504. The electronic device displays the first image on the display screen of the electronic device.

It should be noted that the content of S504 refers to S404 in Figure 4 and will not be described again.

The above S503 to S504 describe the process of the electronic device displaying the first image on the display screen of the electronic device based on the color correction coefficient. In this way, images with higher color quality and closer to real color effects can be displayed to the user, thereby improving the display effect of the image.

The technical solution provided by the embodiment shown in FIG. 5 above provides a solution for color correction based on the second color correction model. Wherein, image processing of the image to be processed is implemented by setting a processing sub-model for image processing in the second color correction model. Prediction of the color correction coefficient is achieved by setting a prediction sub-model for predicting the color correction coefficient in the second color correction model. By setting a correction sub-model for color correction in the first color correction model, color correction of the image after image processing is achieved. In this way, a model with both color correction function and image processing function is provided, which can not only perform image processing on the image to be processed, but also effectively improve the color quality of the image, thereby improving the effect of image processing. Furthermore, by displaying the color-corrected image on the display screen of the electronic device, an image with higher color quality and closer to the real color effect can be displayed to the user, thereby improving the display effect of the image.

Illustratively, FIG. 6 is a schematic diagram of before and after color correction provided by an embodiment of the present application. Referring to Figure 6, the image shown in A in Figure 6 is used to refer to the image to be processed in the embodiment of the present application, that is, the image before color correction. The image shown as B in FIG. 6 is used to refer to the first image after color correction in the embodiment of the present application. For example, refer to the "roof" A1 shown in A in Figure 6. It should be noted that in the image to be processed, the color of the "roof" A1 is not obvious, and its actual color cannot be determined by the naked eye. After color correction using the color correction method provided by the embodiment of the present application, refer to the "roof" B1 shown in B in Figure 6. In the first image, the color of the "roof" B1 is relatively obvious and can be clearly understood by the naked eye. The color of the roof is red. For another example, refer to the "tree" A2 shown in A in Figure 6. It should be noted that in the image to be processed, the color of the "tree" A2 is relatively unobvious, and its actual color cannot be clear to the naked eye, and it is even impossible to distinguish between trees and trees. building. After applying the color correction method provided by the embodiment of the present application for color correction, refer to the "tree" B2 shown in B in Figure 6. In the first image, the color of the "tree" B2 is relatively obvious, and the naked eye can clearly distinguish between the tree and the tree. buildings, and it is clear that the color of trees is green. It can be seen that by applying the color correction method provided by the embodiment of the present application on the image shown in A, the color correction on different color channels can be realized from the pixel dimension, achieving color correction for the image, thereby improving the efficiency of image processing. Effect.

Regarding the color correction model mentioned in Figure 4 or Figure 5 above, before implementing this solution, it is necessary to iteratively train the initial model based on image training data to obtain the color correction model.

Among them, during any iterative training process, the image training data is input into the model obtained after the previous iterative training, the color correction coefficient of the image training data is obtained through the model, and the image training data is processed based on the color correction coefficient. Color correction to obtain a color-corrected output image. Based on the output image and the sample image corresponding to the image training data, adjust the model parameters. In this way, a color correction model training method is provided, which uses image training data to iteratively train the initial model, and can train a model with better color correction capabilities.

Regarding the first color correction model shown in Figure 4, Figure 7 is a schematic flowchart of a training method for the first color correction model provided by an embodiment of the present application. Referring to Figure 7, taking any iterative training process in model training as an example, the method includes the following S701-S705:

S701. During any iterative training process, the electronic device inputs the image training data into the model obtained after the previous iterative training.

Among them, the image training data refers to the training data of the initial model.

S702: Predict the color correction coefficient of the image training data through the prediction sub-model of the model.

In some embodiments, feature extraction is performed on the image training data through the feature extraction layer of the prediction sub-model to obtain color features of the image training data on different color channels. Furthermore, through the convolution layer of the prediction sub-model, the color features of the image training data on different color channels are convolved to obtain the color correction coefficients of the image training data on different color channels.

In this embodiment, by setting up a feature extraction layer and a convolution layer in the prediction sub-model, the feature extraction layer is used to extract the color features of the image training data on different color channels, and the convolution layer is used to convolve the extracted color features. product processing to obtain the color correction coefficient. In this way, a prediction sub-model based on the feature extraction layer and the convolution layer is provided, which can predict the color correction coefficient of the image training data.

The feature extraction layer may include at least two layers of target model structures. The target model structure includes a convolution layer, a batch normalization layer and an activation function layer.

In this way, a feature extraction layer based on at least two layers of target model structure is provided. Among them, the target model structure includes a convolution layer, a batch normalization layer and an activation function layer. In this way, the feature extraction layer formed has good feature extraction capabilities and can extract richer and more effective color features, thereby predicting the color correction coefficients of image training data based on the extracted color features, which can improve the accuracy of prediction of color correction coefficients. sex.

It should be noted that the relevant content of the prediction sub-model in S702 above can be found in S402 in Figure 4 and will not be described again.

S703. Use the correction sub-model of the model to perform color correction on the image training data based on the color correction coefficient to obtain the output image.

In some embodiments, the color correction coefficient includes a first correction coefficient on different color channels of each pixel included in the image training data. Correspondingly, the above-mentioned color correction process may include: based on the first correction coefficient of each pixel point included in the image training data on different color channels, the corresponding color channel of each pixel point included in the image training data is determined. The color values are subjected to the first adjustment process to obtain the output image.

In the above embodiment, a method of performing color correction based on the first correction coefficient of each pixel on different color channels included in the color correction coefficient is provided. Among them, by combining the first correction coefficients on different color channels and the color values on the corresponding color channels for the first adjustment process, the color values on the color channels can be corrected from the pixel dimension, thus achieving image training. Color correction of data, resulting in output images with higher color quality.

In other embodiments, the color correction coefficient includes a first correction coefficient and a second correction coefficient on different color channels for each pixel included in the image training data. Correspondingly, the above-mentioned color correction process may include: based on the second correction coefficient of each pixel point included in the image training data on different color channels, adjusting the color correction coefficient of each pixel point included in the image training data on the corresponding color channel. The color value is subjected to a second adjustment process to obtain an image after the second adjustment process. Based on the first correction coefficients of each pixel point included in the image training data on different color channels, perform a first adjustment process on the color value of each pixel point included in the second adjusted image on the corresponding color channel, Get this output image.

In the above embodiment, a method of performing color correction is provided by combining the first correction coefficient and the second correction coefficient of each pixel on different color channels included in the color correction coefficient. Among them, the second adjustment process is first performed by combining the second correction coefficients on different color channels and the color values on the corresponding color channels, so that the preliminary correction of the color values on the color channels can be quickly and efficiently implemented from the pixel point dimension. Furthermore, by performing the first adjustment process in combination with the first correction coefficients on different color channels and the color values on the corresponding color channels after the second adjustment process, it is possible to further realize the second adjustment of the color values on the color channels from the pixel point dimension. correction, thereby achieving color correction of the image training data, thereby obtaining an output image with higher color quality.

In some embodiments, before performing the above-mentioned second adjustment process, the second correction coefficient is also normalized. Then, based on the second correction coefficient after normalization, a subsequent mediation process is performed.

It should be noted that the relevant content of the color correction in S703 above can be found in S403 in Figure 4 and will not be described again.

S704. The electronic device adjusts model parameters based on the output image and the sample image corresponding to the image training data.

Among them, the sample image is an image whose color quality meets the preset requirements. It should be understood that sample images are used to refer to images with higher color quality corresponding to image training data. For example, the sample image may be an image with high color quality collected by a high-precision camera.

In a possible implementation, the above-mentioned process of adjusting model parameters may be: based on the output image and the sample image of the image training data, determine the model loss value of this iterative training. Furthermore, the model parameters are adjusted according to the model loss value.

Among them, the model loss value is used to represent the difference between the output image of the model and the sample image of the image training data.

In a possible implementation, the model loss value is a cross entropy loss value (cross entropyloss). Correspondingly, the electronic device determines the output image and the image training data based on the output image and the sample image of the image training data. The cross entropy loss value between the sample images is used to obtain the model loss value of this iterative training process, and then based on the model loss value, the above process of adjusting the model parameters is performed.

In another possible implementation, the model loss value is a mean square error loss value (MSE). Correspondingly, the electronic device determines the output image and the sample image of the image training data based on the output image and the sample image of the image training data. The mean square error loss value between the sample images of the image training data is used to obtain the model loss value of this iterative training process, and then based on the model loss value, the above process of adjusting the model parameters is performed.

In the above embodiment, by determining the model loss value, since the model loss value is used to represent the difference between the output image of the model and the sample image of the image training data, the model parameters are adjusted according to the model loss value, It can improve the learning ability of the model, thereby training a model with better learning ability. Of course, in another possible implementation, the electronic device can also obtain other types of model loss values to perform the above process of adjusting model parameters based on the model loss values. The embodiments of the present application do not limit this.

After adjusting the model parameters, the electronic device also determines whether the model training meets the target conditions, and then executes S705 if the model training does not meet the target conditions. When the model training meets the target conditions, the model trained in this iterative process is obtained as the first color correction model.

S705. When the model after adjusting the model parameters does not meet the target conditions, the electronic device performs the next iterative training based on the model after adjusting the model parameters until the model meets the target conditions.

In some embodiments, the target condition satisfies at least one of the following conditions: the number of iterations of model training reaches the target number; or the model loss value is less than or equal to the target threshold. Among them, the target number is the preset number of training iterations, for example, the number of iterations reaches 100. The embodiment of the present application does not limit the setting of the target number of times. The target threshold is a preset fixed threshold, such as the model loss value is less than 0.0001. The embodiment of the present application does not limit the setting of the target threshold.

In the embodiment shown in Figure 7, by setting the prediction sub-model and the correction sub-model in the model, in any iterative training process, the prediction sub-model is used to predict the color correction coefficient of the image training data, and the correction sub-model is used to predict the image The training data is color corrected to obtain the output image. Furthermore, based on the output image and the sample image, the model parameters are adjusted to achieve training optimization of the model, thereby training a model with a color correction function.

Regarding the second color correction model shown in Figure 5, Figure 8 is a schematic flowchart of a training method for the second color correction model provided by an embodiment of the present application. Referring to Figure 8, taking any iterative training process in model training as an example, the method includes the following S801-S806:

S801. During any iterative training process, the electronic device inputs the image training data into the model obtained after the previous iterative training.

It should be noted that the embodiment of the present application can add a color coefficient prediction module based on the original image processing model (such as the Unet network model) to perform model training, so as to obtain the prediction function of both color correction coefficients through training. and models of image processing functions. Therefore, in some embodiments, the image training data may adopt the training data of the original processing model. Of course, in other embodiments, the image training data can also be reacquired.

S802: Perform image processing on the image training data through the processing sub-model of the model to obtain a processed image.

Among them, the image processing can be image enhancement processing, image super-resolution processing, image restoration processing, image repair processing, image deblurring processing, image denoising processing, image rain removal processing, image dehazing processing, quality improvement processing or high dynamic range. Process at least one item.

In the above-described embodiments, various types of image processing functions are shown. In this way, at least one of the functions of image enhancement, image super-resolution, image restoration, image repair, image deblurring, image denoising, image rain removal, image dehazing, quality improvement or high dynamic range can be combined to train to obtain at least one of them. A model of image processing functions.

It should be noted that the relevant content of the processing sub-model in S802 above can be found in S501 in Figure 5 and will not be described again.

S803. Predict the color correction coefficient of the image training data through the prediction sub-model of the model.

In some embodiments, feature extraction is performed on the image training data through the feature extraction layer of the prediction sub-model to obtain color features of the image training data on different color channels. Furthermore, through the convolution layer of the prediction sub-model, the color features of the image training data on different color channels are convolved to obtain the color correction coefficients of the image training data on different color channels.

It should be noted that the relevant content of the above S803 can be found in S502 in Figure 5 and will not be described again.

S804: Perform color correction on the processed image based on the color correction coefficient through the correction sub-model of the model to obtain the output image.

It should be noted that the relevant content of the above S804 can be found in S503 in Figure 5 and will not be described again.

S805: The electronic device adjusts model parameters based on the output image and the sample image corresponding to the image training data.

In some embodiments, the sample image may be an image with higher color quality in the training data of the original processing model. Or, in other embodiments, the sample image may be an image with higher color quality collected using a high-precision camera.

It should be noted that the relevant content of adjusting the model parameters in S805 above can be found in S704 in Figure 7 and will not be described again.

After adjusting the model parameters, the electronic device also determines whether the model training meets the target conditions, and then executes S806 if the model training does not meet the target conditions. When the model training meets the target conditions, the model trained in this iterative process is obtained as the second color correction model.

S806. When the model after adjusting the model parameters does not meet the target conditions, the electronic device performs the next iterative training based on the model after adjusting the model parameters until the model meets the target conditions.

In the embodiment shown in Figure 8, by setting the processing sub-model, the prediction sub-model and the correction sub-model in the model, in any iterative training process, the processing sub-model is used to perform image processing on the image training data, and the prediction sub-model is used to perform image processing. The model predicts the color correction coefficient of the image training data, uses the correction sub-model to perform color correction on the image training data, and obtains the output image. Furthermore, based on the output image and the sample image, the model parameters are adjusted to achieve training optimization of the model, thereby training a model that has both color correction function and image processing function.

It should be noted that the electronic device used to perform the model training process in the above-mentioned Figures 7 and 8 can be the same as the electronic device used to perform the model application process in the above-mentioned Figures 4 and 5, or can be the same as the above-mentioned Figure 4 and Figure 5. The electronic equipment used to perform the model application process is different in 5. For example, in some embodiments, the electronic device used to execute the model application process in FIGS. 4 and 5 may be a terminal, and the electronic device used to execute the model training process in FIGS. 7 and 8 may be a server.

It can be understood that, in order to implement the above functions, the electronic device (such as a terminal) in the embodiment of the present application includes a corresponding hardware structure and/or software module to perform each function. Persons skilled in the art should easily realize that, in conjunction with the units and algorithm steps of each example described in the embodiments disclosed herein, the embodiments of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is performed by hardware or computer software driving the hardware depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of the embodiments of the present application.

FIG. 9 is a schematic diagram of the framework of a color correction model training device provided by an embodiment of the present application. Referring to FIG. 9 , the training device of the color correction model includes an image acquisition module 901 , a coefficient acquisition module 902 and a display module 903 . in,

Image acquisition module 901, used to acquire images to be processed;

The coefficient acquisition module 902 is used to obtain the color correction coefficient of the image to be processed, and the color correction coefficient is used to correct the color of each pixel point included in the image to be processed on different color channels;

The display module 903 is configured to display a first image on the display screen of the electronic device based on the color correction coefficient. The first image is an image obtained by performing color correction on the image to be processed based on the color correction coefficient.

The technical solution provided by the embodiment of the present application can correct the color on different color channels from the pixel dimension by obtaining the color correction coefficient of the image to be processed, thereby achieving color correction for the image to improve the effect of image processing. . Furthermore, by displaying the color-corrected image on the display screen of the electronic device, an image with higher color quality and closer to the real color effect can be displayed to the user, thereby improving the display effect of the image.

In some embodiments, the image to be processed is a second image; or, the image to be processed is an image obtained by image processing the second image;

Wherein, the second image is an image of the first application to be displayed on the display screen.

In some embodiments, when the image to be processed is an image obtained by performing image processing on the second image, the image acquisition module 901 is specifically used to:

Receive an operation from the user to trigger the first application to display an image;

In response to the operation of triggering the first application to display an image, obtain the second image to be displayed;

Perform image processing on the second image to obtain the image to be processed;

The display module 903 is specifically used for:

The first image is displayed in the interface of the first application.

In some embodiments, the coefficient acquisition module 902 is specifically used for:

Input the image to be processed into the first color correction model, and predict the color correction coefficient of the image to be processed through the prediction sub-model of the first color correction model;

The display module 903 is specifically used for:

Through the correction sub-model of the first color correction model, perform color correction on the image to be processed based on the color correction coefficient to obtain the first image;

The first image is displayed on the display screen of the electronic device.

In some embodiments, the coefficient acquisition module 902 is specifically used for:

Through the feature extraction layer of the prediction sub-model, feature extraction is performed on the image to be processed to obtain the color features of the image to be processed on different color channels;

Through the convolution layer of the prediction sub-model, the color features of the image to be processed on different color channels are convolved to obtain the color correction coefficients of the image to be processed on different color channels.

In some embodiments, when the image to be processed is an image obtained by performing image processing on the second image, the image acquisition module 901 is specifically used to:

Input the second image into the second color correction model, perform image processing on the second image through the processing sub-model of the second color correction model, and obtain the image to be processed;

The coefficient acquisition module 902 is specifically used for:

Predict the color correction coefficient of the second image through the prediction sub-model of the second color correction model as the color correction coefficient of the image to be processed;

The display module 903 is specifically used for:

Through the correction sub-model of the second color correction model, perform color correction on the image to be processed based on the color correction coefficient to obtain the first image;

The first image is displayed on the display screen of the electronic device.

In some embodiments, the coefficient acquisition module 902 is specifically used for:

Through the feature extraction layer of the prediction sub-model, perform feature extraction on the second image to obtain the color features of the second image on different color channels;

Through the convolution layer of the prediction sub-model, the color features of the second image on different color channels are convolved to obtain the color correction coefficients of the second image on different color channels.

In some embodiments, the feature extraction layer includes at least two layers of target model structure, and the target model structure includes a convolution layer, a batch normalization layer and an activation function layer.

In some embodiments, the color correction coefficient includes a first correction coefficient on different color channels for each pixel included in the image to be processed;

The display module 903 includes a correction module for:

Based on the first correction coefficients of each pixel point included in the image to be processed on different color channels, a first adjustment process is performed on the color value of each pixel point included in the image to be processed on the corresponding color channel to obtain the first correction coefficient. an image.

In some embodiments, the color correction coefficient includes a first correction coefficient and a second correction coefficient on different color channels for each pixel included in the image to be processed;

The display module 903 includes a correction module for:

Based on the second correction coefficients of each pixel point included in the image to be processed on different color channels, a second adjustment process is performed on the color value of each pixel point included in the image to be processed on the corresponding color channel to obtain a second Adjust the processed image;

Based on the first correction coefficients of each pixel point included in the image to be processed on different color channels, perform a first adjustment process on the color value of each pixel point included in the image after the second adjustment process on the corresponding color channel, Get the first image.

In some embodiments, the device further includes a processing module for:

The second correction coefficient is normalized.

In some embodiments, the device further includes a processing module for:

Perform image processing on the first image to obtain a processed first image;

The display module 903 is used for:

The image-processed first image is displayed on the display screen of the electronic device.

In some embodiments, the image processing is image enhancement processing, image super-resolution processing, image restoration processing, image repair processing, image deblurring processing, image denoising processing, image rain removal processing, image dehazing processing, and quality improvement processing. or at least one of high dynamic range processing.

Figure 10 is a schematic framework diagram of a color correction model training device provided by an embodiment of the present application. Referring to Figure 10, the training device of the color correction model includes a training module 1001.

The training module 1001 is used to iteratively train the initial model based on image training data to obtain the color correction model;

Among them, during any iterative training process, the image training data is input into the model obtained after the previous iterative training, the color correction coefficient of the image training data is obtained through the model, and the image training data is processed based on the color correction coefficient. Color correction is used to obtain a color-corrected output image. The color correction coefficient is used to correct the color of each pixel included in the image training data on different color channels; based on the samples corresponding to the output image and the image training data Image, adjust the model parameters. The sample image is an image whose color quality meets the preset requirements.

The technical solution provided by the embodiment of the present application provides a training method for a color correction model, which uses image training data to iteratively train the initial model. During the model training process, based on the model's output image and the sample image of the image training data, the model parameters are adjusted in order to optimize the training of the model and train a model with better color correction capabilities.

In some embodiments, the training module 1001 is specifically used to:

Predict the color correction coefficient of the image training data through the prediction sub-model of the model;

Through the correction sub-model of the model, the image training data is color corrected based on the color correction coefficient to obtain the output image.

In some embodiments, the training module 1001 is specifically used to:

Perform image processing on the image training data through the processing sub-model of the model to obtain the image after image processing;

Predict the color correction coefficient of the image training data through the prediction sub-model of the model;

Through the correction sub-model of the model, the color correction is performed on the processed image based on the color correction coefficient to obtain the output image.

In some embodiments, the image processing is image enhancement processing, image super-resolution processing, image restoration processing, image repair processing, image deblurring processing, image denoising processing, image rain removal processing, image dehazing processing, and quality improvement processing. or at least one of high dynamic range processing.

In some embodiments, the training module 1001 is specifically used to:

Through the feature extraction layer of the prediction sub-model, feature extraction is performed on the image training data to obtain the color features of the image training data on different color channels;

Through the convolution layer of the prediction sub-model, the color features of the image training data on different color channels are convolved to obtain the color correction coefficients of the image training data on different color channels.

In some embodiments, the feature extraction layer includes at least two layers of target model structure, and the target model structure includes a convolution layer, a batch normalization layer and an activation function layer.

In some embodiments, the color correction coefficient includes a first correction coefficient on different color channels for each pixel included in the image training data;

This training module 1001 is specifically used for:

Based on the first correction coefficient of each pixel point included in the image training data on different color channels, perform a first adjustment process on the color value of each pixel point included in the image training data on the corresponding color channel to obtain the output image.

In some embodiments, the color correction coefficient includes a first correction coefficient and a second correction coefficient on different color channels for each pixel included in the image training data;

This training module 1001 is specifically used for:

Based on the second correction coefficients of each pixel point included in the image training data on different color channels, a second adjustment process is performed on the color value of each pixel point included in the image training data on the corresponding color channel to obtain a second Adjust the processed image;

Based on the first correction coefficients of each pixel point included in the image training data on different color channels, perform a first adjustment process on the color value of each pixel point included in the second adjusted image on the corresponding color channel, Get this output image.

In some embodiments, the device further includes a processing module for:

The second correction coefficient is normalized.

An embodiment of the present application also provides an electronic device, including: a display screen, a processor, and a memory. The display screen provides a display function. The processor is connected to the memory, the memory is used to store program codes, and the processor executes the program codes stored in the memory, thereby realizing the color correction method provided by the embodiment of the present application.

Embodiments of the present application also provide a computer-readable storage medium. Program code is stored on the computer-readable storage medium. When the program code is run on an electronic device, it causes the electronic device to execute the electronic device in the above method embodiment. Each function or step performed.

Embodiments of the present application also provide a computer program product, including program code. When the program code is run on an electronic device, the electronic device causes the electronic device to perform various functions or steps performed by the electronic device in the above method embodiments.

Among them, the electronic devices, computer-readable storage media or computer program products provided by the embodiments of the present application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the corresponding methods provided above. The beneficial effects of this method will not be repeated here.

Through the above description of the embodiments, those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be allocated as needed. It is completed by different functional modules, that is, the internal structure of the device (such as electronic equipment) is divided into different functional modules to complete all or part of the functions described above. For the specific working processes of the systems, devices (such as electronic equipment) and units described above, reference can be made to the corresponding processes in the foregoing method embodiments, which will not be described again here.

In the several embodiments provided in this application, it should be understood that the disclosed systems, devices (such as electronic devices) and methods can be implemented in other ways. For example, the above-described embodiments of devices (such as electronic equipment) are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple divisions. Units or components may be combined or integrated into another system, or some features may be omitted, or not implemented. On the other hand, the coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, and the indirect coupling or communication connection of the devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or software functional units.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method in various embodiments of the present application. The aforementioned storage media include: flash memory, mobile hard disk, read-only memory, random access memory, magnetic disk or optical disk and other media that can store program codes.

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application. should be covered by the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Claims (21)

1. A color correction method, characterized in that it is applied to electronic equipment, and the method includes: Get the image to be processed; Obtain the color correction coefficient of the image to be processed, the color correction coefficient is used to correct the color of each pixel point included in the image to be processed on different color channels; Based on the color correction coefficient, a first image is displayed on the display screen of the electronic device, where the first image is an image obtained by performing color correction on the image to be processed based on the color correction coefficient. 2. The method according to claim 1, wherein the image to be processed is a second image; or, the image to be processed is an image obtained by image processing the second image; Wherein, the second image is an image of the first application to be displayed on the display screen. 3. The method according to claim 2, characterized in that, when the image to be processed is an image obtained by image processing the second image, the obtaining the image to be processed includes: Receive an operation from the user to trigger the first application to display an image; In response to the operation of triggering the first application to display an image, obtaining the second image to be displayed; Perform image processing on the second image to obtain the image to be processed; Displaying the first image on the display screen of the electronic device includes: The first image is displayed in the interface of the first application. 4. The method according to claim 2, characterized in that said obtaining the color correction coefficient of the image to be processed includes: Input the image to be processed into a first color correction model, and predict the color correction coefficient of the image to be processed through the prediction sub-model of the first color correction model; Displaying the first image on the display screen of the electronic device based on the color correction coefficient includes: Through the correction sub-model of the first color correction model, perform color correction on the image to be processed based on the color correction coefficient to obtain the first image; The first image is displayed on the display screen of the electronic device. 5. The method of claim 4, wherein predicting the color correction coefficient of the image to be processed through the prediction sub-model of the first color correction model includes: Through the feature extraction layer of the prediction sub-model, perform feature extraction on the image to be processed to obtain the color features of the image to be processed on different color channels; Through the convolution layer of the prediction sub-model, the color features of the image to be processed on different color channels are convolved to obtain the color correction coefficients of the image to be processed on different color channels. 6. The method according to claim 2, characterized in that, when the image to be processed is an image obtained by image processing the second image, the obtaining the image to be processed includes: Input the second image into a second color correction model, perform image processing on the second image through the processing sub-model of the second color correction model, and obtain the image to be processed; The obtaining the color correction coefficient of the image to be processed includes: Predict the color correction coefficient of the second image through the prediction sub-model of the second color correction model as the color correction coefficient of the image to be processed; Displaying the first image on the display screen of the electronic device based on the color correction coefficient includes: Through the correction sub-model of the second color correction model, perform color correction on the image to be processed based on the color correction coefficient to obtain the first image; The first image is displayed on the display screen of the electronic device. 7. The method of claim 6, wherein predicting the color correction coefficient of the second image through the prediction sub-model of the second color correction model includes: Through the feature extraction layer of the prediction sub-model, perform feature extraction on the second image to obtain the color features of the second image on different color channels; Through the convolution layer of the prediction sub-model, the color features of the second image on different color channels are convolved to obtain the color correction coefficients of the second image on different color channels. 8. The method according to claim 5 or 7, characterized in that the feature extraction layer includes at least two layers of target model structure, and the target model structure includes a convolution layer, a batch normalization layer and an activation function layer. 9. The method according to claim 4 or 6, characterized in that the color correction coefficient includes the first correction coefficient on different color channels of each pixel included in the image to be processed; Performing color correction on the image to be processed based on the color correction coefficient to obtain the first image includes: Based on the first correction coefficient of each pixel point included in the image to be processed on different color channels, a first adjustment process is performed on the color value of each pixel point included in the image to be processed on the corresponding color channel, to obtain the first image. 10. The method according to claim 4 or 6, characterized in that the color correction coefficient includes a first correction coefficient and a second correction coefficient on different color channels for each pixel included in the image to be processed; Performing color correction on the image to be processed based on the color correction coefficient to obtain the first image includes: Based on the second correction coefficient of each pixel point included in the image to be processed on different color channels, a second adjustment process is performed on the color value of each pixel point included in the image to be processed on the corresponding color channel to obtain The image after the second adjustment process; Based on the first correction coefficients of each pixel point included in the image to be processed on different color channels, a first correction coefficient is performed on the color value of each pixel point included in the second adjustment processed image on the corresponding color channel. Adjustment processing is performed to obtain the first image. 11. The method according to claim 10, characterized in that, based on the second correction coefficient of each pixel included in the image to be processed on different color channels, each pixel included in the image to be processed is modified. Before the second adjustment process is performed on the color value of the pixel on the corresponding color channel, the method further includes: The second correction coefficient is normalized. 12. A training method for a color correction model, characterized in that it is applied to electronic devices, and the method includes: Iteratively train an initial model based on image training data to obtain the color correction model; Wherein, in any iterative training process, the image training data is input into the model obtained after the previous iterative training, the color correction coefficient of the image training data is obtained through the model, and the color correction coefficient is used to correct the image training data based on the color correction coefficient. The image training data is subjected to color correction to obtain a color-corrected output image, and the color correction coefficient is used to correct the color of each pixel included in the image training data on different color channels; based on the output image Adjust the model parameters for the sample image corresponding to the image training data, and the sample image is an image whose color quality meets the preset requirements. 13. The method of claim 12, wherein the color correction coefficient of the image training data is obtained through the model, and the color correction coefficient is performed on the image training data based on the color correction coefficient to obtain a color Corrected output image, including: Predict the color correction coefficient of the image training data through the prediction sub-model of the model; Through the correction sub-model of the model, color correction is performed on the image training data based on the color correction coefficient to obtain the output image. 14. The method of claim 12, wherein the color correction coefficient of the image training data is obtained through the model, and the color correction coefficient is performed on the image training data based on the color correction coefficient to obtain a color Corrected output image, including: Perform image processing on the image training data through the processing sub-model of the model to obtain a processed image; Predict the color correction coefficient of the image training data through the prediction sub-model of the model; Through the correction sub-model of the model, color correction is performed on the image processed image based on the color correction coefficient to obtain the output image. 15. The method according to claim 13 or 14, characterized in that predicting the color correction coefficient of the image training data through the prediction sub-model of the model includes: Through the feature extraction layer of the prediction sub-model, feature extraction is performed on the image training data to obtain color features of the image training data on different color channels; Through the convolution layer of the prediction sub-model, the color features of the image training data on different color channels are convolved to obtain the color correction coefficients of the image training data on different color channels. 16. The method according to claim 15, wherein the feature extraction layer includes at least two layers of target model structure, and the target model structure includes a convolution layer, a batch normalization layer and an activation function layer. 17. The method of claim 12, wherein the color correction coefficient includes a first correction coefficient on different color channels for each pixel included in the image training data; Performing color correction on the image training data based on the color correction coefficient to obtain a color-corrected output image includes: Based on the first correction coefficient of each pixel point included in the image training data on different color channels, a first adjustment process is performed on the color value of each pixel point included in the image training data on the corresponding color channel, to obtain The output image. 18. The method of claim 12, wherein the color correction coefficient includes a first correction coefficient and a second correction coefficient on different color channels for each pixel included in the image training data; Performing color correction on the image training data based on the color correction coefficient to obtain a color-corrected output image includes: Based on the second correction coefficient of each pixel point included in the image training data on different color channels, a second adjustment process is performed on the color value of each pixel point included in the image training data on the corresponding color channel, to obtain The image after the second adjustment process; Based on the first correction coefficients of each pixel point included in the image training data on different color channels, a first correction coefficient is performed on the color value of each pixel point included in the second adjusted image on the corresponding color channel. Adjustment processing to obtain the output image. 19. The method according to claim 18, characterized in that, based on the second correction coefficient of each pixel included in the image training data on different color channels, each pixel included in the image training data is Before the second adjustment process is performed on the color value of the pixel on the corresponding color channel, the method further includes: The second correction coefficient is normalized. 20. An electronic device, characterized in that it includes a display screen, a memory and a processor; the display screen is provided with a display function; the memory is used to store program code; the processor is used to call the program code to Carry out the method as described in any one of claims 1-11 or 12-19. 21. A computer-readable storage medium, characterized in that it includes program code. When the program code is run on an electronic device, the electronic device causes the electronic device to execute the method described in any one of claims 1-11 or 12-19. Methods.