WO2020199141A1 - Overexposure recovery processing method, device and computer-readable storage medium - Google Patents

Abstract

The embodiment of the present invention provides an overexposure recovery processing method, device, and a computer-readable storage medium, the method includes: obtaining an image to be recovered; performing the overexposure recovery operation on the image to be recovered through a preset overexposure recovery model to obtain a recovery matrix; performing data processing on the recovery matrix to obtain a recovered target image. The embodiment of the present invention uses a neural network model to perform overexposure recovery, can avoid damage to the details of the image to be recovered while performing overexposure recovery on the image to be recovered, further improve the quality of the recovered target image while realizing the overexposure recovery of the image to be recovered.

Description

Overexposure recovery processing method, equipment and computer readable storage medium Technical field

The embodiments of the present invention relate to the field of data processing, and in particular to an overexposure recovery processing method, device, and computer-readable storage medium.

Background technique

In photography, exposure (Exposure) refers to the amount of light allowed into the lens to shine on the photosensitive medium (the film of a film camera or the image sensor of a digital camera) during the photography process. Exposure can be controlled via a combination of aperture, shutter, and sensitivity of the photosensitive medium. Ideally, when the exposure is controlled within a reasonable range, the photo has strong contrast and moderate brightness. Overexposure refers to the fact that the picture is too bright and the photo is white due to too large aperture and too slow shutter. When the photo is exposed for too long or the area is too large, overexposure will occur, and overexposure will lead to the beauty of the photo Poor, it will also cause loss of photo details.

In order to restore the aesthetics of the overexposed photos, the prior art generally implements the recovery of the overexposed photos through a computer vision algorithm that relies on a highlight recovery (Hightlight Recovery) technology.

However, when the above-mentioned method is used to restore the overexposed photo, it can only restore the highlight details to a certain extent, while the area near the center of the overexposure and the overexposure recovery effect under extreme conditions are poor, which will cause the detail of the photo to be lost.

Summary of the invention

Embodiments of the present invention provide an overexposure recovery processing method, equipment, and computer readable storage medium to solve the overexposure recovery effect caused by the overexposure recovery by computer vision algorithms relying on the exposure recovery (Highlight Recovery) technology in the prior art Poor technical problems with loss of photo details.

The first aspect of the embodiments of the present invention is to provide an overexposure recovery processing method, including:

Obtain the image to be restored;

Performing an overexposure recovery operation on the image to be recovered through a preset overexposure recovery model to obtain a recovery matrix;

Data processing is performed on the restoration matrix to obtain a restored target image.

The second aspect of the embodiments of the present invention is to provide an overexposure recovery processing device, including: a memory and a processor;

The memory is used to store program codes;

The processor calls the program code, and when the program code is executed, is used to perform the following operations:

Obtain the image to be restored;

Performing an overexposure recovery operation on the image to be recovered through a preset overexposure recovery model to obtain a recovery matrix;

Data processing is performed on the restoration matrix to obtain a restored target image.

A third aspect of the embodiments of the present invention is to provide a computer-readable storage medium on which a computer program is stored, and the computer program is executed by a processor to implement the method described in the first aspect.

The overexposure recovery processing method, device, and computer readable storage medium provided in this embodiment perform overexposure recovery on the acquired image to be recovered through a preset overexposure recovery model, obtain a recovery matrix, and perform data processing on the recovery matrix , Can obtain the restored target image. Through the use of neural network model for overexposure restoration, it is possible to avoid damage to the details of the image to be restored while performing overexposure restoration of the image to be restored, thereby achieving overexposure restoration of the image to be restored while improving the goal of restoration The quality of the image.

Description of the drawings

In order to explain the technical solutions in the embodiments of the present invention more clearly, the following will briefly introduce the drawings used in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor.

FIG. 1 is a schematic flowchart of an overexposure recovery processing method provided by Embodiment 1 of the present invention;

2 is a schematic flowchart of an overexposure recovery processing method provided by Embodiment 2 of the present invention;

3 is a schematic structural diagram of a model capable of up-sampling and down-sampling according to an embodiment of the present invention;

4 is a schematic flowchart of an overexposure recovery processing method provided by Embodiment 3 of the present invention;

5 is a schematic flowchart of an overexposure recovery processing method provided by Embodiment 4 of the present invention;

FIG. 6 is a schematic structural diagram of an overexposure recovery processing device provided by Embodiment 5 of the present invention.

detailed description

The technical solutions in the embodiments of the present invention will be clearly described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or a central component may also exist. When a component is considered to be "connected" to another component, it can be directly connected to another component or there may be a centered component at the same time.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field of the present invention. The terms used in the description of the present invention herein are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The term "and/or" as used herein includes any and all combinations of one or more related listed items.

In order to solve the technical problems of poor overexposure recovery effect and loss of photo details caused by overexposure recovery by computer vision algorithms relying on the exposure recovery (Hightlight Recovery) technology in the prior art, the present invention provides an overexposure recovery processing method , Equipment and computer-readable storage media. The overexposure recovery processing method, device, and computer-readable storage medium provided by the present invention can be applied to any scene for recovering overexposed images.

Hereinafter, some embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the case of no conflict, the following embodiments and features in the embodiments can be combined with each other.

FIG. 1 is a schematic flowchart of an overexposure recovery processing method provided by Embodiment 1 of the present invention. As shown in FIG. 1, the method includes:

Step 101: Obtain an image to be restored;

Step 102: Perform an overexposure recovery operation on the image to be recovered through a preset overexposure recovery model to obtain a recovery matrix;

Step 103: Perform data processing on the restoration matrix to obtain a restored target image.

The execution subject of this embodiment is an overexposure recovery processing device. First, it is necessary to obtain an image to be restored, where the image to be restored may be an overexposed image obtained by shooting with any overexposure parameter. In order to realize the overexposure restoration of the image to be restored, the image to be restored can be input into a preset overexposure restoration model for overexposure restoration. Wherein, the overexposure recovery model may be obtained by pre-training using a data set to be trained, and it may perform overexposure recovery on an overexposed image obtained by shooting with any kind of overexposure parameter. It should be noted that the use of neural network model for overexposure restoration can avoid damage to the details of the image to be restored while performing overexposure restoration of the image to be restored, thereby achieving overexposure restoration of the image to be restored while improving The quality of the target image after restoration. Further, after inputting the image to be restored into the preset overexposure restoration model, the restoration matrix output by the overexposure restoration model can be received, which is different from the Bayer matrix corresponding to the normal image. Therefore, in order to obtain the normal image, It is necessary to perform further data processing on the restoration model to obtain the restored target image.

Specifically, the restoration matrix is a twelve-channel matrix; on the basis of the foregoing embodiment, step 103 specifically includes:

Matrix transformation is performed on the twelve-channel matrix to obtain a Bayer matrix corresponding to the twelve-channel matrix, and an image corresponding to the Bayer matrix corresponding to the twelve-channel matrix is used as the restored target image.

In this embodiment, after receiving the twelve-channel matrix output by the overexposure recovery model, since the data output by the overexposure recovery model is a twelve-channel recovery matrix, which is different from the Bayer matrix corresponding to the normal image, , It is necessary to perform matrix transformation on the restoration matrix, convert it into a Bayer matrix corresponding to twelve channels, and use the Bayer matrix corresponding to twelve channels as the restored target image. It should be noted that any matrix transformation method can be used to implement the matrix transformation of the twelve-channel matrix, and the present invention is not limited herein.

The overexposure recovery processing method provided in this embodiment performs overexposure recovery on the acquired image to be recovered through a preset overexposure recovery model to obtain a recovery matrix, and by performing data processing on the recovery matrix, the recovered target image can be obtained . Through the use of neural network model for overexposure restoration, it is possible to avoid damage to the details of the image to be restored while performing overexposure restoration of the image to be restored, thereby achieving overexposure restoration of the image to be restored while improving the goal of restoration The quality of the image.

2 is a schematic flow chart of an overexposure recovery processing method provided in Embodiment 2 of the present invention; FIG. 3 is a schematic structural diagram of a model capable of up-sampling and down-sampling provided in an embodiment of the present invention, based on any of the above embodiments Before performing an overexposure recovery operation on the image to be recovered through a preset overexposure recovery model, the method further includes:

Step 201: Obtain a data set to be trained, where the data set to be trained includes at least one image group to be trained;

Step 202: Train a preset model to be trained according to the data set to be trained to obtain the overexposure recovery model.

In this embodiment, before performing overexposure recovery on the image to be recovered through the overexposure recovery model, the overexposure recovery model needs to be trained. Specifically, the data set to be trained can be obtained first, where the data set to be trained includes at least one set of images to be trained, and the image set to be trained includes a factual image shot with normal exposure parameters and shots with different overexposure parameters. A four-channel matrix corresponding to at least one overexposed image. By training the preset model to be trained through the data set to be trained, an overexposure recovery model can be obtained. Since the overexposure recovery model is obtained by training the fact image and the four-channel matrix corresponding to at least one overexposure image taken with different overexposure parameters, the overexposure recovery model can capture the waiting for any overexposure parameter. Restore the image for overexposure restoration.

It should be noted that, on the basis of any of the foregoing embodiments, the model to be trained is a convolutional neural network model.

Further, on the basis of any of the foregoing embodiments, the model to be trained is a model that can perform up-sampling and down-sampling.

In this embodiment, the model to be trained may specifically be a multi-layer neural network model. The input of the multi-layer neural network model is obtained by down-sampling and convolution of the input of the previous layer, and the output of each layer is spliced acquired. The objects of splicing are: 1. The output of the next layer is the result of upsampling convolution; 2. The input of this layer is the result of convolution kernel convolution (Conv) and nonlinear activation (ReLU). As shown in FIG. 3, the multi-layer model to be trained may have a symmetric structure including up-sampling and down-sampling. Down-sampling is realized by, for example, a maximum pooling layer (Max pool), and down-sampling is used to extract input feature images (eg, The process of up-sampling is to restore the feature information extracted by down-sampling to the image level, and then align and stitch with the up-sampled input information to restore the detailed information, and gradually restore the image accuracy. From the perspective of feature map information, the model can abstract the overexposed image through downsampling, extract the input deep information, and retain the shallow information of the image through a convolution and nonlinear activation and cropping, and then combine the deep information with Shallow information splicing, for the shallow information and deep information output by each layer in the downsampling structure, directly input them into the symmetrical upsampling layer of the downsampling layer, so that multiple input images from shallow to deep can be obtained Hierarchical information, so that the texture details in the overexposed area are extracted, and the image texture of the overexposed area can be restored. In the process of cropping and stitching images, the result of convolution on the lower layer image and the input of this layer are aligned at the pixel scale after the convolution kernel convolution (Conv) and nonlinear activation (ReLU).

It should be noted that, unlike other convolutional networks, when using multilayer neural network models, in order to enable the network structure to operate more efficiently, the above models that can perform upsampling and downsampling do not have all The connection layer, therefore, can greatly reduce the parameters that need to be trained. Since the redundant information may include the detailed information of the image to be restored, it can further ensure that the details of the image to be restored are not lost, and improve the accuracy of overexposure restoration To improve the quality of the restored target image.

The overexposure recovery processing method provided in this embodiment adopts the overexposure recovery model to perform overexposure recovery by preliminarily using the to-be-trained data set that includes at least one set of image groups to be trained to train the model to be trained, so as to provide for subsequent overexposure. Exposure recovery provides the basis. In addition, using a model capable of up-sampling and down-sampling as the model to be trained can ensure that the details of the image to be restored are not lost on the basis of realizing the restoration of the model to be restored.

Further, on the basis of any of the foregoing embodiments, the acquiring a data set to be trained includes:

Acquiring first image information collected by an image collecting device, where the first image information is in a Raw format;

The Bayer matrix corresponding to the first image information is converted into a four-channel matrix to obtain the data set to be trained.

In this embodiment, in order to realize the training of the model to be trained, it is first necessary to obtain the data set to be trained. Specifically, it is first necessary to obtain the first image information collected by the image collection device. The image collection device may be any device capable of realizing image collection, and the present invention is not limited herein. It should be noted that the first image information is in a Raw format. The RAW format is an unprocessed and uncompressed format. Therefore, it can retain all the texture information of the image, so that during the training process, the model to be trained can obtain all the texture information of the first image information in the RAW format and restore it . Taking practical applications for example, images in PNG and JPG formats are 8bit data, and the color of the image is often inaccurate and only contains part of the texture information of the image, while the image in Raw format is 16bit data, which contains all of the image. Texture information. Correspondingly, training the model to be trained through the first image information in the Raw format can effectively improve the recovery accuracy of the model to be trained. Further, after the first image information is obtained, the Bayer matrix corresponding to the first image information needs to be converted into a four-channel matrix, and the four-channel matrix is used as the data set to be trained. It should be noted that converting the first image information into a four-channel matrix can reduce the size of the input matrix and improve the training efficiency of the model to be trained.

The overexposure recovery processing method provided in this embodiment acquires first image information collected by an image acquisition device, where the first image information is in Raw format; and converts the Bayer matrix corresponding to the first image information into a four-channel matrix, The data set to be trained is obtained, thereby providing a basis for subsequent overexposure recovery. In addition, using the first image information in Raw format to train the model to be trained can effectively improve the recovery accuracy of the model to be trained; converting the first image information into a four-channel matrix can reduce the size of the input matrix and improve the model to be trained Training efficiency.

Further, on the basis of any of the foregoing embodiments, the training a preset model to be trained according to the data set to be trained to obtain the overexposure recovery model includes:

For each overexposed image in the to-be-trained data set, calculating the overexposure multiple corresponding to the overexposed image according to the overexposure parameter of the overexposed image;

Training a preset model to be trained according to the overexposure image and the overexposure multiple to obtain the overexposure recovery model.

In this embodiment, before training the model to be trained, first, for each overexposure image in the data set to be trained, since it is obtained by shooting with different overexposure parameters, it needs to be calculated according to the overexposure parameters. The overexposure multiple corresponding to the overexposed image. Therefore, the preset model to be trained can be trained subsequently according to the overexposure multiple and the overexposed image to obtain an overexposure recovery model. By training the preset model to be trained according to the overexposure multiple and the overexposed image, it is possible to improve the accuracy of model overexposure recovery on the basis of improving the efficiency of model training.

The overexposure recovery processing method provided in this embodiment calculates the overexposure multiple corresponding to the overexposed image according to the overexposure parameter of the overexposed image for each overexposed image in the data set to be trained; The overexposure image and the overexposure multiple train a preset model to be trained to obtain the overexposure recovery model, so that the accuracy of the overexposure recovery of the model can be improved on the basis of improving model training efficiency.

Further, on the basis of any of the foregoing embodiments, the overexposure parameter includes an aperture value and an exposure time;

Correspondingly, the calculating the overexposure multiple corresponding to the overexposed image according to the overexposure parameter of the overexposed image includes:

The overexposure multiple corresponding to the overexposed image is calculated according to the aperture value and the exposure time.

In this embodiment, the overexposure parameter may specifically include the aperture value and the exposure time. Accordingly, the overexposure multiple corresponding to the overexposed image may also be calculated according to the aperture value and the exposure time. Specifically, the overexposure multiple corresponding to the overexposed image can be calculated according to the aperture value, the exposure time and the preset formula 1, and the overexposure multiple can describe the degree of overexposure of the overexposed image.

Among them, EV is the overexposure multiple corresponding to the overexposed image, N is the aperture value corresponding to the overexposed image, and t is the exposure time corresponding to the overexposed image.

The overexposure recovery processing method provided by this embodiment calculates the overexposure multiple corresponding to the overexposed image according to the aperture value and the exposure time, so that the overexposure multiple corresponding to the overexposed image can be accurately determined.

Further, on the basis of any of the foregoing embodiments, the training a preset model to be trained according to the overexposure image and the overexposure multiple to obtain the overexposure recovery model includes:

Adjusting the parameters of the model to be trained according to the overexposure multiple to obtain an adjusted model to be trained;

Training the adjusted model to be trained according to the overexposed image to obtain the overexposed recovery model.

In this embodiment, after the overexposure factor corresponding to the overexposed image is calculated, the overexposure factor can be used as a priori knowledge, and the parameters of the model to be trained can be adjusted according to the overexposure factor to obtain the adjusted to-be-trained model. Specifically, the training process is to configure the parameters of the model to be trained through gradient descent until the parameters of the model to be trained converge. If the overexposed image is directly input into the model to be trained for recovery during the training process, the amount of calculation in the training process is relatively large and the training efficiency is low. Correspondingly, if the parameters of the model to be trained are adjusted by the overexposure multiple obtained through calculation in advance, the overexposed image is input into the adjusted model to be trained. Since the parameters of the model to be trained have been adjusted once, the subsequent training process The amount of calculation is much smaller than that of directly inputting the overexposed image into the model to be trained, which can effectively improve the training efficiency of the model to be trained, which can save resources. Correspondingly, the adjusted model to be trained can be trained according to the overexposed image to obtain the overexposed recovery model.

The overexposure recovery processing method provided in this embodiment adjusts the parameters of the model to be trained according to the overexposure parameters, and trains the adjusted model to be trained, thereby improving the training efficiency of the model to be trained.

Figure 4 is a schematic flow chart of the overexposure recovery processing method provided in the third embodiment of the present invention. On the basis of any of the above embodiments, the preset model to be trained is trained according to the to-be-trained data set to obtain all Describe the overexposure recovery model, including:

Step 301: Input the four-channel matrix corresponding to the overexposed image into the model to be trained, and obtain the output result of the model to be trained;

Step 302: Calculate the difference between the output result and the fact image corresponding to the overexposed image;

Step 303: Train the model to be trained according to the difference to obtain the overexposure recovery model.

In this embodiment, during the training process, the four-channel matrix corresponding to the overexposed image in the data set to be trained can be input into the model to be trained, and the model to be trained can process the overexposed image and output an output result. Correspondingly, the output result can be received, and the output result can be compared with the fact image corresponding to the overexposed image in the data set to be trained, and the difference between the two can be calculated. If the difference between the two is large, it means that the overexposure recovery performance of the model to be trained is poor, and training needs to be continued. Accordingly, if the difference between the two is small, it means that the output result is sufficiently similar to the fact image. That is, the overexposure recovery performance of the model to be trained is better. Therefore, in order to improve the overexposure recovery performance of the model to be trained, after calculating the difference between the output result and the fact image corresponding to the overexposed image, you can continue to train the model to be trained according to the difference until it is to be trained The model converges, and the overexposure recovery model is obtained.

The overexposure recovery processing method provided in this embodiment calculates the difference between the output result and the fact image corresponding to the overexposed image, and trains the model to be trained based on the difference, thereby improving the performance of the model to be trained. Overexposure recovery performance.

Further, on the basis of any of the foregoing embodiments, the training the model to be trained according to the difference to obtain the overexposure recovery model includes:

Judging whether the difference value is greater than a preset difference value threshold;

If yes, adjust the parameters of the model to be trained according to the difference, until the adjusted difference between the output result of the model to be trained and the fact image is less than a preset difference threshold;

If not, it is determined that the model to be trained converges, and the overexposure recovery model is obtained.

In this embodiment, in order to improve the overexposure recovery performance of the model to be trained, after calculating the difference between the output result and the fact image corresponding to the overexposed image, the training model can be continued to be trained based on the difference . Specifically, it can be judged whether the difference is greater than a preset threshold. If it is greater, it means that the overexposure recovery performance of the model to be trained is poor, and training needs to be continued. At this time, the difference can be based on the difference The parameters are adjusted until the difference between the output result of the model to be trained and the fact image is less than the preset threshold, that is, it is determined that the model to be trained converges, and the overexposure recovery model is obtained. Optionally, if the difference is less than a preset threshold, it indicates that the output result is sufficiently similar to the fact image, that is, the overexposure recovery performance of the model to be trained is good, and at this time, it is determined that the model to be trained has converged. Overexposure recovery model.

The overexposure recovery processing method provided in this embodiment trains the model to be trained according to the difference between the output result and the fact image corresponding to the overexposed image, thereby improving the overexposure recovery performance of the model to be trained.

Figure 5 is a schematic flow chart of the overexposure recovery processing method provided by the fourth embodiment of the present invention. On the basis of any of the above embodiments, the preset model to be trained is trained according to the data set to be trained to obtain all Describe the overexposure recovery model, including:

Step 401: Randomly divide the data set to be trained into a training set and a verification set according to a preset ratio;

Step 402: Train the model to be trained through the training set;

Step 403: Verify the recovery accuracy of the to-be-trained model through the verification set, and obtain a verification result.

Step 404: Continue to train the model to be trained through the training set according to the verification result until the model to be trained converges to obtain the overexposure recovery model.

In this embodiment, in order to improve the overexposure recovery accuracy of the model to be trained, the data in the data set to be trained can be randomly divided into a training set and a verification set according to a preset ratio. The preset ratio may be 8:2, or other ratios, and those skilled in the art can adjust it according to actual application requirements, and the present invention is not limited here. The training model is trained through the data in the training set, the recovery accuracy of the training model is verified through the data in the verification set, and the verification result is obtained, and the training model is continued to be trained based on the verification result until the model converges. Specifically, if the verification result indicates that the recovery accuracy of the model to be trained is high, it indicates that the model to be trained has converged and an overexposure recovery model is obtained; if the verification result indicates that the recovery accuracy of the model to be trained is low, then the model to be trained is characterized Training still needs to be continued. At this time, the data to be trained in the training set can be used to train the model to be trained until the model to be trained converges and an overexposure recovery model is obtained.

The overexposure recovery processing method provided in this embodiment randomly divides the data set to be trained into a training set and a verification set, and trains the training model through the data in the training set, and the recovery accuracy of the training model through the data in the verification set Perform verification and further process the model to be trained based on the verification result, thereby improving the accuracy of the model's recovery.

Further, on the basis of any of the foregoing embodiments, the training a preset model to be trained according to the data set to be trained, and after obtaining the overexposure recovery model, the method further includes:

Acquiring second image information collected by an image collecting device, where the image information is in a Raw format, and the first image information and the second image information are at least partially non-overlapping;

Generating a test set according to the second image information;

The applicability of the model to be trained is tested through the test set.

In this embodiment, since the overexposure recovery model needs to perform overexposure recovery on various types of overexposed images, the need to change the overexposure recovery model has better applicability. Therefore, the second image information collected by the image acquisition device can be obtained. In order to verify the applicability of the model to be trained, the second image information and the first image information do not overlap at least partially. A test set is generated according to the second image information, and the applicability of the model to be trained is tested according to the test set. If it is verified that the applicability of the model to be trained is poor, it is necessary to use a wider set of data to be trained to continue training. If the applicability of the model to be trained is verified to be better, an overexposure recovery model can be obtained, and then The overexposure recovery can be performed according to the overexposure recovery model.

The overexposure recovery processing method provided in this embodiment generates a test set according to the second image information, and tests the applicability of the model to be trained according to the test set, so that the overexposure recovery accuracy of the model to be trained can be ensured. , Improve the applicability of the model to be trained.

6 is a schematic structural diagram of an overexposure recovery processing device according to Embodiment 5 of the present invention. As shown in FIG. 5, the overexposure recovery processing device includes a memory 51 and a processor 52;

The memory 51 is used to store program codes;

The processor 52 calls the program code, and when the program code is executed, is used to perform the following operations:

Obtain the image to be restored;

Performing an overexposure recovery operation on the image to be recovered through a preset overexposure recovery model to obtain a recovery matrix;

Data processing is performed on the restoration matrix to obtain a restored target image.

Further, on the basis of any of the foregoing embodiments, before performing an overexposure recovery operation on the image to be recovered through a preset overexposure recovery model, the processor is further configured to:

Acquiring a data set to be trained, where the data set to be trained includes at least one group of images to be trained;

Training a preset model to be trained according to the data set to be trained to obtain the overexposure recovery model.

Further, on the basis of any of the foregoing embodiments, the processor is configured to: when acquiring the data set to be trained:

Acquiring image information collected by an image collecting device, where the first image information is in a Raw format;

The Bayer matrix corresponding to the image information is converted into a four-channel matrix to obtain the data set to be trained.

Further, on the basis of any of the foregoing embodiments, the image group to be trained includes a fact image captured by a normal exposure parameter and a four-channel matrix corresponding to an overexposure image captured by at least one overexposure parameter.

Further, on the basis of any of the foregoing embodiments, the processor is configured to: when training a preset model to be trained according to the data set to be trained to obtain the overexposure recovery model:

For each overexposed image in the to-be-trained data set, calculating the overexposure multiple corresponding to the overexposed image according to the overexposure parameter of the overexposed image;

Training a preset model to be trained according to the overexposure image and the overexposure multiple to obtain the overexposure recovery model.

Further, on the basis of any of the foregoing embodiments, the overexposure parameter includes an aperture value and an exposure time;

Correspondingly, when the processor calculates the overexposure multiple corresponding to the overexposed image according to the overexposure parameter of the overexposed image, it is configured to:

The overexposure multiple corresponding to the overexposed image is calculated according to the aperture value and the exposure time.

Further, on the basis of any of the foregoing embodiments, the processor trains a preset model to be trained according to the overexposure image and the overexposure multiple to obtain the overexposure recovery model, using in:

Adjusting the parameters of the model to be trained according to the overexposure multiple to obtain an adjusted model to be trained;

Training the adjusted model to be trained according to the overexposed image to obtain the overexposed recovery model.

Further, on the basis of any of the foregoing embodiments, the processor is configured to: when training a preset model to be trained according to the data set to be trained to obtain the overexposure recovery model:

Input the four-channel matrix corresponding to the overexposed image to the model to be trained, and obtain the output result of the model to be trained;

Calculating the difference between the output result and the fact image corresponding to the overexposed image;

Training the model to be trained according to the difference to obtain the overexposure recovery model.

Further, on the basis of any of the foregoing embodiments, when the processor trains the model to be trained according to the difference to obtain the overexposure recovery model, it is configured to:

Judging whether the difference value is greater than a preset difference value threshold;

If yes, adjust the parameters of the model to be trained according to the difference, until the adjusted difference between the output result of the model to be trained and the fact image is less than a preset difference threshold;

If not, it is determined that the model to be trained converges, and the overexposure recovery model is obtained.

Further, on the basis of any of the foregoing embodiments, the processor is configured to: when training a preset model to be trained according to the data set to be trained to obtain the overexposure recovery model:

Randomly dividing the to-be-trained data set into a training set and a verification set according to a preset ratio;

Training the model to be trained through the training set;

Verifying the restoration accuracy of the model to be trained through the verification set to obtain a verification result;

Continue to train the model to be trained through the training set according to the verification result until the model to be trained converges to obtain the overexposure recovery model.

Further, on the basis of any of the foregoing embodiments, after the processor trains a preset model to be trained according to the data set to be trained to obtain the overexposure recovery model, it is further configured to:

Acquiring second image information collected by an image collecting device, where the image information is in a Raw format, and the first image information and the second image information are at least partially non-overlapping;

Generating a test set according to the second image information;

The applicability of the model to be trained is tested through the test set.

Further, on the basis of any of the foregoing embodiments, the restoration matrix is a twelve-channel matrix;

Correspondingly, when the processor performs data processing on the restoration matrix to obtain a restored target image, it is used to:

Matrix transformation is performed on the twelve-channel matrix to obtain a Bayer matrix corresponding to the twelve-channel matrix, and an image corresponding to the Bayer matrix corresponding to the twelve-channel matrix is used as the restored target image.

Further, on the basis of any of the foregoing embodiments, the model to be trained is a convolutional neural network model.

Further, on the basis of any of the foregoing embodiments, the model to be trained is a model that can perform up-sampling and down-sampling.

In addition, this embodiment also provides a computer-readable storage medium on which a computer program is stored, and the computer program is executed by a processor to implement the overexposure recovery processing method described in the foregoing embodiment.

In the several embodiments provided by the present invention, it should be understood that the disclosed device and method may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components can be combined or It can be integrated into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional units in the various embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware, or may be implemented in the form of hardware plus software functional units.

The above-mentioned integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium. The above-mentioned software functional unit is stored in a storage medium and includes several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor execute the method described in the various embodiments of the present invention. Part of the steps. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program code .

Those skilled in the art can clearly understand that, for the convenience and conciseness of the description, only the division of the above-mentioned functional modules is used as an example. In practical applications, the above-mentioned functions can be allocated by different functional modules as required, namely, the device The internal structure is divided into different functional modules to complete all or part of the functions described above. For the specific working process of the device described above, reference may be made to the corresponding process in the foregoing method embodiment, which is not repeated here.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand: It is still possible to modify the technical solutions described in the foregoing embodiments, or equivalently replace some or all of the technical features; these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the embodiments of the present invention range.

Claims (29)

1. An overexposure recovery treatment method, characterized in that it comprises:

   Obtain the image to be restored;

   Performing an overexposure recovery operation on the image to be recovered through a preset overexposure recovery model to obtain a recovery matrix;

   Data processing is performed on the restoration matrix to obtain a restored target image.
2. The method according to claim 1, wherein before the performing an overexposure recovery operation on the image to be recovered through a preset overexposure recovery model, the method further comprises:

   Acquiring a data set to be trained, where the data set to be trained includes at least one group of images to be trained;

   Training a preset model to be trained according to the data set to be trained to obtain the overexposure recovery model.
3. The method according to claim 2, wherein said acquiring a data set to be trained comprises:

   Acquiring first image information collected by an image collecting device, where the first image information is in a Raw format;

   The Bayer matrix corresponding to the first image information is converted into a four-channel matrix to obtain the data set to be trained.
4. The method according to claim 3, wherein the image group to be trained includes a fact image taken with a normal exposure parameter and a four-channel matrix corresponding to an overexposure image taken with at least one overexposure parameter.
5. The method according to any one of claims 2-4, wherein the training a preset model to be trained according to the data set to be trained to obtain the overexposure recovery model comprises:

   For each overexposed image in the to-be-trained data set, calculating the overexposure multiple corresponding to the overexposed image according to the overexposure parameter of the overexposed image;

   Training a preset model to be trained according to the overexposure image and the overexposure multiple to obtain the overexposure recovery model.
6. The method according to claim 5, wherein the overexposure parameter includes an aperture value and an exposure time;

   Correspondingly, the calculating the overexposure multiple corresponding to the overexposed image according to the overexposure parameter of the overexposed image includes:

   The overexposure multiple corresponding to the overexposed image is calculated according to the aperture value and the exposure time.
7. The method according to claim 5 or 6, wherein the training a preset model to be trained according to the overexposed image and the overexposure multiple to obtain the overexposed recovery model comprises:

   Adjusting the parameters of the model to be trained according to the overexposure multiple to obtain an adjusted model to be trained;

   Training the adjusted model to be trained according to the overexposed image to obtain the overexposed recovery model.
8. The method according to any one of claims 2-6, wherein the training a preset model to be trained according to the data set to be trained to obtain the overexposure recovery model comprises:

   Input the four-channel matrix corresponding to the overexposed image to the model to be trained, and obtain the output result of the model to be trained;

   Calculating the difference between the output result and the fact image corresponding to the overexposed image;

   Training the model to be trained according to the difference to obtain the overexposure recovery model.
9. The method according to claim 8, wherein the training the model to be trained according to the difference to obtain the overexposure recovery model comprises:

   Judging whether the difference value is greater than a preset difference value threshold;

   If yes, adjust the parameters of the model to be trained according to the difference, until the adjusted difference between the output result of the model to be trained and the fact image is less than a preset difference threshold;

   If not, it is determined that the model to be trained converges, and the overexposure recovery model is obtained.
10. The method according to any one of claims 2-9, wherein the training a preset model to be trained according to the data set to be trained to obtain the overexposure recovery model comprises:

    Randomly dividing the to-be-trained data set into a training set and a verification set according to a preset ratio;

    Training the model to be trained through the training set;

    Verifying the restoration accuracy of the model to be trained through the verification set to obtain a verification result;

    Continue to train the model to be trained through the training set according to the verification result until the model to be trained converges to obtain the overexposure recovery model.
11. The method according to any one of claims 2-9, wherein the training a preset model to be trained according to the data set to be trained, and after obtaining the overexposure recovery model, further comprising:

    Acquiring second image information collected by an image collecting device, where the image information is in a Raw format, and the first image information and the second image information do not overlap at least partially;

    Generating a test set according to the second image information;

    The applicability of the model to be trained is tested through the test set.
12. The method according to any one of claims 1-11, wherein the restoration matrix is a twelve-channel matrix;

    Correspondingly, the performing data processing on the restoration matrix to obtain the restored target image includes:

    Matrix transformation is performed on the twelve-channel matrix to obtain a Bayer matrix corresponding to the twelve-channel matrix, and an image corresponding to the Bayer matrix corresponding to the twelve-channel matrix is used as the restored target image.
13. The method according to any one of claims 2-12, wherein the model to be trained is a convolutional neural network model.
14. The method according to any one of claims 2-13, wherein the model to be trained is a model capable of up-sampling and down-sampling.
15. An overexposure recovery processing device, which is characterized by comprising: a memory and a processor;

    The memory is used to store program codes;

    The processor calls the program code, and when the program code is executed, is used to perform the following operations:

    Obtain the image to be restored;

    Performing an overexposure recovery operation on the image to be recovered through a preset overexposure recovery model to obtain a recovery matrix;

    Data processing is performed on the restoration matrix to obtain a restored target image.
16. 15. The device according to claim 15, wherein the processor is further configured to: before performing an overexposure recovery operation on the image to be recovered through a preset overexposure recovery model:

    Acquiring a data set to be trained, where the data set to be trained includes at least one group of images to be trained;

    Training a preset model to be trained according to the data set to be trained to obtain the overexposure recovery model.
17. The device according to claim 16, wherein the processor is configured to:

    Acquiring image information collected by the image collecting device, where the first image information is in a Raw format;

    The Bayer matrix corresponding to the image information is converted into a four-channel matrix to obtain the data set to be trained.
18. The device according to claim 17, wherein the group of images to be trained includes a fact image taken with normal exposure parameters and a four-channel matrix corresponding to an overexposure image taken with at least one overexposure parameter.
19. The device according to any one of claims 16-18, wherein the processor trains a preset model to be trained according to the data set to be trained to obtain the overexposure recovery model. in:

    For each overexposed image in the to-be-trained data set, calculating the overexposure multiple corresponding to the overexposed image according to the overexposure parameter of the overexposed image;

    Training a preset model to be trained according to the overexposure image and the overexposure multiple to obtain the overexposure recovery model.
20. The device according to claim 19, wherein the overexposure parameter comprises an aperture value and an exposure time;

    Correspondingly, when the processor calculates the overexposure multiple corresponding to the overexposed image according to the overexposure parameter of the overexposed image, it is configured to:

    The overexposure multiple corresponding to the overexposed image is calculated according to the aperture value and the exposure time.
21. The device according to claim 19 or 20, wherein the processor trains a preset model to be trained according to the overexposed image and the overexposure multiple to obtain the overexposed recovery model For:

    Adjusting the parameters of the model to be trained according to the overexposure multiple to obtain an adjusted model to be trained;

    Training the adjusted model to be trained according to the overexposed image to obtain the overexposed recovery model.
22. The device according to any one of claims 16-20, wherein the processor trains a preset model to be trained according to the data set to be trained to obtain the overexposure recovery model. in:

    Input the four-channel matrix corresponding to the overexposed image to the model to be trained, and obtain the output result of the model to be trained;

    Calculating the difference between the output result and the fact image corresponding to the overexposed image;

    Training the model to be trained according to the difference to obtain the overexposure recovery model.
23. The device according to claim 22, wherein the processor is configured to: when training the model to be trained according to the difference to obtain the overexposure recovery model:

    Judging whether the difference value is greater than a preset difference value threshold;

    If yes, adjust the parameters of the model to be trained according to the difference, until the adjusted difference between the output result of the model to be trained and the fact image is less than a preset difference threshold;

    If not, it is determined that the model to be trained converges, and the overexposure recovery model is obtained.
24. The device according to any one of claims 16-23, wherein when the processor trains a preset model to be trained according to the data set to be trained to obtain the overexposure recovery model, it uses in:

    Randomly dividing the to-be-trained data set into a training set and a verification set according to a preset ratio;

    Training the model to be trained through the training set;

    Verifying the restoration accuracy of the model to be trained through the verification set to obtain a verification result;

    Continue to train the model to be trained through the training set according to the verification result until the model to be trained converges to obtain the overexposure recovery model.
25. The device according to any one of claims 16-23, wherein the processor trains a preset to-be-trained model according to the to-be-trained data set, and obtains the overexposure recovery model, further Used for:

    Acquiring second image information collected by an image collecting device, where the image information is in a Raw format, and the first image information and the second image information do not overlap at least partially;

    Generating a test set according to the second image information;

    The applicability of the model to be trained is tested through the test set.
26. The device according to any one of claims 15-25, wherein the restoration matrix is a twelve-channel matrix;

    Correspondingly, when the processor performs data processing on the restoration matrix to obtain a restored target image, it is used to:

    Matrix transformation is performed on the twelve-channel matrix to obtain a Bayer matrix corresponding to the twelve-channel matrix, and an image corresponding to the Bayer matrix corresponding to the twelve-channel matrix is used as the restored target image.
27. The device according to any one of claims 16-26, wherein the model to be trained is a convolutional neural network model.
28. The device according to any one of claims 16-27, wherein the model to be trained is a model capable of up-sampling and down-sampling.
29. A computer-readable storage medium, characterized in that a computer program is stored thereon, and the computer program is executed by a processor to implement the method according to any one of claims 1-14.

Images