WO2021180131A1 - 一种图像处理方法及电子设备 - Google Patents

一种图像处理方法及电子设备 Download PDF

Info

Publication number
WO2021180131A1
WO2021180131A1 PCT/CN2021/080027 CN2021080027W WO2021180131A1 WO 2021180131 A1 WO2021180131 A1 WO 2021180131A1 CN 2021080027 W CN2021080027 W CN 2021080027W WO 2021180131 A1 WO2021180131 A1 WO 2021180131A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
quality evaluation
evaluation information
image quality
images
Prior art date
Application number
PCT/CN2021/080027
Other languages
English (en)
French (fr)
Inventor
吕帅林
刘星
张运超
张俪耀
左旺孟
任冬伟
刘铭
李钰
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2021180131A1 publication Critical patent/WO2021180131A1/zh

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80Camera processing pipelines; Components thereof
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60Control of cameras or camera modules
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60Control of cameras or camera modules
    • H04N23/62Control of parameters via user interfaces
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60Control of cameras or camera modules
    • H04N23/63Control of cameras or camera modules by using electronic viewfinders
    • H04N23/631Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
    • H04N23/632Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30168Image quality inspection

Definitions

  • This application belongs to the field of image processing technology, and particularly relates to image processing methods and electronic equipment.
  • the present application provides an image processing method and electronic device, which can solve the problem that the image quality of the final output image of the electronic device is worse than the image quality of the image before processing.
  • this application provides an image processing method, including:
  • each of the N second images is obtained by processing the first image through at least one of the M image enhancement models ;
  • N and M are integers greater than zero, each of the M image enhancement models is different, and the N second images are also different;
  • a target image is output, and the target image is at least one of the first image and the N second images.
  • the image quality of the second image may be worse than the image quality of the first image.
  • the second image includes a human face image, and the human face The eyes, nose, glasses frame, etc. in the image are deformed, purple fringing appears on the edge of the second image, and there are artifacts such as ghosting, color aliasing, and zipper effect in the second image. Therefore, the second image is evaluated by the quality evaluation model. The image quality of the image, or the image quality of the first image and the second image is evaluated, and an image with better image quality is finally output from the first image and the second image according to the evaluation result for the user to view.
  • the electronic device when the image quality of the processed second image deteriorates, the electronic device finally outputs the first image to the user to view, so as to solve the problem of outputting the image that has deteriorated image quality after being processed by the image enhancement model in the prior art.
  • the problem for the user can improve the quality of the output image and reduce the probability of outputting a poor quality image, so as to improve the user's visual experience.
  • the quality evaluation model is trained based on a plurality of training samples, and each training sample includes a sample image and the user’s image quality of the sample image. Evaluation information.
  • the sample image used in the process of training the quality evaluation model corresponds to the input image used when the trained quality evaluation model is applied to evaluate the image quality. Since the quality evaluation model is trained using multiple sample images, when the trained quality evaluation model is used to evaluate the image quality of the input image, the accuracy of the image quality evaluation result obtained is higher.
  • the target image includes:
  • the evaluation rule is that the image quality evaluation information is a predetermined number, or the score corresponding to the image quality evaluation information is greater than or equal to a predetermined score threshold.
  • the image quality evaluation information when the image quality evaluation information is a predetermined number, or the score corresponding to the image quality evaluation information is greater than or equal to the predetermined score threshold, it means that the image quality of the image corresponding to the image quality evaluation information is better.
  • the target image determined according to the image quality evaluation information and evaluation rules is the image with better image quality among the first image and N second images, and the electronic device can output the target image with better image quality for the user to view, reducing the output quality The probability of a bad image.
  • the image quality evaluation information is a number or a score.
  • the image quality evaluation information is a number, and the input image is N second images, or In the case of the first image and the N second images, the image quality evaluation information is image quality evaluation information for each second image in the N second images;
  • outputting a target image according to the obtained image quality evaluation information includes:
  • the image quality evaluation information is a predetermined number of target second images.
  • the electronic device can classify the image quality of the input image through the quality evaluation model, so as to determine whether the input image belongs to the category of good image quality or the category of poor image quality.
  • the output image quality evaluation information is a predetermined number; when the input image belongs to the category of poor image quality, the output image quality evaluation information is not a predetermined number. Performing two classifications on the image quality of the input image has less computational overhead, and the image quality evaluation result can be obtained faster.
  • the fifth possible implementation manner of the first aspect in determining whether there is a target second image whose image quality evaluation information is a predetermined number in the N second images After the image, it also includes:
  • the first image is an image with the best image quality, and the first image is output.
  • the outputting the target image according to the obtained image quality evaluation information further includes:
  • any target second image is selected for output.
  • the image quality evaluation information is a score
  • the input image is N second images
  • the image quality evaluation information is image quality evaluation information for each second image in the N second images
  • outputting a target image according to the obtained image quality evaluation information includes:
  • the target second image with the highest score is determined to be the image with the best image quality, and the target second image with the highest score is output. image.
  • the feature information of the first image can be used to assist in evaluating the image quality of the second image. Since the electronic device can use the first image as a reference image, the feature information of the second image can be obtained from the feature information of the first image. According to the difference feature information between the two, the image quality of the second image is evaluated according to the difference feature information, which can improve the accuracy of the image quality evaluation information of the second image. In addition, the image quality evaluation information is represented by scores, which can more accurately describe the quality of the image. The electronic device can determine the target image with the best image quality among the N second images according to the score corresponding to each second image, so as to output the image with the best image quality for the user to view.
  • Also after determining whether there is a target second image with a score greater than a predetermined score threshold in the N second images ,Also includes:
  • the first image is an image with the best image quality, and the first image is output.
  • the output of the target image according to the obtained image quality evaluation information includes:
  • a new one is acquired An image enhancement model, using the acquired new image enhancement model to process the first image to obtain a new second image, and inputting the new second image as an input image into the quality evaluation model for processing , Obtain new image quality evaluation information; wherein the new image enhancement model is an image enhancement model that has not processed the first image;
  • the new image quality evaluation information is the predetermined number, or the score is greater than the predetermined score threshold, then a new second image is output, otherwise it returns to the step of acquiring a new image enhancement model and subsequent steps until The number of return executions reaches a preset number threshold, and the first image is output.
  • the electronic device may preferentially use the image enhancement model 1 with the best image processing effect to process the first image. If the image quality of the processed image is poor, then use the image enhancement model with the second best image processing effect. 2 Process the first image. If the image quality of the processed image is still poor, select the optimal image enhancement model from the optional image enhancement models. 3 Process the first image, the optional image enhancement model Refers to image enhancement models other than the previously used image enhancement models (such as the optimal and suboptimal image enhancement models for image processing). Since the image enhancement model with the best image processing effect is preferentially used to process the first image, in some cases, the second image with better image quality can be obtained without using N image enhancement models. Compared with the case where N image enhancement models are used to process the first image in parallel, part of the system resources can be saved, and the time required to obtain the second image with better image quality can be shortened, so as to improve the efficiency of the output image.
  • the image quality evaluation information is a number
  • the input image is the first image and the Nth
  • the image quality evaluation information is image quality evaluation information for each second image in the first image and the N second images
  • outputting a target image according to the obtained image quality evaluation information includes:
  • the image quality evaluation information of the first image is a predetermined number, and there is at least one target second image whose image quality evaluation information is the predetermined number among the N second images, then the first image and the Select any one of the target second images to output.
  • the image quality of the first image and the second image can be evaluated, and the image quality evaluation information for the first image and the second image can be obtained.
  • the image quality evaluation information can be used to determine whether the image quality of the second image is The image quality of the first image is better than that of the first image, and it can be learned more accurately whether the image quality of the second image obtained after the first image is processed by the image enhancement model is deteriorated.
  • the eleventh possible implementation manner of the first aspect it is predetermined to determine whether there is image quality evaluation information in the first image and the N second images. After the second image of the digital target, it also includes:
  • the image quality evaluation information of the first image is not a predetermined number, and there is at least one target second image whose image quality evaluation information is a predetermined number among the N second images, select from the target second image Any image output.
  • the twelfth possible implementation manner of the first aspect it is predetermined that whether there is image quality evaluation information in the first image and the N second images After the second image of the digital target, it also includes:
  • the first image is output.
  • the image quality evaluation information is a score
  • the input image is the first image and N number
  • the image quality evaluation information is image quality evaluation information of each second image in the first image and the N second images
  • outputting a target image according to the obtained image quality evaluation information includes:
  • the scores of the first image and the scores of the N second images from the first image and the N second images, determine that the image with the highest score is the image with the best image quality, and output The image with the highest score.
  • the electronic device can directly compare the respective scores of the first image and the N second images, filter out the image with the highest score, and then determine the image with the best image quality, which can more accurately determine the image with the best quality image.
  • the acquiring the first image includes:
  • the performing image fusion processing on the multi-frame RAW image to obtain the first image includes:
  • the multi-frame RAW images are divided into at least two groups, and image fusion processing is performed on each group of RAW images to obtain at least two first images.
  • the first image can be obtained by image fusion processing on the acquired RAW image
  • the second image obtained after the first image is processed by the image enhancement model and the image quality of the second image can be evaluated through the quality evaluation model, or Evaluate the image quality of the first image and the second image, and finally output a better image quality image from the first image and the second image for the user to view according to the evaluation result, which can improve the image quality of the output image.
  • the image quality evaluation information is the image quality evaluation information of the second image
  • the target image is the second image
  • the image quality evaluation information is not a predetermined number, or the score corresponding to the image quality evaluation information is less than the predetermined score threshold, the target image is the First image
  • the image quality evaluation information is image quality evaluation information for the first image and the second image, and the image quality evaluation information is used to indicate whether the image quality of the second image is When the image quality of the first image is better than that of the first image, if the image quality evaluation information is a predetermined number, then the target image is the second image; if the image quality evaluation information is not a predetermined number, then the target The image is the first image;
  • the image quality evaluation information includes the image quality evaluation information corresponding to the first image and the second image
  • the image quality evaluation information of the first image is a predetermined number
  • the image quality evaluation information of the second image is a predetermined number
  • the target image is any one of the second image and the first image; if the image quality evaluation information of the first image is not predetermined If the image quality evaluation information of the second image is a predetermined number, the target image is the second image; if the image quality evaluation information of the first image is a predetermined number, and the second image If the image quality evaluation information of is not a predetermined number, the target image is the first image; or,
  • the target image is the second image; if the image quality of the second image is The score corresponding to the evaluation information is less than the score corresponding to the image quality evaluation information of the first image, then the target image is the first image; if the score corresponding to the image quality evaluation information of the second image is equal to the first image A score corresponding to the image quality evaluation information of an image, then the target image is any one of the second image and the first image;
  • the image quality evaluation information is the image quality evaluation information of the second image of each of the N second images, if there is image quality in the N second images If the evaluation information is a predetermined number of target second images, the target image is at least one of the target second images; if there is no target second image of which image quality evaluation information is a predetermined number among the N second images, Then the target image is the first image; or,
  • the target image is at least one of the target second images; if the N If there is no target second image whose score corresponding to the image quality evaluation information is greater than or equal to a predetermined score threshold in the second image, the target image is the first image;
  • the target image is the The image quality evaluation information in the first image and the N second images is any image in which the image quality evaluation information is a predetermined number, or the target image is corresponding to the image quality evaluation information in the first image and the N second images Any image with a score greater than or equal to a predetermined score threshold, or the target image is the image with the highest score corresponding to the image quality evaluation information among the first image and the N second images.
  • images with better image quality can be selected from the first image and N second images through a predetermined number or a predetermined score threshold for output.
  • this application provides an image processing method, including:
  • the first image is input to M image enhancement models for processing to obtain N second images, and all the images are displayed or saved.
  • N and M are positive integers
  • each of the M image enhancement models is different, and the N second images are also different.
  • the image quality evaluation model is first used to evaluate the image quality of the acquired first image, and whether to process the first image is determined according to the evaluation result.
  • the image enhancement model is used to process the first image, and the second image is output; when the image quality of the first image is poor, the first image is output to the user for viewing to reduce
  • the data processing speed of mobile phones can be improved due to the consumption of resources occupied by image processing.
  • the first image is displayed or saved. image.
  • the first image quality evaluation information is a number or a score.
  • the image quality of the first image meets the requirements It means that the first image quality evaluation information is a predetermined number.
  • the electronic device can classify the image quality of the first image into two categories through the quality evaluation model, so as to determine whether the first image belongs to the category of good image quality or the category of poor image quality.
  • the output image quality evaluation information is a predetermined number; when the first image belongs to the category of poor image quality, the output image quality evaluation information is not a predetermined number. Performing two classifications on the image quality of the first image has less computational overhead, and the image quality evaluation result of the first image can be obtained faster.
  • the image quality evaluation information is a score
  • the image quality of the first image meets the requirements It means that the score of the first image is greater than or equal to a preset threshold.
  • the first image is acquired by an electronic device in a high-zoom photography mode.
  • the first image obtained may cause image distortion due to shaking, distortion of the facial features such as the eyes of the portrait, or blurry image (for example, It is difficult to recognize the subject), etc. Therefore, it is necessary to evaluate the image quality of the first image. If the image quality of the first image is poor, then display the first image to the user for viewing, or save the first image to the gallery; if the image quality of the first image is better, then process the first image to obtain Nth Two images, and display or save at least one of the N second images.
  • the data processing speed of the electronic device can be increased by saving resources consumed by processing the first image with poor image quality.
  • the preset threshold value is 0.25, and both M and N are 1.
  • the sixth possible implementation manner of the second aspect after the N second images are obtained, further include:
  • the second image quality evaluation information is a number or a score.
  • the target image is the first image and the N second images, and the first image 2.
  • the image quality evaluation information is an image with a predetermined number, or an image with a score corresponding to the second image quality evaluation information greater than a predetermined score threshold.
  • this application provides an image processing method, including:
  • each of the N second images is obtained by processing the first image through at least one of the M image enhancement models ;
  • N and M are integers greater than zero, each of the M image enhancement models is different, and the N second images are also different;
  • a target image is output, and the target image is at least one of the first image and the N second images.
  • one second image can be obtained, or at least two second images can also be obtained, which is not limited here.
  • M can be equal to N, and M can also be greater than N.
  • the image quality evaluation information may be image quality evaluation information for N second images, or image quality evaluation information for the first image and N second images.
  • the quality evaluation model when the input image of the quality evaluation model is N second images, the quality evaluation model is used to evaluate the image quality of the N second images, and the obtained image quality evaluation information is the image quality evaluation information of the N second images.
  • the quality evaluation model can be used to assist in evaluating the image quality of the N second images through the first image, and the obtained image quality evaluation information is N Image quality evaluation information of the second image; through the first image to assist in evaluating the image quality of the second image, the accuracy of the evaluation result of the second image can be improved.
  • the quality evaluation model can also be used to evaluate the image quality of the first image and N second images, and the obtained image quality evaluation information is for the first image.
  • the image quality evaluation information of the image and the N second images; the electronic device can use the image quality evaluation information to determine whether the image quality of the second image is better than the image quality of the first image, and can more accurately know the passage of the first image Whether the image quality of the second image obtained after the image enhancement model has deteriorated.
  • the target image can be the image with the best image quality among the first image and the N second images.
  • the image with the best image quality refers to the image with the image quality evaluation information of the predetermined number in the first image and the N second images. Or an image whose score corresponding to the image quality evaluation information is greater than a predetermined score threshold.
  • the method of outputting the target image can be to display the target image or save the target image to the gallery.
  • the image quality of the second image may be worse than the image quality of the first image.
  • the second image includes a human face image, and the human face The eyes, nose, glasses frame, etc. in the image are deformed, purple fringing appears on the edge of the second image, and there are artifacts such as ghosting, color aliasing, and zipper effect in the second image. Therefore, the second image is evaluated by the quality evaluation model. The image quality of the image, or the image quality of the first image and the second image is evaluated, and an image with better image quality is finally output from the first image and the second image according to the evaluation result for the user to view.
  • the electronic device when the image quality of the processed second image deteriorates, the electronic device finally outputs the first image to the user to view, so as to solve the problem of outputting the image that has deteriorated image quality after being processed by the image enhancement model in the prior art.
  • the problem for the user can improve the quality of the output image and reduce the probability of outputting a poor quality image, so as to improve the user's visual experience.
  • the quality evaluation model is obtained by training based on a plurality of training samples, and each training sample includes a sample image and image quality evaluation information of the sample image by a user.
  • the training samples of the quality evaluation model may include the output after the image enhancement model processes the original image.
  • the quality evaluation model can be used to assist in evaluating the image quality of the second image through the first image to output the image quality evaluation information of the second image.
  • the training sample of the evaluation model may include the original image, the sample image output after the original image is processed by the image enhancement model, and the image quality evaluation information marked by the user on the sample image.
  • the training samples of the quality evaluation model can include the original image and the image enhancement model.
  • the sample image output after processing the original image, the image quality evaluation information marked by the user on the original image, and the image quality evaluation information marked by the user on the sample image.
  • the sample image used in the process of training the quality evaluation model corresponds to the input image used when the trained quality evaluation model is applied to evaluate the image quality. Since the quality evaluation model is trained using multiple sample images, when the trained quality evaluation model is used to evaluate the image quality of the input image, the accuracy of the image quality evaluation result obtained is higher.
  • the target image includes a target image determined according to the obtained image quality evaluation information and an evaluation rule, and the evaluation rule is that the image quality evaluation information is a predetermined number. , Or, the score corresponding to the image quality evaluation information is greater than or equal to the predetermined score threshold.
  • the image quality evaluation information when the image quality evaluation information is a predetermined number, or the score corresponding to the image quality evaluation information is greater than or equal to the predetermined score threshold, it means that the image quality of the image corresponding to the image quality evaluation information is better.
  • the target image determined according to the image quality evaluation information and evaluation rules is the image with better image quality among the first image and N second images, and the electronic device can output the target image with better image quality for the user to view, reducing the output quality The probability of a bad image.
  • the image quality evaluation information may be a number or a score used to represent the image quality.
  • the image quality evaluation information can also be represented by letters or words, which is not limited here.
  • the image quality evaluation information may be "0" or "1”, “YES” or “NO”, “Yes” or “No”.
  • the image processing method in this embodiment may include the following three solutions.
  • the first image is image A
  • the second image is image B
  • Option 1 When the input image of the quality evaluation model is image B, the quality evaluation model is used to evaluate the image quality of image B. If the electronic device determines that the image quality of image B is the best according to the image quality evaluation information output by the quality evaluation model Good, then output image B, otherwise output image A.
  • the quality evaluation model can be used to assist in evaluating the image quality of image B through image A. If the electronic device outputs image quality evaluation information according to the quality evaluation model It is determined that the image quality of image B is the best, then image B is output, otherwise image A is output.
  • Solution 3 When the input images of the quality evaluation model are image A and image B, the quality evaluation model is used to evaluate the image quality of image A and image B. If the electronic device determines the image quality of image B according to the image quality evaluation information output by the quality evaluation model If the image quality is better than the image quality of image A, then output image B; if the electronic device determines that the image quality of image B is worse than the image quality of image A according to the image quality evaluation information output by the quality evaluation model, then output image A; if the electronic device According to the image quality evaluation information output by the quality evaluation model, it is determined that the image quality of the image B is the same as the image quality of the image A, and then any one of the image A and the image B is selected for output.
  • the image quality evaluation information can be represented by identification information.
  • the identification information can be numbers, letters, words, etc., and the electronic device can determine the meaning of the numbers, words, or letters in the identification information according to a pre-established correspondence or rule , Thereby outputting image A or image B. For example, for scheme one and scheme two, "0" is used to indicate that the image quality of image B is poor, and "1" is used to indicate that the image quality of image B is good.
  • the electronic device determines that the image quality evaluation result corresponding to the image quality evaluation information is that the image quality of image B is the best, and image B is the target image, and image B is output; when the image quality evaluation information is 0, The electronic device determines that the image quality evaluation result corresponding to the image quality evaluation information is that the image quality of image A is the best, and image A is the target image, and image A is output.
  • image quality evaluation information is 1
  • the electronic device determines that the image quality evaluation result corresponding to the image quality evaluation information is that the image quality of image B is the best, and image B is the target image, and image B is output; when the image quality evaluation information is 0,
  • the electronic device determines that the image quality evaluation result corresponding to the image quality evaluation information is that the image quality of image A is the best, and image A is the target image, and image A is output.
  • scheme three use "0" to indicate that the image quality of image B is worse than that of image A, use "1" to indicate that the image quality of image B is better than that of image A,
  • the electronic device determines according to the image quality evaluation information that the image quality of image B is better than that of image A, and image B is the target image, and image B is output; when the image quality evaluation information is 0, the electronic device According to the image quality evaluation information, the device determines that the image quality of image B is worse than that of image A, image A is the target image, and image A is output; when the image quality evaluation information is 2, the electronic device determines the image quality of image B according to the image quality evaluation information.
  • the image quality is the same as that of the image A, and both the image A and the image B are the target images, and any one of the image A and the image B can be selected for output.
  • the image quality evaluation information may be represented by scores.
  • the image quality evaluation information is the score corresponding to image B, and the electronic device compares the score corresponding to image B with a predetermined score threshold.
  • the score corresponding to image B is less than or equal to the predetermined score threshold, it means The image quality of image B is poor, image A is the target image, and the electronic device outputs image A.
  • the score corresponding to image B is greater than the predetermined score threshold, it means that the image quality of image B is good, image B is the target image, and the electronic device outputs image B .
  • the image quality evaluation information includes the score A of the image A and the score B of the image B.
  • the electronic device compares the score A corresponding to the image B with the score B corresponding to the image B.
  • the score B is greater than the score A, it represents the image
  • the image quality of B is better than that of image A, image B is the target image, and the electronic device outputs image B; when score B is less than score A, it means that the image quality of image B is worse than that of image A, and image A is the target Image, the electronic device outputs image A.
  • score B is equal to score A, it means that the image quality of image B is the same as that of image A.
  • Both image A and image B are target images, and the electronic device selects from image A and image B Any image output.
  • the image quality evaluation information can be represented by a number "0" or "1", or the image quality evaluation information can be represented by a score, and the image quality evaluation information can be expressed in a flexible manner and can intuitively reflect the image quality.
  • the image quality evaluation information is a number, and the input image is N second images, or the first image and the N second images ,
  • the image quality evaluation information is image quality evaluation information for each of the N second images;
  • outputting a target image according to the obtained image quality evaluation information includes:
  • the target second image whose image quality evaluation information is a predetermined number in the N second images it is determined that the target second image whose image quality evaluation information is a predetermined number is the image with the best image quality, and all the images are output.
  • the image quality evaluation information is a predetermined number of target second images.
  • the predetermined number may be "1".
  • the electronic device can classify the image quality of the input image through the quality evaluation model, so as to determine whether the input image belongs to the category of good image quality or the category of poor image quality.
  • the output image quality evaluation information is a predetermined number; when the input image belongs to the category of poor image quality, the output image quality evaluation information is not a predetermined number. Performing two classifications on the image quality of the input image has less computational overhead, and the image quality evaluation result can be obtained faster.
  • the method further includes:
  • the first image is an image with the best image quality, and the first image is output.
  • the image quality evaluation information of the N second images is not a predetermined number, which means that the image quality of the N second images is worse than the image quality of the first image.
  • the first image is output for the user to view. That is, when the image quality of the N second images obtained after the first image is processed by the image enhancement model deteriorates, the first image with better image quality can be output for the user to view, thereby improving the image quality of the output image.
  • the determining the image with the best image quality among the first image and the N second images according to the image quality evaluation information, and outputting the image with the best quality Good images also include:
  • any target second image is selected for output.
  • any image with better image quality may be output.
  • the image quality evaluation information is a score and the input image is N second images, or when the first image and the N second images ,
  • the image quality evaluation information is image quality evaluation information for each of the N second images
  • outputting a target image according to the obtained image quality evaluation information includes:
  • the target second image with the highest score is determined to be the image with the best image quality, and the target second image with the highest score is output. image.
  • the feature information of the first image can be used to assist in evaluating the image quality of the second image. Since the electronic device can use the first image as a reference image, the feature information of the second image can be obtained from the feature information of the first image. According to the difference feature information between the two, the image quality of the second image is evaluated according to the difference feature information, which can improve the accuracy of the image quality evaluation information of the second image. In addition, the image quality evaluation information is represented by scores, which can more accurately describe the quality of the image. The electronic device can determine the target image with the best image quality among the N second images according to the score corresponding to each second image, so as to output the image with the best image quality for the user to view.
  • the method further includes:
  • the first image is an image with the best image quality, and the first image is output.
  • an image with a score greater than a predetermined score threshold is an image with better image quality
  • the electronic device outputs the first image for the user to view.
  • the outputting the target image according to the obtained image quality evaluation information includes:
  • a new one is acquired An image enhancement model, using the acquired new image enhancement model to process the first image to obtain a new second image, and inputting the new second image as an input image into the quality evaluation model for processing , Obtain new image quality evaluation information; wherein the new image enhancement model is an image enhancement model that has not processed the first image;
  • the new image quality evaluation information is the predetermined number, or the score is greater than the predetermined score threshold, then a new second image is output, otherwise it returns to the step of acquiring a new image enhancement model and subsequent steps until The number of return executions reaches a preset number threshold, and the first image is output.
  • the preset threshold of times is set according to the total number of image enhancement models in the electronic device.
  • the preset threshold of times may be equal to the total number of image enhancement models minus one. For example, when the total number of image enhancement models is 3, the preset number threshold may be 2.
  • the electronic device may preferentially use the image enhancement model 1 with the best image processing effect to process the first image. If the image quality of the processed image is poor, then use the image enhancement model with the second best image processing effect. 2 Process the first image. If the image quality of the processed image is still poor, select the optimal image enhancement model from the optional image enhancement models. 3 Process the first image, the optional image enhancement model Refers to image enhancement models other than the previously used image enhancement models (such as the optimal and suboptimal image enhancement models for image processing). Since the image enhancement model with the best image processing effect is preferentially used to process the first image, in some cases, the second image with better image quality can be obtained without using N image enhancement models. Compared with the case where N image enhancement models are used to process the first image in parallel, part of the system resources can be saved, and the time required to obtain the second image with better image quality can be shortened, so as to improve the efficiency of the output image.
  • the image quality evaluation information is a number
  • the input image is a first image and N second images
  • the image quality evaluation information is Image quality evaluation information of each second image in the first image and the N second images
  • outputting a target image according to the obtained image quality evaluation information includes:
  • the image quality evaluation information of the first image is a predetermined number, and there is at least one target second image whose image quality evaluation information is the predetermined number among the N second images, then the first image and the Select any one of the target second images to output.
  • the image quality of the first image and the second image can be evaluated, and the image quality evaluation information for the first image and the second image can be obtained.
  • the image quality evaluation information can be used to determine whether the image quality of the second image is The image quality of the first image is better than that of the first image, and it can be learned more accurately whether the image quality of the second image obtained after the first image is processed by the image enhancement model is deteriorated.
  • the method further includes:
  • the image quality evaluation information of the first image is not a predetermined number, and there is at least one target second image whose image quality evaluation information is a predetermined number among the N second images, select from the target second image Any image output.
  • the image quality evaluation information is a predetermined number, it indicates that the image quality is good. Therefore, when the image quality evaluation information of the first image is not a predetermined number, and at least one image quality evaluation information in the N second images is When pre-determining the number of target second images, it means that the image quality of the first image is poor, and there is at least one target second image with better image quality among the N second images. At this time, the electronic device can output any target The second image is for the user to view.
  • the method further includes:
  • the first image is output.
  • the electronic device outputs the first image to the user for viewing.
  • the image quality evaluation information is a score
  • the input image is a first image and N second images
  • the image quality evaluation information is Image quality evaluation information of each second image in the first image and the N second images
  • outputting a target image according to the obtained image quality evaluation information includes:
  • the scores of the first image and the scores of the N second images from the first image and the N second images, determine that the image with the highest score is the image with the best image quality, and output The image with the highest score.
  • the image with the highest score among the first image and the N second images is the image with the best image quality.
  • the electronic device can directly compare the respective scores of the first image and the N second images, filter out the image with the highest score, and then determine the image with the best image quality, which can more accurately determine the image with the best quality image.
  • the acquiring the first image includes:
  • RAW images are images in RAW format.
  • RAW is an unprocessed and uncompressed format.
  • RAW can be conceptualized as "raw image coded data" or more vividly called “digital film”. It can be understood as:
  • RAW image is the original data that the image sensor converts the captured light source signal into digital signal.
  • the image sensor may include a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor and a Charge Coupled Device (CCD) image sensor.
  • CMOS Complementary Metal-Oxide-Semiconductor
  • CCD Charge Coupled Device
  • the first image can be obtained by image fusion processing on the acquired RAW image
  • the second image obtained after the first image is processed by the image enhancement model and the image quality of the second image can be evaluated through the quality evaluation model, or Evaluate the image quality of the first image and the second image, and finally output a better image quality image from the first image and the second image for the user to view according to the evaluation result, which can improve the image quality of the output image.
  • the performing image fusion processing on the multiple frames of RAW images to obtain the first image includes:
  • the multi-frame RAW images are divided into at least two groups, and image fusion processing is performed on each group of RAW images to obtain at least two first images.
  • a group of RAW images corresponds to a first image.
  • multiple frames of RAW images can be grouped, and the grouped RAW images can be subjected to image fusion processing to obtain the first image, and the image quality of the fused image can be enhanced to obtain the second image.
  • the image of the second image is evaluated. Quality, or evaluate the image quality of the first image and the image quality of the second image, and output a target image with the best image quality according to the evaluation result, and a picture with better image quality can be taken.
  • the target image may be the second image or the first image. Since the mobile phone always outputs images with better image quality, the possibility of storing images with poor image quality in the mobile phone is lower, and the user is less likely to view images with poor image quality, which can improve the user's visual experience.
  • the target image is the second image; if the image quality evaluation information is not a predetermined number, or the score corresponding to the image quality evaluation information Less than a predetermined score threshold, the target image is the first image;
  • the image quality evaluation information is image quality evaluation information for the first image and the second image, and the image quality evaluation information is used to indicate whether the image quality of the second image is When the image quality of the first image is better than that of the first image, if the image quality evaluation information is a predetermined number, then the target image is the second image; if the image quality evaluation information is not a predetermined number, then the target The image is the first image;
  • the image quality evaluation information includes the image quality evaluation information corresponding to the first image and the second image
  • the image quality evaluation information of the first image is a predetermined number
  • the image quality evaluation information of the second image is a predetermined number
  • the target image is any one of the second image and the first image; if the image quality evaluation information of the first image is not predetermined If the image quality evaluation information of the second image is a predetermined number, the target image is the second image; if the image quality evaluation information of the first image is a predetermined number, and the second image If the image quality evaluation information of is not a predetermined number, the target image is the first image; or,
  • the target image is the second image; if the image quality of the second image is The score corresponding to the evaluation information is less than the score corresponding to the image quality evaluation information of the first image, then the target image is the first image; if the score corresponding to the image quality evaluation information of the second image is equal to the first image A score corresponding to the image quality evaluation information of an image, then the target image is any one of the second image and the first image;
  • the image quality evaluation information is the image quality evaluation information of the second image of each of the N second images, if there is image quality in the N second images If the evaluation information is a predetermined number of target second images, the target image is at least one of the target second images; if there is no target second image of which image quality evaluation information is a predetermined number among the N second images, Then the target image is the first image; or,
  • the target image is at least one of the target second images; if the N If there is no target second image whose score corresponding to the image quality evaluation information is greater than or equal to a predetermined score threshold in the second image, the target image is the first image;
  • the target image is the The image quality evaluation information in the first image and the N second images is any image in which the image quality evaluation information is a predetermined number, or the target image is corresponding to the image quality evaluation information in the first image and the N second images Any image with a score greater than or equal to a predetermined score threshold, or the target image is the image with the highest score corresponding to the image quality evaluation information among the first image and the N second images.
  • images with better image quality can be selected from the first image and N second images through a predetermined number or a predetermined score threshold for output.
  • this application provides an image processing method, including:
  • the first image is input to M image enhancement models for processing to obtain N second images, and all the images are displayed or saved.
  • N and M are positive integers
  • each of the M image enhancement models is different, and the N second images are also different.
  • the electronic device acquires the first image in portrait mode, scenery mode, indoor mode, telephoto mode (hereinafter referred to as high-power zoom mode), and so on.
  • the first image can be a preview image or a captured photo.
  • the image quality evaluation model is first used to evaluate the image quality of the acquired first image, and whether to process the first image is determined according to the evaluation result.
  • the image enhancement model is used to process the first image, and the second image is output; when the image quality of the first image is poor, the first image is output to the user for viewing to reduce
  • the data processing speed of mobile phones can be improved due to the consumption of resources occupied by image processing.
  • the first image when it is determined that the image quality of the first image does not meet the requirements according to the first image quality evaluation information, the first image is displayed or saved.
  • the first image quality evaluation information is a number or a score.
  • the image quality evaluation information can be represented by a number "0" or "1”, or the image quality evaluation information can be represented by a score, and the image quality evaluation information can be expressed in a flexible manner, which can more intuitively reflect the image quality.
  • the first image quality evaluation information when the first image quality evaluation information is a number, that the image quality of the first image meets a requirement means that the first image quality evaluation information is a predetermined number.
  • the electronic device can classify the image quality of the first image into two categories through the quality evaluation model, so as to determine whether the first image belongs to the category of good image quality or the category of poor image quality.
  • the output image quality evaluation information is a predetermined number; when the first image belongs to the category of poor image quality, the output image quality evaluation information is not a predetermined number. Performing two classifications on the image quality of the first image has less computational overhead, and the image quality evaluation result of the first image can be obtained faster.
  • the image quality of the first image meets the requirement means that the score of the first image is greater than or equal to a preset Threshold.
  • the image quality evaluation information is represented by a score, and the score can more accurately describe the image quality of the first image.
  • the electronic device compares the score of the first image with a preset threshold, and can more accurately determine whether the image quality of the first image meets the requirements.
  • the first image is obtained by the electronic device in a high-power zoom photographing mode.
  • the first image obtained may cause image distortion due to shaking, distortion of the facial features such as the eyes of the portrait, or blurry image (for example, It is difficult to recognize the subject), etc. Therefore, it is necessary to evaluate the image quality of the first image. If the image quality of the first image is poor, then display the first image to the user for viewing, or save the first image to the gallery; if the image quality of the first image is better, then process the first image to obtain Nth Two images, and display or save at least one of the N second images.
  • the data processing speed of the electronic device can be increased by saving resources consumed by processing the first image with poor image quality.
  • the preset threshold value is 0.25, and both M and N are 1.
  • the electronic device obtains a first image, uses a quality evaluation model to evaluate the image quality of the first image, and obtains a score corresponding to the first image.
  • the score of the first image is less than 0.25, it means that the image quality of the first image does not meet the requirements and save the first image; when the score of the first image is greater than or equal to 0.25, it means that the image quality of the first image meets the requirements, and the first image is
  • the image enhancement model is not input for processing, which can save some resources and increase the data processing speed of the electronic device.
  • the method further includes:
  • the target image can be the image with the best image quality among the first image and the N second images.
  • the image with the best image quality refers to the first image and the N second images in which the second image quality evaluation information is a predetermined number.
  • the quality evaluation model is used to evaluate the image quality of the first image; when the image quality of the first image is poor, the first image is output to the user for viewing; When the image quality of an image is good, the first image is input to the image enhancement model for processing to obtain the second image, and the quality evaluation model is used to evaluate the image quality of the second image. If the image quality of the second image is good, the second image is output to The user views, if the image quality of the second image is poor, the first image is output to the user for viewing. In this way, the probability of outputting poor quality images can be reduced, and the user experience can be improved.
  • the second image quality evaluation information is a number or a score.
  • the target image is the first image and the N second images
  • the second image quality evaluation information is a predetermined digital image, or the An image whose score corresponding to the second image quality evaluation information is greater than a predetermined score threshold.
  • the electronic device can determine the target image with better image quality from the first image and N second images according to the second image quality evaluation information, and output the target image with better image quality to the user for viewing, reducing the output quality of the target image.
  • the probability of the image is a predetermined number, or the score corresponding to the image quality evaluation information is greater than or equal to the predetermined score threshold, it means that the image quality of the image corresponding to the image quality evaluation information is good, therefore,
  • the electronic device can determine the target image with better image quality from the first image and N second images according to the second image quality evaluation information, and output the target image with better image quality to the user for viewing, reducing the output quality of the target image. The probability of the image.
  • this application provides an image processing device, including:
  • the acquiring unit is configured to acquire a first image and N second images, where each second image of the N second images is performed on the first image through at least one image enhancement model of the M image enhancement models. Obtained after processing; where N and M are integers greater than zero, each of the M image enhancement models is different, and the N second images are also different;
  • An image quality evaluation unit configured to process an input image input quality evaluation model to obtain image quality evaluation information, where the input image includes the N second images, or includes the first image and the N second images image;
  • the image output unit is configured to output a target image according to the obtained image quality evaluation information, where the target image is at least one of the first image and the N second images.
  • one second image can be obtained, or at least two second images can also be obtained, which is not limited here.
  • M can be equal to N, and M can also be greater than N.
  • the corresponding beneficial effects of the image processing device provided by the fifth aspect are the same as the beneficial effects of the image processing methods of the first and third aspects, and will not be repeated here.
  • the quality evaluation model is obtained through training based on a plurality of training samples, and each training sample includes a sample image and image quality evaluation information of a user on the sample image.
  • the target image includes a target image determined according to the obtained image quality evaluation information and an evaluation rule, and the evaluation rule is that the image quality evaluation information is a predetermined number. , Or, the score corresponding to the image quality evaluation information is greater than or equal to the predetermined score threshold.
  • the image quality evaluation information may be a number or a score used to represent the image quality.
  • the image quality evaluation information can also be represented by letters or words, which is not limited here.
  • the image quality evaluation information may be "0" or "1”, “YES” or “NO”, “Yes” or “No”.
  • the image quality evaluation information is a number, and the input image is N second images, or the first image and the N second images ,
  • the image quality evaluation information is image quality evaluation information for each of the N second images;
  • the image output unit is specifically configured to:
  • the target second image whose image quality evaluation information is a predetermined number in the N second images it is determined that the target second image whose image quality evaluation information is a predetermined number is the image with the best image quality, and all the images are output.
  • the image quality evaluation information is a predetermined number of target second images.
  • the predetermined number may be "1".
  • the image output unit after determining whether there is a target second image whose image quality evaluation information is a predetermined number among the N second images, is further configured to: If there is no target second image whose image quality evaluation information is a predetermined number among the N second images, it is determined that the first image is an image with the best image quality, and the first image is output.
  • the image output unit is further configured to:
  • any target second image is selected for output.
  • the image quality evaluation information is a score and the input image is N second images, or when the first image and the N second images ,
  • the image quality evaluation information is image quality evaluation information for each of the N second images
  • the image output unit is specifically configured to:
  • the target second image with the highest score is determined to be the image with the best image quality, and the target second image with the highest score is output. image.
  • the image output unit is further configured to: after determining whether there is a target second image with a score greater than a predetermined score threshold in the N second images, if the N second images If there is no target second image with a score greater than a predetermined score threshold in the second image, it is determined that the first image is an image with the best image quality, and the first image is output.
  • the image output unit is further configured to:
  • a new one is acquired An image enhancement model, using the acquired new image enhancement model to process the first image to obtain a new second image, and inputting the new second image as an input image into the quality evaluation model for processing , Obtain new image quality evaluation information; wherein the new image enhancement model is an image enhancement model that has not processed the first image;
  • the new image quality evaluation information is the predetermined number, or the score is greater than the predetermined score threshold, then a new second image is output, otherwise it returns to the step of acquiring a new image enhancement model and subsequent steps until The number of return executions reaches a preset number threshold, and the first image is output.
  • the image quality evaluation information is a number
  • the input image is a first image and N second images
  • the image quality evaluation information is Image quality evaluation information of each second image in the first image and the N second images
  • the image output unit is specifically configured to:
  • the image quality evaluation information of the first image is a predetermined number, and there is at least one target second image whose image quality evaluation information is the predetermined number among the N second images, then the first image and the Select any one of the target second images to output.
  • the image output unit determines whether there is a target second image whose image quality evaluation information is a predetermined number among the first image and the N second images, it also uses At:
  • the image quality evaluation information of the first image is not a predetermined number, and there is at least one target second image whose image quality evaluation information is a predetermined number among the N second images, select from the target second image Any image output.
  • the image output unit is further configured to:
  • the first image is output.
  • the image quality evaluation information is a score
  • the input image is a first image and N second images
  • the image quality evaluation information is Image quality evaluation information of each second image in the first image and the N second images
  • the image output unit is also used for:
  • the scores of the first image and the scores of the N second images from the first image and the N second images, determine that the image with the highest score is the image with the best image quality, and output The image with the highest score.
  • the acquiring unit includes:
  • RAW image acquisition unit for acquiring multiple frames of RAW images
  • the image fusion unit is used to perform image fusion processing on the multiple frames of RAW images to obtain a first image.
  • the fused image corresponding to the RAW image can be used as a reference image, and the fused image corresponding to each group of RAW images can be used to assist in evaluating the image quality of the enhanced image corresponding to the group of RAW images, which can improve the image quality evaluation information of the enhanced image Accuracy.
  • the image fusion unit is specifically configured to:
  • the multi-frame RAW images are divided into at least two groups, and image fusion processing is performed on each group of RAW images to obtain at least two first images.
  • the target image is the second image; if the image quality evaluation information is not a predetermined number, or the score corresponding to the image quality evaluation information Less than a predetermined score threshold, the target image is the first image;
  • the image quality evaluation information is image quality evaluation information for the first image and the second image, and the image quality evaluation information is used to indicate whether the image quality of the second image is When the image quality of the first image is better than that of the first image, if the image quality evaluation information is a predetermined number, then the target image is the second image; if the image quality evaluation information is not a predetermined number, then the target The image is the first image;
  • the image quality evaluation information includes the image quality evaluation information corresponding to the first image and the second image
  • the image quality evaluation information of the first image is a predetermined number
  • the image quality evaluation information of the second image is a predetermined number
  • the target image is any one of the second image and the first image; if the image quality evaluation information of the first image is not predetermined If the image quality evaluation information of the second image is a predetermined number, the target image is the second image; if the image quality evaluation information of the first image is a predetermined number, and the second image If the image quality evaluation information of is not a predetermined number, the target image is the first image; or,
  • the target image is the second image; if the image quality of the second image is The score corresponding to the evaluation information is less than the score corresponding to the image quality evaluation information of the first image, then the target image is the first image; if the score corresponding to the image quality evaluation information of the second image is equal to the first image A score corresponding to the image quality evaluation information of an image, then the target image is any one of the second image and the first image;
  • the image quality evaluation information is the image quality evaluation information of the second image of each of the N second images, if there is image quality in the N second images If the evaluation information is a predetermined number of target second images, the target image is at least one of the target second images; if there is no target second image of which image quality evaluation information is a predetermined number among the N second images, Then the target image is the first image; or,
  • the target image is at least one of the target second images; if the N If there is no target second image whose score corresponding to the image quality evaluation information is greater than or equal to a predetermined score threshold in the second image, the target image is the first image;
  • the target image is the The image quality evaluation information in the first image and the N second images is any image in which the image quality evaluation information is a predetermined number, or the target image is corresponding to the image quality evaluation information in the first image and the N second images Any image with a score greater than or equal to a predetermined score threshold, or the target image is the image with the highest score corresponding to the image quality evaluation information among the first image and the N second images.
  • this application provides an image processing device, including:
  • the first evaluation unit is configured to input the first image into a quality evaluation model for processing to obtain first image quality evaluation information
  • An image processing unit configured to input the first image into M image enhancement models for processing when it is determined that the image quality of the first image meets the requirements according to the first image quality evaluation information, to obtain N second images , And display or save one or more of the N second images; where N and M are positive integers, each of the M image enhancement models is different, and the N image enhancement models are different. The two images are also different.
  • the electronic device acquires the first image in portrait mode, scenery mode, indoor mode, telephoto mode (hereinafter referred to as high-power zoom mode), and so on.
  • the first image can be a preview image or a captured photo.
  • the first image is displayed or saved.
  • the first image quality evaluation information is a number or a score.
  • the first image quality evaluation information when the first image quality evaluation information is a number, that the image quality of the first image meets the requirement means that the first image quality evaluation information is a predetermined number.
  • the image quality of the first image meets the requirement means that the score of the first image is greater than or equal to a preset Threshold.
  • the first image is obtained by the electronic device in a high-power zoom photographing mode.
  • the preset threshold value is 0.25, and both M and N are 1.
  • the image processing apparatus further includes:
  • the second evaluation unit is configured to input the N second images, or the first image and the N second images as input images after the N second images are obtained by the image processing unit
  • the quality evaluation model processes the image to be processed and the input quality evaluation model is processed to obtain second image quality evaluation information
  • the output unit is configured to display or save a target image according to the second image quality evaluation information, where the target image is at least one of the first image and the N second images.
  • the second image quality evaluation information is a number or a score.
  • the target image is the first image and the N second images
  • the second image quality evaluation information is a predetermined digital image, or the An image whose score corresponding to the second image quality evaluation information is greater than a predetermined score threshold.
  • the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor.
  • the electronic device executes the image processing method in any possible implementation manner of the foregoing first aspect or the third aspect, or executes the image processing method in any possible implementation manner of the foregoing second aspect or the fourth aspect.
  • this application provides an electronic device that includes a storage module, a processing module, and a computer program stored in the storage module and capable of running on the processing module.
  • the electronic device executes the computer program
  • the electronic device is caused to execute the image processing method in any possible implementation manner of the foregoing first aspect or the third aspect, or execute the image processing method in any possible implementation manner of the foregoing second aspect or the fourth aspect.
  • the present application provides a computer-readable storage medium that stores a computer program that, when executed by a processor, causes an electronic device to perform the above-mentioned first aspect or third aspect.
  • this application provides a computer program product that, when the computer program product runs on an electronic device, causes the electronic device to execute the image processing method of any one of the possible implementations of the first aspect or the third aspect, Or execute the image processing method in any one of the possible implementation manners of the second aspect or the fourth aspect described above.
  • the image quality of the second image is evaluated through the quality evaluation model, or the image quality of the first image and the second image is evaluated, and based on the evaluation
  • images with better image quality among the first image and the second image are finally output for the user to view. That is to say, when the image quality of the second image obtained after the first image processing deteriorates, the electronic device finally outputs image A for the user to view, so as to solve the problem of the image quality change obtained after the image enhancement model is processed in the prior art.
  • the problem of poor image output to the user can improve the image quality of the output image, and can reduce the probability of outputting a poor quality image, so as to improve the user's visual experience.
  • FIG. 1 is a scene diagram provided by an embodiment of the present application.
  • FIG. 2 is a schematic flowchart of an image processing method provided by the first embodiment of the present application.
  • FIG. 3 is a noisy image provided by an embodiment of the present application.
  • FIG. 4 is an image obtained after denoising the noisy image in FIG. 3 provided by an embodiment of the present application.
  • FIG. 5 is an image with color aliasing provided by an embodiment of the present application.
  • Fig. 6 is a schematic diagram of a user interface provided by an embodiment of the present application.
  • FIG. 7 is a schematic flowchart of an image processing method provided by a second embodiment of the present application.
  • FIG. 8 is a schematic flowchart of an image processing method provided by the third embodiment of the present application.
  • FIG. 9 is a schematic flowchart of an image processing method provided by the fourth embodiment of the present application.
  • FIG. 10 is a schematic flowchart of an image processing method provided by a fifth embodiment of the present application.
  • FIG. 11 is a schematic flowchart of a training quality evaluation model provided by an embodiment of the present application.
  • FIG. 12 is a schematic flowchart of an image processing method provided by a sixth embodiment of the present application.
  • FIGS. 13a to 13d are schematic diagrams of a user interface provided by another embodiment of the present application.
  • FIG. 14 is a schematic diagram of a user interface provided by still another embodiment of the present application.
  • FIG. 15 is a schematic diagram of a user interface provided by another embodiment of the present application.
  • FIG. 16 is a schematic diagram of a user interface provided by still another embodiment of the present application.
  • FIG. 17 is a schematic flowchart of an image processing method provided by a seventh embodiment of the present application.
  • FIG. 18 is a schematic flowchart of an image processing method provided by the eighth embodiment of the present application.
  • FIG. 19 is a schematic flowchart of an image processing method according to a ninth embodiment of the present application.
  • FIG. 21 is a schematic flowchart of an image processing method according to an eleventh embodiment of the present application.
  • FIG. 22 is a schematic flowchart of an image processing method according to a twelfth embodiment of the present application.
  • FIG. 23 is a schematic diagram of a method for processing multi-frame RAW images according to an embodiment of the present application.
  • FIG. 24 is a schematic diagram of a method for processing multi-frame RAW images according to another embodiment of the present application.
  • FIG. 25 is a schematic diagram of a method for processing multi-frame RAW images according to still another embodiment of the present application.
  • FIG. 26 is a schematic structural diagram of an image processing device according to an embodiment of the present application.
  • FIG. 27 is a schematic structural diagram of an image processing apparatus according to another embodiment of the present application.
  • FIG. 28 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • the term “if” can be construed as “when” or “once” or “in response to determination” or “in response to detecting “.
  • the phrase “if determined” or “if detected [described condition or event]” can be interpreted as meaning “once determined” or “in response to determination” or “once detected [described condition or event]” depending on the context ]” or “in response to detection of [condition or event described]”.
  • shooting devices such as smartphones usually use a scene recognition algorithm to automatically identify the shooting scene of an image, and then use an image enhancement algorithm or model corresponding to the shooting scene to process the image to enhance image quality.
  • the smart phone obtains image A
  • the moon is identified by the scene recognition algorithm
  • the shooting scene of image A is the moon.
  • the moon enhancement algorithm or moon enhancement model can be used to process the moon in image A to enhance the details of the moon.
  • the image B with the clear outline of the moon is obtained, and the image B is output as the final image for users to view;
  • the scene recognition algorithm is used to recognize the face, then the shooting scene of image A can be used for portrait shooting.
  • the resolution algorithm or the face super-resolution model performs super-resolution processing on the face image to enhance the sharpness of the face to obtain the image B, and the image B is output as the final image for the user to view.
  • the current image processing method can identify the shooting scene corresponding to the image A, it cannot predict whether the image B that is finally output after the processing has the problem of image quality deterioration relative to the image A.
  • the scene recognition algorithm is usually a scene recognition algorithm based on deep learning. Since the scene recognition algorithm based on deep learning usually requires manual labeling of a large number of scenes for training, the processed scenes may not be accurately labeled due to unobvious discrimination or high complexity of the scene, and it is difficult to cover all the scenes.
  • the recognition algorithm does not cover the moon or human face in a certain scene, and the processed output image may have artifacts.
  • the low-level image information is relative to the high-level image information.
  • the high-level image information mainly includes the description of the semantics of the image (the relationship between the scene and the target), and the low-level image information mainly includes the pixels, image blocks, edges, corners, and textures of the image.
  • the texture of the image is the inherent feature of the image related to the surface structure and material of the object, and it reflects the global feature of the image.
  • the texture of an image can be described as: the spatial distribution law of changes in the gray level of pixels in a neighborhood, including many important image information such as the surface structure and the relationship with the surrounding environment.
  • Image de-raining, de-hazing, image de-noising, de-blurring, and super-resolution are all low-level image processing problems.
  • the purpose of removing rain from an image is to remove rain lines (raindrops) in an image with rain while retaining the original structural features of the image.
  • the purpose of defogging is to remove the fog in the foggy image while retaining the original structural features of the image.
  • artifacts include but are not limited to artifacts, bright spots, checkboard artifacts, bright edges caused by sharpening, burrs, shot noise, sawtooth effects, etc.
  • Artifacts include streaks, chrominance noise, and sparse data noise.
  • the checkerboard effect usually refers to the "checkerboard grid-like artifacts" that often appear in the image, especially the dark parts.
  • the sawtooth effect is also called the zipper effect, which refers to the edge junction or color mutation area of the image, such as a stepped shape.
  • FIG. 1 is a scene diagram provided by an embodiment of the present application.
  • user A can use an electronic device (for example, a mobile phone, a tablet computer, etc.) to take pictures of user B or the surrounding environment to obtain image A.
  • the user can also download the image A from the Internet through the electronic device, for example, download the image A shared through a social application, and can also select the image A from the gallery or album of the electronic device. Since image A may have at least any one of low resolution, image noise, spectacle reflection, and eye reflection, image A needs to be image processed.
  • the image processing method can be: the electronic device obtains the image A to be processed, inputs the image A into M image enhancement models, and processes the image A through the M image enhancement models to obtain N images B; transfers the N images B, or
  • the image A and N images B are input into the trained quality evaluation model for processing to obtain the image quality evaluation information output by the quality evaluation model, and the image with the best image quality among the image A and the N images B is output according to the image quality evaluation information.
  • the quality evaluation model is trained using multiple training samples, and each training sample includes a sample image and image quality evaluation information marked by the user on the sample image.
  • N and M are integers greater than zero, each of the M image enhancement models is different, and the N second images are also different.
  • Option 1 When the input image of the quality evaluation model is image B, the quality evaluation model is used to evaluate the image quality of image B. If the electronic device determines that the image quality of image B is the best according to the image quality evaluation information output by the quality evaluation model Good, then output image B, otherwise output image A;
  • the training samples of the quality evaluation model may include sample images output after the image enhancement model processes the original images, and image quality evaluation information marked by the user on the sample images.
  • the quality evaluation model can be used to assist in evaluating the image quality of image B through image A. If the electronic device outputs image quality evaluation information according to the quality evaluation model Determine the image quality of image B is the best, then output image B, otherwise output image A;
  • the training samples of the quality evaluation model may include the original image, the sample image output after the original image is processed by the image enhancement model, and the image quality evaluation information marked by the user on the sample image.
  • Solution 3 When the input images of the quality evaluation model are image A and image B, the quality evaluation model is used to evaluate the image quality of image A and image B.
  • the electronic device determines the image quality of image B according to the image quality evaluation information output by the quality evaluation model If the image quality is better than the image quality of image A, then output image B; if the electronic device determines that the image quality of image B is worse than the image quality of image A according to the image quality evaluation information output by the quality evaluation model, then output image A; if the electronic device According to the image quality evaluation information output by the quality evaluation model, it is determined that the image quality of the image B is the same as the image quality of the image A, and then any one of the image A and the image B is selected for output.
  • the training samples of the quality evaluation model can include the original image, the sample image output after the image enhancement model processes the original image, the image quality evaluation information marked by the user on the original image, and the image quality marked by the user on the sample image. Evaluation information.
  • the image quality evaluation information can be represented by identification information.
  • the identification information can be numbers, letters, words, etc., and the electronic device can determine the meaning of the numbers, words, or letters in the identification information according to a pre-established correspondence or rule , Thereby outputting image A or image B. For example, for scheme one and scheme two, "0" is used to indicate that the image quality of image B is poor, and "1" is used to indicate that the image quality of image B is good.
  • the electronic device determines that the image quality evaluation result corresponding to the image quality evaluation information is that image B has the best image quality and outputs image B; when the image quality evaluation information is 0, the electronic device determines the image quality evaluation The image quality evaluation result corresponding to the information is that image A has the best image quality, and image A is output.
  • image quality evaluation result corresponding to the information is that image A has the best image quality, and image A is output.
  • the electronic device determines that the image quality of image B is better than the image quality of image A according to the image quality evaluation information, and outputs image B; when the image quality evaluation information is 0, the electronic device determines the image quality according to the image quality evaluation information Determine that the image quality of image B is worse than that of image A, and output image A; when the image quality evaluation information is 2, the electronic device determines that the image quality of image B is the same as the image quality of image A according to the image quality evaluation information. Select either image from A and image B to output.
  • the image quality evaluation information can be represented by scores.
  • the image quality evaluation information is the score corresponding to image B, and the electronic device compares the score corresponding to image B with a predetermined score threshold.
  • the image quality evaluation information includes the score A of the image A and the score B of the image B. The electronic device compares the score A corresponding to the image B with the score B corresponding to the image B.
  • the electronic device When the score B is greater than the score A, it represents the image The image quality of B is better than that of image A, and the electronic device outputs image B; when score B is less than score A, it means that the image quality of image B is worse than that of image A, and the electronic device outputs image A, when score B is equal to When the score is A, it means that the image quality of the image B is the same as the image quality of the image A, and the electronic device selects any image from the image A and the image B to output.
  • the image quality of image B obtained after processing image A using the image enhancement model may be worse than that of image A.
  • image B includes a face image, and the eyes, Deformation of the nose, glasses frame, etc., purple fringing on the edge of image B, overlap, color aliasing, zipper effect and other defects in image B. Therefore, the image quality of image B is evaluated through the quality evaluation model, or the image is evaluated The image quality of image A and image B, and finally output the better image quality of image A and image B for the user to view according to the evaluation result.
  • the electronic device when the image quality of the processed image B deteriorates, the electronic device finally outputs the image A to the user to view, so as to solve the problem of outputting the image with the deteriorated image quality obtained after the image enhancement model processing to the user in the prior art.
  • the problem can improve the image quality of the output image, and can reduce the probability of outputting a poor quality image, so as to improve the user's visual experience.
  • FIG. 2 is a schematic flowchart of an image processing method provided by the first embodiment of the present application.
  • the main body of execution of the image processing method is an electronic device.
  • Electronic devices include but are not limited to mobile phones, notebook computers, tablet computers, wearable devices (including watches), personal digital assistants (PDAs), car machines, virtual reality (Virtual Reality, VR) devices, etc.
  • PDAs personal digital assistants
  • car machine refers to the abbreviation of in-vehicle infotainment products installed in the car.
  • the image input to the quality evaluation model is the second image obtained by using the image enhancement model to enhance the image quality of the first image.
  • the quality evaluation model is used to evaluate the image quality of the second image;
  • the image quality evaluation information of the image determines whether the image quality of the second image is good or bad; if the image quality of the second image is good, output the second image to the user for viewing; if the image quality of the second image is poor, output the first image to the user Check.
  • the image processing method includes the following steps:
  • the mobile phone may obtain the first image to be processed in response to the photographing instruction triggered by the user after starting the photographing application.
  • the mobile phone may also obtain the first image to be processed selected by the user when the image processing application is started.
  • the first image may be an image selected by the user from the gallery or album of the mobile phone.
  • the images in the gallery or album can be photos taken and saved by mobile phone users, or images taken by other users and shared with mobile phone users.
  • the mobile phone inputs the first image into the image enhancement model to perform image enhancement processing to enhance the image quality of the first image to obtain the second image.
  • Image enhancement refers to adding some information or features to the original image, selectively highlighting the features of interest in the image, and suppressing or concealing some unwanted features in the image. For example, improve the clarity of the original image, reduce image noise and false colors.
  • Pseudo-color refers to the color streaks and noise that appear in the dark part of the photo.
  • one second image can be obtained, and at least two second images can also be obtained, which is not limited here.
  • the image enhancement model is used to process the first image at least twice, at least two second images can be obtained. At least two second images may also be different.
  • Image enhancement processing includes but is not limited to super-resolution, denoising, demosaicing, and image restoration. Normally, image restoration and super-resolution, denoising, and demosaicing are independent. That is, in an example, after the mobile phone performs image restoration processing on the first image, S103 is performed; or the mobile phone may perform at least two of the following processing on the first image: super-resolution, denoising, and demosaicing, and then perform S103.
  • the mobile phone after the mobile phone performs image restoration processing on the first image, it can also perform at least any one of super-resolution, denoising, and demosaicing, and then perform S103; the mobile phone can also perform super-resolution first After at least any one of rate, denoising, and demosaicing, image restoration processing is performed on the first image, and then S103 is performed.
  • the processing scenarios of image restoration may include, but are not limited to: restoring old photos, removing fences or fences from photos taken across fences or fences, removing rain lines (raindrops) from rainy images, and removing fog from foggy images , Remove the glass etc. from the image taken through the glass.
  • the type of image processing performed on the first image may be determined according to the image characteristics of the first image, or may be determined according to an instruction triggered by a user, or an image processing function selected by the user.
  • the mobile phone can use the image enhancement model to super-resolution the first image when it detects that the resolution of the first image is less than or equal to the preset resolution threshold, or when it detects that the user triggers an instruction to indicate the super-resolution of the image.
  • the second image is processed and the resolution of the second image is greater than the resolution of the first image; when image noise is detected in the first image (for example, as shown in Figure 3), or the denoising instruction is detected, the image
  • the enhanced model performs denoising processing on the first image to obtain the second image.
  • the image noise of the second image is less than that of the first image; when it is detected that there is mosaic in the first image, or the denoising is detected
  • the image enhancement model is used to perform demosaic processing on the first image; when an image blur area is detected in the first image, or an image restoration instruction is detected, the image enhancement model is used to restore the first image
  • the first image may be an old photo, a rain line (raindrop) image, or a foggy image.
  • the first image may be blurred due to color fading, damage, raindrops, or fog.
  • the preset resolution threshold and gray value difference threshold can be set according to the actual situation, and there is no limitation here.
  • Super-resolution in this scheme refers to the improvement of the original image resolution through software.
  • Denoising refers to the process of reducing noise in digital images.
  • Demosaicing is a digital image processing algorithm. The purpose is to reconstruct a full-color image from incomplete color samples output by a photosensitive element covered with a color filter array (CFA). Pixel complete combination of red, green and blue (RGB) three primary colors. Demosaicing is also called color filter array interpolation (CFA interpolation) or color reconstruction (Color reconstruction).
  • CFA interpolation color filter array interpolation
  • Color reconstruction Color reconstruction
  • Demosaicing has the following characteristics: to avoid false color artifacts, such as aliases or zippering, that is, sudden and unnatural intensity changes in neighboring pixels, giving a feeling of zipper-like textures ) And purple fringe (Purple fringe) noise; try to preserve the image resolution; under the hardware limitation of the camera, realize fast and effective calculation processing with lower computational complexity; the algorithm is easy to analyze to reduce noise ( Noise reduction) is more accurate.
  • Image restoration refers to the use of prior knowledge of the degradation process to restore the original features of the degraded image, so as to improve the overall quality of the image.
  • Image blur is one of the manifestations of image degradation.
  • the image enhancement model may be an image processing model that has been trained in the prior art.
  • the image enhancement model may be an image processing model with a single image processing function, or an image processing model with at least two image processing functions.
  • a single image processing function refers to the realization of super-resolution, denoising, demosaicing or image restoration.
  • the image enhancement model may be a super-resolution model, a denoising model, a demosaicing model, or an image restoration model.
  • the image enhancement model may be a super-resolution model, a denoising model, a demosaicing model, or an image restoration model.
  • at least two image processing models can be used to implement it.
  • the denoising model can be used to process the first image to obtain the denoised image P1, and the image P1 is input to the super-resolution
  • the model is processed to obtain the image P2, and then the image P2 is input to the demosaicing model for processing to obtain the second image.
  • the same image enhancement model can implement at least two of super-resolution, denoising, demosaicing, and image restoration.
  • the image enhancement model can be composed of one image processing model, or can be formed by concatenating at least two sub-models with different image processing functions.
  • the selected image enhancement model can be a concatenation of the denoising model, the super-resolution model, and the demosaicing model.
  • the first image can be denoised, super-resolution and demosaiced in sequence.
  • Super-resolution models include but are not limited to: Super-Resolution Convolutional Neural Networks (SRCNN), Fast Super-Resolution Convolutional Neural Networks (FSRCNN), and effective sub-pixels Convolutional Neural Network (Efficient Sub-Pixel Convolutional Neural Network, ESPCN), Super-Resolution Generative Adversarial Network (SRGAN), Enhanced Super-Resolution Generative Adversarial Networks, ESRGAN) and so on.
  • SRCNN Super-Resolution Convolutional Neural Networks
  • FSRCNN Fast Super-Resolution Convolutional Neural Networks
  • ESPCN Effective Sub-Pixel Convolutional Neural Network
  • SRGAN Super-Resolution Generative Adversarial Network
  • ESRGAN Enhanced Super-Resolution Generative Adversarial Networks
  • Denoising models include, but are not limited to: denoising models based on deep neural networks.
  • Demosaicing models include but are not limited to: Alternating Direction Method of Multipliers (ADMM), demosaicing models based on deep neural networks, etc.
  • ADMM Alternating Direction Method of Multipliers
  • Image restoration models include but are not limited to: image restoration models based on deep neural networks, etc.
  • the first image is processed by the image enhancement model to obtain the second image
  • the resolution of the second image is higher, the image noise is reduced, or the mosaic in the second image is removed, and the second image is removed Raindrops, fog, etc., but there may be artifacts in the second image, for example, there is color aliasing in the second image (as shown in Figure 5). Therefore, in order to reduce the output of poor quality images for users to view, to improve The user's visual experience needs to evaluate the image quality of the second image, so as to output the first image or the second image for the user to view according to the evaluation result.
  • the mobile phone can input the second image into the quality evaluation model, extract the feature information corresponding to the second image for measuring image quality through the quality evaluation model, and process the extracted feature information to evaluate the image quality of the second image and obtain the quality Image quality evaluation information corresponding to the second image output by the evaluation model.
  • S103 may be: input the N second images into a quality evaluation model for processing to obtain image quality evaluation information corresponding to the second image.
  • the image quality evaluation information is image quality evaluation information for each of the N second images.
  • the image quality evaluation information may be an image quality score, and the image quality evaluation information may also be identification information used to indicate that the image quality of the second image is good or bad, and the identification information may be numbers, letters, words, and the like. For example, the identification information may be "0" or "1", “0” indicates that the image quality of the second image is poor, and "1" indicates that the image quality of the second image is good.
  • the identification information can also be “YES” or “NO”, “YES” indicates that the image quality of the second image is good, and “NO” indicates that the image quality of the second image is poor; the identification information can also be “YES” or “ No, “Yes” means that the image quality of the second image is good, and “No” means that the image quality of the second image is poor.
  • the image quality score can be any integer from 0 to 100, or any value between 0, 1, and 0-1.
  • the image quality score is 0-1 For any decimals in between, one decimal place or two decimal places can be kept, and there is no restriction here.
  • the image quality score is greater than or equal to the predetermined score threshold, it indicates that the image quality of the second image is good; when the image quality score is less than the predetermined score threshold, it indicates that the image quality of the second image is poor; when the image quality score is equal to the predetermined score threshold, Indicates that the image quality of the second image is the same as the image quality of the first image.
  • the predetermined score threshold such as 75 or 80, can be set according to specific conditions, and is not limited here.
  • the quality evaluation model may be obtained by training a deep learning network based on multiple training samples using a machine learning algorithm.
  • a training sample includes a sample image and the image quality evaluation information marked by the user.
  • the deep learning network can be a convolutional neural network or an adversarial network.
  • the network type and network structure of the deep learning network are not limited.
  • the input of the quality evaluation model is the sample image and the labeled image quality evaluation information corresponding to the sample image
  • the output of the quality evaluation model is the predicted image quality evaluation information corresponding to the sample image.
  • the quality evaluation model can also be constructed based on the image quality evaluation (Natural Image Quality Evaluator, NIQE) algorithm.
  • NIQE Natural Image Quality Evaluator
  • the NIQE algorithm obtains some statistical data (natural scene statistic, NSS) from natural images to characterize image quality.
  • the multiple sample images may be images obtained after multiple original images are input to the image enhancement model for processing, and the multiple sample images may also be composed of the original image and the image after the original image is degraded.
  • the image quality evaluation information corresponding to each of the multiple sample images may be partly the same or completely different.
  • the marked image quality evaluation information can be information used to indicate whether the image quality is good or bad, or it can be an image quality level.
  • the plurality of sample images may include sample images with good image quality and sample images with poor image quality.
  • the multiple sample images may also include multiple sample images belonging to different image quality levels.
  • the image quality evaluation factor corresponds to the image processing function of the image enhancement model
  • the characteristic information corresponding to the image quality evaluation factor can be extracted from the sample image, and based on the extracted
  • the feature information evaluates the image quality of the sample image, and outputs predicted image quality evaluation information corresponding to the sample image.
  • the evaluation factor of the image quality can be the sharpness of the image, and the sharpness evaluation function can be used to evaluate the image quality.
  • the gray level difference of the clear image is greater, so the evaluation factor of the sharpness evaluation function can be the gray value. The larger the gray value difference of adjacent pixels, the greater the sharpness of the image. The higher the image quality, the better.
  • the evaluation factor of image quality can be image noise.
  • the existing image noise estimation algorithm can be used to estimate the image noise in the sample image, and then estimate The smaller the value of image noise, the better the image quality.
  • the evaluation factor of image quality can be the color scale of each pixel point, and the color scale is an index standard for the brightness of the image.
  • the value range of the color scale of each color is [0,255]; when the adjacent pixels appear abrupt and unnatural intensity changes, it means that there is color aliasing or zipper effect in the image, so the adjacent pixels in the image The smaller the difference of the gradation values of the pixels, the better the image quality.
  • the evaluation factor of image quality can be image visibility or clarity; the greater the difference in gray values of adjacent pixels, the greater the visibility of the image The higher the high or sharpness, the better the image quality.
  • the greater the difference in gray values of adjacent pixels the higher the definition; for scenes where rain or fog in the image is removed, the greater the difference in gray values of adjacent pixels, The higher the visibility.
  • there is spectacle reflection or eye reflection in the image if the gray value difference of adjacent pixels is larger, it means that the reflection degree of spectacles or eyes is smaller, the image corresponding to spectacles or eyes is clearer, and the facial features are The smaller the image distortion, the better the image quality.
  • the image quality evaluation factor may be a combination of the evaluation factors corresponding to each of the at least two image processing functions.
  • the evaluation factor of the image quality may be image noise and image clarity.
  • the first image obtained by the mobile phone performing S101 may have any defects such as mosaic, image noise, low resolution, blurred image, etc., or at least two defects, therefore, when the mobile phone performs S102,
  • the image detection algorithm can be used to determine the defects in the first image.
  • the first image is processed by the corresponding image enhancement model to obtain the second image; in step S103, the quality evaluation model is used for the second image.
  • the image quality of the second image can be evaluated by the evaluation factor corresponding to the defect existing in the first image, and the image quality evaluation information of the second image can be obtained.
  • the image detection algorithm can be used to detect whether there is at least any one of image noise, mosaic, blurred area, poor image resolution, etc. in the first image type.
  • the image detection algorithm can separately detect whether there are image noise, mosaic, blurred areas or poor image resolution in the image, and it can also detect whether there are image noise, mosaic, blurred areas, and poor image resolution in the image at the same time. problem.
  • the mobile phone can use the mosaic detection algorithm to detect whether there is mosaic in the first image; when it detects the presence of mosaic in the first image, the image enhancement model is used to demosaicate the first image to obtain the first image.
  • Second image after that, input the second image into the quality evaluation model for processing, extract the level value of all colors of each pixel in the second image, and determine the adjacent two according to the level value of all colors of each pixel. And determine the image quality evaluation information of the second image according to the difference of the color gradation values of two pixels.
  • the image quality evaluation information is identification information used to indicate image quality
  • the quality The image quality evaluation information output by the evaluation model is "1". If the average difference between the color scale values of two adjacent pixels is greater than the predetermined color scale value difference threshold, the image quality evaluation information output by the quality evaluation model is "0".
  • the image quality evaluation information is used to represent the image quality score
  • at least two difference intervals can be set for the average difference of the color scale values of two adjacent pixels, and a corresponding score value can be set for each difference interval , So that the mobile phone can determine the image quality score corresponding to the second image according to the difference interval to which the average difference of the color scale values of the two adjacent pixel points corresponding to the second image belongs.
  • the mosaic detection algorithm may include, but is not limited to, the Canny edge detection algorithm, the mosaic detection algorithm based on template matching, and the like.
  • Canny edge detection is performed on the first image using the Canny edge detection algorithm to detect the edge in the first image, and obtain the gradient model or binary image corresponding to the first image.
  • the mosaic area usually presents a pile of square or similar squares.
  • the squares and similar squares can be roughly divided into complete squares and incomplete squares with missing sides. Therefore, the mobile phone can detect the first Whether there is a block-shaped or block-like area in the gradient pattern or binary image corresponding to the image, it can be determined whether there is mosaic in the first image.
  • the principle of the mosaic detection algorithm based on template matching is roughly as follows: use the Canny edge detection operator to perform edge detection on the first image to obtain the gradient pattern or binary image corresponding to the first image, and determine the first image based on the edge template of the mosaic image Whether there is mosaic in the corresponding gradient pattern or binary image, where the shape of any edge of the gradient pattern or binary image corresponding to the first image matches the edge template of the mosaic image, then there is mosaic.
  • the edge template of the mosaic image can be set based on the characteristic that the edge of the mosaic area usually presents a regular square after edge detection.
  • the mobile phone can use the image noise detection method to detect the presence of image noise in the first image, and use the mosaic detection algorithm to detect whether there is mosaic in the first image.
  • the first image is denoised and de-masked through the image enhancement model to obtain the second image, and the quality evaluation model is used to process the second image, and extract the second image corresponding to the measurement image
  • the quality feature information is to process the extracted feature information to evaluate the image quality of the second image, and obtain the image quality evaluation information corresponding to the second image output by the quality evaluation model.
  • the feature information corresponding to the second image for measuring image quality includes image noise feature information and the gradation value of all colors of each pixel.
  • the image quality evaluation factor of the second image includes the value of image noise and the difference between the color scale values of two adjacent pixels.
  • the value of image noise is determined according to the characteristic information of image noise.
  • the difference of the gradation values of the dots is determined according to the gradation values of all the colors of each pixel. The smaller the value of image noise and the smaller the difference between the gradation values of two adjacent pixels, the better the image quality.
  • the image quality evaluation information is identification information used to indicate image quality
  • the image quality evaluation information is identification information used to indicate image quality
  • the image quality evaluation information output by the quality evaluation model is "1"
  • the image quality evaluation information output by the quality evaluation model is "0"
  • each evaluation factor can be set to correspond to The weight value of, and set different intervals for the value corresponding to each average factor, and set the corresponding score for each interval. For example, set at least two difference intervals for the average difference of the color scale values of two adjacent pixels, and set a corresponding score for each difference interval; set at least two noise value intervals for the value of image noise , And set the corresponding score for each noise value interval.
  • the mobile phone can determine the score corresponding to each average factor according to the interval to which the value corresponding to each evaluation factor belongs, and determine the image quality score corresponding to the second image according to the score corresponding to each average factor and the weight value of each evaluation factor.
  • the sum of the weight values corresponding to all the evaluation factors is 1, and the weight value corresponding to each evaluation factor can be the same or different.
  • An evaluation factor with a large weight value indicates a greater impact on image quality.
  • Image noise detection methods may include Gaussian noise detection methods based on principal component analysis (PCA), noise detection methods based on signal dependent noise (signal dependent noise, SDN) models, and the like.
  • PCA principal component analysis
  • SDN signal dependent noise
  • the mobile phone uses the image enhancement model to perform super-resolution processing on the first image, and after obtaining the second image, the second image is input to the quality
  • the evaluation model is processed, and the sharpness characteristic information corresponding to the second image used to measure the image quality is extracted.
  • the sharpness characteristic information can be the gray value of the pixel; the mobile phone can calculate the adjacent two pixels according to the gray value of each pixel. The difference between the gray values of each pixel is determined based on the difference between the gray values of two adjacent pixels to determine the image quality evaluation information of the second image.
  • the average difference of the gray values of two adjacent pixels can be calculated to determine the image quality evaluation information of the second image, or the image quality of the second image can be determined by calculating the square of the gray values of two adjacent pixels. Evaluation information. Among them, the larger the average difference between the gray values of two adjacent pixels, or the larger the square of the gray values of two adjacent pixels, the better the image quality.
  • the mobile phone can detect the blurry area in the first image, use the image enhancement model to perform image restoration processing on the first image, and then input the second image into the quality evaluation model for processing, and extract the image of each pixel in the second image.
  • Gray value and calculate the gray value difference between two adjacent pixels according to the gray value of each pixel, and determine the second image according to the difference between the gray values of two adjacent pixels Image quality evaluation information. The larger the difference in gray value, the better the image quality.
  • S104 Determine to output the second image or the first image according to the image quality evaluation information.
  • the image quality evaluation information is used to indicate the image quality of the second image.
  • the mobile phone can judge whether the second image is the image with the best image quality among the first image and the second image according to the image quality evaluation information, and if the judgment result is that the second image is the image with the best image quality among the first image and the second image , Then the mobile phone outputs the second image. If the judgment result is that the second image is not the image with the best image quality among the first image and the second image, the mobile phone outputs the first image.
  • the image quality evaluation information may be identification information or an image quality score for representing image quality
  • the image quality information of the second image is a predetermined identification, or the image quality score of the second image is greater than a predetermined score threshold
  • the second image is the image with the best image quality among the first image and the second image.
  • the predetermined identifier can be "1", "YES” or "YES”.
  • the mobile phone when the image quality evaluation information is identification information used to indicate that the image quality of the second image is good or bad, and the identification information can be numbers, characters, or letters, the mobile phone can be based on the pre-established The corresponding relationship or rule determines the meaning of the number, text or letter corresponding to the identification information of the second image, and determines whether the second image is the image with the best image quality among the first image and the second image according to the specific meaning, Then, according to the judgment result, the image with the best image quality is output for the user to view.
  • the image quality evaluation information is identification information used to indicate that the image quality of the second image is good or bad
  • the identification information can be numbers, characters, or letters
  • the corresponding relationship or rule determines the meaning of the number, text or letter corresponding to the identification information of the second image, and determines whether the second image is the image with the best image quality among the first image and the second image according to the specific meaning, Then, according to the judgment result, the image with the best image quality is output for the user to view
  • the mobile phone when the image quality evaluation information of the second image is "1", "YES” or “Yes”, it means that the image quality of the second image is good, and the second image is the first image and the second image has the best image quality Image, then the mobile phone outputs the second image; when the image quality evaluation information of the second image is "0" or "NO” or “No”, it means that the image quality of the second image is poor, and the first image is the first image and For the image with the best image quality in the second image, the mobile phone outputs the first image.
  • N second images are obtained in S102, and the image quality evaluation information is the image quality evaluation information for each second image in the N second images.
  • the image quality evaluation information is used for
  • the mobile phone can determine whether there is a target second image whose image quality evaluation information is a predetermined identification in the N second images, and according to the judgment result from the first image and the second image
  • the predetermined flag can be "1" or "YES” or "Yes”.
  • the target second image whose image quality evaluation information is the predetermined mark is the image with the best image quality, and the image quality evaluation information is output as The second image of the target of the predetermined identification. If there are multiple target second images whose image quality evaluation information is a predetermined identifier in the N second images, then any target second image is selected for output. If there is no target second image whose image quality evaluation information is a predetermined identifier among the N second images, it is determined that the first image is an image with the best image quality, and the first image is output.
  • the mobile phone compares the image quality score corresponding to the second image with a predetermined score threshold, thereby judging whether the image quality of the second image is good or bad According to the judgment result, the image with the best image quality is determined from the first image and the second image. If the image quality score of the second image is greater than or equal to the predetermined score threshold, it means that the image with the best image quality among the first image and the second image is the second image, then the second image is output; if the image quality score is less than the predetermined score threshold , Indicates that the image with the best image quality among the first image and the second image is the first image, then the first image is output.
  • the predetermined score threshold can be 70, 75, or 80.
  • the predetermined score threshold can be 0.7 , 0.75 or 0.8, but it is not limited to this, and a predetermined score threshold can also be set according to actual needs.
  • N second images are obtained in S102, and the image quality evaluation information is the image quality evaluation information for each of the N second images.
  • the image quality evaluation information is the image quality
  • the mobile phone can determine whether there is a target second image with an image quality score greater than a predetermined score threshold among the N second images, and determine the image with the best image quality from the first image and the second image according to the judgment result.
  • the target second image is determined to be the image with the best image quality, and the target second image is output; if there are N second images If there is no target second image with an image quality score greater than or equal to the predetermined score threshold, it is determined that the first image is an image with the best image quality, and the first image is output. Wherein, if there are at least two target second images in the N second images, any target second image is selected for output, or the target second image with the highest score is output.
  • FIG. 6 is a schematic diagram of a user interface provided by an embodiment of the present application.
  • the user clicks on the icon of the camera application in the mobile phone, the mobile phone starts the camera application, the user can point the camera of the mobile phone at the location of the photographed object, and click the camera button to trigger the camera command.
  • the mobile phone obtains image A, in which the glasses reflect light.
  • the mobile phone can use the image enhancement model to process image A, for example, perform super-resolution and de-reflecting processing on image A.
  • the camera mode of the mobile phone may be a high-power zoom mode.
  • the image evaluation model is used to process image B1 to evaluate the image quality of image B1 to obtain image quality evaluation information of image B1, based on the image quality evaluation information of image B1 From image A and image B1, the image with the best image quality is output, and the image with the best image quality is output.
  • the image quality evaluation information is represented by the image quality score
  • the mobile phone can compare the image quality score of image B1 with a predetermined score threshold.
  • image B1 For the images with the best image quality in image A and image B1, the mobile phone outputs image B1, the user can see that the glasses in image B1 are not reflective, and the facial features or glasses of the portrait in image B1 are not deformed.
  • image quality evaluation information is represented by "0" or "1”
  • image B1 is the image with the best image quality among image A and image B1, and the mobile phone outputs image B1 .
  • the method for the mobile phone to output the image B1 may be to save the image B1 in the gallery, or to display the image B1 on the display interface.
  • the image enhancement model is used to process the image A to obtain the image B2
  • the image evaluation model is used to process the image B2 to evaluate the image quality of the image B2, and obtain the image quality evaluation information of the image B2, according to the image quality evaluation information of the image B2 From image A and image B2, the image with the best image quality is output, and the image with the best image quality is output.
  • the glasses in image B2 are not reflective, but the glasses are deformed, when the image quality evaluation information is represented by the image quality score, the mobile phone compares the image quality score of image B2 with the predetermined score threshold, and the comparison result is the image of image B2 The quality score is less than the predetermined score threshold.
  • image A is the image with the best image quality among image A and image B1, image B2 is discarded, and image A is output by the mobile phone.
  • image quality evaluation information is represented by "0" or "1”
  • the image quality evaluation information of image B2 is "0”
  • image A is the image with the best image quality among image A and image B1
  • the mobile phone outputs image A.
  • the way for the mobile phone to output image A may be to save image A in a gallery, or to display image A on the display interface.
  • the mobile phone processes the acquired first image, obtains the second image, and evaluates the image quality of the second image. If the image quality of the second image is poor, output the first image to the user; The image quality of the second image is good, and the second image is output to the user.
  • images with good image quality will not be output to users, which can reduce the probability of outputting images with poor quality and improve user experience.
  • FIG. 7 is a schematic flowchart of an image processing method provided by the second embodiment of the present application.
  • the quality evaluation model is used to evaluate the image quality of the second image.
  • the difference between FIG. 2 and FIG. 7 is that the image input to the quality evaluation model is different, and the method for the quality evaluation model to evaluate the image quality of the second image is different.
  • the input of the quality evaluation model is the second image, and the quality evaluation model is used to evaluate the image quality of the second image based on the image feature information of the second image; and in S203 of FIG.
  • the input of the quality evaluation model is The first image and the second image
  • the quality evaluation model is used to evaluate the image quality of the second image based on the feature information of the first image and the feature information of the second image. That is to say, Figure 7 uses the first image as a reference image, obtains the difference feature information between the feature information of the second image and the feature information of the first image, and evaluates the image quality of the second image according to the difference feature information. Improve the accuracy of the evaluation result of the second image.
  • S203 which is specifically as follows:
  • the mobile phone can use the quality evaluation model to extract the first feature information used to measure image quality from the first image, and extract the second feature information used to measure image quality from the second image.
  • the processing is performed to obtain the image quality evaluation information corresponding to the second image, so as to realize the auxiliary evaluation of the image quality of the second image through the first feature information.
  • Processing the first feature information and the second feature information may include: comparing the first feature information and the second feature information to obtain the difference feature information, and evaluate the image quality of the second image according to the difference feature information, so as to obtain the corresponding image of the second image.
  • the first feature information may be all the feature information of the first image, or part of the feature information of the first image. All feature information is the feature information of each pixel in the first image; partial feature information can be the feature information of some pixels in the first image, for example, pixels in the first image that can reflect or represent the image quality of the first image point.
  • the first image is super-resolution processed by the image enhancement model, and after the second image is obtained, the first image and the second image are input.
  • the quality evaluation model can extract the first sharpness feature information corresponding to the first image and the second sharpness feature information corresponding to the second image through the quality evaluation model. Through processing, the image quality evaluation information of the second image is obtained.
  • the image enhancement model is used to denoise the first image.
  • the second image is obtained, the first image and the second image are input to the quality evaluation model, which can be extracted through the quality evaluation model
  • the first image noise characteristic information corresponding to the first image and the second image noise characteristic information corresponding to the second image are processed, and the image quality evaluation of the second image is obtained by processing the first image noise characteristic information and the second image noise characteristic information information.
  • the image enhancement model is used to demosaicate the first image.
  • the first image and the second image are input into the quality evaluation model, and the first image can be extracted through the quality evaluation model.
  • the first color scale feature information corresponding to an image and the second color scale feature information corresponding to the second image are processed, and the image quality evaluation information of the second image is obtained by processing the first color scale feature information and the second color scale feature information .
  • the mobile phone can use the image enhancement model to perform image restoration processing after detecting the blurred area in the first image.
  • the first image and the second image can be input into the quality evaluation model, and the first image can be extracted through the quality evaluation model.
  • the first sharpness feature information corresponding to an image and the second sharpness feature information of the second image are processed to obtain the image quality evaluation information of the second image.
  • the image quality evaluation information may be an image quality score, and the image quality evaluation information may also be identification information used to indicate that the image quality of the second image is good or bad, for example, "0", or "1", "0". "Indicates that the image quality of the second image is poor, "1" indicates that the image quality of the second image is good; the identification information can also be “Yes” or “No”, “Yes” indicates that the image quality of the second image is good, “No” Indicates that the image quality of the second image is poor.
  • the identification information may also be “YES” or “NO”, “YES” indicates that the image quality of the second image is good, and "NO” indicates that the image quality of the second image is poor.
  • the image quality evaluation information is an image quality score
  • the image quality score can be any integer from 0 to 100, or can be any value between 0, 1, and 0-1.
  • the image quality score is greater than or equal to the predetermined score threshold, it indicates that the image quality is good; when the image quality score is greater than or equal to the predetermined score threshold and less than the predetermined score threshold, it indicates that the image quality is poor.
  • the input of the quality evaluation model in S203 is the original image, the sample image corresponding to the original image, the image quality evaluation information of the mark corresponding to the sample image, and the output of the quality evaluation model is the predicted image quality evaluation information corresponding to the sample image.
  • the sample image corresponding to the original image may be an image output after the original image is processed by the image enhancement model, or may be an image obtained after degradation processing is performed on the original image, and there is no limitation here.
  • S203 is specifically: input the first image and the N second images into the quality evaluation model to obtain the image corresponding to the second image Quality evaluation information.
  • the image quality evaluation information is image quality evaluation information for each of the N second images.
  • the feature information of the first image can be used to assist in evaluating the image quality of the second image. Since the mobile phone can use the first image as a reference image, the difference between the feature information of the second image and the feature information of the first image can be obtained. The feature information evaluates the image quality of the second image based on the difference feature information, which can improve the accuracy of the image quality evaluation information of the second image.
  • FIGS. 2 and 7 the scheme of evaluating the image quality of the second image through the quality evaluation model is introduced; the scheme of evaluating the image quality of the first image and the second image through the quality evaluation model is introduced below.
  • FIG. 8 is a schematic flowchart of an image processing method provided by the third embodiment of the present application.
  • the inputs of the quality evaluation model are both the first image and the second image.
  • the difference between FIG. 8 and FIG. 7 is that the quality evaluation model in FIG. 7 is used to evaluate the image quality of the second image, while the quality evaluation model in FIG. 8 is used to evaluate the image quality of the first image and the second image.
  • S303 to S306 which are specifically as follows:
  • the quality evaluation model can be used to extract the first feature information corresponding to the first image and the second feature information corresponding to the second image, and process the first feature information and the second feature information to obtain the first image and Image quality evaluation information of the second image.
  • Processing the first feature information and the second feature information may include: comparing the first feature information and the second feature information to obtain the difference feature information, and evaluate the image quality of the second image according to the difference feature information, so as to obtain the corresponding image of the second image.
  • Image quality evaluation information may be all the feature information of the first image, or part of the feature information of the first image. All feature information is the feature information of each pixel in the first image; partial feature information can be the feature information of some pixels in the first image, for example, pixels in the first image that can reflect or represent the image quality of the first image point.
  • the image quality evaluation information may be identification information used to indicate whether the image quality of the second image is better than the image quality of the first image, and the identification information may be numbers, letters, characters, or the like.
  • the image quality evaluation information may also include: first image quality evaluation information corresponding to the first image, and second image quality evaluation information corresponding to the second image.
  • the image quality evaluation information may include a first identifier used to indicate whether the image quality of the first image is good or bad, and a second identifier used to indicate whether the image quality of the second image is good or bad; it may also include the first image. The corresponding first image quality score and the second image quality score corresponding to the second image.
  • the image quality evaluation information when the image quality evaluation information is identification information used to indicate whether the image quality of the second image is better than the image quality of the first image, the image quality evaluation information may be the difference obtained by comparing the first feature information and the second feature information.
  • the feature information is determined according to the difference feature information; the first feature information can be all feature information of the first image, or part of the feature information of the first image. You can use "0", “NO” or "No” to indicate that the image quality of the second image is worse than that of the first image.
  • the image quality of the first image is better than that of the first image, and other numbers (for example, the number 2) indicate that the image quality of the second image is the same as the image quality of the first image.
  • the first image quality evaluation information may be determined by the first image quality evaluation information of the first image.
  • Information determination; the second image quality evaluation information can be determined by the second feature information of the second image, or the difference feature information can be obtained by comparing the first feature information and the second feature information, which is determined according to the difference feature information.
  • the first image quality evaluation information and the second image quality evaluation information may be image quality scores.
  • the image quality score can be any integer from 0-100, or any value from 0-1.
  • the input of the quality evaluation model in S303 is: the original image, the image quality evaluation information of the mark corresponding to the original image, the sample image corresponding to the original image, and the image quality evaluation information of the mark corresponding to the sample image.
  • the output of the quality evaluation model is The predicted first image quality evaluation information corresponding to the original image, and the predicted second image quality evaluation information corresponding to the sample image.
  • the sample image corresponding to the original image may be an image output after the original image is processed by the image enhancement model, or may be an image obtained after degradation processing is performed on the original image, and there is no limitation here.
  • S303 is specifically: inputting the first image and the N second images into the quality evaluation model to obtain the image corresponding to the second image Quality evaluation information.
  • the image quality evaluation information is image quality evaluation information of each second image in the first image and the N second images.
  • S304 Determine to output the second image or the first image according to the image quality evaluation information.
  • the mobile phone can judge whether the second image is the image with the best image quality among the first image and the second image according to the image quality evaluation information, and if the judgment result is that the second image is the image with the best image quality among the first image and the second image , Then output the second image. If the judgment result is that the second image is not the image with the best image quality among the first image and the second image, then the first image is output.
  • the image quality evaluation information can be the identification information or the image quality score used to indicate the image quality
  • the image with the best image quality can be the image with the predetermined identification as the image quality evaluation information, or the image with the best image quality can also be An image with an image quality score greater than or equal to a predetermined score threshold, or an image with the best image quality may also be the image with the highest image quality score in the first image and the second image.
  • the mobile phone can determine the position of the numbers, characters or letters in the identification information according to the pre-established correspondence or rules.
  • the meaning of the expression is to determine whether the image quality of the second image is better than the image quality of the first image, so as to determine the image with the best image quality in the first image and the second image to output the image with the best image quality.
  • the image is good, then output the second image; when the image quality evaluation information is "0", “NO” or "No", it means that the image quality of the second image is worse than that of the first image, and the first image is the first image.
  • One image and the second image with the best image quality, then the first image is output; when the image quality evaluation information is 2, it means that the image quality of the second image is the same as the image quality of the first image. Select any one of the second images to output.
  • the image quality evaluation information is the image quality evaluation information for each of the first image and the N second images, then when N ⁇ 1, the image
  • the image quality evaluation information may include a first identification for indicating the image quality of the first image and a second identification for respectively indicating the image quality of the second image, and the quantity of the second identification is N number.
  • the mobile phone can determine whether there is an image whose image quality evaluation information is a predetermined identifier in the first image and the N second images, and determine the image with the best image quality from the first image and the second image according to the judgment result.
  • the predetermined identifier can be "1" or "YES” or "YES".
  • the image quality evaluation information of the first image is a predetermined identifier, and there is at least one target second image whose image quality evaluation information is the predetermined identifier in the N second images, select from the first image and the target second image Any image output. If the image quality evaluation information of the first image is not the predetermined identifier, and there is at least one target second image whose image quality evaluation information is the predetermined identifier in the N second images, then any one of the target second images is selected for output. If the image quality evaluation information of the first image is not the predetermined identifier, and there is no target second image whose image quality evaluation information is the predetermined identifier in the N second images, the first image is output.
  • the image quality evaluation information includes: the first image quality score corresponding to the first image and the second image quality score corresponding to each of the N second images.
  • the mobile phone can follow the first image The first image quality score of the first image and the second image quality score of the N second images. From the first image and the N second images, determine that the image with the highest image quality score is the first image and the image of the N second images The best quality image to output the image with the highest score.
  • the mobile phone can also determine whether there is an image with an image quality score greater than a predetermined score threshold in the first image and the N second images, and determine the image with the best image quality from the first image and the second image according to the judgment result.
  • the image with the image quality score greater than or equal to the predetermined score threshold is output.
  • the number of images with an image quality score greater than or equal to the predetermined score threshold is at least two, any one of them can be selected for output.
  • the mobile phone can compare the first image quality score corresponding to the first image with the second image quality score corresponding to the N second images, and filter out the image with the highest image quality score, so as to obtain the image with the best image quality.
  • the image quality score of any second image is the highest
  • the second image is output
  • the image quality score of the first image is the highest
  • the first image is output.
  • any one of the second images is selected for output. It should be noted that, when the image quality scores of the first image and any of the second images are tied for first, any one of the first image and the second image whose image quality scores are tied for the first can be selected for output.
  • the mobile phone can compare the first image quality score corresponding to the first image and the second image quality score corresponding to the second image.
  • the image quality score is compared with the threshold value of the score interval to determine the score interval to which the first image quality score belongs and the score interval to which the second image quality score belongs. If the first image quality score and the second image quality score belong to the same score interval, it means that the image quality of the second image is the same as the image quality of the first image, and the mobile phone can select any image from the first image and the second image to output .
  • the first image quality score and the second image quality score are compared to determine which image has the highest image quality in the first image or the second image. good. Among them, the image quality of the image belonging to the high score interval is the best. Specifically, if it is determined according to the comparison result that the first image quality score and the second image quality score belong to different score intervals, and the second image quality score is greater than the first image quality score, it means that the image quality of the second image is better than that of the first image. If the first image quality score and the second image quality score belong to different score intervals according to the comparison result, and the second image quality score is less than the first image quality score, then the second image is output. The image quality of the image is worse than the image quality of the first image, and the first image is output.
  • the first image belongs to the first score interval, and the N second images belong to the second score interval, if the first image quality score of the first image is less than the second image
  • the second image quality score of the mobile phone can determine that the N second images belonging to the second interval are the images with the best image quality among the first image and the N second images, and output any second image, or output image quality The second image with the highest score. If the first image quality score of the first image is greater than the second image quality score of the second image, then the first image is an image with the best image quality, and the first image is output.
  • the mobile phone can set the threshold of the first score interval and the threshold of the second score interval. Thresholds are compared. If the maximum threshold of the first score interval is less than or equal to the minimum threshold of the second score interval, then the second image belonging to the second score interval is the image with the best image quality, and any image belonging to the second score interval can be output. A second image; if the minimum threshold of the first score interval is greater than the maximum threshold of the second score interval, then the first image and the second image belonging to the first score interval are the images with the best image quality, and the first score can be output Any second image or first image in the interval.
  • the second image obtained is the image B1 in FIG. 6.
  • the image A in FIG. 6 is processed in S302, the second image obtained is the image B1 in FIG. 6.
  • the mobile phone inputs the image A and the image B1 in FIG. 6 into the image quality model for processing, and obtains the image quality score of the image A and the image quality score of the image B1.
  • the mobile phone compares the image quality score of image B with the image quality score of image A, and the comparison result is that the image quality score of image B is greater than the image quality score of image A. Therefore, the mobile phone outputs image B1. At this time, the user can see that there is no reflection from the glasses in the image, and the image has a high definition.
  • the second image obtained is image B2 in Fig. 6.
  • image A in Fig. 6 there is reflection of glasses in image A, and part of the image in the area where the glasses is located is relatively blurred; There is no reflection in the glasses, and the sharpness of the image is high, but the glasses in image B2 are deformed.
  • the mobile phone inputs image A and image B2 in Figure 6 into the image quality model for processing, and obtains the image quality score of image A and the image quality score of image B2.
  • the mobile phone compares the image quality score of image B with the image quality score of image A, and obtains The result of the comparison is that the image quality score of image A is greater than the image quality score of image B. Therefore, the image quality of image A is higher than the image quality of image B2, and the mobile phone outputs image A.
  • the user sees the reflection of the glasses in the image, and part of the image in the area where the glasses is located is relatively blurry.
  • the image quality of the first image and the second image can be evaluated, and image quality evaluation information for the first image and the second image can be obtained.
  • the image quality evaluation information can be used to determine whether the image quality of the second image is better than that of the first image.
  • the image quality of an image is good, and it is possible to know more accurately whether the image quality of the second image obtained after the first image is processed by the image enhancement model is deteriorated.
  • FIG. 9 is a schematic flowchart of an image processing method according to a fourth embodiment of the present application. It includes the following steps:
  • the mobile phone acquires the first image, which can be acquired in portrait mode, scenery mode, indoor mode, telephoto mode (hereinafter referred to as high-power zoom mode), and so on.
  • high-power zoom mode For ease of description, the following takes a picture in a high-power zoom mode as an example for description.
  • the user starts the camera application in the mobile phone, and can control the mobile phone to enter the high-power zoom mode or telephoto shooting state to take portraits.
  • High-power zoom modes include, but are not limited to, 3x zoom, 5x zoom, 10x zoom, 30x zoom, or 50x zoom.
  • the zoom factor can be specifically determined according to the zoom capability of the high-power zoom camera.
  • the first image may be a preview image acquired by the mobile phone in a high-power zoom mode, or may be a photo acquired by the mobile phone in response to the user's photo quality in the high-power zoom mode.
  • the first image obtained may be distorted due to jitter, the facial features of the portrait, such as eyes, are deformed, or the image is blurred (for example, it is difficult to recognize the subject) Etc. Therefore, it is necessary to evaluate the image quality of the first image.
  • S402. Input the first image into a quality evaluation model for processing to obtain an image quality score of the first image.
  • the mobile phone inputs the first image into the quality evaluation model for processing, extracts feature information used to measure image quality from the first image, and processes the feature information of the first image to obtain an image quality score corresponding to the first image.
  • the image quality score is used to represent the image quality of the first image.
  • S403 Determine whether the image quality score of the first image is greater than or equal to a preset threshold.
  • the preset threshold value can be 0.25, but it is not limited to this, and other values can also be set according to the actual situation, and there is no limitation here.
  • the mobile phone compares the image quality score of the first image with a preset threshold, so as to determine whether the first image meets the requirements according to the comparison result.
  • the image quality score is greater than or equal to the preset threshold, it indicates that the image quality of the first image is good, and the first image meets the requirements, and the first image can be processed continuously, and S404 is executed.
  • the image quality score is less than the preset threshold, it means that the image quality of the first image is poor, and the first image does not meet the requirements. In most cases, even if the first image is processed, the image quality cannot be improved. In order to save mobile phone resources, Go to S405.
  • the mobile phone inputs the first image into the image enhancement model for processing to obtain the second image, and displays or saves the second image.
  • the mobile phone may use a face recognition algorithm to detect whether the first image includes a face image, and if the first image includes a face image, the mobile phone inputs the first image
  • the image enhancement model performs super-resolution processing on the face image through the image enhancement model, enhances the clarity of the face image in the first image, obtains a second image, and displays or saves the second image.
  • the image enhancement model can also be used to perform super-resolution processing to enhance the clarity of images other than human faces to obtain a second image, and then display or save the second image for users to view;
  • the image may be an image of body parts other than the face in the portrait, for example, a person's limbs, hair, hair accessories, clothing, etc., and may also be an image of other sceneries, buildings, etc. except the portrait in the first image.
  • the mobile phone can perform super-resolution processing on the first image through the image enhancement model to enhance the overall clarity of the first image to obtain the second image, and then display or save the second image For users to view.
  • the mobile phone can use the image enhancement model to perform super-resolution processing on the first image to obtain the second image when it detects that the resolution of the first image is less than or equal to the preset resolution threshold.
  • the resolution of the image is greater than the resolution of the first image; when image noise is detected in the first image, the image enhancement model is used to denoise the first image to obtain the second image; when it is detected that there is an image in the first image
  • the image enhancement model is used to restore the first image.
  • the first image can be a remake of an old photo, an image with rain lines (raindrops), or a foggy image.
  • the first image may fade due to color,
  • the image is blurred due to damage, rain or fog.
  • the first image may also be input into M image enhancement models for processing to obtain N Second image, and display or save N second images.
  • N and M are positive integers, each of the M image enhancement models is different, and the N second images are also different.
  • the same image enhancement model is used to process the first image, one second image can be obtained, and at least two second images can also be obtained, which is not limited here.
  • the image enhancement model is used to process the first image at least twice, at least two second images can be obtained.
  • the mobile phone can save N second images, or display N second images, so that the user can select the images that need to be saved. There is no restriction on the way of displaying the N second images.
  • the mobile phone starts the camera application, if it detects that the high zoom mode is entered, the first preview image can be collected; the image quality of the first preview image is evaluated through the quality evaluation model, and the image quality score of the first preview image is obtained; The image quality score of the preview image is compared with the preset threshold. If the image quality score of the first preview image is greater than or equal to the preset threshold, it indicates that the image quality of the first preview image is better.
  • the mobile phone detects whether a face image is included in the first photo. If the mobile phone detects a face image in the first preview image, super-resolution processing is performed on the face image in the first preview image through the image enhancement model to enhance the first preview image.
  • the definition of the face image in the preview image is obtained, and the second preview image is obtained, and the second preview image is displayed on the preview interface. If the image quality score of the first preview image is less than the preset threshold, it means that the image of the first preview image is poor, and the first preview image is no longer processed, and the first preview image is displayed on the preview interface to reduce the consumption of processing images Resources to improve the processing speed of mobile phones.
  • the mobile phone After the mobile phone starts the camera application and enters the high zoom mode, if a user-triggered camera command is detected, the first photo is obtained in response to the camera command, and the image quality of the first photo is evaluated through the quality evaluation model to obtain the first photo The image quality score of the first photo; the mobile phone compares the image quality score of the first photo with a preset threshold. If the image quality score of the first photo is greater than or equal to the preset threshold, it means that the image quality of the first photo is better.
  • the mobile phone detects whether a face image is included in the first photo. If a face image is detected in the first photo, super-resolution processing is performed on the face image in the first photo through the image enhancement model to enhance the first photo.
  • the sharpness of the face image is obtained, and the second photo is obtained, and the second photo is saved to the gallery for the user to view. If the image quality score of the first photo is less than the preset threshold, it means that the image of the first photo is poor, and the first photo is no longer processed, and the first photo is saved to the library for users to view, so as to reduce the consumption of image processing Resources to improve the data processing speed of mobile phones.
  • the image quality score represents the image quality of the first image as an example for description.
  • “0” or “1” can also be used as an example. Indicates the image quality of the first image. Among them, “0” may be used to indicate that the image quality of the first image is poor, and the first image does not meet the requirements; “1” may be used to indicate that the image quality of the first image is good, and the first image meets the requirements.
  • the mobile phone When the image quality of the first image is evaluated by the quality evaluation model, if the mobile phone detects that the quality evaluation model outputs "1", the first image is input to the image enhancement model for processing to obtain the second image and output the second image; if the mobile phone When it is detected that the quality evaluation model outputs "0", the first image is output.
  • the image quality of the first image is evaluated first. If the image quality of the first image is poor, the first image is displayed to the user for viewing, or the first image is saved to the gallery; If the image quality of the first image is good, the first image is processed to obtain the second image, and the second image is displayed to the user for viewing, or the second image is saved to the gallery.
  • the data processing speed of mobile phones can be increased by saving resources consumed by processing images with poor image quality.
  • FIG. 10 is a schematic flowchart of an image processing method according to a fifth embodiment of the present application.
  • the quality evaluation model is also used to evaluate the image quality of the second image. If the image quality score of the second image is greater than or equal to the predetermined score threshold, the second image is output. Image; if the image quality score of the second image is less than the predetermined score threshold, output the first image.
  • S406 to S407 which are specifically as follows:
  • the method for evaluating the image quality of the second image in S406 is the same as the method for evaluating the image quality of the first image in S402, and will not be repeated here.
  • N second images are acquired in S404 and N ⁇ 2, then in S406, the number of second images input to the quality evaluation model is N.
  • S407 Determine whether the image quality score of the second image is greater than or equal to a predetermined score threshold.
  • the predetermined score threshold can be 75, but it is not limited to this. It can also be 70, 80, 85 or other values. It can be specifically based on the actual application process, the image Set the corresponding score when the quality is better, and there is no restriction here.
  • the predetermined score threshold may be 0.75.
  • the input image of the quality evaluation model is the second image
  • the output of the quality evaluation model is the image quality score corresponding to the second image as an example to illustrate how to output the first image and the second image The way to achieve the best image quality in the medium.
  • the output of the quality evaluation model may also be identification information used to indicate the image quality of the second image
  • the mobile phone may correspond to the second image according to the The identification information of displays or saves the image with the best image quality in the first image and the second image, and the identification information can be numbers, letters, words, etc.
  • the input image of the quality evaluation model can be the first image and N second images
  • the output of the quality evaluation model is the image quality score corresponding to each of the N second images or is used to represent the image Quality identification information, N is a positive integer
  • the mobile phone can output the first image and the N second image with the best image quality according to the image quality score corresponding to each of the N second images or the identification information used to indicate the image quality image.
  • the output of the quality evaluation model may also be the first image and N second images, each corresponding to the image Quality score or identification information used to indicate image quality.
  • the mobile phone can output the first image and N second images according to the respective image quality scores or identification information used to indicate image quality.
  • the image with the best image quality among the images For the specific implementation method, refer to the related descriptions of S303 to S304 in the embodiment corresponding to FIG. 8, which will not be repeated here.
  • the image with the best image quality is an image whose image quality evaluation information is a predetermined identifier among the first image and the N second images, or an image whose score corresponding to the image quality evaluation information is greater than a predetermined score threshold.
  • the predetermined identifier can be "1", "Y” or "Yes".
  • the quality evaluation model is used to evaluate the image quality of the first image; when the image quality of the first image is poor, the first image is output to the user for viewing; When the image quality of an image is good, the first image is input to the image enhancement model for processing to obtain the second image, and the quality evaluation model is used to evaluate the image quality of the second image. If the image quality of the second image is good, the second image is output to The user views, if the image quality of the second image is poor, the first image is output to the user for viewing. In this way, the probability of outputting poor quality images can be reduced, and the user experience can be improved.
  • this application lists a flow of training quality evaluation models, see FIG. 11, which is a schematic flowchart of a training quality evaluation model provided by an embodiment of the application.
  • the quality evaluation model can be trained through the following steps:
  • the method further includes: obtaining a test sample set, the test sample set includes a plurality of test samples, and each test sample has a marked Image quality evaluation information.
  • the image quality evaluation information of the labels corresponding to multiple training samples are not completely the same, and the image quality evaluation information of the labels corresponding to multiple test samples are not completely the same. That is, the training sample set includes training samples with different image quality, and the test sample set includes test samples with different image quality.
  • a training sample includes a training sample image and labeled image quality evaluation information corresponding to the training sample image;
  • a test sample includes a test sample image and image quality evaluation information of the mark corresponding to the test sample image.
  • the image quality of the training sample images needs to be evaluated; during the testing process, the image quality of the test sample images needs to be evaluated.
  • the quality evaluation model in FIG. 2 uses the trained quality evaluation model to evaluate the image quality
  • the image input to the quality evaluation model is the second image obtained after the first image is processed by the input image enhancement model.
  • the quality evaluation model in FIG. 9 and FIG. 10 uses the trained quality evaluation model to evaluate image quality
  • the image input to the quality evaluation model is an image that has not been processed by the image enhancement model.
  • a training sample when training the quality evaluation model shown in Figure 7, includes an original training image, at least one training sample image obtained after processing the original training image, and the training sample image Marked image quality evaluation information.
  • a test sample includes an original test image and at least one test sample image obtained after processing the original test image, and the test sample image has marked image quality evaluation information.
  • the original training image is used to assist in evaluating the image quality of the training sample image.
  • the original test image is used to assist in evaluating the image quality of the test sample image.
  • the image input to the quality evaluation model includes the first image of the input image enhancement model and the second image output by the image enhancement model, and the quality evaluation model outputs the image quality evaluation corresponding to the second image Information, the first image is used to assist in evaluating the image quality of the second image.
  • the training sample image can be obtained after processing the original training image using the image enhancement model
  • the test sample image can be obtained after processing the original test image using the image enhancement model
  • the training sample image can also be obtained by degrading the original training image, and the test sample image can also be a photographed picture or a picture downloaded from the Internet.
  • a SLR camera is used to shoot multiple original training images with good image quality, and the original training images are blurred, noise-added or mosaic-processed to obtain training sample images.
  • Different training sample images can be made from the same original training image. It can also be obtained from different original training images, and there is no limitation here.
  • the picture taken by the mobile phone is used as the original test sample, and the original test sample is degraded to obtain the test sample image.
  • the original training images in the training samples may also have marked image quality evaluation information
  • the original test images in the test samples may also have marked image quality evaluation information. Marked image quality evaluation information.
  • the image input to the quality evaluation model includes the first image of the input image enhancement model and the second image output by the image enhancement model.
  • the quality evaluation model outputs the first image and the second image. Corresponding image quality evaluation information.
  • the image quality evaluation information of the labels corresponding to multiple training sample images are not completely the same, that is, the training sample set includes training sample images with different image quality.
  • the image quality evaluation information of the marks corresponding to the multiple test sample images are not completely the same.
  • the training sample image is different from the test sample image.
  • S002 Input the training samples into the initial quality evaluation model for processing, and obtain predicted image quality evaluation information corresponding to each training sample.
  • the training samples are input into the initial quality evaluation model, and the initial quality evaluation model is used to extract feature information used to measure image quality from the training samples, and the extracted feature information is analyzed to obtain the predicted image quality evaluation information corresponding to the training samples.
  • a training sample when a training sample includes a training sample image and labeled image quality evaluation information corresponding to the training sample image, the feature information used to measure the image quality is extracted from the training sample image, and the training sample The corresponding predicted image quality evaluation information is the predicted image quality evaluation information corresponding to the training sample image.
  • the initial The quality evaluation model processes the original training images and training sample images of the training samples, and determines the predicted image quality evaluation information corresponding to the training sample images.
  • the initial quality evaluation model can be used to extract the first feature information used to measure image quality from the original training image, and the second feature information used to measure image quality can be extracted from the training sample image, and the corresponding training sample is analyzed. The first feature information and the second feature information of, determine the predicted image quality evaluation information corresponding to the training sample image in the training sample.
  • this method can assist in evaluating the image quality of the training sample image through the feature information of the original training image, it can more accurately evaluate whether the training sample image is better or worse than the original training image, which can improve training.
  • the accuracy of the predicted image quality evaluation information of the sample image can improve the accuracy of the predicted image quality evaluation information of the sample image.
  • the original quality evaluation model can be used to process the original training image and the sample training image of the training sample respectively. Determine the predicted first image quality evaluation information corresponding to the original training image, and determine the predicted second image quality evaluation information corresponding to the sample training image.
  • the initial quality evaluation model can be used to extract the first feature information corresponding to the original training image of the same training sample, and the second feature information corresponding to the training sample image, and analyze the first feature information to obtain the prediction corresponding to the original training image.
  • the second feature information is analyzed to obtain the predicted second image quality evaluation information corresponding to the sample training image.
  • S003 Determine the first evaluation accuracy of the initial quality evaluation model according to the marked image quality evaluation information and predicted image quality evaluation information corresponding to the training sample.
  • the labeled image quality evaluation information and predicted image quality evaluation information corresponding to the training samples can be compared, and the first evaluation accuracy rate of the initial quality evaluation model can be determined according to the comparison results corresponding to each training sample.
  • the marked image quality evaluation information and the predicted image quality evaluation information are the same or matching target training samples are screened out, and the number of target training samples is calculated as compared with the total number of training samples participating in the training. Ratio, get the first evaluation accuracy of the initial quality evaluation model.
  • the first evaluation accuracy L/K, L ⁇ K. L represents: in this training, the number of marked image quality evaluation information and predicted image quality evaluation information are the same or matching target training samples; K represents the total number of training samples participating in this training.
  • a training sample when a training sample includes a training sample image and labeled image quality evaluation information corresponding to the training sample image, after determining the predicted image quality evaluation information corresponding to the training sample image according to the training sample image , The image quality evaluation information of the mark corresponding to the training sample image and the predicted image quality evaluation information can be compared to obtain the comparison result corresponding to the training sample.
  • a training sample when a training sample includes an original training image, at least one training sample image obtained after processing the original training image, and the training sample image has marked image quality evaluation information, the After the image and the training sample image determine the predicted image quality evaluation information corresponding to the training sample image, the labeled image quality evaluation information corresponding to the training sample image and the predicted image quality evaluation information can be compared to obtain the comparison result corresponding to the training sample.
  • the first image quality evaluation information predicted corresponding to the original training image of the training sample is determined, and the training is determined After the predicted second image quality evaluation information corresponding to the sample training image of the sample, compare the labeled image quality evaluation information corresponding to the original training image and the predicted first image quality evaluation information to obtain the first comparison result, and compare the training sample images
  • the corresponding labeled image quality evaluation information and the predicted second image quality evaluation information obtain a second comparison result; the comparison result corresponding to the training sample is obtained according to the first comparison result and the second comparison result of the same training sample.
  • the comparison result corresponding to the training sample is the same or matching.
  • the comparison result corresponding to the training sample is not the same or does not match.
  • a preset loss function can also be used to calculate the loss value between the marked image quality evaluation information corresponding to the training sample and the predicted image quality evaluation information, and the evaluation accuracy of the initial quality evaluation model can be expressed by the loss value. .
  • the smaller the loss value the higher the evaluation accuracy of the initial quality evaluation model. It is understandable that when the first loss value corresponding to the original training image of the same training sample and the second loss value corresponding to the sample training image are calculated, the larger loss value is used as the loss value of the training sample.
  • the loss function includes, but is not limited to, the cross-entropy loss function.
  • S004 Determine whether the first evaluation accuracy is greater than or equal to a first accuracy threshold.
  • the first accuracy threshold is used to measure whether the evaluation accuracy of the initial quality evaluation model meets the requirements. When the first evaluation accuracy is less than the first accuracy threshold, the evaluation accuracy of the initial quality evaluation model does not meet the requirements, and S005 is executed. When the first evaluation accuracy is greater than or equal to the first accuracy threshold, the evaluation accuracy of the initial quality evaluation model has met the requirements, and S006 can be executed, or S008 can be jumped to to end the training.
  • the first accuracy threshold can be 85%, 90%, 95%, etc., but is not limited to this, and can be set according to actual requirements, and there is no limitation here.
  • the loss value threshold corresponding to the first accuracy threshold can be set.
  • the initial quality evaluation model If the evaluation accuracy does not meet the requirements, perform S005; when the loss value is less than or equal to the loss threshold, the evaluation accuracy of the initial quality evaluation model meets the requirements, you can perform S006 or jump to S008 to end the training.
  • Methods of adjusting parameters include but are not limited to stochastic gradient descent algorithm, power update algorithm, etc.
  • the training samples used when S002 is executed for the Nth time may be the same or different from the training samples used for S002 for the N+1th time. For example, when S002 is executed for the first time, training samples 1 to 500 are used for training, and when S002 is executed for the second time, samples 501 to 1000 are used for training.
  • a test set may be used to test the trained quality evaluation model.
  • S006 to S008 may also be executed.
  • S006 to S007 are optional steps, that is to say, in a possible implementation, the process of training the quality evaluation model may include S001 to S005, S008; in another possible implementation, The process of training the quality evaluation model may include S001 to S008.
  • S006 ⁇ S008 are as follows:
  • the marked image quality evaluation information and predicted image quality evaluation information corresponding to the test samples can be compared, and the second evaluation accuracy of the initial quality evaluation model can be determined according to the comparison results corresponding to each test sample.
  • the method of calculating the accuracy of the second evaluation is the same as the method of calculating the accuracy of the first evaluation in S003. Please refer to the relevant description in S003, which is not limited here.
  • the test samples used in the testing process correspond to the training samples used in the training process.
  • a training sample includes a training sample image and the image quality evaluation information of the mark corresponding to the training sample image
  • a test sample includes a test sample image and the image quality evaluation information of the mark corresponding to the test sample image.
  • a training sample includes an original training image, at least one training sample image obtained after processing the original training image, and the training sample image has labeled image quality evaluation information
  • a test sample includes an original test image, at least one For the test sample image obtained after processing the original test image, the test sample image has marked image quality evaluation information.
  • the original training image in the training sample can also have labeled image quality evaluation information
  • the original test image in the test sample can also be It has marked image quality evaluation information; during the testing process, it is necessary to evaluate the respective image quality of the original test image and the test sample image.
  • S007 Determine whether the second evaluation accuracy is greater than or equal to a second accuracy threshold.
  • the second accuracy threshold is used to measure whether the test result meets the standard.
  • the second evaluation accuracy is less than the second accuracy threshold, the test does not meet the standard, and S005 is executed, and the initial quality evaluation model needs to be continuously trained.
  • the second evaluation accuracy rate is greater than or equal to the second accuracy threshold, the test meets the standard, and S008 is executed.
  • the second accuracy threshold may be the same as or different from the first accuracy threshold, and there is no limitation here.
  • S008 Stop training the initial quality evaluation model, and obtain the quality evaluation model after training.
  • the test samples in the test set are used to test the accuracy of the trained quality evaluation model to verify the accuracy and reliability of the output results of the quality evaluation model. If the test fails, fine-tune the parameters of the quality evaluation model and continue training; if the test passes, stop training to obtain the trained quality evaluation model. Since the accuracy of the trained quality evaluation model is up to the test standard, when the quality evaluation model is used to evaluate the image quality, the accuracy and reliability of the results output by the quality evaluation model can be improved.
  • the above introduces a scheme for evaluating the image quality after processing the first image with one image enhancement model; the following describes the use of at least two image enhancement models to process the first image to obtain at least two second images, and evaluate at least two The image quality of the second image is to output the best image quality program.
  • FIG. 12 is a schematic flowchart of an image processing method according to a sixth embodiment of the present application.
  • the image processing method includes the following steps:
  • S501 is the same as S101. For details, refer to the related description in S101, which is not repeated here.
  • S502. Use at least two image enhancement models to process the first image to obtain at least two second images.
  • At least two image enhancement models perform the same processing on the first image.
  • At least two image enhancement models are both used to implement the same image processing function, for example, at least two image enhancement models are both used to implement super-resolution, denoising, demosaicing or image restoration. That is, the at least two image enhancement models may both be super-resolution models, denoising models, demosaicing models, or image restoration models.
  • an image enhancement model can also implement any combination of at least two of super-resolution, image denoising, demosaicing, and image restoration.
  • the sample images used for training the image enhancement model may have at least two situations as follows: resolution is less than a preset resolution threshold, image noise, mosaic, blurred image areas, and so on.
  • an image enhancement model can be obtained by concatenating at least two sub-models, and the image processing functions of the at least two sub-models are different. That is, an image enhancement model can be formed by concatenating at least two of the super-resolution model, denoising model, demosaicing model, and image restoration model; in this way, the image enhancement model can perform super-resolution, image denoising, At least two of demosaicing and image restoration.
  • the serial connection order of the sub-models can be determined according to the image processing priority. For example, the priority of image processing can be: denoising>super-resolution>de-mosaic.
  • An image enhancement model can be formed by sequential concatenation of a denoising model, a super-resolution model, and a demosaicing model.
  • the mobile phone can input the first image into at least two image enhancement models, and the at least two image enhancement models process the first image in a parallel processing manner to obtain the second image output by each image enhancement model.
  • the user can open the image processing application and load the first image to be processed, and the user can select the target image from the image processing function options displayed in the user interface (UI) of the image processing application Processing function options.
  • Image processing options include, but are not limited to: face-lift, stove-pipe, image restoration, super-resolution, denoising, demosaicing, etc.
  • the mobile phone acquires at least one target image processing function option selected by the user, and calls at least two image enhancement models corresponding to the target image processing function option to process the first image. "Slim face” and "skinny legs" can correspond to an image enhancement model with an image restoration function.
  • the mobile phone when it acquires the first image to be processed, it can acquire image features in the first image, and select at least two image enhancement models to process the first image according to the image features. For example, a user starts a camera application in a mobile phone to take a photo, and when the mobile phone collects a first image, at least two image enhancement models are selected to process the first image according to the image characteristics of the first image.
  • the mobile phone when the mobile phone detects that the resolution of the first image is less than or equal to the preset resolution threshold, it may use at least two super-resolution models to perform super-resolution processing on the first image to obtain each super-resolution The second image output by each model.
  • the preset resolution threshold can be set according to the actual situation.
  • the mobile phone When the mobile phone detects the presence of image noise in the first image, it uses at least two denoising models to denoise the first image to obtain the second image output by each denoising model.
  • the image noise of the second image There is less image noise than the first image.
  • the mobile phone When the mobile phone detects that there is a mosaic in the first image, it may use at least two demosaic models to demosaic the first image to obtain a second image output by each demosaic model.
  • the mobile phone When the mobile phone detects that there is an image blur area in the first image, for example, it detects that the first image is an old photo retaken, and the first image is blurred due to the presence of rain lines (raindrops), fog, mirrors, etc., use at least The two image restoration models respectively perform restoration processing on the first image to obtain the second image output by each image restoration model.
  • the sharpness (or visibility) of the second image is greater than the sharpness (or visibility) of the first image.
  • the resolution is less than a preset resolution threshold, there is image noise, there is mosaic, there is an area where the image is blurred, and so on.
  • the mobile phone detects that the resolution of the first image is less than the preset resolution threshold and there is When there is image noise and mosaic in the first image, at least two image enhancement models that can achieve super-resolution, image denoising and demosaicing are used to process the first image in parallel to obtain the second output of each image enhancement model. image.
  • the first image A can be subjected to image denoising, super-resolution and demosaicing through the first image enhancement model to obtain the second image B1; through the second image enhancement The model performs image denoising, super-resolution and demosaic processing on the first image A to obtain the second image B2; performs image denoising, super-resolution and demosaic processing on the first image A through the Nth image enhancement model to obtain The second image BN.
  • the mobile phone detects that the resolution of the first image is less than the preset resolution threshold and the first image
  • at least two image enhancement models obtained by concatenating a denoising model and a super-resolution model are selected, and the first image is processed in parallel to obtain a second image output by each image enhancement model.
  • the denoising model in each image enhancement model can be used to denoise the first image
  • the super-resolution model in each image enhancement model can be used to perform super-resolution processing on the denoised image to obtain A second image.
  • the denoising model 1 in the first image enhancement model is used to denoise the first image to obtain image 1, and image 1 is input to the super in the first image enhancement model.
  • the resolution model 1 performs super-resolution processing to obtain a second image B1.
  • Use the denoising model N in the Nth image enhancement model to denoise the first image to obtain the image N, and input the image N into the super-resolution model N in the Nth image enhancement model for super-resolution processing to obtain the second image BN.
  • S503 Process the at least two second image input quality evaluation models to obtain image quality evaluation information corresponding to each of the second images.
  • the mobile phone can use the quality evaluation model to process the image B1 to obtain the image quality evaluation information corresponding to the image B1; use the quality evaluation model to analyze the image B2 Perform processing to obtain image quality evaluation information corresponding to image B2; ...; use a quality evaluation model to process image BN to obtain image quality evaluation information corresponding to image BN.
  • the image quality evaluation information may be an image quality score, and the image quality score may be any integer from 0 to 100, or any value from 0-1.
  • the image quality evaluation information may also be image quality classification information, and the image quality classification information may be identified by integers or decimals in the range of 0-1.
  • S504 and S505 are parallel steps. After S503 is executed, the mobile phone can execute S504 or S505.
  • S504 The difference between S504 and S505 is that in S504, the mobile phone can directly display at least two images with the best image quality in the second image type; in S505, the mobile phone can display the first image and at least two second images, the image quality is the best Image.
  • S504 According to the image quality evaluation information corresponding to each second image, select an image with the best image quality from at least two second images, and output the image with the best image quality.
  • the mobile phone can compare the image quality evaluation information of all the first images to determine the target with the best image quality from at least two second images The second image. For example, when the image quality evaluation information is the image quality score, the mobile phone can compare the image quality score of each second image, filter out the target second image with the highest image quality score from at least two second images, and output The second image with the highest image quality score.
  • the method of outputting the second image with the highest image quality score may be to display the second image with the highest image quality score, or to save the second image with the highest image quality score to the gallery.
  • the mobile phone may also compare the image quality score of each second image with a predetermined score threshold, filter out target second images with an image quality score greater than or equal to the predetermined score threshold, and output the target second image. For example, suppose the number of second images is 3, which are image B1, image B2, and image B3. If the image quality score of image B1 is 0.8, the image quality score of image B2 is 0.7, and the image quality score of image B3 is 0.5 , The predetermined score threshold is 0.7.
  • the mobile phone can simultaneously display on the display interface Image B1 and image B2 can also display prompt information so that the user can select the image that needs to be saved.
  • the user can select and save the image B1 and the image B2, or save the image B1 and the image B2 through the dialog box.
  • the user can select and save the image B1 by clicking on the image B1, and the image B1 is light gray.
  • the mobile phone saves the image B1 and the image B2, and the image B1 and the image B2 are both light gray.
  • the user can click on the box on the left side of image B1, and “ ⁇ ” is displayed in the box, and “Save” is displayed on the right side of image B1, indicating that the user has selected image B1 and the phone saves image B1.
  • target second images may be displayed; when part of the target second images may also be displayed, for example, target second images with higher image quality scores may be displayed preferentially.
  • the method of displaying all target second images may be: the mobile phone displays a plurality of target second images in the UI interface In the image sequence, the user can slide left or right to view any target second image in the image sequence, so that the image to be saved can be selected from multiple target second images. As shown in Figure 13d, the user clicks the "photograph button" to trigger a photographing instruction to take a photo.
  • the mobile phone obtains a first image in response to the photographing instruction triggered by the user, and processes the first image through M image enhancement models to obtain N second images; Input the N second images into the quality evaluation model for processing, or input the first image and N second images into the quality evaluation model for processing to obtain image quality evaluation information, and determine image B1 to image Bi according to the obtained image quality evaluation information It is the image with the best image quality among the first image and the N second images; when the user clicks the icon on the left of the "photograph button", the mobile phone displays image B1 to image Bi. Among them, the mobile phone can display the image B1 in the main display area, and display the image sequence composed of the image B1 to the image Bi below the image B1.
  • the user can also swipe left on the touch screen anywhere on the touch screen to trigger the phone to move the image sequence to the left according to the sliding distance, so that the user can view any image that is behind image B1, such as image B3; the user can also swipe to the right to touch
  • the gesture triggers the mobile phone to move the image sequence to the right according to the sliding distance, so that the user can view the image as needed and select the image that needs to be saved.
  • the mobile phone can also recognize the second image with the highest image quality score as the target image with the best image quality according to the corresponding image quality score of each second image, and output the second image with the highest image quality score .
  • the number of second images is 3, which are image B1, image B2, and image B3. If the image quality score of image B1 is 0.8, the image quality score of image B2 is 0.7, and the image quality score of image B3 is 0.5 , Then, the mobile phone determines that image B1 is the image with the best image quality according to the image quality scores of images B1 to B3, and outputs image B1.
  • the number of second images with the highest image quality score is at least two, either one or at least two may be displayed for the user to select.
  • the mobile phone can display image B1 and image B2 on the display interface at the same time, and can also display prompt information for the user to choose The image that needs to be saved.
  • FIG. 14 is a schematic diagram of a user interface provided by still another embodiment of the present application.
  • image B1 eliminates spectacles reflection, and the facial features of the portrait and glasses are not deformed;
  • image B2 eliminates spectacles reflection, and the glasses are deformed in the image,
  • image B3 eliminates part of the spectacles reflection, and the facial features of the portrait are deformed .
  • the mobile phone compares the corresponding image quality scores of image B1 to image B3, and the comparison result is: image quality score of image B1> image quality score of image B2> image quality score of image B3, indicating the image quality of image B1>
  • the target image filtered by the mobile phone from at least two second images by the image quality score may be an image with the highest definition and no artifacts, or an image with higher definition and the least artifacts.
  • the target image can be an image with the least image noise (or no image noise) and no artifacts, or an image with less image noise and the least artifacts.
  • the target image can be an image with the least image noise (or no image noise) and no shot noise, or an image with less image noise and the least shot noise.
  • the target image can be an image without color aliasing and zipper effect, or an image with the least color aliasing and zipper effect.
  • S505 According to the image quality evaluation information corresponding to each second image, determine an image with the best image quality from the first image and at least two second images, and output the image with the best image quality.
  • the mobile phone can determine the image quality from the first image and at least two second images by comparing the image quality evaluation information of all the second images.
  • the best target image and output the image with the best image quality.
  • the image with the best image quality can be the second image or the first image.
  • the way to output the image with the best image quality can be to display the image with the best image quality, or it can be to save the image with the best image quality to the gallery.
  • the image quality evaluation information may be identification information or scores for indicating image quality.
  • the image quality evaluation information is image quality evaluation information for each of the N second images.
  • the mobile phone can determine whether there is a target second image whose image quality evaluation information is a predetermined identification in the N second images, according to the judgment result
  • the image with the best image quality is determined from the first image and the N second images. Among them, if there is a target second image whose image quality evaluation information is a predetermined mark in the N second images, it is determined that the target second image whose image quality evaluation information is the predetermined mark is the image with the best image quality, and the target second image is output. image.
  • any target second image is selected for output, or all target second images are output. If there is no target second image whose image quality evaluation information is a predetermined identifier among the N second images, it is determined that the first image is an image with the best image quality, and the first image is output.
  • the mobile phone can determine whether there is a target second image with an image quality score greater than a predetermined score threshold in the N second images, and determine the image quality from the first image and the second image according to the judgment result The best image. If there is a target second image with an image quality score greater than or equal to the predetermined score threshold in the N second images, the target second image is determined to be the image with the best image quality, and the target second image is output; if there are N second images If there is no target second image with an image quality score greater than or equal to the predetermined score threshold, it is determined that the first image is an image with the best image quality, and the first image is output.
  • any target second image can be selected for output, all target second images can be output, or part of the target second images can be output.
  • the target image with a higher image quality score can be displayed first, for example, the target second image with the highest score is output.
  • Any target second image can be output. It should be noted that when the mobile phone detects that the image quality score of any second image is greater than or equal to the predetermined score threshold, it outputs the second image and ends the image processing flow, so that the first image with better image quality can be output more quickly. Two images.
  • the mobile phone may also display all target second images on the display interface for the user to select the desired choice.
  • the number of target second images is large (for example, 3 or more)
  • a thumbnail corresponding to the target second image can be generated, and the thumbnail can be displayed on the display interface, and the user can browse the thumbnails included in the thumbnail All target second images, select the images that need to be saved.
  • the number of second images is 3, which are image B1, image B2, and image B3. If the image quality score of image B1 is 0.8, the image quality score of image B2 is 0.7, and the image score of image B3 is 0.5, it is predetermined The score threshold is 0.7; then the mobile phone screens out target second images with image quality scores greater than or equal to 0.7 based on the image quality scores of all the second images, including image B1 and image B2, and simultaneously displays image B1 and image B2 on the display interface , For users to choose.
  • the number of second images is 5, which are image B1, image B2, image B3, image B4, and image B5.
  • image B1 has an image quality score of 0.6
  • image B2 has an image quality score of 0.5
  • image B3 The image score of image B4 is 0.8
  • the image quality score of image B4 is 0.8
  • the image quality score of image B5 is 0.7
  • the predetermined score threshold is 0.7
  • the target second image with an image quality score greater than or equal to 0.7 selected by the mobile phone includes image B3 , Image B4 and image B5, generate a thumbnail composed of image B3, image B4, and image B5, and display the thumbnail on the display interface, so that the user can browse the image B3, image B4, and image B5 included in the thumbnail, thereby Select the image to be saved.
  • the size of the thumbnail can be adjusted according to the size of the mobile phone display screen. There are no restrictions on the arrangement and order of each image in the thumbnail.
  • the at least two target second images may be sorted by image quality scores and then displayed.
  • the mobile phone can display image B3 before image B2, or display image B3 in a larger Display area.
  • the display screen is divided into a first display area and a second display area.
  • the first display area may be above or to the left of the second display area, and the area of the first display area may be larger than the area of the second display area (for example, the first display area).
  • the area ratio of the first display area to the second display area is 2: 1), the image B3 is displayed in the first display area, and the image B2 is displayed in the second display area. As shown in FIG.
  • the size of the image B3 displayed on the display interface of the mobile phone is larger than the size of the image B2. It should be noted that the method for the user to select the desired image based on the UI interface as shown in FIG. 15 is similar to the selection method in FIG. 13, and will not be repeated here.
  • the target second image includes image B3, image B4, and image B5, image quality score of image B3> image quality score of image B4> image quality score of image B5;
  • the quality scores are sorted from high to low, and then the corresponding image list or thumbnail is generated.
  • the arrangement of the images in the image list or thumbnail is: image B3-image B4-image B5.
  • the image B3, the image B4, and the image B5 are arranged vertically, and the image B3 is displayed at the forefront.
  • the method for the user to select the desired image based on the UI interface as shown in FIG. 16 is similar to the selection method in FIG. 13, and will not be repeated here.
  • At least two image enhancement models may be used to process the first image to obtain at least two second images, the target image with the best image quality may be selected from the at least two second images, and the The target image with the best image quality is displayed for the user to view.
  • the image with the best image quality can be determined from the first image and at least two second images, and the image with the best image quality can be output.
  • they can be sorted and then displayed to the user. The user can quickly learn the image quality of each image, so that the desired image can be quickly selected.
  • FIG. 17 is a schematic flowchart of an image processing method according to a seventh embodiment of the present application.
  • the input of the quality evaluation model is the second image, and the quality evaluation model is used to evaluate the image quality of the second image based on the image feature information of the second image;
  • the input of the quality evaluation model is The first image and the second image, and the quality evaluation model is used to evaluate the image quality of the second image based on the feature information of the first image and the feature information of the second image.
  • S603 is as follows:
  • the mobile phone uses the quality evaluation model to determine the image quality evaluation information corresponding to the image B1 based on the image A and the image B1; uses the quality evaluation model to determine the image quality evaluation information corresponding to the image B1 based on the image A and Image B2 determines image quality evaluation information corresponding to image B2; ...; using a quality evaluation model to determine image quality evaluation information corresponding to image BN based on image A and image BN.
  • the first image (for example: image A) is used to assist in evaluating the image quality of each second image.
  • image A For a specific implementation manner of evaluating the image quality of the second image with the aid of the first image, refer to the related description of S203 in the embodiment corresponding to FIG. 7, which will not be repeated here.
  • the feature information of the first image is used to assist in evaluating the image quality of the second image. Since the mobile phone can use the first image as a reference image to evaluate the image quality of the second image, the quality of the second image can be evaluated more accurately. The image quality can improve the accuracy of the image quality evaluation information of the second image.
  • FIG. 18 is a schematic flowchart of an image processing method according to an eighth embodiment of the present application.
  • the difference between Fig. 18 and Fig. 12 lies in S703 ⁇ S704.
  • S703 in Fig. 17 adds a step of evaluating the first image by using a quality evaluation model.
  • Fig. 12 is a sample image with the best image quality selected from at least two second images.
  • the sample image with the best image quality is selected for output from at least two second images and the first image. details as follows:
  • S703. Process the first image and the at least two second image input quality evaluation models to obtain first image quality evaluation information corresponding to the first image, and a first image corresponding to each of the second images. 2. Image quality evaluation information.
  • the mobile phone can use the quality evaluation model to process image A to obtain image quality evaluation information corresponding to image A; the mobile phone can use the quality evaluation model to Image B1 is processed to obtain image quality evaluation information corresponding to image B1; image B2 is processed using a quality evaluation model to obtain image quality evaluation information corresponding to image B2; ...; image BN is processed using a quality evaluation model to obtain an image Image quality evaluation information corresponding to BN.
  • the mobile phone can use the quality evaluation model to process image A to obtain image quality evaluation information corresponding to image A; the mobile phone can use the quality evaluation model Process image A and image B1 to obtain image quality evaluation information corresponding to image B1; use quality evaluation model to process image A and image B2 to obtain image quality evaluation information corresponding to image B2; ...; use quality evaluation model to Image A and image BN are processed to obtain image quality evaluation information corresponding to image BN.
  • the image quality evaluation information corresponding to image A can be determined by the first image quality evaluation information of image A; the image quality evaluation information of images B1 to BN can be determined by the second feature information of images B1 to BN, respectively, or
  • the difference characteristic information is obtained by comparing the first characteristic information and the second characteristic information, which is determined according to the difference characteristic information.
  • the first feature information may be all the feature information of the image A, or part of the feature information of the image A. All feature information is feature information of each pixel in image A, and partial feature information is feature information of some pixels in image A, for example, pixels in the first image that can reflect or represent the image quality of the first image.
  • S704 and S705 are parallel steps. After performing S703, the mobile phone can perform S704 or S705.
  • the image quality evaluation information corresponding to the first image and the second image quality evaluation information corresponding to each of the second images determine that the image quality is the best among the first image and the at least two second images. Good image, output the image with the best image quality.
  • the image with the best image quality may be determined from the first image and all the second images based on the first image quality evaluation information and all the second image quality evaluation information.
  • the image with the best image quality may be the first image and the at least two second images
  • the image quality evaluation information is an image with a predetermined identifier, or an image with an image quality score greater than or equal to a predetermined score threshold, or an image quality An image with a score greater than or equal to a predetermined score threshold and with the highest image quality score.
  • the predetermined identifier can be "1" or "YES” or "YES".
  • the image with the best image quality may be the first image or the second image.
  • the mobile phone can determine that the first image and the Whether there is an image whose image quality evaluation information is a predetermined identifier among the N second images, the image with the best image quality is determined from the first image and the second image according to the judgment result.
  • the image quality evaluation information of the first image is a predetermined identifier
  • there is at least one target second image whose image quality evaluation information is the predetermined identifier in the N second images select from the first image and the target second image Any image output.
  • the image quality evaluation information of the first image is not the predetermined identifier, and there is at least one target second image whose image quality evaluation information is the predetermined identifier in the N second images, then any one of the target second images is selected for output. If the image quality evaluation information of the first image is not the predetermined identifier, and there is no target second image whose image quality evaluation information is the predetermined identifier in the N second images, the first image is output.
  • the mobile phone can use the first image quality score of the first image and the second image quality scores of the N second images from the first image And among the N second images, it is determined that the image with the highest image quality score is the image with the best image quality among the first image and the N second images, and the image with the highest score is output.
  • the mobile phone can also determine whether there is an image with an image quality score greater than a predetermined score threshold in the first image and the N second images, and determine the image with the best image quality from the first image and the second image according to the judgment result.
  • the image with the image quality score greater than or equal to the predetermined score threshold is output.
  • the number of images with an image quality score greater than or equal to the predetermined score threshold is at least two, any one of them can be selected for output, or all images with an image quality score greater than or equal to the predetermined score threshold can be displayed, so that the user can choose to save Image.
  • all images with image quality scores greater than or equal to a predetermined score threshold may be sorted and displayed according to the image quality scores.
  • the image quality of the first image and the second image can be evaluated separately, the first image quality evaluation information corresponding to the first image and the second image quality evaluation information corresponding to the second image can be obtained, and the first image can be directly compared.
  • the quality evaluation information and the second image quality evaluation information thereby selecting an image with the best image quality from the first image and the second image, and displaying the image with the best image quality to the user.
  • FIG. 19 is a schematic flowchart of an image processing method according to a ninth embodiment of the present application.
  • the image processing method shown in FIG. 19 includes the following steps:
  • S801 is the same as S101. For details, please refer to the relevant description in S101, which will not be repeated here.
  • S803 Input the image B1 into a quality evaluation model for processing, and obtain image quality evaluation information corresponding to the image B1.
  • S804 Determine whether the image B1 is a target image according to the image quality evaluation information corresponding to the image B1.
  • S804 is basically the same as S104, please refer to the related description in S104, which will not be repeated here.
  • the target image refers to an image whose image quality meets the requirements.
  • the image quality evaluation information of image B1 is a predetermined identifier, or the image quality score corresponding to the image quality evaluation information of image B1 is greater than or equal to the predetermined score threshold, it means that image B1 is the target image, and S805 is executed to output image B1;
  • the image quality evaluation information of B1 is not a predetermined identifier, or the image quality score corresponding to the image quality evaluation information of image B1 is less than the predetermined score threshold, it means that image B1 is not the target image.
  • image B1 is not output, and image A is not output. Go to S806.
  • the predetermined identifier can be "1", "YES” or "YES”.
  • S808 Determine whether the image B2 is a target image according to the image quality evaluation information corresponding to the image B2.
  • the image quality evaluation information of the image B2 is a predetermined identifier, or the image quality score corresponding to the image quality evaluation information of the image B2 is greater than or equal to the predetermined score threshold, it means that the image B2 is the target image, and S809 is executed to output the output image B2 ;
  • the image quality evaluation information of the image B2 is not a predetermined identifier, or the image quality score corresponding to the image quality evaluation information of the image B2 is less than the predetermined score threshold, it means that the image B2 is not the target image, and the image B2 is not output.
  • the number of image enhancement models is 2
  • the mobile phone outputs image A after performing S808 to determine that image B2 is not the target image. If the number of image enhancement models is greater than 2, the mobile phone executes S808 after determining that image B2 is not the target image, then executes S810.
  • N is an integer and N ⁇ 3.
  • S812 Determine whether the image BN is a target image according to the image quality evaluation information corresponding to the image BN.
  • the image quality evaluation information of the image BN is a predetermined identifier, or the image quality score corresponding to the image quality evaluation information of the image BN is greater than or equal to the predetermined score threshold, it means that the image BN is the target image, and S813 is executed to output the image BN;
  • the image quality evaluation information of the image BN is not a predetermined identifier, or the image quality score corresponding to the image quality evaluation information of the image BN is less than the predetermined score threshold, it means that the image BN is not a target image, and the image BN is not output.
  • the image enhancement model N is not the last one of the N image enhancement models preset in the mobile phone, after the mobile phone executes S812 to determine that the image BN is not the target image, N is incremented by 1, and then returns to the application in step 810
  • the image enhancement model N processes the image A to obtain the image BN output by the image enhancement model N. If the image enhancement model N is the last of the N image enhancement models preset in the mobile phone, after performing S812 and determining that the image BN is not the target image, S814 is performed.
  • the image enhancement model may be determined according to the image processing function selected by the user, or may be determined by the image characteristics of the image A, which is not limited here.
  • N image enhancement models can be used to implement the same image processing function, that is, N image enhancement models can be used to implement denoising, super-resolution, demosaicing or image restoration. Each of the N image enhancement models can also be used to implement at least two image processing functions.
  • the N image enhancement models can be sorted according to the degree of image processing effect corresponding to the model, that is, the model with the best image processing effect is preferentially selected from the candidate image enhancement models to process image A.
  • the N image enhancement models can also be used to implement different image processing functions.
  • the image enhancement model 1 may be the image enhancement model with the best denoising effect, the best super-resolution effect, or the best demosaicing effect. .
  • each of the N image enhancement models is used to implement at least two image processing functions, it is assumed that if there are at least two situations in image A: the resolution is less than the preset resolution threshold, and the image noise , Mosaic, then each of the N image enhancement models is used to perform denoising, super-resolution, and demosaicing on image A.
  • the image enhancement model 1 can be an image enhancement model with the best denoising effect, the best super-resolution effect, and the best de-mosaic effect.
  • image enhancement model 1 and image enhancement model 2 can be denoising model
  • image enhancement model 3 and image enhancement model 4 can be super-resolution model
  • image enhancement model 5 and image enhancement model 6 can be demosaicing Model.
  • the image enhancement model 1 can be the image enhancement model with the best denoising effect
  • the image enhancement model 3 can be the image enhancement model with the best super-resolution effect
  • the image enhancement model 5 can be the enhancement model with the best demosaic effect.
  • the image processing effects of different image enhancement models of the same type may be different, the severity of artifacts in the images processed by the model may be different, and the image output to the user needs to balance the artifacts and Therefore, it is necessary to use a quality evaluation model to evaluate the image quality of the second image obtained after the first image is processed by the image enhancement model, so as to determine whether it is necessary to use other image enhancement models for the first image based on the image quality evaluation results.
  • the image is processed, so as to output a second image with better image quality for the user to view as much as possible.
  • the image enhancement model has a certain degree of stability, and its image enhancement effect is predictable, and it is uncertain whether the image obtained by the image enhancement model processing may contain artifacts. Unforeseeable, in this way, image A is processed through an image enhancement model with a better image enhancement effect, and the probability of obtaining image B with better image quality is greater. Therefore, in this embodiment, the mobile phone prefers image enhancement A better image enhancement model processes image A, so that the probability of obtaining image B with better image quality is greater, and the time required to obtain image B with better image quality can be shortened to improve the output image efficient.
  • the mobile phone detects that the resolution of image A is less than the preset resolution threshold, it can use super-resolution model 1 to process image A to obtain image B1, and then use the quality evaluation model to evaluate the image quality of image B1 to obtain the image of image B1 Quality score. The mobile phone judges whether the image quality score of image B1 is greater than or equal to the predetermined score threshold.
  • image quality score of image B1 is greater than or equal to the predetermined score threshold, then output image B1 to the user for viewing. This situation reflects that although the sharpness of image B1 is greater than The clarity of image B2, but the artifacts in image B1 may be more serious than the artifacts in image B2. If the image quality score of the image B1 is less than the predetermined score threshold, then the image A is output to the user to view; this situation reflects the existence of artifacts in both the image B1 and the image B, and the artifacts in B1 may be more serious than the artifacts in B2.
  • the mobile phone when it detects that the resolution of image A is lower than the resolution threshold, it acquires at least two super-resolution models, for example, super-resolution model 1, super-resolution model 2, and super-resolution model 3.
  • super-resolution model 1 the clarity of the image processed by the super-resolution model 2> the clarity of the image processed by the super-resolution model 3
  • the mobile phone adopts the super-resolution model 1 pair of images A performs super-resolution processing to obtain image B1.
  • the mobile phone uses the image evaluation model to process the image B1 to obtain the image quality evaluation information corresponding to the image B1; determine whether to output the image B1 according to the image quality score corresponding to the image B1.
  • the mobile phone If the image quality score corresponding to the image B1 is greater than or equal to the predetermined score threshold, the mobile phone outputs the image B1 to the user to view, and the image processing flow ends.
  • the user can see that the sharpness of the image B1 is good, and there are no artifacts or a small amount of artifacts in the image B1. For example, there is no obvious deformation or misalignment of the scene, objects, human eyes, nose, or glasses frame in the image B1.
  • the image quality score corresponding to the image B1 is less than the predetermined score threshold, it means that the image B1 is blurry, or there are serious artifacts in the image B1; the mobile phone uses the super-resolution model 2 to perform super-resolution processing on the image A to obtain the image B2.
  • the mobile phone uses the image evaluation model to process the image B2 to obtain the image quality score corresponding to the image B2; according to the image quality score corresponding to the image B2, it is judged whether to output the image B2. If the image quality score corresponding to image B2 is greater than or equal to the predetermined score threshold, the mobile phone outputs image B2 to the user for viewing, and the image processing flow ends.
  • the image B2 that the user can see has better clarity, and there are no artifacts or a small amount of artifacts in the image B2.
  • the mobile phone uses super-resolution model 3 to perform super-resolution processing on image A to obtain image B3.
  • the mobile phone uses the image evaluation model to process the image B3 to obtain the image quality score corresponding to the image B3; the image quality score corresponding to the image B3 can be compared with a predetermined score threshold, and whether to output the image B3 is determined according to the comparison result. If the comparison result is that the image quality score corresponding to image B3 is greater than or equal to the predetermined score threshold, the mobile phone outputs image B3 to the user for viewing, and the image processing flow ends. If the comparison result is that the image quality score corresponding to the image B3 is less than the predetermined score threshold, it means that the image B3 is blurry, or there are serious artifacts in the image B3, and the image A is output for the user to view.
  • the mobile phone can preferentially use the image enhancement model 1 with the best image processing effect to process image A. If the image quality of the processed image is poor, then use the image enhancement model 2 with the second best image processing effect. Image A is processed. If the image quality of the processed image is still poor, then select the optimal image enhancement model from the optional image enhancement model 3 to process image A.
  • the optional image enhancement model refers to the previous Image enhancement models other than the image enhancement models that have been used (such as the optimal and suboptimal image enhancement models for image processing). Since the image enhancement model with the best image processing effect is preferentially used to process the image A, in some cases, N image enhancement models are not needed to obtain the image B with better image quality. Compared with the case where N image enhancement models are used to process image A in parallel, some system resources can be saved, and the time required to obtain image B with better image quality can be shortened, so as to improve the efficiency of the output image.
  • image A can be used to assist in evaluating the image quality of the second image (for example: B1 to BN).
  • image B1 The quality evaluation model is processed to obtain the image quality evaluation information corresponding to the image B1" is replaced with "the image A and the image B1 are input to the quality evaluation model for processing, and the image quality evaluation information corresponding to the image B1 is obtained".
  • input image B2 into the quality evaluation model for processing to obtain image quality evaluation information corresponding to image B2 is replaced with "input image A and image B2 into the quality evaluation model for processing to obtain image quality evaluation information corresponding to image B2 ".
  • in S811 "input image BN into the quality evaluation model for processing to obtain image quality evaluation information corresponding to image BN" is replaced with "input image A and image BN into the quality evaluation model for processing to obtain image quality evaluation information corresponding to image BN ".
  • the second image processed by image A is input to the quality evaluation model, and the quality evaluation model is used to evaluate the image quality of the second image.
  • the quality evaluation can be used
  • the model determines the image quality evaluation information for the image A and the second image (for example: B1 to BN), determines the target image based on the image quality evaluation information for the image A and the second image, and outputs the target image.
  • FIG. 21 is a schematic flowchart of an image processing method according to an eleventh embodiment of the present application.
  • the difference from FIG. 19 is S903 to S904, S907 to S908, and S911 to S912, S914. details as follows:
  • S904 Determine whether the image B1 is a target image according to the image quality evaluation information for the image A and the image B1.
  • the image quality evaluation information is the identification information used to indicate whether the image quality of the image B1 is better than the image quality of the image A
  • the image quality evaluation information of the image B1 is a predetermined identification, it indicates the image quality ratio of the image B1
  • the image quality of the image A is good, and the image B1 is the target image, then execute S905 to output the image B1; if the image quality evaluation information of the image B1 is not a predetermined identifier, it means that the image quality of the image B1 is worse than the image quality of the image A, and the image B1 If it is not the target image, then the image B1 is not output, and S906 is executed.
  • the predetermined identifier can be "1", "YES” or "YES”.
  • the image quality evaluation information includes image quality score A of image A and image quality score B of image B1, if image quality score B>image quality score A, it means that the image quality of image B1 is higher than the image quality of image A.
  • image B1 is the target image, then execute S905 to output image B1; if image quality score B ⁇ image quality score A, it means that the image quality of image B1 is worse than that of image A, or the image quality of image B1 and image A If the image quality is the same, and the image B1 is not the target image, then it is determined that the image B1 is not output, and S906 is executed.
  • S907 Input the image A and the image B2 into the quality evaluation model for processing to obtain image quality evaluation information for the image A and the image B2.
  • S908 Determine whether the image B2 is the target image according to the image quality evaluation information for the image A and the image B2.
  • the mobile phone outputs image A after performing S908 to determine that image B2 is not the target image. If the number of image enhancement models is greater than 2, the mobile phone executes S908 after determining that image B2 is not the target image, then executes S910.
  • S911 Input image A and image BN into a quality evaluation model for processing, and obtain image quality evaluation information for image A and image BN.
  • the image quality of the first image and the second image can be evaluated separately, and the quality evaluation information of the first image and the second image can be used to determine whether the image quality of the second image is better than the image quality of the first image. It can be learned more accurately whether the image quality of the first image is deteriorated after being processed by the image enhancement model.
  • the image enhancement model is first used to perform image processing on the first image to obtain the second image.
  • the second image, or the first image and the second image are simultaneously input to the training
  • the latter quality evaluation model is processed to obtain the image quality evaluation information output by the quality evaluation model, and the first image or the second image is determined to be output according to the image quality evaluation information.
  • 17-21 can be modified into the following processing logic: first use a quality evaluation model to evaluate the image quality of the first image, If the image quality evaluation information of the first image indicates that the image quality of the first image is good, use the image enhancement model to perform image processing on the first image to obtain the second image, and output the second image to the user; if the image of the first image The quality evaluation information indicates that the image quality of the first image is poor, and the first image is output to the user.
  • FIG. 22 is a schematic flowchart of an image processing method according to a twelfth embodiment of the present application.
  • RAW images are images in RAW format.
  • the RAW format is an unprocessed and uncompressed format.
  • RAW can be conceptualized as "raw image coded data" or more vividly called “digital film”.
  • the RAW image is the original data that the image sensor converts the captured light source signal into a digital signal.
  • the image sensor may include a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor and a Charge Coupled Device (CCD) image sensor.
  • CMOS Complementary Metal-Oxide-Semiconductor
  • CCD Charge Coupled Device
  • the user starts the camera application in the mobile phone, and the mobile phone acquires multiple frames of RAW images in response to the camera instruction triggered by the user.
  • the exposure level of the image is related to the brightness of the shooting environment, for example, the intensity of the ambient light, the brightness of the shooting environment, and so on.
  • images can be divided into long exposure images, medium exposure images, and short exposure images.
  • the exposure degrees corresponding to the first images of multiple frames may be the same or different.
  • the multiple-frame RAW images may all be long-exposure images, medium-exposure images, or short-exposure images, and the multi-frame images may also include at least two of the long-exposure images, medium-exposure images, and short-exposure images.
  • the obtained RAW images of multiple frames may all be short exposure images, or may include a small amount of medium exposure images or long exposure images . That is, the number of short-exposure images acquired is greater than the number of long-exposure images. For example, 6 frames of RAW images are acquired, including 1 frame of long exposure image, 3 frames of short exposure image, and 2 frames of medium exposure image, or 5 frames of short exposure image and 1 frame of long exposure image.
  • the acquired RAW images of multiple frames may all be long-exposure images, or may include a small amount of medium-exposure images or short-exposure images.
  • the number of long-exposure images acquired is greater than the number of short-exposure images. For example, 6 frames of RAW images are acquired, including 5 frames of long exposure images and 1 frame of short exposure images, or all 6 frames of RAW images are long exposure images.
  • the multiple frames of RAW images that can be obtained may all be short-exposure images, or may include a small amount of medium-exposure images or long-exposure images. That is, the number of short-exposure images acquired is greater than the number of long-exposure images.
  • the acquired 6 frames of RAW images include 1 frame of long exposure image, 2 frames of medium exposure image, and 3 frames of short exposure image.
  • the number of images contained in each group of RAW images can be the same or different.
  • a set of RAW images may include at least two frames of RAW images.
  • 6 RAW images obtained in S1001 can be divided into 2 groups or 3 groups.
  • RAW image 1, RAW image 3, and RAW image 5 are a group
  • RAW image 2, RAW image 4, and RAW image 6 are a group
  • RAW image 1 and RAW image 2 are a set
  • RAW image 3, RAW image 4, RAW image 5, and RAW image 6 are a set
  • RAW image 1 and RAW image 3 are a group
  • RAW image 2 and RAW image 4 are a group
  • RAW image 5 and RAW image 6 are a group.
  • S1003 Perform image fusion processing on each group of the RAW images to obtain a fusion image corresponding to each group of the RAW images.
  • the mobile phone merges all the images contained in each group of RAW images into one image.
  • image registration technology can be used to perform image fusion processing on each group of RAW images to obtain a fused image.
  • the relevant information about the image data of the same target can be fused together to expand the time and space information contained in the image. Reduce uncertainty and increase reliability.
  • Image registration methods include but are not limited to optical flow registration methods.
  • Image registration is the process of matching and superimposing two or more images acquired at different times, different sensors (imaging equipment) or under different conditions (weather, illuminance, camera position and angle, etc.).
  • the specific process is as follows: firstly perform feature extraction on two images to obtain feature points; find matching feature point pairs through similarity measurement; then obtain the image space coordinate transformation parameters through the matched feature point pairs; finally perform image matching by the coordinate transformation parameters allow.
  • Image enhancement processing can be performed on the fusion image to enhance the image quality of the fusion image, and obtain an enhanced image corresponding to the fusion image.
  • any one or any combination of at least two of super-resolution, denoising, demosaicing, and image restoration processing can be performed on the fused image.
  • an image enhancement model can be used to process the fused image.
  • a specific implementation method of processing the fused image refer to the related description of processing the first image in S102 in the embodiment corresponding to FIG. 2, which will not be repeated here.
  • the target image is an image that can be viewed by the user, and the image format of the target image may be an RGB image.
  • FIG. 23 is a schematic diagram of a method for processing a multi-frame RAW image provided by an embodiment of the present application.
  • the mobile phone obtains N frames of RAW images in S1001 and divides them into 2 groups.
  • the odd-numbered frames can be grouped together, and the even-numbered frames can be grouped together.
  • the mobile phone performs image fusion processing on the first group of RAW images to obtain a fused image 1; and processes the fused image 1 to obtain an enhanced image 1.
  • the mobile phone performs image fusion processing on the second set of RAW images to obtain a fused image 2; and processes the fused image 2 to obtain an enhanced image 2.
  • the mobile phone inputs the enhanced image 1 and the enhanced image 2 into the quality evaluation model for processing, and obtains the image quality evaluation information of the enhanced image 1 and the image quality evaluation information of the enhanced image 2.
  • the image quality evaluation information of the enhanced image 1 and the image quality evaluation information of the enhanced image 2 the target image with the best image quality is determined from the enhanced image 1 and the enhanced image 2.
  • the image quality evaluation information is the image quality score.
  • the mobile phone compares the image quality score of the enhanced image 1 with the image quality score of the enhanced image 2.
  • the target When the image quality score of the enhanced image 1 is greater than the image quality score of the enhanced image 2, the target When the image is enhanced image 1, output enhanced image 1; when the image quality score of enhanced image 1 is less than the image quality score of enhanced image 2, the target image is enhanced image 2, and output enhanced image 2; when the image quality score of enhanced image 1 is equal to
  • the image quality score of the enhanced image 2 is enhanced, the image quality of the enhanced image 1 and the enhanced image 2 are the same, and the enhanced image 1 and the enhanced image 2 are both target images, and any one of the enhanced image 1 and the enhanced image 2 is selected for output.
  • the fusion image corresponding to each group of RAW images and the enhanced image input quality evaluation model can also be processed to obtain image quality evaluation information corresponding to each enhanced image.
  • the mobile phone compares the image quality score of enhanced image 1 with the image quality score of enhanced image 2. When the image quality score of enhanced image 1 is greater than the image quality score of enhanced image 2, the target image is enhanced image 1, and enhanced image 1 is output ; When the image quality score of enhanced image 1 is less than the image quality score of enhanced image 2, the target image is enhanced image 2, and enhanced image 2 is output; when the image quality score of enhanced image 1 is equal to the image quality score of enhanced image 2, the enhancement The image quality of the image 1 and the enhanced image 2 are the same.
  • the enhanced image 1 and the enhanced image 2 are both target images, and any one of the enhanced image 1 and the enhanced image 2 is selected for output.
  • the fused image corresponding to the RAW image can be used as the reference image, and the fused image corresponding to each group of RAW images can be used to assist in evaluating the image quality of the enhanced image corresponding to the group of RAW images, which can improve the accuracy of the image quality evaluation information of the enhanced image. .
  • the fusion image corresponding to each group of RAW images and the enhanced image input quality evaluation model can also be processed to obtain image quality evaluation information corresponding to each fusion image and each enhanced image Corresponding image quality evaluation information, and based on all the image quality evaluation information, determine the target image with the best image quality from the fused image and the enhanced image, and output the target image.
  • the target image can be any fused image or any enhanced image.
  • multiple frames of RAW images may be grouped, and the grouped RAW images may be subjected to image fusion processing to obtain a fused image, and the image quality of the fused image may be enhanced to obtain an enhanced image, and then the image quality of the enhanced image may be evaluated, or Evaluate the image quality of the fusion image and enhance the image quality of the image, output the target image with the best image quality according to the evaluation result, and take a picture with better image quality.
  • the target image can be an enhanced image or a fused image. Since the mobile phone always outputs images with better image quality, the possibility of storing images with poor image quality in the mobile phone is lower, and the user is less likely to view images with poor image quality, which can improve the user's visual experience.
  • FIG. 26 shows a schematic block diagram of the structure of an image processing apparatus provided by an embodiment of the present application. part.
  • the units included in the image processing device are used to execute the steps in the embodiments corresponding to FIGS. 2, 7, 8, 12, and FIGS. 17-25.
  • the image processing apparatus 1 may include:
  • the acquiring unit 110 is configured to acquire a first image and N second images, where each second image of the N second images is performed on the first image through at least one of the M image enhancement models.
  • the image is obtained after processing; where N and M are integers greater than zero, each of the M image enhancement models is different, and the N second images are also different;
  • the image quality evaluation unit 120 is configured to process an input image input quality evaluation model to obtain image quality evaluation information.
  • the input image includes the N second images, or includes the first image and the N th Two images;
  • the image output unit 130 is configured to output a target image according to the obtained image quality evaluation information, where the target image is at least one of the first image and the N second images.
  • one second image can be obtained, or at least two second images can also be obtained, which is not limited here.
  • M can be equal to N, and M can also be greater than N.
  • the quality evaluation model is obtained through training based on a plurality of training samples, and each training sample includes a sample image and image quality evaluation information of the user on the sample image.
  • the target image includes a target image determined according to the obtained image quality evaluation information and evaluation rules, and the evaluation rule is that the image quality evaluation information is a predetermined number, or image The score corresponding to the quality evaluation information is greater than or equal to the predetermined score threshold.
  • the image quality evaluation information may be numbers or scores used to represent image quality.
  • the image quality evaluation information can also be represented by letters or words, which is not limited here.
  • the image quality evaluation information may be "0" or "1”, “YES” or “NO”, “Yes” or “No”.
  • the image quality evaluation information is a number and the input image is N second images, or the first image and the N second images
  • the image The quality evaluation information is image quality evaluation information for each of the N second images
  • the image output unit 130 is specifically configured to:
  • the target second image whose image quality evaluation information is a predetermined number in the N second images it is determined that the target second image whose image quality evaluation information is a predetermined number is the image with the best image quality, and all the images are output.
  • the image quality evaluation information is a predetermined number of target second images.
  • the predetermined number may be "1".
  • the image output unit after determining whether there is a target second image whose image quality evaluation information is a predetermined number among the N second images, is further configured to: if the N second images If there is no target second image whose image quality evaluation information is a predetermined number in the second image, it is determined that the first image is an image with the best image quality, and the first image is output.
  • the image output unit 130 is further configured to:
  • any target second image is selected for output.
  • the image quality evaluation information is a score and the input image is N second images, or when the first image and the N second images, the image The quality evaluation information is image quality evaluation information for each of the N second images;
  • the image output unit 130 is specifically configured to:
  • the target second image with the highest score is determined to be the image with the best image quality, and the target second image with the highest score is output. image.
  • the image output unit 130 is further configured to: after determining whether there is a target second image with a score greater than a predetermined score threshold in the N second images, if the N second images If there is no target second image with a score greater than a predetermined score threshold in the image, it is determined that the first image is an image with the best image quality, and the first image is output.
  • the image output unit 130 is further configured to:
  • a new one is acquired An image enhancement model, using the acquired new image enhancement model to process the first image to obtain a new second image, and inputting the new second image as an input image into the quality evaluation model for processing , Obtain new image quality evaluation information; wherein the new image enhancement model is an image enhancement model that has not processed the first image;
  • the new image quality evaluation information is the predetermined number, or the score is greater than the predetermined score threshold, then a new second image is output, otherwise it returns to the step of acquiring a new image enhancement model and subsequent steps until The number of return executions reaches a preset number threshold, and the first image is output.
  • the image quality evaluation information is a number
  • the input image is a first image and N second images
  • the image quality evaluation information is a comparison between the first image and the Image quality evaluation information of each second image in the N second images
  • the image output unit 130 is specifically configured to:
  • the image quality evaluation information of the first image is a predetermined number, and there is at least one target second image whose image quality evaluation information is the predetermined number among the N second images, then the first image and the Select any one of the target second images to output.
  • the image output unit is further configured to:
  • the image quality evaluation information of the first image is not a predetermined number, and there is at least one target second image whose image quality evaluation information is a predetermined number among the N second images, select from the target second image Any image output.
  • the image output unit 130 is further configured to:
  • the first image is output.
  • the image quality evaluation information is a score
  • the input image is a first image and N second images
  • the image quality evaluation information is a comparison between the first image and the Image quality evaluation information of each second image in the N second images
  • the image output unit 130 is further configured to:
  • the scores of the first image and the scores of the N second images from the first image and the N second images, determine that the image with the highest score is the image with the best image quality, and output The image with the highest score.
  • the acquiring unit includes:
  • RAW image acquisition unit for acquiring multiple frames of RAW images
  • the image fusion unit is used to perform image fusion processing on the multiple frames of RAW images to obtain a first image.
  • the fusion image corresponding to the RAW image can be used as the reference image, and the fusion image corresponding to each group of RAW images can be used to assist in evaluating the image quality of the enhanced image corresponding to the group of RAW images, which can improve the image quality evaluation information of the enhanced image. Accuracy.
  • the image fusion unit is specifically configured to:
  • the multi-frame RAW images are divided into at least two groups, and image fusion processing is performed on each group of RAW images to obtain at least two first images.
  • the target image is the second image; if the image quality evaluation information is not a predetermined number, or the score corresponding to the image quality evaluation information is less than the predetermined score threshold , The target image is the first image;
  • the image quality evaluation information is image quality evaluation information for the first image and the second image, and the image quality evaluation information is used to indicate whether the image quality of the second image is When the image quality of the first image is better than that of the first image, if the image quality evaluation information is a predetermined number, then the target image is the second image; if the image quality evaluation information is not a predetermined number, then the target The image is the first image;
  • the image quality evaluation information includes the image quality evaluation information corresponding to the first image and the second image
  • the image quality evaluation information of the first image is a predetermined number
  • the image quality evaluation information of the second image is a predetermined number
  • the target image is any one of the second image and the first image; if the image quality evaluation information of the first image is not predetermined If the image quality evaluation information of the second image is a predetermined number, the target image is the second image; if the image quality evaluation information of the first image is a predetermined number, and the second image If the image quality evaluation information of is not a predetermined number, the target image is the first image; or,
  • the target image is the second image; if the image quality of the second image is The score corresponding to the evaluation information is less than the score corresponding to the image quality evaluation information of the first image, then the target image is the first image; if the score corresponding to the image quality evaluation information of the second image is equal to the first image A score corresponding to the image quality evaluation information of an image, then the target image is any one of the second image and the first image;
  • the image quality evaluation information is the image quality evaluation information of the second image of each of the N second images, if there is image quality in the N second images If the evaluation information is a predetermined number of target second images, the target image is at least one of the target second images; if there is no target second image of which image quality evaluation information is a predetermined number among the N second images, Then the target image is the first image; or,
  • the target image is at least one of the target second images; if the N If there is no target second image whose score corresponding to the image quality evaluation information is greater than or equal to a predetermined score threshold in the second image, the target image is the first image;
  • the target image is the The image quality evaluation information in the first image and the N second images is any image in which the image quality evaluation information is a predetermined number, or the target image is corresponding to the image quality evaluation information in the first image and the N second images Any image with a score greater than or equal to a predetermined score threshold, or the target image is the image with the highest score corresponding to the image quality evaluation information among the first image and the N second images.
  • the image processing apparatus may be an electronic device, such as a mobile phone, or a chip in the electronic device, or a functional module integrated in the electronic device.
  • the chip or the functional module may be located in a control center (for example, a console) of the user terminal to control the user terminal to implement the image processing method provided in this application.
  • FIG. 27 shows a schematic block diagram of the structure of an image processing apparatus provided by another embodiment of the present application. part.
  • each unit included in the image processing apparatus is used to execute each step in the embodiment corresponding to FIG. 9 and FIG. 10.
  • the image processing device 2 may include:
  • the acquiring unit 210 is configured to acquire the first image
  • the first evaluation unit 220 is configured to input the first image into a quality evaluation model for processing to obtain first image quality evaluation information
  • the image processing unit 230 is configured to, when it is determined that the image quality of the first image meets the requirements according to the first image quality evaluation information, input the first image into M image enhancement models for processing to obtain N second images. Image, and display or save at least one or more of the N second images; wherein, N and M are positive integers, each of the M image enhancement models is different, and the N The second image is also different.
  • the electronic device acquires the first image in portrait mode, scenery mode, indoor mode, telephoto mode (hereinafter referred to as high-power zoom mode), and so on.
  • the first image can be a preview image or a captured photo.
  • the first image is displayed or saved.
  • the first image quality evaluation information is a number or a score.
  • the first image quality evaluation information when the first image quality evaluation information is a number, that the image quality of the first image meets the requirement means that the first image quality evaluation information is a predetermined number.
  • the first image quality evaluation information is a score
  • that the image quality of the first image meets the requirement means that the score of the first image is greater than or equal to a preset threshold.
  • the first image is obtained by the electronic device in a high-zoom photography mode.
  • the preset threshold value is 0.25, and both M and N are 1.
  • the image processing apparatus further includes:
  • the second evaluation unit is configured to input the N second images, or the first image and the N second images as input images after the N second images are obtained by the image processing unit 230
  • the quality evaluation model processes the to-be-processed image input quality evaluation model for processing to obtain the second image quality evaluation information
  • the output unit is configured to display or save a target image according to the second image quality evaluation information, where the target image is at least one of the first image and the N second images.
  • the second image quality evaluation information is a number or a score.
  • the target image is the first image and the N second images
  • the second image quality evaluation information is a predetermined digital image, or the second image quality An image whose score corresponding to the evaluation information is greater than a predetermined score threshold.
  • the image processing apparatus may be an electronic device, such as a mobile phone, or a chip in the electronic device, or a functional module integrated in the electronic device.
  • the chip or the functional module may be located in a control center (for example, a console) of the user terminal to control the user terminal to implement the image processing method provided in this application.
  • FIG. 28 is a schematic structural diagram of an electronic device provided by an embodiment of this application.
  • the electronic device 28 includes: at least one processor 310 (only one is shown in FIG. 28), and a memory 320 is stored in the memory 320 and can run on the at least one processor 310
  • the computer program 321 and the display device 330 may be a touch screen.
  • the processor 310 implements the steps in any of the foregoing image processing method embodiments when the computer program 321 is executed.
  • the electronic device 3 may include, but is not limited to, a processor 310, a memory 320, and a display device 330.
  • FIG. 28 is only an example of the electronic device 3, and does not constitute a limitation on the electronic device 3. It may include more or less components than those shown in the figure, or a combination of certain components, or different components. , For example, can also include input and output devices, network access devices, cameras, etc.
  • the so-called processor 310 may be a central processing unit (Central Processing Unit, CPU), and the processor 310 may also be other general-purpose processors, digital signal processors (Digital Signal Processors, DSPs), and application specific integrated circuits (Application Specific Integrated Circuits). , ASIC), ready-made programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc.
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the memory 320 may be an internal storage unit of the electronic device 3 in some embodiments, such as a hard disk or a memory of the electronic device 3. In some other embodiments, the memory 320 may also be an external storage device of the electronic device 3, such as a Smart Media Card (SMC), a Secure Digital (SD) card, and a flash memory card on the electronic device 3. Card) and so on. Further, the memory 320 may also include both an internal storage unit of the electronic device 3 and an external storage device.
  • the memory 320 is used to store an operating system, an application program, a boot loader (Boot Loader), data, and other programs, such as the program code of the computer program. The memory 320 can also be used to temporarily store data that has been output or will be output.
  • the disclosed device and method may be implemented in other ways.
  • the system embodiment described above is merely illustrative.
  • the division of the modules or units is only a logical function division.
  • there may be other division methods for example, multiple units or components may be It can be combined or integrated into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.
  • the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the computer program can be stored in a computer-readable storage medium.
  • the steps of the foregoing image processing method embodiments can be implemented.
  • the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file, or some intermediate forms.
  • the computer-readable medium may at least include: any entity or device capable of carrying computer program code to the electronic device 3, recording medium, computer memory, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), electric carrier signal, telecommunication signal and software distribution medium.
  • ROM read-only memory
  • RAM random access memory
  • electric carrier signal telecommunication signal and software distribution medium.
  • U disk mobile hard disk, floppy disk or CD-ROM, etc.
  • computer-readable media cannot be electrical carrier signals and telecommunication signals.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

本申请适用于通信技术领域,提供了一种图像处理方法及电子设备。该方法包括:获取待处理的第一图像,将第一图像输入图像增强模型进行处理,得到第二图像;将第二图像,或将第一图像和第二图像,输入质量评价模型进行处理,获得图像质量评价信息;根据获得的图像质量评价信息确定第一图像和第二图像中图像质量较好的目标图像,并输出目标图像。本方案中,由第一图像处理后得到的第二图像的图像质量变差时,电子设备最终输出第一图像给用户查看,可提高输出图像的质量,减少输出质量差的图像的概率,以提高用户体验。

Description

一种图像处理方法及电子设备
本申请要求于2020年03月13日提交国家知识产权局、申请号为202010179848.0、申请名称为“一种图像处理方法及电子设备”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
技术领域
本申请属于图像处理技术领域,尤其涉及图像处理方法及电子设备。
背景技术
由于拍摄环境、拍摄设备的硬件条件、图像压缩等因素的影响,拍摄到的图像往往存在视觉效果较差、分辨率较低、含有噪声或反光等情况,因此需要进行图像质量增强。当前智能手机等拍摄设备可以利用场景识别算法自动识别出图像的拍摄场景,进而使用该拍摄场景对应的图像增强算法或模型对图像进行处理,以增强图像质量。
然而,现有技术存在一个问题,当使用图像增强算法或模型对图像进行处理后,处理后的图像质量可能会更差,而智能手机依然会将处理后的图像作为最终的图像输出,严重影响了图像处理的效果及用户体验。
发明内容
本申请提供了一种图像处理方法及电子设备,可以解决电子设备最终输出的图像的图像质量比处理前的图像的图像质量更差的问题。
第一方面,本申请提供了一种图像处理方法,包括:
获取第一图像和N个第二图像,所述N个第二图像中的每一个第二图像是通过M个图像增强模型中的至少一个图像增强模型对所述第一图像进行处理后获得的;其中,N和M为大于零的整数,所述M个图像增强模型中的每个图像增强模型均不同,所述N个第二图像也存在不同;
将输入图像输入质量评价模型进行处理,获得图像质量评价信息,所述输入图像包括所述N个第二图像,或者包括所述第一图像以及所述N个第二图像;
根据所获得的所述图像质量评价信息输出目标图像,所述目标图像是所述第一图像和所述N个第二图像中的至少一个图像。
本实现方式中,由于采用图像增强模型对第一图像进行处理后得到的第二图像的图像质量可能比第一图像的图像质量更差,例如,第二图像中包括人脸图像,且人脸图像中的眼睛、鼻子、眼镜框等出现变形等情况、第二图像的边缘出现紫边、第二图像中存在叠影、色彩混叠、拉链效应等瑕疵,因此,通过质量评价模型评价第二图像的图像质量,或者评价第一图像和第二图像的图像质量,并根据评价结果最终从第一图像和第二图像中输出图像质量较好图像给用户查看。也就是说,处理后的第二图像的图像质量变差时,电子设备最终输出第一图像给用户查看,以解决现有技术中将经过图像增强模型处理后得到的图像质量变差的图像输出给用户的问题,能够提高输出图像的质量,可以减少输出质量差的图像的概率,以提高用户的视觉体验。
结合第一方面,在第一方面的第一种可能的实现方式中,所述质量评价模型是基于多个训练样本训练得到,每个训练样本包括样本图像以及用户对所述样本图像的图像质量评价信息。
本实现方式中,在训练质量评价模型的过程中所采用的样本图像,与应用训练后的质量评价模型评价图像质量时所采用的输入图像相对应。由于质量评价模型是采用多个样本图像训练得到,因此,采用训练后的质量评价模型评价输入图像的图像质量时,获得的图像质量评价结果的准确度较高。
结合第一方面或第一方面的第一种可能的实现方式,在第一方面的第二种可能的实现方式中,所述目标图像包括:
根据所获得的所述图像质量评价信息和评价规则确定的目标图像,所述评价规则是图像质量评价信息为预定数字,或,图像质量评价信息对应的分数大于或等于预定分值阈值。
在本实现方式中,由于当图像质量评价信息为预定数字,或者,图像质量评价信息对应的分数大于或等于预定分值阈值时,表示该图像质量评价信息对应的图像的图像质量较好,因此,根据图像质量评价信息和评价规则确定的目标图像是第一图像和N个第二图像中,图像质量较好的图像,电子设备可以输出图像质量较好的目标图像给用户查看,减少输出质量差的图像的概率。
结合第一方面或第一方面的上述任一种可能的实现方式,在第一方面的第三种可能的实现方式中,所述图像质量评价信息为数字或分数。
结合第一方面的第三种可能的实现方式,在第一方面的第四种可能的实现方式中,当所述图像质量评价信息为数字,且所述输入图像为N个第二图像,或者所述第一图像以及所述N个第二图像时,所述图像质量评价信息为对所述N个第二图像中每一个第二图像的图像质量评价信息;
相应的,根据所获得的所述图像质量评价信息输出目标图像,包括:
判断所述N个第二图像中是否存在图像质量评价信息为预定数字的目标第二图像;
若所述N个第二图像中存在图像质量评价信息为预定数字的目标第二图像,则确定所述图像质量评价信息为预定数字的目标第二图像为图像质量最好的图像,并输出所述图像质量评价信息为预定数字的目标第二图像。
在本实现方式中,电子设备可以通过质量评价模型对输入图像的图像质量进行二分类,从而判断输入图像是属于图像质量好的一类,还是属于图像质量差的一类。当输入图像是属于图像质量好的一类时,输出的图像质量评价信息为预定数字;当输入图像是属于图像质量差的一类时,输出的图像质量评价信息不是预定数字。对输入图像的图像质量进行二分类,计算开销较小,可以较快的得到图像质量评价结果。
结合第一方面的第四种可能的实现方式,在第一方面的第五种可能的实现方式中,在判断所述N个第二图像中是否存在图像质量评价信息为预定数字的目标第二图像之后,还包括:
若所述N个第二图像中不存在图像质量评价信息为预定数字的目标第二图像,则确定所述第一图像为图像质量最好的图像,输出所述第一图像。
结合第一方面的第四种可能的实现方式,在第一方面的第六种可能的实现方式中, 所述根据所获得的所述图像质量评价信息输出目标图像,还包括:
若所述N个第二图像中存在多个图像质量评价信息为预定数字的目标第二图像,则选择任一目标第二图像输出。
结合第一方面的第三种可能的实现方式,在第一方面的第七种可能的实现方式中,若所述图像质量评价信息为分数,且所述输入图像为N个第二图像,或者所述第一图像以及所述N个第二图像时,所述图像质量评价信息为对所述N个第二图像中每一个第二图像的图像质量评价信息;
相应的,根据所获得的所述图像质量评价信息输出目标图像,包括:
判断所述N个第二图像中是否存在分数大于预定分数阈值的目标第二图像;
若所述N个第二图像中存在分数大于预定分数阈值的目标第二图像,则确定所述分数最高的目标第二图像为图像质量最好的图像,并输出所述分数最高的目标第二图像。
在本实现方式中,可以通过第一图像的特征信息辅助评价第二图像的图像质量,由于电子设备可以将第一图像作为参考图像,获取第二图像的特征信息于第一图像的特征信息之间的差异特征信息,根据差异特征信息评价第二图像的图像质量,可以提高第二图像的图像质量评价信息的准确度。另外,图像质量评价信息通过分数来表示,可以更准确地描述图像质量的好坏程度。电子设备可以通过每个第二图像对应的分数,确定N个第二图像中图像质量最好的目标图像,从而输出图像质量最好的图像给用户查看。
结合第一方面的第七种可能的实现方式,在第一方面的第八种可能的实现方式中,在判断所述N个第二图像中是否存在分数大于预定分数阈值的目标第二图像之后,还包括:
若所述N个第二图像中不存在分数大于预定分数阈值的目标第二图像,则确定所述第一图像为图像质量最好的图像,输出所述第一图像。
结合第一方面的第三种可能的实现方式,在第一方面的第九种可能的实现方式中,所述根据所获得的所述图像质量评价信息输出目标图像,包括:
若所述N个第二图像中不存在图像质量评价信息为预定数字的目标第二图像,或者所述N个第二图像中不存在分数大于预定分数阈值的目标第二图像,则获取新的图像增强模型,采用获取的所述新的图像增强模型对所述第一图像进行处理,获得新的第二图像,并将所述新的第二图像作为输入图像输入所述质量评价模型进行处理,获得新的图像质量评价信息;其中,所述新的图像增强模型为未对所述第一图像进行处理过的图像增强模型;
若确定所述新的图像质量评价信息为所述预定数字,或者所述分数大于所述预定分数阈值,则输出新的第二图像,否则返回执行获取新的图像增强模型步骤以及后续步骤,直到返回执行的次数达到预设的次数阈值,输出所述第一图像。
在本实现方式中,电子设备可以优先采用图像处理效果最优的图像增强模型1对第一图像进行处理,如果处理后的图像的图像质量较差,那么采用图像处理效果次优的图像增强模型2对第一图像进行处理,如果处理后的图像的图像质量还是较差,再从可选的图像增强模型中选择最优的图像增强模型3对第一图像进行处理,可选的图 像增强模型是指除前面已使用的图像增强模型(如图像处理效果最优以及次优的图像增强模型)之外的图像增强模型。由于优先采用图像处理效果最优的图像增强模型对第一图像进行处理,有些情况不需要使用到N个图像增强模型,就可以获得图像质量较好的第二图像。相对于采用N个图像增强模型对第一图像并行处理的情况而言,可以节省部分系统资源,可以缩短获取到图像质量较好的第二图像所需的时间,以提高输出图像的效率。
结合第一方面的第三种可能的实现方式,在第一方面的第十种可能的实现方式中,当所述图像质量评价信息为数字,且所述输入图像为第一图像和N个第二图像时,所述图像质量评价信息为对所述第一图像和N个第二图像中每一个第二图像的图像质量评价信息;
相应的,根据所获得的所述图像质量评价信息输出目标图像,包括:
判断所述第一图像和N个第二图像中是否存在图像质量评价信息为预定数字的图像;
若所述第一图像的图像质量评价信息为预定数字,且所述N个第二图像中存在至少一个图像质量评价信息为预定数字的目标第二图像,则从所述第一图像和所述目标第二图像中选择任一图像输出。
在本实现方式中,可以评价第一图像和第二图像的图像质量,得到针对第一图像以及第二图像的图像质量评价信息,可以通过图像质量评价信息,来判断第二图像的图像质量是否比第一图像的图像质量好,能够更准确地获知由第一图像经过图像增强模型处理后得到的第二图像的图像质量是否变差。
结合第一方面的第十种可能的实现方式,在第一方面的第十一种可能的实现方式中,在判断所述第一图像和N个第二图像中是否存在图像质量评价信息为预定数字的目标第二图像之后,还包括:
若所述第一图像的图像质量评价信息不是预定数字,且所述N个第二图像中存在至少一个图像质量评价信息为预定数字的目标第二图像,则从所述目标第二图像中选择任一图像输出。
结合第一方面的第十种可能的实现方式,在第一方面的第十二种可能的实现方式中,在判断所述第一图像和N个第二图像中是否存在图像质量评价信息为预定数字的目标第二图像之后,还包括:
若所述第一图像的图像质量评价信息不是预定数字,且所述N个第二图像中不存在图像质量评价信息为所述预定数字的目标第二图像,则输出所述第一图像。
结合第一方面的第三种可能的实现方式,在第一方面的第十三种可能的实现方式中,当所述图像质量评价信息为分数,且所述输入图像为第一图像和N个第二图像时,所述图像质量评价信息为对所述第一图像和N个第二图像中每一个第二图像的图像质量评价信息;
相应的,根据所获得的所述图像质量评价信息输出目标图像,包括:
根据第一图像的分数和所述N个第二图像的分数,从所述第一图像和所述N个第二图像中,确定分数最高的图像为所述图像质量最好的图像,并输出所述分数最高的图像。
本实现方式,电子设备可以直接比较第一图像和N个第二图像各自对应的分数,筛选出分数最高的图像,进而确定图像质量最好的图像,这样可以更准确的确定图像质量最好的图像。
结合第一方面,或者,第一方面的第一种至第八种、第十种至第十二种任一种可能的实现方式,在第一方面的第十四种可能的实现方式中,所述获取第一图像包括:
获取多帧RAW图像;
对所述多帧RAW图像进行图像融合处理,得到第一图像。
结合第一方面的第十四种可能的实现方式,在第一方面的第十五种可能的实现方式中,所述对所述多帧RAW图像进行图像融合处理,得到第一图像,包括:
将所述多帧RAW图像划分为至少两组,对每组RAW图像进行图像融合处理得到至少两个第一图像。
本实现方式,可以对获取到的RAW图像进行图像融合处理得到第一图像,采用图像增强模型对第一图像进行处理后得到的第二图像,通过质量评价模型评价第二图像的图像质量,或者评价第一图像和第二图像的图像质量,并根据评价结果最终从第一图像和第二图像中输出图像质量较好图像给用户查看,能够提高输出的图像的图像质量。
结合第一方面,在第一方面的第十六种可能的实现方式中,
当N=M=1,所述图像质量评价信息为所述第二图像的图像质量评价信息时,若所述图像质量评价信息为预定数字,或者所述图像质量评价信息对应的分数大于或等于预定分数阈值,则所述目标图像为所述第二图像;若所述图像质量评价信息不是预定数字,或者所述图像质量评价信息对应的分数小于预定分数阈值,则所述目标图像为所述第一图像;
当N=M=1,所述图像质量评价信息为针对所述第一图像和所述第二图像的图像质量评价信息,所述图像质量评价信息用于表示所述第二图像的图像质量是否比所述第一图像的图像质量好时,若所述图像质量评价信息为预定数字,则所述目标图像为所述第二图像;若所述图像质量评价信息不是预定数字,则所述目标图像为所述第一图像;
当N=M=1,所述图像质量评价信息包括所述第一图像和所述第二图像各自对应的图像质量评价信息时,若所述第一图像的图像质量评价信息为预定数字,且所述第二图像的图像质量评价信息为预定数字,则所述目标图像为所述第二图像和所述第一图像中的任一图像;若所述第一图像的图像质量评价信息不是预定数字,且所述第二图像的图像质量评价信息为预定数字,则所述目标图像为所述第二图像;若所述第一图像的图像质量评价信息为预定数字,且所述第二图像的图像质量评价信息不是预定数字,则所述目标图像为所述第一图像;或者,
若所述第二图像的图像质量评价信息对应的分数大于所述第一图像的图像质量评价信息对应的分数,则所述目标图像为所述第二图像;若所述第二图像的图像质量评价信息对应的分数小于所述第一图像的图像质量评价信息对应的分数,则所述目标图像为所述第一图像;若所述第二图像的图像质量评价信息对应的分数等于所述第一图像的图像质量评价信息对应的分数,则所述目标图像为所述第二图像和所述第一图像 中的任一图像;
当N=M≥2,所述图像质量评价信息为所述N个第二图像中每一个第二图像的第二图像的图像质量评价信息时,若所述N个第二图像中存在图像质量评价信息为预定数字的目标第二图像,则所述目标图像为至少一个所述目标第二图像;若所述N个第二图像中不存在图像质量评价信息为预定数字的目标第二图像,则所述目标图像为所述第一图像;或者,
若所述N个第二图像中存在图像质量评价信息对应的分数大于或等于预定分值阈值的目标第二图像,则所述目标图像为至少一个所述目标第二图像;若所述N个第二图像中不存在图像质量评价信息对应的分数大于或等于预定分值阈值的目标第二图像,则所述目标图像为所述第一图像;
当N=M≥2,所述图像质量评价信息为针对第一图像和所述N个第二图像中每一个第二图像的第二图像的图像质量评价信息时,所述目标图像为所述第一图像和所述N个第二图像中图像质量评价信息为预定数字的任一图像,或者,所述目标图像为所述第一图像和所述N个第二图像中图像质量评价信息对应的分数大于或等于预定分值阈值的任一图像,或者,所述目标图像为所述第一图像和所述N个第二图像中图像质量评价信息对应的分数最高的图像。
本实现方式,可以通过预定数字或预定分数阈值,从第一图像和N个第二图像中筛选出图像质量较好的图像进行输出。
第二方面,本申请提供了一种图像处理方法,包括:
获取第一图像;
将所述第一图像输入质量评价模型进行处理,得到第一图像质量评价信息;
当根据所述第一图像质量评价信息确定所述第一图像的图像质量符合要求时,将所述第一图像输入M个图像增强模型进行处理,得到N个第二图像,并显示或保存所述N个第二图像中的一个或者多个;其中,N和M为正整数,所述M个图像增强模型中的每个图像增强模型均不同,所述N个第二图像也存在不同。
在本实现方式中,先采用图像质量评价模型评价获取到的第一图像的图像质量,根据评价结果确定是否对第一图像进行处理。其中,当第一图像的图像质量较好时,采用图像增强模型对第一图像进行处理,输出第二图像;当第一图像的图像质量较差时,输出第一图像给用户查看,以减少因处理图像所占用的资源的消耗,可以提高手机的数据处理速度。
结合第二方面,在第二方面的第一种可能的实现方式中,当根据所述第一图像质量评价信息确定所述第一图像的图像质量不符合要求时,显示或保存所述第一图像。
结合第二方面或第二方面的第一种可能的实现方式,在第二方面的第二种可能的实现方式中,所述第一图像质量评价信息为数字或分数。
结合第二方面的第二种可能的实现方式,在第二方面的第三种可能的实现方式中,当所述第一图像质量评价信息为数字时,所述第一图像的图像质量符合要求是指所述第一图像质量评价信息为预定数字。
在本实现方式中,电子设备可以通过质量评价模型对第一图像的图像质量进行二分类,从而判断第一图像是属于图像质量好的一类,还是属于图像质量差的一类。当第一图像是属于图像质量好的一类时,输出的图像质量评价信息为预定数字;当第一图像是属于图像质量差的一类时,输出的图像质量评价信息不是预定数字。对第一图像的图像质量进行二分类,计算开销较小,可以较快的得到第一图像的图像质量评价结果。
结合第二方面的第二种可能的实现方式,在第二方面的第四种可能的实现方式中,当所述第一图像质量评价信息为分数时,所述第一图像的图像质量符合要求是指所述第一图像的分数大于或等于预设阈值。
结合第二方面或第二方面的上述任一种可能的实现方式,在第二方面的第五种可能的实现方式中,所述第一图像由电子设备在高倍变焦拍照模式下获取得到。
本实现方式中,由于电子设备在高倍变焦模式下拍摄人像照片时,获取到的第一图像,可能会因抖动而导致图像失真、图像中的人像的眼睛等五官变形或者图像较模糊(例如,较难辨认被拍摄对象)等,因此,需要评价第一图像的图像质量。如果第一图像的图像质量较差,那么将第一图像显示给用户查看,或将第一图像保存至图库;如果第一图像的图像质量较好,那么对第一图像进行处理得到N个第二图像,并显示或保存N个第二图像中的至少一个图像。通过节省因处理图像质量差的第一图像而消耗的资源,可以提高电子设备的数据处理速度。
结合第二方面或第二方面的上述任一种可能的实现方式,在第二方面的第六种可能的实现方式中,所述预设阈值为0.25,M和N均为1。
结合第二方面或第二方面的第一种至第四种任一可能的实现方式,在第二方面的第六种可能的实现方式中,所述得到所述N个第二图像之后,还包括:
将所述N个第二图像,或者所述第一图像以及所述N个第二图像作为输入图像输入所述质量评价模型进行处理待处理图像输入质量评价模型进行处理,得到第二图像质量评价信息;
根据所述第二图像质量评价信息输出目标图像,所述目标图像是所述第一图像和所述N个第二图像中的至少一个图像。
结合第二方面的第六种可能的实现方式,在第二方面的第七种可能的实现方式中,所述第二图像质量评价信息为数字或分数。
结合第二方面的第七种可能的实现方式,在第二方面的第八种可能的实现方式中,所述目标图像是所述第一图像和所述N个第二图像中,所述第二图像质量评价信息为预定数字的图像,或者所述第二图像质量评价信息对应的分数大于预定分数阈值的图像。
第三方面,本申请提供了一种图像处理方法,包括:
获取第一图像和N个第二图像,所述N个第二图像中的每一个第二图像是通过M个图像增强模型中的至少一个图像增强模型对所述第一图像进行处理后获得的;其中, N和M为大于零的整数,所述M个图像增强模型中的每个图像增强模型均不同,所述N个第二图像也存在不同;
将输入图像输入质量评价模型进行处理,获得图像质量评价信息,所述输入图像包括所述N个第二图像,或者包括所述第一图像以及所述N个第二图像;
根据所获得的所述图像质量评价信息输出目标图像,所述目标图像是所述第一图像和所述N个第二图像中的至少一个图像。
其中,一个图像增强模型对第一图像进程处理后,可以得到一个第二图像,也可以得到至少两个第二图像,此处不做限制。也就是说,M可以与N相等,M也可以大于N。
图像质量评价信息可以是针对N个第二图像的图像质量评价信息,也可以是针对第一图像和N个第二图像的图像质量评价信息。其中,当质量评价模型的输入图像为N个第二图像时,质量评价模型用于评价N个第二图像的图像质量,获得的图像质量评价信息是N个第二图像的图像质量评价信息。当质量评价模型的输入图像为第一图像和N个第二图像时,质量评价模型可以用于通过第一图像辅助评价N个第二图像的图像质量图像质量,获得的图像质量评价信息是N个第二图像的图像质量评价信息;通过第一图像辅助评价第二图像的图像质量,可以提高第二图像的评价结果的准确度。当质量评价模型的输入图像为第一图像和N个第二图像时,质量评价模型还可以用于评价第一图像和N个第二图像的图像质量,获得的图像质量评价信息是针对第一图像和N个第二图像的图像质量评价信息;电子设备可以通过图像质量评价信息,来判断第二图像的图像质量是否比第一图像的图像质量好,能够更准确地获知由第一图像经过图像增强模型处理后得到的第二图像的图像质量是否变差。
目标图像可以为第一图像和N个第二图像中图像质量最好的图像,图像质量最好的图像是指第一图像和N个第二图像中,图像质量评价信息为预定数字的图像,或者图像质量评价信息对应的分数大于预定分数阈值的图像。
输出目标图像的方式可以是显示目标图像,也可以是将目标图像保存至图库。
本实现方式中,由于采用图像增强模型对第一图像进行处理后得到的第二图像的图像质量可能比第一图像的图像质量更差,例如,第二图像中包括人脸图像,且人脸图像中的眼睛、鼻子、眼镜框等出现变形等情况、第二图像的边缘出现紫边、第二图像中存在叠影、色彩混叠、拉链效应等瑕疵,因此,通过质量评价模型评价第二图像的图像质量,或者评价第一图像和第二图像的图像质量,并根据评价结果最终从第一图像和第二图像中输出图像质量较好图像给用户查看。也就是说,处理后的第二图像的图像质量变差时,电子设备最终输出第一图像给用户查看,以解决现有技术中将经过图像增强模型处理后得到的图像质量变差的图像输出给用户的问题,能够提高输出图像的质量,可以减少输出质量差的图像的概率,以提高用户的视觉体验。
在第三方面的一种可能的实现方式中,所述质量评价模型是基于多个训练样本训练得到,每个训练样本包括样本图像以及用户对所述样本图像的图像质量评价信息。
需要说明的是,当质量评价模型的输入图像为第二图像,质量评价模型用于评价第二图像的图像质量时,质量评价模型的训练样本可以包括图像增强模型对原始图像进行处理后输出的样本图像、以及用户对样本图像标记的图像质量评价信息。
当质量评价模型的输入图像为第一图像和第二图像,质量评价模型可以用于通过第一图像辅助评价第二图像的图像质量图像质量,以输出第二图像的图像质量评价信息时,质量评价模型的训练样本可以包括原始图像、图像增强模型对原始图像进行处理后输出的样本图像、以及用户对样本图像标记的图像质量评价信息。
当质量评价模型的输入图像为第一图像和第二图像时,质量评价模型用于评价第一图像和第二图像的图像质量时,质量评价模型的训练样本可以包括原始图像、图像增强模型对原始图像进行处理后输出的样本图像、用户对原始图像标记的图像质量评价信息以及用户对样本图像标记的图像质量评价信息。
本实现方式中,在训练质量评价模型的过程中所采用的样本图像,与应用训练后的质量评价模型评价图像质量时所采用的输入图像相对应。由于质量评价模型是采用多个样本图像训练得到,因此,采用训练后的质量评价模型评价输入图像的图像质量时,获得的图像质量评价结果的准确度较高。
在第三方面的一种可能的实现方式中,所述目标图像包括根据所获得的所述图像质量评价信息和评价规则确定的目标图像,所述评价规则是所述图像质量评价信息为预定数字,或,图像质量评价信息对应的分数大于或等于预定分值阈值。
在本实现方式中,由于当图像质量评价信息为预定数字,或者,图像质量评价信息对应的分数大于或等于预定分值阈值时,表示该图像质量评价信息对应的图像的图像质量较好,因此,根据图像质量评价信息和评价规则确定的目标图像是第一图像和N个第二图像中,图像质量较好的图像,电子设备可以输出图像质量较好的目标图像给用户查看,减少输出质量差的图像的概率。
在第三方面的一种可能的实现方式中,图像质量评价信息可以为用于表示图像质量的数字或分数。
需要说明的是,图像质量评价信息还可以通过字母或文字来表示,此处不做限制。例如,图像质量评价信息可以是“0”或“1”、“YES”或“NO”,“是”或“否”。
示例性的,本实施例中的图像处理方法可以包括以下三种方案,下面以第一图像为图像A,第二图像为图像B,N=M=1为例进行说明。
具体的,方案一,当质量评价模型的输入图像为图像B时,质量评价模型用于评价图像B的图像质量,若电子设备根据质量评价模型输出的图像质量评价信息确定图像B的图像质量最好,则输出图像B,否则输出图像A。
方案二,当质量评价模型的输入图像为图像A和图像B时,质量评价模型可以用于通过图像A辅助评价图像B的图像质量图像质量,若电子设备根据质量评价模型输出的图像质量评价信息确定图像B的图像质量最好,则输出图像B,否则输出图像A。
方案三,当质量评价模型的输入图像为图像A和图像B时,质量评价模型用于评价图像A和图像B的图像质量,若电子设备根据质量评价模型输出的图像质量评价信息确定图像B的图像质量比图像A的图像质量好,则输出图像B;若电子设备根据质量评价模型输出的图像质量评价信息确定图像B的图像质量比图像A的图像质量差,则输出图像A;若电子设备根据质量评价模型输出的图像质量评价信息确定图像B的图像质量与图像A的图像质量相同,则从图像A和图像B中选择任一图像输出。
示例性的,图像质量评价信息可以通过标识信息表示,标识信息可以是数字、字 母、文字等,电子设备可以根据预先建立的对应关系或规则确定标识信息中的数字、文字或字母所表示的含义,从而输出图像A或图像B。例如,针对方案一和方案二,用“0”表示图像B的图像质量差,用“1”表示图像B的图像质量好。当图像质量评价信息为1时,电子设备确定图像质量评价信息对应的图像质量评价结果为图像B的图像质量最好,图像B为目标图像,输出图像B;当图像质量评价信息为0时,电子设备确定图像质量评价信息对应的图像质量评价结果为图像A的图像质量最好,图像A为目标图像,输出图像A。针对方案三,用“0”表示图像B的图像质量比图像A的图像质量差,用“1”表示图像B的图像质量比图像A的图像质量好,其它数字表示图像B的图像质量与图像A的图像质量相同。当图像质量评价信息为1时,电子设备根据图像质量评价信息确定图像B的图像质量比图像A的图像质量好,图像B为目标图像,输出图像B;当图像质量评价信息为0时,电子设备根据图像质量评价信息确定图像B的图像质量比图像A的图像质量差,图像A为目标图像,输出图像A;当图像质量评价信息为2时,电子设备根据图像质量评价信息确定图像B的图像质量与图像A的图像质量相同,图像A和图像B均为目标图像,可以从图像A和图像B中选择任一图像输出。
示例性的,图像质量评价信息可以通过分数表示。例如,针对方案一和方案二,图像质量评价信息是图像B对应的分数,电子设备将图像B对应的分数与预定分数阈值进行比较,当图像B对应的分数小于或等于预定分数阈值时,表示图像B的图像质量差,图像A为目标图像,电子设备输出图像A,当图像B对应的分数大于预定分数阈值时,表示图像B的图像质量好,图像B为目标图像,电子设备输出图像B。针对方案三,图像质量评价信息包括图像A的分数A和图像B的分数B,电子设备将图像B对应的分数A与图像B对应的分数B进行比较,当分数B大于分数A时,表示图像B的图像质量比图像A的图像质量好,图像B为目标图像,电子设备输出图像B;当分数B小于分数A时,表示图像B的图像质量比图像A的图像质量差,图像A为目标图像,电子设备输出图像A,当分数B等于分数A时,表示图像B的图像质量与图像A的图像质量相同,图像A和图像B均为目标图像,电子设备从图像A和图像B中选择任一图像输出。
在实现方式中,图像质量评价信息可以通过数字“0”或“1”来表示,或者,图像质量评价信息可以通过分数来表示,图像质量评价信息表示方式较灵活,可以直观的体现图像质量。
在第三方面的一种可能的实现方式中,当所述图像质量评价信息为数字,且所述输入图像为N个第二图像,或者所述第一图像以及所述N个第二图像时,所述图像质量评价信息为对所述N个第二图像中每一个第二图像的图像质量评价信息;
相应的,根据所获得的所述图像质量评价信息输出目标图像,包括:
判断所述N个第二图像中是否存在图像质量评价信息为预定数字的目标第二图像;
若所述N个第二图像中存在图像质量评价信息为预定数字的目标第二图像,则确定所述图像质量评价信息为预定数字的目标第二图像为图像质量最好的图像,并输出所述图像质量评价信息为预定数字的目标第二图像。
其中,预定数字可以为“1”。
在本实现方式中,电子设备可以通过质量评价模型对输入图像的图像质量进行二分类,从而判断输入图像是属于图像质量好的一类,还是属于图像质量差的一类。当输入图像是属于图像质量好的一类时,输出的图像质量评价信息为预定数字;当输入图像是属于图像质量差的一类时,输出的图像质量评价信息不是预定数字。对输入图像的图像质量进行二分类,计算开销较小,可以较快的得到图像质量评价结果。
在第三方面的一种可能的实现方式中,在判断所述N个第二图像中是否存在图像质量评价信息为预定数字的目标第二图像之后,还包括:
若所述N个第二图像中不存在图像质量评价信息为预定数字的目标第二图像,则确定所述第一图像为图像质量最好的图像,输出所述第一图像。
在本实现方式中,N个第二图像的图像质量评价信息都不是预定数字,说明N个第二图像的图像质量均比第一图像的图像质量差,此时输出第一图像给用户查看。也就是说,第一图像经过图像增强模型处理后得到的N个第二图像的图像质量变差时,可以输出图像质量较好的第一图像给用户查看,提高输出的图像的图像质量。在第一方面的一种可能的实现方式中,所述根据所述图像质量评价信息确定所述第一图像和所述N个第二图像中图像质量最好的图像,并输出所述质量最好的图像,还包括:
若所述N个第二图像中存在多个图像质量评价信息为预定数字的目标第二图像,则选择任一目标第二图像输出。
在本实现方式中,当N个第二图像中存在至少两个图像质量较好的图像时,可以输出任一个图像质量较好的图像。
在第三方面的一种可能的实现方式中,若所述图像质量评价信息为分数,且所述输入图像为N个第二图像,或者所述第一图像以及所述N个第二图像时,所述图像质量评价信息为对所述N个第二图像中每一个第二图像的图像质量评价信息;
相应的,根据所获得的所述图像质量评价信息输出目标图像,包括:
判断所述N个第二图像中是否存在分数大于预定分数阈值的目标第二图像;
若所述N个第二图像中存在分数大于预定分数阈值的目标第二图像,则确定所述分数最高的目标第二图像为图像质量最好的图像,并输出所述分数最高的目标第二图像。
在本实现方式中,可以通过第一图像的特征信息辅助评价第二图像的图像质量,由于电子设备可以将第一图像作为参考图像,获取第二图像的特征信息于第一图像的特征信息之间的差异特征信息,根据差异特征信息评价第二图像的图像质量,可以提高第二图像的图像质量评价信息的准确度。另外,图像质量评价信息通过分数来表示,可以更准确地描述图像质量的好坏程度。电子设备可以通过每个第二图像对应的分数,确定N个第二图像中图像质量最好的目标图像,从而输出图像质量最好的图像给用户查看。
在第三方面的一种可能的实现方式中,在判断所述N个第二图像中是否存在分数大于预定分数阈值的目标第二图像之后,还包括:
若所述N个第二图像中不存在分数大于预定分数阈值的目标第二图像,则确定所述第一图像为图像质量最好的图像,输出所述第一图像。
在本实现方式中,由于分数大于预定分数阈值的图像为图像质量较好的图像,当 N个第二图像中不存在分数大于预定分数阈值的目标第二图像时,说明N个第二图像的图像质量比第一图像的图像质量差,此时,电子设备输出第一图像给用户查看。
在第三方面的一种可能的实现方式中,所述根据所获得的所述图像质量评价信息输出目标图像,包括:
若所述N个第二图像中不存在图像质量评价信息为预定数字的目标第二图像,或者所述N个第二图像中不存在分数大于预定分数阈值的目标第二图像,则获取新的图像增强模型,采用获取的所述新的图像增强模型对所述第一图像进行处理,获得新的第二图像,并将所述新的第二图像作为输入图像输入所述质量评价模型进行处理,获得新的图像质量评价信息;其中,所述新的图像增强模型为未对所述第一图像进行处理过的图像增强模型;
若确定所述新的图像质量评价信息为所述预定数字,或者所述分数大于所述预定分数阈值,则输出新的第二图像,否则返回执行获取新的图像增强模型步骤以及后续步骤,直到返回执行的次数达到预设的次数阈值,输出所述第一图像。
预设的次数阈值根据电子设备内的图像增强模型的总数量设置。预设的次数阈值可以等于图像增强模型的总数量减一。例如,当图像增强模型的总数量为3时,预设的次数阈值可以为2。
在本实现方式中,电子设备可以优先采用图像处理效果最优的图像增强模型1对第一图像进行处理,如果处理后的图像的图像质量较差,那么采用图像处理效果次优的图像增强模型2对第一图像进行处理,如果处理后的图像的图像质量还是较差,再从可选的图像增强模型中选择最优的图像增强模型3对第一图像进行处理,可选的图像增强模型是指除前面已使用的图像增强模型(如图像处理效果最优以及次优的图像增强模型)之外的图像增强模型。由于优先采用图像处理效果最优的图像增强模型对第一图像进行处理,有些情况不需要使用到N个图像增强模型,就可以获得图像质量较好的第二图像。相对于采用N个图像增强模型对第一图像并行处理的情况而言,可以节省部分系统资源,可以缩短获取到图像质量较好的第二图像所需的时间,以提高输出图像的效率。
在第三方面的一种可能的实现方式中,当所述图像质量评价信息为数字,且所述输入图像为第一图像和N个第二图像时,所述图像质量评价信息为对所述第一图像和N个第二图像中每一个第二图像的图像质量评价信息;
相应的,根据所获得的所述图像质量评价信息输出目标图像,包括:
判断所述第一图像和N个第二图像中是否存在图像质量评价信息为预定数字的图像;
若所述第一图像的图像质量评价信息为预定数字,且所述N个第二图像中存在至少一个图像质量评价信息为预定数字的目标第二图像,则从所述第一图像和所述目标第二图像中选择任一图像输出。
在本实现方式中,可以评价第一图像和第二图像的图像质量,得到针对第一图像以及第二图像的图像质量评价信息,可以通过图像质量评价信息,来判断第二图像的图像质量是否比第一图像的图像质量好,能够更准确地获知由第一图像经过图像增强模型处理后得到的第二图像的图像质量是否变差。
在第三方面的一种可能的实现方式中,在判断所述第一图像和N个第二图像中是否存在图像质量评价信息为预定数字的目标第二图像之后,还包括:
若所述第一图像的图像质量评价信息不是预定数字,且所述N个第二图像中存在至少一个图像质量评价信息为预定数字的目标第二图像,则从所述目标第二图像中选择任一图像输出。
本实现方式中,由于图像质量评价信息为预定数字,表示图像质量较好,因此,当第一图像的图像质量评价信息不是预定数字,且N个第二图像中存在至少一个图像质量评价信息为预定数字的目标第二图像时,说明,第一图像的图像质量较差,并且N个第二图像中存在至少一个图像质量较好的目标第二图像,此时,电子设备可以输出任一个目标第二图像给用户查看。
在第三方面的一种可能的实现方式中,在判断所述第一图像和N个第二图像中是否存在图像质量评价信息为预定数字的目标第二图像之后,还包括:
若所述第一图像的图像质量评价信息不是预定数字,且所述N个第二图像中不存在图像质量评价信息为所述预定数字的目标第二图像,则输出所述第一图像。
本实现方式中,N个第二图像中不存在图像质量评价信息为预定数字的目标第二图像,说明,由第一图像处理得到的N个第二图像的图像质量变差了,此时,电子设备输出第一图像给用户查看。
在第三方面的一种可能的实现方式中,当所述图像质量评价信息为分数,且所述输入图像为第一图像和N个第二图像时,所述图像质量评价信息为对所述第一图像和N个第二图像中每一个第二图像的图像质量评价信息;
相应的,根据所获得的所述图像质量评价信息输出目标图像,包括:
根据第一图像的分数和所述N个第二图像的分数,从所述第一图像和所述N个第二图像中,确定分数最高的图像为所述图像质量最好的图像,并输出所述分数最高的图像。
由于分数越高表示图像质量越好,因此,第一图像和N个第二图像中分数最高的图像,即为图像质量最好的图像。
本实现方式,电子设备可以直接比较第一图像和N个第二图像各自对应的分数,筛选出分数最高的图像,进而确定图像质量最好的图像,这样可以更准确的确定图像质量最好的图像。
在第三方面的一种可能的实现方式中,所述获取第一图像包括:
获取多帧RAW图像;
对所述多帧RAW图像进行图像融合处理,得到第一图像。
RAW图像是RAW格式的图像。RAW是未经处理、也未经压缩的格式,可以把RAW概念化为“原始图像编码数据”或更形象的称为“数字底片”。可以理解为:RAW图像就是图像传感器将捕捉到的光源信号转化为数字信号的原始数据。图像传感器可以包括互补金属氧化物半导体(Complementary Metal-Oxide-Semiconductor,CMOS)图像传感器,电荷耦合器件(Charge Coupled Device,CCD)图像传感器。
本实现方式,可以对获取到的RAW图像进行图像融合处理得到第一图像,采用图像增强模型对第一图像进行处理后得到的第二图像,通过质量评价模型评价第二图 像的图像质量,或者评价第一图像和第二图像的图像质量,并根据评价结果最终从第一图像和第二图像中输出图像质量较好图像给用户查看,能够提高输出的图像的图像质量。
在第三方面的一种可能的实现方式中,所述对所述多帧RAW图像进行图像融合处理,得到第一图像,包括:
将所述多帧RAW图像划分为至少两组,对每组RAW图像进行图像融合处理得到至少两个第一图像。
其中,一组RAW图像对应一个第一图像。
本实施例中,可以将多帧RAW图像进行分组,并对分组后的RAW图像进行图像融合处理得到第一图像,并增强融合图像的图像质量得到第二图像,之后,评价第二图像的图像质量,或者评价第一图像的图像质量以及第二图像的图像质量,根据评价结果输出图像质量最好的目标图像,可以拍出图像质量较好的照片。目标图像可以是第二图像,或者第一图像。由于手机始终输出图像质量较好的图像,因此,手机中存储图像质量较差的图像的可能性较低,用户查看到图像质量较差的图像的可能性变小,可以提高用户视觉体验。
在第三方面的一种可能的实现方式中,当N=M=1,所述图像质量评价信息为所述第二图像的图像质量评价信息时,若所述图像质量评价信息为预定数字,或者所述图像质量评价信息对应的分数大于或等于预定分数阈值,则所述目标图像为所述第二图像;若所述图像质量评价信息不是预定数字,或者所述图像质量评价信息对应的分数小于预定分数阈值,则所述目标图像为所述第一图像;
当N=M=1,所述图像质量评价信息为针对所述第一图像和所述第二图像的图像质量评价信息,所述图像质量评价信息用于表示所述第二图像的图像质量是否比所述第一图像的图像质量好时,若所述图像质量评价信息为预定数字,则所述目标图像为所述第二图像;若所述图像质量评价信息不是预定数字,则所述目标图像为所述第一图像;
当N=M=1,所述图像质量评价信息包括所述第一图像和所述第二图像各自对应的图像质量评价信息时,若所述第一图像的图像质量评价信息为预定数字,且所述第二图像的图像质量评价信息为预定数字,则所述目标图像为所述第二图像和所述第一图像中的任一图像;若所述第一图像的图像质量评价信息不是预定数字,且所述第二图像的图像质量评价信息为预定数字,则所述目标图像为所述第二图像;若所述第一图像的图像质量评价信息为预定数字,且所述第二图像的图像质量评价信息不是预定数字,则所述目标图像为所述第一图像;或者,
若所述第二图像的图像质量评价信息对应的分数大于所述第一图像的图像质量评价信息对应的分数,则所述目标图像为所述第二图像;若所述第二图像的图像质量评价信息对应的分数小于所述第一图像的图像质量评价信息对应的分数,则所述目标图像为所述第一图像;若所述第二图像的图像质量评价信息对应的分数等于所述第一图像的图像质量评价信息对应的分数,则所述目标图像为所述第二图像和所述第一图像中的任一图像;
当N=M≥2,所述图像质量评价信息为所述N个第二图像中每一个第二图像的第 二图像的图像质量评价信息时,若所述N个第二图像中存在图像质量评价信息为预定数字的目标第二图像,则所述目标图像为至少一个所述目标第二图像;若所述N个第二图像中不存在图像质量评价信息为预定数字的目标第二图像,则所述目标图像为所述第一图像;或者,
若所述N个第二图像中存在图像质量评价信息对应的分数大于或等于预定分值阈值的目标第二图像,则所述目标图像为至少一个所述目标第二图像;若所述N个第二图像中不存在图像质量评价信息对应的分数大于或等于预定分值阈值的目标第二图像,则所述目标图像为所述第一图像;
当N=M≥2,所述图像质量评价信息为针对第一图像和所述N个第二图像中每一个第二图像的第二图像的图像质量评价信息时,所述目标图像为所述第一图像和所述N个第二图像中图像质量评价信息为预定数字的任一图像,或者,所述目标图像为所述第一图像和所述N个第二图像中图像质量评价信息对应的分数大于或等于预定分值阈值的任一图像,或者,所述目标图像为所述第一图像和所述N个第二图像中图像质量评价信息对应的分数最高的图像。
本实现方式,可以通过预定数字或预定分数阈值,从第一图像和N个第二图像中筛选出图像质量较好的图像进行输出。
第四方面,本申请提供了一种图像处理方法,包括:
获取第一图像;
将所述第一图像输入质量评价模型进行处理,得到第一图像质量评价信息;
当根据所述第一图像质量评价信息确定所述第一图像的图像质量符合要求时,将所述第一图像输入M个图像增强模型进行处理,得到N个第二图像,并显示或保存所述N个第二图像中的一个或者多个;其中,N和M为正整数,所述M个图像增强模型中的每个图像增强模型均不同,所述N个第二图像也存在不同。
其中,电子设备在人像模式下、景色模式、室内模式、长焦模式(下面称为高倍变焦模式)等等下获取第一图像。第一图像可以是预览图像,也可以是拍摄的照片。
在本实现方式中,先采用图像质量评价模型评价获取到的第一图像的图像质量,根据评价结果确定是否对第一图像进行处理。其中,当第一图像的图像质量较好时,采用图像增强模型对第一图像进行处理,输出第二图像;当第一图像的图像质量较差时,输出第一图像给用户查看,以减少因处理图像所占用的资源的消耗,可以提高手机的数据处理速度。
在第四方面的一种可能的实现方式中,当根据所述第一图像质量评价信息确定所述第一图像的图像质量不符合要求时,显示或保存所述第一图像。
在第四方面的一种可能的实现方式中,所述第一图像质量评价信息为数字或分数。
在本实现方式中,图像质量评价信息可以通过数字“0”或“1”来表示,或者,图像质量评价信息可以通过分数来表示,图像质量评价信息表示方式较灵活,可以更直观的体现图像质量。
在第四方面的一种可能的实现方式中,当所述第一图像质量评价信息为数字时, 所述第一图像的图像质量符合要求是指所述第一图像质量评价信息为预定数字。
在本实现方式中,电子设备可以通过质量评价模型对第一图像的图像质量进行二分类,从而判断第一图像是属于图像质量好的一类,还是属于图像质量差的一类。当第一图像是属于图像质量好的一类时,输出的图像质量评价信息为预定数字;当第一图像是属于图像质量差的一类时,输出的图像质量评价信息不是预定数字。对第一图像的图像质量进行二分类,计算开销较小,可以较快的得到第一图像的图像质量评价结果。
在第四方面的一种可能的实现方式中,当所述第一图像质量评价信息为分数时,所述第一图像的图像质量符合要求是指所述第一图像的分数大于或等于预设阈值。
本实现方式中,图像质量评价信息通过分数来表示,而分数可以更准确地描述第一图像的图像质量的好坏程度。电子设备将第一图像的分数与预设阈值进行比较,可以更准确地确定第一图像的图像质量是否符合要求。
在第四方面的一种可能的实现方式中,所述第一图像由电子设备在高倍变焦拍照模式下获取得到。
本实现方式中,由于电子设备在高倍变焦模式下拍摄人像照片时,获取到的第一图像,可能会因抖动而导致图像失真、图像中的人像的眼睛等五官变形或者图像较模糊(例如,较难辨认被拍摄对象)等,因此,需要评价第一图像的图像质量。如果第一图像的图像质量较差,那么将第一图像显示给用户查看,或将第一图像保存至图库;如果第一图像的图像质量较好,那么对第一图像进行处理得到N个第二图像,并显示或保存N个第二图像中的至少一个图像。通过节省因处理图像质量差的第一图像而消耗的资源,可以提高电子设备的数据处理速度。
在第四方面的一种可能的实现方式中,所述预设阈值为0.25,M和N均为1。
本实现方式中,电子设备获取到一个第一图像,采用质量评价模型评价第一图像的图像质量,得到第一图像对应的分数。当第一图像的分数小于0.25时,表示第一图像的图像质量不符合要求,保存第一图像;当第一图像的分数大于或等于0.25时,表示第一图像的图像质量符合要求,将第一图像输入图像增强模型处理后得到一个第二图像,显示或保存第二图像。由于第一图像不符合要求时,不输入图像增强模型处理,这样可以节省一部分资源,提高电子设备的数据处理速度。
在第四方面的一种可能的实现方式中,所述得到所述N个第二图像之后,还包括:
将所述N个第二图像,或者所述第一图像以及所述N个第二图像作为输入图像输入所述质量评价模型进行处理待处理图像输入质量评价模型进行处理,得到第二图像质量评价信息;
根据所述第二图像质量评价信息显示或保存目标图像,所述目标图像是所述第一图像和所述N个第二图像中的至少一个图像。
目标图像可以为第一图像和N个第二图像中图像质量最好的图像,图像质量最好的图像是指第一图像和N个第二图像中,第二图像质量评价信息为预定数字的图像, 或者第二图像质量评价信息对应的分数大于预定分数阈值的图像。
本实现方式中,在采用图像增强模型对第一图像处理前,采用质量评价模型评价第一图像的图像质量;当第一图像的图像质量较差时,输出第一图像给用户查看;当第一图像的图像质量较好时,将第一图像输入图像增强模型进行处理得到第二图像,采用质量评价模型评价第二图像的图像质量,如果第二图像的图像质量好,输出第二图像给用户查看,如果第二图像的图像质量差,输出第一图像给用户查看。这种方式,可以减少输出质量差的图像的概率,提高用户体验。
在第四方面的一种可能的实现方式中,所述第二图像质量评价信息为数字或分数。
在第四方面的一种可能的实现方式中,所述目标图像是所述第一图像和所述N个第二图像中,所述第二图像质量评价信息为预定数字的图像,或者所述第二图像质量评价信息对应的分数大于预定分数阈值的图像。
本实现方式中,由于当图像质量评价信息为预定数字,或者,图像质量评价信息对应的分数大于或等于预定分值阈值时,表示该图像质量评价信息对应的图像的图像质量较好,因此,电子设备可以根据第二图像质量评价信息,从第一图像和N个第二图像中,确定图像质量较好的目标图像,并输出图像质量较好的目标图像给用户查看,减少输出质量差的图像的概率。
第五方面,本申请提供了一种图像处理装置,包括:
获取单元,用于获取第一图像和N个第二图像,所述N个第二图像中的每一个第二图像是通过M个图像增强模型中的至少一个图像增强模型对所述第一图像进行处理后获得的;其中,N和M为大于零的整数,所述M个图像增强模型中的每个图像增强模型均不同,所述N个第二图像也存在不同;
图像质量评价单元,用于将输入图像输入质量评价模型进行处理,获得图像质量评价信息,所述输入图像包括所述N个第二图像,或者包括所述第一图像以及所述N个第二图像;
图像输出单元,用于根据所获得的所述图像质量评价信息输出目标图像,所述目标图像是所述第一图像和所述N个第二图像中的至少一个图像。
其中,一个图像增强模型对第一图像进程处理后,可以得到一个第二图像,也可以得到至少两个第二图像,此处不做限制。也就是说,M可以与N相等,M也可以大于N。
第五方面提供的图像处理装置对应的有益效果与第一方面、第三方面的图像处理方法的有益效果相同,此处不赘述。
在第五方面的一种可能的实现方式中,所述质量评价模型是基于多个训练样本训练得到,每个训练样本包括样本图像以及用户对所述样本图像的图像质量评价信息。
在第五方面的一种可能的实现方式中,所述目标图像包括根据所获得的所述图像质量评价信息和评价规则确定的目标图像,所述评价规则是所述图像质量评价信息为预定数字,或,图像质量评价信息对应的分数大于或等于预定分值阈值。
在第五方面的一种可能的实现方式中,图像质量评价信息可以为用于表示图像质量的数字或分数。
需要说明的是,图像质量评价信息还可以通过字母或文字来表示,此处不做限制。例如,图像质量评价信息可以是“0”或“1”、“YES”或“NO”,“是”或“否”。
在第五方面的一种可能的实现方式中,当所述图像质量评价信息为数字,且所述输入图像为N个第二图像,或者所述第一图像以及所述N个第二图像时,所述图像质量评价信息为对所述N个第二图像中每一个第二图像的图像质量评价信息;
相应的,所述图像输出单元具体用于:
判断所述N个第二图像中是否存在图像质量评价信息为预定数字的目标第二图像;
若所述N个第二图像中存在图像质量评价信息为预定数字的目标第二图像,则确定所述图像质量评价信息为预定数字的目标第二图像为图像质量最好的图像,并输出所述图像质量评价信息为预定数字的目标第二图像。
其中,预定数字可以为“1”。
在第五方面的一种可能的实现方式中,所述图像输出单元,在判断所述N个第二图像中是否存在图像质量评价信息为预定数字的目标第二图像之后,还用于:若所述N个第二图像中不存在图像质量评价信息为预定数字的目标第二图像,则确定所述第一图像为图像质量最好的图像,输出所述第一图像。
在第五方面的一种可能的实现方式中,所述图像输出单元还用于:
若所述N个第二图像中存在多个图像质量评价信息为预定数字的目标第二图像,则选择任一目标第二图像输出。
在第五方面的一种可能的实现方式中,若所述图像质量评价信息为分数,且所述输入图像为N个第二图像,或者所述第一图像以及所述N个第二图像时,所述图像质量评价信息为对所述N个第二图像中每一个第二图像的图像质量评价信息;
相应的,所述图像输出单元具体用于:
判断所述N个第二图像中是否存在分数大于预定分数阈值的目标第二图像;
若所述N个第二图像中存在分数大于预定分数阈值的目标第二图像,则确定所述分数最高的目标第二图像为图像质量最好的图像,并输出所述分数最高的目标第二图像。
在五方面的一种可能的实现方式中,所述图像输出单元还用于,在判断所述N个第二图像中是否存在分数大于预定分数阈值的目标第二图像之后,若所述N个第二图像中不存在分数大于预定分数阈值的目标第二图像,则确定所述第一图像为图像质量最好的图像,输出所述第一图像。
在第五方面的一种可能的实现方式中,所述所述图像输出单元还用于:
若所述N个第二图像中不存在图像质量评价信息为预定数字的目标第二图像,或者所述N个第二图像中不存在分数大于预定分数阈值的目标第二图像,则获取新的图像增强模型,采用获取的所述新的图像增强模型对所述第一图像进行处理,获得新的第二图像,并将所述新的第二图像作为输入图像输入所述质量评价模型进行处理,获得新的图像质量评价信息;其中,所述新的图像增强模型为未对所述第一图像进行处理过的图像增强模型;
若确定所述新的图像质量评价信息为所述预定数字,或者所述分数大于所述预定分数阈值,则输出新的第二图像,否则返回执行获取新的图像增强模型步骤以及后续 步骤,直到返回执行的次数达到预设的次数阈值,输出所述第一图像。
在第五方面的一种可能的实现方式中,当所述图像质量评价信息为数字,且所述输入图像为第一图像和N个第二图像时,所述图像质量评价信息为对所述第一图像和N个第二图像中每一个第二图像的图像质量评价信息;
相应的,所述图像输出单元具体用于:
判断所述第一图像和N个第二图像中是否存在图像质量评价信息为预定数字的图像;
若所述第一图像的图像质量评价信息为预定数字,且所述N个第二图像中存在至少一个图像质量评价信息为预定数字的目标第二图像,则从所述第一图像和所述目标第二图像中选择任一图像输出。
在第五方面的一种可能的实现方式中,所述图像输出单元在判断所述第一图像和N个第二图像中是否存在图像质量评价信息为预定数字的目标第二图像之后,还用于:
若所述第一图像的图像质量评价信息不是预定数字,且所述N个第二图像中存在至少一个图像质量评价信息为预定数字的目标第二图像,则从所述目标第二图像中选择任一图像输出。
在第五方面的一种可能的实现方式中,所述图像输出单元还用于:
若所述第一图像的图像质量评价信息不是预定数字,且所述N个第二图像中不存在图像质量评价信息为所述预定数字的目标第二图像,则输出所述第一图像。
在第五方面的一种可能的实现方式中,当所述图像质量评价信息为分数,且所述输入图像为第一图像和N个第二图像时,所述图像质量评价信息为对所述第一图像和N个第二图像中每一个第二图像的图像质量评价信息;
相应的,所述图像输出单元还用于:
根据第一图像的分数和所述N个第二图像的分数,从所述第一图像和所述N个第二图像中,确定分数最高的图像为所述图像质量最好的图像,并输出所述分数最高的图像。
在第五方面的一种可能的实现方式中,获取单元包括:
RAW图像获取单元,用于获取多帧RAW图像;
图像融合单元,用于对所述多帧RAW图像进行图像融合处理,得到第一图像。
在该实现方式中,可以将RAW图像对应的融合图像作为参考图像,通过每组RAW图像对应的融合图像辅助评价该组RAW图像对应的增强图像的图像质量,可以提高增强图像的图像质量评价信息的准确度。
在第五方面的一种可能的实现方式中,所述图像融合单元具体用于:
将所述多帧RAW图像划分为至少两组,对每组RAW图像进行图像融合处理得到至少两个第一图像。
在第五方面的一种可能的实现方式中,当N=M=1,所述图像质量评价信息为所述第二图像的图像质量评价信息时,若所述图像质量评价信息为预定数字,或者所述图像质量评价信息对应的分数大于或等于预定分数阈值,则所述目标图像为所述第二图像;若所述图像质量评价信息不是预定数字,或者所述图像质量评价信息对应的分数小于预定分数阈值,则所述目标图像为所述第一图像;
当N=M=1,所述图像质量评价信息为针对所述第一图像和所述第二图像的图像质量评价信息,所述图像质量评价信息用于表示所述第二图像的图像质量是否比所述第一图像的图像质量好时,若所述图像质量评价信息为预定数字,则所述目标图像为所述第二图像;若所述图像质量评价信息不是预定数字,则所述目标图像为所述第一图像;
当N=M=1,所述图像质量评价信息包括所述第一图像和所述第二图像各自对应的图像质量评价信息时,若所述第一图像的图像质量评价信息为预定数字,且所述第二图像的图像质量评价信息为预定数字,则所述目标图像为所述第二图像和所述第一图像中的任一图像;若所述第一图像的图像质量评价信息不是预定数字,且所述第二图像的图像质量评价信息为预定数字,则所述目标图像为所述第二图像;若所述第一图像的图像质量评价信息为预定数字,且所述第二图像的图像质量评价信息不是预定数字,则所述目标图像为所述第一图像;或者,
若所述第二图像的图像质量评价信息对应的分数大于所述第一图像的图像质量评价信息对应的分数,则所述目标图像为所述第二图像;若所述第二图像的图像质量评价信息对应的分数小于所述第一图像的图像质量评价信息对应的分数,则所述目标图像为所述第一图像;若所述第二图像的图像质量评价信息对应的分数等于所述第一图像的图像质量评价信息对应的分数,则所述目标图像为所述第二图像和所述第一图像中的任一图像;
当N=M≥2,所述图像质量评价信息为所述N个第二图像中每一个第二图像的第二图像的图像质量评价信息时,若所述N个第二图像中存在图像质量评价信息为预定数字的目标第二图像,则所述目标图像为至少一个所述目标第二图像;若所述N个第二图像中不存在图像质量评价信息为预定数字的目标第二图像,则所述目标图像为所述第一图像;或者,
若所述N个第二图像中存在图像质量评价信息对应的分数大于或等于预定分值阈值的目标第二图像,则所述目标图像为至少一个所述目标第二图像;若所述N个第二图像中不存在图像质量评价信息对应的分数大于或等于预定分值阈值的目标第二图像,则所述目标图像为所述第一图像;
当N=M≥2,所述图像质量评价信息为针对第一图像和所述N个第二图像中每一个第二图像的第二图像的图像质量评价信息时,所述目标图像为所述第一图像和所述N个第二图像中图像质量评价信息为预定数字的任一图像,或者,所述目标图像为所述第一图像和所述N个第二图像中图像质量评价信息对应的分数大于或等于预定分值阈值的任一图像,或者,所述目标图像为所述第一图像和所述N个第二图像中图像质量评价信息对应的分数最高的图像。
第六方面,本申请提供了一种图像处理装置,包括:
获取单元,用于获取第一图像;
第一评价单元,用于将所述第一图像输入质量评价模型进行处理,得到第一图像质量评价信息;
图像处理单元,用于当根据所述第一图像质量评价信息确定所述第一图像的图像质量符合要求时,将所述第一图像输入M个图像增强模型进行处理,得到N个第二图 像,并显示或保存所述N个第二图像中的一个或者多个;其中,N和M为正整数,所述M个图像增强模型中的每个图像增强模型均不同,所述N个第二图像也存在不同。
其中,电子设备在人像模式下、景色模式、室内模式、长焦模式(下面称为高倍变焦模式)等等下获取第一图像。第一图像可以是预览图像,也可以是拍摄的照片。
第六方面提供的图像处理装置对应的有益效果与第二方面、第四方面的图像处理方法的有益效果相同,此处不赘述。
在第六方面的一种可能的实现方式中,当根据所述第一图像质量评价信息确定所述第一图像的图像质量不符合要求时,显示或保存所述第一图像。
在第六方面的一种可能的实现方式中,所述第一图像质量评价信息为数字或分数。
在第六方面的一种可能的实现方式中,当所述第一图像质量评价信息为数字时,所述第一图像的图像质量符合要求是指所述第一图像质量评价信息为预定数字。
在第六方面的一种可能的实现方式中,当所述第一图像质量评价信息为分数时,所述第一图像的图像质量符合要求是指所述第一图像的分数大于或等于预设阈值。
在第六方面的一种可能的实现方式中,所述第一图像由电子设备在高倍变焦拍照模式下获取得到。
在第六方面的一种可能的实现方式中,所述预设阈值为0.25,M和N均为1。
在第六方面的一种可能的实现方式中,所述图像处理装置还包括:
第二评价单元,用于在所述图像处理单元得到所述N个第二图像之后,将所述N个第二图像,或者所述第一图像以及所述N个第二图像作为输入图像输入所述质量评价模型进行处理待处理图像输入质量评价模型进行处理,得到第二图像质量评价信息;
输出单元,用于根据所述第二图像质量评价信息显示或保存目标图像,所述目标图像是所述第一图像和所述N个第二图像中的至少一个图像。
在第六方面的一种可能的实现方式中,所述第二图像质量评价信息为数字或分数。
在第六方面的一种可能的实现方式中,所述目标图像是所述第一图像和所述N个第二图像中,所述第二图像质量评价信息为预定数字的图像,或者所述第二图像质量评价信息对应的分数大于预定分数阈值的图像。
第七方面,本申请提供了一种电子设备,包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序,所述处理器执行所述计算机程序时使所述电子设备执行上述第一方面或第三方面的任一种可能的实现方式的图像处理方法,或者执行上述第二方面或第四方面的任一种可能的实现方式的图像处理方法。
第八方面,本申请提供了一种电子设备,包括存储模块、处理模块以及存储在所述存储模块中并可在所述处理模块上运行的计算机程序,所述处理模块执行所述计算机程序时使所述电子设备执行上述第一方面或第三方面的任一种可能的实现方式的图像处理方法,或者执行上述第二方面或第四方面的任一种可能的实现方式的图像处理方法。第九方面,本申请提供了一种计算机可读存储介质,所述计算机可读存储介质 存储有计算机程序,所述计算机程序被处理器执行时使得电子设备执行上述第一方面或第三方面的任一种可能的实现方式的图像处理方法,或者执行上述第二方面或第四方面的任一种可能的实现方式的图像处理方法。
第十方面,本申请提供了一种计算机程序产品,当计算机程序产品在电子设备上运行时,使得电子设备执行上述第一方面或第三方面中任一种可能的实现方式的图像处理方法,或者执行上述第二方面或第四方面中的任一种可能的实现方式的图像处理方法。
本申请与现有技术相比存在的有益效果是:
本方案中,在通过图像增强模型对第一图像进行处理,得到第二图像之后,通过质量评价模型评价第二图像的图像质量,或者评价第一图像和第二图像的图像质量,并根据评价结果最终输出第一图像和第二图像中图像质量较好的图像给用户查看。也就是说,由第一图像处理后得到的第二图像的图像质量变差时,电子设备最终输出图像A给用户查看,以解决现有技术中将经过图像增强模型处理后得到的图像质量变差的图像输出给用户的问题,能够提高输出的图像的图像质量,可以减少输出质量差的图像的概率,以提高用户的视觉体验。
附图说明
为了更清楚地说明本申请实施例中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动性的前提下,还可以根据这些附图获得其他的附图。
图1是本申请一实施例提供的一种场景图;
图2是本申请第一实施例提供的一种图像处理方法的示意性流程图;
图3是本申请一实施例提供的一种带噪图像;
图4是本申请一实施例提供的对图3中的带噪图像进行去噪后得到的图像;
图5是本申请一实施例提供的一种存在色彩混叠的图像;
图6是本申请一实施例提供的一种用户界面的示意图;
图7是本申请第二实施例提供的一种图像处理方法的示意性流程图;
图8是本申请第三实施例提供的一种图像处理方法的示意性流程图;
图9是本申请第四实施例提供的一种图像处理方法的示意性流程图;
图10是本申请第五实施例提供的一种图像处理方法的示意性流程图;
图11是本申请实施例提供的一种训练质量评价模型的示意性流程图;
图12是本申请第六实施例提供的一种图像处理方法的示意性流程图;
图13a~图13d是本申请另一实施例提供的一种用户界面的示意图;
图14是本申请再一实施例提供的一种用户界面的示意图;
图15是本申请又一实施例提供的一种用户界面的示意图;
图16是本申请再一实施例提供的一种用户界面的示意图;
图17是本申请第七实施例提供的一种图像处理方法的示意性流程图;
图18是本申请第八实施例提供的一种图像处理方法的示意性流程图;
图19是本申请第九实施例提供的一种图像处理方法的示意性流程图;
图20是本申请第十实施例提供的一种图像处理方法的示意性流程图;
图21是本申请第十一实施例提供的一种图像处理方法的示意性流程图;
图22是本申请第十二实施例提供的一种图像处理方法的示意性流程图;
图23是本申请一实施例提供的一种处理多帧RAW图像的方法的示意图;
图24是本申请另一实施例提供的一种处理多帧RAW图像的方法的示意图;
图25是本申请再一实施例提供的一种处理多帧RAW图像的方法的示意图。
图26是本申请实一施例提供的一种图像处理装置的结构示意图;
图27是本申请实另一施例提供的一种图像处理装置的结构示意图;
图28是本申请实施例提供的一种电子设备的结构示意图。
具体实施方式
以下描述中,为了说明而不是为了限定,提出了诸如特定系统结构、技术之类的具体细节,以便透彻理解本申请实施例。然而,本领域的技术人员应当清楚,在没有这些具体细节的其它实施例中也可以实现本申请。在其它情况中,省略对众所周知的系统、装置、电路以及方法的详细说明,以免不必要的细节妨碍本申请的描述。
应当理解,当在本申请说明书和所附权利要求书中使用时,术语“包括”指示所描述特征、整体、步骤、操作、元素和/或组件的存在,但并不排除一个或多个其它特征、整体、步骤、操作、元素、组件和/或其集合的存在或添加。
还应当理解,在本申请说明书和所附权利要求书中使用的术语“和/或”是指相关联列出的项中的一个或多个的任何组合以及所有可能组合,并且包括这些组合。
如在本申请说明书和所附权利要求书中所使用的那样,术语“如果”可以依据上下文被解释为“当...时”或“一旦”或“响应于确定”或“响应于检测到”。类似地,短语“如果确定”或“如果检测到[所描述条件或事件]”可以依据上下文被解释为意指“一旦确定”或“响应于确定”或“一旦检测到[所描述条件或事件]”或“响应于检测到[所描述条件或事件]”。
另外,在本申请说明书和所附权利要求书的描述中,术语“第一”、“第二”、“第三”等仅用于区分描述,而不能理解为指示或暗示相对重要性。
在本申请说明书中描述的参考“一个实施例”或“一些实施例”等意味着在本申请的一个或多个实施例中包括结合该实施例描述的特定特征、结构或特点。由此,在本说明书中的不同之处出现的语句“在一个实施例中”、“在一些实施例中”、“在其他一些实施例中”、“在另外一些实施例中”等不是必然都参考相同的实施例,而是意味着“一个或多个但不是所有的实施例”,除非是以其他方式另外特别强调。术语“包括”、“包含”、“具有”及它们的变形都意味着“包括但不限于”,除非是以其他方式另外特别强调。
目前智能手机等拍摄设备通常利用场景识别算法自动识别出图像的拍摄场景,进而使用该拍摄场景对应的图像增强算法或模型对图像进行处理,以增强图像质量。
例如,智能手机获取到图像A,如果利用场景识别算法识别出月亮,那么图像A的拍摄场景为拍月亮,可以采用月亮增强算法或月亮增强模型对图像A中的月亮进行处理,以增强月亮细节的清晰度,得到月亮轮廓清晰的图像B,并将图像B作为最终的图像输出,供用户查看;如果利用场景识别算法识别出人脸,那么图像A的拍摄场 景为拍人像可以采用人脸超分辨率算法或人脸超分辨率模型对人脸图像进行超分辨处理,以增强人脸的清晰度,得到图像B,并将图像B作为最终的图像输出,供用户查看。
目前的图像处理方法,虽然可以识别图像A对应的拍摄场景,但无法预测处理后最终输出的图像B相对于图像A是否存在图像质量变差的问题。
另外,由于图像处理主要关注底层(low level)图像信息的处理,而底层图像信息很难量化,因此,场景识别算法通常是基于深度学习的场景识别算法。由于基于深度学习的场景识别算法通常需要人工标注大量的场景进行训练得到,而处理场景可能因区分度不明显或场景复杂度较高等原因而无法准确标注,也很难覆盖所有的场景,一旦场景识别算法没有覆盖到某场景下的月亮或人脸,处理后输出的图像可能会存在瑕疵(artifacts)。
其中,low level图像信息是相对于high level图像信息而言的。high level图像信息主要包括图像的语义(场景、目标间的相互关系)描述,low level图像信息主要包括图像的像素、图像块、边缘、角点、纹理等。图像的纹理是与物体表面结构和材质有关的图像的内在特征,反映出来的是图像的全局特征。图像的纹理可以描述为:一个邻域内像素的灰度级发生变化的空间分布规律,包括表面组织结构、与周围环境关系等许多重要的图像信息。
图像去雨、去雾、与图像去噪、去模糊、超分辨等都属于low level的图像处理问题。简单来说,图像去雨的目的是将有雨图像中雨线(雨点)去除,同时保留图像原有的结构特征。去雾的目的是去除有雾图像中的雾,同时保留图像原有的结构特征。
其中,artifacts包括但不限于伪影、亮斑、棋盘效应(check board artifacts)、锐化造成的亮边、毛刺、散粒噪声、锯齿效应等。伪影包括条纹、色度噪声和稀疏数据噪声。棋盘效应通常是指图像中尤其是深色部分常出现的“棋盘格子状伪影”。锯齿效应也称拉链效应,是指在图像的边缘交界或颜色突变区域,例如呈阶梯状。
综上所述,现有技术存在一个问题,当使用图像增强算法或模型对图像进行处理后,得到的图像质量可能会更差,而智能手机依然会将处理后的图像作为最终的图像输出,严重影响了图像处理的效果及用户体验图像处理。
为解决上述问题,本申请提供一种low level的图像处理方法。参见图1,图1是本申请一实施例提供的一种场景图。如图1所示用户A可以用电子设备(例如,手机、平板电脑等)对用户B或对周围环境进行拍照,得到图像A。用户也可以通过电子设备从互联网下载图像A,例如下载通过社交应用分享的图像A,还可以从电子设备的图库或相册中选择图像A。由于图像A可能会存在分辨率较低、含有图像噪声、眼镜反光、眼睛反光中的至少任一种情况,因此需要对图像A进行图像处理。图像处理方法可以为:电子设备获取待处理的图像A,将图像A输入M个图像增强模型,通过M个图像增强模型对图像A进行处理后获得N个图像B;将N个图像B,或者将图像A和N个图像B输入训练后的质量评价模型进行处理,获得质量评价模型输出的图像质量评价信息,根据图像质量评价信息输出图像A和N个图像B中图像质量最好的图像。其中,质量评价模型采用多个训练样本训练得到,每个训练样本包括样本图像以及用户对样本图像标记的图像质量评价信息。N和M为大于零的整数,所述M个图像增 强模型中的每个图像增强模型均不同,所述N个第二图像也存在不同。
具体的,方案一,当质量评价模型的输入图像为图像B时,质量评价模型用于评价图像B的图像质量,若电子设备根据质量评价模型输出的图像质量评价信息确定图像B的图像质量最好,则输出图像B,否则输出图像A;
其中,针对方案一,质量评价模型的训练样本可以包括图像增强模型对原始图像进行处理后输出的样本图像、以及用户对样本图像标记的图像质量评价信息。
方案二,当质量评价模型的输入图像为图像A和图像B时,质量评价模型可以用于通过图像A辅助评价图像B的图像质量图像质量,若电子设备根据质量评价模型输出的图像质量评价信息确定图像B的图像质量最好,则输出图像B,否则输出图像A;
其中,针对方案二,质量评价模型的训练样本可以包括原始图像、图像增强模型对原始图像进行处理后输出的样本图像、以及用户对样本图像标记的图像质量评价信息。方案三,当质量评价模型的输入图像为图像A和图像B时,质量评价模型用于评价图像A和图像B的图像质量,若电子设备根据质量评价模型输出的图像质量评价信息确定图像B的图像质量比图像A的图像质量好,则输出图像B;若电子设备根据质量评价模型输出的图像质量评价信息确定图像B的图像质量比图像A的图像质量差,则输出图像A;若电子设备根据质量评价模型输出的图像质量评价信息确定图像B的图像质量与图像A的图像质量相同,则从图像A和图像B中选择任一图像输出。
其中,针对方案三,质量评价模型的训练样本可以包括原始图像、图像增强模型对原始图像进行处理后输出的样本图像、用户对原始图像标记的图像质量评价信息以及用户对样本图像标记的图像质量评价信息。
示例性的,图像质量评价信息可以通过标识信息表示,标识信息可以是数字、字母、文字等,电子设备可以根据预先建立的对应关系或规则确定标识信息中的数字、文字或字母所表示的含义,从而输出图像A或图像B。例如,针对方案一和方案二,用“0”表示图像B的图像质量差,用“1”表示图像B的图像质量好。当图像质量评价信息为1时,电子设备确定图像质量评价信息对应的图像质量评价结果为图像B的图像质量最好,输出图像B;当图像质量评价信息为0时,电子设备确定图像质量评价信息对应的图像质量评价结果为图像A的图像质量最好,输出图像A。针对方案三,用“0”表示图像B的图像质量比图像A的图像质量差,用“1”表示图像B的图像质量比图像A的图像质量好,其它数字表示图像B的图像质量与图像A的图像质量相同。当图像质量评价信息为1时,电子设备根据图像质量评价信息确定图像B的图像质量比图像A的图像质量好,输出图像B;当图像质量评价信息为0时,电子设备根据图像质量评价信息确定图像B的图像质量比图像A的图像质量差,输出图像A;当图像质量评价信息为2时,电子设备根据图像质量评价信息确定图像B的图像质量与图像A的图像质量相同,从图像A和图像B中选择任一图像输出。
示例性的,图像质量评价信息可以通过分数表示,例如,针对方案一和方案二,图像质量评价信息是图像B对应的分数,电子设备将图像B对应的分数与预定分数阈值进行比较,当图像B对应的分数小于或等于预定分数阈值时,表示图像B的图像质量差,电子设备输出图像A,当图像B对应的分数大于预定分数阈值时,表示图像B的图像质量好,电子设备输出图像B。针对方案三,图像质量评价信息包括图像A的 分数A和图像B的分数B,电子设备将图像B对应的分数A与图像B对应的分数B进行比较,当分数B大于分数A时,表示图像B的图像质量比图像A的图像质量好,电子设备输出图像B;当分数B小于分数A时,表示图像B的图像质量比图像A的图像质量差,电子设备输出图像A,当分数B等于分数A时,表示图像B的图像质量与图像A的图像质量相同,电子设备从图像A和图像B中选择任一图像输出。
本方案中,由于采用图像增强模型对图像A进行处理后得到的图像B的图像质量可能比图像A的图像质量更差,例如,图像B中包括人脸图像,且人脸图像中的眼睛、鼻子、眼镜框等出现变形等情况、图像B的边缘出现紫边、图像B中存在叠影、色彩混叠、拉链效应等瑕疵,因此,通过质量评价模型评价图像B的图像质量,或者评价图像A和图像B的图像质量,并根据评价结果最终从图像A和图像B中输出图像质量较好的那一个给用户查看。也就是说,处理后的图像B的图像质量变差时,电子设备最终输出图像A给用户查看,以解决现有技术中将经过图像增强模型处理后得到的图像质量变差的图像输出给用户的问题,能够提高输出的图像的图像质量,可以减少输出质量差的图像的概率,以提高用户的视觉体验。
图像处理
为了使本申请的目的、技术方案和优点更加清楚,下面将结合附图对本申请的图像处理方法的具体实现方式作进一步地详细描述。
请参阅图2,图2是本申请第一实施例提供的一种图像处理方法的示意性流程图。图像处理方法的执行主体为电子设备。电子设备包括但不限于手机、笔记本电脑、平板电脑、可穿戴设备(包括手表)、个人数字助理(Personal Digital Assistant,PDA)、车机、虚拟现实(Virtual Reality,VR)设备等。其中,车机指的是安装在汽车里面的车载信息娱乐产品的简称。在本实施例中,输入质量评价模型的图像为采用图像增强模型对第一图像进行图像质量增强后得到的第二图像,质量评价模型用于评价第二图像的图像质量;电子设备根据第二图像的图像质量评价信息判断第二图像的图像质量是好还是差;如果第二图像的图像质量好,输出第二图像给用户查看;如果第二图像的图像质量差,输出第一图像给用户查看。下面以手机为例进行说明,图像处理方法包括以下步骤:
S101、获取待处理的第一图像。
手机可以在启动拍照应用之后,响应于用户触发的拍照指令,获取待处理的第一图像。
手机也可以在启动图像处理应用时,获取用户选择的待处理的第一图像。第一图像可以是用户从手机的图库或相册中选择的图像。图库或相册中的图像可以是手机用户拍摄并保存的照片,也可以是其他用户拍摄并分享给手机用户的图像。
S102、采用图像增强模型对所述第一图像进行处理,得到第二图像。
手机将第一图像输入图像增强模型进行图像增强处理,以增强第一图像的图像质量,得到第二图像。图像增强是指:对原始图像附加一些信息或特征,有选择性突出图像中感兴趣的特征,抑制或掩盖图像中某些不需要的特征。例如提高原始图像的清晰度,降低图像噪点和伪色等。伪色是指照片暗部出现的彩色条纹及噪点。
采用同一个图像增强模型对第一图像进行处理时,可以获得一个第二图像,也可以获得至少两个第二图像,此处不做限制。例如,当采用图像增强模型对第一图像处理至少两次时,可以得到至少两个第二图像。至少两个第二图像也可以存在不同。
图像增强处理包括但不限于超分辨率、去噪、去马赛克、图像复原。通常情况下,图像复原与超分辨率、去噪、去马赛克这三种处理是独立的。即,在一种示例中,手机对第一图像进行图像复原处理后,执行S103;或者手机可以对第一图像进行以下至少两种处理:超分辨率、去噪、去马赛克,然后执行S103。当然,在另一种示例中,手机对第一图像进行图像复原处理后,还可以执行超分辨率、去噪、去马赛克中的至少任一种,再执行S103;手机也可以先执行超分辨率、去噪、去马赛克中的至少任一种后,再对第一图像进行图像复原处理,然后执行S103。其中,图像复原的处理场景可以包括但不限于:对老照片进行复原、对隔着栅栏或围栏拍摄的照片去除栅栏或围栏、对有雨图像去除雨线(雨点)、对有雾图像去除雾、对隔着玻璃拍摄的图像去除玻璃等。
需要说明的是,对第一图像执行的图像处理类型可以根据第一图像的图像特征确定、也可以根据用户触发的指令,或用户选择的图像处理功能来确定。
例如,手机可以在检测到第一图像的分辨率小于或等于预设分辨率阈值,或者检测到用户触发用于表示图像超分辨率的指令时,采用图像增强模型对第一图像进行超分辨率处理得到第二图像,第二图像的分辨率大于第一图像的分辨率;在检测到第一图像中存在图像噪声(例如,如图3所示),或者检测到去噪指令时,采用图像增强模型对第一图像进行去噪处理得到第二图像,如图4所示,第二图像的图像噪声比第一图像的图像噪声少;在检测到第一图像中存在马赛克,或检测到去马赛克指令时,采用图像增强模型对第一图像进行去马赛克处理;在检测到第一图像中存在图像模糊的区域,或检测到图像复原的指令时,采用图像增强模型对第一图像进行复原处理,比如,第一图像可以是翻拍的老照片、有雨线(雨点)图像、有雾图像,第一图像可能会因色彩褪色、损坏、有雨点或雾等原因导致图像模糊。预设分辨率阈值、灰度值差值阈值可以根据实际情况进行设置,此处不做限制。
超分辨率在本方案中是指通过软件的方法提高原有图像的分辨率。去噪是指减少数字图像中噪声的过程。
去马赛克(demosaicing)是一种数位影像处理算法,目的是从覆有滤色阵列(Color filter array,CFA)的感光元件所输出的不完全色彩取样中,重建出全彩影像,即重建出各像素完整的红绿蓝(red green blue,RGB)三原色组合。去马赛克也称为滤色阵列内插法(CFA interpolation)或色彩重建法(Color reconstruction)。去马赛克具备以下特点:避免错误颜色噪声(False color artifacts)产生,例如色彩混叠(Aliases)或出现拉链状(Zippering,即邻近像素出现突兀且不自然的强度改变,有一种拉链状纹路的感觉)以及紫边(Purple fringe)噪声;尽量保留影像分辨率;在相机内的硬件限制下,以较低计算复杂度(Computational complexity)实现快速有效的运算处理;算法易于分析,以使降噪(Noise reduction)更精确。
图像复原(image restoration)即利用退化过程的先验知识,去恢复已被退化图像的本来面目,以提高图像的整体质量。图像模糊是图像退化的表现之一。
图像增强模型可以是现有技术中已训练的图像处理模型。图像增强模型可以是具有单一图像处理功能的图像处理模型,也可以是具有至少两种图像处理功能的图像处理模型。单一图像处理功能是指实现超分辨率、去噪、去马赛克或图像复原。
当图像增强模型是具有单一图像处理功能的图像处理模型时,图像增强模型可以是超分辨率模型、去噪模型、去马赛克模型或图像复原模型。此时,当需要实现至少两种图像处理时,可以采用至少两种图像处理模型来实现。例如,当需要对第一图像进行去噪、超分辨率、去马赛克三种处理时,可以采用去噪模型对第一图像进行处理,得到去噪后的图像P1,将图像P1输入超分辨率模型进行处理,得到图像P2,之后,将图像P2输入去马赛克模型进行处理,得到第二图像。
当图像增强模型具有至少两种图像处理功能的图像处理模型时,同一个图像增强模型可以实现超分辨率、去噪、去马赛克、图像复原中的至少两种。图像增强模型可以由一个图像处理模型构成,也可以由至少两个具有不同的图像处理功能的子模型串接而成。例如,当需要对第一图像进行去噪、超分辨率、去马赛克三种处理时,选择的图像增强模型可以是由去噪模型、超分辨率模型、以及去马赛克模型串接而成,从而能够依次对第一图像进行去噪、超分辨率以及去马赛克。
超分辨率模型包括但不限于:超分辨率卷积神经网络(Super-Resolution Convolutional Neural Networks,SRCNN)、快速超分辨率卷积神经网络(Fast Super-Resolution Convolutional Neural Networks,FSRCNN)、有效亚像素卷积神经网络(Efficient Sub-Pixel Convolutional Neural Network,ESPCN)、超分辨率生成对抗网络(Super-Resolution Generative Adversarial Network,SRGAN),增强型超分辨率生成对抗网络(Enhanced Super-Resolution Generative Adversarial Networks,ESRGAN)等。
去噪模型包括但不限于:基于深度神经网络的去噪模型。
去马赛克模型包括但不限于:交替方向乘子法(Alternating Direction Method of Multipliers,ADMM)、基于深度神经网络的去马赛克模型等。
图像复原模型包括但不限于:基于深度神经网络的图像复原模型等。
由于第一图像在经过图像增强模型处理后得到第二图像,虽然第二图像的分辨率变高了、图像噪声变少了、或者去除了第二图像中的马赛克、去除了第二图像中的雨点、雾等,但是,第二图像中可能存在artifacts,例如,第二图像中存在色彩混叠(如图5所示),因此,为了减少输出质量差的图像供用户查看的情况,以提高用户的视觉体验,需要评价第二图像的图像质量,以根据评价结果输出第一图像或第二图像给用户查看。
S103、将所述第二图像输入质量评价模型进行处理,得到所述第二图像对应的图像质量评价信息。
手机可以将第二图像输入质量评价模型,通过质量评价模型提取第二图像对应的用于衡量图像质量的特征信息,对提取到的特征信息进行处理,以评价第二图像的图像质量,获取质量评价模型输出的第二图像对应的图像质量评价信息。
其中,如果在S102中获得N个第二图像,S103可以为:将所述N个第二图像输入质量评价模型进行处理,得到所述第二图像对应的图像质量评价信息。所述图像质 量评价信息为对所述N个第二图像中每一个第二图像的图像质量评价信息。
图像质量评价信息可以是图像质量分数,图像质量评价信息也可以是用于表示第二图像的图像质量好或差的标识信息,标识信息可以是数字、字母、文字等。例如,标识信息可以是“0”或“1”,“0”表示第二图像的图像质量差,“1”表示第二图像的图像质量好。再例如,标识信息也可以是“YES”或“NO”,“YES”表示第二图像的图像质量好,“NO”表示第二图像的图像质量差;标识信息也可以是“是”或“否”,“是”表示第二图像的图像质量好,“否”表示第二图像的图像质量差。
当图像质量评价信息是图像质量分数时,图像质量分数可以是0-100中的任一整数,也可以是0、1以及0-1之间的任一数值,当图像质量分数为0-1之间的任意小数时,可以保留一位小数,也可以保留2位小数,此处不做限制。当图像质量分数大于或等于预定分数阈值时,表示第二图像的图像质量好;当图像质量分数小于预定分数阈值时,表示第二图像的图像质量差;当图像质量分数等于预定分数阈值时,表示第二图像的图像质量与第一图像的图像质量相同。预定分数阈值例如75或80等,可以根据具体情况设置,此处不做限定。质量评价模型可以是采用机器学习算法基于多个训练样本对深度学习网络训练得到。一个训练样本包括一个样本图像及用户对其标记的图像质量评价信息。深度学习网络可以是卷积神经网络、对抗网络,在此,对深度学习网络的网络类型及网络结构不做限定。在训练的过程中,质量评价模型的输入为样本图像以及样本图像对应的标记的图像质量评价信息,质量评价模型的输出为样本图像对应的预测的图像质量评价信息。
质量评价模型也可以是基于图像质量评价(Natural Image Quality Evaluator,NIQE)算法构建得到。NIQE算法是从自然图像中获取一些统计数据(natural scene statistic,NSS),用来刻画图像质量。
多个样本图像可以是多个原始图像输入图像增强模型进行处理后得到的图像,多个样本图像也可以由原始图像、对原始图像做退化处理后的图像组成。多个样本图像各自对应的图像质量评价信息可以部分相同,也可以完全不同。标记的图像质量评价信息可以是用于表示图像质量好或坏的信息,也可以是图像质量等级。例如,多个样本图像可以包括图像质量好的样本图像以及图像质量差的样本图像。多个样本图像也可以包括多个属于不同图像质量等级的样本图像。
可以理解的是,在训练质量评价模型时,图像质量的评价因子与图像增强模型的图像处理功能相对应,可以从样本图像中提取与图像质量的评价因子相对应的特征信息,根据提取到的特征信息评价样本图像的图像质量,并输出样本图像对应的预测的图像质量评价信息。
当图像增强模型的图像处理功能为超分辨率时,在训练质量评价模型时,图像质量的评价因子可以为图像的清晰度,可以采用清晰度评价函数评价图像质量。由于相对于模糊图像而言,清晰图像的灰度差异更大,因此,清晰度评价函数的评价因子可以为灰度值,相邻像素点的灰度值差值越大,表示图像的清晰度越高,图像质量越好。
当图像增强模型的图像处理功能为图像去噪时,在训练质量评价模型时,图像质量的评价因子可以为图像噪声,可以采用现有的图像噪声估计算法估计样本图像中的图像噪声,估计出的图像噪声的值越小,表示图像质量越好。
当图像增强模型的图像处理功能为去马赛克时,在训练质量评价模型时,图像质量的评价因子可以为每个像素点的各种色彩的色阶,色阶是表示图像亮度强弱的指数标准,每个颜色的色阶的取值范围都是[0,255];由于邻近像素出现突兀且不自然的强度改变时,意味着图像中存在色彩混叠或拉链效应等,因此,图像中邻近像素点的色阶值的差值越小,表示图像质量越好。
当图像增强模型的图像处理功能为图像复原时,在训练质量评价模型时,图像质量的评价因子可以为图像可见度或清晰度;相邻像素点的灰度值差值越大,表示图像可见度越高或清晰度越高,图像质量越好。例如,对老照片复原时,相邻像素点的灰度值差值越大,清晰度越高;对于去除图像中的雨点或雾的场景,相邻像素点的灰度值差值越大,可见度越高。再例如,对于图像中存在眼镜反光或眼睛反光时,如果相邻像素点的灰度值差值越大,意味着眼镜或眼睛的反光程度越小,眼镜或眼睛对应的图像越清晰,并且五官图像的形变越小,表示图像质量越好。
当图像增强模型的图像处理功能包括去噪、超分辨率、去马赛克中的至少两种时,图像质量的评价因子可以为至少两种图像处理功能各自对应的评价因子的组合。例如,当图像增强模型的图像处理功能为去噪以及超分辨率时,在训练质量评价模型时,图像质量的评价因子可以为图像噪声以及图像的清晰度。
需要说明的是,由于手机在执行S101所获得的第一图像可能存在马赛克、图像噪声、分辨率较低、图像模糊等任一种缺陷,或者至少两种缺陷,因此,手机在执行S102时,可以采用图像检测算法,确定第一图像存在的缺陷,针对第一图像中存在的缺陷,采用相应的图像增强模型对第一图像进行处理,获得第二图像;在执行S103采用质量评价模型对第二图像进行处理时,可以通过与第一图像存在的缺陷相对应的评价因子来评价第二图像的图像质量,得到第二图像的图像质量评价信息。其中,图像检测算法可以用于检测第一图像种是否存在图像噪声、马赛克、模糊区域、图像分辨率差等至少任一种。该图像检测算法可以分别检测图像中是否存在的图像噪声、马赛克、模糊区域或者图像分辨率差的问题,也可以同时检测出图像中是否存在的图像噪声、马赛克、模糊区域和图像分辨率差的问题。
为了便于描述,针对第一图像存在一种问题的情况以第一图像存在马赛克为例进行说明,针对第一图像存在至少两种问题的情况以第一图像存在马赛克和图像噪声为例进行说明。具体如下:
针对第一图像存在马赛克的情况:手机可以通过马赛克检测算法检测第一图像种是否存在马赛克;在检测到第一图像中存在马赛克时,采用图像增强模型对第一图像进行去马赛克处理,得到第二图像,之后,将第二图像输入质量评价模型进行处理,提取第二图像中的每个像素点的所有色彩的色阶值,根据每个像素点的所有色彩的色阶值确定相邻两个像素点的色阶值的差值,并根据相邻两个像素点的色阶值的差值确定第二图像的图像质量评价信息。
相邻两个像素点的色阶值的差值越小,表示图像质量越好。示例性的,当图像质量评价信息为用于表示图像质量的标识信息时,若相邻两个像素点的色阶值的平均差值小于或等于预定的色阶值的差值阈值,则质量评价模型输出的图像质量评价信息为“1”,若相邻两个像素点的色阶值的平均差值大于预定的色阶值的差值阈值,则质量 评价模型输出的图像质量评价信息为“0”。当图像质量评价信息为用于表示图像质量分数时,可以针对相邻两个像素点的色阶值的平均差值设置至少两个差值区间,并为每个差值区间设置相应的分值,从而使得手机能够根据第二图像对应的相邻两个像素点的色阶值的平均差值所属的差值区间,确定第二图像对应的图像质量分数。
其中,马赛克检测算法可以包括但不限于Canny边缘检测算法、基于模板匹配的马赛克检测算法等。
Canny边缘检测算法检测马赛克的原理大致为:采用Canny边缘检测算法对第一图像进行Canny边缘检测,以检测第一图像中的边缘,得到第一图像对应的梯度模图或二值图。由于马赛克区域经过边缘检测后,通常呈现一堆方块状或类方块状的区域,方块和类方块大体可以分为完备的正方形、分别缺一边的不完备正方形,因此,手机可以检测第一图像对应的梯度模图或二值图种是否存在方块状或类方块状的区域,即可确定第一图像是否存在马赛克。
基于模板匹配的马赛克检测算法的原理大致为:采用Canny边缘检测算子对第一图像进行边缘检测,得到第一图像对应的梯度模图或二值图,基于马赛克图像边缘模板来确定第一图像对应的梯度模图或二值图是否存在马赛克,其中,第一图像对应的梯度模图或二值图中任一边缘的形状与马赛克图像边缘模板相匹配,那么存在马赛克。马赛克图像边缘模板可以基于马赛克区域经过边缘检测后,边缘通常呈现规则的正方形这一特点而设置。
针对第一图像存在马赛克和图像噪声的情况:手机可以采用图像噪声检测方法来检测第一图像中存在图像噪声,以及采用马赛克检测算法检测第一图像种是否存在马赛克,在确定第一图像存在图像噪声和马赛克时,通过图像增强模型对第一图像进行去噪处理和去马塞克处理,得到第二图像,采用质量评价模型对第二图像进行处理,提取第二图像对应的用于衡量图像质量的特征信息,对提取到的特征信息进行处理,以评价第二图像的图像质量,获取质量评价模型输出的第二图像对应的图像质量评价信息。第二图像对应的用于衡量图像质量的特征信息包括图像噪声特征信息以及每个像素点的所有色彩的色阶值。此时,第二图像的图像质量的评价因子包括图像噪声的值以及相邻两个像素点的色阶值的差值,图像噪声的值是根据图像噪声特征信息确定的,相邻两个像素点的色阶值的差值是根据每个像素点的所有色彩的色阶值确定的。图像噪声的值越小且相邻两个像素点的色阶值的差值越小,表示图像质量越好。
需要说明的是,示例性的,当图像质量评价信息为用于表示图像质量的标识信息时,若图像噪声的值小于或等于预定的图像噪声阈值,且相邻两个像素点的色阶值的平均差值小于或等于预定的色阶值的差值阈值,则质量评价模型输出的图像质量评价信息为“1”;若图像噪声的值大于预定的图像噪声阈值,或,相邻两个像素点的色阶值的平均差值大于预定的色阶值的差值阈值,则质量评价模型输出的图像质量评价信息为“0”。
示例性的,当图像质量评价信息为图像质量分数,第二图像的图像质量的评价因子包括至少两个时,手机在根据评价因子评价第二图像的图像质量时,可以设置每个评价因子对应的权重值,以及针对每个平均因子对应的值设置不同的区间,并为每个区间设置相应的分数。例如,针对相邻两个像素点的色阶值的平均差值设置至少两个 差值区间,并为每个差值区间设置相应的分值;针对图像噪声的值设置至少两个噪声值区间,并为每个噪声值区间设置相应的分值。手机可以根据每个评价因子对应的值所属的区间,确定每个平均因子对应的分数,根据每个平均因子对应的分数以及每个评价因子的权重值确定第二图像对应的图像质量分数。其中,所有所有评价因子对应的权重值之和为1,每个评价因子对应的权重值可以相同,也可以不同。权重值大的评价因子,表示对图像质量的影响较大。
图像噪声检测方法可以包括基于主成分分析(Principal Component Analysis,PCA)的高斯噪声检测方法、基于信号相关噪声(signal dependent noise,SDN)模型的噪声检测方法等。
可以理解的是,手机在检测到第一图像的分辨率小于或等于预设分辨率阈值,采用图像增强模型对第一图像进行超分辨率处理,得到第二图像之后,将第二图像输入质量评价模型进行处理,提取第二图像对应的用于衡量图像质量的清晰度特征信息,清晰度特征信息可以为像素点的灰度值;手机可以根据每个像素点的灰度值计算相邻两个像素点的灰度值的差值,根据相邻两个像素点的灰度值的差值确定第二图像的图像质量评价信息。例如,可以计算相邻两个像素点的灰度值的平均差值确定第二图像的图像质量评价信息,或通过计算相邻两个像素点的灰度值的平方确定第二图像的图像质量评价信息。其中,相邻两个像素点的灰度值的平均差值越大,或相邻两个像素点的灰度值的平方越大,表示图像质量越好。
手机可以在检测到第一图像中存在图像模糊的区域,采用图像增强模型对第一图像进行图像复原处理之后,将第二图像输入质量评价模型进行处理,提取第二图像中每个像素点的灰度值,并根据每个像素点的灰度值计算相邻两个像素点之间的灰度值的差值,根据相邻两个像素点的灰度值的差值确定第二图像的图像质量评价信息。灰度值的差值越大,表示图像质量越好。
S104、根据所述图像质量评价信息确定输出所述第二图像或者所述第一图像。
图像质量评价信息用于指示第二图像的图像质量。手机可以根据图像质量评价信息判断第二图像是否为第一图像和第二图像中图像质量最好的图像,如果判断结果为第二图像是第一图像和第二图像中图像质量最好的图像,那么手机输出第二图像。如果判断结果为第二图像不是第一图像和第二图像中图像质量最好的图像,那么手机输出第一图像。其中,由于图像质量评价信息可以是用于表示图像质量的标识信息或图像质量分数,因此,当第二图像的图像质量信息为预定标识,或者,第二图像的图像质量分数大于预定分数阈值时,第二图像为第一图像和第二图像中图像质量最好的图像。预定标识可以为“1”、“YES”或“是”。
具体的,在一种可能的实现方式中,当图像质量评价信息是用于表示第二图像的图像质量好或差的标识信息,标识信息可以是数字、文字或字母时,手机可以根据预先建立的对应关系或规则确定第二图像的标识信息所对应的数字、文字或字母所表示的含义,并根据具体含义判断第二图像是否为第一图像和第二图像中图像质量最好的图像,进而根据判断结果输出图像质量最好的图像,以供用户查看。其中,当第二图像的图像质量评价信息为“1”、“YES”或“是”时,表示第二图像的图像质量好,第二图像为第一图像和第二图像中图像质量最好的图像,那么手机输出第二图像;当 第二图像的图像质量评价信息为“0”或“NO”或“否”时,表示第二图像的图像质量差,第一图像为第一图像和第二图像中图像质量最好的图像,手机输出第一图像。
需要说明的是,在S102中获得N个第二图像,图像质量评价信息为对N个第二图像中每一个第二图像的图像质量评价信息,当N≥2,图像质量评价信息是用于表示第二图像的图像质量好或差的标识信息时,手机可以判断N个第二图像中是否存在图像质量评价信息为预定标识的目标第二图像,根据判断结果从第一图像和第二图像中确定图像质量最好的图像,预定标识可以为“1”或“YES”或“是”。若N个第二图像中存在图像质量评价信息为预定标识的目标第二图像,则确定图像质量评价信息为预定标识的目标第二图像为图像质量最好的图像,并输出图像质量评价信息为预定标识的目标第二图像。若N个第二图像中存在多个图像质量评价信息为预定标识的目标第二图像,则选择任一目标第二图像输出。若N个第二图像中不存在图像质量评价信息为预定标识的目标第二图像,则确定第一图像为图像质量最好的图像,输出第一图像。
在另一种可能的实现方式中,当图像质量评价信息是图像质量分数时,手机将第二图像对应的图像质量分数与预定分数阈值进行比较,从而判断第二图像的图像质量是好还是差,根据判断结果从第一图像和第二图像中确定图像质量最好的图像。如果第二图像的图像质量分数大于或等于预定分数阈值,表示第一图像和第二图像中,图像质量最好的图像为第二图像,那么输出第二图像;如果图像质量分数小于预定分数阈值,表示第一图像和第二图像中,图像质量最好的图像为第一图像,那么输出第一图像。当图像质量分数可以是0-100中的任一整数时,预定分数阈值可以是70、75或80,当图像质量分数可以是0-1中的任一整数或者小数时,预定分数阈值可以0.7、0.75或0.8,但并不限于此,也可以根据实际需要设置预定分数阈值。
需要说明的是,在S102中获得N个第二图像,图像质量评价信息为对N个第二图像中每一个第二图像的图像质量评价信息,当N≥2,图像质量评价信息是图像质量分数时,手机可以判断N个第二图像中是否存在图像质量分数大于预定分数阈值的目标第二图像,根据判断结果从第一图像和第二图像中确定图像质量最好的图像。若N个第二图像中存在图像质量分数大于或等于预定分数阈值的目标第二图像,则确定目标第二图像为图像质量最好的图像,输出目标第二图像;若N个第二图像中不存在图像质量分数大于或等于预定分数阈值的目标第二图像,则确定第一图像为图像质量最好的图像,输出第一图像。其中,若N个第二图像中存在至少两个目标第二图像,则选择任一目标第二图像输出,或者输出分数最高的目标第二图像。
下面结合具体的应用场景说明图像处理过程。请一并参阅图6,图6是本申请一实施例提供的一种用户界面的示意图。
如图6所示,用户点击手机中的拍照应用的图标,手机启动拍照应用,用户可以将手机的摄像头对准被拍摄对象所在的位置,并点击拍照按钮触发拍照指令。手机响应于用户触发的拍照指令,获取到图像A,该图像A中眼镜存在反光。手机可以采用图像增强模型对图像A进行处理,比如,对图像A进行超分辨率以及去除反光处理。其中,手机的拍照模式可以为高倍变焦模式。
如果采用图像增强模型对图像A处理后得到图像B1,那么采用图像评价模型对图像B1进行处理,以评价图像B1的图像质量,得到图像B1的图像质量评价信息,根据图像B1的图像质量评价信息从图像A和图像B1中图像质量最好的图像,并输出图像质量最好的图像。当图像质量评价信息通过图像质量分数表示时,手机可以将图像B1的图像质量分数与预定分数阈值进行比较,如果得到的比较结果为图像B1的图像质量分数大于或等于预定分数阈值,那么图像B1为图像A和图像B1中图像质量最好的图像,手机输出图像B1,用户可以看到的图像B1中眼镜没有反光,并且图像B1中的人像的五官或眼镜没有变形。当图像质量评价信息通过“0”或“1”表示时,如果图像B1的图像质量评价信息为“1”,那么图像B1为图像A和图像B1中图像质量最好的图像,手机输出图像B1。具体的,手机输出图像B1的方式可以是将图像B1保存到图库中,还可以在显示界面显示图像B1。
如果采用图像增强模型对图像A处理后得到图像B2,那么采用图像评价模型对图像B2进行处理,以评价图像B2的图像质量,得到图像B2的图像质量评价信息,根据图像B2的图像质量评价信息从图像A和图像B2中图像质量最好的图像,并输出图像质量最好的图像。由于图像B2中眼镜虽然没有反光,但眼镜变形了,当图像质量评价信息通过图像质量分数表示时,手机将图像B2的图像质量分数与预定分数阈值进行比较,得到的比较结果为图像B2的图像质量分数小于预定分数阈值,因此,图像A为图像A和图像B1中图像质量最好的图像,将图像B2丢弃,手机输出图像A。当图像质量评价信息通过“0”或“1”表示时,图像B2的图像质量评价信息为“0”,图像A为图像A和图像B1中图像质量最好的图像,手机输出图像A。具体的,手机输出图像A的方式可以为将图像A保存到图库中,还可以在显示界面显示图像A。
本申请实施例中,手机对获取到的第一图像进行处理之后,得到第二图像,并评价第二图像的图像质量,如果第二图像的图像质量差,输出第一图像给用户;如果第二图像的图像质量好,输出第二图像给用户。通过输出图像质量好的图像给用户查看,图像质量差的图像不会输出给用户,可以减少输出质量差的图像的概率,提高用户体验。
上面介绍了一种评价第二图像的图像质量的图像处理流程,下面将介绍另一种评价第二图像的图像质量的图像处理流程。参见图7,图7是本申请第二实施例提供的一种图像处理方法的示意性流程图。图2以及图7中,质量评价模型用于评价第二图像的图像质量。图2与图7的区别在于,输入质量评价模型的图像不同,以及质量评价模型评价第二图像的图像质量的方法不同。图2的S103中,质量评价模型的输入为第二图像,质量评价模型用于基于第二图像的图像特征信息评价第二图像的图像质量;而图7的S203中,质量评价模型的输入为第一图像以及第二图像,质量评价模型用于基于第一图像的特征信息以及第二图像的特征信息评价第二图像的图像质量。也就是说,图7是将第一图像作为参考图像,获取第二图像的特征信息于第一图像的特征信息之间的差异特征信息,根据差异特征信息评价第二图像的图像质量,这样可以提高第二图像的评价结果的准确度。具体的,图7对应的实施例与图2对应的实施例的区别在于S203,具体如下:
S203、将所述第一图像以及所述第二图像输入质量评价模型进行处理,得到所述 第二图像对应的图像质量评价信息。
手机可以采用质量评价模型从第一图像中提取用于衡量图像质量的第一特征信息,从第二图像中提取用于衡量图像质量的第二特征信息,对第一特征信息以及第二特征信息进行处理,得到第二图像对应的图像质量评价信息,从而实现通过第一特征信息辅助评价第二图像的图像质量。对第一特征信息以及第二特征信息进行处理可以包括:比较第一特征信息以及第二特征信息,得到差异特征信息,根据差异特征信息评价第二图像的图像质量,从而得到第二图像对应的图像质量评价信息。第一特征信息可以是第一图像的全部特征信息,也可以是第一图像的部分特征信息。全部特征信息是第一图像中每个像素点的特征信息;部分特征信息可以是第一图像中的部分像素点的特征信息,例如第一图像中可以反映或代表第一图像的图像质量的像素点。
例如,手机在检测到第一图像的分辨率小于或等于预设分辨率阈值,采用图像增强模型对第一图像进行超分辨率处理,得到第二图像之后,将第一图像以及第二图像输入质量评价模型,可以通过质量评价模型提取第一图像对应的第一清晰度特征信息,以及第二图像对应的第二清晰度特征信息,对第一清晰度特征信息以及第二清晰度特征信息进行处理,得到第二图像的图像质量评价信息。
手机在检测到第一图像中存在图像噪声,采用图像增强模型对第一图像进行去噪处理,得到第二图像之后,将第一图像以及第二图像输入质量评价模型,可以通过质量评价模型提取第一图像对应的第一图像噪声特征信息,以及第二图像对应的第二图像噪声特征信息,对第一图像噪声特征信息以及第二图像噪声特征信息进行处理,得到第二图像的图像质量评价信息。
手机在检测到第一图像中存在马赛克,采用图像增强模型对第一图像进行去马赛克处理,得到第二图像之后,将第一图像以及第二图像输入质量评价模型,可以通过质量评价模型提取第一图像对应的第一色阶特征信息,以及第二图像对应的第二色阶特征信息,对第一色阶特征信息以及第二色阶特征信息进行处理,得到第二图像的图像质量评价信息。
手机可以在检测到第一图像中存在图像模糊的区域,采用图像增强模型进行图像复原处理,得到第二图像之后,将第一图像以及第二图像输入质量评价模型,可以通过质量评价模型提取第一图像对应的第一清晰度特征信息,以及第二图像的第二清晰度特征信息,对第一清晰度特征信息以及第二清晰度特性信息进行处理,得到第二图像的图像质量评价信息。
在S203中,图像质量评价信息可以是图像质量分数,图像质量评价信息也可以是用于表示第二图像的图像质量好或差的标识信息,例如,“0”、或“1”,“0”表示第二图像的图像质量差,“1”表示第二图像的图像质量好;标识信息也可以是“是”或“否”,“是”表示第二图像的图像质量好,“否”表示第二图像的图像质量差。标识信息也可以是“YES”或“NO”,“YES”表示第二图像的图像质量好,“NO”表示第二图像的图像质量差。
当图像质量评价信息是图像质量分数时,图像质量分数可以是0-100中的任一整数,也可以是0、1以及0-1之间的任一数值。当图像质量分数大于或等于预定分数阈值时,表示图像质量好;当图像质量分数大于或等于预定分数阈值小于预定分数阈值 时,表示图像质量差。其中,S203中的质量评价模型的输入为原始图像,原始图像对应的样本图像、样本图像对应的标记的图像质量评价信息,质量评价模型的输出为样本图像对应的预测的图像质量评价信息。原始图像对应的样本图像可以是原始图像经过图像增强模型处理后输出的图像,也可以是对原始图像进行退化处理后得到的图像,此处不做限制。
需要说明的是,如果在S202中获得N个第二图像,S203具体为:将所述第一图像以及所述N个第二图像输入质量评价模型进行处理,得到所述第二图像对应的图像质量评价信息。所述图像质量评价信息为对所述N个第二图像中每一个第二图像的图像质量评价信息。
本实施例,可以通过第一图像的特征信息辅助评价第二图像的图像质量,由于手机可以将第一图像作为参考图像,获取第二图像的特征信息于第一图像的特征信息之间的差异特征信息,根据差异特征信息评价第二图像的图像质量,可以提高第二图像的图像质量评价信息的准确度。
在图2以及图7中,介绍了通过质量评价模型评价第二图像的图像质量的方案;下面介绍通过质量评价模型评价第一图像以及第二图像的图像质量的方案。参见图8,图8是本申请第三实施例提供的一种图像处理方法的示意性流程图。图7与图8中,质量评价模型的输入均为第一图像以及第二图像。图8与图7的区别在于,图7中质量评价模型用于评价第二图像的图像质量,而图8中的质量评价模型用于评价第一图像以及第二图像的图像质量。具体的,图8对应的实施例与图7对应的实施例的区别在于S303~S306,具体如下:
S303、将所述第一图像以及所述第二图像输入质量评价模型进行处理,得到针对所述第一图像以及所述第二图像的图像质量评价信息。
具体的,可以采用质量评价模型提取到第一图像对应的第一特征信息,以及第二图像对应的第二特征信息,对第一特征信息以及第二特征信息进行处理,得到针对第一图像以及第二图像的图像质量评价信息。对第一特征信息以及第二特征信息进行处理可以包括:比较第一特征信息以及第二特征信息,得到差异特征信息,根据差异特征信息评价第二图像的图像质量,从而得到第二图像对应的图像质量评价信息。第一特征信息可以是第一图像的全部特征信息,也可以是第一图像的部分特征信息。全部特征信息是第一图像中每个像素点的特征信息;部分特征信息可以是第一图像中的部分像素点的特征信息,例如第一图像中可以反映或代表第一图像的图像质量的像素点。
图像质量评价信息可以是用于表示第二图像的图像质量是否比第一图像的图像质量好的标识信息,标识信息可以是数字、字母、文字等。图像质量评价信息也可以包括:所述第一图像对应的第一图像质量评价信息,以及所述第二图像对应的第二图像质量评价信息。例如,图像质量评价信息可以包括用于表示第一图像的图像质量是好还是差的第一标识以及用于表示第二图像的图像质量是好还是差的第二标识;还可以包括第一图像对应的第一图像质量分数以及第二图像对应的第二图像质量分数。
其中,当图像质量评价信息是用于表示第二图像的图像质量是否比第一图像的图像质量好的标识信息时,图像质量评价信息可以是通过比较第一特征信息以及第二特 征信息得到差异特征信息,根据差异特征信息而确定;第一特征信息可以是第一图像的全部特征信息,也可以是第一图像的部分特征信息。可以用“0”、“NO”或“否”,表示图像第二的图像质量比第一图像的图像质量差,用“1”、“YES”或“是”,表示第二图像的图像质量比第一图像的图像质量好,其它数字(例如,数字2)表示第二图像的图像质量与第一图像的图像质量相同。
当图像质量评价信息包括:第一图像对应的第一图像质量评价信息,以及第二图像对应的第二图像质量评价信息时,第一图像质量评价信息可以由第一图像的第一图像质量评价信息确定;第二图像质量评价信息可以由第二图像的第二特征信息确定,也可以通过比较第一特征信息以及第二特征信息得到差异特征信息,根据差异特征信息而确定。第一图像质量评价信息以及第二图像质量评价信息,可以为图像质量分数。图像质量分数可以是0-100中的任一整数,也可以是0-1中的任一数值。
在S303中的质量评价模型的输入为:原始图像、原始图像对应的标记的图像质量评价信息,原始图像对应的样本图像,以及样本图像对应的标记的图像质量评价信息,质量评价模型的输出为原始图像对应的预测的第一图像质量评价信息,样本图像对应的预测的第二图像质量评价信息。原始图像对应的样本图像可以是原始图像经过图像增强模型处理后输出的图像,也可以是对原始图像进行退化处理后得到的图像,此处不做限制。
需要说明的是,如果在S302中获得N个第二图像,S303具体为:将所述第一图像以及所述N个第二图像输入质量评价模型进行处理,得到所述第二图像对应的图像质量评价信息。所述图像质量评价信息为对所述第一图像和N个第二图像中每一个第二图像的图像质量评价信息。
S304、根据所述图像质量评价信息确定输出所述第二图像或者所述第一图像。
手机可以根据图像质量评价信息判断第二图像是否为第一图像和第二图像中图像质量最好的图像,如果判断结果为第二图像是第一图像和第二图像中图像质量最好的图像,那么输出第二图像。如果判断结果为第二图像不是第一图像和第二图像中图像质量最好的图像,那么输出第一图像。由于图像质量评价信息可以是用于表示图像质量的标识信息或图像质量分数,因此,图像质量最好的图像可以是图像质量评价信息为预定标识的图像,或者图像质量最好的图像也可以是图像质量分数大于或等于预定分值阈值的图像,或者图像质量最好的图像还可以是第一图像和第二图像中图像质量分数最高的图像。
当图像质量评价信息是用于表示第二图像的图像质量是否比第一图像的图像质量好的标识信息时,手机可以根据预先建立的对应关系或规则确定标识信息中的数字、文字或字母所表示的含义,判断第二图像的图像质量是否比第一图像的图像质量好,从而确定第一图像和第二图像中图像质量最好的图像,以输出图像质量最好的图像。当图像质量评价信息为“1”、“YES”或“是”时,表示第二图像的图像质量比第一图像的图像质量好,第二图像为第一图像和第二图像中图像质量最好的图像,那么输出第二图像;当图像质量评价信息为“0”、“NO”或“否”时,表示第二图像的图像质量比第一图像的图像质量差,第一图像为第一图像和第二图像中图像质量最好的图像,那么输出第一图像;当图像质量评价信息为2时,表示第二图像的图像质量与 第一图像的图像质量相同,从第一图像和第二图像中选择任一图像输出。
需要说明的是,如果在S302中获得N个第二图像,图像质量评价信息为对第一图像和N个第二图像中每一个第二图像的图像质量评价信息,那么当N≥1,图像质量评价信息通过标识信息表示时,图像质量评价信息可以包括用于表示第一图像的图像质量的第一标识以及用于分别表示第二图像的图像质量的第二标识,第二标识的数量为N个。手机可以判断第一图像和N个第二图像中是否存在图像质量评价信息为预定标识的图像,根据判断结果从第一图像和第二图像中确定图像质量最好的图像。预定标识可以为“1”或“YES”或“是”。其中,若第一图像的图像质量评价信息为预定标识,且N个第二图像中存在至少一个图像质量评价信息为预定标识的目标第二图像,则从第一图像和目标第二图像中选择任一图像输出。若第一图像的图像质量评价信息不是预定标识,且N个第二图像中存在至少一个图像质量评价信息为预定标识的目标第二图像,则从目标第二图像中选择任一图像输出。若第一图像的图像质量评价信息不是预定标识,且N个第二图像中不存在图像质量评价信息为预定标识的目标第二图像,则输出第一图像。
当图像质量评价信息为分数,图像质量评价信息包括:第一图像对应的第一图像质量分数以及N个第二图像各自对应的第二图像质量分数,N≥1时,手机可以根据第一图像的第一图像质量分数和N个第二图像的第二图像质量分数,从第一图像和N个第二图像中,确定图像质量分数最高的图像为第一图像和N个第二图像中图像质量最好的图像,以输出分数最高的图像。手机还可以判断第一图像和N个第二图像中是否存在图像质量分数大于预定分数阈值的图像,根据判断结果从第一图像和第二图像中确定图像质量最好的图像。若低于图像和N个第二图像中存在图像质量分数大于或等于预定分数阈值的图像,输出图像质量分数大于或等于预定分数阈值的图像。当图像质量分数大于或等于预定分数阈值的图像的数量为至少两个时,可以从中选择任一个进行输出。
例如,手机可以将第一图像对应的第一图像质量分数以及N个第二图像对应的第二图像质量分数进行比较,筛选出图像质量分数最高的图像,从而得到图像质量最好的图像。当任一第二图像的图像质量分数最高时,输出第二图像,当第一图像的图像质量分数最高时,输出第一图像。当图像质量分数最高的第二图像至少有2个时,从中选择任一个第二图像进行输出。需要说明的是,当第一图像和任一第二图像的图像质量分数并列第一时,可以从图像质量分数并列第一的第一图像和第二图像中选择任一个输出。
再例如,当图像质量信息通过图像质量分数来表示,通过至少两个不同的分数区间表示不同的图像质量时,手机可以将第一图像对应的第一图像质量分数、第二图像对应的第二图像质量分数,与分数区间的阈值进行比较,从而确定第一图像质量分数所属的分数区间和第二图像质量分数所属的分数区间。如果第一图像质量分数和第二图像质量分数属于相同的分数区间,表示第二图像的图像质量与第一图像的图像质量相同,手机可以从第一图像和第二图像中选择任一图像输出。如果第一图像质量分数和第二图像质量分数所属的分数区间不同时,再通过比较第一图像质量分数和第二图像质量分数,来确定第一图像和第二图像中哪个图像的图像质量最好。其中,属于分 数高的分数区间的图像的图像质量最好。具体的,如果根据比较结果确定第一图像质量分数和第二图像质量分数属于不同的分数区间,并且第二图像质量分数大于第一图像质量分数时,表示第二图像的图像质量比第一图像的图像质量好,那么输出第二图像;如果根据比较结果确定第一图像质量分数和第二图像质量分数属于不同的分数区间,并且第二图像质量分数小于第一图像质量分数时,表示第二图像的图像质量比第一图像的图像质量差,输出第一图像。
需要说明的是,当第二图像的数量为N个,第一图像属于第一分数区间,N个第二图像属于第二分数区间时,如果第一图像的第一图像质量分数小于第二图像的第二图像质量分数,那么手机可以确定属于第二区间的N个第二图像均为第一图像和N第二图像中图像质量最好的图像,输出任一第二图像,或者输出图像质量分数最高的第二图像。如果如果第一图像的第一图像质量分数大于第二图像的第二图像质量分数,那么第一图像是图像质量最好的图像,输出第一图像。
当第二图像的数量为N个,部分第二图像和第一图像属于第一分数区间,部分第二图像属于第二分数区间时,手机可以将第一分数区间的阈值以及第二分数区间的阈值进行比较,如果第一分数区间的最大阈值小于或等于第二分数区间的最小阈值,那么属于第二分数区间的第二图像为图像质量最好的图像,可以输出属于第二分数区间的任一第二图像;如果第一分数区间的最小阈值大于第二分数区间的最大阈值,那么属于第一分数区间的第一图像以及第二图像为图像质量最好的图像,可以输出属于第一分数区间的任一第二图像或第一图像。
示例性的,如果在S302中对图6中的图像A处理,获得的第二图像为图6中的图像B1。图6中,图像A中存在眼镜反光,眼镜所在区域的部分图像比较模糊;图像B1中的眼镜不存在反光,图像的清晰度较高。手机将图6中的图像A以及图像B1输入图像质量模型进行处理,得到图像A的图像质量分数以及图像B1的图像质量分数。手机比较图像B的图像质量分数与图像A的图像质量分数,得到的比较结果为图像B的图像质量分数大于图像A的图像质量分数,因此,手机输出图像B1。此时,用户看到得图像中眼镜不存在反光,图像的清晰度较高。
如果在S302中对图6中的图像A处理,获得的第二图像为图6中的图像B2,图6中,图像A中存在眼镜反光,眼镜所在区域的部分图像比较模糊;图像B2中的眼镜不存在反光,图像的清晰度较高,但是图像B2中的眼镜发生变形了。手机将图6中的图像A以及图像B2输入图像质量模型进行处理,得到图像A的图像质量分数以及图像B2的图像质量分数,手机比较图像B的图像质量分数与图像A的图像质量分数,得到的比较结果为图像A的图像质量分数大于图像B的图像质量分数。因此,图像A的图像质量高于图像B2的图像质量,手机输出图像A。此时,用户看到得图像中存在眼镜反光,眼镜所在区域的部分图像比较模糊。
本实施例,可以评价第一图像和第二图像的图像质量,得到针对第一图像以及第二图像的图像质量评价信息,可以通过图像质量评价信息,来判断第二图像的图像质量是否比第一图像的图像质量好,能够更准确地获知由第一图像经过图像增强模型处理后得到的第二图像的图像质量是否变差。
上面在图2、图7~图8对应的实施例中,介绍了第一图像经过图像增强模型处理 得到第二图像,并采用质量评价模型评价第二图像的图像质量。下面介绍另一种方案:先采用质量评价模型评价第一图像的图像质量,根据评价结果确定是否对第一图像进行处理。其中,当第一图像的图像质量较好时,采用图像增强模型对第一图像进行处理,输出第二图像;当第一图像的图像质量较差时,输出第一图像给用户查看,以减少因处理图像所占用的资源的消耗,可以提高手机的数据处理速度。参见图9,图9是本申请第四实施例提供的一种图像处理方法的示意性流程图。具体包括以下步骤:
S401、获取第一图像。
手机获取第一图像,可以在人像模式下、景色模式、室内模式、长焦模式(下面称为高倍变焦模式)等等下获取该第一图像。为了便于说明,下面以在高倍变焦模式下进行拍照为例进行说明。用户启动手机中的拍照应用,可以控制手机进入高倍变焦模式或长焦拍摄状态,拍摄人像。高倍变焦模式包括但不限于3倍变焦、5倍变焦、10倍变焦、30倍变焦或50倍变焦等。变焦的倍数具体可以根据高倍变焦摄像头的变焦能力而定。
第一图像可以是手机在高倍变焦模式下获取到的预览图像,也可以是手机在高倍变焦模式下响应于用户的拍照质量而获取到的照片。
由于在高倍变焦模式下拍摄人像照片时,获取到的第一图像,可能会因抖动而导致图像失真、图像中的人像的眼睛等五官变形或者图像较模糊(例如,较难辨认被拍摄对象)等,因此,需要评价第一图像的图像质量。
S402、将所述第一图像输入质量评价模型进行处理,得到所述第一图像的图像质量分数。
手机将第一图像输入质量评价模型进行处理,从第一图像中提取用于衡量图像质量的特征信息,并对第一图像的特征信息进行处理,得到第一图像对应的图像质量分数。图像质量分数用于表示第一图像的图像质量。
S403、判断所述第一图像的图像质量分数是否大于或等于预设阈值。
预设阈值可以为0.25,但并不限于此,还可以根据实际情况设置其他值,此处不做限制。手机将第一图像的图像质量分数与预设阈值进行比较,从而根据比较结果判断第一图像是否符合要求。当图像质量分数大于或等于预设阈值时,表示第一图像的图像质量较好,第一图像符合要求,可以继续对第一图像进行处理,执行S404。当图像质量分数小于预设阈值时,表示第一图像的图像质量较差,第一图像不符合要求,大多数情况下即使对第一图像进行处理也无法提高图像质量,为了节省手机的资源,执行S405。
S404、当所述第一图像的图像质量分数大于或等于预设阈值时,将所述第一图像输入图像增强模型进行处理,得到第二图像,显示或保存所述第二图像。
当图像质量分数大于或等于预设阈值时,手机将第一图像输入图像增强模型进行处理,得到第二图像,并显示或保存第二图像。
示例性的,当图像质量分数大于或等于预设阈值时,手机可以采用人脸识别算法检测第一图像中是否包括人脸图像,如果第一图像中包括人脸图像,手机将第一图像输入图像增强模型,通过图像增强模型对人脸图像进行超分辨率处理,增强第一图像中的人脸图像的清晰度,得到第二图像,并显示或保存第二图像。当然,还可以通过 图像增强模型对进行超分辨率处理,增强除人脸之外的图像的清晰度,得到第二图像,然后显示或保存第二图像以供用户查看;除人脸之外的图像可以是人像中除了脸部的其他身体部位的图像,比如,人的肢体、头发、发饰、着装等,还可以是第一图像中除了人像之外的其他景物、建筑物等的图像。或者,如果第一图像中不包括人脸图像,手机可以通过图像增强模型对第一图像进行超分辨率处理,增强第一图像的整体的清晰度得到第二图像,然后显示或保存第二图像,以供用户查看。
在一种可能的实现方式中,手机可以在检测到第一图像的分辨率小于或等于预设分辨率阈值时,采用图像增强模型对第一图像进行超分辨率处理得到第二图像,第二图像的分辨率大于第一图像的分辨率;在检测到第一图像中存在图像噪声时,采用图像增强模型对第一图像进行去噪处理得到第二图像;在检测到第一图像中存在图像模糊的区域时,采用图像增强模型对第一图像进行复原处理,比如,第一图像可以是翻拍的老照片、有雨线(雨点)图像、有雾图像,第一图像可能会因色彩褪色、损坏、有雨点或雾等原因导致图像模糊。
需要说明的是,在一种可能的实现方式中,当所述第一图像的图像质量分数大于或等于预设阈值时,还可以将第一图像输入M个图像增强模型进行处理,得到N个第二图像,并显示或保存N个第二图像。N和M为正整数,M个图像增强模型中的每个图像增强模型均不同,N个第二图像也存在不同。采用同一个图像增强模型对第一图像进行处理时,可以获得一个第二图像,也可以获得至少两个第二图像,此处不做限制。例如,当采用图像增强模型对第一图像处理至少两次时,可以得到至少两个第二图像。此时,手机可以保存N个第二图像,也可以显示N个第二图像,以便用户选择需要保存的图像。显示N个第二图像的方式不做限制。
S405、当所述第一图像的图像质量分数小于预设阈值时,显示或保存第一图像。
下面结合具体场景说明对本实施例的图像处理过程。
例如,手机启动拍照应用,如果检测到进入高倍变焦模式时,可以采集第一预览图像;通过质量评价模型评价第一预览图像的图像质量,得到第一预览图像的图像质量分数;手机将第一预览图像的图像质量分数与预设阈值进行比较,如果第一预览图像的图像质量分数大于或等于预设阈值,表示第一预览图像的图像质量较好。手机检测第一照片中是否包括人脸图像,如果手机在第一预览图像中检测到人脸图像,那么通过图像增强模型对第一预览图像中的人脸图像进行超分辨率处理,以增强第一预览图像中的人脸图像的清晰度,得到第二预览图像,并在预览界面显示第二预览图像。如果第一预览图像的图像质量分数小于预设阈值,表示第一预览图像的图像较差,不再对第一预览图像做处理,在预览界面显示第一预览图像,以减少因处理图像而消耗的资源,提高手机的处理速度。
再例如,手机启动拍照应用,进入高倍变焦模式之后,如果检测到用户触发的拍照指令,那么响应于该拍照指令获取第一照片,通过质量评价模型评价第一照片的图像质量,得到第一照片的图像质量分数;手机将第一照片的图像质量分数与预设阈值进行比较,如果第一照片的图像质量分数大于或等于预设阈值,表示第一照片的图像质量较好。手机检测第一照片中是否包括人脸图像,如果在第一照片中检测到人脸图像,那么通过图像增强模型对第一照片中的人脸图像进行超分辨率处理,以增强第一 照片中的人脸图像的清晰度,得到第二照片,并将第二照片保存至图库以供用户查看。如果第一照片的图像质量分数小于预设阈值,表示第一照片的图像较差,不再对第一照片做处理,将第一照片保存至图库以供用户查看,以减少因处理图像而消耗的资源,提高手机的数据处理速度。
可以理解的是,本实施例在S401~S405中是以图像质量分数表示第一图像的图像质量为例进行说明,在一种可能的实现方式中,也可以用“0”或“1”来表示第一图像的图像质量。其中,可以用“0”表示第一图像的图像质量差,第一图像不符合要求;用“1”表示第一图像的图像质量好,第一图像符合要求。当通过质量评价模型评价第一图像的图像质量时,如果手机检测到质量评价模型输出“1”,将第一图像输入图像增强模型进行处理,得到第二图像,并输出第二图像;如果手机检测到质量评价模型输出“0”时,输出第一图像。
本实施例中,获取到第一图像时,先评价第一图像的图像质量,如果第一图像的图像质量较差,那么将第一图像显示给用户查看,或将第一图像保存至图库;如果第一图像的图像质量较好,那么对第一图像进行处理得到第二图像,并将第二图像显示给用户查看,或将第二图像保存至图库。通过节省因处理图像质量差的图像而消耗的资源,可以提高手机的数据处理速度。
参见图10,图10是本申请第五实施例提供的一种图像处理方法的示意性流程图。图10与图9的区别在于,图10中在得到第二图像之后,还采用质量评价模型评价第二图像的图像质量,如果第二图像的图像质量分数大于或等于预定分数阈值,输出第二图像;如果第二图像的图像质量分数小于预定分数阈值,输出第一图像。具体的,图10与图9的区别在于S406~S407,具体如下:
S406、将所述第二图像输入质量评价模型进行处理,得到所述第二图像的图像质量分数。
S406中评价第二图像的图像质量的方法,与S402中评价第一图像的图像质量的方法相同,此处不赘述。
需要说明的是,如果在S404中获取到N个第二图像且N≥2,那么,在S406中,输入质量评价模型的第二图像的数量为N。
S407、判断所述第二图像的图像质量分数是否大于或等于预定分数阈值。
当采用0-100之间的数值来表示图像质量分数时,预定分数阈值可以为75,但并不限于此,也可以为70、80、85或其他值,具体可以根据实际应用过程中,图像质量较好时对应的分数进行设置,此处不做限制。
当采用0-1之间的数值来表示图像质量分数时,预定分数阈值可以为0.75。
S408、当第二图像的图像质量分数大于或等于预定分数阈值时,显示或保存第二图像。
S409、当第二图像的图像质量分数小于预定分数阈值时,显示或保存第一图像。
需要说明的是,在本实施例中,以质量评价模型的输入图像为第二图像,质量评价模型的输出为第二图像对应的图像质量分数为例,说明如何输出第一图像和第二图像中图像质量最好的图像的实现方式。在一种可能的实现方式中,当质量评价模型的输入图像为第二图像时,质量评价模型的输出也可以为用于表示第二图像的图像质量 的标识信息,手机可以根据第二图像对应的标识信息显示或保存第一图像和第二图像中图像质量最好的图像,标识信息可以是数字、字母、文字等。具体实现方法参见图6对应的实施例中S103~S104的相关描述,此处不赘述。在另一种可能的实现方式中,质量评价模型的输入图像可以为第一图像以及N个第二图像,质量评价模型的输出为N个第二图像各自对应的图像质量分数或用于表示图像质量的标识信息,N为正整数;手机可以根据N个第二图像各自对应的图像质量分数或用于表示图像质量的标识信息,输出第一图像和N个第二图像中图像质量最好的图像。具体实现方法参见图7对应的实施例中S203~S204的相关描述,此处不赘述。在另一种可能的实现方式中,质量评价模型的输入图像为第一图像以及N个第二图像时,质量评价模型的输出还可以为第一图像和N个第二图像,各自对应的图像质量分数或用于表示图像质量的标识信息,手机可以根据第一图像和N个第二图像,各自对应的图像质量分数或用于表示图像质量的标识信息,输出第一图像和N个第二图像中图像质量最好的图像。具体实现方法参见图8对应的实施例中S303~S304的相关描述,此处不赘述。图像质量最好的图像是第一图像和N个第二图像中,图像质量评价信息为预定标识的图像,或者图像质量评价信息对应的分数大于预定分数阈值的图像。预定标识可以为“1”、“Y”或“是”。
本实施例中,在采用图像增强模型对第一图像处理前,采用质量评价模型评价第一图像的图像质量;当第一图像的图像质量较差时,输出第一图像给用户查看;当第一图像的图像质量较好时,将第一图像输入图像增强模型进行处理得到第二图像,采用质量评价模型评价第二图像的图像质量,如果第二图像的图像质量好,输出第二图像给用户查看,如果第二图像的图像质量差,输出第一图像给用户查看。这种方式,可以减少输出质量差的图像的概率,提高用户体验。
可选的,本申请列举了一种训练质量评价模型的流程,参见图11,图11是本申请实施例提供的一种训练质量评价模型的示意性流程图。质量评价模型可以通过以下步骤训练得到:
S001、获取训练样本集,所述训练样本集包括多个训练样本,每个训练样本具有标记的图像质量评价信息。
可选的,当需要采用测试样本集对训练后的质量评价模型进行测试时,在S006之前还包括:获取测试样本集,所述测试样本集包括多个测试样本,每个测试样本具有标记的图像质量评价信息。
多个训练样本对应的标记的图像质量评价信息不完全相同,多个测试样本对应的标记的图像质量评价信息不完全相同。也就是说,训练样本集中包括图像质量不同的训练样本,测试样本集中包括图像质量不同的测试样本。
在一种可能的实现方式中,例如,在训练如图2、图9和图10中的质量评价模型时,一个训练样本包括一个训练样本图像以及训练样本图像对应的标记的图像质量评价信息;一个测试样本包括一个测试样本图像以及测试样本图像对应的标记的图像质量评价信息。在训练过程中,需要评价训练样本图像的图像质量;在测试过程中,需要评价测试样本图像的图像质量。图2中的质量评价模型在采用训练后的质量评价模型评价图像质量时,输入质量评价模型的图像为第一图像输入图像增强模型进行处理 后得到的第二图像。图9和图10中的质量评价模型在采用训练后的质量评价模型评价图像质量时,输入质量评价模型的图像是没有经过图像增强模型处理的图像。
在另一种可能的实现方式中,例如,在训练如图7中的质量评价模型时,一个训练样本包括一个原始训练图像、至少一个对原始训练图像处理后得到的训练样本图像,训练样本图像具有标记的图像质量评价信息。一个测试样本包括一个原始测试图像、至少一个对原始测试图像处理后得到的测试样本图像,测试样本图像具有标记的图像质量评价信息。原始训练图像用于辅助评价训练样本图像的图像质量。原始测试图像用于辅助评价测试样本图像的图像质量。在采用训练后的质量评价模型评价图像质量时,输入质量评价模型的图像包括输入图像增强模型的第一图像以及图像增强模型输出的第二图像,质量评价模型输出第二图像对应的图像质量评价信息,第一图像用于辅助评价第二图像的图像质量。
其中,训练样本图像可以采用图像增强模型对原始训练图像处理后得到,测试样本图像可以采用图像增强模型对原始测试图像处理后得到。
训练样本图像也可以对原始训练图像进行退化处理后得到,测试样本图像也可以是拍摄的图片或从互联网下载的图片。例如,采用单反相机拍摄多张图像质量较好的原始训练图像,对原始训练图像进行模糊处理、加噪处理或加马赛克处理,得到训练样本图像,不同的训练样本图像可以由同一张原始训练图像得到,也可以由不同的原始训练图像得到,此处不做限制。将手机拍摄的图片作为原始测试样本,对原始测试样本退化处理后得到测试样本图像。
在另一种可能的实现方式中,例如,在训练如图8中的质量评价模型时,训练样本中的原始训练图像还可以具有标记的图像质量评价信息,测试样本中的原始测试图像还可以具有标记的图像质量评价信息。在训练过程中,需要评价原始训练图像以及训练样本图像各自的图像质量;在测试过程中,需要评价原始测试图像以及测试样本图像各自的图像质量。在采用训练后的质量评价模型评价图像质量时,输入质量评价模型的图像包括输入图像增强模型的第一图像以及图像增强模型输出的第二图像,质量评价模型输出第一图像以及第二图像各自对应的图像质量评价信息。
可以理解的是,多个训练样本图像对应的标记的图像质量评价信息不完全相同,也就是说,训练样本集中包括图像质量不同的训练样本图像。多个测试样本图像对应的标记的图像质量评价信息不完全相同。训练样本图像与测试样本图像不同。
S002、将训练样本输入初始的质量评价模型进行处理,得到每个训练样本对应的预测的图像质量评价信息。
将训练样本输入初始的质量评价模型,采用初始的质量评价模型从训练样本中提取用于衡量图像质量的特征信息,分析提取到的特征信息,得到训练样本对应的预测的图像质量评价信息。
在一种可能的实现方式中,当一个训练样本包括一个训练样本图像以及训练样本图像对应的标记的图像质量评价信息时,用于衡量图像质量的特征信息从训练样本图像中提取得到,训练样本对应的预测的图像质量评价信息,为训练样本图像对应的预测的图像质量评价信息。
在另一种可能的实现方式中,当一个训练样本包括一个原始训练图像、至少一个 对原始训练图像处理后得到的训练样本图像,训练样本图像具有标记的图像质量评价信息时,可以采用初始的质量评价模型处理训练样本的原始训练图像以及训练样本图像,确定训练样本图像对应的预测的图像质量评价信息。具体的,可以采用初始的质量评价模型从原始训练图像中提取用于衡量图像质量的第一特征信息,从训练样本图像中提取用于衡量图像质量的第二特征信息,分析同一个训练样本对应的第一特征信息以及第二特征信息,确定该训练样本中的训练样本图像对应的预测的图像质量评价信息。由于这种方式可以通过原始训练图像的特征信息辅助评价训练样本图像的图像质量,可以更准确地评价训练样本图像相对于原始训练图像而言,图像质量变得更好还是更差,可以提高训练样本图像的预测的图像质量评价信息的准确度。
在另一种可能的实现方式中,当训练样本中的原始训练图像也具有标记的图像质量评价信息时,可以采用初始的质量评价模型分别对训练样本的原始训练图像和样本训练图像进行处理,确定原始训练图像对应的预测的第一图像质量评价信息,以及确定样本训练图像对应的预测的第二图像质量评价信息。具体的,可以采用初始的质量评价模型提取到同一个训练样本的原始训练图像对应的第一特征信息,以及训练样本图像对应的第二特征信息,分析第一特征信息得到原始训练图像对应的预测的第一图像质量评价信息,分析第二特征信息得到样本训练图像对应的预测的第二图像质量评价信息。
S003、根据训练样本对应的标记的图像质量评价信息以及预测的图像质量评价信息,确定初始的质量评价模型的第一评价准确度。
可以比较训练样本对应的标记的图像质量评价信息以及预测的图像质量评价信息,根据各训练样本对应的比较结果确定初始的质量评价模型的第一评价准确率。例如,根据各训练样本对应的比较结果,筛选出标记的图像质量评价信息以及预测的图像质量评价信息相同或匹配的目标训练样本,并计算目标训练样本的数量与参与训练的训练样本的总数的比值,得到初始的质量评价模型的第一评价准确度。其中,第一评价准确度=L/K,L≤K。L表示:在本次训练中,标记的图像质量评价信息以及预测的图像质量评价信息相同或匹配的目标训练样本的数量;K表示参与本次训练的训练样本的总数。
在一种可能的实现方式中,当一个训练样本包括一个训练样本图像以及训练样本图像对应的标记的图像质量评价信息时,在根据训练样本图像确定训练样本图像对应的预测的图像质量评价信息之后,可以比较训练样本图像对应的标记的图像质量评价信息以及预测的图像质量评价信息,得到训练样本对应的比较结果。
在另一种可能的实现方式中,当一个训练样本包括一个原始训练图像、至少一个对原始训练图像处理后得到的训练样本图像,训练样本图像具有标记的图像质量评价信息时,在根据原始训练图像以及训练样本图像确定训练样本图像对应的预测的图像质量评价信息之后,可以比较训练样本图像对应的标记的图像质量评价信息以及预测的图像质量评价信息,得到训练样本对应的比较结果。
在另一种可能的实现方式中,当训练样本中的原始训练图像也具有标记的图像质量评价信息时,在确定训练样本的原始训练图像对应的预测的第一图像质量评价信息,以及确定训练样本的样本训练图像对应的预测的第二图像质量评价信息之后,比较原 始训练图像对应的标记的图像质量评价信息以及预测的第一图像质量评价信息,得到第一比较结果,以及比较训练样本图像对应的标记的图像质量评价信息以及预测的第二图像质量评价信息,得到第二比较结果;根据同一个训练样本的第一比较结果以及第二比较结果得到该训练样本对应的比较结果。其中,当第一比较结果与第二比较结果均为相同或匹配时,该训练样本对应的比较结果为相同或匹配。当第一比较结果或第二比较结果为不相同或不匹配时,该训练样本对应的比较结果为不相同或不匹配。
示例性地,还可以采用预设的损失函数计算训练样本对应的标记的图像质量评价信息以及预测的图像质量评价信息之间的损失值,可以通过损失值表示初始的质量评价模型的评价准确度。损失值越小,初始的质量评价模型的评价准确度越高。可以理解的是,当计算得到同一训练样本的原始训练图像对应的第一损失值和样本训练图像对应的第二损失值时,以较大的损失值作为该训练样本的损失值。损失函数包括但不限于交叉熵损失函数。
S004、判断所述第一评价准确度是否大于或等于第一准确度阈值。
第一准确度阈值用于衡量初始的质量评价模型的评价准确率是否符合要求。第一评价准确度小于第一准确度阈值时,初始的质量评价模型的评价准确度不符合要求,执行S005。第一评价准确度大于或等于第一准确度阈值时,初始的质量评价模型的评价准确度已符合要求,可以执行S006,也可以跳转至S008,结束训练。
第一准确度阈值可以为85%、90%或95%等,但并不限于此,可以根据实际要求进行设置,此处不做限制。
可以理解的是,当通过损失值表示初始的质量评价模型的第一评价准确度时,可以设置第一准确度阈值对应的损失值阈值,当损失值大于损失值阈值时,初始的质量评价模型的评价准确度不符合要求,执行S005;当损失值小于或等于损失值阈值时,初始的质量评价模型的评价准确度已符合要求,可以执行S006,也可以跳转至S008,结束训练。
S005、调整初始的质量评价模型的参数,返回S002。
当第一评价准确度小于第一准确度阈值时,调整初始的质量评价模型的参数,之后,跳转至S002,继续执行S002~S004,以继续训练初始的质量评价模型。
调整参数的方法包括但不限于随机梯度下降算法、动力更新算法等。
可以理解的是,当第N次执行S002采用的训练样本,与第N+1次执行S002采用的训练样本可以相同,也可以不同。例如,在第一次执行S002时采用训练样本1~500进行训练,第二次执行S002时采用样本501~1000进行训练。
可选的,为了验证质量评价模型的准确度,还可以采用测试集对经过训练的质量评价模型进行测试。当S004中的判断结果为第一评价准确度大于或等于第一准确度阈值时,还可以执行S006~S008。需要说明的是,S006~S007为可选的步骤,也就是说在一种可能的实现方式中,训练质量评价模型的流程可以包括S001~S005、S008;在另一种可能的实现方式中,训练质量评价模型的流程可以包括S001~S008。S006~S008具体如下:
S006、采用所述测试集对初始的质量评价模型进行测试,得到初始的质量评价模型的第二评价准确度。
可以比较测试样本对应的标记的图像质量评价信息以及预测的图像质量评价信息,根据各测试样本对应的比较结果确定初始的质量评价模型的第二评价准确度。计算第二评价准确度的方法与S003中计算第一评价准确度的方法相同,请见S003中的相关描述,此处不做限制。
其中,测试过程中采用的测试样本与训练过程中采用的训练样本相对应。例如,当一个训练样本包括一个训练样本图像以及训练样本图像对应的标记的图像质量评价信息时;一个测试样本包括一个测试样本图像以及测试样本图像对应的标记的图像质量评价信息。
再例如,当一个训练样本包括一个原始训练图像、至少一个对原始训练图像处理后得到的训练样本图像,训练样本图像具有标记的图像质量评价信息时,一个测试样本包括一个原始测试图像、至少一个对原始测试图像处理后得到的测试样本图像,测试样本图像具有标记的图像质量评价信息。
再例如,当训练样本中的原始训练图像还可以具有标记的图像质量评价信息,在训练过程中,需要评价原始训练图像以及训练样本图像各自的图像质量时,测试样本中的原始测试图像还可以具有标记的图像质量评价信息;在测试过程中,需要评价原始测试图像以及测试样本图像各自的图像质量。
S007、判断所述第二评价准确度是否大于或等于第二准确度阈值。
第二准确度阈值用于衡量测试结果是否达标。当第二评价准确度小于第二准确度阈值时,测试不达标,执行S005,需要继续训练初始的质量评价模型。当第二评价准确率大于或等于第二准确度阈值时,测试达标,执行S008。
第二准确度阈值可以与第一准确度阈值相同,也可以不同,此处不做限制。
其中,为了提高图像质量模型的评价图像质量的准确度,在执行S007之后,可以跳转至S005,微调初始的质量评价模型的参数,之后,跳转至S002,继续执行S002~S004,以继续训练初始的质量评价模型。
S008、停止训练初始的质量评价模型,得到训练后的所述质量评价模型。
本申请实施例,在采用训练集中的训练样本训练初始的质量评价模型的过程中,当初始的质量评价模型的准确度符合要求时,采用测试集中的测试样本测试经过训练的质量评价模型的准确度,以验证质量评价模型输出的结果的准确度及可靠性,如果测试不通过,微调质量评价模型的参数,继续训练;如果测试通过,停止训练,得到训练后的质量评价模型。由于训练后的质量评价模型的准确度是测试达标的,采用该质量评价模型评价图像质量时,可以提高质量评价模型输出的结果的准确度及可靠性。
上面介绍了采用一个图像增强模型对第一图像进行处理后,评价图像质量的方案;下面介绍采用至少两个图像增强模型对第一图像进行处理得到至少两个第二图像,并评价至少两个第二图像的图像质量,以输出图像质量最好的图像的方案。
参见图12,图12是本申请第六实施例提供的一种图像处理方法的示意性流程图。图像处理方法包括以下步骤:
S501、获取待处理的第一图像。
S501与S101相同,具体参见S101中的相关描述,此处不赘述。
S502、采用至少两个图像增强模型对所述第一图像进行处理,得到至少两个第二图像。
至少两个图像增强模型对第一图像执行相同的处理。
在一种可能的实现方式中,至少两个图像增强模型均用于实现同一种图像处理功能,例如,至少两个图像增强模型均用于实现超分辨率、去噪、去马赛克或图像复原。即,至少两个图像增强模型可以都为超分辨率模型、去噪模型、去马赛克模型或图像复原模型。
在另一种可能的实现方式中,一个图像增强模型也可以实现超分辨率、图像去噪、去马赛克、图像复原中的至少两种的任意组合。训练该图像增强模型所采用的样本图像可以存在以下至少两种情况:分辨率小于预设分辨率阈值、图像噪声、马赛克、图像模糊的区域等。
预设分辨率阈值在另一种可能的实现方式中,一个图像增强模型可以由至少两个子模型串接得到,至少两个子模型的图像处理功能各不相同。即,一个图像增强模型可以由超分辨率模型、去噪模型、去马赛克模型、图像复原模型中的至少两个串接而成;这样该图像增强模型就可以执行超分辨率、图像去噪、去马赛克、图像复原中的至少两种。其中,子模型的串接顺序可以根据图像处理优先级来确定。例如,图像处理优先级可以为:去噪>超分辨率>去马赛克。一个图像增强模型可以由一个去噪模型、一个超分辨率模型、以及一个去马赛克模型顺序串接而成。
手机可以将第一图像分别输入至少两个图像增强模型,至少两个图像增强模型采用并行处理方式对第一图像进行处理,获取每个图像增强模型输出的第二图像。
在一种可能的实现方式中,用户可以打开图像处理应用,加载待处理的第一图像,用户可以从图像处理应用的用户界面(User Interface,UI)中显示的图像处理功能选项中选择目标图像处理功能选项。图像处理功能选项包括但不限于:瘦脸、瘦腿、图像复原、超分辨率、去噪、去马赛克等。手机获取用户选中的至少一个目标图像处理功能选项,并调用目标图像处理功能选项对应的至少两个图像增强模型对第一图像进行处理。“瘦脸”以及“瘦腿”可以对应于具有图像复原功能的图像增强模型。
在另一种可能的实现方式中,手机可以在获取到待处理的第一图像时,可以获取第一图像中的图像特征,根据图像特征选择至少两个图像增强模型对第一图像进行处理。例如,用户启动手机中的拍照应用进行拍照,手机采集到第一图像时,根据第一图像的图像特征选择至少两个图像增强模型对第一图像进行处理。
示例性的,手机可以在检测到第一图像的分辨率小于或等于预设分辨率阈值时,分别采用至少两个超分辨率模型对第一图像进行超分辨率处理,得到每个超分辨率模型各自输出的第二图像。预设分辨率阈值可以根据实际情况进行设置。
手机可以在检测到第一图像中存在图像噪声时,分别采用至少两个去噪模型对第一图像进行去噪处理,得到每个去噪模型各自输出的第二图像,第二图像的图像噪声比第一图像的图像噪声少。
手机可以在检测到第一图像中存在马赛克时,采用至少两个去马赛克模型分别对第一图像进行去马赛克处理,得到每个去马赛克模型各自输出的第二图像。
手机可以在检测到第一图像中存在图像模糊的区域时,例如,检测到第一图像是 翻拍的老照片、第一图像因存在雨线(雨点)、雾、镜面等导致图像模糊,采用至少两个图像复原模型分别对第一图像进行复原处理,得到每个图像复原模型各自输出的第二图像。第二图像的清晰度(或可见度)大于第一图像的清晰度(或可见度)。
可以理解的是,第一图像中也可以存在以下至少两种情况:分辨率小于预设分辨率阈值、存在图像噪声、存在马赛克、存在图像模糊的区域等。
示例性的,当一个图像增强模型可以实现超分辨率、图像去噪、去马赛克中的至少两种时,手机在检测到第一图像的分辨率小于预设分辨率阈值、第一图像中存在图像噪声以及第一图像中存在马赛克时,采用至少两个均可以实现超分辨率、图像去噪以及去马赛克的图像增强模型对第一图像进行并行处理,得到每个图像增强模型输出的第二图像。例如,图像增强模型的总数为N,N>1,可以通过第一图像增强模型对第一图像A进行图像去噪、超分辨率以及去马赛克处理,得到第二图像B1;通过第二图像增强模型对第一图像A进行图像去噪、超分辨率以及去马赛克处理,得到第二图像B2;通过第N图像增强模型对第一图像A进行图像去噪、超分辨率以及去马赛克处理,得到第二图像BN。
当一个图像增强模型可以由超分辨率模型、去噪模型、去马赛克模型中的至少两个串接得到时,手机在检测到第一图像的分辨率小于预设分辨率阈值以及第一图像中存在图像噪声时,选取至少两个由一个去噪模型以及一个超分辨率模型串接得到的图像增强模型,对第一图像进行并行处理,得到每个图像增强模型输出的第二图像。其中,可以采用每个图像增强模型中的去噪模型对第一图像进行去噪处理,以及采用每个图像增强模型中的超分辨率模型对去噪处理后的图像进行超分辨率处理,得到一个第二图像。
例如,图像增强模型的总数为N,N>1,采用第一图像增强模型中的去噪模型1对第一图像进行去噪处理得到图像1,将图像1输入第一图像增强模型中的超分辨率模型1进行超分辨率处理,得到第二图像B1。采用第N图像增强模型中的去噪模型N对第一图像进行去噪处理得到图像N,将图像N输入第N图像增强模型中的超分辨率模型N进行超分辨率处理,得到第二图像BN。
S503、将所述至少两个第二图像输入质量评价模型进行处理,得到每个所述第二图像对应的图像质量评价信息。
例如,当S502中输出的第二图像包括B1、B2、……、BN时,手机可以采用质量评价模型对图像B1进行处理,得到图像B1对应的图像质量评价信息;采用质量评价模型对图像B2进行处理,得到图像B2对应的图像质量评价信息;……;采用质量评价模型对图像BN进行处理,得到图像BN对应的图像质量评价信息。
在S503中,图像质量评价信息可以是图像质量分数,图像质量分数可以是0-100中的任一整数,也可以是0-1中的任一数值。图像质量评价信息也可以是图像质量分类信息,图像质量分类信息可以用0-1中的整数或小数来标识。可选的,采用质量评价模型评价第二图像的图像质量的方法可以参见图2对应的实施例中S103的相关描述。S504和S505是并列步骤,手机在执行S503之后,可以执行S504或S505。S504和S505的区别在于,S504中,手机可以直接显示至少两个第二图像种图像质量最好的图像;S505中,手机可以显示第一图像和至少两个第二图像中,图像质量最好的图像。
S504、根据每个第二图像对应的所述图像质量评价信息,从至少两个第二图像中选择图像质量最好的图像,输出所述图像质量最好的图像。
由于图像质量评价信息用于标识第二图像的图像质量的好坏程度,因此,手机可以通过比较所有第一图像的图像质量评价信息,从至少两个第二图像中确定图像质量最好的目标第二图像。例如,当图像质量评价信息为图像质量分数时,手机可以将每个第二图像的图像质量分数进行比较,从至少两个第二图像中筛选出图像质量分数最高的目标第二图像,并输出图像质量分数最高的第二图像。输出图像质量分数最高的第二图像的方式可以是显示图像质量分数最高的第二图像,也可以是将图像质量分数最高的第二图像保存至图库。
或者,手机也可以将每个第二图像的图像质量分数与预定分数阈值进行比较,筛选出图像质量分数大于或等于预定分数阈值的目标第二图像,并输出目标第二图像。例如,假设第二图像的数量为3个,分别为图像B1、图像B2以及图像B3,如果图像B1的图像质量分数为0.8、图像B2的图像质量分数为0.7、图像B3的图像质量分数为0.5,预定分数阈值为0.7。由于图像B1的图像质量分数和图像B2的图像质量分数大于或等于0.7,因此,图像B1以及图像B2均为目标第二图像,如图13a~图13c所示,手机可以在显示界面上同时显示图像B1以及图像B2,还可以显示提示信息,以便用户选择需要保存的图像。其中,如图13a所示,用户可以通过对话框中来选中并保存图像B1、图像B2,或者保存图像B1和图像B2。如图13b所示,用户可以通过点击图像B1来选中图像B1并进行保存,图像B1呈淡灰色。如果用户选中图像B1和图像B2,那么手机保存图像B1和图像B2,图像B1和图像B2都呈淡灰色。如图13c所示,用户可以点击图像B1左侧的方框,该方框中显示“√”,在图像B1的右侧显示“保存”,标识用户已选中图像B1,手机保存图像B1。
需要说明的是,如果目标第二图像的数量较多,可以显示所有的目标第二图像;也可以显示部分目标第二图像时,例如,可以优先显示图像质量分数较高的目标第二图像。
示例性的,当目标第二图像的数量较多(比如,大于或等于3个)时,显示所有目标第二图像的方法可以是:手机在UI界面中显示由多个目标第二图像组成的图像序列,用户可以向左滑动或向右滑动查看图像序列中的任一目标第二图像,从而能够从多个目标第二图像中选择想要保存的图像。如图13d所示,用户点击“拍照按钮”触发拍照指令进行拍照,手机响应于用户触发的拍照指令得到第一图像,通过M个图像增强模型对第一图像进行处理得到N个第二图像;将N个第二图像输入质量评价模型进行处理,或者将第一图像和N个第二图像输入质量评价模型进行处理,得到图像质量评价信息,根据获得的图像质量评价信息确定图像B1~图像Bi为第一图像和N个第二图像中图像质量最好的图像;当用户点击“拍照按钮”左边的图标时,手机显示图像B1~图像Bi。其中,手机可以在主显示区域显示图像B1,在图像B1下方显示由图像B1~图像Bi组成的图像序列。用户点击显示于图像B1右下角的“选择图标”可以选中图像B1进行保存。用户也可以在触摸屏的任意位置通过向左滑动触摸手势触发手机根据滑动距离向左移动图像序列,以便用户查看排在图像B1之后的任一图像,例如图像B3;用户还可以通过向右滑动触摸手势触发手机根据滑动距离向右移动图像 序列,以便用户根据需要查看图像,并选中需要保存的图像。
需要说明的是,手机也可以根据每个第二图像各自对应的图像质量分数,将图像质量分数最高的第二图像识别为图像质量最好的目标图像,并输出图像质量分数最高的第二图像。例如,假设第二图像的数量为3个,分别为图像B1、图像B2以及图像B3,如果图像B1的图像质量分数为0.8、图像B2的图像质量分数为0.7、图像B3的图像质量分数为0.5,那么,手机根据图像B1~图像B3的图像质量分数确定图像B1为图像质量最好的图像,输出图像B1。当图像质量分数最高的第二图像的数量为至少两个时,可以显示任一个,也可以显示至少两个,以供用户选择。例如,当图像B1的图像质量分数和图像B2的图像质量分数并列第一时,如图13所示,手机可以在显示界面上同时显示图像B1以及图像B2,还可以显示提示信息,以便用户选择需要保存的图像。
请一并参见图14,图14是本申请再一实施例提供的一种用户界面的示意图。如图14所示,图像B1消除了眼镜反光,且人像的五官以及眼镜均没有变形;图像B2消除了眼镜反光,且图像中眼镜发生变形,图像B3消除了部分眼镜反光,且人像的五官变形。手机将图像B1~图像B3各自对应的图像质量分数进行比较,得到的比较结果为:图像B1的图像质量分数>图像B2的图像质量分数>图像B3的图像质量分数,表示图像B1的图像质量>图像B2的图像质量>图像B3的图像质量,手机显示图像B1。
示例性的,手机通过图像质量分数从至少两个第二图像中,筛选出来的目标图像可以是清晰度最高且不存在artifacts的图像,也可以是清晰度较高且artifacts最少的图像。目标图像可以是图像噪声最少(或没有图像噪声)且不存在artifacts的图像,也可以是图像噪声较少且artifacts最少的图像。目标图像可以是图像噪声最少(或没有图像噪声)且不存在散粒噪声的图像,也可以是图像噪声较少且散粒噪声最少的图像。目标图像可以是不存在色彩混叠以及拉链效应的图像,也可以是色彩混叠以及拉链效应最少的图像。
S505、根据每个第二图像对应的所述图像质量评价信息,从第一图像和至少两个第二图像中确定图像质量最好的图像,并输出所述图像质量最好的的图像。
由于图像质量评价信息用于标识第二图像的图像质量的好坏程度,因此,手机可以通过比较所有第二图像的图像质量评价信息,从第一图像和至少两个第二图像中确定图像质量最好的目标图像,并输出图像质量最好的图像。图像质量最好的图像可以是第二图像,也可以是第一图像。输出图像质量最好的图像的方式可以是显示图像质量最好的图像,也可以是将图像质量最好的图像保存至图库。图像质量评价信息可以为用于表示图像质量的标识信息或分数。
例如,假设在S502中获得N个第二图像,N≥2,图像质量评价信息为对N个第二图像中每一个第二图像的图像质量评价信息。
当图像质量评价信息是用于表示第二图像的图像质量好或差的标识信息时,手机可以判断N个第二图像中是否存在图像质量评价信息为预定标识的目标第二图像,根据判断结果从第一图像和N个第二图像中确定图像质量最好的图像。其中,若N个第二图像中存在图像质量评价信息为预定标识的目标第二图像,则确定图像质量评价信息为预定标识的目标第二图像为图像质量最好的图像,并输出目标第二图像。若N个 第二图像中存在至少两个图像质量评价信息为预定标识的目标第二图像,则选择任一目标第二图像输出,或者输出所有目标第二图像。若N个第二图像中不存在图像质量评价信息为预定标识的目标第二图像,则确定第一图像为图像质量最好的图像,输出第一图像。
当图像质量评价信息是图像质量分数时,手机可以判断N个第二图像中是否存在图像质量分数大于预定分数阈值的目标第二图像,根据判断结果从第一图像和第二图像中确定图像质量最好的图像。若N个第二图像中存在图像质量分数大于或等于预定分数阈值的目标第二图像,则确定目标第二图像为图像质量最好的图像,输出目标第二图像;若N个第二图像中不存在图像质量分数大于或等于预定分数阈值的目标第二图像,则确定第一图像为图像质量最好的图像,输出第一图像。若N个第二图像中存在至少两个目标第二图像,可以选择任一目标第二图像输出,可以输出全部目标第二图像,也可以输出部分目标第二图像。当输出部分目标第二图像时,可以优先显示图像质量分数较高的目标图像,例如,输出分数最高的目标第二图像,当存在至少两个目标第二图像的图像质量分数并列第一时,可以输出任一个目标第二图像。需要说明的是,手机可以在检测到任一第二图像的图像质量分数大于或等于预定分数阈值时,输出该第二图像,结束图像处理流程,这样可以更快地输出图像质量较好的第二图像。
可以理解的是,当N个第二图像中存在至少两个目标第二图像时,手机也可以在显示界面显示全部目标第二图像,以供用户从中选择需要的选择。当目标第二图像的数量较多(比如,3个或3个以上)时,可以生成目标第二图像对应的缩略图,并在显示界面显示该缩略图,用户可以浏览该缩略图中包括的所有目标第二图像,选择需要保存的图像。
例如,第二图像的数量为3个,分别为图像B1、图像B2以及图像B3,如果图像B1的图像质量分数为0.8、图像B2的图像质量分数为0.7、图像B3的图像分数为0.5,预定分数阈值为0.7;那么手机根据所有第二图像的图像质量分数,筛选出的图像质量分数大于或等于0.7的目标第二图像包括图像B1和图像B2,并在显示界面同时显示图像B1和图像B2,以供用户选择。
再例如,第二图像的数量为5个,分别为图像B1、图像B2、图像B3、图像B4以及图像B5,如果图像B1的图像质量分数为0.6、图像B2的图像质量分数为0.5、图像B3的图像分数为0.8,图像B4的图像质量分数为0.8、图像B5的图像质量分数为0.7,预定分数阈值为0.7;那么手机筛选出的图像质量分数大于或等于0.7的目标第二图像包括图像B3、图像B4以及图像B5,生成由图像B3、图像B4以及图像B5组成的缩略图,并在显示界面显示该缩略图,以便用户浏览该缩略图中包括的图像B3、图像B4以及图像B5,从而选择需要保存的图像。其中,缩略图的大小可以根据手机显示屏的尺寸进行调整。缩略图中各图像的排列方式和排列顺序不做限制。
为了减少用户从目标第二图像中挑选图像所花费的时间,当目标第二图像的数量为至少两个时,可以将至少两个目标第二图像按图像质量分数进行排序后进行显示。
例如,假设目标第二图像包括图像B2以及图像B3,图像B2的图像质量分数小于图像B3的图像质量分数;手机可以将图像B3放在图像B2之前进行显示,也可以将图像B3显示于较大的显示区域。比如,将显示屏分为第一显示区域以及第二显示 区域,第一显示区域可以在第二显示区域的上方或左边,第一显示区域的面积可以大于第二显示区域的面积(例如,第一显示区域与第二显示区域的面积比例为2:1),图像B3显示在第一显示区域,图像B2显示在第二显示区域。如图15所示,手机显示界面中显示的图像B3的尺寸大于图像B2的尺寸。需要说明的是,用户基于如图15所示的UI界面选择需要的图像的方法类似于图13中的选择方式,此处不赘述。
再例如,假设目标第二图像包括图像B3、图像B4以及图像B5,图像B3的图像质量分数>图像B4的图像质量分数>图像B5的图像质量分数;将图像B3、图像B4以及图像B5按图像质量分数从高到低的顺序进行排序,之后,生成相应的图像列表或缩略图。图像列表或缩略图中图像的排列方式为:图像B3-图像B4-图像B5。如图16所示,图像B3、图像B4以及图像B5呈纵向排列,并且图像B3显示在最前面。需要说明的是,用户基于如图16所示的UI界面选择需要的图像的方法类似于图13中的选择方式,此处不赘述。
本申请实施例中,可以采用至少两个图像增强模型对第一图像进行处理,从而得到至少两个第二图像,可以从至少两个第二图像中选择图像质量最好的目标图像,并将图像质量最好的目标图像显示给用户查看或者,可以从第一图像和至少两个第二图像中确定图像质量最好的图像,并输出图像质量最好的图像。当图像质量最好的图像为至少两个时,可以对其进行排序后再显示给用户,用户可以快速地了解到各图像的图像质量,从而能够快速的选择需要的图像。
为了提高第二图像的图像质量评价信息的准确度,可以通过第一图像辅助评价第二图像的图像质量。参见图17,图17是本申请第七实施例提供的一种图像处理方法的示意性流程图。图17对应的实施例与图12对应的实施例的区别在于S603,输入质量评价模型的图像不同,以及质量评价模型评价第二图像的图像质量的方法不同。图12的S503中,质量评价模型的输入为第二图像,质量评价模型用于基于第二图像的图像特征信息评价第二图像的图像质量;而图17的S603中,质量评价模型的输入为第一图像以及第二图像,质量评价模型用于基于第一图像的特征信息以及第二图像的特征信息评价第二图像的图像质量。S603具体如下:
S603、将所述第一图像以及所述至少两个第二图像输入质量评价模型进行处理,每个所述第二图像对应的图像质量评价信息。
例如,当S602中输出的第二图像包括B1、B2、……、BN时,手机采用质量评价模型基于图像A以及图像B1确定图像B1对应的图像质量评价信息;采用质量评价模型基于图像A以及图像B2确定图像B2对应的图像质量评价信息;……;采用质量评价模型基于图像A以及图像BN确定图像BN对应的图像质量评价信息。
第一图像(例如:图像A)用于辅助评价每个第二图像的图像质量。借助第一图像评价第二图像的图像质量的具体实现方式参见图7对应的实施例中S203的相关描述,此处不赘述。
本实施例中,通过第一图像的特征信息辅助评价第二图像的图像质量,由于手机可以以第一图像为参考图像,来评价第二图像的图像质量,可以更准确地评价第二图像的图像质量,可以提高第二图像的图像质量评价信息的准确度。
参见图18,图18是本申请第八实施例提供的一种图像处理方法的示意性流程图。 图18与图12的区别在于S703~S704,图17的S703中增加了采用质量评价模型评价第一图像的步骤,图12中是从至少两个第二图像中选择图像质量最好的样本图像进行输出,图17中是从至少两个第二图像以及第一图像中,选择图像质量最好的样本图像进行输出。具体如下:
S703、将所述第一图像以及所述至少两个第二图像输入质量评价模型进行处理,得到所述第一图像对应的第一图像质量评价信息,以及每个所述第二图像对应的第二图像质量评价信息。
例如,当S702中输出的第二图像包括B1、B2、……、BN时,手机可以采用质量评价模型对图像A进行处理,得到图像A对应的图像质量评价信息;手机可以采用质量评价模型对图像B1进行处理,得到图像B1对应的图像质量评价信息;采用质量评价模型对图像B2进行处理,得到图像B2对应的图像质量评价信息;……;采用质量评价模型对图像BN进行处理,得到图像BN对应的图像质量评价信息。
再例如,当S702中输出的第二图像包括B1、B2、……、BN时,手机可以采用质量评价模型对图像A进行处理,得到图像A对应的图像质量评价信息;手机可以采用质量评价模型对图像A和图像B1进行处理,得到图像B1对应的图像质量评价信息;采用质量评价模型对图像A和图像B2进行处理,得到图像B2对应的图像质量评价信息;……;采用质量评价模型对图像A和图像BN进行处理,得到图像BN对应的图像质量评价信息。
其中,图像A对应的图像质量评价信息可以由图像A的第一图像质量评价信息确定;图像B1~图像BN的图像质量评价信息可以分别由图像B1~图像BN的第二特征信息确定,也可以通过比较第一特征信息以及第二特征信息得到差异特征信息,根据差异特征信息而确定。
第一特征信息可以是图像A的全部特征信息,也可以是图像A的部分特征信息。全部特征信息是图像A中每个像素点的特征信息,部分特征信息是图像A中的部分像素点的特征信息,例如第一图像中可以反映或代表第一图像的图像质量的像素点。本实施例中,获得第一图像对应的第一图像质量评价信息,以及每个第二图像对应的第二图像质量评价信息的具体实现方式,参见图8对应的实施例中S303的相关描述,此处不赘述。S704和S705是并列步骤,手机在执行S703之后,可以执行S704或S705。
S704、根据所述第一图像对应的第一图像质量评价信息,以及每个所述第二图像对应的第二图像质量评价信息,从第一图像和至少两个第二图像中确定图像质量最好的图像,输出所述图像质量最好的图像。
在S704中,可以基于第一图像质量评价信息以及所有的第二图像质量评价信息,从第一图像以及所有的第二图像中确定图像质量最好的图像。其中,图像质量最好的图像可以是第一图像和至少两个第二图像中,图像质量评价信息为预定标识的图像,或者图像质量分数大于或等于预定分数阈值的图像,还可以是图像质量分数大于或等于预定分数阈值,且图像质量分数最高的图像。预定标识可以为“1”或“YES”或“是”。图像质量最好的图像可能是第一图像,也可能是第二图像。
例如,假设在S502中获得N个第二图像,N≥2,当第一图像质量评价信息和第一图像质量评价信息均为用于表示图像质量的标识信息时,手机可以判断第一图像和N 个第二图像中是否存在图像质量评价信息为预定标识的图像,根据判断结果从第一图像和第二图像中确定图像质量最好的图像。其中,若第一图像的图像质量评价信息为预定标识,且N个第二图像中存在至少一个图像质量评价信息为预定标识的目标第二图像,则从第一图像和目标第二图像中选择任一图像输出。若第一图像的图像质量评价信息不是预定标识,且N个第二图像中存在至少一个图像质量评价信息为预定标识的目标第二图像,则从目标第二图像中选择任一图像输出。若第一图像的图像质量评价信息不是预定标识,且N个第二图像中不存在图像质量评价信息为预定标识的目标第二图像,则输出第一图像。
当第一图像质量评价信息和第二图像质量评价信息均为图像质量分数时,手机可以根据第一图像的第一图像质量分数和N个第二图像的第二图像质量分数,从第一图像和N个第二图像中,确定图像质量分数最高的图像为第一图像和N个第二图像中图像质量最好的图像,输出分数最高的图像。手机还可以判断第一图像和N个第二图像中是否存在图像质量分数大于预定分数阈值的图像,根据判断结果从第一图像和第二图像中确定图像质量最好的图像。若第一图像和N个第二图像中存在图像质量分数大于或等于预定分数阈值的图像,输出图像质量分数大于或等于预定分数阈值的图像。当图像质量分数大于或等于预定分数阈值的图像的数量为至少两个时,可以从中选择任一个进行输出,也可以显示所有图像质量分数大于或等于预定分数阈值的图像,以便用户从中选择需要保存的图像。
为了减少用户从挑选图像所花费的时间,当目标第二图像的数量为至少两个时,可以将所有图像质量分数大于或等于预定分数阈值的图像按图像质量分数进行排序后进行显示。
本实施例,可以分别评价第一图像与第二图像的图像质量,得到第一图像对应的第一图像质量评价信息,以及第二图像对应的第二图像质量评价信息,可以直接比较第一图像质量评价信息以及第二图像质量评价信息,从而从第一图像以及第二图像中选择图像质量最好的图像,并将图像质量最好的图像显示给用户。
上面描述了通过至少两个图像增强模型并行处理第一图像的方案,下面介绍采用至少两个图像增强模型串行处理第一图像的方案。
参见图19,图19是本申请第九实施例提供的一种图像处理方法的示意性流程图。图19所示的图像处理方法包括以下步骤:
S801、获取待处理的图像A。
S801与S101相同,具体参见S101中的相关描述,此处不赘述。
S802、采用图像增强模型1对图像A进行处理,得到图像增强模型1输出的图像B1。
S803、将图像B1输入质量评价模型进行处理,得到图像B1对应的图像质量评价信息。
S804、根据图像B1对应的图像质量评价信息确定图像B1是否为目标图像。
S804与S104基本相同,参见S104中的相关描述,此处不赘述。
其中,目标图像是指图像质量符合要求的图像。当图像B1的图像质量评价信息为预定标识,或者图像B1的图像质量评价信息对应的图像质量分数大于或等于预定 分数阈值时,表示图像B1是目标图像,执行S805,以输出图像B1;当图像B1的图像质量评价信息不是预定标识,或者图像B1的图像质量评价信息对应的图像质量分数小于预定分数阈值时,表示图像B1不是目标图像,此时,不输出图像B1,且不输出图像A,执行S806。预定标识可以为“1”、“YES”或“是”。
S805、当图像B1是目标图像时,输出图像B1。
S806、当图像B1不是目标图像时,采用图像增强模型2对图像A进行处理,得到图像增强模型2输出的图像B2。
S807、将图像B2输入质量评价模型进行处理,得到图像B2对应的图像质量评价信息。
S808、根据图像B2对应的图像质量评价信息确定图像B2是否为目标图像。
其中,当图像B2的图像质量评价信息为预定标识,或者图像B2的图像质量评价信息对应的图像质量分数大于或等于预定分数阈值时,表示图像B2是目标图像,执行S809,以输出输出图像B2;当图像B2的图像质量评价信息不是预定标识,或者图像B2的图像质量评价信息对应的图像质量分数小于预定分数阈值时,表示图像B2不是目标图像,不输出图像B2。如果图像增强模型的数量为2时,手机在执行S808确定图像B2不是目标图像后,输出图像A。如果图像增强模型的数量大于2,手机在执行S808确定图像B2不是目标图像后,执行S810。
S809、当图像B2是目标图像时,输出图像B2。
S810、当图像B2不是目标图像时,采用图像增强模型N对图像A进行处理,得到图像增强模型N输出的图像BN。
N为整数且N≥3。
S811、将图像BN输入质量评价模型进行处理,得到图像BN对应的图像质量评价信息。
S812、根据图像BN对应的图像质量评价信息确定图像BN是否为目标图像。
其中,当图像BN的图像质量评价信息为预定标识,或者图像BN的图像质量评价信息对应的图像质量分数大于或等于预定分数阈值时,表示图像BN是目标图像,执行S813,以输出图像BN;当图像BN的图像质量评价信息不是预定标识,或者图像BN的图像质量评价信息对应的图像质量分数小于预定分数阈值时,表示图像BN不是目标图像,不输出图像BN。
需要说明的是,如果图像增强模型N不是手机内预置的N个图像增强模型中的最后一个,手机在执行完S812确定图像BN不是目标图像之后,N递增1,返回执行步骤810中的采用图像增强模型N对图像A进行处理,得到图像增强模型N输出的图像BN。如果图像增强模型N是手机内预置的N个图像增强模型中的最后一个,那么在执行完S812确定图像BN不是目标图像之后,执行S814。
S813、当图像BN是目标图像时,输出图像BN。
S814、当图像BN不是目标图像时,输出图像A。
可以理解的是,图像增强模型可以根据用户选择的图像处理功能来确定,也可以由图像A的图像特征来确定,此处不做限制。
N个图像增强模型可以用于实现同一种图像处理功能,即N个图像增强模型可以 用于实现去噪、超分辨率、去马赛克或图像复原。N个图像增强模型中每个图像增强模型也可以用于实现至少两种图像处理功能。N个图像增强模型可以按模型对应的图像处理效果的优劣程度来排序,即从备选的图像增强模型中优先选用图像处理效果最优的模型对图像A进行处理。N个图像增强模型还可以用于实现不同的图像处理功能。
示例性的,当N个图像增强模型用于实现同一种图像处理功能时,例如,图像增强模型1可以为去噪效果最好、超分辨率效果最好或去马赛克效果最好的图像增强模型。
示例性的,当N个图像增强模型中每个图像增强模型用于实现至少两种图像处理功能时,假设如果图像A中存在以下至少两种情况:分辨率小于预设分辨率阈值、图像噪声、马赛克,那么,N个图像增强模型中的每个模型均用于对图像A执行去噪、超分辨率以及去马赛克。图像增强模型1可以为去噪效果最好、超分辨率效果最好以及去马赛克效果最好的图像增强模型。
示例性的,当N个图像增强模型用于实现不同的图像处理功能时,用于实现同一种图像处理功能的图像增强模型的数量可以为一个,也可以为至少两个。如果图像A中存在以下至少两种情况:分辨率小于预设分辨率阈值、图像噪声、马赛克时,可以按照图像处理优先级对图像A做相应的处理,图像处理优先级可以为:去噪>超分辨率>去马赛克。如果N=3,那么图像增强模型1可以为去噪模型、图像增强模型2可以为超分辨率模型、图像增强模型3可以为去马赛克模型。如果N=6,那么图像增强模型1以及图像增强模型2可以为去噪模型、图像增强模型3以及图像增强模型4可以为超分辨率模型、图像增强模型5以及图像增强模型6可以为去马赛克模型。图像增强模型1可以为去噪效果最好的图像增强模型,图像增强模型3可以为超分辨率效果最好的图像增强模型,图像增强模型5可以为去马赛克效果最好的增强模型。
由于图像增强模型的特性决定了:同种类型的不同的图像增强模型的图像处理效果可能不同,经过模型处理后的图像存在artifacts的严重程度可能不同,而输出给用户看的图像需要平衡artifacts以及图像处理效果,因此,需要采用质量评价模型评价图像增强模型对第一图像进行处理后得到的第二图像的图像质量,以根据图像质量评价结果来确定是否需要采用其他的图像增强模型对第一图像进行处理,从而尽可能输出图像质量较好的第二图像给用户查看。
由于对于训练后的图像增强模型而言,图像增强模型具有一定的稳定性,其图像增强效果是可预见的,而图像增强模型处理后得到的图像是否可能会存在artifacts这个是不确定的,也不可预见的,这样一来,通过图像增强效果较好的图像增强模型对图像A进行处理,获得图像质量较好的图像B的概率较大,因此,在本实施例中,手机优先选择图像增强效果较好的图像增强模型对图像A进行处理,这样可以获取到图像质量较好的图像B的概率较大,可以缩短获取到图像质量较好的图像B所需的时间,以提高输出图像的效率。
示例性的,假设,已知超分辨率模型1的图像增强效果比超分辨率模型2的图像增强效果好,但经过超分辨率模型处理后的图像存在的artifacts可能比经过超分辨率模型处理后的图像存在的artifacts严重。手机在检测到图像A的分辨率小于预设分辨率阈值时,可以采用超分辨率模型1对图像A进行处理得到图像B1,然后采用质量评 价模型评价图像B1的图像质量,得到图像B1的图像质量分数。手机判断图像B1的图像质量分数是否大于或等于预定分数阈值,如果图像B1的图像质量分数大于或等于预定分数阈值,那么输出图像B1给用户查看,这种情况反映出虽然图像B1的清晰度大于图像B2的清晰度,但是图像B1存在的artifacts可能比图像B2存在的artifacts更严重。如果图像B1的图像质量分数小于预定分数阈值,那么输出图像A给用户查看;这种情况反映出图像B1以及图像B种均存在的artifacts,且B1存在的artifacts可能比B2存在的artifacts更严重。
示例性的,手机在检测到图像A的分辨率低于分辨率阈值时,获取至少两个超分辨率模型,例如,超分辨率模型1、超分辨率模型2以及超分辨率模型3,假设已知超分辨率模型1处理后的图像的清晰度>超分辨率模型2处理后的图像的清晰度>超分辨率模型3处理后的图像的清晰度;手机采用超分辨率模型1对图像A进行超分辨率处理,得到图像B1。手机采用图像评价模型对图像B1进行处理,得到图像B1对应的图像质量评价信息;根据图像B1对应的图像质量分数判断是否输出图像B1。如果图像B1对应的图像质量分数大于或等于预定分数阈值,手机输出图像B1给用户查看,结束图像处理流程。用户可以看到图像B1的清晰度较好,并且图像B1中没有artifacts或者存在少量的artifacts,例如,图像B1中的景、物、人眼、鼻子或眼镜框等不存在明显的变形或错位。如果图像B1对应的图像质量分数小于预定分数阈值,说明图像B1较模糊,或者图像B1中存在较严重的artifacts;手机采用超分辨率模型2对图像A进行超分辨率处理,得到图像B2。手机采用图像评价模型对图像B2进行处理,得到图像B2对应的图像质量分数;根据图像B2对应的图像质量分数判断是否输出图像B2。如果图像B2对应的图像质量分数大于或等于预定分数阈值,手机输出图像B2给用户查看,结束图像处理流程。用户可以看到的图像B2的清晰度较好,并且图像B2中没有artifacts或者存在少量的artifacts。
如果图像B2对应的图像质量分数小于预定分数阈值,说明图像B2较模糊,或者图像B2中存在较严重的artifacts。手机采用超分辨率模型3对图像A进行超分辨率处理,得到图像B3。手机采用图像评价模型对图像B3进行处理,得到图像B3对应的图像质量分数;可以将图像B3对应的图像质量分数与预定分数阈值进行比较,根据比较结果判断是否输出图像B3。如果比较结果为图像B3对应的图像质量分数大于或等于预定分数阈值,手机输出图像B3给用户查看,结束图像处理流程。如果比较结果为图像B3对应的图像质量分数小于预定分数阈值,说明图像B3较模糊,或者图像B3中存在较严重的artifacts,输出图像A给用户查看。
在本实施例中,手机可以优先采用图像处理效果最优的图像增强模型1对图像A进行处理,如果处理后的图像的图像质量较差,那么采用图像处理效果次优的图像增强模型2对图像A进行处理,如果处理后的图像的图像质量还是较差,再从可选的图像增强模型中选择最优的图像增强模型3对图像A进行处理,可选的图像增强模型是指除前面已使用的图像增强模型(如图像处理效果最优以及次优的图像增强模型)之外的图像增强模型。由于优先采用图像处理效果最优的图像增强模型对图像A进行处理,有些情况不需要使用到N个图像增强模型,就可以获得图像质量较好的图像B。 相对于采用N个图像增强模型对图像A并行处理的情况,可以节省部分系统资源,可以缩短获取到图像质量较好的图像B所需的时间,以提高输出图像的效率。
基于图20,在另一种实施例中,可以通过图像A辅助评价第二图像(例如:B1~BN)的图像质量,具体地,如图20所示,在S803中,“将图像B1输入质量评价模型进行处理,得到图像B1对应的图像质量评价信息”替换为“将图像A以及图像B1输入质量评价模型进行处理,得到图像B1对应的图像质量评价信息”。在S807中,“将图像B2输入质量评价模型进行处理,得到图像B2对应的图像质量评价信息”替换为“将图像A以及图像B2输入质量评价模型进行处理,得到图像B2对应的图像质量评价信息”。在S811中,“将图像BN输入质量评价模型进行处理,得到图像BN对应的图像质量评价信息”替换为“将图像A以及图像BN输入质量评价模型进行处理,得到图像BN对应的图像质量评价信息”。
具体实现方式参见图7对应的实施例中S203的相关描述,此处不赘述。
基于图19,区别于图19中,将由图像A处理的到的第二图像输入质量评价模型,采用质量评价模型评价第二图像的图像质量的方案,在另一种实施例,可以采用质量评价模型确定针对图像A和第二图像(例如:B1~BN)的图像质量评价信息,根据针对图像A和第二图像的图像质量评价信息确定目标图像,并输出目标图像。
参见图21,图21是本申请第十一实施例提供的一种图像处理方法的示意性流程图。与图19的区别在于S903~S904、S907~S908以及S911~S912、S914。具体如下:
S903、将图像A以及图像B1输入质量评价模型进行处理,得到针对图像A和图像B1的图像质量评价信息。
具体实现方式参见图8对应的实施例中S303的相关描述,此处不赘述。
S904、根据所述针对图像A和图像B1的图像质量评价信息,确定图像B1是否为目标图像。
示例性的,当图像质量评价信息为用于表示图像B1的图像质量是否比图像A的图像质量好的标识信息时,如果图像B1的图像质量评价信息为预定标识,表示图像B1的图像质量比图像A的图像质量好,图像B1为目标图像,那么执行S905,以输出图像B1;如果图像B1的图像质量评价信息不是预定标识,表示图像B1的图像质量比图像A的图像质量差,图像B1不是目标图像,那么不输出图像B1,执行S906。预定标识可以为“1”、“YES”或“是”。
示例性的,当图像质量评价信息包括图像A的图像质量分数A和图像B1的图像质量分数B时,如果图像质量分数B>图像质量分数A,表示图像B1的图像质量比图像A的图像质量好,图像B1为目标图像,那么执行S905,以输出图像B1;如果图像质量分数B≤图像质量分数A,表示图像B1的图像质量比图像A的图像质量差,或者,图像B1和图像A的图像质量相同,图像B1不是目标图像,那么确定不输出图像B1,执行S906。S907、将图像A以及图像B2输入质量评价模型进行处理,得到针对图像A和图像B2的图像质量评价信息。
S908、根据针对图像A和图像B2的图像质量评价信息,确定图像B2是否为目标图像。
S908的实现方式参见S904中的描述,此处不赘述。
需要说明的是,如果图像增强模型的数量为2时,手机在执行S908确定图像B2不是目标图像后,输出图像A。如果图像增强模型的数量大于2,手机在执行S908确定图像B2不是目标图像后,执行S910。
S911、将图像A以及图像BN输入质量评价模型进行处理,得到针对图像A和图像BN的图像质量评价信息。
S912、根据针对图像A和图像BN的图像质量评价信息,确定图像BN是否为目标图像。
S912的实现方式参见S904中的描述,此处不赘述。
S914、当图像BN不是目标图像时,输出图像A。
本实施例,可以分别评价第一图像与第二图像的图像质量,可以针对第一图像和第二图像的质量评价信息,来判断第二图像的图像质量是否比第一图像的图像质量好,能够更准确地获知第一图像经过图像增强模型处理后,图像质量是否变差。
需要说明的是,图12、图17-图21中,先采用图像增强模型对第一图像进行图像处理,得到第二图像,将第二图像,或者将第一图像和第二图像同时输入训练后的质量评价模型进行处理,获得质量评价模型输出的图像质量评价信息,根据图像质量评价信息确定输出第一图像或第二图像。在一种可能的实现方式中,也可以参照图9的处理逻辑,将图12、图17-图21对应的实施例变形为如下处理逻辑:先采用质量评价模型评价第一图像的图像质量,如果第一图像的图像质量评价信息表示第一图像的图像质量好,采用图像增强模型对第一图像进行图像处理,得到第二图像,并将第二图像输出给用户;如果第一图像的图像质量评价信息表示第一图像的图像质量差,输出第一图像给用户。
参见图22,图22是本申请第十二实施例提供的一种图像处理方法的示意性流程图。
S1001、获取多帧RAW图像。
多帧RAW图像的内容可以不完全相同。RAW图像是RAW格式的图像。RAW格式是未经处理、也未经压缩的格式,可以把RAW概念化为“原始图像编码数据”或更形象的称为“数字底片”。RAW图像就是图像感应器将捕捉到的光源信号转化为数字信号的原始数据。图像传感器可以包括互补金属氧化物半导体(Complementary Metal-Oxide-Semiconductor,CMOS)图像传感器,电荷耦合器件(Charge Coupled Device,CCD)图像传感器。上述RAW图像是本领域现有的概念,也可以理解为未经处理、也未经压缩的图像,在此不再进行赘述。
例如,用户开启手机中的拍照应用,手机响应于用户触发的拍照指令,获取多帧待RAW图像。图像的曝光程度跟拍照环境的明亮程度有关,例如,环境光线的强弱程度、拍摄环境亮度等。根据曝光程度,可以将图像分为长曝光图像、中曝光图像以及短曝光图像。长曝光图像对应的曝光程度>中曝光图像对应的曝光程度>短曝光图像对应的曝光程度。多帧第一图像对应的曝光程度可以相同,也可以不同。多帧RAW图像可以均为长曝光图像、中曝光图像或短曝光图像,多帧图像中也可以包括长曝光图像、中曝光图像以及短曝光图像中的至少两种。
示例性的,在光线充足的环境中(如白天)拍照时,在响应于拍照指令时,获取 到的多帧RAW图像可以均为短曝光图像,也可以包括少量的中曝光图像或长曝光图像。即获取到的短曝光图像的数量大于长曝光图像的数量。例如,获取到6帧RAW图像,其中包括1帧长曝光图像、3帧短曝光图像以及2帧中曝光图像,或者,包括5帧短曝光图像以及1帧长曝光图像。
在光线较暗的环境中(比如,晚上)拍照时,获取到的多帧RAW图像可以均为长曝光图像,也可以包括少量的中曝光图像或短曝光图像。获取到的长曝光图像的数量大于短曝光图像的数量。例如,获取到6帧RAW图像,其中包括5帧长曝光图像以及1帧短曝光图像,或者,6帧RAW图像均为长曝光图像。
在高亮环境中(比如,聚光灯下)拍照或者逆光拍照时,可以获取到的多帧RAW图像可以均为短曝光图像,也可以包括少量的中曝光图像或长曝光图像。即获取到的短曝光图像的数量大于长曝光图像的数量。例如,获取到的6帧RAW图像,包括1帧长曝光图像、2帧中曝光图像,3帧短曝光图像。
S1002、将所述多帧RAW图像进行分组,得到至少两组RAW图像。
每组RAW图像所包含的图像数量可以相同,也可以不同。一组RAW图像可以包括至少两帧RAW图像。
例如,在S1001中获取到6帧RAW图像,可以分成2组,也可以分成3组。
比如,RAW图像1、RAW图像3以及RAW图像5为一组,RAW图像2、RAW图像4以及RAW图像6为一组。或者,RAW图像1以及RAW图像2为一组,RAW图像3、RAW图像4、RAW图像5以及RAW图像6为一组。或者,RAW图像1以及RAW图像3为一组、RAW图像2以及RAW图像4为一组,RAW图像5以及RAW图像6为一组。
S1003、将每组所述RAW图像进行图像融合处理,得到每组所述RAW图像对应的融合图像。
手机将每组RAW图像中包含的所有图像融合成一张图像。具体的,可以采用图像配准技术对每组RAW图像进行图像融合处理,得到一张融合图像,可以将关于同一目标的图像数据的相关信息融合到一起,以扩大图像所含有的时间空间信息,减少不确定性,增加可靠性。图像配准方法包括但不限于光流配准法。
图像配准(Image registration)就是将不同时间、不同传感器(成像设备)或不同条件下(天候、照度、摄像位置和角度等)获取的两幅或多幅图像进行匹配、叠加的过程。具体流程如下:首先对两幅图像进行特征提取得到特征点;通过进行相似性度量找到匹配的特征点对;然后通过匹配的特征点对得到图像空间坐标变换参数;最后由坐标变换参数进行图像配准。
S1004、对每个所述融合图像进行处理,得到每个所述融合图像对应的增强图像。
可以对融合图像进行图像增强处理,以增强融合图像的图像质量,得到融合图像对应的增强图像。具体可以对融合图像进行超分辨率、去噪、去马赛克以及图像复原处理中的任一种或者至少两种的任意组合。
其中,当检测到以下场景为:透过玻璃拍照、拍摄已过塑的照片、融合图像中存在雨滴、雾时,对融合图像进行图像复原处理。
在S1004中,可以采用图像增强模型对融合图像进行处理。对融合图像进行处理 的具体实现方法可以参见图2对应的实施例中的S102中对第一图像进行处理的相关描述,此处不赘述。
S1005、将每个所述增强图像输入质量评价模型进行处理,得到每个所述增强图像对应的图像质量评价信息。
S1005的具体实现方法参见图2对应的实施例中S103的相关描述,此处不赘述。
S1006、根据每个所述增强图像对应的图像质量评价信息,从所有的所述增强图像中确定图像质量最好的目标图像。
具体实现方式参见图12对应的实施例中的S504的相关描述,此处不赘述。其中,目标图像是可以给用户查看的图像,目标图像的图像格式可以是RGB图像。
S1007、输出图像质量最好的目标图像。
示例性的,请一并参见图23,图23是本申请实施例提供的一种处理多帧RAW图像的方法的示意图。手机在S1001中获取到N帧RAW图像,并将其分成2组。可以将奇数帧归为一组,偶数帧归为一组。
手机对第一组RAW图像进行图像融合处理,得到一张融合图像1;并对融合图像1进行处理,得到增强图像1。手机对第二组RAW图像进行图像融合处理,得到一张融合图像2;并对融合图像2进行处理,得到增强图像2。
手机将增强图像1以及增强图像2输入质量评价模型进行处理,得到增强图像1的图像质量评价信息以及增强图像2的图像质量评价信息。根据增强图像1的图像质量评价信息以及增强图像2的图像质量评价信息,从增强图像1和增强图像2中确定图像质量最好的目标图像。其中,图像质量评价信息为图像质量分数,手机将增强图像1的图像质量分数与增强图像2的图像质量分数进行比较,当增强图像1的图像质量分数大于增强图像2的图像质量分数时,目标图像为增强图像1,输出增强图像1;当增强图像1的图像质量分数小于增强图像2的图像质量分数时,目标图像为增强图像2,输出增强图像2;当增强图像1的图像质量分数等于增强图像2的图像质量分数时,增强图像1和增强图像2的图像质量相同,增强图像1和增强图像2均为目标图像,从增强图像1和增强图像2中选择任一图像输出。
可以理解的是,如图24所示,在另一实施例中,也可以将每组RAW图像对应的融合图像以及增强图像输入质量评价模型进行处理,得到每个增强图像对应的图像质量评价信息,手机将增强图像1的图像质量分数与增强图像2的图像质量分数进行比较,当增强图像1的图像质量分数大于增强图像2的图像质量分数时,目标图像为增强图像1,输出增强图像1;当增强图像1的图像质量分数小于增强图像2的图像质量分数时,目标图像为增强图像2,输出增强图像2;当增强图像1的图像质量分数等于增强图像2的图像质量分数时,增强图像1和增强图像2的图像质量相同,增强图像1和增强图像2均为目标图像,从增强图像1和增强图像2中选择任一图像输出。
本实施例可以将RAW图像对应的融合图像作为参考图像,通过每组RAW图像对应的融合图像辅助评价该组RAW图像对应的增强图像的图像质量,可以提高增强图像的图像质量评价信息的准确度。
如图25所示,在一种实施例中,还可以将每组RAW图像对应的融合图像以及增强图像输入质量评价模型进行处理,得到每个融合图像对应的图像质量评价信息以及 每个增强图像对应的图像质量评价信息,并根据所有的图像质量评价信息,从融合图像以及增强图像中确定图像质量最好的目标图像,并输出目标图像。目标图像可以是任一融合图像,也可以是任一增强图像。具体实现方法参见前文从第一图像以及第二图像中确定图像质量最好的目标图像的相关描述,此处不赘述。
本实施例中,可以将多帧RAW图像进行分组,并对分组后的RAW图像进行图像融合处理得到融合图像,并增强融合图像的图像质量得到增强图像,之后,评价增强图像的图像质量,或者评价融合图像的图像质量以及增强图像的图像质量,根据评价结果输出图像质量最好的目标图像,可以拍出图像质量较好的照片。目标图像可以是增强图像,或者融合图像。由于手机始终输出图像质量较好的图像,因此,手机中存储图像质量较差的图像的可能性较低,用户查看到图像质量较差的图像的可能性变小,可以提高用户视觉体验。
应理解,上述实施例中各步骤的序号的大小并不意味着执行顺序的先后,各过程的执行顺序应以其功能和内在逻辑确定,而不应对本申请实施例的实施过程构成任何限定。
对应于上文实施例应用于电子设备的图像处理方法,图26示出了本申请一实施例提供的图像处理装置的结构示意框图,为了便于说明,仅示出了与本申请实施例相关的部分。本实施例中,图像处理装置包括的各单元用于执行图2、图7、图8、图12、以及图17~图25对应的实施例中的各步骤,具体参见上文的相关描述,此处不赘述。图像处理装置1可以包括:
获取单元110,用于获取第一图像和N个第二图像,所述N个第二图像中的每一个第二图像是通过M个图像增强模型中的至少一个图像增强模型对所述第一图像进行处理后获得的;其中,N和M为大于零的整数,所述M个图像增强模型中的每个图像增强模型均不同,所述N个第二图像也存在不同;
图像质量评价单元120,用于将输入图像输入质量评价模型进行处理,获得图像质量评价信息,所述输入图像包括所述N个第二图像,或者包括所述第一图像以及所述N个第二图像;
图像输出单元130,用于根据所获得的所述图像质量评价信息输出目标图像,所述目标图像是所述第一图像和所述N个第二图像中的至少一个图像。
其中,一个图像增强模型对第一图像进程处理后,可以得到一个第二图像,也可以得到至少两个第二图像,此处不做限制。也就是说,M可以与N相等,M也可以大于N。
在一种可能的实现方式中,所述质量评价模型是基于多个训练样本训练得到,每个训练样本包括样本图像以及用户对所述样本图像的图像质量评价信息。
在一种可能的实现方式中,所述目标图像包括根据所获得的所述图像质量评价信息和评价规则确定的目标图像,所述评价规则是所述图像质量评价信息为预定数字,或,图像质量评价信息对应的分数大于或等于预定分值阈值。
在一种可能的实现方式中,图像质量评价信息可以为用于表示图像质量的数字或 分数。
需要说明的是,图像质量评价信息还可以通过字母或文字来表示,此处不做限制。例如,图像质量评价信息可以是“0”或“1”、“YES”或“NO”,“是”或“否”。
在一种可能的实现方式中,当所述图像质量评价信息为数字,且所述输入图像为N个第二图像,或者所述第一图像以及所述N个第二图像时,所述图像质量评价信息为对所述N个第二图像中每一个第二图像的图像质量评价信息;
相应的,所述图像输出单元130具体用于:
判断所述N个第二图像中是否存在图像质量评价信息为预定数字的目标第二图像;
若所述N个第二图像中存在图像质量评价信息为预定数字的目标第二图像,则确定所述图像质量评价信息为预定数字的目标第二图像为图像质量最好的图像,并输出所述图像质量评价信息为预定数字的目标第二图像。
其中,预定数字可以为“1”。
在一种可能的实现方式中,所述图像输出单元,在判断所述N个第二图像中是否存在图像质量评价信息为预定数字的目标第二图像之后,还用于:若所述N个第二图像中不存在图像质量评价信息为预定数字的目标第二图像,则确定所述第一图像为图像质量最好的图像,输出所述第一图像。
在一种可能的实现方式中,所述图像输出单元130还用于:
若所述N个第二图像中存在多个图像质量评价信息为预定数字的目标第二图像,则选择任一目标第二图像输出。
在一种可能的实现方式中,若所述图像质量评价信息为分数,且所述输入图像为N个第二图像,或者所述第一图像以及所述N个第二图像时,所述图像质量评价信息为对所述N个第二图像中每一个第二图像的图像质量评价信息;
相应的,所述图像输出单元130具体用于:
判断所述N个第二图像中是否存在分数大于预定分数阈值的目标第二图像;
若所述N个第二图像中存在分数大于预定分数阈值的目标第二图像,则确定所述分数最高的目标第二图像为图像质量最好的图像,并输出所述分数最高的目标第二图像。
在一种可能的实现方式中,所述图像输出单元130还用于,在判断所述N个第二图像中是否存在分数大于预定分数阈值的目标第二图像之后,若所述N个第二图像中不存在分数大于预定分数阈值的目标第二图像,则确定所述第一图像为图像质量最好的图像,输出所述第一图像。
在一种可能的实现方式中,所述所述图像输出单元130还用于:
若所述N个第二图像中不存在图像质量评价信息为预定数字的目标第二图像,或者所述N个第二图像中不存在分数大于预定分数阈值的目标第二图像,则获取新的图像增强模型,采用获取的所述新的图像增强模型对所述第一图像进行处理,获得新的第二图像,并将所述新的第二图像作为输入图像输入所述质量评价模型进行处理,获得新的图像质量评价信息;其中,所述新的图像增强模型为未对所述第一图像进行处理过的图像增强模型;
若确定所述新的图像质量评价信息为所述预定数字,或者所述分数大于所述预定 分数阈值,则输出新的第二图像,否则返回执行获取新的图像增强模型步骤以及后续步骤,直到返回执行的次数达到预设的次数阈值,输出所述第一图像。
在一种可能的实现方式中,当所述图像质量评价信息为数字,且所述输入图像为第一图像和N个第二图像时,所述图像质量评价信息为对所述第一图像和N个第二图像中每一个第二图像的图像质量评价信息;
相应的,所述图像输出单元130具体用于:
判断所述第一图像和N个第二图像中是否存在图像质量评价信息为预定数字的图像;
若所述第一图像的图像质量评价信息为预定数字,且所述N个第二图像中存在至少一个图像质量评价信息为预定数字的目标第二图像,则从所述第一图像和所述目标第二图像中选择任一图像输出。
在一种可能的实现方式中,所述图像输出单元在判断所述第一图像和N个第二图像中是否存在图像质量评价信息为预定数字的目标第二图像之后,还用于:
若所述第一图像的图像质量评价信息不是预定数字,且所述N个第二图像中存在至少一个图像质量评价信息为预定数字的目标第二图像,则从所述目标第二图像中选择任一图像输出。
在一种可能的实现方式中,所述图像输出单元130还用于:
若所述第一图像的图像质量评价信息不是预定数字,且所述N个第二图像中不存在图像质量评价信息为所述预定数字的目标第二图像,则输出所述第一图像。
在一种可能的实现方式中,当所述图像质量评价信息为分数,且所述输入图像为第一图像和N个第二图像时,所述图像质量评价信息为对所述第一图像和N个第二图像中每一个第二图像的图像质量评价信息;
相应的,所述图像输出单元130还用于:
根据第一图像的分数和所述N个第二图像的分数,从所述第一图像和所述N个第二图像中,确定分数最高的图像为所述图像质量最好的图像,并输出所述分数最高的图像。
在第三方面的一种可能的实现方式中,获取单元包括:
RAW图像获取单元,用于获取多帧RAW图像;
图像融合单元,用于对所述多帧RAW图像进行图像融合处理,得到第一图像。
本实施例中,可以将RAW图像对应的融合图像作为参考图像,通过每组RAW图像对应的融合图像辅助评价该组RAW图像对应的增强图像的图像质量,可以提高增强图像的图像质量评价信息的准确度。
在一种可能的实现方式中,所述图像融合单元具体用于:
将所述多帧RAW图像划分为至少两组,对每组RAW图像进行图像融合处理得到至少两个第一图像。
在一种可能的实现方式中,当N=M=1,所述图像质量评价信息为所述第二图像的图像质量评价信息时,若所述图像质量评价信息为预定数字,或者所述图像质量评价信息对应的分数大于或等于预定分数阈值,则所述目标图像为所述第二图像;若所述图像质量评价信息不是预定数字,或者所述图像质量评价信息对应的分数小于预定分 数阈值,则所述目标图像为所述第一图像;
当N=M=1,所述图像质量评价信息为针对所述第一图像和所述第二图像的图像质量评价信息,所述图像质量评价信息用于表示所述第二图像的图像质量是否比所述第一图像的图像质量好时,若所述图像质量评价信息为预定数字,则所述目标图像为所述第二图像;若所述图像质量评价信息不是预定数字,则所述目标图像为所述第一图像;
当N=M=1,所述图像质量评价信息包括所述第一图像和所述第二图像各自对应的图像质量评价信息时,若所述第一图像的图像质量评价信息为预定数字,且所述第二图像的图像质量评价信息为预定数字,则所述目标图像为所述第二图像和所述第一图像中的任一图像;若所述第一图像的图像质量评价信息不是预定数字,且所述第二图像的图像质量评价信息为预定数字,则所述目标图像为所述第二图像;若所述第一图像的图像质量评价信息为预定数字,且所述第二图像的图像质量评价信息不是预定数字,则所述目标图像为所述第一图像;或者,
若所述第二图像的图像质量评价信息对应的分数大于所述第一图像的图像质量评价信息对应的分数,则所述目标图像为所述第二图像;若所述第二图像的图像质量评价信息对应的分数小于所述第一图像的图像质量评价信息对应的分数,则所述目标图像为所述第一图像;若所述第二图像的图像质量评价信息对应的分数等于所述第一图像的图像质量评价信息对应的分数,则所述目标图像为所述第二图像和所述第一图像中的任一图像;
当N=M≥2,所述图像质量评价信息为所述N个第二图像中每一个第二图像的第二图像的图像质量评价信息时,若所述N个第二图像中存在图像质量评价信息为预定数字的目标第二图像,则所述目标图像为至少一个所述目标第二图像;若所述N个第二图像中不存在图像质量评价信息为预定数字的目标第二图像,则所述目标图像为所述第一图像;或者,
若所述N个第二图像中存在图像质量评价信息对应的分数大于或等于预定分值阈值的目标第二图像,则所述目标图像为至少一个所述目标第二图像;若所述N个第二图像中不存在图像质量评价信息对应的分数大于或等于预定分值阈值的目标第二图像,则所述目标图像为所述第一图像;
当N=M≥2,所述图像质量评价信息为针对第一图像和所述N个第二图像中每一个第二图像的第二图像的图像质量评价信息时,所述目标图像为所述第一图像和所述N个第二图像中图像质量评价信息为预定数字的任一图像,或者,所述目标图像为所述第一图像和所述N个第二图像中图像质量评价信息对应的分数大于或等于预定分值阈值的任一图像,或者,所述目标图像为所述第一图像和所述N个第二图像中图像质量评价信息对应的分数最高的图像。
在本实施例中,图像处理装置可以是电子设备,例如手机,或者是电子设备中的芯片,或者是集成在电子设备中的功能模块。其中,该芯片或者该功能模块可以位于用户终端的控制中心(例如,控制台),控制用户终端实现本申请提供的图像处理方法。
需要说明的是,上述装置/单元之间的信息交互、执行过程等内容,由于与本申请 的图像处理方法实施例基于同一构思,其具体功能及带来的技术效果,具体可参见前文的图像处理方法的实施例部分,此处不再赘述。
对应于上文实施例应用于电子设备的图像处理方法,图27示出了本申请另一实施例提供的图像处理装置的结构示意框图,为了便于说明,仅示出了与本申请实施例相关的部分。本实施例中,图像处理装置包括的各单元用于执行图9以及图10对应的实施例中的各步骤,具体参见上文的相关描述,此处不赘述。图像处理装置2可以包括:
获取单元210,用于获取第一图像;
第一评价单元220,用于将所述第一图像输入质量评价模型进行处理,得到第一图像质量评价信息;
图像处理单元230,用于当根据所述第一图像质量评价信息确定所述第一图像的图像质量符合要求时,将所述第一图像输入M个图像增强模型进行处理,得到N个第二图像,并显示或保存所述N个第二图像中的至少一个或多个;其中,N和M为正整数,所述M个图像增强模型中的每个图像增强模型均不同,所述N个第二图像也存在不同。
其中,电子设备在人像模式下、景色模式、室内模式、长焦模式(下面称为高倍变焦模式)等等下获取第一图像。第一图像可以是预览图像,也可以是拍摄的照片。
在一种可能的实现方式中,当根据所述第一图像质量评价信息确定所述第一图像的图像质量不符合要求时,显示或保存所述第一图像。
在一种可能的实现方式中,所述第一图像质量评价信息为数字或分数。
在一种可能的实现方式中,当所述第一图像质量评价信息为数字时,所述第一图像的图像质量符合要求是指所述第一图像质量评价信息为预定数字。
在一种可能的实现方式中,当所述第一图像质量评价信息为分数时,所述第一图像的图像质量符合要求是指所述第一图像的分数大于或等于预设阈值。
在一种可能的实现方式中,所述第一图像由电子设备在高倍变焦拍照模式下获取得到。
在一种可能的实现方式中,所述预设阈值为0.25,M和N均为1。
在一种可能的实现方式中,图像处理装置还包括:
第二评价单元,用于在图像处理单元230得到所述N个第二图像之后,将所述N个第二图像,或者所述第一图像以及所述N个第二图像作为输入图像输入所述质量评价模型进行处理待处理图像输入质量评价模型进行处理,得到第二图像质量评价信息;
输出单元,用于根据所述第二图像质量评价信息显示或保存目标图像,所述目标图像是所述第一图像和所述N个第二图像中的至少一个图像。
在一种可能的实现方式中,所述第二图像质量评价信息为数字或分数。
在一种可能的实现方式中,所述目标图像是所述第一图像和所述N个第二图像中,所述第二图像质量评价信息为预定数字的图像,或者所述第二图像质量评价信息对应的分数大于预定分数阈值的图像。
在本实施例中,图像处理装置可以是电子设备,例如手机,或者是电子设备中的芯片,或者是集成在电子设备中的功能模块。其中,该芯片或者该功能模块可以位于用户终端的控制中心(例如,控制台),控制用户终端实现本申请提供的图像处理方法。
需要说明的是,上述装置/单元之间的信息交互、执行过程等内容,由于与本申请的图像处理方法实施例基于同一构思,其具体功能及带来的技术效果,具体可参见前文的图像处理方法的实施例部分,此处不再赘述。
参见图28,图28为本申请一实施例提供的电子设备的结构示意图。如图28所示,该电子设备的28包括:至少一个处理器310(图28中仅示出一个)、存储器320存储在所述存储器320中并可在所述至少一个处理器310上运行的计算机程序321、以及显示设备330,显示设备330可以是触摸显示屏,所述处理器310执行所述计算机程序321时实现上述任意图像处理方法实施例中的各步骤。
电子设备3可包括,但不仅限于,处理器310、存储器320以及显示设备330。本领域技术人员可以理解,图28仅仅是电子设备3的举例,并不构成对电子设备3的限定,可以包括比图示更多或更少的部件,或者组合某些部件,或者不同的部件,例如还可以包括输入输出设备、网络接入设备、摄像头等。
所称处理器310可以是中央处理单元(Central Processing Unit,CPU),该处理器310还可以是其他通用处理器、数字信号处理器(Digital Signal Processor,DSP)、专用集成电路(Application Specific Integrated Circuit,ASIC)、现成可编程门阵列(Field-Programmable Gate Array,FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
存储器320在一些实施例中可以是电子设备3的内部存储单元,例如电子设备3的硬盘或内存。存储器320在另一些实施例中也可以是电子设备3的外部存储设备,例如电子设备3上的智能存储卡(Smart Media Card,SMC),安全数字(Secure Digital,SD)卡,闪存卡(Flash Card)等。进一步地,存储器320还可以既包括电子设备3的内部存储单元也包括外部存储设备。存储器320用于存储操作系统、应用程序、引导装载程序(Boot Loader)、数据以及其他程序等,例如所述计算机程序的程序代码等。所述存储器320还可以用于暂时地存储已经输出或者将要输出的数据。
所属领域的技术人员可以清楚地了解到,为了描述的方便和简洁,仅以上述各功能单元、模块的划分进行举例说明,实际应用中,可以根据需要而将上述功能分配由不同的功能单元、模块完成,即将所述装置的内部结构划分成不同的功能单元或模块,以完成以上描述的全部或者部分功能。实施例中的各功能单元、模块可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中,上述集成的单元既可以采用硬件的形式实现,也可以采用软件功能单元的形式实现。另外,各功能单元、模块的具体名称也只是为了便于相互区分,并不用于 限制本申请的保护范围。上述系统中单元、模块的具体工作过程,可以参考前述方法实施例中的对应过程,在此不再赘述。
在上述实施例中,对各个实施例的描述都各有侧重,某个实施例中没有详述或记载的部分,可以参见其它实施例的相关描述。
本领域普通技术人员可以意识到,结合本文中所公开的实施例描述的各示例的单元及算法步骤,能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行,取决于技术方案的特定应用和设计约束条件。专业技术人员可对每个特定的应用来使用不同方法来实现所描述的功能,但是这种实现不应认为超出本申请的范围。
在本申请所提供的实施例中,应该理解到,所揭露的装置和方法,可以通过其它的方式实现。例如,以上所描述的系统实施例仅仅是示意性的,例如,所述模块或单元的划分,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,例如多个单元或组件可以结合或者可以集成到另一个系统,或一些特征可以忽略,或不执行。另一点,所显示或讨论的相互之间的耦合或直接耦合或通讯连接可以是通过一些接口,装置或单元的间接耦合或通讯连接,可以是电性,机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元显示的部件可以是或者也可以不是物理单元,即可以位于一个地方,或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外,在本申请各个实施例中的各功能单元可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现,也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时,可以存储在一个计算机可读取存储介质中。基于这样的理解,本申请实现上述实施例方法中的全部或部分流程,可以通过计算机程序来指令相关的硬件来完成,所述的计算机程序可存储于一计算机可读存储介质中,该计算机程序在被处理器执行时,可实现上述各个图像处理方法实施例的步骤。其中,所述计算机程序包括计算机程序代码,所述计算机程序代码可以为源代码形式、对象代码形式、可执行文件或某些中间形式等。所述计算机可读介质至少可包括:能够将计算机程序代码携带到电子设备3的任何实体或装置、记录介质、计算机存储器、只读存储器(Read-Only Memory,ROM)、随机存取存储器(Random Access Memory,RAM)、电载波信号、电信信号以及软件分发介质。例如U盘、移动硬盘、磁碟或者光盘等。在某些司法管辖区,根据立法和专利实践,计算机可读介质不可以是电载波信号和电信信号。
以上所述实施例仅用以说明本申请的技术方案,而非对其限制;尽管参照前述实施例对本申请进行了详细的说明,本领域的普通技术人员应当理解:其依然可以对前述各实施例所记载的技术方案进行修改,或者对其中部分技术特征进行等同替换;而这些修改或者替换,并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围,均应包含在本申请的保护范围之内。

Claims (23)

  1. 一种图像处理方法,其特征在于,包括:
    获取第一图像和N个第二图像,所述N个第二图像中的每一个第二图像是通过M个图像增强模型中的至少一个图像增强模型对所述第一图像进行处理后获得的;其中,N和M为大于零的整数,所述M个图像增强模型中的每个图像增强模型均不同,所述N个第二图像也存在不同;
    将输入图像输入质量评价模型进行处理,获得图像质量评价信息,所述输入图像包括所述N个第二图像,或者包括所述第一图像以及所述N个第二图像;
    根据所获得的所述图像质量评价信息输出目标图像,所述目标图像是所述第一图像和所述N个第二图像中的至少一个图像。
  2. 根据权利要求1所述的图像处理方法,其特征在于,其中,所述质量评价模型是基于多个训练样本训练得到,每个训练样本包括样本图像以及用户对所述样本图像的图像质量评价信息。
  3. 根据权利要求1或2所述的图像处理方法,其特征在于,所述目标图像包括:
    根据所获得的所述图像质量评价信息和评价规则确定的目标图像,所述评价规则是图像质量评价信息为预定数字,或,图像质量评价信息对应的分数大于或等于预定分值阈值。
  4. 根据权利要求1-3任一项所述的图像处理方法,其特征在于,所述图像质量评价信息为数字或分数。
  5. 根据权利要求4所述的图像处理方法,其特征在于,当所述图像质量评价信息为数字,且所述输入图像为N个第二图像,或者所述第一图像以及所述N个第二图像时,所述图像质量评价信息为对所述N个第二图像中每一个第二图像的图像质量评价信息;
    相应的,根据所获得的所述图像质量评价信息输出目标图像,包括:
    判断所述N个第二图像中是否存在图像质量评价信息为预定数字的目标第二图像;
    若所述N个第二图像中存在图像质量评价信息为预定数字的目标第二图像,则确定所述图像质量评价信息为预定数字的目标第二图像为图像质量最好的图像,并输出所述图像质量评价信息为预定数字的目标第二图像。
  6. 根据权利要求5所述的图像处理方法,其特征在于,在判断所述N个第二图像中是否存在图像质量评价信息为预定数字的目标第二图像之后,还包括:
    若所述N个第二图像中不存在图像质量评价信息为预定数字的目标第二图像,则确定所述第一图像为图像质量最好的图像,输出所述第一图像。
  7. 根据权利要求5所述的图像处理方法,其特征在于,所述根据所获得的所述图像质量评价信息输出目标图像,还包括:
    若所述N个第二图像中存在多个图像质量评价信息为预定数字的目标第二图像,则选择任一目标第二图像输出。
  8. 根据权利要求4所述的图像处理方法,其特征在于,若所述图像质量评价信息为分数,且所述输入图像为N个第二图像,或者所述第一图像以及所述N个第二图像时,所述图像质量评价信息为对所述N个第二图像中每一个第二图像的图像质量评价 信息;
    相应的,根据所获得的所述图像质量评价信息输出目标图像,包括:
    判断所述N个第二图像中是否存在分数大于预定分数阈值的目标第二图像;
    若所述N个第二图像中存在分数大于预定分数阈值的目标第二图像,则确定所述分数最高的目标第二图像为图像质量最好的图像,并输出所述分数最高的目标第二图像。
  9. 根据权利要求8所述的图像处理方法,其特征在于,在判断所述N个第二图像中是否存在分数大于预定分数阈值的目标第二图像之后,还包括:
    若所述N个第二图像中不存在分数大于预定分数阈值的目标第二图像,则确定所述第一图像为图像质量最好的图像,输出所述第一图像。
  10. 根据权利要求4所述的图像处理方法,其特征在于,所述根据所获得的所述图像质量评价信息输出目标图像,包括:
    若所述N个第二图像中不存在图像质量评价信息为预定数字的目标第二图像,或者所述N个第二图像中不存在分数大于预定分数阈值的目标第二图像,则获取新的图像增强模型,采用获取的所述新的图像增强模型对所述第一图像进行处理,获得新的第二图像,并将所述新的第二图像作为输入图像输入所述质量评价模型进行处理,获得新的图像质量评价信息;其中,所述新的图像增强模型为未对所述第一图像进行处理过的图像增强模型;
    若确定所述新的图像质量评价信息为所述预定数字,或者所述分数大于所述预定分数阈值,则输出新的第二图像,否则返回执行获取新的图像增强模型步骤以及后续步骤,直到返回执行的次数达到预设的次数阈值,输出所述第一图像。
  11. 根据权利要求4所述的图像处理方法,其特征在于,当所述图像质量评价信息为数字,且所述输入图像为第一图像和N个第二图像时,所述图像质量评价信息为对所述第一图像和N个第二图像中每一个第二图像的图像质量评价信息;
    相应的,根据所获得的所述图像质量评价信息输出目标图像,包括:
    判断所述第一图像和N个第二图像中是否存在图像质量评价信息为预定数字的图像;
    若所述第一图像的图像质量评价信息为预定数字,且所述N个第二图像中存在至少一个图像质量评价信息为预定数字的目标第二图像,则从所述第一图像和所述目标第二图像中选择任一图像输出。
  12. 根据权利要求11所述的图像处理方法,其特征在于,在判断所述第一图像和N个第二图像中是否存在图像质量评价信息为预定数字的目标第二图像之后,还包括:
    若所述第一图像的图像质量评价信息不是预定数字,且所述N个第二图像中存在至少一个图像质量评价信息为预定数字的目标第二图像,则从所述目标第二图像中选择任一图像输出;或者,
    若所述第一图像的图像质量评价信息不是预定数字,且所述N个第二图像中不存在图像质量评价信息为所述预定数字的目标第二图像,则输出所述第一图像。
  13. 根据权利要求4所述的图像处理方法,其特征在于,当所述图像质量评价信息为分数,且所述输入图像为第一图像和N个第二图像时,所述图像质量评价信息为 对所述第一图像和N个第二图像中每一个第二图像的图像质量评价信息;
    相应的,根据所获得的所述图像质量评价信息输出目标图像,包括:
    根据第一图像的分数和所述N个第二图像的分数,从所述第一图像和所述N个第二图像中,确定分数最高的图像为所述图像质量最好的图像,并输出所述分数最高的图像。
  14. 根据权利要求1-2、5-13任一项所述的图像处理方法,其特征在于,所述获取第一图像包括:
    获取多帧RAW图像;
    对所述多帧RAW图像进行图像融合处理,得到第一图像。
  15. 根据权利要求14所述的图像处理方法,其特征在于,所述对所述多帧RAW图像进行图像融合处理,得到第一图像,包括:
    将所述多帧RAW图像划分为至少两组,对每组RAW图像进行图像融合处理得到至少两个第一图像。
  16. 根据权利要求1所述的图像处理方法,其特征在于,
    当N=M=1,所述图像质量评价信息为所述第二图像的图像质量评价信息时,若所述图像质量评价信息为预定数字,或者所述图像质量评价信息对应的分数大于或等于预定分数阈值,则所述目标图像为所述第二图像;若所述图像质量评价信息不是预定数字,或者所述图像质量评价信息对应的分数小于预定分数阈值,则所述目标图像为所述第一图像;
    当N=M=1,所述图像质量评价信息为针对所述第一图像和所述第二图像的图像质量评价信息,所述图像质量评价信息用于表示所述第二图像的图像质量是否比所述第一图像的图像质量好时,若所述图像质量评价信息为预定数字,则所述目标图像为所述第二图像;若所述图像质量评价信息不是预定数字,则所述目标图像为所述第一图像;
    当N=M=1,所述图像质量评价信息包括所述第一图像和所述第二图像各自对应的图像质量评价信息时,若所述第一图像的图像质量评价信息为预定数字,且所述第二图像的图像质量评价信息为预定数字,则所述目标图像为所述第二图像和所述第一图像中的任一图像;若所述第一图像的图像质量评价信息不是预定数字,且所述第二图像的图像质量评价信息为预定数字,则所述目标图像为所述第二图像;若所述第一图像的图像质量评价信息为预定数字,且所述第二图像的图像质量评价信息不是预定数字,则所述目标图像为所述第一图像;或者,
    若所述第二图像的图像质量评价信息对应的分数大于所述第一图像的图像质量评价信息对应的分数,则所述目标图像为所述第二图像;若所述第二图像的图像质量评价信息对应的分数小于所述第一图像的图像质量评价信息对应的分数,则所述目标图像为所述第一图像;若所述第二图像的图像质量评价信息对应的分数等于所述第一图像的图像质量评价信息对应的分数,则所述目标图像为所述第二图像和所述第一图像中的任一图像;
    当N=M≥2,所述图像质量评价信息为所述N个第二图像中每一个第二图像的第二图像的图像质量评价信息时,若所述N个第二图像中存在图像质量评价信息为预定 数字的目标第二图像,则所述目标图像为至少一个所述目标第二图像;若所述N个第二图像中不存在图像质量评价信息为预定数字的目标第二图像,则所述目标图像为所述第一图像;或者,
    若所述N个第二图像中存在图像质量评价信息对应的分数大于或等于预定分值阈值的目标第二图像,则所述目标图像为至少一个所述目标第二图像;若所述N个第二图像中不存在图像质量评价信息对应的分数大于或等于预定分值阈值的目标第二图像,则所述目标图像为所述第一图像;
    当N=M≥2,所述图像质量评价信息为针对第一图像和所述N个第二图像中每一个第二图像的第二图像的图像质量评价信息时,所述目标图像为所述第一图像和所述N个第二图像中图像质量评价信息为预定数字的任一图像,或者,所述目标图像为所述第一图像和所述N个第二图像中图像质量评价信息对应的分数大于或等于预定分值阈值的任一图像,或者,所述目标图像为所述第一图像和所述N个第二图像中图像质量评价信息对应的分数最高的图像。
  17. 一种图像处理方法,其特征在于,包括:
    获取第一图像;
    将所述第一图像输入质量评价模型进行处理,得到第一图像质量评价信息;
    当根据所述第一图像质量评价信息确定所述第一图像的图像质量符合要求时,将所述第一图像输入M个图像增强模型进行处理,得到N个第二图像,并显示或保存所述N个第二图像中的一个或者多个;其中,N和M为正整数,所述M个图像增强模型中的每个图像增强模型均不同,所述N个第二图像也存在不同。
  18. 根据权利要求17所述的图像处理方法,其特征在于,当根据所述第一图像质量评价信息确定所述第一图像的图像质量不符合要求时,显示或保存所述第一图像;或者所述第一图像由电子设备在高倍变焦拍照模式下获取得到。
  19. 根据权利要求17或18所述的图像处理方法,其特征在于,所述第一图像质量评价信息为数字或分数,
    当所述第一图像质量评价信息为数字时,所述第一图像的图像质量符合要求是指所述第一图像质量评价信息为预定数字;或者,
    当所述第一图像质量评价信息为分数时,所述第一图像的图像质量符合要求是指所述第一图像的分数大于或等于预设阈值。
  20. 根据权利要求19所述的图像处理方法,其特征在于,所述预设阈值为0.25,M和N均为1。
  21. 根据权利要求17-18、20任一项所述的图像处理方法,其特征在于,所述得到N个第二图像之后,还包括:
    将所述N个第二图像,或者所述第一图像以及所述N个第二图像作为输入图像输入所述质量评价模型进行处理待处理图像输入质量评价模型进行处理,得到第二图像质量评价信息;
    根据所述第二图像质量评价信息输出目标图像,所述目标图像是所述第一图像和所述N个第二图像中的至少一个图像。
  22. 一种电子设备,包括存储器、处理器以及存储在所述存储器中并可在所述处 理器上运行的计算机程序,其特征在于,所述处理器执行所述计算机程序时使得所述电子设备执行如权利要求1至16任一项所述的图像处理方法,或如权利要求17至21任一项所述的图像处理方法。
  23. 一种计算机可读存储介质,所述计算机可读存储介质存储有计算机程序,其特征在于,所述计算机程序被处理器执行时使得电子设备执行如权利要求1至16任一项所述的图像处理方法,或如权利要求17至21任一项所述的图像处理方法。
PCT/CN2021/080027 2020-03-13 2021-03-10 一种图像处理方法及电子设备 WO2021180131A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010179848.0A CN113395440A (zh) 2020-03-13 2020-03-13 一种图像处理方法及电子设备
CN202010179848.0 2020-03-13

Publications (1)

Publication Number Publication Date
WO2021180131A1 true WO2021180131A1 (zh) 2021-09-16

Family

ID=77616279

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/080027 WO2021180131A1 (zh) 2020-03-13 2021-03-10 一种图像处理方法及电子设备

Country Status (2)

Country Link
CN (1) CN113395440A (zh)
WO (1) WO2021180131A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113962965A (zh) * 2021-10-26 2022-01-21 腾讯科技(深圳)有限公司 图像质量评价方法、装置、设备以及存储介质

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116684687B (zh) * 2023-08-01 2023-10-24 蓝舰信息科技南京有限公司 基于数字孪生技术的增强可视化教学方法
CN117333733A (zh) * 2023-09-06 2024-01-02 中移数智科技有限公司 裂缝检测模型训练方法、装置、电子设备及存储介质
CN116916151B (zh) * 2023-09-12 2023-12-08 荣耀终端有限公司 拍摄方法、电子设备和存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1819624A (zh) * 2005-02-07 2006-08-16 三星电子株式会社 进行数据处理的设备和方法及其记录介质
CN101901482A (zh) * 2009-05-31 2010-12-01 汉王科技股份有限公司 判断去雾增强图像质量效果的方法
CN109934776A (zh) * 2018-12-25 2019-06-25 北京奇艺世纪科技有限公司 模型生成方法、视频增强方法、装置及计算机可读存储介质
CN109949250A (zh) * 2019-03-29 2019-06-28 北京奇艺世纪科技有限公司 一种图像处理方法及装置
CN110738611A (zh) * 2019-09-20 2020-01-31 网宿科技股份有限公司 一种视频画质增强方法、系统及设备

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109191382B (zh) * 2018-10-18 2023-12-05 京东方科技集团股份有限公司 图像处理方法、装置、电子设备及计算机可读存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1819624A (zh) * 2005-02-07 2006-08-16 三星电子株式会社 进行数据处理的设备和方法及其记录介质
CN101901482A (zh) * 2009-05-31 2010-12-01 汉王科技股份有限公司 判断去雾增强图像质量效果的方法
CN109934776A (zh) * 2018-12-25 2019-06-25 北京奇艺世纪科技有限公司 模型生成方法、视频增强方法、装置及计算机可读存储介质
CN109949250A (zh) * 2019-03-29 2019-06-28 北京奇艺世纪科技有限公司 一种图像处理方法及装置
CN110738611A (zh) * 2019-09-20 2020-01-31 网宿科技股份有限公司 一种视频画质增强方法、系统及设备

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113962965A (zh) * 2021-10-26 2022-01-21 腾讯科技(深圳)有限公司 图像质量评价方法、装置、设备以及存储介质
CN113962965B (zh) * 2021-10-26 2023-06-09 腾讯科技(深圳)有限公司 图像质量评价方法、装置、设备以及存储介质

Also Published As

Publication number Publication date
CN113395440A (zh) 2021-09-14

Similar Documents

Publication Publication Date Title
WO2021180131A1 (zh) 一种图像处理方法及电子设备
WO2017101626A1 (zh) 一种实现图像处理的方法及装置
WO2021022983A1 (zh) 图像处理方法和装置、电子设备、计算机可读存储介质
CN106899781B (zh) 一种图像处理方法及电子设备
CN109076159B (zh) 电子设备及其操作方法
WO2018176925A1 (zh) Hdr图像的生成方法及装置
CN101690160B (zh) 用于使用自动聚焦统计的运动检测的方法、系统及设备
CN107948517A (zh) 预览画面虚化处理方法、装置及设备
CN112258404A (zh) 图像处理方法、装置、电子设备和存储介质
CN111753869A (zh) 图像处理方法、图像处理装置、存储介质、图像处理系统及已学习模型制造方法
JP2009500977A (ja) ファインダー上に画像焦点情報を表示するための改善されたシステム及び方法
CN111131688B (zh) 一种图像处理方法、装置及移动终端
CN111325698A (zh) 图像处理方法、装置及系统、电子设备
CN113658197B (zh) 图像处理方法、装置、电子设备及计算机可读存储介质
Choi et al. A method for fast multi-exposure image fusion
US20230033956A1 (en) Estimating depth based on iris size
CN113962873A (zh) 一种图像去噪方法、存储介质及终端设备
US20220398704A1 (en) Intelligent Portrait Photography Enhancement System
CN108810407B (zh) 一种图像处理方法、移动终端及计算机可读存储介质
US20080199073A1 (en) Red eye detection in digital images
WO2023151210A1 (zh) 图像处理方法、电子设备及计算机可读存储介质
EP3070669A1 (en) Method and apparatus for color smoothing in an alpha matting process
CN111354058B (zh) 一种图像着色方法、装置、图像采集设备及可读存储介质
CN114387443A (zh) 一种图像处理方法、存储介质及终端设备
CN108431867B (zh) 一种数据处理方法及终端

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21767926

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21767926

Country of ref document: EP

Kind code of ref document: A1