CN118247155A - Model training method, image enhancement processing method and related devices - Google Patents

Model training method, image enhancement processing method and related devices Download PDF

Info

Publication number
CN118247155A
CN118247155A CN202211630596.4A CN202211630596A CN118247155A CN 118247155 A CN118247155 A CN 118247155A CN 202211630596 A CN202211630596 A CN 202211630596A CN 118247155 A CN118247155 A CN 118247155A
Authority
CN
China
Prior art keywords
image
feature
characteristic
current
images
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211630596.4A
Other languages
Chinese (zh)
Inventor
冷文华
陈志豪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing Opper Software Technology Co ltd
Original Assignee
Nanjing Opper Software Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing Opper Software Technology Co ltd filed Critical Nanjing Opper Software Technology Co ltd
Priority to CN202211630596.4A priority Critical patent/CN118247155A/en
Publication of CN118247155A publication Critical patent/CN118247155A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761—Proximity, similarity or dissimilarity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Multimedia (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • General Engineering & Computer Science (AREA)
  • Molecular Biology (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Processing (AREA)

Abstract

The embodiment of the application provides a model training method, an image enhancement processing method and a related device, which comprise the following steps: acquiring a first characteristic image and a second characteristic image, wherein the first characteristic image is obtained by adjusting image parameters of the first characteristic image according to an image parameter adjusting instruction of a user under preset backlight parameters, and the image parameters comprise at least one of the following: contrast, brightness, saturation; processing the first characteristic image through an image enhancement processing model to obtain a third characteristic image; determining a target loss function according to the third characteristic image and the second characteristic image; and adjusting network parameters of the image enhancement processing model according to the target loss function. The application is beneficial to generating the image which accords with the preference of the user and improving the display effect of the image in the low backlight scene.

Description

Model training method, image enhancement processing method and related devices
Technical Field
The application belongs to the technical field of data processing, and particularly relates to a model training method, an image enhancement processing method and a related device.
Background
With the continuous development of electronic equipment technology, electronic equipment such as mobile phones has become an indispensable existence in people's lives. The endurance of the electronic device is limited due to the battery capacity and the like, so that the time of once use of the electronic device after charging is as long as possible to be an important direction of people's attention, and the display brightness of the screen of the electronic device is an important factor affecting the endurance of the electronic device, so that the power consumption of the electronic device is reduced by lowering the backlight of the electronic device, and the endurance of the electronic device is improved. Therefore, how to achieve the purpose that the display effect meets the requirement of human eyes more is a main challenge at present without affecting the look and feel of people for the display content of the electronic equipment while ensuring that the brightness of the screen is reduced and the power consumption of the screen is reduced.
Disclosure of Invention
The embodiment of the application provides a model training method, an image enhancement processing method and a related device, which are used for generating images meeting user preference and improving the display effect of the images in a low-backlight scene.
In a first aspect, an embodiment of the present application provides a model training method, including:
acquiring a first characteristic image and a second characteristic image, wherein the first characteristic image is obtained by adjusting image parameters of the first characteristic image according to an image parameter adjusting instruction of a user under preset backlight parameters, and the image parameters comprise at least one of the following: contrast, brightness, saturation;
processing the first characteristic image through an image enhancement processing model to obtain a third characteristic image;
Determining an objective loss function according to the third characteristic image and the second characteristic image;
And adjusting network parameters of the image enhancement processing model according to the target loss function.
In a second aspect, an embodiment of the present application provides an image enhancement processing method, including:
acquiring a reference image displayed by electronic equipment under preset backlight parameters;
processing the reference image through an image enhancement processing model to obtain a target image, wherein the image enhancement processing model is trained by the model training method in the first aspect;
And displaying the target image.
In a third aspect, an embodiment of the present application provides a model training apparatus, including:
the device comprises an acquisition unit, a display unit and a display unit, wherein the acquisition unit is used for acquiring a first characteristic image and a second characteristic image, the first characteristic image is obtained by adjusting image parameters of the first characteristic image according to an image parameter adjusting instruction of a user under preset backlight parameters, and the image parameters comprise at least one of the following: contrast, brightness, saturation;
The processing unit is used for processing the first characteristic image through an image enhancement processing model to obtain a third characteristic image;
A determining unit configured to determine a target loss function from the third feature image and the second feature image;
And the adjusting unit is used for adjusting the network parameters of the image enhancement processing model according to the target loss function.
In a fourth aspect, an embodiment of the present application provides an image enhancement processing apparatus, including:
the acquisition unit is used for acquiring a reference image displayed by the electronic equipment under the preset backlight parameters;
The processing unit is used for processing the reference image through an image enhancement processing model to obtain a target image, and the image enhancement processing model is trained by the model training method in the first aspect;
and the display unit is used for displaying the target image.
In a fifth aspect, an embodiment of the present application provides an electronic device comprising a processor, a memory, a communication interface, and one or more programs stored in the memory and configured to be executed by the processor, the programs comprising instructions for performing the steps in the first or second aspects of the embodiment of the present application.
In a sixth aspect, embodiments of the present application provide a computer storage medium having stored thereon a computer program/instruction for execution by a processor to perform the steps of the first or second aspects described above.
In a seventh aspect, embodiments of the present application provide a chip comprising a processor for invoking and running a computer program from memory, such that a device on which the chip is mounted performs the steps as described in the first or second aspects above.
In an eighth aspect, embodiments of the present application provide a computer program product, wherein the computer program product comprises a non-transitory computer readable storage medium storing a computer program operable to cause a computer to perform part or all of the steps described in the first or second aspects of the embodiments of the present application. The computer program product may be a software installation package.
It can be seen that in the embodiment of the present application, the electronic device may acquire a first feature image, and a second feature image parameter obtained by adjusting an image parameter of the first feature image by a user under a preset backlight parameter, and may process the first feature image through an image enhancement processing model to obtain a third feature image, and then determine a target loss function according to the third feature image and the second feature image, and adjust a network parameter of the image enhancement processing model according to the target loss function, where the image parameter may include at least one of contrast, brightness, and saturation. It can be seen that, when the image enhancement processing model is trained, the training data used includes the first feature image that is adjusted based on the user instruction under the specific backlight condition, and since at least one of the contrast, brightness and saturation of the first feature image is adjusted based on the user requirement, the image enhancement processing model trained by the first feature image can generate an image that better meets the user preference, and is beneficial to improving the display effect of the image in the specific backlight scene, such as the low backlight scene.
Drawings
In order to more clearly illustrate the embodiments of the application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic structural diagram of an electronic device according to an embodiment of the present application;
FIG. 2a is a schematic flow chart of a model training method according to an embodiment of the present application;
FIG. 2b is a schematic diagram of an image enhancement processing model according to an embodiment of the present application;
FIG. 2c is an architectural diagram representation of a multi-scale Laplacian block provided by an embodiment of the application;
Fig. 3a is a schematic flow chart of an image enhancement processing method according to an embodiment of the present application;
FIG. 3b is a schematic diagram of a test result provided by an embodiment of the present application;
FIG. 3c is a schematic diagram of another test result provided by an embodiment of the present application;
FIG. 3d is a schematic diagram of another test result provided by an embodiment of the present application;
FIG. 4a is a functional block diagram of a model training device according to an embodiment of the present application;
FIG. 4b is a block diagram of functional units of another model training apparatus according to an embodiment of the present application;
FIG. 5a is a block diagram showing functional units of an image enhancement processing apparatus according to an embodiment of the present application;
Fig. 5b is a functional block diagram of another image enhancement processing apparatus according to an embodiment of the present application.
Detailed Description
In order that those skilled in the art will better understand the present application, a technical solution in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The terms first, second and the like in the description and in the claims and in the above-described figures are used for distinguishing between different objects and not necessarily for describing a sequential or chronological order. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.
Referring to fig. 1, fig. 1 is a schematic structural diagram of an electronic device according to an embodiment of the application. As shown, the electronic device 110 includes a processor 120, a memory 130, a communication interface 140, and one or more programs 131, wherein the one or more programs 131 are stored in the memory 130 and configured to be executed by the processor 120, and the one or more programs 131 include instructions for performing any of the following method embodiments. In a specific implementation, the processor 120 is configured to perform any step performed by the electronic device in the method embodiment described below, and when performing a data transmission operation such as receiving, the communication interface 140 is optionally invoked to complete the corresponding operation. The electronic equipment can be a server or a terminal equipment, the image enhancement processing model can be trained through the electronic equipment, meanwhile, a reference image to be displayed under the preset backlight parameters of the electronic equipment can be obtained through the electronic equipment, and then the reference image is processed through the trained image enhancement processing model, so that a target image is output and displayed.
Referring to fig. 2a, fig. 2a is a schematic flow chart of a model training method according to an embodiment of the application. The method may be performed by an electronic device, as shown in fig. 2a, the method comprising the following steps.
In step 201, a first feature image and a second feature image are acquired.
The first characteristic image is obtained by adjusting image parameters of the first characteristic image according to an image parameter adjusting instruction of a user under a preset backlight parameter, and the image parameters comprise at least one of the following: contrast, brightness, saturation.
In a specific implementation, the backlight parameter, that is, the screen display brightness of the electronic device, may be obtained by reducing a preset backlight parameter (for example, the backlight parameter conventionally set by using a mobile phone may be a default value of the backlight parameter in the mobile phone) by a certain value, for example, a backlight value reduced by 5% on the conventionally set backlight parameter may be used as the preset backlight parameter, and the first feature image is displayed under the preset backlight parameter, and a user image parameter adjustment instruction is received, so that at least one image parameter of contrast, brightness and saturation of the first feature image is adjusted, thereby obtaining the second feature image. At this time, the user can adjust the image parameters according to personal preference to achieve the most comfortable visual effect under the preset backlight parameters.
The electronic device can acquire a plurality of groups of characteristic composition training data sets to train the image enhancement processing model, and each group of characteristic images comprises a first characteristic image before user adjustment and a second characteristic image after user adjustment.
In a specific implementation, the first feature images in the same training data set are all obtained by adjusting image parameters under the same preset backlight parameters, the electronic device can also respectively acquire a plurality of corresponding different training data sets according to different preset backlight parameters, so that a plurality of trained image enhancement processing models can be obtained according to the training of the plurality of different training data sets, the trained image enhancement processing models corresponding to the preset backlight parameters can be used for processing reference images to be displayed under the condition of different preset backlight parameters, and accordingly adjusted target images which accord with user preferences and are better in display effect under the preset backlight parameters are obtained and displayed.
And 202, processing the first characteristic image through an image enhancement processing model to obtain a third characteristic image.
And step 203, determining a target loss function according to the third characteristic image and the second characteristic image.
And step 204, adjusting network parameters of the image enhancement processing model according to the target loss function.
In a specific implementation, the electronic device may repeatedly execute steps 201-204 until the value of the objective loss function meets a preset condition (e.g., the value of the objective loss function is lower than a preset value), thereby obtaining a trained image enhancement processing model. In the process of continuously adjusting the network parameters of the image enhancement processing model according to the target loss function until the trained image enhancement processing model is obtained, the image enhancement processing model can continuously learn the difference and the similarity between the user adjustment image (namely the first characteristic image) and the model processing image (namely the third characteristic image), and the trained image enhancement processing model can output the optimal image according with the user preference by comparing the difference between the user adjustment image and the model processing image and adjusting the network parameters of the image enhancement processing model.
It can be seen that in the embodiment of the present application, the electronic device may acquire a first feature image, and a second feature image parameter obtained by adjusting an image parameter of the first feature image by a user under a preset backlight parameter, and may process the first feature image through an image enhancement processing model to obtain a third feature image, and then determine a target loss function according to the third feature image and the second feature image, and adjust a network parameter of the image enhancement processing model according to the target loss function, where the image parameter may include at least one of contrast, brightness, and saturation. It can be seen that, when the image enhancement processing model is trained, the training data used includes the first feature image that is adjusted based on the user instruction under the specific backlight condition, and since at least one of the contrast, brightness and saturation of the first feature image is adjusted based on the user requirement, the image enhancement processing model trained by the first feature image can generate an image that better meets the user preference, and is beneficial to improving the display effect of the image in the specific backlight scene, such as the low backlight scene.
In one possible example, the image enhancement processing model includes N sub-networks, where N is an integer greater than 1, and the processing the first feature image by the image enhancement processing model to obtain a third feature image includes: carrying out Laplacian decomposition on the first characteristic image to obtain N fourth characteristic images with different resolutions; performing the following operations for each fourth feature image in the N fourth feature images to obtain N third feature images: and processing the current fourth characteristic image through a sub-network corresponding to the current fourth characteristic image in the N sub-networks to obtain a third characteristic image corresponding to the current fourth characteristic image, wherein the N sub-networks are in one-to-one correspondence with the N fourth characteristic images.
The laplace decomposition may be performed on the first feature image by, for example, performing downsampling or the like to obtain a plurality of fourth feature images with different resolutions. The plurality of decomposed fourth feature images may include a fourth feature image having the same resolution as the first feature image (i.e., a resolution adjustment amount of 0), and may further include other fourth feature images having a resolution smaller than the first feature image. Specifically, the plurality of fourth feature images may be divided into a plurality of different levels, the first level may be a fourth feature image having the same resolution as the first feature image, and the fourth feature image of each level except the first level may be obtained by decimating pixels of odd lines and columns (or even lines and columns) in the fourth feature image of the previous level.
For example, taking N as 3 as an example, assuming that the resolution of the first feature image is 256×256, the plurality of fourth feature images may include: a fourth feature image with a resolution of 64 x 64, a fourth feature image with a resolution of 128 x 128, and a fourth feature image with a resolution of 256 x 256.
The image enhancement processing model comprises a plurality of sub-networks which are in one-to-one correspondence with the fourth characteristic images, when the first characteristic image is processed, each sub-network can correspondingly process the fourth characteristic image of one level, that is, different from the enhancement of a single network, the input of different sub-networks of the image enhancement processing model comprises different detail information, the different sub-networks learn the information corresponding to the characteristic images of the corresponding level, so that the different sub-networks can process the images of different fineness degrees, the different sub-networks can also enhance the images of different fineness degrees, and finally the output of the plurality of different sub-networks is synthesized to obtain the output of the whole image enhancement processing model, and the combination of low-frequency information and high-frequency information is realized.
For example, the sub-network corresponding to the coarsest level, i.e., the level with the lowest resolution of the fourth feature image, may adjust the global illumination, while the sub-networks corresponding to the fourth feature image of the other levels may enhance the underlying details (e.g., contrast, sharpness, etc.) at a finer pyramid level.
In this example, the image enhancement processing model includes a plurality of sub-networks, and the electronic device may adjust the first feature image to a plurality of fourth feature images with different resolutions, and process the fourth feature images through the plurality of sub-networks to obtain a third feature image for training the image enhancement processing model, which is beneficial to improving the fineness of image processing of the trained image enhancement processing model and improving the display effect of the image processed by the model.
In one possible example, the processing, by a sub-network corresponding to a current fourth feature image in the N sub-networks, the current fourth feature image to obtain the third feature image corresponding to the current fourth feature image includes: processing the current fourth characteristic image through a sub-network corresponding to the current fourth characteristic image and outputting a fifth characteristic image corresponding to the current fourth characteristic image; and performing jump connection processing on the current fourth characteristic image and a fifth characteristic image corresponding to the current fourth characteristic image to obtain a third characteristic image corresponding to the current fourth characteristic image.
In specific implementation, referring to fig. 2b, fig. 2b is a schematic architecture diagram of an image enhancement processing model according to an embodiment of the present application. Taking N equal to 3 as an example, the image enhancement processing model may include three subnetworks (Net 1, net2 and Net 3), and 3 fourth feature images (L1, L2 and L3) may be obtained by subjecting the first feature image to laplacian pyramid decomposition, where the third feature image corresponding to L1 isIs obtained by jumping connection of the characteristic images output by L1 and Net1, wherein the characteristic images output by Net1, L1 andIs the same. Similarly, the third characteristic image corresponding to L3 isL3 corresponding third feature image, namely
In this example, the image enhancement processing model includes a plurality of sub-networks, and the electronic device may adjust the first feature image to a plurality of fourth feature images with different resolutions, and process the fourth feature images through the plurality of sub-networks to obtain a third feature image for training the image enhancement processing model, which is beneficial to improving the fineness of image processing of the trained image enhancement processing model and improving the display effect of the image processed by the model.
In one possible example, the determining the target loss function from the third feature image and the second feature image includes: the following operations are executed for an a-th third feature image in the N third feature images to determine N sixth feature images, a is smaller than or equal to N, and the larger the value of a of the a-th third feature image is, the larger the resolution of a fourth feature image corresponding to the a-th third feature image is: if a is equal to 1, the a third characteristic image is used as a sixth characteristic image corresponding to the a third characteristic image; if a is greater than 1, carrying out residual connection processing on the a third characteristic image and a characteristic image obtained by upsampling the a-1 third characteristic image to obtain a sixth characteristic image corresponding to the a third characteristic image;
Determining a first loss function and a second loss function different from the first loss function from the N sixth feature images and the second feature image; determining a third loss function according to M third feature images with the largest resolution of a fourth feature image corresponding to the N third feature images and the second feature images, wherein M is smaller than N; determining loss weights respectively corresponding to the first loss function, the second loss function and the third loss function; the target loss function is determined from the first loss function, the second loss function, the third loss function, and the loss weight.
Taking N equal to 3 as an example, please continue to refer to fig. 2b, wherein a is equal to 1, namely, the first third feature image is L3, the second third feature image is L2, the third feature image is L1, and at this time, the sixth feature image corresponding to L3I.e. equal toL2-corresponding sixth feature imageI.e. the last third feature imagePost-upsampling ANDThe sixth characteristic image corresponding to L1 obtained after the connection processing is determined in the same way as L2, wherein the method is similar to that of the method for determining theThe up-sampling process can keep the resolution of each image which needs to be connected to be consistent, and the processing is convenient.
When the third loss function is determined, the output of the sub-network of each level is not directly compared with the second characteristic image adjusted by the user, but the sub-network output of part of the coarse image level is removed, the third loss function is determined only according to the output of the sub-network of the finest first several levels, and the local sharpness of the enhancement result can be maintained to a certain extent. Taking still the image enhancement processing model architecture shown in fig. 3a as an example, assuming that M is equal to 2, when determining the third loss function, one can rely onAndAnd determining the user-adjusted second feature image without using/>, when determining the third loss function
In a specific implementation, taking the first loss function, the third loss function, and the second loss function as examples, where L d、Ll and L c are respectively taken, assuming that the loss weight corresponding to the first loss function is ω d, the loss weight corresponding to the third loss function is ω l, and the loss weight corresponding to the second loss function is L c as examples, the target loss function L total may be determined by the following formula:
Ltotal＝ωdLd+ωlLl+ωcLc。
in this example, residual connection is performed between different sub-networks to generate a plurality of sixth feature images, so that the low-frequency information and the high-frequency information can be reasonably combined, the first loss function and the second loss function are determined through the plurality of sixth feature images and the second feature images adjusted by the user, the third loss function is determined according to a finer part of the third feature images and the second feature images used for adjustment, and the target loss function used for adjusting the network parameters of the image enhancement processing model is determined according to the loss weights corresponding to the loss functions, which is beneficial to improving the fineness of image processing of the trained image enhancement processing model and improving the display effect of the image processed by the model.
In one possible example, the determining a first loss function from the N sixth feature images and the second feature image includes: the resolution of the second characteristic image is adjusted to obtain N seventh characteristic images, the N seventh characteristic images are in one-to-one correspondence with the N sixth characteristic images, and the resolution of each seventh characteristic image is the same as the resolution of the sixth characteristic image corresponding to each seventh characteristic image; the following operation is executed for each sixth feature image in the N sixth feature images to determine N first parameters, wherein the first parameters corresponding to the current sixth feature image are determined according to the current sixth feature image, a seventh feature image corresponding to the current sixth feature image and the pixel number of the current sixth feature image; and determining the first loss function according to the N first parameters.
For example, assuming that the sixth feature image includes image 1, image 2, and image 3, and the resolutions are 64×64, 128×128, and 256×256, respectively, after the resolution of the second feature image is adjusted, image 4 with resolution of 64×64, image 5 with resolution of 128×128, and image 6 with resolution of 256×256 are obtained, at this time, a first parameter 1 may be determined according to image 1 and image 4, a first parameter 2 may be determined according to image 2 and image 5, a first parameter 3 may be determined according to image 3 and image 5, and a first loss function may be determined according to the first parameter 1, first parameter 2, and first parameter 3.
In a specific implementation, since the image enhancement processing model includes N sub-networks, each sub-network is used for processing fourth feature images with different resolutions, and each sub-network can obtain a sixth feature image with the same resolution as the output fourth feature image, when comparing the second feature image with the N sixth feature images, the resolution of the second feature image can be adjusted, and the second feature image is also adjusted to be a plurality of seventh feature images with different resolutions, and N first parameters are determined respectively with the N sixth feature images with the same resolution, and finally a first loss function is determined, and when determining the loss function, information with different fineness degrees in the images is comprehensively compared.
Specifically, the first loss function may be a true loss of data, by which the final output of the sub-network corresponding to each laplacian pyramid level is constrained (i.e., in fig. 2bAnd) Euclidean distance (i.e. mean square loss) from the true output (i.e. the N seventh feature images processed from the user adjusted second feature image). Still taking the architecture of the image enhancement processing model shown in fig. 2b as an example, the first loss function L d can be determined by the following formula:
Wherein, I.e.Seventh feature image of same resolution, T k isThe number of pixels included in the display panel. From the pixel average point of view, this mean square loss may lead to enhancement results that are similar to the corresponding true values.
In this example, the resolution of the second feature image is adjusted to obtain N seventh feature images corresponding to the N sixth feature images one by one, and for each sixth feature image in the N sixth feature images, according to the current sixth feature image, the seventh feature image corresponding to the current sixth feature image, and the number of pixels of the current sixth feature image, a first parameter corresponding to the current sixth feature image is determined, so that a first loss function is determined according to the determined N first parameters, which is beneficial to improving the display effect of the image processed by the trained image enhancement processing model.
In one possible example, the determining a second loss function from the N sixth feature images and the second feature image includes: the resolution of the second characteristic image is adjusted to obtain N seventh characteristic images, the N seventh characteristic images are in one-to-one correspondence with the N sixth characteristic images, and the resolution of each seventh characteristic image is the same as the resolution of the sixth characteristic image corresponding to each seventh characteristic image; performing the following operation for each of the N sixth feature images to determine N second parameters; determining a second parameter corresponding to the current sixth feature image according to the current sixth feature image, a seventh feature image corresponding to the current sixth feature image, the height of the current sixth feature image and the width of the current sixth feature image; and determining the second loss function according to the N second parameters.
Wherein the second loss function may be a color loss, in order to guarantee the final output of the sub-network in the image enhancement processing model (i.e. in fig. 2bAnd) The color vector formed by R, G, B channels is the same as the corresponding true value, i.e. the direction of the seventh feature image, and the color loss can be constructed by using cosine similarity, and at this time, still taking the architecture of the image enhancement processing model shown in fig. 2b as an example, the second loss function L c can be determined by the following formula:
wherein H k and W k characterize the enhancement results in the k-th level Laplacian pyramid (i.e. ) I characterizesIs used to characterize the inner product operation. Wherein the closer the color of the true phase (i.e., the seventh feature image) of the enhancement result is, the closer the color loss L c is to 0.
In this example, the resolution of the second feature image is adjusted to obtain N seventh feature images corresponding to the N sixth feature images one by one, and for each sixth feature image in the N sixth feature images, a second parameter corresponding to the current sixth feature image is determined according to the current sixth feature image, the height of the current sixth feature image, and the width of the current sixth feature image, so that a second loss function is determined according to the determined N second parameters, which is beneficial to improving the display effect of the image processed by the trained image enhancement processing model.
In one possible example, the determining a third loss function according to M third feature images with the largest resolution of the fourth feature image among the N third feature images and the second feature image includes: the resolution of the second characteristic image is adjusted to obtain M eighth characteristic images, the M eighth characteristic images are in one-to-one correspondence with the M third characteristic images, and the resolution of each eighth characteristic image is the same as the resolution of the third characteristic image corresponding to each eighth characteristic image; the following operations are performed for each of the M third feature images to determine N third parameters: determining a third parameter corresponding to the current third feature image according to the current third feature image, an eighth feature image corresponding to the current third feature image and the pixel number of the current third feature image; and determining the third loss function according to the M third parameters.
Where the presence of the averaging term in the mean square loss is considered to generally produce a blurred enhancement result. The third loss function may be a laplace loss defined by absolute value error, and the difference between the third feature image of the sub-network corresponding to the input image with finer constraint part and the true eighth feature image adjusted by the user is still exemplified by the architecture of the image enhancement processing model shown in fig. 2b, and when M is equal to 2, the third loss function L l may be determined by the following formula:
Wherein, I.e.Eighth feature image of the same resolution, T k isThe number of pixels included in the display panel.
In this example, the resolution of the second feature image is adjusted to obtain M eighth feature images corresponding to the M third feature images one to one, and for each eighth feature image in the M eighth feature images, a third parameter corresponding to the current third feature image is determined according to the current eighth feature image, the eighth feature image corresponding to the current third feature image, and the number of pixels of the current third feature image, so that a third loss function is determined according to the determined M third parameters, which is beneficial to improving the display effect of the image processed by the trained image enhancement processing model.
In one possible example, the processing, by the sub-network corresponding to the current fourth feature image in the N sub-networks, the current fourth feature image to obtain a third feature image corresponding to the current fourth feature image includes: performing P-level convolution processing on the current fourth characteristic image, wherein each level of convolution processing performs the following operations: processing input feature images of current level convolution processing by adopting a plurality of convolution networks with different parameters, and carrying out residual connection processing on output feature images of the convolution networks to obtain the output feature images of the current level convolution processing, wherein the input feature images of the first level convolution processing are the fourth feature images, and the input feature images of other level convolution processing except the first level convolution processing are the output feature images of the previous level convolution processing; performing P-level deconvolution on the characteristic image obtained by the P-level convolution; and determining a third characteristic image corresponding to the current fourth characteristic image according to the characteristic image obtained by the P-th-level deconvolution process.
IN a specific implementation, the internal architecture of each sub-network may be as shown IN fig. 2b, where each P IN the image enhancement processing model shown IN fig. 2b is equal to 3, that is, each sub-network may include 3-level convolution processing (each level convolution processing includes inputting a multi-scale laplace block including a residual block, performing laplace pyramid reconstruction after processing and outputting) and 3-level deconvolution processing (each level deconvolution processing includes processing an input feature image through the multi-scale laplace block including the residual block and outputting the processed result to the deconvolution block after the laplace pyramid reconstruction), as shown IN fig. 2b, each level convolution processing and deconvolution processing includes a multi-scale laplace block (multiscale Laplacian-residual block, MSLB), the internal architecture of each multi-scale laplace block may be as shown IN fig. 2c, and a plurality of convolution networks with different parameters may be adopted IN the multi-scale laplace block (each convolution network may perform convolution operation, and IN the 3x3conv of fig. 2c may be a kernel convolution operation, and may be activated by a plurality of convolution functions IN 3x and may be a contrast function, and a contrast function may be input to 3 lu, and a contrast image may be input to a contrast-enhanced image after the contrast-enhanced image is outputted by a contrast function, and a contrast-enhanced image is outputted by a contrast function is outputted after the contrast-enhanced by a contrast function is outputted by a plurality of 3 x.
In this example, each of the multiple sub-networks of the image enhancement processing model may include a multi-level convolution process and a deconvolution process, and in each level convolution process, residual connection may be performed on the feature images output by different parameter networks through the multi-scale laplace block, so as to provide multi-scale information, which is beneficial to further improving the display effect of the image processed by the trained image enhancement processing model.
Referring to fig. 3a, fig. 3a is a flowchart illustrating an image enhancement processing method according to an embodiment of the application. The method may be performed by an electronic device, as shown in fig. 3a, the method comprising the following steps.
Step 301, obtaining a reference image displayed by the electronic device under a preset backlight parameter.
And step 302, processing the reference image through an image enhancement processing model to obtain a target image.
The image enhancement processing model is trained by any model training method. The image processing steps in the image processing model in the model training method can be cited in the image enhancement processing method, and are not described herein.
Step 303, displaying the target image.
In a specific implementation, the electronic device may be configured to start executing step 301 when it is detected that the backlight parameter is adjusted to a preset backlight parameter (for example, the power saving mode of the mobile phone or the screen brightness in a dark environment is reduced), or when an image enhancement request instruction of a user is received, so that image enhancement processing is performed on each frame of image displayed by the electronic device through a trained image enhancement processing model to obtain a target image, and the target image is displayed through a display screen of the electronic device, so that the display effect of the image under the preset backlight parameter is optimized.
In the embodiment of the application, because the training data used in training the image enhancement processing model comprises the first characteristic image which is adjusted based on the user instruction under the specific backlight condition, at least one of the contrast, brightness and saturation of the first characteristic image is adjusted based on the user demand, the trained image enhancement processing model is obtained by training the first characteristic image, and the reference image displayed by the electronic equipment under the preset backlight parameter is processed by the trained image enhancement processing model, so that the target image is output and displayed, the output target image better accords with the user preference, and the display effect of the target image in the low backlight scene is better.
For example, referring to fig. 3b, 3c and 3d, a test image (i.e. a reference image) is randomly selected and input into an image enhancement processing model to obtain an output image (i.e. a target image), the OLED display screen results are shown in fig. 3b and the LCD display screen results are shown in fig. 3c, and it can be seen from the figure that the color tone is overall bright under the condition of low backlight, and the color reality and saturation are maintained, so that the observation effect of the user can be ensured not to be affected for the image containing text information. The image observed by the display screen is shown in fig. 3d, it can be seen that the visual effect of the image after being observed and enhanced under the condition of reducing backlight is similar to or even better than the effect of the original image observed under the normal backlight, that is, the image enhancement processing model trained by the model training method can effectively improve the display effect of the image when the reference image is processed.
In accordance with the above embodiments, referring to fig. 4a, fig. 4a is a functional unit block diagram of a model training device according to an embodiment of the present application. The model training apparatus 40 includes:
An obtaining unit 401, configured to obtain a first feature image and a second feature image, where the first feature image is obtained by adjusting, under a preset backlight parameter, an image parameter of the first feature image according to an image parameter adjustment instruction of a user, where the image parameter includes at least one of: contrast, brightness, saturation;
a processing unit 402, configured to process the first feature image through an image enhancement processing model to obtain a third feature image;
a determining unit 403, configured to determine a target loss function according to the third feature image and the second feature image;
an adjusting unit 404, configured to adjust network parameters of the image enhancement processing model according to the objective loss function.
In one possible example, the image enhancement processing model includes N sub-networks, where N is an integer greater than 1, and the processing unit 402 is specifically configured to: carrying out Laplacian decomposition on the first characteristic image to obtain N fourth characteristic images with different resolutions; performing the following operations for each fourth feature image in the N fourth feature images to obtain N third feature images: and processing the current fourth characteristic image through a sub-network corresponding to the current fourth characteristic image in the N sub-networks to obtain a third characteristic image corresponding to the current fourth characteristic image, wherein the N sub-networks are in one-to-one correspondence with the N fourth characteristic images.
In one possible example, in the aspect that the current fourth feature image is processed through a sub-network corresponding to the current fourth feature image in the N sub-networks to obtain the third feature image corresponding to the current fourth feature image, the processing unit 402 is specifically configured to: processing the current fourth characteristic image through a sub-network corresponding to the current fourth characteristic image and outputting a fifth characteristic image corresponding to the current fourth characteristic image; and performing jump connection processing on the current fourth characteristic image and a fifth characteristic image corresponding to the current fourth characteristic image to obtain a third characteristic image corresponding to the current fourth characteristic image.
In one possible example, the determining unit 403 is specifically configured to: the following operations are executed for an a-th third feature image in the N third feature images to determine N sixth feature images, a is smaller than or equal to N, and the larger the value of a of the a-th third feature image is, the larger the resolution of a fourth feature image corresponding to the a-th third feature image is: if a is equal to 1, the a third characteristic image is used as a sixth characteristic image corresponding to the a third characteristic image; if a is greater than 1, carrying out residual connection processing on the a third characteristic image and a characteristic image obtained by upsampling the a-1 third characteristic image to obtain a sixth characteristic image corresponding to the a third characteristic image;
Determining a first loss function and a second loss function different from the first loss function from the N sixth feature images and the second feature image; determining a third loss function according to M third feature images with the largest resolution of a fourth feature image corresponding to the N third feature images and the second feature images, wherein M is smaller than N; determining loss weights respectively corresponding to the first loss function, the second loss function and the third loss function; the target loss function is determined from the first loss function, the second loss function, the third loss function, and the loss weight.
In one possible example, in said determining a first loss function from said N sixth feature images and said second feature images, said determining unit 403 is specifically configured to: the resolution of the second characteristic image is adjusted to obtain N seventh characteristic images, the N seventh characteristic images are in one-to-one correspondence with the N sixth characteristic images, and the resolution of each seventh characteristic image is the same as the resolution of the sixth characteristic image corresponding to each seventh characteristic image; the following operation is executed for each sixth feature image in the N sixth feature images to determine N first parameters, wherein the first parameters corresponding to the current sixth feature image are determined according to the current sixth feature image, a seventh feature image corresponding to the current sixth feature image and the pixel number of the current sixth feature image; and determining the first loss function according to the N first parameters.
In one possible example, in said determining a second loss function from said N sixth feature images and said second feature images, said determining unit 403 is specifically configured to: the resolution of the second characteristic image is adjusted to obtain N seventh characteristic images, the N seventh characteristic images are in one-to-one correspondence with the N sixth characteristic images, and the resolution of each seventh characteristic image is the same as the resolution of the sixth characteristic image corresponding to each seventh characteristic image; performing the following operation for each of the N sixth feature images to determine N second parameters; determining a second parameter corresponding to the current sixth feature image according to the current sixth feature image, a seventh feature image corresponding to the current sixth feature image, the height of the current sixth feature image and the width of the current sixth feature image; and determining the second loss function according to the N second parameters.
In one possible example, the determining unit 403 is specifically configured to determine, in terms of the M third feature images with the largest resolution according to the corresponding fourth feature image of the N third feature images and the second feature image, a third loss function: the resolution of the second characteristic image is adjusted to obtain M eighth characteristic images, the M eighth characteristic images are in one-to-one correspondence with the M third characteristic images, and the resolution of each eighth characteristic image is the same as the resolution of the third characteristic image corresponding to each eighth characteristic image; the following operations are performed for each of the M third feature images to determine N third parameters: determining a third parameter corresponding to the current third feature image according to the current third feature image, an eighth feature image corresponding to the current third feature image and the pixel number of the current third feature image; and determining the third loss function according to the M third parameters.
In one possible example, in the aspect that the processing is performed on the current fourth feature image through a sub-network corresponding to the current fourth feature image in the N sub-networks to obtain a third feature image corresponding to the current fourth feature image, the processing unit 402 is specifically configured to: performing P-level convolution processing on the current fourth characteristic image, wherein each level of convolution processing performs the following operations: processing input feature images of current level convolution processing by adopting a plurality of convolution networks with different parameters, and carrying out residual connection processing on output feature images of the convolution networks to obtain the output feature images of the current level convolution processing, wherein the input feature images of the first level convolution processing are the fourth feature images, and the input feature images of other level convolution processing except the first level convolution processing are the output feature images of the previous level convolution processing; performing P-level deconvolution on the characteristic image obtained by the P-level convolution; and determining a third characteristic image corresponding to the current fourth characteristic image according to the characteristic image obtained by the P-th-level deconvolution process.
It can be understood that, since the method embodiment and the apparatus embodiment are different presentation forms of the same technical concept, the content of the method embodiment portion in the present application should be synchronously adapted to the apparatus embodiment portion, which is not described herein.
In the case of integrated units, please refer to fig. 4b, fig. 4b is a block diagram illustrating functional units of another model training apparatus according to an embodiment of the present application. In fig. 4b, the model training apparatus 400 includes: a processing module 412 and a communication module 411. The processing module 412 is used to control and manage the actions of the model training device, e.g., the steps of the acquisition unit 401, the processing unit 402, the determination unit 403, and the adjustment unit 404, and/or other processes for performing the techniques described herein. The communication module 411 is used for interaction between the model training apparatus and other devices. As shown in fig. 4b, the model training apparatus may further comprise a storage module 413, the storage module 413 being configured to store program code and data of the model training apparatus.
The processing module 412 may be a Processor or controller, such as a central processing unit (Central Processing Unit, CPU), a general purpose Processor, a digital signal Processor (DIGITAL SIGNAL Processor, DSP), an ASIC, FPGA or other programmable logic device, transistor logic device, hardware components, or any combination thereof. Which may implement or perform the various exemplary logic blocks, modules and circuits described in connection with this disclosure. The processor may also be a combination that performs the function of a computation, e.g., a combination comprising one or more microprocessors, a combination of a DSP and a microprocessor, and the like. The communication module 411 may be a transceiver, an RF circuit, or a communication interface, etc. The memory module 413 may be a memory.
All relevant contents of each scenario related to the above method embodiment may be cited to the functional description of the corresponding functional module, which is not described herein. The model training apparatus 400 may perform the model training method shown in fig. 2 a.
In accordance with the above embodiments, referring to fig. 5a, fig. 5a is a functional unit block diagram of an image enhancement processing device according to an embodiment of the present application. The image enhancement processing device 50 includes:
an obtaining unit 501, configured to obtain a reference image displayed by an electronic device under a preset backlight parameter;
The processing unit 502 is configured to process the reference image through an image enhancement processing model, to obtain a target image, where the image enhancement processing model is obtained by training by any one of the model training methods described above;
a display unit 503 for displaying the target image.
It can be understood that, since the method embodiment and the apparatus embodiment are different presentation forms of the same technical concept, the content of the method embodiment portion in the present application should be synchronously adapted to the apparatus embodiment portion, which is not described herein.
In the case of using integrated units, refer to fig. 5b, fig. 5b is a block diagram illustrating functional units of another image enhancement processing apparatus according to an embodiment of the present application. In fig. 5b, the image enhancement processing apparatus 500 includes: a processing module 512 and a communication module 511. The processing module 512 is configured to control and manage actions of the image enhancement processing device, such as the steps of the acquisition unit 501, the processing unit 502, and the display unit 503, and/or other processes for performing the techniques described herein. The communication module 511 is used for interaction between the image enhancement processing apparatus and other devices. As shown in fig. 5b, the image enhancement processing apparatus may further comprise a storage module 513, the storage module 513 being for storing program code and data of the image enhancement processing apparatus.
The processing module 512 may be a Processor or controller, such as a central processing unit (Central Processing Unit, CPU), a general purpose Processor, a digital signal Processor (DIGITAL SIGNAL Processor, DSP), an ASIC, FPGA or other programmable logic device, transistor logic device, hardware components, or any combination thereof. Which may implement or perform the various exemplary logic blocks, modules and circuits described in connection with this disclosure. The processor may also be a combination that performs the function of a computation, e.g., a combination comprising one or more microprocessors, a combination of a DSP and a microprocessor, and the like. The communication module 511 may be a transceiver, an RF circuit, a communication interface, or the like. The memory module 513 may be a memory.
All relevant contents of each scenario related to the above method embodiment may be cited to the functional description of the corresponding functional module, which is not described herein. The image enhancement processing apparatus 500 may perform the image enhancement processing method shown in fig. 3 a.
The foregoing description of the embodiments of the present application has been presented primarily in terms of a method-side implementation. It will be appreciated that, in order to achieve the above-described functions, the electronic device includes a hardware structure and a software module for performing the respective functions. Those of skill in the art will readily appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as hardware or combinations of hardware and computer software. Whether a function is implemented as hardware or computer software driven hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The embodiment of the application can divide the functional units of the electronic device according to the method example, for example, each functional unit can be divided corresponding to each function, and two or more functions can be integrated in one processing unit. The integrated units may be implemented in hardware or in software functional units. It should be noted that, in the embodiment of the present application, the division of the units is schematic, which is merely a logic function division, and other division manners may be implemented in actual practice.
The embodiment of the application also provides a chip, wherein the chip comprises a processor, and the processor is used for calling and running the computer program from the memory, so that the device provided with the chip executes part or all of the steps described in the electronic device in the embodiment of the method.
The embodiment of the application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program makes a computer execute part or all of the steps of any one of the above method embodiments, and the computer includes an electronic device.
Embodiments of the present application also provide a computer program product comprising a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform part or all of the steps of any one of the methods described in the method embodiments above. The computer program product may be a software installation package, said computer comprising an electronic device.
It should be noted that, for simplicity of description, the foregoing method embodiments are all described as a series of acts, but it should be understood by those skilled in the art that the present application is not limited by the order of acts described, as some steps may be performed in other orders or concurrently in accordance with the present application. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily required for the present application.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments.
In the several embodiments provided by the present application, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, such as the above-described division of units, merely a division of logic functions, and there may be additional manners of dividing in actual implementation, such as multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, or may be in electrical or other forms.
The units described above as separate components may or may not be physically separate, and components shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units described above, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable memory. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a memory, comprising several instructions for causing a computer device (which may be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the above-mentioned method of the various embodiments of the present application. And the aforementioned memory includes: a usb disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Those of ordinary skill in the art will appreciate that all or a portion of the steps in the various methods of the above embodiments may be implemented by a program that instructs associated hardware, and the program may be stored in a computer readable memory, which may include: flash disk, read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disk or optical disk.
The foregoing has outlined rather broadly the more detailed description of embodiments of the application, wherein the principles and embodiments of the application are explained in detail using specific examples, the above examples being provided solely to facilitate the understanding of the method and core concepts of the application; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present application, the present description should not be construed as limiting the present application in view of the above.
Although the present invention is disclosed above, the present invention is not limited thereto. Variations and modifications, including combinations of the different functions and implementation steps, as well as embodiments of the software and hardware, may be readily apparent to those skilled in the art without departing from the spirit and scope of the invention.

Claims (13)

1. A method of model training, comprising:
acquiring a first characteristic image and a second characteristic image, wherein the first characteristic image is obtained by adjusting image parameters of the first characteristic image according to an image parameter adjusting instruction of a user under preset backlight parameters, and the image parameters comprise at least one of the following: contrast, brightness, saturation;
processing the first characteristic image through an image enhancement processing model to obtain a third characteristic image;
Determining an objective loss function according to the third characteristic image and the second characteristic image;
And adjusting network parameters of the image enhancement processing model according to the target loss function.
2. The method of claim 1, wherein the image enhancement processing model includes N sub-networks, N being an integer greater than 1, and wherein the processing the first feature image by the image enhancement processing model to obtain a third feature image includes:
Carrying out Laplacian decomposition on the first characteristic image to obtain N fourth characteristic images with different resolutions;
Performing the following operations for each fourth feature image in the N fourth feature images to obtain N third feature images:
And processing the current fourth characteristic image through a sub-network corresponding to the current fourth characteristic image in the N sub-networks to obtain a third characteristic image corresponding to the current fourth characteristic image, wherein the N sub-networks are in one-to-one correspondence with the N fourth characteristic images.
3. The method according to claim 2, wherein the processing the current fourth feature image through a sub-network corresponding to the current fourth feature image in the N sub-networks to obtain the third feature image corresponding to the current fourth feature image includes:
Processing the current fourth characteristic image through a sub-network corresponding to the current fourth characteristic image and outputting a fifth characteristic image corresponding to the current fourth characteristic image;
and performing jump connection processing on the current fourth characteristic image and a fifth characteristic image corresponding to the current fourth characteristic image to obtain a third characteristic image corresponding to the current fourth characteristic image.
4. A method according to claim 3, wherein said determining an objective loss function from said third feature image and said second feature image comprises:
The following operations are executed for an a-th third feature image in the N third feature images to determine N sixth feature images, a is smaller than or equal to N, and the larger the value of a of the a-th third feature image is, the larger the resolution of a fourth feature image corresponding to the a-th third feature image is:
If a is equal to 1, the a third characteristic image is used as a sixth characteristic image corresponding to the a third characteristic image;
if a is greater than 1, carrying out residual connection processing on the a third characteristic image and a characteristic image obtained by upsampling the a-1 third characteristic image to obtain a sixth characteristic image corresponding to the a third characteristic image;
Determining a first loss function and a second loss function different from the first loss function from the N sixth feature images and the second feature image;
Determining a third loss function according to M third feature images with the largest resolution of a fourth feature image corresponding to the N third feature images and the second feature images, wherein M is smaller than N;
determining loss weights respectively corresponding to the first loss function, the second loss function and the third loss function;
The target loss function is determined from the first loss function, the second loss function, the third loss function, and the loss weight.
5. The method of claim 4, wherein the determining a first loss function from the N sixth feature images and the second feature image comprises:
The resolution of the second characteristic image is adjusted to obtain N seventh characteristic images, the N seventh characteristic images are in one-to-one correspondence with the N sixth characteristic images, and the resolution of each seventh characteristic image is the same as the resolution of the sixth characteristic image corresponding to each seventh characteristic image;
the following operations are performed for each of the N sixth feature images to determine N first parameters:
determining a first parameter corresponding to a current sixth feature image according to the current sixth feature image, a seventh feature image corresponding to the current sixth feature image and the pixel number of the current sixth feature image;
And determining the first loss function according to the N first parameters.
6. The method of claim 4, wherein the determining a second loss function from the N sixth feature images and the second feature image comprises:
The resolution of the second characteristic image is adjusted to obtain N seventh characteristic images, the N seventh characteristic images are in one-to-one correspondence with the N sixth characteristic images, and the resolution of each seventh characteristic image is the same as the resolution of the sixth characteristic image corresponding to each seventh characteristic image;
performing the following operation for each of the N sixth feature images to determine N second parameters;
Determining a second parameter corresponding to the current sixth feature image according to the current sixth feature image, a seventh feature image corresponding to the current sixth feature image, the height of the current sixth feature image and the width of the current sixth feature image;
And determining the second loss function according to the N second parameters.
7. The method of claim 4, wherein determining the third loss function from the M third feature images with the largest resolution of the corresponding fourth feature image of the N third feature images and the second feature image comprises:
The resolution of the second characteristic image is adjusted to obtain M eighth characteristic images, the M eighth characteristic images are in one-to-one correspondence with the M third characteristic images, and the resolution of each eighth characteristic image is the same as the resolution of the third characteristic image corresponding to each eighth characteristic image;
The following operations are performed for each of the M third feature images to determine N third parameters:
Determining a third parameter corresponding to the current third feature image according to the current third feature image, an eighth feature image corresponding to the current third feature image and the pixel number of the current third feature image;
And determining the third loss function according to the M third parameters.
8. The method according to claim 2, wherein the processing the current fourth feature image through a sub-network corresponding to the current fourth feature image in the N sub-networks to obtain a third feature image corresponding to the current fourth feature image includes:
Performing P-level convolution processing on the current fourth characteristic image, wherein each level of convolution processing performs the following operations: processing input feature images of current level convolution processing by adopting a plurality of convolution networks with different parameters, and carrying out residual connection processing on output feature images of the convolution networks to obtain the output feature images of the current level convolution processing, wherein the input feature images of the first level convolution processing are the fourth feature images, and the input feature images of other level convolution processing except the first level convolution processing are the output feature images of the previous level convolution processing;
performing P-level deconvolution on the characteristic image obtained by the P-level convolution;
and determining a third characteristic image corresponding to the current fourth characteristic image according to the characteristic image obtained by the P-th-level deconvolution process.
9. An image enhancement processing method, comprising:
acquiring a reference image displayed by electronic equipment under preset backlight parameters;
Processing the reference image through an image enhancement processing model to obtain a target image, wherein the image enhancement processing model is trained by the model training method according to any one of claims 1 to 8;
And displaying the target image.
10. A model training device, comprising:
the device comprises an acquisition unit, a display unit and a display unit, wherein the acquisition unit is used for acquiring a first characteristic image and a second characteristic image, the first characteristic image is obtained by adjusting image parameters of the first characteristic image according to an image parameter adjusting instruction of a user under preset backlight parameters, and the image parameters comprise at least one of the following: contrast, brightness, saturation;
The processing unit is used for processing the first characteristic image through an image enhancement processing model to obtain a third characteristic image;
A determining unit configured to determine a target loss function from the third feature image and the second feature image;
And the adjusting unit is used for adjusting the network parameters of the image enhancement processing model according to the target loss function.
11. An image enhancement processing apparatus, comprising:
the acquisition unit is used for acquiring a reference image displayed by the electronic equipment under the preset backlight parameters;
a processing unit, configured to process the reference image through an image enhancement processing model, to obtain a target image, where the image enhancement processing model is obtained by training the model training method according to any one of claims 1 to 8;
and the display unit is used for displaying the target image.
12. An electronic device comprising a processor, a memory, a communication interface, and one or more programs stored in the memory and configured to be executed by the processor, the programs comprising instructions for performing the steps of the method of any of claims 1-8 or 9.
13. A computer readable storage medium having stored thereon a computer program/instructions, characterized in that the computer program/instructions are executed by a processor to perform the steps of the method according to any of claims 1-8 or 9.
CN202211630596.4A 2022-12-19 2022-12-19 Model training method, image enhancement processing method and related devices Pending CN118247155A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211630596.4A CN118247155A (en) 2022-12-19 2022-12-19 Model training method, image enhancement processing method and related devices

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211630596.4A CN118247155A (en) 2022-12-19 2022-12-19 Model training method, image enhancement processing method and related devices

Publications (1)

Publication Number Publication Date
CN118247155A true CN118247155A (en) 2024-06-25

Family

ID=91557812

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211630596.4A Pending CN118247155A (en) 2022-12-19 2022-12-19 Model training method, image enhancement processing method and related devices

Country Status (1)

Country Link
CN (1) CN118247155A (en)

Similar Documents

Publication Publication Date Title
Wang et al. Image quality assessment: from error visibility to structural similarity
CN110363753B (en) Image quality evaluation method and device and electronic equipment
CN109817170B (en) Pixel compensation method and device and terminal equipment
CN104094312B (en) Control to the video processnig algorithms based on the perceptual quality characteristic for measuring
CN108416722B (en) Image display method, storage medium and terminal equipment
CN108447040A (en) histogram equalization method, device and terminal device
CN113052923A (en) Tone mapping method, tone mapping apparatus, electronic device, and storage medium
CN113837980A (en) Resolution adjusting method and device, electronic equipment and storage medium
CN113837945B (en) Display image quality optimization method and system based on super-resolution reconstruction
CN112200719B (en) Image processing method, electronic device, and readable storage medium
CN113240576A (en) Method and device for training style migration model, electronic equipment and storage medium
CN102663683B (en) Image enhancement method and image enhancement system
CN111383299B (en) Image processing method and device and computer readable storage medium
CN111754412B (en) Method and device for constructing data pair and terminal equipment
CN118247155A (en) Model training method, image enhancement processing method and related devices
CN112272294A (en) Display image compression method, device, equipment and computer storage medium
CN111489289B (en) Image processing method, image processing device and terminal equipment
CN117078849A (en) Normal map generation method and device
CN109993694B (en) Method and device for generating super-resolution image
CN110191340A (en) Video frame processing method, device, equipment and storage medium
CN106780400B (en) Image processing method and device
CN107330905A (en) A kind of image processing method, device and storage medium
CN113902639A (en) Image processing method, image processing device, electronic equipment and storage medium
DE102019105508A1 (en) Spatio-temporal image metrics for rendered animations
CN116309274B (en) Method and device for detecting small target in image, computer equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination