CN114418887A - Image enhancement method and device, electronic equipment and storage medium - Google Patents

Image enhancement method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN114418887A
CN114418887A CN202210060864.7A CN202210060864A CN114418887A CN 114418887 A CN114418887 A CN 114418887A CN 202210060864 A CN202210060864 A CN 202210060864A CN 114418887 A CN114418887 A CN 114418887A
Authority
CN
China
Prior art keywords
image
current
renderer
current image
rendering
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210060864.7A
Other languages
Chinese (zh)
Other versions
CN114418887B (en
Inventor
李鑫
何栋梁
张琦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to CN202210060864.7A priority Critical patent/CN114418887B/en
Publication of CN114418887A publication Critical patent/CN114418887A/en
Application granted granted Critical
Publication of CN114418887B publication Critical patent/CN114418887B/en
Priority to KR1020230005149A priority patent/KR20230112063A/en
Priority to JP2023003837A priority patent/JP2023105807A/en
Priority to US18/155,901 priority patent/US20230230205A1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/60—Image enhancement or restoration using machine learning, e.g. neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4046—Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

The application discloses an image enhancement method, an image enhancement device, electronic equipment and a storage medium, relates to the technical field of artificial intelligence, and particularly relates to computer vision and deep learning technology. The specific implementation scheme is as follows: acquiring an original image, and taking the original image as a current image; if the current image meets the preset enhancement condition, selecting one renderer from a plurality of pre-trained renderers as the current renderer; inputting the current image into a current renderer, and outputting an image of the current image enhanced on the corresponding dimension of the current renderer through the current renderer; and taking the image of the current image enhanced on the corresponding dimension of the current renderer as the current image, and repeatedly executing the operation until the current image does not meet the enhancement condition. The embodiment of the application can adaptively enhance the image in a large range on multiple dimensions, is time-saving and labor-saving, has obvious effect and is suitable for various application scenes.

Description

Image enhancement method and device, electronic equipment and storage medium
Technical Field
The present disclosure relates to the field of artificial intelligence technologies, and further relates to computer vision and deep learning technologies, and in particular, to an image enhancement method and apparatus, an electronic device, and a storage medium.
Background
When a photo is taken or a video is recorded at ordinary times, the problems of overexposure/underexplosion, insufficient color brightness and the like exist in the shot photo or the recorded video due to light, shooting equipment and the like, and the following two methods are mainly used for solving the problem of poor color of the image/video at present: (1) the later manual adjustment is performed by a color matching operator, and the more common software is Photoshop, Lightrom and the like. The disadvantages of manual toning are evident: time and labor are wasted, the toning result is limited by the level of a toning technician, and mass rapid toning and satisfactory results are difficult to achieve. (2) At present, some algorithms can solve the problem of brightness enhancement of low-illumination images or the problem of hue and saturation enhancement, but the self-adaption enhancement of the images in a large range on multiple dimensions is difficult to achieve, for example, for dim-light images, few algorithms can process the images on the multiple dimensions, and the existing algorithms can only solve the problem under a specific scene.
Disclosure of Invention
The disclosure provides an image enhancement method, an image enhancement device, an electronic device and a storage medium.
In a first aspect, the present application provides a method of image enhancement, the method comprising:
acquiring an original image, and taking the original image as a current image;
if the current image meets the preset enhancement condition, selecting one renderer from a plurality of pre-trained renderers as the current renderer;
inputting the current image into the current renderer, and outputting an image of the current image enhanced on a corresponding dimension of the current renderer through the current renderer; and taking the image of the current image enhanced on the corresponding dimension of the current renderer as the current image, and repeatedly executing the operations until the current image does not meet the enhancement condition.
In a second aspect, the present application provides an image enhancement apparatus, the apparatus comprising: the device comprises an acquisition module, a selection module and an enhancement module; wherein the content of the first and second substances,
the acquisition module is used for acquiring an original image and taking the original image as a current image;
the selecting module is used for selecting one renderer from a plurality of pre-trained renderers as a current renderer if the current image meets a preset enhancing condition;
the enhancement module is used for inputting the current image into the current renderer and outputting an image of the current image enhanced on the corresponding dimension of the current renderer through the current renderer; and taking the image of the current image enhanced on the corresponding dimension of the current renderer as the current image, and repeatedly executing the operations until the current image does not meet the enhancement condition.
In a third aspect, an embodiment of the present application provides an electronic device, including:
one or more processors;
a memory for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the image enhancement method of any embodiment of the present application.
In a fourth aspect, the present application provides a storage medium, on which a computer program is stored, which when executed by a processor implements the image enhancement method according to any embodiment of the present application.
In a fifth aspect, a computer program product is provided, which when executed by a computer device implements the image enhancement method of any of the embodiments of the present application.
According to the technology of this application, it wastes time and energy to have solved the mode that adopts artifical mixing of colors among the prior art, and the result of transferring is subject to mixing of colors engineer's level, hardly accomplish big batch quick mixing of colors and the technical problem that the result is all satisfied, and current algorithm hardly accomplishes multidimension ground self-adaptation regulation on a large scale, the technical problem who only is applicable to specific scene, adopt the technical scheme that this application provided, can be on multidimension ground self-adaptation reinforcing image on a large scale, labour saving and time saving, the effect is obvious, be applicable to various application scenes.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
fig. 1 is a first flowchart of an image enhancement method according to an embodiment of the present disclosure;
fig. 2 is a second flowchart of an image enhancement method provided in an embodiment of the present application;
FIG. 3 is a schematic structural diagram of a prediction network provided in an embodiment of the present application;
fig. 4 is a schematic structural diagram of a rendering network provided in an embodiment of the present application;
fig. 5 is a schematic diagram of a third flow of an image enhancement method provided in an embodiment of the present application;
FIG. 6 is a schematic structural diagram of an image enhancement apparatus provided in an embodiment of the present application;
fig. 7 is a block diagram of an electronic device for implementing an image enhancement method according to an embodiment of the present application.
Detailed Description
Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Example one
Fig. 1 is a first flowchart of an image enhancement method provided in an embodiment of the present application, where the method may be performed by an image enhancement apparatus or an electronic device, where the apparatus or the electronic device may be implemented by software and/or hardware, and the apparatus or the electronic device may be integrated in any intelligent device with a network communication function. As shown in fig. 1, the image enhancement method may include the steps of:
s101, acquiring an original image, and taking the original image as a current image.
In this step, the electronic device may first obtain an original image; and then the original image is taken as the current image. The image in the embodiment of the present application may be a photo, or may be a frame in a video.
And S102, if the current image meets the preset enhancement condition, selecting one renderer from the plurality of pre-trained renderers as the current renderer.
In this step, if the current image meets the preset enhancement condition, the electronic device may select one renderer from the plurality of pre-trained renderers as the current renderer. The renderer in the embodiment of the present application may include, but is not limited to: an exposure renderer, a hue renderer, and a saturation renderer. Therefore, the three renderers need to be trained first. Taking an exposure renderer as an example, firstly, a batch of well-colored images are respectively adjusted to different values according to three dimensions through Lightroom software, then corresponding images are stored, for example, for a well-colored image I, image data is generated by rendering with software according to different dimensions, and each dimension generates plus and minus 50 images. Taking exposure as an example, assuming that the range of exposure is (-100, 100), then images of exposure-100, -98, -96, …, 96, 98, 100 are generated with 2 steps, respectively; an exposure renderer may be trained using the generated image as an output, the original image as an input, and the adjusted exposure as an input parameter.
S103, inputting the current image into the current renderer, and outputting an image of the current image enhanced on the corresponding dimension of the current renderer through the current renderer; and taking the image of the current image enhanced on the corresponding dimension of the current renderer as the current image, and repeatedly executing the operation until the current image does not meet the enhancement condition.
In this step, the electronic device may input the current image to the current renderer, and output an image in which the current image is enhanced in a dimension corresponding to the current renderer by the current renderer; and taking the image of the current image enhanced on the corresponding dimension of the current renderer as the current image, and repeatedly executing the operation until the current image does not meet the enhancement condition. Specifically, the electronic device may first input the current image to a first network and a second network in a current renderer; obtaining a rendering value corresponding to the current image in a current renderer through a first network based on the current image; and then transmitting the rendering value to a second network, and rendering the current image through the second network based on the rendering value to obtain an image of the current image enhanced on the corresponding dimension of the current renderer. The first network in the embodiment of the present application may be an Operator Module (Operator Module); the second Network may be a Render Network (Render Network). Alternatively, the prediction network may be common among a plurality of different renderers.
The image enhancement method provided by the embodiment of the application comprises the steps of firstly obtaining an original image and taking the original image as a current image; if the current image meets the preset enhancement condition, selecting one renderer from a plurality of pre-trained renderers as the current renderer; then inputting the current image into a current renderer, and outputting an image of the current image enhanced on the corresponding dimension of the current renderer through the current renderer; and taking the image of the current image enhanced on the corresponding dimension of the current renderer as the current image, and repeatedly executing the operation until the current image does not meet the enhancement condition. That is to say, the present application may adaptively select one renderer from a plurality of renderers, and use the renderer to render the current image in the corresponding dimension. In the existing image enhancement method, manual adjustment is performed later by a color matching operator, or some algorithms are adopted, so that the image enhancement method can only enhance the image in respective dimensionality and cannot process the image in multiple dimensionalities. Because the technical means of training a plurality of renderers in advance and enhancing the current image one by one through the plurality of renderers are adopted, the technical problems that the manual color mixing mode in the prior art is time-consuming and labor-consuming, the color mixing result is limited by the level of a color mixer, large-batch rapid color mixing is difficult to achieve, and the result is satisfactory are solved, and the technical problems that the multi-dimensional large-range self-adaptive adjustment is difficult to achieve by the conventional algorithm and only the method is suitable for a specific scene are solved; moreover, the technical scheme of the embodiment of the application is simple and convenient to implement, convenient to popularize and wide in application range.
Example two
Fig. 2 is a schematic flowchart of a second process of the image enhancement method according to the embodiment of the present application. Further optimization and expansion are performed based on the technical scheme, and the method can be combined with the various optional embodiments. As shown in fig. 2, the image enhancement method may include the steps of:
s201, acquiring an original image, and taking the original image as a current image.
S202, if the current image meets the preset enhancement condition, selecting one renderer from the plurality of pre-trained renderers as the current renderer.
S203, inputting the current image into a first network and a second network in the current renderer; and obtaining a rendering value corresponding to the current image in the current renderer through a first network based on the current image.
In this step, the electronic device may input the current image to the first network and the second network in the current renderer; and obtaining a rendering value corresponding to the current image in the current renderer through a first network based on the current image. Specifically, the electronic device may input the current image to a convolutional neural network, and perform feature extraction on the current image through the convolutional neural network to obtain a feature map corresponding to the current image; then, calculating the mean value, the variance and the maximum value of the feature map on each channel; and predicting a rendering value corresponding to the current image in the current renderer according to the mean value, the variance and the maximum value of the feature map on each channel. In order to increase the speed, the method can firstly carry out down-sampling on the current image, then carry out feature extraction on the down-sampled image by using a CNN network, then calculate the mean value, the variance and the maximum value of each channel of the feature map, then splice into a new feature full-connection layer to predict the rendering value (value) of the current renderer, then render the current image by using the value to obtain a new image, and can improve the color of the degraded image after three basic operations.
S204, transmitting the rendering value to a second network, and rendering the current image through the second network based on the rendering value to obtain an image of the current image enhanced in the corresponding dimension of the current renderer; and taking the image of the current image enhanced on the corresponding dimension of the current renderer as the current image, and repeatedly executing the operation until the current image does not meet the enhancement condition.
In this step, the electronic device may transmit the rendering value to the second network, and render the current image based on the rendering value through the second network to obtain an image in which the current image is enhanced in a dimension corresponding to the current renderer; and taking the image of the current image enhanced on the corresponding dimension of the current renderer as the current image, and repeatedly executing the operation until the current image does not meet the enhancement condition. Specifically, the electronic device may input the current image to a first rendering unit in the second network to obtain a rendering result output by the first rendering unit; then, the rendering result and the rendering value output by the first rendering unit are input to a second rendering unit in a second network to obtain the rendering result output by the second rendering unit, and the rendering result output by the second rendering unit is used as an image of the current image enhanced in the corresponding dimension of the current renderer; wherein the first rendering unit includes: two fully connected layers and an activation function; the second rendering unit includes: two fully connected layers and two activation functions.
Fig. 3 is a schematic structural diagram of a prediction network provided in an embodiment of the present application. As shown in fig. 3, the prediction network may include: the system comprises a convolutional neural network (CNN Feature Extractor), a mean calculation unit (mean), a variance calculation unit (std), a maximum calculation unit (max), a splicing unit (Concat) and a full connection layer (FC). Specifically, the current image is down-sampled, then the feature extraction is performed on the sampled image by using a convolutional neural network, then the mean Value, the variance and the maximum Value are calculated for each channel of the feature map, then a new rendering Value (Value) of the feature over-full-connected layer prediction current renderer is spliced, then the Value is used for rendering the current image to obtain a new image, and the color of the degraded image can be improved after three basic operations.
Fig. 4 is a schematic structural diagram of a rendering network provided in an embodiment of the present application. As shown in fig. 4, the rendering network may include: a first rendering unit and a second rendering unit; wherein the first rendering unit includes: two fully connected layers and an activation function; the second rendering unit includes: two fully connected layers and two activation functions. Specifically, the structure of the first rendering unit is: a full connectivity layer (FC), an activation function (Leaky ReLU), a full connectivity layer (FC); the structure of the second rendering unit is as follows: an activation function (Leaky ReLU), a full connectivity layer (FC), an activation function (Leaky ReLU), a full connectivity layer (FC).
The image enhancement method provided by the embodiment of the application comprises the steps of firstly obtaining an original image and taking the original image as a current image; if the current image meets the preset enhancement condition, selecting one renderer from a plurality of pre-trained renderers as the current renderer; then inputting the current image into a current renderer, and outputting an image of the current image enhanced on the corresponding dimension of the current renderer through the current renderer; and taking the image of the current image enhanced on the corresponding dimension of the current renderer as the current image, and repeatedly executing the operation until the current image does not meet the enhancement condition. That is to say, the present application may adaptively select one renderer from a plurality of renderers, and use the renderer to render the current image in the corresponding dimension. In the existing image enhancement method, manual adjustment is performed later by a color matching operator, or some algorithms are adopted, so that the image enhancement method can only enhance the image in respective dimensionality and cannot process the image in multiple dimensionalities. Because the technical means of training a plurality of renderers in advance and enhancing the current image one by one through the plurality of renderers are adopted, the technical problems that the manual color mixing mode in the prior art is time-consuming and labor-consuming, the color mixing result is limited by the level of a color mixer, large-batch rapid color mixing is difficult to achieve, and the result is satisfactory are solved, and the technical problems that the multi-dimensional large-range self-adaptive adjustment is difficult to achieve by the conventional algorithm and only the method is suitable for a specific scene are solved; moreover, the technical scheme of the embodiment of the application is simple and convenient to implement, convenient to popularize and wide in application range.
EXAMPLE III
Fig. 5 is a schematic diagram of a third flow of an image enhancement method according to an embodiment of the present application. Further optimization and expansion are performed based on the technical scheme, and the method can be combined with the various optional embodiments. As shown in fig. 5, the image enhancement method may include the steps of:
s501, acquiring an original image, and taking the original image as a current image.
S502, if the current image meets the preset enhancement condition, selecting one renderer from the plurality of pre-trained renderers as the current renderer.
S503, down-sampling the current image to obtain a down-sampled image.
In this step, the electronic device may perform downsampling on the current image to obtain a downsampled image. For example, assuming that the current image is a 1024 × 1024 image, a 256 × 256 image can be obtained after downsampling.
S504, the down-sampling image is used as a current image, the current image is input into a convolution neural network, and feature extraction is carried out on the current image through the convolution neural network to obtain a feature map corresponding to the current image.
And S505, calculating the mean value, the variance and the maximum value of the feature map on each channel.
S506, predicting a rendering value corresponding to the current image in the current renderer according to the mean value, the variance and the maximum value of the feature map on each channel.
And S507, transmitting the rendering value to a second network, and rendering the current image through the second network based on the rendering value to obtain an image of the current image enhanced in the corresponding dimension of the current renderer.
In the specific embodiment of the application, in order to increase the speed, the current image is downsampled, then the downsampled image is subjected to feature extraction by using a convolutional neural network, then the mean Value, the variance and the maximum Value of each channel of a feature map are solved, then a new feature full-connection layer is spliced to predict the rendering Value (Value) of the current renderer, then the current image is rendered by using the Value to obtain a new image, and the color of the degraded image can be improved after three basic operations.
And S508, taking the image of the current image enhanced on the corresponding dimension of the current renderer as the current image, and repeatedly executing the operation until the current image does not meet the enhancement condition.
The image enhancement method provided by the embodiment of the application comprises the steps of firstly obtaining an original image and taking the original image as a current image; if the current image meets the preset enhancement condition, selecting one renderer from a plurality of pre-trained renderers as the current renderer; then inputting the current image into a current renderer, and outputting an image of the current image enhanced on the corresponding dimension of the current renderer through the current renderer; and taking the image of the current image enhanced on the corresponding dimension of the current renderer as the current image, and repeatedly executing the operation until the current image does not meet the enhancement condition. That is to say, the present application may adaptively select one renderer from a plurality of renderers, and use the renderer to render the current image in the corresponding dimension. In the existing image enhancement method, manual adjustment is performed later by a color matching operator, or some algorithms are adopted, so that the image enhancement method can only enhance the image in respective dimensionality and cannot process the image in multiple dimensionalities. Because the technical means of training a plurality of renderers in advance and enhancing the current image one by one through the plurality of renderers are adopted, the technical problems that the manual color mixing mode in the prior art is time-consuming and labor-consuming, the color mixing result is limited by the level of a color mixer, large-batch rapid color mixing is difficult to achieve, and the result is satisfactory are solved, and the technical problems that the multi-dimensional large-range self-adaptive adjustment is difficult to achieve by the conventional algorithm and only the method is suitable for a specific scene are solved; moreover, the technical scheme of the embodiment of the application is simple and convenient to implement, convenient to popularize and wide in application range.
Example four
Fig. 6 is a schematic structural diagram of an image enhancement device according to an embodiment of the present application. As shown in fig. 6, the apparatus 600 includes: an acquisition module 601, a selection module 602 and an enhancement module 603; wherein the content of the first and second substances,
the obtaining module 601 is configured to obtain an original image, and use the original image as a current image;
the selecting module 602 is configured to select one renderer from the pre-trained plurality of renderers as a current renderer if the current image meets a preset enhancement condition;
the enhancing module 603 is configured to input the current image to the current renderer, and output, by the current renderer, an image in which the current image is enhanced in a dimension corresponding to the current renderer; and taking the image of the current image enhanced on the corresponding dimension of the current renderer as the current image, and repeatedly executing the operations until the current image does not meet the enhancement condition.
Further, the enhancement module 603 is specifically configured to input the current image to a first network and a second network in the current renderer; obtaining a rendering value corresponding to the current image in the current renderer through the first network based on the current image; and transmitting the rendering value to the second network, and rendering the current image through the second network based on the rendering value to obtain an image of the current image enhanced on the corresponding dimension of the current renderer.
Further, the enhancing module 603 is specifically configured to input the current image to a convolutional neural network, and perform feature extraction on the current image through the convolutional neural network to obtain a feature map corresponding to the current image; calculating the mean, variance and maximum value of the feature map on each channel; and predicting a rendering value corresponding to the current image in the current renderer based on the mean value, the variance and the maximum value of the feature map on each channel.
Further, the enhancing module 603 is further configured to perform downsampling on the current image to obtain a downsampled image; and taking the downsampled image as the current image, and executing the operation of inputting the current image into a convolutional neural network.
Further, the enhancing module 603 is specifically configured to input the current image to a first rendering unit in the second network, and obtain a rendering result output by the first rendering unit; inputting the rendering result and the rendering value output by the first rendering unit into a second rendering unit in the second network to obtain the rendering result output by the second rendering unit, and taking the rendering result output by the second rendering unit as the image of the current image enhanced in the dimension corresponding to the current renderer; wherein the first rendering unit includes: two fully connected layers and an activation function; the second rendering unit includes: two fully connected layers and two activation functions.
Further, the apparatus further comprises: a training module 604 (not shown in the figure), configured to extract an image from an image library corresponding to any one renderer as a current image if the any one renderer of the plurality of renderers does not satisfy a convergence condition corresponding to the renderer; training the any one renderer by using the current image until the any one renderer meets the convergence condition.
The image enhancement device can execute the method provided by any embodiment of the application, and has the corresponding functional modules and beneficial effects of the execution method. For details of the technique not described in detail in this embodiment, reference may be made to the image enhancement method provided in any embodiment of the present application.
EXAMPLE five
The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.
FIG. 7 illustrates a schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 7, the device 700 comprises a computing unit 701, which may perform various suitable actions and processes according to a computer program stored in a Read Only Memory (ROM)702 or a computer program loaded from a storage unit 708 into a Random Access Memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other by a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.
Various components in the device 700 are connected to the I/O interface 705, including: an input unit 706 such as a keyboard, a mouse, or the like; an output unit 707 such as various types of displays, speakers, and the like; a storage unit 708 such as a magnetic disk, optical disk, or the like; and a communication unit 709 such as a network card, modem, wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
Computing unit 701 may be a variety of general purpose and/or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The calculation unit 701 performs the respective methods and processes described above, such as the image enhancement method. For example, in some embodiments, the image enhancement method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of a computer program may be loaded onto and/or installed onto device 700 via ROM 702 and/or communications unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the image enhancement method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the image enhancement method by any other suitable means (e.g. by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), blockchain networks, and the internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service are overcome.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved. In the technical scheme of the disclosure, the acquisition, storage, application and the like of the personal information of the related user all accord with the regulations of related laws and regulations, and do not violate the good customs of the public order.
The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

Claims (15)

1. A method of image enhancement, the method comprising:
acquiring an original image, and taking the original image as a current image;
if the current image meets the preset enhancement condition, selecting one renderer from a plurality of pre-trained renderers as the current renderer;
inputting the current image into the current renderer, and outputting an image of the current image enhanced on a corresponding dimension of the current renderer through the current renderer; and taking the image of the current image enhanced on the corresponding dimension of the current renderer as the current image, and repeatedly executing the operations until the current image does not meet the enhancement condition.
2. The method of claim 1, wherein inputting the current image to the current renderer, outputting, by the current renderer, an image of the current image enhanced in a dimension corresponding to the current renderer, comprises:
inputting the current image to a first network and a second network in the current renderer; obtaining a rendering value corresponding to the current image in the current renderer through the first network based on the current image;
and transmitting the rendering value to the second network, and rendering the current image through the second network based on the rendering value to obtain an image of the current image enhanced on the corresponding dimension of the current renderer.
3. The method of claim 2, wherein obtaining, based on the current image, a corresponding rendering value of the current image in the current renderer through the first network comprises:
inputting the current image into a convolutional neural network, and performing feature extraction on the current image through the convolutional neural network to obtain a feature map corresponding to the current image;
calculating the mean, variance and maximum value of the feature map on each channel;
and predicting a rendering value corresponding to the current image in the current renderer based on the mean value, the variance and the maximum value of the feature map on each channel.
4. The method of claim 3, prior to inputting the current image to a convolutional neural network, the method further comprising:
down-sampling the current image to obtain a down-sampled image; and taking the downsampled image as the current image, and executing the operation of inputting the current image into a convolutional neural network.
5. The method of claim 2, wherein rendering the current image based on the rendering value over the second network to obtain an image in which the current image is enhanced in a dimension corresponding to the current renderer comprises:
inputting the current image to a first rendering unit in the second network to obtain a rendering result output by the first rendering unit;
inputting the rendering result and the rendering value output by the first rendering unit into a second rendering unit in the second network to obtain the rendering result output by the second rendering unit, and taking the rendering result output by the second rendering unit as the image of the current image enhanced in the dimension corresponding to the current renderer; wherein the first rendering unit includes: two fully connected layers and an activation function; the second rendering unit includes: two fully connected layers and two activation functions.
6. The method of claim 1, further comprising:
if any renderer of the plurality of renderers does not meet the corresponding convergence condition, extracting an image from an image library corresponding to the any renderer to serve as a current image; training the any one renderer by using the current image until the any one renderer meets the convergence condition.
7. An image enhancement device, the device comprising: the device comprises an acquisition module, a selection module and an enhancement module; wherein the content of the first and second substances,
the acquisition module is used for acquiring an original image and taking the original image as a current image;
the selecting module is used for selecting one renderer from a plurality of pre-trained renderers as a current renderer if the current image meets a preset enhancing condition;
the enhancement module is used for inputting the current image into the current renderer and outputting an image of the current image enhanced on the corresponding dimension of the current renderer through the current renderer; and taking the image of the current image enhanced on the corresponding dimension of the current renderer as the current image, and repeatedly executing the operations until the current image does not meet the enhancement condition.
8. The apparatus of claim 7, the enhancement module to input the current image to a first network and a second network in the current renderer; obtaining a rendering value corresponding to the current image in the current renderer through the first network based on the current image; and transmitting the rendering value to the second network, and rendering the current image through the second network based on the rendering value to obtain an image of the current image enhanced on the corresponding dimension of the current renderer.
9. The apparatus according to claim 8, wherein the enhancement module is specifically configured to input the current image into a convolutional neural network, and perform feature extraction on the current image through the convolutional neural network to obtain a feature map corresponding to the current image; calculating the mean, variance and maximum value of the feature map on each channel; and predicting a rendering value corresponding to the current image in the current renderer based on the mean value, the variance and the maximum value of the feature map on each channel.
10. The apparatus of claim 9, the enhancement module further configured to down-sample the current image to obtain a down-sampled image; and taking the downsampled image as the current image, and executing the operation of inputting the current image into a convolutional neural network.
11. The apparatus according to claim 8, wherein the enhancement module is specifically configured to input the current image to a first rendering unit in the second network, and obtain a rendering result output by the first rendering unit; inputting the rendering result and the rendering value output by the first rendering unit into a second rendering unit in the second network to obtain the rendering result output by the second rendering unit, and taking the rendering result output by the second rendering unit as the image of the current image enhanced in the dimension corresponding to the current renderer; wherein the first rendering unit includes: two fully connected layers and an activation function; the second rendering unit includes: two fully connected layers and two activation functions.
12. The apparatus of claim 7, further comprising: the training module is used for extracting an image from an image library corresponding to any renderer to serve as a current image if any renderer in the renderers does not meet the corresponding convergence condition; training the any one renderer by using the current image until the any one renderer meets the convergence condition.
13. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.
14. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-6.
15. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-6.
CN202210060864.7A 2022-01-19 2022-01-19 Image enhancement method and device, electronic equipment and storage medium Active CN114418887B (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
CN202210060864.7A CN114418887B (en) 2022-01-19 2022-01-19 Image enhancement method and device, electronic equipment and storage medium
KR1020230005149A KR20230112063A (en) 2022-01-19 2023-01-13 Image enhancement method and device, electronic equipment and storage medium
JP2023003837A JP2023105807A (en) 2022-01-19 2023-01-13 Image enhancement method, device, electronic device, and storage medium
US18/155,901 US20230230205A1 (en) 2022-01-19 2023-01-18 Image enhancement method and apparatus, electronic device, and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210060864.7A CN114418887B (en) 2022-01-19 2022-01-19 Image enhancement method and device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN114418887A true CN114418887A (en) 2022-04-29
CN114418887B CN114418887B (en) 2022-12-20

Family

ID=81274574

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210060864.7A Active CN114418887B (en) 2022-01-19 2022-01-19 Image enhancement method and device, electronic equipment and storage medium

Country Status (4)

Country Link
US (1) US20230230205A1 (en)
JP (1) JP2023105807A (en)
KR (1) KR20230112063A (en)
CN (1) CN114418887B (en)

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102427547A (en) * 2011-11-15 2012-04-25 清华大学 Multi-angle stereo rendering apparatus
US20130318206A1 (en) * 2012-05-28 2013-11-28 Gemtek Technology Co., Ltd. Render, controller and managing methods thereof
CN103700385A (en) * 2012-09-27 2014-04-02 深圳市快播科技有限公司 Media player, playing method, and video post-processing method in hardware acceleration mode
CN103810671A (en) * 2012-11-15 2014-05-21 腾讯科技（深圳）有限公司 Method and system for color stretching of RGB mode images
CN107845074A (en) * 2017-10-24 2018-03-27 维沃移动通信有限公司 A kind of image rendering method and mobile terminal
CN109218820A (en) * 2018-11-14 2019-01-15 广州市百果园信息技术有限公司 A kind of video renderer and Video Rendering method
US20190188535A1 (en) * 2017-12-15 2019-06-20 Google Llc Machine-Learning Based Technique for Fast Image Enhancement
CN113453073A (en) * 2021-06-29 2021-09-28 北京百度网讯科技有限公司 Image rendering method and device, electronic equipment and storage medium
US11170476B1 (en) * 2020-10-15 2021-11-09 Aeva, Inc. Techniques for fast point cloud filtering using a series cascaded filter

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102427547A (en) * 2011-11-15 2012-04-25 清华大学 Multi-angle stereo rendering apparatus
US20130318206A1 (en) * 2012-05-28 2013-11-28 Gemtek Technology Co., Ltd. Render, controller and managing methods thereof
CN103700385A (en) * 2012-09-27 2014-04-02 深圳市快播科技有限公司 Media player, playing method, and video post-processing method in hardware acceleration mode
CN103810671A (en) * 2012-11-15 2014-05-21 腾讯科技（深圳）有限公司 Method and system for color stretching of RGB mode images
CN107845074A (en) * 2017-10-24 2018-03-27 维沃移动通信有限公司 A kind of image rendering method and mobile terminal
US20190188535A1 (en) * 2017-12-15 2019-06-20 Google Llc Machine-Learning Based Technique for Fast Image Enhancement
CN109218820A (en) * 2018-11-14 2019-01-15 广州市百果园信息技术有限公司 A kind of video renderer and Video Rendering method
US11170476B1 (en) * 2020-10-15 2021-11-09 Aeva, Inc. Techniques for fast point cloud filtering using a series cascaded filter
CN113453073A (en) * 2021-06-29 2021-09-28 北京百度网讯科技有限公司 Image rendering method and device, electronic equipment and storage medium

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
MINUI HONG 等: "StyleMix: Separating Content and Style for Enhanced Data Augmentation", 《IEEE》 *
季健等: "面向移动设备的可编程像素渲染器的设计", 《计算机工程与应用》 *
高瞻等: "渲染器与Web服务器耦合实现远程体渲染的交互优化", 《中国图象图形学报》 *

Also Published As

Publication number Publication date
JP2023105807A (en) 2023-07-31
KR20230112063A (en) 2023-07-26
US20230230205A1 (en) 2023-07-20
CN114418887B (en) 2022-12-20

Similar Documents

Publication Publication Date Title
CN112862877B (en) Method and apparatus for training an image processing network and image processing
CN113436105A (en) Model training and image optimization method and device, electronic equipment and storage medium
CN114792355A (en) Virtual image generation method and device, electronic equipment and storage medium
CN114120413A (en) Model training method, image synthesis method, device, equipment and program product
CN114418887B (en) Image enhancement method and device, electronic equipment and storage medium
CN115759209B (en) Quantification method and device of neural network model, electronic equipment and medium
CN116402914A (en) Method, device and product for determining stylized image generation model
CN116363429A (en) Training method of image recognition model, image recognition method, device and equipment
CN114882313B (en) Method, device, electronic equipment and storage medium for generating image annotation information
CN112927319B (en) Model training method, image processing method, device, equipment and storage medium
CN115660991A (en) Model training method, image exposure correction method, device, equipment and medium
CN115019057A (en) Image feature extraction model determining method and device and image identification method and device
CN114494782A (en) Image processing method, model training method, related device and electronic equipment
CN114445898A (en) Face living body detection method, device, equipment, storage medium and program product
CN113344213A (en) Knowledge distillation method, knowledge distillation device, electronic equipment and computer readable storage medium
CN113554550A (en) Training method and device of image processing model, electronic equipment and storage medium
CN113379750A (en) Semi-supervised learning method of semantic segmentation model, related device and product
CN113542620B (en) Special effect processing method and device and electronic equipment
CN113283305B (en) Face recognition method, device, electronic equipment and computer readable storage medium
CN114219744B (en) Image generation method, device, equipment and storage medium
CN113344199B (en) Method for training separable convolutional network, road side equipment and cloud control platform
CN113362218B (en) Data processing method and device, electronic equipment and storage medium
CN112991206B (en) Image processing method, device, equipment and storage medium
CN114693950B (en) Training method and device of image feature extraction network and electronic equipment
CN116363260B (en) Image generation method and device and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant