CN115439367A - Image enhancement method and device, electronic equipment and storage medium - Google Patents
Image enhancement method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115439367A CN115439367A CN202211154177.8A CN202211154177A CN115439367A CN 115439367 A CN115439367 A CN 115439367A CN 202211154177 A CN202211154177 A CN 202211154177A CN 115439367 A CN115439367 A CN 115439367A
- Authority
- CN
- China
- Prior art keywords
- image
- vector
- sample
- hidden
- label
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 75
- 239000013598 vector Substances 0.000 claims abstract description 204
- 238000003062 neural network model Methods 0.000 claims abstract description 46
- 230000002708 enhancing effect Effects 0.000 claims abstract description 10
- 230000004048 modification Effects 0.000 claims description 93
- 238000012986 modification Methods 0.000 claims description 93
- 238000000605 extraction Methods 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 7
- 238000012549 training Methods 0.000 description 35
- 230000008569 process Effects 0.000 description 19
- 230000009466 transformation Effects 0.000 description 13
- 230000006870 function Effects 0.000 description 11
- 238000010586 diagram Methods 0.000 description 10
- 238000009826 distribution Methods 0.000 description 7
- 230000001965 increasing effect Effects 0.000 description 6
- 241000282326 Felis catus Species 0.000 description 5
- 238000013528 artificial neural network Methods 0.000 description 5
- 238000013135 deep learning Methods 0.000 description 5
- 238000013527 convolutional neural network Methods 0.000 description 4
- 239000011159 matrix material Substances 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000013145 classification model Methods 0.000 description 2
- 230000003247 decreasing effect Effects 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 241000124008 Mammalia Species 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000036961 partial effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000002829 reductive effect Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000013179 statistical model Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The application provides an image enhancement method, an image enhancement device, electronic equipment and a storage medium, wherein the method comprises the following steps: extracting the characteristics of an original image by using a neural network model to obtain a hidden characteristic vector; modifying the hidden feature vector to obtain a modified hidden vector; and carrying out image enhancement on the original image according to the modified hidden vector to obtain an enhanced image corresponding to the original image. The method comprises the steps of extracting features of an original image by using a neural network model, modifying an obtained hidden feature vector to obtain a modified hidden vector, and then enhancing the image of the original image according to the modified hidden vector.
Description
Technical Field
The present application relates to the technical field of computer vision, image processing, and deep learning, and in particular, to an image enhancement method and apparatus, an electronic device, and a storage medium.
Background
The image enhancement is to obtain a network with stronger generalization capability by transforming the training picture in order to reduce the overfitting phenomenon of the network, so as to better adapt to an application scene, emphasize some concerned features and inhibit non-concerned features.
At present, in order to increase the training data set of the neural network model, the image enhancement of the original image is generally performed in a random manner, which includes: and performing image enhancement in random modes such as flip transformation (flip), zoom transformation (zoom), shift transformation (shift), scale transformation (scale), contrast transformation (contrast), noise (noise) addition and random cropping. However, in the specific implementation process, it is found that after the original image is subjected to image enhancement in a random manner and the obtained enhanced image is used to train the model, the generalization performance of the models is not obviously improved. Therefore, it is difficult for the enhanced image obtained by enhancing the image in a random manner to improve the generalization performance of the model.
Disclosure of Invention
An embodiment of the present application aims to provide an image enhancement method, an image enhancement device, an electronic device, and a storage medium, which are used for solving the problem that it is difficult for an enhanced image obtained by image enhancement to improve the model generalization performance.
The embodiment of the application provides an image enhancement method, which comprises the following steps: carrying out feature extraction on an original image by using a neural network model to obtain a hidden feature vector; modifying the hidden feature vector to obtain a modified hidden vector; and carrying out image enhancement on the original image according to the modified hidden vector to obtain an enhanced image corresponding to the original image. In the implementation process of the scheme, the original image is subjected to feature extraction by using the neural network model, the obtained hidden feature vector is modified to obtain a modified hidden vector, and then the original image is subjected to image enhancement according to the modified hidden vector.
Optionally, in an embodiment of the present application, the neural network model is a self-coding network; performing feature extraction on the original image by using a neural network model, wherein the feature extraction comprises the following steps: performing feature extraction on an original image by using an encoder in a self-coding network; and performing image enhancement on the original image according to the modified hidden vector, wherein the image enhancement comprises the following steps: and performing image enhancement on the original image according to the modified hidden vector by using a generator in the self-coding network. In the implementation process of the scheme, the encoder and the generator in the self-coding network are adopted to perform image enhancement on the original image, so that the encoder and the generator in the self-coding network can better generate an enhanced image for improving the model generalization performance, and therefore, the obtained enhanced image can be used for improving the model generalization performance, and the model generalization performance is effectively improved.
Optionally, in an embodiment of the present application, the neural network model is a hyper-network; and performing image enhancement on the original image according to the modified hidden vector, wherein the image enhancement comprises the following steps: modifying the network parameters of the hyper-network by using the modified hidden vector to obtain a modified hyper-network; and carrying out image enhancement on the original image through the modified hyper-network. In the implementation process of the scheme, the modified hyper-network is used for carrying out image enhancement on the original image, so that the modified hyper-network can better generate an enhanced image for improving the model generalization performance, therefore, the obtained enhanced image can be used for improving the model generalization performance, and the model generalization performance is effectively improved.
Optionally, in this embodiment of the present application, modifying the hidden feature vector includes: and acquiring a modification vector, and adding the modification vector on the basis of the hidden feature vector. In the implementation process of the scheme, the modification vector is obtained, the modification vector is added on the basis of the hidden feature vector, and the obtained modified hidden vector is used for guiding and generating the enhanced image capable of improving the model generalization performance, so that the model generalization performance can be improved by using the obtained enhanced image, and the model generalization performance is effectively improved.
Optionally, in this embodiment of the present application, obtaining the modification vector includes: obtaining a sample image and a sample label, wherein the sample label is an enhanced image obtained by enhancing the sample image; carrying out image enhancement on the sample image by using a neural network model to obtain a prediction enhanced image; a modification vector is determined from the image pixel loss values between the prediction enhanced image and the sample labels. In the implementation process of the scheme, the modification vector is determined according to the image pixel loss value between the prediction enhanced image and the sample label, so that the problem that the modification vector is difficult to determine is avoided, and the accuracy rate of determining the modification vector is effectively improved.
Optionally, in this embodiment of the present application, obtaining the modification vector includes: obtaining a sample image and a sample label, wherein the sample label is a category label to which the sample image belongs; performing category prediction on the sample image to obtain a predicted image category; and determining a modification vector according to the classification loss value between the prediction image class and the class label. In the implementation process of the scheme, the modification vector is determined according to the classification loss value between the predicted image class and the class label, so that the problem that the modification vector is difficult to determine is avoided, and the accuracy of determining the modification vector is effectively improved.
Optionally, in this embodiment of the present application, obtaining the modification vector includes: obtaining a specimen image and a specimen label, the specimen label comprising: the enhanced image and the category label corresponding to the sample image; carrying out image enhancement on the sample image by using a neural network model to obtain a prediction enhanced image, and carrying out category prediction on the sample image to obtain a predicted image category; calculating an image pixel loss value between the prediction enhanced image and the sample label, and calculating a classification loss value between the prediction image class and the class label; a modification vector is determined based on the image pixel loss value and the classification loss value. In the implementation process of the scheme, the modification vector is determined according to the image pixel loss value and the classification loss value, so that the problem that the modification vector is difficult to determine is avoided, and the accuracy of determining the modification vector is effectively improved.
An embodiment of the present application further provides an image enhancement apparatus, including: the characteristic vector obtaining module is used for extracting the characteristics of the original image by using the neural network model to obtain a hidden characteristic vector; the characteristic vector modification module is used for modifying the hidden characteristic vector to obtain a modified hidden vector; and the enhanced image obtaining module is used for carrying out image enhancement on the original image according to the modified hidden vector to obtain an enhanced image corresponding to the original image.
Optionally, in an embodiment of the present application, the neural network model is a self-coding network; a feature vector obtaining module comprising: the image feature extraction submodule is used for extracting features of the original image by using an encoder in a self-coding network; an enhanced image acquisition module comprising: and the original image enhancement submodule is used for enhancing the image of the original image according to the modified hidden vector by using a generator in the self-coding network.
Optionally, in an embodiment of the present application, the neural network model is a hyper-network; an enhanced image acquisition module comprising: the network parameter modification submodule is used for modifying the network parameters of the super network by using the modified hidden vector to obtain a modified super network; and the network image enhancement sub-module is used for carrying out image enhancement on the original image through the modified hyper-network.
Optionally, in an embodiment of the present application, the feature vector modification module includes: and the vector increasing and decreasing modification submodule is used for acquiring the modification vector and increasing the modification vector on the basis of the hidden feature vector.
Optionally, in this embodiment of the present application, the vector increase/decrease modification sub-module includes: a first data acquisition unit configured to acquire a sample image and a sample label, the sample label being an enhanced image obtained by image enhancement of the sample image; the enhanced image prediction unit is used for carrying out image enhancement on the sample image by using the neural network model to obtain a prediction enhanced image; a first vector determination unit for determining a modification vector based on image pixel loss values between the prediction enhanced image and the sample labels.
Optionally, in this embodiment of the present application, the vector increase/decrease modification sub-module includes: the second data acquisition unit is used for acquiring a sample image and a sample label, wherein the sample label is a category label to which the sample image belongs; the image type prediction unit is used for performing type prediction on the sample image to obtain a predicted image type; and the second vector determining unit is used for determining the modification vector according to the classification loss value between the prediction image class and the class label.
Optionally, in this embodiment of the present application, the vector increase/decrease modification sub-module includes: a third data acquisition unit for acquiring a specimen image and a specimen label, the specimen label including: the enhanced image and the category label corresponding to the sample image; the data type prediction unit is used for carrying out image enhancement on the sample image by using the neural network model to obtain a prediction enhanced image and carrying out type prediction on the sample image to obtain a predicted image type; the image loss calculation unit is used for calculating an image pixel loss value between the prediction enhanced image and the sample label and calculating a classification loss value between the prediction image class and the class label; a third vector determination unit for determining a modification vector based on the image pixel loss value and the classification loss value.
An embodiment of the present application further provides an electronic device, including: a processor and a memory, the memory storing processor-executable machine-readable instructions, the machine-readable instructions when executed by the processor performing the method as described above.
Embodiments of the present application also provide a computer-readable storage medium having a computer program stored thereon, where the computer program is executed by a processor to perform the method as described above.
Additional features and advantages of embodiments of the present application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of embodiments of the present application.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic flow chart of an image enhancement method provided by an embodiment of the present application;
FIG. 2 is a schematic diagram illustrating image enhancement using a self-coding network provided by an embodiment of the present application;
FIG. 3 is a schematic diagram illustrating image enhancement using a hyper-network according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of an image enhancement device provided in an embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device provided in an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, as presented in the figures, is not intended to limit the scope of the embodiments of the present application, as claimed, but is merely representative of selected embodiments of the present application. All other embodiments obtained by a person skilled in the art based on the embodiments of the present application without any creative effort belong to the protection scope of the embodiments of the present application.
It is to be understood that "first" and "second" in the embodiments of the present application are used to distinguish similar objects. Those skilled in the art will appreciate that the terms "first," "second," etc. do not denote any order or quantity, nor do the terms "first," "second," etc. denote any order or importance.
Before describing the image enhancement method provided by the embodiment of the present application, some concepts related to the embodiment of the present application are described:
deep Learning (Deep Learning) is an algorithm for characterizing and Learning data in machine Learning, is a branch of machine Learning, and is also an algorithm for characterizing and Learning data by taking an artificial neural network as an architecture.
A Convolutional Neural Network (CNN), which is an artificial Neural network, in which artificial neurons of the artificial Neural network can respond to peripheral units and perform large-scale image processing; the convolutional neural network may include convolutional and pooling layers.
KL divergence (KLD), which is called relative entropy in information systems, randomness in continuous time sequences, information gain in statistical model inference, and information divergence (asymmetry), is a measure of the asymmetry of the difference between two probability distributions P and Q.
It should be noted that the image enhancement method provided in the embodiment of the present application may be executed by an electronic device, where the electronic device refers to a device terminal or a server having a function of executing a computer program, and the device terminal includes, for example: a smart phone, a personal computer, a tablet computer, a personal digital assistant, or a mobile internet device, etc. A server refers to a device that provides computing services over a network, such as: the server system comprises an x86 server and a non-x 86 server, wherein the non-x 86 server comprises: mainframe, minicomputer, and UNIX server.
Application scenarios to which the image enhancement method is applicable are described below, where the application scenarios include, but are not limited to: the method comprises the following steps of image compression (namely, an original image is an uncompressed image, the original image is enhanced by using the image enhancement method, and an obtained enhanced image is an image with compressed pixels), data enhancement of super-resolution reconstruction deep learning, image classification and other application scenes. The application scenarios of super-resolution reconstruction here are specifically as follows: the image enhancement method is used for extracting and modifying the hidden feature vector from the original image, the original image is enhanced according to the modified hidden feature vector, the resolution of the obtained enhanced image is larger than that of the original image, and after the enhanced image with higher resolution is obtained, the enhanced image can be used in an application scene of image classification. The application scenario of the data enhancement of deep learning here is specifically, for example: the training data set comprises a class A images and a class B images, and if the number of the class A images is less than that of the class B images, the class A images can be subjected to image enhancement by using the image enhancement method, so that more class A images are obtained, and the number of the class A images is approximately equal to that of the class B images.
Please refer to fig. 1, which illustrates a flowchart of an image enhancement method provided in an embodiment of the present application; the embodiment of the application provides an image enhancement method, which comprises the following steps:
step S110: and (4) carrying out feature extraction on the original image by using a neural network model to obtain a hidden feature vector.
A Neural Network Model (Neural Network Model) refers to a Neural Network Model obtained by training an untrained Neural Network by using preset training data, and particularly relates to a Network Model for extracting features of an original image. The neural network model herein may specifically adopt a self-encoding network (Auto-Encoder Networks) or a Hyper network (Hyper Networks), etc.
The original image is an image that needs image enhancement, and the original image may be an RGB color image (i.e., a three-dimensional matrix formed by three channels of RGB), or may be a black-and-white image. If the original image is a three-dimensional matrix of an RGB color image, the enhanced image obtained after image enhancement is a new three-dimensional matrix different from the original image.
Latent Feature Vector (LFV), also known as Latent Space Representation (LSR) or Latent Space Vector (LSR Vector), refers to a Latent Space Vector that is extracted from an image by a neural network model.
Step S120: and modifying the hidden feature vector to obtain a modified hidden vector.
Since there are many ways to modify the hidden feature vector in step S120, the ways to modify the hidden feature vector will be described in detail below, and will not be described herein again.
Step S130: and carrying out image enhancement on the original image according to the modified hidden vector to obtain an enhanced image corresponding to the original image.
Since there are many image enhancement methods in step S130, the image enhancement methods will be described in detail below, and will not be described herein again.
In the implementation process, the original image is subjected to feature extraction by using the neural network model, the obtained hidden feature vector is modified to obtain a modified hidden vector, and then the original image is subjected to image enhancement according to the modified hidden vector.
It will be appreciated that when the image is enhanced using the image enhancement method described above, it may be for all images in one data set to be enhanced. That is, for an image data set, the image enhancement method may be used to perform image enhancement on all original images (that is, perform image enhancement according to the modified hidden feature vector), and add the enhanced images into a training data set of the neural network model (the training data set includes a sample image and a sample label), thereby completing an expansion function on the image training data. The image enhancement method provided by the embodiment of the application is different from the traditional data enhancement method based on image transformation or noise introduction such as mirror image transformation, inversion transformation and the like, and the image enhancement method can be understood as that the original characteristics of parts in the original image which are irrelevant to classification are replaced according to the modified hidden vector, so that the newly generated enhanced image can not be restored to the original image by using a matrix transformation mode, and the diversity of the image is enriched from another dimension. In addition, the image enhancement method provided by the embodiment of the application can be used in combination with an image-based transformation enhancement method such as mirror transformation and inverse transformation, namely, a part of the image in the data set is subjected to image enhancement by using the image enhancement method, and another part of the image in the data set is subjected to image enhancement by using the image-based transformation enhancement method, so that the multiple of data expansion is effectively increased.
In addition, the data expansion function of the image data set may be completed by performing enhancement on a specific image (for example, a few sample images with a certain type of features) in one data set, that is, screening one image from the image data set to determine the image as an original image, enhancing the original image by using the image enhancement method, and adding an enhanced image obtained by enhancement to the image data set. When the image dataset has commonality and is used as training data of a neural network, image enhancement can be performed on the few sample images of a certain class of features in the image dataset, so that the number of the few sample images is increased, and the problem of data category imbalance caused by too few sample images of a certain class of features in the image dataset is solved. When the image enhancement is performed on the few-sample image with a certain type of features in the image data set, because the neural network model (such as a self-coding network or a super network) updates the network parameters of the neural network model according to the type of features in the training process of extracting the type of features from the neural network model, the updated network parameters can be directly used when the feature extraction (and the subsequent image enhancement) is performed on the few-sample image with the same type of features, and therefore, the same type of features of the few-sample image are shared through the network parameters in the model training process and the image enhancement process of the neural network model, so that the calculation cost of performing the data enhancement on the few-sample image with the same type of features is reduced, and the performance of performing the image enhancement by using the neural network model is improved.
Examples of enhancing a specific image in the image data set include: assuming that an image data set includes images of a cat and images of a dog, the image data set has a commonality that both images are images of a mammal, when the number of images of the cat is less than that of the dog, the images of the cat can be understood as low-sample images, and the low-sample images can be enhanced by using the image enhancement method to enhance the number of images of the cat, so as to solve the problem that the number of images of the cat in the image data set is too low to cause data category imbalance. Another example is: if an image data set comprises face images of men and face images of women, the common property of the image data set is that the face images are face images, when the number of the face images of women is less than that of the face images of men, the face image of women can be understood as a low sample image, and the low sample image can be enhanced by using the image enhancement method to enhance the number of the face images of women, so that the problem that the data types are unbalanced due to too low number of the face images of women in the image data set is solved.
Please refer to fig. 2, which is a schematic diagram illustrating image enhancement using a self-coding network according to an embodiment of the present application; as a first optional implementation of the image enhancement method, the neural network model may be a self-encoding network (Auto-Encoder Networks), and the self-encoding network may include: an Encoder (Encoder) and a generator (Decoder); the implementation of the step S110 may include:
step S111: and performing feature extraction on the original image by using an encoder in a self-coding network to obtain a hidden feature vector.
An Encoder (Encoder) refers to an Encoder in a self-encoding network, and may specifically adopt various structures, for example, a linear layer (also referred to as a full connection layer) and an activation function layer may be adopted to form the Encoder, a CNN network (for example, res net22, res net38, res net50, res net101, and res net152, etc.) may also be adopted, and a transform-based network (for example, viT, etc.) may also be adopted.
Specific examples of the implementation of step S111 include: it is assumed that a collection of original images can be represented as D imgs ＝{x 1 ,x 2 ,…,x n Then can beSet D of original images using an Encoder (Encoder) in a self-encoding network imgs ＝{x 1 ,x 2 ,…,x n Performing feature extraction, and representing a set of obtained hidden feature vectors as H = { H = } 1 ,h 2 ,…,h n }. For each original image in the original image set, the above-mentioned formula for each original image processing procedure may be represented as h = E (x), where h represents a hidden feature vector, E represents an Encoder (Encoder) in a self-encoding network, and x represents an original image.
As a first optional implementation of step S120, the process of modifying the hidden feature vector may include:
step S121: and acquiring a modification vector, and adding the modification vector on the basis of the hidden feature vector.
There are many embodiments for obtaining the modification vector in step S121, and these three embodiments will be exemplified in detail below: in a first alternative embodiment, a modification vector is obtained according to an image pixel loss value; in a second alternative embodiment, a modification vector is obtained according to the classification loss value; in a third alternative embodiment, the modification vector is obtained based on the image pixel loss value and the classification loss value. Since these three embodiments will be exemplified in detail below, they will not be described in detail.
The embodiment of adding the modification vector on the basis of the hidden feature vector in step S121 is, for example: using the formula h a ＝h+h aug Adding a modification vector on the basis of the hidden feature vector to obtain a modified hidden vector; wherein h is a Representing the hidden vector after modification, h representing the hidden feature vector before modification, h aug Representing a modification vector.
As a first optional implementation of the step S130, the process of image enhancement may include:
step S131: and performing image enhancement on the original image by using a generator in the self-coding network according to the modified hidden vector to obtain an enhanced image corresponding to the original image.
Image enhancement using self-coding network in the above step S131For example: using a generator (Decoder) in a self-coding network to perform image enhancement on an original image according to the modified hidden vector, wherein the image enhancement process can be expressed as x by using a formula r ＝D(h a |θ D ) Wherein x is r Representing the enhanced image obtained, and the resolution of the enhanced image may be greater than or equal to the resolution, θ, of the original image D The network parameters of the self-coding network are represented, and the self-coding network can update the network parameters of the self-coding network in the training process of extracting a certain class of features, so that the updated network parameters can be directly used when the feature extraction (and the subsequent image enhancement) is carried out on the images with the same class of features, and the network parameters of the self-coding network can be shared in the set of original images (i.e. the image data set for training other models), h a Represents the modified hidden vector and D represents the generator (Decoder) in the self-coding network.
Please refer to fig. 3, which is a schematic diagram illustrating image enhancement using a super network according to an embodiment of the present application; as a second optional implementation manner of the image enhancement method, the neural network model may also be a Hyper network (Hyper Networks), that is, a Hyper network may be used to extract the hidden feature vector of the original image, add a modification vector on the basis of the hidden feature vector to obtain a modified hidden vector, and input the modified hidden vector into the Hyper network, so that the Hyper network generates an enhanced image according to unknown information (i.e., information represented by the modified hidden vector) of the original image. Specifically, the image enhancement in step S130 may include:
step S132: and modifying the network parameters of the hyper-network by using the modified hidden vector to obtain the modified hyper-network.
Step S133: and carrying out image enhancement on the original image through the modified hyper-network to obtain an enhanced image corresponding to the original image.
The embodiment of the above steps S132 to S133 is, for example: assuming that the original image is a 600 × 800 pixel image, extracting coordinate information of pixel points of the original image, inputting the coordinate information of the pixel points of the original image into the super network, extracting a hidden feature vector of the original image by using the super network, then obtaining a modification vector, adding the modification vector on the basis of the hidden feature vector, obtaining a modified hidden vector, and inputting the modified hidden vector into the super network. After the super network receives the coordinate information of the pixel point of the original image and the modified hidden vector, the modified hidden vector can be used for modifying the network parameter of the super network to obtain the modified super network, so that the original image is subjected to image enhancement through the modified super network, that is, the modified super network can predict the RGB three-channel pixel value corresponding to the coordinate information according to the coordinate information of the pixel point of the original image and the modified hidden vector, and finally, the enhanced image is restored according to the coordinate information and the RGB three-channel pixel value corresponding to the coordinate information.
It will be appreciated that the above-described process of image enhancement using a hyper-network may be formulated as x r ＝D(c；h a |θ D ) (ii) a Wherein x is r Representing the enhanced image obtained, and the resolution of the enhanced image may be greater than or equal to the resolution, θ, of the original image D The network parameters of the super network are represented, and because the super network can update the network parameters of the super network in the training process of extracting a certain class of features, the updated network parameters can be directly used when the feature extraction (and the subsequent image enhancement) is carried out on the images with the same class of features, so the network parameters of the super network can be shared in the set of original images (namely, the image data set for training other models), h a Representing the modified hidden vector, c representing the coordinate information of the pixel point of the original image, and D representing a generator (Decoder) in the self-coding network.
As a first alternative implementation of step S121, the manner of obtaining the modification vector according to the image pixel loss value may include:
step S121a: and acquiring a sample image and a sample label, wherein the sample label is an enhanced image obtained by enhancing the sample image.
The sample image and the sample label may be obtained separately, for example: manually collecting a sample image, manually enhancing the sample image, and marking the obtained enhanced image as a sample label; of course, the sample image and the sample label may also be acquired together packed into the first training data set.
The obtaining manner of the first training data set in step S121a includes: the first acquisition mode is that a target object is shot by using terminal equipment such as a video camera, a video recorder or a color camera, a sample image is manually collected and is manually enhanced, the obtained enhanced image is marked as a sample label, and a first training data set is obtained; then the terminal device sends a first training data set to the electronic device, then the electronic device receives the first training data set sent by the terminal device, and the electronic device can store the first training data set into a file system, a database or mobile storage equipment; a second obtaining manner, obtaining a first training data set stored in advance, specifically for example: acquiring a first training data set from a file system, a database or a mobile storage device; in a third obtaining mode, software such as a browser is used for obtaining the first training data set on the internet, or other application programs are used for accessing the internet to obtain the first training data set.
Step S121b: and performing image enhancement on the sample image by using the neural network model to obtain a predicted enhanced image.
Step S121c: a modification vector is determined from the image pixel loss values between the prediction enhanced image and the sample labels.
The embodiments of the above steps S121b to S121c are, for example: carrying out image enhancement on the sample image by using a neural network model to obtain a prediction enhanced image; according to KL divergence formula L (x, x) r )＝||x-x r || 2 Calculating an image pixel loss value between the enhanced image and the sample label, and determining a modification vector according to the image pixel loss value; where x denotes the original image, x r Representing the enhanced image obtained, L (x, x) r ) Representing the KL divergence between the original image and the enhanced image. The KL powderThe degree is a measure of the number of additional average bits required to encode samples of the probability distribution subject to P using the Q-based probability distribution; typically, P represents the true distribution of the data, and Q represents the theoretical distribution of the data, an estimated model distribution, or an approximate distribution of P. Finally, the above specific way of determining the modification vector according to the loss value of the image pixel is as follows: randomly initializing modification vector, using formula
Calculating the initialized modification vector and an image pixel loss value to obtain a final modification vector; wherein, h on the left aug H on the right side of the vector representing the modification vector calculated this time aug Indicating the initialized modification vector (or the last time the modification vector was calculated), alpha indicates the preset learning rate hyperparameter,
this gradient value representing a loss value of a pixel of the image,
representing the present gradient value of the modification vector.
As a second alternative implementation of step S121, the manner of obtaining the modification vector according to the classification loss value may include:
step S121d: and acquiring a sample image and a sample label, wherein the sample label is a category label to which the sample image belongs.
The sample image and the sample label may be obtained separately, for example: manually collecting a sample image, and manually identifying a sample label of the sample image; of course, the sample image and the sample label may also be acquired together packed into the second training data set.
The obtaining manner of the second training data set in step S121d includes: the first acquisition mode is that a target object is shot by using terminal equipment such as a video camera, a video recorder or a color camera, a class label to which a sample image belongs is manually identified and marked as a sample label, and a second training data set is acquired; then the terminal device sends a second training data set to the electronic device, then the electronic device receives the second training data set sent by the terminal device, and the electronic device can store the second training data set into a file system, a database or mobile storage equipment; a second obtaining manner, obtaining a second training data set stored in advance, specifically for example: acquiring a second training data set from a file system, a database or a mobile storage device; in a third obtaining mode, software such as a browser is used for obtaining the second training data set on the internet, or other application programs are used for accessing the internet to obtain the second training data set.
Step S121e: and performing category prediction on the sample image to obtain a predicted image category.
Step S121f: and determining a modification vector according to the classification loss value between the prediction image class and the class label.
The embodiments of the above steps S121e to S121f are, for example: performing class prediction on the sample image by using an image classification model (such as a LeNet network model, an AlexNet network model, a VGG network model, a GoogLeNet network model, a ResNet network model and the like) to obtain a predicted image class; according to cross entropy loss function
Or Euclidean distance loss function
Calculating a classification loss value between the classification of the predicted image and the classification label; wherein L is 1 Representing the classification loss value, N representing the total number of original images, y i Indicates the category label, p, corresponding to the ith original image i And indicating the prediction image category corresponding to the ith original image. Finally, the above specific way of determining the modification vector according to the classification loss value is, for example: randomly initializing modification vector, using formula
Calculating the initialized modification vector and the classification loss value to obtain a final modification vector; wherein, h on the left aug H on the right side of the vector representing the modification vector calculated this time aug Indicating the initialized modification vector (or the last time the modification vector was calculated), alpha indicates the preset learning rate hyperparameter,
a present gradient value representing a classification loss value,
representing the present gradient value of the modification vector.
As a third alternative implementation of step S121, the manner of obtaining the modification vector according to the image pixel loss value and the classification loss value may include:
step S121g: obtaining a specimen image and a specimen label, the specimen label comprising: the corresponding enhanced image and the class label of the sample image.
The implementation principle and implementation manner of step S121g are similar to those of step S121a and step S121d, and therefore, the implementation principle and implementation manner will not be described here, and if it is not clear, reference may be made to the description of step S121a and step S121 d.
Step S121h: and performing image enhancement on the sample image by using the neural network model to obtain a prediction enhanced image, and performing class prediction on the sample image to obtain a predicted image class.
The embodiment of the step S121h is, for example: the method includes the steps of performing image enhancement on a sample image by using a neural network model (such as the self-coding network or the super network) to obtain a prediction enhanced image, and performing class prediction on the sample image by using an image classification model (such as a LeNet network model, an AlexNet network model, a VGG network model, a GoogLeNet network model, a ResNet network model and the like) to obtain a prediction image class.
Step S121i: and calculating an image pixel loss value between the prediction enhanced image and the sample label, and calculating a classification loss value between the prediction image class and the class label.
An embodiment of the step S121i includes: image pixel loss values between the prediction enhanced image and the sample labels are calculated using a KL divergence formula, mean Square Error (MSE) loss function, exponential loss function (exponential loss), or the like, and classification loss values between the prediction image classes and the class labels are calculated using a cross entropy loss function, or the like.
Step S121j: a modification vector is determined based on the image pixel loss value and the classification loss value.
The embodiment of step S121j described above is, for example: using the formula
Calculating the loss value of the image pixel and the classification loss value to obtain a final modification vector; wherein, h on the left aug H on the right side of the vector representing the modification vector calculated this time aug Indicating the initialized modification vector (or the last time the modification vector was calculated), alpha indicating the preset learning rate hyperparameter,
a present gradient value representing a sum of the image pixel loss value and the classification loss value,
representing the present gradient value of the modification vector.
Optionally, in a specific practical process, if a value of the hidden feature vector or the modified hidden vector is greater than a preset threshold, the hidden feature vector or the modified hidden vector may be normalized, and the normalized hidden feature vector or the modified hidden vector may be used to participate in the above calculation.
Please refer to fig. 4, which illustrates a schematic structural diagram of an image enhancement apparatus provided in an embodiment of the present application; the embodiment of the present application provides an image enhancement apparatus 200, including:
the feature vector obtaining module 210 is configured to perform feature extraction on the original image by using a neural network model to obtain a hidden feature vector.
And a feature vector modification module 220, configured to modify the hidden feature vector to obtain a modified hidden vector.
An enhanced image obtaining module 230, configured to perform image enhancement on the original image according to the modified hidden vector, so as to obtain an enhanced image corresponding to the original image.
Optionally, in an embodiment of the present application, the neural network model is a self-coding network; a feature vector obtaining module comprising:
and the image feature extraction sub-module is used for extracting features of the original image by using an encoder in the self-encoding network.
An enhanced image acquisition module comprising:
and the original image enhancement submodule is used for enhancing the image of the original image according to the modified hidden vector by using a generator in the self-coding network.
Optionally, in an embodiment of the present application, the neural network model is a hyper-network; an enhanced image acquisition module comprising:
and the network parameter modification submodule is used for modifying the network parameters of the super network by using the modified hidden vector to obtain the modified super network.
And the network image enhancement sub-module is used for carrying out image enhancement on the original image through the modified hyper-network.
Optionally, in an embodiment of the present application, the feature vector modification module includes:
and the vector increasing and decreasing modification submodule is used for acquiring a modification vector and increasing the modification vector on the basis of the hidden feature vector.
Optionally, in this embodiment of the present application, the vector increase/decrease modification sub-module includes:
the image processing apparatus includes a first data acquisition unit configured to acquire a specimen image and a specimen label, which is an enhanced image obtained by image-enhancing the specimen image.
And the enhanced image prediction unit is used for carrying out image enhancement on the sample image by using the neural network model to obtain a prediction enhanced image.
A first vector determination unit for determining a modification vector based on image pixel loss values between the prediction enhanced image and the sample labels.
Optionally, in this embodiment of the present application, the vector increase/decrease modification sub-module includes:
and the second data acquisition unit is used for acquiring the sample image and a sample label, wherein the sample label is a category label to which the sample image belongs.
And the image type prediction unit is used for performing type prediction on the sample image to obtain a predicted image type.
And the second vector determining unit is used for determining the modification vector according to the classification loss value between the prediction image class and the class label.
Optionally, in this embodiment of the present application, the vector increase/decrease modifying submodule includes:
a third data acquisition unit for acquiring a specimen image and a specimen label, the specimen label including: and the enhanced image and the class label correspond to the sample image.
And the data type prediction unit is used for performing image enhancement on the sample image by using the neural network model to obtain a prediction enhanced image, and performing type prediction on the sample image to obtain a predicted image type.
And the image loss calculating unit is used for calculating an image pixel loss value between the prediction enhanced image and the sample label and calculating a classification loss value between the prediction image class and the class label.
And a third vector determination unit for determining a modification vector according to the image pixel loss value and the classification loss value.
It should be understood that the apparatus corresponds to the above-mentioned embodiment of the image enhancement method, and can perform the steps related to the above-mentioned embodiment of the method, and the specific functions of the apparatus can be referred to the above description, and the detailed description is appropriately omitted here to avoid redundancy. The device includes at least one software function that can be stored in memory in the form of software or firmware (firmware) or solidified in the Operating System (OS) of the device.
Please refer to fig. 5, which illustrates a schematic structural diagram of an electronic device according to an embodiment of the present application. An electronic device 300 provided in an embodiment of the present application includes: a processor 310 and a memory 320, the memory 320 storing machine readable instructions executable by the processor 310, the machine readable instructions when executed by the processor 310 performing the method as above.
Embodiments of the present application further provide a computer-readable storage medium 330, where the computer-readable storage medium 330 stores a computer program, and the computer program is executed by the processor 310 to perform the above method.
The computer-readable storage medium 330 may be implemented by any type of volatile or nonvolatile storage device or combination thereof, such as a Static Random Access Memory (SRAM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), an Erasable Programmable Read-Only Memory (EPROM), a Programmable Read-Only Memory (PROM), a Read-Only Memory (ROM), a magnetic Memory, a flash Memory, a magnetic disk, or an optical disk.
It should be noted that, in the present specification, the embodiments are all described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments may be referred to each other. For the device-like embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The apparatus embodiments described above are merely illustrative and, for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
In addition, functional modules of the embodiments in the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part. Furthermore, in the description herein, reference to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the embodiments of the present application. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
In this document, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.
The above description is only an alternative embodiment of the embodiments of the present application, but the scope of the embodiments of the present application is not limited thereto, and any person skilled in the art can easily conceive of changes or substitutions within the technical scope of the embodiments of the present application, and all the changes or substitutions should be covered by the scope of the embodiments of the present application.
Claims (10)
1. An image enhancement method, comprising:
carrying out feature extraction on an original image by using a neural network model to obtain a hidden feature vector;
modifying the hidden feature vector to obtain a modified hidden vector;
and performing image enhancement on the original image according to the modified hidden vector to obtain an enhanced image corresponding to the original image.
2. The method of claim 1, wherein the neural network model is a self-coding network; the method for extracting the features of the original image by using the neural network model comprises the following steps:
performing feature extraction on the original image by using an encoder in the self-coding network;
the image enhancement of the original image according to the modified hidden vector comprises:
and performing image enhancement on the original image according to the modified hidden vector by using a generator in the self-coding network.
3. The method of claim 1, wherein the neural network model is a hyper-network; the image enhancement of the original image according to the modified hidden vector comprises:
modifying the network parameters of the hyper-network by using the modified hidden vector to obtain a modified hyper-network;
and performing image enhancement on the original image through the modified hyper-network.
4. The method of claim 1, wherein the modifying the latent feature vector comprises:
and acquiring a modification vector, and adding the modification vector on the basis of the hidden feature vector.
5. The method of claim 4, wherein obtaining the modification vector comprises:
acquiring a sample image and a sample label, wherein the sample label is an enhanced image obtained by enhancing the sample image;
performing image enhancement on the sample image by using the neural network model to obtain a prediction enhanced image;
determining the modification vector from image pixel loss values between the prediction enhanced image and the sample label.
6. The method of claim 4, wherein obtaining the modification vector comprises:
obtaining a sample image and a sample label, wherein the sample label is a category label to which the sample image belongs;
performing category prediction on the sample image to obtain a predicted image category;
determining the modification vector according to a classification loss value between the predicted image class and the class label.
7. The method of claim 4, wherein obtaining the modification vector comprises:
obtaining a specimen image and a specimen label, the specimen label comprising: the enhanced image and the category label corresponding to the sample image;
performing image enhancement on the sample image by using the neural network model to obtain a prediction enhanced image, and performing category prediction on the sample image to obtain a predicted image category;
calculating an image pixel loss value between the prediction enhanced image and the sample label, and calculating a classification loss value between the prediction image class and the class label;
determining the modification vector based on the image pixel loss value and the classification loss value.
8. An image enhancement apparatus, comprising:
the characteristic vector obtaining module is used for extracting the characteristics of the original image by using a neural network model to obtain a hidden characteristic vector;
the characteristic vector modification module is used for modifying the hidden characteristic vector to obtain a modified hidden vector;
and the enhanced image obtaining module is used for carrying out image enhancement on the original image according to the modified hidden vector to obtain an enhanced image corresponding to the original image.
9. An electronic device, comprising: a processor and a memory, the memory storing machine-readable instructions executable by the processor, the machine-readable instructions, when executed by the processor, performing the method of any of claims 1 to 7.
10. A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, is adapted to carry out the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211154177.8A CN115439367A (en) | 2022-09-21 | 2022-09-21 | Image enhancement method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211154177.8A CN115439367A (en) | 2022-09-21 | 2022-09-21 | Image enhancement method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115439367A true CN115439367A (en) | 2022-12-06 |
Family
ID=84248916
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211154177.8A Pending CN115439367A (en) | 2022-09-21 | 2022-09-21 | Image enhancement method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115439367A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117274316A (en) * | 2023-10-31 | 2023-12-22 | 广东省水利水电科学研究院 | River surface flow velocity estimation method, device, equipment and storage medium |
-
2022
- 2022-09-21 CN CN202211154177.8A patent/CN115439367A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117274316A (en) * | 2023-10-31 | 2023-12-22 | 广东省水利水电科学研究院 | River surface flow velocity estimation method, device, equipment and storage medium |
| CN117274316B (en) * | 2023-10-31 | 2024-05-03 | 广东省水利水电科学研究院 | River surface flow velocity estimation method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110163235B (en) | Training of image enhancement model, image enhancement method, device and storage medium | |
| Ren et al. | Deep Robust Single Image Depth Estimation Neural Network Using Scene Understanding. | |
| CN110807757B (en) | Image quality evaluation method and device based on artificial intelligence and computer equipment | |
| CN109493297B (en) | Low-quality face image enhancement method, system, equipment and storage medium | |
| CN111292264A (en) | Image high dynamic range reconstruction method based on deep learning | |
| CN113409361B (en) | Multi-target tracking method and device, computer and storage medium | |
| CN113066034A (en) | Face image restoration method and device, restoration model, medium and equipment | |
| CN113269722A (en) | Training method for generating countermeasure network and high-resolution image reconstruction method | |
| CN115205150A (en) | Image deblurring method, device, equipment, medium and computer program product | |
| CN114283352A (en) | Video semantic segmentation device, training method and video semantic segmentation method | |
| CN111784693A (en) | Image quality evaluation method and device, electronic equipment and storage medium | |
| CN114419102B (en) | Multi-target tracking detection method based on frame difference time sequence motion information | |
| CN112802076A (en) | Reflection image generation model and training method of reflection removal model | |
| CN116977674A (en) | Image matching method, related device, storage medium and program product | |
| CN115439367A (en) | Image enhancement method and device, electronic equipment and storage medium | |
| CN115984307A (en) | Video object segmentation method and device, electronic equipment and storage medium | |
| CN110942097A (en) | Imaging-free classification method and system based on single-pixel detector | |
| Zhang et al. | Embarrassingly simple binarization for deep single imagery super-resolution networks | |
| CN117115786B (en) | Depth estimation model training method for joint segmentation tracking and application method | |
| Li et al. | ROI-Based Deep Image Compression with Swin Transformers | |
| CN114998814B (en) | Target video generation method and device, computer equipment and storage medium | |
| CN116805290A (en) | Image restoration method and device | |
| CN117078970A (en) | Picture identification method and device, electronic equipment and storage medium | |
| US20230135978A1 (en) | Generating alpha mattes for digital images utilizing a transformer-based encoder-decoder | |
| CN116977247A (en) | Image processing method, device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |