CN111815529B - Low-quality image classification enhancement method based on model fusion and data enhancement - Google Patents
Low-quality image classification enhancement method based on model fusion and data enhancement Download PDFInfo
- Publication number
- CN111815529B CN111815529B CN202010607913.5A CN202010607913A CN111815529B CN 111815529 B CN111815529 B CN 111815529B CN 202010607913 A CN202010607913 A CN 202010607913A CN 111815529 B CN111815529 B CN 111815529B
- Authority
- CN
- China
- Prior art keywords
- image
- classified
- enhancement
- neural network
- probability vector
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/254—Fusion techniques of classification results, e.g. of results related to same input data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Abstract
The invention relates to a low-quality image classification enhancement method based on model fusion and data enhancement, which comprises the following steps of: s1: establishing an image set; s2: performing data enhancement on the image set; s3: constructing and training a VGG16 convolutional neural network model; s4: constructing and training a ResNet convolution neural network model; s5: inputting an image to be classified; s6: acquiring a first probability vector and a second probability vector of an image to be classified; s7: acquiring a fusion probability vector, acquiring the image type of an image to be classified, and if the image type is clear, entering a step S9, otherwise, entering a step S8; s8: enhancing the image to be classified to obtain an enhanced image, and inputting the enhanced image serving as the image to be classified into the step S6; s9: and outputting the image. Compared with the prior art, the method and the device have the advantages that two modes of data enhancement and model fusion are adopted to improve the accuracy of classification, the images can be effectively classified, and the enhancement effect is good.
Description
Technical Field
The invention relates to a low-quality image classification enhancement method, in particular to a low-quality image classification enhancement method based on model fusion and data enhancement.
Background
When a camera shoots a picture, a low-quality image is usually shot due to poor lighting environment or unstable self-stability, and the like, so that the image details are lost due to the fact that shadow, low brightness or blurred pictures exist on the low-quality image, and subsequent operations such as identification and analysis are difficult to perform on the image.
In a low-quality image enhancement method under extreme weather conditions disclosed in chinese patent No. cn201610079472.X, etc., a classification enhancement method for distinguishing a haze image and a rain and snow image according to chromaticity component values is proposed. The method adopts a pure physics algorithm, can distinguish a few image types, has poor robustness, and is very easy to have misjudgment and the like. In an imaging identification method and system in severe weather disclosed in chinese patent CN201811484514.3, a method for enhancing and then identifying collected low-quality images in severe weather is provided, an electronic image stabilization algorithm is used to eliminate motion blur of the images or an adaptive image defogging algorithm is used to eliminate cloud and fog interference of the images, and then a VGG16 convolutional neural network is used to identify and classify the images. However, the invention does not mention how to distinguish whether the acquired image is a low-quality image, and only one enhancement algorithm is adopted to perform enhancement processing on the low-quality images of all categories, so that the enhancement effect is general.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide a low-quality image classification enhancement method based on model fusion and data enhancement.
The purpose of the invention can be realized by the following technical scheme:
a low-quality image classification enhancement method based on model fusion and data enhancement comprises the following steps:
s1: establishing an image set, wherein the image set comprises a clear image, a fuzzy image and a low-brightness image;
s2: performing data enhancement on the images in the image set;
s3: constructing a VGG16 convolutional neural network model, and training the VGG16 convolutional neural network model through an image set;
s4: constructing a ResNet convolutional neural network model, and training the ResNet convolutional neural network model through an image set;
s5: inputting an image to be classified;
s6: inputting an image to be classified into a VGG16 convolutional neural network model to obtain a first probability vector of the image to be classified, inputting the image to be classified into a ResNet convolutional neural network model to obtain a second probability vector of the image to be classified;
s7: fusing the first probability vector and the second probability vector to obtain a fused probability vector, and acquiring the image type of the image to be classified, if the image type is clear, entering a step S9, otherwise entering a step S8;
s8: selecting an image enhancement algorithm corresponding to the image type to enhance the image to be classified to obtain an enhanced image, and inputting the enhanced image serving as the image to be classified into the step S6;
s9: and outputting the image.
Preferably, the first probability vector is:
[λ a1 ，λ a2 ，λ a3 ]
wherein λ is a1 ，λ a2 ，λ a3 Probability that the images acquired in the VGG16 convolutional neural network model are sharp images, blurred images and low-brightness images respectively,
the second probability vector is:
[λ b1 ，λ b2 ，λ b3 ]
wherein λ is b1 ，λ b2 ，λ b3 Respectively the probability that the images acquired in the ResNet convolutional neural network model are sharp images, blurred images and low-brightness images,
the fusion probability vector is as follows:
[λ a1 +λ b1 ，λ a2 +λ b2 ，λ a3 +λ b3 ]
and the image type represented by the maximum value in the fusion probability vector is the image type of the image to be classified.
Preferably, the classifier of the VGG16 convolutional neural network model comprises two fully-connected layers, wherein the second fully-connected layer adopts a Softmax activation function, and the second fully-connected layer maps the input values of the first fully-connected layer between intervals (0,1) to obtain the first probability vector.
Preferably, the classifier of the ResNet convolutional neural network model includes two fully-connected layers, wherein the second fully-connected layer uses a Softmax activation function, and the second fully-connected layer maps the input value of the first fully-connected layer to an interval (0,1) to obtain a second probability vector.
Preferably, the data enhancement is one or more of flipping, rotating, scaling, cropping, shifting, adding noise, and modifying contrast, at random, of the image data.
Preferably, the data enhancement in step S2 is to perform horizontal rotation and vertical rotation on the image data, and step S2 specifically is to:
s21: taking the vertex of the lower left corner of the image as an origin O, establishing an xyz space coordinate system, wherein the size of the image is a multiplied by b, and the image is superposed with an xy plane;
s22: randomly judging whether the image is horizontally rotated or not, wherein the probability of horizontal rotation or not is 0.5, if the image is horizontally rotated, rotating the image 180 degrees by taking (x = a/2, z = 0) as a rotating shaft, and if the image is not horizontally rotated, entering the step S22;
s23: and randomly judging whether the image is vertically rotated or not, wherein the probability of vertical rotation or not is 0.5, if the image is vertically rotated, rotating the image by 180 degrees by taking (y = b/2, z = 0) as a rotating shaft, and finishing data enhancement, otherwise, the image is not vertically rotated, and finishing data enhancement.
Preferably, when the number of times of enhancement of the image is greater than the maximum number of times of enhancement, the image is output:
s5: inputting an image to be classified, and setting the enhancement times p =0;
s6: inputting an image to be classified into a VGG16 convolutional neural network model to obtain a first probability vector of the image to be classified, inputting the image to be classified into a ResNet convolutional neural network model to obtain a second probability vector of the image to be classified;
s7: fusing the first probability vector and the second probability vector to obtain a fused probability vector, and acquiring the image type of the image to be classified, if the image type is clear, entering a step S9, otherwise entering a step S8;
s8: selecting an image enhancement algorithm corresponding to the image type to enhance the image to be classified, obtaining an enhanced image, setting the enhancement times p = p +1, if p is larger than or equal to the maximum enhancement times, entering the step S9, and otherwise, inputting the enhanced image serving as the image to be classified into the step S6;
s9: and outputting the image.
Preferably, the VGG16 convolutional neural network model comprises 13 convolutional layers, 3 fully-connected layers and 5 pooling layers.
Preferably, in S8, a GAN-based blind motion blur removal algorithm is used to perform enhancement processing on the image with the blurred image as the image type.
Preferably, in S8, a low-brightness image enhancement algorithm based on a corresponding camera model is used to perform enhancement processing on an image of which the image type is a low-brightness image.
Compared with the prior art, the invention has the following advantages:
(1) Compared with the prior art, the image classification method based on the convolutional neural network has the advantages that the convolutional neural network is adopted for classifying the image, the accuracy rate is higher compared with that of a traditional classifier based on physics, and meanwhile, the classification accuracy rate is improved by adopting two modes of data enhancement and model fusion;
(2) The invention integrates two algorithms special for enhancing two types of images, namely fuzzy images and low-brightness images, and can effectively classify and restore the detailed information of the images;
(3) In order to prevent two types of conditions from existing on one image, the method sets the maximum enhancement times, and performs classification enhancement on the image for multiple times to realize the maximum enhancement effect;
(4) The random image data enhancement mode is utilized to perform data enhancement of horizontal rotation and vertical rotation on the image data, and the classification accuracy is effectively improved.
Drawings
FIG. 1 is a flow chart of the present invention;
FIG. 2 is a flow chart of an embodiment of the present invention;
FIG. 3 is a block diagram of the VGG16 model;
FIG. 4 is a block diagram of the VGG16 model;
FIG. 5 is a block diagram of a residual unit;
FIG. 6 is a block diagram of the ResNet model;
fig. 7 is a structural diagram of the DeblurGAN model.
Detailed Description
The invention is described in detail below with reference to the figures and specific embodiments. Note that the following description of the embodiments is merely a substantial example, and the present invention is not intended to be limited to the application or the use thereof, and is not limited to the following embodiments.
Examples
A low-quality image classification enhancement method based on model fusion and data enhancement is disclosed, as shown in FIG. 1, and comprises the following steps:
s1: an image set is established.
In the image set of the present embodiment, the image set includes three types of images, i.e., a sharp image, a blurred image, and a low-brightness image, and includes a training set, a test set, and a verification set, where 300 images of each type are included in the training set, 50 images of each type are included in the test set, and 50 images of each type are included in the verification set.
S2: data enhancement is performed on the images in the image set.
Data enhancement is one or more of flipping, rotating, scaling, cropping, shifting, adding noise, and modifying contrast on the image data at random.
In this embodiment, the data enhancement in step S2 is to perform horizontal rotation and vertical rotation on the image data, where step S2 specifically is:
s21: taking the vertex of the lower left corner of the image as an origin O, establishing an xyz space coordinate system, wherein the size of the image is a multiplied by b, and the image is superposed with an xy plane;
s22: randomly judging whether the image is horizontally rotated or not, wherein the probability of horizontal rotation or not is 0.5, if the image is horizontally rotated, rotating the image 180 degrees by taking (x = a/2, z = 0) as a rotating shaft, and if the image is not horizontally rotated, entering the step S22;
s23: and randomly judging whether the image is vertically rotated or not, wherein the probability of vertical rotation or not is 0.5, if the image is vertically rotated, rotating the image by 180 degrees by taking (y = b/2, z = 0) as a rotating shaft, and finishing data enhancement, otherwise, the image is not vertically rotated, and finishing data enhancement.
S3: and constructing a VGG16 convolutional neural network model, and training the VGG16 convolutional neural network model through the image set.
As shown in fig. 3 and 4, the model structure of the VGG16 includes 13 Convolutional layers (volumetric Layer), 3 Fully connected layers (full connected Layer), and 5 pooling layers (Pool Layer). The convolutional layers and the pooling layers can be divided into different blocks (blocks), which are numbered from front to back as blocks 1-5, and each Block comprises a plurality of convolutional layers and pooling layers, for example, block4 comprises 3 convolutional layers (Conv 3-512) and 1 pooling layer (Maxpool); and within the same block, the number of channels of the convolutional layers is the same, for example: block2 contains 2 convolutional layers, each convolutional layer is represented by Conv2-128, namely the convolutional core is 3 x 3, and the number of channels is 128; block3 contains 3 convolutional layers, each denoted by Conv3-256, i.e., the convolutional kernel is 3 × 3 × 3, and the number of channels is 256. The input structure of the VGG16 model is 224 × 224 × 3, where the number of channels is doubled, from 64 to 128 to 256 in sequence, until 512 remains the same and is no longer doubled, and the height and width of the image is halved, from 224 → 112 → 56 → 28 → 14 → 7.
S4: and constructing a ResNet convolutional neural network model, and training the ResNet convolutional neural network model through the image set.
The ResNet convolution neural network solves the degradation problem of the deep network through residual learning: as network depth increases, network accuracy saturates and even decreases. As shown in fig. 5 and 6, the structure of the ResNet model refers to the VGG19 structure, on the basis of which a residual unit is modified and added through a short circuit mechanism.
The residual unit has two layers, the first layer:
F(x)＝W 2 σ(W 1 x)
where σ represents the nonlinear activation function ReLu, W 1 And W 2 Two weights in the first layer are represented.
Then the first layer is connected with the second layer through a short circuit, the ReLu function of the second layer obtains output y,
y＝F(x)+x
where F (x) is the output of the first layer.
The ResNet is mainly characterized in that the convolution with the convolution kernel size of 2 is directly used for down-sampling, and a global average pooling layer is adopted to replace a full-connection layer. Compared with the common network, the ResNet network imposes a short circuit mechanism between every two layers, so that residual error learning is formed.
S5: inputting an image to be classified;
s6: and inputting the image to be classified into a VGG16 convolutional neural network model to obtain a first probability vector of the image to be classified, inputting the image to be classified into a ResNet convolutional neural network model to obtain a second probability vector of the image to be classified.
S7: and fusing the first probability vector and the second probability vector to obtain a fused probability vector, and acquiring the image type of the image to be classified, if the image type is clear, entering a step S9, and if not, entering a step S8.
The invention adopts model fusion of a VGG16 convolution neural network model and a ResNet convolution neural network model. Model fusion is to directly add probability vectors predicted by two or more basic models, and take the class with the highest probability after addition as the prediction class of the image.
In this embodiment, the probability vector is represented in a Onehot encoding form. The Onehot coding is a common class label coding form in a neural network, and converts a class label with an integer value into a binary value, wherein the index of the label value is marked as 1, and the other indexes are 0. In this embodiment, there are 3 category labels for the sharp image, the blurred image, and the low-luminance image, which correspond to label values 0,1, and 2, respectively. The label value 0 is converted into [1,0,0] after being subjected to One-Hot encoding, the label value 1 is converted into [0,1,0], and the label value 2 is converted into [0,0,1].
In this embodiment, the two model classifiers each include two fully-connected layers, and the second fully-connected layer employs a Softmax activation function, and the second fully-connected layer maps the input values from the first fully-connected layer to intervals (0,1), and obtains a first probability vector and a second probability vector. Specifically, the Softmax function maps the input values of the first fully-connected layer between intervals (0,1), forming, for example, [ λ [ [ λ ] 1 ，λ 2 ，λ 3 ]Of the probability vector of i For each class of probability, the sum of which is 1, usually taken as λ i The largest median is taken as the predicted class.
The formula of the Softmax function is:
wherein Z is i And C is the output value of the ith node, and the number of output nodes, namely the number of classified categories.
In this embodiment, the pre-trained VGG16 convolutional neural network model and ResNet convolutional neural network model are adopted to output the first probability vector [ λ [ lambda ] ], respectively a1 ，λ a2 ，λ a3 ]And a second probability vector [ lambda ] b1 ，λ b2 ，λ b3 ]。
Wherein, λ a1 ，λ a2 ，λ a3 Probability of the images acquired in the VGG16 convolutional neural network model being respectively a sharp image, a blurred image and a low-brightness image, lambda b1 ，λ b2 ，λ b3 Respectively, the probability that the image obtained in the ResNet convolutional neural network model is a sharp image, a blurred image, and a low-brightness image, respectively.
Adding the two probability vectors to obtain a fusion probability vector
[λ a1 +λ b1 ，λ a2 +λ b2 ，λ a3 +λ b3 ]
And the image type represented by the maximum value in the fusion probability vector is the image type of the image to be classified.
S8: and (6) selecting an image enhancement algorithm corresponding to the image type to enhance the image to be classified to obtain an enhanced image, and inputting the enhanced image serving as the image to be classified into the step S6.
Specifically, a GAN-based blind motion blur removal algorithm is adopted to enhance an image with a blurred image as an image type, and a corresponding camera model-based low-brightness image enhancement algorithm is adopted to enhance an image with a low-brightness image as an image type.
Further, a GAN-based blind motion blur removal algorithm:
the algorithm treats image blurring removal as an end-to-end task through a GAN network, and a clear image is automatically generated according to a blurred image by using a generator through learning the blurred image and the clear image. The blind deblurring in the algorithm aims to restore a clear image Is by giving a blurred image IB under the condition of no blur kernel, deblurring Is carried out by adopting a generator, and a discrimination network Is introduced in the training process to carry out training and learning in a countermeasure mode.
As shown IN fig. 7, the generator contains two downsampling convolution modules, 9 residual modules (containing one convolution, IN and ReLU), and two upsampling transposed convolution modules, while also introducing global residual concatenation. This structure may allow faster training and at the same time better performance. Besides the generator, the algorithm also defines a discriminator in the training process, and adopts WGAN (Wasserstein GAN) with a penalty term to carry out countertraining.
The loss function of the algorithm comprises two parts: content loss and antagonism loss, the formula is:
L＝L CAN +λL x
wherein L is GAN To combat losses, L x For content loss, λ is the weight.
In countering loss, using WGAN-GP as a penalty function, the loss is calculated as:
wherein, I B In order to input a blurred image, the image is displayed,
in order to be a generator,
to discriminate the network.
The content loss adopts the perception loss, and is based on the difference between the generated and target image CNN feature mapping, and the formula is as follows:
wherein, I s For the generated image, I B In order to input a blurred image, the image is displayed,
to be a generator, phi i，j Is a feature map obtained by the jth convolution (after activation) before the ith max pooling layer in the VGG19 network, W i，j And H i，j Is a dimension function graph.
Further, a low brightness picture enhancement algorithm based on the corresponding model of the camera:
for a picture taken by a camera, the pixel value is not proportional to the brightness value reflected by the object. The nonlinear transformation Function from the luminance value sensed by the Camera photosensitive element to the actual pixel value of the image is called a Camera Response Function (CRF). The algorithm firstly waits for a camera response model by analyzing the relation between pictures with different exposure degrees, then obtains an exposure contrast image of the image by using a picture brightness component estimation method, and finally enhances the low-brightness picture by using the camera response model and the exposure contrast image.
The camera response model is defined as:
P＝f(E)
where E is the irradiance of the picture, P is the pixel value of the picture, and f should satisfy:
the algorithm uses a luminance Transform Function (BTF) to estimate f.
BTF is a mapping function between two graphs of the same scene but different exposure levels:
R＝g(P 0 ，k)
wherein P is 1 And P 0 Is an image with different exposure degrees in the same scene, and k is an exposure ratio.
CRM can therefore be calculated from g (f (E), k) = f (kE).
S9: and outputting the image.
The experimental environment in this example is: the accuracy of the system Win10, the display card Tesla P100, the deep learning framework Tensorflow2.0, the VGG16, the ResNet model and the fused model is shown in the following table, and the accuracy of the fused model is higher than that of any single model, which is shown in the following table.
In an embodiment of the present invention, in order to prevent the image processed by the two enhancement algorithms from being blurred or with low brightness, as shown in fig. 2, a maximum enhancement number is set, and when the enhancement number of the image is greater than or equal to the maximum enhancement number, the image is output:
s1: establishing an image set, wherein the image set comprises a clear image, a fuzzy image and a low-brightness image;
s2: performing data enhancement on the images in the image set;
s3: constructing a VGG16 convolutional neural network model, and training the VGG16 convolutional neural network model through an image set;
s4: constructing a ResNet convolutional neural network model, and training the ResNet convolutional neural network model through an image set;
s5: inputting an image to be classified, and setting the enhancement times p =0;
s6: inputting an image to be classified into a VGG16 convolutional neural network model to obtain a first probability vector of the image to be classified, inputting the image to be classified into a ResNet convolutional neural network model to obtain a second probability vector of the image to be classified;
s7: fusing the first probability vector and the second probability vector to obtain a fused probability vector, and acquiring the image category of the image to be classified, if the image type is clear, entering a step S9, otherwise entering a step S8;
s8: selecting an image enhancement algorithm corresponding to the image type to enhance the image to be classified, obtaining an enhanced image, setting the enhancement times p = p +1, if p is larger than or equal to the maximum enhancement times, entering the step S9, and otherwise, inputting the enhanced image serving as the image to be classified into the step S6;
s9: and outputting the image.
The above embodiments are merely examples and do not limit the scope of the present invention. These embodiments may be implemented in other various manners, and various omissions, substitutions, and changes may be made without departing from the technical spirit of the present invention.
Claims (8)
1. A low-quality image classification enhancement method based on model fusion and data enhancement is characterized by comprising the following steps:
s1: establishing an image set, wherein the image set comprises a clear image, a fuzzy image and a low-brightness image;
s2: performing data enhancement on the image in the image set, wherein the data enhancement is one or more operations of turning, rotating, scaling, cutting, shifting, adding noise and modifying contrast on the image data randomly;
s3: constructing a VGG16 convolutional neural network model, and training the VGG16 convolutional neural network model through an image set;
s4: constructing a ResNet convolutional neural network model, and training the ResNet convolutional neural network model through an image set;
s5: inputting an image to be classified;
s6: inputting an image to be classified into a VGG16 convolutional neural network model to obtain a first probability vector of the image to be classified, inputting the image to be classified into a ResNet convolutional neural network model to obtain a second probability vector of the image to be classified;
s7: fusing the first probability vector and the second probability vector to obtain a fused probability vector, and acquiring the image type of the image to be classified, if the image type is clear, entering a step S9, otherwise entering a step S8;
s8: selecting an image enhancement algorithm corresponding to the image type to enhance the image to be classified to obtain an enhanced image, and inputting the enhanced image serving as the image to be classified into the step S6;
s9: outputting an image;
and in the S8, the image with the image type of the blurred image is enhanced by adopting a GAN-based blind motion blur removal algorithm.
2. The method of claim 1, wherein the first probability vector is:
[λ a1 ，λ a2 ，λ a3 ]
wherein λ is a1 ，λ a2 ，λ a3 Probability that the images acquired in the VGG16 convolutional neural network model are sharp images, blurred images and low-brightness images respectively,
the second probability vector is:
[λ b1 ，λ b2 ，λ b3 ]
wherein λ is b1 ，λ b2 ，λ b3 Respectively the probability that the images acquired in the ResNet convolutional neural network model are sharp images, blurred images and low-brightness images,
the fusion probability vector is as follows:
[λ a1 +λ b1 ，λ a2 +λ b2 ，λ a3 +λ ba ]
and the image type represented by the maximum value in the fusion probability vector is the image type of the image to be classified.
3. The method of claim 1, wherein the classifier of the VGG16 convolutional neural network model comprises two fully-connected layers, wherein the second fully-connected layer uses a Softmax activation function, and the second fully-connected layer maps the input values from the first fully-connected layer to the intervals (0,1) to obtain the first probability vector.
4. The method of claim 1, wherein the classifier of the ResNet convolutional neural network model comprises two fully-connected layers, wherein the second fully-connected layer uses a Softmax activation function, and the second fully-connected layer maps the input values from the first fully-connected layer to the intervals (0,1) to obtain the second probability vector.
5. The method according to claim 1, wherein the data enhancement in step S2 is performed by horizontally rotating and vertically rotating the image data, and the step S2 specifically comprises:
s21: taking the vertex of the lower left corner of the image as an origin O, establishing an xyz space coordinate system, wherein the size of the image is a multiplied by b, and the image is superposed with an xy plane;
s22: randomly judging whether the image is horizontally rotated or not, wherein the probability of horizontal rotation or not is 0.5, if the image is horizontally rotated, rotating the image by 180 degrees by taking (x = a/2, z = 0) as a rotating shaft, and otherwise, entering the step S22;
s23: and randomly judging whether the image is vertically rotated or not, wherein the probability of vertical rotation or not is 0.5, if the image is vertically rotated, rotating the image by 180 degrees by taking (y = b/2, z = 0) as a rotating shaft, and finishing data enhancement, otherwise, the image is not vertically rotated, and finishing data enhancement.
6. The method for classifying and enhancing the low-quality image based on the model fusion and the data enhancement as claimed in claim 1, wherein when the enhancing times of the image is greater than or equal to the maximum enhancing times, the image is output:
s5: inputting an image to be classified, and setting the enhancement times p =0;
s6: inputting the image to be classified into a VGG16 convolution neural network model to obtain a first probability vector of the image to be classified, inputting the image to be classified into a ResNet convolution neural network model to obtain a second probability vector of the image to be classified;
s7: fusing the first probability vector and the second probability vector to obtain a fused probability vector, and acquiring the image type of the image to be classified, if the image type is clear, entering a step S9, otherwise entering a step S8;
s8: selecting an image enhancement algorithm corresponding to the image type to enhance the image to be classified, obtaining an enhanced image, setting the enhancement times p = p +1, if p is larger than or equal to the maximum enhancement times, entering the step S9, and otherwise, inputting the enhanced image serving as the image to be classified into the step S6;
s9: and outputting the image.
7. The method of claim 1, wherein the VGG16 convolutional neural network model comprises 13 convolutional layers, 3 fully-connected layers and 5 pooling layers.
8. The method according to claim 1, wherein in S8, an image with a low-brightness image type is enhanced by using a low-brightness image enhancement algorithm based on a corresponding camera model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010607913.5A CN111815529B (en) | 2020-06-30 | 2020-06-30 | Low-quality image classification enhancement method based on model fusion and data enhancement |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010607913.5A CN111815529B (en) | 2020-06-30 | 2020-06-30 | Low-quality image classification enhancement method based on model fusion and data enhancement |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111815529A CN111815529A (en) | 2020-10-23 |
| CN111815529B true CN111815529B (en) | 2023-02-07 |
Family
ID=72856666
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010607913.5A Active CN111815529B (en) | 2020-06-30 | 2020-06-30 | Low-quality image classification enhancement method based on model fusion and data enhancement |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111815529B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112541877B (en) * | 2020-12-24 | 2024-03-19 | 广东宜教通教育有限公司 | Defuzzification method, system, equipment and medium for generating countermeasure network based on condition |
| CN113469083A (en) * | 2021-07-08 | 2021-10-01 | 西安电子科技大学 | SAR image target classification method and system based on anti-sawtooth convolution neural network |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106650721A (en) * | 2016-12-28 | 2017-05-10 | 吴晓军 | Industrial character identification method based on convolution neural network |
| CN106875352A (en) * | 2017-01-17 | 2017-06-20 | 北京大学深圳研究生院 | A kind of enhancement method of low-illumination image |
| CN107169450A (en) * | 2017-05-15 | 2017-09-15 | 中国科学院遥感与数字地球研究所 | The scene classification method and system of a kind of high-resolution remote sensing image |
| CN108734667A (en) * | 2017-04-14 | 2018-11-02 | Tcl集团股份有限公司 | A kind of image processing method and system |
| CN109522945A (en) * | 2018-10-31 | 2019-03-26 | 中国科学院深圳先进技术研究院 | One kind of groups emotion identification method, device, smart machine and storage medium |
| CN109801224A (en) * | 2018-12-04 | 2019-05-24 | 北京奇艺世纪科技有限公司 | A kind of image processing method, device, server and storage medium |
| CN109934293A (en) * | 2019-03-15 | 2019-06-25 | 苏州大学 | Image-recognizing method, device, medium and obscure perception convolutional neural networks |
| CN110264424A (en) * | 2019-06-20 | 2019-09-20 | 北京理工大学 | A kind of fuzzy retinal fundus images Enhancement Method based on generation confrontation network |
| CN110428011A (en) * | 2019-08-06 | 2019-11-08 | 华南理工大学 | A kind of deep learning image fault classification method towards video transmission quality |
| CN110956201A (en) * | 2019-11-07 | 2020-04-03 | 江南大学 | Image distortion type classification method based on convolutional neural network |
| CN111127435A (en) * | 2019-12-25 | 2020-05-08 | 福州大学 | No-reference image quality evaluation method based on double-current convolutional neural network |
-
2020
- 2020-06-30 CN CN202010607913.5A patent/CN111815529B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106650721A (en) * | 2016-12-28 | 2017-05-10 | 吴晓军 | Industrial character identification method based on convolution neural network |
| CN106875352A (en) * | 2017-01-17 | 2017-06-20 | 北京大学深圳研究生院 | A kind of enhancement method of low-illumination image |
| CN108734667A (en) * | 2017-04-14 | 2018-11-02 | Tcl集团股份有限公司 | A kind of image processing method and system |
| CN107169450A (en) * | 2017-05-15 | 2017-09-15 | 中国科学院遥感与数字地球研究所 | The scene classification method and system of a kind of high-resolution remote sensing image |
| CN109522945A (en) * | 2018-10-31 | 2019-03-26 | 中国科学院深圳先进技术研究院 | One kind of groups emotion identification method, device, smart machine and storage medium |
| CN109801224A (en) * | 2018-12-04 | 2019-05-24 | 北京奇艺世纪科技有限公司 | A kind of image processing method, device, server and storage medium |
| CN109934293A (en) * | 2019-03-15 | 2019-06-25 | 苏州大学 | Image-recognizing method, device, medium and obscure perception convolutional neural networks |
| CN110264424A (en) * | 2019-06-20 | 2019-09-20 | 北京理工大学 | A kind of fuzzy retinal fundus images Enhancement Method based on generation confrontation network |
| CN110428011A (en) * | 2019-08-06 | 2019-11-08 | 华南理工大学 | A kind of deep learning image fault classification method towards video transmission quality |
| CN110956201A (en) * | 2019-11-07 | 2020-04-03 | 江南大学 | Image distortion type classification method based on convolutional neural network |
| CN111127435A (en) * | 2019-12-25 | 2020-05-08 | 福州大学 | No-reference image quality evaluation method based on double-current convolutional neural network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111815529A (en) | 2020-10-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109584248B (en) | Infrared target instance segmentation method based on feature fusion and dense connection network | |
| CN112233038B (en) | True image denoising method based on multi-scale fusion and edge enhancement | |
| CN111209952B (en) | Underwater target detection method based on improved SSD and migration learning | |
| EP3937481A1 (en) | Image display method and device | |
| CN107274445B (en) | Image depth estimation method and system | |
| CN111402146B (en) | Image processing method and image processing apparatus | |
| US10614736B2 (en) | Foreground and background detection method | |
| CN111754446A (en) | Image fusion method, system and storage medium based on generation countermeasure network | |
| CN111915525B (en) | Low-illumination image enhancement method capable of generating countermeasure network based on improved depth separation | |
| CN110866879B (en) | Image rain removing method based on multi-density rain print perception | |
| US10706558B2 (en) | Foreground and background detection method | |
| CN111260738A (en) | Multi-scale target tracking method based on relevant filtering and self-adaptive feature fusion | |
| CN110288550B (en) | Single-image defogging method for generating countermeasure network based on priori knowledge guiding condition | |
| CN110148088B (en) | Image processing method, image rain removing method, device, terminal and medium | |
| CN109903315B (en) | Method, apparatus, device and readable storage medium for optical flow prediction | |
| CN110532959B (en) | Real-time violent behavior detection system based on two-channel three-dimensional convolutional neural network | |
| CN111815529B (en) | Low-quality image classification enhancement method based on model fusion and data enhancement | |
| WO2010043771A1 (en) | Detecting and tracking objects in digital images | |
| CN112819858B (en) | Target tracking method, device, equipment and storage medium based on video enhancement | |
| CN111242026A (en) | Remote sensing image target detection method based on spatial hierarchy perception module and metric learning | |
| CN112115979A (en) | Fusion method and device of infrared image and visible image | |
| CN111582074A (en) | Monitoring video leaf occlusion detection method based on scene depth information perception | |
| CN113034417A (en) | Image enhancement system and image enhancement method based on generation countermeasure network | |
| CN115272437A (en) | Image depth estimation method and device based on global and local features | |
| Chen et al. | Visual depth guided image rain streaks removal via sparse coding |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |