CN114462558A - Data-augmented supervised learning image defect classification method and system - Google Patents

Data-augmented supervised learning image defect classification method and system Download PDF

Info

Publication number
CN114462558A
CN114462558A CN202210382999.5A CN202210382999A CN114462558A CN 114462558 A CN114462558 A CN 114462558A CN 202210382999 A CN202210382999 A CN 202210382999A CN 114462558 A CN114462558 A CN 114462558A
Authority
CN
China
Prior art keywords
image
layer
supervised learning
neural network
representing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210382999.5A
Other languages
Chinese (zh)
Inventor
郭波
张渴望
张建
谢云敏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanchang Institute of Technology
Original Assignee
Nanchang Institute of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanchang Institute of Technology filed Critical Nanchang Institute of Technology
Priority to CN202210382999.5A priority Critical patent/CN114462558A/en
Publication of CN114462558A publication Critical patent/CN114462558A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation

Abstract

The invention provides a method and a system for classifying defects of supervised learning images with data augmentation, wherein the method comprises the following steps: acquiring an image to be trained; inputting the image to be trained into an interested region feature extraction module to obtain an image interested feature region; constructing a data augmentation model, and performing data augmentation on the image interested characteristic region to obtain a data set after data augmentation; constructing a supervised learning neural network model, and training the supervised learning neural network model by using the data set after data augmentation; and (4) putting the image of the area to be predicted into the trained supervised learning neural network model for prediction to obtain an image classification result. The method has the advantages of small requirement on manual marking, good classification and identification performance, higher robustness and strong expandability.

Description

Data-augmented supervised learning image defect classification method and system
Technical Field
The invention relates to the technical field of computer image defect classification, in particular to a method and a system for classifying image defects through data augmentation and supervised learning.
Background
At present, for a deep learning image classification detection technology with better performance, large-scale manual labeling is often required to meet the training requirement of a high-precision model. However, manually annotating large amounts of data, resulting in long annotation times and inefficiencies, is impractical for certain application scenarios. In the traditional method for manually detecting the defects of the products, because workers work for a long time, the labor intensity is high, visual fatigue is easy to generate, and the defect detection quality is not uniform.
Deep learning based classification of defect images is a common defect detection technique for automatically identifying and detecting image defects. However, the accuracy of defect detection is often poor due to the fact that the defect image data sets required for neural network training are few or it is difficult to obtain sufficient defect data sets.
Disclosure of Invention
In view of the above-mentioned situation, the main objective of the present invention is to provide a method and a system for classifying defects in supervised learning images with augmented data to solve the above-mentioned technical problems.
The embodiment of the invention provides a data-augmented supervised learning image defect classification method, which comprises the following steps:
step one, obtaining an image to be trained;
inputting the image to be trained into an interested region feature extraction module to obtain an image interested feature region;
constructing a data augmentation model, and performing data augmentation on the image interested characteristic region to obtain a data set after data augmentation;
constructing a supervised learning neural network model, and training the supervised learning neural network model by using the data set after data augmentation;
and fifthly, placing the image of the area to be predicted into the trained supervised learning neural network model for prediction to obtain an image classification result.
The invention provides a supervised learning image defect classification method for data augmentation, which comprises the steps of firstly, obtaining an image to be trained; inputting an image to be trained into an interested region feature extraction module to obtain an image interested feature region, then constructing a data augmentation model, and performing data augmentation on the image interested feature region to obtain a data set after data augmentation; then, a supervised learning neural network model is constructed, and the supervised learning neural network model is trained by utilizing the data set after data augmentation; and finally, the image of the area to be predicted is put into the trained supervised learning neural network model for prediction to obtain an image classification result. After the defect images are extracted through the region-of-interest feature extraction module, the calculated amount can be reduced by utilizing the data augmentation neural network, and the problems of insufficient quantity and unbalance of different defect images are solved; in addition, the supervised learning neural network model increases the image negative value characteristic information and reduces the number of corresponding neural network layers, so that the fitting speed can be increased and the training time can be shortened. The method has the advantages of small requirement on manual marking, good classification and identification performance, higher robustness and strong expandability and data augmentation.
The method for classifying the defects of the supervised learning image with the augmented data comprises the following steps of:
inputting the image to be trained into an interested region feature extraction module for interested extraction to obtain a defect image;
and carrying out normalization processing on the defect image to obtain the interested characteristic region of the image.
In the third step, the method for constructing the data augmentation model and performing data augmentation on the feature region of interest of the image includes the following steps:
inputting the image interesting feature region into a generator model, and performing up-sampling on the image interesting feature region through a transposed convolution layer in the generator model to obtain an up-sampling defect image, wherein the generator model comprises the transposed convolution layer, a batch normalization layer and an activation function layer which are sequentially connected;
inputting the up-sampling defect image into a batch normalization layer for normalization processing to obtain a normalized up-sampling defect image;
performing function mapping on the normalized up-sampling defect image through the activation function layer to output a generated feature image from the generator model;
inputting the generated characteristic image into a discrimination model, and judging whether the generated characteristic image is consistent with a real defect image;
and if the generated characteristic image is consistent with the real defect image, outputting the generated characteristic image from the discrimination model.
The method for classifying the defects of the supervised learning image with the augmented data comprises the following steps of inputting the upsampled defect image into a batch normalization layer for normalization, wherein a corresponding normalization formula is as follows:
Figure 118992DEST_PATH_IMAGE001
wherein the content of the first and second substances,
Figure 387162DEST_PATH_IMAGE002
indicates input to
Figure 317072DEST_PATH_IMAGE003
Layer one
Figure 278075DEST_PATH_IMAGE004
The mean of the neuron numbers of individual neurons,
Figure 424760DEST_PATH_IMAGE005
is shown as
Figure 231042DEST_PATH_IMAGE003
Layer one
Figure 281038DEST_PATH_IMAGE006
The magnitude of the value of each of the neurons,
Figure 147362DEST_PATH_IMAGE007
representing batches of training data neurons
Figure 282809DEST_PATH_IMAGE008
The standard deviation of the degree of activation of (a),
Figure 892781DEST_PATH_IMAGE006
the number of the neuron is indicated by a sequence number,
Figure 797284DEST_PATH_IMAGE008
is shown as
Figure 100089DEST_PATH_IMAGE003
Layer one
Figure 713646DEST_PATH_IMAGE006
Magnitude of normalized values for individual neurons.
In the method for classifying the defects of the supervised learning image with the augmented data, the normalized upsampled defect image is subjected to function mapping through the activation function layer, so that a generated feature image is output from the generator model, and a corresponding activation function is represented as:
Figure 392889DEST_PATH_IMAGE009
wherein the content of the first and second substances,
Figure 151897DEST_PATH_IMAGE010
a first activation function is represented that is,
Figure 625604DEST_PATH_IMAGE011
to representA certain neuron node value in a specific layer of the neural network model,
Figure 470063DEST_PATH_IMAGE012
a random number representing a normal distribution of input neurons,
Figure 421839DEST_PATH_IMAGE013
the variance is expressed in terms of the number of peaks,
Figure 300933DEST_PATH_IMAGE014
representing the number of samples;
the discriminant formula corresponding to the discriminant model is expressed as:
Figure 945541DEST_PATH_IMAGE015
wherein the content of the first and second substances,
Figure 775831DEST_PATH_IMAGE016
representing minimizing the discriminant probability of generating a sample for the generator model,
Figure 531298DEST_PATH_IMAGE017
representing the maximum discriminant probability of the generated samples for the discriminant model,
Figure 999319DEST_PATH_IMAGE018
represents the cross-entropy loss of the binary function,
Figure 814829DEST_PATH_IMAGE019
the expected value of the true sample is represented,
Figure 633880DEST_PATH_IMAGE020
which represents the expected value of the generated sample,
Figure 193037DEST_PATH_IMAGE021
representing the probability of whether the real defect image is real or not,
Figure 781145DEST_PATH_IMAGE022
representation generator modelProbability of whether the generated feature image of the profile output is true,
Figure 501976DEST_PATH_IMAGE023
a generated feature image representing the output of the generator model,
Figure 306859DEST_PATH_IMAGE024
representing the noise of the input generator model.
The method for classifying the defects of the supervised learning image with the augmented data comprises the following steps of:
inputting the data set after data augmentation into a first layer in a supervised learning neural network model for convolution to obtain a second layer of image features and a third layer of image features;
fusing the second layer image features with the third layer image features to obtain fourth layer fused image features, and pooling the fourth layer fused image features to obtain fifth layer image features;
and performing convolution on the fifth layer image characteristics and the sixth layer image characteristics, pooling the seventh layer image characteristics, and finally completing the training of the supervised learning neural network model through a one-dimensional operation.
The method for classifying the defects of the supervised learning image with the augmented data comprises the following steps of inputting a data set with the augmented data into a first layer in a supervised learning neural network model for convolution to obtain a second layer image feature and a third layer image feature, wherein a formula for performing convolution operation is as follows:
Figure 404128DEST_PATH_IMAGE025
wherein the content of the first and second substances,
Figure 112321DEST_PATH_IMAGE026
representing position in input function image
Figure 4053DEST_PATH_IMAGE027
The gray-level value at the location of the location,
Figure 797697DEST_PATH_IMAGE028
representing a convolution kernel
Figure 698657DEST_PATH_IMAGE029
The magnitude of the value at the location of the position,
Figure 995777DEST_PATH_IMAGE030
representing the abscissa in the input function image,
Figure 323990DEST_PATH_IMAGE031
representing the ordinate in the input function image,
Figure 837886DEST_PATH_IMAGE032
representing the abscissa corresponding to the convolution kernel,
Figure 542537DEST_PATH_IMAGE033
representing the ordinate corresponding to the convolution kernel;
in the step of fusing the second-layer image features and the third-layer image features to obtain fourth-layer fused image features, a formula for performing feature fusion is represented as:
Figure 959743DEST_PATH_IMAGE034
wherein the content of the first and second substances,
Figure 458857DEST_PATH_IMAGE035
representing the fused image characteristics after the fusion of the cascade function,
Figure 961514DEST_PATH_IMAGE036
which represents a vector concatenation operation, is shown,
Figure 204276DEST_PATH_IMAGE037
a second activation function is represented that is,
Figure 475989DEST_PATH_IMAGE038
feature information representing a convolution image output from a second layer in the supervised learning neural network model,
Figure 146005DEST_PATH_IMAGE039
feature information representing the convolution image output by the third layer in the supervised learning neural network model,
Figure 634492DEST_PATH_IMAGE040
representing a global average pooling operation.
The supervised learning neural network model comprises a plurality of supervised learning neural network layers, and the calculation formula of each neuron in the supervised learning neural network layers is expressed as:
Figure 415367DEST_PATH_IMAGE041
wherein the content of the first and second substances,
Figure 807165DEST_PATH_IMAGE042
is shown as
Figure 648082DEST_PATH_IMAGE043
Layer one
Figure 125331DEST_PATH_IMAGE004
The magnitude of the value of each of the neurons,
Figure 709896DEST_PATH_IMAGE044
is shown as
Figure 956201DEST_PATH_IMAGE045
In a layer of
Figure 968019DEST_PATH_IMAGE046
Of individual neuronThe size of the numerical value is,
Figure 899941DEST_PATH_IMAGE047
is shown as
Figure 288197DEST_PATH_IMAGE045
In a layer of
Figure 389008DEST_PATH_IMAGE046
The nerve cell and the first
Figure 571727DEST_PATH_IMAGE048
The weights are connected to the layer neurons,
Figure 23568DEST_PATH_IMAGE049
which represents a non-linear activation function,
Figure 949936DEST_PATH_IMAGE050
is shown as
Figure 170833DEST_PATH_IMAGE045
The total number of neurons in a layer.
The method for classifying the defects of the supervised learning image with the augmented data comprises the following steps of training a supervised learning neural network model, wherein the training of the supervised learning neural network model comprises forward propagation and backward propagation, the forward propagation of the supervised learning neural network model is completed through a convolutional layer and a pooling layer, the supervised learning neural network model carries out multi-class cross entropy function calculation through the backward propagation, and the expression of the multi-class cross entropy function is as follows:
Figure 790033DEST_PATH_IMAGE051
wherein the content of the first and second substances,
Figure 696547DEST_PATH_IMAGE052
represents a multi-class cross-entropy function,
Figure 426606DEST_PATH_IMAGE053
representing a sample
Figure 236430DEST_PATH_IMAGE054
The cross-entropy function of (a) is,
Figure 292111DEST_PATH_IMAGE055
indicating the number of categories that are required,
Figure 452965DEST_PATH_IMAGE056
has a value of 0 or 1; when the sample is
Figure 721135DEST_PATH_IMAGE054
The true class of
Figure 385466DEST_PATH_IMAGE057
Taking 1, otherwise, taking 0;
Figure 346468DEST_PATH_IMAGE058
representing a sample
Figure 493154DEST_PATH_IMAGE054
Belong to the category
Figure 565015DEST_PATH_IMAGE059
The prediction function of (a) is determined,
Figure 615010DEST_PATH_IMAGE060
representing the number of samples;
the formula of the weight update corresponding to the multi-classification cross entropy function is represented as:
Figure 215756DEST_PATH_IMAGE061
wherein the content of the first and second substances,
Figure 616782DEST_PATH_IMAGE062
which represents the weight update learning rate,
Figure 961175DEST_PATH_IMAGE063
indicating the updated weight value of the weight value,
Figure 131257DEST_PATH_IMAGE064
which represents the current weight value of the current weight,
Figure 902903DEST_PATH_IMAGE065
representing a sample
Figure 24181DEST_PATH_IMAGE054
The prediction function of (2):
weight update learning rate
Figure 437845DEST_PATH_IMAGE066
Expressed as:
Figure 196853DEST_PATH_IMAGE067
。
the invention also provides a system for classifying the defects of the supervised learning image with augmented data, wherein the system comprises:
the image acquisition module is used for acquiring an image to be trained;
the interest extraction module is used for inputting the image to be trained into the interest region feature extraction module to obtain an image interest feature region;
the data augmentation module is used for constructing a data augmentation model and augmenting the interested characteristic region of the image to obtain a data set after data augmentation;
the model training module is used for constructing a supervised learning neural network model and training the supervised learning neural network model by using the data set after data augmentation;
and the result output module is used for putting the image of the area to be predicted into the trained supervised learning neural network model for prediction so as to obtain an image classification result.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
FIG. 1 is a flowchart illustrating a method for classifying defects in supervised learning images with augmented data according to a first embodiment of the present invention;
FIG. 2 is a flowchart illustrating a method for classifying defects in supervised learning images with augmented data according to a second embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a supervised learning neural network model according to a second embodiment of the present invention;
FIG. 4 is a diagram illustrating a comparison of the structure of a supervised learning neural network model, AlexNet and VGG16 neural network in an embodiment of the present invention;
FIG. 5 is a graph comparing the accuracy of a training with data augmentation and the accuracy of a test without data augmentation according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a system for classifying defects in supervised learning images with augmented data according to a third embodiment of the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.
These and other aspects of embodiments of the invention will be apparent with reference to the following description and attached drawings. In the description and drawings, particular embodiments of the invention have been disclosed in detail as being indicative of some of the ways in which the principles of the embodiments of the invention may be practiced, but it is understood that the scope of the embodiments of the invention is not limited correspondingly. On the contrary, the embodiments of the invention include all changes, modifications and equivalents coming within the spirit and terms of the claims appended hereto.
Referring to fig. 1, the present invention provides a method for classifying defects of supervised learning images with data augmentation, wherein the method includes the following steps:
and S101, acquiring an image to be trained.
Specifically, the image to be trained generally refers to an image captured by a camera, and the image captured by the camera in daily life is generally a color RGB image. The RGB image is composed of three channels R, G, B (red, green, blue), and each channel is set to 256 values, i.e., 0 to 255, according to the range that can be recognized by human eyes.
S102, inputting the image to be trained into an interested region feature extraction module to obtain an image interested feature region.
Specifically, the region of interest (ROI) refers to a region to be processed, which is delineated from a processed image in a manner of a square frame, a circle, an ellipse, an irregular polygon, or the like in machine vision and image processing, and is also a region where a target object is located. In specific implementation, the region of interest can be manually circled with a specific identifier, such as frame selection; and scanning and matching the original color image by utilizing the template image of the target object so as to find an image area with the similarity degree higher than a threshold value with the template image of the target object in the original color image. The image area is an interested area, so that the interested area is automatically extracted; or the automatic identification of the region of interest of the original color image can be carried out based on the trained neural network model.
In this step, the method for inputting the image to be trained to the interested region feature extraction module to obtain the interested feature region of the image includes the following steps:
s1021, inputting the image to be trained into an interested region feature extraction module for interested extraction to obtain a defect image;
s1022, normalizing the defect image to obtain the interested characteristic region of the image.
S103, constructing a data augmentation model, and performing data augmentation on the image interested characteristic region to obtain a data set after data augmentation.
Machine learning allows for improved generalization performance of models by collecting more trainable data in order to achieve recognition with sufficient accuracy. The GAN generation of the antagonistic neural network is a two-person zero-sum game idea (two-player game), two models of a G (Generator model) network and a D (discrimination model) network are trained simultaneously, and after multiple times of antagonistic adjustment, the two models reach Nash balance (Nash equilibrium). After training, the anti-neural network can efficiently and accurately generate a characteristic image, and the integrity of the characteristics of the defect image is ensured as much as possible. In the field of data augmentation, GAN generation has great advantages for antagonistic neural networks.
In the present invention, the method for generating an input image matrix (image interesting feature region) input into a data augmentation model includes the steps of:
generating a random noise image, wherein the size of the random noise image is equal to that of an actual image, and the size of the random noise image is row col, wherein row represents the number of image lines, and col represents the number of image columns;
combining the random noise image with the actual image, wherein the specific combination mode comprises the following steps: respectively reading pixel gray values at the same positions of the random noise image and the actual image, summing the read pixel gray values and averaging to obtain an average value, putting the average value into an image matrix at a corresponding image position, and ending at an image position (row, col) to finally obtain an input image matrix of the data augmentation model.
The image matrix size is row × col, row represents the number of image rows, and col represents the number of image columns. It should be added that, by the above arrangement, the fitting speed of the data augmentation model can be increased, and the training time of the data augmentation model can be reduced.
Further, the input image matrix generated as described above is input to GAN to generate a G (Generator model) network against a neural network model. The Generator model is built by a five-layer neural network, and the first layer adopts transposed convolution-stridgeConv 2D for up-sampling. Meanwhile, in order to prevent gradient disappearance in the convolution process and accelerate the convergence speed of the model, a batch normalization layer Bacth normalization is added. Then, a full connection layer Dense and the first three layers of Francination-stridgeConv 2D transpose convolution layers, each layer adopts the form of an activation function Leaky Relu to carry out function mapping, and then a characteristic defect image is output from a Generator model. And (2) judging the image by entering a D (discrimination model) network and a real defect image, completing construction of a Discriminator model by a three-layer neural network, similarly adopting a form of a Leaky Relu of each layer of activation function to improve the generalization capability of the model, and outputting an image when the generated characteristic image is judged to be consistent with the defect image (see figure 3).
Specifically, in this step, a data augmentation model is constructed, and a method for augmenting the data of the image interested feature region includes the following steps:
and S1031, inputting the image interesting feature region into a generator model, and performing up-sampling on the image interesting feature region through a transposed convolution layer in the generator model to obtain an up-sampling defect image, wherein the generator model comprises the transposed convolution layer, a batch normalization layer and an activation function layer which are sequentially connected.
S1032, inputting the up-sampling defect image into a batch normalization layer for normalization processing to obtain a normalized up-sampling defect image.
In this step, in order to prevent the gradient from disappearing and accelerate the model convergence speed, a batch normalization layer backnormalization is added for normalization, and the corresponding normalization formula is as follows:
Figure 670560DEST_PATH_IMAGE001
wherein the content of the first and second substances,
Figure 515019DEST_PATH_IMAGE002
indicates input to
Figure 466795DEST_PATH_IMAGE003
Layer one
Figure 345889DEST_PATH_IMAGE004
The mean of the neuron numbers of individual neurons,
Figure 724918DEST_PATH_IMAGE005
denotes the first
Figure 555208DEST_PATH_IMAGE003
Layer one
Figure 310675DEST_PATH_IMAGE006
The magnitude of the value of each of the neurons,
Figure 44275DEST_PATH_IMAGE007
representing batches of training data neurons
Figure 859785DEST_PATH_IMAGE005
The standard deviation of the degree of activation of (a),
Figure 678836DEST_PATH_IMAGE006
the number of the neuron is shown as a number,
Figure 972414DEST_PATH_IMAGE008
denotes the first
Figure 826101DEST_PATH_IMAGE003
Layer one
Figure 812511DEST_PATH_IMAGE006
Magnitude of normalized values for individual neurons.
And S1033, performing function mapping on the normalized up-sampling defect image through the activation function layer to output a generated characteristic image from the generator model.
In this step, the corresponding activation function is represented as:
Figure 351815DEST_PATH_IMAGE009
wherein the content of the first and second substances,
Figure 449084DEST_PATH_IMAGE010
a first activation function is represented that is,
Figure 891698DEST_PATH_IMAGE011
represents a certain neuron node value in a specific layer of the neural network model,
Figure 314589DEST_PATH_IMAGE013
the variance is represented as a function of time,
Figure 108232DEST_PATH_IMAGE012
a random number representing a normal distribution of input neurons,
Figure 743613DEST_PATH_IMAGE013
the variance is represented as a function of time,
Figure 40733DEST_PATH_IMAGE014
representing the number of samples.
S1034, inputting the generated characteristic image into a discrimination model, and judging whether the generated characteristic image is consistent with the real defect image.
Wherein, the discriminant formula corresponding to the discriminant model is expressed as:
Figure 368946DEST_PATH_IMAGE015
wherein the content of the first and second substances,
Figure 148421DEST_PATH_IMAGE016
representing minimizing the discriminant probability of generating a sample for the generator model,
Figure 853072DEST_PATH_IMAGE017
representing the maximum discriminant probability of the generated samples for the discriminant model,
Figure 270278DEST_PATH_IMAGE018
represents the cross-entropy loss of the binary function,
Figure 503813DEST_PATH_IMAGE019
the expected value of the true sample is represented,
Figure 6470DEST_PATH_IMAGE020
which represents the expected value of the generated sample,
Figure 983653DEST_PATH_IMAGE021
representing the probability of whether the real defect image is real or not,
Figure 520945DEST_PATH_IMAGE022
representing the probability of whether the generated feature image output by the generator model is authentic,
Figure 190961DEST_PATH_IMAGE023
a generated feature image representing the output of the generator model,
Figure 679449DEST_PATH_IMAGE024
representing the noise of the input generator model.
And S1035, if the generated characteristic image is consistent with the real defect image, outputting the generated characteristic image from the discriminant model.
And S104, constructing a supervised learning neural network model, and training the supervised learning neural network model by using the data set after data augmentation.
In this step, a supervised learning neural network model is constructed, and the method for training the supervised learning neural network model by using the data set after data augmentation comprises the following steps:
and S1041, inputting the data set after the data augmentation into a first layer in a supervised learning neural network model for convolution to obtain a second layer image feature and a third layer image feature.
The formula for performing the convolution operation is expressed as:
Figure 460323DEST_PATH_IMAGE025
wherein the content of the first and second substances,
Figure 852121DEST_PATH_IMAGE026
representing position in input function image
Figure 693038DEST_PATH_IMAGE027
The gray-level value at the location of the location,
Figure 170287DEST_PATH_IMAGE028
representing a convolution kernel
Figure 426956DEST_PATH_IMAGE029
The magnitude of the value at the location of the position,
Figure 797894DEST_PATH_IMAGE030
representing the abscissa in the input function image,
Figure 183614DEST_PATH_IMAGE031
representing the ordinate in the input function image,
Figure 7214DEST_PATH_IMAGE032
representing the abscissa corresponding to the convolution kernel,
Figure 270836DEST_PATH_IMAGE033
representing the ordinate to which the convolution kernel corresponds. Note that, in the first layer, the Input data (150 × 3) obtained by amplifying the Input data has a 3-channel 150 × 150 pixel matrix.
S1042, fusing the second layer image features and the third layer image features to obtain fourth layer fused image features, and pooling the fourth layer fused image features to obtain fifth layer image features.
In the fourth layer of the supervised learning neural network model, the formula for feature fusion is expressed as:
Figure 496281DEST_PATH_IMAGE034
wherein the content of the first and second substances,
Figure 554367DEST_PATH_IMAGE035
representing the fused image characteristics after the fusion of the cascade function,
Figure 130842DEST_PATH_IMAGE036
which represents a vector concatenation operation, is shown,
Figure 932575DEST_PATH_IMAGE037
it is shown that the second activation function is,
Figure 12527DEST_PATH_IMAGE038
feature information representing a convolution image output by the second layer in the supervised learning neural network model,
Figure 5628DEST_PATH_IMAGE039
feature information representing the convolution image output by the third layer in the supervised learning neural network model,
Figure 803820DEST_PATH_IMAGE040
representing a global average pooling operation.
It should be noted here that for the second activation function described above
Figure 143666DEST_PATH_IMAGE037
The specific expression is as follows:
Figure 343703DEST_PATH_IMAGE068
wherein the content of the first and second substances,
Figure 9171DEST_PATH_IMAGE069
represents a certain neuron node value in a specific layer of the neural network model,
Figure 29079DEST_PATH_IMAGE070
representing a neuron node activation value in a particular layer of the neural network model.
In the invention, a supervised learning neural network model firstly adopts the steps of sliding from left to right and from top to bottom according to Stride and carrying out convolution calculation with a 3 x 3 convolution kernel (filter) on the original information of an input layer of a visual perception area to obtain a mapping output feature map (figure map)）I*gThen comparing the output characteristic diagramI*gTaking the negative value to obtain a characteristic diagram-I*g. Wherein the content of the first and second substances,gin the form of a filter or convolution kernel,Iand outputting the characteristic diagram of the previous layer. In the third layer of the supervised learning neural network model, the neural network will output a feature map in this embodimentI*gAnd get the negative characteristic diagram-I*gAnd transmitting the data to a next-stage neuron, and respectively acting a functional relation on each node, namely adopting the second activation function to map the characteristic information of each node.
And S1043, performing convolution on the fifth layer image characteristic and the sixth layer image characteristic, pooling the seventh layer image characteristic, and finally completing training of the supervised learning neural network model through a one-dimensional operation.
In this embodiment, the supervised learning neural network model includes a plurality of supervised learning neural network layers, and the calculation formula of each neuron in the supervised learning neural network layers is represented as:
Figure 172616DEST_PATH_IMAGE041
wherein the content of the first and second substances,
Figure 227159DEST_PATH_IMAGE042
is shown as
Figure 296484DEST_PATH_IMAGE043
Layer one
Figure 69268DEST_PATH_IMAGE004
The magnitude of the value of each of the neurons,
Figure 16496DEST_PATH_IMAGE044
is shown as
Figure 925546DEST_PATH_IMAGE045
In a layer of
Figure 932816DEST_PATH_IMAGE046
The magnitude of the value of each of the neurons,
Figure 192896DEST_PATH_IMAGE047
is shown as
Figure 943815DEST_PATH_IMAGE045
In a layer of
Figure 972950DEST_PATH_IMAGE046
The nerve cell and the first
Figure 220015DEST_PATH_IMAGE048
The weights are connected to the layer neurons,
Figure 967392DEST_PATH_IMAGE049
which represents a non-linear activation function,
Figure 256422DEST_PATH_IMAGE050
is shown as
Figure 140064DEST_PATH_IMAGE045
The total number of neurons in a layer.
Further, in the present invention, the training of the supervised learning neural network model includes forward propagation and backward propagation. The forward propagation of the supervised learning neural network model is completed through a convolutional layer and a pooling layer, the supervised learning neural network model performs multi-class cross entropy function calculation through backward propagation, and the expression of the multi-class cross entropy function is as follows:
Figure 223558DEST_PATH_IMAGE051
wherein the content of the first and second substances,
Figure 192651DEST_PATH_IMAGE052
represents a multi-class cross-entropy function,
Figure 19792DEST_PATH_IMAGE053
representing a sample
Figure 23520DEST_PATH_IMAGE054
The cross-entropy function of (a) is,
Figure 42030DEST_PATH_IMAGE055
indicating the number of categories that are required,
Figure 232840DEST_PATH_IMAGE056
has a value of 0 or 1; when the sample is
Figure 129251DEST_PATH_IMAGE054
The true class of
Figure 721907DEST_PATH_IMAGE057
Taking 1, otherwise, taking 0;
Figure 678362DEST_PATH_IMAGE058
representing a sample
Figure 90888DEST_PATH_IMAGE054
Belong to the category
Figure 790991DEST_PATH_IMAGE059
The prediction function of (a) is determined,
Figure 238153DEST_PATH_IMAGE060
representing the number of samples.
The formula of the weight update corresponding to the multi-classification cross entropy function is represented as:
Figure 598465DEST_PATH_IMAGE061
wherein the content of the first and second substances,
Figure 29446DEST_PATH_IMAGE062
which represents the weight update learning rate,
Figure 2081DEST_PATH_IMAGE063
indicating the updated weight value of the weight value,
Figure 569329DEST_PATH_IMAGE064
which represents the current weight value of the current weight,
Figure 602007DEST_PATH_IMAGE065
representing a sample
Figure 254705DEST_PATH_IMAGE054
The prediction function of (2):
weight update learning rate
Figure 31031DEST_PATH_IMAGE066
Expressed as:
Figure 718365DEST_PATH_IMAGE067
。
it should be added that the Adam optimizer is mainly used for updating the value of the weight update learning rate, but if the parameter value is greater than 1, explosion of model parameter update occurs, and therefore needs to be avoided in the model training process. Meanwhile, too small a parameter value may reduce update efficiency, and thus various conditions need to be defined.
In the present invention, the details of the implementation of step S104 are as follows:
first layer in supervised learning neural network model
Figure 420479DEST_PATH_IMAGE071
Obtaining the second layer in the supervised learning neural network model by convolution after acquiring the image data
Figure 560474DEST_PATH_IMAGE072
Third layer in supervised learning neural network model
Figure 874911DEST_PATH_IMAGE073
：
Figure 416751DEST_PATH_IMAGE072
:Conv2D(3*3*16)-m_tanh-strides=3；
Figure 791232DEST_PATH_IMAGE073
:DeConv2D(3*3*16)-m_tanh-strides=3；
To pair
Figure 418522DEST_PATH_IMAGE072
A layer,
Figure 536651DEST_PATH_IMAGE073
The layers are fused to obtain
Figure 198576DEST_PATH_IMAGE074
:
Add（
Figure 242494DEST_PATH_IMAGE072
，
Figure 91501DEST_PATH_IMAGE073
）；
To pair
Figure 13320DEST_PATH_IMAGE074
Making them pass through a pool to obtain
Figure 529752DEST_PATH_IMAGE075
:
MaxPooling2D（3*3）；
To pair
Figure 246036DEST_PATH_IMAGE075
Performing convolution to obtain
Figure 582339DEST_PATH_IMAGE076
：
Conv2D(3*3*32)-m_tanh-strides=3；
To pair
Figure 42270DEST_PATH_IMAGE076
Performing convolution to obtain
Figure 413209DEST_PATH_IMAGE077
：
Conv2D(3*3*32)-m_tanh-strides=3；
To pair
Figure 798929DEST_PATH_IMAGE077
Making them pass through a pool to obtain
Figure 888107DEST_PATH_IMAGE078
:
MaxPooling2D（3*3）；
To pair
Figure 682888DEST_PATH_IMAGE078
Performing 1-dimensional operation to obtain
Figure 377175DEST_PATH_IMAGE079
:
Flatten(x8)；
To pair
Figure 231998DEST_PATH_IMAGE079
Performing full connection operation to obtain
Figure 418260DEST_PATH_IMAGE080
:
Dense(256)-m_tanh；
To pair
Figure 344628DEST_PATH_IMAGE080
Performing full connection operation to obtain
Figure 64060DEST_PATH_IMAGE081
:
Dense(256)-m_tanh；
To pair
Figure 417681DEST_PATH_IMAGE081
Performing full connection operation to obtain
Figure 91239DEST_PATH_IMAGE082
:
Dense(64)-m_tanh；
To pair
Figure 821297DEST_PATH_IMAGE082
Performing full connection operation to obtain
Figure 896701DEST_PATH_IMAGE083
:
Dense(1)-sigmoid；
Where m _ tanh represents the activation function, sigmoid represents the sigmoid function, strides =3 represents a step size of 3, Conv2D (a × b × c) represents the convolution layer with convolution kernel a × b × c, where a × b represents the convolution kernel size and c represents the number of convolution kernels; DeConv2D (a × b × c) indicates that the convolution kernel is a × b × c, and the convolution layer is overall negative in value, i.e., -Conv2D (a × b × c), where a × b represents the convolution kernel size and c represents the number of convolution kernels; MaxPooling2D (3 × 3) represents the largest pooling layer, where 3 × 3 represents the pooling window; dense (64) represents the fully-connected layer, where 64 is the number of neurons in the fully-connected layer and Flatten () represents a one-dimensional operation.
And S105, placing the image of the area to be predicted into the trained supervised learning neural network model for prediction to obtain an image classification result.
The invention provides a supervised learning image defect classification method for data augmentation, which comprises the steps of firstly, obtaining an image to be trained; inputting an image to be trained into an interested region feature extraction module to obtain an image interested feature region, then constructing a data augmentation model, and performing data augmentation on the image interested feature region to obtain a data set after data augmentation; then, a supervised learning neural network model is constructed, and the supervised learning neural network model is trained by utilizing the data set after data augmentation; and finally, the image of the area to be predicted is put into the trained supervised learning neural network model for prediction to obtain an image classification result. After the defect images are extracted through the region-of-interest feature extraction module, the calculated amount can be reduced by utilizing the data augmentation neural network, and the problems of insufficient quantity and unbalance of different defect images are solved; in addition, the supervised learning neural network model increases the image negative value characteristic information and reduces the number of corresponding neural network layers, so that the fitting speed can be increased and the training time can be shortened. The method has the advantages of small requirement on manual marking, good classification and identification performance, higher robustness and strong expandability and data augmentation.
Referring to fig. 2, in a second embodiment of the present invention, the method is described in detail by taking defect classification of the welding image as an example, and specifically includes steps S201 to S204:
s201, acquiring a welding original image.
S202, extracting the welding original image through an interested area feature extraction module to obtain a welding defect area.
The welding defect Region is preprocessed, the welding image obtained by the wide dynamic sensor is utilized, and a Region of interest (ROI), namely the welding defect Region, is extracted.
Specifically, a proper threshold value is set for binarization operation, mean value filtering and opening operation modes are sequentially carried out to serve as the basis of relevance operation, then the gray level relation between a specific pixel point and surrounding pixel points is converted, and finally a welding defect area is extracted, so that the image processing difficulty is reduced, and the computer operation time is reduced. Furthermore, normalization operation is carried out on the extracted images, and further training preparation is carried out on the input supervised learning neural network. Additionally, the normalization operation has already been explained in the first embodiment, and is not described herein again.
And step S203, constructing a data augmentation model, and performing data augmentation on the welding defect area to obtain a welding defect data set after data augmentation.
Referring to fig. 3, when the GAN generates an antagonistic neural network model, and the image matrix is input into the GAN to generate a G (Generator model) network of the antagonistic neural network, the Generator model is constructed by a five-layer neural network, and upsampling is performed using a transposed convolution-gradient 2D. Meanwhile, in order to prevent gradient disappearance and accelerate the model convergence speed, a batch normalization layer Bacth normalization is added. The fully connected layer Dense, and the first three Francionlenv 2D layers are transposed into convolutional layers, each layer taking the form of the Activate function Leaky Relu.
And then, outputting an image with welding defect characteristics from the Generator model, judging the image with the welding defect characteristics in a D (Discrimization model) network, completing the construction of the Discrimizer model by a three-layer neural network, and similarly adopting the form of a Leaky Relu of each layer of activation function to improve the generalization capability of the model. Only when the generated characteristic image is judged to be consistent with the welding defect image, the image is output.
And S204, training the supervised learning neural network model by using the welding defect data set after data augmentation so as to realize image defect classification.
In this embodiment, the supervised learning neural network model is far less deep than two conventional models, compared with AlexNet and VGG 16. When a deep neural network is used, the number of layers of the network is large, and a large number of image elements can be carried, so that more complex data relation mapping can be realized.
Meanwhile, too high number of layers often causes problems of over-fitting, under-fitting, gradient disappearance, gradient explosion, and the like. However, the supervised learning neural network model avoids the disadvantages caused by an excessively high number of layers to a certain extent, the number of all parameters (total parameters) or trainable parameters (trainable parameters) is greatly reduced, the operation amount is reduced, and the accuracy of model defect classification is considered (see fig. 4).
And in the process of constructing the welding defect detection framework, the neural network for effectively classifying the welding defect image is realized. Generally, a feature map obtained by a convolutional neural network usually ignores learning of negative value features, a sigmoid (x) function is used for feature mapping operation, a derivative value of back propagation is calculated quickly by using the formula according to the characteristics of the mapping relation, and the back propagation is realized quickly.
Further, the defect classification effect of the supervised learning neural network model is shown in fig. 5, and as the number of operation cycles increases, the accuracy of data augmentation performed by using the GAN generation antagonistic neural network is greatly improved compared with that without data augmentation, and stable detection accuracy is achieved faster than that without data augmentation. When the operation period is about 40 times, the detection precision of the supervised learning neural network model is greatly improved and is relatively kept stable.
In the invention, the weld image defects are classified by the supervised learning neural network model obtained by pre-training, and the accuracy comparison is carried out by adopting known traditional models such as AlexNet, VGG16 and the like. After the comparison model and the supervised learning neural network model are subjected to fine tuning by adopting 1% of test data after training is finished, the supervised learning neural network model provided by the invention has higher accuracy which reaches 96.15%.
The time required by each neural network model in a single training process is taken as the vertical axis, and the time spent by each training of the model, the AlexNet model and the VGG16 in 50 training processes is plotted into a box chart for comparison. The running time range of the model is much more stable than AlexNet and VGG16 according to the length of the box body. The model takes the shortest time for each training on average and has the most stable time distribution. The most uniform operation time distribution of the model can be seen from the middle bit line.
In order to verify the generalization capability of the model provided by the invention, a handwritten character recognition system (MNIST data set) of the American post office is selected to carry out a model generalization effect test. The data set was formed by collecting data sets of handwritten numerical data of residents of the united states, which contained a training set of 60000 examples and a test set of 10000 examples in total. The fact proves that: the data set is well applied to a supervised learning neural network model and is suitable for deep learning neural network classification. The accuracy of the model is higher classification level, 98.05%, when the MNIST data set is classified and compared by the supervised learning neural network model.
In conclusion, the method can reduce the calculated amount and solve the problems of insufficient quantity and unbalance of different defect images by utilizing the data augmented neural network after the ROI defect images are extracted; the supervised learning neural network model increases image negative value characteristic information, reduces the number of corresponding neural network layers, and can accelerate the fitting rate and reduce the training time; the method has the advantages of small requirement on manual marking, good classification and identification performance, higher robustness and strong expandability and data augmentation.
Referring to fig. 6, a third embodiment of the present invention further provides a system for classifying defects in supervised learning images with augmented data, wherein the system includes:
the image acquisition module is used for acquiring an image to be trained;
the interest extraction module is used for inputting the image to be trained into the interest region feature extraction module to obtain an image interest feature region;
the data augmentation module is used for constructing a data augmentation model and augmenting the interested characteristic region of the image to obtain a data set after data augmentation;
the model training module is used for constructing a supervised learning neural network model and training the supervised learning neural network model by using the data set after data augmentation;
and the result output module is used for putting the image of the area to be predicted into the trained supervised learning neural network model for prediction so as to obtain an image classification result.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent should be subject to the appended claims.

Claims (10)

1. A method for classifying image defects through data augmentation and supervised learning is characterized by comprising the following steps:
step one, obtaining an image to be trained;
inputting the image to be trained into an interested region feature extraction module to obtain an image interested feature region;
constructing a data augmentation model, and performing data augmentation on the image interested characteristic region to obtain a data set after data augmentation;
constructing a supervised learning neural network model, and training the supervised learning neural network model by using the data set after data augmentation;
and fifthly, placing the image of the area to be predicted into the trained supervised learning neural network model for prediction to obtain an image classification result.
2. The method for classifying image defects through data augmentation and supervised learning as recited in claim 1, wherein in the second step, the method for inputting the image to be trained to the region-of-interest feature extraction module to obtain the image feature-of-interest region comprises the following steps:
inputting the image to be trained into an interested region feature extraction module for interested extraction to obtain a defect image;
and carrying out normalization processing on the defect image to obtain the interested characteristic region of the image.
3. The method for classifying the image defects through data augmentation and supervised learning as recited in claim 1, wherein in the third step, the method for constructing the data augmentation model and performing data augmentation on the image interest feature region comprises the following steps:
inputting the image interesting feature region into a generator model, and performing up-sampling on the image interesting feature region through a transposed convolution layer in the generator model to obtain an up-sampling defect image, wherein the generator model comprises the transposed convolution layer, a batch normalization layer and an activation function layer which are sequentially connected;
inputting the up-sampling defect image into a batch normalization layer for normalization processing to obtain a normalized up-sampling defect image;
performing function mapping on the normalized up-sampling defect image through the activation function layer to output a generated feature image from the generator model;
inputting the generated characteristic image into a discrimination model, and judging whether the generated characteristic image is consistent with a real defect image;
and if the generated characteristic image is consistent with the real defect image, outputting the generated characteristic image from the discrimination model.
4. The method according to claim 3, wherein the step of inputting the upsampled defect image into a batch normalization layer for normalization processing comprises a corresponding normalization formula:
Figure 590389DEST_PATH_IMAGE001
wherein the content of the first and second substances,
Figure 835426DEST_PATH_IMAGE002
indicates input to the first
Figure 84005DEST_PATH_IMAGE003
Layer one
Figure 338269DEST_PATH_IMAGE004
The mean of the neuron numbers of individual neurons,
Figure 621482DEST_PATH_IMAGE005
denotes the first
Figure 37420DEST_PATH_IMAGE003
Layer one
Figure 773295DEST_PATH_IMAGE006
The magnitude of the value of each of the neurons,
Figure 565670DEST_PATH_IMAGE007
representing batches of training data neurons
Figure 703391DEST_PATH_IMAGE005
The standard deviation of the degree of activation of (a),
Figure 290230DEST_PATH_IMAGE006
the number of the neuron is shown as a number,
Figure 513401DEST_PATH_IMAGE008
is shown as
Figure 846818DEST_PATH_IMAGE003
Layer one
Figure 839045DEST_PATH_IMAGE006
The magnitude of the normalized values of the individual neurons.
5. The method according to claim 4, wherein in the step of performing function mapping on the normalized upsampled defect image through the activation function layer to output the generated feature image from the generator model, the corresponding activation function is represented as:
Figure 862364DEST_PATH_IMAGE009
wherein the content of the first and second substances,
Figure 572831DEST_PATH_IMAGE010
a first activation function is represented that is,
Figure 441430DEST_PATH_IMAGE011
represents a certain neuron node value in a specific layer of the neural network model,
Figure 553743DEST_PATH_IMAGE012
a random number representing a normal distribution of input neurons,
Figure 482384DEST_PATH_IMAGE013
the variance is represented as a function of time,
Figure 680147DEST_PATH_IMAGE014
representing the number of samples;
the discriminant formula corresponding to the discriminant model is expressed as:
Figure 352437DEST_PATH_IMAGE015
wherein the content of the first and second substances,
Figure 319256DEST_PATH_IMAGE016
representing minimizing the discriminant probability of generating a sample for the generator model,
Figure 684379DEST_PATH_IMAGE017
representation-to-discriminant model maximization generation sampleThe probability of the discrimination of the present invention,
Figure 103859DEST_PATH_IMAGE018
represents the cross-entropy loss of the binary function,
Figure 579839DEST_PATH_IMAGE019
a desired value that represents a true sample of,
Figure 401165DEST_PATH_IMAGE020
which represents the expected value of the generated sample,
Figure 937188DEST_PATH_IMAGE021
representing the probability of whether the real defect image is real or not,
Figure 843964DEST_PATH_IMAGE022
representing the probability of whether the generated feature image output by the generator model is authentic,
Figure 855127DEST_PATH_IMAGE023
a generated feature image representing the output of the generator model,
Figure 796538DEST_PATH_IMAGE024
representing the noise of the input generator model.
6. The method for classifying defects of supervised learning image with augmented data as claimed in claim 5, wherein in the fourth step, a supervised learning neural network model is constructed, and the method for training the supervised learning neural network model by using the data set after augmented data comprises the following steps:
inputting the data set after data augmentation into a first layer in a supervised learning neural network model for convolution to obtain a second layer of image features and a third layer of image features;
fusing the second layer image features with the third layer image features to obtain fourth layer fused image features, and pooling the fourth layer fused image features to obtain fifth layer image features;
and performing convolution on the fifth-layer image features and the sixth-layer image features, pooling the seventh-layer image features, and finally completing training on the supervised learning neural network model through a one-dimensional operation.
7. The method according to claim 6, wherein in the step of inputting the data-augmented data set into the first layer of the supervised learning neural network model for convolution to obtain the second layer image features and the third layer image features, the formula for performing convolution operation is as follows:
Figure 503463DEST_PATH_IMAGE025
wherein, the first and the second end of the pipe are connected with each other,
Figure 897535DEST_PATH_IMAGE026
representing position in input function image
Figure 715319DEST_PATH_IMAGE027
The gray-level value at the location of the location,
Figure 245657DEST_PATH_IMAGE028
representing a convolution kernel
Figure 123483DEST_PATH_IMAGE029
The magnitude of the value at the location of the position,
Figure 4852DEST_PATH_IMAGE030
representing the abscissa in the input function image,
Figure 626326DEST_PATH_IMAGE031
representing input functionsThe ordinate in the image is that of the image,
Figure 276750DEST_PATH_IMAGE032
representing the abscissa corresponding to the convolution kernel,
Figure 200844DEST_PATH_IMAGE033
representing the ordinate corresponding to the convolution kernel;
in the step of fusing the second-layer image features and the third-layer image features to obtain fourth-layer fused image features, a formula for performing feature fusion is represented as:
Figure 428563DEST_PATH_IMAGE034
wherein the content of the first and second substances,
Figure 729094DEST_PATH_IMAGE035
representing the fused image characteristics after the fusion of the cascade function,
Figure 624238DEST_PATH_IMAGE036
which represents a vector concatenation operation, is shown,
Figure 453653DEST_PATH_IMAGE037
it is shown that the second activation function is,
Figure 434248DEST_PATH_IMAGE038
feature information representing a convolution image output by the second layer in the supervised learning neural network model,
Figure 272891DEST_PATH_IMAGE039
feature information representing the convolution image output by the third layer in the supervised learning neural network model,
Figure 759891DEST_PATH_IMAGE040
representing a global average pooling operation.
8. The method according to claim 7, wherein the supervised learning neural network model comprises a plurality of supervised learning neural network layers, and the calculation formula of each neuron in the supervised learning neural network layers is represented as:
Figure 25788DEST_PATH_IMAGE041
wherein the content of the first and second substances,
Figure 228099DEST_PATH_IMAGE042
denotes the first
Figure 870433DEST_PATH_IMAGE043
Layer one
Figure 474590DEST_PATH_IMAGE004
The magnitude of the value of each of the neurons,
Figure 911387DEST_PATH_IMAGE044
denotes the first
Figure 600994DEST_PATH_IMAGE045
In a layer of
Figure 781440DEST_PATH_IMAGE046
The magnitude of the value of each of the neurons,
Figure 240103DEST_PATH_IMAGE047
is shown as
Figure 847802DEST_PATH_IMAGE045
In a layer of
Figure 24705DEST_PATH_IMAGE046
The nerve cell and the first
Figure 743263DEST_PATH_IMAGE048
The weights are connected to the layer neurons,
Figure 322012DEST_PATH_IMAGE049
which represents a non-linear activation function,
Figure 100612DEST_PATH_IMAGE050
denotes the first
Figure 764811DEST_PATH_IMAGE045
Total number of neurons in a layer.
9. The method of claim 8, wherein the training of the supervised learning neural network model comprises forward propagation and backward propagation, wherein the forward propagation of the supervised learning neural network model is completed by a convolutional layer and a pooling layer, and the supervised learning neural network model performs multi-class cross entropy function calculation by backward propagation, and the expression of the multi-class cross entropy function is as follows:
Figure 287060DEST_PATH_IMAGE051
wherein the content of the first and second substances,
Figure 717385DEST_PATH_IMAGE052
represents a multi-class cross-entropy function,
Figure 666887DEST_PATH_IMAGE053
representing a sample
Figure 552803DEST_PATH_IMAGE054
The cross-entropy function of (a) is,
Figure 878742DEST_PATH_IMAGE055
indicating the number of categories that are required,
Figure 166504DEST_PATH_IMAGE056
has a value of 0 or 1; when the sample is
Figure 286907DEST_PATH_IMAGE054
The true class of
Figure 925699DEST_PATH_IMAGE057
Taking 1, otherwise, taking 0;
Figure 789749DEST_PATH_IMAGE058
representing a sample
Figure 932018DEST_PATH_IMAGE054
Belong to the category
Figure 488901DEST_PATH_IMAGE059
The prediction function of (a) is determined,
Figure 349410DEST_PATH_IMAGE060
representing the number of samples;
the formula of the weight update corresponding to the multi-classification cross entropy function is represented as:
Figure 17151DEST_PATH_IMAGE061
wherein the content of the first and second substances,
Figure 13926DEST_PATH_IMAGE062
which represents the weight update learning rate,
Figure 741711DEST_PATH_IMAGE063
indicating the updated weight value of the weight value,
Figure 89516DEST_PATH_IMAGE064
which represents the current weight value of the current weight,
Figure 560948DEST_PATH_IMAGE065
representing a sample
Figure 415159DEST_PATH_IMAGE054
The prediction function of (2).
10. A data augmented supervised learning image defect classification system, the system comprising:
the image acquisition module is used for acquiring an image to be trained;
the interest extraction module is used for inputting the image to be trained into the interest region feature extraction module to obtain an image interest feature region;
the data augmentation module is used for constructing a data augmentation model and augmenting the interested characteristic region of the image to obtain a data set after data augmentation;
the model training module is used for constructing a supervised learning neural network model and training the supervised learning neural network model by using the data set after data augmentation;
and the result output module is used for putting the image of the area to be predicted into the trained supervised learning neural network model for prediction so as to obtain an image classification result.
CN202210382999.5A 2022-04-13 2022-04-13 Data-augmented supervised learning image defect classification method and system Pending CN114462558A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210382999.5A CN114462558A (en) 2022-04-13 2022-04-13 Data-augmented supervised learning image defect classification method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210382999.5A CN114462558A (en) 2022-04-13 2022-04-13 Data-augmented supervised learning image defect classification method and system

Publications (1)

Publication Number Publication Date
CN114462558A true CN114462558A (en) 2022-05-10

Family

ID=81418443

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210382999.5A Pending CN114462558A (en) 2022-04-13 2022-04-13 Data-augmented supervised learning image defect classification method and system

Country Status (1)

Country Link
CN (1) CN114462558A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116503858A (en) * 2023-05-06 2023-07-28 华东师范大学 Immunofluorescence image classification method and system based on generation model

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108107051A (en) * 2017-12-19 2018-06-01 无锡先导智能装备股份有限公司 lithium battery defect detecting system and method based on machine vision
CN109102019A (en) * 2018-08-09 2018-12-28 成都信息工程大学 Image classification method based on HP-Net convolutional neural networks
CN110097103A (en) * 2019-04-22 2019-08-06 西安电子科技大学 Based on the semi-supervision image classification method for generating confrontation network
CN111583250A (en) * 2020-05-14 2020-08-25 上海深至信息科技有限公司 Deep learning-based ultrasonic image mitral valve positioning method and system
CN111711820A (en) * 2020-05-13 2020-09-25 哈尔滨工业大学 Crack image compression sampling method based on generation countermeasure network
CN113077388A (en) * 2021-04-25 2021-07-06 中国人民解放军国防科技大学 Data-augmented deep semi-supervised over-limit learning image classification method and system
CN113205153A (en) * 2021-05-26 2021-08-03 华侨大学 Training method of pediatric pneumonia auxiliary diagnosis model and model obtained through training
CN113255714A (en) * 2020-02-11 2021-08-13 北京三星通信技术研究有限公司 Image clustering method and device, electronic equipment and computer readable storage medium
CN113256623A (en) * 2021-06-29 2021-08-13 南昌工程学院 FPC defect detection method based on improved MASK RCNN

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108107051A (en) * 2017-12-19 2018-06-01 无锡先导智能装备股份有限公司 lithium battery defect detecting system and method based on machine vision
CN109102019A (en) * 2018-08-09 2018-12-28 成都信息工程大学 Image classification method based on HP-Net convolutional neural networks
CN110097103A (en) * 2019-04-22 2019-08-06 西安电子科技大学 Based on the semi-supervision image classification method for generating confrontation network
CN113255714A (en) * 2020-02-11 2021-08-13 北京三星通信技术研究有限公司 Image clustering method and device, electronic equipment and computer readable storage medium
CN111711820A (en) * 2020-05-13 2020-09-25 哈尔滨工业大学 Crack image compression sampling method based on generation countermeasure network
CN111583250A (en) * 2020-05-14 2020-08-25 上海深至信息科技有限公司 Deep learning-based ultrasonic image mitral valve positioning method and system
CN113077388A (en) * 2021-04-25 2021-07-06 中国人民解放军国防科技大学 Data-augmented deep semi-supervised over-limit learning image classification method and system
CN113205153A (en) * 2021-05-26 2021-08-03 华侨大学 Training method of pediatric pneumonia auxiliary diagnosis model and model obtained through training
CN113256623A (en) * 2021-06-29 2021-08-13 南昌工程学院 FPC defect detection method based on improved MASK RCNN

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
M.戈帕尔著: "《机器学习及其应用》", 30 April 2020, 北京：机械工业出版社 *
方志军，高永彬，吴晨谋编著: "《TensorFlow应用案例教程》", 31 August 2020, 北京：中国铁道出版社 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116503858A (en) * 2023-05-06 2023-07-28 华东师范大学 Immunofluorescence image classification method and system based on generation model
CN116503858B (en) * 2023-05-06 2024-01-23 华东师范大学 Immunofluorescence image classification method and system based on generation model

Similar Documents

Publication Publication Date Title
CN110321923B (en) Target detection method, system and medium for fusion of different-scale receptive field characteristic layers
US20190228268A1 (en) Method and system for cell image segmentation using multi-stage convolutional neural networks
CN108665005B (en) Method for improving CNN-based image recognition performance by using DCGAN
CN109684922B (en) Multi-model finished dish identification method based on convolutional neural network
CN107451565B (en) Semi-supervised small sample deep learning image mode classification and identification method
CN110287777B (en) Golden monkey body segmentation algorithm in natural scene
CN107871316B (en) Automatic X-ray film hand bone interest area extraction method based on deep neural network
CN111611972B (en) Crop leaf type identification method based on multi-view multi-task integrated learning
CN114998220B (en) Tongue image detection and positioning method based on improved Tiny-YOLO v4 natural environment
CN110969171A (en) Image classification model, method and application based on improved convolutional neural network
CN111986125A (en) Method for multi-target task instance segmentation
CN113221956B (en) Target identification method and device based on improved multi-scale depth model
CN112288026B (en) Infrared weak and small target detection method based on class activation diagram
Lin et al. Determination of the varieties of rice kernels based on machine vision and deep learning technology
CN111160389A (en) Lithology identification method based on fusion of VGG
CN115239672A (en) Defect detection method and device, equipment and storage medium
CN114462558A (en) Data-augmented supervised learning image defect classification method and system
CN112784494B (en) Training method of false positive recognition model, target recognition method and device
CN111340051A (en) Picture processing method and device and storage medium
CN115147341A (en) Fabric surface defect classification depth network method based on biological vision inspiration
CN116206227B (en) Picture examination system and method for 5G rich media information, electronic equipment and medium
CN113052215A (en) Sonar image automatic target identification method based on neural network visualization
CN113191361A (en) Shape recognition method
CN111860601A (en) Method and device for predicting large fungus species
CN112560824B (en) Facial expression recognition method based on multi-feature adaptive fusion

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20220510