CN117710763A - Image noise recognition model training method, image noise recognition method and device - Google Patents
Image noise recognition model training method, image noise recognition method and device Download PDFInfo
- Publication number
- CN117710763A CN117710763A CN202311579876.1A CN202311579876A CN117710763A CN 117710763 A CN117710763 A CN 117710763A CN 202311579876 A CN202311579876 A CN 202311579876A CN 117710763 A CN117710763 A CN 117710763A
- Authority
- CN
- China
- Prior art keywords
- image
- noise
- label
- gain
- tag
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012549 training Methods 0.000 title claims abstract description 136
- 238000000034 method Methods 0.000 title claims abstract description 84
- 230000004927 fusion Effects 0.000 claims abstract description 85
- 239000013604 expression vector Substances 0.000 claims abstract description 78
- 230000015654 memory Effects 0.000 claims description 25
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000010276 construction Methods 0.000 claims description 7
- 230000008569 process Effects 0.000 abstract description 13
- 238000012545 processing Methods 0.000 abstract description 6
- 239000013598 vector Substances 0.000 description 18
- 230000006870 function Effects 0.000 description 16
- 238000000605 extraction Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 6
- 230000004913 activation Effects 0.000 description 4
- 238000001514 detection method Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000002372 labelling Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000013178 mathematical model Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T90/00—Enabling technologies or technologies with a potential or indirect contribution to GHG emissions mitigation
Abstract
The invention relates to the technical field of image processing, and discloses an image noise identification model training method, an image noise identification method and an image noise identification device, wherein the image noise identification method comprises the following steps: acquiring a sample image and an image tag thereof; calculating to obtain a label fusion gain expression vector corresponding to the sample image by using the sample image and the image label; constructing and obtaining a noise training data set based on the image tag and the tag fusion gain expression vector; based on a noise training data set, training an initial image noise recognition model to obtain a target image noise recognition model, wherein the target image noise recognition model is used for carrying out image noise recognition. The dimensionality of the data characteristic expression is improved, the overfitting in the network training process is avoided, and the noise data is not required to be marked manually.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to an image noise identification model training method, an image noise identification method and an image noise identification device.
Background
In recent years, with the rapid development of artificial intelligence technology, the status of large-scale data in artificial intelligence research is becoming more important. In the application process of large-scale data, the data annotation is key, and the quality of the data annotation directly influences the quality of the algorithm. Data labeling is a process of giving labels of attribute items or characteristic items to be labeled of data, and noise data is one of main factors affecting the quality of data labeling. Noise refers to data with incorrect annotation of data, namely data with inconsistent labels, and labels of noise data are called noise labels. In large-scale data, noise is always inevitably present. Neither the data acquisition method nor the data labeling method can fundamentally prevent the occurrence of noise data. Noise data can easily cause the problem of over fitting of the deep neural network, and has a certain influence on the training effect and generalization performance of the network, thereby causing the decline of the learning effect and training quality of the algorithm.
In the related technology, the related research of noise label detection is mostly an indirect identification method, and the inherent characteristic attribute of the noise label is easily ignored by the indirect identification method; meanwhile, the method considers that the distribution of the noise data accords with a certain physical or mathematical rule, and a mathematical model or model training strategy is artificially designed to infer the noise data, but the artificially designed model and strategy may have limitations. Therefore, the indirect recognition method may cause recognition errors, missed detection, and the like.
Disclosure of Invention
In view of the above, the invention provides an image noise recognition model training method, an image noise recognition method and an image noise recognition device, so as to solve the problem that the existing noise label recognition has limitation.
In a first aspect, the present invention provides a training method for an image noise recognition model, the method comprising:
acquiring a sample image and an image tag thereof;
calculating to obtain a label fusion gain expression vector corresponding to the sample image by using the sample image and the image label;
constructing and obtaining a noise training data set based on the image tag and the tag fusion gain expression vector;
based on the noise training data set, training the initial image noise recognition model to obtain a target image noise recognition model, wherein the target image noise recognition model is used for image noise recognition.
According to the invention, the data characteristic expression method for the label fusion gain expression of the noise data is designed, so that the characteristics of the noise data are directly extracted, and whether noise is generated or not is directly judged, the inherent characteristics of the noise data can be accurately mined, and the accurate judgment is realized. The data feature expression method of the label fusion gain expression is utilized to improve the dimension of the data feature expression, increase the information quantity and avoid the over fitting in the network training process. Based on the existing data, the automatic generation of the noise training data set is realized, and the noise data is not required to be marked manually.
In an optional implementation manner, by using the sample image and the image label, a label fusion gain expression vector corresponding to the sample image is calculated, which includes:
performing feature recognition on the sample image to obtain an image category probability corresponding to the sample image;
calculating the difference between the image category probability and the image label to obtain a label difference corresponding to the sample image;
training the gain network based on the category probability and the label difference to obtain a trained gain network;
and based on the trained gain network, performing gain on the label difference, the image category probability and the image label to obtain a label fusion gain expression vector.
In the mode, the trained gain network is used for carrying out gain on the label difference, the image category probability and the image label, the dimensionality of vectors such as the label difference, the category probability and the image label is improved, the information quantity of noise characteristics is expanded, the over fitting of network training can be effectively avoided in the process of directly identifying the noise characteristics, and more prior information can be brought to the noise learning network.
In an alternative embodiment, based on a trained gain network, the label difference, the image category probability and the image label are gained to obtain a label fusion gain expression vector, which includes:
inputting the label difference into a trained gain network to obtain a gain label difference;
inputting the image category probability and the image label into a trained gain network to obtain category probability gain and image label gain;
calculating the difference between the class probability gain and the image label gain to obtain a label gain difference;
and combining the gain label difference with the label gain difference to obtain a label fusion gain expression vector.
In the mode, the label fusion gain expression vector is obtained by fusing the label difference, the image category probability and the image label, so that the information quantity of the noise characteristic expression vector is further enhanced, and the overfitting of the noise learning network is avoided.
In an alternative embodiment, the constructing a noise training data set based on the image label and the label fusion gain expression vector includes:
taking the image tag and the tag fusion gain expression vector as negative sample data of a noise training data set;
based on a preset proportion, calculating to obtain a pseudo tag corresponding to the image tag, calculating to obtain a tag fusion gain expression vector corresponding to the pseudo tag by utilizing the pseudo tag and the sample image, and taking the tag fusion gain expression vector corresponding to the pseudo tag and the pseudo tag as positive sample data of a noise training data set;
combining the negative sample data with the positive sample data results in a noise training data set.
In the mode, noise data is artificially manufactured by modifying the labels of the data set, a training sample is provided for the image noise recognition model, the image noise recognition model is convenient to directly recognize noise characteristic data, and the robustness and accuracy of noise recognition are further improved.
In an optional implementation manner, based on a preset proportion, calculating to obtain a pseudo tag corresponding to the image tag includes:
dividing the image tag into a first data set, a second data set and a third data set based on a preset proportion;
And modifying the image labels in the first data set, the second data set and the third data set to obtain pseudo labels corresponding to the image labels.
In the mode, noise is artificially manufactured by dividing the noise data into three types, so that the manufactured noise data are distributed relatively evenly in different confidence ranges, and the accuracy of the image noise recognition model obtained through training is further improved.
In a second aspect, the present invention provides an image noise recognition method, including:
acquiring an image to be identified and an image tag thereof;
based on the image to be identified and the image label thereof, calculating to obtain a label fusion gain expression vector corresponding to the image to be identified;
inputting a label fusion gain expression vector corresponding to an image to be identified into an image noise identification model, identifying the noise probability that the image to be identified belongs to noise, and judging whether the image to be identified belongs to noise or not based on the noise probability, wherein the image noise identification model is obtained by training by using the image noise identification model training method according to any one of the first aspect.
According to the invention, the inherent characteristics of the noise data can be accurately mined by using the trained noise recognition model, so that whether the image to be recognized is the noise data or not can be directly judged, and whether the label of the image to be recognized is correct or not can be judged.
In a third aspect, the present invention provides an image noise recognition model training apparatus, the apparatus comprising:
the first image acquisition module is used for acquiring a sample image and an image tag thereof;
the first fusion gain calculation module is used for calculating a label fusion gain expression vector corresponding to the sample image by using the sample image and the image label;
the noise training data set construction module is used for constructing and obtaining a noise training data set based on the image label and the label fusion gain expression vector;
the model training module is used for training the initial image noise recognition model based on the noise training data set to obtain a target image noise recognition model, and the target image noise recognition model is used for carrying out image noise recognition.
In a fourth aspect, the present invention provides an image noise recognition apparatus, the apparatus comprising:
the second image acquisition module is used for acquiring an image to be identified and an image tag thereof;
the second fusion gain calculation module is used for calculating a label fusion gain expression vector corresponding to the image to be identified based on the image to be identified and the image label thereof;
the noise recognition module is used for inputting the label fusion gain expression vector corresponding to the image to be recognized into an image noise recognition model, recognizing the noise probability that the image to be recognized belongs to noise, and judging whether the image to be recognized belongs to noise or not based on the noise probability, wherein the image noise recognition model is trained by the image noise recognition model training device of the third aspect.
In a fifth aspect, the present invention provides a computer device comprising: the processor executes the computer instructions, thereby executing the image noise recognition model training method according to the first aspect or any implementation manner corresponding to the first aspect or executing the image noise recognition method according to the second aspect.
In a sixth aspect, the present invention provides a computer-readable storage medium having stored thereon computer instructions for causing a computer to perform the image noise recognition model training method of the first aspect or any one of its corresponding embodiments or the image noise recognition method of the second aspect.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flow chart of an image noise recognition model training method according to an embodiment of the present invention.
FIG. 2 is a flow chart of image noise recognition model training and image noise recognition according to an embodiment of the present invention.
FIG. 3 is a flow chart of another image noise recognition model training method according to an embodiment of the present invention.
Fig. 4 is a schematic diagram of calculating a tag fusion gain expression vector according to an embodiment of the present invention.
Fig. 5 is a flow chart of yet another image noise recognition model training method according to an embodiment of the present invention.
Fig. 6 is a flowchart of an image noise recognition method according to an embodiment of the present invention.
Fig. 7 is a block diagram of a structure of an image noise recognition model training apparatus according to an embodiment of the present invention.
Fig. 8 is a block diagram of an image noise recognition apparatus according to an embodiment of the present invention.
Fig. 9 is a schematic diagram of a hardware structure of a computer device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In the related technology, the related research of noise label detection is mostly an indirect identification method, and the inherent characteristic attribute of the noise label is easily ignored by the indirect identification method; meanwhile, the method considers that the distribution of the noise data accords with a certain physical or mathematical rule, and a mathematical model or model training strategy is artificially designed to infer the noise data, but the artificially designed model and strategy may have limitations. Therefore, the indirect recognition method may cause recognition errors, missed detection, and the like.
In order to solve the above-mentioned problems, in the embodiments of the present invention, an image noise recognition model training method is provided for a computer device, and it should be noted that an execution body of the image noise recognition model training device may be an image noise recognition model training device, and the image noise recognition model training device may be implemented as part or all of the computer device in a manner of software, hardware or a combination of software and hardware, where the computer device may be a terminal, a client, or a server, and the server may be a server, or may be a server cluster formed by multiple servers. In the following method embodiments, the execution subject is a computer device.
The computer device in this embodiment is suitable for use in detecting and identifying use scenes for image noise tags. According to the image noise identification model training method provided by the invention, the data characteristic expression method of label fusion gain expression for noise data is designed, the characteristics of the noise data are directly extracted, and whether noise is generated or not is directly judged, so that the inherent characteristics of the noise data can be accurately mined, and accurate judgment is realized. The data feature expression method of the label fusion gain expression is utilized to improve the dimension of the data feature expression, increase the information quantity and avoid the over fitting in the network training process. Based on the existing data, the automatic generation of the noise training data set is realized, and the noise data is not required to be marked manually.
In accordance with an embodiment of the present invention, there is provided an image noise recognition model training method embodiment, it being noted that the steps shown in the flowchart of the figures may be performed in a computer system such as a set of computer executable instructions, and, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in an order other than that shown or described herein.
In this embodiment, an image noise recognition model training method is provided, which may be used in the above-mentioned computer device, and fig. 1 is a flowchart of an image noise recognition model training method according to an embodiment of the present invention, as shown in fig. 1, where the flowchart includes the following steps:
step S101, a sample image and an image tag thereof are acquired.
In one example, all images d= { x, y } in the dataset are acquired, where x is sample image data and y is an image tag in a one-hot encoded form, the form of the image tag is not limiting in the present invention. Let y argmaxy The actual category to which the image tag refers.
Step S102, calculating to obtain a label fusion gain expression vector corresponding to the sample image by using the sample image and the image label.
In one example, the network f is extracted by inputting the data into an image feature i Training is carried out to realize the image feature extraction network f i Is used for calculating the weight of the (c). The ResNet network is used as the image feature extraction network, and the kind of the image feature extraction network is not limited in the present invention. Image feature extraction network f i The vector obtained through Softmax function processing is recorded as s, s is called image category probability, and the length of the vector is equal to the number n of categories in the data set. By designing a gain network f g And training the network to obtain a trained gain network. And calculating to obtain the label fusion gain expression vector corresponding to the sample image by using the trained gain network.
And step S103, constructing and obtaining a noise training data set based on the image label and the label fusion gain expression vector.
In an example, to directly determine whether the data is noise or not by using the two-class network, there must be a data set (noise training data set, noise Training Set, NTS) corresponding to the problem and including positive and negative samples of noise or not, and the noise training data set is constructed by using the original data set D and by constructing a pseudo tag.
Step S104, training the initial image noise recognition model based on the noise training data set to obtain a target image noise recognition model.
In the embodiment of the invention, the target image noise recognition model is used for image noise recognition.
In an example, the image noise recognition model takes NTS as training samples, inputs as fusion gain expression vector u, and consists of 5 layers of full-connection layers, wherein the length of layer 1 is 512, the lengths of layers 3 and 4 are 1024, the activation functions are all ReLU, the last layer is the full-connection layer with length 1 as the output layer of the network, and the activation function is Sigmoid. The image noise recognition model is input according to 1024 data samples of each batch for training until all data are traversed 50 times.
The input label of the image noise recognition model is fused with a gain expression vector u, and the gain expression vector u is output as the probability f that the image belongs to noise f (u) the loss function in the training process of the image noise identification model is a two-class cross entropy loss function, and the loss function is shown in the following formula:
wherein v is an image label of an image sample in the noise training data set NTS, v=1 indicates that the image sample is a positive sample (i.e. noise data), v=0 indicates that the image sample is a negative sample (i.e. non-noise data), log is a log-to-fetch operation, u is a label fusion gain expression vector, f f (u) probability of image belonging to noise, L f The loss function value for the noise recognition model.
In an implementation scenario, fig. 2 is a flow chart of image noise recognition model training and image noise recognition according to an embodiment of the present invention. As shown in fig. 2, an original image x is input into an image feature extraction network f i And obtaining the class probability s corresponding to the image. In the training process of the image noise identification model, a pseudo tag is generated on the basis of the image tag yForming noise training data set NTS containing positive sample and negative sample, combining image label y and pseudo label +.>Generating a label fusion gain expression vector u of data in a noise training data set NTS with class probability s, and training an image noise recognition model f by using the noise training data set f And learning is carried out, and the trained image noise recognition model can recognize and obtain the noise probability corresponding to the original image.
According to the image noise recognition model training method, the data feature expression method for the label fusion gain expression of the noise data is designed, the features of the noise data are directly extracted, whether noise is generated or not is directly judged, inherent features of the noise data can be accurately mined, and accurate judgment is achieved. The data feature expression method of the label fusion gain expression is utilized to improve the dimension of the data feature expression, increase the information quantity and avoid the over fitting in the network training process. Based on the existing data, the automatic generation of the noise training data set is realized, and the noise data is not required to be marked manually.
In this embodiment, an image noise recognition model training method is provided, which may be used in the above-mentioned computer device, and fig. 3 is a flowchart of another image noise recognition model training method according to an embodiment of the present invention, as shown in fig. 3, where the flowchart includes the following steps:
step S301, a sample image and an image tag thereof are acquired. Please refer to step S101 in the embodiment shown in fig. 1 in detail, which is not described herein.
Step S302, calculating to obtain a label fusion gain expression vector corresponding to the sample image by using the sample image and the image label.
Specifically, the step S302 includes:
in step S3021, feature recognition is performed on the sample image, so as to obtain an image category probability corresponding to the sample image.
In one example, let y argmaxy The actual category to which the image tag refers. By inputting the data into the image feature extraction network f i Training is carried out to realize the image feature extraction network f i Is used for calculating the weight of the (c). A res net network is used as the image feature extraction network. Image feature extraction network f i The vector obtained through Softmax function processing is recorded as s, s is the image category probability, and the length of the vector is equal to the number n of categories in the data set.
Step S3022, calculating the difference between the image category probability and the image label, to obtain the label difference corresponding to the sample image.
In an example, tag difference u l Is the difference between the image class probability s and the image label y, calculates the label difference u l The formula of (2) is as follows:
u l ＝s-y
step S3023, training the gain network based on the category probability and the label difference to obtain a trained gain network.
In one example, the gain network is composed of a decoding network and an encoding network, the input of the decoding network is a vector with length of n, and the decoding network f g-de The full-connection layer consists of 6 layers, wherein the lengths of the 1 st layer and the 2 nd layer are 128, the lengths of the 3 rd layer and the 4 th layer are 256, the lengths of the 5 th layer and the 6 th layer are 512, and the activation functions are all ReLU. The input of the coding network is the decoding network f g-de The output of (a), the coding network f g-en The full-connection layer consists of 5 layers, wherein the lengths of the 1 st layer and the 2 nd layer are 256, the lengths of the 3 rd layer and the 4 th layer are 128, the length of the 5 th layer is n, and the activation functions are ReLU.
Training of the gain network includes: the training data of the gain network consists of image class probability s and image labels y of all images in the data set, and training is carried out according to 512 input networks in each batch until all the data are traversed for 50 times. For the input x of the gain network, its output is f g (x) The loss function in the gain network training process is the sum of Euclidean distances between each original input vector and the vector obtained after decoding-encoding processing, and the formula of the loss function is as follows:
wherein x is i Is the i-th input vector of the gain network, f g (x i ) D (a, b) is the Euclidean distance of the vector a and the vector b, which is the output result of the gain network to the ith input vector.
For a trained gain network f g Using only the decoding network f g-de The gain of the partial image class probability vector s increases the information quantity of the input vector and avoids information overfitting.
Step S3024, performing gain on the label difference, the image category probability and the image label based on the trained gain network, to obtain a label fusion gain expression vector.
In some alternative embodiments, step S3024 includes:
and a step a1, inputting the label difference into a trained gain network to obtain a gain label difference.
In one example, the tag difference u l Decoding network f input to a trained gain network g-de In (1) obtaining the gain tag difference u d The specific formula is as follows:
u d ＝f g-de (u l )
and a step a2, inputting the image category probability and the image label into a trained gain network to obtain category probability gain and image label gain.
In one example, the class probability s and the image tag y are input to a decoding network f of a trained gain network, respectively g-de In (3) obtaining the class probability gain f g-de (s) and image tag gain f g-de (y)。
And a3, calculating the difference between the class probability gain and the image label gain to obtain the label gain difference.
In one example, the tag gain difference u of the image to be identified is calculated by the following formula e ：
u e ＝f g-de (s)-f g-de (y)
And a step a4, combining the gain label difference with the label gain difference to obtain a label fusion gain expression vector.
In one example, the gain is tag difference u d Difference from the tag gain u e Splicing is carried out in sequence to obtain a label fusion gain expression vector u:
u＝[u d u e ]
in an implementation scenario, fig. 4 is a schematic diagram of calculating a label fusion gain expression vector according to an embodiment of the present invention, as shown in fig. 4, an image feature extraction network f i Calculating the calculated image class probability s obtained through Softmax function processing, and calculating the difference label difference u between the image class probability s and the image label y l The label difference u l Decoding network f input to a trained gain network g-de In (1) obtaining the gain tag difference u d The class probability s and the image label y are respectively input into a decoding network f of the trained gain network g-de In (3) obtaining the class probability gain f g-de (s) and image tag gain f g-de (y) calculating a tag gain difference u of the image to be recognized e The gain is subjected to label difference u d Difference from the tag gain u e And splicing in sequence to obtain the label fusion gain expression vector u.
In the mode, the label fusion gain expression vector is obtained by fusing the label difference, the image category probability and the image label, so that the information quantity of the noise characteristic expression vector is further enhanced, and the overfitting of the noise learning network is avoided.
Step S303, constructing and obtaining a noise training data set based on the image label and the label fusion gain expression vector. Please refer to step S103 in the embodiment shown in fig. 1 in detail, which is not described herein.
Step S304, training the initial image noise recognition model based on the noise training data set to obtain the target image noise recognition model. Please refer to step S103 in the embodiment shown in fig. 1 in detail, which is not described herein.
According to the image noise recognition model training method, the trained gain network is used for carrying out gain on the label difference, the image category probability and the image label, the dimensionality of vectors such as the label difference, the category probability and the image label is improved, the information quantity of noise features is expanded, the over fitting of network training can be effectively avoided in the process of directly recognizing the noise features, and more priori information can be brought to the noise learning network. The label fusion gain expression vector is obtained by fusing the label difference, the image category probability and the image label, so that the information quantity of the noise characteristic expression vector is further enhanced, and the overfitting of a noise learning network is avoided.
In this embodiment, an image noise recognition model training method is provided, which may be used in the above-mentioned computer device, and fig. 5 is a flowchart of another image noise recognition model training method according to an embodiment of the present invention, as shown in fig. 5, where the flowchart includes the following steps:
In step S501, a sample image and an image tag thereof are acquired. Please refer to step S301 in the embodiment shown in fig. 3 in detail, which is not described herein.
Step S502, calculating to obtain a label fusion gain expression vector corresponding to the sample image by using the sample image and the image label. Please refer to step S302 in the embodiment shown in fig. 3 in detail, which is not described herein.
Step S503, based on the image label and the label fusion gain expression vector, constructing and obtaining a noise training data set.
Specifically, the step S503 includes:
in step S5031, the image tag and the tag fusion gain expression vector are used as negative sample data of the noise training data set.
In an example, the data of the noise training data set NTS includes data consisting of two parts nts= { u, v }, where u is a tag fusion gain expression vector; v is a noise label, 1 is noise, and 0 is non-noise. And calculating all image data x and original labels y in the original data set D to obtain a label fusion gain expression vector u, and enabling a noise label v=0 of the data to represent non-noise data to form a negative sample of NTS.
Step S5032, based on the preset proportion, calculating to obtain a pseudo tag corresponding to the image tag, calculating to obtain a tag fusion gain expression vector corresponding to the pseudo tag by using the pseudo tag and the sample image, and taking the tag fusion gain expression vector corresponding to the pseudo tag and the pseudo tag as positive sample data of the noise training data set.
In some alternative embodiments, step S5032 includes:
and b1, dividing the image label into a first data set, a second data set and a third data set based on a preset proportion.
And b2, modifying the image labels in the first data set, the second data set and the third data set to obtain pseudo labels corresponding to the image labels.
In one example, 1/3 of the data in the original data set D is randomly selected as the first data set, and the actual category y in the first data set is made argmaxy For each image tag y in the first dataset, =0, dividing y by y at image tag y argmaxy Randomly selecting one component y from other components k1 Modify y k1 Has a value of (0.99,1)]A random number within the range; dividing y at image tag y argmaxy And y is k1 Randomly selecting one component y from other components k2 Modifying its value to 1-y k1 。
(2) Randomly selecting half data from the rest 2/3 of the original data set D as a second data set, and enabling the actual category y in the second data set to be argmaxy For each image tag y in the second data set, =0, dividing y by y at image tag y argmaxy Randomly selecting one component y from other components k1 Modify y k1 Has a value of (0.95,0.99)]A random number within the range; dividing y at image tag y argmaxy And y is k1 Randomly selecting one component y from other components k2 Modifying its value to (0, 1-y) k1 ]A random number within the range; dividing y at image tag y argmaxy 、y k1 And y is k2 Randomly selecting one component y from other components k3 Modifying its value to 1-y k1 -y k2 。
(3) In the case of using the remaining data which are not selected as the third data set, for each image tag y in the third data set, the respective component y is ensured i Sum of 1 and the subscript of the largest component is not the actual class y of the original label argmaxy On the premise, each component in y is randomly assigned.
(4) The label for each image obtained in the above steps (1) - (3) is called pseudo labelUsing the image data x and pseudo tag +.>The tag fusion gain expression vector u is calculated, and the data v=1 of the tag fusion gain expression vector represents noise data, and the noise data form positive samples of NTS.
Step S5033, combining the negative sample data with the positive sample data results in a noise training data set.
Step S504, training the initial image noise recognition model based on the noise training data set to obtain a target image noise recognition model. Please refer to step S304 in the embodiment shown in fig. 3 in detail, which is not described herein.
According to the image noise recognition model training method, the noise data is artificially manufactured by modifying the labels of the data set, a training sample is provided for the image noise recognition model, the image noise recognition model can directly recognize the noise characteristic data conveniently, and the robustness and accuracy of noise recognition are further improved. Noise is artificially manufactured by dividing the noise data into three types, so that the manufactured noise data are distributed relatively evenly in different confidence ranges, and the accuracy of the image noise recognition model obtained by training is further improved.
In this embodiment, an image noise recognition method is provided, which may be used in the above-mentioned computer device, and fig. 6 is a flowchart of the image noise recognition method according to an embodiment of the present invention, as shown in fig. 6, where the flowchart includes the following steps:
step S601, an image to be identified and an image tag thereof are acquired.
Step S602, based on the image to be identified and the image label thereof, calculating to obtain a label fusion gain expression vector corresponding to the image to be identified.
Step S603, inputting the label fusion gain expression vector of the image to be identified into an image noise identification model, identifying the noise probability that the image to be identified belongs to noise, and judging whether the image to be identified belongs to noise or not based on the noise probability.
In the embodiment of the invention, the image noise recognition model is trained by using the image noise recognition model training method in any one of the above embodiments.
In one example, for a trained image feature extraction network f i Gain network f g Noise learning network f f () For the image data i= { x, y } to be determined as noise data, whether the label thereof is a noise label is determined according to the following steps:
(1) Inputting image data x into an image feature extraction network f i The class probability s is obtained.
(2) Based on trained gain network f by using class probability s and data label y g And 3, obtaining a label fusion gain expression vector u.
(3) Inputting the label fusion gain expression vector u to a trained noise learning network f f () In obtaining the output f of the network f (u) the output may be considered as the probability that the image belongs to noise.
(4) E.g. f f (u)>0.5, the data is considered to be noise data, and the label is a noise label; otherwise, the data is considered to be non-noise data, and the label is a non-noise label.
According to the image noise identification method, whether the image to be identified is noise data or not can be directly judged by using the trained noise identification model, whether the label of the image to be identified is correct or not is judged, inherent characteristics of the noise data can be accurately mined, and accurate judgment is achieved.
The embodiment also provides an image noise recognition model training device, which is used for implementing the above embodiment and the preferred implementation manner, and is not described in detail. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. While the means described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.
The present embodiment provides an image noise recognition model training device, as shown in fig. 7, including:
a first image acquisition module 701, configured to acquire a sample image and an image tag thereof. Please refer to step S101 in the embodiment shown in fig. 1 in detail, which is not described herein.
The first fusion gain calculation module 702 is configured to calculate, using the sample image and the image label, a label fusion gain expression vector corresponding to the sample image. Please refer to step S102 in the embodiment shown in fig. 1 in detail, which is not described herein.
The noise training data set construction module 703 is configured to construct a noise training data set based on the image tag and the tag fusion gain expression vector. Please refer to step S103 in the embodiment shown in fig. 1 in detail, which is not described herein.
The model training module is used for training the initial image noise recognition model based on the noise training data set to obtain a target image noise recognition model, and the target image noise recognition model is used for carrying out image noise recognition. Please refer to step S104 in the embodiment shown in fig. 1 in detail, which is not described herein.
In some alternative embodiments, the first fused gain calculation module 702 includes:
and the feature recognition unit is used for carrying out feature recognition on the sample image to obtain the image category probability corresponding to the sample image.
And the label difference calculating unit is used for calculating the difference between the image category probability and the image label to obtain the label difference corresponding to the sample image.
The gain network training unit is used for training the gain network based on the category probability and the label difference to obtain a trained gain network.
The vector gain unit is used for carrying out gain on the label difference, the image category probability and the image label based on the trained gain network to obtain a label fusion gain expression vector.
In some alternative embodiments, the vector gain unit includes:
and the gain label difference calculating subunit is used for inputting the label difference into the trained gain network to obtain the gain label difference.
And the class sample probability gain calculation subunit is used for inputting the image class probability and the image label into a trained gain network to obtain class probability gain and image label gain.
And the label gain difference calculating subunit is used for calculating the difference between the class probability gain and the image label gain to obtain the label gain difference.
And the vector gain subunit is used for combining the gain label difference with the label gain difference to obtain a label fusion gain expression vector.
In some alternative embodiments, the noise training data set construction module 703 includes:
and the negative sample data construction unit is used for taking the image label, the sample image and the label fusion gain expression vector as negative sample data of the noise training data set.
The positive sample data construction unit is used for calculating to obtain a corresponding pseudo tag in the image tag based on a preset proportion, calculating to obtain a tag fusion gain expression vector corresponding to the pseudo tag by utilizing the pseudo tag and the sample image, and taking the tag fusion gain expression vector corresponding to the pseudo tag and the pseudo tag as positive sample data of the noise training data set.
And the training set construction unit is used for combining the negative sample data with the positive sample data to obtain a noise training data set.
In some alternative embodiments, the positive sample data constructing unit includes:
the data set dividing subunit is used for dividing the image label into a first data set, a second data set and a third data set based on a preset proportion.
The pseudo tag obtaining subunit is configured to modify the image tags in the first data set, the second data set, and the third data set to obtain pseudo tags corresponding to the image tags.
Further functional descriptions of the above respective modules and units are the same as those of the above corresponding embodiments, and are not repeated here.
The image noise recognition model training apparatus in this embodiment is presented in the form of functional units, where the units refer to ASIC (Application Specific Integrated Circuit ) circuits, processors and memories executing one or more software or fixed programs, and/or other devices that can provide the above described functionality.
The embodiment also provides an image noise training device, which is used for implementing the above embodiment and the preferred implementation, and is not described in detail. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. While the means described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.
The present embodiment provides an image noise recognition apparatus, as shown in fig. 8, including:
a second image acquisition module 801, configured to acquire an image to be identified and an image tag thereof. Please refer to step S601 in the embodiment shown in fig. 6 in detail, which is not described herein.
The second fusion gain calculation module 802 is configured to calculate, based on the image to be identified and the image tag thereof, a tag fusion gain expression vector corresponding to the image to be identified. Please refer to step S602 in the embodiment shown in fig. 6 in detail, which is not described herein.
The noise recognition module 803 is configured to input a label fusion gain expression vector corresponding to the image to be recognized into an image noise recognition model, recognize to obtain a noise probability that the image to be recognized belongs to noise, and determine whether the image to be recognized belongs to noise based on the noise probability. Please refer to step S603 in the embodiment shown in fig. 6, which is not described herein.
Further functional descriptions of the above respective modules and units are the same as those of the above corresponding embodiments, and are not repeated here.
The image noise recognition means in this embodiment are presented in the form of functional units, here referred to as ASIC (Application Specific Integrated Circuit ) circuits, processors and memories executing one or more software or fixed programs, and/or other devices that can provide the above described functionality.
The embodiment of the invention also provides computer equipment, which is provided with the image noise recognition model training device shown in the figure 7 and the image noise recognition device shown in the figure 8.
Referring to fig. 9, fig. 9 is a schematic structural diagram of a computer device according to an alternative embodiment of the present invention, as shown in fig. 9, the computer device includes: one or more processors 10, memory 20, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are communicatively coupled to each other using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the computer device, including instructions stored in or on memory to display graphical information of the GUI on an external input/output device, such as a display device coupled to the interface. In some alternative embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple computer devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). One processor 10 is illustrated in fig. 9.
The processor 10 may be a central processor, a network processor, or a combination thereof. The processor 10 may further include a hardware chip, among others. The hardware chip may be an application specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general-purpose array logic, or any combination thereof.
Wherein the memory 20 stores instructions executable by the at least one processor 10 to cause the at least one processor 10 to perform the methods shown in implementing the above embodiments.
The memory 20 may include a storage program area that may store an operating system, at least one application program required for functions, and a storage data area; the storage data area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, memory 20 may optionally include memory located remotely from processor 10, which may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
Memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk, or solid state disk; the memory 20 may also comprise a combination of the above types of memories.
The computer device also includes a communication interface 30 for the computer device to communicate with other devices or communication networks.
The embodiments of the present invention also provide a computer readable storage medium, and the method according to the embodiments of the present invention described above may be implemented in hardware, firmware, or as a computer code which may be recorded on a storage medium, or as original stored in a remote storage medium or a non-transitory machine readable storage medium downloaded through a network and to be stored in a local storage medium, so that the method described herein may be stored on such software process on a storage medium using a general purpose computer, a special purpose processor, or programmable or special purpose hardware. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, a solid state disk or the like; further, the storage medium may also comprise a combination of memories of the kind described above. It will be appreciated that a computer, processor, microprocessor controller or programmable hardware includes a storage element that can store or receive software or computer code that, when accessed and executed by the computer, processor or hardware, implements the methods illustrated by the above embodiments.
Although embodiments of the present invention have been described in connection with the accompanying drawings, various modifications and variations may be made by those skilled in the art without departing from the spirit and scope of the invention, and such modifications and variations fall within the scope of the invention as defined by the appended claims.
Claims (10)
1. A method for training an image noise recognition model, the method comprising:
acquiring a sample image and an image tag thereof;
calculating a label fusion gain expression vector corresponding to the sample image by using the sample image and the image label;
based on the image tag and the tag fusion gain expression vector, constructing and obtaining a noise training data set;
training the initial image noise recognition model based on the noise training data set to obtain a target image noise recognition model, wherein the target image noise recognition model is used for image noise recognition.
2. The method according to claim 1, wherein the calculating, using the sample image and the image label, a label fusion gain expression vector corresponding to the sample image includes:
performing feature recognition on the sample image to obtain an image category probability corresponding to the sample image;
Calculating the difference between the image category probability and the image label to obtain a label difference corresponding to the sample image;
training the gain network based on the category probability and the label difference to obtain a trained gain network;
and based on the trained gain network, performing gain on the label difference, the image category probability and the image label to obtain the label fusion gain expression vector.
3. The method of claim 2, wherein the obtaining the label fusion gain expression vector based on the trained gain network by gain the label difference, the image class probability, and the image label comprises:
inputting the label difference into the trained gain network to obtain a gain label difference;
inputting the image category probability and the image label into the trained gain network to obtain category probability gain and image label gain;
calculating the difference between the class probability gain and the image label gain to obtain a label gain difference;
and combining the gain label difference with the label gain difference to obtain the label fusion gain expression vector.
4. The method of claim 1, wherein constructing a noise training data set based on the image tag and the tag fusion gain expression vector comprises:
taking the image label and the label fusion gain expression vector as negative sample data of the noise training data set;
calculating a pseudo tag corresponding to the image tag based on a preset proportion, calculating a tag fusion gain expression vector corresponding to the pseudo tag by using the pseudo tag and the sample image, and taking the tag fusion gain expression vector corresponding to the pseudo tag and the pseudo tag as positive sample data of the noise training data set;
combining the negative sample data with the positive sample data results in the noise training data set.
5. The method according to claim 4, wherein the calculating, based on the preset ratio, the pseudo tag corresponding to the image tag includes:
dividing the image tag into a first data set, a second data set and a third data set based on the preset proportion;
and modifying the image labels in the first data set, the second data set and the third data set to obtain pseudo labels corresponding to the image labels.
6. An image noise recognition method, the method comprising:
acquiring an image to be identified and an image tag thereof;
based on the image to be identified and the image label thereof, calculating to obtain a label fusion gain expression vector corresponding to the image to be identified;
inputting a label fusion gain expression vector corresponding to the image to be identified into an image noise identification model, identifying the noise probability that the image to be identified belongs to noise, and judging whether the image to be identified belongs to noise or not based on the noise probability, wherein the image noise identification model is trained by using the image noise identification model training method according to any one of claims 1-5.
7. An image noise recognition model training apparatus, the apparatus comprising:
the first image acquisition module is used for acquiring a sample image and an image tag thereof;
the first fusion gain calculation module is used for calculating a label fusion gain expression vector corresponding to the sample image by using the sample image and the image label;
the noise training data set construction module is used for constructing a noise training data set based on the image tag and the tag fusion gain expression vector;
The model training module is used for training the initial image noise recognition model based on the noise training data set to obtain a target image noise recognition model, and the target image noise recognition model is used for carrying out image noise recognition.
8. An image noise recognition apparatus, characterized in that the apparatus comprises:
the second image acquisition module is used for acquiring an image to be identified and an image tag thereof;
the second fusion gain calculation module is used for calculating a label fusion gain expression vector corresponding to the image to be identified based on the image to be identified and the image label thereof;
the noise recognition module is used for inputting the label fusion gain expression vector corresponding to the image to be recognized into an image noise recognition model, recognizing the noise probability that the image to be recognized belongs to noise, and judging whether the image to be recognized belongs to noise or not based on the noise probability, wherein the image noise recognition model is trained by the image noise recognition model training device according to claim 7.
9. A computer device, comprising:
a memory and a processor, the memory and the processor being communicatively connected to each other, the memory having stored therein computer instructions, the processor executing the computer instructions to perform the image noise recognition model training method of any one of claims 1 to 5 or to perform the image noise recognition method of claim 6.
10. A computer-readable storage medium having stored thereon computer instructions for causing a computer to perform the image noise recognition model training method of any one of claims 1 to 5 or the image noise recognition method of claim 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311579876.1A CN117710763A (en) | 2023-11-23 | 2023-11-23 | Image noise recognition model training method, image noise recognition method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311579876.1A CN117710763A (en) | 2023-11-23 | 2023-11-23 | Image noise recognition model training method, image noise recognition method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117710763A true CN117710763A (en) | 2024-03-15 |
Family
ID=90156084
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311579876.1A Pending CN117710763A (en) | 2023-11-23 | 2023-11-23 | Image noise recognition model training method, image noise recognition method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117710763A (en) |
-
2023
- 2023-11-23 CN CN202311579876.1A patent/CN117710763A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108399428B (en) | Triple loss function design method based on trace ratio criterion | |
| WO2018108129A1 (en) | Method and apparatus for use in identifying object type, and electronic device | |
| CN109086654B (en) | Handwriting model training method, text recognition method, device, equipment and medium | |
| WO2022105125A1 (en) | Image segmentation method and apparatus, computer device, and storage medium | |
| CN108734210B (en) | Object detection method based on cross-modal multi-scale feature fusion | |
| US20210295114A1 (en) | Method and apparatus for extracting structured data from image, and device | |
| CN108830329B (en) | Picture processing method and device | |
| CN107111782B (en) | Neural network structure and method thereof | |
| CN111914559B (en) | Text attribute extraction method and device based on probabilistic graphical model and computer equipment | |
| CN110399800B (en) | License plate detection method and system based on deep learning VGG16 framework and storage medium | |
| CN110909868A (en) | Node representation method and device based on graph neural network model | |
| US11714921B2 (en) | Image processing method with ash code on local feature vectors, image processing device and storage medium | |
| US20180365594A1 (en) | Systems and methods for generative learning | |
| CN110968725B (en) | Image content description information generation method, electronic device and storage medium | |
| CN113065525A (en) | Age recognition model training method, face age recognition method and related device | |
| JP2015036939A (en) | Feature extraction program and information processing apparatus | |
| CN110135428B (en) | Image segmentation processing method and device | |
| CN111723852A (en) | Robust training method for target detection network | |
| CN114612921A (en) | Form recognition method and device, electronic equipment and computer readable medium | |
| Altun et al. | SKETRACK: stroke-based recognition of online hand-drawn sketches of arrow-connected diagrams and digital logic circuit diagrams | |
| CN111401394B (en) | Image annotation method and device and computer readable storage medium | |
| CN110717037A (en) | Method and device for classifying users | |
| CN111161238A (en) | Image quality evaluation method and device, electronic device, and storage medium | |
| CN111079930A (en) | Method and device for determining quality parameters of data set and electronic equipment | |
| CN117710763A (en) | Image noise recognition model training method, image noise recognition method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |