CN113128472B - Multi-label labeling method based on intelligent collaborative learning - Google Patents
Multi-label labeling method based on intelligent collaborative learning Download PDFInfo
- Publication number
- CN113128472B CN113128472B CN202110534862.2A CN202110534862A CN113128472B CN 113128472 B CN113128472 B CN 113128472B CN 202110534862 A CN202110534862 A CN 202110534862A CN 113128472 B CN113128472 B CN 113128472B
- Authority
- CN
- China
- Prior art keywords
- label
- image
- training set
- vectors
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/24323—Tree-organised classifiers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Software Systems (AREA)
- Medical Informatics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a multi-label labeling algorithm taking an improved label embedding method as a core, which comprises the following steps: constructing a national costume pattern label system and a data set, forming a set of label system related to the national costume pattern by analyzing the characteristics and the content of the national costume pattern, and constructing a set of national costume pattern data set on the basis of the set of label system; performing feature extraction on the national costume pattern image by using a deep learning method; expressing the relation among the labels by constructing a label network, and optimizing the weight value representation of the label network; compressing the label network by a Laplace characteristic dimension reduction method and reserving the structure of the original label network; and automatically labeling the national costume patterns by combining the ensemble learning and the multi-label classification learner.
Description
Technical Field
The invention relates to the technical field of machine learning, in particular to a multi-label labeling method based on intelligent collaborative learning.
Background
The national dress pattern field, a certain image relates to a plurality of aspects of the cultural field, including fields such as name, meaning, configuration, color and the like. For such image data with ambiguity, all contents of the image data cannot be completely explained by using a single label, while multi-label labeling can accurately and completely describe cultural contents of national dress patterns, and a user can acquire rich cultural contents of the national dress patterns in an automatic labeling mode.
The multi-label labeling is a popular research direction in the field of current machine learning, and the multi-label labeling method based on label embedding provided by researchers is suitable for numerous and complicated national dress pattern data, so that the problems of overlarge label space and unbalanced sample label categories are solved well.
However, the label embedding method has some problems in terms of label space dimension reduction, and particularly when the number of labels is too large, problems of inaccurate dimension reduction extraction, too low speed and the like are generated, so that the pre-operation of label embedding and optimization of the pre-operation need to be performed.
Disclosure of Invention
The invention aims to overcome the defects in the prior art and provide a multi-label labeling method based on intelligent collaborative learning. The national costume pattern culture content can be automatically marked.
The invention provides a multi-label labeling method based on intelligent collaborative learning, which is characterized by comprising the following steps: the method comprises the following steps:
step 1, acquiring a national costume pattern image, and performing manual marking to form a set of complete national costume pattern data and corresponding label contents, wherein each label content comprises a plurality of independent labels;
step 2, dividing the ethnic clothing pattern data and the corresponding label content into a training set and a verification set;
step 3, training the algorithm model in the training set, and specifically comprising the following steps:
3.1, extracting characteristic vectors of the ethnic clothing pattern data in the training set through a residual error network, wherein all the characteristic vectors form an image characteristic space;
3.2, taking each independent label of the training set as a node in the network structure, forming an edge between two labels when the two labels simultaneously appear in a sample example, and forming the weight of the edge through the co-occurrence frequency of the labels to complete the construction of the label network, wherein the weight calculation formula of each edge is as follows:
w (i, j) represents the weight of the edge between label i and label j, wherein the numerator represents the ethnicity of label i and label j in the training setThe number of times of simultaneous occurrence in the label content of the clothing pattern, the denominator represents the total number of the occurrence of the label i and the label j in the label content of the national clothing pattern of the training set, y s,i 1 indicates the presence of the label i, y in the sample s s,j 1 indicates the presence of label j in sample s; w is a i ，w j Respectively representing weight vectors formed by the weights of edges taking the label i and the label j as end points in the label network,
c is a constant value, and c is more than 0 and less than 1;
3.3, reducing the dimension of the label network through Laplace feature mapping to obtain a label embedding space, and determining a label embedding vector corresponding to each image feature vector through a polymerization method;
3.4, training the decision tree integration regression model by using a training set, wherein input data are image feature vectors of each image in the training set and corresponding label embedded vectors;
step 4, using the verification set to label multiple labels
4.1, extracting and verifying feature vectors of the centralized ethnic clothing pattern data through a residual error network, wherein all the feature vectors form an image feature space;
4.2, performing regression on the image feature space by using the trained set-pair decision tree integrated regression model, and outputting a label embedded vector corresponding to the image feature vector;
and 4.3, performing multi-label classification on the label embedded vector obtained after regression by using a classifier to obtain a final prediction result of the verification set, namely an image labeling result.
The innovation of the invention is that:
1. and constructing a label network to express the relation between the labels, and optimizing the weight value of the label network through a kernel function.
2. The deep learning model, the embedding method, the ensemble learning method and the multi-label classification method are combined to form a new multi-label labeling algorithm, and a better labeling effect is achieved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flowchart of a multi-label labeling method based on intelligent collaborative learning according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Specifically, the multi-label labeling method based on intelligent collaborative learning provided by the embodiment includes the following steps:
s110, acquiring national clothing pattern images by means of digital scanning of books, and manually marking the images through marking software to form a set of complete national clothing pattern data and corresponding label contents, wherein each label content comprises a plurality of independent labels.
S120, processing the images and the label contents into a computer 0/1 adjacency matrix by using a data set construction program, and forming a training set and a verification set which are suitable for an algorithm model. In this example, the data set is randomly divided into two parts, a training set and a validation set, in a 9:1 ratio.
S130, training the algorithm model in a training set, and specifically comprising the following steps:
a1, extracting the characteristic vectors of the national costume pattern data in the training set through a residual error network, wherein all the characteristic vectors form an image characteristic space.
In the step, an objective function BCEWithLoitsLoss which is suitable for multi-label classification is utilized to extract image characteristics of the Resnet model which is pre-trained by ImageNet through a ethnic clothing pattern data set.
The specific process is as follows:
the input picture 224 x 224 sized area was randomly chosen and the input picture samples were increased with horizontal flipping and standard color enhancement was used. And removing the original full connection layer in Resnet, reconstructing the full connection layer, and extracting the image characteristics of the ethnic clothing pattern data set by using the Resnet model pre-trained by ImageNet by using a target function BCEWithLoctissoss suitable for multi-label classification. The 2048 dimensional image features are exported from the Resnet fully connected layer onwards. BCELoss may calculate a binary loss between the predicted output and the true output as a criterion for single label classification. BCEwithLogitsLoss is a combination of a Sigmoid function and a BCELoss function. The calculation models used in this step are all existing models, see the paper: he K, Zhang X, Ren S, et al. deep reactive Learning for Image registration [ C ]. IEEE Conference on Computer Vision and Pattern Registration (CVPR). IEEE Computer Society,2016 (see).
a2, taking each label in the training set as a node in the network structure, forming an edge between two labels when the two labels appear in a sample example at the same time, and forming the weight of the edge through the co-occurrence frequency of the labels to complete the construction of the label network, wherein the weight calculation formula of each edge is as follows:
w (i, j) represents the weight of the edge between the label i and the label j, wherein the numerator represents the number of times that the label i and the label j simultaneously appear in the label content of the ethnic clothing pattern of the training set, and the denominator represents the number of times that the label i and the label j simultaneously appear in the training setTotal number of occurrences in tag content of national dress pattern, y s,i 1 indicates the presence of the label i, y in the sample s s,j 1 indicates the presence of label j in sample s; w is a i ，w j Each weight vector represents a weight of an edge having a label i and a label j as end points in the label network, represents an approximation relationship between the label and another label, and considers the degree of association of the label from the viewpoint of a sample example. The incidence relation of the two labels can be further considered from the perspective of the labels by utilizing the kernel function, if the two labels coexist with more same labels, the two labels can be similar, otherwise, the two labels are more independent.
The weight value constructed by the kernel function has the characteristics that,
the larger the tag i is, the more similar the tag j is, and vice versa, the more independent. c is constant, and c is more than 0 and less than 1.
a3, reducing dimensions of the label network through Laplacian feature mapping to obtain a label embedding space, and determining a label embedding vector corresponding to each image feature vector through a polymerization method. In this step, the laplace feature dimension reduction method is a known method, and is described in the literature: belkin M, Niyogi P.Laplacian eigenmaps and spectral techniques for embedding and curing [ C]585-591, also known, see: for vector v i ，ξ(v i )＝v i1 +v i2 +v i3 +...v in Mathematically, a vector aggregation operation (vector summation).
a4, selecting a more direct regression method to map the feature space to the label embedding space.
In this embodiment, the regression method adopts an integrated regression model for the decision tree, and the input data is the image feature vector and the corresponding label embedded vector of each image in the training set. The decision tree regression integration algorithm adopts the square average error as the judgment standard of the purity of the decision tree nodes, and the number N of the decision trees is 10.
Considering that the feature space dimension is high, the formed decision tree may have a deep depth and a poor generalization capability, the embodiment performs post-pruning on a single decision tree by using the CART pruning method. The decision tree integration regression model is an existing regression learning method, and is described in the literature: liaw A, Wiener M, classification and regression by random forest [ J ]. R news,2002,2(3):18-22.
Step 4, using the verification set to label multiple labels
b1, extracting the characteristic vectors of the ethnic clothing pattern data in the verification set through a residual error network, wherein all the characteristic vectors form an image characteristic space. This step can be referred to as step a1 and will not be described in detail herein.
b2, performing regression on the image feature space by using the trained set-pair decision tree integrated regression model, and outputting a label embedded vector corresponding to the image feature vector.
b3, performing multi-label classification on the label embedding vector obtained after regression by using a classifier to obtain a final prediction result of the verification set, namely an image labeling result.
In this example, the ML-KNN multi-label classification learner is used to perform multi-label classification. The ML-KNN multi-label classification learner firstly needs to use training set data for training, input data are label embedding vectors and original label vectors corresponding to image features in a training set, and the original label vectors are formed by independent labels contained in the content of each label. The original label vector and the regression result (label embedded vector) are used as the input of the ML-KNN multi-label classifier to train, so that the original characteristic input can be reserved to a certain extent, and the decoding deviation caused by regression error is reduced. The ML-KNN classifier belongs to the prior art category, see literature: zhang M L, Zhou Z H.ML-KNN A lazy learning approach to multi-label learning [ J ]. Pattern recognition,2007,40(7): 2038-.
Experiments were performed on a national costume pattern dataset. The smaller the value of One-error, Coverage, Ranking-loss and Hamming-Los indexes adopted by the evaluation indexes is, the better the performance of the algorithm on the indexes is, and the higher the value of Average-precision is, the better the performance of the algorithm on the indexes is.
The results of the experiment are as follows:
in addition to the above embodiments, the present invention may have other embodiments. All technical solutions formed by adopting equivalent substitutions or equivalent transformations fall within the protection scope of the claims of the present invention.
Claims (7)
1. A multi-label labeling method based on intelligent collaborative learning is characterized in that: the method comprises the following steps:
step 1, acquiring a national costume pattern image, and performing manual marking to form a set of complete national costume pattern data and corresponding label contents, wherein each label content comprises a plurality of independent labels;
step 2, dividing the ethnic clothing pattern data and the corresponding label content into a training set and a verification set;
step 3, training the algorithm model in the training set, and specifically comprising the following steps:
3.1, extracting characteristic vectors of the ethnic clothing pattern data in the training set through a residual error network, wherein all the characteristic vectors form an image characteristic space;
3.2, taking each independent label of the training set as a node in a network structure, forming an edge between two labels when the two labels simultaneously appear in a sample example, and forming the weight of the edge through the co-occurrence frequency of the labels to complete the construction of the label network, wherein the weight calculation formula of each edge is as follows:
w (i, j) represents the weight of the edge between the label i and the label j, wherein the numerator represents the number of times that the label i and the label j simultaneously appear in the label content of the national costume pattern of the training set, the denominator represents the total number of the label i and the label j appearing in the label content of the national costume pattern of the training set, y s,i 1 indicates the presence of the label i, y in the sample s s,j 1 indicates the presence of label j in sample s; w is a i ，w j Respectively representing weight vectors formed by the weights of edges taking the label i and the label j as end points in the label network,
c is a constant value, and c is more than 0 and less than 1;
3.3, reducing the dimension of the label network through Laplace feature mapping to obtain a label embedding space, and determining a label embedding vector corresponding to each image feature vector through a polymerization method;
3.4, training the decision tree integration regression model by using a training set, wherein input data are image feature vectors of each image in the training set and corresponding label embedded vectors;
step 4, using the verification set to label multiple labels
4.1, extracting and verifying feature vectors of the centralized ethnic clothing pattern data through a residual error network, wherein all the feature vectors form an image feature space;
4.2, performing regression on the image feature space by using the trained set-pair decision tree integrated regression model, and outputting a label embedded vector corresponding to the image feature vector;
and 4.3, performing multi-label classification on the label embedded vector obtained after regression by using a classifier to obtain a final prediction result of the verification set, namely an image labeling result.
2. The multi-label labeling method based on intelligent collaborative learning according to claim 1, characterized in that: in the step 1, after the book is scanned by using the scanner, the national clothing pattern is identified by manual visual observation through the image marking tool, and the corresponding marking file is obtained by manual marking.
3. The multi-label labeling method based on intelligent collaborative learning according to claim 1, characterized in that: in the step 1, the data set is randomly divided into a training set and a verification set according to a ratio of 9: 1.
4. The multi-label labeling method based on intelligent collaborative learning according to claim 1, characterized in that: in the step 3.1, the image feature extraction is carried out on the ethnic clothing pattern data set by the Resnet model after ImageNet pre-training by using the target function BCEWithLoitsLoss suitable for multi-label classification.
5. The multi-label labeling method based on intelligent collaborative learning according to claim 1, characterized in that: in step 3.4, the decision tree regression integration algorithm adopts the square mean error as the judgment standard of the node impure degree of the decision tree, and the number N of the decision trees is 10.
6. The multi-label labeling method based on intelligent collaborative learning according to claim 1, characterized in that: and 4.3, performing multi-label classification by using an ML-KNN multi-label classification learning device.
7. The multi-label labeling method based on intelligent collaborative learning according to claim 6, characterized in that: and 4.3, training the ML-KNN multi-label classification learning device by using the training set, wherein input data are label embedded vectors and original label vectors corresponding to the image features in the training set, and the original label vectors are formed by independent labels contained in the content of each label.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110534862.2A CN113128472B (en) | 2021-05-17 | 2021-05-17 | Multi-label labeling method based on intelligent collaborative learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110534862.2A CN113128472B (en) | 2021-05-17 | 2021-05-17 | Multi-label labeling method based on intelligent collaborative learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113128472A CN113128472A (en) | 2021-07-16 |
| CN113128472B true CN113128472B (en) | 2022-09-20 |
Family
ID=76782086
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110534862.2A Active CN113128472B (en) | 2021-05-17 | 2021-05-17 | Multi-label labeling method based on intelligent collaborative learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113128472B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104216993A (en) * | 2014-09-10 | 2014-12-17 | 武汉科技大学 | Tag-co-occurred tag clustering method |
| CN104881689A (en) * | 2015-06-17 | 2015-09-02 | 苏州大学张家港工业技术研究院 | Method and system for multi-label active learning classification |
| CN110163234A (en) * | 2018-10-10 | 2019-08-23 | 腾讯科技（深圳）有限公司 | A kind of model training method, device and storage medium |
-
2021
- 2021-05-17 CN CN202110534862.2A patent/CN113128472B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104216993A (en) * | 2014-09-10 | 2014-12-17 | 武汉科技大学 | Tag-co-occurred tag clustering method |
| CN104881689A (en) * | 2015-06-17 | 2015-09-02 | 苏州大学张家港工业技术研究院 | Method and system for multi-label active learning classification |
| CN110163234A (en) * | 2018-10-10 | 2019-08-23 | 腾讯科技（深圳）有限公司 | A kind of model training method, device and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| Double Attention for Multi-Label Image Classification;HAIYING ZHAO 等;《IEEE ACCESS》;20201214;第8卷;全文 * |
| 基于多标签学习的卷积神经网络的图像标注方法;高耀东 等;《计算机应用》;20171231;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113128472A (en) | 2021-07-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106599854B (en) | Automatic facial expression recognition method based on multi-feature fusion | |
| CN112131978B (en) | Video classification method and device, electronic equipment and storage medium | |
| CN107480620B (en) | Remote sensing image automatic target identification method based on heterogeneous feature fusion | |
| CN112734775A (en) | Image annotation, image semantic segmentation and model training method and device | |
| CN102054178A (en) | Chinese painting image identifying method based on local semantic concept | |
| CN109002755B (en) | Age estimation model construction method and estimation method based on face image | |
| CN110188708A (en) | A kind of facial expression recognizing method based on convolutional neural networks | |
| CN104850822B (en) | Leaf identification method under simple background based on multi-feature fusion | |
| CN105760858A (en) | Pedestrian detection method and apparatus based on Haar-like intermediate layer filtering features | |
| CN107622280B (en) | Modularized processing mode image saliency detection method based on scene classification | |
| CN109886161A (en) | A kind of road traffic index identification method based on possibility cluster and convolutional neural networks | |
| JP4553300B2 (en) | Content identification device | |
| CN110413825B (en) | Street-clapping recommendation system oriented to fashion electronic commerce | |
| CN105718932A (en) | Colorful image classification method based on fruit fly optimization algorithm and smooth twinborn support vector machine and system thereof | |
| CN110852327A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN110046544A (en) | Digital gesture identification method based on convolutional neural networks | |
| CN110599463A (en) | Tongue image detection and positioning algorithm based on lightweight cascade neural network | |
| CN113705579A (en) | Automatic image annotation method driven by visual saliency | |
| CN108647703B (en) | Saliency-based classification image library type judgment method | |
| CN114187520A (en) | Building extraction model and application method thereof | |
| CN106650798A (en) | Indoor scene recognition method combining deep learning and sparse representation | |
| CN112001438A (en) | Multi-mode data clustering method for automatically selecting clustering number | |
| CN114782979A (en) | Training method and device for pedestrian re-recognition model, storage medium and terminal | |
| CN109658523A (en) | The method for realizing each function operation instruction of vehicle using the application of AR augmented reality | |
| CN113128472B (en) | Multi-label labeling method based on intelligent collaborative learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |