CN110516092B - Automatic image annotation method based on K nearest neighbor and random walk algorithm - Google Patents
Automatic image annotation method based on K nearest neighbor and random walk algorithm Download PDFInfo
- Publication number
- CN110516092B CN110516092B CN201910822291.5A CN201910822291A CN110516092B CN 110516092 B CN110516092 B CN 110516092B CN 201910822291 A CN201910822291 A CN 201910822291A CN 110516092 B CN110516092 B CN 110516092B
- Authority
- CN
- China
- Prior art keywords
- image
- probability
- label
- prediction
- vector
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/53—Querying
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24147—Distances to closest patterns, e.g. nearest neighbour classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/29—Graphical models, e.g. Bayesian networks
Abstract
The invention provides an automatic image labeling method based on K neighbor and random walk algorithm, belonging to the field of image retrieval. The method is characterized in that a vertex set of a probability graph model is constructed by adopting a K nearest neighbor algorithm, an edge set of a random walk graph is constructed based on the interrelation among labels in a training image label set, and the labels of an image to be labeled are automatically labeled by the random walk algorithm. The invention has good adaptability, accuracy and universality.
Description
Technical Field
The invention belongs to the field of image retrieval, and aims at the problem that manually marked images cannot compete with massive image marking tasks on the Internet, and automatic marking of the images is realized by using a K neighbor and random walk algorithm.
Background
The invention realizes automatic labeling of massive images of the Internet and provides basic support for image retrieval. The traditional manual image labeling method has large workload, and the subjectivity and the inaccuracy are inevitably brought, so that the automatic image labeling of a computer is imperative. The automatic labeling of the image is to make a computer automatically add semantic keywords capable of reflecting the content of the image to the image, and the use of the automatic labeling can effectively improve the difficulty of the current image retrieval.
The K-nearest neighbor algorithm is a typical representative of the lazy learning method, has no obvious training process, and has the advantages of high classification precision, insensitivity to abnormal values and no data input assumption. The random walk algorithm was first described mathematically by einstein in 1926. Since many entities in nature move in unpredictable ways, random walk algorithms are used to describe such unstable movement processes in which a mobile node randomly selects a direction and speed to move from a current location to a new location. The random walk algorithm is a global optimization method and has the advantage of being not easy to fall into a local minimum value.
Disclosure of Invention
The invention aims to improve the accuracy of similarity calculation between images by comprehensively considering the difference between discrete features and continuous features of the images and based on Hamming loss and a Gaussian kernel function, reduce the time complexity and the space complexity of the algorithm by constructing a probability graph model based on a K nearest neighbor algorithm, and realize automatic annotation of the images by performing random walk on the probability graph model. The method has good adaptability, accuracy and universality.
The image automatic labeling method based on the K nearest neighbor and the random walk algorithm comprises the following steps:
step (a 1): extracting the features of the image to form a training setAnd test setWhere i is the number of the image,for the ith image xiIs determined by the feature vector of (a),for continuous features, D1Is the number of consecutive features that are, for discrete features, D is the total number of features,is xiLabel set y ofiThe tag vector of (a) is determined,as a total label set,/qThe number of the Q label in the L is Q, the Q is the number of the label, and the Q is the total number of the labels;
step (a 2): computing x based on Hamming losses and Gaussian functionsiAnd xjSimilarity (x) betweeni,xj) (i, j ═ 1,2, … m + n); step (a 3): probability graph model constructed based on K nearest neighborsWhereinIs composed ofThe set of the vertices of (a) is,is composed ofThe edge set of (1);
step (a 4): prediction probability vector of image belonging to each label based on random walk algorithmWhereinIs xiBelong toqThe prediction probability of (i ═ m +1, m +2, …, m + n, Q ═ 1,2, …, Q);
step (a 5): construction of a predicted tag set y 'of an image based on predicted probabilities'i(i=m+1,m+2,…,m+n)。
The method for calculating the similarity between the images based on the Hamming loss and the Gaussian kernel function comprises the following steps:
step (B1) constructs x based on Hamming lossesiAnd xjSimilarity of each discrete feature therebetween(i, j ═ 1,2, … m + n, and i and j cannot be greater than m at the same time);
step (B2) constructs x based on Gaussian kernel functioniAnd xjSimilarity of each successive feature therebetweenWhereinRepresenting a vectorAndthe distance between the two adjacent electrodes is less than the total distance,is a regulatory factor;
step (B3) is performed by applying NomSimiliity (x)i,xj) And NumSimiliity (x)i,xj) Weighted summation, construction of xiAnd xjSimilarity between them
The method for constructing the probability map model based on the K nearest neighbor comprises the following steps:
step (C1): structure xiIn the training setMiddle K nearest neighborWherein K isThe number of the middle elements, x represents χtrainAll images in (1) and xiAfter the similarity between the images is sorted from big to small, x is one of the first K most similar images;
step (C3): structure of the deviceEdge set ofWhereinWherein s isThe number of the middle vertex is,are each vrAnd vsA set of labels corresponding to the image;
step (C4): constructing a probabilistic graphical modelThe steps of constructing the prediction probability of each label of the image based on the random walk algorithm are as follows:
step (D1): the structure is based onTransition matrix of WhereinIs composed ofRow r +1 and column s +1,
step (D2): the structure is based onIs a hop probability vectorWherein 1 isK+1A K +1 dimensional column vector with each component being 1;
step (D3): the structure is based onFormula of random walkWherein k is the iteration number, alpha is the probability of jumping during random walk,is the probability distribution vector at the kth random walk time,a probability distribution vector corresponding to 0 is k;
step (D4): continuously iterating until obtaining stable probability distribution vectorWhereinSatisfy the requirement ofIs an iteration error;
step (D5): calculating a final probability distribution vectorWhereinIs 1qIn thatA prior probability of (1);
a step (D6) of calculating a prediction probability vector of each label to which the image belongsWhereinIs xiBelong toqThe probability of prediction of (a) is,the steps of constructing the image prediction label set based on the label prediction probability adopted by the invention are as follows:
step (E4) ofFrom big to smallSelecting front length (y'i) And (4) taking the set of labels corresponding to the individual probabilities as an image prediction label set to finish automatic labeling of the image.
The invention has the following advantages:
1. the invention can label all types of images and has strong universality.
2. The invention can process images containing continuous features and discrete features and has strong adaptability.
3. The method is used for automatically marking the image based on the K nearest neighbor and the random walk algorithm, and is high in robustness and accuracy.
Drawings
FIG. 1 is a flow chart of the present invention for computing inter-image similarity based on Hamming loss and Gaussian kernel function
FIG. 2 is a flow chart of the present invention for constructing a probabilistic graphical model based on K-nearest neighbors
FIG. 3 is a flow chart of the present invention for constructing the prediction probability of each label belonging to an image based on the random walk algorithm
FIG. 4 is a flow chart of constructing an image prediction tag set based on tag prediction probability according to the present invention
FIG. 5 is a simplified flow chart of the present invention
Detailed Description
The invention relates to a method for constructing image similarity measurement based on Hamming loss and Gaussian kernel function, which constructs a probability graph model based on a K nearest neighbor algorithm and realizes automatic annotation of images by random walk on the probability graph model.
The process of calculating the similarity between images based on Hamming loss and Gaussian kernel function is as follows:
(1) as shown in fig. 1, the similarity between images with respect to discrete features is calculated by hamming loss, and the similarity between images with respect to continuous features is calculated by gaussian kernel function;
(2) and carrying out weighted average on the similarity between the discrete features and the similarity between the continuous features to obtain the similarity between the images.
The process of constructing the probability map model based on the K neighbors is as follows:
(1) as shown in fig. 2, a vertex set of the probability map model is constructed based on K neighbors of the image, and an edge set of the probability map model is constructed based on the correlation between corresponding label sets of the image;
(2) and combining the vertex set and the edge set into a probability graph model.
The process of constructing the prediction probability of each label of the image based on the random walk algorithm is as follows:
(1) constructing a state transition matrix and a jump probability vector in a random walk process based on a probability map model diagram as shown in FIG. 3;
(2) obtaining probability vectors of the image on each vertex in the probability map model through random walk;
(3) the probability that the image belongs to each label is calculated based on the probability distribution vector.
The process of constructing the image prediction tag set based on the tag prediction probability is as follows:
(1) as shown in fig. 4, the average length of the tag sets of the images belonging to the respective tags and the length of the predicted tag set of the image are calculated;
(2) a set of predictive labels for the image is constructed.
Claims (1)
1. An automatic image labeling method based on K neighbors and random walks is characterized in that similarity between images is calculated based on Hamming loss and a Gaussian kernel function, a probability model graph is constructed based on the K neighbors, prediction probabilities of the images belonging to all labels are constructed based on the random walks, an image prediction label set is constructed based on the label prediction probabilities, and the method sequentially comprises the following steps:
step (a 1): extracting the features of the image to form a training setAnd test setWhere i is the number of the image,for the ith image xiIs determined by the feature vector of (a),for continuous features, D1Is the number of consecutive features that are,for discrete features, D is the total number of features,is xiLabel set y ofiThe vector of the labels is then used to,L={l1,l2,…,lQas the total label set, lqThe number of the Q label in the L is Q, the number of the label is Q, and the number of the total label is Q;
step (a 2): computing x based on Hamming losses and Gaussian functionsiAnd xjSimilarity (x) betweeni,xj)(i,j=1,2,…m+n);
Step (a 3): probability model graph constructed based on K nearest neighborsWhereinIs composed ofThe set of the vertices of (a) is,is composed ofThe edge set of (1);
step (a 4): prediction summary for image belonging to each label based on random walk algorithmRate vectorWhereinIs xiBelong toqThe prediction probability of (i ═ m +1, m +2, …, m + n, Q ═ 1,2, …, Q);
step (a 5): construction of a predicted tag set y 'of an image based on predicted probabilities'i(i=m+1,m+2,…,m+n);
The method for calculating the similarity between the images based on the Hamming loss and the Gaussian kernel function comprises the following steps:
step (B1) constructs x based on Hamming lossesiAnd xjSimilarity between features with respect to each otherAnd i and j cannot be greater than m simultaneously);
step (B2) constructs x based on Gaussian kernel functioniAnd xjSimilarity between successive featuresWhereinRepresenting a vectorAndthe distance between the two adjacent electrodes is less than the total distance,is a regulatory factor;
step (B3) is performed by applying NomSimiliity (x)i,xj) And NumSimiliity (x)i,xj) Weighted summation, construction of xiAnd xjSimilarity between them
The method for constructing the probability model graph based on the K neighbors comprises the following steps:
step (C1): structure xiIn the training setMiddle K nearest neighbor Wherein K isNumber of elements (A) in (B), where x belongs to xiAt xtrainK in (1) is near neighbor represented in xtrainThe image corresponding to all the characteristic vectors in the image is xiIn the sequence of similarity from large to small, x is positioned in the first K;
step (C3): structure of the deviceEdge set ofWhereinWherein s isThe number of the second vertex in the list,are each vr、vsA set of labels for the corresponding image;
The method for constructing the prediction probability of the image belonging to each label based on the random walk comprises the following steps:
step (D1): the structure is based onTransition matrix of WhereinIs composed ofRow r +1 and column s +1,
step (D2): the structure is based onIs a hop probability vectorWherein 1 isK+1Is a K +1 dimensional column vector with a component of 1;
step (D3): the structure is based onFormula of random walkWhere k is the probability of the occurrence of a jump when the iteration number alpha is random walk,is the probability distribution vector at the kth random walk time,a probability distribution vector corresponding to 0 is k;
step (D4): continuously iterating until obtaining stable probability distribution vectorWhereinSatisfy the requirement ofIs an iteration error;
step (D5): calculating a final probability distribution vectorWhereinIs 1qIn thatA prior probability of (1);
a step (D6) of calculating a prediction probability vector of each label to which the image belongs WhereinIs xiBelong toqThe probability of prediction of (a) is,
the image prediction label set is constructed based on the label prediction probability, and the steps are as follows:
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910822291.5A CN110516092B (en) | 2019-09-02 | 2019-09-02 | Automatic image annotation method based on K nearest neighbor and random walk algorithm |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910822291.5A CN110516092B (en) | 2019-09-02 | 2019-09-02 | Automatic image annotation method based on K nearest neighbor and random walk algorithm |
Publications (2)
Publication Number | Publication Date |
---|---|
CN110516092A CN110516092A (en) | 2019-11-29 |
CN110516092B true CN110516092B (en) | 2020-12-01 |
Family
ID=68629210
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201910822291.5A Active CN110516092B (en) | 2019-09-02 | 2019-09-02 | Automatic image annotation method based on K nearest neighbor and random walk algorithm |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN110516092B (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116049476A (en) * | 2023-03-06 | 2023-05-02 | 福建福清核电有限公司 | Remote sensing image semantic locating method based on multi-level likelihood expansion |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105808752A (en) * | 2016-03-10 | 2016-07-27 | 大连理工大学 | CCA and 2PKNN based automatic image annotation method |
CN109800818A (en) * | 2019-01-25 | 2019-05-24 | 宝鸡文理学院 | A kind of image meaning automatic marking and search method and system |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102298605B (en) * | 2011-06-01 | 2013-04-17 | 清华大学 | Image automatic annotation method and device based on digraph unequal probability random search |
CN108268510B (en) * | 2016-12-30 | 2022-01-28 | 华为技术有限公司 | Image annotation method and device |
US11899669B2 (en) * | 2017-03-20 | 2024-02-13 | Carnegie Mellon University | Searching of data structures in pre-processing data for a machine learning classifier |
CN109670552B (en) * | 2018-12-24 | 2023-03-10 | 苏州大学 | Image classification method, device and equipment and readable storage medium |
-
2019
- 2019-09-02 CN CN201910822291.5A patent/CN110516092B/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105808752A (en) * | 2016-03-10 | 2016-07-27 | 大连理工大学 | CCA and 2PKNN based automatic image annotation method |
CN109800818A (en) * | 2019-01-25 | 2019-05-24 | 宝鸡文理学院 | A kind of image meaning automatic marking and search method and system |
Non-Patent Citations (2)
Title |
---|
"基于K近邻的多标签学习方法研究";余强;《中国优秀硕士学位论文全文数据库 信息科技辑》;20140915(第09期);第I138-729页 * |
"聚类分析及其在图像处理中的应用";肖宇;《中国博士学位论文全文数据库 信息科技辑》;20130515(第5期);第I138-53页 * |
Also Published As
Publication number | Publication date |
---|---|
CN110516092A (en) | 2019-11-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Gao et al. | A semi-supervised convolutional neural network-based method for steel surface defect recognition | |
WO2022068196A1 (en) | Cross-modal data processing method and device, storage medium, and electronic device | |
CN110309331B (en) | Cross-modal deep hash retrieval method based on self-supervision | |
CN110472090B (en) | Image retrieval method based on semantic tags, related device and storage medium | |
CN111080400B (en) | Commodity recommendation method and system based on gate control graph convolution network and storage medium | |
Yaseen et al. | Deep learning hyper-parameter optimization for video analytics in clouds | |
WO2018150812A1 (en) | Balancing active learning | |
Adhikari et al. | Iterative bounding box annotation for object detection | |
CN110009430B (en) | Cheating user detection method, electronic device and computer readable storage medium | |
Islam et al. | InceptB: a CNN based classification approach for recognizing traditional bengali games | |
CN114186084A (en) | Online multi-mode Hash retrieval method, system, storage medium and equipment | |
CN110929119A (en) | Data annotation method, device, equipment and computer storage medium | |
CN110516092B (en) | Automatic image annotation method based on K nearest neighbor and random walk algorithm | |
CN113610610B (en) | Session recommendation method and system based on graph neural network and comment similarity | |
ElAlami | Unsupervised image retrieval framework based on rule base system | |
Zhang et al. | NAS4FBP: Facial beauty prediction based on neural architecture search | |
CN115424012A (en) | Lightweight image semantic segmentation method based on context information | |
Janković Babić | A comparison of methods for image classification of cultural heritage using transfer learning for feature extraction | |
Bahrami et al. | Automatic image annotation using an evolutionary algorithm (IAGA) | |
Shavit et al. | From Xception to NEXcepTion: New Design Decisions and Neural Architecture Search | |
CN113987170A (en) | Multi-label text classification method based on convolutional neural network | |
CN112101086A (en) | Face clustering method based on link prediction | |
Aggarwal et al. | Object Detection Based Approaches in Image Classification: A Brief Overview | |
CN113743168B (en) | Urban flyer identification method based on micro-depth neural network search | |
CN116977021B (en) | Automatic pushing method for system butt joint based on big data |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |