CN101685464B - Method for automatically labeling images based on community potential subject excavation - Google Patents
Method for automatically labeling images based on community potential subject excavation Download PDFInfo
- Publication number
- CN101685464B CN101685464B CN2009100999166A CN200910099916A CN101685464B CN 101685464 B CN101685464 B CN 101685464B CN 2009100999166 A CN2009100999166 A CN 2009100999166A CN 200910099916 A CN200910099916 A CN 200910099916A CN 101685464 B CN101685464 B CN 101685464B
- Authority
- CN
- China
- Prior art keywords
- image
- community
- label
- candidate
- marked
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method for automatically labeling images based on community potential subject excavation, which comprises the following steps: 1) adopting a hidden Dirichlet allocation model to excavate implicit subjects in a single community; 2) after obtaining the probability distribution of image labels and the implicit subjects by analyzing community potential subjects, deleting the image labels of which the probability of the community image labels and the implicit subjects is smaller than a set value k to perform 'de-noising' filtration on the community image labels; 3) generating image candidate labeling labels of the images to be labeled by propagating similar image labels; 4) optimizing the image candidate labeling labels according to the relativity between the image candidate labeling labels and the implicit subjects of the images; and 5) obtaining the final labeling result of the images through the information fusion of a plurality of communities. The method makes full use of the information on different communities in which the images are positioned and the information on the community potential subjects in a social shared network to label the images, and compared with the conventional labeling method, the generated labeling result is more accurate.
Description
Technical field
The present invention relates to the automatic mark field of image, relate in particular to a kind of method that marks automatically based on the image of social sharing network.
Background technology
We is along with the fast development of network and multimedia technology, and the amount of images on the internet is explosive increase.According to statistics, 2008, Google index Web webpage scale reached 1,000,000,000,000, and wherein view data is above tens.In recent years, shared network has caused Internet user's special concern, marks on the Flickr of website the masses that provide digital picture to share, and the image of its index surpasses 3,000,000,000, and increases fast with every month speed of millions of.
The image tag information that the Internet user manually adds for the Flickr image is that the high-efficiency management and the retrieval of image brought very big facility.But, analyse in depth discovery by result to the manual mark of Flickr image, the label of 64% image all is less than or equals 3.How the image of a large amount of no labels or label deficiency is added automatically or improves it and have the hot issue that label is a current research.
Different with normal image, the internet is shared image and is had following several characteristics:
Shared network picture quality is uneven, by different user by different cameral at different time from different angles or use the difference skill of taking pictures to take and obtain;
The shared network picture material is abundant, the label entry of Flickr image has surpassed 100,000,000 3 thousand ten thousand, has contained more than 6,000 ten thousand notions, has included various content, incident and object or the like such as landscape, building, personal portrait, active clip;
Shared network image, semantic complexity, an image often comprises a plurality of different subject informations simultaneously, may both comprise subject informations such as " Sky ", " Clouds " such as an image, has also comprised subject informations such as " Water ", " River " simultaneously.
Because the shared network image has These characteristics, therefore be difficult to use traditional algorithm that it is effectively marked.The shared image of analysing in depth on the Flickr can be found a notable feature: when the user according to time, place or incident with image uploading behind album, can further it be recommended in the corresponding community and go according to image subject.Community among the Flickr is meant the image collection that comprises a certain particular topic, and when the user uploads the image on community the time that does not meet the community theme, the keeper can delete these unrelated images, and this has just guaranteed the consistance on the community image subject.Therefore, can utilize the subject information of image place community that image is marked.Simultaneously, can further be subdivided into the fact of a plurality of sub-topicses again at a certain community theme, can imply theme to community and excavate, the combining image visual similarity is finally obtained meticulousr annotation results then.
Summary of the invention
The objective of the invention is to overcome the deficiencies in the prior art, a kind of method of the automatic image annotation that excavates based on the potential theme of community is provided.
The method of the automatic image annotation that excavates based on the potential theme of community comprises the steps:
1) adopt latent Di Li Cray apportion model that the implicit theme in the single community is excavated;
2) by after potential subject analysis obtains the probability distribution of image tag and implicit theme to community, deletion community image tag and implicit theme probability come the community image tag is carried out " denoising " filtration less than the image tag of setting value k;
3) propagate the image candidate who produces image to be marked by the similar image label and mark label;
4) marking between the implicit theme of label and image correlativity according to the image candidate marks label to the image candidate and is optimized;
5) obtain the final annotation results of image by many community information fusion.
Described image candidate by similar image label propagation generation image to be marked marks the step of label: for image I to be marked in the community
u, image I to be marked
uAnd the probability between the image tag w calculates from following formula:
Wherein P (w|J) represents image tag w occurrence number among the training image J, P (I
u| J) represent image I to be marked
uAnd the visual similarity between the training image J, choose and image I to be marked
uThe highest pairing image tag w of 10 width of cloth training image J of visual similarity is as image I to be marked
uThe candidate mark label, i.e. P (w|I
u) be worth 10 maximum image tag w as image I to be marked
uThe image candidate mark label.
Correlativity marks the step that label is optimized to the image candidate between the described implicit theme that marks label and image according to the image candidate:
1) marks label w by calculating in all implicit themes that two image candidates mark probability product between the label and obtaining the image candidate
kAnd w
lBetween implicit topic similarity, computing formula is:
The probability distribution of ф presentation video label and implicit theme wherein;
2) marking label and other image candidate by the computed image candidate marks implicit topic relativity sum between the label and obtains the image candidate and mark label w
iWith image I to be marked
uThe correlativity of implicit theme, computing formula is:
P (w wherein
j| w
i) the presentation video candidate marks label w
jAnd w
iBetween implicit topic similarity;
3) recomputate the image candidate and mark label w
iWith image I to be marked
uProbability, computing formula is: P ' (w
i| I
u)=P (w
i| I
u) * R (w
i, I
u), P (w|I wherein
u) represent image I to be marked
uWith image tag w
iBetween probability, R (w
i, I
u) the presentation video candidate marks label w
iWith image I to be marked
uThe correlativity of implicit theme.
Described step of image being carried out final mark by many community information fusion:
1) by being chosen at the theme that the most frequent image tag is represented community appears in the community from the title of each community, just in WordNet " entity " semantic tree, find the node of representing this community by this image tag then, constitute the HD between each community;
2) by the HD between each community, each community is carried out final mark by merging to image from top to bottom successively, for obtaining a new father node by the markup information of each child node community is averaged between each community that contains common ancestor's node, delete child node, reach the purpose of fusion;
3) mark the final annotation results that preceding 5 values of label obtain image to be marked by choosing the image candidate.
The present invention has made full use of the information of the different community at image place, and the potential subject information that utilizes the community place comes the mark label is carried out " denoising " and optimization, therefore more accurate than traditional result that mask method marked, markup information is also more extensive.
Description of drawings
Fig. 1 is based on the method flow diagram of the automatic image annotation of the potential theme excavation of community.
Fig. 2 is automatic image annotation result of the present invention.
Fig. 3 is latent Di Li Cray apportion model.
Embodiment
The method of the automatic image annotation that excavates based on the potential theme of community comprises the steps:
1) adopt latent Di Li Cray apportion model that the implicit theme in the single community is excavated;
2) by after potential subject analysis obtains the probability distribution of image tag and implicit theme to community, deletion community image tag and implicit theme probability come the community image tag is carried out " denoising " filtration less than the image tag of setting value k;
3) propagate the image candidate who produces image to be marked by the similar image label and mark label;
4) marking between the implicit theme of label and image correlativity according to the image candidate marks label to the image candidate and is optimized;
5) obtain the final annotation results of image by many community information fusion.
The latent Di Li Cray apportion model of described employing is as follows to the step that the implicit theme in the single community excavates:
1) latent Di Li Cray apportion model is commonly used to text is carried out subject analysis; Concern between image (document) d, implicit theme z, the image tag w in the latent Di Li Cray apportion model (as Fig. 3) mainly by implicit variable θ and φ decision, wherein θ presentation video d theme distributes, the β z label that is the theme distributes, and α, β are the prior probability of implicit variable θ, φ.Prior probability α, β obey the Di Li Cray and distribute, and T is a community theme sum, and D is the community total number of images, N
dBe each image tagged total number of labels;
2) because direct computed image is concentrated the probability more complicated between implicit theme z and image d and the image tag w, adopt Gibbs to sample usually and simplify the LDA Model Calculation; For i image tag token, the image tag index of this token is w
i, token corresponding image index is d
i, Gibbs sampling alternately is considered each image tag token, by calculating the number of times that other token is assigned to each theme, estimates which theme current token is assigned to.In this process, theme is recycled sampling, and the theme conditional probability is:
Wherein, z
i=j represents that theme j is given token i, z by assignment
-iExpression except token i the theme of other image tag token distribute, " " represents other all Given informations, such as all other image tag index w
-i, image index d
-iAnd prior probability α, β.C
WTWith C
DTBe respectively that size is W*T, D*T dimension matrix.
Presentation video label w is assigned to the number of times of theme j,
The image tag of presentation video d the inside is assigned to the number of times (not comprising current label token i) of theme j;
3) in each Gibbs sampling, all images label all is assigned to some themes in the image set.When Gibbs sampling by iteration behind enough number of times, the theme probability just approaches priori Di Li Cray and distributes.After the Gibbs sampling finished, the label-theme distribution ф and the theme-image θ that have just obtained finding the solution distributed, and the theme conditional probability is:
Wherein
Presentation video label w is assigned to the number of times of theme j,
The image tag of presentation video d the inside is assigned to the number of times of theme j, the number of W presentation video label, and T represents the number of theme, α, β represent prior probability.
Described image candidate by similar image label propagation generation image to be marked marks the step of label: for image I to be marked in the community
u, image I to be marked
uAnd the probability between the image tag w calculates from following formula:
Wherein P (w|J) represents image tag w occurrence number among the training image J, P (I
u| J) represent image I to be marked
uAnd the visual similarity between the training image J, choose and image I to be marked
uThe highest pairing image tag w of 10 width of cloth training image J of visual similarity is as image I to be marked
uThe candidate mark label, i.e. P (w|I
u) be worth 10 maximum image tag w as image I to be marked
uThe image candidate mark label.
Correlativity marks the step that label is optimized to the image candidate between the described implicit theme that marks label and image according to the image candidate:
1) marks label w by calculating in all implicit themes that two image candidates mark probability product between the label and obtaining the image candidate
kAnd w
lBetween implicit topic similarity, computing formula is:
The probability distribution of ф presentation video label and implicit theme wherein;
2) marking label and other image candidate by the computed image candidate marks implicit topic relativity sum between the label and obtains the image candidate and mark label w
iWith image I to be marked
uThe correlativity of implicit theme, computing formula is:
P (w wherein
j| w
i) the presentation video candidate marks label w
jAnd w
iBetween implicit topic similarity;
3) recomputate the image candidate and mark label w
iWith image I to be marked
uProbability, computing formula is: P ' (w
i| I
u)=P (w
i| I
u) * R (w
i, I
u), P (w|I wherein
u) represent image I to be marked
uWith image tag w
iBetween probability, R (w
i, I
u) the presentation video candidate marks label w
iWith image I to be marked
uThe correlativity of implicit theme.
Described step of image being carried out final mark by many community information fusion:
1) by being chosen at the theme that the most frequent image tag is represented community appears in the community from the title of each community, just in WordNet " entity " semantic tree, find the node of representing this community by this image tag then, constitute the HD between each community;
2) by the HD between each community, each community is carried out final mark by merging to image from top to bottom successively, for obtaining a new father node by the markup information of each child node community is averaged between each community that contains common ancestor's node, delete child node, reach the purpose of fusion;
3) mark the final annotation results that preceding 5 values of label obtain image to be marked by choosing the image candidate.
The present invention has made full use of the information of the different community in image place in the social sharing network, and utilize the potential subject information in community place to come the mark label is carried out " denoising " and optimization, therefore more accurate than the annotation results of traditional mask method generation, markup information is also more extensive.
As shown in Figure 1, the method for the automatic image annotation that excavates based on the potential theme of community specifies as follows:
1), finds N the different community at this image place for an image to be marked;
2) distributing mould to imply theme to the latent Di Li Cray of each community utilization excavates;
3) according to the correlativity of community label and the implicit theme of community the community label being carried out " denoising " filters;
4) propagate the image candidate who produces image to be marked by the similar image label and mark label;
5) marking between the implicit theme of label and image correlativity according to the image candidate marks label to the image candidate and is optimized;
6) come image is marked by many community information fusion;
7) obtain the final image annotation results of image to be marked.
Embodiment 1
Fig. 2 has provided an object lesson of the automatic image annotation that excavates based on the potential theme of community.
1) chooses an image to be marked, find 3 different community at this image place: community 1 " Water, Oceans, Lakes, Rivers, Creeks ", community 2 " Sky ﹠amp; Clouds ", community 3 " Beautiful Scenery ";
2) utilizing latent Di Li Cray to distribute mould to imply theme respectively to 3 community excavates;
3) according to the correlativity of community label and the implicit theme of community 3 community labels being carried out " denoising " filters;
4) propagate the image candidate who produces image to be marked by the similar image label and mark label " river sanwater antonio bexar county courthouse blue clouds sea ";
5) marking between the implicit theme of label and image correlativity according to the candidate marks label to the candidate and is optimized and obtains the image candidate and mark label " river san water bexar blue courthouse antonio countyclouds sea ";
6) obtain the image candidate by 2 community information fusion and mark label " clouds river san skywater bexar blue courthouse antonio count "; Obtain the image candidate by 3 community information fusion and mark label " sky blue clouds river water san landscape bexar courthouse mountains "
7) mark the final image annotation results " sky blue clouds river water " that preceding 5 values of label obtain image to be marked by choosing the image candidate.
Can see from top example, different with traditional image labeling method is, the present invention has made full use of the information of the different community at image place in the social sharing network, and the potential subject information that utilizes the community place comes the mark label is carried out " denoising " and optimization, therefore more accurate than traditional annotation results that mask method produced, markup information is also more extensive.
Claims (1)
1. the method based on the automatic image annotation of the potential theme excavation of community is characterized in that comprising the steps:
1) adopt latent Di Li Cray apportion model that the implicit theme in the single community is excavated;
2) by after potential subject analysis obtains the probability distribution of image tag and implicit theme to community, deletion community image tag and implicit theme probability come the community image tag is carried out " denoising " filtration less than the image tag of setting value k;
3) propagate the image candidate who produces image to be marked by the similar image label and mark label;
4) marking between the implicit theme of label and image correlativity according to the image candidate marks label to the image candidate and is optimized;
5) obtain the final annotation results of image by many community information fusion;
Described image candidate by similar image label propagation generation image to be marked marks the step of label: for image I to be marked in the community
u, image I to be marked
uAnd the probability between the image tag w calculates from following formula:
Wherein P (w|J) represents image tag w occurrence number among the training image J, P (I
u| J) represent image I to be marked
uAnd the visual similarity between the training image J, T is a community theme sum, chooses and image I to be marked
uThe highest pairing image tag w of 10 width of cloth training image J of visual similarity marks label as the candidate of image I u to be marked, i.e. P (w|I
u) be worth 10 maximum image tag w as image I to be marked
uThe image candidate mark label;
Correlativity marks the step that label is optimized to the image candidate between the described implicit theme that marks label and image according to the image candidate:
1) marks label w by calculating in all implicit themes that two image candidates mark probability product between the label and obtaining the image candidate
kAnd w
lBetween implicit topic similarity, computing formula is:
The probability distribution of ф presentation video label and implicit theme wherein, T is a community theme sum;
2) marking label and other image candidate by the computed image candidate marks implicit topic relativity sum between the label and obtains the image candidate and mark label w
iWith image I to be marked
uThe correlativity of implicit theme, computing formula is:
P (w wherein
j| w
i) the presentation video candidate marks label w
jAnd w
iBetween implicit topic similarity;
3) recomputate the image candidate and mark label w
iWith image I to be marked
uProbability, computing formula is: P ' (w
i| I
u)=P (w
i| I
u) * R (w
i, I
u), P (w wherein
i| I
u) represent image I to be marked
uWith image tag w
iBetween probability, R (w
i, I
u) the presentation video candidate marks label w
iWith image I to be marked
uThe correlativity of implicit theme;
Described step of image being carried out final mark by many community information fusion:
1) by being chosen at the theme that the most frequent image tag is represented community appears in the community from the title of each community, just in WordNet " entity " semantic tree, find the node of representing this community by this image tag then, constitute the HD between each community;
2) by the HD between each community, each community is carried out final mark by merging to image from top to bottom successively, for obtaining a new father node by the markup information of each child node community is averaged between each community that contains common ancestor's node, delete child node, reach the purpose of fusion;
3) mark the final annotation results that preceding 5 values of label obtain image to be marked by choosing the image candidate.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN2009100999166A CN101685464B (en) | 2009-06-18 | 2009-06-18 | Method for automatically labeling images based on community potential subject excavation |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN2009100999166A CN101685464B (en) | 2009-06-18 | 2009-06-18 | Method for automatically labeling images based on community potential subject excavation |
Publications (2)
Publication Number | Publication Date |
---|---|
CN101685464A CN101685464A (en) | 2010-03-31 |
CN101685464B true CN101685464B (en) | 2011-08-24 |
Family
ID=42048628
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN2009100999166A Expired - Fee Related CN101685464B (en) | 2009-06-18 | 2009-06-18 | Method for automatically labeling images based on community potential subject excavation |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN101685464B (en) |
Families Citing this family (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102496146B (en) * | 2011-11-28 | 2014-03-05 | 南京大学 | Image segmentation method based on visual symbiosis |
CN102760149B (en) * | 2012-04-05 | 2015-02-25 | 中国人民解放军国防科学技术大学 | Automatic annotating method for subjects of open source software |
CN103678335B (en) * | 2012-09-05 | 2017-12-08 | 阿里巴巴集团控股有限公司 | The method of method, apparatus and the commodity navigation of commodity sign label |
CN102902821B (en) * | 2012-11-01 | 2015-08-12 | 北京邮电大学 | The image high-level semantics mark of much-talked-about topic Network Based, search method and device |
CN103065157A (en) * | 2012-12-24 | 2013-04-24 | 南京邮电大学 | Image labeling method based on activation diffusion theory |
CN103927309B (en) * | 2013-01-14 | 2017-08-11 | 阿里巴巴集团控股有限公司 | A kind of method and device to business object markup information label |
CN103631890B (en) * | 2013-11-15 | 2017-05-17 | 北京奇虎科技有限公司 | Method and device for mining image principal information |
CN103714178B (en) * | 2014-01-08 | 2017-01-25 | 北京京东尚科信息技术有限公司 | Automatic image marking method based on word correlation |
CN105095228A (en) * | 2014-04-28 | 2015-11-25 | 华为技术有限公司 | Method and apparatus for monitoring social information |
CN106021365A (en) * | 2016-05-11 | 2016-10-12 | 上海迪目信息科技有限公司 | High-dimension spatial point covering hypersphere video sequence annotation system and method |
CN109145936B (en) * | 2018-06-20 | 2019-07-09 | 北京达佳互联信息技术有限公司 | A kind of model optimization method and device |
CN113408633B (en) * | 2021-06-29 | 2023-04-18 | 北京百度网讯科技有限公司 | Method, apparatus, device and storage medium for outputting information |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1920820A (en) * | 2006-09-14 | 2007-02-28 | 浙江大学 | Image meaning automatic marking method based on marking significance sequence |
CN101082912A (en) * | 2006-06-01 | 2007-12-05 | 上海杰图软件技术有限公司 | Method for annotating electronic map through photograph collection having position information |
-
2009
- 2009-06-18 CN CN2009100999166A patent/CN101685464B/en not_active Expired - Fee Related
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101082912A (en) * | 2006-06-01 | 2007-12-05 | 上海杰图软件技术有限公司 | Method for annotating electronic map through photograph collection having position information |
CN1920820A (en) * | 2006-09-14 | 2007-02-28 | 浙江大学 | Image meaning automatic marking method based on marking significance sequence |
Non-Patent Citations (1)
Title |
---|
陈烨等.基于社群隐含主题挖掘和多社群信息融合的自动图像标注.《中国图象图形学报》.2010,第15卷(第6期),第944-950页. * |
Also Published As
Publication number | Publication date |
---|---|
CN101685464A (en) | 2010-03-31 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN101685464B (en) | Method for automatically labeling images based on community potential subject excavation | |
CN101872346B (en) | Method for generating video navigation system automatically | |
Qian et al. | Tagging photos using users' vocabularies | |
CN103065153B (en) | A kind of video key frame extracting method based on color quantization and cluster | |
CN103810303B (en) | Image search method and system based on focus object recognition and theme semantics | |
CN109033998A (en) | Remote sensing image atural object mask method based on attention mechanism convolutional neural networks | |
CN104536956A (en) | A Microblog platform based event visualization method and system | |
JP2011198364A (en) | Method of adding label to medium document and system using the same | |
CN103620545A (en) | Hierarchical, zoomable presentations of media sets | |
CN101295360B (en) | Semi-supervision image classification method based on weighted graph | |
CN103440640A (en) | Method for clustering and browsing video scenes | |
CN104683885A (en) | Video key frame abstract extraction method based on neighbor maintenance and reconfiguration | |
CN108549647A (en) | The method without accident in mark language material active predicting movement customer service field is realized based on SinglePass algorithms | |
CN106055667A (en) | Method for extracting core content of webpage based on text-tag density | |
JP2011055169A (en) | Electronic apparatus and image processing method | |
CN101894129B (en) | Video topic finding method based on online video-sharing website structure and video description text information | |
CN110678861A (en) | Image selection suggestions | |
CN107291778A (en) | The collection method and device of data | |
CN106529492A (en) | Video topic classification and description method based on multi-image fusion in view of network query | |
CN104199838A (en) | User model building method based on label disambiguation | |
Min et al. | Multimodal spatio-temporal theme modeling for landmark analysis | |
Zhang et al. | A remote sensing land cover classification algorithm based on attention mechanism | |
Lin et al. | Discovering multirelational structure in social media streams | |
Shi et al. | Multi-temporal urban semantic understanding based on GF-2 remote sensing imagery: from tri-temporal datasets to multi-task mapping | |
CN103218460A (en) | Image label complementing method based on optimal linear sparsity reconstruction |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
C14 | Grant of patent or utility model | ||
GR01 | Patent grant | ||
CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20110824 Termination date: 20140618 |
|
EXPY | Termination of patent right or utility model |