CN103853797B - A kind of picture retrieval method and system based on n member picture indices structures - Google Patents
A kind of picture retrieval method and system based on n member picture indices structures Download PDFInfo
- Publication number
- CN103853797B CN103853797B CN201210523756.5A CN201210523756A CN103853797B CN 103853797 B CN103853797 B CN 103853797B CN 201210523756 A CN201210523756 A CN 201210523756A CN 103853797 B CN103853797 B CN 103853797B
- Authority
- CN
- China
- Prior art keywords
- picture
- text
- image
- index
- retrieval
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/5866—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Library & Information Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of picture retrieval method and system based on n member picture indices structures, it is related to image retrieval technologies field.Method disclosed by the invention includes：When receiving the search operaqtion of user, judge the form of retrieval content of user's input for textual form, then the text intrinsic vectorization based on n member picture indices is carried out to the text that user inputs to handle, using carrying out picture retrieval under index of the result in text label, when the form of the retrieval content of user's input is graphic form, picture semantic automatic marking based on n member picture indices structures is carried out to the picture that user inputs, extract n member pictures, for carrying out picture retrieval in index of the TF IDF characteristic vectors in the text label of semantic tagger of the n member pictures of extraction, finally by the picture retrieved is according to sequencing of similarity and exports.The invention also discloses the picture retrieval system based on n member picture indices structures.Technical scheme improves recall precision and effect.
Description
Technical field
The present invention relates to image search method and system, and in particular to figure of the one kind based on first (n-gram) picture indices of n
Piece search method and system, are mainly used in field of image search.
Background technology
At present, picture retrieval is broadly divided into two ways, text based picture retrieval (text-based image
) and the picture retrieval based on content (content-based image retrieval) retrieval.Traditional based on text
In this picture retrieval system (TBIR), picture is typically that user passes through required for keyword retrieval after artificial mark
Picture.The distinct disadvantage of this mode is that picture must be by being manually labeled, in today of information huge explosion, this mode
It is unpractical.To overcome the shortcoming of text based picture retrieval, the picture retrieval mode based on content is in last century 80
Age arises at the historic moment, wherein Chang in 1984 has done the work of initiative in this respect.The so-called picture retrieval based on content
(CBIR), refer to by extracting the original bottom visual signature of picture (such as color characteristic, textural characteristics, shape facility etc.) to figure
Piece is indexed, and the mode of the low-level image feature progress picture searching eventually through picture.It is more famous it is commercial based on
The picture retrieval instrument of content has QBIC, Photobook, Virage, VisualSEEK, Netra and SIMPLIcity.
Currently conventional picture retrieval system, be all mostly the picture that image data is concentrated is extracted higher-dimension low-level image feature to
Amount, is indexed by being set up to these higher-dimension low-level image features vector, or the image to being marked with picture, passes through text label pair
Image sets up index.User is then by submitting text or diagram picture to retrieve directory system.However, by this method
Searching system retrieval effectiveness and efficiency it is unsatisfactory, its main cause is to be retrieved inherently to exist by low-level image feature
" semantic gap " problem, and by the directory system set up to higher-dimension low-level image feature with the drastically increasing of the quantity of index picture
It is long, recall precision it is very low, therefore the index picture quantity of photo current search engine is also limited, the figure of its user search
Piece effect is undesirable.And current most of picture retrieval systems all do not use the spatial signature information carried in picture.
The current main method for solving " semantic gap " problem is and current most of picture searchings by carrying out automatic marking to picture
Engine is not by picture automatic marking technology Successful utilization into picture retrieval system.
However, the development of current text retrieval is quite ripe, it, which indexes foundation and retrieval technique, certain product
It is tired, therefore correlation technique can be used for reference in terms of text retrieval, improve the performance of current picture retrieval system.
The content of the invention
The technical problems to be solved by the invention are to provide a kind of picture retrieval side based on n-gram picture indices structures
Method and system, to improve picture retrieval efficiency and effect.
In order to solve the above-mentioned technical problem, the invention discloses a kind of picture retrieval side based on n member picture indices structures
Method, including：
When receiving the search operaqtion of user, the form of the retrieval content of user's input is judged；
When the form for the retrieval content that user inputs is textual form, the text that user inputs is carried out based on n members to scheme
The text intrinsic vectorization processing of piece index, is carried out using under index of the text intrinsic vector result in text label
Picture retrieval, by the picture retrieved is according to sequencing of similarity and exports；
When the form for the retrieval content that user inputs is graphic form, the picture that user inputs is carried out based on n members to scheme
The picture semantic automatic marking of piece index structure, based on n meta-models extract n member pictures, for the n member pictures of extraction word frequency-
Picture retrieval is carried out in reverse index of document-frequency (TF-IDF) characteristic vector in the text label of semantic tagger, will be retrieved
To picture be ranked up and export according to similarity.
It is preferred that the above method also includes：
Before user carries out search operaqtion, the index based on n member images is built, constructed index is included with image n members
For index, using image labeling and picture details as the index structure of index object, and with picture be labeled as index, to scheme
As the index structure that n is first and picture details are index object.
It is preferred that in the above method, the process for building the index based on n member images is as follows：
Image data collection with text marking is pre-processed, is concentrated from pretreated view data and extracts " figure
As lemma "；
The corresponding image dictionary for including image n members is built according to " the image lemma " extracted；
According to constructed image dictionary, the picture that the image data with text marking is concentrated is cut, extracted
Corresponding image n members, set up the picture indices based on n meta-models.
It is preferred that the above method, is carried out at the text intrinsic vector based on n member picture indices to the text that user inputs
Reason refers to：
The content of text inputted according to user, is retrieved based on n member picture indices structures, according to the image n retrieved
The probability weights of member, intrinsic vector processing is carried out to content of text.
It is preferred that the above method, utilizes the lower figure of index of the text intrinsic vector result in text label
Piece is retrieved, and the picture retrieved is referred to according to sequencing of similarity and output：
The text that user inputs is carried out after vectorization, the value after being handled according to vectorization is under the index in text label
Picture carry out Similarity Measure, the picture retrieved is sorted and exported according to the obtained size of similarity is calculated.
The invention also discloses a kind of picture retrieval system based on n member picture indices structures, including judging unit, first
Indexing units and the second indexing units, wherein：
The judging unit, when receiving the search operaqtion of user, judges the form of the retrieval content of user's input, works as user
The form of the retrieval content of input be textual form when, the text that user inputs is sent to first indexing units, when with
When the form of the retrieval content of family input is graphic form, the picture that user inputs is sent to second indexing units；
First indexing units, the text intrinsic vector based on n member picture indices is carried out to the text that user inputs
Processing, using picture retrieval is carried out under index of the text intrinsic vector result in text label, by the figure retrieved
Piece is according to sequencing of similarity and exports；
Second indexing units, the picture semantic based on n member picture indices structures is carried out certainly to the picture that user inputs
Dynamic mark, n member pictures are extracted based on n meta-models, special for word frequency-reverse document-frequency (TF-IDF) of the n member pictures of extraction
Levy in index of the vector in the text label of semantic tagger and carry out picture retrieval, the picture retrieved is carried out according to similarity
Sort and export.
It is preferred that said system also includes：Based on n member picture indices construction units, the index based on n member images is set up,
It is index that the index, which is included with image n members, using image labeling and picture details as the index structure of index object, and with
Picture is labeled as index, using image n members and picture details as the index structure of index object.
It is preferred that in said system, it is described to be divided into based on n member picture indices construction units：
" image dictionary " builds part, and the image data collection with text marking is pre-processed, from pretreated
View data, which is concentrated, extracts " image lemma ", and the corresponding image for including image n members is built according to " the image lemma " extracted
Dictionary；
Index construct part, the image dictionary according to constructed by " image dictionary " builds part, to text mark
The picture that the image data of note is concentrated is cut, and is extracted corresponding image n members, is set up the picture indices based on n meta-models.
It is preferred that in said system, the text that first indexing units are inputted to user carries out being based on n member picture indices
Text intrinsic vectorization processing refer to：
The content of text inputted according to user, is retrieved based on n member picture indices structures, according to the image n retrieved
The probability weights of member, intrinsic vector processing is carried out to content of text.
It is preferred that in said system, first indexing units are using text intrinsic vector result in text mark
Picture retrieval is carried out under index in label, the picture retrieved is referred to according to sequencing of similarity and output：
The text that user inputs is carried out after vectorization, the value after being handled according to vectorization is under the index in text label
Picture carry out Similarity Measure, the picture retrieved is sorted and exported according to the obtained size of similarity is calculated.
Technical scheme, can be with effectively by text based picture retrieval and the picture retrieval mode based on content
Combine, effectively raise recall precision and effect.
Embodiment
Fig. 1 is picture retrieval schematic flow sheet of the present embodiment based on n-gram picture indices structures；
Fig. 2 is the flow chart of extraction " image lemma " in the present embodiment；
Fig. 3 is the exemplary plot of image cutting and extraction n-gram in the present embodiment；
Fig. 4 is using image n-gram as index, using semantic label and image as the image index topology example of index content
Figure；
Fig. 5 is using image, semantic label as index, using image n-gram and image as the image index structure of index content
Exemplary plot；
Fig. 6 is the picture semantic automatic marking schematic flow sheet based on n-gram picture indices structures.
Embodiment
For the object, technical solutions and advantages of the present invention are more clearly understood, below in conjunction with accompanying drawing to skill of the present invention
Art scheme is described in further detail.It should be noted that in the case where not conflicting, in embodiments herein and embodiment
Feature can arbitrarily be mutually combined.
Embodiment 1
The present embodiment provides a kind of picture retrieval method based on n-gram picture indices structures, and this method includes two kinds of inspections
Rope mode：That is the picture retrieval of the picture retrieval of textual form and graphic form.The implementation principle of this method is as shown in Figure 1.
Specifically include following steps 100 to 400：
Step 100, when receiving the search operaqtion of user, the form of the retrieval content of user's input is judged, if text shape
Formula then enters step 200 (a), if graphic form then enters step 200 (b)；
Step 200 (a), is carried out at the text intrinsic vector based on n-gram picture indices to the text that user inputs
Reason, into step 300 (a).
Specifically, the content of text that this step is inputted according to user, to being retrieved based on n-gram picture indices structures,
The image n-gram obtained according to retrieval probability weights, intrinsic vector processing is carried out to content of text.
Step 200 (b), carries out the picture semantic based on n-gram picture indices structures to the picture that user inputs and marks automatically
Note, based on n-gram model extraction n-gram pictures, into step 300 (b).
The step first to the picture that user inputs extract image n-gram operation, so extract the feature of picture to
Amount, then the semantic tagger processing based on n-gram picture indices structures is carried out to picture.
Step 300 (a), the text intrinsic vector result inputted using user is schemed in the index in text label
Piece is retrieved, and the similarity of the picture retrieved is calculated, into step 400.
In this step, the text that user inputs is carried out after vectorization, according to the value after vectorization to corresponding text rope
Picture under drawing carries out Similarity Measure.
Step 300 (b), for TF (the Term Frequency Term Frequency, word of the n-gram pictures of extraction
Frequently text label of-IDF (Inverse Document Frequency, the reverse document-frequency) characteristic vectors in semantic tagger)
Picture retrieval is carried out in interior index, the picture retrieved is ranked up and exported according to similarity.
The picture that the step is inputted to user is carried out after meaning automatic marking, the characteristic vector extracted to picture, in language
Similarity Measure is carried out in picture under the text index of justice mark.
Step 400, after Similarity Measure is carried out, the picture retrieved is sorted and according to this according to the size of similarity
Sort the just list for returning and retrieving to user.
It should also be noted that based on the above method, also having some preferred schemes, search operaqtion is carried out in user
Before, the index based on n-gram images is also built, it is index that constructed index, which is included with image n members, with image labeling and figure
Piece details are the index structure of index object, and are labeled as index with picture, with image n members and picture details
For the index structure of index object.
Specifically, the process for building the index based on n-gram images is as follows：
Image data collection with text marking is pre-processed, is concentrated from pretreated view data and extracts " figure
As lemma "；
The corresponding image dictionary for including image n members is built according to " the image lemma " extracted；
According to constructed image dictionary, the picture that the image data with text marking is concentrated is cut, extracted
Corresponding image n members, set up the picture indices based on n meta-models.
Below by taking the preferred scheme for including index operation of the structure based on n-gram images as an example, describe in detail above-mentioned
Picture retrieval process based on n-gram picture indices structures.
The first step, learns image lemma, then " the image word by learning to obtain by the image data collection randomly selected
Member " structure " image dictionary ".
Wherein, the process of study " image lemma " is as shown in Fig. 2 comprise the following steps：
First, textual cutting is carried out to the picture of selection, the mode of textual cutting can be according to different application need
Ask and be designed.A kind of example of the picture textual cutting method provided in the embodiment of the present invention is to be evenly divided into picture
Size is m*n image fritter (such as Fig. 3), and each fritter is considered as one " word " in similar text-processing, and every width
Image is considered as " article " accordingly, and the method not limited to this of textual cutting is carried out to picture.
Secondly the characteristics of the underlying image including but not limited to color of image for the equal-sized image fritter being cut into, is extracted
Feature, image texture characteristic.And merged its multiple low-level image feature, so as to obtain an energy response diagram as a variety of bottoms of fritter
The characteristic vector of layer feature.
Then, to the characteristic vector of obtained each image fritter, cluster operation is carried out using clustering method, finally by
The typical data point for representing respective cluster class is chosen as " image lemma ".Corresponding numbering is assigned to " the image lemma " of acquisition
(such as Fig. 3).A kind of embodiment (such as Fig. 2) that the present invention is used, is to be k-means by the characteristic vector to all image fritters
Cluster operation, predefines the quantity of clustering cluster, is obtained " image lemma " by obtaining the barycenter of k-means cluster results.
Finally, study is obtained after " image lemma ", is exactly by constructing " image dictionary ", in order to further indicate that image
Space characteristics, add n-gram in " image dictionary ", for any one " image lemma ", n-1 " figures adjacent thereto
As lemma " " image lemma " sequence is constituted, all these " image lemma " sequences are all added into " image word as an item
In allusion quotation ", while adding other " image lemma " sequences that its length is less than n, constitute " image dictionary ".For example, it is assumed that extract
" image lemma " is 1,2,3, and it is 2 to choose n, then " image dictionary " item that obtained " image dictionary " is included is：(1)、(2)、
(3), (1,1), (1,2), (1,3), (2,1), (2,2), (2,3), (3,1), (3,2), (3,3).For extracting " image lemma "
Quantity is K, from n in 2 embodiment, the gram quantity that " image dictionary " is included is K*K+K.
Second step, judges the form of the retrieval content of user's input；
Wherein, why this step is judged the form of retrieval content that user inputs, and is to determine that system is answered
The next step operation taken, and appropriate pretreatment is done to corresponding input.If user's input is content of text, need
Make participle to content of text and remove the processing of stop words；If user's input is picture, then then need to enter picture
The corresponding format conversion of row and size normalization processing.
3rd step, judge user input for textual form when, carry out the text based on n-gram picture indices structures
Intrinsic vectorization processing, or judge user input for graphic form when, progress the figure based on n-gram picture indices structures
Piece meaning automatic marking；
4th step, the text intrinsic vector result inputted using user or picture feature based on n-gram models to
Amount, is retrieved in the picture that text label is indexed；
If in the step, user input be textual form, to user input text carry out based on n-gram figure
The text intrinsic vectorization processing of piece index, specifically the method for text intrinsic vector is：First in such as Fig. 5 index structure
It is middle to be retrieved, using corresponding n-gram Nweight values as text vectorization component of a vector weights, for containing many
The text of individual participle, last vectorial each component value is added, and the intrinsic vector for obtaining the text of user's input is represented.
If user's input is picture, picture semantic as shown in Figure 6 is carried out certainly to the image data that user inputs
Dynamic mark, and the TF-IDF characteristic vectors of picture are extracted, the TF-IDF calculations that the present embodiment is used are as follows：
N in formulaI, j--- " image dictionary " item is in image djIn appearance frequency；
∑knK, j--- image djIn all items there is frequency summation.
In formula | D | --- the total number of images of picture library；
|{j：ti∈dj| --- include " image dictionary " item ti amount of images (i.e. nI, j≠ 0 number of files).
5th step, the result finally to retrieval are ranked up according to similarity, and export retrieval result.
In the step, if user's input is textual form, text vector table is carried out to the text that user inputs
After showing, Similarity Measure is carried out in the picture under the corresponding text label that user inputs is indexed to obtained vector；
If user's input is graphic form, the picture that user inputs is carried out after meaning automatic marking, in mark
Similarity Measure is carried out in the picture under tab indexes afterwards, and returns to the weights of Similarity Measure.
Finally all pictures that retrieval is obtained are ranked up according to the weights size of Similarity Measure, will be incited somebody to action according to sequence
Just list returns to user.
Embodiment 2
The present embodiment introduces a kind of picture retrieval system based on n-gram picture indices structures, and the system at least includes sentencing
Disconnected unit, the first indexing units and the second indexing units.
Judging unit, when receiving the search operaqtion of user, judges the form of the retrieval content of user's input, when user's input
Retrieval content form be textual form when, the text that user inputs is sent to the first indexing units, when user input
When the form for retrieving content is graphic form, the picture that user inputs is sent to the second indexing units；
First indexing units, are carried out at the text intrinsic vector based on n-gram picture indices to the text that user inputs
Reason, using picture retrieval is carried out under index of the text intrinsic vector result in text label, by the picture retrieved
According to sequencing of similarity and export；
Wherein, the text that the first indexing units are inputted to user carries out the text intrinsic vector based on n-gram picture indices
When changing processing, the content of text inputted according to user is retrieved based on n-gram picture indices structures, according to the figure retrieved
As n-gram probability weights, intrinsic vector processing is carried out to content of text.
And the first indexing units under index of the text intrinsic vector result in text label using carrying out picture
Retrieval, by the picture retrieved according to sequencing of similarity and when exporting, is mainly carried out after vectorization to the text that user inputs,
Value after being handled according to vectorization carries out Similarity Measure to the picture under the index in text label, the phase obtained according to calculating
The picture retrieved is sorted and exported like the size of degree.
Second indexing units, carry out the picture semantic based on n-gram picture indices structures automatic to the picture that user inputs
Mark, based on n-gram model extraction n-gram pictures, for extraction n-gram pictures TF-IDF characteristic vectors in semanteme
Picture retrieval is carried out in index in the text label of mark, the picture retrieved is ranked up and exported according to similarity.
Also some preferred schemes, on the basis of said system, increase has based on n-gram picture indices construction units,
The unit sets up the index based on n-gram images, and the index set up is included using image n-gram as index, with image labeling
With picture details be index object index structure, and with picture be labeled as index, with image n-gram and picture
Details are the index structure of index object.
Specifically, it can be divided into again based on n-gram picture indices construction unit, " image dictionary " builds part and index
Build part.
" image dictionary " builds part, and the major function of the part is that, according to image data set, study includes image n-
Gram image dictionary, specifically, the part are pre-processed to the image data collection with text marking, from pretreated
View data, which is concentrated, extracts " image lemma ", and the corresponding figure for including image n-gram is built according to " the image lemma " extracted
As dictionary；
Wherein, build " image dictionary " and learn image lemma firstly the need of the image data collection by randomly selecting, then
Pass through " image lemma " structure " image dictionary " for learning to obtain.
Learn the method and step of " image lemma " as shown in Fig. 2 comprising the following steps that description：
The first step, the picture to selection carry out textual cutting, and the mode of textual cutting can be according to different applications
Demand is designed.A kind of example of the picture textual cutting method provided in the embodiment of the present invention is by picture even partition
Into the image fritter (such as Fig. 1) that size is m*n, each fritter is considered as one " word " in similar text-processing, and every
Width image is considered as " article " accordingly, and the method not limited to this of textual cutting is carried out to picture.
The characteristics of the underlying image for the equal-sized image fritter that second step, extraction are cut into includes but is not limited to image face
Color characteristic, image texture characteristic.And merged its multiple low-level image feature, so as to obtain an energy response diagram as fritter is a variety of
The characteristic vector of low-level image feature.
3rd step, to the characteristic vector of obtained each image fritter, carries out cluster operation using clustering method, finally leads to
Cross and choose the typical data point for representing respective cluster class as " image lemma ".Corresponding numbering is assigned to " the image lemma " of acquisition
(such as Fig. 1).A kind of embodiment (such as Fig. 2) that the present invention is used, is to be k-means by the characteristic vector to all image fritters
Cluster operation, predefines the quantity of clustering cluster, is obtained " image lemma " by obtaining the barycenter of k-means cluster results.
And learn to obtain after " image lemma ", it is exactly by constructing " image dictionary ", in order to further indicate that the sky of image
Between feature, n-gram are added in " image dictionary ", for any one " image lemma ", n-1 " images adjacent thereto
Lemma " constitutes " image lemma " sequence, and all these " image lemma " sequences all are added into " image word as an item
In allusion quotation ", while adding other " image lemma " sequences that its length is less than n, constitute " image dictionary ".For example, it is assumed that extract
" image lemma " is 1,2,3, and it is 2 to choose n, then " image dictionary " item that obtained " image dictionary " is included is：(1)、(2)、
(3), (1,1), (1,2), (1,3), (2,1), (2,2), (2,3), (3,1), (3,2), (3,3).For extracting " image lemma "
Quantity is K, from n in 2 embodiment, the gram quantity that " image dictionary " is included is K*K+K.
Index construct part, the major function of the part is, to image data set, to set up and be based on n- according to " image dictionary "
Gram image index.Specifically, the image dictionary according to constructed by " image dictionary " builds part, to text marking
The picture that image data is concentrated is cut, and is extracted corresponding image n members, is set up the picture indices based on n meta-models.And set up
The image index based on n-gram include two class index structures：One kind be using image n-gram as index, with image labeling and
Picture details are the index structure of index object；Another is to be labeled as index with picture, with image n-gram and figure
Piece details are the index structure of index object.
Below again by taking the image index based on n-gram for including above two index structure of structure as an example, illustrate figure
The detailed process of piece retrieval.
1. using image n-gram as index, using image labeling and image details as index object, as shown in figure 4, figure
Middle Mnode is main index node, is the item in " image dictionary " in master index node, including unigram and bigram.(1,1)
For image bigram, the content of master index node index includes two parts：1st, comprising " image dictionary " item in master index node
All pictures details, by taking Mnode as an example, picture of its lower index is all comprising " image dictionary " item (1,1)
The details of picture；2nd, the subindex node comprising text marking label (sun) and its correspondence weights (Lweightsun)
(Cnodel).By taking Cnodel as an example, subindex node includes the text label sun occurred in view data and by calculating
Obtained corresponding weights Lweightsun.What Lweightsun reacted is " image dictionary " item and subindex in master index node
The relation of text label in node, the computational methods that the present invention is used are as follows：
Wherein：
N in formula ((1,1) | sun) --- in all pictures with sun labels, include the number of (1,1)；
N (n-gram | sun) --- in the index picture with sun labels, include all n-gram number；
Nimg (sun) --- the number of all pictures with sun labels；
Nimg (All) --- the quantity of all pictures in data set；
N ((1,1)) --- image data concentrates the quantity of all (1,1)；
N (n-gram) --- image data concentrates all n-gram quantity
What is indexed under subindex node is both to have included " image dictionary " item in master index node (Mnode), while band again
There are the details of all pictures of text label in subindex node, by taking Cnodel as an example, picture of its lower index is included
(1,1) " image dictionary " item, while carrying sun labels again.
2. using image, semantic label as index, using image n-gram and image as index content, as shown in figure 5, in figure
Mnode is main index node, the text label concentrated for image data in master index node, as shown in figure 4, sun is picture number
According to a text label of concentration.The content of master index node index includes two parts：1st, this text label is carried in data set
All pictures details, by taking Mnode as an example, content of its lower index is that all pictures comprising text label sun are detailed
Thin information；2nd, the subindex node comprising corresponding " image dictionary " item ((1)) and its correspondence weights (Nweightsun)
(Cnodel).By taking Cnodel as an example, subindex node includes (1) in " image dictionary " and the correspondence obtained by calculating
Weights Nweight (1).Nweight (1) reactions are the text label in master index node and " scheming in subindex node
The latent layer relation of picture dictionary " item, what the present invention was provided is specifically calculated as follows：
In formula：N ((1) | sun) --- the number of (1) is included for the picture with label sun；
N (n-gram | sun) --- all n-gram quantity is included for the picture with label sun.
What is indexed under subindex node is both to have carried the text label in master index node (Mnode), while again comprising son
The details of all pictures of " image dictionary " item in index node, by taking Cnodel as an example, the picture of its lower index is carried
Sun text labels, while including (1) " image dictionary " item again.
One of ordinary skill in the art will appreciate that all or part of step in the above method can be instructed by program
Related hardware is completed, and described program can be stored in computer-readable recording medium, such as read-only storage, disk or CD
Deng.Alternatively, all or part of step of above-described embodiment can also use one or more integrated circuits to realize.Accordingly
Each module/unit in ground, above-described embodiment can be realized in the form of hardware, it would however also be possible to employ the shape of software function module
Formula is realized.The application is not restricted to the combination of the hardware and software of any particular form.
It is described above, it is only the preferred embodiments of the present invention, is not intended to limit the scope of the present invention.It is all this
Within the spirit and principle of invention, any modification, equivalent substitution and improvements done etc. should be included in the protection model of the present invention
Within enclosing.
Claims (8)
1. a kind of picture retrieval method based on n member picture indices structures, it is characterised in that this method includes：
The index based on n member images is built, it is index that constructed index, which is included with image n members, detailed with image labeling and picture
Thin information is the index structure of index object, and is labeled as index with picture, using image n members and picture details as rope
Draw the index structure of object；
When receiving the search operaqtion of user, the form of the retrieval content of user's input is judged；
When the form for the retrieval content that user inputs is textual form, the text that user inputs is carried out being based on n member picture ropes
The text intrinsic vectorization processing drawn, using carrying out picture under index of the text intrinsic vector result in text label
Retrieval, by the picture retrieved is according to sequencing of similarity and exports；
When the form for the retrieval content that user inputs is graphic form, the picture that user inputs is carried out being based on n member picture ropes
The picture semantic automatic marking of guiding structure, n member pictures are extracted based on n meta-models, for the word frequency-reverse of the n member pictures of extraction
Document-frequency TF-IDF characteristic vectors carry out picture retrieval in the index in the text label of semantic tagger, by the figure retrieved
Piece is ranked up and exported according to similarity.
2. the method as described in claim 1, it is characterised in that the process for building the index based on n member images is as follows：
Image data collection with text marking is pre-processed, is concentrated from pretreated view data and extracts " image word
Member "；
The corresponding image dictionary for including image n members is built according to " the image lemma " extracted；
According to constructed image dictionary, the picture that the image data with text marking is concentrated is cut, extracts corresponding
Image n member, set up the picture indices based on n meta-models.
3. method as claimed in claim 1 or 2, it is characterised in that carry out being based on n member picture indices to the text that user inputs
Text intrinsic vectorization processing refer to：
The content of text inputted according to user, is retrieved based on n member picture indices structures, according to the image n members retrieved
Probability weights, intrinsic vector processing is carried out to content of text.
4. method as claimed in claim 3, it is characterised in that using text intrinsic vector result in text label
Index under carry out picture retrieval, by the picture retrieved according to sequencing of similarity and export refer to：
The text that user inputs is carried out after vectorization, the value after being handled according to vectorization is to the figure under the index in text label
Piece carries out Similarity Measure, and the picture retrieved is sorted and exported according to the size for calculating obtained similarity.
5. a kind of picture retrieval system based on n member picture indices structures, it is characterised in that the system includes being based on n member pictures
Index construct unit, judging unit, the first indexing units and the second indexing units, wherein：
It is described to be based on n member picture indices construction units, the index based on n member images is set up, the index is included with image n members for rope
Draw, using image labeling and picture details as the index structure of index object, and with picture be labeled as index, with image n
Member and the index structure that picture details are index object；
The judging unit, when receiving the search operaqtion of user, judges the form of the retrieval content of user's input, when user's input
Retrieval content form be textual form when, the text that user inputs is sent to first indexing units, when user is defeated
When the form of the retrieval content entered is graphic form, the picture that user inputs is sent to second indexing units；
First indexing units, carry out the text intrinsic vectorization based on n member picture indices to the text that user inputs and handle,
Using carrying out picture retrieval under index of the text intrinsic vector result in text label, by the picture retrieved according to
Sequencing of similarity is simultaneously exported；
Second indexing units, carry out the picture semantic based on n member picture indices structures to the picture that user inputs and mark automatically
Note, n member pictures are extracted based on n meta-models, for word frequency-reverse document-frequency TF-IDF characteristic vectors of the n member pictures of extraction
Picture retrieval is carried out in index in the text label of semantic tagger, the picture retrieved is ranked up simultaneously according to similarity
Output.
6. system as claimed in claim 5, it is characterised in that described to be divided into based on n member picture indices construction units：
" image dictionary " builds part, the image data collection with text marking is pre-processed, from pretreated image
" image lemma " is extracted in data set, the corresponding image dictionary for including image n members is built according to " the image lemma " extracted；
Index construct part, the image dictionary according to constructed by " image dictionary " builds part, to text marking
The picture that image data is concentrated is cut, and is extracted corresponding image n members, is set up the picture indices based on n meta-models.
7. the system as described in claim 5 or 6, it is characterised in that the text that first indexing units are inputted to user enters
Text intrinsic vectorization processing of the row based on n member picture indices refers to：
The content of text inputted according to user, is retrieved based on n member picture indices structures, according to the image n members retrieved
Probability weights, intrinsic vector processing is carried out to content of text.
8. system as claimed in claim 7, it is characterised in that first indexing units are handled using text intrinsic vectorization
As a result picture retrieval is carried out under the index in text label, the picture retrieved is referred to according to sequencing of similarity and output：
The text that user inputs is carried out after vectorization, the value after being handled according to vectorization is to the figure under the index in text label
Piece carries out Similarity Measure, and the picture retrieved is sorted and exported according to the size for calculating obtained similarity.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210523756.5A CN103853797B (en) | 2012-12-07 | 2012-12-07 | A kind of picture retrieval method and system based on n member picture indices structures |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210523756.5A CN103853797B (en) | 2012-12-07 | 2012-12-07 | A kind of picture retrieval method and system based on n member picture indices structures |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103853797A CN103853797A (en) | 2014-06-11 |
| CN103853797B true CN103853797B (en) | 2017-10-17 |
Family
ID=50861464
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210523756.5A Active CN103853797B (en) | 2012-12-07 | 2012-12-07 | A kind of picture retrieval method and system based on n member picture indices structures |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103853797B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104504110B (en) * | 2014-12-30 | 2019-05-03 | 百度在线网络技术（北京）有限公司 | Searching method and device |
| US10068380B2 (en) * | 2016-11-17 | 2018-09-04 | Adobe Systems Incorporated | Methods and systems for generating virtual reality environments from electronic documents |
| CN109871856B (en) * | 2017-12-04 | 2022-03-04 | 北京京东尚科信息技术有限公司 | Method and device for optimizing training sample |
| CN110390030A (en) * | 2019-06-28 | 2019-10-29 | 中山大学 | The storage method and device of pictorial information |
| CN110781326A (en) * | 2019-10-25 | 2020-02-11 | 湖南省公安厅 | Picture retrieval and acquisition method and device and picture storage system |
| CN116595043A (en) * | 2023-04-28 | 2023-08-15 | 南京司泰克智能科技有限公司 | Big data retrieval method and device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1920820A (en) * | 2006-09-14 | 2007-02-28 | 浙江大学 | Image meaning automatic marking method based on marking significance sequence |
| CN101587478A (en) * | 2008-05-20 | 2009-11-25 | 株式会社理光 | Methods and devices for training, automatically labeling and searching images |
| CN102549571A (en) * | 2009-05-15 | 2012-07-04 | 谷歌公司 | Landmarks from digital photo collections |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5224851B2 (en) * | 2008-02-27 | 2013-07-03 | インターナショナル・ビジネス・マシーンズ・コーポレーション | Search engine, search system, search method and program |
-
2012
- 2012-12-07 CN CN201210523756.5A patent/CN103853797B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1920820A (en) * | 2006-09-14 | 2007-02-28 | 浙江大学 | Image meaning automatic marking method based on marking significance sequence |
| CN101587478A (en) * | 2008-05-20 | 2009-11-25 | 株式会社理光 | Methods and devices for training, automatically labeling and searching images |
| CN102549571A (en) * | 2009-05-15 | 2012-07-04 | 谷歌公司 | Landmarks from digital photo collections |
Non-Patent Citations (1)
| Title |
|---|
| 基于文本和视觉信息融合的web图像检索;黄鹏;《中国博士学位论文全文数据库 信息科技辑》;20090215;第I138-30页，第3章第3.2节，第6章第6.2.4节 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103853797A (en) | 2014-06-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Bhagat et al. | Image annotation: Then and now | |
| CN101438285B (en) | Method for domain identification of documents in a document database | |
| CN104537116B (en) | A kind of books searching method based on label | |
| CN103853797B (en) | A kind of picture retrieval method and system based on n member picture indices structures | |
| CN101271476B (en) | Relevant feedback retrieval method based on clustering in network image search | |
| Clinchant et al. | Xrce’s participation in wikipedia retrieval, medical image modality classification and ad-hoc retrieval tasks of imageclef 2010 | |
| CN106651696B (en) | Approximate question pushing method and system | |
| Noaman et al. | Naive Bayes classifier based Arabic document categorization | |
| CN108647322B (en) | Method for identifying similarity of mass Web text information based on word network | |
| CN107844493B (en) | File association method and system | |
| Lee et al. | MAP-based image tag recommendation using a visual folksonomy | |
| CN105426529A (en) | Image retrieval method and system based on user search intention positioning | |
| Wang et al. | Grouping web image search result | |
| CN114443847A (en) | Text classification method, text processing method, text classification device, text processing device, computer equipment and storage medium | |
| JP4426041B2 (en) | Information retrieval method by category factor | |
| CN113987175A (en) | Text multi-label classification method based on enhanced representation of medical topic word list | |
| Hasan et al. | Content based document classification using soft cosine measure | |
| Wu et al. | Searching online book documents and analyzing book citations | |
| Tian et al. | Automatic image annotation with real-world community contributed data set | |
| El-Hajj et al. | An optimal approach for text feature selection | |
| Xie et al. | The accuracy and value of machine-generated image tags: design and user evaluation of an end-to-end image tagging system | |
| CN113516202A (en) | Webpage accurate classification method for CBL feature extraction and denoising | |
| Richter et al. | Leveraging community metadata for multimodal image ranking | |
| Coelho et al. | Image abstraction in crossmedia retrieval for text illustration | |
| CN103853795A (en) | Image indexing method and device based on n-gram model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |