CN104298758A - Multi-perspective target retrieval method - Google Patents

Abstract

The invention discloses a multi-perspective target retrieval method. The multi-perspective target retrieval method includes steps of acquiring a retrieval target input by a user and view sets of objects in database; utilizing the image characteristics extraction algorithm to extract characteristics of the retrieval target and the objects in the database; clustering the view sets by a clustering method after characteristics extraction and extracting representative views of each type; determining corresponding initial weight of each representative view according to scale of the belonging type and updating the weights by means of relation between the representative views and generating the final weights; establishing a weighted bipartite graph by means of the representative views of the two view sets and their weights; seeking the optimal matching of the weighted bipartite graph by the bipartite graph matching algorithm and acquiring similarity between the retrieval target and each object in the database, sequencing and outputting the sequencing result as retrieval results. By the multi-perspective target retrieval method, accuracy of multi-perspective target retrieval is improved.

Description

A kind of method of various visual angles target retrieval

Technical field

The present invention relates to field of image search, particularly relate to a kind of method of various visual angles target retrieval.

Background technology

In actual life, the existence of object has spatiality, and human eye is also 3 D stereo to the perception of object.The 2 d plane picture of the just object that traditional camera technique obtains, and RGB – D (three primary colors add distance) video camera, such as: Kinect said three-dimensional body sense video camera, two-dimensional signal can not only be obtained, can also obtain corresponding depth information, this just compensate for the defect of traditional cameras.Three-dimensional model and image are compared, and the perception details that it can be expressed is abundanter, more presses close to the sense of reality of human eye solid, so be more suitable for the cognition impression of the mankind.

The acquisition problem of three-dimensional model is the large problem that must consider.If all need to set up a set of three-dimensional model at every turn, workload will be very huge, simultaneously can energy time of inevitably at substantial, and not be that ordinary person can complete, obviously very unrealistic.Obtain three-dimensional model is all oneself set up or rely on spatial digitizer before, realize difficulty inconvenient again, nowadays this situation has obtained huge improvement, can be downloaded by NetFind three-dimensional model easily, therefore nowadays sharable three-dimensional model quantity presents blasting type rising tendency.Therefore, must accomplish to rely on network to make full use of existing model resource ^[1].The fast development of network technology and numerous search engine system appear as three-dimensional model resource share and propagation brings great convenience factor.Therefore, help user from the mass data daily database, to retrieve desired model quickly and accurately, namely study three-dimensional model search technology, become current problem demanding prompt solution and study hotspot.

Various visual angles target retrieval algorithm is mainly divided into two classes: text based retrieval and content-based retrieval ^[2].Text based retrieval technology due to implementation algorithm simple, very ripe, application also widely, but due to the inherent shortcoming that it exists self, the quantity of information of text carrying very little, therefore and be not suitable for three-dimensional model search the abundant informations such as the physical dimension of three-dimensional body, topological structure, material texture cannot be described accurately and effectively.By contrast, the feature that content-based retrieval has is: less human intervention, visual effect true to nature, retrieval rate are high.Utilize machine automatically to calculate and extract the internal characteristics of three-dimensional model, by the calculating of specific algorithm realization for distortion in interrogation model and database, thus set up characteristic key index, reach browsing and search function of will realizing.

During the similarity of current various visual angles target retrieval algorithm between calculating two objects, mostly only calculate the Euclidean distance that two objects correspond to each other between view, and do not consider the relevance between each view of same object and importance, retrieval accuracy has much room for improvement.

Summary of the invention

The invention provides a kind of method of various visual angles target retrieval, described below:

A method for various visual angles target retrieval, said method comprising the steps of:

(1) the view collection of object in the searched targets of user's input and database is obtained;

(2) image characteristics extraction algorithm is used to carry out feature extraction to the view collection of object in described searched targets and database;

(3) adopt clustering method to carry out cluster to the view collection after feature extraction, and extract the representational view of each class;

(4) determine the corresponding initial weight of each width representational view according to the scale of place class, utilize the relation between representational view to carry out weight renewal, generate final weight;

(5) representational view of two view collection and their weighted value is utilized to build weighting bipartite graph;

(6) use bipartite graph matching algorithm to seek the Optimum Matching of described weighting bipartite graph, obtain the similarity between each object in searched targets and database, and sort with this, using the result after sequence as search and output.

The beneficial effect of technical scheme provided by the invention is: the present invention is by carrying out cluster to the view collection of the three-dimensional body obtained, extract representational view, provide weight, in conjunction with bipartite graph Optimum Matching, obtain the similarity between searched targets and database object, improve the accuracy of various visual angles target retrieval.Weighted value after renewal contains the information such as the scale of contact between representational view and cluster.The similarity utilizing bipartite graph matching to obtain contains the relevance between two model representation views, than merely compute euclidian distances, and better effects if.

Accompanying drawing explanation

Fig. 1 is a kind of process flow diagram of various visual angles target retrieval method.

Fig. 2 is that in ETH database, the standard-Cha full curve of looking into of three kinds of algorithms compares.

Fig. 3 is that NN, FT and ST of three kinds of algorithms in ETH database compares.

Embodiment

For making the object, technical solutions and advantages of the present invention clearly, below embodiment of the present invention is described further in detail.

Embodiments provide a kind of method of various visual angles target retrieval, see Fig. 1, the method comprises:

101: obtain the view collection of object in the searched targets of user's input and database;

Wherein, the view collection of object in the searched targets of user's input and database is the corresponding one group of two dimension view representing these three dimensional objects.These two dimension views can be obtained real three dimensional object shooting by real camera, and the Softcam that also can pass through 3D program software (such as 3D MAX) obtains the shooting of virtual three-dimensional object.

102: use image characteristics extraction algorithm to carry out feature extraction to the view collection of object in searched targets and database;

The Image Visual Feature extraction algorithm of current popular can be adopted to carry out feature extraction and sign to view collection, and without loss of generality, the embodiment of the present invention have employed can the gradient orientation histogram of Efficient Characterization picture shape and architectural feature ^[3](Histogram of Oriented Gradient is called for short HOG operator) carries out characteristic present.

The circular of HOG operator is: the gradient calculating each width view regional area respectively, uses statistical means to construct gradient orientation histogram, thus forms gradient orientation histogram operator feature, is used for describing former view.

103: adopt clustering method to carry out cluster to the view collection after feature extraction, and extract the representational view of each class;

The view clustering algorithm of current popular can be adopted to carry out cluster to the view collection after feature extraction, without loss of generality, have employed classical K-means ^[4]clustering method.

K-means clustering method is specially: the exact amount K first determining wanted cluster, and it, as cluster centre, to remaining each view, according to the distance of itself and each cluster centre, is assigned to nearest class by initial selected K view.Recalculate the mean value of view in each class, form new cluster centre.Repeat this process, until clustering convergence.View set representations as a three-dimensional model M is wherein be the width two dimension view in V, i is view sequence number, and M is three-dimensional model, n _mit is the number of view.Adopt each the width view of HOG operator representation in 102, and then try to achieve two width views with between Euclidean distance:

$d(v_i^M,v_j^M)={(f_i-f_j)}^T(f_i-f_j)$

Wherein i, j are view sequence number, f _iand f _jbe respectively with hOG proper vector, T represents matrix transpose.

View collection thus after carrying out K-means cluster, obtain K view subset, i.e. V={V ₁, V ₂..., V _k, and the view in each view subset is visually similar view.Calculate each the width view in each class and the Euclidean distance sum in such between other views, choose the representational view of the minimum view of Euclidean distance sum as such, obtain K width representational view collection { rv ₁, rv ₂..., rv _i..., rv _k, wherein rv _ibe the i-th width representational view, i is representational view sequence number.Concrete, the value of K is generally determined by subjectivity, and the number of views that Primary Reference view is concentrated determines, selects K=15 in this experiment.

104: determine the corresponding initial weight of each width representational view according to the scale of place class, utilize the relation between representational view to carry out weight renewal, generate final weight;

Concrete method is as follows:

1) initial weight is generated;

According to formula

$p_{{\mathrm{rv}}_i}^0=\frac{\left|N\left(i\right)\right|}{\left|A\right|}$

Try to achieve the initial weight of each width representational view | N (i) | be the view number in i-th class; | A| is the view number in model M; And then obtain initial weight value vector

2) final weight is generated;

The weighted value of every width representational view only depends on that the scale of place class is not accurate enough.When a width representational view and another width representational view quite close to time, this problem is just more obvious.Therefore, must consider that the relation between the representational view selected carries out weight renewal.

First, structure associated diagram describes the relation between representational view.Wherein each node on behalf one width representational view, the limit between two nodes represents two width representational view rv ₁and rv ₂between correlativity r (rv ₁, rv ₂).

According to formula

$r({\mathrm{rv}}_1,{\mathrm{rv}}_2)=\exp (-\frac{d{({\mathrm{rv}}_1,{\mathrm{rv}}_2)}^2}{{\mathrm{\σ}}^2})$

Try to achieve two width representational view rv ₁and rv ₂between correlativity; Wherein the value of σ is generally determined by experience, selects variance between all representational view as parameter in the present embodiment; D (rv ₁, rv ₂) represent two width representational view rv ₁and rv ₂between Euclidean distance.

Secondly, according to formula

$t({\mathrm{rv}}_1,{\mathrm{rv}}_2)=\frac{r({\mathrm{rv}}_1,{\mathrm{rv}}_2)}{{\mathrm{\Σ}}_{i}r({\mathrm{rv}}_1,{\mathrm{rv}}_i)}$

Try to achieve from representational view rv ₁to representational view rv ₂transition probability; Wherein, r (rv ₁, rv ₂) represent two width representational view rv ₁and rv ₂between correlativity.

Finally, according to formula

$\left\{\begin{array}{c}{p}_{{\mathrm{rv}}_1}^{n+1}=\mathrm{\γ}{p}_{{\mathrm{rv}}_1}^0+(1-\mathrm{\γ})\underset{i\≠1}{\mathrm{\Σ}}t({\mathrm{rv}}_i,{\mathrm{rv}}_1)p_i^n\\ p_{{\mathrm{rv}}_2}^{n+1}=\mathrm{\γ}{p}_{{\mathrm{rv}}_2}^0+(1-\mathrm{\γ})\underset{i\≠2}{\mathrm{\Σ}}t({\mathrm{rv}}_i,{\mathrm{rv}}_2)p_i^n\\ .\\ .\\ .\\ p_{{\mathrm{rv}}_k}^{n+1}=\mathrm{\γ}{p}_{{\mathrm{rv}}_k}^0+(1-\mathrm{\γ})\underset{i\≠k}{\mathrm{\Σ}}t({\mathrm{rv}}_i,{\mathrm{rv}}_k)p_i^n\end{array}\right.$

Try to achieve the final weight of each width representational view respectively; ..., after (n+1)th iteration respectively, the 1st width representational view, the 2nd width representational view ..., the weight of kth width representational view; ..., the 1st width representational view respectively, the 2nd width representational view ..., the initial weight of kth width representational view; Be the parameter determining initial weight value significance level, in the present embodiment, select γ=0.8; T (rv _i, rv _k) be transition probability from the i-th width representational view to kth width representational view; be the weighted value of the i-th width representational view after n-th iteration, k is clusters number, 1≤i≤k.

By experience through iteration several times, this process will converge stopping, and arranging iterations in the present embodiment is 5.And then obtain final weighted value vector $p^f=(p_{{\mathrm{rv}}_1}^f,p_{{\mathrm{rv}}_2}^f,...,p_{{\mathrm{rv}}_k}^f).$

105: utilize the representational view of two view collection and corresponding weighted value to build weighting bipartite graph;

If the representational view collection of searched targets A, wherein represent the 1st width representational view of searched targets A respectively, the 2nd width representational view ..., n-th _awidth representational view, n _ait is the representational view collection number of searched targets; the representational view collection of an object B in database, wherein represent the 1st width representational view of searched targets B respectively, the 2nd width representational view ..., n-th _bwidth representational view, n _bit is the representational view collection number of this object; with represent the weighted value set of an object B in searched targets A and database respectively.Successively all objects in searched targets A and database are weighted to the structure of bipartite graph.Concrete, the method building weighting bipartite graph is as follows:

1) foundation newly gathers R ';

Because the representational view number in representational view collection Q with R is not necessarily identical, so first will dimension be unified.N is supposed in the present embodiment _a>=n _b, n _a-n _bindividual new element adds in R.If j=1,2 ..., n _aif, j>n _b, then for sky, be 0.Thus ensure that two view collection have the representational view of equal number, be convenient to ensuing calculating and compare.So just set up and newly gather R '.

2) the weighted value g on limit is calculated _{i, j};

Each limit g in weighting bipartite graph _{i, j}(i, j=1,2 ..., n _a) represent the representational view of searched targets A with the representational view of an object in database between contact.

According to formula

$g_{i,j}=\left\{\begin{array}{cc}\frac{1}{2}(p_{{\mathrm{rv}}_i^a}^f+p_{{\mathrm{rv}}_j^b}^f)\×d\left({\mathrm{rv}}_i^a{\mathrm{rv}}_j^b\right)& \mathrm{if\; j}\≤n_b,\\ 0& \mathrm{otherwise}.\end{array}\right.$

Try to achieve the weighted value g on every bar limit _{i, j}; Wherein with the representational view of searched targets A respectively with the representational view of an object B in database weighted value; representative with between Euclidean distance.

3) weighting bipartite graph is built;

In the present embodiment, jointly set up weighting bipartite graph G={Q, R ', U} by the representational view collection R ' of an object B in the representational view collection Q of searched targets A and database.Wherein in node set Q, each node represents the width representational view in representational view collection Q; In node set R ', each node represents the width representational view in representational view collection R '; Limit set U={g _{i, j}, the weighted connections between all representational view representing an object B in all representational view in searched targets A and database.

Successively all objects in searched targets A and database are weighted to the structure of bipartite graph.

106: use bipartite graph matching algorithm to seek the Optimum Matching of weighting bipartite graph, obtain the similarity between each object in searched targets and database, and sort with this, using the result after sequence as search and output.

The bipartite graph matching algorithm of current popular can be adopted to ask the Optimum Matching of weighting bipartite graph, without loss of generality, have employed Kuhn-Munkres algorithm ^[5].

1) Optimum Matching is asked

To the weighting bipartite graph G={Q formed, R ', U}, use Kuhn-Munkres algorithm, under the constraint of mating one to one, can obtain the subgraph Λ that weights are minimum _m, as the Optimum Matching of this bipartite graph, and weights are sued for peace the Similarity value obtained in searched targets A and database between an object B.

According to the objective function Equation of most authority two points coupling

$\begin{array}{c}{\mathrm{\Λ}}_M=\mathrm{ar\; gma}{x}_{{\mathrm{\Λ}}_k\∈\mathrm{\Λ}}\underset{1\≤i\≤n}{\mathrm{\Σ}}{c}_{a_k\left(i\right),b_k\left(i\right)}\\ =\mathrm{ar\; gma}{x}_{{\mathrm{\Λ}}_k\∈\mathrm{\Λ}}\underset{1\≤i\≤n}{\mathrm{\Σ}}(G-g_{a_k\left(i\right),b_k\left(i\right)})\end{array}$

And Similarity value formula

$S_{\mathrm{Match}}={\max }_{{\mathrm{\Λ}}_k\∈\mathrm{\Λ}}\underset{1\≤i\≤n}{\mathrm{\Σ}}(G-g_{a_k\left(i\right),b_k\left(i\right)})$

Try to achieve Optimum Matching Λ _mand corresponding similarity S _match; Λ _krepresent bipartite graph matching; Λ is all possible bipartite graph matching; it is the element of n × n limit efficiency matrix C ^[6]; a in bipartite graph _k(i) and b _ki () is the weighted value on the limit that two matched node are formed; G is than max (g _ij) slightly larger constant, argmax function stand finds the parameter with maximal value, max function stand maximizing.

2) sequencing of similarity

According to the Similarity value S between object each in searched targets and database _match, sort from big to small, S _matchlarger both representatives similarity is higher.Using the result after sequence as search and output.

Experiment

1, experimental data base

Testing database used is the online ETH database shared, and has 80 three-dimensional models, comprises 8 classes, every class 10 objects in this database.Apple, car, milk cow, cup, doggie, horse, pears, tomato respectively.

2, evaluation criteria

Four kinds of evaluation criterias are applied in experiment ^[7], specific as follows:

(1) arest neighbors (Nearest neighbor is called for short NN): arest neighbors is closest to the number percent belonging to query categories coupling in inquiry.

(2) one-level precision ratio (First tier is called for short FT): the response of K arest neighbors coupling, wherein K is the radix of query categories.In this experiment, K=10.

(3) secondary precision ratio (Second tier is called for short ST): the response of 2K arest neighbors coupling, wherein K is the radix of query categories.

(4) standard-Cha full curve (Precision-Recall) is looked into: the average response (AverageRecall of the Performance Evaluation aspect of three-dimensional body retrieval, be called for short AR) and mean accuracy (Average Precision is called for short AP).

Try to achieve AR and AP according to following formula, make and look into standard-Cha full curve:

$\mathrm{Recall}=\frac{N_z}{N_r}$

Wherein, Recall is response value; N _zthe quantity of correct searching object; N _rthe quantity of all related objects.

$\mathrm{Precision}=\frac{N_z}{N_{\mathrm{all}}}$

Wherein, Precision is accuracy value; N _allthe quantity of all searching objects.

$\mathrm{AR}={\mathrm{\Σ}}_{i=1}^{N_m}\mathrm{Recall}\left(i\right)$

Wherein, N _mthe quantity of three-dimensional model class; Recall (i) is the response value of the i-th class.

$\mathrm{AP}={\mathrm{\Σ}}_{i=1}^{N_m}\mathrm{Rrecision}\left(i\right)$

Wherein, Precision (i) is the accuracy value of the i-th class.

3, algorithm is contrasted

In experiment, this method and following two kinds of methods are contrasted:

ED ^[8](A 3D Model Retrieval Approach Based on the Elevation Descriptor), is again the 3D searching algorithm based on height descriptors.

CCFV ^[9](Camera Constraint-Free View-Based 3D Object Retrieval), is again the 3D searching algorithm based on view under free-viewing angle.

4, experimental result

In ETH database, the standard-Cha full curve of looking into of three kinds of algorithms compares as Fig. 2.Wherein, ordinate represents essence (Precision), and (Recall) is responded in horizontal ordinate representative.Look into standard-Cha full curve and transverse and longitudinal area that coordinate encloses is larger, represent retrieval performance more excellent.

In ETH database, NN, FT and ST of three kinds of algorithms compare as Fig. 3.NN, FT and ST value is larger, represents retrieval performance more excellent.

Looking in standard-Cha full curve, this method curve and transverse and longitudinal area that coordinate encloses maximum, be obviously better than ED and CCFV; In ETH database, this method is compared with CCFV algorithm, and its NN, FT, ST index exceeds 16.25%, 6%, 4.25% respectively; Compared with ED algorithm, its NN, FT, ST index exceeds 17.5%, 13.88%, 13% respectively.As shown in experimental result, this method can reach better retrieval performance relative to ED and CCFV.

List of references

[1] Jia Hui, Liu Jianyuan, Zhang Jiangang. 3 d model library semantic net builds and search method research [J]. Xian Institute of Posts and Telecoms's journal, 2012,17 (3): 53-57.

[2] Zheng Baichuan. content-based 3D model index technical research [D]. Zhejiang University, 2004.

[3]Dalal?N,Triggs?B.Histograms?of?oriented?gradients?for?human?detection[C].//Computer?Vision?and?Pattern?Recognition,2005.CVPR?2005.IEEE?Computer?Society?Conference?on.IEEE,2005:886-893.

[4] the .K-means Research of Clustering Algorithms [J] such as Wang Qian, Wang Cheng, Feng Zhenyuan. electronic design engineering, 2012,20 (7) .DOI:10.3969/j.issn.1674-6236.2012.07.008.

[5] Huajian is new. and the web service based on semanteme finds and algorithm research [D]. Institutes Of Technology Of Changsha, 2010.

[6]Gao?Y,Dai?Q,Wang?M,et?al.3D?model?retrieval?using?weighted?bipartite?graph?matching[J].Signal?Processing:Image?Communication,2011,26(1):39-47.

[7]Gao?Y,Dai?Q,Zhang?N?Y.3D?model?comparison?using?spatial?structure?circular?descriptor[J].Pattern?Recognition,2010,43(3):1142-1151.

[8]Shih?J?L,Lee?C?H,Wang?J?T.A?new?3D?model?retrieval?approach?based?on?the?elevation?descriptor[J].Pattern?Recognition,2007,40(1):283-295.

[9]Gao?Y,Tang?J,Hong?R,et?al.Camera?constraint-free?view-based?3-D?object?retrieval[J].Image?Processing,IEEE?Transactions?on,2012,21(4):2269-2281.

It will be appreciated by those skilled in the art that accompanying drawing is the schematic diagram of a preferred embodiment, the invention described above embodiment sequence number, just to describing, does not represent the quality of embodiment.

The foregoing is only preferred embodiment of the present invention, not in order to limit the present invention, within the spirit and principles in the present invention all, any amendment done, equivalent replacement, improvement etc., all should be included within protection scope of the present invention.

Claims (6)

1. a method for various visual angles target retrieval, is characterized in that, said method comprising the steps of:

(1) the view collection of object in the searched targets of user's input and database is obtained;

(2) image characteristics extraction algorithm is used to carry out feature extraction to the view collection of object in described searched targets and database;

(3) adopt clustering method to carry out cluster to the view collection after feature extraction, and extract the representational view of each class;

(4) determine the corresponding initial weight of each width representational view according to the scale of place class, utilize the relation between representational view to carry out weight renewal, generate final weight;

(5) representational view of two view collection and their weighted value is utilized to build weighting bipartite graph;

(6) use bipartite graph matching algorithm to seek the Optimum Matching of described weighting bipartite graph, obtain the similarity between each object in searched targets and database, and sort with this, using the result after sequence as search and output.

2. the method for a kind of various visual angles target retrieval according to claim 1, is characterized in that, described utilization image characteristics extraction algorithm is specially the operation that the view collection of object in described searched targets and database carries out feature extraction:

Calculate the gradient of each width view regional area respectively, use statistical means to construct gradient orientation histogram, thus form gradient orientation histogram operator feature, be used for describing former view.

3. the method for a kind of various visual angles target retrieval according to claim 1, is characterized in that, described employing clustering method is specially the operation that the view collection after feature extraction carries out cluster:

First determine the exact amount K of wanted cluster, it, as cluster centre, to remaining each view, according to the distance of itself and each cluster centre, is assigned to nearest class by initial selected K view; Recalculate the mean value of view in each class, form new cluster centre; Repeat this process, until clustering convergence.

4. the method for a kind of various visual angles target retrieval according to claim 1, it is characterized in that, described representational view is specially:

Calculate the Euclidean distance sum between each width view and other views in each class, choose the view minimum with other view Euclidean distance sums representatively property view.

5. the method for a kind of various visual angles target retrieval according to claim 1, it is characterized in that, described initial weight is specially:

$p_{{\mathrm{rv}}_i}^0=\frac{\left|N\left(i\right)\right|}{\left|A\right|}$

Wherein, | N (i) | be view number in i-th cluster; | A| is view number in model M.

6. the method for a kind of various visual angles target retrieval according to claim 1, is characterized in that, described final weight is specially:

$\left\{\begin{array}{c}{p}_{{\mathrm{rv}}_1}^{n+1}={\mathrm{\γp}}_{{\mathrm{rv}}_1}^0+(1-\mathrm{\γ})\underset{i\≠1}{\mathrm{\Σ}}t({\mathrm{rv}}_i,{\mathrm{rv}}_1)p_i^n\\ p_{{\mathrm{rv}}_2}^{n+1}={\mathrm{\γp}}_{{\mathrm{rv}}_2}^0+(1-\mathrm{\γ})\underset{i\≠2}{\mathrm{\Σ}}t({\mathrm{rv}}_i,{\mathrm{rv}}_2)p_i^n\\ .\\ .\\ .\\ p_{{\mathrm{rv}}_k}^{n+1}={\mathrm{\γp}}_{{\mathrm{rv}}_k}^0+(1-\mathrm{\γ})\underset{i\≠k}{\mathrm{\Σ}}t({\mathrm{rv}}_i,{\mathrm{rv}}_k)p_i^n\end{array}\right.$

Wherein, after (n+1)th iteration respectively, the 1st width representational view, the 2nd width representational view ..., the weight of kth width representational view; the 1st width representational view respectively, the 2nd width representational view ..., the initial weight of kth width representational view; γ is the parameter determining original weighted value significance level; T (rv _i, rv _k) be transition probability from i-th representational view to a kth representational view; be the weighted value of the i-th width representational view after n-th iteration, k is clusters number, 1≤i≤k.