CN102789489A - Image retrieval method and system based on hand-held terminal - Google Patents

Abstract

The invention realizes an image retrieval system. The system allows a user to photograph an interested object or an interested scene through the hand-held terminal. After a picture is uploaded to a server, the system matches the existing picture in a database with the picture photographed by the user to discover the object or the scene photographed by the user, thereby determining an interesting point or a photographing position of the user; and then the related information is returned to the hand-held terminal of the user. A picture matching algorithm developed by the invention has the characteristics of resisting photographing angles, scales, rotation and light conversion, the matching precision achieves above 95.34 percent, and the application level is achieved.

Description

Image search method and system based on handheld terminal

Technical field

The present invention relates to the picture search technical field, a kind of image search method and system based on handheld terminal is provided.

Background technology

Present main flow search engine all is based on literal, yet the request of oneself inquiring about all can not perhaps inconveniently be described under many circumstances by domestic consumer with literal.The mode that the present invention allows the user to take pictures through handheld terminal is expressed oneself query requests with picture, and is directly perceived and make things convenient for, and satisfied the actual demand of people's lives.At present the similar image retrieval algorithm of success mostly be based on the partial interest point (Local Interesting Points, LIPS).Yet because it is very high to extract the algorithm time complexity of required LIPs, existed algorithms is difficult to real-time network picture retrieval.Algorithm of the present invention roughly levels off on precision based on the traditional algorithm of LIPs, but on time complexity, is far smaller than classic method, has satisfied the picture retrieval needs of real-time network environment.

Summary of the invention

The objective of the invention is to overcome the deficiency of prior art, propose the image matching method of a kind of anti-shooting angle, yardstick, rotation, illumination conversion.

The present invention is for realizing that above-mentioned purpose adopts following technical scheme:

1, a kind of image search method based on handheld terminal is characterized in that may further comprise the steps:

1) the camera routine interface that calls handheld terminal carries out photo acquisition;

2) the networking transmission interface that calls handheld terminal is sent to server with picture;

3) result who at last server is returned shows in user interface.

In the such scheme, server process adopts treatment step to comprise after receiving picture in the said step 2:

21) accept the picture that client-side program is uploaded;

22) picture being carried out paster decomposes;

23) paster is mapped in the paster space;

24) with the storehouse in existing picture mate;

25) according to the point of interest of matching result predictive user or the position of taking pictures;

26) relevant information is sent to client-side program.

In the such scheme, said step 22) the paster decomposition method may further comprise the steps in:

31) make up quadrature paster spatial model:

A, every width of cloth picture is modeled as the combination of one group of paster, the statistical nature that on contained all the member's pasters of a width of cloth picture, draws is used for representing the characteristic of this picture;

B, through collecting large-scale picture; And the paster of all pictures gathered and carry out cluster analysis; Find the paster that compacts a most set

through the right similarity of all pasters in weighing

; Construct a similarity matrix R; R is carried out obtaining an eigenvectors after the spectral factorization; The vector of this group paster during vector is

is expressed, and the space that this eigenvectors is opened is the paster space of asking;

C, use spectral factorization (Spectral Decomposition) are optimized the orthogonality in space, and process is following: at first, calculate

In all pasters to (u _i, u _j) the cosine similarity and put into matrix R=[r _Ij] _{M * m}, wherein

$r_{\mathrm{ij}}=\mathrm{Sim}({\stackrel{\→}{u}}_i,{\stackrel{\→}{u}}_j)=\cos ({\stackrel{\→}{u}}_i,{\stackrel{\→}{u}}_j)=\frac{{\stackrel{\→}{u}}_i{\stackrel{\→}{u}}_j}{\left|{\stackrel{\→}{u}}_i\right|\left|{\stackrel{\→}{u}}_j\right|}$

Be paster u _iAnd u _jCorresponding minutia vector then, carries out spectral factorization to R, R=V Λ V ^T=(V Λ ^1/2V ^T) (V Λ ^1/2V ^T)=C ^TC

Wherein Λ is the diagonal matrix that has comprised the R eigenwert; The corresponding vector of all pasters in V is the characteristic of correspondence vector matrix, and Matrix C has comprised

.The actual base vector that comprises one group of linear space of C that obtain this moment, the space that all base vectors are opened is the quadrature paster space of being asked.

32) be independent of the feature representation technology of conversion: 31) on the basis in said paster space, arbitrary non-ly in the paster space, be expressed as with reference to paster

:

$C^T\stackrel{\→}{u}=R_u$

$\stackrel{\→}{u}={\left(C^T\right)}^{-1}{R}_u$

R wherein _uBe the m dimensional vector, expressed paster u with In all relations with reference to paster.This moment, any image T can be expressed as the vector sum of the vector that all corresponding pasters of this picture form in the paster space in the paster space, promptly

$T=\underset{k=1}{\overset{n}{\mathrm{\Σ}}}{\stackrel{\→}{u}}_k$

33) Adaptive Matching Algorithm: the picture that will be regarded as the candidate above the picture of certain threshold value with source picture analogies degree; Use hidden Markov model (Hidden Markov Model) that the spatial relation between paster is carried out modeling; Come to recognize automatically the pattern that comprises in the picture; Then, in all candidate's picture set, search for these patterns to find final result.

With respect to the traditional image matching technique, the model that the present invention proposes has following several advantages:

One, the balance of speed and precision: with respect to the method (as: color moment that lattice extracts) of global characteristics, the feature representation that the present invention proposes has certain resistance transducing power, thereby precision is higher; Traditional relatively local feature method (as: SIFT, SURF etc.), the method algorithm time complexity that the present invention proposes is lower, and detection speed is faster, is more suitable for large scale network and calculates.

Two, extensibility: the vector in the paster space that the present invention makes up can calculate mutually, for further mining algorithm has stayed development space.The structure that adds the space has been considered orthogonality (be each base vector not linear dependence), can guarantee the correlation computations mathematical model of convergence linear space more in the space, thereby more accurate.

Three, core of the present invention is the images match model based on the paster space.The characteristics of image expression is carried out in paster space of this model construction, has improved the mode of in the classic method all pasters in two width of cloth Target Photos (perhaps unique point) being compared one by one, thereby has reduced the time complexity of copy picture detection algorithm.Simultaneously, picture is carried out the method for paster decomposition and broken the locus strong constraint between local feature point in the picture, reduced of the influence of various copy picture editor modes, thereby improved the robustness of algorithm feature representation.

Description of drawings

Fig. 1 is a system block diagram of the present invention;

Fig. 2 is an image paster decomposing schematic representation of the present invention;

Fig. 3 is a system flow synoptic diagram of the present invention.

Embodiment

Embodiment of the present invention are made up of client algorithm and server end algorithm two parts.

The objective of the invention is to overcome the deficiency of prior art, propose the image matching method of a kind of anti-shooting angle, yardstick, rotation, illumination conversion.

Technical scheme of the present invention is:

At first make up the paster spatial model that to express picture; Picture can be tried one's best in the expression in this space do not receive the influence of various picture conversion, be created on this basis then and can both satisfy network (being mainly cell phone network) picture Matching Algorithm on speed and the precision.So correlative study is divided into three parts: quadrature paster spatial model, be independent of the feature representation technology of conversion and Adaptive Matching Algorithm.Below do respectively and briefly introduce:

Quadrature paster spatial model: this model is modeled as the combination (as shown in Figure 2) of one group of paster (a little image block) with every width of cloth picture, and the statistical nature that therefore on contained all the member's pasters of a width of cloth picture, draws can be used for representing the characteristic of this picture.

Through collecting large-scale picture; And the paster of all pictures gathered and carry out cluster analysis; Can find the paster that compacts most set

this set to comprise the most representative one group of paster, react the various minutias that picture possibly carry.Through the right similarity of all pasters in weighing

, can construct a similarity matrix R.R is carried out can obtaining an eigenvectors after the spectral factorization.The vector of this group paster during vector is

is expressed.The space that this eigenvectors is opened is the paster space of asking.Because

is the most representative paster group; Therefore any paster (even not in

) can be weighed its entrained details classification through the similarity of all pasters in weighing with

.For example, if a paster X follows

In two paster u _iAnd u _jThe most similar, explain that then the entrained details of X is u _iAnd u _jThe combination of entrained minutia.So the vector x that the similarity of all pasters constitutes among each picture and

has been expressed the distribution of its entrained details statistics.

But because with reference to also having certain similarity between the paster (i.e. all pasters in

), direct in theory to make up the result that the method in space obtains with paster and their similarity be nonopiate.This nonorthogonality can influence the accuracy of net result.Therefore can use spectral factorization (Spectral Decomposition) that the orthogonality in space is optimized.Process is following: at first, calculate

In all pasters to (u _i, u _j) the cosine similarity and put into matrix R=[r _Ij] m * m, wherein

$r_{\mathrm{ij}}=\mathrm{Sim}({\stackrel{\→}{u}}_i,{\stackrel{\→}{u}}_j)=\cos ({\stackrel{\→}{u}}_i,{\stackrel{\→}{u}}_j)=\frac{{\stackrel{\→}{u}}_i{\stackrel{\→}{u}}_j}{\left|{\stackrel{\→}{u}}_i\right|\left|{\stackrel{\→}{u}}_j\right|}---\left(1\right)$

Be paster u _iAnd u _jCorresponding minutia vector (can use color moment, small echo texture etc.).Then, matrix R is carried out spectral factorization,

R＝VΛV ^T＝(VΛ ^1/2V ^T)(VΛ ^1/2V ^T)＝C ^TC (2)

C ^TBe the transposed matrix of Matrix C, wherein Λ is the diagonal matrix that has comprised the R eigenwert, and V is the characteristic of correspondence vector matrix.The corresponding vector of all pasters can be expressed as the actual base vector that comprises one group of linear space of

C during Matrix C had comprised

.These base vectors open space be exactly the quadrature paster space that we require.Its orthogonality can guarantee that the expression of paster in the space has global coherency (Global Consistency).Last arbitrary non-ly can in the paster space, be expressed as with reference to paster

:

$C^T\stackrel{\→}{u}=R_u$

$\stackrel{\→}{u}={\left(C^T\right)}^{-1}{R}_u---\left(3\right)$

R wherein _uBe the m dimensional vector, expressed paster u with In all relations (being cosine similarity shown in the formula 1) with reference to paster.Any image T can be expressed as the vector sum of the vector that all corresponding pasters of this picture form in the paster space in the paster space, promptly

$T=\underset{k=1}{\overset{n}{\mathrm{\Σ}}}{\stackrel{\→}{u}}_k---\left(4\right)$

Through above-mentioned model, any image can be mapped in the paster space and be expressed as a vector.Therefore similarity between image can be judged through the similarity (or distance) of calculating their corresponding vectors simply.This similarity account form has been avoided multiple spot characteristic matching mode in twos, and the time complexity of algorithm is from O (N ²) ease down to O (cN) (wherein c is a constant, represents the dimension in paster space).

Be independent of the feature representation technology of conversion: the paster that the paster spatial model carries out image decomposes breaks up the strong position constraint of each local feature on the picture.Because the influence that local detail is brought by various editor's characteristic changes is very little,, can more effectively deal with the influence of the editing operations such as rotation, displacement, montage and small-scale convergent-divergent of image so the vector that obtains in paster space expression robustness is higher.

Adapting to image matching algorithm: judge that whether two width of cloth pictures comprise unified object or scene, can weigh according to their similarity simply.But,, make the pattern that has (Pattern) of some object itself also lose owing to broken up the position constraint of local feature thereupon.Therefore, in the research only similarity as a kind of reference value of selecting candidate's copy, that is: will be regarded as candidate's picture with the picture that source picture analogies degree surpasses certain threshold value.In further matching algorithm, use hidden Markov model that the spatial relation between paster is carried out modeling, can recognize the pattern that comprises in the picture automatically like this.Then, in all candidate's picture set, search for these patterns to find final result.The discovery of these patterns does not need to train in advance, is adaptive.

Touch image matching system operational scheme constructed on the basis of type in the paster space as shown in Figure 3.System is divided into off-line training and two parts of online coupling.In the off-line part, at first collect a large amount of pictures as training set, all pasters to training set carry out carrying out cluster analysis after paster extracts again, use said method structure paster space then.In the online compatible portion, when each tests picture arrival, at first carry out paster and extract, re-use the paster space that trains each test picture is carried out paster expression and other vector quantization expression of picture level.Carry out similarity according to vector expression then and calculate, the foundation that its result judges as the copy picture of same object or scene (promptly comprise to).

Claims (3)

1. image search method based on handheld terminal is characterized in that may further comprise the steps:

1) the camera routine interface that calls handheld terminal carries out photo acquisition;

2) the networking transmission interface that calls handheld terminal is sent to server with picture;

3) result who at last server is returned shows in user interface.

2. a kind of image search method based on handheld terminal according to claim 1 is characterized in that: server process adopts treatment step to comprise after receiving picture in the said step 2:

21) accept the picture that client-side program is uploaded;

22) picture being carried out paster decomposes;

23) paster is mapped in the paster space;

24) with the storehouse in existing picture mate;

25) according to the point of interest of matching result predictive user or the position of taking pictures;

26) relevant information is sent to client-side program.

3. a kind of image search method based on handheld terminal according to claim 2 is characterized in that: the paster decomposition method may further comprise the steps said step 22):

31) make up quadrature paster spatial model:

A, every width of cloth picture is modeled as the combination of one group of paster, the statistical nature that on contained all the member's pasters of a width of cloth picture, draws is used for representing the characteristic of this picture;

B, through collecting large-scale picture; And the paster of all pictures gathered and carry out cluster analysis; Find the paster that compacts a most set

through the right similarity of all pasters in weighing

; Construct a similarity matrix R; R is carried out obtaining an eigenvectors after the spectral factorization; The vector of this group paster during vector is

is expressed, and the space that this eigenvectors is opened is the paster space of asking;

C, use spectral factorization (Spectral Decomposition) are optimized the orthogonality in space, and process is following: at first, calculate

In all pasters to (u _i, u _j) the cosine similarity and put into matrix R=[r _Ij] _{M * m}, wherein

$r_{\mathrm{ij}}=\mathrm{Sim}({\stackrel{\→}{u}}_i,{\stackrel{\→}{u}}_j)=\cos ({\stackrel{\→}{u}}_i,{\stackrel{\→}{u}}_j)=\frac{{\stackrel{\→}{u}}_i{\stackrel{\→}{u}}_j}{\left|{\stackrel{\→}{u}}_i\right|\left|{\stackrel{\→}{u}}_j\right|}$

Be paster u _iAnd u _jCorresponding minutia vector then, carries out spectral factorization to R,

R＝VΛV ^T＝(VΛ ^1/2V ^T)(VΛ ^1/2V ^T)＝C ^TC

Wherein Λ is the diagonal matrix that has comprised the R eigenwert; The corresponding vector of all pasters in V is the characteristic of correspondence vector matrix, and Matrix C has comprised

.The actual base vector that comprises one group of linear space of C that obtain this moment, the space that all base vectors are opened is the quadrature paster space of being asked.

32) be independent of the feature representation technology of conversion: 31) on the basis in said paster space, arbitrary non-ly in the paster space, be expressed as with reference to paster

:

$C^T\stackrel{\→}{u}=R_u$

$\stackrel{\→}{u}={\left(C^T\right)}^{-1}{R}_u$

R wherein _uBe the m dimensional vector, expressed paster u with

In all relations with reference to paster.This moment, any image T can be expressed as the vector sum of the vector that all corresponding pasters of this picture form in the paster space in the paster space, promptly

$T=\underset{k=1}{\overset{n}{\mathrm{\Σ}}}{\stackrel{\→}{u}}_k$

33) Adaptive Matching Algorithm: the picture that will be regarded as the candidate above the picture of certain threshold value with source picture analogies degree; Use hidden Markov model (Hidden Markov Model) that the spatial relation between paster is carried out modeling; Come to recognize automatically the pattern that comprises in the picture; Then, in all candidate's picture set, search for these patterns to find final result.

Images