CN104200240B - A kind of Sketch Searching method based on content-adaptive Hash coding - Google Patents

Abstract

The invention discloses a kind of Sketch Searching method based on content-adaptive Hash coding, it is characterised in that including step in detail below：Constrained based on appearance constraints and diversity, sketch or profile diagram to being retrieved choose candidate window according to its content-adaptive for feature extraction, realize that the information that whole image is included is evenly distributed in each window；Conspicuousness detection method based on key point detects the conspicuousness of characteristic window；Hash algorithm based on local sensitivity combines the local visual feature of sketch or profile diagram, conspicuousness and structure space feature, is compiled into feature Hash codes；The feature Hash codes of sketch or profile diagram are indexed, the similarity between sketch is measured by calculating the Hamming distance between feature Hash codes, and by similarity it is high return to user.Can have precision higher, wider adaptability and stronger matching capacity using the present invention.

Description

A kind of Sketch Searching method based on content-adaptive Hash coding

Technical field

The present invention relates to image processing field, more particularly to a kind of sketch inspection based on content-adaptive Hash coding Suo Fangfa.

Background technology

In recent years, by the use of sketch (such as vision material such as hand-drawing graphics, picture and threedimensional model information) as input come Retrieval is always the study hotspot of computer vision field.Because becoming increasingly popular with touch-screen equipment, people's more biasing Like that this mode can be preferably in the input and interaction that complete various information with computer using modes such as gesture, felt pens Give expression to the intention of user and operate extremely simple.Meanwhile, all kinds of retrieval tasks are carried out by way of cartographical sketching, this is to make With handheld input device (such as Apple iPhone/iPad, Microsoft Surface and other all kinds of panel computers etc.) User provides very strong convenience.

In all kinds of sketch interactive tasks, sketch visual information carrier different from hand-drawing graphics, picture and 3D models etc. Matching problem occupied an important position in whole sketch interactive computing field, same this is also based on the search method of sketch Basic research problems.In order to obtained in retrieval tasks as far as possible with the cognitive of people, intuitive visual impression always as a result, it is desirable to Having a kind of efficient algorithm can accurately and rapidly measure the similarity between input sketch and the information that is retrieved, based on sketch Search method research be exactly be intended to solve this problem.From the point of view of real value, on the one hand, the search method based on sketch It is closely related with Practical Project technology application, do not only have very strong field applicability, or it is various based on the main of sketch application Core technology.On the other hand, it also has very strong scientific research value, and it explores computer recognition and artificially creates vision figure The basic method of shape.

In traditional retrieval tasks, a sketch is generally viewed as a series of set of Freehandhand-drawing strokes, is represented with this The information such as one skeleton of object, profile.However, the content such as other appearance details that object is possessed, color, texture, But inevitably be lost during sketch is converted into.Due to the characteristic, search method and tradition based on sketch The searching algorithm based on picture compared to there is very big difference.Recently for a period of time, carried out in the field Research work always strive to answer a problem：How can reasonably extract and be contained in relatively each sketch Characteristic information

In the recent period, the method based on segmentation (segmentation) is widely used, and is led in Sketch Searching and identification The verified this kind of algorithm in domain is very effective.The stroke that they would generally include sketch during realization is divided Cut, computation complexity is reduced with this.Then the extraction of the attributes such as topology, geometry, then to the stroke after segmentation is carried out.So And, carrying out accurate stroke segmentation for sketch is difficult to accomplish, particularly with the picture comprising natural landscape or 3D moulds For type, this step becomes increasingly difficult.Because the contour feature of such visual information typically include many noises, A perfect stroke segmentation almost impossible mission is carried out to them.Thus, exactly this shortcoming is greatly limited Made applicability of the class method based on stroke segmentation when be used to picture or 3D models is retrieved based on sketch.

Other research work are referred from traditional picture retrieval algorithm, and sketch is compared by provincial characteristics (patch) Similarity complete retrieval tasks.First, a sketch is equably divided into zonule one by one, is then therefrom extracted not Same visual signature description.Under normal conditions, the grid or dense grade that this kind of method will be overlapped with region are big Window is covered on whole sketch, and the feature distribution situation of the sketch is described with this.This is highly suitable for common picture, because For a pictures have usually contained the minutias such as abundant color or texture, but, for only including limited stroke This kind of sparse contour images of sketch are inapplicable.In this case, the most information that sketch is included all can The image-region the inside of several highly significants is covered, and causes remaining region almost vacancy.Due to this extreme not The phenomenon of balance, can produce many invalid Feature Descriptors, and these characteristic similarities of description after participating in are calculated In after can substantially reduce the validity of calculating, be additionally, since was needed to it before being indexed to them using hash algorithm Binaryzation is carried out, this can cause that only visual information is further lost, so as to allow situation to become even worse.

Another problem of search method based on provincial characteristics is the comparison for calculation methods to similarity between sketch Weakness, it is not strict accurate.One cartographical sketching typically include lines stroke miscellaneous rather than color or texture Different objects are showed etc. content, thus sketch has very big difference with traditional picture both ways：Huge Otherness in class (because each sketch drafting person more or less has the subjective understanding of oneself for same thing) and compared with Small class inherited (due to the missing of this kind of very important visual information such as color or texture).Even which results in identical The sketch of theme remains unchanged and contains the provincial characteristics of a large amount of dissmilarities.Therefore, " similar picture has usually contained a large amount of similar Provincial characteristics ", the concept that this is widely used in picture retrieval algorithm for based on sketch search method for not Accurately, its definition constraint to similarity excessively it is strict thus become not effective enough.

Therefore, in view of the analysis of the above is it can be found that the existing retrieval technique based on sketch and imperfection, need Improve and develop.Present invention search method of the research based on provincial characteristics, because it is applicable compared to the method split based on stroke Property it is higher, and concentration solve two problems present in the above-mentioned search method based on provincial characteristics.

The content of the invention

It is an object of the invention to provide a kind of Sketch Searching method based on content-adaptive Hash coding, it is intended to solve The existing Sketch Searching method based on Hash coding does not take into account sketch own content characteristic distributions when visual signature is extracted, and adopts Method can substantially reduce the validity of calculating, and weak to the comparison for calculation methods of similarity between sketch, not strictly Accurate problem.

Technical scheme is as follows：A kind of Sketch Searching method based on content-adaptive Hash coding, it includes Step in detail below：

Step A：Constrained based on appearance constraints and diversity, to the sketch or profile diagram that are retrieved according to its content-adaptive Candidate window is chosen on ground is used for feature extraction, realizes that the information that whole image is included is evenly distributed in each window；

Step B：Conspicuousness detection method based on key point detects the conspicuousness of characteristic window；

Step C：Hash algorithm based on local sensitivity is by the local visual feature of sketch or profile diagram, conspicuousness and knot Conformational space feature combines, and is compiled into feature Hash codes；

Step D：The feature Hash codes of sketch or profile diagram are indexed, by calculating the Hamming between feature Hash codes Distance measures the similarity between sketch, and similarity result high is returned into user.

Described Sketch Searching method, wherein, picture and 3D models can also be retrieved, enter to picture and 3D models Before row retrieval, it is pre-processed, convert them into contour line picture.

Described Sketch Searching method, wherein, picture is respectively with 3D model conversations into the method for contour line picture：It is right The conspicuousness detection algorithm of jointing edge extraction algorithm and picture is wanted to calculate the remarkable configuration figure of the picture in picture；For 3D models, the corresponding contour projection charts of model are calculated according to the matching algorithm based on visual angle.

Described Sketch Searching method, wherein, for the candidate window Algorithms of Selecting of feature extraction：First it is the grass of input Figure sets the grid of initialization n*n；M*m initial seed of uniform sampling from grid again；Then it is the window in each grid Calculate HOG features h_i；Finally calculate overall situation HOG features

Described Sketch Searching method, wherein, described appearance constraints are denoted as C_app, it is embodied as：C_app(h):=F_app (h)≥k_app×F_app(H)

Wherein, whereinF_appIt is the target equation of appearance constraints, works as F_appObtained by calculating When value is higher, it represents that the visual signature information that this feature window is included is more.

Described Sketch Searching method, wherein, described diversity constraint is denoted as C_var, it is embodied as：C_var(h):= F_var(h)≤k_var×F_var(H)

Wherein, whereinF_varIt is the target equation of diversity constraint, If F_varValue is relatively low and F_appIt is higher, then show that the value of each dimension in window feature vector h is higher.

Described Sketch Searching method, wherein, the conspicuousness detection method based on key point detects the notable of characteristic window The specific method of property is：First, it is special for each using Harris-Laplace detectors as sketch conspicuousness extracting tool Levy window w_i, define its conspicuousness k_iFor：

Wherein, Number (S_i) represent in characteristic window w_iIt is notable that middle use Harris-Laplace detectors are extracted Point number；Area(w_i) it is w_iNumber comprising pixel.

Described Sketch Searching method, wherein, the method for being compiled into feature Hash codes is：Make f_iIt is from characteristic window w_iIn Its binaryzation is first vector by the characteristic vector for extractingThen, it then follows the calculating process of similar hash algorithm, according to each Window is correspondingAnd k_iValue is calculated the feature Hash codes of the window；Then, by sketch difference in the horizontal and vertical directions It is divided into two, obtains four locus of separation, the candidate window being pointed on each locus carries out Hash volume respectively Code；Finally, breathed out so as to obtain representing the feature of whole sketch by the Hash codes on four locus successively head and the tail splicing Uncommon code.

Described Sketch Searching method, wherein, Feature Descriptor also can select the Feature Descriptor or part of Scale invariant Linear Gabor characteristic description.

Described Sketch Searching method, wherein, select the HOG features without normalized to describe each subwindow Comprising visual information.

Beneficial effects of the present invention：The present invention by during sketch or profile diagram feature is extracted by itself Hold distribution situation to take into account so that all of feature is tried one's best and is uniformly distributed in candidate window, and the feature for being extracted Vector can more characterize single sketch its own the characteristics of.Compared to the searching algorithm split based on stroke, these features receive picture, 3D The influence of the factors such as the noise and texture that are included in model silhouette figure is smaller；It is based on for the search method of Hash compared to other, Algorithm proposed by the invention has precision higher, wider adaptability and stronger matching capacity.The application of this method Can be generalized to other carries out the research field that Hash is encoded to key technology with to characteristic information, is of universal significance.

Brief description of the drawings

Fig. 1 is the result schematic diagram retrieved on various data sets using the present invention.

Fig. 2 a, 2b, 2c are sketch characteristic window Selection Strategy schematic diagrames.

Fig. 3 a, 3b, 3c are the notable window calculation result schematic diagrams of sketch.

Fig. 4 is sketch provincial characteristics hash algorithm flow chart.

Fig. 5 is to calculate picture contour images intermediate result schematic diagram.

Fig. 6 a, 6b are the curve maps for comparing this method difference assembly property.

Fig. 7 a, 7b are the Performance comparision curve maps retrieved using algorithms of different.

Specific embodiment

To make the objects, technical solutions and advantages of the present invention clearer, clear and definite, develop simultaneously embodiment pair referring to the drawings The present invention is further described.

Searching algorithm based on sketch proposed by the invention mainly includes following three part：With reference to input sketch Own feature, according to two class constraintss adaptively selected characteristic extract window；The key point included using window is believed Cease adaptively to detect the conspicuousness of each characteristic window；Two category feature information are mutually tied with the structural information of sketch more than Close, they are compiled into for creating aspect indexing by Hash codes by LSH algorithms.Next, this specification to be again divided into this several Part is illustrated to the present invention.

As shown in Figure 2 a, a sketch is given, the uniform grid of n*n is divided into first.Then, as shown in Figure 2 b, M*m point is equably chosen on the crosspoint of these grids as the initial seed for producing all characteristic windows, is place then It is subwindow set Δ w (x, y, i) in the seed point definition on sketch (x, y) position, Δ w (x, y, i) illustrates all surrounding The i-th circle window on seed.As shown in Figure 2 c, wherein black round dot represents the seed point of selection, then around the seed point 4 closest subwindows are exactly the 1st circle window x for surrounding the seed point, are designated as Δ w (x, y, 1)；Around the 1st circle window Mouthful and all subwindows closest therewith are exactly the 2nd circle window for surrounding the seed pointIt is designated as Δ w (x, y, 2).

For window w (x, y) for finally producing suitable size to each initial seed, it is necessary to successively iteratively Δ W (x, y, i) is added to w (x, y), shows that its characteristic information for including has filled until w (x, y) meets some requirements to constrain Foot, or when w (x, y) becomes an illegal window, such as window overflowed whole sketch edge or it became Greatly (more than the whole a quarter of sketch).

Therefore, the selection and design of constraint becomes to be even more important, and this is directly connected to the characteristic window of final selection Quality, the characteristic information that most window included is may result in if constraint is excessively strict not enough；Conversely, Window Mouth can then become too much, so that substantial amounts of region is repeatedly included in multiple windows so that characteristic information becomes undue Redundancy, increases the amount of calculation that latter step characteristic similarity compares.The present invention proposes following two effective constraints policies.

Due to the HOG (Histogram of Oriented Gradients, gradient orientation histogram) of image, feature is It is widely used in the computer vision research such as object detection and image retrieval field and achieves good effect, therefore, this HOG feature of the method selection without normalized describes the visual information that each subwindow is included.In the HOG of standard During feature calculation, influence of the shadow to image is often removed using normalized, and sketch is by stroke and the back of the body Black-and-white two color image that scape is constituted and in the absence of light change, thus have no effect on feature using not normalized HOG features and carry The precision for taking, and have benefited from the reduction of amount of calculation, arithmetic speed is accelerated to a certain extent.Note vector h={ b₁,b₂,…, b_nIt is the HOG features extracted from window w (x, y), Δ h (x, y, i) is the HOG of window set corresponding to Δ w (x, y, i) Characteristic vector；Note H is the HOG features H vectors of whole sketch, because histogram is with additive property, therefore when calculating grass The HOG characteristic vectors h of all subwindows in figure_iAfterwards, the HOG histograms spy of full figure can be directly obtained by the way that these values are added up H is levied, then it is defined asThe appearance constraints of sketch characteristic window are defined as below, C is denoted as_app, its definition is such as Under：

C_app(h):=F_app(h)≥k_app×F_app(H) formula 1

Wherein, whereinF_appIt is the target equation of appearance constraints, substantially it is calculated The average of HOG features h.Work as F_appWhen value obtained by calculating is higher, it represent the visual signature information that is included of this feature window compared with It is many, conversely, relatively low F_appValue then represents that whole window is almost sky.Obviously, constraints C_appEnsure that each Window Mouth needs to possess visual signature information abundant enough.

Second constraint that this method is proposed is referred to as diversity constraint, is designated as C_var, it is defined as follows：

C_var(h):=F_var(h)≤k_var×F_var(H) formula 2

Wherein, whereinF_varIt is the target equation of diversity constraint, It is actually the variance of HOG features h.If F_varValue is relatively low and F_appIf higher, then show in window feature vector h Each dimension value it is higher.It therefore meets C_varThe window of constraint will include more diversified feature, rather than Straight line or line segment of single direction etc., this multifarious feature have proved to be highly useful in Sketch Searching problem.

It is worth noting that, parameter k_appAnd k_varSketch overall situation HOG features H is controlled to produced by two kinds of constraintss Influence.Found k by specific experiment_appAnd k_var0.8 and 1 is respectively set to be possible to obtain best retrieval performance.Fixed More than justice after two class constraints, with each initial seed position as starting point, iteratively the subwindow of surrounding is added It is added to wherein until everywhere window all meets this two classes constraint, this completes the selection of feature candidate window.Abide by The algorithm for following above window Selection Strategy is designated as window considerations constraint Algorithms of Selecting, and the work of above-mentioned whole algorithm has been summarized below Flow：

Step a：The grid of n*n is initialized for input sketch；

Step b：M*m initial seed of uniform sampling from grid；

Step c：It is the window calculation HOG features h in each grid_i；

Step d：Calculate overall situation HOG features H.

Specific algorithm is as follows：

Candidate window " iteratively " is produced using " two constraints " due to being previously mentioned, so here with puppet Code provides a kind of implementation method example of simple, intuitive, helps user to understand how to use two constraints.

It is worth noting that, constraint C_appTarget equation F_appCan being incremented calculate, that is to say, that its meet etc. Formula：

F_app(h+ Δs h)=F_app(h)+F_app(Δ h) (3) formula 3

As window ceaselessly increases, it is only necessary to by the F of characteristic vector corresponding to incremental portion subwindow_appValue is added to In the result of preceding an iteration, and simultaneously need not recalculate F on new window obtained by each iteration_appValue.Therefore, Verify whether characteristic window meets constraint C in each iterative process_appIt is very rapid.Although F_varCan not being incremented Calculate, but until each feature candidate window is meeting constraints C_appIt need not all be calculated before, so, total comes See, the calculating process of the whole characteristic window of this method is relatively quick.

In an experiment it can be seen that, either from Scale invariant Feature Descriptor, local linear Gabor characteristic description Sub (Gabor Local Line-based Feature, GALIF) or previously described HOG Feature Descriptors, using this hair Bright characteristic window Selection Strategy proposed above can significantly increase the precision of retrieval.

After the selection for completing sketch characteristic window, this method also proposed a kind of each window of detection in sketch is input into The method of conspicuousness.For sketch, its local features is characterized, use the conspicuousness extraction algorithm of distinguished point based, example Such as multiple dimensioned Gauss model, Hessian algorithms and Harris-Laplace detectors, the conspicuousness in region is based on than using Extraction algorithm comes more effectively.This is that due to different from the picture comprising continuum, a sketch typically include many Independent separate lines and point.The conspicuousness extraction algorithm of distinguished point based is just designed to find significant in image Point, so the node or flex point that are very suitable for detecting in sketch, this is also the key message place included in sketch. In the present invention, it is found through experiments that Harris-Laplace detectors possess more preferable property when sketch key point is detected Can, therefore, use it as sketch conspicuousness extracting method of the invention.

Belong to the candidate window w of W for each_i(wherein W is the result that algorithm 1 is returned by mentioned earlier, is all selections Characteristic window set), define its conspicuousness for k_i, be can be calculated according to equation below：

Wherein, Number (S_i) represent in characteristic window w_iIt is notable that middle use Harris-Laplace detectors are extracted Point number；Area(w_i) it is w_iNumber comprising pixel.Intuitively, can draw：When a window is smaller and comprising more Significant point when, the window is more notable.

It should be noted that in order to prevent the conspicuousness of characteristic window excessively sensitive for its size, resolution ratio, above formula is adopted Its size is characterized with the secondary radical sign of window pixel level area.Fig. 3 is illustrated as a sketch detection characteristic window is notable Property intermediate result, wherein Fig. 3 a be input into sketch example；Soft dot in Fig. 3 b is represented and used in this sketch All key points that Harris-Laplace detectors are found out；Fig. 3 c square frames have been indicated by formula 4 calculated preceding 3 The most significant candidate feature window of the individual mutual area coverage less than 20%.

After vision description and its conspicuousness that calculate candidate feature window, in order that obtain these information can have Effect degree of being used for like compare and index among, it is necessary to these contents are organically integrated.Based on the similar of Hash Degree metric algorithm can effectively retrieve large-scale data set, while ensure high calculating speed, thus, the present invention The hash algorithm of local sensitivity has been used to be encoded come the feature to extracting, details are provided below.This method is in reality Algorithm is described using HOG as window feature during existing.

Make f_iIt is from characteristic window w_iIn the characteristic vector that extracts, then by by f_iIn preceding 40% value highest position be set to 1st, remaining is set to -1, you can by f_iTwo-value turns to vectorIt is pointed out that having benefited from two kinds of spies of this algorithm proposition Levy window choose constraints, and then be effectively guaranteed each window can be comprising enough characteristic informations so that two During value, the loss of information is reduced significantly.Then, the present invention have references to the meter of similar hash algorithm (Sim-hash) Calculation process, it is corresponding by each windowAnd k_iValue is calculated the feature Hash codes of the window.

In actually calculating, by window conspicuousness k_iAsWeight.The spatial information that one sketch is included is examined in sketch A highly useful category feature is had proven in rope field.Therefore, in order to the spatial relationship bag of local feature in sketch Containing in feature Hash codes, first, sketch is divided into two in the horizontal and vertical directions respectively, obtains four spaces of separation In position, such as Fig. 4 shown in flow (a).Then, method as described above is pointed to the candidate window on each locus Hash coding is carried out respectively, shown in flow (b)~(e) in whole process such as Fig. 4.Finally, by four locus Hash codes head and the tail splice successively, such as shown in flow (f), you can obtain representing the feature Hash codes of whole sketch.It is given to appoint Two contour images of sketch, picture or 3D model projections of meaning, the Hamming distance (Hamming between their feature Hash codes Distance) it is exactly its similarity.

Following introduction is of the invention to implement details, including before retrieval to picture and the pre- place of 3D models Reason process, the specific setting of parameter, characteristic index scheme and whole query process.

Sketch is typically a kind of black white image comprising lines of outline, and picture has usually contained abundant color and each The texture of formula various kinds, 3D models are then a kind of dough sheet set in three dimensions, it is clear that the feature of these three data is completely not Together, it is impossible to directly they are compared, are retrieved.Therefore, between retrieving starts, in order to allow user to make with sketch Come retrieving image and 3D models, it is necessary to be pre-processed to this two category information to be input into, convert them to the wheel of class sketch Wide image.

For given one secondary picture, picture in its entirety is calculated first by Canny or other image edge extraction algorithms Profile diagram E_c, but it is clear that the contour images inevitably contain many due to the false edges produced by background texture Lines.In order to find the body matter (namely the part being retrieved desired by the picture) of picture expression, it is necessary to use picture Be marked for the contents of the section by conspicuousness detection algorithm, and its corresponding notable figure is designated as S.Then, this method is also used Maximum filter (Maximum Filter, MF) has done a filtering process to S, this makes it possible to slightly expand picture Marking area, it is to avoid cause the outline of picture body matter to be lost due to the segmentation errors of algorithm.Finally, according to following public affairs Formula can just calculate the remarkable configuration figure E of the picture, and calculating process is as shown in Figure 5.

For the 3D model for giving, the present invention is according to Mathias Eitz et al. in paper " Sketch-Based Matching (View-based matching) algorithm based on visual angle proposed in Shape Retrieval " calculates model pair The contour projection charts answered.In calculating process, in order to ensure that the stability of result is extracted in projection, the present invention is relaxed using Lloyd Algorithm is that each 3D model is equably sampled out about 14 projection view angles from its corresponding encirclement sphere by loop iteration. Also, enlightening contour line (Suggestive Contours) is used to extract appropriate contour line from model projection figure Bar.

Afterwards, the picture remarkable configuration figure or projected outline's figure of 3D models for either being calculated according to the above method, all They are cut out come from baseline results and zoomed to the resolution ratio of 160*160 with a smallest square bounding box, with this To reduce the influence that picture size size and deformation are produced to retrieval result.Then, according to the calculating process of algorithm 1, will be input into The profile diagram of sketch, picture or model is divided into the grid of 80*80, and therefrom uniform sampling goes out 15*15 seed.It is special at each Levy in subwindow, calculate comprising 8 not normalized HOG feature histograms in direction as the h in constraints, thus each Characteristic vector contains 8 dimensions.All of characteristic window has all been scaled to the block of pixels of 16*16 afterwards, in order to connect down To carry out Visual Feature Retrieval Process.Finally, due to not normalized HOG feature histograms have been computed before, they are entered After row normalization, HOG features can just be used to describe the characteristic vector f of each characteristic window.

Give an arbitrary sketch as input, in measurement database the profile diagram of all sketches, picture or model and The Hamming distance of feature Hash codes between it, and ascending sequence, can just find most like with sketch grass in database Figure, picture or 3D models.Because all Hash codes are binaryzations, thus only by simple displacement and or computing just The Hamming distance between it can be very rapidly calculated, even if characteristic is not indexed can also have retrieval very high Speed.In order to further speed up retrieval performance, the multiple Index Algorithm (Fast of Hamming space Hash proposed according to Norouzi et al. Search in Hamming Space with Multi-Index Hashing) it is combined with the present invention, to characteristic According to being indexed, so allowing for this method can complete inquiry request each time within the sublinear time.Summarize as follows The flow of whole search method of the invention：

First, it is that all pictures in database or 3D models produce suitable profile diagram E；Secondly according to constraint Algorithms of Selecting is that all profile diagram E produce candidate feature window w；Again for each characteristic window w calculate corresponding HOG features f with And conspicuousness k；Then all HOG features f and conspicuousness k in profile diagram E are that each profile diagram E produces a feature to breathe out Uncommon code h；Finally for all of feature Hash codes h creates index I.

Second step according to retrieving is to the 4th step for input sketch S calculates its feature Hash codes h_s；From index I Calculate this Query Result R and return to user

In order to support and verifying research method and key technology proposed by the invention, in three marks being widely used On quasi- data set, this method Performance comparision is carried out into the searching algorithm based on sketch of other newest forefronts respectively. Magic Sketch data sets are set up by Liang et al., wherein contain 1100 sketches altogether, according to painted content it Be divided into 55 classifications respectively.These sketches are with MPEG-CE1 trademark image databases, the UK of trademark patent office of Britain PTO trademark databases, refrigerator electric elements figure and engineering drawing are reference, wherein representational shape are selected, by 10 People draws and obtains, and the data set is used to verify the validity of each part that this method proposes algorithm.TU Berlin Data set is constructed by Eitz et al., wherein containing 31 kinds of different themes, each theme is owned by an example Sketch and its corresponding 40 test pictures.The data set is for verifying that this method is used for the picture retrieval based on sketch Validity during task.PSB data sets are set up by Eitz et al., and each sketch that it is included is all readily identified, And corresponding to the class 3D models in Princeton 3 d model libraries (Princeton Shape Benchmark, PSB).Cause This, the validity when data set can be used to verifying that this method is used for the 3D model index tasks based on sketch.

In order to evaluate the performance of the retrieval technique based on sketch proposed by the invention, this method uses recall ratio (Recall), precision ratio (Precision) and data set baseline scores (Benchmark Score) are used as Performance Evaluating Indexes. In order to more intuitively evaluate existing method, (Precision-Recall curves, PR curves) provides recall ratio in graph form Or the relation between the size of precision ratio and candidate's graphic result collection.With increasing for returning result number, recall ratio will gradually Increase and precision ratio will then be gradually reduced, this is determined by the computational methods of precision ratio and recall ratio.When return structure number Window increase when, the denominator of precision ratio is bigger, and precision ratio is lower；And for recall ratio, window is bigger, the correlation of return The number of result is more, and the molecule of recall ratio is bigger, and the value of recall ratio is bigger.It is obvious that in recall ratio figure and looking into standard In rate figure, curve then retrieval effectiveness more high is better, because in the case where the candidate vector figure of same number is returned, curve is got over Gao Ze represents corresponding recall ratio or precision ratio is higher；Vice versa.Data set baseline scores are by Mathias Eitz et al. In paper " Sketch-Based Image Retrieval:Benchmark and Bag-of-Features Proposed in Descriptors ", be another index for evaluating data searching algorithm performance.The index is made by comparing After being input into the inquiry given under same retrieval data set, the legitimate reading of ranking result and artificial mark obtained by searching algorithm Between Kendall coefficient of rank correlations judge the quality of the searching algorithm.Baseline scores obtained by searching algorithm are higher, then table Cognitive result of the retrieval result of the bright algorithm closer to the mankind.In addition, the present invention is also tested for searching algorithm completion The time that primary retrieval task is spent, to assess the run time performance of proposed scheme.

In order to verify for selecting characteristic window two constraint C that this algorithm is proposed_app(formula 1) and C_var(formula 2) validity, first, according to Konstantinos Bozas in paper " Large Scale Sketch Based Image The algorithm proposed in Retrieval Using Patch Hashing " realizes one carries out spy using grid (Grid) is overlapped Levy the algorithm datum line of extraction.Then, its same only use is constrained into C_appAnd constraint C is used simultaneously_appAnd C_varThis algorithm enter Go and compared.In order to verify the characteristic window conspicuousness k that this algorithm is proposed_iValidity, algorithm more than realize in k_i It is arranged to 1, and they are same using all C_app、C_varAnd k_iRealization compare.Fig. 6 a show each as described above The retrieval performance that class algorithm is obtained on Magic Sketch data sets, therefrom it can be found that two Windows of this algorithm proposition Mouth selection constraint is complementary, also, removes C_app、C_varAnd k_iIn any one component can all cause the retrieval performance of this algorithm Decrease.Therefore, the C that this algorithm is proposed_app、C_varAnd k_iIt is indispensable, each part has its validity.

Additionally, the performance characteristics in order to further show this method, will use SIFT, GALIF and HOG feature to describe Son carry out feature extraction on uniform grid after searching algorithm realize respectively with using they selected by this algorithm take out spy Levy extract region on carry out feature extraction algorithm realize compare.The PR curves of Fig. 6 b show that above-mentioned algorithm is realized Comparative result on Magic Sketch data sets.It can be found that no matter which kind of Feature Descriptor used from figure, in this algorithm The result that the result of feature extraction is superior on uniform grid is carried out on selected feature extraction region, so as to demonstrate this Versatility of the method to different characteristic description.It is worth noting that, have benefited from GALIF Feature Descriptors used it is multi-direction, Multiple dimensioned feature sampling policy characterizes visual signature substituting traditional histogram strategy, thus it can be found that knot from figure Closing this method can obtain optimal retrieval performance with GALIF Feature Descriptors.But, due to calculating GALIF Feature Descriptors Huge time overhead can be brought, therefore, in order to ensure the availability of algorithm, in the standard implementation of this method, still use HOG Feature Descriptors calculate the visual signature of sketch.

It is last in experiment, this algorithm also has been entered to compare with the searching algorithm based on sketch of other forefronts.In TU On Berlin data sets, this algorithm is same to use bag of words (Bag-of-Words, BW), critical shape (Key Shapes, KW) Searching algorithm with min-hash (Min-hash, MH) is compared, and data set baseline scores are used to judge searching algorithm Performance, as a result as shown in table 1.Additionally, on Magic Sketch data sets, this algorithm is same to be based on having inclined SVM (BSVM), geometry The algorithm of spatial relationship (Spatial Relations, SR) and geometric space neighbouring (Spatial Proximity, SP) enters Go and compared；On PSB data sets, this algorithm is with based on diffusion tensor (Diffusion Tensor, DT), grid SIFT (SIFT-Grid) compared with the searching algorithm of GALIF (GALIF-Grid).On the two data sets, mark is used Accurate PR curves evaluate retrieval performance, as a result distinguish as shown in figs. 7 a and 7b.Experimental result above shows this method There is retrieval precision higher compared to other searching algorithms, performance is more preferably.Referring to Fig. 1, this method is respectively show not With the retrieval result example on data set.

The baseline scores of the algorithms of different of table 1

Searching algorithm based on sketch proposed by the invention about spends 1.87 on Magic Sketch standard data sets Second completes a Sketch Searching task.The result is to be equipped with the desk-top of four core CPU, 16GB internal memories of Intel 3.39GHz It is actually measured on computer, realize that code uses MATLAB to program and without parallel optimization.It is worth noting that, this method On the premise of remaining able to meet basic requirement of real-time, there is retrieval precision higher compared to other algorithms.

It should be appreciated that application of the invention is not limited to above-mentioned citing, and for those of ordinary skills, can To be improved according to the above description or converted, all these modifications and variations should all belong to the guarantor of appended claims of the present invention Shield scope.

Claims (9)

1. it is a kind of based on content-adaptive Hash coding Sketch Searching method, it is characterised in that including step in detail below：

Step A：Constrained based on appearance constraints and diversity, sketch or profile diagram to being retrieved are selected according to its content-adaptive Candidate window is taken for feature extraction, realizes that the information that whole image is included is evenly distributed in each window；

Step B：Conspicuousness detection method based on key point detects the conspicuousness of characteristic window；

Step C：Hash algorithm based on local sensitivity is empty by the local visual feature of sketch or profile diagram, conspicuousness and structure Between feature combine, be compiled into feature Hash codes；

Step D：The feature Hash codes of sketch or profile diagram are indexed, by calculating the Hamming distance between feature Hash codes To measure the similarity between sketch, and similarity result high is returned into user,

Wherein, in the step B, the specific method of the conspicuousness of the conspicuousness detection method detection characteristic window based on key point For：First, using Harris-Laplace detectors as sketch conspicuousness extracting tool, for each characteristic window w_i, it is fixed Adopted its conspicuousness k_iFor：

Wherein, Number (S_i) represent in characteristic window w_iThe significant point that middle use Harris-Laplace detectors are extracted Number；Area(w_i) it is w_iNumber comprising pixel.

2. Sketch Searching method according to claim 1, it is characterised in that picture and 3D models can also be retrieved, Before being retrieved to picture and 3D models, it is pre-processed, convert them into contour line picture.

3. Sketch Searching method according to claim 2, it is characterised in that by picture and 3D model conversations into line drawing The method of picture is respectively：Jointing edge extraction algorithm and the conspicuousness detection algorithm of picture is wanted to calculate the picture for picture Remarkable configuration figure；For 3D models, the corresponding contour projection charts of model are calculated according to the matching algorithm based on visual angle.

4. Sketch Searching method according to claim 1, it is characterised in that the candidate window for feature extraction is chosen and calculated Method：First for the sketch installation of input initializes the grid of n*n；M*m initial seed of uniform sampling from grid again；Then it is Window calculation HOG features h in each grid_i；Finally calculate overall situation HOG features

5. the Sketch Searching method according to claim 1 or 3, it is characterised in that described appearance constraints are denoted as C_app, tool Body surface is shown as：C_app(h):=F_app(h)≥k_app×F_app(H)

Wherein,H represents the corresponding HOG histograms of oriented gradients characteristic vector of the window, and H is represented HOG characteristic vectors corresponding to whole input sketch, n represents the dimension of HOG characteristic vectors h, b_iRepresent the in characteristic vector h Value in i dimension, k_appIt is predefined coefficient, for controlling global parameter F_app(H) influence, F_appIt is appearance constraints Target equation, works as F_appWhen value obtained by calculating is higher, it represents that the visual signature information that this feature window is included is more.

6. Sketch Searching method according to claim 5, it is characterised in that described diversity constraint is denoted as C_var, specifically It is expressed as：C_var(h):=F_var(h)≤k_var×F_var(H)

Wherein,H represents that the corresponding HOG histograms of oriented gradients of the window is special Vector is levied, H represents the HOG characteristic vectors corresponding to whole input sketch, and n represents the dimension of HOG characteristic vectors h, b_iRepresent Value in i-th dimension of characteristic vector h, k_varIt is predefined coefficient, for controlling global parameter F_var(H) influence, F_var It is the target equation of diversity constraint, if F_varValue is relatively low and F_appIt is higher, then show every in window feature vector h The value of one dimension is higher.

7. Sketch Searching method according to claim 1, it is characterised in that the method for being compiled into feature Hash codes is：Make f_i It is from characteristic window w_iIn the characteristic vector that extracts be first vector by its binaryzationThen, it then follows the meter of similar hash algorithm Calculation process, it is corresponding according to each windowAnd k_iValue is calculated the feature Hash codes of the window；Then, sketch is existed respectively It is divided into two on both horizontally and vertically, obtains four locus of separation, is pointed to the candidate's window on each locus Mouth carries out Hash coding respectively；Finally, splice so as to be represented by the Hash codes on four locus successively head and the tail The feature Hash codes of whole sketch.

8. Sketch Searching method according to claim 4, it is characterised in that Feature Descriptor also can select Scale invariant Feature Descriptor or local linear Gabor characteristic description.

9. Sketch Searching method according to claim 4, it is characterised in that selected the HOG without normalized special Levy to describe the visual information that each subwindow is included.