CN106326288B

CN106326288B - Image search method and device

Info

Publication number: CN106326288B
Application number: CN201510375311.0A
Authority: CN
Inventors: 童志军; 陈宇; 安山; 孙佰贵; 张洪明
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2015-06-30
Filing date: 2015-06-30
Publication date: 2019-12-03
Anticipated expiration: 2035-06-30
Also published as: CN106326288A

Abstract

This application discloses a kind of image search method and devices.Wherein, described image searching method includes: the target interest region for obtaining image to be searched；The local feature vectors and deep learning feature vector in target interest region are extracted respectively；The local feature vectors, deep learning feature vector correspondence are utilized and preset local weighted index, predetermined depth Weighted Index executes Feature Dimension Reduction respectively and handles, and Fusion Features are carried out to local feature vectors after dimensionality reduction and dimensionality reduction deep learning feature vector using default splicing Weighted Index, it obtains and improves the target feature vector that the target interest provincial characteristics describes precision；It is scanned for according to the target feature vector, obtains the search result based on the image to be searched.

Description

Image search method and device

Technical field

The application belongs to picture search field, specifically, being related to a kind of image search method and device.

Background technique

With the development of science and technology, image is in e-commerce, information propagate play the role of it is great.Since image can be given The impression of people's " What You See Is What You Get ", the mode that user obtains merchandise news are changed by the original search based on text based on figure The search of picture.In general, picture search be divided into again similarity and with money search (so-called same money search, refer to search with The identical commodity image of style in image to be searched), the development of deep learning feature vector so that image feature descriptive power It improves, image similarity has reached the requirement of user, but existing system in same money commercial articles searching and is not so good as people Meaning, needs user to carry out manually deleting choosing in the result that image search engine returns, the same money recall rate in search result is not high. So-called recall rate refers to the ratio of associated picture number all in the associated picture number searched out and image library.

Raising in existing search technique is generally divided into two classes with the method for money recall rate, and one is be based on various features simultaneously The combined searching method of row, another kind is the searching method of the feature serial combination based on bis- minor sort of ReRank, so-called ReRank refers to reordering technique, i.e., the technology of two minor sorts is carried out on the basis of the result of first time search.Below to two class sides Method illustrates respectively.

The searching method of various features the parallel combined, usually by color, texture, gradient (such as SIFT, HOG, LBP, Gabor) local feature descriptions' and global characteristics description such as, normalize respectively, be then stitched together as a feature to Amount.PCA (Principle Component Analysis Principal Component Analysis Algorithm) dimensionality reduction is used to spliced feature vector The feature vector expression for learning to the end.Various features direct splicing and the method for dimensionality reduction in various features the parallel combined, will The feature (such as local feature and global characteristics) of different dimensions is stitched together, the feature after PCA dimensionality reduction is combined, though The descriptive power of each feature vector is so combined, but the weight defaulted between the feature vectors of various dimensions is equal, does not have Reach the optimal expression ability of assemblage characteristic vector.Therefore, the same money that the searching method based on various features the parallel combined obtains Recall rate is unsatisfactory, also needs artificial screening.

The searching method (herein referred to as " bis- minor sort of ReRank ") of feature serial combination based on bis- minor sort of ReRank, Search for the first time using deep learning DCNN (Deep Convolutional Neural Networks) feature vector first Rope sequence, then in the subset of search result, uses local feature vectors or the deep learning feature vector of other attributes Two minor sorts are carried out, it is same to can effectively improve search by ReRank in the case where first time search recall rate is high for this method The Top10 hit rate of money.But bis- minor sort of ReRank extremely relies on the recall rate of search for the first time, if first time search subset In do not include commodity with money, the subsequent search result based on bis- minor sort of ReRank do not include the commodity of same money equally.Therefore work as In the case where lacking in first search subset with money recall rate, bis- minor sort of ReRank is then possible to fail.Therefore based on this search The same money recall rate of method has unstability, and ideal search result is equally not achieved.

Summary of the invention

In view of this, the technical problem to be solved by the application is to provide a kind of image search method and devices.

In order to solve the above-mentioned technical problem, this application discloses a kind of image search methods, comprising:

Obtain the target interest region of image to be searched；

The local feature vectors and deep learning feature vector in target interest region are extracted respectively；

Local weighted index, predetermined depth are preset to the local feature vectors, corresponding utilize of deep learning feature vector Weighted Index executes Feature Dimension Reduction processing respectively, and using default splicing Weighted Index to local feature vectors after dimensionality reduction and dimensionality reduction Deep learning feature vector carries out Fusion Features, obtain improve the target interest provincial characteristics describe the target signature of precision to Amount；

It is scanned for according to the target feature vector, obtains the search result based on the image to be searched.

In order to solve the above-mentioned technical problem, disclosed herein as well is a kind of image search apparatus, comprising:

First obtains module, for obtaining the target interest region of image to be searched；

Extraction module, for extract respectively target interest region local feature vectors and deep learning feature to Amount；

Dimensionality reduction Fusion Module, for corresponding using default local to the local feature vectors, deep learning feature vector After Weighted Index, predetermined depth Weighted Index execute Feature Dimension Reduction processing respectively, and the default splicing Weighted Index of utilization is to dimensionality reduction Local feature vectors and dimensionality reduction deep learning feature vector carry out Fusion Features, obtain the raising target interest provincial characteristics and retouch State the target feature vector of precision；

Search module obtains searching based on the image to be searched for scanning for according to the target feature vector Hitch fruit.

Compared with prior art, the application can be obtained including following technical effect:

1) the embodiment of the present application, which passes through, treats the local feature vectors in target interest region in search image, deep learning spy Sign vector it is corresponding using preset local weighted index, predetermined depth Weighted Index executes Feature Dimension Reduction processing respectively, and using pre- If splicing Weighted Index carries out Fusion Features to local feature vectors after dimensionality reduction and dimensionality reduction deep learning feature vector, realize pair Different characteristic vector carries out individual features dimensionality reduction or fusion treatment using different Weighted Indexes, enables to different dimensions feature The composite behaviour of vector (local feature vectors and deep learning feature vector) is optimal, and improves figure to be searched when commercial articles searching The feature descriptive power in the target interest region of picture, so that position is forward in the search result criticized back with money commodity, and The position of similar commodity rearward, improves the precision and recall rate of same money commercial articles searching.Simultaneously compared to various features direct splicing The method of dimensionality reduction, the target feature vector that image search method provided by the embodiments of the present application finally obtains treat search image mesh The characterisation accuracy for marking interest region is thinner higher, and the same money recall rate of search result is higher.

2) precision is described due to improving target interest provincial characteristics, therefore the embodiment of the present application effectively improves same money and calls together The rate of returning, then if reducing there will be effect and even keeping away using the same money recall rate of the embodiment of the present application as first time search result The problem of exempting from the failure of bis- minor sort of ReRank, improves the same money recall rate based on bis- minor sort of ReRank.

Certainly, any product for implementing the application must be not necessarily required to reach all the above technical effect simultaneously.

Detailed description of the invention

The drawings described herein are used to provide a further understanding of the present application, constitutes part of this application, this Shen Illustrative embodiments and their description please are not constituted an undue limitation on the present application for explaining the application.In the accompanying drawings:

Fig. 1 is a kind of image search method flow diagram of the embodiment of the present application；

Fig. 2 is another image search method flow diagram of the embodiment of the present application；

Fig. 3 is the generation method flow diagram of Weighted Index in a kind of image search method of the embodiment of the present application；

Fig. 4 is the generation method flow diagram of Weighted Index in another image search method of the embodiment of the present application；

Fig. 5 is the generation method flow diagram of Weighted Index in another image search method of the embodiment of the present application；

Fig. 6 is the depth convolutional neural networks configuration schematic diagram of the embodiment of the present application；

Fig. 7 is a kind of image search apparatus modular structure schematic diagram of the embodiment of the present application；

Fig. 8 is that a kind of Weighted Index of image search apparatus of the embodiment of the present application generates the signal of sub-device modular structure Figure.

Specific embodiment

Presently filed embodiment is described in detail below in conjunction with accompanying drawings and embodiments, how the application is applied whereby Technological means solves technical problem and reaches the realization process of technical effect to fully understand and implement.

It is described further below with implementation method of the first embodiment to the application.Referring to Fig. 1, the present embodiment provides A kind of image search method flow diagram, this method comprises:

Step 100, the commodity body of image to be searched is obtained.It should be noted that the figure to be searched of the embodiment of the present application As that can be any format, the electronic image of arbitrary size, these formats include but is not limited to JPG, PNG, TIF, BMP.It is described The acquisition modes of image to be searched can be to be directly downloaded on the net, is also possible to mobile phone or camera is taken pictures upload.The application is real The commodity body for applying example is the object-image section of commodity image, it can for the target interest identified from image to be searched Region.

Step 101, the local feature vectors and deep learning feature vector of the commodity body are extracted respectively.

Step 102, local weighted finger is preset to the local feature vectors, the corresponding utilization of deep learning feature vector Number, predetermined depth Weighted Index execute Feature Dimension Reduction processing respectively, and special to part after dimensionality reduction using default splicing Weighted Index It levies vector sum dimensionality reduction deep learning feature vector and carries out Fusion Features, obtain the mesh for improving the commodity body characterisation accuracy Mark feature vector.

Step 103, it is scanned for according to the target feature vector, obtains the search knot based on the image to be searched Fruit.

In order to preferably improve the precision and recall rate of same money commercial articles searching, need to improve the feature of commercial articles searching engine to Ability to express is measured, so that position is forward in the search result of return with money commodity, to improve the conclusion of the business conversion ratio of commodity.

The embodiment of the present application passes through local feature vectors, the deep learning feature vector for treating commodity body in search image It is corresponding using preset local weighted index, predetermined depth Weighted Index executes Feature Dimension Reduction processing respectively, and utilize default splicing Weighted Index carries out Fusion Features to local feature vectors after dimensionality reduction and dimensionality reduction deep learning feature vector, realizes to different spies It levies vector and individual features dimensionality reduction or fusion treatment is carried out using different Weighted Indexes, enable to different dimensions feature vector The composite behaviour of (local feature vectors and deep learning feature vector) is optimal, image to be searched when improving commercial articles searching The feature descriptive power of commodity body, so that position is forward in the search result criticized back with money commodity, and similar commodity Position rearward, improve the precision and recall rate of same money commercial articles searching.Compared to various features direct splicing and the side of dimensionality reduction Method, the target feature vector that image search method provided by the embodiments of the present application finally obtains treat search image commodity body Characterisation accuracy is thinner higher, and the same money recall rate of search result is higher.

In addition, due to improving commodity body characterisation accuracy, therefore the embodiment of the present application effectively improves same money and calls together The rate of returning, then if reducing there will be effect and even keeping away using the same money recall rate of the embodiment of the present application as first time search result The problem of exempting from the failure of bis- minor sort of ReRank, improves the same money recall rate based on bis- minor sort of ReRank.

It is described further below with implementation method of the second embodiment to the application.Referring to Fig. 2, the present embodiment provides A kind of image search method flow diagram, this method comprises:

Step 200, when receiving the image to be searched of input, the commodity body of the image to be searched is extracted.Specifically , the method for extracting the commodity body of image to be searched can be such as SLIC super-pixel segmentation, aobvious for commodity body dividing method The methods of the detection of work property, GrabCut；Or commodity body detection method (such as Adaboost iterative algorithm, R-CNN are deep Spend learning algorithm), the detection of commodity body is carried out by treating search image, to remove background image in image to be searched Interference, obtains the commodity body of image to be searched.The commodity body of the embodiment of the present application is the object-image section of commodity image, It can be with the target interest region to be identified from image to be searched.

It should be noted that the image to be searched of the embodiment of the present application can be any format, the electronic chart of arbitrary size Picture, these formats include but is not limited to JPG, PNG, TIF, BMP.The acquisition modes of the image to be searched can be online direct Downloading, is also possible to mobile phone or camera is taken pictures upload.

Step 201, the local feature vectors and deep learning feature vector of the commodity body are extracted respectively.

Specifically, the extraction of the local feature vectors of commodity body can be accomplished by the following way:

Sub-step 2011 extracts multiple Dense SIFT (dense Scale invariant features transform) feature of the commodity body Description；

Sub-step 2012 uses Fisher to each Feature Descriptor according to preset GMM mixed Gauss model Vector is encoded, and the local feature vectors of the commodity body are obtained.

Specifically, the extraction of the deep learning feature vector of commodity body can be accomplished by the following way: by the quotient Product main body inputs preset depth convolutional neural networks, obtains the deep learning feature vector of the commodity body.

Step 202, it is corresponding to the local feature vectors, deep learning feature vector using preset local weighted index, Predetermined depth Weighted Index executes Feature Dimension Reduction processing respectively, and using default splicing Weighted Index to local feature after dimensionality reduction to Amount and dimensionality reduction deep learning feature vector carry out Fusion Features, and the target for obtaining the raising commodity body characterisation accuracy is special Levy vector.Specifically, step 202 may include:

Sub-step 2021 carries out Feature Dimension Reduction processing using local feature vectors described in local weighted exponent pair are preset, obtains Local feature vectors after to dimensionality reduction.

Sub-step 2022 carries out at Feature Dimension Reduction the deep learning feature vector using predetermined depth Weighted Index Reason, obtains deep learning feature vector after dimensionality reduction.

Sub-step 2023 is spliced after the dimensionality reduction deep learning feature vector after local feature vectors and dimensionality reduction, and will be spelled The feature vector obtained after connecing is normalized, and obtains normalization characteristic vector.

Sub-step 2024 carries out Feature Dimension Reduction processing to the normalization characteristic vector using default splicing Weighted Index, Obtain target feature vector.

The embodiment of the present application treats local feature vectors, the depth of commodity body in search image before feature vector splicing Local weighted index is preset in the corresponding utilization of learning characteristic vector, predetermined depth Weighted Index executes Feature Dimension Reduction processing respectively, The feature descriptive power of local feature vectors, deep learning feature vector is improved, and using default after feature vector splicing Splice Weighted Index and Feature Dimension Reduction processing is carried out to the normalization characteristic vector, by local feature vectors, deep learning feature The feature description of vector has complementary advantages, removes crudely and store essence, so that target feature vector treats commodity body in search image Feature description is optimal.

It should be noted that row successively, can not synchronize execution, be also possible to son 2021,2022 sequence of sub-step Step 2022 is before sub-step 2021.

Step 203, it is scanned for according to the target feature vector, obtains the search knot based on the image to be searched Fruit.

It is worth noting that the embodiment of the present application is greater than the figure to be searched of 100*100 particularly with commodity body resolution ratio The search result of picture has preferably with money recall rate, this is because the resolution ratio of commodity body is bigger, the picture extracted is more The information of scale is bigger, and the local feature vectors of commodity body, deep learning feature vector be also in the image to be searched of acquisition Much, more accurate, the same money recall rate for finally obtaining search result is higher.But calculating and storage based on physical device system Ability, commodity body resolution ratio are greater than 100*100 and to be less than 2560*2560.In order to make the local feature vectors and depth that obtain It spends the precision of learning characteristic vector and the calculating of device systems and storage capacity reaches balance, actually use point of commodity body Resolution is the image of 256*256, which can obtain the local feature vectors for reaching enough accuracy and deep learning feature Vector also can be such that the calculating of device systems and storage capacity is optimal, to obtain highest same money recall rate.

For the default Weighted Index in aforementioned first and second embodiment, the below realization with 3rd embodiment to the application The generation method that Weighted Index is preset in method is described further.Refer to referring to Fig. 3, the embodiment of the present application provides a kind of weight Several generation methods, comprising:

Step 300, prepare training sample, training sample include Positive training sample to and negative training sample pair, step 300 tool Body can be realized by following steps:

Sub-step 3001 extracts multiple images to be retrieved in all commodity images in the default tranining database, And it obtains and corresponding search result is obtained according to each image to be retrieved；

Sub-step 3002 is ranked up each search result, searches for knot after obtaining sequence corresponding with image to be retrieved Fruit；

Image to be retrieved is formed Positive training sample with the top n result of search result after corresponding sequence by sub-step 3003 It is right, and image to be retrieved is formed into negative training sample pair with N number of result of remaining result in search result after corresponding sequence；Its In, N is positive integer.

Step 301, commodity image all in default tranining database is obtained, quotient in all commodity images is extracted The product features vector of product main body, and the matrix A of m × n is obtained according to the product features vector of extraction, wherein m represents commodity spy The dimension of vector is levied, n represents the number of training sample；

Step 302, matrix A is used at Principal Component Analysis Algorithm PCA (Principle Component Analysis) Reason, obtains the dimensionality reduction matrix B of l × m, wherein m > l, l are positive integer；

Step 303, matrix W is initialized using matrix B, and initial using the sampling feature vectors iteration optimization of training sample Matrix W ' after change obtains the default Weighted Index for carrying out dimensionality reduction and fusion to feature vector.Herein, initial using B Changing matrix W can be understood as using B as initialization matrix W, i.e., the value of matrix B is assigned to W, mathematical expression is as follows: W:=B, W '=W.Specifically, step 303 initialize in the following manner after matrix W ':

Sub-step 3031 initializes matrix W using matrix B, the matrix W ' after being initialized；

Sub-step 3032, using stochastic gradient descent algorithm SGD (Stochastic Gradient Descent) to weighting Formula is iterated optimization, with matrix W ' described in iteration optimization, obtains default Weighted Index；

Wherein, the weighted formula are as follows:

The y_ijFor the label of training sample pair, subscript i and j represent i-th of sample for forming training sample pair and j-th Sample；Work as y_ijWhen=1, Positive training sample pair is represented；Work as y_ijWhen=- 1, negative training sample pair is represented；B is to be learned positive and negative Training sample is to classification thresholds；φ_iWith φ_jConstitute a pair of sample feature vector of training sample pair to be entered；W is to be learned Weight matrix, dimension is m × n, and m is much smaller than n.

As can be seen from the above description, matrix B, W, W ' are essentially same matrix in sub-step 3031, and sub-step 3032 will Initial value of the W '=W as weighted formula uses stochastic gradient descent algorithm SGD (Stochastic Gradient Descent) iteration optimization weighted formula realizes the iteration optimization to matrix W (i.e. W ').

In the present embodiment, the product features vector can be special for commodity local feature vectors or commodity deep learning The commodity for levying vector or commodity local feature vectors and commodity deep learning feature vector splice feature vector.Correspondingly, step The sampling feature vectors of the 303 corresponding training samples utilized be sample local feature vectors or sample deep learning characteristic vector, Or the sample splicing feature vector of sample local feature vectors and sample deep learning characteristic vector, corresponding obtained default weighting Index is to preset local weighted index or predetermined depth Weighted Index, or preset splicing Weighted Index.That is: 1) working as step When the product features vector of rapid 301 commodity bodies extracted is commodity local feature vectors, step 303 utilizes the sample of training sample Matrix W described in this local feature vectors iteration optimization obtains local weighted for carrying out presetting for dimensionality reduction to local feature vectors Index；2) when the product features vector for the commodity body extracted when step 301 is commodity deep learning feature vector, step 303 Matrix W described in sample deep learning characteristic vector iteration optimization using training sample, obtain for deep learning feature to Amount carries out the predetermined depth Weighted Index of dimensionality reduction；3) the product features vector for the commodity body extracted when step 301 is commodity spelling When connecing feature vector, matrix W described in sample splicing feature vector iteration optimization of the step 303 using training sample is used for The default splicing Weighted Index of dimensionality reduction, fusion is carried out to splicing feature vector.

After the embodiment of the present application is by using weighted formula iteration initialization matrix W, obtained Weighted Index is enabled to Image feature vector with money commodity is less than b-1 by distance after weighting, and the commodity image feature of similar money or different money Distance is greater than b+1 after weighting, that is, while reducing inter- object distance, increases between class distance, so that in the top with money commodity.

It is described further below with implementation method of the fourth embodiment to the application.Due to the data volume one of training sample As in million M or more, and the execution equipment (such as PC machine) of Weighted Index generation method need to be by all training samples in operation Data volume be loaded into memory and can generate Weighted Index, if therefore the data volume of training sample is greater than the memory for executing equipment When can not then generate Weighted Index.In order to solve this problem, the embodiment of the present application provides a kind of generation method of Weighted Index, when When the data volume of the training sample is greater than preset data amount threshold value, batch processing is carried out to the training sample, obtains more batches Training subsample, the data volume of each batch of trained subsample is no more than preset data threshold value.Data volume=training of training sample The feature vector dimension of the number * training sample of sample.

The core concept of the embodiment of the present application is: each batch of trained subsample is successively carried out as currently batch training sample Generation method as described in 3rd embodiment executes the generation side that equipment only executes Weighted Index according to a collection of training sample every time Method, and using the Weighted Index obtained according to current batch of training sample as the initialization matrix of next group training sample, until root It is target Weighted Index according to the Weighted Index that last batch of training sample obtains --- default Weighted Index.The embodiment of the present application can To solve the problems, such as that Weighted Index can not be generated when the data volume of training sample is magnanimity.

Specifically, the generation method of default Weighted Index provided by the embodiments of the present application can specifically include:

A batch training subsample is chosen as first and trains subsample, and utilizes the sample of first training subsample Eigen vector iteration optimization initializes matrix, obtains the first Weighted Index；

A batch training subsample is chosen in residue batch training subsample as second batch training subsample, and utilizes second The first Weighted Index described in the sampling feature vectors iteration optimization of training subsample is criticized, the second Weighted Index is obtained；

Another batch of trained subsample is chosen as third batch in residue batch training subsample and trains subsample, and utilizes the Second Weighted Index described in the sampling feature vectors iteration optimization of two batches of trained subsamples；

And it repeats selection next group training subsample and iteration optimization in residue batch training subsample and accordingly adds The process of index is weighed, until described more batches trained subsamples are all iterated optimization, obtains default Weighted Index.

In order to more clearly explain the embodiment of the present application, it is assumed that obtain r batches of training subsamples, r is just whole greater than 1 Number, we use B_kIndicate current batch of training subsample, wherein k is positive integer and is not more than r.We can also be to r batches of training increments Originally it is ranked up, obtains with sequence { B₁, B_k..., B_r, k=1,2,3 ..., r } and the r crowd training subsamples that indicate, wherein { B_k, k =1,2,3 ... ..., r } indicate kth batch training subsample.Specifically as shown in figure 4, default weighting provided by the embodiments of the present application refers to Several generation methods may include:

Step 400, the step 300 of 3rd embodiment is seen.

Step 4011, determine the training sample data volume be greater than preset data amount threshold value, to the training sample into Row batch processing, obtains r batches of training subsamples, and r is the positive integer greater than 1.The data volume of each batch of trained subsample no more than Preset data threshold value.

A batch training subsample is chosen as first and trains subsample B₁, and utilize first training subsample B₁ Sampling feature vectors by step 402 to step 4062 iteration optimization initialize matrix W, obtain the first Weighted Index.Step 402,404 and step 4062 correspond to step 301,302 and 3032 of 3rd embodiment, details are not described herein.Implement with third Unlike example step 3031, step 4061 is identical as step 3031 in k=1, as k > 1, step 4061 specifically: By kth -1 crowd trained subsample B_k-1Weighted Index W_k-1'=W weighted input formula, and use stochastic gradient descent algorithm SGD (Stochastic Gradient Descent) is iterated optimization to weighted formula, with matrix W described in iteration optimization_k-1' is (i.e. Matrix of the initial value W '=W after k-1 iteration optimization), obtain kth batch training subsample B_kWeighted Index --- matrix W_k'.

When according to first training subsample B₁After obtaining the first Weighted Index, judge whether k++ is not more than < r, if so, Then follow the steps 4012；If not, then it is assumed that obtain target Weighted Index, as obtain default Weighted Index.

Step 4012, a batch training subsample is chosen in residue batch training subsample as second batch training subsample B₂, and utilize second batch training subsample B₂Sampling feature vectors iteration optimization described in the first Weighted Index, obtain the second weighting Index；

Another batch of trained subsample is chosen in residue batch training subsample as third batch training subsample B₃, and utilize Second batch trains subsample B₂Sampling feature vectors iteration optimization described in the second Weighted Index；

And it repeats and chooses next group training subsample B in residue batch training subsample_iAnd iteration optimization is corresponding The process of Weighted Index, until described more crowdes trained subsample { B₁... B_i... B_k, B_r, k=1,2,3 ..., r } all it is iterated Optimization obtains default Weighted Index.

The implementation method of the application is described further with the 5th embodiment below.In order to make final target signature to The characterisation accuracy that amount treats commodity body in search image is thinner higher, first embodiment to fourth embodiment extract to Searching for the dimension of feature vector of image commodity body, the higher the better, and the sample characteristics that while generating default Weighted Index uses to It is consistent with the feature vector dimension of commodity body in the image to be searched of extraction to measure dimension, therefore when generating default Weighted Index Also the higher the better for the feature vector dimension of the training sample of extraction.But by it is aforementioned it is known that training sample data volume= The feature vector dimension of the number * training sample of training sample, therefore when the feature vector dimension of training sample is up to tens of thousands of dimensions, Weighted Index can not be generated by still resulting in execution equipment, and to solve the above-mentioned problems, the embodiment of the present application also provides another The generation method of kind Weighted Index: right when the dimension of the sampling feature vectors of the training sample is greater than default dimension threshold value The dimension of the sampling feature vectors carries out segment processing, obtains multistage sample characteristics subvector, each section of sample characteristics subvector Dimension no more than default dimension threshold value.

The core concept of the embodiment of the present application is: carrying out as described in 3rd embodiment every section of sample characteristics subvector Generation method, corresponding obtained multiple default Weighted Indexes.The embodiment of the present application can solve when the sample of training sample is special Sign vector can not generate Weighted Index problem when being high dimensional feature, specifically, default dimension threshold value is a Wan Wei.It should be understood that It is that sampling feature vectors can splice feature vector for sample local feature vectors, sample depth feature vector, sample, then right The default Weighted Index that should be obtained is to preset local weighted index, predetermined depth Weighted Index, default splicing Weighted Index.

When carrying out picture search using obtained multiple default Weighted Indexes, step 102 and second in first embodiment Step 202 can carry out as follows in embodiment: to the feature vector of the commodity body of extraction (such as: local feature to Amount, deep learning feature vector) dimension carry out segment processing, obtain multistage feature subvector；Wherein, the feature of commodity body The number of segment of subvector is identical as the number of segment of sample characteristics subvector.Every section of feature subvector is multiplied and is corresponded to preset Weighted Index, Correspondence obtains feature subvector after multistage dimensionality reduction, then feature subvector after multistage dimensionality reduction is stitched together, and obtains feature after dimensionality reduction Vector.

In order to more clearly explain the embodiment of the present application, it is assumed that (it is special to be assumed to be sample part to sampling feature vectors Sign vector) dimension carry out segment processing and obtain t section sample characteristics subvectors, t is the positive integer greater than 1, we use S_xIt indicates Each section of sample characteristics subvector, wherein x is positive integer and is not more than t, is finally obtained with sequence { S₁, S_x..., S_t, x=1,2, 3 ..., t } indicate t section sample characteristics subvector, wherein { S_x, x=1,2,3 ... ..., t } indicate xth section sample characteristics to Amount.Specifically as shown in figure 5, the generation method of default Weighted Index provided by the embodiments of the present application may include:

Matrix is initialized using the multistage sample characteristics subvector difference iteration optimization of the training sample, it is multiple right to obtain That answers presets local weighted index { A₁, A_x... ..., A_t, x=1,2,3 ... ..., t }.

When using obtain it is multiple preset local weighted index and carry out picture search when, in first embodiment step 102 and The dimension of the feature vector (being assumed to be local feature vectors) for the commodity body that step 202 pair is extracted in second embodiment is divided Section processing, obtains t sections of local feature subvectors, we use T_xEach section of local feature subvector is indicated, with sequence { T₁, T_x..., T_t, x=1,2,3 ..., t } indicate t section local feature subvector.

It recycles and above-mentioned presets local weighted index { A_x, x=1,2,3 ... ..., t } and corresponding multiplied by { T_x, x=1,2,3 ..., T }, correspondence obtains local feature subvector { T after multistage dimensionality reduction_x’, x=1,2,3 ..., t }.

Splice local feature subvector after the multistage dimensionality reduction, obtains local feature vectors after dimensionality reduction.

It should be understood that when carrying out picture search, for depth after dimensionality reduction in first embodiment and second embodiment Feature vector, target feature vector can be obtained by the generating mode of local feature vectors after above-mentioned dimensionality reduction.That is:

1. the depth characteristic vector to commodity body carries out segment processing, multistage depth characteristic subvector is obtained；Wherein, quotient The number of segment of the feature subvector of product main body is identical as the number of segment of sample characteristics subvector.Every section of depth characteristic subvector is multiplied pair again Should be corresponding to obtain depth characteristic subvector after multistage dimensionality reduction with predetermined depth Weighted Index, then by depth characteristic after multistage dimensionality reduction Subvector is stitched together, and obtains depth characteristic vector after dimensionality reduction.

2. the normalization characteristic vector to commodity body carries out segment processing, multistage normalization characteristic subvector is obtained；Its In, the number of segment of the feature subvector of commodity body is identical as the number of segment of sample characteristics subvector.Again by every section of normalization characteristic Vector multiply it is corresponding with default splicing Weighted Index, it is corresponding to obtain normalization characteristic subvector after multistage dimensionality reduction, then by multistage dimensionality reduction Normalization characteristic subvector is stitched together afterwards, the target feature vector after obtaining dimensionality reduction.

It will be apparent to a skilled person that when the data volume of training sample is magnanimity, fourth embodiment and the Five embodiments can merge progress: i.e. can also be to current batch when operate described in fourth embodiment current batch of training subsample The sampling feature vectors of training subsample carry out operation described in the 5th embodiment；It can also be in the sample to every section of trained subsample When eigen vector operate described in the 5th embodiment can also behaviour described in fourth embodiment be carried out every section of trained subsample Make.The combination of the two, which can more be efficiently solved, can not generate weighting when the sampling feature vectors of training sample are high dimensional feature Index number problem.

It is described further below with implementation method of the sixth embodiment to the application.The embodiment of the present application provides a kind of figure As searching method, two processes are generally comprised: the generating process of default Weighted Index and the process of picture search.

One, the generating process of Weighted Index is preset

The process can substantially specifically include three steps as shown in 3rd embodiment:

1) preparation of training sample

The recall rate of the Top10 with money commodity is improved, is substantially to reduce query (image to be retrieved) and with money commodity The distance of pair, and increase at a distance from the query and commodity pair of similar money and different moneys.It is similar in order to complete fine granularity Inquiry learning needs to collect by the positive sample pair (i.e. Positive training sample to) of same money commodity composition and by similar money, different moneys The negative sample pair (i.e. negative training sample to) of commodity composition.

The collection of positive negative sample pair, the result that can be sorted based on DCNN deep learning feature according to Euclidean distance.Example If we can prepare the merchandising database of a 100W, 10W query is randomly selected, each query is arranged using Euclidean distance Sequence obtains 8192 search results；Each query is formed into positive sample pair with the Top20 in corresponding search result, it is random to take out 20 samples between each query and ranking [21,8192] are taken to form negative sample pair；A 200w's available in this way The training set of the negative sample pair of positive sample pair and 200w.

It should be understood that the embodiment of the present application can also be according to COS distance or European when specifically collecting sample The combination of distance and COS distance is ranked up.

The selection of positive sample pair, can also be by the master map and pair of each commodity in addition to can be by way of searching order Figure is used as positive sample pair, because the master map of commodity and width figure often describe the different perspectives or different face of the same commodity Color, style are just as；In addition it can be synthesized by variation patterns such as scale, translation, rotation, color, Gamma gamma corrections Sample obtains the composite diagram of each commodity, constitutes positive sample pair by commodity itself and composite diagram.

2) extraction of feature vector

The extraction of feature vector may include the sample local feature vectors of the commodity body of all images in training sample Extraction and sample deep learning characteristic vector extraction.

2.1) extraction of the commodity local feature vectors of commodity body

The extraction of commodity local feature vectors can be carried out with the following method for the commodity body of every image: will Input subject image normalizes to 300 by long side, and carries out dimensional variation by scale factor, generates the image gold word of 5 scales Tower, it is 128 that SIFT feature, which describes sub- dimension, and the patch size of characteristic vector pickup is 24X24, offset 1, full figure extraction Dense SIFT feature description.SIFT feature is described into sub- dimension using PCA and drops to 64, uses the first-order statistics of GMM model Amount and second-order statistic are as feature representation, and the Gauss model number of GMM is 512, and final characteristic dimension is 65536 (64*2* 512=65536).

It should be understood that the commodity local feature vectors of commodity body in addition to select Fisher Vector local feature, The features such as BOW, Sparse Coding, VLAD can also be selected, in addition parameter configuration can be according to practical problem in feature extraction Be adjusted, 2.1) in parameter it is for reference only, not uniquely.

2.2) extraction of the commodity deep learning feature vector of commodity body

Mentioning for commodity deep learning feature vector can be carried out with the following method for the commodity body of every image It takes:

2.2.1) DCNN network configuration and training

The configuration of depth convolutional neural networks is as shown in fig. 6, a total of 2 convolutional layers, and 5 pooling layers, 9 Inception layers, 3 full articulamentums and 3 softmax layers.Softmax1 and softmax2 is added primarily to preventing BP (Back Propagation) training gradient decaying, and the middle level features of the available commodity body of output of these layers are retouched It states, is the supplement to the corresponding high-level characteristic of softmax3.Training parameter weight is initialized using random number, initially LearningRate is set as 0.01, model can be allowed to restrain faster, when nicety of grading stablize when, turn down LearningRate after Continuous training, until model converges to a good value.The weight coefficient of depth convolutional neural networks is obtained i.e. after the completion of training For deep learning model, for extracting the commodity deep learning feature vector of commodity body.

2.2.2) DCNN feature extraction

Feature is extracted after network configuration is removed data input layer and softmax classifier layer, by three full convolutional layers Merging features rise as last commodity deep learning feature vector.

It should be noted that the extraction of commodity deep learning feature vector can select the deep learning mould in addition to DCNN Type, such as AutoEncoder, DBM.Model initialization can select existing disclosed model parameter in extraction process, or The initialization model parameter by the way of the Pretrain of layer wise, then stochastic gradient descent method is used on this basis Finetune (iteration optimization) model parameter.By these methods more accurate model parameter can be obtained with acceleration model training.

3) generation of Weighted Index

In order to reduce the distance between same money product features, and increase the distance between similar money and different moneys, needs Using in the generating process of aforementioned default Weighted Index 1) prepare positive negative sample pair feature vector using weighted formula into Row iteration optimization, exports W and b；By SGD (Stochastic Gradient Descent) iteration optimization W and b, corresponded to Default Weighted Index W '.After being iterated optimization by above-mentioned utilization weighted formula (i.e. distance study function), it can make Characteristics of image with money commodity is less than b-1 by distance after weighting, and the commodity image characteristic weighing of similar money or different money Distance is greater than b+1 afterwards.y_ijFor the label of sample pair, subscript i and j represent i-th of the sample and j-th of sample of composition training sample pair This；Work as y_ijWhen=1, Positive training sample pair is represented；Work as y_ijWhen=- 1, negative training sample pair is represented；B is positive and negative sample to be learned This is to classification thresholds, φ_iWith φ_jConstituting a pair of of feature vector of training sample pair to be entered, (herein, training sample is to can Think Positive training sample pair, or negative training sample to), W is weight matrix to be learned, and dimension is m × n, and m is far small In n, so that the dimension of the feature vector after weighting is much smaller than the dimension of primitive character, thus while lifting feature descriptive power Achieve the purpose that dimensionality reduction.Weighted formula is as follows:

It should be understood that metric learning function (i.e. aforementioned weighted formula) could alternatively be similar learning distance metric Algorithm, as positive sample pair is used only to study mahalanobis distance matrix, such as ITML by two multivariate Gaussian cores of optimization in MAHAL Relative entropy learns mahalanobis distance matrix, if KISSME learns distance matrix by two Gaussian Profile likelihood ratios of optimization, when When distance study function replaces with the optimizing expression in these methods, still belong to the range of the embodiment of the present application.

Specifically, the generation method of Weighted Index may include:

A the product features vector for) extracting default all commodity of tranining database, obtains the matrix A of a m × n, wherein m The dimension of product features vector is represented, n represents the number of training sample, and product features vector herein can be Fisher The splicing vector of Vector local feature vectors, DCNN deep learning feature vector or both feature；

B) learn the dimensionality reduction matrix B to l × m using PCA to matrix A, wherein m > l, l are positive integer, preferred l= 256；

C matrix W, the matrix W ' after being initialized) are initialized using B.It is used alternatingly positive sample pair'sWith it is negative Sample pair'sWeighted input formula iteration optimization W ', final output target Weighted Index W ＂ (i.e. target weighting matrix), Target Weighted Index is the default Weighted Index to be generated.Herein, it can be understood as making using B using B initialization matrix W To initialize matrix W, i.e., the value of matrix B is assigned to W, mathematical expression is as follows: W:=B, W '=W.As can be seen from the above description, square Battle array B, W, W ' are essentially same matrix, C) using W '=W as the initial value of weighted formula, use stochastic gradient descent algorithm SGD (Stochastic Gradient Descent) iteration optimization weighted formula realizes the iteration optimization to matrix W (i.e. W ').

It will be appreciated that model initialization can select random number, PCA dimensionality reduction matrix initialisation might not be used； If in addition positive sample pair or negative sample pair is used only in training sample, also the similarity weight matrix of available robust.

It is worth noting that: the product features vector in A) can be Fisher Vector commodity local feature vectors, The commodity of DCNN commodity deep learning feature vector or both feature splice feature vector, then C) in the corresponding training utilized The sampling feature vectors of sample be sample local feature vectors or sample deep learning characteristic vector or sample local feature to The sample of amount and sample deep learning characteristic vector splices feature vector；Then corresponding obtained default Weighted Index is default part Weighted Index or predetermined depth Weighted Index, or default splicing Weighted Index.

Since the training sample of fine granularity similarity-based learning is generally million or more, and in order to make final target signature Vector treats thinner higher, the spy of the image commodity body to be searched of extraction of characterisation accuracy of commodity body in search image The higher the better for the dimension of sign vector, and generates the to be searched of the sampling feature vectors dimension and extraction used when presetting Weighted Index The feature vector dimension of commodity body is consistent in image, therefore the feature of the training sample extracted when generating default Weighted Index Also the higher the better for vector dimension, and the feature vector dimension of usual training sample is up to tens of thousands of dimensions, and common PC machine memory can not expire Sufficient training requirement at this time can pre-process in batches training sample or be segmented to the sampling feature vectors of training sample pre- Processing.

It pre-processes in batches: training sample is packaged into batch one by one, batch training is divided to can solve Massive Sample Problem is loaded into memory only one batch, the weight matrix W that a preceding batch is generated every time_i-1(such as the 4th implementation First Weighted Index of example description), W is generated as batch next time_i(such as second Weighted Index of fourth embodiment description) Initial value so that algorithm training do not influenced by sample size.This process sees the description of aforementioned fourth embodiment.

Segmentation pretreatment:, can be to characteristic dimension segment processing, such as by a characteristic length when characteristic dimension is very high For the vector of m, it is divided into 5 sections, every segment length is m/5, and being iterated obtains corresponding weight respectively to every section of feature subvector Matrix W_j(such as default Weighted Index of the 5th embodiment description), wherein j=1,2,3,4,5, implement when in progress such as first In the image search method of example, second embodiment and sixth embodiment when the dimension-reduction treatment of commodity body feature vector, by 5 sections Feature subvector is multiplied by corresponding weight matrix W respectively_j(such as default Weighted Index of the 5th embodiment description), and splice As the feature vector of final expression characteristic.This process sees the description of aforementioned 5th embodiment.

When the default Weighted Index of above-mentioned generation is applied in picture search, commodity body in search image can be treated Local feature vectors, deep learning feature vector correspondence utilize and preset local weighted index, predetermined depth Weighted Index is held respectively The processing of row Feature Dimension Reduction, and using default splicing Weighted Index to local feature vectors after dimensionality reduction and dimensionality reduction deep learning feature to Amount carries out Fusion Features, realizes and is carried out at individual features dimensionality reduction or fusion to different characteristic vector using different Weighted Indexes Reason, enables to the composite behaviour of different dimensions feature vector (local feature vectors and deep learning feature vector) to reach most It is excellent, the feature representation ability characteristics descriptive power of the commodity body of image to be searched when improving commercial articles searching, so that same money Commodity position in the search result criticized back is forward, and the position of similar commodity is rearward, improves the precision of same money commercial articles searching And recall rate.

It should be understood that the generating process of above-mentioned default Weighted Index can be off-line training process, or Line training process.

Two, the process of picture search

The process substantially may include three steps:

1) extraction of feature vector

Obtain the commodity body of image to be searched；

The local feature vectors and deep learning feature vector of the commodity body are extracted respectively.

The extraction of local feature vectors and deep learning feature vector can refer to the aforementioned " generation of default Weighted Index 2 in journey "), no longer illustrate herein.

2) the dimensionality reduction fusion of feature vector

2.1) dimensionality reduction of feature vector

Local feature vectors, the deep learning feature vector for treating the commodity body of search image are corresponding using default local Weighted Index, predetermined depth Weighted Index execute Feature Dimension Reduction processing respectively.This process sees aforementioned first embodiment to The description of five embodiments, no longer illustrates herein.

2.2) splicing of feature vector

Splice after the dimensionality reduction deep learning feature vector after local feature vectors and dimensionality reduction, and the spy that will be obtained after splicing Sign vector is normalized, and obtains normalization characteristic vector.This process sees aforementioned first embodiment to the 5th implementation The description of example, no longer illustrates herein.

2.3) splice the dimensionality reduction of vector

Feature Dimension Reduction processing is carried out to the normalization characteristic vector using default splicing Weighted Index, obtains target signature Vector.This process sees the description of aforementioned first embodiment to the 5th embodiment, no longer illustrates herein.

3) picture search

It is scanned for according to the target feature vector, obtains the search result based on the image to be searched.This process The description of aforementioned first embodiment to the 5th embodiment is seen, is no longer illustrated herein.

The implementation of the application is described further with the 7th embodiment below.The embodiment of the present application provides a kind of figure As searcher, comprising:

First obtains module 701, for obtaining the commodity body of image to be searched；

Extraction module 702, for extracting the local feature vectors and deep learning feature vector of the commodity body respectively；

Dimensionality reduction Fusion Module 703, for setting a trap in advance to the local feature vectors, corresponding utilize of deep learning feature vector Portion's Weighted Index, predetermined depth Weighted Index execute Feature Dimension Reduction processing respectively, and using default splicing Weighted Index to dimensionality reduction Local feature vectors and dimensionality reduction deep learning feature vector carry out Fusion Features afterwards, obtain and improve the commodity body feature description The target feature vector of precision；

Search module 704 is obtained for being scanned for according to the target feature vector based on the image to be searched Search result.

Further, the dimensionality reduction Fusion Module includes:

First partial dimensionality reduction unit, for using preset local feature vectors described in local weighted exponent pair carry out feature drop Dimension processing, obtains local feature vectors after dimensionality reduction；

First depth dimensionality reduction unit, it is special for being carried out using predetermined depth Weighted Index to the deep learning feature vector Dimension-reduction treatment is levied, deep learning feature vector after dimensionality reduction is obtained；

First concatenation unit, for splicing after the dimensionality reduction deep learning feature vector after local feature vectors and dimensionality reduction, And the feature vector obtained after splicing is normalized, obtain normalization characteristic vector；

First splicing dimensionality reduction unit, for carrying out feature to the normalization characteristic vector using default splicing Weighted Index Dimension-reduction treatment obtains target feature vector.

Further, the resolution ratio of the commodity body is greater than 100*100.

Further, the resolution ratio of the commodity body is 256*256.

Further, the extraction module includes the local shape factor for extracting the local feature vectors of commodity body Unit；

The local shape factor unit includes:

Subelement is extracted, for extracting multiple Feature Descriptors of the commodity body；

Coded sub-units, for using Fisher to each Feature Descriptor according to preset GMM mixed Gauss model Vector is encoded, and the local feature vectors of the commodity body are obtained.

Further, the extraction module includes deep learning feature extraction unit: for inputting the commodity body Preset depth convolutional neural networks obtain the deep learning feature vector of the commodity body.

Further, described first module is obtained, is specifically used for detecting image to be searched, removes in the image to be searched The interference of background image obtains the commodity body of the image to be searched.

Present apparatus embodiment is corresponded to each other with the feature in above-mentioned first, second embodiment, therefore can be found in first, second The associated description of method flow part in embodiment, details are not described herein.

The implementation of the application is described further with the 8th embodiment below.The embodiment of the present application provides a kind of figure As searcher, the present embodiment has the 7th embodiment roughly the same, is to further include that Weighted Index generates sub-device except different, Default Weighted Index is generated for being iterated optimization using the feature vector of training sample in default tranining database, wherein The default Weighted Index includes presetting local weighted index, predetermined depth Weighted Index, default splicing Weighted Index.Specifically , Weighted Index generates sub-device and includes:

Second obtains module 801, for obtaining commodity image all in default tranining database, extracts described all The product features vector of commodity body in commodity image, and the matrix A of m × n is obtained according to the product features vector of extraction, wherein M represents the dimension of product features vector, and n represents the number of training sample；

Dimensionality reduction module 802 obtains the dimensionality reduction matrix B of l × n for being handled using Principal Component Analysis Algorithm matrix A, In, m > l, l are positive integer；

Iteration module 803 for using matrix B as initialization matrix W, and utilizes the sampling feature vectors of training sample Matrix W described in iteration optimization obtains the default Weighted Index for carrying out dimensionality reduction and fusion to feature vector.

Further, described second the product features vector that module obtains is obtained as commodity local feature vectors or commodity The commodity of deep learning feature vector or commodity local feature vectors and commodity deep learning feature vector splice feature vector；

Then the sampling feature vectors of the corresponding training sample utilized of the second acquisition module be sample local feature vectors, Or the sample splicing feature of sample deep learning characteristic vector or sample local feature vectors and sample deep learning characteristic vector Vector；

Then the corresponding obtained default Weighted Index of the iteration module is to preset local weighted index or predetermined depth weighting Index, or default splicing Weighted Index.

Further, the training sample include Positive training sample to and negative training sample pair, the generating means also wrap Training sample generation module is included, the training sample generation module includes:

Extracting unit, for extracting multiple figures to be retrieved in all commodity images in the default tranining database Picture, and obtain and corresponding search result is obtained according to each image to be retrieved；

Sequencing unit is searched for after obtaining sequence corresponding with image to be retrieved for being ranked up to each search result As a result；

Generation unit, for the top n result composition of search result after image to be retrieved and corresponding sequence just to be trained sample This is right, and image to be retrieved is formed negative training sample pair with N number of result of remaining result in search result after corresponding sequence；Its In, N is positive integer.

Further, the iteration module includes:

Initialization unit, for initializing matrix W, the matrix W ' after being initialized using matrix B；

Iterative optimization unit, it is excellent with iteration for being iterated optimization to weighted formula using stochastic gradient descent algorithm Change the matrix W ', obtains default Weighted Index；

Wherein, the weighted formula are as follows:

The y_ijFor the label of training sample, Positive training sample is to being 1, and negative training sample is to being -1；B be it is to be learned just Negative training sample is to classification thresholds；φ_iWith φ_jConstitute a pair of sample feature of training sample to be entered；W is weight to be learned Matrix, dimension is m × n, and m is much smaller than n.

Further, it further includes module in batches that the Weighted Index, which generates sub-device, for working as the number of the training sample When being greater than preset data amount threshold value according to amount, batch processing is carried out to the training sample, obtains more batches of trained subsamples；Then

The iteration module is specifically used for:

Another batch of trained subsample is chosen as third batch in residue batch training subsample and trains subsample, and utilizes the Second Weighted Index described in the feature vector iteration optimization of two batches of trained subsamples；

Further, it further includes segmentation module that the Weighted Index, which generates sub-device, for working as the sample of the training sample When the dimension of eigen vector is greater than default dimension threshold value, segment processing is carried out to the dimension of the sampling feature vectors, is obtained Multistage feature subvector；Then

The iteration module is specifically used for:

Matrix is initialized using the multistage feature subvector difference iteration optimization of the training sample, correspondence obtains multiple pre- If Weighted Index.Specifically, the dimensionality reduction Fusion Module includes:

Second local dimensionality reduction unit obtains multistage local feature for carrying out segment processing to the local feature vectors Subvector, wherein the number of segment of local feature subvector is identical as the number of segment of sample characteristics subvector；By every section of local feature Subvector, which multiplies, to be corresponded to preset local weighted index, and correspondence obtains local feature subvector after multistage dimensionality reduction；Splice the multistage Feature subvector gets up after dimensionality reduction, obtains local feature vectors after dimensionality reduction；

Second depth dimensionality reduction unit obtains multistage depth characteristic for carrying out segment processing to the depth characteristic vector Subvector, wherein the number of segment of depth characteristic subvector is identical as the number of segment of sample characteristics subvector；By every section of depth characteristic Subvector, which multiplies, to be corresponded to predetermined depth Weighted Index, and correspondence obtains depth characteristic subvector after multistage dimensionality reduction；Splice the multistage Feature subvector gets up after dimensionality reduction, obtains depth characteristic vector after dimensionality reduction；

Second concatenation unit, for splicing after the dimensionality reduction deep learning feature vector after local feature vectors and dimensionality reduction, And the feature vector obtained after splicing is normalized, obtain normalization characteristic vector；

Second splicing dimensionality reduction unit obtains multistage normalization for carrying out segment processing to the normalization characteristic vector Feature subvector, wherein the number of segment of normalization characteristic subvector is identical as the number of segment of sample characteristics subvector；It will return described in every section One change feature subvector, which multiplies, to be corresponded to default splicing Weighted Index, and correspondence obtains normalization characteristic subvector after multistage dimensionality reduction；It spells Feature subvector after the multistage dimensionality reduction is connect, normalization characteristic vector after dimensionality reduction is obtained.

Present apparatus embodiment is corresponded to each other with the feature in above-mentioned third, the four, the 5th, sixth embodiment, therefore can be joined See third, the four, the 5th, in sixth embodiment method flow part associated description, details are not described herein.

The embodiment of the present application solves in same money commercial articles searching that hit rate is low in TopN (such as Top10) with money picture Problem.Either based on traditional SIFT (Scale-invariant feature transform, scale invariant feature conversion) Etc. image search method of the local feature vectors still based on deep learning feature vector, although can guarantee the phase of search result Like property, but picture in the top is frequently not the same money commodity that user wants.The embodiment of the present application by product features to Amount (local feature vectors or deep learning feature vector) be weighted using corresponding Weighted Index, improve product features to The feature descriptive power of amount while reducing inter- object distance, increases between class distance, so that in the top with money commodity；And And characteristic dimension is reduced in above process, it reduces feature vector memory space and search calculates the time.

The embodiment of the present application by provide picture search sequence in feature combining weights study (i.e. by Weighted Index into The fusion of row dimensionality reduction) method, effectively in conjunction with the feature of different dimensions and reduce the dimension of feature, solve same money in searching order The low problem of recall rate, and reduce feature committed memory size and characteristic distance in search Project Realization and calculate the time.This Application embodiment does not depend on any Preprocessing Technique and empirical parameter, so having versatility for commercial articles searching field And robustness.

Those skilled in the art can select Multiple Kernel Learning it is to be noted that it is well known that in classification problem (Multi-kernel learning) selects different kernel functions to different feature vectors, and the weight of each core of training is selected Best kernel function combines classification.Although the combination of eigenvectors based on Multiple Kernel Learning, can with each feature of dynamic learning to The kernel function of amount reaches the optimal of combination of eigenvectors, but it is fundamentally based on classification problem, can not be in searching order problem Middle application.Therefore the mode of aforementioned Multiple Kernel Learning can not give the application with technical inspiration.

In conclusion the embodiment of the present application it is available following the utility model has the advantages that

1) the embodiment of the present application passes through commodity body local feature vectors, the deep learning feature vector for treating search image It is corresponding using preset local weighted index, predetermined depth Weighted Index executes Feature Dimension Reduction processing respectively, and utilize default splicing Weighted Index carries out Fusion Features to local feature vectors after dimensionality reduction and dimensionality reduction deep learning feature vector, realizes to different spies It levies vector and individual features dimensionality reduction or fusion treatment is carried out using different Weighted Indexes, enable to different dimensions feature vector The composite behaviour of (local feature vectors and deep learning feature vector) is optimal, image to be searched when improving commercial articles searching The feature representation ability characteristics descriptive power of commodity body, so that being leaned on position in the search result criticized back with money commodity Before, and the position of similar commodity is rearward, improves the precision and recall rate of same money commercial articles searching.It is directly spelled compared to various features It connects and the method for dimensionality reduction, the target feature vector that image search method provided by the embodiments of the present application finally obtains treats search graph As the characterisation accuracy of commodity body is thinner higher, the same money recall rate of search result is higher.

2) due to improving commodity body characterisation accuracy, therefore the embodiment of the present application effectively improves same money and recalls Rate, then if reducing there will be effect and even avoiding using the same money recall rate of the embodiment of the present application as first time search result The problem of bis- minor sort of ReRank fails, improves the same money recall rate based on bis- minor sort of ReRank.

3) the embodiment of the present application is also excessive in training sample amount, when the feature vector dimension of training sample is very high, by right Training sample fragment, the way of the feature vector dimension of training sample efficiently solve mass data, and high dimensional feature can not give birth to At Weighted Index, and then the problem of same money recall rate can not be improved.

4) traditional classification study algorithm (such as multicore feature learning) can not be applied in searching order, it is different from the past with Learning algorithm (such as multicore feature learning) for the purpose of classification, the embodiment of the present application add different characteristic vector using different It weighs index and carries out individual features dimensionality reduction or fusion treatment, enable to different dimensions feature vector (local feature vectors and depth Learning characteristic vector) composite behaviour be optimal, the feature representation of the commodity body of image to be searched when improving commercial articles searching Ability characteristics descriptive power, so that position is forward in the search result criticized back with money commodity, and the position of similar commodity Rearward, the precision and recall rate of same money commercial articles searching are improved.

In a typical configuration, calculating equipment includes one or more processors (CPU), input/output interface, net Network interface and memory.

Memory may include the non-volatile memory in computer-readable medium, random access memory (RAM) and/or The forms such as Nonvolatile memory, such as read-only memory (ROM) or flash memory (flash RAM).Memory is computer-readable medium Example.

Computer-readable medium includes permanent and non-permanent, removable and non-removable media can be by any method Or technology come realize information store.Information can be computer readable instructions, data structure, the module of program or other data. The example of the storage medium of computer includes, but are not limited to phase change memory (PRAM), static random access memory (SRAM), moves State random access memory (DRAM), other kinds of random access memory (RAM), read-only memory (ROM), electric erasable Programmable read only memory (EEPROM), flash memory or other memory techniques, read-only disc read only memory (CD-ROM) (CD-ROM), Digital versatile disc (DVD) or other optical storage, magnetic cassettes, tape magnetic disk storage or other magnetic storage devices Or any other non-transmission medium, can be used for storage can be accessed by a computing device information.As defined in this article, it calculates Machine readable medium does not include non-temporary computer readable media (transitory media), such as the data-signal and carrier wave of modulation.

As used some vocabulary to censure specific components in the specification and claims.Those skilled in the art answer It is understood that hardware manufacturer may call the same component with different nouns.This specification and claims are not with name The difference of title is as the mode for distinguishing component, but with the difference of component functionally as the criterion of differentiation.Such as logical The "comprising" of piece specification and claim mentioned in is an open language, therefore should be construed to " include but do not limit In "." substantially " refer within the acceptable error range, those skilled in the art can within a certain error range solve described in Technical problem basically reaches the technical effect.In addition, " coupling " word includes any direct and indirect electric property coupling herein Means.Therefore, if it is described herein that a first device is coupled to a second device, then representing the first device can directly electrical coupling It is connected to the second device, or the second device indirectly electrically coupled through other devices or coupling means.Specification Subsequent descriptions be implement the application better embodiment, so it is described description be for the purpose of the rule for illustrating the application, It is not intended to limit the scope of the present application.The protection scope of the application is as defined by the appended claims.

It should also be noted that, the terms "include", "comprise" or its any other variant are intended to nonexcludability Include, so that commodity or system including a series of elements not only include those elements, but also including not clear The other element listed, or further include for this commodity or the intrinsic element of system.In the feelings not limited more Under condition, the element that is limited by sentence "including a ...", it is not excluded that in the commodity or system for including the element also There are other identical elements.

Several preferred embodiments of the present invention have shown and described in above description, but as previously described, it should be understood that this Utility model is not limited to forms disclosed herein, and should not be regarded as an exclusion of other examples, and can be used for various Other combinations, modification and environment, and above-mentioned introduction or related fields can be passed through within the scope of the inventive concept described herein Technology or knowledge be modified.And changes and modifications made by those skilled in the art do not depart from the spirit and model of the utility model It encloses, then it all should be in the protection scope of the appended claims for the utility model.

Claims

1. a kind of image search method characterized by comprising

Obtain the target interest region of image to be searched；

Local weighted index, predetermined depth weighting are preset to the local feature vectors, corresponding utilize of deep learning feature vector Index executes Feature Dimension Reduction processing respectively, and using default splicing Weighted Index to local feature vectors after dimensionality reduction and dimensionality reduction depth Learning characteristic vector carries out Fusion Features, obtains and improves the target feature vector that the target interest provincial characteristics describes precision；

2. image search method according to claim 1, which is characterized in that described to the local feature vectors, depth Learning characteristic vector correspondence utilizes and presets local weighted index, predetermined depth Weighted Index executes Feature Dimension Reduction processing respectively, and Fusion Features packet is carried out to local feature vectors after dimensionality reduction and dimensionality reduction deep learning feature vector using default splicing Weighted Index It includes:

Feature Dimension Reduction processing is carried out using local feature vectors described in local weighted exponent pair are preset, obtains local feature after dimensionality reduction Vector；

Feature Dimension Reduction processing is carried out to the deep learning feature vector using predetermined depth Weighted Index, obtains depth after dimensionality reduction Learning characteristic vector；

Splice after the dimensionality reduction deep learning feature vector after local feature vectors and dimensionality reduction, and by the feature obtained after splicing to Amount is normalized, and obtains normalization characteristic vector；

Using default splicing Weighted Index to the normalization characteristic vector carry out Feature Dimension Reduction processing, obtain target signature to Amount.

3. image search method according to claim 1, which is characterized in that further include: using in default tranining database The sampling feature vectors of training sample are iterated optimization and generate default Weighted Index, wherein the default Weighted Index includes Preset local weighted index, predetermined depth Weighted Index, default splicing Weighted Index.

4. image search method according to claim 3, which is characterized in that described to utilize training in default tranining database The sampling feature vectors of sample are iterated the default Weighted Index of optimization generation

Commodity image all in default tranining database is obtained, target interest region in all commodity images is extracted Product features vector, and the matrix A of m × n is obtained according to the product features vector of extraction, wherein m represents product features vector Dimension, n represent the number of training sample；

Matrix A is handled using Principal Component Analysis Algorithm, obtains the dimensionality reduction matrix B of l × m, wherein m > l, l are positive integer；

Use matrix B as initialization matrix W, and using matrix W described in the sampling feature vectors iteration optimization of training sample, obtains To the default Weighted Index for carrying out dimensionality reduction and fusion to feature vector.

5. image search method according to claim 4, which is characterized in that the product features vector is that commodity part is special Levy the commodity of vector or commodity deep learning feature vector or commodity local feature vectors and commodity deep learning feature vector Splice feature vector；

Then the corresponding sampling feature vectors utilized are sample local feature vectors or sample deep learning characteristic vector or sample The sample of local feature vectors and sample deep learning characteristic vector splices feature vector；

Then corresponding obtained default Weighted Index is to preset local weighted index or predetermined depth Weighted Index, or preset splicing Weighted Index.

6. image search method according to claim 4, which is characterized in that the training sample includes Positive training sample pair It is described to obtain before presetting commodity image all in tranining database with negative training sample pair further include:

Multiple images to be retrieved are extracted in all commodity images in the default tranining database, and are obtained according to each to be checked Rope image obtains corresponding search result；

Each search result is ranked up, search result after sequence corresponding with image to be retrieved is obtained；

The top n result of search result after image to be retrieved and corresponding sequence is formed into Positive training sample pair, and by figure to be retrieved As forming negative training sample pair with N number of result of remaining result in search result after corresponding sequence；Wherein, N is positive integer.

7. image search method according to claim 4, which is characterized in that when the data volume of the training sample is greater than in advance If when data-quantity threshold, carrying out batch processing to the training sample, obtaining more batches of trained subsamples；Described preset then is generated to add Weighing index includes:

A batch training subsample is chosen as first and trains subsample, and is special using the sample of first training subsample It levies vector iteration optimization and initializes matrix, obtain the first Weighted Index；

A batch training subsample is chosen in residue batch training subsample as second batch training subsample, and is instructed using second batch Practice the first Weighted Index described in the sampling feature vectors iteration optimization of subsample, obtains the second Weighted Index；

Another batch of trained subsample is chosen in residue batch training subsample as third batch training subsample, and utilizes second batch Second Weighted Index described in the feature vector iteration optimization of training subsample；

And it repeats selection next group training subsample and iteration optimization respective weight in residue batch training subsample and refers to Several processes obtains default Weighted Index until described more batches trained subsamples are all iterated optimization.

8. image search method according to claim 4, which is characterized in that when the sampling feature vectors of the training sample Dimension when being greater than default dimension threshold value, segment processing is carried out to the dimension of the sampling feature vectors, it is special to obtain multistage sample Levy subvector；Then generating the default Weighted Index includes:

Matrix is initialized using the multistage sample characteristics subvector difference iteration optimization of the training sample, correspondence obtains multiple pre- If Weighted Index.

9. image search method according to claim 8, which is characterized in that described to the local feature vectors, depth Learning characteristic vector correspondence utilizes and presets local weighted index, predetermined depth Weighted Index executes Feature Dimension Reduction processing respectively, and Fusion Features are carried out to local feature vectors after dimensionality reduction and dimensionality reduction deep learning feature vector using default splicing Weighted Index, are obtained The target feature vector that the target interest provincial characteristics describes precision must be improved, comprising:

Segment processing is carried out to the local feature vectors, obtains multistage local feature subvector, wherein local feature subvector Number of segment it is identical as the number of segment of sample characteristics subvector；It is local weighted to preset that every section of local feature subvector is multiplied into correspondence Index, correspondence obtain local feature subvector after multistage dimensionality reduction；Splice feature subvector after the multistage dimensionality reduction, is dropped Local feature vectors after dimension；

Segment processing is carried out to the deep learning feature vector, obtains multistage depth characteristic subvector, wherein depth characteristic The number of segment of vector is identical as the number of segment of sample characteristics subvector；Every section of depth characteristic subvector is multiplied corresponding with predetermined depth Weighted Index, correspondence obtain depth characteristic subvector after multistage dimensionality reduction；Splice feature subvector after the multistage dimensionality reduction, obtains Depth characteristic vector after to dimensionality reduction；

Segment processing is carried out to the normalization characteristic vector, obtains multistage normalization characteristic subvector, wherein normalization characteristic The number of segment of subvector is identical as the number of segment of sample characteristics subvector；Every section of normalization characteristic subvector is multiplied corresponding with default Splice Weighted Index, correspondence obtains normalization characteristic subvector after multistage dimensionality reduction；Splice feature subvector after the multistage dimensionality reduction Get up, obtains normalization characteristic vector after dimensionality reduction.

10. image search method according to claim 1, which is characterized in that the resolution ratio in target interest region is big In 100*100.

11. image search method according to claim 10, which is characterized in that the resolution ratio in target interest region is 256*256。

12. image search method according to claim 1, which is characterized in that the local feature in target interest region The extraction of vector includes:

Extract multiple Feature Descriptors in target interest region；

Each Feature Descriptor is encoded using Fisher Vector according to preset GMM mixed Gauss model, is obtained The local feature vectors in target interest region.

13. image search method according to claim 1, which is characterized in that the deep learning in target interest region The extraction of feature vector includes:

Target interest region is inputted into preset depth convolutional neural networks, obtains the depth in target interest region Practise feature vector.

14. a kind of image search apparatus characterized by comprising

Extraction module, for extracting the local feature vectors and deep learning feature vector in target interest region respectively；

Dimensionality reduction Fusion Module, for corresponding local weighted using presetting to the local feature vectors, deep learning feature vector Index, predetermined depth Weighted Index execute Feature Dimension Reduction processing respectively, and using default splicing Weighted Index to part after dimensionality reduction Feature vector and dimensionality reduction deep learning feature vector carry out Fusion Features, obtain and improve the target interest provincial characteristics description essence The target feature vector of degree；

Search module obtains the search knot based on the image to be searched for scanning for according to the target feature vector Fruit.

15. image search apparatus according to claim 14, which is characterized in that the dimensionality reduction Fusion Module includes:

First partial dimensionality reduction unit, for using preset local feature vectors described in local weighted exponent pair carry out Feature Dimension Reduction at Reason, obtains local feature vectors after dimensionality reduction；

First depth dimensionality reduction unit, for carrying out feature drop to the deep learning feature vector using predetermined depth Weighted Index Dimension processing, obtains deep learning feature vector after dimensionality reduction；

First concatenation unit, for splicing after the dimensionality reduction deep learning feature vector after local feature vectors and dimensionality reduction, and will The feature vector obtained after splicing is normalized, and obtains normalization characteristic vector；

First splicing dimensionality reduction unit, for carrying out Feature Dimension Reduction to the normalization characteristic vector using default splicing Weighted Index Processing obtains target feature vector.

16. image search apparatus according to claim 14, which is characterized in that described image searcher further include: add It weighs index and generates sub-device, for being iterated optimization life using the sampling feature vectors of training sample in default tranining database At default Weighted Index, wherein the default Weighted Index includes presetting local weighted index, predetermined depth Weighted Index, pre- If splicing Weighted Index.

17. image search apparatus according to claim 16, which is characterized in that the Weighted Index generates sub-device packet It includes:

Second obtains module, for obtaining commodity image all in default tranining database, extracts all commodity figures The product features vector in target interest region, and obtains the matrix A of m × n according to the product features vector of extraction as in, wherein m The dimension of product features vector is represented, n represents the number of training sample；

Dimensionality reduction module obtains the dimensionality reduction matrix B of l × m, wherein m > l, l for being handled using Principal Component Analysis Algorithm matrix A For positive integer；

Iteration module for using matrix B as initialization matrix W, and utilizes the sampling feature vectors iteration of training sample excellent Change the matrix W, obtains the default Weighted Index for carrying out dimensionality reduction and fusion to feature vector.

18. image search apparatus according to claim 17, which is characterized in that described second obtains the commodity that module obtains Feature vector is that commodity local feature vectors or commodity deep learning feature vector or commodity local feature vectors and commodity are deep The commodity for spending learning characteristic vector splice feature vector；

Then the sampling feature vectors of the corresponding training sample utilized of the second acquisition module are sample local feature vectors or sample The sample of this deep learning feature vector or sample local feature vectors and sample deep learning characteristic vector splice feature to Amount；

Then the corresponding obtained default Weighted Index of the iteration module refers to preset local weighted index or predetermined depth weighting Number, or default splicing Weighted Index.

19. image search apparatus according to claim 17, which is characterized in that the training sample includes Positive training sample To and negative training sample pair, it further includes training sample generation module that the Weighted Index, which generates sub-device, and the training sample is raw Include: at module

Extracting unit, for extracting multiple images to be retrieved in all commodity images in the default tranining database, and It obtains and corresponding search result is obtained according to each image to be retrieved；

Sequencing unit obtains search result after sequence corresponding with image to be retrieved for being ranked up to each search result；

Generation unit, for image to be retrieved to be formed Positive training sample pair with the top n result of search result after corresponding sequence, And image to be retrieved is formed into negative training sample pair with N number of result of remaining result in search result after corresponding sequence；Wherein, N For positive integer.

20. image search apparatus according to claim 17, which is characterized in that the Weighted Index generates sub-device and also wraps Module in batches is included, for being carried out to the training sample when the data volume of the training sample is greater than preset data amount threshold value Batch processing obtains more batches of trained subsamples；Then

The iteration module is specifically used for:

21. image search apparatus according to claim 17, which is characterized in that the Weighted Index generates sub-device and also wraps Segmentation module is included, for when the dimension of the sampling feature vectors of the training sample is greater than default dimension threshold value, to the sample The dimension of eigen vector carries out segment processing, obtains multistage sample characteristics subvector；Then the iteration module is specifically used for:

22. image search apparatus according to claim 21, which is characterized in that the dimensionality reduction Fusion Module includes:

Second local dimensionality reduction unit, for carrying out segment processing to the local feature vectors, obtain multistage local feature to Amount, wherein the number of segment of local feature subvector is identical as the number of segment of sample characteristics subvector；By every section of local feature to Amount, which multiplies, to be corresponded to preset local weighted index, and correspondence obtains local feature subvector after multistage dimensionality reduction；Splice the multistage dimensionality reduction Feature subvector gets up afterwards, obtains local feature vectors after dimensionality reduction；

Second depth dimensionality reduction unit obtains multistage depth characteristic for carrying out segment processing to the deep learning feature vector Subvector, wherein the number of segment of depth characteristic subvector is identical as the number of segment of sample characteristics subvector；By every section of depth characteristic Subvector, which multiplies, to be corresponded to predetermined depth Weighted Index, and correspondence obtains depth characteristic subvector after multistage dimensionality reduction；Splice the multistage Feature subvector gets up after dimensionality reduction, obtains depth characteristic vector after dimensionality reduction；

Second concatenation unit, for splicing after the dimensionality reduction deep learning feature vector after local feature vectors and dimensionality reduction, and will The feature vector obtained after splicing is normalized, and obtains normalization characteristic vector；

Second splicing dimensionality reduction unit obtains multistage normalization characteristic for carrying out segment processing to the normalization characteristic vector Subvector, wherein the number of segment of normalization characteristic subvector is identical as the number of segment of sample characteristics subvector；By every section of normalization Feature subvector, which multiplies, to be corresponded to default splicing Weighted Index, and correspondence obtains normalization characteristic subvector after multistage dimensionality reduction；Splicing institute Feature subvector after multistage dimensionality reduction is stated, normalization characteristic vector after dimensionality reduction is obtained.

23. image search apparatus according to claim 14, which is characterized in that the resolution ratio in target interest region is big In 100*100.

24. image search apparatus according to claim 23, which is characterized in that the resolution ratio in target interest region is 256*256。

25. image search apparatus according to claim 14, which is characterized in that the extraction module includes for extracting mesh Mark the local shape factor unit of interest region local feature vectors；

The local shape factor unit includes:

Subelement is extracted, for extracting multiple Feature Descriptors in target interest region；

Coded sub-units, for using Fisher to each Feature Descriptor according to preset GMM mixed Gauss model Vector is encoded, and the local feature vectors in target interest region are obtained.

26. image search apparatus according to claim 14, which is characterized in that the extraction module includes deep learning spy It levies extraction unit: for target interest region to be inputted preset depth convolutional neural networks, obtaining the target interest The deep learning feature vector in region.