CN108415937A

CN108415937A - A kind of method and apparatus of image retrieval

Info

Publication number: CN108415937A
Application number: CN201810069845.4A
Authority: CN
Inventors: 张雷; 陈杰
Original assignee: Bo Yun Vision (beijing) Technology Co Ltd
Current assignee: Bo Yun Vision (beijing) Technology Co Ltd
Priority date: 2018-01-24
Filing date: 2018-01-24
Publication date: 2018-08-17

Abstract

An embodiment of the present invention provides a kind of method and apparatus of image retrieval, it is relatively low to solve existing search method precision, the problem of needing some specific features that can just improve precision as auxiliary in retrieval.The method of image retrieval therein, including：From the first depth characteristic is extracted in image to be retrieved and from image to be retrieved extract First partial feature；First depth characteristic and First partial feature are matched respectively with the second depth characteristic of multiple candidate images and the second local feature stored in property data base, obtain matched second depth characteristic and the second local feature；Output image corresponding with matched second depth characteristic and the second local feature.

Description

A kind of method and apparatus of image retrieval

Technical field

The present invention relates to field of image recognition more particularly to a kind of method and apparatus of image retrieval.

Background technology

Nowadays road monitoring probe and video camera spread all over street bayonet.Public security and traffic system are various in scheduling, criminal investigation etc. The image data for being required for these monitoring probes of heavy dependence to be collected into security protection task.But for a large amount of video image number According to wanting to find target from these vedio datas only according to manpower, not only inefficiency, but also need to put into a large amount of people Power material resources, high cost.

Although existing technology can carry out the inspection of target using deep learning using these a large amount of vedio datas Rope, but often precision is relatively low, and because needing some specific features that can just improve precision as auxiliary in retrieval, the present invention carries The image search method of confession can in the case that picture require it is lower, effectively improve retrieval precision.

Invention content

In view of this, an embodiment of the present invention provides a kind of method and apparatus of image retrieval, solves conventional images inspection The retrieval precision of rope technology is relatively low, and needs the problem of some specific features are as assisting just improving precision.

According to an aspect of the invention, there is provided a kind of method of image retrieval, including：It is extracted from image to be retrieved First depth characteristic and the extraction First partial feature from image to be retrieved；By the first depth characteristic and First partial feature It is matched, is obtained respectively with the second depth characteristic of multiple candidate images and the second local feature stored in property data base To matched second depth characteristic and the second local feature；Output and matched second depth characteristic and the second local feature Corresponding image.

In one embodiment, by the first depth characteristic and First partial feature with stored in property data base it is multiple Before second depth characteristic of candidate image and the second local feature are matched respectively, further include：Pass through dimension-reduction treatment pair First depth characteristic of image to be retrieved carries out compressed encoding, obtains the third depth characteristic of image to be retrieved；By figure to be retrieved The third depth characteristic of picture is matched with the 4th depth characteristic of multiple candidate images of characteristic library storage, is matched The 4th depth characteristic, the 4th depth characteristic of plurality of candidate image is to multiple candidate images by dimension-reduction treatment Two depth characteristics carry out what compressed encoding obtained；Obtain image corresponding with matched 4th depth characteristic, wherein deep by first The second depth characteristic and second of the multiple candidate images stored in degree feature and First partial feature and property data base Local feature is matched respectively, including：By the first depth characteristic of image to be retrieved in property data base with it is matched Second depth characteristic of the corresponding image of the 4th depth characteristic is matched；By the First partial feature and feature of image to be retrieved Second local feature of the multiple candidate images stored in database is matched.

In one embodiment, compressed encoding is carried out to the first depth characteristic of image to be retrieved by dimension-reduction treatment, including： Using ITQ (iterative quantization method), compressed encoding is carried out to the first depth characteristic of image to be retrieved by dimension-reduction treatment.

In one embodiment, the method for image retrieval further includes：The second depth characteristic is extracted from multiple candidate images；It is logical It crosses dimension-reduction treatment and compressed encoding is carried out to the second depth characteristic of multiple candidate images, obtain the 4th depth characteristic, it is deep by the 4th Characteristic storage is spent in property data base.

In one embodiment, the multiple times that will be stored in the first depth characteristic and First partial feature and property data base The second depth characteristic and the second local feature for selecting image are matched respectively, including：Calculate the first depth characteristic with it is multiple The similarity of second depth characteristic of candidate image obtains the first similarity of multiple candidate images；Calculate First partial feature With the similarity of the second local feature of multiple candidate images, the second similarity of multiple candidate images is obtained；To multiple candidates The first similarity and the second similarity of image are weighted, and obtain ranking operation result；It is true according to ranking operation result Fixed matched second depth characteristic and the second local feature.

In one embodiment, image is vehicle image, and method further includes：For each vehicle collecting cart in multiple vehicles Image；Extract the second depth characteristic and the second local feature of vehicle image；By the second depth characteristic of vehicle image with And second local feature be stored in property data base.

In one embodiment, vehicle image is collected for each vehicle in multiple vehicles, specifically includes step：Acquisition regards Frequency stream information；For each vehicle, multiple images for collecting dimension are filtered out from Video stream information according to multiple collection dimensions； Information of vehicles is obtained from multiple images for collecting dimension；The vehicle for each vehicle after being arranged according to information of vehicles Image.

In one embodiment, the vehicle image for each vehicle after being arranged according to information of vehicles, specifically includes Step：Multistage screening is carried out according to information of vehicles；The result of multistage screening is concluded and classified, being directed to after being arranged is each The vehicle image of vehicle.

In one embodiment, multiple collection dimensions include following at least two dimension：Bayonet scene dimension, including catalogue card At least one of mouth, microcaloire mouth and monitoring；Weather dimension, including rainy day, greasy weather, snowy day, Qiang Guangtian and normal weather At least one of；Road dimension, including high speed, major trunk roads, bypass, tunnel, crossroad, T-shaped road junction, corner and lane At least one of road；Angle dimension including overlook front, overlook the back side, overlook side, head-up front, head-up the back side and Look squarely at least one of side；Target accounting dimension, including target account for image be more than 40%, target account for image 20% to Between 40% and target accounts for image and is less than at least one of 20%；Image resolution ratio dimension, including 1260*1080,1080* 720, at least one of 720*360 and 360*144；Between time dimension, including 0 point to 6 points, between 6 points to 9 points, 9 points To between 15 points, between 15 points to 21 points and at least one of between 21 points to 0 point；Target occlusion dimension, including target At least one of part is blocked and target is not blocked；Destination number dimension, including single target quantity and multiple At least one of destination number.

In one embodiment, information of vehicles includes following at least one information：The car plate of vehicle in the picture, model and Color, wherein multistage screening is carried out according to information of vehicles, including：Known for car plate from multiple images for collecting dimension Not, to collect the vehicle image of same car plate；It is screened again for model and color from the vehicle image of same car plate, with Collect the image of same vehicle.

In one embodiment, from extracting the first depth characteristic in image to be retrieved and extract first from image to be retrieved Local feature, including：The first depth spy is extracted from image to be retrieved in the way of convolutional neural networks or Recognition with Recurrent Neural Network Sign, and First partial feature is extracted from image to be retrieved in the way of CDVS (compact description of visual search).

In one embodiment, loss function used in the training process of convolutional neural networks includes：tripletloss And/or softmaxloss, wherein convolutional neural networks include that normalization layer and full articulamentum, tripletloss loss functions are set It sets after normalizing layer, softmaxloss loss functions are arranged after full articulamentum.

According to another aspect of the present invention, a kind of device of image retrieval is provided, including：

Extraction module, the first depth characteristic for being configured to extract in picture to be retrieved, and be configured to extract to be retrieved First partial feature in picture；Matching module is configured to match the first depth characteristic and characteristic in picture to be retrieved Second depth characteristic of the multiple candidate images stored in library, and obtain matched second depth characteristic；And it matches to be retrieved Second local feature of the multiple candidate images stored in the First partial feature and property data base in picture, and matched The second local feature；Output module is configured as output to corresponding with matched second depth characteristic and the second local feature Image.

In one embodiment, the device of image retrieval further includes：Compressed encoding module is configured to treat by dimension-reduction treatment The first depth characteristic for retrieving image carries out compressed encoding, obtains the third depth characteristic of image to be retrieved, and by dimensionality reduction at It manages and the 4th depth characteristic of image to be retrieved is obtained to the second depth characteristic progress compressed encoding of multiple candidate images, wherein The 4th depth of the multiple candidate images stored in third depth characteristic and property data base in picture to be retrieved is matched with module Feature is spent, and obtains the corresponding image of matched 4th depth characteristic, wherein matching module concrete configuration is：By figure to be retrieved Second depth characteristic of the image corresponding with matched 4th depth characteristic in the first depth characteristic and property data base of picture It is matched by matching module；Matching module concrete configuration is by the First partial feature and property data base of image to be retrieved Second local feature of multiple candidate images of middle storage is matched by matching module.

In one embodiment, the first compressed encoding module concrete configuration is：Using ITQ (iterative quantization method), pass through drop Dimension processing carries out compressed encoding to the first depth characteristic of image to be retrieved.

In one embodiment, the device of image retrieval further includes：Characteristic memory module is configured to store multiple candidates The second depth characteristic, the second local feature and the 4th depth characteristic of image, wherein extraction module concrete configuration is from multiple The second depth characteristic is extracted in candidate image and the second local feature is extracted from multiple candidate images.

In one embodiment, matching module concrete configuration is：Calculate the second of the first depth characteristic and multiple candidate images The similarity of depth characteristic obtains the first similarity of multiple candidate images, calculates First partial feature and multiple candidate images The second local feature similarity, obtain the second similarity of multiple candidate images, it is similar to the first of multiple candidate images Degree and the second similarity are weighted, and obtain ranking operation as a result, and determining matched second according to ranking operation result Depth characteristic and the second local feature.In one embodiment, the device of image retrieval further includes：Image collection module, is configured to Vehicle image is collected for each vehicle in multiple vehicles, wherein extraction module is additionally configured to：Extract the second of vehicle image Depth characteristic and the second local feature for extracting vehicle image, and characteristic memory module is additionally configured to：By vehicle figure Second depth characteristic of picture and the second local feature are stored in characteristic memory module.

In one embodiment, the device of image retrieval further includes：Video flowing acquisition module is configured to obtain video flowing letter Breath；Various dimensions screening module is configured to be directed to each vehicle, be filtered out from Video stream information according to multiple collection dimensions multiple Collect the image of dimension；Information of vehicles acquisition module is configured to obtain information of vehicles from multiple images for collecting dimension；Vehicle Image management module is configured to the vehicle image for each vehicle after being arranged according to information of vehicles.

In one embodiment, vehicle image management module further includes：Multistage screening module, be configured to according to information of vehicles into Row multistage screening.

In one embodiment, various dimensions screening module concrete configuration be include following at least two dimension：Bayonet scene is tieed up Degree, including at least one of catalogue bayonet, microcaloire mouth and monitoring；Weather dimension, including rainy day, greasy weather, snowy day, Qiang Guangtian And at least one of normal weather；Road dimension, including high speed, major trunk roads, bypass, tunnel, crossroad, T-shaped road At least one of mouth, corner and tunnel；Angle dimension including overlook front, overlook the back side, overlook side, head-up front, Look squarely at least one of the back side and head-up side；Target accounting dimension, including target account for image and account for figure more than 40%, target Picture is between 20% to 40% and target accounts for image and is less than at least one of 20%；Image resolution ratio dimension, including 1260* 1080, at least one of 1080*720,720*360 and 360*144；Between time dimension, including 0 point to 6 points, 6 points to 9 Between point, between 9 points to 15 points, between 15 points to 21 points and at least one of between 21 points to 0 point；Target occlusion is tieed up Degree, including target part is blocked and at least one of target is not blocked；Destination number dimension, including single target number At least one of amount and multiple destination numbers.

In one embodiment, information of vehicles includes following at least one information：Vehicle car plate in the picture, model and Color, and multistage screening module concrete configuration is：It is identified for car plate from multiple images for collecting dimension, to collect The vehicle image of same car plate is screened from the vehicle image of same car plate for model and color again later, to collect The image of same vehicle.

In one embodiment, extraction module concrete configuration is：Using convolutional neural networks or Recognition with Recurrent Neural Network to be checked The first depth characteristic is extracted in rope image and is calculated using CDVS (compact description of visual search), LBP (local binary patterns) The mode of son, sift (Scale invariant features transform matching algorithm) or surf (accelerating robust feature algorithm) is from image to be retrieved Extract First partial feature.

In one embodiment, the first depth characteristic extraction module concrete configuration is：The training process institute of convolutional neural networks The loss function used includes：Tripletloss and/or softmaxloss, wherein convolutional neural networks include normalization layer and Full articulamentum, tripletloss loss functions are arranged after normalizing layer, and the setting of softmaxloss loss functions is connecting entirely After connecing layer.

An embodiment of the present invention provides a kind of method of image retrieval, by extract depth characteristic in image to be retrieved and Depth characteristic and local feature in local feature, with property data base are compared respectively, are finally exported in property data base With the image of the depth characteristic and local characteristic matching of image to be retrieved.Can in the case that picture require it is lower, effectively Retrieval precision is improved, it is relatively low to solve existing search method precision, needs some specific features just may be used as auxiliary in retrieval The problem of to improve precision.

Description of the drawings

Fig. 1 show the flow diagram of the method for the image retrieval of one embodiment of the invention offer.

Fig. 2 show the schematic diagram of the image retrieval of one embodiment of the invention offer.

Fig. 3 show the flow diagram of the method for the image retrieval of another embodiment of the present invention offer.

Fig. 4 show the schematic diagram of the image retrieval of another embodiment of the present invention offer.

Fig. 5 show the flow diagram of the method for the vehicle image collection of one embodiment of the invention offer.

Fig. 6 show the flow diagram of the method for the image retrieval of another embodiment of the present invention offer.

Fig. 7 show the schematic diagram of the image retrieval of another embodiment of the present invention offer.

Fig. 8 show the structural schematic diagram of the device of the image retrieval of one embodiment of the invention offer.

Fig. 9 show the structural schematic diagram of the device of the image retrieval of another embodiment of the present invention offer.

Specific implementation mode

Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete Site preparation describes, it is clear that described embodiment is only a part of the embodiment of the present invention, instead of all the embodiments.Based on this Embodiment in invention, every other reality obtained by those of ordinary skill in the art without making creative efforts Example is applied, shall fall within the protection scope of the present invention.

Fig. 1 show the flow diagram of the image retrieval of one embodiment of the invention offer.The method of Fig. 1 can be filled by calculating It sets, such as server, executes.As shown in Figure 1, the method for image retrieval includes：

100：Extract the first depth characteristic and First partial feature in image to be retrieved.

Image to be retrieved can be the image of vehicle image, the image of people or other mobile objects, and the present invention does not make this It limits.

Depth characteristic for example can be the depth characteristic extracted in the way of CNN (convolutional neural networks), reality of the invention It applies example to be not limited to this, for example, it can be carry in the way of the deep neural network of Recognition with Recurrent Neural Network or other structures The depth characteristic taken.Local feature can be the depth characteristic extracted in the way of CDVS (compact description of visual search), The embodiment of the present invention is not limited to this, for example, local feature can utilize LBP (local binary patterns) operator, sift (rulers Degree invariant features Transformation Matching algorithm) or the local feature that extracts of surf (accelerating robust feature algorithm) mode.

110：By the first depth characteristic of image to be retrieved and First partial feature and multiple candidates in property data base Second depth characteristic of image and the second local feature are matched respectively, obtain matched second depth characteristic and second Local feature.

Second depth characteristic and the second local shape factor mode can be with the first depth characteristic and the second local features Mode is identical or different.First depth characteristic matched with the second depth characteristic both can refer to it is similar or identical, for example, the two Similarity then thinks that the two is similar more than preset threshold value, and similarity can be characterized with Euclidean distance or Hamming distance etc..Class As, First partial feature and the matching of the second local feature can also be characterized with depth characteristic similar mode.

It according to an embodiment of the invention, can be flat according to the weighting of the similarity of the similarity and local feature of depth characteristic Mean value determines whether to match, for example, server can calculate the first depth characteristic and second depth of multiple candidate images is special The similarity of sign obtains the first similarity of multiple candidate images；Calculate the second of First partial feature and multiple candidate images The similarity of local feature obtains the second similarity of multiple candidate images；The first similarity to multiple candidate images and Two similarities are weighted, and obtain ranking operation result；Matched second depth characteristic is determined according to ranking operation result With the second local feature.

120：According to matched second depth characteristic and the second local feature, corresponding image is exported.Specifically, can To collect multiple candidate images first, and the second depth characteristic and the second local feature of multiple candidate images are extracted, in advance The second depth characteristic of storage multiple images and the second local feature and multiple images and its depth are special in property data base It seeks peace the correspondence of local feature, the property data base as retrieval.When being retrieved, first with for example same skill Art means extract the first depth characteristic and First partial feature of picture to be retrieved, by the corresponding depth characteristic of picture to be retrieved and Local feature is matched respectively with the depth characteristic of multiple candidate images in above-mentioned property data base and local feature, then The matching similarity of the depth characteristic and local feature of an image can be calculated separately, can for example pass through weighting scheme handle later The two is added the similarity total score for obtaining the image, and the depth that the highest top N of similarity is discharged in a manner of such as ranking is special It seeks peace local feature, corresponding image is found according to the depth characteristic and local feature, finally export the image needed.

The schematic diagram of the image retrieval provided as shown in Figure 2 for one embodiment of the invention, it can be seen from the figure that be checked Rope image and the image of characteristic library storage all have respective depth characteristic and a local feature, and when matching uses respective depth Degree feature and local feature carry out matched mode progress respectively, finally export image corresponding with matching characteristic as retrieval As a result, the matching due to the global alignment with depth characteristic and local feature, can play the effect for improving retrieval precision, below The matching retrieval mode of this fusion depth characteristic and local feature is referred to as precise search.

An embodiment of the present invention provides a kind of method of image retrieval, by extract depth characteristic in image to be retrieved and Depth characteristic and local feature in local feature, with property data base are compared respectively, are finally exported in property data base With the image of the depth characteristic and local characteristic matching of image to be retrieved.It can accomplish to identify the target in any image, solve The prior art be necessarily required to that the features such as car plate could be retrieved to image request to be retrieved height, it is limited to ask Topic.

Optionally, as another embodiment, the method for the image retrieval of Fig. 1 further includes：It is extracted from multiple candidate images Second depth characteristic；Compressed encoding is carried out to the second depth characteristic of multiple candidate images by dimension-reduction treatment, it is deep to obtain the 4th Feature is spent, the 4th depth characteristic is stored in property data base.

For example, it can be ITQ (iterative quantization method) to carry out compressed encoding above by dimension-reduction treatment.This method can reach To the purpose for shortening retrieval time.The embodiment of the present invention is not limited to this, and compressed encoding is carried out above by dimension-reduction treatment Process can also realize that other hash methods can also achieve the purpose that above-mentioned shortening retrieval time by other hash methods. The compressed encoding feature of image in image and property data base to be retrieved is matched later, specially：By third depth characteristic and 4th depth characteristic is matched, and the 4th depth characteristic is to carry out compressed encoding to the second depth characteristic in property data base It obtains, wherein the process of compressed encoding is realized using above-mentioned ITQ (iterative quantization method) or other hash methods. During matched, above-mentioned third depth characteristic and the 4th matched process of depth characteristic can use Hamming distance to calculate phase Like degree.Image corresponding with matched 4th depth characteristic is finally obtained, and the 4th depth characteristic can be stored in database In.

Optionally, the process that final retrieval image is exported as another embodiment, in the image search method of Fig. 1 is specific For：After obtaining image corresponding with matched 4th depth characteristic, the first depth characteristic to image to be retrieved and matched Second depth characteristic of the 4th depth characteristic correspondence image is matched, and the first similarity is obtained.To the first of image to be retrieved Local feature and the second local feature of matched 4th depth characteristic correspondence image are matched, and the second similarity is obtained.It is right First similarity and the second similarity are weighted, and can give the first similarity and the second similarity according to significance level Assign weights, the two weights be added 1.It is above-mentioned to the first depth characteristic of image to be retrieved and matching during matched The 4th depth characteristic correspondence image the second depth characteristic carry out matching and to the First partial feature of image to be retrieved and Second local feature of matched 4th depth characteristic correspondence image, which carries out matching, to use Euclidean distance to calculate similarity.Most Image corresponding with matched second depth characteristic and the second local feature is exported according to the operation result of weighting afterwards.

Optionally, as another embodiment, when image is vehicle image, the method for Fig. 1 further includes：For multiple vehicles In each vehicle collect vehicle image；Extract the second depth characteristic and the second local feature of vehicle image；By vehicle figure Second depth characteristic of picture and the second local feature are stored in property data base.

Specifically, vehicle image is collected for each vehicle in multiple vehicles, specifically includes step：Obtain video flowing letter Breath；For each vehicle, multiple images for collecting dimension are filtered out from Video stream information according to multiple collection dimensions；From multiple It collects in the image of dimension and obtains information of vehicles；The vehicle image for each vehicle after being arranged according to information of vehicles.

Specifically, multistage screening can be carried out according to information of vehicles, and the result of multistage screening is concluded and classified, obtained The vehicle image for each vehicle after to arrangement.

According to an embodiment of the invention, information of vehicles includes following at least one information：The car plate of vehicle in the picture, type Number and color, wherein according to information of vehicles carry out multistage screening, including：From it is multiple collect dimensions images in for car plate into Row identification, to collect the vehicle image of same car plate；It is sieved again for model and color from the vehicle image of same car plate Choosing, to collect the image of same vehicle.

In one embodiment, loss function used in the training process of convolutional neural networks includes：tripletloss And/or softmaxloss.For example, convolutional neural networks include normalization layer and full articulamentum, tripletloss loss functions It is arranged after normalizing layer, while softmaxloss loss functions are arranged after full articulamentum.

The flow diagram of the method for the image retrieval of another embodiment of the present invention offer is provided.The implementation of Fig. 3 Example is the example of the embodiment of Fig. 1.The method of Fig. 3 includes following content.

300：Extract the first depth characteristic in picture to be retrieved.

310：Extract the First partial feature in picture to be retrieved.

320：Compressed encoding is carried out to the first depth characteristic of image to be retrieved by dimension-reduction treatment, it is special to obtain third depth Sign.

330：Third depth characteristic and the 4th depth characteristic are matched, obtained corresponding with matched 4th depth characteristic Image.4th depth characteristic is to carry out compressed encoding to the second depth characteristic in property data base to obtain, wherein pressing The process for reducing the staff code is realized using above-mentioned ITQ (iterative quantization method) or other hash methods.

350：By the first depth characteristic of image to be retrieved and First partial feature and matched 4th depth characteristic pair Second depth characteristic of the image answered and the second local feature are matched.

360：Second depth of the first depth characteristic of image to be retrieved and matched 4th depth characteristic correspondence image is special Sign matching, obtains the first similarity.

370：Second part of the First partial feature of image to be retrieved and matched 4th depth characteristic correspondence image is special Sign matching, obtains the second similarity.

380：First similarity and the second similarity are weighted, it can be similar to first according to significance level Degree and the second similarity assign weights, the two weights be added 1.

390：It is exported according to the operation result of weighting corresponding with matched second depth characteristic and the second local feature Image.

Above-mentioned third depth characteristic and the 4th matched process of depth characteristic can use Hamming distance to calculate similarity. It should be noted that the compressed encoding and matching the process of the compressed corresponding depth characteristic and being examined prior to above-mentioned raising The precise search step of Suo Jingdu (is step 330) in detail in this figure, it is therefore an objective to reduce the quantity of precise search image, reduce fine The retrieval amount (database images after reduction are hereinafter image after screening) of retrieval, overall retrieval rate is improved with this, The matching process between feature after this compressed encoding is referred to as quick-searching hereinafter.

Fig. 4 show the schematic diagram of the image retrieval of another embodiment of the present invention offer, it can be seen from the figure that quick Precise search after retrieval is the fusion matching process to depth and local feature.Above-mentioned first similarity and the second similarity The mode that similarity can be calculated by using Euclidean distance is obtained.The advantages of ranking operation is can be according to significance level to not Same similarity assigns weights.

In one embodiment of this invention, the weights for assigning the first similarity and the second similarity 0.5 respectively are equivalent to One similarity and the second similarity occupy same proportion, the advantage of doing so is that balancing single feature for output result It influences, increases matched accuracy.

In one embodiment of this invention, the process for compressed encoding being carried out above by dimension-reduction treatment passes through ITQ (iteration amounts Change method) it realizes.ITQ (iterative quantization method) is to carry out binary conversion treatment to original depth characteristic, passes through binary conversion treatment Depth characteristic size can narrow down to the 1/256 of former depth characteristic size, this mode makes depth characteristic can in matching With keep precision without lossing too much in the case of, substantially shorten match time.The method of iterative quantization is to having obtained Depth characteristic is normalized, and obtains the training dataset of ITQ (iterative quantization method).Then the training data is carried out PCA dimension-reduction treatment, this process are that training dataset (2048*N, N are number of samples) is mapped to characteristic dimension (2048* 2048), then by extracting the feature vector and characteristic value of preceding 256 dimension, the feature distribution that training dataset 2048 is tieed up maps To the feature distribution of 256*256, the feature distribution V after dimensionality reduction is obtained with this.Finally utilize Random-Rotation matrix R to this feature point Cloth is rotated so that following quantization loss function is minimum：

Wherein B is binary coded matrix,It indicates binary coded matrix B and passes through Random-Rotation square The Euclidean distance of the feature distribution V of battle array R so that the Euclidean distance is minimum, i.e., the quantization loss function is minimum, finally so that two-value The information of characteristic loss after change is as few as possible.ITQ (iterative quantization method) fixes R before this, then optimizes B.Then B is fixed, Then optimize R.The continuous iteration of the two steps, algorithm will converge on a locally optimal solution.Obtain spin matrix R it Afterwards, it is multiplied with the depth characteristic 2048float features after normalization, obtaining as a result, set 1 more than 0, setting to 0 less than 0, from And obtain it is compressed after 256bit features.

In one embodiment of this invention, above-mentioned image search method can be applied to vehicle image retrieval.It collects first every Then the vehicle image of a vehicle extracts the second depth characteristic and the second local feature of the vehicle image, should after extraction It is spare when the second depth characteristic and the second part characteristic storage to be retrieved in property data base.

Fig. 5 show the flow diagram that the vehicle image of one embodiment of the invention offer is collected.One in the present invention is real It applies in example, the process for collecting vehicle image includes：

500：The real-time video flow data of traffic camera is obtained from public security system or other mechanisms.

510：By multiple collection dimensions from video flowing, the vehicle image in video flowing is screened, filters out symbol Multiple pictures for collecting dimension condition are closed as the vehicle image collected.

520：By the support of third party software, information of vehicles is obtained from the vehicle image of the collection after screening, then lead to It crosses and utilizes the information of vehicles, finally obtain all images of each car for extracting depth characteristic, hereinafter referred to as vehicle figure Image set.

The advantage of doing so is that ensure that each image has good grounds, when retrieving matched image, when can be most fast Between find required information so that image retrieval has more rich function.

530：The picture of collection is identified by Car license recognition, filters out all images of same car plate.

540：All images of above-mentioned same car plate are screened again using color and vehicle information, are filtered out all The vehicle of same car plate identifies multiple vehicles and color if under the vehicle image of same car plate, can also use artificial The mode of screening is screened, it is therefore an objective to exclude the error image screened in step 530.

Finally by all image collections of each car after debug image to together, obtaining vehicle image collection.Pass through Image that multistage screening obtains is by arranging and concluding, it is ensured that the data in the vehicle image collection of feature to be extracted be it is accurate, Also it is that subsequent extraction characteristic procedure reduces unnecessary workload.

In one embodiment of this invention, above by using information of vehicles, obtain for extract depth characteristic each All images of vehicle, the step of specifically including a multistage screening, carry out inducing classification processing, can obtain after multistage screening To vehicle image collection.As shown in figure 5, multi-level screening procedure includes step 530 and step 540.

In one embodiment of this invention, above-mentioned multiple collection dimension conditions include 9 collection dimensions：

At least one of bayonet scene dimension, including catalogue bayonet, microcaloire mouth and monitoring；

At least one of weather dimension, including rainy day, greasy weather, snowy day, Qiang Guangtian and normal weather；

In road dimension, including high speed, major trunk roads, bypass, tunnel, crossroad, T-shaped road junction, corner and tunnel It is at least one；

Angle dimension including vertical view front overlook the back side, overlook side, looking squarely positive, the head-up back side and head-up side At least one of；

Target accounting dimension, including target account for image and are more than 40%, and target accounts for image between 20% to 40% and mesh Mark accounts for image and is less than at least one of 20%；

At least one of image resolution ratio dimension, including 1260*1080,1080*720,720*360 and 360*144；

Between time dimension, including 0 point to 6 points, between 6 points to 9 points, between 9 points to 15 points, between 15 points to 21 points with And at least one of between 21 points to 0 point；

Target occlusion dimension, including target part is blocked and at least one of target is not blocked；

Destination number dimension, including at least one of single target quantity and multiple destination numbers.

It should be noted that the condition that above-mentioned 9 dimensions are screening images can also be used as an attribute mark after screening Note is in image file, and thus each image will include 9 attributes of above-mentioned 9 dimensions, and in addition to this, each image also wraps Containing 3 other attributes, it is respectively：The ID of place group obtains the time of vehicle image and the ID of collecting device.So most All will include 12 attributes (9 dimensional attribute+3 other attributes), therefore in all images of each car screened eventually After being matched to the image for needing to retrieve, more information can be obtained by 12 attributes by matching image, satisfaction makes Demand of the user to tracking vehicle other information.

In one embodiment of this invention, the extraction of above-mentioned all depth characteristics may be by CNN (convolutional Neural nets Network) or the mode of RNN (Recognition with Recurrent Neural Network) carry out.And CDVS (visual search may be used in the extraction of above-mentioned local feature Compact description son) mode carry out.

In one embodiment of this invention, the depth model that above-mentioned convolutional neural networks use is from 50 layers of residual error network (Resnet-50) it is developed on the basis of.Using the process of the model extraction depth characteristic of CNN (convolutional neural networks), Including building depth model, depth model includes mode input layer, model convolutional layer and model loss layer.Followed by depth mould Type loads training data, obtains depth characteristic.The structure of depth model comprises the following steps：

The input layer of model is built, mode input layer can need individually according to loss function or the different of model structure Design, the embodiment of the present invention carry out vehicle image using tripletloss loss functions and softmaxloss loss functions Training.Since tripletloss is triple loss function, by 2 different images of same vehicle and 1 different vehicle Vehicle image put together, form a triple data.Triple data are transferred to model from the input layer of model Convolutional layer is propagated forward in layer.

Then the convolutional layer for adjusting model, mainly adjusts convolution kernel.This part uses modularized design, entire model set With 2 kinds of totally 15 similar modules.

Module is initiated with BatchNormalize (batch normalizes) layer, is the output data to last layer, next layer The layer of operation is normalized in input data, can preferably ensure that model is restrained, to reach training effect.

Next the layer connected, 2 kinds of modules are different, the first has connect 3 convolutional layers, and convolution kernel is respectively 1*1,3* 3 and 1*1, and using the output of a upper module as inputting and replicating one time, in this module, be added with last output.The Two kinds are then to have connect 4 convolutional layers, and the output that the output of one of convolutional layer is last with this module is added.Convolutional layer is set up Later, one layer of down-sampled layer, the output layer as network when using finally are increased.

The use of model loss function tripletloss and softmaxloss enable model to be easier to restrain, obtain more Good training effect.In principle, tripletloss be in order to by the feature of same vehicle draw closer to the feature of different vehicles is drawn It is farther, to make model obtain better feature differentiation ability.Specific design is the down-sampled layer in last layer of the preceding paragraph Later, increase a normalization layer, and then one tripletloss layers (or loss function).One is connect again on down-sampled layer A full articulamentum connects softmaxloss layers (or loss functions) after full articulamentum, and so far the structure of convolutional neural networks is just It completes.

It can start to adjust the parameter in network, that is, the mistake of image training characteristics one by one after the completion of network struction Journey：All image datas are loaded into data input layer first, up to loss layer, then data are propagated forward by input layer Loss layer calculates loss (loss) value, and penalty values start to return according to chain type Rule for derivation, each with the passback of loss values The parameter of layer starts automatically with new, and then new one group of data input, obtains new loss values, start iterative cycles.Until loss Value drops to the threshold value being artificially arranged or reaches the iterations upper limit, and training terminates.

So far, the structure of entire feature and training process just complete.When extracting feature, it is only necessary to be passed to vehicle image Mode input layer, takes out feature from down-sampled layer, has just obtained the second depth characteristic of vehicle.

In one embodiment of this invention, the method for above-mentioned CDVS (compact description of visual search) is searched for vision The feature description of rope has certain rotation and scaling invariance, that is, together including for the feature of image texture and edge After rotation and scaling, the feature of extraction a degree of can remain unchanged one image.Such characteristic makes CDVS in complexity Robustness under environment is more preferable, this feature is fine for the matching capacity of same target, makes together with depth characteristic in screening With more properly, this is also to choose the reason of it is as supplemental characteristic.

When being stored in property data base, input vehicle image passes through CDVS feature extractions, will produce two kinds of spies of global and local Sign, the embodiment of the present invention have only used CDVS local features.The second local feature generated and the second depth characteristic and compression The 4th depth characteristic after coding is saved in together as the feature of this vehicle image in property data base.

Fig. 6 show the flow diagram of the method for the image retrieval of another embodiment of the present invention offer.The embodiment of Fig. 6 It is to be illustrated by taking the retrieval of vehicle image as an example.As included method from can be seen that the step of vehicle image is retrieved shown in Fig. 6 Including following content：

600：It is initially the process of image collection from this step.

For example, the real-time video flow data of traffic camera can be obtained from public security system or other mechanisms.

605：Image is collected from video flowing.

By multiple collection dimensions, the vehicle image in video flowing is screened, filters out and meets multiple collection dimensions The picture of condition is as the vehicle image collected.

610：Obtain information of vehicles.

By the support of third party software, information of vehicles, wherein vehicle are obtained from the vehicle image of the collection after screening Information includes：The model of the license board information of vehicle, vehicle color and vehicle.

615：It screens to obtain all images of same car plate by car plate.

It is first screened one time by the license board information of above-mentioned vehicle, filters out all images of same car plate, this screening Process can use Car license recognition model be screened.

620：By color and Type selection and classify, finally obtains all images of each car.

All images of above-mentioned same car plate are screened once again, be specifically using vehicle model and colouring information into Row screening and classification, can also be by artificial if identifying a variety of vehicle vehicles and color under the vehicle image of same car plate It screens again one time, has obtained all images of accurate each car, so far the collection process of vehicle image collection is completed.Below by needle The extraction training of feature is carried out to the vehicle image collection.

625：The depth characteristic of vehicle image collection is extracted by CNN (convolutional neural networks), specially：Using residual from 50 layers The depth model structure depth model developed on the basis of poor network (Resnet-50), depth model includes mode input Layer, model convolutional layer and model loss layer.Followed by training data is loaded to depth model, obtain depth characteristic.Depth model Structure comprise the following steps：

CDVS (compact description of visual search), the feature of extraction vehicle image collection are also needed to simultaneously, because CDVS (depending on Feel compact description of search) it is characterized in being directed to the feature description of visual search, including for the feature of image texture and edge, With certain rotation and scaling invariance, that is, same image, after rotation and scaling, the feature of extraction can be to a certain degree Remain unchanged.Such characteristic makes robustness of the CDVS under complex environment more preferable, of this feature for same target It is fine with ability, it is used together more properly with depth characteristic in screening, this is also to choose the reason of it is as supplemental characteristic.

630：Compressed encoding is carried out to above-mentioned second depth characteristic by ITQ (iterative quantization method), specially：ITQ is (repeatedly For quantization method) it is that binary conversion treatment is carried out to original depth characteristic, it can be with by the depth characteristic size of binary conversion treatment The 1/256 of former depth characteristic size is narrowed down to, this mode allows depth characteristic keeping precision not excessive in matching In the case of loss, substantially shorten match time.The method of iterative quantization is that the depth characteristic obtained is normalized Processing, obtains the training dataset of ITQ (iterative quantization method).Then PCA dimension-reduction treatment, this mistake are carried out to the training data Journey is that training dataset (2048*N, N are number of samples) is mapped to characteristic dimension (2048*2048), then by extraction before The feature distribution that training dataset 2048 is tieed up, is mapped to the feature distribution of 256*256 by the feature vector and characteristic value of 256 dimensions, The feature distribution V after dimensionality reduction is obtained with this.Finally this feature distribution is rotated using Random-Rotation matrix R so that as follows It is minimum to quantify loss function：

After having extracted the 4th depth characteristic, by the 4th depth characteristic and the second above-mentioned depth characteristic and second game Portion's feature is collectively stored in spare in property data base, and so far, the set-up procedure of image retrieval finishes, wherein in property data base Three features are contained, are the 4th depth characteristic, the second depth characteristic and the second local feature respectively.

635：It is initially the process of image retrieval from this step, the depth characteristic of image to be retrieved and local feature is carried It takes out, above-mentioned CNN (convolutional neural networks) and CDVS (compact description of visual search) is used to extract the respectively One depth characteristic and First partial feature, the process and step 625 of extraction are identical.

640：The first depth characteristic of dimension-reduction treatment, obtains third depth characteristic.

Compressed encoding is carried out to above-mentioned first depth characteristic using ITQ (iterative quantization method), obtains third depth characteristic, Specific method and step 630 are identical.So far, image to be retrieved also has depth characteristic corresponding with property data base, office Depth characteristic after portion's feature and compressed encoding.

645：The third depth characteristic of picture to be retrieved is matched with the 4th depth characteristic in property data base, is obtained To image corresponding with matched 4th depth characteristic.

Similarity is calculated using Hamming distance during matched, ranking is then carried out according to the height of similarity.

The process of obtaining can obtain the corresponding image of similarity ranking top N, and the depth of ranking top N correspondence image is special Local feature of seeking peace will carry out fine match with the two features of picture to be retrieved.

655：The first depth characteristic and First partial feature of image to be retrieved are corresponding with matched 4th depth characteristic Image the second depth characteristic and the second local feature matched.

Specially：First depth characteristic of picture to be retrieved and First partial feature is corresponding with above-mentioned ranking top N Second depth characteristic of image and the second local feature are matched, the second depth of ranking top N correspondence image here What feature and the second part were characterized in extracting by step 625 in advance.That is just with the upper of a part The feature in property data base is stated, this part is obtained by step 645.

660：The first similarity and the second similarity obtained to step 655 matching is weighted, and is transported according to weighting It calculates result and determines matched second depth characteristic and the second local feature.

Specially：By the matching of step 655, it is (special by the first depth of image to be retrieved the first similarity has been obtained Second depth characteristic matching primitives of ranking of seeking peace top N correspondence image come out) and the second similarity (pass through figure to be retrieved What the First partial feature of picture and the second of ranking top N correspondence image the local characteristic matching were calculated), it should be noted that It is that depth characteristic similarity mode and local characteristic similarity matching here are calculated by Euclidean distance.

Then the weights for assigning 0.5 respectively to the first similarity and the second similarity, after the two is multiplied by weights after calculating Score be added, the score to be sorted.

665：Export the corresponding image of above-mentioned sequence score rank top N.So far, retrieving finishes.

For the ease of understanding that the step in Fig. 6, Fig. 7 show the signal of the image retrieval of another embodiment of the present invention offer Figure.It can be seen from the figure that passing through the collection image that various dimensions screen from video flowing, each is obtained using multistage screening All images of vehicle as feature extraction vehicle data collection, later use CNN (convolutional neural networks) and CDVS (vision is searched Compact description of rope) the second depth characteristic and the second local feature are extracted respectively, then the second depth characteristic is used again ITQ (iterative quantization method) (ITQ) extracts the 4th depth characteristic, these three features are stored in property data base, in case inspection Suo Yong.

After obtaining image to be retrieved, extracted respectively using the above method the first depth characteristic, First partial feature and Third depth characteristic, the 4th depth characteristic that then first Rapid matching third depth characteristic and property data base store before, just Step filters out the image of ranking top N, is directed to the second depth characteristic and the second local feature of ranking top N image again later, It is accurately matched with the first depth characteristic of image to be retrieved and First partial feature, the first phase that depth characteristic matching generates The two, is multiplied by the score after weights by the weights for assigning 0.5 respectively like the second similarity that degree and local feature generate after calculating It is added, obtains sequence score, finally export the corresponding image of sequence score rank top N.

Fig. 8 show the structural schematic diagram of the device of the image retrieval of one embodiment of the invention offer.As shown in figure 8, figure As the device 10 of retrieval includes：

Extraction module 45, the first depth characteristic for being configured to extract in picture to be retrieved, and be configured to extract to be checked First partial feature in rope picture.

Matching module 60, be configured to match stored in the first depth characteristic in picture to be retrieved and property data base it is more Second depth characteristic of a candidate image, and obtain matched second depth characteristic；Match the First partial in picture to be retrieved Second local feature of the multiple candidate images stored in feature and property data base, and obtain matched second local feature.

Output module 75 is configured as output to image corresponding with matched second depth characteristic and the second local feature.

Extraction module 45 correspond to above-mentioned steps 100, matching module 60 correspond to above-mentioned steps 110, output module 75 correspond on State step 120.

Specifically, multiple candidate images can be collected first, and multiple candidate images are extracted by extraction module 45 The second depth characteristic and the second local feature, the second depth characteristic and the of multiple images is stored in property data base in advance The correspondence of two local features and multiple images and its depth characteristic and local feature, the characteristic as retrieval Library.When being retrieved, the first depth characteristic and first game of picture to be retrieved are extracted first with for example same technological means Portion's feature, by the corresponding depth characteristic of picture to be retrieved and local feature and multiple candidate images in above-mentioned property data base Depth characteristic and local feature are matched respectively by matching module 60, then can calculate separately the depth characteristic of an image With the matching similarity of local feature, for example the two addition can be shown that the similarity of the image is total by weighting scheme later Point, the depth characteristic and local feature of the highest top N of similarity are discharged in a manner of such as ranking, according to the depth characteristic and Local feature finds corresponding image, the image needed finally by the output of output module 75.

Fig. 9 show another embodiment of the present invention offer image retrieval device structural schematic diagram, with lower module with Above-described embodiment corresponds to, and is for realizing the device of above-described embodiment step, and details are not described herein for specific steps and related description, The device 10 of the image retrieval includes：

Video flowing acquisition module 15 is configured to obtain Video stream information.

Image collection module 20, concrete configuration are to collect vehicle image for each vehicle in multiple vehicles.

Various dimensions screening module 25 is configured to be directed to each vehicle, be sieved from Video stream information according to multiple collection dimensions Select multiple images for collecting dimension.

Information of vehicles acquisition module 30 is configured to obtain information of vehicles from multiple images for collecting dimension.Multistage screening Module 35 is configured to carry out multistage screening according to information of vehicles.

Vehicle image management module 40 is configured to the vehicle figure for each vehicle after being arranged according to information of vehicles Picture.

Characteristic memory module 50 is configured to store the second depth characteristic of multiple candidate images, the second local feature And the 4th depth characteristic.

Compressed encoding module 55 is configured to carry out compression volume to the first depth characteristic of image to be retrieved by dimension-reduction treatment Code, obtains the third depth characteristic of image to be retrieved, and by dimension-reduction treatment to the second depth characteristic of multiple candidate images into Row compressed encoding obtains the 4th depth characteristic of image to be retrieved.

Matching module 60, be configured to match stored in First partial feature in picture to be retrieved and property data base it is more Second local feature of a candidate image, and obtain matched second local feature and match the first depth in picture to be retrieved Second depth characteristic of the multiple candidate images stored in degree feature and property data base, and it is special to obtain matched second depth Sign.And it is configured to that the first similarity of multiple candidate images and the second similarity is weighted, obtains ranking operation As a result, and determining matched second depth characteristic and the second local feature according to ranking operation result.

Those of ordinary skill in the art may realize that lists described in conjunction with the examples disclosed in the embodiments of the present disclosure Member and algorithm steps can be realized with the combination of electronic hardware or computer software and electronic hardware.These functions are actually It is implemented in hardware or software, depends on the specific application and design constraint of technical solution.Professional technician Each specific application can be used different methods to achieve the described function, but this realization is it is not considered that exceed The scope of the present invention.

It is apparent to those skilled in the art that for convenience and simplicity of description, the system of foregoing description, The specific work process of device and unit, can refer to corresponding processes in the foregoing method embodiment, and details are not described herein.

In several embodiments provided herein, it should be understood that disclosed systems, devices and methods, it can be with It realizes by another way.For example, the apparatus embodiments described above are merely exemplary, for example, the unit It divides, only a kind of division of logic function, formula that in actual implementation, there may be another division manner, such as multiple units or component It can be combined or can be integrated into another system, or some features can be ignored or not executed.Another point, it is shown or The mutual coupling, direct-coupling or communication connection discussed can be the indirect coupling by some interfaces, device or unit It closes or communicates to connect, can be electrical, machinery or other forms.

The unit illustrated as separating component may or may not be physically separated, aobvious as unit The component shown may or may not be physical unit, you can be located at a place, or may be distributed over multiple In network element.Some or all of unit therein can be selected according to the actual needs to realize the mesh of this embodiment scheme 's.

In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, it can also It is that each unit physically exists alone, it can also be during two or more units be integrated in one unit.

It, can be with if the function is realized in the form of SFU software functional unit and when sold or used as an independent product It is stored in a computer read/write memory medium.Based on this understanding, technical scheme of the present invention is substantially in other words The part of the part that contributes to existing technology or the technical solution can be expressed in the form of software products, the meter Calculation machine software product is stored in a storage medium, including some instructions are used so that a computer equipment (can be People's computer, server or network equipment etc.) it performs all or part of the steps of the method described in the various embodiments of the present invention. And storage medium above-mentioned includes：USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), arbitrary access are deposited The various media that can store program ver-ify code such as reservoir (RAM, Random Access Memory), magnetic disc or CD.

The above description is merely a specific embodiment, but scope of protection of the present invention is not limited thereto, any Those familiar with the art in the technical scope disclosed by the present invention, can easily think of the change or the replacement, and should all contain Lid is within protection scope of the present invention.Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. a kind of method of image retrieval, which is characterized in that including：

From the first depth characteristic is extracted in image to be retrieved and from the image to be retrieved extract First partial feature；

By the multiple candidate images stored in first depth characteristic and the First partial feature and property data base Second depth characteristic and the second local feature are matched respectively, obtain matched second depth characteristic and the second part is special Sign；

Output image corresponding with matched second depth characteristic and the second local feature.

2. the method for image retrieval according to claim 1, which is characterized in that it is described by first depth characteristic with And the second depth characteristic and the second part of the multiple candidate images stored in the First partial feature and property data base Before feature is matched respectively, further include：

Compressed encoding is carried out to the first depth characteristic of the image to be retrieved by dimension-reduction treatment, obtains the image to be retrieved Third depth characteristic；

It is deep by the 4th of the third depth characteristic of the image to be retrieved and multiple candidate images of the characteristic library storage the Degree feature is matched, and matched 4th depth characteristic and image corresponding with matched 4th depth characteristic are obtained, 4th depth characteristic of wherein the multiple candidate image is the second depth by dimension-reduction treatment to the multiple candidate image Feature carries out what compressed encoding obtained,

Wherein, the multiple times that will be stored in first depth characteristic and the First partial feature and property data base The second depth characteristic and the second local feature for selecting image are matched respectively, including：

It will be special with matched 4th depth in the first depth characteristic of the image to be retrieved and the property data base The second depth characteristic for levying corresponding image is matched；

It will be special with matched 4th depth in the First partial feature of the image to be retrieved and the property data base The second local feature for levying corresponding image is matched.

3. the method for image retrieval according to claim 2, which is characterized in that further include：

The second depth characteristic is extracted from the multiple candidate image；

Compressed encoding is carried out to the second depth characteristic of the multiple candidate image by dimension-reduction treatment, obtains the 4th depth spy Sign, the 4th depth characteristic is stored in the property data base.

4. the method for image retrieval according to any one of claims 1 to 3, which is characterized in that described by described Second depth characteristic of the multiple candidate images stored in one depth characteristic and the First partial feature and property data base And second local feature matched respectively, including：

The similarity for calculating first depth characteristic and the second depth characteristic of the multiple candidate image, obtains multiple candidates First similarity of image；

The similarity for calculating the First partial feature and the second local feature of the multiple candidate image, obtains multiple candidates Second similarity of image；

The first similarity and the second similarity of the multiple candidate image are weighted, ranking operation result is obtained；

Matched second depth characteristic and the second local feature are determined according to the ranking operation result.

5. the method for image retrieval according to any one of claims 1 to 3, which is characterized in that described image is vehicle Image, the method further include：

Vehicle image is collected for each vehicle in multiple vehicles；

Extract the second depth characteristic and the second local feature of the vehicle image；

Second depth characteristic of the vehicle image and the second local feature are stored in the property data base.

6. the method for image retrieval according to claim 5, which is characterized in that each vehicle in multiple vehicles Collect vehicle image, specifically include step：

Obtain Video stream information；

For each vehicle, the multiple figure for collecting dimension is filtered out from the Video stream information according to multiple collection dimensions Picture；

Information of vehicles is obtained from the multiple image for collecting dimension；

Multistage screening is carried out according to the information of vehicles；

The result of the multistage screening is concluded and classified, the vehicle image for each vehicle after the arrangement is obtained.

7. the method for image retrieval according to any one of claims 1 to 3, which is characterized in that described to be retrieved The first depth characteristic is extracted in image and First partial feature is extracted from the image to be retrieved, including：

The first depth characteristic, and profit are extracted from the image to be retrieved in the way of convolutional neural networks or Recognition with Recurrent Neural Network With compact description of visual search, local binary pattern operator, Scale invariant features transform matching algorithm or accelerate steady special Sign algorithmic approach extracts First partial feature from the image to be retrieved, wherein

Loss function includes used in the training process of the convolutional neural networks：Tripletloss and/or Softmaxloss,

The wherein described convolutional neural networks include normalization layer and full articulamentum, and the tripletloss loss functions setting exists After the normalization layer, the softmaxloss loss functions are arranged after the full articulamentum.

8. a kind of device of image retrieval, which is characterized in that including：

Extraction module, the first depth characteristic for being configured to extract in picture to be retrieved, and be configured to extract picture to be retrieved In First partial feature；

Matching module, be configured to match the first depth characteristic in the picture to be retrieved with stored in property data base it is multiple Second depth characteristic of candidate image, and obtain matched second depth characteristic；And match in the picture to be retrieved Second local feature of the multiple candidate images stored in one local feature and property data base, and obtain matched second part Feature；

Output module is configured as output to image corresponding with matched second depth characteristic and the second local feature.

9. the device of image retrieval according to claim 8, which is characterized in that further include：

Compressed encoding module is configured to carry out compression volume to the first depth characteristic of the image to be retrieved by dimension-reduction treatment Code obtains the third depth characteristic of image to be retrieved, and special to the second depth of the multiple candidate image by dimension-reduction treatment Sign carries out compressed encoding and obtains the 4th depth characteristic of image to be retrieved, wherein the matching module matches the picture to be retrieved In third depth characteristic and property data base in the 4th depth characteristic of multiple candidate images that stores, and obtain matched the The corresponding image of four depth characteristics,

Wherein, the matching module concrete configuration is：By the first depth characteristic of the image to be retrieved and the characteristic Second depth characteristic of the image corresponding with matched 4th depth characteristic in library is matched by matching module；

The matching module concrete configuration is to deposit the First partial feature of the image to be retrieved with the property data base Second local feature of multiple candidate images of storage is matched by matching module.

10. the device of the image retrieval according to claim 8 or 9, the matching module concrete configuration are：Described in calculating The similarity of first depth characteristic and the second depth characteristic of the multiple candidate image, obtains the first phase of multiple candidate images Like degree, the similarity of the First partial feature and the second local feature of the multiple candidate image is calculated, multiple times are obtained The second similarity for selecting image is weighted the first similarity and the second similarity of the multiple candidate image, obtains To ranking operation as a result, and determining that matched second depth characteristic and the second part are special according to the ranking operation result Sign.