CN107291825A - With the search method and system of money commodity in a kind of video - Google Patents

With the search method and system of money commodity in a kind of video Download PDF

Info

Publication number
CN107291825A
CN107291825A CN201710382041.5A CN201710382041A CN107291825A CN 107291825 A CN107291825 A CN 107291825A CN 201710382041 A CN201710382041 A CN 201710382041A CN 107291825 A CN107291825 A CN 107291825A
Authority
CN
China
Prior art keywords
commodity
money
video
feature
characteristic
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201710382041.5A
Other languages
Chinese (zh)
Inventor
史培培
王涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing QIYI Century Science and Technology Co Ltd
Original Assignee
Beijing QIYI Century Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing QIYI Century Science and Technology Co Ltd filed Critical Beijing QIYI Century Science and Technology Co Ltd
Priority to CN201710382041.5A priority Critical patent/CN107291825A/en
Publication of CN107291825A publication Critical patent/CN107291825A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/73Querying
    • G06F16/732Query formulation
    • G06F16/7335Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Image Analysis (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention provides the search method and system in a kind of video with money commodity, this method includes:Image to be retrieved is obtained, image to be retrieved includes commodity to be retrieved;The first characteristic of division and first of commodity to be retrieved in image to be retrieved is extracted with money feature;Each commodity that each frame of video for video is included, obtain the second characteristic of division and second of each commodity with money feature;Calculate commodity to be retrieved the first characteristic of division and first with money feature and the second characteristic of division of each commodity and second with the target similarity between money feature;The frame of video that target similarity is more than corresponding to the commodity of predetermined threshold value is defined as to the frame of video of the same money commodity comprising commodity to be retrieved;Wherein, characteristic of division represents the feature classified for different types of commodity, with similar features of the same commodity of money character representation under different external appearance characteristics.The present invention can realize lookup in video to same money commodity, and improve the lookup degree of accuracy.

Description

With the search method and system of money commodity in a kind of video
Technical field
The present invention relates to data retrieval technology field, in more particularly to a kind of video with money commodity search method and be System.
Background technology
With the development of internet, e-commerce website constantly emerges with video website, traditional based on text retrieval skill Art has been difficult to the demand for meeting people, and picture is as perception medium of the people to world around, using picture as basic input, from Information needed for being retrieved in high-volume database has been provided with powerful researching value.
It is desirable to find the commodity of money identical with the commodity in picture in video by way of picture, but it is due to The number of video frames of video is huge, if being difficult to find identical commodity, work therein by way of manually searching Amount is also huge.
Therefore, same money commodity can be searched in correlation technique in video using traditional image retrieval algorithm, still, should Method carries out characteristic matching using the color of image, texture, shape, edge or feature point feature mostly.For example, Scale invariant Eigentransformation (Scale-invariant feature transform, SIFT) method, but this kind of searching algorithm has spy Levy a little more, the problem of characteristic storage amount is larger, and the same money commodity included in the video frame figure exist posture, angle, During the great varieties such as contrast, illumination, the recognition accuracy of the algorithm will be substantially reduced.
Therefore, correlation technique also proposed the method that image retrieval is carried out using depth characteristic, this method is compared to above-mentioned The degree of accuracy of conventional retrieval method improves, but is due to the size of same commodity in video to be found, appearance The change such as state, angle is very big, and background is also more complicated, so that it is to be checked to cause this method still can not be found from video The same money commodity of the commodity of rope, the degree of accuracy is relatively low.
As can be seen here, the image search method in correlation technique there is no method to realize and same money commodity searched from video, search The degree of accuracy it is relatively low.
The content of the invention
The invention provides the search method and system in a kind of video with money commodity, to solve the image in correlation technique Search method can not be realized searches same money commodity, the problem of degree of accuracy of lookup is relatively low from video.
In order to solve the above problems, according to an aspect of the present invention, the invention discloses in a kind of video with money commodity Search method, including:
Image to be retrieved is obtained, the image to be retrieved includes commodity to be retrieved;
The first characteristic of division of commodity to be retrieved and first described in the image to be retrieved is extracted with money feature;
Each commodity that each frame of video for the video is included, obtain the second characteristic of division of each commodity With second with money feature;
The first characteristic of division and first of the commodity to be retrieved is calculated with second point of money feature and each commodity Category feature and second is with the target similarity between money feature;
The frame of video that the target similarity is more than corresponding to the commodity of predetermined threshold value is defined as comprising described to be retrieved The frame of video of the same money commodity of commodity;
Wherein, characteristic of division represents the feature classified for different types of commodity, same with money character representation Similar features of the commodity under different external appearance characteristics.
Alternatively, first characteristic of division and first for calculating the commodity to be retrieved is with money feature and each business Second characteristic of division of product and second with the target similarity between money feature, including:
By first characteristic of division and described first with the connection of money feature, the first classification is obtained with money feature;
By second characteristic of division and described second with the connection of money feature, the second classification is obtained with money feature;
First classification is calculated with money feature and second classification with the similarity of money feature, target is obtained similar Degree.
Alternatively, first characteristic of division and first for calculating the commodity to be retrieved is with money feature and each business Second characteristic of division of product and second with the target similarity between money feature, including:
Calculate between the first characteristic of division of the commodity to be retrieved and the second characteristic of division of each commodity One similarity, and the first of the commodity to be retrieved is calculated with money feature and each commodity second between money feature The second similarity;
Weighted mean operation is made to first similarity and second similarity using the weight parameter pre-set, Obtain target similarity.
Alternatively, first characteristic of division and the first same money for extracting commodity to be retrieved described in the image to be retrieved The step of feature, including:
The image to be retrieved is inputted to advance trained disaggregated model and with money model;
Feature extraction is carried out to the image to be retrieved respectively using the disaggregated model and the same money model, institute is obtained The first characteristic of division and described first is stated with money feature;
Wherein, the disaggregated model is that obtained convolutional neural networks CNN models are trained using categorized data set;
The same money model is the CNN models got using the same amount of money constructed in advance according to training;
Wherein, the same money data set constructed in advance includes multiclass image, is included per class image and is directed to same commodity Different external appearance characteristics multiple pictures.
Alternatively, each commodity that each frame of video for the video is included, obtain each commodity Second characteristic of division and second with money feature the step of, including:
Each commodity that each frame of video for the video is included, extract described each from each frame of video Second characteristic of division of commodity and second is with money feature;Or,
Each commodity that each frame of video for the video is included, obtain the every of the video from the first database Second characteristic of division of each commodity that one frame of video is included and second with money feature, wherein, in first database store There is the second characteristic of division and second of each commodity extracted in advance from each frame of video of the video with money feature.
Alternatively, described to obtain image to be retrieved, the image to be retrieved is included before commodity to be retrieved, and this method is also wrapped Include:
Each frame of video of the video is obtained, and is stored to the second database;
To each frame of video stored in second database, each commodity that each frame of video is included are extracted The second characteristic of division and second with money feature, and store to first database.
According to another aspect of the present invention, the invention also discloses the searching system in a kind of video with money commodity, including:
Image collection module, for obtaining image to be retrieved, the image to be retrieved includes commodity to be retrieved;
First extraction module, for extracting the first characteristic of division of commodity to be retrieved described in the image to be retrieved and Together money feature;
Feature acquisition module, each commodity included for each frame of video for the video are obtained described each Second characteristic of division of commodity and second is with money feature;
Computing module, for calculate the commodity to be retrieved the first characteristic of division and first with money feature with it is described each Second characteristic of division of commodity and second is with the target similarity between money feature;
Determining module, bag is defined as the target similarity to be more than to the frame of video corresponding to the commodity of predetermined threshold value The frame of video of same money commodity containing the commodity to be retrieved;
Wherein, characteristic of division represents the feature classified for different types of commodity, same with money character representation Similar features of the commodity under different external appearance characteristics.
Alternatively, the computing module includes:
First connection submodule, for first characteristic of division and described first to be connected with money feature, obtains first Classification is with money feature;
Second connects submodule, for second characteristic of division and described second to be connected with money feature, obtains second point Similar money feature;
First calculating sub module, classifies with the phase of money feature for calculating first classification with money feature and described second Like spending, target similarity is obtained.
Alternatively, the computing module also includes:
Second calculating sub module, for calculating the first characteristic of division of the commodity to be retrieved and each commodity The first similarity between two characteristic of divisions;
3rd calculating sub module, for calculating the first of the commodity to be retrieved with the of money feature and each commodity Two with the second similarity between money feature;
4th calculating sub module, for using the weight parameter pre-set to first similarity and second phase Make weighted mean operation like degree, obtain target similarity.
Alternatively, first extraction module includes:
Input submodule, for the image to be retrieved to be inputted to advance trained disaggregated model and with money mould Type;
First extracting sub-module, for using the disaggregated model and the same money model respectively to the image to be retrieved Feature extraction is carried out, first characteristic of division and described first is obtained with money feature;
Wherein, the disaggregated model is that obtained CNN models are trained using categorized data set;
The same money model is the CNN models got using the same amount of money constructed in advance according to training;
Wherein, the same money data set constructed in advance includes multiclass image, is included per class image and is directed to same commodity Different external appearance characteristics multiple pictures.
Alternatively, the feature acquisition module includes:
Second extracting sub-module, each commodity included for each frame of video for the video, from described each The second characteristic of division and second of each commodity is extracted in frame of video with money feature;Or,
Acquisition submodule, each commodity included for each frame of video for the video, from the first database The second characteristic of division and second for each commodity that each frame of video of the video is included is obtained with money feature, wherein, it is described Be stored with first database the second characteristic of division of each commodity for being extracted in advance from each frame of video of the video and Second with money feature.
Alternatively, the system also includes:
Acquisition module, for obtaining each frame of video of the video, and is stored to the second database;
Second extraction module, for each frame of video to being stored in second database, extracts each video Second characteristic of division of each commodity that frame is included and second is stored to first database with money feature.
Compared with prior art, the present invention includes advantages below:
The present invention is by obtaining the characteristic of division of the commodity to be retrieved in image to be retrieved and with money feature and video The characteristic of division for each commodity that each frame of video is included and with money feature, and calculate the commodity to be retrieved in image to be retrieved Feature and video in each commodity feature between target similarity, feature here is related to characteristic of division and special with money Levy, thus find in video by target similarity be more than predetermined threshold value commodity corresponding to frame of video, realize regarding Same money commodity are searched in frequency, and improve the lookup degree of accuracy.
In addition, the present invention with same money feature by the characteristic of division of commodity by being first connected processing, then, then to contrast business Product each connection processing after two features carry out similarity comparison calculate, obtain target similarity.Can be similar in progress Degree is when comparing, combining classification feature and with money feature, and the data compared of similarity can be made more comprehensive.
In addition, the present invention is similar by first being carried out respectively according to the type of feature to the different types of feature for contrasting commodity Degree is calculated, then, then the similarity on same money feature, the similarity on characteristic of division that are obtained to calculating make weighted average Computing, so as to obtain target similarity.Due to being individually compared to different types of feature, therefore, with the retrieval of money commodity Accuracy rate it is higher.
Brief description of the drawings
Fig. 1 be the present invention a kind of video in money commodity search method embodiment step flow chart;
Fig. 2 be the present invention a kind of video in money commodity searching system embodiment block diagram;
Fig. 3 be the present invention another video in money commodity searching system embodiment structured flowchart.
Embodiment
In order to facilitate the understanding of the purposes, features and advantages of the present invention, it is below in conjunction with the accompanying drawings and specific real Applying mode, the present invention is further detailed explanation.
Reference picture 1, show the present invention a kind of video in money commodity search method embodiment step flow chart, Specifically it may include steps of:
Step 101, image to be retrieved is obtained, the image to be retrieved includes commodity to be retrieved;
Commodity to be retrieved (such as certain cap is included in image to be retrieved (i.e. one picture), the picture wherein it is possible to obtain Son), that is, need to search the frame of video with the cap in picture with money in some video.
Step 102, the first characteristic of division of commodity to be retrieved described in the image to be retrieved and first is extracted with money spy Levy;
Wherein it is possible to extract the characteristic of division of certain cap in the picture and with money feature.
Step 103, each commodity that each frame of video for the video is included, obtain the second of each commodity Characteristic of division and second is with money feature;
Likewise, also needing to each commodity included to each frame of video of video to be checked, each commodity are obtained Characteristic of division and with money feature.
Wherein, characteristic of division represents the feature classified for different types of commodity, i.e. characteristic of division can be distinguished Which kind of classification the commodity belong to, and the similar features with the same commodity of money character representation under different external appearance characteristics, that is, are directed to Multiple pictures of same commodity, and cause in each picture because the posture of commodity, angle, contrast, illumination etc. are different The external feature of commodity is not fully identical, but still there are similar features under different external appearance characteristics for same commodity , thus in order to find same money commodity, it is necessary in obtaining the frame figure of frame of video a certain commodity similar features and extraction figure The similar features of commodity to be retrieved in piece.
Step 104, the first characteristic of division and first of the commodity to be retrieved is calculated with money feature and each commodity The second characteristic of division and second with the target similarity between money feature;
Wherein it is possible to calculate one group of feature (including the first characteristic of division and first is with money feature) and the institute of commodity to be retrieved State the target similarity between one group of feature (including the second characteristic of division and second is with money feature) of each commodity.
Step 105, the corresponding frame of video of commodity that target similarity is more than predetermined threshold value is defined as comprising described to be checked The frame of video of the same money commodity of rope commodity.
Finally, the corresponding frame of video of commodity that target similarity is more than predetermined threshold value is defined as comprising commodity to be retrieved With the frame of video of money commodity.
By means of the technical scheme of the above embodiment of the present invention, the present invention is by obtaining the business to be retrieved in image to be retrieved The characteristic of division of product with the characteristic of division for each commodity that each frame of video is included in money feature and video and with money spy Levy, and it is similar to the target between the feature of each commodity in video to calculate the feature of commodity to be retrieved in image to be retrieved Degree, feature here is related to characteristic of division and with money feature, and target similarity is more than into default threshold so as to find in video Frame of video corresponding to the commodity of value, realizes and searches same money commodity in video, and improve the lookup degree of accuracy.
Alternatively, in one embodiment, when performing step 102, it can be realized using following scheme:Treated described Retrieval image is inputted to advance trained disaggregated model and with money model;Using the disaggregated model and the same money model Feature extraction is carried out to the image to be retrieved respectively, first characteristic of division and described first is obtained with money feature.
Wherein, the disaggregated model is that obtained CNN (convolutional neural networks) model is trained using categorized data set;It is described It is the CNN models got using the same amount of money constructed in advance according to training with money model;Wherein, categorized data set training is being utilized During CNN models, it is possible to use conventional network structure, such as vgg, googlenet, resnet etc..The embodiment of the present invention is used Be inception21k models.And vgg, googlenet can also be utilized when using with amount of money according to collection training CNN models, Resnet etc..The embodiment of the present invention uses googlenet models.
Wherein, found by many experiments of inventor, classification CNN disaggregated models and CNN use different with money model Network structure is higher compared to the degree of accuracy for the feature extracted using identical network structure.
Wherein, categorized data set includes multiclass image, and what it is per class iamge description is all same type of things (such as bird Class, aircraft class, cat class etc., wherein, the image of bird of the birds image comprising multiple kinds), the other feature of class of things emphatically.Its In, categorized data set can utilize disclosed data set, such as Imagenet data sets (image for including 1000 classifications).
Likewise, with money data set also include multiclass image, but its per class image include both for same business Multiple pictures of the different external appearance characteristics of product (such as certain cap, necklace, dress ornament etc.), wherein, with amount of money according to concentration for every Performance characteristic, the external appearance characteristic of the picture of individual commodity change greatly, so may insure that the Characterizations ability of same money feature is accurate. Wherein, with money data set voluntarily constructed according to the demand of portraying of same money feature by inventor.
Alternatively, in one embodiment, when performing step 103, it can be realized by any one following mode:
Mode A:Each commodity that each frame of video for the video is included, institute is extracted from each frame of video The second characteristic of division and second of each commodity is stated with money feature;
Wherein, the feature extraction of each commodity included for each frame of video in video can equally use above-mentioned CNN disaggregated models and CNN in embodiment is with money model.
Wherein, the feature extraction mode of mode one ensure that accessed is characterized in that state is newest.
Or, mode B:Each commodity that each frame of video for the video is included, obtain institute from the first database The second characteristic of division and second for each commodity that each frame of video of video is included is stated with money feature, wherein, first number It is same according to the second characteristic of division and second for the advance each commodity extracted from each frame of video of the video that are stored with storehouse Money feature.
I.e. mode B uses the process extracted in advance, extracts stored after each feature to corresponding first data in advance Storehouse (i.e. feature database), mode B feature extraction mode can reduce the retrieval time of same money commodity, lift recall precision.
Alternatively, in one embodiment, before step 101 is performed, method according to embodiments of the present invention can be with Including:Each frame of video of the video is obtained, and is stored to the second database;It is each to what is stored in second database Frame of video, extracts the second characteristic of division and second for each commodity that each frame of video is included with money feature, and store To first database.
That is, the embodiment of the present invention can obtain each frame of video (i.e. two field picture) in video in advance, and by video Each frame of video all preserve to the second database (composition image library);Each frame of video (i.e. two field picture) in image library is directed to again Included in each commodity extract and (cried here with money feature (being called second here with money feature) and characteristic of division accordingly Make the second characteristic of division) store to above-mentioned first database (so that constitutive characteristic storehouse).
So, the embodiment of the present invention can be pre-created the image library (the second database) of video, and with image library Comprising the corresponding feature database of each commodity (the first database), so, it is possible to use each video of database realizing video The same money feature of each commodity included in frame, the extraction in advance of characteristic of division, lift image retrieval efficiency.
In addition, in one embodiment, when performing step 104, two can be realized using following optional mode one Target Similarity Measure between individual commodity:By first characteristic of division and described first with the connection of money feature, first is obtained Classification is with money feature;By second characteristic of division and described second with the connection of money feature, the second classification is obtained with money feature;Meter First classification is calculated with money feature and second classification with the similarity of money feature, target similarity is obtained.
In addition, in another embodiment, when performing step 104, can also be realized using following optional mode two Target Similarity Measure between two commodity:Calculate the first characteristic of division and each commodity of the commodity to be retrieved The first similarity between second characteristic of division, and the first of the commodity to be retrieved is calculated with money feature and each business The second of product is with the second similarity between money feature;Using the weight parameter pre-set to first similarity and described Second similarity makees weighted mean operation, obtains target similarity.
Wherein, in mode one, the characteristic of division of commodity first can be connected processing (i.e. characteristic vector with same money feature Between connection processing), then, then to contrast commodity each connection processing after two features carry out similarity-rough set calculating, Obtain target similarity.
In mode two, then according to the type of feature similarity is first carried out respectively to the different types of feature for contrasting commodity (that is, the same money feature calculation similarity that commodity A and commodity B characteristic of division calculate similarity, commodity A and commodity B) is calculated, so Afterwards, then to calculating the similarity on same money feature, the similarity on characteristic of division obtained makees weighted mean operation, so that Obtain target similarity.Wherein, the embodiment of the present invention can pre-set the weighted value of the similarity of characteristic of division, with money feature Between similarity weighted value.
So, the embodiment of the present invention can be required according to the retrieval accuracy of same money commodity, target in different ways Similarity Measure scheme (that is, aforesaid way one or mode two), wherein, mode two is compared to mode one, due to being to inhomogeneity The feature of type is individually compared, therefore, and the accuracy rate of retrieval is higher;And mode one goes for inspection compared to mode two In the relatively low scene of the rope degree of accuracy, specifically it can be adjusted flexibly and be selected according to actual scene.
In addition, above-mentioned by the embodiment of data storage to the first database and the second database, if current scene Need the calculation of progress aforesaid way one, then, can be by each frame of video institute in video when to data database storing Comprising the same money feature extracted of each commodity and characteristic of division, make feature connection processing, and the same money after connection is handled Characteristic of division is stored to the first database (i.e. feature database).So, same money characteristic of division can be stored into database in advance, Lift recall precision.
And if current scene needs to carry out the calculation of aforesaid way two, then, can when to data database storing Stored respectively to two with the same money feature and characteristic of division of being extracted each commodity that each frame of video is included in video First database (i.e. one with money property data base, a characteristic classification data storehouse).So, it is easy to independently obtaining for two category features Take, so that when realizing that the first characteristic of division and the second characteristic of division carry out Similarity Measure, the first same money can also be carried out parallel Feature and the second Similarity Measure with money feature, lift recall precision.
The above method is briefly described with reference to the searching system shown in Fig. 2.
First, input inquiry image module 21 obtains image I to be checked, and inputs to the He of characteristic of division extraction module 22 With money characteristic extracting module 23;
Then, characteristic of division extraction module 22 extracts image I characteristic of division fc using CNN disaggregated models and inputted to inspection Rope module 24, characteristic of division fc dimension is 1024 dimensions here;With money characteristic extracting module 23 using CNN with money model extraction Image I same money feature fs is simultaneously inputted to retrieval module 24, and characteristic of division fs dimension is 1024 dimensions here;
Then, retrieval module 24 makees normalized respectively to image I characteristic of division fc and with money feature fs, is returned The one characteristic of division fnc and normalized same money feature fns changed;
Then, retrieval module 24 can use different retrieval sides according to the constituted mode difference of feature in feature database 27 Formula, specifically:
On the one hand, if stored in feature database 27 in above-described embodiment by each image in image library 26 The same money feature and characteristic of division of each commodity are attached, and obtained classification (wherein, is also passed through before proceeding with money feature Cross above-mentioned normalized), then retrieval mode one will be used:
Module 24 is retrieved to concatenate and (connect) normalized characteristic of division fnc and normalized same money feature fns to one Rise, obtain normalized same money characteristic of division fcon=(fnc, fns), fcon dimension is 2048 dimensions;
Then, retrieval module 24 is by the every of normalized same money characteristic of division fcon and each frame of video in feature database 27 Same money characteristic of division after each concatenation of individual commodity carries out target Similarity Measure, determines that the target similarity is more than default The target two field picture of threshold value;Or, according to the order of target similarity from big to small to each in each frame of video for retrieving The same money characteristic of division of commodity is ranked up, finally, and the mesh of predetermined quantity is found in image library 26 corresponding with feature database 27 Mark the larger multiple target two field pictures of similarity;
Then, the time point further according to target two field picture finds the frame of video hair of corresponding video time point in video Deliver to result output module 25;
Finally, as a result output module 25 by comprising defeated with the frame of video of the commodity of money with the commodity in image I to be checked Go out.
Wherein, between above-mentioned calculating feature similarity (whether similarity or the first similarity, second similar Degree) when, can be determined by calculating the modes such as Euclidean distance, COS distance, coefficient correlation between feature two features it Between similarity degree.
Wherein, calculate that to obtain Euclidean distance smaller, similarity degree is bigger between representing two features;Calculate and obtain remaining Chordal distance is smaller, and similarity degree is bigger between representing two features;Calculate that to obtain coefficient correlation bigger, represent two features it Between similarity degree it is bigger.
Wherein, the embodiment of the present invention carries out the explanation of Similarity Measure exemplified by calculating coefficient correlation, and for it is European away from Computational methods from, COS distance are prior art, and those skilled in the art is referred to carry out the calculating of similarity.
In addition, it should be noted that when the calculation of the similarity used is different, the size of corresponding predetermined threshold value Also it is different, i.e. the size of the predetermined threshold value can be adjusted flexibly according to the calculation of currently employed similarity, be made Match with corresponding calculation.
On the other hand, if system includes two feature databases, store in image library 26 respectively in above-described embodiment Each image in each commodity same money feature and characteristic of division, i.e. feature database 1 store each frame of video in video Comprising each commodity characteristic of division, and feature database 2 then stores each business that each frame of video is included in the video The same money feature (wherein, also going through above-mentioned normalized before storing) of product, then retrieval mode two will be used:
Retrieve module 24 by image I normalized characteristic of division fnc and normalized same money feature fns respectively with two Each normalized characteristic of division and normalized same money feature in feature database carry out Similarity Measure (wherein, image I point Category feature is calculated with the characteristic of division in feature database, and image I same money feature and the same money feature in feature database are calculated), Obtain characteristic of division similarity dc and with money characteristic similarity ds;Then, using default classified weight ω 1 and default same money Weights omega 2 obtains unified similar to characteristic of division similarity dc and with money characteristic similarity ds weighted averages (as shown in Equation 1) Spend dw;
Dw=(ω 1*dc+ ω 2*ds)/(ω 1+ ω 2) formula (1)
Wherein, the weights omega 1 (such as 0.3) and (example of weights omega 2 with money characteristic similarity of above-mentioned characteristic of division similarity Such as, concrete numerical value 0.7) is the degree of accuracy highest numerical value of the retrieval that is determined by inventor's many experiments with money commodity. Wherein, ω 1 and the sums of ω 2 are 1, and according to the difference of actual scene, two weights can also be adjusted, but need to ensure it And for 1.
Then, retrieval module 24 arranges the obtained unified similarity dw for each commodity according to order from big to small Sequence;The target two field picture of similarity maximum of predetermined quantity is found in an image library 26 corresponding with two feature databases (i.e. The characteristic of division of target two field picture and be respectively stored in money feature in above-mentioned database 1 and database 2);Or retrieval module 24 determine that the similarity is more than the target two field picture of predetermined threshold value in image library 26;
Then, the time point further according to target two field picture finds the frame of video hair of corresponding video time point in video Deliver to result output module 25;
Finally, as a result output module 25 by comprising defeated with the frame of video of the commodity of money with the commodity in image I to be checked Go out.
Calculate that to obtain Euclidean distance smaller, similarity degree is bigger between representing two features;Calculate obtain cosine away from From smaller, similarity degree is bigger between representing two features;Calculate that to obtain coefficient correlation bigger, represent phase between two features It is bigger like degree.
Wherein, the embodiment of the present invention carries out the explanation of Similarity Measure exemplified by calculating coefficient correlation, and for it is European away from Computational methods from, COS distance are prior art, and those skilled in the art is referred to carry out the calculating of similarity.
In addition, it should be noted that when the calculation of the similarity used is different, the size of corresponding predetermined threshold value Also it is different, i.e. the size of the predetermined threshold value can be adjusted flexibly according to the calculation of currently employed similarity, be made Match with corresponding calculation.
If for example, using the computational methods of Euclidean distance or COS distance, the distance obtained according to calculating is entered Row takes the target two field picture of the minimum predetermined quantity of distance from the sequence of small arrival.
To sum up, the embodiment of the present invention utilizes deep learning Algorithm for Training CNN with money model and CNN disaggregated models, and will Two kinds of model extractions to feature merge, and retrieval result is ranked up according to similarity measurement criterion, finds matching Commodity, substantially increase the accuracy rate of retrieval.Also, because same money is characterized in that one kind more becomes more meticulous, it is better able to description figure As a kind of feature of similitude.Therefore will be blended with money feature and characteristic of division, using the feature after fusion as retrieval according to According to, so that in the case that the change such as the size of same commodity, posture, angle is very big in video, and background is also more complicated, Also same money commodity can be accurately retrieved in video, improve the degree of accuracy of same money commodity.
It should be noted that for embodiment of the method, in order to be briefly described, therefore it to be all expressed as to a series of action group Close, but those skilled in the art should know, the embodiment of the present invention is not limited by described sequence of movement, because according to According to the embodiment of the present invention, some steps can be carried out sequentially or simultaneously using other.Secondly, those skilled in the art also should Know, embodiment described in this description belongs to preferred embodiment, the involved action not necessarily present invention is implemented Necessary to example.
Corresponding with the method that the embodiments of the present invention are provided, reference picture 3 is shown same in a kind of video of the invention The structured flowchart of the searching system embodiment of money commodity, can specifically include following module:
Image collection module 31, for obtaining image to be retrieved, the image to be retrieved includes commodity to be retrieved;
First extraction module 32, for extract the first characteristic of division of commodity to be retrieved described in the image to be retrieved and First with money feature;
Feature acquisition module 33, each commodity included for each frame of video for the video obtain described every Second characteristic of division of one commodity and second is with money feature;
Computing module 34, for calculate the commodity to be retrieved the first characteristic of division and first with money feature with it is described every Second characteristic of division of one commodity and second is with the target similarity between money feature;
Determining module 35, is defined as the target similarity to be more than to the frame of video corresponding to the commodity of predetermined threshold value The frame of video of same money commodity comprising the commodity to be retrieved;
Wherein, characteristic of division represents the feature classified for different types of commodity, same with money character representation Similar features of the commodity under different external appearance characteristics.
Alternatively, the computing module 34 includes:
First connection submodule, for first characteristic of division and described first to be connected with money feature, obtains first Classification is with money feature;
Second connects submodule, for second characteristic of division and described second to be connected with money feature, obtains second point Similar money feature;
First calculating sub module, classifies with the phase of money feature for calculating first classification with money feature and described second Like spending, target similarity is obtained.
Alternatively, the computing module 34 also includes:
Second calculating sub module, for calculating the first characteristic of division of the commodity to be retrieved and each commodity The first similarity between two characteristic of divisions;
3rd calculating sub module, for calculating the first of the commodity to be retrieved with the of money feature and each commodity Two with the second similarity between money feature;
4th calculating sub module, for using the weight parameter pre-set to first similarity and second phase Make weighted mean operation like degree, obtain target similarity.
Alternatively, first extraction module 32 includes:
Input submodule, for the image to be retrieved to be inputted to advance trained disaggregated model and with money mould Type;
First extracting sub-module, for using the disaggregated model and the same money model respectively to the image to be retrieved Feature extraction is carried out, first characteristic of division and described first is obtained with money feature;
Wherein, the disaggregated model is that obtained CNN models are trained using categorized data set;
The same money model is the CNN models got using the same amount of money constructed in advance according to training;
Wherein, the same money data set constructed in advance includes multiclass image, is included per class image and is directed to same commodity Different external appearance characteristics multiple pictures.
Alternatively, the feature acquisition module 33 includes:
Second extracting sub-module, each commodity included for each frame of video for the video, from described each The second characteristic of division and second of each commodity is extracted in frame of video with money feature;Or,
Acquisition submodule, each commodity included for each frame of video for the video, from the first database The second characteristic of division and second for each commodity that each frame of video of the video is included is obtained with money feature, wherein, it is described Be stored with first database the second characteristic of division of each commodity for being extracted in advance from each frame of video of the video and Second with money feature.
Alternatively, the system also includes:
Acquisition module, for obtaining each frame of video of the video, and is stored to the second database;
Second extraction module, for each frame of video to being stored in second database, extracts each video Second characteristic of division of each commodity that frame is included and second is stored to first database with money feature.
For system embodiment, because it is substantially similar to embodiment of the method, so description is fairly simple, it is related Part illustrates referring to the part of embodiment of the method.
Each embodiment in this specification is described by the way of progressive, what each embodiment was stressed be with Between the difference of other embodiment, each embodiment identical similar part mutually referring to.
It should be understood by those skilled in the art that, the embodiment of the embodiment of the present invention can be provided as method, device or calculate Machine program product.Therefore, the embodiment of the present invention can using complete hardware embodiment, complete software embodiment or combine software and The form of the embodiment of hardware aspect.Moreover, the embodiment of the present invention can use it is one or more wherein include computer can With in the computer-usable storage medium (including but is not limited to magnetic disk storage, CD-ROM, optical memory etc.) of program code The form of the computer program product of implementation.
The embodiment of the present invention is with reference to method according to embodiments of the present invention, terminal device (system) and computer program The flow chart and/or block diagram of product is described.It should be understood that can be by computer program instructions implementation process figure and/or block diagram In each flow and/or square frame and the flow in flow chart and/or block diagram and/or the combination of square frame.These can be provided Computer program instructions are set to all-purpose computer, special-purpose computer, Embedded Processor or other programmable data processing terminals Standby processor is to produce a machine so that held by the processor of computer or other programmable data processing terminal equipments Capable instruction is produced for realizing in one flow of flow chart or multiple flows and/or one square frame of block diagram or multiple square frames The device for the function of specifying.
These computer program instructions, which may be alternatively stored in, can guide computer or other programmable data processing terminal equipments In the computer-readable memory worked in a specific way so that the instruction being stored in the computer-readable memory produces bag The manufacture of command device is included, the command device is realized in one flow of flow chart or multiple flows and/or one side of block diagram The function of being specified in frame or multiple square frames.
These computer program instructions can be also loaded into computer or other programmable data processing terminal equipments so that Series of operation steps is performed on computer or other programmable terminal equipments to produce computer implemented processing, so that The instruction performed on computer or other programmable terminal equipments is provided for realizing in one flow of flow chart or multiple flows And/or specified in one square frame of block diagram or multiple square frames function the step of.
Although having been described for the preferred embodiment of the embodiment of the present invention, those skilled in the art once know base This creative concept, then can make other change and modification to these embodiments.So, appended claims are intended to be construed to Including preferred embodiment and fall into having altered and changing for range of embodiment of the invention.
Finally, in addition it is also necessary to explanation, herein, such as first and second or the like relational terms be used merely to by One entity or operation make a distinction with another entity or operation, and not necessarily require or imply these entities or operation Between there is any this actual relation or order.Moreover, term " comprising ", "comprising" or its any other variant meaning Covering including for nonexcludability, so that process, method, article or terminal device including a series of key elements are not only wrapped Those key elements, but also other key elements including being not expressly set out are included, or also include being this process, method, article Or the intrinsic c key elements of terminal device.In the absence of more restrictions, by wanting that sentence "including a ..." is limited Element, it is not excluded that also there is other identical element in the process including the key element, method, article or terminal device.
Above in a kind of video provided by the present invention with the search method and a kind of video of money commodity with money commodity Searching system, be described in detail, specific case used herein is carried out to the principle and embodiment of the present invention Illustrate, the explanation of above example is only intended to the method and its core concept for helping to understand the present invention;Simultaneously for this area Those skilled in the art, according to the present invention thought, will change in specific embodiments and applications, to sum up Described, this specification content should not be construed as limiting the invention.

Claims (12)

1. with the search method of money commodity in a kind of video, it is characterised in that including:
Image to be retrieved is obtained, the image to be retrieved includes commodity to be retrieved;
The first characteristic of division of commodity to be retrieved and first described in the image to be retrieved is extracted with money feature;
Each commodity that each frame of video for the video is included, obtain the second characteristic of division and the of each commodity Two with money feature;
The first characteristic of division and first of the commodity to be retrieved is calculated with money feature and the second classification spy of each commodity Second is sought peace with the target similarity between money feature;
The frame of video that the target similarity is more than corresponding to the commodity of predetermined threshold value is defined as to include the commodity to be retrieved Same money commodity frame of video;
Wherein, characteristic of division represents the feature classified for different types of commodity, with the same commodity of money character representation Similar features under different external appearance characteristics.
2. according to the method described in claim 1, it is characterised in that first characteristic of division for calculating the commodity to be retrieved And first with the second characteristic of division of money feature and each commodity and second with the target similarity between money feature, bag Include:
By first characteristic of division and described first with the connection of money feature, the first classification is obtained with money feature;
By second characteristic of division and described second with the connection of money feature, the second classification is obtained with money feature;
First classification is calculated with money feature and second classification with the similarity of money feature, target similarity is obtained.
3. according to the method described in claim 1, it is characterised in that first characteristic of division for calculating the commodity to be retrieved And first with the second characteristic of division of money feature and each commodity and second with the target similarity between money feature, bag Include:
Calculate the first phase between the first characteristic of division of the commodity to be retrieved and the second characteristic of division of each commodity Like spending, and the first of the commodity to be retrieved is calculated with money feature and each commodity second with the between money feature Two similarities;
Weighted mean operation is made to first similarity and second similarity using the weight parameter pre-set, obtained Target similarity.
4. according to the method described in claim 1, it is characterised in that described to extract business to be retrieved described in the image to be retrieved First characteristic of division of product and first with money feature the step of, including:
The image to be retrieved is inputted to advance trained disaggregated model and with money model;
Feature extraction is carried out to the image to be retrieved respectively using the disaggregated model and the same money model, described the is obtained One characteristic of division and described first is with money feature;
Wherein, the disaggregated model is that obtained convolutional neural networks CNN models are trained using categorized data set;
The same money model is the CNN models got using the same amount of money constructed in advance according to training;
Wherein, the same money data set constructed in advance includes multiclass image, is included per class image for same commodity not With multiple pictures of external appearance characteristic.
5. according to the method described in claim 1, it is characterised in that it is every that each frame of video for the video is included One commodity, obtain each commodity the second characteristic of division and second with money feature the step of, including:
Each commodity that each frame of video for the video is included, extract each commodity from each frame of video The second characteristic of division and second with money feature;Or,
Each commodity that each frame of video for the video is included, each of the video is obtained from the first database and is regarded Second characteristic of division of each commodity that frequency frame is included and second with money feature, wherein, be stored with first database pre- The second characteristic of division and second of each commodity first extracted from each frame of video of the video is with money feature.
6. method according to claim 5, it is characterised in that the acquisition image to be retrieved, the image bag to be retrieved Before commodity to be retrieved, methods described also includes:
Each frame of video of the video is obtained, and is stored to the second database;
To each frame of video stored in second database, the of each commodity that each frame of video is included is extracted Two characteristic of divisions and second are stored to first database with money feature.
7. with the searching system of money commodity in a kind of video, it is characterised in that including:
Image collection module, for obtaining image to be retrieved, the image to be retrieved includes commodity to be retrieved;
First extraction module, for extracting the first characteristic of division of commodity to be retrieved described in the image to be retrieved and first same Money feature;
Feature acquisition module, each commodity included for each frame of video for the video obtain each commodity The second characteristic of division and second with money feature;
Computing module, for calculating the first characteristic of division and first of the commodity to be retrieved with money feature and each commodity The second characteristic of division and second with the target similarity between money feature;
Determining module, is defined as including institute for the target similarity to be more than to the frame of video corresponding to the commodity of predetermined threshold value State the frame of video of the same money commodity of commodity to be retrieved;
Wherein, characteristic of division represents the feature classified for different types of commodity, with the same commodity of money character representation Similar features under different external appearance characteristics.
8. system according to claim 7, it is characterised in that the computing module includes:
First connection submodule, for first characteristic of division and described first to be connected with money feature, obtains the first classification With money feature;
Second connects submodule, for second characteristic of division and described second to be connected with money feature, obtains the second classification same Money feature;
First calculating sub module, for calculating first classification with money feature and second classification with the similar of money feature Degree, obtains target similarity.
9. system according to claim 7, it is characterised in that the computing module also includes:
Second calculating sub module, second point for calculating the first characteristic of division of the commodity to be retrieved and each commodity The first similarity between category feature;
3rd calculating sub module, for calculating the first of the commodity to be retrieved with the second same of money feature and each commodity The second similarity between money feature;
4th calculating sub module, for using the weight parameter pre-set to first similarity and second similarity Make weighted mean operation, obtain target similarity.
10. system according to claim 7, it is characterised in that first extraction module includes:
Input submodule, for the image to be retrieved to be inputted to advance trained disaggregated model and with money model;
First extracting sub-module, for being carried out respectively to the image to be retrieved using the disaggregated model and the same money model Feature extraction, obtains first characteristic of division and described first with money feature;
Wherein, the disaggregated model is that obtained CNN models are trained using categorized data set;
The same money model is the CNN models got using the same amount of money constructed in advance according to training;
Wherein, the same money data set constructed in advance includes multiclass image, is included per class image for same commodity not With multiple pictures of external appearance characteristic.
11. system according to claim 7, it is characterised in that the feature acquisition module includes:
Second extracting sub-module, each commodity included for each frame of video for the video, from each video The second characteristic of division and second of each commodity is extracted in frame with money feature;Or,
Acquisition submodule, each commodity included for each frame of video for the video are obtained from the first database Second characteristic of division of each commodity that each frame of video of the video is included and second with money feature, wherein, described first Be stored with the second characteristic of division and second of each commodity extracted in advance from each frame of video of the video in database With money feature.
12. system according to claim 11, it is characterised in that the system also includes:
Acquisition module, for obtaining each frame of video of the video, and is stored to the second database;
Second extraction module, for each frame of video to being stored in second database, extracts each frame of video institute Comprising each commodity the second characteristic of division and second with money feature, and store to first database.
CN201710382041.5A 2017-05-26 2017-05-26 With the search method and system of money commodity in a kind of video Pending CN107291825A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710382041.5A CN107291825A (en) 2017-05-26 2017-05-26 With the search method and system of money commodity in a kind of video

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710382041.5A CN107291825A (en) 2017-05-26 2017-05-26 With the search method and system of money commodity in a kind of video

Publications (1)

Publication Number Publication Date
CN107291825A true CN107291825A (en) 2017-10-24

Family

ID=60095210

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710382041.5A Pending CN107291825A (en) 2017-05-26 2017-05-26 With the search method and system of money commodity in a kind of video

Country Status (1)

Country Link
CN (1) CN107291825A (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109063589A (en) * 2018-07-12 2018-12-21 杭州电子科技大学 Instrument and equipment on-line monitoring method neural network based and system
CN110427517A (en) * 2019-07-18 2019-11-08 华戎信息产业有限公司 A kind of figure based on scene lexicographic tree searches video method, device and computer readable storage medium
CN110647630A (en) * 2019-09-30 2020-01-03 浙江执御信息技术有限公司 Method and device for detecting same-style commodities
CN110807095A (en) * 2018-08-01 2020-02-18 北京京东尚科信息技术有限公司 Article matching method and device
CN111126179A (en) * 2019-12-05 2020-05-08 北京奇艺世纪科技有限公司 Information acquisition method and device, storage medium and electronic device
CN111309969A (en) * 2020-01-16 2020-06-19 湘潭大学 Video retrieval method matched with text information
CN111783808A (en) * 2019-07-23 2020-10-16 北京沃东天骏信息技术有限公司 Method and apparatus for generating information
CN112148924A (en) * 2019-06-28 2020-12-29 杭州海康威视数字技术股份有限公司 Luggage case retrieval method and device and electronic equipment
WO2021082505A1 (en) * 2019-10-28 2021-05-06 深圳市商汤科技有限公司 Picture processing method, apparatus and device, storage medium, and computer program

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103853724A (en) * 2012-11-29 2014-06-11 三星电子(中国)研发中心 Multimedia data sorting method and device
CN105808732A (en) * 2016-03-10 2016-07-27 北京大学 Integration target attribute identification and precise retrieval method based on depth measurement learning
CN106021575A (en) * 2016-05-31 2016-10-12 北京奇艺世纪科技有限公司 Retrieval method and device for same commodities in video
CN106202362A (en) * 2016-07-07 2016-12-07 Tcl集团股份有限公司 Image recommendation method and image recommendation device
CN106407281A (en) * 2016-08-26 2017-02-15 北京奇艺世纪科技有限公司 Image retrieval method and device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103853724A (en) * 2012-11-29 2014-06-11 三星电子(中国)研发中心 Multimedia data sorting method and device
CN105808732A (en) * 2016-03-10 2016-07-27 北京大学 Integration target attribute identification and precise retrieval method based on depth measurement learning
CN106021575A (en) * 2016-05-31 2016-10-12 北京奇艺世纪科技有限公司 Retrieval method and device for same commodities in video
CN106202362A (en) * 2016-07-07 2016-12-07 Tcl集团股份有限公司 Image recommendation method and image recommendation device
CN106407281A (en) * 2016-08-26 2017-02-15 北京奇艺世纪科技有限公司 Image retrieval method and device

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109063589A (en) * 2018-07-12 2018-12-21 杭州电子科技大学 Instrument and equipment on-line monitoring method neural network based and system
CN110807095A (en) * 2018-08-01 2020-02-18 北京京东尚科信息技术有限公司 Article matching method and device
CN112148924A (en) * 2019-06-28 2020-12-29 杭州海康威视数字技术股份有限公司 Luggage case retrieval method and device and electronic equipment
CN110427517A (en) * 2019-07-18 2019-11-08 华戎信息产业有限公司 A kind of figure based on scene lexicographic tree searches video method, device and computer readable storage medium
CN110427517B (en) * 2019-07-18 2023-04-25 华戎信息产业有限公司 Picture searching video method and device based on scene dictionary tree and computer readable storage medium
CN111783808A (en) * 2019-07-23 2020-10-16 北京沃东天骏信息技术有限公司 Method and apparatus for generating information
CN110647630A (en) * 2019-09-30 2020-01-03 浙江执御信息技术有限公司 Method and device for detecting same-style commodities
WO2021082505A1 (en) * 2019-10-28 2021-05-06 深圳市商汤科技有限公司 Picture processing method, apparatus and device, storage medium, and computer program
CN111126179A (en) * 2019-12-05 2020-05-08 北京奇艺世纪科技有限公司 Information acquisition method and device, storage medium and electronic device
CN111309969A (en) * 2020-01-16 2020-06-19 湘潭大学 Video retrieval method matched with text information

Similar Documents

Publication Publication Date Title
CN110866140B (en) Image feature extraction model training method, image searching method and computer equipment
CN107291825A (en) With the search method and system of money commodity in a kind of video
CN109948425B (en) Pedestrian searching method and device for structure-aware self-attention and online instance aggregation matching
US9633045B2 (en) Image ranking based on attribute correlation
Wang et al. Sketch-based 3d shape retrieval using convolutional neural networks
CN109960742B (en) Local information searching method and device
CN106682233A (en) Method for Hash image retrieval based on deep learning and local feature fusion
CN110825899A (en) Clothing image retrieval method integrating color features and residual network depth features
CN105589938A (en) Image retrieval system and retrieval method based on FPGA
CN103186538A (en) Image classification method, image classification device, image retrieval method and image retrieval device
CN110399895A (en) The method and apparatus of image recognition
CN110503076A (en) Video classification methods, device, equipment and medium based on artificial intelligence
Termritthikun et al. NU-InNet: Thai food image recognition using convolutional neural networks on smartphone
CN106096028A (en) Historical relic indexing means based on image recognition and device
CN114332680A (en) Image processing method, video searching method, image processing device, video searching device, computer equipment and storage medium
CN111339812A (en) Pedestrian identification and re-identification method based on whole or partial human body structural feature set, electronic equipment and storage medium
CN110059156A (en) Coordinate retrieval method, apparatus, equipment and readable storage medium storing program for executing based on conjunctive word
CN109993187A (en) A kind of modeling method, robot and the storage device of object category for identification
CN115131698A (en) Video attribute determination method, device, equipment and storage medium
CN106407281A (en) Image retrieval method and device
EP3748460A1 (en) Search system, search method, and program
Guo Research on sports video retrieval algorithm based on semantic feature extraction
CN112650869B (en) Image retrieval reordering method and device, electronic equipment and storage medium
CN116089639A (en) Auxiliary three-dimensional modeling method, system, device and medium
CN104715071B (en) A kind of specific pedestrian retrieval method described based on imperfect text

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20171024

RJ01 Rejection of invention patent application after publication