CN106202362A - Image recommendation method and image recommendation device - Google Patents

Image recommendation method and image recommendation device Download PDF

Info

Publication number
CN106202362A
CN106202362A CN201610532365.8A CN201610532365A CN106202362A CN 106202362 A CN106202362 A CN 106202362A CN 201610532365 A CN201610532365 A CN 201610532365A CN 106202362 A CN106202362 A CN 106202362A
Authority
CN
China
Prior art keywords
image
described
retrieved
characteristics
retrieval
Prior art date
Application number
CN201610532365.8A
Other languages
Chinese (zh)
Inventor
冯研
Original Assignee
Tcl集团股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tcl集团股份有限公司 filed Critical Tcl集团股份有限公司
Priority to CN201610532365.8A priority Critical patent/CN106202362A/en
Publication of CN106202362A publication Critical patent/CN106202362A/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • G06F16/951Indexing; Web crawling techniques

Abstract

The present invention provides a kind of image recommendation method and image recommendation device.Wherein, this image recommendation method includes: obtain image to be retrieved;Convolutional neural networks model is used to extract the characteristics of image of described image to be retrieved from described image to be retrieved, wherein, described convolutional neural networks model obtains based on the training of commodity image training set, and described commodity image training set comprises the multiple image sets according to merchandise classification and commodity parametric classification;Characteristics of image based on the image described to be retrieved extracted, the target image of the Image Feature Matching of retrieval characteristics of image and described image to be retrieved;One or more target image is pushed based on retrieval result.The technical scheme that the present invention provides can be effectively improved the accuracy of image retrieval.

Description

Image recommendation method and image recommendation device

Technical field

The present invention relates to field of computer technology, be specifically related to a kind of image recommendation method and image recommendation device.

Background technology

Along with the fast development of the Internet, increasing multimedia messages occurs in our life, and image is Typical Representative in multimedia messages, comprises abundant visual information, and it is interested to retrieve oneself from the image of magnanimity Image be extremely difficult thing, single these large nuber of images once-overs are required for the biggest workload.Image recommendation is Recommended by the matching degree between feature and the user preferences feature of image itself, and the feature of image itself can be led to Cross image text markup information and image content information is stated.

At present, image recommendation system is broadly divided into two classes: image recommendation system based on keyword and image content-based Image recommendation system.Image recommendation system based on keyword is with the word tag information of image as foundation, user is felt Keyword message corresponding to image of interest mates with the keyword message of image in image library, by image the highest for matching degree Recommend user.The image recommendation system of image content-based is to its content characteristic of the image zooming-out in image library, by user The content characteristic of image of interest mates with the content characteristic of image in image library, and the highest for matching degree is recommended use Family.

Traditional image recommendation model system also exists problems with:

1. image recommendation system based on keyword is affected very big by subjective factors, and same image is for different user institute circle Fixed keyword can gap, and may reflect that identical keyword describes for different Graphical User, this makes Retrieval result varies, and reduces user significantly to the degree of belief of product and satisfaction.

2. in the image recommendation system of image content-based, the foundation of image index and the realization of image recommendation are all based on The extraction of pixel characteristic, i.e. utilizes image processing method and mathematical method extraction table from original gray-scale map or coloured image Levy the pixel characteristic of image information.In actual application environment, the shooting of image can be by illumination, shooting angle, post-production And the impact of many extraneous factors such as capture apparatus, the color of image, brightness and saturation all can change a lot, and make The pixel characteristic that must extract is the most widely different, due to doing by extraneous factors such as the shooting environmental of image and capture apparatus Disturbing, therefore, be only based on the content that pixel characteristic can not describe in image accurately, different images likely can extract Going out identical pixel characteristic, incoherent image is also possible to extract closely similar pixel characteristic, and this will be substantially reduced use The family user satisfaction to product.

Summary of the invention

The present invention provides a kind of image recommendation method and image recommendation device, for improving the accuracy of image retrieval.

First aspect present invention provides a kind of image recommendation method, including:

Obtain image to be retrieved;

Convolutional neural networks model is used to extract the characteristics of image of above-mentioned image to be retrieved from above-mentioned image to be retrieved, its In, above-mentioned convolutional neural networks model based on commodity image training set training obtain, above-mentioned commodity image training set comprise according to Merchandise classification and multiple image sets of commodity parametric classification;

Characteristics of image based on the image above-mentioned to be retrieved extracted, retrieval characteristics of image and the image of above-mentioned image to be retrieved The target image of characteristic matching;

One or more target image is pushed based on retrieval result.

Second aspect present invention provides a kind of image recommendation device, including:

Acquiring unit, is used for obtaining image to be retrieved;

Image characteristics extraction unit, is used for using convolutional neural networks model to extract above-mentioned treating from above-mentioned image to be retrieved The characteristics of image of retrieval image, wherein, above-mentioned convolutional neural networks model obtains based on the training of commodity image training set, above-mentioned business Product training set of images comprises the multiple image sets according to merchandise classification and commodity parametric classification;

Retrieval unit, the characteristics of image of the image above-mentioned to be retrieved for extracting based on above-mentioned image characteristics extraction unit, The target image of the Image Feature Matching of retrieval characteristics of image and above-mentioned image to be retrieved;

Push unit, pushes one or more target for the retrieval result obtained based on the retrieval of above-mentioned retrieval unit Image.

Therefore, the present invention, when carrying out image recommendation, uses convolutional neural networks model from image to be retrieved Extract characteristics of image also to retrieve based on the characteristics of image extracted, due to this convolutional neural networks model be based on comprise according to The commodity image training set training of multiple image sets of merchandise classification and commodity parametric classification obtains, accordingly, with respect to traditional Keyword describes or pixel characteristic, uses the characteristics of image of this convolutional neural networks model extraction can reflect image more accurately Content and to external world factor have the strongest robustness, so that retrieve the target image obtained more based on this characteristics of image Accurately, improve the accuracy of image retrieval.

Accompanying drawing explanation

In order to be illustrated more clearly that the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing In having technology to describe, the required accompanying drawing used is briefly described, it should be apparent that, the accompanying drawing in describing below is only this Some embodiments of invention, for those of ordinary skill in the art, on the premise of not paying creative work, also may be used To obtain other accompanying drawing according to these accompanying drawings.

One embodiment schematic flow sheet of image recommendation method that Fig. 1-a provides for the present invention;

The convolutional neural networks model training schematic flow sheet that Fig. 1-b provides for the present invention;

A kind of image characteristics extraction schematic flow sheet that Fig. 1-c provides for the present invention;

SAE model schematic under a kind of application scenarios that Fig. 1-d provides for the present invention;

One example structure schematic diagram of image recommendation device that Fig. 2 provides for the present invention;

One example structure schematic diagram of image recommendation system that Fig. 3-a provides for the present invention;

A kind of webcrawler module handling process schematic diagram that Fig. 3-b provides for the present invention.

Detailed description of the invention

For making the goal of the invention of the present invention, feature, the advantage can be the most obvious and understandable, below in conjunction with the present invention Accompanying drawing in embodiment, is clearly and completely described the technical scheme in the embodiment of the present invention, it is clear that described reality Executing example is only a part of embodiment of the present invention, and not all embodiments.Based on the embodiment in the present invention, the common skill in this area The every other embodiment that art personnel are obtained under not making creative work premise, broadly falls into the model of present invention protection Enclose.

The embodiment of the present invention provides a kind of image recommendation method, refers to the image shown in Fig. 1-a, in the embodiment of the present invention Recommendation method may include steps of:

Step 101, obtain image to be retrieved;

In the embodiment of the present invention, an image retrieval control can be provided the user, when user triggers this image retrieval control Time, i.e. enter step 101.

In a step 101, the retrieval of similar image can be carried out for the image of user's captured in real-time, then apply in one In scene, step 101 may include that the image obtaining current shooting gained is as image to be retrieved.Specifically, step is being entered When 101, start photographic head (or starting photographic head when receiving the instruction of instruction captured in real-time further), in order to by this Photographic head carries out the shooting of image, when user trigger this photographic head take pictures action time, obtain current shooting gained image make For image to be retrieved.Further, when entering step 101, it is also possible to prompting user can shoot real time imaging as figure to be retrieved Picture.

Or, it is also possible to the image for user's collection carries out the retrieval of similar image, then in another kind of application scenarios, Step 101 may include that and obtains image chosen in current photo library as image to be retrieved.Specifically, step is being entered When 101, multiple images of comprising with the form display photos storehouse of preview (or select from photo library receiving instruction further When selecting the instruction of image, the multiple images comprised with the form display photos storehouse of preview), in order to user is at multiple images of display In carry out the selection of image to be retrieved, further, when entering step 101, it is also possible to prompting user can select from photo library Image is as image to be retrieved.

Certainly, the embodiment of the present invention can also obtain image to be retrieved otherwise, is not construed as limiting herein.

Step 102, use convolutional neural networks model extract the figure of above-mentioned image to be retrieved from above-mentioned image to be retrieved As feature;

In the embodiment of the present invention, above-mentioned convolutional neural networks model obtains based on the training of commodity image training set, above-mentioned business Product training set of images comprises the multiple image sets according to merchandise classification and commodity parametric classification.Wherein, merchandise classification instruction commodity Type (such as clothing, footwear, case and bag class etc.), commodity parameter instruction commodity characteristic information (color of such as commodity, material Matter, style etc.).As a example by clothing commodity, owing to clothing has relative to case and bag class, articles for daily use and electronic product etc. Background is complicated, and the features such as image subject decorative pattern, pattern are changeable are, and user's style to image subject when image retrieval, color Precision requires the highest, therefore, can select multiple training images (such as 10000 respectively for the clothing of 150 common kinds Or more number) and add the training carrying out convolutional neural networks model in above-mentioned commodity image training set to, above-mentioned commodity figure As 150 class image of clothing in training set are that the information such as the color according to clothing, pattern, style, material, sleeve length carry out intersection point Class, such as part women's dress classification information can be as shown in table 1:

Table 1

In the embodiment of the present invention, convolutional neural networks model is a kind of non-full connection and the neural network structure of multilamellar, often One layer is made up of multiple two dimensional surfaces, and each plane is made up of multiple independent neurons.It is the same with other neural network model, Convolutional neural networks model is also made up of input layer, hidden layer and output layer.Hidden layer has two kinds of special structure sheafs: volume Lamination and pond sample level, convolutional layer is made up of multiple characteristic planes, and each characteristic plane is made up of neuron, and same feature is put down Neuron on face has identical link weight.Convolutional neural networks is the most also a kind of mapping being input to output, It can learn substantial amounts of being input on the premise of the accurate mathematical expression formula that need not between any input and output and export Between mapping relations, it is only necessary to being trained convolutional neural networks by known pattern, convolutional neural networks just has defeated Enter the mapping ability between output.What convolutional neural networks model performed is the training having guidance, so in its sample set Sample is analogous to [input information, export information] such information to composition.Concrete, convolutional neural networks model training Schematic flow sheet can be as shown in Fig. 1-b, and for the classification information of each image in above-mentioned commodity image training set, (classification information includes Merchandise classification and commodity parameter) image is labelled, and using the content that labels as the standard output result of single sample (or can also need not label and directly determine the standard output result of single sample according to classification information), as depending on It is trained obtaining final spendable convolutional neural networks model according to the convolutional neural networks set up.Wherein, convolution god Following several process specifically can be included: forward conduction stage, back-propagation phase, calculating network parameter through the training process of network Residual error (network parameter residual error includes: output layer residual error, next layer be the convolutional layer of pond sample level (i.e. pooling layer) residual error, The residual error of pooling layer that next layer is convolutional layer)) and calculate convolutional layer and be connected the derivative of the weights of layer, bias.Can Choosing, convolutional neural networks model in the embodiment of the present invention has a hidden layer 7 layers, reaches to receive after training iterations 100,000 times Hold back.

In a step 102, use the convolutional neural networks model that training obtains from the image to be retrieved that step 101 obtains Extract the characteristics of image of above-mentioned image to be retrieved.Concrete, the image characteristics extraction flow process of step 102 can as shown in fig 1-c, Including:

Step A1, the pixel value vector of image to be retrieved is inputted convolutional neural networks model;

Illustrating, if the size of image to be retrieved is m*n, the value of the pixel of the i-th row jth row is xi,j, then by vector {x1,1, x1,2..., xm,n-1, xm,nAs this image to be retrieved pixel value vector input above-mentioned convolutional neural networks model In.

Step A2, use the initialized multiple convolution kernels view data to being currently entered in this convolutional neural networks model Do convolution, obtain convolution character image data;

Step A3, pond maximum to above-mentioned convolution characteristic work sampling, it is thus achieved that the characteristic image after the sampling of maximum pond.

Step A4, detect whether next layer is convolutional layer;

In the embodiment of the present invention, whether next layer of detection convolutional neural networks model current layer is convolutional layer, if volume Lamination, then enter step A2, if not convolutional layer, then enters step A5.

It should be noted that when being entered step A2 by step A1, the above-mentioned view data being currently entered is step A1 In mention image to be retrieved pixel value vector, by step A4 enter step A2 time, the above-mentioned view data being currently entered It is the character image data after step A3 obtains the sampling of maximum pond.

Step A5, enter next layer and perform full concatenation operation;

Concrete, the full concatenation operation process in step A5 is referred to prior art and describes, and here is omitted.

Whether step A6, detection currently arrive last layer of convolutional neural networks model;

If current last layer arriving above-mentioned convolutional neural networks model being detected, then enter step A7, if detecting Current last layer not arriving above-mentioned convolutional neural networks model, then perform step A5.

The characteristics of image that step A7, output are extracted;

It should be noted that the image characteristics extraction flow process shown in Fig. 1-c is only one signal, implement based on the present invention Convolutional neural networks model in example, can also use other image based on convolutional neural networks model in other embodiments Feature extraction algorithm extracts the characteristics of image of above-mentioned image to be retrieved, does not limits.

Further, if the characteristics of image using above-mentioned convolutional neural networks model extraction to go out has the image spy of thousands of dimension Levy, then can also carry out Data Dimensionality Reduction to extracting the characteristics of image obtained, on the one hand, Data Dimensionality Reduction can solve " dimension calamity Difficult ", solve the imbalance between abundant information and poor knowledge, reduce complexity;On the other hand, can preferably be appreciated and understood by The data obtained.Then step 102 specifically can also include: carries from above-mentioned image to be retrieved using convolutional neural networks model The characteristics of image taken carries out dimension-reduction treatment, using special as the image of above-mentioned image to be retrieved for the characteristics of image obtained after dimension-reduction treatment Levy.

Optionally, stack own coding neutral net (SAE, Stack Auto-Encoders) is used to carry out characteristics of image Dimension-reduction treatment, wherein, SAE is a neutral net being made up of multilamellar sparse own coding device, the output of preceding layer own coding device Input as later layer own coding device.Concrete, the primitive image features of the thousands of dimensions for extracting, with its training one certainly Encoder, it is possible to obtain the single order feature being originally inputted, can obtain correspondence for each primitive image features inputted Single order feature.Using these single order features as the input of next sparse own coding device, obtain original graph with they training study As the second order feature of feature, upper single order feature is input in the second layer sparse own coding device that trains, just can obtain often The second order feature activation value that individual single order feature is corresponding, two the sparse own coding devices trained the most at last build a bag altogether Containing the SAE model of the characteristics of image dimensionality reduction of a hidden layer, an input layer and an output layer, this final energy of SAE model Realizing our dimensionality reduction to image feature data, as a example by the characteristics of image of 6 dimensions, then corresponding SAE model can be such as Fig. 1-d institute Showing, wherein, x1~x6 represents the primitive image features (i.e. input layer) of 6 different dimensions, h respectively1 (1)~h4 (1)Represent 4 respectively Individual single order feature (i.e. hidden layer), h1 (2)~h3 (2)Represent 3 second order features (i.e. output layer) respectively.The circle putting on "+1 " is Bias node, namely intercept item.

Certainly, in other embodiments, it would however also be possible to employ other characteristics of image dimension reduction method is to using above-mentioned convolutional Neural The characteristics of image that network model extracts from above-mentioned image to be retrieved carries out dimension-reduction treatment, by special for the image obtained after dimension-reduction treatment Levy the characteristics of image as above-mentioned image to be retrieved.

Step 103, characteristics of image based on the image above-mentioned to be retrieved extracted, retrieval characteristics of image and above-mentioned figure to be retrieved The target image of the Image Feature Matching of picture;

In step 103, characteristics of image based on the image above-mentioned to be retrieved that step 102 obtains, can build from advance Image retrieval data base in retrieve the target image of Image Feature Matching of characteristics of image and above-mentioned image to be retrieved.

In order to improve image retrieval efficiency, optionally, local sensitivity Hash (LSH, Locality Sensitive is used Hashing) algorithm is that image feature vector sets up index, then step 103 specifically includes: determined above-mentioned to be checked by hash function The Hash bucket that the characteristics of image of rope image is mapped;Image corresponding to each element already present in above-mentioned Hash bucket is determined For above-mentioned target image;Wherein, in above-mentioned Hash bucket, each element already present is the most right beforehand through above-mentioned hash function The characteristics of image of each image carries out mapping and obtains, and the characteristics of image of each image above-mentioned is by above-mentioned convolutional neural networks mould Type extracts from each image above-mentioned respectively and obtains.

Wherein, above-mentioned employing LSH algorithm is that the process that image feature vector foundation indexes specifically comprises the steps that 1, takes sample set In all image feature vector P={x1,x2,…,xdMaximum coordinate value among C in all dimensions, image feature vector P is mapped to Hamming space Hd’In, Hd’=C*d, mapping method is that all dimensional information of vector P are converted into binary sequence v P (), this conversion formula is as follows: v (p)=Unaryc(x1)……Unaryc(xd), wherein, Unaryc(xd) it is expressed as xdAfter individual 1 Immediately following C-xdThe C of individual 0 ties up binary vector;2, definition hash function g (p)=p | I, p | I is the projection on I direction of the p vector, Use g (p) that image feature vector P is carried out computing so that all images are mapped in Hash bucket, wherein, characteristics of image The image that vector distance is close can be mapped in identical Hash bucket, and distant image is then mapped to different Hash In Tong.Further, after certain Hash bucket is filled, then when having new characteristics of image to arrive, it is impossible to be mapped to this Hash bucket In, then map it to other less than and ensure higher collision probability Hash bucket in;3, these Hash buckets are compressed to Kazakhstan In uncommon table.

Further, it is also possible to the mode using LSH algorithm and distributed system to combine is that characteristics of image sets up index, its In, above-mentioned distributed system is embodied in: during building above-mentioned image retrieval data base, uses the collection of multiple stage machine Group extracts characteristics of image from image, and (wherein, image includes sample image and image to be retrieved, and sample image can pass through network Reptile mode and information retrieval mode based on static template obtain, and the image characteristics extraction mode of sample image is referred to treat The image characteristics extraction mode (the image characteristics extraction process that i.e. step 102 is recorded) of retrieval image), wherein one is main control computer Device, its task is to be assigned on other machine carry out image characteristics extraction by image uniform, and all images all can be distributed to Carry out image characteristics extraction on other machine, then by the host machine in index cluster, the characteristics of image extracted uniformly is divided Being fitted on other index machine and carry out image index operation, every machine maintains a Hash table and a hash function.Enter one Step, during building above-mentioned image retrieval data base, arrives when the characteristics of image of a sample image is hashed Function Mapping In certain Hash bucket of Hash table, then Linked Storage Structure can be used to store the mark (mark of such as image of this sample image Symbol (ID, IDentity), it is not necessary to the characteristics of image of this storage image again.

Step 104, based on retrieval result push one or more target image;

At step 104, can owning the Image Feature Matching of the characteristics of image retrieved and above-mentioned image to be retrieved Target image pushes, to recommend all target images to user, or, it is also possible to from the characteristics of image retrieved with above-mentioned In all target images of the Image Feature Matching of image to be retrieved, push former the target images that similarity is high.

Further, when step 103 determines, by hash function, the Hash bucket that the characteristics of image of image to be retrieved is mapped, And when the image corresponding to each element already present in above-mentioned Hash bucket is defined as above-mentioned target image, step 104 is permissible Including: if already present element number is little in above-mentioned Hash bucket (the Hash bucket that the characteristics of image of image the most to be retrieved is mapped) In default image recommendation number, then push all target images based on retrieval result;If already present element in above-mentioned Hash bucket Number is more than above-mentioned image recommendation number, then: calculate the similarity of each target image and above-mentioned image to be retrieved;According to similarity Being pushed top n target image by height sequence on earth, wherein, above-mentioned N is equal to above-mentioned image recommendation number.

Concrete, each target image can be measured by Jie Kade similarity coefficient similar to above-mentioned image to be retrieved Degree, according to the Jie Kade similarity coefficient of each target image calculated Yu above-mentioned image to be retrieved, according to the similar system of Jie Kade Number sorts from high to low, picks out top n target image and is pushed to user.

Further, step 104 can also include: pushes the merchandise news associated with this target image pushed.Specifically , can find, according to the mark (such as ID) of this target image pushed, the commodity that the target image pushed with this associates Information, and push corresponding merchandise news while pushing target image.

Therefore, in the embodiment of the present invention when carrying out image recommendation, use convolutional neural networks model to be retrieved Image in extract characteristics of image and based on extract characteristics of image retrieve, due to this convolutional neural networks model be based on Comprise to train according to the commodity image training set of merchandise classification and multiple image sets of commodity parametric classification and obtain, therefore, relatively Describing or pixel characteristic in traditional keyword, the characteristics of image using this convolutional neural networks model extraction can be more accurately Reflection picture material and to external world factor have the strongest robustness, so that retrieve the target obtained based on this characteristics of image Image is the most accurate, improves the accuracy of image retrieval.

A kind of image recommendation device provided the embodiment of the present invention below is described, as in figure 2 it is shown, the present invention implements Image recommendation device 200 in example includes:

Acquiring unit 201, is used for obtaining image to be retrieved;

Image characteristics extraction unit 202, is used for using convolutional neural networks model to extract from above-mentioned image to be retrieved Stating the characteristics of image of image to be retrieved, wherein, above-mentioned convolutional neural networks model obtains based on the training of commodity image training set, on State commodity image training set and comprise the multiple image sets according to merchandise classification and commodity parametric classification;

Retrieval unit 203, the image of the image above-mentioned to be retrieved for extracting based on image characteristics extraction unit 202 is special Levy, the target image of the Image Feature Matching of retrieval characteristics of image and above-mentioned image to be retrieved;

Push unit 204, pushes one or more mesh for the retrieval result obtained based on retrieval unit 203 retrieval Logo image.

Optionally, retrieval unit 203 includes: first determines unit, for determining above-mentioned figure to be retrieved by hash function The Hash bucket that the characteristics of image of picture is mapped;Second determines unit, for by each element institute already present in above-mentioned Hash bucket Corresponding image is defined as above-mentioned target image;Wherein, in above-mentioned Hash bucket, each element already present is beforehand through above-mentioned Hash function characteristics of image to each image respectively carries out mapping and obtains, and the characteristics of image of each image above-mentioned is by above-mentioned Convolutional neural networks model extracts from each image above-mentioned respectively and obtains.

Optionally, image characteristics extraction unit 202 is specifically additionally operable to: to using described convolutional neural networks model from described The characteristics of image extracted in image to be retrieved carries out dimension-reduction treatment, using the characteristics of image that obtains after dimension-reduction treatment as described to be checked The characteristics of image of rope image.

Optionally, retrieval unit 203 specifically for: based on the characteristics of image of image described to be retrieved extracted, from image Searching database is retrieved the target image of characteristics of image and the Image Feature Matching of described image to be retrieved.Wherein, described figure The mode using local sensitivity hash algorithm and distributed system to combine as searching database is that characteristics of image sets up index.

Optionally, push unit 204 specifically for: when in above-mentioned Hash bucket already present element number be not more than preset During image recommendation number, push all target images based on retrieval result;When in above-mentioned Hash bucket, already present element number is more than During above-mentioned image recommendation number, calculate the similarity of each target image and above-mentioned image to be retrieved, according to similarity by height on earth Sequence push top n target image, wherein, above-mentioned N be equal to above-mentioned image recommendation number.

Optionally, push unit 204 is additionally operable to push the merchandise news associated with this target image pushed.

Optionally, acquiring unit 201 specifically for: obtain current shooting gained image as image to be retrieved;Or, Obtain image chosen in current photo library as image to be retrieved.

Should be understood that the function of each functional module of image recommendation device in the embodiment of the present invention can be according to above-mentioned side Method in method embodiment implements, and it implements process and can refer to the associated description in said method embodiment, herein Repeat no more.

Therefore, the image recommendation device in the embodiment of the present invention, when carrying out image recommendation, uses convolutional Neural net Network model extracts characteristics of image from image to be retrieved and retrieves, due to this convolutional Neural based on the characteristics of image extracted Network model is to train according to the commodity image training set of merchandise classification and multiple image sets of commodity parametric classification based on comprising Obtaining, describe or pixel characteristic accordingly, with respect to traditional keyword, the image using this convolutional neural networks model extraction is special Levy can reflect more accurately picture material and to external world factor there is the strongest robustness, so that based on this characteristics of image The target image that retrieval obtains is the most accurate, improves the accuracy of image retrieval.

Being described a kind of image recommendation system below, this image recommendation system uses the image recommendation as shown in 1-a Method carries out image recommendation.As shown in Fig. 3-a, this image recommendation system includes following several part: webcrawler module, image Characteristic extracting module, index module and image recommendation module.

Wherein, webcrawler module is mainly used in extracting web page source file and extracting image and commodity from web page source file Information.The concrete processing procedure of webcrawler module as shown in Fig. 3-b, including:

Step S11, by web crawler obtain web page source file;

Web crawler is typically from some or URL (URL, the Uniform of several Initial pages Resource Locator) set about, obtain the URL on Initial page, during crawling webpage, constantly from current page The new URL of middle acquisition puts in URL queue, until meeting the stop condition also preset.Web crawler is first from website Page starts, and resolves downwards, and the process that crawls of crawlers is broadly divided into following step: a, parses from website homepage The URL of the initial directory page of each classification;B, parse the URL of next layer of catalogue page from each initial directory page classified, directly To the URL being resolved to last layer catalogue page;C, parse list page URL from last layer catalogue page;D, parse from list page All commodity URL that this page comprises, join in URL queue to be crawled;E, from this URL queue take out relevant to commodity URL, by the web page source file download of its correspondence to local.

Step S12, from above-mentioned web page source file, carry out information retrieval based on static template;

The contents such as very many web page tag and advertisement link, these contents are comprised due to the content of web page source file own It is invalid information for web crawlers person, accordingly, it would be desirable to obtained from web page source file by Web information extraction technique Useful information, i.e. with Web as information source, obtains the information of specified type from Web, and is carried out structured storage, can For program.

The purpose of step S12 is to obtain the merchandise classification in webpage and commodity parameter from above-mentioned web page source file Information, its detailed process is as follows: analyze each web page source file, obtains the theme (i.e. merchandise classification) in web page source file, business The position of the information such as product parameter information, to determine extraction bound information, carries out uniqueness judgement simultaneously to bound, by really Fixed bound information saves as static template;Read static template, mate business by the bound information of this static template Product source web page, obtains image link information and the commodity parameter information of commodity;The commodity parameter information got is processed (the commodity parameter information branch storage that such as will get), so that parameter information standardization;To the commodity parameter after processing Information is made denoising and (the commodity parameter information after processing such as is removal HTML (HTML, HyperText Markup Language) process of label so that the commodity parameter information finally given only comprises the text message of parameter), Further, if the commodity parameter information obtained comprises unicode coding, in addition it is also necessary to by character conversion, unicode is encoded Be converted to the Chinese character of correspondence;Export final merchandise classification and commodity parameter information;Will according to the image address extracted Image is downloaded and stored in local file system.

The information that step S13, storage are extracted;

In step s 13, commodity image webcrawler module crawled and corresponding merchandise news (such as merchandise classification With commodity parameter) it is stored on distributed type assemblies with document form, and ID and the corresponding merchandise news of commodity image are deposited Storage is in data base, in order to can read corresponding image from this distributed type assemblies according to image ID and obtain in this data base Corresponding merchandise news.

After getting commodity image relevant information by webcrawler module, the retrieval to image to be realized and recommend, need The content information of image is extracted as digitized feature, on the one hand facilitates computer to carry out computing, on the other hand relative to The description to picture material that visually-perceptible is the most abstract, is converted into digitized feature and Measurement of Similarity between Two Images more may be used Quantify.In the embodiment of the present invention, realize the extraction to characteristics of image, specifically, this image by above-mentioned image characteristics extraction module The image characteristics extraction process of characteristic extracting module is referred to the description of step 102 in Fig. 1-a illustrated embodiment, the most no longer Repeat.

Above-mentioned index module is set up for output based on above-mentioned webcrawler module and above-mentioned image characteristics extraction module Index, in order to above-mentioned image recommendation module carries out the retrieval of target image based on the index that above-mentioned index module is advised.

The concrete structure of above-mentioned image recommendation module is referred to the image recommendation device shown in Fig. 2, and it implemented Journey can refer to the associated description in the embodiment of the method shown in Fig. 1-a, and here is omitted.

In several embodiments provided herein, it should be understood that disclosed methods, devices and systems can lead to The mode crossing other realizes.

It should be noted that for aforesaid each method embodiment, in order to simplicity describes, therefore it is all expressed as a series of Combination of actions, but those skilled in the art should know, the present invention is not limited by described sequence of movement because According to the present invention, some step can use other order or carry out simultaneously.Secondly, those skilled in the art also should know Knowing, it might not be all this that embodiment described in this description belongs to preferred embodiment, involved action and module Bright necessary.

In the above-described embodiments, the description to each embodiment all emphasizes particularly on different fields, and does not has the portion described in detail in certain embodiment Point, may refer to the associated description of other embodiments.

It is more than to a kind of image recommendation method provided by the present invention and the description of image recommendation device, for this area Those skilled in the art, according to the thought of the embodiment of the present invention, the most all can change it Place, to sum up, this specification content should not be construed as limitation of the present invention.

Claims (10)

1. an image recommendation method, it is characterised in that including:
Obtain image to be retrieved;
Convolutional neural networks model is used to extract the characteristics of image of described image to be retrieved from described image to be retrieved, wherein, Described convolutional neural networks model obtains based on the training of commodity image training set, and described commodity image training set comprises according to commodity Classification and multiple image sets of commodity parametric classification;
Characteristics of image based on the image described to be retrieved extracted, retrieval characteristics of image and the characteristics of image of described image to be retrieved The target image of coupling;
One or more target image is pushed based on retrieval result.
Image recommendation method the most according to claim 1, it is characterised in that described based on the image described to be retrieved extracted Characteristics of image, the target image of the Image Feature Matching of retrieval characteristics of image and described image to be retrieved, including:
The Hash bucket that the characteristics of image of described image to be retrieved is mapped is determined by hash function;
Image corresponding to each element already present in described Hash bucket is defined as described target image;
Wherein, in described Hash bucket, each element already present is figure to each image respectively beforehand through described hash function Obtaining as feature carries out mapping, the characteristics of image of each image described is respectively from described by described convolutional neural networks model Each image extracts and obtains.
Image recommendation method the most according to claim 1 and 2, it is characterised in that described use convolutional neural networks model The characteristics of image of described image to be retrieved is extracted from described image to be retrieved, including:
The characteristics of image using described convolutional neural networks model to extract from described image to be retrieved is carried out dimension-reduction treatment, will The characteristics of image obtained after dimension-reduction treatment is as the characteristics of image of described image to be retrieved.
Image recommendation method the most according to claim 1 and 2, it is characterised in that described described to be retrieved based on extract The characteristics of image of image, the target image of the Image Feature Matching of retrieval characteristics of image and described image to be retrieved, particularly as follows:
Characteristics of image based on the image described to be retrieved extracted, retrieves characteristics of image from image retrieval data base and treats with described The target image of the Image Feature Matching of retrieval image;
Wherein, the mode that described image retrieval data base uses local sensitivity hash algorithm and distributed system to combine is image Feature sets up index.
Image recommendation method the most according to claim 1 and 2, it is characterised in that described acquisition image to be retrieved includes:
Obtain the image of current shooting gained as image to be retrieved;
Or,
Obtain image chosen in current photo library as image to be retrieved.
6. an image recommendation device, it is characterised in that including:
Acquiring unit, is used for obtaining image to be retrieved;
Image characteristics extraction unit, is used for using convolutional neural networks model to extract from described image to be retrieved described to be retrieved The characteristics of image of image, wherein, described convolutional neural networks model obtains based on the training of commodity image training set, described commodity figure As training set comprises the multiple image sets according to merchandise classification and commodity parametric classification;
Retrieval unit, the characteristics of image of the image described to be retrieved for extracting based on described image characteristics extraction unit, retrieval The target image of the Image Feature Matching of characteristics of image and described image to be retrieved;
Push unit, pushes one or more target figure for the retrieval result obtained based on the retrieval of described retrieval unit Picture.
Image recommendation device the most according to claim 6, it is characterised in that
Described retrieval unit includes:
First determines unit, for determining, by hash function, the Hash bucket that the characteristics of image of described image to be retrieved is mapped;
Second determines unit, for the image corresponding to each element already present in described Hash bucket is defined as described target Image;
Wherein, in described Hash bucket, each element already present is figure to each image respectively beforehand through described hash function Obtaining as feature carries out mapping, the characteristics of image of each image described is respectively from described by described convolutional neural networks model Each image extracts and obtains.
8. according to the image recommendation device described in claim 6 or 7, it is characterised in that described image characteristics extraction unit is concrete It is additionally operable to: the characteristics of image using described convolutional neural networks model to extract from described image to be retrieved is carried out at dimensionality reduction Reason, using the characteristics of image that obtains after dimension-reduction treatment as the characteristics of image of described image to be retrieved.
9. according to the image recommendation device described in claim 6 or 7, it is characterised in that described retrieval unit specifically for: based on The characteristics of image of the image described to be retrieved extracted, retrieves characteristics of image and described image to be retrieved from image retrieval data base The target image of Image Feature Matching;
Wherein, the mode that described image retrieval data base uses local sensitivity hash algorithm and distributed system to combine is image Feature sets up index.
10. according to the image recommendation device described in claim 6 or 7, it is characterised in that described acquiring unit specifically for: obtain Take the image of current shooting gained as image to be retrieved;Or, obtain image chosen in current photo library as to be checked Rope image.
CN201610532365.8A 2016-07-07 2016-07-07 Image recommendation method and image recommendation device CN106202362A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610532365.8A CN106202362A (en) 2016-07-07 2016-07-07 Image recommendation method and image recommendation device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610532365.8A CN106202362A (en) 2016-07-07 2016-07-07 Image recommendation method and image recommendation device

Publications (1)

Publication Number Publication Date
CN106202362A true CN106202362A (en) 2016-12-07

Family

ID=57473541

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610532365.8A CN106202362A (en) 2016-07-07 2016-07-07 Image recommendation method and image recommendation device

Country Status (1)

Country Link
CN (1) CN106202362A (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106886599A (en) * 2017-02-28 2017-06-23 北京京东尚科信息技术有限公司 Image search method and device
CN107291825A (en) * 2017-05-26 2017-10-24 北京奇艺世纪科技有限公司 With the search method and system of money commodity in a kind of video
CN107316042A (en) * 2017-07-18 2017-11-03 盛世贞观(北京)科技有限公司 A kind of pictorial image search method and device
CN107729557A (en) * 2017-11-08 2018-02-23 北京大学 A kind of classification of inventory information, search method and device
CN108509466A (en) * 2017-04-14 2018-09-07 腾讯科技(深圳)有限公司 A kind of information recommendation method and device

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030021481A1 (en) * 2001-07-25 2003-01-30 Nec Corporation Image retrieval apparatus and image retrieving method
CN104991959A (en) * 2015-07-21 2015-10-21 北京京东尚科信息技术有限公司 Method and system for retrieving same or similar image based on content
CN105512289A (en) * 2015-12-07 2016-04-20 郑州金惠计算机系统工程有限公司 Image retrieval method based on deep learning and Hash

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030021481A1 (en) * 2001-07-25 2003-01-30 Nec Corporation Image retrieval apparatus and image retrieving method
CN104991959A (en) * 2015-07-21 2015-10-21 北京京东尚科信息技术有限公司 Method and system for retrieving same or similar image based on content
CN105512289A (en) * 2015-12-07 2016-04-20 郑州金惠计算机系统工程有限公司 Image retrieval method based on deep learning and Hash

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106886599A (en) * 2017-02-28 2017-06-23 北京京东尚科信息技术有限公司 Image search method and device
CN108509466A (en) * 2017-04-14 2018-09-07 腾讯科技(深圳)有限公司 A kind of information recommendation method and device
CN107291825A (en) * 2017-05-26 2017-10-24 北京奇艺世纪科技有限公司 With the search method and system of money commodity in a kind of video
CN107316042A (en) * 2017-07-18 2017-11-03 盛世贞观(北京)科技有限公司 A kind of pictorial image search method and device
CN107729557A (en) * 2017-11-08 2018-02-23 北京大学 A kind of classification of inventory information, search method and device

Similar Documents

Publication Publication Date Title
Liu et al. Multi-task deep visual-semantic embedding for video thumbnail selection
CN103733177B (en) Strengthen RUNTIME VIEW
US8897505B2 (en) System and method for enabling the use of captured images through recognition
JP2010530998A (en) Image-based information retrieval method and system
CN104021150B (en) Face recognition with social networks auxiliary
US8630513B2 (en) System and method for providing objectified image renderings using recognition information from images
Wu et al. 3d shapenets: A deep representation for volumetric shapes
Du et al. Fusion of difference images for change detection over urban areas
US7809722B2 (en) System and method for enabling search and retrieval from image files based on recognized information
US7809192B2 (en) System and method for recognizing objects from images and identifying relevancy amongst images and information
KR20110033179A (en) Annotating images
US20140149376A1 (en) System for collecting interest graph by relevance search incorporating image recognition system
RU2387004C2 (en) Method and system for calculating unit significance value in display page
CN103329126B (en) Utilize the search of joint image-audio query
Torralba et al. Labelme: Online image annotation and applications
Wang et al. Image saliency: From intrinsic to extrinsic context
EP2883158B1 (en) Identifying textual terms in response to a visual query
CN101960481A (en) Method and system for managing digital photos
Ansari et al. Pixel-based image forgery detection: A review
CN101896901A (en) Interactive concept learning in image search
CN107018486A (en) Handle the method and system of virtual query
US10055391B2 (en) Method and apparatus for forming a structured document from unstructured information
CN101853299B (en) Image searching result ordering method based on perceptual cognition
CN109145078A (en) The application page of the machine application is indexed
Mei et al. Contextual in-image advertising

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination