CN103544216B - The information recommendation method and system of a kind of combination picture material and keyword - Google Patents
The information recommendation method and system of a kind of combination picture material and keyword Download PDFInfo
- Publication number
- CN103544216B CN103544216B CN201310436726.5A CN201310436726A CN103544216B CN 103544216 B CN103544216 B CN 103544216B CN 201310436726 A CN201310436726 A CN 201310436726A CN 103544216 B CN103544216 B CN 103544216B
- Authority
- CN
- China
- Prior art keywords
- picture
- matrix
- keyword
- image content
- similarity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/5866—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
- G06F16/5838—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9535—Search customisation based on user profiles and personalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
Abstract
The present invention discloses the information recommendation method and system of a kind of combination picture material and keyword, wherein, method includes step：The keyword message and the image content information comprising color characteristic and textural characteristics of the picture of picture library are extracted, keyword message and image content information are expressed as vector space model, obtain corresponding keyword message matrix and image content information matrix；Keyword message matrix and image content information matrix are processed using Sparse model, the similarity in calculating picture library between picture obtains similarity table；Target Photo according to user search inquires about the similar pictures of the Target Photo from similarity table and forms raw recommendation list；Raw recommendation list is arranged to obtain consequently recommended list and show.
Description
Technical field
The present invention relates to image retrieval and information recommendation field, more particularly to a kind of combination picture material and keyword letter
Breath recommends method and system.
Background technology
With internet and the high speed development of ecommerce, online multimedia messages also increase in explosion type, because
This people is also more and more to the Search Requirement of multimedia messages, such as shopping online, social networks, video sharing platform etc.
It is required for retrieving multimedia messages.Multimedia messages on internet generally include image, Voice ＆ Video.Wherein scheme
The retrieval of picture is a focus and trend of the Internet, applications particularly ecommerce.For example, only by picture self-information
To retrieve the commodity of correlation so as on the one hand facilitate user to inquire about, another aspect help system effectively makes recommendation to user.
The retrieval for interconnecting picture on network has two directions of main flow：First is retrieval based on keyword
(Keyword-Based Retrieval)；Second is retrieval (Content-Based based on the content of image itself
Retrieval)。
, it is necessary to first carry out keyword, Ran Houcai to all of image in the image indexing system based on keyword
Image can be scanned for using global search technology, such as AltaVista, Yahoo!, Google image search engine etc..This
There are two aspects in the method for kind:One is that this method needs more artificial participation, and with the increasing of picture number
Plus, this method is difficult to realize；Second Problem is that the information content that image is included is huge, and different people are for same figure
The understanding of picture is also differed, and this results in the standard unified to the mark neither one of image, and light is to be difficult to accomplish with keyword
Describe exactly and retrieving multimedia information.
The information retrieval of image content-based is that the content and context semantic environment of multimedia object are retrieved, such as
Color in image, the scene in texture, or video, segment are analyzed and feature extraction, and are carried out based on these features
Similitude is matched, and the system of information retrieval based on contents has Photobook, the U.S. of QBIC, MIT multi-media Laboratory of IBM exploitations
VisualSEEK image query systems of Columbia University's exploitation etc..Content-based information retrieval has stronger objective
Property, but, due to these features not real semantic information of representative image, content-based retrieval result does not often make us full
Meaning, therefore most systems are also based on the image retrieval of keyword at present.
Image indexing system whether based on keyword is also based on the image indexing system of picture material all in the presence of not
Foot.Therefore, prior art has yet to be improved and developed.
The content of the invention
In view of above-mentioned the deficiencies in the prior art, it is an object of the invention to provide a kind of combination picture material and keyword
Information recommendation method and system, it is intended to otherwise solve the problems, such as that conventional images searching system workload is big, retrieval result is inaccurate.
Technical scheme is as follows：
A kind of information recommendation method of combination picture material and keyword, wherein, including step：
A, the keyword message of the picture of extraction picture library and the picture material letter comprising color characteristic and textural characteristics
Breath, vector space model is expressed as by keyword message and image content information, obtains corresponding keyword message matrix and figure
As content information matrix；
B, keyword message matrix and image content information matrix are processed using Sparse model, calculate picture
Similarity in storehouse between picture obtains similarity table；
C, the similar pictures of the Target Photo are inquired about from similarity table according to the Target Photo of user search and is formed
Raw recommendation list；
D, arrangement raw recommendation list obtain consequently recommended list and show.
Described combination picture material and the information recommendation method of keyword, wherein, in the step A, extract and include face
The process of the image content information of color characteristic and textural characteristics is specifically included：
By picture according to preassigned quantity piecemeal；
Each piecemeal by color cell process and obtains the histogram comprising several minizones, each minizone generation
A kind of color of table；
The textural characteristics at abstract image edge；
Color characteristic and textural characteristics to extracting are quantified and Regularization, obtain the CEDD Nogatas of every pictures
Figure.
Described combination picture material and the information recommendation method of keyword, wherein, linear sparse model in the step B
Algorithmic formula it is as follows：
Wherein, M, F represent keyword message matrix (m respectivelyij)m×nWith image content information matrix (fij)p×n, S is represented to be needed
Similarity matrix between the picture to be calculated, | | | |F、||·||1Frobenius norms and 1- norms, m, p point are represented respectively
The key characteristics quantity and image content features quantity of all pictures are not represented, and n represents picture number, α representative image contents
The factor of influence of information matrix, β, λ are regularization parameter.
Described combination picture material and the information recommendation method of keyword, wherein, F is normalized：
Wherein, the i-th rows of F (i, j) representing matrix F, jth column element；Min { F (i) } all units of the rows of representing matrix F i-th
The minimum value of element；The maximum of Max { F (i) } representing matrix F the i-th row all elements, 0≤i<p,0≤j<n.
Described combination picture material and the information recommendation method of keyword, wherein, the step B also includes：
When there is new picture to be added to picture library, the computing formula updated using sparse linear model incremental calculates new picture
With the similarity between other pictures in picture library, sparse linear model is updated.
Described combination picture material and the information recommendation method of keyword, wherein, the arrangement raw recommendation list is obtained
Further included to consequently recommended list and the step of displaying：
When multiple raw recommendation lists are obtained, the picture in all raw recommendation lists is filtered, and to filtering
Picture afterwards is ranked up and obtains consequently recommended list.
Described combination picture material and the information recommendation method of keyword, wherein, the arrangement raw recommendation list is obtained
Specifically included to consequently recommended list and the step of displaying：
Picture in raw recommendation list is filtered and is sorted and obtained consequently recommended list, shown described consequently recommended
List.
Described combination picture material and the information recommendation method of keyword, wherein, the target according to user search
Picture is inquired about the similar pictures of the Target Photo from similarity table and is specifically included the step of forming raw recommendation list：
Obtain the ID of the Target Photo of user search in user search picture, ID according to Target Photo is from similarity table
In inquire similar pictures related to Target Photo and form raw recommendation list.
A kind of information recommendation system of combination picture material and keyword, wherein, including：
Matrix conversion module, for extracting the keyword message of the picture of picture library and special comprising color characteristic and texture
The image content information levied, vector space model is expressed as by keyword message and image content information, obtains corresponding key
Word information matrix and image content information matrix；
Similarity calculation module, for utilizing Sparse model to keyword message matrix and image content information matrix
Processed, the similarity in calculating picture library between picture obtains similarity table；
Raw recommendation list acquisition module, the mesh is inquired about for the Target Photo according to user search from similarity table
Mark on a map piece similar pictures and form raw recommendation list；
Consequently recommended list acquisition module, obtains consequently recommended list and shows for arranging raw recommendation list.
Described combination picture material and the information recommendation system of keyword, wherein, the matrix conversion module includes：
Blocking unit, for by picture according to preassigned quantity piecemeal；
Color processing unit, for by color cell process by each piecemeal obtaining comprising the straight of several minizones
Fang Tu, each minizone represents a kind of color；
Texture processing unit, for the textural characteristics at abstract image edge；
CEDD histogram acquiring units, for extract color characteristic and textural characteristics quantified and regularization at
Reason, obtains the CEDD histograms of every pictures.
Described combination picture material and the information recommendation system of keyword, wherein, the similarity calculation module includes：
Algorithm unit, for according to similar between keyword message matrix and image content information matrix computations picture
Degree, wherein algorithmic formula is as follows：
Wherein, M, F represent keyword message matrix (m respectivelyij)m×nWith image content information matrix (fij)p×n, S is represented to be needed
Similarity matrix between the picture to be calculated, | | | |F、||·||1Frobenius norms and 1- norms, m, p point are represented respectively
The key characteristics quantity and image content features quantity of all pictures are not represented, and n represents picture number, α representative image contents
The factor of influence of information matrix, β, λ are regularization parameter.
Described combination picture material and the information recommendation system of keyword, wherein, the similarity calculation module includes：
Normalized unit, for being normalized to matrix F：
Wherein, the i-th rows of F (i, j) representing matrix F, jth column element；Min { F (i) } all units of the rows of representing matrix F i-th
The minimum value of element；The maximum of Max { F (i) } representing matrix F the i-th row all elements, 0≤i<p,0≤j<n.
Described combination picture material and the information recommendation system of keyword, wherein, similarity calculation module also includes：
Incremental computations unit, for when there is new picture to be added to picture library, being updated using sparse linear model incremental
Computing formula calculates the similarity between other pictures in new picture and picture library, updates sparse linear model.
Beneficial effect：The present invention calculates figure using Sparse models coupling keyword message and image content information
Similarity between piece, feature extraction is carried out including using CEDD models to picture material.Strong adaptability of the present invention, energy
Enough recommendation requests for meeting user in real time, while to be generally higher than that in the effect recommended single model is used, wherein in MRR
Amplification in this index can be close to 100%.
Brief description of the drawings
Fig. 1 is the flow chart of the information recommendation method preferred embodiment that the present invention combines picture material and keyword.
Fig. 2 is the online part of information recommendation system and the system of offline part that the present invention combines picture material and keyword
Block diagram.
Fig. 3 is the particular flow sheet of step S101 in method shown in Fig. 1.
Fig. 4 is the system block diagram of the information recommendation system preferred embodiment that the present invention combines picture material and keyword.
Fig. 5 is the concrete structure block diagram of matrix conversion module in system shown in Figure 4.
Fig. 6 is prec@n and recall@n index design sketch in recommendation effect using information recommendation system of the invention.
Fig. 7 is MAP, NDCG, MRR and AUC index design sketch in recommendation effect using information recommendation system of the invention.
Specific embodiment
The present invention provides the information recommendation method and system of a kind of combination picture material and keyword, to make mesh of the invention
, technical scheme and effect it is clearer, clear and definite, the present invention is described in more detail below.It should be appreciated that described herein
Specific embodiment be only used to explain the present invention, be not intended to limit the present invention.
Fig. 1 is referred to, Fig. 1 is the stream of the information recommendation method preferred embodiment that the present invention combines picture material and keyword
Cheng Tu, as illustrated, it includes：
S101, the keyword message of the picture of extraction picture library and the picture material comprising color characteristic and textural characteristics
Information, vector space model is expressed as by keyword message and image content information, obtain corresponding keyword message matrix and
Image content information matrix；
S102, keyword message matrix and image content information matrix are processed using Sparse model, calculated
Similarity in picture library between picture obtains similarity table；
S103, the similar pictures and shape of inquiring about the Target Photo from similarity table according to the Target Photo of user search
Into raw recommendation list；
S104, arrangement raw recommendation list obtain consequently recommended list and show.
Method in the present invention be not only applicable to picture retrieval recommend, it may also be used for the retrieval of various multimedia messages and
Recommend, such as on e-commerce website or TV shopping platform, many existing picture description informations of commodity also have attribute to close
Key word information, such as suitcase commodity, can both provide the picture of suitcase, can also provide and be closed including suitcase brand, type, size etc.
Key word information.So information recommendation system range of application of the invention is very wide, the citing in the present embodiment is not represented to this
The limitation of the range of application of invention.
The invention mainly comprises offline and online two parts, wherein step S101 and step S102 is offline part, and
Step S103 and step S104 is online part.With e-commerce website（Picture is commodity picture）Commending system as a example by,
Offline part therein, it is to set up a machine learning model effectively to combine and the row of calculating picture material and keyword
Sequence forms maximally related other commodity of each commodity, and the model can realize the renewal of increment type simultaneously（Describe later）, from
And new commodity are quickly effectively processed, with preferable scalability；Online part therein is then to commodity according to user
The commercial product recommending list of different commodity is browsed or combines the features such as buying behavior, so as to provide unified dependent merchandise
Recommendation list.
Information recommendation system of the invention, its framework are as shown in Fig. 2 online is partly to user on image retrieval platform
Retrieval and recommendation service are provided, searcher is mainly to picture library（Picture library refers to the database of mass picture information of being stored with,
For example for e-commerce website, it has the commodity picture storehouse comprising magnanimity commodity picture, and picture herein not only includes it
The picture material of itself, also comprising keyword message associated with it）Retrieved, the Image ID that record user search is crossed, so
The picture for inquiring correlation from similarity table afterwards forms raw recommendation list.Recommendation service is mainly made up of 3 parts：It is original
Recommendation list, filtering and sequence, recommendation service browsing or buying behavior acquisition raw recommendation list according to user first, it
System is filtered according to the preference or merchandise news of user afterwards, for example, undercarriage or the commodity that cannot buy
Automatic fitration is fallen；Because user may browse multiple commodity, therefore multiple commercial product recommending lists can be obtained, therefore system is also needed
The commodity repeated in these lists are filtered, final system is carried out to the list after filtering according to their weight
Sequence forms final recommendation list.
The purpose of offline part is to calculate picture similarity to obtain similarity table.The present invention is that have using sparse linear model
Effect image content information and keyword message are combined and calculates a unified picture similarity.
In step S101, two parts are extracted from picture：Keyword message and image content information, with commodity figure
As a example by piece, keyword message is mainly the attribute information of commodity, such as brand, type, size, price etc., and picture material
Information is mainly the natural characteristics such as color, the texture in image.
Keyword message can naturally be expressed as into a vector space model, and it is generally all a sparse square
Battle array, row represents attribute information, and row represent each merchandise items.
And image content information must be converted into a vector space model by associated picture Processing Algorithm, for example, pass through
The methods such as color histogram, wavelet transformation, Scale invariant features transform carry out color, texture, edge angle point information to image etc.
It is analyzed, each pictures is all converted into the vector on a feature space；And image content information is frequently not dilute
Thin, therefore, the present invention can utilize related algorithm that image content information is converted into sparse matrix, generally according to the spy of image
Levy during image is converted into vector, the vectorial degree of rarefication that different algorithms is produced is inconsistent, and the degree of rarefication pair of vector
The efficiency of algorithm and the degree of accuracy can be had a certain impact, and the present invention substantially may be used using the image vector degree of rarefication that CEDD algorithms are produced
To meet demand, if necessary to further adjustment degree of rarefication, sparse coding can be also used（Conventional image statisticses method）
Mode adjust.The purpose of do so can mainly be accelerated to calculate the efficiency of picture similarity.Asked by effectively iteration
Solution preocess, this model can picture be arrived in study on the basis of two different data sources of picture material and keyword well
Similarity, therefore the accuracy rate of Similarity Measure is substantially increased, such that it is able to be given in the picture library of magnanimity such as commodity storehouse
User more accurately recommends maximally related commodity.
For keyword message, the process that it is extracted can use existing keyword message extractive technique, and the present invention is not
Repeat again.For image content information, its extraction process as shown in figure 3, including：
S201, by picture according to preassigned quantity piecemeal；
S202, each piecemeal by color cell process obtain comprising 24 histograms of minizone, each cell
Between represent a kind of color；
S203, the textural characteristics using MPEG-7 abstract images edge；
S204, color characteristic and textural characteristics to extracting are quantified and Regularization, obtain every pictures
CEDD histograms.
CEDD histograms (Color and Edge Directivity Descriptornorm) are current popular
Abstract image color characteristic and textural characteristics model, extract and only account for 54 bytes of storage space, Er Qiexiang per pictures characteristic information
Than models such as MPEG-7, computing cost is smaller, therefore is widely used in image information retrieval field, and obtains preferable
Retrieval effectiveness.CEDD histograms are made up of texture cell and color cell, and texture cell includes 6 pieces of texture regions, color cell
Comprising 24 pieces of color regions, therefore 6 × 24 pieces of regions are included altogether.
In step s 102, keyword message matrix and image content information matrix can be carried out using Sparse model
Treatment is so as to calculate similarity.
The model commonly used in Collaborative Filtering Recommendation System includes neighbourhood model and matrix decomposition model.Neighbourhood model is pushed away
Computing cost is small when recommending, speed is fast, and its effect is set up and calculated exactly on article similarity basis；Matrix decomposition model enters
Effect is better than neighbourhood model when row is recommended, but computing cost is larger, it is impossible to carry out real-time recommendation.Therefore neighbourhood model is widely used
In real-time recommendation occasion, recommended during responding user's new behavior.
The method for calculating article similarity generally uses the equidistant measure formulas of cosine similarity, sets up in vector space mould
It is separate based on attribute it is assumed that being therefore difficult to calculate article similarity exactly on the basis of type；Other the method can not have
When effect ground combines two kinds of item information data sources, therefore the advantage that multiple data sources information is complementary to one another can not be played.
The sparse linear model that the present embodiment is used can then efficiently solve above mentioned problem, be believed by combining keyword
Breath and image content information, are modeled using machine learning algorithm, therefore calculate article similarity compared to based on data mapping
Method, sparse linear model can more accurately calculate similarity between article such as picture, be pushed away in real time while disclosure satisfy that
The requirement recommended.
In the present embodiment, the algorithmic formula of Sparse model is as follows：
Wherein, M, F represent keyword message matrix (m respectivelyij)m×nWith image content information matrix (fij)p×n, S is represented to be needed
Similarity matrix between the picture to be calculated, | | | |F、||·||1Frobenius norms and 1- norms, m, p point are represented respectively
The key characteristics quantity and image content features quantity of all pictures are not represented, and n represents picture number, wherein, α representative images
The factor of influence of content information matrix, for adjusting percentage contribution of the image content information when picture similarity is calculated, β, λ are equal
It is regularization parameter, for preventing the Sparse model over-fitting.
Keyword message matrix M element value 0 or 1, and image content information matrix F element value is whole between 0~7
Number, it is therefore desirable to be normalized to image content information matrix F.The present embodiment is adopted with the following method often to go F matrix and distinguished
It is normalized.
Wherein the i-th rows of F (i, j) representing matrix F, jth column element；Min { F (i) } representing matrix F the i-th row all elements
Minimum value；The maximum of Max { F (i) } representing matrix F the i-th row all elements, 0≤i<p,0≤j<n.
Additionally, in step s 102, it is similar between the picture and other pictures in order to calculate for the new picture of addition
Degree, the present embodiment completes Similarity Measure process using the method that sparse linear model incremental updates, and this process need not be again
The similarity between all commodity is all calculated, therefore, it is possible to quickly, efficiently carry out model modification such that it is able to recommend new
Picture.The computing formula that sparse linear model incremental updates is as follows：
Wherein, M (, n), F (, n) be the corresponding keyword feature vector of new picture and image content features
Vector；S (, n) it is similarity between new picture and other pictures；||·||2Represent 2- norms.
In last online part, user can be retrieved by system, such as when retrieving a Target Photo, obtain and use
The ID of the Target Photo of family retrieval, the ID according to Target Photo inquire the similar diagram related to Target Photo from similarity table
Piece forms raw recommendation list.
After raw recommendation list is obtained, raw recommendation list need to be arranged, it is arranged includes arranging raw recommendation
Picture in table is filtered and the treatment sorted is so as to obtain consequently recommended list.
Simultaneity factor can also be to the list after filtering, and weight according to picture is ranked up to raw recommendation list, institute
The weight stated refers to preference of the user to picture, such as, for commodity picture, the weight reflects user to corresponding business
The preference of product.Consequently recommended row are obtained according to being ranked up successively from high to low to the preference of picture according to user
Table.Weighted value herein can be the Similarity value for above calculating, it would however also be possible to employ existing method calculates user to each
The preference of picture, such as counting user are calculated to number of clicks, click frequency, the browsing time of picture etc. information
Weight.
Further, since the picture of user search may have multiple, then the raw recommendation list for obtaining has been likely to multiple,
The picture in all raw recommendation lists can now be filtered, being then ranked up combination to the picture after filtering obtains one
Individual consequently recommended list.For example extract and sort in each raw recommendation list preceding several pictures, be combined into one it is final
Recommendation list, also can be according to the weight of each Target Photo of retrieval（Weight herein can also be obtained according to foregoing similar approach）
Come the picture number for determining to be extracted needed for each raw recommendation list, a consequently recommended list is obtained.
So that the present invention is applied to e-commerce website as an example, illustrate, it is assumed that original article number is n, add one newly
During commodity, M, F, s-matrix respectively become m × (n+1), p × (n+1), (n+1) × (n+1) matrixes, i.e. matrix M, F is added respectively
The corresponding keyword feature vector of new commodity and image content features vector, constitute matrix M, F (n+1)th and arrange；Matrix S adds a line
One row, i.e. similarity between new commodity and other commodity；By according to above-mentioned computing formula solve can calculate S (, n),
The namely similarity between new commodity and other commodity, the corresponding keyword of new commodity is added additionally, due to matrix M, F
When characteristic vector and image content features vector, on similarity influence very little between original article, therefore can be with calculated
Old matrix S (, k), 0≤k<N, without recalculate S (, k), 0≤k<N, so as to complete sparse linear model
Incremental update, saves the substantial amounts of calculating time.
Implementation result of the invention is illustrated below by specific embodiment.
Data Collection：Experiment uses women's bag data, the keyword message and picture material of 440 women's bags of collection in Taobao
Information.The keyword message descriptive labelling information such as including color, brand, sex, style, size；Image content information is women's bag
Picture.
Rating collection：For each women's bag, crucial 20 women's bags that WD is similar or picture material is similar of screening allow
Three people are evaluated, and provide the scoring of each women's bag and other women's bag similarities, are averaged as final scoring.Scoring level
0-5 points, 0 represents least similar, and 5 represent most like.Average mark >=3 are screened as test set, for the effect of assessment algorithm.
Evaluation index：The conventional evaluation index of commending system includes prec@n, recall@n, MAP, NDCG, MRR, AUC.
prec@n：For assessing during n commercial product recommending before commending system is produced, the dependent merchandise of return is in recommendation list
Shared ratio, i.e. accuracy rate.
recall@n：For assessing during n commercial product recommending before commending system is produced, the dependent merchandise of return is in whole correlations
Shared ratio, i.e. recall rate in commodity.
MAP：For assess commending system return recommended project correspondence real user like preference ranking it is average accurate
Consider the sequencing information of commercial product recommending list dependent merchandise when rate calculates the index, thus in recommendation list dependent merchandise sequence
More forward, MAP may be higher.
NDCG：For assessing the matching journey between commending system generation commercial product recommending list and the actual list of test set
Degree, the index considers the sequencing information of recommendation list commodity.The main thought of the index is that the commodity that user likes are come
Than can to a greater extent increase Consumer's Experience behind coming before recommendation list, thus correlation commodity sequence higher it is forward when
NDCG values are higher, otherwise NDCG values are relatively low.
MRR：It is the inverted degree of accuracy as it of sequence using model answer in the system of being evaluated provides result, it is then right
All of problem is averaged, i.e., all users are averaged.
AUC：ROC (receiver operator curve) area under a curve is represented, it weighs a commending system energy
The enough commodity for what extent liking user and the commodity not liked are distinguished.
Above-mentioned evaluation index is evaluates the conventional mechanism of searching algorithm, and particular technique is the details present invention repeat no more.
During assessment commending system, the desired value such as prec@n, recall@n, MAP, NDCG, MRR, AUC is higher, shows that this is pushed away
The recommendation results for recommending method generation are more accurate, with preferably recommendation effect.
Tested on above-mentioned data set, These parameters are estimated respectively, to evaluate the effect of sparse linear model
Really.
Be can see from Fig. 4 and Fig. 5, in all 6 conventional evaluation indexes, using model of the invention
（SpareLinearMode）Substantially than the model based on keyword is used alone（Texual）Or image content-based
（Visual）Model it is all good, particularly in MRR this index, amplification close to 100%, wherein it can also be seen that based on key
Model of the model of word generally than image content-based is good, but incorporates image content information to aftereffect in model of the invention
Fruit is significantly improved, and this also demonstrates model of the invention and combines both approaches well such that it is able to preferably
Recommend related commodity to user.
Based on the above method, the present invention also provides the information recommendation system of a kind of combination picture material and keyword, such as Fig. 6
It is shown, including：
Matrix conversion module 100, for extracting the keyword message of the picture of picture library and comprising color characteristic and texture
The image content information of feature, vector space model is expressed as by keyword message and image content information, obtains corresponding pass
Key word information matrix and image content information matrix；
Similarity calculation module 200, for utilizing Sparse model to keyword message matrix and image content information
Matrix is processed, and the similarity in calculating picture library between picture obtains similarity table；
Raw recommendation list acquisition module 300, institute is inquired about for the Target Photo according to user search from similarity table
State the similar pictures of Target Photo and form raw recommendation list；
Consequently recommended list acquisition module 400, obtains consequently recommended list and shows for arranging raw recommendation list.Close
It has been described in detail in method above in the ins and outs of above-mentioned module, therefore has been repeated no more.
Further, as shown in fig. 7, the matrix conversion module 100 includes：
Blocking unit 110, for by picture according to preassigned quantity piecemeal；
Color processing unit 120, obtains comprising several minizones for by color cell process each piecemeal
Histogram, each minizone represents a kind of color；
Texture processing unit 130, for the textural characteristics at abstract image edge；
CEDD histograms acquiring unit 140, is quantified and regularization for the color characteristic and textural characteristics to extracting
Treatment, obtains the CEDD histograms of every pictures.Ins and outs on above-mentioned modular unit are existing detailed in method above
State, therefore repeat no more.
Further, the similarity calculation module includes：
Algorithm unit, for according to similar between keyword message matrix and image content information matrix computations picture
Degree, wherein algorithmic formula is as follows：
Wherein, M, F represent keyword message matrix (m respectivelyij)m×nWith image content information matrix (fij)p×n, S is represented to be needed
Similarity matrix between the picture to be calculated, | | | |F、||·||1Frobenius norms and 1- norms, m, p point are represented respectively
The key characteristics quantity and image content features quantity of all pictures are not represented, and n represents picture number.
Further, the similarity calculation module includes：
Normalized unit, for being normalized to matrix F：
Wherein, the i-th rows of F (i, j) representing matrix F, jth column element；Min { F (i) } all units of the rows of representing matrix F i-th
The minimum value of element；The maximum of Max { F (i) } representing matrix F the i-th row all elements, 0≤i<p,0≤j<n.On above-mentioned
The ins and outs of modular unit have been described in detail in method above, therefore repeat no more.
Further, similarity calculation module also includes：
Incremental computations unit, for when there is new picture to be added to picture library, being updated using sparse linear model incremental
Computing formula calculates the similarity between other pictures in new picture and picture library, updates sparse linear model.
In sum, the present invention calculates figure using Sparse models coupling keyword message and image content information
Similarity as between, feature extraction is carried out including using CEDD models to picture material.Strong adaptability of the present invention, energy
Enough recommendation requests for meeting user in real time, while to be generally higher than that in the effect recommended single model is used, wherein in MRR
Amplification in this index can be close to 100%.
It should be appreciated that application of the invention is not limited to above-mentioned citing, and for those of ordinary skills, can
To be improved according to the above description or converted, all these modifications and variations should all belong to the guarantor of appended claims of the present invention
Shield scope.
Claims (13)
1. the information recommendation method of a kind of combination picture material and keyword, it is characterised in that including step：
A, the keyword message of the picture of extraction picture library and the image content information comprising color characteristic and textural characteristics, will
Keyword message and image content information are expressed as vector space model, obtain corresponding keyword message matrix and picture material
Information matrix；
B, keyword message matrix and image content information matrix are processed using Sparse model, in calculating picture library
Similarity between picture obtains similarity table；
C, the similar pictures of the Target Photo are inquired about from similarity table according to the Target Photo of user search and forms original
Recommendation list；
D, arrangement raw recommendation list obtain consequently recommended list and show.
2. the information recommendation method of combination picture material according to claim 1 and keyword, it is characterised in that the step
In rapid A, the process for extracting the image content information comprising color characteristic and textural characteristics is specifically included：
By picture according to preassigned quantity piecemeal；
Each piecemeal by color cell process and obtains the histogram comprising several minizones, each minizone represents one
Plant color；
The textural characteristics at abstract image edge；
Color characteristic and textural characteristics to extracting are quantified and Regularization, obtain the CEDD histograms of every pictures.
3. the information recommendation method of combination picture material according to claim 1 and keyword, it is characterised in that the step
The algorithmic formula of linear sparse model is as follows in rapid B：
,；
Wherein, M, F represent keyword message matrix respectivelyWith image content information matrix,Expression needs meter
Similarity matrix between the picture of calculation,、Frobenius norms and 1- norms are represented respectively, and m, p represent all figures respectively
The key characteristics quantity and image content features quantity of piece, n represent picture number, the shadow of α representative image content information matrixes
The factor is rung, β, λ are regularization parameter.
4. the information recommendation method of combination picture material according to claim 3 and keyword, it is characterised in that enter to F
Row normalized：
；
Wherein,The rows of representing matrix F i-th, jth column element；The minimum of representing matrix F the i-th row all elements
Value；The maximum of representing matrix F the i-th row all elements,。
5. the information recommendation method of combination picture material according to claim 1 and keyword, it is characterised in that the step
Rapid B also includes：
When there is new picture to be added to picture library, the computing formula updated using Sparse model incremental calculates new picture with figure
The similarity between other pictures in valut, updates Sparse model.
6. the information recommendation method of combination picture material according to claim 1 and keyword, it is characterised in that described whole
Reason raw recommendation list obtains consequently recommended list and is further included the step of displaying：
When multiple raw recommendation list is obtained, the picture in all raw recommendation lists is filtered, and to filtering after
Picture is ranked up and obtains consequently recommended list.
7. the information recommendation method of combination picture material according to claim 1 and keyword, it is characterised in that described whole
Reason raw recommendation list obtains consequently recommended list and is specifically included the step of displaying：
Picture in raw recommendation list is filtered and is sorted and obtained consequently recommended list, shown the consequently recommended row
Table.
8. the information recommendation method of combination picture material according to claim 1 and keyword, it is characterised in that described
The similar pictures of the Target Photo are inquired about from similarity table according to the Target Photo of user search and raw recommendation list is formed
The step of specifically include：
The ID of the Target Photo of user search is obtained in user search picture, the ID according to Target Photo is looked into from similarity table
The similar pictures for asking Target Photo form raw recommendation list.
9. the information recommendation system of a kind of combination picture material and keyword, it is characterised in that including：
Matrix conversion module, for extracting the keyword message of the picture of picture library and comprising color characteristic and textural characteristics
Image content information, vector space model is expressed as by keyword message and image content information, obtains corresponding keyword letter
Breath matrix and image content information matrix；
Similarity calculation module, for being carried out to keyword message matrix and image content information matrix using Sparse model
Treatment, the similarity in calculating picture library between picture obtains similarity table；
Raw recommendation list acquisition module, the target figure is inquired about for the Target Photo according to user search from similarity table
The similar pictures of piece simultaneously form raw recommendation list；
Consequently recommended list acquisition module, obtains consequently recommended list and shows for arranging raw recommendation list.
10. the information recommendation system of combination picture material according to claim 9 and keyword, it is characterised in that described
Matrix conversion module includes：
Blocking unit, for by picture according to preassigned quantity piecemeal；
Color processing unit, the Nogata comprising several minizones is obtained for by color cell process each piecemeal
Figure, each minizone represents a kind of color；
Texture processing unit, for the textural characteristics at abstract image edge；
CEDD histogram acquiring units, are quantified and Regularization for the color characteristic and textural characteristics to extracting, and are obtained
To the CEDD histograms of every pictures.
The information recommendation system of 11. combination picture materials according to claim 9 and keyword, it is characterised in that described
Similarity calculation module includes：
Algorithm unit, for according to the similarity between keyword message matrix and image content information matrix computations picture, its
Middle algorithmic formula is as follows：
,；
Wherein, M, F represent keyword message matrix respectivelyWith image content information matrix,Expression needs meter
Similarity matrix between the picture of calculation,、Frobenius norms and 1- norms are represented respectively, and m, p represent all figures respectively
The key characteristics quantity and image content features quantity of piece, n represent picture number, the shadow of α representative image content information matrixes
The factor is rung, β, λ are regularization parameter.
The information recommendation system of 12. combination picture materials according to claim 11 and keyword, it is characterised in that described
Similarity calculation module includes：
Normalized unit, for being normalized to matrix F：
；
Wherein,The rows of representing matrix F i-th, jth column element；The minimum of representing matrix F the i-th row all elements
Value；The maximum of representing matrix F the i-th row all elements,。
The information recommendation system of 13. combination picture materials according to claim 9 and keyword, it is characterised in that similar
Degree computing module also includes：
Incremental computations unit, for when there is new picture to be added to picture library, using the calculating of Sparse model incremental renewal
Formula calculates the similarity between other pictures in new picture and picture library, updates Sparse model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310436726.5A CN103544216B (en) | 2013-09-23 | 2013-09-23 | The information recommendation method and system of a kind of combination picture material and keyword |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310436726.5A CN103544216B (en) | 2013-09-23 | 2013-09-23 | The information recommendation method and system of a kind of combination picture material and keyword |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103544216A CN103544216A (en) | 2014-01-29 |
| CN103544216B true CN103544216B (en) | 2017-06-06 |
Family
ID=49967668
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310436726.5A Active CN103544216B (en) | 2013-09-23 | 2013-09-23 | The information recommendation method and system of a kind of combination picture material and keyword |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103544216B (en) |
Families Citing this family (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104166836B (en) * | 2014-05-08 | 2017-07-14 | 清华大学深圳研究生院 | A kind of multiple dimensioned engineering truck recognition methods of piecemeal modeled based on multiple features and system |
| CN105227973A (en) * | 2014-06-27 | 2016-01-06 | 中兴通讯股份有限公司 | Based on information recommendation method and the device of scene Recognition |
| KR20160051390A (en) * | 2014-11-03 | 2016-05-11 | 삼성전자주식회사 | Electronic device and method for providing filter n electronic device |
| CN104881798A (en) * | 2015-06-05 | 2015-09-02 | 北京京东尚科信息技术有限公司 | Device and method for personalized search based on commodity image features |
| CN106909497B (en) * | 2015-12-22 | 2021-01-19 | 阿里巴巴集团控股有限公司 | Test method and device |
| CN105608609B (en) * | 2016-02-17 | 2018-02-16 | 北京金山安全软件有限公司 | Method and device for pushing travel information and electronic equipment |
| CN106127525A (en) * | 2016-06-27 | 2016-11-16 | 浙江大学 | A kind of TV shopping Method of Commodity Recommendation based on sorting algorithm |
| CN107766853B (en) * | 2016-08-16 | 2021-08-06 | 阿里巴巴集团控股有限公司 | Image text information generation and display method and electronic equipment |
| CN108874813B (en) * | 2017-05-10 | 2022-07-29 | 腾讯科技（北京）有限公司 | Information processing method, device and storage medium |
| CN107301238A (en) * | 2017-06-28 | 2017-10-27 | 北京京东尚科信息技术有限公司 | The processing method and system of user's request |
| CN107833082B (en) * | 2017-09-15 | 2022-04-12 | 唯品会(海南)电子商务有限公司 | Commodity picture recommendation method and device |
| CN107609201A (en) * | 2017-10-25 | 2018-01-19 | 广东工业大学 | A kind of recommended models generation method and relevant apparatus based on commending system |
| CN108363804B (en) * | 2018-03-01 | 2020-08-21 | 浙江工业大学 | Local model weighted fusion Top-N movie recommendation method based on user clustering |
| CN108920654B (en) * | 2018-06-29 | 2021-10-29 | 泰康保险集团股份有限公司 | Question and answer text semantic matching method and device |
| CN109344314B (en) * | 2018-08-20 | 2021-11-16 | 腾讯科技（深圳）有限公司 | Data processing method and device and server |
| CN109145994B (en) * | 2018-08-28 | 2021-01-05 | 昆明理工大学 | Image similarity judgment method based on improved color variance algorithm |
| CN110955822B (en) * | 2018-09-25 | 2024-02-06 | 北京京东尚科信息技术有限公司 | Commodity searching method and device |
| CN109446430B (en) * | 2018-11-29 | 2021-10-01 | 西安电子科技大学 | Product recommendation method and device, computer equipment and readable storage medium |
| CN110321966B (en) * | 2019-07-11 | 2022-02-15 | 同方知网数字出版技术股份有限公司 | Method for accelerating book similarity analysis speed |
| CN110705612A (en) * | 2019-09-18 | 2020-01-17 | 重庆邮电大学 | Sentence similarity calculation method, storage medium and system with mixed multi-features |
| CN111428040B (en) * | 2020-04-01 | 2023-08-18 | 中移雄安信息通信科技有限公司 | Target object determination method, device, equipment and computer storage medium |
| CN112581161B (en) * | 2020-12-04 | 2024-01-19 | 上海明略人工智能(集团)有限公司 | Object selection method and device, storage medium and electronic equipment |
| CN113822735A (en) * | 2021-02-24 | 2021-12-21 | 北京沃东天骏信息技术有限公司 | Goods recommendation method and device, storage medium and electronic equipment |
| CN113129269A (en) * | 2021-03-23 | 2021-07-16 | 东北林业大学 | Method for automatically classifying concrete surface cavities by selecting variables from image texture features |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103226616A (en) * | 2013-05-16 | 2013-07-31 | 南京龙渊微电子科技有限公司 | Image content retrieval system and image content sparse learning method thereof |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7457825B2 (en) * | 2005-09-21 | 2008-11-25 | Microsoft Corporation | Generating search requests from multimodal queries |
-
2013
- 2013-09-23 CN CN201310436726.5A patent/CN103544216B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103226616A (en) * | 2013-05-16 | 2013-07-31 | 南京龙渊微电子科技有限公司 | Image content retrieval system and image content sparse learning method thereof |
Non-Patent Citations (2)
| Title |
|---|
| 图像检索中的特征表示模型和多信息源融合方式的研究;陆文婷;《中国博士学位论文全文数据库信息科技辑》;20120115(第1期);第I138-64页 * |
| 基于半监督和主动学习相结合的图像的检索研究;林良思;《中国优秀硕士学位论文全文数据库信息科技辑》;20110615(第6期);第I138-537页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103544216A (en) | 2014-01-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103544216B (en) | The information recommendation method and system of a kind of combination picture material and keyword | |
| CN107220365B (en) | Accurate recommendation system and method based on collaborative filtering and association rule parallel processing | |
| CN105224699B (en) | News recommendation method and device | |
| CN103744966B (en) | A kind of item recommendation method, device | |
| CN110674407B (en) | Hybrid recommendation method based on graph convolution neural network | |
| CN111444395B (en) | Method, system and equipment for obtaining relation expression between entities and advertisement recall system | |
| CN110532479A (en) | A kind of information recommendation method, device and equipment | |
| CN108717407B (en) | Entity vector determination method and device, and information retrieval method and device | |
| CN108334592B (en) | Personalized recommendation method based on combination of content and collaborative filtering | |
| CN110175895B (en) | Article recommendation method and device | |
| CN108563690B (en) | Collaborative filtering recommendation method based on object-oriented clustering | |
| CN112052387B (en) | Content recommendation method, device and computer readable storage medium | |
| De et al. | Discriminative link prediction using local, community, and global signals | |
| Jin et al. | Reinforced similarity integration in image-rich information networks | |
| CN107885852A (en) | A kind of APP based on APP usage records recommends method and system | |
| CN109460519B (en) | Browsing object recommendation method and device, storage medium and server | |
| CN111651678B (en) | Personalized recommendation method based on knowledge graph | |
| Zhou et al. | Relevance feature mapping for content-based multimedia information retrieval | |
| CN101950400A (en) | Network shopping guiding method | |
| CN109840833A (en) | Bayes's collaborative filtering recommending method | |
| CN108920647B (en) | Low-rank matrix filling TOP-N recommendation method based on spectral clustering | |
| Liu et al. | Fast recommendation on latent collaborative relations | |
| CN112749330B (en) | Information pushing method, device, computer equipment and storage medium | |
| JP2018073429A (en) | Retrieval device, retrieval method, and retrieval program | |
| CN111858972A (en) | Movie recommendation method based on family knowledge graph |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |