CN103530304A - On-line recommendation method, system and mobile terminal based on self-adaption distributed computation - Google Patents
On-line recommendation method, system and mobile terminal based on self-adaption distributed computation Download PDFInfo
- Publication number
- CN103530304A CN103530304A CN201310171026.8A CN201310171026A CN103530304A CN 103530304 A CN103530304 A CN 103530304A CN 201310171026 A CN201310171026 A CN 201310171026A CN 103530304 A CN103530304 A CN 103530304A
- Authority
- CN
- China
- Prior art keywords
- online
- matrix
- model
- treating apparatus
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9535—Search customisation based on user profiles and personalisation
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses an on-line recommendation method, system and mobile terminal based on self-adaption distributed computation. The self-adaption distributed computation is achieved mainly through that each processing device samples large data and then trains matrix decomposition models in a self-adaption mode, then incremental on-line model updating is carried out in a spontaneous mode, and finally weighting integrated recommendation is carried out on each model. According to the recommendation system, a cluster is built without any distributed computation model, mass data can be processed effectively through a single or multiple ordinary computers, the stability is good, the expandability is high, cost can be saved greatly and development efficiency is improved greatly.
Description
Technical field
The present invention relates to intelligent recommendation technical field, relate in particular to a kind of online recommend method, system and mobile terminal calculating based on adapter distribution.
Background technology
How from the large data of magnanimity, finding the interested information of user, how to allow information be subject to users' welcome, is a very difficult thing.The task of commending system is exactly contact user and information, helps user to find, to own valuable information, information can be presented in face of its interesting user, thereby realize information consumer and informant's doulbe-sides' victory.
Commending system is mainly the behavior by analysis user, to its modeling, thereby comes the interest of predictive user to make recommendation by model.Main method can be divided into information filtering, collaborative filtering, and the model based on matrix decomposition.Information filtering is to user, to recommend and the article of liking before them similar other article in terms of content on the basis based on article content.Collaborative filtering finds similar user to make recommendation with similar article by the behavioral data of analysis user.Model based on matrix decomposition is by finding that implicit feature (such as classification) comes contact user interest and article.This model is determined the weight of article in this classification by the method for matrix decomposition on user's behavioral data, then calculates the interest level of user to article, thereby user is recommended.The algorithm major part of information filtering and collaborative filtering is all some statistical methods on the interior perhaps user behavior data of article, and model based on matrix decomposition is a kind of method of machine learning, can better learn out the relation between user and article, therefore this model has been widely applied in the commending system of current main flow.
Although the modelling effect based on matrix decomposition is good, normally as a kind of computation model of off-line.Because it requires to load whole data in internal memory the inside, and Time Calculation complexity is also very high.On common machines, be difficult to utilize matrix decomposition model to dispose commending system to the large data of magnanimity.Also have at present and propose to utilize distributed computing platform such as MPI (Message Passing Interface) or MapReduce carry out rapidly distributed matrix decomposition operation in large-scale group system, simultaneously by increment type model realization online updating and recommendation.Yet this method need to be built high-performance computer, the framework complexity of simultaneously disposing group system and Distributed Calculation is high, and is unfavorable for the maintenance and expansion of system.
In view of this, how a kind of fast, stable, reliable for large design data, effectively based on matrix decomposition, at line model, current intelligent recommendation system is played to vital effect.
Summary of the invention
In view of deficiency of the prior art, the object of the invention is to provide a kind of online recommend method and system of calculating based on adapter distribution.Be intended to solve the computation complexity facing when intelligent recommendation system in prior art utilizes matrix decomposition model to process the large data of magnanimity high, safeguard the problems such as expansion difficulty.
Technical scheme of the present invention is as follows:
The online recommend method that adapter distribution calculates, for by treating apparatus, mass data information processing rear line being recommended, wherein, described online recommend method comprises the following steps:
A, according to the processing power for the treatment of apparatus, distribute after adopting matrix sampling algorithm based on norm to sample from mass data information, make each treating apparatus can process alone the data message of distribution;
B, the matrix decomposition model of utilization based on amount of bias are trained distributed data message, obtain a score in predicting model;
C, by increment type online updating method, upgrade described score in predicting model;
D, by the score in predicting model after the integrated renewal of weighting, obtain the final recommendation list to user.
The described online recommend method calculating based on adapter distribution, wherein, in described steps A, adopts the matrix sampling algorithm based on norm specifically to comprise the following steps:
A1, obtain data matrix corresponding to mass data information;
A2, the row and column of described data matrix is sampled simultaneously, obtain a submatrix; And guarantee that according to the first of vector or second norm data and the degree of approximation between mass data that the submatrix after sampling comprises are less than predetermined error threshold.
The described online recommend method calculating based on adapter distribution, wherein, in described step C, increment type online updating method comprises known users is predicted and new user/article are predicted.
The described online recommend method calculating based on adapter distribution, wherein, the weight during weighting described in described step D is integrated is to distribute according to the processing power of each treating apparatus.
The described online recommend method calculating based on adapter distribution, wherein, described treating apparatus is computing machine, the processing power of described treating apparatus comprises internal memory and the arithmetic capability of computing machine.
The online commending system that adapter distribution calculates, for by treating apparatus, mass data information processing rear line being recommended, wherein, described online commending system comprises:
Adaptive load balancing unit, for according to the processing power for the treatment of apparatus, distributes after adopting matrix sampling algorithm based on norm to sample from mass data information, makes each treating apparatus can process alone the data message of distribution;
Distributed matrix resolving cell, for utilizing the matrix decomposition model based on amount of bias to train distributed data message, obtains a score in predicting model;
Increment type online updating unit, for upgrading described score in predicting model by increment type online updating method;
Online integration recommendation unit, obtains the final recommendation list to user for the score in predicting model by after the integrated renewal of weighting.
The described online commending system calculating based on adapter distribution, wherein, described treating apparatus is computing machine, the processing power of described treating apparatus comprises internal memory and the arithmetic capability of computing machine.
The described online commending system calculating based on adapter distribution, wherein, in described increment type online updating unit, increment type online updating method comprises known users is predicted and new user/article are predicted.
The described online commending system calculating based on adapter distribution, wherein, the weight during weighting described in described Online integration recommendation unit is integrated is to distribute according to the processing power of each treating apparatus.
, wherein, comprise the above-mentioned online commending system calculating based on adapter distribution.
Beneficial effect:
Online recommend method, system and the mobile terminal calculating based on adapter distribution of the present invention, wherein, described commending system does not need to remove to set up cluster by any distributed computing platform.And the processing mass data that can effectively process on the logical treating apparatus of separate unit or many Daeporis, good stability not only, extensibility is high, but also can greatly cost-saving and development efficiency.
Accompanying drawing explanation
Fig. 1 is the process flow diagram of the online recommend method calculating based on adapter distribution of the present invention.
Fig. 2 is the structured flowchart of the online commending system calculating based on adapter distribution of the present invention.
Fig. 3 is the framework schematic diagram of the commending system based on matrix decomposition of prior art.
Fig. 4 is the framework schematic diagram of the online commending system calculating based on adapter distribution of the present invention.
Embodiment
The invention provides a kind of online recommend method, system and mobile terminal calculating based on adapter distribution, for making object of the present invention, technical scheme and effect clearer, clear and definite, below the present invention is described in more detail.Should be appreciated that specific embodiment described herein, only in order to explain the present invention, is not intended to limit the present invention.
Refer to Fig. 1, it is the process flow diagram of the online recommend method calculating based on adapter distribution of the present invention.The described online recommend method calculating based on adapter distribution, for by treating apparatus, mass data information processing rear line being recommended, as shown in Figure 1, described online recommend method comprises the following steps:
S1, according to the processing power for the treatment of apparatus, distribute after adopting matrix sampling algorithm based on norm to sample from mass data information, make each treating apparatus can process alone the data message of distribution;
S2, the matrix decomposition model of utilization based on amount of bias are trained distributed data message, obtain a score in predicting model;
S3, by increment type online updating method, upgrade described score in predicting model;
S4, by the score in predicting model after the integrated renewal of weighting, obtain the final recommendation list to user.
For above-mentioned steps, be described in detail respectively below:
Described step S1 is the processing power according to treating apparatus, and the matrix sampling algorithm of employing based on norm distributes after sampling from mass data information, makes each treating apparatus can process alone the data message of distribution.
Conventionally to the algorithm of large data processing, be all based on MapReduce model.First data are carried out to burst, then on many machines (being treating apparatus), fragment data is processed simultaneously, finally each data relevant in Map are summed up together.The advantage of this model is first; by this distributed treatment framework of MapReduce; can not only be for the treatment of large-scale data; and a lot of loaded down with trivial details details can be stashed; such as; the standby management of automatically parallelizing, load balancing and calamity etc., like this by the greatly development of the person of simplifying procedures; The retractility of the second, MapReduce is very good, that is to say, every increase by one station server, and it just can be linked into much the same computing power in cluster, and most of distributed treatment frameworks in the past all differ greatly with MapReduce aspect retractility.But shortcoming is: the first, input Interval data is become to the fragment of fixed size, then by MapReduce platform processes, processing delay is directly proportional to the length of data slot, the expense of initialization process task like this.Little segmentation meeting reduces and postpones, and increases additional overhead, and the management of the dependence between segmentation more complicated (for example segmentation may need the information of previous segmentation); Otherwise large segmentation meeting increases delay.Optimized fragment size depends on concrete application; The second, in order to support Stream Processing, MapReduce need to be transformed into the pattern of Pipeline, rather than Reduce directly exports; Consider efficiency, intermediate result is preferably only kept in internal memory etc.These changes increase the complexity of original MapReduce framework greatly, are unfavorable for the maintenance and expansion of system.
And in the present invention, adopt a kind of dynamic adaptivity load-balancing technique, namely according to the processing power of each computing machine, carry out distribute data sheet, every treating apparatus (in the present embodiment, described treating apparatus is computing machine) participate in calculating in the data that can distribute at it independently, and do not need to each other to communicate by letter by message mechanism.The benefit of doing is like this: the first, if fault has appearred in certain computing machine, other computing machine still can independently carry out computing; The second, because we do not need to set up cluster, so any computing machine can be used fully, like this can be greatly cost-saving; The 3rd, extensibility is good, and any computing machine can seamlessly access in this system, and does not but bring the expense of extra operation and management simultaneously.
In addition, for mass data information (being that data message amount is enough large), adopt the random sampling based on matrix norm can guarantee to be in theory similar to as much as possible raw data, the data of sampling also have certain representativeness simultaneously.Take film recommended website as main, in thousands of portions film, evaluated, viewed film often conventionally just several thousand, the film number that has statistics to show that active user on average watches is also no more than 3000, so utilize the method for current existing matrix decomposition to do effective recommendation to user in the film that we can only sample out at these.In the present embodiment, adopt the matrix sampling algorithm based on norm specifically to comprise the following steps:
S11, obtain data matrix corresponding to mass data information;
S12, the row and column of described data matrix is sampled simultaneously, obtain a submatrix; And guarantee that according to the first of vector or second norm data and the degree of approximation between mass data that the submatrix after sampling comprises are less than predetermined error threshold.
Specifically, we adopt the following grab sample algorithm based on norm:
First input: original matrix
, the number of samples of row and column is p and q; Then output packet is containing the submatrix of the capable q row of p, simultaneously according to the ratio of the second norm calculation row and column in whole matrix norm, with the norm ratio that example (sampling of row is similar) generates each row as follows that is sampled to being listed as
, such as
.R is normalized:
, the value after normalization just can be used as the probability of this sample sampling, as follows the interval P of generating probability:
be created at random the number between [0,1], if in certain interval of P, just extract that corresponding sample.
Described step S2 is that the matrix decomposition model of utilization based on amount of bias trained distributed data message, obtains a score in predicting model.
Traditional matrix disassembling method based on SVD (Singular Value Decomposition) need to all import data in internal memory, and computation complexity is nonlinear, cannot process the data of magnanimity at all.The most of SGD (Stochastic Gradient Descent) adopting of matrix decomposition model generally using in commending system carrys out training data, by such method, matrix decomposition model can solve the sparse property problem of data effectively, but still limited to large data-handling capacity.There is recently document to propose then to utilize the distributed computing platform of MapReduce to carry out the problem of parallel processing matrix decomposition by subdivision matrix, but this has proposed very high requirement to hardware, also need to be familiar with very much how with these distributed computing platforms, effectively to calculate simultaneously.
And in the present invention, by adaptivity load balancing, can realize and not need to set up group system and just can carry out the task of Distributed Calculation, and every machine can effectively calculate individually according to the ability of self.Because we utilize matrix sampling model to realize to extract from data matrix the information of the expression original matrix that a small amount of row and column just can be similar to.Based on this theory, can on the data matrix after sampling, carry out independently matrix decomposition computing.So only need on the logical machine of every Daepori, preserve certain sample data both can obtain a score in predicting model fast.This score in predicting model Time & Space Complexity is not very high, and can guarantee validity and the stability of commending system.Therefore this distributed matrix decomposition model has efficient processing power to the large data of magnanimity.Its specific practice is as follows: because user behavior data comprises several very important information conventionally: the global mean value of the scoring of all records, the amount of bias of user's scoring and article are accepted the amount of bias of scoring.By hidden class and amount of bias, come the model of associated user and article to be called the matrix decomposition based on amount of bias.So, in technical scheme of the present invention, adopt a kind of matrix decomposition model SVDFeature based on amount of bias, be described as:
Y is target scoring,
average mark,
global characteristics amount of bias,
user's amount of bias,
article amount of bias,
user's recessive character,
article recessive characters.Method based on SGD, can obtain the correlation parameter of these amount of bias, as shown in table 1:
Table 1
Described step S3 is for to upgrade described score in predicting model by increment type online updating method.
When the list of user's generating recommendations, need to calculate the interest weight of user to all article, rank then, returns to N article of weight maximum.Therefore so, when number of articles is a lot, the time complexity of this process is very high, and the speed that generates user's recommendation list is very slow, can not calculate in real time online, and need off-line to be stored in database all users' recommendation results is computed in advance.Therefore, matrix decomposition model is all static method conventionally, can not carry out online recommendation in real time, that is to say, when user has had after new behavior, his recommendation list can not change.And in actual application, user's data constantly increase, therefore online recommendation is very important.At present document has proposed some online matrix increment type models and can upgrade fast the single model having calculated, but how in the environment of Distributed Calculation, effectively processes replacement problem and play a part very important to whole commending system effect.
In the present invention, the increment type online updating of single matrix decomposition model is expanded in distributed computing environment.Again, need to consider two kinds of situations: the first, known user; The second, new user or new article.To known user (user who had trained on certain machine), this programme adopt one fast update strategy directly carry out online updating; To new user (comprise known user but do not train on this machine) and new article, adopt another one fast update strategy carry out model modification.This online updating model proposing in distributed computing environment can upgrade model more effectively fast, thereby can accelerate efficiently the speed of online recommendation, while is resolution system cold start-up problem effectively, comprises if the user who newly adds is recommended article and the article that newly add are recommended to user.
Its specific practice is as follows: the matrix decomposition model based on amount of bias that integrating step S2 obtains, then add global characteristics, user characteristics and article characteristics to for user like making personalized recommendation, yet this model itself is not supported incrementally updating.Can not distributed earth parallel computation, so this model still cannot be processed the data of magnanimity.We introduce on this basis Distributed Calculation module and just can well address this problem, because model of the present invention is to be based upon on the basis of on single common machines, data from the sample survey being trained, therefore can utilize the algorithm of lower two kinds of online updatings to upgrade the model on each machine.Renewal has two kinds of operations: the first, existing user is predicted; The second, new article and new user are predicted.As shown in table 2 below, wherein, S is the submatrix after sampling, and W is article characteristics matrix, the eigenmatrix that H is user.To above-mentioned amount of bias, also can adopt similar mode to upgrade, as shown in table 3.
Table 2
Table 3
Described step S4 is for to obtain the final recommendation list to user by the score in predicting model after the integrated renewal of weighting.
The current matrix decomposition algorithm for large data, owing to being all based on MapReduce model, so just can provide final prediction scoring after must all data being summed up together, has limited its effect so greatly.If have a node to break down, all can affect to final result.
And how the model of the integrated recommendation that the present invention proposes more effectively obtains recommendation results in order to solve exactly in this distributed computing framework.Such as certain machine is when breaking down, and how to guarantee that whole system provides recommendation results fast and effectively.The integrated recommendation that the present invention proposes is actually a weighted sum to every machine prediction scoring.Because the ability of every machine processing data is different, the recommendation effect of corresponding training model out also can be different, and in the online updating of increment type, the effect of each model modification also has difference.And can finally obtain a more rational overall recommendation results by the recommendation results obtaining on every machine of weighting.Ideally, every machine can provide recommendation results timely, and the result obtaining after weighting is like this best, if but in to ageing exigent application, this model also can obtain relatively good recommendation results.Because the matrix decomposition model on every machine can be made separately recommendation, as long as can guarantee that like this machine can normally move the recommendation that can provide to a certain degree.Thereby and for certain user, can also optionally provide some reasonable models and make recommendation.Therefore, integrated recommendation of the present invention is that a kind of extensibility is high, the recommended models of good stability.
Specifically, integrated recommendation is mainly that the recommendation results to obtaining on all machines is weighted summation, and weight is mainly to calculate according to the processing power of every machine, concrete be exactly according to every machine sampling number give weight.The ranks number of supposing every machine sampling is respectively
with
, the weight of every machine can simple computation be
so the computing formula of integrated recommendation is:
Wherein,
it is the score in predicting that j platform machine is made.Finally as long as right
sort and just can provide the recommendation of Top K.Integrated recommendation can each machine of balance processing power and efficiently provide online fast recommendation.If in order further to raise the efficiency, can also adopt the mode of just recommending of hitting, such as which not machine of this user's data from the sample survey certain existing user, as long as comprised this user in the data of sampling on every or several machines, just can skip over.
The present invention also provides a kind of online commending system calculating based on adapter distribution, and for by treating apparatus, mass data information processing rear line being recommended, as shown in Figure 2, described online commending system comprises:
Adaptive load balancing unit 100, for according to the processing power for the treatment of apparatus, distributes after adopting matrix sampling algorithm based on norm to sample from mass data information, makes each treating apparatus can process alone the data message of distribution;
Distributed matrix resolving cell 200, for utilizing the matrix decomposition model based on amount of bias to train distributed data message, obtains a score in predicting model;
Increment type online updating unit 300, for upgrading described score in predicting model by increment type online updating method;
Online integration recommendation unit 400, obtains the final recommendation list to user for the score in predicting model by after the integrated renewal of weighting.
In said system, the function of various piece is all described in detail in said method, here superfluous having stated no longer just.
Conventionally the framework based on matrix decomposition model is mainly divided into online and two parts of calculated off-line, as shown in Figure 3.Thereby online part is mainly according to existing model, user preferences to be calculated, sorted and filters to provide recommendation results.Off-line part is mainly the recommended models that on behavioral data, user's information itself and the Information base of article itself user, generator matrix decomposes, if have new user behavior data, newly-increased user or the information of article, all partly upgrade the then model of re invocation matrix decomposition at off-line and carry out the training of model.Can find out that this model is less compliant in line and recommends, because it cannot be in time to newly-increased user behavior data, or newly-increased user, or newly-increased article provide effective recommendation.To existing user, also will can provide final recommendation results by calculating his hobby to all article, then sorting, filter, because the quantity of article is huge, this calculates can be more consuming time simultaneously.If online user's quantity is considerable, this calculated amount can be very large, and the handling capacity of system also can be more and more less.
The on-line intelligence commending system of the adaptivity Distributed Calculation that the present invention proposes, as shown in Figure 4.Then adaptivity Distributed Calculation mainly adaptively realizes training matrix decomposition model after large sampling of data by every machine, carry out the model online updating of increment type spontaneously, finally each model is weighted to integrated recommendation.Be distributed and in linearize with the difference of commending system maximum of the prior art in Fig. 3: matrix decomposition and model modification are based upon on the framework of adaptivity Distributed Calculation, and carry out at off-line except matrix decomposition training pattern, the renewal of model and recommendation are because the superiority on this framework has realized in linearize.
In short, the present invention proposes a kind of on-line intelligence suggested design of effective and feasible adaptivity Distributed Calculation.It has proposed adaptivity distributed computing framework for the first time innovatively, thereby can adaptivity Distributed Calculation referring in the system forming at the machine by common that every computing machine can be done load balancing automatically according to the processing power of oneself realizes Distributed Calculation.This commending system does not need to remove to set up cluster by any distributed computing platform.And the processing mass data that can effectively process in separate unit or many common computer, good stability not only, extensibility is high, but also can greatly cost-saving and development efficiency.This scheme has effectively utilized matrix decomposition model simultaneously, and in distributed computing environment, model is upgraded fast, finally by the more satisfactory recommendation results of integrated each model.By this scheme, only need to utilize one or many common computer to provide an efficient recommendation solution of processing mass data.
In addition, the present invention also provides a kind of mobile terminal (as mobile phone), and it is provided with the above-mentioned online commending system calculating based on adapter distribution, makes user obtain anywhere or anytime recommendation information by mobile terminal.
In sum, online recommend method, system and the mobile terminal calculating based on adapter distribution of the present invention, wherein, adaptivity Distributed Calculation is mainly processed device-adaptive by each training matrix decomposition model after large sampling of data is realized, the model online updating that carries out increment type is finally weighted integrated recommendation to each model then spontaneously.This commending system does not need to remove to set up cluster by any distributed computing platform.And the processing mass data that can effectively process in separate unit or many common computer, good stability not only, extensibility is high, but also can greatly cost-saving and development efficiency.The on-line intelligence commending system of the adaptivity Distributed Calculation based on matrix decomposition that in short, the present invention proposes has the following benefit:
Easily deployment property: the adaptivity Load Balancing Model of matrix sampling can guarantee that any computing machine can be used for recommending;
Extensibility and stability: we do not need to build group system, and between every computing machine, do not need communication, if a machine breaks down, other machines can be made recommendation equally.
Diversity: avoid occurring " long tail effect ", if just extract the most reputable several thousand films, the viewed chance of not pandemic so film can be more and more less.Adaptivity Load Balancing Model can guarantee the diversity of recommending to the full extent.
Real-time: for the real-time that guarantees to recommend, we have introduced increment type online updating model, can upgrade fast existing subscriber's recommendation list and make recommendation.
Accuracy: based on the better relation between associated article and user of amount of bias matrix decomposition algorithm, thereby obtain recommendation effect more accurately, and can further optimize recommendation effect by integrated recommended models.
Should be understood that, application of the present invention is not limited to above-mentioned giving an example, and for those of ordinary skills, can be improved according to the above description or convert, and all these improvement and conversion all should belong to the protection domain of claims of the present invention.
Claims (10)
1. the online recommend method calculating based on adapter distribution, for by treating apparatus, mass data information processing rear line being recommended, is characterized in that, described online recommend method comprises the following steps:
A, according to the processing power for the treatment of apparatus, distribute after adopting matrix sampling algorithm based on norm to sample from mass data information, make each treating apparatus can process alone the data message of distribution;
B, the matrix decomposition model of utilization based on amount of bias are trained distributed data message, obtain a score in predicting model;
C, by increment type online updating method, upgrade described score in predicting model;
D, by the score in predicting model after the integrated renewal of weighting, obtain the final recommendation list to user.
2. the online recommend method calculating based on adapter distribution according to claim 1, is characterized in that, in described steps A, adopts the matrix sampling algorithm based on norm specifically to comprise the following steps:
A1, obtain data matrix corresponding to mass data information;
A2, the row and column of described data matrix is sampled simultaneously, obtain a submatrix; And guarantee that according to the first of vector or second norm data and the degree of approximation between mass data that the submatrix after sampling comprises are less than predetermined error threshold.
3. the online recommend method calculating based on adapter distribution according to claim 2, is characterized in that, in described steps A 2, specifically comprises the following steps:
A21, the row and column of data matrix is sampled simultaneously, the number of samples of row and column is respectively p and q, exports a submatrix that comprises the capable q row of p;
A22, according to the ratio of the second norm calculation row and column in whole matrix norm, generate the norm ratio of each row and column;
A23, above-mentioned norm ratio is normalized after, obtain the probability of sample sampling, and generate corresponding probability interval;
A24, generate one at random and be greater than 0 number that is less than 1, judge that it whether in above-mentioned probability interval, extracts the sample corresponding with it in this way.
4. the online recommend method calculating based on adapter distribution according to claim 1, is characterized in that, in described step C, increment type online updating method comprises known users is predicted and new user/article are predicted.
5. the online recommend method calculating based on adapter distribution according to claim 1, is characterized in that, the weight during weighting described in described step D is integrated is to distribute according to the processing power of each treating apparatus.
6. the online recommend method calculating based on adapter distribution according to claim 5, is characterized in that, in described step D, the weight of weighting in integrated is to distribute specifically and comprise according to the processing power of each treating apparatus:
The ranks number of the sampling of D1, each treating apparatus is respectively
with
;
D2, weight is set
.
7. according to the online recommend method calculating based on adapter distribution described in claim 1 or 4, it is characterized in that, described treating apparatus is computing machine, and the processing power of described treating apparatus comprises internal memory and the arithmetic capability of computing machine.
8. the online commending system calculating based on adapter distribution, for by treating apparatus, mass data information processing rear line being recommended, is characterized in that, described online commending system comprises:
Adaptive load balancing unit, for according to the processing power for the treatment of apparatus, distributes after adopting matrix sampling algorithm based on norm to sample from mass data information, makes each treating apparatus can process alone the data message of distribution;
Distributed matrix resolving cell, for utilizing the matrix decomposition model based on amount of bias to train distributed data message, obtains a score in predicting model;
Increment type online updating unit, for upgrading described score in predicting model by increment type online updating method;
Online integration recommendation unit, obtains the final recommendation list to user for the score in predicting model by after the integrated renewal of weighting.
9. the online commending system calculating based on adapter distribution according to claim 8, is characterized in that, in described increment type online updating unit, increment type online updating method comprises known users is predicted and new user/article are predicted.
10. a mobile terminal, is characterized in that, comprises the online commending system calculating based on adapter distribution claimed in claim 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310171026.8A CN103530304B (en) | 2013-05-10 | 2013-05-10 | On-line recommendation method, system and mobile terminal based on self-adaption distributed computation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310171026.8A CN103530304B (en) | 2013-05-10 | 2013-05-10 | On-line recommendation method, system and mobile terminal based on self-adaption distributed computation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103530304A true CN103530304A (en) | 2014-01-22 |
| CN103530304B CN103530304B (en) | 2017-05-10 |
Family
ID=49932323
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310171026.8A Active CN103530304B (en) | 2013-05-10 | 2013-05-10 | On-line recommendation method, system and mobile terminal based on self-adaption distributed computation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103530304B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107491813A (en) * | 2017-08-29 | 2017-12-19 | 天津工业大学 | A kind of long-tail group recommending method based on multiple-objection optimization |
| WO2018149337A1 (en) * | 2017-02-15 | 2018-08-23 | 腾讯科技（深圳）有限公司 | Information distribution method, device, and server |
| CN105183841B (en) * | 2015-09-06 | 2019-03-26 | 南京游族信息技术有限公司 | The recommended method of frequent item set and deep learning is combined under big data environment |
| CN110413878A (en) * | 2019-07-04 | 2019-11-05 | 四川金赞科技有限公司 | User based on adaptive elastomeric network-commodity preference prediction meanss and method |
| CN110941771A (en) * | 2019-12-24 | 2020-03-31 | 哈尔滨理工大学 | Commodity parallel dynamic pushing method in e-commerce platform |
| CN111259249A (en) * | 2020-01-20 | 2020-06-09 | 北京百度网讯科技有限公司 | Data screening method, device, equipment and storage medium |
| CN112818254A (en) * | 2021-02-03 | 2021-05-18 | 广东牛牛智能科技有限公司 | Farmer and civil industry personalized employment recommendation method and system based on intelligent logic collaborative filtering |
| KR102593134B1 (en) * | 2022-12-16 | 2023-10-24 | 고려대학교산학협력단 | Method for resizing embedding table for user devices and apparatus for same |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080243997A1 (en) * | 2007-03-29 | 2008-10-02 | Motorola, Inc. | Distributed content item recommendation system and method of operation therefor |
| US20100332408A1 (en) * | 2000-06-23 | 2010-12-30 | Ekhaus Michael A | Method and System for High Performance Model-Based Personalization |
| CN102426686A (en) * | 2011-09-29 | 2012-04-25 | 南京大学 | Internet information product recommending method based on matrix decomposition |
| CN102750360A (en) * | 2012-06-12 | 2012-10-24 | 清华大学 | Mining method of computer data for recommendation systems |
-
2013
- 2013-05-10 CN CN201310171026.8A patent/CN103530304B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100332408A1 (en) * | 2000-06-23 | 2010-12-30 | Ekhaus Michael A | Method and System for High Performance Model-Based Personalization |
| US20080243997A1 (en) * | 2007-03-29 | 2008-10-02 | Motorola, Inc. | Distributed content item recommendation system and method of operation therefor |
| CN102426686A (en) * | 2011-09-29 | 2012-04-25 | 南京大学 | Internet information product recommending method based on matrix decomposition |
| CN102750360A (en) * | 2012-06-12 | 2012-10-24 | 清华大学 | Mining method of computer data for recommendation systems |
Non-Patent Citations (3)
| Title |
|---|
| TIANQI CHEN等: "SVDFeature: A Toolkit for Feature-based Collaborative Filtering", 《JOURNAL OF MACHINE LEARNING RESEARCH》 * |
| 李改等: "基于矩阵分解的协同过滤算法", 《计算机工程与应用》 * |
| 谢兴: "社会网络中兴趣发现与信息组织的研究", 《万方数据》 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105183841B (en) * | 2015-09-06 | 2019-03-26 | 南京游族信息技术有限公司 | The recommended method of frequent item set and deep learning is combined under big data environment |
| WO2018149337A1 (en) * | 2017-02-15 | 2018-08-23 | 腾讯科技（深圳）有限公司 | Information distribution method, device, and server |
| CN107491813A (en) * | 2017-08-29 | 2017-12-19 | 天津工业大学 | A kind of long-tail group recommending method based on multiple-objection optimization |
| CN110413878A (en) * | 2019-07-04 | 2019-11-05 | 四川金赞科技有限公司 | User based on adaptive elastomeric network-commodity preference prediction meanss and method |
| CN110413878B (en) * | 2019-07-04 | 2022-04-15 | 五五海淘(上海)科技股份有限公司 | User-commodity preference prediction device and method based on adaptive elastic network |
| CN110941771A (en) * | 2019-12-24 | 2020-03-31 | 哈尔滨理工大学 | Commodity parallel dynamic pushing method in e-commerce platform |
| CN110941771B (en) * | 2019-12-24 | 2022-08-23 | 哈尔滨理工大学 | Commodity parallel dynamic pushing method in e-commerce platform |
| CN111259249A (en) * | 2020-01-20 | 2020-06-09 | 北京百度网讯科技有限公司 | Data screening method, device, equipment and storage medium |
| CN111259249B (en) * | 2020-01-20 | 2023-08-22 | 北京百度网讯科技有限公司 | Data screening method, device, equipment and storage medium |
| CN112818254A (en) * | 2021-02-03 | 2021-05-18 | 广东牛牛智能科技有限公司 | Farmer and civil industry personalized employment recommendation method and system based on intelligent logic collaborative filtering |
| CN112818254B (en) * | 2021-02-03 | 2022-06-14 | 广东牛牛智能科技有限公司 | Chinese-intelligence-logic-collaborative-filtering-based individual employment recommendation method and system for rural workers |
| KR102593134B1 (en) * | 2022-12-16 | 2023-10-24 | 고려대학교산학협력단 | Method for resizing embedding table for user devices and apparatus for same |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103530304B (en) | 2017-05-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103530304A (en) | On-line recommendation method, system and mobile terminal based on self-adaption distributed computation | |
| US20230281448A1 (en) | Method and apparatus for information recommendation, electronic device, computer readable storage medium and computer program product | |
| WO2022057658A1 (en) | Method and apparatus for training recommendation model, and computer device and storage medium | |
| CN106776928B (en) | Position recommendation method based on memory computing framework and fusing social contact and space-time data | |
| CN110266745B (en) | Information flow recommendation method, device, equipment and storage medium based on deep network | |
| CN103886047B (en) | Towards the online recommendation method of distribution of stream data | |
| CN110147882B (en) | Neural network model training method, crowd diffusion method, device and equipment | |
| CN111008332A (en) | Content item recommendation method, device, server and storage medium | |
| CN108052979A (en) | The method, apparatus and equipment merged to model predication value | |
| CN112052387A (en) | Content recommendation method and device and computer readable storage medium | |
| CN113392317A (en) | Label configuration method, device, equipment and storage medium | |
| CN109087138A (en) | Data processing method and system, computer system and readable storage medium storing program for executing | |
| CN112084413A (en) | Information recommendation method and device and storage medium | |
| CN109933610A (en) | Data processing method, device, computer equipment and storage medium | |
| CN116151235A (en) | Article generating method, article generating model training method and related equipment | |
| Ozturk et al. | Investigating the trans-boundary of air pollution between the BRICS and its neighboring countries: an empirical analysis | |
| CN118043802A (en) | Recommendation model training method and device | |
| CN103870671A (en) | Method and device for extracting user sample from Cookies | |
| CN110020214B (en) | Knowledge-fused social network streaming event detection system | |
| CN117035059A (en) | Efficient privacy protection recommendation system and method for communication | |
| CN116975686A (en) | Method for training student model, behavior prediction method and device | |
| CN116976429A (en) | Distributed training method, device, electronic equipment, storage medium and program product | |
| CN115795156A (en) | Material recall and neural network training method, device, equipment and storage medium | |
| CN116204709A (en) | Data processing method and related device | |
| CN110035126A (en) | A kind of document handling method, calculates equipment and storage medium at device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |