CN105243063B - The method and apparatus of information recommendation - Google Patents
The method and apparatus of information recommendation Download PDFInfo
- Publication number
- CN105243063B CN105243063B CN201410274081.4A CN201410274081A CN105243063B CN 105243063 B CN105243063 B CN 105243063B CN 201410274081 A CN201410274081 A CN 201410274081A CN 105243063 B CN105243063 B CN 105243063B
- Authority
- CN
- China
- Prior art keywords
- source data
- recommendation
- dimension
- client
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The invention discloses a kind of method and apparatus of information recommendation, belong to data processing field.The described method includes: obtaining source data；Determine the dimension and the corresponding MapReduce model of each dimension of the source data；The source data is calculated parallel using determining each MapReduce model；The recommendation information being calculated is stored in Hbase system；When receiving the recommendation request of client, recommendation information related with the user of the client is obtained from the Hbase system and is sent to the client.Described device includes: to obtain module, determining module, computing module, memory module and recommending module.Present invention greatly enhances the speed of data processing, reduce the complexity of calculating, save computing overhead, improve the efficiency and accuracy of information recommendation.
Description
Technical field
The present invention relates to data processing field, in particular to a kind of method and apparatus of information recommendation.
Background technique
With the development of information technology, data are more and more important to enterprise, and data volume is also exponentially increasing, data
The technologies such as storage and processing becoming hot spot.And valuable content how is extracted from huge data, and do with this
Various recommendations have become mainstream business at present.
There is a kind of recommended technology at present, is that data are calculated based on collaborative filtering, and is pushed away what is be calculated
It recommends result and is sent to user's progress information recommendation.But the algorithm complexity is high, algorithm is opened for large-scale data set
Pin can become very huge, slow so as to cause data processing speed, result is inaccurate, and database corruption is also resulted in when serious.
Summary of the invention
In view of this, the embodiment of the invention provides a kind of method and apparatus of information recommendation, when improving information recommendation
Speed and accuracy.The technical solution is as follows:
In a first aspect, providing a kind of method of information recommendation, which comprises
Obtain source data；
Determine the dimension and the corresponding MapReduce model of each dimension of the source data；
The source data is calculated parallel using determining each MapReduce model；
The recommendation information being calculated is stored in Hbase system；
When receiving the recommendation request of client, obtained from the Hbase system related with the user of the client
Recommendation information and be sent to the client.
Optionally, the acquisition source data, comprising:
According to preset period timing acquisition source data；And/or
When source data changes, changed source data is obtained in real time.
Further, described that the source data is calculated parallel using determining each MapReduce model, it wraps
It includes:
For the source data of timing acquisition, corresponding MapReduce model is utilized simultaneously according to preset static policies timing
Row is calculated；
For the source data obtained in real time, according to preset dynamic strategy utilize in real time corresponding MapReduce model into
Row calculates.
Optionally, the recommendation information that will be calculated is stored in Hbase system, comprising:
The User ID of client and recommendation list are stored in Hbase system, the User ID of the client is as master
Key, the recommendation list include the recommendation information of each dimension of the user.
Further, the recommendation information of each dimension is spliced by designated symbols in the recommendation list, each dimension
Recommendation information include: Refer ID, recommend reason and weight.
Second aspect, provides a kind of device of information recommendation, and described device includes:
Module is obtained, for obtaining source data；
Determining module, for determining the dimension and the corresponding MapReduce model of each dimension of the source data；
Computing module, for being calculated parallel the source data using determining each MapReduce model；
Memory module, for the recommendation information being calculated to be stored in Hbase system；
Recommending module, for being obtained and the client from the Hbase system when receiving the recommendation request of client
The related recommendation information of the user at end is simultaneously sent to the client.
Optionally, the acquisition module includes:
Timing acquisition unit, for according to preset period timing acquisition source data；And/or
Real-time acquiring unit, for obtaining changed source data in real time when source data changes.
Further, the computing module includes:
Computing unit utilizes corresponding for the source data for timing acquisition according to preset static policies timing
MapReduce model is calculated parallel；For the source data obtained in real time, correspondence is utilized in real time according to preset dynamic strategy
MapReduce model calculated.
Optionally, the memory module includes:
Storage unit, for the User ID of client and recommendation list to be stored in Hbase system, the client
For User ID as major key, the recommendation list includes the recommendation information of each dimension of the user.
Further, the recommendation information of each dimension is spliced by designated symbols in the recommendation list, each dimension
Recommendation information include: Refer ID, recommend reason and weight.
Technical solution provided in an embodiment of the present invention has the benefit that acquisition source data；Determine the source data
Dimension and the corresponding MapReduce model of each dimension；Using determining each MapReduce model parallel to the source
Data are calculated；The recommendation information being calculated is stored in Hbase system；When the recommendation request for receiving client
When, recommendation information related with the client is obtained from the Hbase system and is sent to the client, realizes information
Recommend, is suitable for the MapReduce model of large-scale data set operation due to using, and use different MapReduce
Model carries out parallel processing to the source data of different dimensions, greatly improves the speed of data processing, reduces answering for calculating
Miscellaneous degree, saves computing overhead, improves the efficiency and accuracy of information recommendation.In addition, storing recommendation using Hbase system
Breath has the advantages such as high reliability, high-performance, scalable, improves the safety of data storage.
Detailed description of the invention
To describe the technical solutions in the embodiments of the present invention more clearly, make required in being described below to embodiment
Attached drawing is briefly described, it should be apparent that, drawings in the following description are only some embodiments of the invention, for
For those of ordinary skill in the art, without creative efforts, it can also be obtained according to these attached drawings other
Attached drawing.
Fig. 1 is the method flow diagram for the information recommendation that one embodiment of the invention provides；
Fig. 2 be another embodiment of the present invention provides two degree of friend recommendation schematic diagrames；
Fig. 3 be another embodiment of the present invention provides information recommendation method flow diagram；
Fig. 4 be another embodiment of the present invention provides information recommendation interaction schematic diagram；
Fig. 5 be another embodiment of the present invention provides information recommendation structure drawing of device；
Fig. 6 be another embodiment of the present invention provides information recommendation structure drawing of device.
Specific embodiment
To make the object, technical solutions and advantages of the present invention clearer, below in conjunction with attached drawing to embodiment party of the present invention
Formula is described in further detail.
The present embodiments relate to Hadoop platform, a distributed system infrastructures.Two spies of Hadoop platform
Property is, using HDFS (Hadoop Distributed File System, Hadoop distributed file system) storing data, and
The operation of big data is carried out based on MapReduce model.MapReduce model is a kind of programming model, by specifying one
One group of key-value pair is mapped to one group of new key-value pair by Map function, and guarantees to own by specifying concurrent Reduce function
Each of key-value pair of mapping shares identical key group.
Referring to Fig. 1, one embodiment of the invention provides a kind of method of information recommendation, comprising:
101: obtaining source data.
Wherein, source data can store on one or multiple servers.
102: determining the dimension and the corresponding MapReduce model of each dimension of the source data.
The dimension of source data is determined by customer relationship chain.In general, a customer relationship chain is exactly a dimension, including
But be not limited to it is following any one: it is common friend, Boss ticket, common IP, common group, address list, stranger's communication, common
Grouping etc..
103: the source data being calculated parallel using determining each MapReduce model.
Wherein, each dimension calculates the source data of the dimension using corresponding MapReduce model, multiple
The source data of dimension is respectively adopted respective MapReduce model and is calculated parallel, to greatly improve the place of data
Reason ability, improves calculating speed and efficiency.
Moreover, new MapReduce model parallel processing can be designed when source data increases new dimension, to existing
MapReduce model operation do not impact, do not interfere with each other, have good scalability.
In the present embodiment, the source data of different dimensions corresponds to different MapReduce models, specifically, can be according to dimension
The characteristic of degree designs MapReduce model, and the present embodiment is not specifically limited in this embodiment.
For example, source data is " PC and mobile communication for several times+full dose friend relation to ", the format of PC and mobile communication for several times
Are as follows: user UID1, user UIDn, the number that user UID1 is communicated on PC with user UIDn, user UID1 is on mobile phone
The number communicated with user UIDn.Wherein, user UID1 and user UIDn is the user of instant messaging, user's UIDn generation
Several can carry out the user of instant messaging with user UID1 with table, and n be positive integer, user UID1 can in user UIDn
Any one carries out mutually instant messaging.The full dose friend relation is to referring to all friend relation pair in instant messaging application, lattice
Formula is UIDi, UIDj.It is not the user of good friend each other that purpose, which is for the user's recommendation for carrying out instant messaging,.Recommendation information is close
Friend refers to that communication number of days reaches specified and just recommends mutually for X days or more, and above-mentioned communication number is also possible to communicate number of days.Firstly,
Determine that the dimension of the source data for communication dimension, designs MapReduce model corresponding with the dimension and pushed away as follows
Recommend calculating: described except processing again refers to that each user UIDn remembers firstly, carry out the communication number of PC and mobile phone except again handling
Record a communication total degree, specially PC communicates the cumulative of number and mobile communication number.Then, according to full dose friend relation
It is right, by the good friend in user UID1 and user UIDn to deletion, obtain the user pair set of non-friend relation.Finally, in the collection
Communication user UIDn list is extracted in closing and is sorted from high to low according to communication total degree, and recommendation list is obtained.As user UID1
Client when sending recommendation request, which can directly be recommended to client, or the recommendation can also be arranged
The user UIDn that number of days is more than given number of days is communicated in table recommends client as close friend.
For example, source data is " buddy list of the buddy list+D of A ", recommendation information is " two degree of good friends ".Referring to fig. 2, A
Buddy list include B and C, the buddy list of D includes B and E, and A and D are good friends.Dimension can be determined according to source data
It for buddy list, designs MapReduce model corresponding with the dimension and carries out recommendation calculating as follows: being arranged in the good friend of A
The good friend B that D is filtered out in table (B, C), obtains result C, the recommendation information as D.A is filtered out in the buddy list (B, E) of D
Good friend B obtain result E, the recommendation information as A.When A requests to recommend, using E as friend recommendation to A, when D requests to recommend
When, using C as friend recommendation to D.
For example, source data is " group relation+full dose friend relation to ", target is not to be to the member in same group
The progress of good friend is recommended mutually.The dimension for determining source data is " common group ".According to the corresponding of dimension design
MapReduce model carries out recommendation calculating as follows: firstly, all members in same group are done cartesian product, obtaining
To multiple groups friend couple；It then, will be the friend of good friend to filtering out；Finally, by the friend of the friend's centering obtained after filtering
It is friendly to be mutually used as recommendation information.When receiving the recommendation request of some user, carried out all friends of the user as good friend
Recommend.
104: the recommendation information being calculated is stored in Hbase system.
Hbase system has the characteristics that high reliability, high performance, is towards column, telescopic distributed memory system, especially
It is suitable for the storages of mass data.And the storage based on column mode, there is no any association, the efficiency of inquiry between table and table
It is very high.
The recommendation information may include much information, such as instant messaging ID, phone number, group ID, this implementation
Example is not specifically limited in this embodiment.
105: when receiving the recommendation request of client, being obtained from the Hbase system related with the user of the client
Recommendation information and be sent to the client.
Wherein, recommendation information can be stored according to the ID (Identity, identity recognition number) of user, therefore, received
To some client recommendation request when, corresponding recommendation information can be inquired according to the User ID of the client, thus
Recommend the user of the client.
The client that the present embodiment is related to can be mobile phone, PC machine, tablet computer etc., and the present embodiment does not do this specifically
It limits.
In the present embodiment, optionally, source data is obtained, may include:
According to preset period timing acquisition source data；And/or
When source data changes, changed source data is obtained in real time.
Further, the source data is calculated parallel using determining each MapReduce model, may include:
For the source data of timing acquisition, corresponding MapReduce model is utilized simultaneously according to preset static policies timing
Row is calculated；
For the source data obtained in real time, according to preset dynamic strategy utilize in real time corresponding MapReduce model into
Row calculates.
In the present embodiment, optionally, the recommendation information being calculated is stored in Hbase system, may include:
The User ID of client and recommendation list are stored in Hbase system, the User ID of the client as major key,
The recommendation list includes the recommendation information of each dimension of the user.
Further, the recommendation information of each dimension is spliced by designated symbols in the recommendation list, each dimension
Recommendation information include: Refer ID, recommend reason and weight.
The above method provided in this embodiment obtains source data；Determine the dimension and each dimension pair of the source data
The MapReduce model answered；The source data is calculated parallel using determining each MapReduce model；It will calculate
Obtained recommendation information is stored in Hbase system；When receiving the recommendation request of client, obtained from the Hbase system
It takes recommendation information related with the user of the client and is sent to the client, realize information recommendation, due to using
It is suitable for the MapReduce model of large-scale data set operation, and using different MapReduce model to different dimensions
Source data carry out parallel processing, greatly improve the speed of data processing, reduce the complexity of calculating, save operation
Expense improves the efficiency and accuracy of information recommendation.In addition, storing recommendation information using Hbase system, have highly reliable
Property, high-performance, the advantages such as scalable, improve the safety of data storage.
Referring to Fig. 3, another embodiment of the present invention provides a kind of methods of information recommendation, comprising:
301: according to preset period timing acquisition source data；And/or when source data changes, hair is obtained in real time
The source data for changing.
Wherein it is possible to by realizing that FTP (File Transfer Protocol, File Transfer Protocol) client functionality comes
Timing acquisition source data is executed, which can according to need setting, and the present embodiment does not limit specific value.Periodically
The source data of acquisition can store in HDFS, which can be an equipment, or be also possible to a group system.
Obtaining source data in real time can be using HTTP (HyperText Transfer Protocol, Hyper text transfer association
View) mode or GET (obtaining order) mode realizes, it is preferable that it is realized using HTTP mode and obtains source data in real time, peace
Quan Xinggeng high.The source data obtained in real time can store in Hbase system, which may include an equipment, or
Person's multiple devices.
Timing acquisition is suitable for the not high recommendation request of timeliness, obtains be more suitable for that timeliness is more demanding to be pushed away in real time
Recommend request.It is therefore preferred that timing acquisition is combined application with real-time acquisition, it can adapt to various recommendation requests, apply
It is more flexible.
302: determining the dimension and the corresponding MapReduce model of each dimension of the source data.
In the present embodiment, corresponding MapReduce model, and every kind can be designed for every kind of dimension in advance
Proposed algorithm in MapReduce model is all different.Depending on algorithm foundation dimension, more targetedly, therefore, it is recommended that meter
The result of calculation is more acurrate, realizes intelligent recommendation.
303: for the source data of timing acquisition, utilizing corresponding MapReduce mould according to preset static policies timing
Type is calculated parallel.
Wherein, static policies are used for the source data of timing acquisition, since the source data of timing acquisition is usually all lot number
According to data volume is larger, but timeliness is of less demanding, it is therefore possible to use the static policies that timing calculates.It such as can be in terms of every month
Calculation is once or the calculating of per two weeks is primary etc., and the present embodiment is not specifically limited in this embodiment.
The source data of timing acquisition is usually directed to multiple dimensions, therefore, can carry out to the source data of multiple dimension
Parallel processing is calculated parallel using respective MapReduce model.
304: for the source data obtained in real time, utilizing corresponding MapReduce mould in real time according to preset dynamic strategy
Type is calculated.
Wherein, dynamic strategy is used for the source data obtained in real time, since what is obtained in real time is all changed data, leads to
Regular data amount is smaller, but timeliness is higher, it is therefore possible to use the dynamic strategy calculated in real time.In most cases, it obtains in real time
The source data taken is all the data of sole user, in the address list as changed in some mobile phone or some PC machine increase newly i.e.
When communication good friend etc., the present embodiment is not specifically limited in this embodiment.
Above-mentioned two step 303 and 304 can carry out simultaneously, to achieve the effect that parallel processing.
In addition, on the basis of using MapReduce model, further, can also be used in the present embodiment
Combiner function and Partitioner function etc. carry out optimization algorithm, effectively to improve the performance of algorithm, do not do herein excessive
Explanation.
By carrying out real-time calculation processing to the source data obtained in real time, it can guarantee the high-timeliness recommended, realize
Real-time recommendation.
305: the User ID of client and recommendation list being stored in Hbase system, the User ID conduct of the client
Major key, the recommendation list include the recommendation information of each dimension of the user.
Preferably, the recommendation information of each dimension is spliced by designated symbols in the recommendation list, and each dimension pushes away
Information is recommended to include: Refer ID, recommend reason and weight.
In the present embodiment, the User ID of client and recommendation list can be designed to a main table storage.For example, can be with
As shown in table 1.
Table 1
Wherein, major key KEY is the user UID that length is 10, cannot be sky.Reasonlist is recommendation list, including
The recommendation information of all dimensions.It wherein, is KEY with UID, it is ensured that the speed of storage and inquiry.In general, UID is greater than 8
Long number less than 10, the UID in KEY less than 10 need to mend 0 in front to guarantee length for 10.Build table
Afterwards, region division can be carried out to table according to the distribution of KEY, being uniformly distributed for data storage is guaranteed with this, is deposited with reaching
The optimization of storage and inquiry velocity.Recommendation list symbolization splices the form storage of the recommendation information of various dimensions, so as to
To digitize recommendation information, memory space is more saved than written form storage.The separator of splicing can according to need fixed
Justice, the present embodiment are not specifically limited in this embodiment.
As shown in table 1, recommendation list may include a plurality of record, and every record includes: to recommend UID, recommend reason and power
Weight；A plurality of record can be sorted from large to small according to recommendation weight.Wherein, the separator between each item record can be " "；It pushes away
Recommending UID, recommending the separator between reason and weight can be " # "；Separator between multiple recommendation reasons can be " | "；
Recommending reason includes KEY and VALUE, and separator can be ": " etc..For example, a recommendation list comprising two records
It is as follows: 289367798#0:1#5 $ 11111111#0:1#4.First is recorded as " 289367798#0:1#5 ", then, by this
UID user recommends key；Wherein, the UID of recommended user is " 289367798 "；Recommending reason is " 0:1 ", and " 0 " is KEY value,
" 1 " is VALUE value, and different VALUE values can correspond to different meanings, is defined as needed, such as " 0 " corresponding Boss ticket dimension
It spends (reason), " 1 " corresponding Boss ticket talk times；Calculating weight according to rule is 5.Article 2 is recorded as " 11111111#
0:1#4 ", then, this UID user is recommended into key；Wherein, the UID of recommended user be " 11111111 ", recommend reason be " 0:
1 ", weight 4.It is noted that when there is the increase of new dimension, only need to after recommendation list additional record, simple side
Just, it easily extends.
In addition, some numerical results can also be stored in distributed cache system to further increase inquiry velocity
It unites in Memcached, whether when there is client to send recommendation request, inquire from Memcached has corresponding recommendation first
Information, if so, the recommendation information is directly then returned to client；It is corresponded to if it is not, being inquired from Hbase system again
Recommendation information recommended, thus avoid every time recommend all from Hbase system queries, alleviate the pressure of Hbase system, pole
The earth improves inquiry velocity.
Therefore, the data storage in the present embodiment can consist of three parts, HDFS, Hbase and Memcached.Wherein,
The source data of timing acquisition is stored in HDFS, is suitable for mass data storage；The source data obtained in real time and parallel meter
The result of calculation is stored in Hbase, and data are safer, efficient；Memcahed is used to cache some numerical results, with further
Improve query capability.
306: when receiving the recommendation request of client, being obtained from the Hbase system related with the user of the client
Recommendation information and be sent to the client.
The above method can be executed by the device of information recommendation.The recommendation request of client can be sent out by server
The device is given, returns to recommendation information to client after executing inquiry by the device.The process can be as shown in figure 4, specific stream
Journey is as follows: S1, client send recommendation request to server；S2, server, which receive, to be transmitted to information after the recommendation request and pushes away
Recommend device；After S3, information recommending apparatus receive the request, the recommendation information of the client is inquired to Hbase；S4, information recommendation
The recommendation information that inquiry obtains is sent to server by device, is transmitted to client by server and is recommended.
The above method provided in this embodiment, according to preset period timing acquisition source data；And/or when source data is sent out
When changing, changed source data is obtained in real time；Determine that dimension and each dimension of the source data are corresponding
MapReduce model；For the source data of timing acquisition, corresponding MapReduce is utilized according to preset static policies timing
Model is calculated parallel；For the source data obtained in real time, utilized in real time according to preset dynamic strategy corresponding
MapReduce model is calculated；The User ID of client and recommendation list are stored in Hbase system, the client
For User ID as major key, which includes the recommendation information of each dimension of the client；When receiving pushing away for client
When recommending request, recommendation information related with the user of the client is obtained from the Hbase system and is sent to the client；It realizes
Information recommendation is suitable for the MapReduce model of large-scale data set operation due to using, and using different
MapReduce model carries out parallel processing to the source data of different dimensions, greatly improves the speed of data processing, reduces
The complexity of calculating, saves computing overhead, improves the efficiency and accuracy of information recommendation.In addition, being deposited using Hbase system
Recommendation information is stored up, there are the advantages such as high reliability, high-performance, scalable, improves the safety of data storage.
Referring to Fig. 5, another embodiment of the present invention provides a kind of devices of information recommendation, comprising:
Module 501 is obtained, for obtaining source data；
Determining module 502, for determining the dimension and the corresponding MapReduce model of each dimension of the source data；
Computing module 503, for being calculated parallel the source data using determining each MapReduce model；
Memory module 504, for the recommendation information being calculated to be stored in Hbase system；
Recommending module 505, for being obtained and the client from the Hbase system when receiving the recommendation request of client
The related recommendation information of the user at end is simultaneously sent to the client.
Referring to Fig. 6, in the present embodiment, optionally, obtaining module 501 may include:
Timing acquisition unit 501a, for according to preset period timing acquisition source data；And/or
Real-time acquiring unit 501b, for obtaining changed source data in real time when source data changes.
In the present embodiment, further, computing module 503 may include:
Computing unit 503a utilizes correspondence according to preset static policies timing for the source data for timing acquisition
MapReduce model calculated parallel；For the source data obtained in real time, utilized in real time pair according to preset dynamic strategy
The MapReduce model answered is calculated.
In the present embodiment, optionally, memory module 504 may include:
Storage unit, for the User ID of client and recommendation list to be stored in Hbase system, the use of the client
For family ID as major key, which includes the recommendation information of each dimension of the user.
In the present embodiment, further, the recommendation information of each dimension is spelled by designated symbols in the recommendation list
It connects, the recommendation information of each dimension includes: Refer ID, recommends reason and weight.
In the present embodiment, above-mentioned apparatus can also logically be divided are as follows: first interface layer, data computation layer, data
Accumulation layer and second interface layer.Wherein, first interface layer includes above-mentioned acquisition module 501, is responsible for obtaining source data；Data calculate
Layer includes above-mentioned determining module 502 and computing module 503, is responsible for being calculated using MapReduce model；Data storage layer packet
Above-mentioned memory module 504 is included, storage calculated result is responsible for；Second interface layer includes above-mentioned recommending module 505, responsible and client
Interaction, sends recommendation information in client request, to complete to recommend.
Above-mentioned apparatus provided in this embodiment can execute the method that any of the above-described embodiment of the method provides, and be detailed in method reality
The description in example is applied, is not repeated herein.
Above-mentioned apparatus provided in this embodiment obtains source data；Determine the dimension and each dimension pair of the source data
The MapReduce model answered；The source data is calculated parallel using determining each MapReduce model；It will calculate
Obtained recommendation information is stored in Hbase system；When receiving the recommendation request of client, obtained from the Hbase system
It takes recommendation information related with the user of the client and is sent to the client, realize information recommendation, due to using
It is suitable for the MapReduce model of large-scale data set operation, and using different MapReduce model to different dimensions
Source data carry out parallel processing, greatly improve the speed of data processing, reduce the complexity of calculating, save operation
Expense improves the efficiency and accuracy of information recommendation.In addition, storing recommendation information using Hbase system, have highly reliable
Property, high-performance, the advantages such as scalable, improve the safety of data storage.
Those of ordinary skill in the art will appreciate that realizing that all or part of the steps of above-described embodiment can pass through hardware
It completes, relevant hardware can also be instructed to complete by program, the program can store in a kind of computer-readable
In storage medium, storage medium mentioned above can be read-only memory, disk or CD etc..
The foregoing is merely presently preferred embodiments of the present invention, is not intended to limit the invention, it is all in spirit of the invention and
Within principle, any modification, equivalent replacement, improvement and so on be should all be included in the protection scope of the present invention.
Claims (6)
1. a kind of method of information recommendation, which is characterized in that the described method includes:
According to preset period timing acquisition source data；And/or
When source data changes, changed source data is obtained in real time；
If the source data is computer and mobile communication number and full dose friend relation pair, the source data is determined
Dimension is to communicate dimension, if the source data is the buddy list of the first user and the buddy list of second user, determines institute
The dimension of source data is stated as good friend's List Dimension, if the source data is group relation and full dose friend relation pair, determines institute
The dimension for stating source data is common group's dimension, determines the corresponding MapReduce model of each dimension；
For the source data of timing acquisition, using corresponding MapReduce model and advance according to preset static policies timing
Row calculates, and for the source data obtained in real time, is carried out in real time using corresponding MapReduce model according to preset dynamic strategy
It calculates；
The recommendation information being calculated is stored in Hbase system, the part recommendation information in the Hbase system also stores
In distributed cache system Memcached；
When receiving the recommendation request of client, if being stored with the corresponding recommendation of the client in Memcached system
The corresponding recommendation information of the client is then sent to the client from Memcached system by information；
If not stored in Memcached system have the corresponding recommendation information of the client, obtained from the Hbase system
Recommendation information related with the user of the client is simultaneously sent to the client.
2. the method according to claim 1, wherein the recommendation information that will be calculated is stored in Hbase
In system, comprising:
The User ID of client and recommendation list are stored in Hbase system, the User ID of the client is as major key, institute
State the recommendation information that recommendation list includes each dimension of the user.
3. according to the method described in claim 2, it is characterized in that, the recommendation information of each dimension is by referring in the recommendation list
Determine symbol to be spliced, the recommendation information of each dimension includes: Refer ID, recommends reason and weight.
4. a kind of device of information recommendation, which is characterized in that described device includes:
Timing acquisition unit, for according to preset period timing acquisition source data；And/or
Real-time acquiring unit, for obtaining changed source data in real time when source data changes；
Determining module determines if being computer and mobile communication number and full dose friend relation pair for the source data
The dimension of the source data is communication dimension, if the source data is the buddy list of the first user and the good friend of second user
List determines that the dimension of the source data is good friend's List Dimension, if the source data is that group relation and full dose good friend are closed
System pair determines that the dimension of the source data is common group's dimension, determines the corresponding MapReduce model of each dimension；
Computing unit utilizes corresponding for the source data for timing acquisition according to preset static policies timing
MapReduce model is calculated parallel；For the source data obtained in real time, correspondence is utilized in real time according to preset dynamic strategy
MapReduce model calculated；
Memory module, the part for the recommendation information being calculated to be stored in Hbase system, in the Hbase system
Recommendation information also is stored in distributed cache system Memcached；
Recommending module, for when receiving the recommendation request of client, if being stored with the client pair in Memcached
The corresponding recommendation information of the client is then sent to the client from Memcached by the recommendation information answered；If
It is not stored in Memcached to have the corresponding recommendation information of the client, then it is obtained and the client from the Hbase system
The related recommendation information of user and be sent to the client.
5. device according to claim 4, which is characterized in that the memory module includes:
Storage unit, for the User ID of client and recommendation list to be stored in Hbase system, the user of the client
For ID as major key, the recommendation list includes the recommendation information of each dimension of the user.
6. device according to claim 5, which is characterized in that the recommendation information of each dimension is by referring in the recommendation list
Determine symbol to be spliced, the recommendation information of each dimension includes: Refer ID, recommends reason and weight.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410274081.4A CN105243063B (en) | 2014-06-18 | 2014-06-18 | The method and apparatus of information recommendation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410274081.4A CN105243063B (en) | 2014-06-18 | 2014-06-18 | The method and apparatus of information recommendation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105243063A CN105243063A (en) | 2016-01-13 |
| CN105243063B true CN105243063B (en) | 2019-11-15 |
Family
ID=55040715
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410274081.4A Active CN105243063B (en) | 2014-06-18 | 2014-06-18 | The method and apparatus of information recommendation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105243063B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106709009A (en) * | 2016-12-26 | 2017-05-24 | 镇江恒驰科技有限公司 | System for processing big data of electric vehicles on basis of cloud platform |
| CN113240536A (en) * | 2021-05-14 | 2021-08-10 | 北京达佳互联信息技术有限公司 | Information acquisition method, device, server, medium and product |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102663627A (en) * | 2012-04-26 | 2012-09-12 | 焦点科技股份有限公司 | Personalized recommendation method |
| CN103345514A (en) * | 2013-07-09 | 2013-10-09 | 焦点科技股份有限公司 | Streamed data processing method in big data environment |
| CN103345698A (en) * | 2013-07-09 | 2013-10-09 | 焦点科技股份有限公司 | Personalized recommendation method based on cloud processing mode and applied in e-business environment |
| CN103500226A (en) * | 2013-10-23 | 2014-01-08 | 中国农业银行股份有限公司 | Method and device for removing sensitivity of sensitive data |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100011020A1 (en) * | 2008-07-11 | 2010-01-14 | Motorola, Inc. | Recommender system |
| CN102663117B (en) * | 2012-04-18 | 2013-11-20 | 中国人民大学 | OLAP (On Line Analytical Processing) inquiry processing method facing database and Hadoop mixing platform |
| CN103853714B (en) * | 2012-11-28 | 2017-06-20 | 中国移动通信集团河南有限公司 | A kind of data processing method and device |
| CN103747060B (en) * | 2013-12-26 | 2017-12-08 | 惠州华阳通用电子有限公司 | A kind of distributed monitoring system and method based on streaming media service cluster |
-
2014
- 2014-06-18 CN CN201410274081.4A patent/CN105243063B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102663627A (en) * | 2012-04-26 | 2012-09-12 | 焦点科技股份有限公司 | Personalized recommendation method |
| CN103345514A (en) * | 2013-07-09 | 2013-10-09 | 焦点科技股份有限公司 | Streamed data processing method in big data environment |
| CN103345698A (en) * | 2013-07-09 | 2013-10-09 | 焦点科技股份有限公司 | Personalized recommendation method based on cloud processing mode and applied in e-business environment |
| CN103500226A (en) * | 2013-10-23 | 2014-01-08 | 中国农业银行股份有限公司 | Method and device for removing sensitivity of sensitive data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105243063A (en) | 2016-01-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11093501B2 (en) | Searching in a database | |
| CN106980669B (en) | A kind of storage of data, acquisition methods and device | |
| CN106897322B (en) | A kind of access method and device of database and file system | |
| US9454558B2 (en) | Managing an index of a table of a database | |
| US8775425B2 (en) | Systems and methods for massive structured data management over cloud aware distributed file system | |
| US8468120B2 (en) | Systems and methods for tracking and reporting provenance of data used in a massively distributed analytics cloud | |
| Phaneendra et al. | Big Data-solutions for RDBMS problems-A survey | |
| US9756119B2 (en) | Apparatus and method for pipelined event processing in a distributed environment | |
| US11176128B2 (en) | Multiple access path selection by machine learning | |
| Gürcan et al. | Real-time processing of big data streams: Lifecycle, tools, tasks, and challenges | |
| Bhardwaj et al. | Big data emerging technologies: A CaseStudy with analyzing twitter data using apache hive | |
| CN108021809A (en) | A kind of data processing method and system | |
| CN103761102B (en) | A kind of uniform data service platform and its implementation | |
| CN103646051A (en) | Big-data parallel processing system and method based on column storage | |
| Gupta et al. | Faster as well as early measurements from big data predictive analytics model | |
| Saxena et al. | Practical real-time data processing and analytics: distributed computing and event processing using Apache Spark, Flink, Storm, and Kafka | |
| CN109947729A (en) | A kind of real-time data analysis method and device | |
| US20180129690A1 (en) | Schema-Free In-Graph Indexing | |
| Zobaed et al. | Big Data in the Cloud. | |
| CN105243063B (en) | The method and apparatus of information recommendation | |
| De Prato et al. | The next wave:" big data"? | |
| US20190364109A1 (en) | Scale out data storage and query filtering using storage pools | |
| US10467193B1 (en) | Real-time ad hoc querying of data records | |
| CN108804502A (en) | Big data inquiry system, method, computer equipment and storage medium | |
| US20190362011A1 (en) | Scale out data storage and query filtering using data pools |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP02 | Change in the address of a patent holder | ||
| CP02 | Change in the address of a patent holder |
Address after: Room 810, 8 / F, 34 Haidian Street, Haidian District, Beijing 100080 Patentee after: BEIJING D-MEDIA COMMUNICATION TECHNOLOGY Co.,Ltd. Address before: 100089 Beijing city Haidian District wanquanzhuang Road No. 28 Wanliu new building block A room 602 Patentee before: BEIJING D-MEDIA COMMUNICATION TECHNOLOGY Co.,Ltd. |