CN109190043B - Recommendation method and device, storage medium, electronic device and recommendation system - Google Patents
Recommendation method and device, storage medium, electronic device and recommendation system Download PDFInfo
- Publication number
- CN109190043B CN109190043B CN201811046294.6A CN201811046294A CN109190043B CN 109190043 B CN109190043 B CN 109190043B CN 201811046294 A CN201811046294 A CN 201811046294A CN 109190043 B CN109190043 B CN 109190043B
- Authority
- CN
- China
- Prior art keywords
- recall
- model
- main
- user
- results
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The disclosure relates to a recommendation method and device, a storage medium, an electronic device and a recommendation system, which are used for solving the problem that the recommendation system in the related art is low in diversity and accuracy. The recommendation method comprises the following steps: collecting user behavior data, and calling a plurality of recall models in a recall model set according to the user behavior data to obtain a plurality of recall results, wherein the plurality of recall models comprise a main recall model and a secondary recall model which has different recall rules from the main recall model; selecting a target recall result meeting a preset condition from recall results of the secondary recall model; and sorting the recall results of the main recall model and the target recall results and recommending the results to a user.
Description
Technical Field
The present disclosure relates to the field of data processing technologies, and in particular, to a recommendation method and apparatus, a storage medium, an electronic device, and a recommendation system.
Background
In the internet scene, the recommendation system is an indispensable part of many products, and can provide high-quality personalized recommendation service for users under the condition that the users do not have explicit behaviors.
For example, in a take-away scenario, a user is required to make a quick decision, and therefore the top page of an application needs to be able to provide the user with a variety of needs. The flow distribution function is achieved through some novel recommendation results, and meanwhile accurate personalized recommendation needs to be provided for the user, so that the commodity selection time of the user is shortened. This provides a need for a recommendation system that is more versatile and accurate.
However, in the related art, the recommendation system can only use one recall model for a specific scenario, for example, the Itembased CF recall model based on the item recommendation algorithm, which results in a single data granularity and algorithm, and cannot provide a more precise and personalized recommendation service.
Disclosure of Invention
The disclosed embodiments mainly aim to provide a recommendation method and apparatus, a storage medium, an electronic device and a recommendation system, so as to solve the problem that the recommendation system in the related art is low in diversity and accuracy.
In order to achieve the above object, a first aspect of the embodiments of the present disclosure provides a recommendation method, where the method includes:
collecting user behavior data, and calling a plurality of recall models in a recall model set according to the user behavior data to obtain a plurality of recall results, wherein the plurality of recall models comprise a main recall model and a secondary recall model which has different recall rules from the main recall model;
selecting a target recall result meeting a preset condition from recall results of the secondary recall model;
and sorting the recall results of the main recall model and the target recall results and recommending the results to a user.
Optionally, the selecting a target recall result meeting a preset condition from the recall results of the secondary recall model includes:
and selecting a recall result with a click rate and/or a conversion rate meeting a threshold condition from recall results of the secondary recall model as the target recall result according to the log data of the secondary recall model.
Optionally, the invoking a plurality of recall models in an online recall model set according to the user behavior data includes:
and respectively using the user behavior data to call the main recall model and the secondary recall model according to a preset flow ratio, wherein the ratio of the flow entering the main recall model is more than 85%, and the ratio of the flow entering the secondary recall model is more than 0.
Optionally, the method further comprises:
determining the exposure rate and the conversion rate of each recall rule of the recall model according to the log data of the recall model;
deletion of recall rules from the recall model having an exposure above a first threshold and a conversion below a second threshold.
Optionally, the recall result is merchant information recommended to the user; the primary recall model adopts an item-based Itembased recall rule or a user Uerbased recall rule, and the secondary recall model adopts at least one of the following recall rules: user interest recall, association rules, matrix decomposition.
Optionally, the method further comprises:
acquiring historical behavior data of a user;
calculating a recall object similar to a main recall object according to the historical behavior data;
and storing the similar recall object of each main recall object into the recall-to-be-recalled set of the recall model corresponding to the main recall object.
Optionally, the calculating a recall object list similar to a master recall object according to the historical behavior data includes:
calculating a recall object similar to a main recall object by adopting at least one of the following calculation methods of similarity according to the historical behavior data: cosine similarity, jaccard similarity, log-likelihood LLR similarity.
A second aspect of the embodiments of the present disclosure provides a recommendation apparatus, including:
the data collection module is used for collecting user behavior data;
the model calling module is used for calling a plurality of recall models in a recall model set according to the user behavior data to obtain a plurality of recall results, wherein the plurality of recall models comprise a main recall model and a secondary recall model which has different recall rules from the main recall model;
the result screening module is used for selecting a target recall result meeting preset conditions from the recall results of the secondary recall model;
and the sequencing recommendation module is used for sequencing the recall result of the main recall model and the target recall result and then recommending the result to a user.
Optionally, the result screening module is configured to:
and selecting a recall result with a click rate and/or a conversion rate meeting a threshold condition from recall results of the secondary recall model as the target recall result according to the log data of the secondary recall model.
Optionally, the model invoking module is configured to:
and respectively using the user behavior data to call the main recall model and the secondary recall model according to a preset flow ratio, wherein the ratio of the flow entering the main recall model is more than 85%, and the ratio of the flow entering the secondary recall model is more than 0.
Optionally, the apparatus further comprises:
the index calculation module is used for determining the exposure rate and the conversion rate of each recall rule of the recall model according to the log data of the recall model;
a rule deletion module to delete recall rules from the recall model having an exposure above a first threshold and a conversion below a second threshold.
Optionally, the recall result is merchant information recommended to the user; the primary recall model adopts an item-based Itembased recall rule or a user Uerbased recall rule, and the secondary recall model adopts at least one of the following recall rules: user interest recall, association rules, matrix decomposition.
Optionally, the apparatus further comprises:
the acquisition module is used for acquiring historical behavior data of a user;
the similarity calculation module is used for calculating a recall object similar to a main recall object according to the historical behavior data;
and the storage module is used for storing the similar recall object of each main recall object into the to-be-recalled set of the recall model corresponding to the main recall object.
Optionally, the similarity calculation module is configured to:
calculating a recall object similar to a main recall object by adopting at least one of the following calculation methods of similarity according to the historical behavior data: cosine similarity, jaccard similarity, log-likelihood LLR similarity.
A third aspect of embodiments of the present disclosure provides a non-transitory computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the steps of the method of the first aspect.
A fourth aspect of the embodiments of the present disclosure provides an electronic device, including:
a memory having a computer program stored thereon;
a processor for executing the computer program in the memory to implement the steps of the method of the first aspect.
A fifth aspect of the embodiments of the present disclosure provides a recommendation system, which includes at least one server on which a log system for storing log data of recall results is deployed, a set of recall models, and an apparatus for performing the method of the first aspect.
By adopting the technical scheme, the following technical effects can be at least achieved:
the recommendation method provided by the embodiment of the disclosure uses the secondary recall model with different recall rules in addition to the main recall model, so that the diversity of the algorithm is improved, and further, the personalization of the recommendation system is improved. And the recall result of the secondary recall model can be fused with the recall result of the main recall model, so that the recommendation accuracy is ensured on the basis of improving the diversity.
Additional features and advantages of embodiments of the present disclosure will be described in detail in the detailed description which follows.
Drawings
The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the disclosure without limiting the disclosure. In the drawings:
fig. 1 is a schematic flow chart of a recommendation method provided by an embodiment of the present disclosure;
FIG. 2 is a schematic structural diagram of a recommendation system provided by an embodiment of the present disclosure;
FIG. 3 is a schematic diagram of on-line recommendation logic in the recommendation system shown in FIG. 2;
FIG. 4 is a schematic diagram of the feedback system in the recommendation system of FIG. 2 screening targeted recall results in a secondary recall model;
FIG. 5 is a schematic structural diagram of another recommendation system provided by the embodiments of the present disclosure;
fig. 6 is a schematic structural diagram of a recommendation device provided in an embodiment of the present disclosure;
fig. 7 is a schematic structural diagram of an electronic device provided in an embodiment of the present disclosure.
Detailed Description
The following detailed description of specific embodiments of the present disclosure is provided in connection with the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating the present disclosure, are given by way of illustration and explanation only, not limitation.
As shown in fig. 1, a recommendation method provided in an embodiment of the present disclosure includes:
s101, collecting user behavior data, and calling a plurality of recall models in a recall model set according to the user behavior data to obtain a plurality of recall results.
Wherein the plurality of recall models includes a primary recall model and a secondary recall model having different recall rules than the primary recall model.
It is noted that the recall in the recommendation system may be, for example, a hit recall, a user interest recall, an association rule, collaborative filtering, matrix factorization, DNN (Deep Neural Network), and the like. Different recall models may be employed for different application scenarios. For example, when the recommended scene is an APP home page of an application program or a page of a large category of different categories, the user is required to quickly find information desired by the user when opening the APP or entering the page of the large category, and therefore recommendation can be performed mainly according to personal preference of the user in such a scene. For another example, when the user enters the product detail page, the user is requested to recommend another product related to the current product, and in such a scenario, the recommendation may be performed mainly for the product on the current detail page and assisted by the preference information of the user. For another example, the user may enter the item list page through the screening item or the search box on the item list page to obtain information, and if there are few or no results searched by the current screening item or the search condition, the user may request to trigger the recommendation logic to perform information recommendation. In the above example, different recalling models may be used for different recommendations.
Therefore, in specific implementation, the recall rules of the major recall model and the minor recall model can be set according to the actual application scenario, wherein the recall rule of the major recall model is closer to the application scenario than the minor recall model, and the recall rule of the minor recall model can be used as a supplement to the diversity. In addition, in different application scenarios, different recall models are adopted, and corresponding recall results are also different, for example, for a social network site, the recall result may be a user, and for a shopping site, the recall result may be a merchant or a commodity.
And S102, selecting a target recall result meeting preset conditions from the recall results of the secondary recall model.
Since the recall results of the sub-recall model are usually mixed with recall results with low accuracy, in order to avoid affecting the accuracy of the recommendation result displayed to the user in the final recommendation, in this step S102, a high-quality recall result can be selected from the recall results of the sub-recall model, for example, a recall result with a high conversion rate, a high click rate, or a high similarity score with the primary recall object is selected from the recall results of all sub-recall models through preset corresponding conditions.
S103, sorting the recall result of the main recall model and the target recall result and recommending the sorted result to a user.
Specifically, most recommendation applications show a recommendation result list to users, and belong to a topN recommendation mode, so after a recall result of the master recall model and a filtered target recall result are obtained as recommendation candidate sets, the recommendation candidate sets need to be ranked, and finally, the ranked recommendation results are provided to upper-layer applications through corresponding interface APIs and are shown to the users.
In the related technology, a single recall model is used, which only can cover preference habits of specific users and cannot have good performance on all user groups. In the technical scheme provided by the embodiment of the disclosure, besides the main recall model, the secondary recall model with different recall rules is used, so that the diversity of the algorithm is improved, and further the individuation of the recommendation system is improved. And the recall result of the secondary recall model can be fused with the recall result of the main recall model, so that the recommendation accuracy is ensured on the basis of improving the diversity.
Specifically, the step S102 may include: and selecting a recall result with a click rate and/or a conversion rate meeting a threshold condition from recall results of the secondary recall model as the target recall result according to the log data of the secondary recall model. That is, the preset condition in step S102 refers to a threshold condition of an index for measuring the accuracy of the recall result, where the index includes a click rate and/or a conversion rate, and both the click rate and the conversion rate can be obtained from log data.
Specifically, the log data includes an exposure log representing recommended display, a click log representing click viewing by a user, and an order forming log representing whether the user places an order to purchase a commodity or not in the case that the recommended object is a commodity or a merchant. Therefore, the click rate of the recall object can be calculated based on the exposure log and the click log, and the conversion rate of the recall object can be calculated based on the click log and the list forming log.
Exemplarily, fig. 2 is a schematic diagram of an application scenario provided by an embodiment of the present disclosure, and the recommendation system shown in fig. 2 includes: an online system 11 and a feedback system 12. The online system 11 includes a recall model set, such as the main recall model shown in fig. 2, and a secondary recall model 1 to a secondary recall model N, and the online system 11 is configured to distribute behavior data of a user to each recall model, obtain a recall result, and record log data of each recall object of the recall result;
the feedback system 12 is configured to determine, according to the log data, a target recall result for which a click rate and/or a conversion rate meet a threshold condition, and fuse the target recall result to a recall result of the master recall model.
Based on the recommendation system shown in fig. 2, a detailed recommendation flow is shown in fig. 3:
after the online system collects behavior data of the user through the upper-layer application, the behavior data are firstly distributed to all recall models to obtain recall results output by each recall model. FIG. 3 is an example of obtaining a recall object based on each recall model, recall object 1 through recall object N + 1. And the feedback system screens target recall objects with click rate and/or conversion rate meeting threshold conditions from recall results of the secondary recall model based on log data in the log system, takes the target recall objects and recall objects of the main recall model as candidate sets, inputs the candidate sets into the sequencing model for sequencing, and finally provides the sequenced recommendation results to an upper application through a corresponding interface API (application programming interface) to be displayed to a user.
Optionally, the step S101 of calling a plurality of recall models in the online recall model set according to the user behavior data includes: and respectively using the user behavior data to call the main recall model and the secondary recall model according to a preset flow ratio, wherein the ratio of the flow entering the main recall model is more than 85%, and the ratio of the flow entering the secondary recall model is more than 0. Still taking the example of fig. 2, after the user behavior data is collected, the primary recall model may be invoked using the behavior data traffic of more than 85% and less than 95%, and the remaining 5% to 15% of the traffic enters the secondary recall models 1 to N. The recall result of the secondary recall model is used as a supplement of the diversity of the main recall model instead of as a lead, so that the recommendation accuracy is ensured.
In a possible implementation manner of the embodiment of the present disclosure, the method further includes: determining the exposure rate and the conversion rate of each recall rule of the recall model according to the log data of the recall model; deletion of recall rules from the recall model having an exposure above a first threshold and a conversion below a second threshold; and/or, removing the recall rule with the exposure rate higher than a third threshold value and the click rate lower than a fourth threshold value from the recall model. That is, if a certain product is recommended many times (i.e. the exposure rate is high), but the number of times that the product is clicked and viewed by the user is small (i.e. the click rate is small), the recall rule indicating that the product is recalled is not accurate enough, and does not meet the user's requirement, so that the product can be deleted from the recall model. Similarly, for the commodities with high exposure rate and low conversion rate, the commodities can be deleted from the recall model, and the accuracy of model recommendation is improved. The above is only an example, and corresponding precision thresholds may also be set for the click rate and the conversion rate, so as to determine whether the recall rule needs to be deleted, that is, when the click rate or the conversion rate is smaller than the corresponding precision threshold, the recall rule is deleted from the recall model.
Still referring to fig. 2, the process of the feedback system calculating the relevant indexes of each recall model based on the log data in the log system is shown in fig. 4. Wherein each of the log data includes an identification of a recall model, an identification of a recall object of the recall model, and an identification of objects of the recall model that are similar to the recall object. Wherein id shown in fig. 4 is an identifier of a recall rule of an online experiment, which is used for a feedback system to identify the online recall rule represented by the log data, cf _ id is an identifier of a recall model, item1 is a master item in the recall model, item2 is an item similar to the master item in the recall model, ctr is a click rate of exposure to click, and cvr is a click to singleton conversion rate.
Specifically, the feedback system can obtain the relationship between the main item of each recall rule and the item recalled by the recall model in the log system, and can know the exposure and click rate of each recalled object from the log system, so that which recall rules are accurately applicable and which recall rules can be deleted according to the exposure and click rate.
Illustratively, assume that the primary recall object in the secondary recall model a is item1, the corresponding similar recall objects are item2, item3, item5, item6, and, as can be seen from the log data, item2, item3, item5, item6 are exposed 100, 200, 50, 400 times, respectively, clicked 80, 100, 30, 10 times, respectively. From this, it was found that the click rates of item2, item3, item5 and item6 were 0.8, 0.5, 0.6 and 0.025, respectively. In this case, recall item2, item3, item5 may be merged with the recall objects of the primary recall model (i.e., sorted recommendations are presented to the user), and further recall rules of recall item6 may be deleted from the secondary recall model a.
The method provided by the embodiment of the disclosure can be used in the field of instant delivery, such as recommendation of take-away merchants, that is, the recall result is merchant information recommended to the user. In this case, the primary recall model employs either an item-based recall rule or a user Userbased-based recall rule, and the secondary recall model employs at least one of the following recall rules: user interest recall, association rules, matrix decomposition.
Optionally, the recommendation method provided in the embodiment of the present disclosure further includes a method for calculating the similarity between the recall objects offline, including: acquiring historical behavior data of a user; calculating a recall object similar to a main recall object according to the historical behavior data; and storing the similar recall object of each main recall object into the recall-to-be-recalled set of the recall model corresponding to the main recall object.
Illustratively, as shown in fig. 5, the recommendation system includes an offline system 13, which is mainly used for extracting the historical behavior data of the user. In particular, historical behavior data of a user that may be used includes data of the user's following behavior over a period of time (e.g., 90 days): a Point of Interest (poi) browsed by a user; clicking data of the poi by a user in the session; the user clicks on the data of the poi behavior after searching within the session.
The historical behavior data of the user extracted by the offline system can be stored in a Hive table of a Hadoop-based data warehouse tool. Wherein the stored table fields are shown in table 1 below:
TABLE 1
And the offline system extracts the obtained historical behavior data of the user and is used for calculating the similarity between the updated articles. For example, the similarity calculation tool can be implemented by using distributed computing Spark of MapReduce algorithm, and can realize the calculation of the similarity between billion-level recalled object items. In specific implementation, similarity calculation among items can be completed through three MapReduce, each MapReduce completes one calculation operator, and the following three calculation operators are provided:
a) a pretreatment operator:
b) norm operator:
c) similarity calculator: si,j＝similarity(doti,j,ni,nj),
The following describes the calculation process by taking the jaccard similarity of a specific calculation vector as an example, where, for example, the vectors required to calculate the jaccard similarity are as follows:
then, the calculation by the preprocessing operator can obtain:
further through norm operator calculation, the following can be obtained:
the Jack similarity is further calculated as:
the foregoing is only an example of the jaccard similarity, and in specific implementation, the similarity between the recall objects may be calculated according to any one of similarity calculation methods of calculating cosine similarity, jaccard similarity, and log-likelihood LLR similarity. The following table 2 shows the calculation operators corresponding to the three common similarity calculation methods.
TABLE 2
Optionally, after obtaining the similarity list of the main recall object and other objects through similarity calculation, the similarity list may be stored in the key value kv system in the following format:
key:item
value:item_1:weight_1,item_2:weight_2,……,item_N:weight_N
wherein item represents a recall object, item _1 to item _ N are other objects similar to item, and weight represents the similarity between items. The similarity list of each main recall object is used for providing each recall model of the online system for article recall, and the recall efficiency is improved by adopting key value storage.
Based on the same inventive concept, the embodiments of the present disclosure further provide a recommendation apparatus for implementing the steps of the recommendation method provided by the above method embodiments, as shown in fig. 6, the apparatus includes:
a data collection module 61 for collecting user behavior data;
a model calling module 62, configured to call, according to the user behavior data, a plurality of recall models in a recall model set to obtain a plurality of recall results, where the plurality of recall models include a primary recall model and a secondary recall model having a different recall rule from the primary recall model;
a result screening module 63, configured to select a target recall result that meets a preset condition from the recall results of the secondary recall model;
and a ranking recommendation module 64, configured to rank the recall result of the master recall model and the target recall result and recommend the ranked recall result to the user.
By adopting the device, the device uses the secondary recall model with different recall rules besides the main recall model, so that the diversity of the algorithm is improved, and the individuation of the recommendation system is further improved. And the recall result of the secondary recall model can be fused with the recall result of the main recall model, so that the recommendation accuracy is ensured on the basis of improving the diversity.
Optionally, the result screening module 63 is configured to:
and selecting a recall result with a click rate and/or a conversion rate meeting a threshold condition from recall results of the secondary recall model as the target recall result according to the log data of the secondary recall model.
Optionally, the model invoking module 62 is configured to:
and respectively using the user behavior data to call the primary recall model and the secondary recall model according to a preset flow ratio, wherein the ratio of the flow entering the primary recall model is 85-95%.
Optionally, the apparatus further comprises:
the index calculation module is used for determining the exposure rate and the conversion rate of each recall rule of the recall model according to the log data of the recall model;
a rule deletion module to delete recall rules from the recall model having an exposure above a first threshold and a conversion below a second threshold.
Optionally, the recall result is merchant information recommended to the user; the primary recall model adopts an item-based Itembased recall rule or a user Uerbased recall rule, and the secondary recall model adopts at least one of the following recall rules: user interest recall, association rules, matrix decomposition.
Optionally, the apparatus further comprises:
the acquisition module is used for acquiring historical behavior data of a user;
the similarity calculation module is used for calculating a recall object similar to a main recall object according to the historical behavior data;
and the storage module is used for storing the similar recall object of each main recall object into the to-be-recalled set of the recall model corresponding to the main recall object.
Optionally, the similarity calculation module is configured to:
calculating a recall object similar to a main recall object by adopting at least one of the following calculation methods of similarity according to the historical behavior data: cosine similarity, jaccard similarity, log-likelihood LLR similarity.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
The disclosed embodiments also provide a non-transitory computer-readable storage medium on which a computer program is stored, which when executed by a processor implements the steps of the recommendation method described above.
An embodiment of the present disclosure further provides an electronic device, including:
a memory having a computer program stored thereon;
a processor for executing the computer program in the memory to implement the steps of the above-mentioned recommended method.
Exemplarily, fig. 7 is a schematic structural diagram of the above electronic device provided by the embodiment of the present disclosure. Wherein the electronic device may be provided as a server. Referring to fig. 7, an electronic device 700 includes a processing component 701 that further includes one or more processors and memory resources, represented by memory 702, for storing instructions, such as applications, that are executable by the processing component 701. The application programs stored in memory 702 may include one or more modules that each correspond to a set of instructions. Furthermore, the processing component 701 is configured to execute instructions to perform the steps of the recommendation method described above.
The electronic device 700 may also include a power component 703 configured to perform power management of the electronic device 700, a wired or wireless network interface 704 configured to connect the electronic device to a network, and an input-output (I/O) interface 705. The electronic device 700 may operate based on an operating system stored in the memory 702, such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, or the like.
The embodiment of the disclosure also provides a recommendation system, which includes at least one server, on which a log system for storing log data of recall results, a recall model set, and an apparatus for executing the recommendation method are deployed. That is to say, the apparatus for executing the recommendation method runs the recall model set, and the system for storing log data may be deployed on the same server or may be deployed on different servers, which is not limited in this disclosure.
The preferred embodiments of the present disclosure are described in detail with reference to the accompanying drawings, however, the present disclosure is not limited to the specific details of the above embodiments, and various simple modifications may be made to the technical solution of the present disclosure within the technical idea of the present disclosure, and these simple modifications all belong to the protection scope of the present disclosure. It should be noted that, in the foregoing embodiments, various features described in the foregoing embodiments may be combined in any suitable manner, and in order to avoid unnecessary repetition, various possible combinations are not separately described in the embodiments of the present disclosure.
In addition, any combination of various embodiments of the present disclosure may be made, and the same should be considered as the disclosure of the present disclosure, as long as it does not depart from the spirit of the present disclosure.
Claims (10)
1. A recommendation method, characterized in that the method comprises:
collecting user behavior data, and calling a plurality of recall models in a recall model set according to the user behavior data to obtain a plurality of recall results, wherein the plurality of recall models comprise a main recall model and a secondary recall model which has different recall rules from the main recall model;
selecting a target recall result meeting a preset condition from recall results of the secondary recall model;
the recall results of the main recall model and the target recall results are ranked and then recommended to a user;
the method further comprises the following steps:
determining the exposure rate and the conversion rate of each recall rule of the recall model according to the log data of the recall model;
recall rules with exposure above a first threshold and conversion below a second threshold are deleted from the recall model.
2. The method of claim 1, wherein said selecting a target recall result from the recall results of the minor recall model that meets a preset condition comprises:
and selecting a recall result with a click rate and/or a conversion rate meeting a threshold condition from recall results of the secondary recall model as the target recall result according to the log data of the secondary recall model.
3. The method of claim 1, wherein said invoking a plurality of recall models from a set of online recall models from the user behavior data comprises:
and respectively using the user behavior data to call the main recall model and the secondary recall model according to a preset flow ratio, wherein the ratio of the flow entering the main recall model is more than 85%, and the ratio of the flow entering the secondary recall model is more than 0.
4. The method of any of claims 1 to 3, wherein the recall result is merchant information recommended to the user; the primary recall model adopts an item-based Itembased recall rule or a user Uerbased recall rule, and the secondary recall model adopts at least one of the following recall rules: user interest recall, association rules, matrix decomposition.
5. The method according to any one of claims 1 to 3, further comprising:
acquiring historical behavior data of a user;
calculating a recall object similar to a main recall object according to the historical behavior data;
and storing the similar recall object of each main recall object into the recall-to-be-recalled set of the recall model corresponding to the main recall object.
6. The method of claim 5, wherein said computing a recall object list from said historical behavioral data that is similar to a master recall object comprises:
calculating a recall object similar to a main recall object by adopting at least one of the following calculation methods of similarity according to the historical behavior data: cosine similarity, jaccard similarity, log-likelihood LLR similarity.
7. A recommendation device, characterized in that the device comprises:
the data collection module is used for collecting user behavior data;
the model calling module is used for calling a plurality of recall models in a recall model set according to the user behavior data to obtain a plurality of recall results, wherein the plurality of recall models comprise a main recall model and a secondary recall model which has different recall rules from the main recall model;
the result screening module is used for selecting a target recall result meeting preset conditions from the recall results of the secondary recall model;
the sequencing recommendation module is used for sequencing the recall result of the main recall model and the target recall result and then recommending the result to a user;
the device further comprises:
the index calculation module is used for determining the exposure rate and the conversion rate of each recall rule of the recall model according to the log data of the recall model;
and the rule deleting module is used for deleting the recall rule with the exposure rate higher than the first threshold and the conversion rate lower than the second threshold from the recall-once model.
8. A non-transitory computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 6.
9. An electronic device, comprising:
a memory having a computer program stored thereon;
a processor for executing the computer program in the memory to carry out the steps of the method of any one of claims 1 to 6.
10. A recommendation system, comprising at least one server on which is deployed a logging system for storing log data of recall results, a set of recall models, and means for performing the method of any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811046294.6A CN109190043B (en) | 2018-09-07 | 2018-09-07 | Recommendation method and device, storage medium, electronic device and recommendation system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811046294.6A CN109190043B (en) | 2018-09-07 | 2018-09-07 | Recommendation method and device, storage medium, electronic device and recommendation system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109190043A CN109190043A (en) | 2019-01-11 |
| CN109190043B true CN109190043B (en) | 2021-04-23 |
Family
ID=64915397
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811046294.6A Active CN109190043B (en) | 2018-09-07 | 2018-09-07 | Recommendation method and device, storage medium, electronic device and recommendation system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109190043B (en) |
Families Citing this family (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109948023B (en) * | 2019-03-08 | 2023-11-03 | 腾讯科技（深圳）有限公司 | Recommended object acquisition method, recommended object acquisition device and storage medium |
| CN110275779B (en) * | 2019-06-20 | 2022-07-08 | 北京百度网讯科技有限公司 | Resource acquisition method, device, equipment and storage medium |
| CN110781341A (en) * | 2019-10-09 | 2020-02-11 | 上海麦克风文化传媒有限公司 | Audio album recommendation method and system fusing multi-strategy recall data sets |
| CN110909200A (en) * | 2019-10-09 | 2020-03-24 | 上海麦克风文化传媒有限公司 | Offline and real-time combined audio album recommendation system recall method and system |
| CN110942376B (en) * | 2019-12-02 | 2023-09-01 | 上海麦克风文化传媒有限公司 | Fusion method of real-time multi-recall strategy of audio product |
| CN111259232B (en) * | 2019-12-03 | 2022-08-12 | 江苏艾佳家居用品有限公司 | Recommendation system optimization method based on personalized recall |
| CN111241388A (en) * | 2019-12-13 | 2020-06-05 | 北京三快在线科技有限公司 | Multi-policy recall method and device, electronic equipment and readable storage medium |
| CN111161021B (en) * | 2019-12-23 | 2021-10-08 | 叮当快药科技集团有限公司 | Quick secondary sorting method for recommended commodities based on real-time characteristics |
| CN111178970B (en) * | 2019-12-30 | 2023-06-30 | 微梦创科网络科技（中国）有限公司 | Advertisement putting method and device, electronic equipment and computer readable storage medium |
| CN111325598B (en) * | 2020-01-22 | 2023-04-28 | 深圳大学 | Article recommendation method and device and terminal equipment |
| CN111340561A (en) * | 2020-03-04 | 2020-06-26 | 深圳前海微众银行股份有限公司 | Information click rate calculation method, device, equipment and readable storage medium |
| CN113407816A (en) * | 2020-03-17 | 2021-09-17 | 北京达佳互联信息技术有限公司 | Object recommendation method and device, electronic equipment and storage medium |
| CN111444438B (en) * | 2020-03-24 | 2023-09-01 | 北京百度网讯科技有限公司 | Method, device, equipment and storage medium for determining quasi-recall rate of recall strategy |
| CN111597454A (en) * | 2020-04-02 | 2020-08-28 | 微梦创科网络科技（中国）有限公司 | Account recommendation method and device |
| CN111538901B (en) * | 2020-04-15 | 2023-06-06 | 达而观信息科技（上海）有限公司 | Article recommendation method and device, server and storage medium |
| CN111539210B (en) * | 2020-04-16 | 2023-08-11 | 支付宝(杭州)信息技术有限公司 | Cross-network entity identification method and device, electronic equipment and medium |
| CN113763010A (en) * | 2020-11-19 | 2021-12-07 | 北京沃东天骏信息技术有限公司 | Information pushing method and device |
| CN112685633A (en) * | 2020-12-30 | 2021-04-20 | 杭州智聪网络科技有限公司 | Information recommendation method and system based on recall model and prediction model |
| CN112785397B (en) * | 2021-03-09 | 2024-03-29 | 中国工商银行股份有限公司 | Product recommendation method, device and storage medium |
| CN113343089A (en) * | 2021-06-11 | 2021-09-03 | 北京完美赤金科技有限公司 | User recall method, device and equipment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108198045A (en) * | 2018-01-30 | 2018-06-22 | 东华大学 | The design method of mixing commending system based on e-commerce website data mining |
| CN108415928A (en) * | 2018-01-18 | 2018-08-17 | 郝宁宁 | A kind of book recommendation method and system based on weighted blend k- nearest neighbor algorithms |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9940606B2 (en) * | 2013-10-30 | 2018-04-10 | Chegg, Inc. | Correlating jobs with personalized learning activities in online education platforms |
| US10049416B2 (en) * | 2013-11-26 | 2018-08-14 | Chegg, Inc. | Job recall services in online education platforms |
| CN106228386B (en) * | 2016-07-12 | 2018-09-25 | 腾讯科技（深圳）有限公司 | A kind of information-pushing method and device |
-
2018
- 2018-09-07 CN CN201811046294.6A patent/CN109190043B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108415928A (en) * | 2018-01-18 | 2018-08-17 | 郝宁宁 | A kind of book recommendation method and system based on weighted blend k- nearest neighbor algorithms |
| CN108198045A (en) * | 2018-01-30 | 2018-06-22 | 东华大学 | The design method of mixing commending system based on e-commerce website data mining |
Non-Patent Citations (1)
| Title |
|---|
| "基于改进的个性化混合推荐算法的研究";钟良月;《中国优秀硕士学位论文全文数据库 社会科学Ⅱ辑》;20180531(第05期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109190043A (en) | 2019-01-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109190043B (en) | Recommendation method and device, storage medium, electronic device and recommendation system | |
| CN107424043B (en) | Product recommendation method and device and electronic equipment | |
| US10685065B2 (en) | Method and system for recommending content to a user | |
| CN109241415B (en) | Project recommendation method and device, computer equipment and storage medium | |
| CN107451894B (en) | Data processing method, device and computer readable storage medium | |
| CN107862022B (en) | Culture resource recommendation system | |
| CN109033101B (en) | Label recommendation method and device | |
| CN110175895B (en) | Article recommendation method and device | |
| CN111708949A (en) | Medical resource recommendation method and device, electronic equipment and storage medium | |
| CN104933100A (en) | Keyword recommendation method and device | |
| CN105205188A (en) | Method and device for recommending purchase material suppliers | |
| CN109685537B (en) | User behavior analysis method, device, medium and electronic equipment | |
| TW201401088A (en) | Search method and apparatus | |
| CN111339406B (en) | Personalized recommendation method, device, equipment and storage medium | |
| Sivaramakrishnan et al. | An effective user clustering-based collaborative filtering recommender system with grey wolf optimisation | |
| CN112883265A (en) | Information recommendation method and device, server and computer readable storage medium | |
| JP2015162109A (en) | Task assignment server, task assignment method, and program | |
| CN111078997A (en) | Information recommendation method and device | |
| JP7092194B2 (en) | Information processing equipment, judgment method, and program | |
| CN114090877A (en) | Position information recommendation method and device, electronic equipment and storage medium | |
| JP5622880B2 (en) | Item recommendation system, item recommendation method, and item recommendation program | |
| CN106575418A (en) | Suggested keywords | |
| CN110874432B (en) | Sorting method, information recommendation method, system and device | |
| CN114707068A (en) | Method, device, equipment and medium for recommending intelligence base knowledge | |
| CN111460300A (en) | Network content pushing method and device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |