CN114741599A - News recommendation method and system based on knowledge enhancement and attention mechanism - Google Patents
News recommendation method and system based on knowledge enhancement and attention mechanism Download PDFInfo
- Publication number
- CN114741599A CN114741599A CN202210421741.1A CN202210421741A CN114741599A CN 114741599 A CN114741599 A CN 114741599A CN 202210421741 A CN202210421741 A CN 202210421741A CN 114741599 A CN114741599 A CN 114741599A
- Authority
- CN
- China
- Prior art keywords
- news
- user
- entity
- candidate
- entities
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9535—Search customisation based on user profiles and personalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/335—Filtering based on additional data, e.g. user or group profiles
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/901—Indexing; Data structures therefor; Storage structures
- G06F16/9024—Graphs; Linked lists
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/02—Knowledge representation; Symbolic representation
Abstract
The invention discloses a news recommendation method and a system based on knowledge enhancement and attention mechanism, wherein the method comprises the following steps: acquiring historical click data of a user on a news link; acquiring candidate news data; inputting both the historical click data and the candidate news data into a trained news recommendation model to obtain news data which is personalized and recommended for a user; the news recommendation model is used for constructing a plurality of user interest representations based on historical click data of a user on a plurality of news links; matching all the user interest representations with each candidate news data to obtain matching results; aggregating all the user interest representations by using different weights to obtain an aggregation result; obtaining a feature vector of the news which is interested by the user based on the matching result and the aggregation result; splicing the feature vector of the news which is interested by the user and the feature vector of the candidate news, and predicting the splicing result to obtain recommended candidate news; and recommending the result to the user.
Description
Technical Field
The invention relates to the technical field of big data and natural language processing, in particular to a news recommendation method and system based on knowledge enhancement and attention mechanism.
Background
The statements in this section merely provide background information related to the present disclosure and may not constitute prior art.
With the rapid development of the internet, the demand of users for online news is also gradually increasing. The news platform has a large amount of news information, and if the news platform is directly selected by a user, great selection trouble is caused to the user. Therefore, it is crucial for the news platform to construct a personalized recommendation system capable of capturing the reading interest of the user and pushing related information. In the recommendation system, there are mainly 3 types of algorithms: content-based methods, collaborative filtering and mixing methods. Due to the fact that collaborative filtering has the characteristic that interaction information of users and articles is sparse and the cold start problem exists, the collaborative filtering is not suitable for news recommendation scenes, and the mainstream news recommendation method is usually based on content or a mixing method.
At present, most news-based recommendation systems only learn the representation of news from a semantic layer, and ignore information of a knowledge level contained in the news. Most of knowledge-level information exists in the knowledge graph, and related research for combining the news recommendation model with the knowledge graph is few. Wang et al propose a depth recommendation model DKN that fuses knowledge-graph information. DKN, firstly, a knowledge graph highly related to the task is constructed, and then the entity information of the knowledge graph is merged into the neural network on the basis of the traditional deep neural network model, thereby realizing the goal of merging the semantic representation of news and the knowledge representation of the entity. Although the method considers information on different levels, experiments prove that the method has better effect than the traditional method without adding entity information, but the importance degree of neighbors in the neighborhood of a certain hop count is not considered.
In addition, news has certain instantaneity, and the deep recommendation model DKN provided by Wang et al, which is used for fusing knowledge map information, only uses a single convolutional neural network based on an attention mechanism to acquire the interests of the user, so that the interests of the user cannot be dynamically and accurately acquired within a period of time.
Disclosure of Invention
In order to solve the defects of the prior art, the invention provides a news recommendation method and a system based on knowledge enhancement and attention mechanism; the method is characterized in that a knowledge graph and an attention network are introduced, the knowledge graph is used for obtaining entities from data, new entity links and new association rules, on the basis of utilizing the knowledge graph, the attention network is combined for carrying out association search on text features in news titles, the weight of related edges, namely the importance degree of the relationship, is calculated in a set step number, mainly reachable paths among news entities are found, reachable paths among the news entities are selected and found, and additional related entities and context representation are obtained so as to enhance text feature representation and obtain rich news features.
In a first aspect, the invention provides a news recommendation method based on knowledge enhancement and attention mechanism;
the news recommendation method based on knowledge enhancement and attention mechanism comprises the following steps:
acquiring historical click data of a user on a news link; acquiring candidate news data;
inputting both historical click data and candidate news data into the trained news recommendation model to obtain news data which is recommended for a user in an individualized manner;
the news recommendation model constructs a plurality of user interest representations based on historical click data of a user on a plurality of news links; matching all the user interest representations with each candidate news data to obtain matching results; aggregating all the user interest representations by using different weights to obtain an aggregation result; obtaining a feature vector of the news which is interested by the user based on the matching result and the aggregation result; splicing the feature vector of the news which is interested by the user and the feature vector of the candidate news, and predicting the splicing result to obtain recommended candidate news; and recommending the result to the user.
In a second aspect, the invention provides a news recommendation system based on knowledge enhancement and attention mechanisms;
news recommendation system based on knowledge enhancement and attention mechanism, comprising:
an acquisition module configured to: acquiring historical click data of a user on a news link; acquiring candidate news data;
a recommendation module configured to: inputting both the historical click data and the candidate news data into a trained news recommendation model to obtain news data which is personalized and recommended for a user;
the news recommendation model constructs a plurality of user interest representations based on historical click data of a user on a plurality of news links; matching all the user interest representations with each candidate news data to obtain matching results; aggregating all the user interest representations by using different weights to obtain an aggregation result; obtaining a feature vector of the news which is interested by the user based on the matching result and the aggregation result; splicing the feature vector of the news which is interested by the user and the feature vector of the candidate news, and predicting the splicing result to obtain recommended candidate news; and recommending the result to the user.
In a third aspect, the present invention further provides an electronic device, including:
a memory for non-transitory storage of computer readable instructions; and
a processor for executing the computer readable instructions,
wherein the computer readable instructions, when executed by the processor, perform the method of the first aspect.
In a fourth aspect, the present invention also provides a storage medium storing non-transitory computer readable instructions, wherein the non-transitory computer readable instructions, when executed by a computer, perform the instructions of the method of the first aspect.
In a fifth aspect, the invention also provides a computer program product comprising a computer program for implementing the method of the first aspect when run on one or more processors.
Compared with the prior art, the invention has the beneficial effects that:
(1) compared with the traditional news-based recommendation method, the method has the advantages that under the condition that the training efficiency is guaranteed, the data of a plurality of news platforms are used, so that the interest of users is more deeply mined, the click through rate CTR of news is more accurately estimated under different news data conditions, and the accuracy of the overall CTR estimation is improved;
(2) the invention introduces a knowledge graph and an attention network, uses the knowledge graph to obtain entities, new entity links and new association rules from data, and performs association search on text features in news titles by combining the attention network on the basis of using the knowledge graph, calculates the weight of related edges in proper steps, namely the importance degree of the relationship, mainly finds out reachable paths among news entities, selects the reachable paths among the news entities, and obtains additional related entities and context representation so as to enhance text feature representation and obtain rich news features.
(3) The invention introduces a plurality of convolutional neural networks based on attention mechanism, the attention mechanism refers to the processing mode of human vision, and focuses attention on a key area, and the essence of the mechanism is to select information playing a key role in a task from a plurality of information, so that the complexity of the task is reduced; the method dynamically acquires the current interest of the user through the convolutional neural network based on the attention mechanism, namely the user behavior is linked with the current behavior through the attention mechanism, so that the real-time interest of the user is well estimated, and the prediction capability is improved; therefore, the expression vector of the user interest changes with the difference of news, the expression capability under the limited dimensionality is improved, and the different interests of the user can be better quantified.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the invention and together with the description serve to explain the invention and not to limit the invention.
FIG. 1 is a flowchart of a method according to a first embodiment;
FIG. 2 is a schematic diagram of a convolutional neural network based on an attention mechanism according to a first embodiment;
fig. 3 is a schematic diagram of an encoder according to the first embodiment.
Detailed Description
It is to be understood that the following detailed description is exemplary and is intended to provide further explanation of the invention as claimed. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the invention. As used herein, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise, and it should be understood that the terms "comprises" and "comprising", and any variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The embodiments and features of the embodiments of the present invention may be combined with each other without conflict.
All data are obtained according to the embodiment and are legally applied on the data on the basis of compliance with laws and regulations and user consent.
Example one
The embodiment provides a news recommendation method based on knowledge enhancement and attention mechanism;
as shown in fig. 1, a news recommendation method based on knowledge enhancement and attention mechanism includes:
s101: acquiring historical click data of a user on a news link; acquiring candidate news data;
s102: inputting both the historical click data and the candidate news data into a trained news recommendation model to obtain news data which is personalized and recommended for a user;
the news recommendation model constructs a plurality of user interest representations based on historical click data of a user on a plurality of news links; matching all the user interest representations with each candidate news data to obtain matching results; aggregating all the user interest representations by using different weights to obtain an aggregation result; obtaining a feature vector of the news which is interested by the user based on the matching result and the aggregation result; splicing the feature vectors of the news in which the user is interested and the feature vectors of the candidate news, and predicting a splicing result to obtain recommended candidate news; and recommending the result to the user.
Further, the news recommendation model includes: m first encoders and a second encoder; wherein M is a positive integer;
the first encoder is used for encoding news titles browsed by the user to obtain user interest representations;
the second encoder is used for encoding the candidate news headlines to obtain the feature vectors of the candidate news;
the output ends of two adjacent first encoders are connected with the input end of a corresponding first attention-based convolutional neural network; wherein the number of the first attention mechanism-based convolutional neural networks is M-1;
the output ends of all the first encoders and the output ends of all the second encoders are connected with the input end of the second attention-based convolutional neural network; wherein the number of the second attention-based convolutional neural networks is one;
the output end of the second attention-based convolutional neural network and the output ends of all the first attention-based convolutional neural networks are connected with the multiplier;
the multiplier outputs a feature vector of news which is interesting to the user;
the output end of the multiplier and the output end of the second encoder are both connected with the input end of the splicing unit, so that the characteristic vector of the news which is interesting to the user is spliced with the characteristic vector of the candidate news;
the output end of the splicing unit is connected with the input end of the multilayer sensor, and the output end of the multilayer sensor is used for outputting recommended candidate news.
Further, the internal structure of the first encoder and the second encoder is the same.
Further, as shown in fig. 3, the first encoder includes: the system comprises an embedding layer, a graph attention network layer, a convolutional neural network layer and a maximum pooling layer which are connected in sequence.
Further, the internal structure of the first attention-based convolutional neural network is the same as that of the second attention-based convolutional neural network.
Further, as shown in fig. 2, the network structure of the first attention-based convolutional neural network includes, connected in sequence: the system comprises an attention distribution calculation layer, an interest weight acquisition layer, a basic weight acquisition layer, a final weight acquisition layer and an aggregation characteristic acquisition layer.
Further, the operation principle of the first attention-based convolutional neural network is as follows:
the attention distribution calculation layer is used for converting a user historical news click sequence into a low-dimensional real number vector through the Embedding layer so as to be used as an attention signal, coding a user behavior sequence and calculating attention distribution;
the interest weight acquisition layer is used for calculating weighted average of attention and converting the weighted average into interest weight through an attention mechanism according to the similarity, so that the interest of the user is deeply mined;
the basic weight acquisition layer is used for carrying out attention mechanism operation on the user behavior sequence again according to the vector of the object to be recommended to obtain basic weight;
the final weight acquisition layer is used for carrying out corresponding averaging on the interest weight and the basic weight obtained through the attention mechanism so as to obtain the final weight of each article interacted by the user;
and the aggregation characteristic acquisition layer is used for performing weighted fusion on the input sequence and taking the obtained vector as the aggregation characteristic of the input sequence.
Further, constructing a plurality of user interest representations based on historical click data of a user on a plurality of news links; the method specifically comprises the following steps:
(11): extracting an entity from a history clicked news title by adopting an existing knowledge map to obtain the entity, a connecting edge between the entities and an association rule between the entities so as to obtain a relationship construction subgraph;
(12): adopting a graph attention network to perform relevance search on each entity of the relation construction subgraph, calculating the weight of a connecting edge between the entities, and selecting the entity with the weight higher than a set threshold value;
(13): and performing feature representation on the selected entity by adopting a convolutional neural network to obtain user interest representation of the currently browsed news title.
Further, the step (11) of extracting an entity from a history clicked news title by using an existing knowledge graph to obtain the entity, the connecting edges between the entities and the association rules between the entities to further obtain a relationship building sub-graph specifically includes:
(111) identifying entities from news headlines, and obtaining a news entity set E after processing all news headlinesnews；
(112) From news entity set EnewsExtracting a sub-graph from the existing knowledge graph: for all the relations in the existing knowledge graph, the set E is combined withnewsSetting the threshold value of the word frequency of the entity same as the middle entity as n times, when the word frequency of the two entities exceeds n times, constructing the relation between the two entities, and eliminating any e not in connection in the subgraphi∈EnewsNodes and corresponding edges on the path; n is a positive integer, e.g., 10;
(113) and mapping the entities and the relations in the knowledge graph subgraph to a low-dimensional vector space by adopting a knowledge representation learning method (TransE) to obtain the characteristic vectors of the entities and the relations for subsequent use.
Historical click news of user i as
Wherein the content of the first and second substances,
is the title of the jth news that user i clicked on, NiIs the total number of news clicked by user i;
for each news headline, it is composed of a series of words, i.e., t ═ w1,w2,…]Wherein each word may be associated with an entity e in the knowledge-graph;
the history click news record of a certain user i is
The corresponding news characteristic is expressed as
Wherein
Is the title of the jth news that user i clicked on, NiIs the total number of news clicks by user i.
A knowledge-graph, represented as a set of entity-relationship-entity triples:
wherein ε and
representing a collection of entities and relationships, respectively, (h, r, t) representing the existence of a relationship r from h to t. The entity features are entity vectors in news headlines, each word may correspond to an entity vector in the knowledge-graph, and if there is no corresponding entity, the zero vector is used for filling.
It should be appreciated that in order to utilize knowledge-graph information in the model, the present invention constructs a knowledge-graph that is highly relevant to the task, using the knowledge-graph to retrieve entities from the data, as well as new entity links and new association rules.
Further, the (12): adopting a graph attention network to perform relevance search on each entity of the relation construction subgraph, calculating the weight of a connecting edge between the entities, and selecting the entities with the weight higher than a set threshold value; the method specifically comprises the following steps:
(121): learning embedded vectors for each entity and relationship using TransE
TransE performs embedding of entities/relationships on the knowledge graph and leaves the features unchanged.
Let
To represent the set of head entities h in the triples. An entity can be represented as:
wherein:
represents a vector concatenation, represents ehAnd etIs the entity vector learned from TransE. Pi (h, r, t) is an attention weight value for controlling the quantity of information which needs to be transmitted to the current entity by the neighbor node, and is obtained by calculation through a double-layer fully-connected neural network:
wherein: coefficients are normalized using the softmax function. Only the neighborhood of a single-hop graph is enabled by using the graph attention network, and the TransE embedded vector is used as a node characteristic, so that the graph attention network can be used for realizing the methodThe training parameter is { W0,W1,w2,b1,b2}。
(122): by enriching the entity representation, the related context information of the entity is enriched. Context embedding is calculated as the average of its context entities:
(123): the problem exists with entity vectors that belong to a different vector space than word vectors.
Firstly, a mapping function is introduced to map the entity vector sum to a vector space which is the same as the text characteristic, and an entity characteristic matrix is obtained.
g(e)＝tanh(Me+b)
Wherein:
is a matrix of transformations that can be trained,
are trainable parameters.
Then, the entity context feature matrix is mapped into a vector space which is the same as the text feature by adopting a mapping function which is the same as the entity feature, so that the entity context feature matrix is obtained.
Finally, three-channel input is constructed by the text feature matrix, the entity feature matrix and the entity context feature matrix, and a news feature vector e (t) is extracted through CNN, wherein,
is the output of the ith convolution kernel, and m is the number of convolution kernels:
exemplarily, the graph attention network is adopted to perform relevance search on each extracted entity, calculate the weight of a connecting edge between the entities, and select the entity with the weight higher than a set threshold; based on the concept of the attention network GAT, relevance search is carried out on each entity appearing in news, the weight of the relevant edges, namely the importance degree of the relation, is calculated in a set step number, mainly reachable paths among the news entities are found, reachable paths among the news entities are selected and found, and the entities with high importance degree are selected for embedding. And richer news characteristic representation is obtained, and the real-time interest of the user is better estimated, so that the accuracy of estimation of the Click Through rate Click-Through-rate (CTR) is improved.
Further, the different weights are used for aggregating all the user interest representations to obtain an aggregation result; the method specifically comprises the following steps:
and adopting a first attention-based convolutional neural network to aggregate all the user interest representations by using different weights to obtain an aggregation result.
Further, the different weights are used for aggregating all the user interest representations to obtain an aggregation result; the method specifically comprises the following steps:
and carrying out weighted summation on all the user interest representations by using different set weights to obtain an aggregation result.
Illustratively, the aggregating all the user interest representations by using different weights to obtain an aggregation result; the method specifically comprises the following steps:
for the current candidate news tjUsing a convolutional neural network mechanism based on an attention mechanism to calculate the current interest feature representation of the user:
e(tj)＝tanh(Wte(tj)+bt) Formula (8)
Wherein: sjIs used for measuring historical click news
For candidate news e (t)j) M 1, …, Ni. In addition, { Ww,Wt,bw,btIs a trainable parameter, Ww,Wt∈Rd×d，bw，bt，v,
d is the dimension of news embedding. Convolutional neural network based on attention mechanism
The embedding of two news headlines is received as input and output impact weights.
The dynamic interest characteristics e (i) of user i are calculated as follows:
further, all the user interest representations are matched with each candidate news data to obtain a matching result; the method specifically comprises the following steps:
and matching all the user interest representations with each candidate news data by adopting a second attention-based convolutional neural network to obtain a matching result.
Further, obtaining a feature vector of the news which is interested by the user based on the matching result and the aggregation result; the method specifically comprises the following steps:
and multiplying the matching result and the aggregation result to obtain the feature vector of the news which is interested by the user.
It should be appreciated that for news headlines browsed by the User, candidate news features are considered, candidate news are matched with each click of news using the attention-based convolutional neural network ann (attention And DNN for User Interest extraction), And feature representations of the current interests of the User are aggregated. The invention considers that news clicked by a user has different influences on candidate news. A plurality of attention-based convolutional neural networks are used in the model to automatically match the news of each click with candidate news and aggregate the current interests of the user with different weights. The ANN is used as a user interest extractor to acquire the current interest of the user in the candidate news according to the historical news clicked by the user.
Further, splicing the feature vector of the news which is interested by the user and the feature vector of the candidate news; the method specifically comprises the following steps:
and serially splicing the feature vector of the news which is interested by the user and the feature vector of the candidate news.
Further, predicting the splicing result to obtain recommended candidate news; the method specifically comprises the following steps:
inputting the splicing result into a multilayer perceptron to obtain a recommended probability value of recommended candidate news;
and sorting the candidate news according to the sequence of the recommendation probability values from large to small, and recommending the news with the top sorting to the user.
It should be understood that the spliced feature vector is used as the input of the MLP network, and the high-order feature extraction and the final result prediction are performed through the MLP network, so that the advantages are complemented, and the accurate prediction of the CTR of the article is obtained.
Illustratively, the splicing result is predicted to obtain recommended candidate news; the method specifically comprises the following steps:
predicting whether user i will click on candidate news headlines tjAnd considering the dynamic interest features of the user and the candidate news features, and using a second attention-based convolutional neural network
To predict the probability of the user clicking on the candidate news, the formula is as follows:
wherein the content of the first and second substances,
represents the network output after softmax layer, represents the sample tjA predicted probability of being clicked; the optimization algorithm selects an Adam algorithm, the optimizer selects an Adam optimizer, the AUC value is used for judgment, the value range of the AUC is between 0.5 and 1, the closer the AUC value is to 1, the higher the prediction authenticity is.
The method includes the steps that a knowledge graph and a graph attention network are adopted to conduct relevance search on text features in news titles, extra relevant entities and context representation are obtained to enhance text feature representation, and text, entity and context information are integrated through CNN; constructing a user feature representation: dynamically gathering user interest representations by adopting a plurality of convolutional neural networks based on an attention mechanism, enhancing the effect of extracting the user interest, and further acquiring the characteristic representation of the user; obtaining an accurate prediction result of CTR: and splicing the user characteristic representation and the candidate news characteristic representation to be used as the input of the MLP network, and performing high-order characteristic extraction and final result prediction through the MLP network to obtain the accurate prediction of the candidate news CTR.
In conclusion, the news recommending method based on knowledge enhancement and attention mechanism can recommend news which better meets the expectation or the demand of the user, and has higher prediction accuracy compared with the existing news recommending method.
Example two
The embodiment provides a news recommendation system based on knowledge enhancement and attention mechanism;
news recommendation system based on knowledge enhancement and attention mechanism, comprising:
an acquisition module configured to: acquiring historical click data of a user on a news link; acquiring candidate news data;
an acquisition module configured to: inputting both the historical click data and the candidate news data into a trained news recommendation model to obtain news data which is personalized and recommended for a user;
the news recommendation model constructs a plurality of user interest representations based on historical click data of a user on a plurality of news links; matching all the user interest representations with each candidate news data to obtain matching results; aggregating all the user interest representations by using different weights to obtain an aggregation result; obtaining a feature vector of the news which is interested by the user based on the matching result and the aggregation result; splicing the feature vector of the news which is interested by the user and the feature vector of the candidate news, and predicting the splicing result to obtain recommended candidate news; and recommending the result to the user.
It should be noted here that the above-mentioned obtaining module and obtaining module correspond to steps S101 and S102 in the first embodiment, and the above-mentioned modules are the same as the examples and application scenarios implemented by the corresponding steps, but are not limited to the disclosure of the first embodiment. It should be noted that the modules described above as part of a system may be implemented in a computer system such as a set of computer-executable instructions.
In the foregoing embodiments, the descriptions of the embodiments have different emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
The proposed system can be implemented in other ways. For example, the above-described system embodiments are merely illustrative, and for example, the division of the above-described modules is merely a logical functional division, and in actual implementation, there may be another division, for example, a plurality of modules may be combined or may be integrated into another system, or some features may be omitted, or not executed.
EXAMPLE III
The present embodiment also provides an electronic device, including: one or more processors, one or more memories, and one or more computer programs; wherein, a processor is connected with the memory, the one or more computer programs are stored in the memory, and when the electronic device runs, the processor executes the one or more computer programs stored in the memory, so as to make the electronic device execute the method according to the first embodiment.
It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general purpose processor, a digital signal processor DSP, an application specific integrated circuit ASIC, an off-the-shelf programmable gate array FPGA or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory may include both read-only memory and random access memory and may provide instructions and data to the processor, and a portion of the memory may also include non-volatile random access memory. For example, the memory may also store device type information.
In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software.
The method in the first embodiment may be directly implemented by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor. The software modules may be located in ram, flash, rom, prom, or eprom, registers, among other storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor. To avoid repetition, it is not described in detail here.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
Example four
The present embodiments also provide a computer-readable storage medium for storing computer instructions, which when executed by a processor, perform the method of the first embodiment.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. The news recommendation method based on knowledge enhancement and attention mechanism is characterized by comprising the following steps:
acquiring historical click data of a user on a news link; acquiring candidate news data;
inputting both historical click data and candidate news data into the trained news recommendation model to obtain news data which is recommended for a user in an individualized manner;
the news recommendation model constructs a plurality of user interest representations based on historical click data of a user on a plurality of news links; matching all the user interest representations with each candidate news data to obtain matching results; aggregating all the user interest representations by using different weights to obtain an aggregation result; obtaining a feature vector of the news which is interested by the user based on the matching result and the aggregation result; splicing the feature vector of the news which is interested by the user and the feature vector of the candidate news, and predicting the splicing result to obtain recommended candidate news; and recommending the result to the user.
2. The news recommendation method based on knowledge enhancement and attention mechanism as claimed in claim 1, wherein said news recommendation model comprises: m first encoders and a second encoder; wherein M is a positive integer;
the first encoder is used for encoding news titles browsed by the user to obtain user interest representations;
the second encoder is used for encoding the candidate news headlines to obtain the feature vectors of the candidate news;
the output ends of two adjacent first encoders are connected with the input end of a corresponding first attention-based convolutional neural network; wherein the number of the first attention mechanism-based convolutional neural networks is M-1;
the output ends of all the first encoders and the output ends of all the second encoders are connected with the input end of the second attention-based convolutional neural network; wherein the number of the second attention mechanism-based convolutional neural networks is one;
the output end of the second attention mechanism-based convolutional neural network and the output ends of all the first attention mechanism-based convolutional neural networks are connected with the multiplier;
the multiplier outputs a feature vector of news which is interesting to the user;
the output end of the multiplier and the output end of the second encoder are both connected with the input end of the splicing unit, so that the characteristic vector of the news which is interesting to the user is spliced with the characteristic vector of the candidate news;
the output end of the splicing unit is connected with the input end of the multilayer sensor, and the output end of the multilayer sensor is used for outputting recommended candidate news.
3. A news recommendation method based on knowledge enhancement and attention mechanism as claimed in claim 2, wherein said first encoder comprises: the system comprises an embedding layer, a graph attention network layer, a convolutional neural network layer and a maximum pooling layer which are connected in sequence.
4. The news recommendation method based on knowledge enhancement and attention mechanism as claimed in claim 1, wherein said building a plurality of user interest representations based on historical click data of a user on a plurality of news links; the method specifically comprises the following steps:
extracting an entity from a history clicked news title by adopting an existing knowledge map to obtain the entity, a connecting edge between the entities and an association rule between the entities so as to obtain a relationship construction subgraph;
adopting a graph attention network to perform relevance search on each entity of the relation construction subgraph, calculating the weight of a connecting edge between the entities, and selecting the entity with the weight higher than a set threshold value;
and performing feature representation on the selected entity by adopting a convolutional neural network to obtain user interest representation of the currently browsed news title.
5. The news recommendation method based on knowledge enhancement and attention mechanism as claimed in claim 4, wherein said using the existing knowledge map to extract an entity from a history clicked news title, obtaining the entity, the connecting edges between the entities and the association rules between the entities, and further obtaining the relationship construction subgraph specifically comprises:
(111) identifying entities from news headlines, and obtaining a news entity set E after processing all news headlinesnews；
(112) From news entity set EnewsExtracting a sub-graph from the existing knowledge graph: for all the relations in the existing knowledge graph, the set E is combined withnewsSetting the threshold value of the word frequency of the entity same as the middle entity as n times, when the word frequency of the two entities exceeds n times, constructing the relation between the two entities, and eliminating any e not in connection in the subgraphi∈EnewsNodes and corresponding edges on the path; n is a positive integer, e.g., 10;
(113) and mapping the entities and the relations in the knowledge map subgraph to a low-dimensional vector space by adopting a knowledge representation learning method TransE to obtain the characteristic vectors of the entities and the relations for subsequent use.
6. The news recommendation method based on knowledge enhancement and attention mechanism as claimed in claim 4, wherein a graph attention network is adopted to perform relevance search on each entity of the relation construction subgraph, the weight of the connection edge between the entities is calculated, and the entity with the weight higher than a set threshold is selected; the method specifically comprises the following steps:
(121): learning embedded vectors for each entity and relationship using TransE
Embedding entities/relations on a knowledge graph by TransE and keeping characteristics unchanged;
let
To represent the set of head entities h in the triples; one entity is represented as:
wherein:
represents a vector concatenation, represents ehAnd etIs an entity vector learned from TransE; pi (h, r, t) is an attention weight value for controlling the quantity of information which needs to be transmitted to the current entity by the neighbor node, and is obtained by calculation through a double-layer fully-connected neural network:
wherein the coefficients are normalized using a softmax function; only the neighborhood of a single-hop graph is enabled by using the graph attention network, a TransE embedded vector is used as a node feature, and trainable parameters are { W }0，W1，W2，b1，b2}；
(122): the related context information of the entity is enriched through enriching the entity representation; context embedding is calculated as the average of its context entities:
(123): the entity vector has the problem of belonging to a different vector space from the word vector;
firstly, introducing a mapping function to map the entity vector sum to a vector space with the same text characteristic to obtain an entity characteristic matrix;
g(e)＝tanh(Me+b)
wherein:
is a matrix of a transformation that can be trained,
is a trainable parameter;
then, mapping the entity context feature matrix into a vector space which is the same as the text feature by adopting a mapping function which is the same as the entity feature to obtain an entity context feature matrix;
finally, three-channel input is constructed by the text feature matrix, the entity feature matrix and the entity context feature matrix, and a news feature vector e (t) is extracted through CNN, wherein,
is the output of the ith convolution kernel, and m is the number of convolution kernels:
7. the news recommendation method based on knowledge enhancement and attention mechanism as claimed in claim 1, wherein said aggregating all user interest representations using different weights results in an aggregated result; the method specifically comprises the following steps:
adopting a first attention-based convolutional neural network, and aggregating all user interest representations by using different weights to obtain an aggregation result;
aggregating all the user interest representations by using different weights to obtain an aggregation result; the method specifically comprises the following steps:
carrying out weighted summation on all user interest representations by using different set weights to obtain an aggregation result;
matching all the user interest representations with each candidate news data to obtain a matching result; the method specifically comprises the following steps:
matching all the user interest representations with each candidate news data by adopting a second attention-based convolutional neural network to obtain a matching result;
obtaining a feature vector of the news which is interested by the user based on the matching result and the aggregation result; the method specifically comprises the following steps:
multiplying the matching result and the aggregation result to obtain a feature vector of the news which is interested by the user;
splicing the feature vector of the news which is interested by the user and the feature vector of the candidate news; the method specifically comprises the following steps:
carrying out series splicing on the feature vector of the news in which the user is interested and the feature vector of the candidate news;
predicting the splicing result to obtain recommended candidate news; the method specifically comprises the following steps:
inputting the splicing result into a multilayer perceptron to obtain a recommended probability value of recommended candidate news;
and sorting the candidate news according to the sequence of the recommendation probability values from large to small, and recommending the news with the top sorting to the user.
8. A news recommendation system based on knowledge enhancement and attention mechanism is characterized by comprising:
an acquisition module configured to: acquiring historical click data of a user on a news link; acquiring candidate news data;
a recommendation module configured to: inputting both the historical click data and the candidate news data into a trained news recommendation model to obtain news data which is personalized and recommended for a user;
the news recommendation model constructs a plurality of user interest representations based on historical click data of a user on a plurality of news links; matching all the user interest representations with each candidate news data to obtain matching results; aggregating all the user interest representations by using different weights to obtain an aggregation result; obtaining a feature vector of the news which is interested by the user based on the matching result and the aggregation result; splicing the feature vector of the news which is interested by the user and the feature vector of the candidate news, and predicting the splicing result to obtain recommended candidate news; and recommending the result to the user.
9. An electronic device, comprising:
a memory for non-transitory storage of computer readable instructions; and
a processor for executing the computer readable instructions,
wherein the computer readable instructions, when executed by the processor, perform the method of any of claims 1-7.
10. A storage medium storing, non-transitory, computer-readable instructions, wherein the non-transitory computer-readable instructions, when executed by a computer, perform the instructions of the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210421741.1A CN114741599A (en) | 2022-04-21 | 2022-04-21 | News recommendation method and system based on knowledge enhancement and attention mechanism |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210421741.1A CN114741599A (en) | 2022-04-21 | 2022-04-21 | News recommendation method and system based on knowledge enhancement and attention mechanism |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114741599A true CN114741599A (en) | 2022-07-12 |
Family
ID=82284712
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210421741.1A Pending CN114741599A (en) | 2022-04-21 | 2022-04-21 | News recommendation method and system based on knowledge enhancement and attention mechanism |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114741599A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115357785A (en) * | 2022-08-05 | 2022-11-18 | 山东省计算中心（国家超级计算济南中心） | Enterprise information recommendation method based on semantic interaction and local activation |
| CN117390290A (en) * | 2023-12-08 | 2024-01-12 | 安徽省立医院(中国科学技术大学附属第一医院) | Method for learning dynamic user interests based on language model of content enhancement |
-
2022
- 2022-04-21 CN CN202210421741.1A patent/CN114741599A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115357785A (en) * | 2022-08-05 | 2022-11-18 | 山东省计算中心（国家超级计算济南中心） | Enterprise information recommendation method based on semantic interaction and local activation |
| CN115357785B (en) * | 2022-08-05 | 2023-06-30 | 山东省计算中心（国家超级计算济南中心） | Enterprise information recommendation method based on semantic interaction and local activation |
| CN117390290A (en) * | 2023-12-08 | 2024-01-12 | 安徽省立医院(中国科学技术大学附属第一医院) | Method for learning dynamic user interests based on language model of content enhancement |
| CN117390290B (en) * | 2023-12-08 | 2024-03-15 | 安徽省立医院(中国科学技术大学附属第一医院) | Method for learning dynamic user interests based on language model of content enhancement |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111681059B (en) | Training method and device of behavior prediction model | |
| CN110837602B (en) | User recommendation method based on representation learning and multi-mode convolutional neural network | |
| CN111797321B (en) | Personalized knowledge recommendation method and system for different scenes | |
| CN108804633B (en) | Content recommendation method based on behavior semantic knowledge network | |
| CN110717098B (en) | Meta-path-based context-aware user modeling method and sequence recommendation method | |
| CN111061946B (en) | Method, device, electronic equipment and storage medium for recommending scenerized content | |
| CN109389151B (en) | Knowledge graph processing method and device based on semi-supervised embedded representation model | |
| JP6884116B2 (en) | Information processing equipment, information processing methods, and programs | |
| CN114741599A (en) | News recommendation method and system based on knowledge enhancement and attention mechanism | |
| CN110532471A (en) | Active Learning collaborative filtering method based on gating cycle unit neural network | |
| CN112990972A (en) | Recommendation method based on heterogeneous graph neural network | |
| CN112989169B (en) | Target object identification method, information recommendation method, device, equipment and medium | |
| CN114780831A (en) | Sequence recommendation method and system based on Transformer | |
| CN114265986B (en) | Information pushing method and system fusing knowledge graph structure and path semantics | |
| CN115048586B (en) | Multi-feature-fused news recommendation method and system | |
| CN112765482A (en) | Product delivery method, device, equipment and computer readable medium | |
| CN117216281A (en) | Knowledge graph-based user interest diffusion recommendation method and system | |
| CN115374792A (en) | Policy text labeling method and system combining pre-training and graph neural network | |
| CN114925270A (en) | Session recommendation method and model | |
| CN113590976A (en) | Recommendation method of space self-adaptive graph convolution network | |
| CN112364245B (en) | Top-K movie recommendation method based on heterogeneous information network embedding | |
| CN112287689B (en) | Judicial second-examination case situation auxiliary analysis method and system | |
| CN110083766B (en) | Query recommendation method and device based on meta-path guiding embedding | |
| CN116361643A (en) | Model training method for realizing object recommendation, object recommendation method and related device | |
| CN115345687A (en) | Cross-website commodity alignment method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |