CN114519600A - Graph neural network CTR estimation algorithm fusing adjacent node variances - Google Patents
Graph neural network CTR estimation algorithm fusing adjacent node variances Download PDFInfo
- Publication number
- CN114519600A CN114519600A CN202111658407.XA CN202111658407A CN114519600A CN 114519600 A CN114519600 A CN 114519600A CN 202111658407 A CN202111658407 A CN 202111658407A CN 114519600 A CN114519600 A CN 114519600A
- Authority
- CN
- China
- Prior art keywords
- user
- article
- node
- representation
- neural network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0241—Advertisements
- G06Q30/0242—Determining effectiveness of advertisements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0241—Advertisements
- G06Q30/0277—Online advertisement
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Strategic Management (AREA)
- Finance (AREA)
- Development Economics (AREA)
- Accounting & Taxation (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Game Theory and Decision Science (AREA)
- Entrepreneurship & Innovation (AREA)
- Economics (AREA)
- Marketing (AREA)
- General Business, Economics & Management (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a graph neural network CTR estimation algorithm fusing adjacent node variances, which comprises the following steps: (1) acquiring click interaction behaviors of a user and an article, and context information such as click occurrence time, article display forms and the like; (2) generating a user-article interactive topological graph, taking a user and an article as nodes, taking the click of the user on the article as an edge, and taking other information as the characteristics of the nodes, and generating an interactive topological graph G (V, E); (3) training a neural network model of the graph; (4) and for the candidate user-item pairs, carrying out forward propagation by using the model to obtain a predicted value of whether the user clicks the item. The method can take the difference of the neighbor nodes into consideration during modeling, obtain better user and article representations, further improve the performance of a recommendation system and improve the accuracy of click rate prediction.
Description
Technical Field
The invention relates to the field of graph neural networks and internet recommendation systems, in particular to a graph neural network CTR estimation algorithm fusing adjacent node variances.
Background
The Internet recommendation system is widely applied to the fields of e-commerce, advertisement, video and the like, and can judge the preference degree of a user on an article through the user characteristics, the article characteristics and the interaction between the user and the article, so that the most appropriate article is recommended to the user, and the article click rate and the user satisfaction degree are improved.
In recent years, recommendation systems based on graph neural networks have been rapidly developed. Compared with the traditional deep learning model, the graph neural network can explore more complex and higher-order interaction relations in a graph consisting of users and articles, and is beneficial to improving the performance of a recommendation system. Graph neural networks typically use a message propagation mechanism, i.e., aggregating the characterizations of neighboring nodes to update the characterization of the central node. Conventional aggregation methods have a maximum value, (weighted) averaging, etc., and by default, these methods ignore a problem specific to graph structure data, i.e., information implicit in the size of the differences between multiple neighbors. Aiming at the problem, the patent provides a graph neural network fusing adjacent node variances to predict the user-item click rate.
Such a situation may arise in graph structure data: two nodes have similar characteristics, the mean of their neighbors is similar, but the difference of their neighboring nodes may be different. For example, the average of the registration time lengths of the customers of the two merchants is almost the same, but the registration time length of the customer of one of the merchants is very small, so that the merchant may have a false trade arbitrage situation. The conventional graph neural network aggregation method only considers the characteristics of the neighbor nodes, but does not consider the difference between the neighbor nodes, which is also a characteristic of the central node, and neglects the problem which does not exist in the traditional structured data and the picture data.
In conclusion, the invention designs a CTR prediction algorithm of a graph neural network fusing adjacent node variances.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide a graph neural network CTR prediction algorithm fusing the variance of adjacent nodes, which can take the difference of the adjacent nodes into consideration during modeling to obtain better user and article characteristics, thereby improving the performance of a recommendation system and improving the accuracy of click rate prediction.
In order to achieve the purpose, the invention is realized by the following technical scheme: a graph neural network CTR estimation algorithm fusing adjacent node variances comprises the following steps:
(1) acquiring click interaction behaviors of a user and an article, and context information such as click occurrence time, article display forms and the like;
(2) generating a user-article interactive topological graph, taking a user and an article as nodes, taking the click of the user on the article as an edge, and taking other information as the characteristics of the nodes, and generating an interactive topological graph G (V, E);
(3) training a neural network model of the graph;
(a) establishing an L-layer graph neural network, splicing the mean value and the variance of the neighbor node representation in each layer into aggregate information for updating the representation of the central node;
(b) after the characteristics of the user and the article are obtained through an L-layer graph neural network, the user-article pair with connection is marked as a positive sample, the user-article pair without connection is marked as a negative sample, the value of the inner product of the user characteristics and the article characteristics through a logistic function is used as a predicted value of whether the user-article has connection, and a loss function is input for model training;
(4) and for the candidate user-item pairs, carrying out forward propagation by using the model to obtain a predicted value of whether the user clicks the item.
The step (3) utilizes python to process the user-article interaction topological graph to train the graph neural network, and specifically comprises the following steps:
(a) the graph neural network is composed of L-layer feature transformation, and the computation method of the t-th layer is as follows: note the book
For the characterization of node v at level t-1,
and the characterization of all the neighbor nodes of the node v at the t-1 layer.
(b) And calculating the variance information of the neighbors of the node v, calculating the variance of the neighbor tokens of the node v in each token dimension, and if the node v only has one neighbor, defining the variance information of the neighbor as a 0 vector. The mathematical representation is:
wherein
To represent
The ith component of (a).
(c) And calculating the aggregation information of the neighbors of the node v, and splicing the representation mean value (the calculation method is the same as the variance information) and the representation variance of the neighbors of the node v together to form an aggregation information vector. The mathematical expression is:
(d) and inputting the aggregation information vector of the neighbor of the node v and the representation of the node v into a full-connection layer for transformation, and then activating through a sigmoid activation function to obtain a new representation of the node v on the layer. The mathematical representation is:
(e) repeating a) -d) to obtain the representation of each node in the topological graph after the L layer. And (3) recording the user-item pairs with connection as positive samples and the user-item pairs without connection as negative samples, and taking the value of the inner product of the user characterization and the item characterization through a logistic function as a predicted value of whether the user-item has connection, wherein the mathematical expression is as follows:
inputting the loss function into the loss function, optimizing the loss function to carry out model training, and storing the trained model parameters.
The invention has the following beneficial effects:
compared with the traditional structured data and the picture data, the picture structure data has the characteristic that neither of the two data can be found, namely, the difference of the neighbor nodes is also a characteristic of the central node. Conventional graph neural networks ignore this because the overall statistical information for all neighbors cannot be extracted, whether by average pooling, maximum pooling, or attention mechanism, resulting in inefficient use of the information.
The invention recognizes the problem for the first time, innovatively provides a new neighbor information aggregation method, brings the difference of neighbor nodes into the processing range of the algorithm, can more fully extract the information of the user-article interaction graph, enables the user characterization and the article characterization to contain more accurate and rich information, and improves the accuracy of click rate prediction.
Drawings
The invention is described in detail below with reference to the drawings and the detailed description;
FIG. 1 is an overall flow chart of the present invention;
FIG. 2 is an interactive topological graph of the present invention (light color is user nodes, dark color is article nodes);
FIG. 3 is a schematic diagram of a neural network of the present invention.
Detailed Description
In order to make the technical means, the creation characteristics, the achievement purposes and the effects of the invention easy to understand, the invention is further described with the specific embodiments.
Referring to fig. 1 to 3, the following technical solutions are adopted in the present embodiment: a graph neural network CTR estimation algorithm fusing adjacent node variances comprises the following steps:
1. based on the service system log, extracting click interaction behaviors of the user and the article, and collecting context information when clicking, such as click occurrence time, article display form and the like.
2. Importing the collected information into a graph database to generate a user-article interaction topological graph: with the user and the article as nodes, the click of the user on the article as an edge, and other information as the characteristics of the nodes, an interaction topology graph G ═ V, E is generated, where V denotes the user node and the article node, and E denotes the edge, as shown in fig. 2.
3. Graph neural network training is performed using python to process the user-item interaction topology, as shown in fig. 3.
a) The graph neural network is composed of L-layer feature transformation, and the computation method of the t-th layer is as follows: note the book
For the characterization of node v at level t-1,
and the characterization of all the neighbor nodes of the node v at the t-1 layer.
b) And calculating the variance information of the neighbors of the node v, calculating the variance of the neighbor tokens of the node v in each token dimension, and if the node v only has one neighbor, defining the variance information of the neighbor as a 0 vector. The mathematical expression is:
wherein
To represent
The ith component of (2).
c) And calculating the aggregation information of the neighbors of the node v, and splicing the representation mean value (the calculation method is the same as the variance information) and the representation variance of the neighbors of the node v together to form an aggregation information vector. The mathematical representation is:
d) and inputting the aggregation information vector of the neighbor of the node v and the representation of the node v into a full-connection layer for transformation, and then activating through a sigmoid activation function to obtain a new representation of the node v on the layer. The mathematical representation is:
e) repeating a) -d) to obtain the representation of each node in the topological graph after the L layer. And (3) recording the user-item pairs with connection as positive samples and the user-item pairs without connection as negative samples, and taking the value of the inner product of the user characterization and the item characterization through a logistic function as a predicted value of whether the user-item has connection, wherein the mathematical expression is as follows:
inputting the loss function into the loss function, optimizing the loss function to carry out model training, and storing the trained model parameters.
4. And (5) model service. And for the candidate user-item pairs, carrying out forward propagation by using the model to obtain a predicted value of whether the user clicks the item, namely a CTR predicted value.
Example 1: a graph neural network CTR estimation algorithm fusing adjacent node variances comprises the following steps:
1. based on the service system log, extracting click interaction behaviors of the user and the article, and collecting context information when clicking, such as click occurrence time, article display form and the like.
a) According to the actual scene needs, the click interaction behavior recorded in the service system log includes, but is not limited to, a user clicking to enter a commodity page, browsing a commodity advertisement for a certain time, and the like.
b) Context information such as the user's age, gender, registration duration, etc., the categories of goods, price, advertisement text, page tones, etc. is collected.
c) For the collected type data, the collected type data can be converted into numerical data through OneHot coding or Embedding; for the collected text type data, the text type data can be converted into numerical data through Word2 Vec; and splicing the obtained numerical data together to serve as the initial characteristic vectors of the user and the commodity.
2. Importing the collected information into a graph database to generate a user-article interaction topological graph: with the user and the article as nodes, the click of the user on the article as an edge, and other information as the characteristics of the nodes, an interaction topology graph G ═ V, E is generated, where V denotes the user node and the article node, and E denotes the edge, as shown in fig. 2.
3. Graph neural network training is performed using python to process the user-item interaction topology, as shown in fig. 3.
d) The features of the user and the commodity are preprocessed through an MLP (multi-layer perceptron) to make the feature dimensions the same, and the feature dimensions can be 64 dimensions or 128 dimensions.
e) The neural network module of the graph generally comprises 2 to 3 layers of feature transformation, and the computation method of the t-th layer is as follows: note the book
For the characterization of node v at level t-1,
and the characterization of all the neighbor nodes of the node v at the t-1 layer.
f) And calculating the variance information of the neighbors of the node v, calculating the variance of the neighbor tokens of the node v in each token dimension, and if the node v only has one neighbor, defining the variance information of the neighbor as a 0 vector. The mathematical representation is:
wherein
To represent
The ith component of (a).
g) And calculating the aggregation information of the neighbors of the node v, and splicing the representation mean value (the calculation method is the same as the variance information) and the representation variance of the neighbors of the node v together to form an aggregation information vector. The mathematical representation is:
h) and inputting the aggregation information vector of the neighbor of the node v and the representation of the node v into a full-connection layer for transformation, and then activating through a sigmoid activation function to obtain a new representation of the node v on the layer. The mathematical representation is:
i) and repeating the steps from b) to e) to obtain the representation of each node in the topological graph after the layers from 2 to 3 are repeated. User-item pairs with a connection are denoted as positive examples and user-item pairs without a connection are denoted as negative examples. Since the probability of the user clicking on the item is much smaller than the probability of not clicking, the number of negative samples is much larger than that of the positive samples, and only a part of the negative samples (for example, the total number is not more than 20 times of the positive samples) are selected to be calculated together with the positive samples in the next step through downsampling.
j) And performing inner product calculation on the user characterization and the article characterization of the positive and negative sample pairs, inputting the inner product into a logistic function, and outputting the output as a predicted value of whether the user-article is connected or not, wherein the mathematical expression is as follows:
inputting the model parameters into a loss function, training the model parameters by using a gradient descent optimization loss function, and storing the trained model. The loss function is generally chosen as a cross-entropy loss function, mathematically represented as:
4. and (5) model service. For a user-article pair needing prediction, an initial feature vector is obtained by using the same information conversion method during model training, the user-article pair is added into an interactive topological graph used during training, forward propagation is carried out by using a trained model, and a prediction value of whether a user clicks an article or not, namely a CTR prediction value is obtained. A user can also correspond to a plurality of candidate items, the click rate of each candidate item is respectively predicted, and the items with the click rate top10 are returned to the user.
The variance information of the neighbor node characteristics is incorporated into the algorithm, the information in the user-article interaction topological graph is more fully extracted, and the accuracy of CTR estimation is effectively improved. The embodiment provides a new neighbor node information aggregation method, which can enable the representations of users and article nodes to learn the variance information of neighbors, thereby more fully representing the characteristics of a central node and improving the prediction performance of an algorithm.
The foregoing shows and describes the general principles and broad features of the present invention and advantages thereof. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are described in the specification and illustrated only to illustrate the principle of the present invention, but that various changes and modifications may be made therein without departing from the spirit and scope of the present invention, which fall within the scope of the invention as claimed. The scope of the invention is defined by the appended claims and equivalents thereof.
Claims (2)
1. A CTR (neural network) estimation algorithm for a graph fusing adjacent node variances is characterized by comprising the following steps of:
(1) acquiring click interaction behaviors of a user and an article, and context information such as click occurrence time, article display forms and the like;
(2) generating a user-article interactive topological graph, taking a user and an article as nodes, taking the click of the user on the article as an edge, and taking other information as the characteristics of the nodes, and generating an interactive topological graph G (V, E);
(3) training a neural network model of the graph;
(a) establishing an L-layer graph neural network, splicing the mean value and the variance of the neighbor node representation in each layer into aggregate information for updating the representation of the central node;
(b) after the characteristics of the user and the article are obtained through an L-layer graph neural network, the user-article pair with connection is marked as a positive sample, the user-article pair without connection is marked as a negative sample, the value of the inner product of the user characteristics and the article characteristics through a logistic function is used as a predicted value of whether the user-article has connection, and a loss function is input for model training;
(4) and for the candidate user-item pairs, carrying out forward propagation by using the model to obtain a predicted value of whether the user clicks the item.
2. The prediction algorithm of the neural network CTR fused with the variance of adjacent nodes as claimed in claim 1, wherein the step (3) of processing the user-item interaction topological graph by python to train the neural network comprises:
(a) the graph neural network is composed of L-layer feature transformation, and the computation method of the t-th layer is as follows: note the book
For the characterization of node v at level t-1,
representing all neighbor nodes of the node v at a t-1 layer;
(b) calculating the variance information of the neighbors of the node v, calculating the variance of the neighbor representation of the node v in each representation dimension, and if the node v only has one neighbor, defining the variance information of the neighbor as a 0 vector; the mathematical representation is:
wherein
To represent
The ith component of (a);
(c) calculating the aggregation information of the neighbors of the node v, and splicing the representation mean value and the representation variance of the neighbors of the node v together to form an aggregation information vector; the mathematical representation is:
(d) inputting the aggregation information vector of the neighbor of the node v and the representation of the node v into a full-connection layer for transformation, and then activating through a sigmoid activation function to obtain a new representation of the node v on the layer; the mathematical representation is:
(e) repeating a) -d) to obtain the representation of each node in the topological graph after the L layer; and (3) recording the user-item pairs with connection as positive samples, recording the user-item pairs without connection as negative samples, and taking the value of the inner product of the user characterization and the item characterization through a logistic function as a predicted value of whether the user-item has connection or not, wherein the mathematical expression is as follows:
inputting the loss function into the loss function, optimizing the loss function to carry out model training, and storing the trained model parameters.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111658407.XA CN114519600A (en) | 2021-12-30 | 2021-12-30 | Graph neural network CTR estimation algorithm fusing adjacent node variances |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111658407.XA CN114519600A (en) | 2021-12-30 | 2021-12-30 | Graph neural network CTR estimation algorithm fusing adjacent node variances |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114519600A true CN114519600A (en) | 2022-05-20 |
Family
ID=81597407
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111658407.XA Pending CN114519600A (en) | 2021-12-30 | 2021-12-30 | Graph neural network CTR estimation algorithm fusing adjacent node variances |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114519600A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114817751A (en) * | 2022-06-24 | 2022-07-29 | 腾讯科技（深圳）有限公司 | Data processing method, data processing device, electronic equipment, storage medium and program product |
-
2021
- 2021-12-30 CN CN202111658407.XA patent/CN114519600A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114817751A (en) * | 2022-06-24 | 2022-07-29 | 腾讯科技（深圳）有限公司 | Data processing method, data processing device, electronic equipment, storage medium and program product |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11995702B2 (en) | Item recommendations using convolutions on weighted graphs | |
| CN108763362B (en) | Local model weighted fusion Top-N movie recommendation method based on random anchor point pair selection | |
| Pan et al. | Study on convolutional neural network and its application in data mining and sales forecasting for E-commerce | |
| CN111080400B (en) | Commodity recommendation method and system based on gate control graph convolution network and storage medium | |
| CN112364976B (en) | User preference prediction method based on session recommendation system | |
| CN111932336A (en) | Commodity list recommendation method based on long-term and short-term interest preference | |
| CN115082147B (en) | Sequence recommendation method and device based on hypergraph neural network | |
| CN110796313B (en) | Session recommendation method based on weighted graph volume and item attraction model | |
| CN112950324B (en) | Knowledge graph assisted pairwise sorting personalized merchant recommendation method and system | |
| EP4242955A1 (en) | User profile-based object recommendation method and device | |
| CN114036406A (en) | Recommendation method and system based on graph contrast learning and social network enhancement | |
| CN112861006A (en) | Recommendation method and system fusing meta-path semantics | |
| CN113610610B (en) | Session recommendation method and system based on graph neural network and comment similarity | |
| CN113641811B (en) | Session recommendation method, system, equipment and storage medium for promoting purchasing behavior | |
| CN113763095B (en) | Information recommendation method and device and model training method and device | |
| CN114519600A (en) | Graph neural network CTR estimation algorithm fusing adjacent node variances | |
| CN116204723A (en) | Social recommendation method based on dynamic hypergraph representation learning | |
| CN114842247B (en) | Characteristic accumulation-based graph convolution network semi-supervised node classification method | |
| CN115222203A (en) | Risk identification method and device | |
| CN115564532A (en) | Training method and device of sequence recommendation model | |
| CN115238191A (en) | Object recommendation method and device | |
| CN115510318A (en) | Training method of user characterization model, user characterization method and device | |
| CN115618079A (en) | Session recommendation method, device, electronic equipment and storage medium | |
| CN115599990A (en) | Knowledge perception and deep reinforcement learning combined cross-domain recommendation method and system | |
| CN114092188A (en) | Recommendation system algorithm of lightweight B2B E-commerce platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication |