CN116402555A - Data alignment method, device, equipment and storage medium for advertisement recommendation - Google Patents
Data alignment method, device, equipment and storage medium for advertisement recommendation Download PDFInfo
- Publication number
- CN116402555A CN116402555A CN202211084782.2A CN202211084782A CN116402555A CN 116402555 A CN116402555 A CN 116402555A CN 202211084782 A CN202211084782 A CN 202211084782A CN 116402555 A CN116402555 A CN 116402555A
- Authority
- CN
- China
- Prior art keywords
- image
- information
- recognition model
- image information
- full
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 239000011159 matrix material Substances 0.000 claims abstract description 44
- 238000010586 diagram Methods 0.000 claims abstract description 33
- 238000011176 pooling Methods 0.000 claims description 42
- 230000006870 function Effects 0.000 claims description 24
- 238000004590 computer program Methods 0.000 claims description 15
- 238000013528 artificial neural network Methods 0.000 claims description 13
- 238000012549 training Methods 0.000 claims description 13
- 238000012545 processing Methods 0.000 claims description 11
- 238000006243 chemical reaction Methods 0.000 claims description 7
- 238000007781 pre-processing Methods 0.000 claims description 7
- 230000009466 transformation Effects 0.000 claims description 6
- 230000004931 aggregating effect Effects 0.000 claims description 5
- 230000008447 perception Effects 0.000 claims description 4
- 239000000463 material Substances 0.000 description 9
- 238000013527 convolutional neural network Methods 0.000 description 8
- 101100481876 Danio rerio pbk gene Proteins 0.000 description 5
- 101100481878 Mus musculus Pbk gene Proteins 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 230000007246 mechanism Effects 0.000 description 4
- 238000003058 natural language processing Methods 0.000 description 3
- 238000012512 characterization method Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000005096 rolling process Methods 0.000 description 2
- 241000251468 Actinopterygii Species 0.000 description 1
- 241000282376 Panthera tigris Species 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 230000008602 contraction Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000001902 propagating effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0241—Advertisements
- G06Q30/0251—Targeted advertisements
- G06Q30/0269—Targeted advertisements based on user profile or attribute
- G06Q30/0271—Personalized advertisement
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0241—Advertisements
- G06Q30/0277—Online advertisement
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/19—Recognition using electronic means
- G06V30/191—Design or setup of recognition systems or techniques; Extraction of features in feature space; Clustering techniques; Blind source separation
- G06V30/19147—Obtaining sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/40—Document-oriented image-based pattern recognition
- G06V30/41—Analysis of document content
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The application relates to the technical field of advertisement recommendation, and particularly discloses a data alignment method, device and equipment for advertisement recommendation and a storage medium. Wherein the method comprises the following steps: acquiring text information and image information of advertisements; acquiring an abstract semantic representation diagram based on the text information, converting side information of the abstract semantic representation diagram into a first adjacency matrix, and inputting the first adjacency matrix into a pre-trained image recognition model to obtain full-image information of a text sequence; inputting the image information into the image recognition model to acquire image-level features of the image information; and aligning the full-image information with the image-level features through the image recognition model, and outputting alignment data. According to the method and the device for the advertisement recommendation system, the whole image information and the image level features of the same advertisement can be aligned, and the aligned whole image information and the aligned image level features are applied to the CTR model, so that the advertisement recommended by the advertisement recommendation system is more accurate.
Description
Technical Field
The present disclosure relates to the field of advertisement recommendation technologies, and in particular, to a data alignment method, apparatus, device, and storage medium for advertisement recommendation.
Background
The commodity information existing in most of the e-commerce platforms generally has multi-mode characteristics of commodities, namely commodity characteristics mainly comprising commodity information, content and the like, and video characteristics mainly comprising images of commodity pictures, advertisement materials and the like. In a commonly used advertisement recommendation system, the feature aspect mainly uses the conventional advertisement feature. However, the embedding vector representation of the multimodal data such as the text information (e.g. the title) and the image information (e.g. the advertisement material picture) which are most intuitively affected by the advertisement Click is not aligned, so that the text information and the image information in the same advertisement are not fully applied to a CTR (Click-Through-Rate) model of the advertisement recommendation system, and therefore, the advertisement recommended by the advertisement recommendation system may not be sufficiently accurate.
Disclosure of Invention
The application provides a data alignment method, device, equipment and storage medium for advertisement recommendation, which are used for aligning full-image information and image-level characteristics of the same advertisement and applying the full-image information and the image-level characteristics to a CTR model, so that advertisements recommended by an advertisement recommendation system are more accurate.
In a first aspect, the present application provides a data alignment method for advertisement recommendation, the method comprising:
acquiring text information and image information of advertisements;
acquiring an abstract semantic representation diagram based on the text information, converting side information of the abstract semantic representation diagram into a first adjacency matrix, and inputting the first adjacency matrix into a pre-trained image recognition model to obtain full-image information of a text sequence;
inputting the image information into the image recognition model to acquire image-level features of the image information;
and aligning the full-image information with the image-level features through the image recognition model, and outputting alignment data.
In a second aspect, the present application further provides a data alignment device for advertisement recommendation, where the device includes:
the acquisition module is used for acquiring text information and image information of the advertisement;
the conversion module is used for acquiring an abstract semantic representation diagram based on the text information, converting side information of the abstract semantic representation diagram into a first adjacent matrix, and inputting the first adjacent matrix into a pre-trained image recognition model to obtain full-picture information of a text sequence;
the input module is used for inputting the image information into the image recognition model and acquiring image-level characteristics of the image information;
and the alignment module is used for aligning the full-image information and the image-level features through the image recognition model and outputting alignment data through the image recognition model.
In a third aspect, the present application also provides a computer device comprising a memory and a processor; the memory is used for storing a computer program; the processor is configured to execute the computer program and implement the data alignment method for advertisement recommendation as described above when the computer program is executed.
In a fourth aspect, the present application also provides a computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to implement a data alignment method for advertisement recommendation as described above.
The application discloses a data alignment method, a device, computer equipment and a storage medium for advertisement recommendation, wherein an abstract semantic representation diagram is obtained based on text information, side information of the abstract semantic representation diagram is converted into a first adjacent matrix, and the first adjacent matrix is input into a pre-trained image recognition model to obtain full-picture information of a text sequence; inputting the image information into the image recognition model to acquire image-level features of the image information; and the image recognition model is used for aligning the full-image information with the image-level features, and the alignment data can be applied to an advertisement recommendation system, so that advertisement recommendation is more accurate.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a data alignment system for advertisement recommendation provided in an embodiment of the present application;
FIG. 2 is a schematic flow chart of a data alignment method for advertisement recommendation provided in an embodiment of the present application;
FIG. 3 is a schematic block diagram of sub-steps of a data alignment method for advertisement recommendation provided in an embodiment of the present application;
FIG. 4 is a schematic block diagram of another data alignment apparatus for advertisement recommendation provided by an embodiment of the present application;
fig. 5 is a schematic block diagram of a computer device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
The flow diagrams depicted in the figures are merely illustrative and not necessarily all of the elements and operations/steps are included or performed in the order described. For example, some operations/steps may be further divided, combined, or partially combined, so that the order of actual execution may be changed according to actual situations.
It is to be understood that the terminology used in the description of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should also be understood that the term "and/or" as used in this specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
For ease of understanding of the embodiments of the present application, some related terms are briefly described below:
a graph is a data structure consisting of a set of vertices V and a set of relationships E (set of edges) between vertices, otherwise known as graph data. The graph is a one-to-many relationship in the data structure, and is generally divided into an undirected graph and an undirected graph.
Wherein, the directed graph refers to a pair of directional graphs, which consists of a group of vertexes and a group of directional edges, and each directional edge is connected with a pair of ordered vertexes. While an undirected graph can be understood as a graph with edges having no direction.
An Adjacency Matrix (Adjacency Matrix) may be a Matrix representing the Adjacency relationship between vertices.
The relative numbers given in the edges or arcs of the graph may be referred to as weights or weights. The weights may represent the distance from one vertex to another, cost, etc.
Homogeneous graph (Homogeneous graph), mainly refers to a graph with only one type of vertices and edges.
Heterogeneous graphs (Heterogeneous graph) mainly refer to graphs with vertex and edge types greater than or equal to two.
GNN (Graph Neural Network) network, i.e. the graph neural network. The graph neural network is a graph domain information processing method based on deep learning and is used for learning a joint sense model of graphs containing a large number of connections. GNNs capture the independence of the graph as information propagates between nodes of the graph.
Generally, in the prior art, text information of an advertisement may be obtained through natural language processing (Natural Language Processing, NLP). The conventional practice is as follows: the encoding is performed using a bi-directional encoder characterization (Bidirectional Encoder Representation from Transformers, BERT) class pre-training model based on a converter as an encoder. The problem of selecting the weight of the BERT class pre-training model is considered; then acquiring the full-image information of the text information represented by the vector; whether or not to perform the dimension reduction operation needs to be considered, and other characteristics may be affected.
In the prior art, the image-level characteristics of the material pictures in the advertisement can also be obtained through a certain strategy. Because one advertisement may correspond to a plurality of material pictures, the manner in which the plurality of material pictures in the same advertisement acquire the image-level features may be to select the ebedding of one material picture according to a certain policy, or to average the image-level features acquired by the plurality of material pictures in the same advertisement.
However, the two ways can not align the image level features of the full-image information and the image information of the text information in the advertisement, and therefore the advertisement recommended by the advertisement recommendation system with the CTR model can not be more accurate.
In view of this, embodiments of the present application provide a data alignment method, apparatus, computer device, and storage medium for advertisement recommendation. The data alignment method for advertisement recommendation can be applied to a server and is used for aligning the image-level features of the full-image information and the image information of the text information in the advertisement. Therefore, the aligned data is applied to the CTR model in the advertisement recommendation system, and the advertisement recommendation system can more accurately recommend advertisements to users. The server may be an independent server or a server cluster.
Some embodiments of the present application are described in detail below with reference to the accompanying drawings. The following embodiments and features of the embodiments may be combined with each other without conflict.
Referring to fig. 1, fig. 1 is a schematic diagram of a data alignment system for advertisement recommendation according to an embodiment of the present application. The system comprises a terminal and a server, wherein the terminal is in communication connection with the server, and advertisements can be recommended through the server and displayed in the terminal.
The terminal comprises electronic equipment such as a mobile phone, a tablet personal computer, a notebook computer, a desktop computer, a personal digital assistant, wearable equipment and the like.
Wherein the server comprises a single server or a cluster of servers.
FIG. 2 is a schematic flow chart of a data alignment method for advertisement recommendation according to an embodiment of the present application; the data alignment method for advertisement recommendation can be applied to a server. The advertisement recommendation system specifically includes steps S101 to S104.
Step S101, acquiring text information and image information of advertisements.
The advertisement may be an advertisement that the user has viewed or operated, such as a clicked advertisement. Generally, the same advertisement comprises text information such as titles, numbers and the like, and the text information is text information; the same advertisement also comprises a plurality of material pictures, and the plurality of material pictures are image information.
In some platforms, such as some host platforms, live platforms such as fighting fish, tiger teeth, etc., and most e-commerce platforms, some advertisements are recommended to the user. However, in most advertising recommendation systems employed by platforms, only conventional advertising features are often used when recommending advertisements to users. Traditional advertising features include, but are not limited to: the type of advertisement that the user has clicked on and/or the content information, the user information, the frequency with which the user clicks on the advertisement, the timing, etc.
In order to enable the advertisement recommendation system to accurately recommend advertisements to users, a general user prepares a large number of conventional advertisement feature samples in advance, and inputs the conventional advertisement feature samples into a CTR model in the advertisement recommendation system to train the CTR model in the advertisement recommendation system. In the actual application process, the advertisement recommendation system recommends advertisements to users through a CTR model.
However, the factors influencing the user to click the advertisement are the most intuitive text information, image information and other multi-mode data; these multimodal data are not applied to the CTR model.
Therefore, in the embodiment of the application, the multi-mode data such as the text information and the image information of the advertisement can be obtained, so that a richer training sample is provided for the follow-up training CTR model, and the advertisement recommendation system can recommend the advertisement more accurately.
In one embodiment of the application, the user can manually mark the text information in the same advertisement, so that the text information can be stripped.
In one embodiment of the present application, text information and image information may also be extracted from the same advertisement by some video editing software.
Of course, text information and image information may be obtained from the same advertisement in some other way, and embodiments of the present application will not be described in detail.
Step S102, an abstract semantic representation diagram is obtained based on the text information, side information of the abstract semantic representation diagram is converted into a first adjacent matrix, and the first adjacent matrix is input into a pre-trained image recognition model to obtain full-view information of a text sequence.
Wherein the abstract semantic representation is AMR (Abstract Meaning Representation). AMR diagrams are rooted, annotated, directed, and loop-free diagrams that are widely used to represent high-level semantic relationships between abstract concepts of unstructured natural text. Unlike syntactic specificity, AMR is a high-level semantic abstraction of graphs. More specifically, different sentences which are similar in semantics may share the same analysis result of the AMR graph, so that the sentence semantic representation method of the AMR graph basically has the capability of representing the semantic of one sentence more completely and more accurately, and can enable the semantic expression to be more accurate.
In step S102, the text information may be first constructed as an AMR map for the text information portion; the text information can be constructed as an AMR map, for example, directly through an AMR parser.
After the AMR diagram is obtained, the abstract semantic representation diagram can be preprocessed, and the preprocessed abstract semantic representation diagram is converted into a homogeneous diagram from a heterogeneous diagram; while the topology of the homogeneity map can be represented as a unified first adjacency matrix.
Wherein the step of preprocessing may include: and discarding the side type information of the abstract semantic representation, and retaining the side connection information of the abstract semantic representation.
It should be noted that, in the embodiment of the present application, the image recognition model may include a GNN network (graph neural network).
The graphic neural network can comprise a pooling layer, a readout layer and an output layer.
Further, as shown in fig. 3, fig. 3 is a schematic block diagram of sub-steps of a data alignment method for advertisement recommendation according to an embodiment of the present application. In this embodiment of the present application, the converting the side information of the abstract semantic representation to the first adjacency matrix, and inputting the side information to a pre-trained image recognition model to obtain full-graph information of the text sequence includes:
and S1021, carrying out pooling treatment on the first adjacent matrix through the pooling layer to obtain a pooled subgraph, and carrying out point multiplication on nodes in the pooled subgraph and a standard tangent function of the importance of the nodes to obtain a pooled subgraph after scaling transformation.
Step S1022, aggregating the scaled and transformed pooled subgraphs through the readout layer to output an aggregated pooled subgraph; and adding the aggregate pooled subgraphs through the output layer to represent the text sequence in a full graph, so as to obtain the full graph information.
It is emphasized that in the embodiments of the present application, the graph neural network uses a topok-based pooling mechanism. The pooling mechanism of TopK is a process of continually discarding nodes that can capture information on different scales of the graph. Unlike local sliding window based pooling operations in convolutional neural networks (Convolutional Neural Network, CNN), the pooling mechanism of TopK can put the scope on the full graph nodes.
Specifically, the TopK-based pooling mechanism may include: firstly, a super parameter k, k epsilon (0, 1) representing the pooling rate can be set for the graphic neural network, and then the graphic neural network is used for learningWe learn a representation of node importance z and sort it down, then downsample N nodes in the full graph to k N And (5) the nodes finally obtain the pooling subgraph after Topk processing and selection. The specific formula may be as follows:
wherein the vector is
Representing a pooling sub-graph after Topk selection, top-rank refers to selecting K with the greatest rank N And elements, wherein k is a number between 0 and 1, N is the total number of nodes, and z represents the importance of the nodes.
In addition, the method for acquiring the node importance z in the formula (1) may be as follows:
a global basis vector can be set
The projection of the node feature vector onto the base vector is taken as the node importance z. The specific formula for acquiring the node importance z is as follows:
wherein, I represents vector concatenation, X represents vector
Is included in the node (a).
Further, a gpool layer (a pooling layer) can be defined by using a formula (3), namely, a pooling sub-graph x after scaling transformation is obtained;
wherein, tan h (z) is calculated as a tan h function of importance z, and tan h represents a standard tangent function.
In the formula (3), by multiplying a tanh (z) by a point, the node characteristics are subjected to primary contraction transformation by utilizing the importance of the node, and gradient learning of the node with high importance is further enhanced.
In this embodiment, a readout layer may be further added behind the gpool layer; and aggregating the scaled and transformed pooling subgraphs through a readout layer to output an aggregated pooling subgraph, so as to realize one-time aggregation of global information of the graph under the scale. The readout layer is specifically implemented by splicing global average pooling and global maximum pooling, and the formula is as follows:
wherein s is (l) Representing the output representing the L-th output layer, x l Representing a pooled sub-graph x representing the output of the L-th output layer, ||represents vector concatenation, and L, N is a positive integer.
In order to obtain a full-scale representation, the s of each layer can be represented by the output layer (l) Adding to obtain a full graph representation S, as shown in equation (5):
wherein Y represents a positive integer.
Step S103, inputting the image information into the image recognition model to acquire the image level characteristics of the image information.
Wherein the image recognition model further comprises a GCN (graph rolling network) comprising a convolution layer, a global max pooling layer and a GCN layer. Processing the image information through a convolution layer to obtain a feature map of the image information; and carrying out pooling treatment on the feature map and the nonlinear function of the GCN layer in the image recognition model through a global maximum pooling layer to obtain image-level features.
In particular, in embodiments of the present application, node representations may be updated by propagating information between nodes of a graph rolling network. And standard coilUnlike the standard convolution acts on the local euclidean structure in the image, the goal of the GCN network in this application is to learn the function f on the graph G. GCN networks can employ characterization
And a corresponding second adjacency matrix A2, and the second adjacency matrix A2 is an input of the GCN network; wherein n is the number of nodes, d is the dimension of the node features, namely the feature dimension after label embedding, and the node features are updated to be H l+1 ∈R n×d ，R nxd I.e. the feature space of the second adjacency matrix. The node representation of each GCN layer can be written as a nonlinear function f as shown in equation (6):
H l+1 ＝f(H l ，A2) (6)
wherein H is l Node representation representing layer i in a GCN network, f (H l A2) is a nonlinear function represented by the nodes of the first layer.
From equation (6), f can be calculated GMP ，f GMP Representing a function of the global maximization layer.
In addition, resNet-101 may be used as a fundamental model for the experiment in the embodiments of the present application. After the nonlinear function represented by the node is obtained, the nonlinear functions of the feature map and the GCN layer in the image recognition model can be subjected to pooling processing through the global maximum pool layer in the image recognition model, so that the image-level feature P is obtained. The specific formula may be as follows:
P＝f GMP (f CNN (I，θ CNN ))∈R D (7)
wherein θ CNN Is a model parameter, D is an output dimension, f CNN A function representing a CNN network, the function being known, I representing a picture, R D Representing an image level feature space.
Therefore, if the resolution of the input image is 448×448, 2048×14×14 feature maps are obtained from the conv5_x layer, and then the image-level features can be obtained using formula (7).
Step S104, aligning the full-image information with the image-level features through the image recognition model, and outputting alignment data.
The graph recognition can process the full graph information and the image level features to align the full graph information and the image level features.
In the embodiment of the application, the image recognition model includes GNN (graph neural network). The training steps may include:
acquiring a full-image information sample and an image-level feature sample of advertisements, wherein each advertisement comprises at least more than one full-image information sample and at least one image-level feature sample; inputting the full-image information sample into the image neural network for training, and inputting the image-level feature sample into the image convolution network for training to obtain the image recognition model; and outputting the classification information through a perception machine layer in the image recognition model, wherein the classification information is used for judging whether the full-image information and the image-level features are aligned or not.
According to the scheme, the aligned data, namely the aligned full-image information or the image-level characteristics, can be obtained through the advertisement clicked or watched by the user, and the partial data can be applied to the advertisement recommendation system, so that the advertisement recommendation system can recommend advertisements more accurately later, and the experience of the user is improved.
In addition, since different modes of data in advertisements are represented differently, for example, image information and text information are some. Therefore, through multi-modal learning, multi-modal information can be better processed, and richer characteristic information can be obtained. The image recognition model has the GNN network and the GCN network, so that multi-mode learning can be realized.
Referring to fig. 4, fig. 4 is a schematic block diagram of a data alignment device for advertisement recommendation for performing the foregoing data alignment method for advertisement recommendation according to an embodiment of the present application. The data alignment device for advertisement recommendation can be configured on a server.
As shown in fig. 4, the data alignment device 400 for advertisement recommendation includes: an acquisition module 401, a conversion module 402, an input module 403, and an alignment module 404.
The acquiring module 401 is configured to acquire text information and image information of an advertisement;
the conversion module 402 is configured to obtain an abstract semantic representation based on the text information, convert side information of the abstract semantic representation into a first adjacency matrix, and input the first adjacency matrix into a pre-trained image recognition model to obtain full-graph information of a text sequence;
an input module 403, configured to input the image information into the image recognition model, and obtain an image level feature of the image information;
an alignment module 404, configured to align the full-image information and the image-level features through the image recognition model and output alignment data.
In an embodiment, the conversion module 402 is further configured to: and carrying out abstract semantic processing on the text information to obtain the abstract semantic representation.
In an embodiment, the conversion module 402 is further configured to: preprocessing the abstract semantic representation graph, and converting the preprocessed abstract semantic representation graph from a heterogeneous graph to a homogeneous graph, wherein the preprocessing comprises discarding edge type information of the abstract semantic representation graph, and reserving edge connection information of the abstract semantic representation graph; based on the homogeneity map, a topological structure representing the homogeneity map is obtained, and a first adjacency matrix is obtained.
In an embodiment, the image recognition model comprises a pooling layer, a readout layer and an output layer; the conversion module 402 is further configured to: carrying out pooling treatment on the first adjacent matrix through a pooling layer to obtain a pooled subgraph, and carrying out point multiplication on nodes in the pooled subgraph and standard tangent functions of the importance of the nodes to obtain a pooled subgraph after scaling transformation; aggregating the scaled and transformed pooling subgraphs through a readout layer to output an aggregated pooling subgraph; and adding the aggregate pooled subgraphs through an output layer to represent the text sequence in a full graph to obtain full graph information.
In an embodiment, the image recognition model comprises a convolution layer, a global max pooling layer and a GCN layer; the input module 403 is further configured to: processing the image information through the convolution layer to obtain a feature map of the image information; and carrying out pooling treatment on nonlinear functions of the GCN layer in the feature map and the image recognition model through the global maximum pool layer to obtain image-level features.
In an embodiment, the input module 403 is further configured to: acquiring a second adjacency matrix of the image information; and constructing the GCN layer in the image recognition model as a nonlinear function based on the second adjacency matrix to obtain a nonlinear function.
In an embodiment, the data alignment device for advertisement recommendation is further configured to: acquiring a full-image information sample and an image-level feature sample of advertisements, wherein each advertisement comprises at least more than one full-image information sample and at least one image-level feature sample; inputting the full-image information sample into the image neural network for training, and inputting the image-level feature sample into the image convolution network for training to obtain the image recognition model; and outputting the classification information through a perception machine layer in the image recognition model, wherein the classification information is used for judging whether the full-image information and the image-level features are aligned or not.
It should be noted that, for convenience and brevity of description, the specific working process of the apparatus and each module described above may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.
The apparatus described above may be implemented in the form of a computer program which is executable on a computer device as shown in fig. 5.
The data alignment device 400 for advertisement recommendation provided by the application can obtain aligned data, namely aligned full-image information or image-level characteristics, and the partial data can be applied to an advertisement recommendation system, so that advertisement recommendation is more accurate, and user experience is improved.
In addition, since different modes of data in advertisements are represented differently, for example, image information and text information are some. Therefore, through multi-modal learning, multi-modal information can be better processed, and richer characteristic information can be obtained. The image recognition model has the GNN network and the GCN network, so that multi-mode learning can be realized.
Referring to fig. 5, fig. 5 is a schematic block diagram of a computer device according to an embodiment of the present application. The computer device may be a server.
With reference to FIG. 5, the computer device includes a processor, memory, and a network interface connected by a system bus, where the memory may include a non-volatile storage medium and an internal memory.
The non-volatile storage medium may store an operating system and a computer program. The computer program comprises program instructions that, when executed, cause the processor to perform any of a number of data alignment methods for advertisement recommendation.
The processor is used to provide computing and control capabilities to support the operation of the entire computer device.
The internal memory provides an environment for the execution of a computer program in a non-volatile storage medium that, when executed by the processor, causes the processor to perform any of a number of data alignment methods for advertisement recommendation.
The network interface is used for network communication such as transmitting assigned tasks and the like. It will be appreciated by those skilled in the art that the structure shown in fig. 5 is merely a block diagram of some of the structures associated with the present application and is not limiting of the computer device to which the present application may be applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
It should be appreciated that the processor may be a central processing unit (Central Processing Unit, CPU), but may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field-programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. Wherein the general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
Wherein in one embodiment the processor is configured to run a computer program stored in the memory to implement the steps of:
acquiring text information and image information of advertisements; acquiring an abstract semantic representation diagram based on the text information, converting side information of the abstract semantic representation diagram into a first adjacency matrix, and inputting the first adjacency matrix into a pre-trained image recognition model to obtain full-image information of a text sequence; inputting the image information into the image recognition model to acquire image-level features of the image information; and aligning the full-image information with the image-level features through the image recognition model, and outputting alignment data.
In one embodiment, the processor, when implementing the data alignment method for advertisement recommendation, is configured to implement: and carrying out abstract semantic processing on the text information to obtain the abstract semantic representation.
In one embodiment, the processor, when implementing the data alignment method for advertisement recommendation, is configured to implement: preprocessing the abstract semantic representation graph, and converting the preprocessed abstract semantic representation graph from a heterogeneous graph to a homogeneous graph, wherein the preprocessing comprises discarding edge type information of the abstract semantic representation graph, and reserving edge connection information of the abstract semantic representation graph; based on the homogeneity map, a topological structure representing the homogeneity map is obtained, and a first adjacency matrix is obtained.
In one embodiment, the processor, when implementing the data alignment method for advertisement recommendation, is configured to implement: the image recognition model comprises a pooling layer, a readout layer and an output layer; the step of converting the side information of the abstract semantic representation into a first adjacency matrix and inputting the first adjacency matrix into a pre-trained image recognition model to obtain the full-graph information of the text sequence comprises the following steps:
carrying out pooling treatment on the first adjacent matrix through a pooling layer to obtain a pooled subgraph, and carrying out point multiplication on nodes in the pooled subgraph and standard tangent functions of the importance of the nodes to obtain a pooled subgraph after scaling transformation; aggregating the scaled and transformed pooling subgraphs through a readout layer to output an aggregated pooling subgraph;
and adding the aggregate pooled subgraphs through an output layer to represent the text sequence in a full graph to obtain full graph information.
In one embodiment, the processor, when implementing the data alignment method for advertisement recommendation, is configured to implement: the image recognition model comprises a convolution layer, a global maximum pooling layer and a GCN layer; inputting the image information into the image recognition model to obtain image-level features of the image information, including: processing the image information through the convolution layer to obtain a feature map of the image information; and carrying out pooling treatment on nonlinear functions of the GCN layer in the feature map and the image recognition model through the global maximum pool layer to obtain image-level features.
In one embodiment, the processor, when implementing the data alignment method for advertisement recommendation, is configured to implement: acquiring a second adjacency matrix of the image information; and constructing the GCN layer in the image recognition model as a nonlinear function based on the second adjacency matrix to obtain the nonlinear function.
In one embodiment, the processor is used for realizing a data alignment method for advertisement recommendation, wherein the data alignment method is used for obtaining a full-image information sample and an image-level characteristic sample of advertisements, and each advertisement comprises at least more than one full-image information sample and at least one image-level characteristic sample; inputting the full-image information sample into the image neural network for training, and inputting the image-level feature sample into the image convolution network for training to obtain the image recognition model; and outputting the classification information through a perception machine layer in the image recognition model, wherein the classification information is used for judging whether the full-image information and the image-level features are aligned or not.
According to the scheme, the aligned data, namely the aligned full-image information or the image-level characteristics, can be obtained, and the partial data can be applied to an advertisement recommendation system, so that advertisement recommendation is more accurate, and user experience is improved.
An embodiment of the present application further provides a computer readable storage medium, where the computer readable storage medium stores a computer program, where the computer program includes program instructions, and the processor executes the program instructions to implement the data alignment method for any advertisement recommendation provided in the embodiment of the present application.
The computer readable storage medium may be an internal storage unit of the computer device according to the foregoing embodiment, for example, a hard disk or a memory of the computer device. The computer readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), or the like, which are provided on the computer device.
While the invention has been described with reference to certain preferred embodiments, it will be understood by those skilled in the art that various changes and substitutions of equivalents may be made and equivalents will be apparent to those skilled in the art without departing from the scope of the invention. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A data alignment method for advertisement recommendation, comprising:
acquiring text information and image information of advertisements;
acquiring an abstract semantic representation diagram based on the text information, converting side information of the abstract semantic representation diagram into a first adjacency matrix, and inputting the first adjacency matrix into a pre-trained image recognition model to obtain full-image information of a text sequence;
inputting the image information into the image recognition model to acquire image-level features of the image information;
and aligning the full-image information with the image-level features through the image recognition model, and outputting alignment data.
2. The data alignment method for advertisement recommendation according to claim 1, wherein the obtaining an abstract semantic representation based on the text information comprises:
and carrying out abstract semantic processing on the text information to obtain the abstract semantic representation.
3. The data alignment method for advertisement recommendation according to claim 1, wherein converting the side information of the abstract semantic representation into a first adjacency matrix comprises:
preprocessing the abstract semantic representation graph, and converting the preprocessed abstract semantic representation graph from a heterogeneous graph to a homogeneous graph, wherein the preprocessing comprises discarding edge type information of the abstract semantic representation graph, and reserving edge connection information of the abstract semantic representation graph;
based on the homogeneity map, a topological structure representing the homogeneity map is obtained, and a first adjacency matrix is obtained.
4. The data alignment method for advertisement recommendation according to claim 1, wherein the image recognition model comprises a pooling layer, a readout layer, and an output layer; the step of converting the side information of the abstract semantic representation into a first adjacency matrix and inputting the first adjacency matrix into a pre-trained image recognition model to obtain the full-graph information of the text sequence comprises the following steps:
carrying out pooling treatment on the first adjacent matrix through a pooling layer to obtain a pooled subgraph, and carrying out point multiplication on nodes in the pooled subgraph and standard tangent functions of the importance of the nodes to obtain a pooled subgraph after scaling transformation;
aggregating the scaled and transformed pooling subgraphs through a readout layer to output an aggregated pooling subgraph;
and adding the aggregate pooled subgraphs through an output layer to represent the text sequence in a full graph to obtain full graph information.
5. The data alignment method for advertisement recommendation according to claim 1, wherein the image recognition model comprises a convolution layer, a global max pooling layer, a GCN layer; inputting the image information into the image recognition model to obtain image-level features of the image information, including:
processing the image information through the convolution layer to obtain a feature map of the image information;
and carrying out pooling treatment on nonlinear functions of the GCN layer in the feature map and the image recognition model through the global maximum pool layer to obtain image-level features.
6. The data alignment method for advertisement recommendation of claim 5, further comprising:
acquiring a second adjacency matrix of the image information;
and constructing the GCN layer in the image recognition model as a nonlinear function based on the second adjacency matrix to obtain the nonlinear function.
7. The data alignment method for advertisement recommendation according to any one of claims 1-6, further comprising:
acquiring a full-image information sample and an image-level feature sample of advertisements, wherein each advertisement comprises at least more than one full-image information sample and at least one image-level feature sample;
inputting the full-image information sample into the image neural network for training, and inputting the image-level feature sample into the image convolution network for training to obtain the image recognition model;
and outputting the classification information through a perception machine layer in the image recognition model, wherein the classification information is used for judging whether the full-image information and the image-level features are aligned or not.
8. A data alignment apparatus for advertisement recommendation, comprising:
the acquisition module is used for acquiring text information and image information of the advertisement;
the conversion module is used for acquiring an abstract semantic representation diagram based on the text information, converting side information of the abstract semantic representation diagram into a first adjacent matrix, and inputting the first adjacent matrix into a pre-trained image recognition model to obtain full-picture information of a text sequence;
the input module is used for inputting the image information into the image recognition model and acquiring image-level characteristics of the image information;
and the alignment module is used for aligning the full-image information and the image-level features through the image recognition model and outputting alignment data through the image recognition model.
9. A computer device, the computer device comprising a memory and a processor;
the memory is used for storing a computer program;
the processor for executing the computer program and for implementing the data alignment method for advertisement recommendation according to any one of claims 1 to 7 when executing the computer program.
10. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program which, when executed by a processor, causes the processor to implement the data alignment method for advertisement recommendation according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211084782.2A CN116402555A (en) | 2022-09-06 | 2022-09-06 | Data alignment method, device, equipment and storage medium for advertisement recommendation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211084782.2A CN116402555A (en) | 2022-09-06 | 2022-09-06 | Data alignment method, device, equipment and storage medium for advertisement recommendation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116402555A true CN116402555A (en) | 2023-07-07 |
Family
ID=87014787
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211084782.2A Pending CN116402555A (en) | 2022-09-06 | 2022-09-06 | Data alignment method, device, equipment and storage medium for advertisement recommendation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116402555A (en) |
-
2022
- 2022-09-06 CN CN202211084782.2A patent/CN116402555A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10664999B2 (en) | Saliency prediction for a mobile user interface | |
| CN108345890B (en) | Image processing method, device and related equipment | |
| US20150049951A1 (en) | Presenting meaningful information summary for analyzing complex visualizations | |
| CN110378410B (en) | Multi-label scene classification method and device and electronic equipment | |
| WO2019102309A1 (en) | Search query enhancement with context analysis | |
| CN111666416B (en) | Method and device for generating semantic matching model | |
| CN111625645B (en) | Training method and device for text generation model and electronic equipment | |
| CN109087162A (en) | Data processing method, system, medium and calculating equipment | |
| CN112348592A (en) | Advertisement recommendation method and device, electronic equipment and medium | |
| US20230102804A1 (en) | Method of rectifying text image, training method, electronic device, and medium | |
| CN112749300A (en) | Method, apparatus, device, storage medium and program product for video classification | |
| CN109389660A (en) | Image generating method and device | |
| CN112785493A (en) | Model training method, style migration method, device, equipment and storage medium | |
| CN113468330B (en) | Information acquisition method, device, equipment and medium | |
| CN113408507B (en) | Named entity identification method and device based on resume file and electronic equipment | |
| CN111898338B (en) | Text generation method and device and electronic equipment | |
| CN113850714A (en) | Training of image style conversion model, image style conversion method and related device | |
| CN113723072A (en) | RPA (resilient packet Access) and AI (Artificial Intelligence) combined model fusion result acquisition method and device and electronic equipment | |
| CN112434746B (en) | Pre-labeling method based on hierarchical migration learning and related equipment thereof | |
| CN116756281A (en) | Knowledge question-answering method, device, equipment and medium | |
| CN116402555A (en) | Data alignment method, device, equipment and storage medium for advertisement recommendation | |
| US20220358347A1 (en) | Computerized system and method for distilled deep prediction for personalized stream ranking | |
| CN113610856A (en) | Method and device for training image segmentation model and image segmentation | |
| CN113688938A (en) | Method for determining object emotion and method and device for training emotion classification model | |
| CN111950572A (en) | Method, apparatus, electronic device and computer-readable storage medium for training classifier |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |