CN113111273A - Information recommendation method and device, electronic equipment and storage medium - Google Patents
Information recommendation method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113111273A CN113111273A CN202110430370.9A CN202110430370A CN113111273A CN 113111273 A CN113111273 A CN 113111273A CN 202110430370 A CN202110430370 A CN 202110430370A CN 113111273 A CN113111273 A CN 113111273A
- Authority
- CN
- China
- Prior art keywords
- layer
- information
- input
- information recommendation
- splicing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 66
- 239000013598 vector Substances 0.000 claims abstract description 174
- 230000003993 interaction Effects 0.000 claims abstract description 72
- 239000011159 matrix material Substances 0.000 claims abstract description 59
- 238000004364 calculation method Methods 0.000 claims abstract description 54
- 238000012163 sequencing technique Methods 0.000 claims abstract description 24
- 230000007246 mechanism Effects 0.000 claims abstract description 17
- 230000014509 gene expression Effects 0.000 claims description 13
- 230000006870 function Effects 0.000 claims description 12
- 238000013507 mapping Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 8
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 4
- 230000009286 beneficial effect Effects 0.000 abstract description 3
- 230000008569 process Effects 0.000 description 13
- 238000012545 processing Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 238000004422 calculation algorithm Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- XXSCONYSQQLHTH-UHFFFAOYSA-N 9h-fluoren-9-ylmethanol Chemical compound C1=CC=C2C(CO)C3=CC=CC=C3C2=C1 XXSCONYSQQLHTH-UHFFFAOYSA-N 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9538—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9535—Search customisation based on user profiles and personalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Pure & Applied Mathematics (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Computational Mathematics (AREA)
- Computing Systems (AREA)
- Algebra (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses an information recommendation method, which is applied to a sequencing model, wherein the sequencing model comprises an input layer, a full connection layer, an interaction layer and a splicing module, and the information recommendation method comprises the following steps: generating a feature embedding vector of input information by using an input layer; inputting the feature embedded vectors into an interaction layer, splicing all the feature embedded vectors by using the interaction layer to obtain target vectors, and executing calculation of a multi-head attention mechanism among elements on the target vectors to obtain output of the interaction layer; inputting the characteristic embedded vectors into a full-connection layer, and performing matrix multiplication on the characteristic embedded vectors by using the full-connection layer to obtain full-connection layer output; and splicing the interaction layer output and the full-connection layer output by using a splicing module to obtain the ranking score of the input information, and outputting an information recommendation result according to the ranking score. The method and the device can improve the accuracy of information recommendation. The application also discloses an information recommendation device, an electronic device and a storage medium, which have the beneficial effects.
Description
Technical Field
The present disclosure relates to the field of machine learning, and in particular, to an information recommendation method and apparatus, an electronic device, and a storage medium.
Background
The personalized recommendation algorithm can help the user to find favorite contents, and the online duration and the retention rate of the user are improved. The personalized recommendation algorithm is realized based on a ranking model, the ranking model is used for scoring and ranking the alternative information, and N alternative information before ranking is used as recommendation content and pushed to the client.
In the related technology, the ordering calculation is mainly performed by using an ordering model based on the AutoInt algorithm, but because the minimum unit of the operation of the AutoInt algorithm is a feature embedding vector, all the feature embedding vectors need to be the same in size, and the mode of limiting the size of the feature embedding vector influences the expression of the feature embedding vector and reduces the accuracy of information recommendation.
Therefore, how to improve the accuracy of information recommendation is a technical problem that needs to be solved by those skilled in the art.
Disclosure of Invention
The application aims to provide an information recommendation method and device, an electronic device and a storage medium, which can improve the accuracy of information recommendation.
In order to solve the technical problem, the application provides an information recommendation method, which is applied to a sequencing model, wherein the sequencing model comprises an input layer, a full-link layer, an interaction layer and a splicing module, and the information recommendation method comprises the following steps:
generating a feature embedding vector of input information by using the input layer;
inputting the feature embedded vectors into the interaction layer, splicing all the feature embedded vectors by using the interaction layer to obtain target vectors, and performing calculation of a multi-head attention mechanism between elements on the target vectors to obtain interaction layer output;
inputting the characteristic embedded vector into the full-connection layer, and performing matrix multiplication on the characteristic embedded vector by using the full-connection layer to obtain full-connection layer output;
and splicing the interaction layer output and the full-connection layer output by using the splicing module to obtain the ranking score of the input information, and outputting an information recommendation result according to the ranking score.
Optionally, generating a feature embedding vector of the input information by using the input layer includes:
representing the input information as one-hot coding in an embedding layer of the input layer, and performing feature embedding vector conversion on the one-hot coding to obtain an initial feature embedding vector;
and performing corresponding weighting operation on the initial feature embedding vector according to the data type of the input information in a weighting splicing module of the input layer to obtain the feature embedding vector.
Optionally, performing corresponding weighting operation on the initial feature embedding vector according to the data type of the input information to obtain the feature embedding vector, including:
judging whether the input information is non-segmented numerical data or not;
if so, multiplying the initial feature embedding vector by the original numerical value of the input information to obtain the feature embedding vector;
if not, multiplying the initial feature embedding vector by 1 to obtain the feature embedding vector.
Optionally, the obtaining an interaction layer output by performing calculation of a multi-head attention mechanism between elements on the target vector includes:
performing matrix inner product calculation on the target vector and a query matrix and a key matrix of the sequencing model respectively to obtain a first result and a second result, and performing Hadamard product calculation on the ith element of the first result and the kth element of the second result to obtain a mapping function;
performing softmax calculation on the mapping function to obtain an attention weight;
obtaining the fraction weight of the target vector and the value matrix of the sequencing model, and performing matrix inner product calculation to obtain a third result;
and performing weighted calculation on the attention weight and the third result to obtain an element self-attention expression under each head in the multi-head attention mechanism, and multiplying all the element self-attention expressions to obtain the interaction layer output.
Optionally, before generating the feature embedding vector of the input information by using the input layer, the method further includes:
receiving a recommendation request, and determining a requester portrait corresponding to the recommendation request;
respectively combining the requester portraits with resource portraits of a plurality of alternative resources to obtain the request pairs;
and performing feature extraction on the request pair to obtain the input information.
Optionally, outputting an information recommendation result according to the ranking score includes:
and sequencing the alternative resources according to the sequence of the ranking scores from high to low to obtain a resource recommendation sequence, and outputting the information recommendation result according to the resource recommendation sequence.
Optionally, the obtaining the ranking score of the input information by using the splicing module to splice the interaction layer output and the full connection layer output includes:
and splicing the interaction layer output and the full-connection layer output by using the splicing module to obtain a splicing result, and performing weighted calculation on the splicing result to obtain the ranking score of the input information.
The application also provides an information recommendation device, which comprises a sequencing model, wherein the sequencing model comprises an input layer, a full connection layer, an interaction layer and a splicing module;
wherein the input layer is used for generating a feature embedding vector of input information; the interaction layer is used for splicing all the characteristic embedded vectors to obtain target vectors, and calculating a multi-head attention mechanism between the target vector execution elements to obtain interaction layer output; the full connection layer is used for carrying out matrix multiplication on the characteristic embedded vectors to obtain full connection layer output; the splicing module is used for splicing the interaction layer output and the full connection layer output to obtain the ranking score of the input information, and outputting an information recommendation result according to the ranking score.
The application also provides a storage medium, on which a computer program is stored, which when executed implements the steps performed by the above information recommendation method.
The application also provides an electronic device, which comprises a memory and a processor, wherein the memory is stored with a computer program, and the processor realizes the steps executed by the information recommendation method when calling the computer program in the memory.
The application provides an information recommendation method, which is applied to a sequencing model, wherein the sequencing model comprises an input layer, a full connection layer, an interaction layer and a splicing module, and the information recommendation method comprises the following steps: generating a feature embedding vector of input information by using the input layer; inputting the feature embedded vectors into the interaction layer, splicing all the feature embedded vectors by using the interaction layer to obtain target vectors, and performing calculation of a multi-head attention mechanism between elements on the target vectors to obtain interaction layer output; inputting the characteristic embedded vector into the full-connection layer, and performing matrix multiplication on the characteristic embedded vector by using the full-connection layer to obtain full-connection layer output; and splicing the interaction layer output and the full-connection layer output by using the splicing module to obtain the ranking score of the input information, and outputting an information recommendation result according to the ranking score.
The method and the device utilize the input layer to generate the feature embedded vectors of the input information, and splice all the feature embedded vectors into the target vector in the interaction layer so as to perform multi-head attention calculation operation on the target vector. Because the target vector is calculated based on the spliced feature embedded vectors, the calculation between every two feature embedded vectors is not relied, and the size of the feature embedded vectors is not limited. The ranking score of the input information is determined by utilizing the interaction layer output and the full connection layer output, and then the information recommendation result is obtained. According to the method and the device, the size of the characteristic embedding vector is not limited, information contained in the characteristic embedding vector can be sufficiently expressed, and the information recommendation accuracy is further improved. The application also provides an information recommendation device, an electronic device and a storage medium, and the information recommendation device, the electronic device and the storage medium have the beneficial effects and are not repeated herein.
Drawings
In order to more clearly illustrate the embodiments of the present application, the drawings needed for the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings can be obtained by those skilled in the art without inventive effort.
Fig. 1 is an architecture diagram of an information recommendation system according to an embodiment of the present application;
fig. 2 is a flowchart of an information recommendation method according to an embodiment of the present application;
FIG. 3 is a schematic structural diagram of a ranking model according to an embodiment of the present application;
fig. 4 is a flowchart of a method for obtaining a feature embedding vector according to an embodiment of the present disclosure;
FIG. 5 is a flowchart of an interaction layer calculation process according to an embodiment of the present disclosure;
fig. 6 is a schematic diagram illustrating a computing principle of an interaction layer according to an embodiment of the present disclosure;
fig. 7 is a flowchart of a live broadcast room recommendation method for a live broadcast platform according to an embodiment of the present application;
fig. 8 is a block diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
In order to facilitate understanding of the information recommendation method provided in the present application, a system for using the information recommendation method is described below. Referring to fig. 1, fig. 1 is an architecture diagram of an information recommendation system according to an embodiment of the present application, and as shown in fig. 1, the information recommendation system includes a terminal device 101, a server 102, and an information recommendation apparatus 103. The terminal device 101 may be a mobile phone, a tablet computer, or a personal computer, and the terminal device 101 sends the user information of the logged-in user to the server 102. The server 102 is used for the terminal apparatus 101 to provide a resource usage service, and a plurality of resources may exist in the server 102. The server 102 may send the information related to the resource and the user information to the information recommendation device 103, where the information recommendation device 103 runs a ranking model. The information recommendation device 103 may score the matching degree of each resource and the user information by using the ranking model to obtain a ranking result of the matching degree. The information recommendation device 103 may send the matching degree ranking result to the server 102, and the server 102 may send resource recommendation information to the terminal 101 according to the matching degree ranking result so as to recommend the top N ranked resources to the user.
Referring to fig. 2, fig. 2 is a flowchart of an information recommendation method according to an embodiment of the present application.
The specific steps may include:
s201: generating a feature embedding vector of input information by using the input layer;
fig. 3 is a schematic structural diagram of a sorting model provided in an embodiment of the present application, where the sorting model mentioned above may include an input layer, a full connection layer, an interaction layer, and a splicing module. The input layer includes an embedded once and weighted concatenation module. The input information may be data characteristics of data to be sorted, and the embodiment may generate the characteristic embedding vector embedding of the input information by using an input layer.
After the feature embedding vector is obtained, the input layer may transmit the feature embedding vector to the full connection layer and the interaction layer for processing, so the execution order of S102 and S103 is not limited in this embodiment, and after the interaction layer output and the full connection layer output are obtained, S104 may be entered for sorting and information recommendation.
S202: inputting the feature embedded vectors into an interaction layer, splicing all the feature embedded vectors by using the interaction layer to obtain target vectors, and executing calculation of a multi-head attention mechanism among elements on the target vectors to obtain output of the interaction layer;
after the feature embedding vectors are input to the interaction layer, all the feature embedding vectors can be spliced in the interaction layer to obtain one vector, namely a target vector. The target vector includes all elements of the feature embedded vector, and then the calculation operation of a Multi-head Self-attention mechanism (Multi-head Self-attention) between elements is executed on the target vector, so as to obtain the output of the interaction layer. In the process, the elements in the target vector can be processed, and the calculation process is not limited by the size of the feature embedding vector.
In actual use, because the information content of different characteristics is different, redundancy or insufficiency exists in the uniform size. The embodiment is based on the splicing results of all the feature embedded vectors, does not depend on the calculation of every two feature embedded vectors, and therefore the size of the feature embedded vectors does not need to be limited, and information contained in the feature embedded vectors is fully expressed.
S203: inputting the characteristic embedded vectors into a full-connection layer, and performing matrix multiplication on the characteristic embedded vectors by using the full-connection layer to obtain full-connection layer output;
after the feature embedded vectors are input to the full-link layer, the full-link layer may perform matrix multiplication on the feature embedded vectors to obtain full-link layer output.
S204: and splicing the interaction layer output and the full-connection layer output by using the splicing module to obtain the ranking score of the input information, and outputting an information recommendation result according to the ranking score.
On the basis of obtaining the interaction layer output and the full connection layer output, the embodiment can utilize the splicing module to splice the interaction layer output and the full connection layer output to obtain a splicing result, and determine the ranking score of the input information according to the splicing result, so that the information which can be recommended is determined according to the ranking score from high to low, and the information recommendation result is obtained. Specifically, this embodiment may utilize the splicing module to splice the interaction layer output and the full-connection layer output to obtain a splicing result, and perform weighted calculation on the splicing result to obtain the ranking score of the input information.
Further, in this embodiment, the alternative resources may be ranked in order of the ranking score from high to low to obtain a resource recommendation order, and the information recommendation result may be output according to the resource recommendation order.
The method and the device utilize the input layer to generate the feature embedded vectors of the input information, and splice all the feature embedded vectors into the target vector in the interaction layer so as to perform multi-head attention calculation operation on the target vector. Because the target vector is calculated based on the spliced feature embedded vectors, the calculation between every two feature embedded vectors is not relied, and the size of the feature embedded vectors is not limited. The ranking score of the input information is determined by utilizing the interaction layer output and the full connection layer output, and then the information recommendation result is obtained. According to the method and the device, the size of the characteristic embedding vector is not limited, information contained in the characteristic embedding vector can be sufficiently expressed, and the information recommendation accuracy is further improved.
Referring to fig. 4, fig. 4 is a flowchart of a method for obtaining a feature embedding vector according to an embodiment of the present application, where this embodiment is an explanation of a process of generating a feature embedding vector of input information in an embodiment corresponding to fig. 2, and this embodiment may be combined with the embodiment corresponding to fig. 2 to obtain a further implementation manner, and this embodiment may include the following steps:
s401: representing the input information as one-hot encoding in an embedding layer of the input layers;
the input layer of the ranking model includes an embedded layer and a weighted concatenation module, and this embodiment may use the embedded layer to represent the input information as a one-hot encoding.
S402: performing feature embedding vector conversion on the one-hot code to obtain an initial feature embedding vector;
the operation procedures of S401 and S402 are illustrated:
for example, dividing the age into the following intervals [ <18,19-30,30-40,40-50,50-60, >60], then 24 years can be represented as [0,1,0,0,0,0] by one-hot coding, constructing an embedding matrix, which is 6 x 8 given the size of embedding as 8. After 24 years of age to imbedding, it is the 8 elements in the second column of the matrix.
S403: and performing corresponding weighting operation on the initial feature embedding vector according to the data type of the input information in a weighting splicing module of the input layer to obtain the feature embedding vector.
Since the initial feature embedding vector obtained by using the one-hot encoding does not include the meaning of the numerical value itself, the present embodiment may perform a corresponding weighting operation on the initial feature embedding vector according to the data type of the input information to obtain a feature embedding vector for performing the ranking score calculation.
Specifically, the present embodiment may determine whether the input information is non-segmented numerical data; if so, multiplying the initial feature embedding vector by the original numerical value of the input information to obtain the feature embedding vector; if not, multiplying the initial feature embedding vector by 1 to obtain the feature embedding vector. The non-segmented numerical data may include click rate, attention count, and the like.
Referring to fig. 5, fig. 5 is a flowchart of an interaction layer calculation process provided in an embodiment of the present application, where this embodiment is an explanation of a process of inputting a feature embedding vector into an interaction layer for calculation in the embodiment corresponding to fig. 2, and a further embodiment may be obtained by combining this embodiment with the embodiment corresponding to fig. 2, where this embodiment may include the following steps:
s501: splicing all the feature embedded vectors by utilizing an interaction layer to obtain a target vector;
s502: performing matrix inner product calculation on the target vector and a query matrix and a key matrix of the sequencing model respectively to obtain a first result and a second result, and performing Hadamard product calculation on the ith element of the first result and the kth element of the second result to obtain a mapping function;
s503: performing softmax calculation on the mapping function to obtain an attention weight;
s504: obtaining the fraction weight of the target vector and the value matrix of the sequencing model, and performing matrix inner product calculation to obtain a third result;
s505: and performing weighted calculation on the attention weight and the third result to obtain an element self-attention expression under each head in the multi-head attention mechanism, and multiplying all the element self-attention expressions to obtain the interaction layer output.
The query matrix, the key matrix and the value matrix in the above embodiments are preset matrices in a ranking model using a multi-head attention mechanism. Because the embodiment performs calculation based on the elements embedded in the feature vectors, if a matrix inner product is used for calculating a mapping function, an oversized matrix appears, and the matrix is difficult to load by a single-machine GPU, so that the application scene of the embodiment is limited. In order to solve the above problems, the present application uses a hadamard product to calculate a mapping function, thereby avoiding the occurrence of a large matrix, and the hadamard product can make the feature crossing granularity finer to obtain rich feature crossing results.
A specific process of calculation in the interaction layer is described below by an actual example, please refer to fig. 6, where fig. 6 is a schematic diagram of a principle of calculation of the interaction layer provided in the embodiment of the present application, e 1-eM in fig. 6 represent feature embedded vectors, all the feature embedded vectors are spliced to obtain a target vector, Vm is an m-th value of all the feature embedded vectors after being spliced, and the target vector is transformed by a query matrix, a key matrix, and a value matrix to obtain a value Vm. Vm means the product of the left value and the attention weight, with a size equal to the length of the concatenation of all feature-embedded vectors.
The calculation process in the interaction layer is described by the following formulas 1 to 4:
equation 1:
equation 2:
equation 3:
equation 4:
in the above formula, T is the form of the feature embedded vector after splicing, h represents the head (i.e. granularity) of the multi-head attention mechanism,
in the form of a query matrix, the query matrix,
in the form of a key matrix, the key matrix,
in order to be a value matrix, the value matrix,<>indicating a matrix inner product calculation, an-indicates a hadamard product calculation. Alpha is attention weight, i represents the ith element in T, k represents the kth element in T, phi represents mapping function, N represents the length of T after splicing of the feature embedding vector,
self-attention expression, v, representing the ith element obtained under the h-th headiA self-attention expression of the ith element after processing of a plurality of heads is represented. A header may be understood as a set of parameters, each header corresponding to a set of Wquery, Wkey, and Wvalue parameters, and several headers corresponding to several sets of the above parameters.
In this embodiment, the matrix inner product calculation may be performed on the T and the query matrix to obtain the query, the matrix inner product calculation may be performed on the T and the key matrix to obtain the key, and the matrix inner product calculation may be performed on the T and the value matrix to obtain the value. Using formula 2 to calculate the mapping function phi of the ith element and the kth element in the T under the h head by using the query and the key as Hadamard codes(h)(Ti,Tk) The attention weight attention score is obtained by performing softmax calculation by using the formula 1And multiplying the attention weight by the value by using a formula 3, summing to obtain the self-attention expression of the ith element under the h-th head, and calculating the self-attention expression of the ith element after the multiple heads are processed by using a formula 4 to obtain an interaction layer output result.
The above embodiment does not use the inner product of the matrix in calculating the attention weight, but uses the hadamard product for calculation. If the inner product is used by the original method, a large matrix of T x T can be generated when attention weight is obtained, which is huge in an actual industrial scene, because the dimensionality of T is generally in the level of thousands/ten thousand, and a single GPU is difficult to load the large matrix, which is not beneficial to the application of the embodiment in more scenes. In the embodiment, the formula is adjusted to be in a Hadamard product form, the size of the matrix of the attention weight is still T, and the application scene of the scheme is expanded. Furthermore, in this embodiment, the attention weight is calculated by using the hadamard product, so that the feature crossing granularity is finer, and a richer feature crossing result is obtained.
This example was experimented with several public datasets (movilens, Criteo, and Avazu) in the field of CTR prediction and compared to some currently mainstream methods. The comparison model mainly comprises LR, FM, AFM, deep Cross, NFM, crossNet, CIN, HOFM and AutoInt, the name of the sequencing model provided by the embodiment is BitInt, and the specific effect pair is as follows:
in the table, AUC is the area under the ROC curve, and is an evaluation index of classification, and the larger the value, the better the representation effect. Logloss is a logarithmic loss, with smaller values indicating better model learning. As can be seen from the table above, our model achieves the best results across all data sets.
The embodiment proposes to change the interaction mode on the vector-wise based on multi-head self-attention on the sub-granularity bit-wise layer of the feature embedding vector, so that the size of the feature embedding vector can be freely adjusted, and a better effect is obtained. If the method is based on the original multi-head self-attack method, a super-large matrix appears in the middle step, so that the method cannot be applied in an industrial scene.
The flow described in the above embodiment is explained below by an embodiment in practical use. Referring to fig. 7, fig. 7 is a flowchart of a live broadcast room recommendation method for a live broadcast platform according to an embodiment of the present application, where this embodiment introduces a scheme for a platform to personalized push a live broadcast room for a user, and the specific process is as follows:
a user logs iN a live broadcast platform through a client and sends a push request u to a live broadcast server, the live broadcast server inquires current anchor i1 and i2 … iN which are currently online from an online anchor pool after receiving the push request, and the current user and each anchor are spliced into a request pair { u, i1}, { u, i2} … { u, iN }. And sending the request pair to a feature platform, and further extracting features by using the feature platform to obtain the feature pair of the user and the anchor. And inputting the characteristic pairs into the sorting model so that the sorting model scores based on the characteristic matching degree of the user and the anchor to obtain scores, and sorting according to the scores so as to push information based on a sorting result.
As a possible implementation, the present embodiment may extract the input information by: receiving a recommendation request, and determining a requester portrait corresponding to the recommendation request; respectively combining the requester portraits with resource portraits of a plurality of alternative resources to obtain the request pairs; and performing feature extraction on the request pair to obtain the input information. A requestor portrait is information that describes a user's characteristics and a resource portrait is information that describes a resource's characteristics. In an application scenario of a live broadcast platform, the above features include: basic portraits of users and anchor including age, gender, and school calendar; statistical characteristics including the watching duration of the user, the number of the paid fees and the like of the user; the real-time characteristics comprise the number of watching users in a live broadcast room, the type of the live broadcast room, the playing time length and the like.
An information recommendation device provided by an embodiment of the present application may include:
the information recommendation device comprises a sequencing model, wherein the sequencing model comprises an input layer, a full connection layer, an interaction layer and a splicing module;
wherein the input layer is used for generating a feature embedding vector of input information; the interaction layer is used for splicing all the characteristic embedded vectors to obtain target vectors, and calculating a multi-head attention mechanism between the target vector execution elements to obtain interaction layer output; the full connection layer is used for carrying out matrix multiplication on the characteristic embedded vectors to obtain full connection layer output; the splicing module is used for splicing the interaction layer output and the full connection layer output to obtain the ranking score of the input information, and outputting an information recommendation result according to the ranking score.
The method and the device utilize the input layer to generate the feature embedded vectors of the input information, and splice all the feature embedded vectors into the target vector in the interaction layer so as to perform multi-head attention calculation operation on the target vector. Because the target vector is calculated based on the spliced feature embedded vectors, the calculation between every two feature embedded vectors is not relied, and the size of the feature embedded vectors is not limited. The ranking score of the input information is determined by utilizing the interaction layer output and the full connection layer output, and then the information recommendation result is obtained. According to the method and the device, the size of the characteristic embedding vector is not limited, information contained in the characteristic embedding vector can be sufficiently expressed, and the information recommendation accuracy is further improved.
Further, the process of generating the feature embedding vector of the input information by the input layer includes: representing the input information as one-hot coding in an embedding layer, and performing feature embedding vector conversion on the one-hot coding to obtain an initial feature embedding vector; and performing corresponding weighting operation on the initial feature embedding vector according to the data type of the input information in a weighting splicing module to obtain the feature embedding vector.
Further, the weighted splicing module is configured to determine whether the input information is non-segmented numerical data; if so, multiplying the initial feature embedding vector by the original numerical value of the input information to obtain the feature embedding vector; if not, multiplying the initial feature embedding vector by 1 to obtain the feature embedding vector.
Further, the splicing module is configured to perform matrix inner product calculation on the target vector and the query matrix and the key matrix of the ordering model respectively to obtain a first result and a second result, and perform hadamard product calculation on an ith element of the first result and a kth element of the second result to obtain a mapping function; the attention weight is obtained by performing softmax calculation on the mapping function; the value matrix of the ordering model is used for obtaining the fraction weight of the target vector and calculating the matrix inner product to obtain a third result; and the second result is used for carrying out weighted calculation on the attention weight and the third result to obtain element self-attention expression under each head in the multi-head attention mechanism, and multiplying all the element self-attention expressions to obtain the interaction layer output.
Further, the method also comprises the following steps:
the characteristic extraction module is used for receiving a recommendation request and determining a requester portrait corresponding to the recommendation request before generating a characteristic embedding vector of input information by utilizing the input layer; the system is also used for respectively combining the requester portraits with resource portraits of a plurality of alternative resources to obtain the request pairs; and the system is also used for extracting the characteristics of the request pair to obtain the input information.
Further, the splicing module is used for sequencing the alternative resources according to the sequence of the ranking scores from high to low to obtain a resource recommendation sequence, and outputting the information recommendation result according to the resource recommendation sequence.
Further, the splicing module is used for splicing the interaction layer output and the full-connection layer output by using the splicing module to obtain a splicing result, and performing weighted calculation on the splicing result to obtain the ranking score of the input information.
Since the embodiments of the apparatus portion and the method portion correspond to each other, please refer to the description of the embodiments of the method portion for the embodiments of the apparatus portion, which is not repeated here.
The present application also provides a storage medium having a computer program stored thereon, which when executed, may implement the steps provided by the above-described embodiments. The storage medium may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The present application further provides an electronic device, and referring to fig. 8, a structure diagram of an electronic device provided in an embodiment of the present application, as shown in fig. 8, may include a processor 810 and a memory 820.
Processor 810 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so forth. The processor 810 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). Processor 810 may also include a main processor and a coprocessor, where the main processor is a processor, also called a Central Processing Unit (CPU), for Processing data in the wake state; a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 810 may be integrated with a GPU (Graphics Processing Unit) that is responsible for rendering and drawing the content that the display screen needs to display. In some embodiments, the processor 810 may further include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning.
Memory 820 may include one or more computer-readable storage media, which may be non-transitory. Memory 820 may also include high speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In this embodiment, the memory 820 is at least used for storing a computer program 821, wherein after being loaded and executed by the processor 810, the computer program can implement relevant steps in the information recommendation method disclosed in any of the foregoing embodiments. In addition, the resources stored by the memory 820 may also include an operating system 822, data 823, and the like, and the storage may be transient storage or permanent storage. The operating system 822 may include Windows, Linux, Android, and the like.
In some embodiments, the electronic device may also include a display screen 830, an input-output interface 840, a communication interface 850, sensors 860, a power source 870, and a communication bus 880.
Of course, the structure of the electronic device shown in fig. 8 does not constitute a limitation of the electronic device in the embodiment of the present application, and the electronic device may include more or less components than those shown in fig. 8 or some components in combination in practical applications.
The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description. It should be noted that, for those skilled in the art, it is possible to make several improvements and modifications to the present application without departing from the principle of the present application, and such improvements and modifications also fall within the scope of the claims of the present application.
It is further noted that, in the present specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Claims (10)
1. An information recommendation method is applied to a sequencing model, the sequencing model comprises an input layer, a full connection layer, an interaction layer and a splicing module, and the information recommendation method comprises the following steps:
generating a feature embedding vector of input information by using the input layer;
inputting the feature embedded vectors into the interaction layer, splicing all the feature embedded vectors by using the interaction layer to obtain target vectors, and performing calculation of a multi-head attention mechanism between elements on the target vectors to obtain interaction layer output;
inputting the characteristic embedded vector into the full-connection layer, and performing matrix multiplication on the characteristic embedded vector by using the full-connection layer to obtain full-connection layer output;
and splicing the interaction layer output and the full-connection layer output by using the splicing module to obtain the ranking score of the input information, and outputting an information recommendation result according to the ranking score.
2. The information recommendation method of claim 1, wherein generating feature embedding vectors of input information using the input layer comprises:
representing the input information as one-hot coding in an embedding layer of the input layer, and performing feature embedding vector conversion on the one-hot coding to obtain an initial feature embedding vector;
and performing corresponding weighting operation on the initial feature embedding vector according to the data type of the input information in a weighting splicing module of the input layer to obtain the feature embedding vector.
3. The information recommendation method according to claim 2, wherein performing corresponding weighting operations on the initial feature embedding vector according to the data type of the input information to obtain the feature embedding vector comprises:
judging whether the input information is non-segmented numerical data or not;
if so, multiplying the initial feature embedding vector by the original numerical value of the input information to obtain the feature embedding vector;
if not, multiplying the initial feature embedding vector by 1 to obtain the feature embedding vector.
4. The information recommendation method according to claim 1, wherein the calculating of the multi-head attention mechanism between elements on the target vector results in an interaction layer output, comprising:
performing matrix inner product calculation on the target vector and a query matrix and a key matrix of the sequencing model respectively to obtain a first result and a second result, and performing Hadamard product calculation on the ith element of the first result and the kth element of the second result to obtain a mapping function;
performing softmax calculation on the mapping function to obtain an attention weight;
obtaining the fraction weight of the target vector and the value matrix of the sequencing model, and performing matrix inner product calculation to obtain a third result;
and performing weighted calculation on the attention weight and the third result to obtain an element self-attention expression under each head in the multi-head attention mechanism, and multiplying all the element self-attention expressions to obtain the interaction layer output.
5. The information recommendation method of claim 1, wherein before generating the feature embedding vector of the input information using the input layer, further comprising:
receiving a recommendation request, and determining a requester portrait corresponding to the recommendation request;
respectively combining the requester portraits with resource portraits of a plurality of alternative resources to obtain the request pairs;
and performing feature extraction on the request pair to obtain the input information.
6. The information recommendation method of claim 1, wherein outputting information recommendation results according to the ranking scores comprises:
and sequencing the alternative resources according to the sequence of the ranking scores from high to low to obtain a resource recommendation sequence, and outputting the information recommendation result according to the resource recommendation sequence.
7. The information recommendation method according to any one of claims 1 to 6, wherein the obtaining the ranking score of the input information by using the splicing module to splice the interaction layer output and the full connection layer output comprises:
and splicing the interaction layer output and the full-connection layer output by using the splicing module to obtain a splicing result, and performing weighted calculation on the splicing result to obtain the ranking score of the input information.
8. An information recommendation device is characterized in that the information recommendation device comprises a sequencing model, wherein the sequencing model comprises an input layer, a full connection layer, an interaction layer and a splicing module;
wherein the input layer is used for generating a feature embedding vector of input information; the interaction layer is used for splicing all the characteristic embedded vectors to obtain target vectors, and calculating a multi-head attention mechanism between the target vector execution elements to obtain interaction layer output; the full connection layer is used for carrying out matrix multiplication on the characteristic embedded vectors to obtain full connection layer output; the splicing module is used for splicing the interaction layer output and the full connection layer output to obtain the ranking score of the input information, and outputting an information recommendation result according to the ranking score.
9. An electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the information recommendation method according to any one of claims 1 to 7 when calling the computer program in the memory.
10. A storage medium having stored thereon computer-executable instructions which, when loaded and executed by a processor, carry out the steps of the information recommendation method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110430370.9A CN113111273A (en) | 2021-04-21 | 2021-04-21 | Information recommendation method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110430370.9A CN113111273A (en) | 2021-04-21 | 2021-04-21 | Information recommendation method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113111273A true CN113111273A (en) | 2021-07-13 |
Family
ID=76719439
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110430370.9A Pending CN113111273A (en) | 2021-04-21 | 2021-04-21 | Information recommendation method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113111273A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113570432A (en) * | 2021-07-28 | 2021-10-29 | 北京达佳互联信息技术有限公司 | Resource recommendation determining method, device, equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018212710A1 (en) * | 2017-05-19 | 2018-11-22 | National University Of Singapore | Predictive analysis methods and systems |
| CN110188236A (en) * | 2019-04-22 | 2019-08-30 | 北京达佳互联信息技术有限公司 | A kind of recommended method of music, apparatus and system |
| CN110196946A (en) * | 2019-05-29 | 2019-09-03 | 华南理工大学 | A kind of personalized recommendation method based on deep learning |
| CN111506820A (en) * | 2020-04-30 | 2020-08-07 | 喜大（上海）网络科技有限公司 | Recommendation model, method, device, equipment and storage medium |
| CN112667714A (en) * | 2021-03-17 | 2021-04-16 | 腾讯科技（深圳）有限公司 | User portrait optimization method and device based on deep learning and storage medium |
-
2021
- 2021-04-21 CN CN202110430370.9A patent/CN113111273A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018212710A1 (en) * | 2017-05-19 | 2018-11-22 | National University Of Singapore | Predictive analysis methods and systems |
| CN110188236A (en) * | 2019-04-22 | 2019-08-30 | 北京达佳互联信息技术有限公司 | A kind of recommended method of music, apparatus and system |
| CN110196946A (en) * | 2019-05-29 | 2019-09-03 | 华南理工大学 | A kind of personalized recommendation method based on deep learning |
| CN111506820A (en) * | 2020-04-30 | 2020-08-07 | 喜大（上海）网络科技有限公司 | Recommendation model, method, device, equipment and storage medium |
| CN112667714A (en) * | 2021-03-17 | 2021-04-16 | 腾讯科技（深圳）有限公司 | User portrait optimization method and device based on deep learning and storage medium |
Non-Patent Citations (3)
| Title |
|---|
| JUNJIE LIANG; JINLONG HU; SHOUBIN DONG; VASANT HONAVAR: "Top-N-Rank: A Scalable List-wise Ranking Method for Recommender Systems", 2018 IEEE INTERNATIONAL CONFERENCE ON BIG DATA (BIG DATA), 24 January 2019 (2019-01-24) * |
| 刘伟, 刘柏嵩, 王洋洋: "融合网络图模型和排序模型的论文个性化推荐算法", 数据通信, no. 2, 28 April 2019 (2019-04-28) * |
| 肖海涛, 何鹏, 曾诚: "基于排序学习的Top-k软件服务推荐方法", 计算机应用, vol. 38, 30 June 2018 (2018-06-30) * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113570432A (en) * | 2021-07-28 | 2021-10-29 | 北京达佳互联信息技术有限公司 | Resource recommendation determining method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110909182B (en) | Multimedia resource searching method, device, computer equipment and storage medium | |
| CN106326391B (en) | Multimedia resource recommendation method and device | |
| CN114265979B (en) | Method for determining fusion parameters, information recommendation method and model training method | |
| CN112052387B (en) | Content recommendation method, device and computer readable storage medium | |
| CN112508118B (en) | Target object behavior prediction method aiming at data offset and related equipment thereof | |
| CN108288208B (en) | Display object determination method, device, medium and equipment based on image content | |
| CN112232889A (en) | User interest portrait extension method, device, equipment and storage medium | |
| CN111767394A (en) | Abstract extraction method and device based on artificial intelligence expert system | |
| WO2023087914A1 (en) | Method and apparatus for selecting recommended content, and device, storage medium and program product | |
| CN111460290A (en) | Information recommendation method, device, equipment and storage medium | |
| CN112785005A (en) | Multi-target task assistant decision-making method and device, computer equipment and medium | |
| CN113407850A (en) | Method and device for determining and acquiring virtual image and electronic equipment | |
| CN111104572A (en) | Feature selection method and device for model training and electronic equipment | |
| CN113111273A (en) | Information recommendation method and device, electronic equipment and storage medium | |
| US20220292132A1 (en) | METHOD AND DEVICE FOR RETRIEVING IMAGE (As Amended) | |
| CN110827078B (en) | Information recommendation method, device, equipment and storage medium | |
| CN112069337A (en) | Picture processing method and device, electronic equipment and storage medium | |
| CN113836390A (en) | Resource recommendation method and device, computer equipment and storage medium | |
| CN114817845B (en) | Data processing method, device, electronic equipment and storage medium | |
| CN111783453B (en) | Text emotion information processing method and device | |
| CN114201625A (en) | Multimedia recommendation method and device, electronic equipment and storage medium | |
| CN113868523A (en) | Recommendation model training method, electronic device and storage medium | |
| CN113254788A (en) | Big data based recommendation method and system and readable storage medium | |
| CN112182382A (en) | Data processing method, electronic device, and medium | |
| CN112182379B (en) | Data processing method, electronic device, and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |