CN116188082A - Advertisement click rate estimating method based on factoring machine model - Google Patents

Advertisement click rate estimating method based on factoring machine model Download PDF

Info

Publication number
CN116188082A
CN116188082A CN202211633743.3A CN202211633743A CN116188082A CN 116188082 A CN116188082 A CN 116188082A CN 202211633743 A CN202211633743 A CN 202211633743A CN 116188082 A CN116188082 A CN 116188082A
Authority
CN
China
Prior art keywords
embedding
predicted value
estimated
layer
feature
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211633743.3A
Other languages
Chinese (zh)
Inventor
陆凯
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Life Insurance Company of China Ltd
Original Assignee
Ping An Life Insurance Company of China Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Life Insurance Company of China Ltd filed Critical Ping An Life Insurance Company of China Ltd
Priority to CN202211633743.3A priority Critical patent/CN116188082A/en
Publication of CN116188082A publication Critical patent/CN116188082A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/02Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241Advertisements
    • G06Q30/0251Targeted advertisements
    • G06Q30/0255Targeted advertisements based on user history
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/02Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241Advertisements
    • G06Q30/0242Determining effectiveness of advertisements

Landscapes

  • Business, Economics & Management (AREA)
  • Strategic Management (AREA)
  • Engineering & Computer Science (AREA)
  • Accounting & Taxation (AREA)
  • Development Economics (AREA)
  • Finance (AREA)
  • Economics (AREA)
  • Game Theory and Decision Science (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Marketing (AREA)
  • Physics & Mathematics (AREA)
  • General Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Machine Translation (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The utility model relates to the field of artificial intelligence, and provides an advertisement click rate prediction method based on a factoring machine model, which is characterized in that in a characteristic crossing layer of the factoring machine model, a plurality of characteristic crossing values corresponding to the same dimension in characteristic crossing embedding are accumulated, then a high-order characteristic crossing vector is determined by the characteristic crossing accumulated value, and then the high-order characteristic crossing vector is accumulated to obtain a click rate prediction value, so that the processing process of the high-order characteristic crossing is simplified, the processing process can be calculated on a graphic processor in parallel, and compared with the traditional advertisement click rate prediction method based on the factoring machine model, the high-order characteristic crossing information can be learned on the basis of reducing calculation time and memory occupation, and the prediction accuracy is improved.

Description

Advertisement click rate estimating method based on factoring machine model
Technical Field
The application relates to the technical field of artificial intelligence, in particular to an advertisement click rate estimating method based on a factorization machine model.
Background
With the rapid development of the internet, internet information has also exploded, and how to push personalized advertisements to users so that users can obtain interesting information from the internet environment with high information overload is one of the difficulties faced by large internet manufacturers.
Advertisement Click-Through Rate (CTR) is an important technical means for solving the problem, and CTR is used for predicting the probability of a user clicking a specific advertisement by analyzing historical data of user searching and clicking behaviors. The factorizer (Factorization Machine, FM) model is widely applied to the CTR field, but the related FM model can only perform second-order feature intersection, the estimated accuracy is not high, and exponential computing time and memory resources are required if high-order feature interaction is performed.
Therefore, in the advertisement click rate estimation method based on the FM model, how to learn the high-order feature intersection information on the basis of low resource occupation to improve the estimation accuracy becomes a technical problem to be solved urgently.
Disclosure of Invention
The embodiment of the application mainly aims to provide an advertisement click rate estimation method, device, electronic equipment and computer-readable storage medium based on a factorization machine model, which can learn high-order characteristic crossing information on the basis of low resource occupation to improve estimation accuracy.
In order to achieve the above objective, a first aspect of an embodiment of the present application provides an advertisement click rate estimation method based on a factorization machine model, where the method includes:
The method comprises the steps of obtaining advertisement data to be estimated and a trained factoring machine model, wherein the factoring machine model comprises an embedding layer, a logistic regression layer, a characteristic crossing layer and an output layer;
inputting the advertisement data to be estimated into the embedding layer to obtain a first single-dimensional feature embedding and a first feature cross embedding corresponding to the advertisement data to be estimated;
embedding and inputting the first single-dimensional features into the logistic regression layer to obtain a first predicted value corresponding to the advertisement data to be predicted;
embedding and inputting the first characteristic intersection into the characteristic intersection layer to obtain a second predicted value corresponding to the advertisement data to be predicted;
in the feature cross layer, accumulating a plurality of feature cross values corresponding to the same dimension in the first feature cross embedding to obtain a feature cross accumulated value, determining a high-order feature cross vector according to the feature cross accumulated value, and accumulating each element in the high-order feature cross vector to obtain a second predicted value corresponding to the advertisement data to be estimated;
and inputting the first predicted value and the second predicted value to the output layer to obtain a click rate predicted value corresponding to the advertisement data to be estimated.
According to some embodiments of the invention, the method for estimating advertisement click rate based on factorizer model, determining a higher-order feature cross vector according to the feature cross accumulated value includes:
obtaining a high-order feature cross vector, wherein the size of the high-order feature cross vector is (H, M), H is the length of a hidden vector, and M is a preset order;
updating each element in the high-order feature cross vector according to the feature cross accumulated value and the intermediate result representation;
each element in the high order feature cross vector is updated by the following formula:
Figure BDA0004006795420000021
wherein the A i,j For the elements on the j-th row of the ith column in the higher-order feature cross vector A, the following
Figure BDA0004006795420000022
Embedding E for the first feature intersection interaction The j-th dimension characteristic crossing value of the kth characteristic in the advertisement data to be estimated is L, and B is the intermediate result representation;
the intermediate result representation is determined by the following formula:
B=concat([1],[A 0 ,A 1 ,…,A i ,…A M-2 ])=[0,A 0 ,A 1 ,…,A i ,…,A M-2 ];
wherein the A i Is the i-th row element in the higher order feature cross vector a.
According to some embodiments of the invention, the method for estimating the click rate of the advertisement based on the factorizer model includes inputting the advertisement data to be estimated to the embedding layer to obtain a first single-dimensional feature embedding and a first feature cross embedding corresponding to the advertisement data to be estimated, including:
Inputting the advertisement data to be estimated into the embedding layer, and extracting features of the advertisement data to be estimated through the embedding layer to obtain a first single-dimensional feature embedding corresponding to the advertisement data to be estimated;
and carrying out feature cross processing on the first single-dimensional feature embedding based on a preset hidden vector to obtain a first feature cross embedding corresponding to the advertisement data to be estimated.
According to some embodiments of the invention, the method for estimating the click rate of the advertisement based on the factorizer model, wherein the step of embedding and inputting the first single-dimensional feature into the logistic regression layer to obtain a first predicted value corresponding to the advertisement data to be estimated includes:
inputting the first single-dimensional feature embedding into the logistic regression layer to perform dot product processing on a preset first model parameter and the first single-dimensional feature embedding through the logistic regression layer to obtain a dot product result;
obtaining a first predicted value corresponding to the advertisement data to be predicted according to the dot product result and a preset second model parameter;
the first predicted value is determined by the following formula:
P lr =<E single ,W lr >+b lr
wherein the E single Embedding the W for the first single-dimensional feature lr For the first model parameter, b lr For the second model parameters, the<·>Representing the dot product.
According to some embodiments of the invention, the method for estimating click rate of advertisement based on factoring machine model, where the step of inputting the first predicted value and the second predicted value to the output layer to obtain the predicted click rate value corresponding to the advertisement data to be estimated includes:
inputting the first predicted value and the second predicted value into the output layer, and mapping the first predicted value and the second predicted value through the output layer according to a preset activation function to obtain click rate predicted values corresponding to the advertisement data to be estimated;
the click rate prediction value is determined by the following formula:
p=sigmoid(P lr +P interaction );
wherein, P is the click rate predicted value corresponding to the advertisement data to be predicted, sigmoid (·) is an activation function, and lr for the first predicted value, the P intera□□ion Is the second predicted value.
According to the advertisement click rate estimation method based on the factoring machine model provided by some embodiments of the invention, the factoring machine model is obtained through training:
acquiring a click data sample set, wherein the click data sample set comprises a plurality of click data training samples;
Inputting a plurality of click data training samples to the embedding layer to obtain second single-dimensional feature embedding and second feature cross embedding corresponding to the click data training samples;
embedding and inputting the second single-dimensional features into the logistic regression layer to obtain a third predicted value corresponding to the click data training sample;
embedding and inputting the second characteristic cross into the characteristic cross layer to obtain a fourth predicted value corresponding to the click data training sample;
inputting the third predicted value and the fourth predicted value to the output layer to obtain a training predicted value corresponding to the click data training sample;
determining a loss value of the factoring machine model according to a preset label corresponding to the click data training sample, the training predicted value and a preset loss function;
and updating the model parameters of the factoring machine model based on the loss value until the training ending condition is met, so as to obtain the trained factoring machine model.
According to the advertisement click rate estimation method based on the factoring machine model provided by some embodiments of the present invention, the loss function is determined by the following formula:
L 1 =-ylog(p)-(1-y)log(1-p);
Wherein the L is 1 And taking the y as a preset label corresponding to the click data training sample, and the p as a training predicted value corresponding to the click data training sample.
To achieve the above object, a second aspect of the embodiments of the present application provides an advertisement click rate estimating device based on a factorization machine model, the device including:
the first acquisition module is used for acquiring advertisement data to be estimated and a trained factoring machine model, wherein the factoring machine model comprises an embedding layer, a logistic regression layer, a characteristic crossing layer and an output layer;
the feature embedding acquisition module is used for inputting the advertisement data to be estimated into the embedding layer to obtain a first single-dimensional feature embedding and a first feature cross embedding corresponding to the advertisement data to be estimated;
the first predicted value acquisition module is used for embedding and inputting the first single-dimensional characteristics into the logistic regression layer to obtain a first predicted value corresponding to the advertisement data to be estimated;
the second predicted value acquisition module is used for embedding and inputting the first characteristic intersection into the characteristic intersection layer to obtain a second predicted value corresponding to the advertisement data to be estimated;
In the feature cross layer, accumulating a plurality of feature cross values corresponding to the same dimension in the first feature cross embedding to obtain a feature cross accumulated value, determining a high-order feature cross vector according to the feature cross accumulated value, and accumulating each element in the high-order feature cross vector to obtain a second predicted value corresponding to the advertisement data to be estimated;
the estimating module is used for inputting the first predicted value and the second predicted value to the output layer to obtain the click rate predicted value corresponding to the advertisement data to be estimated.
To achieve the above object, a third aspect of the embodiments of the present application proposes an electronic device, the electronic device comprising a memory, a processor, a computer program stored on the memory and executable on the processor, the computer program implementing the method of the first aspect when executed by the processor.
To achieve the above object, a fourth aspect of the embodiments of the present application proposes a storage medium, which is a computer-readable storage medium, for computer-readable storage, the storage medium storing one or more computer programs executable by one or more processors to implement the method described in the first aspect.
The application provides an advertisement click rate estimating method, an advertisement click rate estimating device, an electronic device and a computer readable storage medium based on a factoring machine model, wherein the factoring machine model based advertisement click rate estimating method comprises the steps of obtaining advertisement data to be estimated and a trained factoring machine model, inputting the advertisement data to be estimated into an embedding layer to obtain first single-dimensional feature embedding and first feature cross embedding corresponding to the advertisement data to be estimated, and then inputting the first single-dimensional feature embedding into a logistic regression layer to obtain a first predicted value corresponding to the advertisement data to be estimated; and then, inputting the first feature cross embedding into a feature cross layer to obtain a second predicted value corresponding to the advertisement data to be estimated, in the feature cross layer, accumulating a plurality of feature cross values corresponding to the same dimension in the first feature cross embedding to obtain a feature cross accumulated value, determining a high-order feature cross vector according to the feature cross accumulated value, accumulating each element in the high-order feature cross vector to obtain a second predicted value corresponding to the advertisement data to be estimated, and finally, inputting the first predicted value and the second predicted value into an output layer to obtain a click rate predicted value corresponding to the advertisement data to be estimated. In the feature cross layer of the factoring machine model, a plurality of feature cross values corresponding to the same dimension in feature cross embedding are accumulated, then a high-order feature cross vector is determined by the feature cross accumulated value, and then the high-order feature cross vector is accumulated to obtain a click rate predicted value, so that the processing process of the high-order feature cross is simplified, parallel calculation can be performed on a graphics processor, and compared with the traditional advertising click rate prediction method based on the factoring machine model, the method can learn high-order feature cross information on the basis of reducing calculation time and memory occupation, and prediction accuracy is improved.
Drawings
FIG. 1 is a schematic flow chart of an advertisement click rate estimation method based on a factorization machine model according to an embodiment of the present application;
FIG. 2 is a schematic flow chart of the substeps of step S140 in FIG. 1;
FIG. 3 is a schematic flow chart of the substeps of step S120 in FIG. 1;
FIG. 4 is a schematic flow chart of the substeps of step S130 in FIG. 1;
FIG. 5 is a schematic flow chart of the substeps of step S150 in FIG. 1;
FIG. 6 is a flowchart of an advertisement click rate estimation method based on a factorization machine model according to another embodiment of the present application;
FIG. 7 is a schematic structural diagram of an advertisement click rate estimating device based on a factorization machine model according to an embodiment of the present application;
fig. 8 is a schematic hardware structure of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
It is noted that unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the present application only and is not intended to be limiting of the present application.
With the rapid development of the internet, internet information has also exploded, and how to push personalized advertisements to users so that users can obtain interesting information from the internet environment with high information overload is one of the difficulties faced by large internet manufacturers.
Advertisement Click-Through Rate (CTR) is an important technical means for solving the problem, and CTR is used for predicting the probability of a user clicking a specific advertisement by analyzing historical data of user searching and clicking behaviors. The factorizer (Factorization Machine, FM) model is widely applied to the CTR field, but the related FM model can only perform second-order feature intersection, the estimated accuracy is not high, and exponential computing time and memory resources are required if high-order feature interaction is performed.
Therefore, in the advertisement click rate estimation method based on the FM model, how to learn the high-order feature intersection information on the basis of low resource occupation to improve the estimation accuracy becomes a technical problem to be solved urgently.
Based on the above, the embodiment of the application provides an advertisement click rate estimation method, an advertisement click rate estimation device, an electronic device and a computer readable storage medium based on a factorization machine model, which can learn high-order characteristic crossing information on the basis of low resource occupation to improve the estimation accuracy.
The embodiment of the application provides an advertisement click rate estimation method, an advertisement click rate estimation device, an electronic device and a computer readable storage medium based on a factoring machine model, and specifically, the following embodiment is used for explaining, first, the advertisement click rate estimation method based on the factoring machine model in the embodiment of the application.
The embodiment of the application can acquire and process the related data based on the artificial intelligence technology. Among these, artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use knowledge to obtain optimal results.
Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a robot technology, a biological recognition technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.
The advertisement click rate estimation method based on the factorization machine model can be applied to a terminal, a server side and software running in the terminal or the server side. In some embodiments, the terminal may be a smart phone, tablet, notebook, desktop, etc.; the server side can be configured as an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, and a cloud server for providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, basic cloud computing services such as big data and artificial intelligent platforms and the like; the software may be an application or the like that implements the factoring machine model-based advertisement click rate estimation method, but is not limited to the above form.
The subject application is operational with numerous general purpose or special purpose computer system environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. The application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
Referring to fig. 1, fig. 1 shows a flowchart of an advertisement click rate estimation method based on a factoring machine model according to an embodiment of the present application, and as shown in fig. 1, the advertisement click rate estimation method based on a factoring machine model includes, but is not limited to, steps S110 to S150:
Step S110, advertisement data to be estimated and a trained factoring machine model are obtained, wherein the factoring machine model comprises an embedded layer, a logistic regression layer, a characteristic crossing layer and an output layer;
illustratively, the advertisement data to be estimated includes target user information including user attribute information such as gender, age, education level, income, occupation, etc., and target advertisement information; user interest information, such as advertisement information clicked or browsed by the user, web page information browsed by the user, purchase information of the user, and the like; geographic location information of the user; terminal equipment information of the user, and the like. The targeted advertising information includes: size information of the advertisement, placement position of the advertisement, type information of the advertisement, etc.
And inputting the advertisement data to be estimated into a trained factoring machine model to obtain a click rate estimated value of the target user on the target advertisement.
Step S120, inputting the advertisement data to be estimated into the embedding layer to obtain a first single-dimensional feature embedding and a first feature cross embedding corresponding to the advertisement data to be estimated;
in some embodiments, referring to fig. 3, fig. 3 shows a schematic flow chart of a sub-step of step S120 in fig. 1, and as shown in fig. 3, the inputting the advertisement data to be estimated into the embedding layer, to obtain a first single-dimensional feature embedding and a first feature cross embedding corresponding to the advertisement data to be estimated, includes, but is not limited to, step S310 and step S320:
Step S310, inputting the advertisement data to be estimated into the embedding layer, so as to extract the characteristics of the advertisement data to be estimated through the embedding layer, and obtaining a first single-dimensional characteristic embedding corresponding to the advertisement data to be estimated;
step S320, performing feature cross processing on the first single-dimensional feature embedding based on a preset hidden vector, to obtain a first feature cross embedding corresponding to the advertisement data to be estimated.
It can be understood that the advertisement data to be estimated includes a plurality of features including a numerical feature and a discrete feature, and the plurality of features are subjected to one-hot (one-hot) encoding by an embedding layer to obtain a high-dimensional sparse feature, and then the high-dimensional sparse feature is mapped into a first single-dimensional feature embedding (embedding), specifically, aiming at the discrete feature, the embedding corresponding to each feature value is obtained; for the numerical characteristics, the numerical characteristics are divided into barrels, namely discretization processing is carried out, and then the corresponding emmbedding of each barrel is taken.
For example, for the gender of the user in the advertisement data to be estimated, the unique thermal code may be used to embed a feature vector with a fixed length, for example, feature vector "01" indicates that the gender is male, and feature vector "10" indicates that the gender is female; for the age of the user, the price of the advertisement commodity and the sales volume of the advertisement commodity in the advertisement data to be estimated, through dividing the intervals, the single-hot encoding can be adopted to embed the advertisement data into feature vectors with fixed lengths, for example, the commodity prices are respectively represented by the feature vectors '0001', '0010', '0100' and '1000' above 1-200, 200-2000, 2000-20000 and 20000.
Each feature is provided with
Figure BDA0004006795420000081
Hidden vector v corresponding to H dimension i And letting the first feature cross-embed as:
Figure BDA0004006795420000082
specifically, the output of the embedded layer may be expressed as the firstFeature cross-embedding
Figure BDA0004006795420000083
Figure BDA0004006795420000084
The size of the vector is (L, H), L is the characteristic quantity of the advertisement data to be estimated, and H is the length of the hidden vector. First one-dimensional feature embedding->
Figure BDA0004006795420000085
A one-dimensional vector of size L.
Step S130, embedding and inputting the first single-dimensional characteristics into the logistic regression layer to obtain a first predicted value corresponding to the advertisement data to be estimated;
in some embodiments, referring to fig. 4, fig. 4 is a schematic flow chart illustrating a sub-step of step S130 in fig. 1, and as shown in fig. 4, the embedding and inputting the first single-dimensional feature into the logistic regression layer to obtain a first predicted value corresponding to the advertisement data to be estimated includes, but is not limited to, step S410 and step S420:
step S410, inputting the first single-dimensional feature embedding to the logistic regression layer, so as to obtain a dot product result by dot product processing on a preset first model parameter and the first single-dimensional feature embedding through the logistic regression layer;
step S420, obtaining a first predicted value corresponding to the advertisement data to be predicted according to the dot product result and a preset second model parameter;
The first predicted value is determined by the following formula:
P lr =<E single ,W lr >+b lr
wherein the E single Embedding the W for the first single-dimensional feature lr For the first model parameter, b lr For the second model parameters, the<·>Representing dot product, in particular, W lr Is a vector with the size of the hidden vector length H, b lr Are individual values.
Step S140, embedding and inputting the first characteristic intersection into the characteristic intersection layer to obtain a second predicted value corresponding to the advertisement data to be estimated;
in the feature cross layer, accumulating a plurality of feature cross values corresponding to the same dimension in the first feature cross embedding to obtain a feature cross accumulated value, determining a high-order feature cross vector according to the feature cross accumulated value, and accumulating each element in the high-order feature cross vector to obtain a second predicted value corresponding to the advertisement data to be estimated;
in some embodiments, referring to fig. 2, fig. 2 is a flowchart illustrating a method for estimating advertisement click rate based on a factorizer model according to another embodiment of the present application, as shown in fig. 2, wherein the determining a higher-order feature cross vector according to the feature cross accumulated value includes, but is not limited to, step S210 and step S220:
Step S210, obtaining a high-order feature cross vector, wherein the size of the high-order feature cross vector is (H, M), H is the length of a hidden vector, and M is a preset order;
step S220, each element in the high-order feature cross vector is updated according to the feature cross accumulated value and the intermediate result representation;
each element in the high order feature cross vector is updated by the following formula:
Figure BDA0004006795420000091
/>
wherein the A i,j For the elements on the j-th row of the ith column in the higher-order feature cross vector A, the following
Figure BDA0004006795420000092
Embedding E for the first feature intersection interaction The j-th dimension characteristic crossing value of the kth characteristic in the advertisement data to be estimated is L, and B is the intermediate result representation;
the intermediate result representation is determined by the following formula:
B=concat([1],[A 0 ,A 1 ,…,A i ,…A M-2 ])=[0,A 0 ,A 1 ,…,A i ,…,A M-2 ];
wherein the A i Is the i-th row element in the higher order feature cross vector a.
It will be appreciated that the first feature cross-embedding may be processed by the feature cross-layer by a predetermined order M, which may take on the values of [2, L ]]Initializing a high order feature cross vector a=0 of (H, M) and a second predicted value P interaction =0, element a thereof i,j Representing the i+1 order feature cross value at the point of the ith dimension up to the jth feature. And traversing the feature cross values of the 1 st to L features in the first feature cross embedding in the same dimension to obtain feature cross accumulated values, and then carrying out iterative computation according to the feature cross accumulated values and intermediate results to obtain feature cross vectors A of each dimension corresponding to the first feature cross embedding.
Each element in its higher order feature cross vector can also be updated by the following formula:
Figure BDA0004006795420000101
thereby updating P interaction
Figure BDA0004006795420000102
By simplifying the processing procedure of the high-order feature intersection, each element in the high-order feature intersection vector is updated in parallel by using the image processor, so that the time complexity of O (L) and the space complexity of O (MH) are realized.
And step S150, inputting the first predicted value and the second predicted value to the output layer to obtain the click rate predicted value corresponding to the advertisement data to be estimated.
In some embodiments, referring to fig. 5, fig. 5 shows a schematic flow chart of the substep of step S150 in fig. 1, and as shown in fig. 5, the inputting the first predicted value and the second predicted value to the output layer to obtain the click rate predicted value corresponding to the advertisement data to be estimated includes, but is not limited to, step S510:
step S510, inputting the first predicted value and the second predicted value to the output layer, so as to obtain a click rate predicted value corresponding to the advertisement data to be estimated by mapping the first predicted value and the second predicted value through the output layer according to a preset activation function;
the click rate prediction value is determined by the following formula:
p=sigmoid(P lr +P interaction );
Wherein, P is the click rate predicted value corresponding to the advertisement data to be predicted, sigmoid (·) is an activation function, and lr for the first predicted value, the P interaction Is the second predicted value.
It will be appreciated that click rate predictors are mapped between 0-1 by a sigmoid function.
In some embodiments, please refer to fig. 6, fig. 6 shows a flowchart of an advertisement click rate estimation method based on a factoring machine model according to another embodiment of the present application, and as shown in fig. 6, the factoring machine model is obtained through training:
step S610, a click data sample set is obtained, wherein the click data sample set comprises a plurality of click data training samples;
step S620, inputting a plurality of click data training samples to the embedding layer, so as to obtain a second single-dimensional feature embedding and a second feature cross embedding corresponding to the click data training samples;
step 630, embedding and inputting the second single-dimensional feature into the logistic regression layer to obtain a third predicted value corresponding to the click data training sample;
step S640, embedding and inputting the second feature intersection to the feature intersection layer to obtain a fourth predicted value corresponding to the click data training sample;
Step S650, inputting the third predicted value and the fourth predicted value to the output layer to obtain a training predicted value corresponding to the click data training sample;
step S660, determining a loss value of the factoring machine model according to a preset label corresponding to the click data training sample, the training predicted value and a preset loss function;
and step S670, updating the model parameters of the factoring machine model based on the loss value until the training ending condition is met, and obtaining the trained factoring machine model.
It can be understood that the click data sample set is subjected to the partition batch processing, one batch comprises a plurality of click data training samples, the click data training samples are effective user behavior sequences, training predicted values corresponding to the samples are obtained by inputting the sequences into a factorizer model, then loss values are calculated according to the training predicted values and sample preset labels, and gradients are calculated reversely and model parameters are updated.
In some embodiments, the loss function is determined by the following formula:
L 1 =-ylog(p)-(1-y)log(1-p);
wherein the L is 1 And taking the y as a preset label corresponding to the click data training sample, and the p as a training predicted value corresponding to the click data training sample.
In a specific embodiment, updating model parameters of the factoring machine model until a loss value obtained through the updated factoring machine model is smaller than a preset value, and obtaining a trained factoring machine model after training is finished; or acquiring a test sample set, acquiring the prediction accuracy of the factoring machine model through the test sample set, if the prediction accuracy is larger than a preset value, finishing training to obtain a trained factoring machine model, otherwise, continuing training the factoring machine model.
In some embodiments, the click rate predicted value of the target user on the target advertisement is obtained by inputting the advertisement data to be predicted into the trained factoring machine model, and then advertisement pushing is performed according to the click rate predicted value corresponding to the advertisement data to be predicted, for example, when the click rate predicted value of the target advertisement is higher than a preset value, the target advertisement is pushed to the target user, otherwise, the target advertisement is not pushed to the target user.
The method comprises the steps of obtaining advertisement data to be estimated and a trained factoring machine model, wherein the factoring machine model comprises an embedding layer, a logistic regression layer, a characteristic crossing layer and an output layer, inputting the advertisement data to be estimated into the embedding layer to obtain first single-dimensional characteristic embedding and first characteristic crossing embedding corresponding to the advertisement data to be estimated, and then inputting the first single-dimensional characteristic embedding into the logistic regression layer to obtain a first predicted value corresponding to the advertisement data to be estimated; and then, inputting the first feature cross embedding into a feature cross layer to obtain a second predicted value corresponding to the advertisement data to be estimated, in the feature cross layer, accumulating a plurality of feature cross values corresponding to the same dimension in the first feature cross embedding to obtain a feature cross accumulated value, determining a high-order feature cross vector according to the feature cross accumulated value, accumulating each element in the high-order feature cross vector to obtain a second predicted value corresponding to the advertisement data to be estimated, and finally, inputting the first predicted value and the second predicted value into an output layer to obtain a click rate predicted value corresponding to the advertisement data to be estimated. In the feature cross layer of the factoring machine model, a plurality of feature cross values corresponding to the same dimension in feature cross embedding are accumulated, then a high-order feature cross vector is determined by the feature cross accumulated value, and then the high-order feature cross vector is accumulated to obtain a click rate predicted value, so that the processing process of the high-order feature cross is simplified, parallel calculation can be performed on a graphics processor, and compared with the traditional advertising click rate prediction method based on the factoring machine model, the method can learn high-order feature cross information on the basis of reducing calculation time and memory occupation, and prediction accuracy is improved.
Referring to fig. 7, the embodiment of the present application further provides an advertisement click rate estimating device 100 based on a factoring machine model, where the advertisement click rate estimating device 100 based on the factoring machine model includes:
a first obtaining module 110, configured to obtain advertisement data to be estimated and a trained factorizer model, where the factorizer model includes an embedding layer, a logistic regression layer, a feature intersection layer, and an output layer;
the feature embedding obtaining module 120 is configured to input the advertisement data to be estimated to the embedding layer, so as to obtain a first single-dimensional feature embedding and a first feature cross embedding corresponding to the advertisement data to be estimated;
a first predicted value obtaining module 130, configured to embed and input the first single-dimensional feature into the logistic regression layer to obtain a first predicted value corresponding to the advertisement data to be estimated;
a second predicted value obtaining module 140, configured to embed and input the first feature intersection to the feature intersection layer, to obtain a second predicted value corresponding to the advertisement data to be estimated;
in the feature cross layer, accumulating a plurality of feature cross values corresponding to the same dimension in the first feature cross embedding to obtain a feature cross accumulated value, determining a high-order feature cross vector according to the feature cross accumulated value, and accumulating each element in the high-order feature cross vector to obtain a second predicted value corresponding to the advertisement data to be estimated;
The estimating module 150 is configured to input the first predicted value and the second predicted value to the output layer, so as to obtain a click rate predicted value corresponding to the advertisement data to be estimated.
It should be noted that, because the content of information interaction and execution process between modules of the above apparatus is based on the same concept as the method embodiment of the present application, specific functions and technical effects thereof may be referred to in the method embodiment section, and details are not repeated herein.
Referring to fig. 8, fig. 8 shows a hardware structure of an electronic device provided in an embodiment of the present application, where the electronic device includes:
the processor 210 may be implemented by a general-purpose CPU (Central Processing Unit ), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, etc., for executing a relevant computer program to implement the technical solutions provided in the embodiments of the present application;
the Memory 220 may be implemented in the form of a Read Only Memory (ROM), a static storage device, a dynamic storage device, or a random access Memory (Random Access Memory, RAM). The memory 220 may store an operating system and other application programs, and when the technical solution provided in the embodiments of the present disclosure is implemented by software or firmware, relevant program codes are stored in the memory 220, and the processor 210 invokes an advertisement click rate estimation method based on the factoring machine model to execute the embodiments of the present disclosure;
An input/output interface 230 for implementing information input and output;
the communication interface 240 is configured to implement communication interaction between the present device and other devices, and may implement communication in a wired manner (e.g. USB, network cable, etc.), or may implement communication in a wireless manner (e.g. mobile network, WIFI, bluetooth, etc.); and a bus 250 for transferring information between each of the components of the device (e.g., processor 210, memory 220, input/output interface 230, and communication interface 240);
wherein processor 210, memory 220, input/output interface 230, and communication interface 240 are communicatively coupled to each other within the device via bus 250.
The embodiment of the application also provides a storage medium, which is a computer readable storage medium and is used for computer readable storage, the storage medium stores one or more computer programs, and the one or more computer programs can be executed by one or more processors to realize the advertisement click rate estimation method based on the factorization machine model.
The memory is a computer-readable storage medium that can be used to store software programs as well as computer-executable programs. In addition, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory optionally includes memory remotely located relative to the processor, the remote memory being connectable to the processor through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The embodiments described in the embodiments of the present application are for more clearly describing the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application, and as those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.
The above described apparatus embodiments are merely illustrative, wherein the units illustrated as separate components may or may not be physically separate, i.e. may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
Those of ordinary skill in the art will appreciate that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.
The terms "first," "second," "third," "fourth," and the like in the description of the present application and in the above-described figures, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that embodiments of the present application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be understood that in this application, "at least one" means one or more, and "a plurality" means two or more. "and/or" for describing the association relationship of the association object, the representation may have three relationships, for example, "a and/or B" may represent: only a, only B and both a and B are present, wherein a, B may be singular or plural. The character "/" generally indicates that the context-dependent object is an "or" relationship. "at least one of" or the like means any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (one) of a, b or c may represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, c may be single or plural.
In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, and for example, the division of the above units is merely a logical function division, and there may be other manners of division in actual implementation, for example, multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described above as separate components may or may not be physically separate, and components shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, including multiple instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method of each embodiment of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing a program.
Preferred embodiments of the present application are described above with reference to the accompanying drawings, and thus do not limit the scope of the claims of the embodiments of the present application. Any modifications, equivalent substitutions and improvements made by those skilled in the art without departing from the scope and spirit of the embodiments of the present application shall fall within the scope of the claims of the embodiments of the present application.

Claims (10)

1. An advertisement click rate estimating method based on a factorization machine model is characterized by comprising the following steps:
the method comprises the steps of obtaining advertisement data to be estimated and a trained factoring machine model, wherein the factoring machine model comprises an embedding layer, a logistic regression layer, a characteristic crossing layer and an output layer;
inputting the advertisement data to be estimated into the embedding layer to obtain a first single-dimensional feature embedding and a first feature cross embedding corresponding to the advertisement data to be estimated;
embedding and inputting the first single-dimensional features into the logistic regression layer to obtain a first predicted value corresponding to the advertisement data to be predicted;
embedding and inputting the first characteristic intersection into the characteristic intersection layer to obtain a second predicted value corresponding to the advertisement data to be predicted;
in the feature cross layer, accumulating a plurality of feature cross values corresponding to the same dimension in the first feature cross embedding to obtain a feature cross accumulated value, determining a high-order feature cross vector according to the feature cross accumulated value, and accumulating each element in the high-order feature cross vector to obtain a second predicted value corresponding to the advertisement data to be estimated;
And inputting the first predicted value and the second predicted value to the output layer to obtain a click rate predicted value corresponding to the advertisement data to be estimated.
2. The method for estimating click rate of advertisement based on factorizer model according to claim 1, wherein said determining a higher-order feature cross vector according to said feature cross accumulated value comprises:
obtaining a high-order feature cross vector, wherein the size of the high-order feature cross vector is (H, M), H is the length of a hidden vector, and M is a preset order;
updating each element in the high-order feature cross vector according to the feature cross accumulated value and the intermediate result representation;
each element in the high order feature cross vector is updated by the following formula:
Figure FDA0004006795410000011
wherein the A i,j For the elements on the j-th row of the ith column in the higher-order feature cross vector A, the following
Figure FDA0004006795410000012
Embedding E for the first feature intersection interaction The j-th dimension characteristic crossing value of the kth characteristic in the advertisement data to be estimated is L, and B is the intermediate result representation;
the intermediate result representation is determined by the following formula:
B=concat([1],[A 0 ,A 1 ,…,A i ,…A M-2 ])=[0,A 0 ,A 1 ,…,A i ,…,A M-2 ];
wherein the A i Is the i-th row element in the higher order feature cross vector a.
3. The method for estimating click rate of advertisement based on factorization machine model according to claim 1, wherein said inputting the advertisement data to be estimated into the embedding layer to obtain the first single-dimensional feature embedding and the first feature cross embedding corresponding to the advertisement data to be estimated comprises:
inputting the advertisement data to be estimated into the embedding layer, and extracting features of the advertisement data to be estimated through the embedding layer to obtain a first single-dimensional feature embedding corresponding to the advertisement data to be estimated;
and carrying out feature cross processing on the first single-dimensional feature embedding based on a preset hidden vector to obtain a first feature cross embedding corresponding to the advertisement data to be estimated.
4. The method for estimating click rate of advertisement based on factorizer model according to claim 1, wherein the embedding the first single-dimensional feature into the logistic regression layer to obtain the first predicted value corresponding to the advertisement data to be estimated comprises:
inputting the first single-dimensional feature embedding into the logistic regression layer to perform dot product processing on a preset first model parameter and the first single-dimensional feature embedding through the logistic regression layer to obtain a dot product result;
Obtaining a first predicted value corresponding to the advertisement data to be predicted according to the dot product result and a preset second model parameter;
the first predicted value is determined by the following formula:
P lr =<E single ,W lr >+b lr
wherein the E single Embedding the W for the first single-dimensional feature lr For the first model parameter, b lr For the second model parameters, the<·>Representing the dot product.
5. The method for estimating click rate of advertisement based on factoring machine model according to claim 1, wherein said inputting the first predicted value and the second predicted value to the output layer to obtain the predicted click rate value corresponding to the advertisement data to be estimated comprises:
inputting the first predicted value and the second predicted value into the output layer, and mapping the first predicted value and the second predicted value through the output layer according to a preset activation function to obtain click rate predicted values corresponding to the advertisement data to be estimated;
the click rate prediction value is determined by the following formula:
p=sigmoid(P lr +P interaction );
wherein, P is the click rate predicted value corresponding to the advertisement data to be predicted, sigmoid (·) is an activation function, and lr for the first predicted value, the P interaction Is the second predicted value.
6. The method for estimating advertisement click rate based on factoring machine model according to claim 1, wherein said factoring machine model is trained by:
acquiring a click data sample set, wherein the click data sample set comprises a plurality of click data training samples;
inputting a plurality of click data training samples to the embedding layer to obtain second single-dimensional feature embedding and second feature cross embedding corresponding to the click data training samples;
embedding and inputting the second single-dimensional features into the logistic regression layer to obtain a third predicted value corresponding to the click data training sample;
embedding and inputting the second characteristic cross into the characteristic cross layer to obtain a fourth predicted value corresponding to the click data training sample;
inputting the third predicted value and the fourth predicted value to the output layer to obtain a training predicted value corresponding to the click data training sample;
determining a loss value of the factoring machine model according to a preset label corresponding to the click data training sample, the training predicted value and a preset loss function;
and updating the model parameters of the factoring machine model based on the loss value until the training ending condition is met, so as to obtain the trained factoring machine model.
7. The factorer model-based advertisement click rate estimation method of claim 6, wherein the loss function is determined by the following formula:
L 1 =-y log(p)-(1-y)log(1-p);
wherein the L is 1 And taking the y as a preset label corresponding to the click data training sample, and the p as a training predicted value corresponding to the click data training sample.
8. An advertisement click rate estimating method based on a factorization machine model, which is characterized in that the device comprises:
the first acquisition module is used for acquiring advertisement data to be estimated and a trained factoring machine model, wherein the factoring machine model comprises an embedding layer, a logistic regression layer, a characteristic crossing layer and an output layer;
the feature embedding acquisition module is used for inputting the advertisement data to be estimated into the embedding layer to obtain a first single-dimensional feature embedding and a first feature cross embedding corresponding to the advertisement data to be estimated;
the first predicted value acquisition module is used for embedding and inputting the first single-dimensional characteristics into the logistic regression layer to obtain a first predicted value corresponding to the advertisement data to be estimated;
the second predicted value acquisition module is used for embedding and inputting the first characteristic intersection into the characteristic intersection layer to obtain a second predicted value corresponding to the advertisement data to be estimated;
In the feature cross layer, accumulating a plurality of feature cross values corresponding to the same dimension in the first feature cross embedding to obtain a feature cross accumulated value, determining a high-order feature cross vector according to the feature cross accumulated value, and accumulating each element in the high-order feature cross vector to obtain a second predicted value corresponding to the advertisement data to be estimated;
the estimating module is used for inputting the first predicted value and the second predicted value to the output layer to obtain the click rate predicted value corresponding to the advertisement data to be estimated.
9. An electronic device, comprising:
at least one processor; the method comprises the steps of,
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores a computer program that is executed by the at least one processor to enable the at least one processor to perform the factorization machine model-based advertisement click rate estimation method of any one of claims 1 to 7.
10. A computer readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the factorization-model-based advertisement click rate estimation method according to any one of claims 1 to 7.
CN202211633743.3A 2022-12-19 2022-12-19 Advertisement click rate estimating method based on factoring machine model Pending CN116188082A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211633743.3A CN116188082A (en) 2022-12-19 2022-12-19 Advertisement click rate estimating method based on factoring machine model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211633743.3A CN116188082A (en) 2022-12-19 2022-12-19 Advertisement click rate estimating method based on factoring machine model

Publications (1)

Publication Number Publication Date
CN116188082A true CN116188082A (en) 2023-05-30

Family

ID=86447979

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211633743.3A Pending CN116188082A (en) 2022-12-19 2022-12-19 Advertisement click rate estimating method based on factoring machine model

Country Status (1)

Country Link
CN (1) CN116188082A (en)

Similar Documents

Publication Publication Date Title
CN109492772B (en) Method and device for generating information
CN104765873A (en) Method and device for determining similarity among users
CN108984555B (en) User state mining and information recommendation method, device and equipment
CN112395487B (en) Information recommendation method and device, computer readable storage medium and electronic equipment
CN112307352B (en) Content recommendation method, system, device and storage medium
CN109117442B (en) Application recommendation method and device
CN112288554B (en) Commodity recommendation method and device, storage medium and electronic device
KR101635283B1 (en) Method for analyzing data based on matrix factorization model and apparatus therefor
WO2023213157A1 (en) Data processing method and apparatus, program product, computer device and medium
CN115982463A (en) Resource recommendation method, device, equipment and storage medium
CN115204436A (en) Method, device, equipment and medium for detecting abnormal reasons of business indexes
CN116775875A (en) Question corpus construction method and device, question answering method and device and storage medium
CN117034963A (en) Model training method, translation method, prediction method, recommendation method and device
CN115828153A (en) Task prediction method, device, equipment and medium based on artificial intelligence
CN116188082A (en) Advertisement click rate estimating method based on factoring machine model
CN113704617A (en) Article recommendation method, system, electronic device and storage medium
CN113094584A (en) Method and device for determining recommended learning resources
CN112446738A (en) Advertisement data processing method, device, medium and electronic equipment
CN112231572A (en) User feature extraction method, device, equipment and storage medium
CN115222486B (en) Article recommendation model training method, article recommendation method, device and storage medium
CN114298118B (en) Data processing method based on deep learning, related equipment and storage medium
CN111768218B (en) Method and device for processing user interaction information
CN116186416A (en) Training method and device for article recommendation model, electronic equipment and storage medium
CN116957641A (en) Click behavior prediction method and device, electronic equipment and storage medium
CN117391713A (en) Information pushing method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination