CN113450127A - Information display method and device, computer equipment and storage medium - Google Patents

Information display method and device, computer equipment and storage medium Download PDF

Info

Publication number
CN113450127A
CN113450127A CN202010225927.0A CN202010225927A CN113450127A CN 113450127 A CN113450127 A CN 113450127A CN 202010225927 A CN202010225927 A CN 202010225927A CN 113450127 A CN113450127 A CN 113450127A
Authority
CN
China
Prior art keywords
displayable information
time period
information
model
estimated
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010225927.0A
Other languages
Chinese (zh)
Inventor
李邦鹏
张梦一
符永顺
贺旭
陈功
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN202010225927.0A priority Critical patent/CN113450127A/en
Publication of CN113450127A publication Critical patent/CN113450127A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241—Advertisements
    • G06Q30/0273—Determination of fees for advertising
    • G06Q30/0275—Auctions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241—Advertisements
    • G06Q30/0277—Online advertisement

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Accounting & Taxation (AREA)
  • Development Economics (AREA)
  • Theoretical Computer Science (AREA)
  • Finance (AREA)
  • Strategic Management (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Game Theory and Decision Science (AREA)
  • Software Systems (AREA)
  • General Business, Economics & Management (AREA)
  • Economics (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Marketing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The application relates to an information display method, an information display device, computer equipment and a storage medium, and relates to the technical field of internet application. The method comprises the following steps: acquiring actual display results of all pieces of displayable information; obtaining the prediction display result of each displayable information corresponding to at least two sub-sequencing models respectively; acquiring calibration coefficients of the displayable information corresponding to the at least two sub-sequencing models respectively based on the actual display result of the displayable information and the prediction display result of the displayable information corresponding to the at least two sub-sequencing models respectively; in the subsequent time, each displayable information is displayed according to the displayable information sorting model and the calibration coefficient of each displayable information corresponding to at least two sub-sorting models respectively, and each displayable information is displayed according to the calibrated sorting result of the displayable information sorting model, so that the accuracy of the displayable information sorting model is improved, and the display effect of each displayable information is improved.

Description

Information display method and device, computer equipment and storage medium
Technical Field
The embodiment of the application relates to the technical field of internet application, in particular to an information display method, an information display device, computer equipment and a storage medium.
Background
With the continuous development of network and computer technologies, online advertisements become an effective means for media owners to convert user traffic into cash profits, and in order to improve the accuracy of online advertisement delivery, after ranking each online advertisement through a ranking model, the online advertisement is selectively displayed according to ranking results.
In the related art, during training of the ranking model, the individual models that make up the exposable information ranking model are typically calibrated by using a validation dataset of model evaluations. For example, the model training device may acquire a training data set and a validation data set, train out a ranking model from the training data set, and then input the validation data set into the ranking model to calibrate the ranking model.
However, the scheme shown in the related art is only suitable for calibrating a single-stage ranking model through a pre-acquired verification data set in a training process, is not suitable for a multi-stage ranking model, and cannot change the model in a subsequent model application process, so that the accuracy of the ranking model is poor.
Disclosure of Invention
The embodiment of the application provides an information display method, an information display device, computer equipment and a storage medium, which can improve the accuracy of sequencing and displaying displayable information, and the technical scheme is as follows:
in one aspect, an information display method is provided, and the method includes:
acquiring actual display results of all pieces of displayable information; the displayable information is information which is predicted by a displayable information sequencing model to receive the prediction probability of the operation of a specified user and is displayed according to the prediction probability obtained by prediction; the displayable information sequencing model comprises at least two cascaded sub-sequencing models, and the actual display result refers to actual data of the specified user operation received after each piece of displayable information is displayed in the display platform;
obtaining a prediction display result of each piece of displayable information corresponding to the at least two sub-sequencing models respectively, wherein the prediction display result is used for indicating the prediction probability of the at least two sub-sequencing models for predicting the piece of displayable information to receive the appointed user operation respectively;
acquiring calibration coefficients of the at least two sub-sequencing models respectively corresponding to the displayable information based on the actual display result of the displayable information and the predicted display result of the at least two sub-sequencing models respectively corresponding to the displayable information;
and correcting the subsequent prediction probabilities of the at least two sub-ordering models on the various displayable information based on the calibration coefficients of the at least two sub-ordering models.
In another aspect, an information presentation apparatus is provided, the apparatus comprising:
the first acquisition module is used for acquiring the actual display result of each piece of displayable information; the displayable information is information which is predicted by a displayable information sequencing model to receive the prediction probability of the operation of a specified user and is displayed according to the prediction probability obtained by prediction; the displayable information sequencing model comprises at least two cascaded sub-sequencing models, and the actual display result refers to actual data of the specified user operation received after each piece of displayable information is displayed in the display platform;
a second obtaining module, configured to obtain predicted display results of the at least two sub-ranking models respectively corresponding to the displayable information, where the predicted display results are used to indicate prediction probabilities of the at least two sub-ranking models predicting that the displayable information receives a specified user operation;
a third obtaining module, configured to obtain, based on an actual display result of each piece of displayable information and a predicted display result of each piece of displayable information corresponding to the at least two sub-ranking models, calibration coefficients of each piece of displayable information corresponding to the at least two sub-ranking models, respectively;
and the display module is used for correcting the subsequent prediction probability of the at least two sub-sequencing models on each displayable information based on the calibration coefficients of the at least two sub-sequencing models.
Optionally, the at least two sub-ranking models include a coarse ranking model and a fine ranking model cascaded after the coarse ranking model;
the second obtaining module includes:
the first obtaining sub-module is used for obtaining calibration coefficients of the various displayable information respectively corresponding to the carefully chosen sequencing model based on the actual display result of the various displayable information and the prediction display result of the various displayable information respectively corresponding to the carefully chosen sequencing model;
and the second obtaining submodule is used for obtaining the calibration coefficients of the various displayable information respectively corresponding to the roughing sequencing model based on the predicted display results of the various displayable information respectively corresponding to the roughing sequencing model and the predicted display results of the various displayable information respectively corresponding to the roughing sequencing model.
Optionally, the first obtaining sub-module includes:
a first obtaining unit, configured to obtain a first calibration coefficient initial value of the refined sorting model and a calibration coefficient target value of the refined sorting model respectively corresponding to each displayable information based on an actual display result of each displayable information and a predicted display result of each displayable information respectively corresponding to the refined sorting model;
and the second obtaining unit is used for obtaining the calibration coefficients of the various displayable information respectively corresponding to the carefully chosen sequencing model based on the first calibration coefficient initial value and the calibration coefficient target values of the various displayable information respectively corresponding to the carefully chosen sequencing model.
Optionally, the first obtaining unit includes:
the first obtaining subunit is configured to obtain a first calibration coefficient initial value of the refined ranking model based on an actual display result of each piece of displayable information in a first time period and a predicted display result of each piece of displayable information corresponding to the refined ranking model in the first time period, respectively;
and the second obtaining subunit is configured to obtain, based on an actual display result of each piece of displayable information in a second time period, a calibration coefficient target value corresponding to each piece of displayable information respectively to the fine sorting model.
Optionally, the actual display result includes an actual exposure number, an actual click number, and an actual conversion number; the prediction display result comprises a prediction click rate and a prediction conversion rate; the calibration coefficients comprise click rate calibration coefficients and conversion rate calibration coefficients;
the first obtaining subunit is configured to:
acquiring the sum of the actual clicks of each piece of displayable information in the first time period and the sum of the actual conversion numbers of each piece of displayable information in the first time period;
acquiring the sum of the first estimated clicks of each displayable information in the first time period and the sum of the first estimated conversions of each displayable information in the first time period, which correspond to the selected sorting model, based on the actual exposure number of each displayable information in the first time period and the estimated click rate and the estimated conversion rate of each displayable information in the first time period, which correspond to the selected sorting model respectively;
acquiring a first click rate calibration coefficient initial value of the selected sorting model based on the sum of actual click numbers of the displayable information in the first time period and the sum of the first estimated click numbers;
and acquiring a first conversion rate calibration coefficient initial value of the refined sorting model based on the sum of the actual conversion numbers of the various pieces of displayable information in the first time period and the sum of the first estimated conversion numbers.
Optionally, the actual display result includes an actual exposure number, an actual click number, and an actual conversion number; the prediction display result comprises a prediction click rate and a prediction conversion rate; the calibration coefficients comprise click rate calibration coefficients and conversion rate calibration coefficients;
the second obtaining subunit is configured to:
acquiring the actual number of clicks of each piece of displayable information in the second time period and the actual number of conversions of each piece of displayable information in the second time period;
acquiring the estimated click rate and the estimated conversion rate of each displayable information in the second time period, which correspond to the selected sorting model, based on the actual exposure number of each displayable information in the second time period and the estimated click rate and the estimated conversion rate of each displayable information in the second time period;
acquiring click rate calibration coefficient target values of the displayable information corresponding to the carefully selected sorting model respectively based on the actual number of clicks of the displayable information in the second time period and the estimated number of clicks of the displayable information in the second time period corresponding to the carefully selected sorting model;
and acquiring conversion rate calibration coefficient target values of the various displayable information respectively corresponding to the selected sorting model based on the actual conversion numbers of the various displayable information in the second time period and the estimated conversion numbers of the various displayable information in the second time period.
Optionally, the second obtaining unit is configured to perform weighted summation on the first calibration coefficient initial value and the calibration coefficient target values of the selected ranking model corresponding to the displayable information respectively, so as to obtain the calibration coefficients of the selected ranking model corresponding to the displayable information respectively.
Optionally, the second obtaining sub-module includes:
a third obtaining unit, configured to obtain a second calibration coefficient initial value of the rough sorting model and a calibration coefficient target value of the rough sorting model based on that each piece of displayable information corresponds to a predicted display result of the fine sorting model and each piece of displayable information corresponds to a predicted display result of the rough sorting model respectively;
and a fourth obtaining unit, configured to obtain, based on the second calibration coefficient initial value and the calibration coefficient target values of which the respective pieces of displayable information respectively correspond to the primary sorting model, calibration coefficients of which the respective pieces of displayable information respectively correspond to the coarse sorting model.
Optionally, the third obtaining unit includes:
a third obtaining subunit, configured to obtain a second calibration coefficient initial value of the rough sorting model based on that each piece of displayable information corresponds to a predicted display result of the fine sorting model in a third time period, and each piece of displayable information corresponds to a predicted display result of the rough sorting model in the third time period;
the fourth obtaining subunit is configured to obtain, based on the predicted display result that each piece of displayable information corresponds to the fine sorting model in a fourth time period, a calibration coefficient target value that each piece of displayable information corresponds to the rough sorting model.
Optionally, the prediction display result includes a prediction click rate and a prediction conversion rate; the calibration coefficients comprise click rate calibration coefficients and conversion rate calibration coefficients;
the third obtaining subunit is configured to obtain, based on the actual exposure number of each piece of displayable information in the third time period and the estimated click rate and the estimated conversion rate of each piece of displayable information in the third time period, a sum of second estimated click rates of each piece of displayable information in the third time period and a sum of second estimated conversion numbers of each piece of displayable information in the third time period, where the each piece of displayable information corresponds to the selected sorting model;
acquiring the sum of the third estimated clicks of each displayable information in the third time period and the sum of the third estimated conversions of each displayable information in the third time period, which correspond to the roughing sorting model, based on the actual exposure number of each displayable information in the third time period and the estimated click rate and the estimated conversion rate of each displayable information in the third time period, which correspond to the roughing sorting model respectively;
acquiring a second click rate calibration coefficient initial value of the roughing sequencing model based on the sum of second estimated click numbers of each piece of displayable information in the third time period and the sum of third estimated click numbers of each piece of displayable information in the third time period;
and acquiring a second conversion rate calibration coefficient initial value of the roughing sequencing model based on the sum of the second estimated conversion numbers of the various displayable information in the third time period and the sum of the third estimated conversion numbers of the various displayable information in the third time period.
Optionally, the prediction display result includes a prediction click rate and a prediction conversion rate; the calibration coefficients comprise click rate calibration coefficients and conversion rate calibration coefficients;
the fourth obtaining subunit is configured to obtain, based on the actual exposure number of each piece of displayable information in the fourth time period and the estimated click rate and the estimated conversion rate of each piece of displayable information in the fourth time period, where each piece of displayable information corresponds to the selected sorting model, an estimated click number and an estimated conversion number of each piece of displayable information in the fourth time period, which correspond to the selected sorting model;
acquiring the estimated click rate and the estimated conversion rate of each displayable information in the fourth time period, which correspond to the roughing sorting model, based on the actual exposure number of each displayable information in the fourth time period and the estimated click rate and the estimated conversion rate of each displayable information in the fourth time period, respectively;
acquiring click rate calibration coefficient target values of the displayable information corresponding to the roughing sorting model respectively based on the estimated click number of the displayable information in the fourth time period corresponding to the fine sorting model and the estimated click number of the displayable information in the fourth time period corresponding to the roughing sorting model;
and acquiring conversion rate calibration coefficient target values of the displayable information respectively corresponding to the roughing sequencing model based on the estimated conversion numbers of the displayable information in the fourth time period corresponding to the fine sequencing model and the estimated conversion numbers of the displayable information in the fourth time period corresponding to the roughing sequencing model.
Optionally, the fourth obtaining unit is configured to perform weighted summation on the second calibration coefficient initial value and the calibration coefficient target values of the rough sorting model corresponding to the respective pieces of displayable information, so as to obtain the calibration coefficients of the rough sorting model corresponding to the respective pieces of displayable information.
In another aspect, a computer device is provided, which includes a processor and a memory, wherein at least one instruction, at least one program, a set of codes, or a set of instructions is stored in the memory, and the at least one instruction, the at least one program, the set of codes, or the set of instructions is loaded and executed by the processor to implement the information presentation method as described above.
In another aspect, a computer-readable storage medium is provided, in which at least one instruction, at least one program, a set of codes, or a set of instructions is stored, which is loaded and executed by a processor to implement the information presentation method as described above.
The technical scheme provided by the application can comprise the following beneficial effects:
in the process of information display, the calibration coefficient of each sub-sorting model can be obtained through the actual display result of each displayable information and the prediction display result of each sub-sorting model with cascade connection of each sub-sorting model of each displayable information sorting model, so that the displayable information sorting model is calibrated, each displayable information is displayed based on the calibrated sorting result of the displayable information sorting model, the displayable information sorting model is updated based on the prediction effect and the actual display effect of the cascade multi-stage displayable information sorting model, and the accuracy of the displayable information sorting model is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.
FIG. 1 illustrates a workflow diagram of an ad exchange platform of a multi-level ranking model shown in an exemplary embodiment of the present application;
FIG. 2 is a diagram illustrating a relationship between an ad exchange platform, a media host, and an advertiser according to an exemplary embodiment of the present application;
FIG. 3 illustrates a flow diagram for training a multi-level ranking model according to an exemplary embodiment of the present application;
FIG. 4 is a schematic diagram illustrating the structure of an information presentation system in accordance with an exemplary embodiment;
FIG. 5 illustrates a flow chart of an information presentation method provided by an exemplary embodiment of the present application;
FIG. 6 illustrates a flow chart of an information presentation method provided by an exemplary embodiment of the present application;
FIG. 7 illustrates a real-time data flow multi-level calibration algorithm framework diagram shown in an exemplary embodiment of the present application;
FIG. 8 illustrates a block diagram of an information presentation device provided in an exemplary embodiment of the present application;
FIG. 9 is a block diagram illustrating the structure of a computer device according to an example embodiment.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.
The embodiment of the application provides an information display method, and the scheme can calibrate a displayable information sequencing model through the actual display effect of displayable information on the basis of the displayable information sequencing model trained by Artificial Intelligence (AI), so that the accuracy of the model is improved. For ease of understanding, the terms referred to in this application are explained below.
1) Artificial intelligence
Artificial intelligence is a theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the realization method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making.
The artificial intelligence technology is a comprehensive subject and relates to the field of extensive technology, namely the technology of a hardware level and the technology of a software level. The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.
2) Machine Learning (Machine Learning, ML)
Machine learning is a multi-field cross discipline, and relates to a plurality of disciplines such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and the like. The special research on how a computer simulates or realizes the learning behavior of human beings so as to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve the performance of the computer. Machine learning is the core of artificial intelligence, is the fundamental approach for computers to have intelligence, and is applied to all fields of artificial intelligence. Machine learning and deep learning generally include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.
3) Media host
The media owner refers to an entity possessing an internet platform, and may be a platform or an individual, such as a WeChat friend circle, a public number, a news application, an electronic journal, a magazine, and the like, and generally has a large user access volume (also referred to as user traffic). For the media owner, the user access may be converted to cash revenue by inserting ad slots in the platform.
4) Advertising owner
The advertiser refers to an entity which displays the advertisement of the advertiser through an advertisement space of an internet platform. For example, a brand owner who displays advertisements in a WeChat friend circle, or an entity who places advertisements in an advertisement board of an electronic newspaper and magazine, displays own advertisements at advertisement positions of a meeting networking platform to transmit information related to the owner, so that the purpose of attracting users is achieved.
5) Online advertising
The online advertisement, also referred to as internet advertisement, refers to an advertisement placed on an advertisement space (e.g., WeChat friend group, public number, news application, electronic newspaper and magazine, etc.) of an internet platform, and the online advertisement may include a picture advertisement, a text advertisement, a keyword advertisement, a ranking advertisement, a video advertisement, etc.
6) Advertisement trading platform (ADX, Ad Exchange)
An ad exchange platform refers to an entity that links a media host and an advertiser. It places the advertiser's ad on an ad spot provided by the media owner. In order to accurately deliver advertisements of advertisers to target groups, the advertisement trading platform generally collects information of users to portray the users, so as to accurately deliver advertisements according to interests, geographic positions or other data of the users.
7) Click Through Rate (CTR Click Through Rate)
The Click through rate refers to the Click through rate of the online advertisement, i.e. the actual number of clicks (Click) of the advertisement is divided by the display amount (Show content) of the advertisement, and is an important index for measuring the effectiveness of the online advertisement.
The calculation formula can be expressed as:
Figure BDA0002427630680000091
8) lightweight predictive Click Rate (LiteCTR, Lite Predict Click TthregouhRate)
The lightweight estimated click rate is the probability of the online advertisement system estimating the click after the online advertisement is put under a certain condition, is an important component of the ranking model, represents the estimated click rate model in the rough ranking under the multi-stage ranking model, and has low complexity.
9) Prediction Click Rate (pCTR, Predict Click Through Rate)
The estimated click rate is the probability of the on-line advertisement system estimating the click after the on-line advertisement is put under a certain condition, is an important component of the ranking model, represents the estimated click rate model in the fine ranking under the multi-stage ranking model, and has high complexity.
10) Conversion (CVR, Conversion Rate)
The conversion rate is an index for measuring the effect of the online advertisement, and refers to the conversion ratio from clicking the online advertisement to becoming an effective activated, registered or paid user, that is, the actual conversion times of the online advertisement is divided by the click rate of the online advertisement.
11) Light weight prediction Conversion Rate (LiteCVR, Lite Predict Conversion Rate)
The lightweight estimated conversion rate is the probability of conversion of the online advertisement system estimated after the online advertisement is clicked under a certain condition, is an important component of the ranking model, represents the conversion rate model in the rough ranking under the multi-level ranking model, and is low in complexity.
12) Estimated transformation ratio (pCVR, Predict Conversion Rate)
The pre-estimated conversion rate is the probability of conversion of the online advertisement system after the online advertisement is clicked under a certain condition, is an important component of the ranking model, represents the conversion rate model in the fine ranking under the multi-stage ranking model, and has high complexity.
13) Thousand people's costs (CPM, Cost Per Mille)
The thousand-person cost is the cost paid by an advertiser after the online advertisement is displayed to one thousand access users on an internet platform, and the calculation formula is as follows:
thousand people cost (advertising fee/number of people arriving) x 1000;
the advertisement fee/number of arriving people is usually expressed in a percentage form, and the number of arriving people refers to the number of effective access users of the online advertisement, for example, the cost paid by an advertiser for a certain online advertisement is 10000 yuan, and the number of arriving people is 5000000, so the cost of thousands of people is:
10000/5000000 × 1000 ═ 2 (yuan);
14) bid (bid)
Bids refer to the price that an advertiser bids on an online advertisement, typically a converted price in oCPM (Optimized Cost of thousand people).
15) oCPM (Optimized Cost of thousand people Per Mille)
The charging mode of the oCPM is the same as that of the CPM, and the value of each advertisement is judged by the online advertisement platform. In this mode, the advertisement group sets the conversion target and cost price of the advertisement, and the online advertisement platform optimizes the advertisement delivery according to the setting of the advertiser, thereby achieving the target as efficiently as possible. The charge of the online advertisement after every thousand times of display is positively correlated with the real-time bid of the online advertisement, wherein the real-time thousand-person cost of the advertisement is as follows:
real-time CPM bid × pCVR × pCTR;
16) multi-level ranking model
Due to the large number of online advertisements, based on engineering efficiency considerations, a plurality (e.g., 2, including roughing and refining) of pCTR, pCVR models are typically implemented to compute CPM for progressively screening online advertisements that best fit the interests of the user, advertiser, and media residence. Wherein:
roughly sorting and ordering real-time CPM (bid multiplied by LiteCTR multiplied by LiteCVR);
selecting and sorting real-time CPM (bid multiplied by pCTR multiplied by pCVR);
when the online advertisement is delivered on the internet of things platform, a bidding manner is generally adopted for delivery, and the bidding manner of the bidding manner is various, and an advertiser can select to offer a bid for exposure (thousands of times) (CPM, Cost Per mill), a bid for Click (CPC, Cost Per Click), or a bid for conversion (CPA, Cost Per Action). Different bidding modes have different applicable scenes, for example, the mainstream bidding mode of searching advertisements is CPC. And mobile Application (APP) advertisements are more concerned about the conversion cost, so the CPA mode is beneficial to promoting information providers to control the conversion cost.
With the development of advertising, in terms of bidding methods for converting bids, bidding methods such as opcm (optimized Cost Per mill), opca (optimized Cost Per action) and the like are gradually derived in addition to CPA. Invariably, CPA, oCPM, oCPA are bid on conversion. Obviously, in the process of advertisement putting according to the conversion bid, the advertisement trading platform needs to know the real conversion amount of the advertisement, can be used for deduction and balance control, and can also carry out real-time optimization on the advertisement effect according to the actual conversion amount, thereby putting the advertisement to a more suitable person.
The current advertisement trading platform sorts and screens advertisements put on advertisement positions of an internet platform through a multi-stage sorting model. Referring to fig. 1, a workflow diagram of an ad exchange platform of a multi-level ranking model according to an exemplary embodiment of the present application is shown, where the workflow of the ad exchange platform includes:
step 110, a user request is received.
The user request refers to a request, received by an advertisement trading platform, of a user to access an internet platform of a media owner and a request, received by the media owner, of the media owner to display an online advertisement, where the advertisement trading platform is used to connect the advertisement owner and the media owner, please refer to fig. 2, which shows a schematic diagram of a relationship among the advertisement trading platform, the media owner and the advertisement owner, shown in an exemplary embodiment of the present application, and as shown in fig. 2, the advertisement trading platform 220 puts an advertisement of the advertisement owner 210 on an advertisement slot of the media owner 230, and at the same time, can collect user information corresponding to the media owner to perform user portrayal, and performs targeted putting on the advertisement of the advertisement owner according to different user portrayals corresponding to different media owners.
Step 120, the advertisement is roughly sorted.
And when the advertisement trading platform obtains a user request, the LiteCTR and LiteCVR values of each advertisement in the roughing sequencing model are calculated by combining the user information and the advertisement information, and the roughing real-time bid of each advertisement, namely the roughing real-time CPM, is calculated according to the LiteCTR and LiteCVR values. And the advertisement trading platform sorts the advertisements according to the final rough-selection real-time CPM, and selects N advertisements with the most front sorting to feed back to the fine-selection sorting model, wherein N is a positive integer.
Step 130, advertisement selection and sorting.
When the advertisement trading platform obtains a user request, the pCTR and pCVR values of each advertisement in the selected sorting model are calculated by combining the user information and the advertisement information, and the real-time bid of each advertisement is calculated according to the pCTR and pCVR values. And the advertisement trading platform ranks the N advertisements fed back by the rough ranking model according to the final refined real-time bidding, namely the refined real-time CPM, selects M advertisements with the top ranking and sends the M advertisements to the media owner, wherein M is less than N, and M is a positive integer.
Step 140, the advertisement display is wined.
The media owner receives the advertisement information provided by the advertisement trading platform, namely M advertisements screened by the advertisement trading platform through the roughing sorting model and the fine sorting model, and the advertisements are displayed on the advertisement positions of the media owner. When the displayed advertisements accumulate enough display times in the media owner, the advertisement trading platform can charge corresponding fees to the advertisers according to the display times.
And 150, clicking and converting data to reflow.
The advertisement trading platform collects historical delivery records and click and conversion records of delivered advertisements. And checking the effect of the delivery system and carrying out the next iteration of the new model.
In the prior art, the advertisement roughing ordering and the advertisement fine-selecting ordering may be accomplished by a multistage ordering model composed of a roughing ordering model and a fine-selecting ordering model, and the multistage ordering model may be applied in a server of an advertisement trading platform, and a basic flow for predicting CTR and CVR is implemented by using a machine learning algorithm, please refer to fig. 3, which shows a flowchart of training the multistage ordering model according to an exemplary embodiment of the present application, and as shown in fig. 3, the flow may include the following steps:
and step 310, data processing.
Optionally, before data processing, the advertisement trading platform needs to collect user information and obtain advertisement information, where the user information may be behavior information of the user on each media host platform and other internet platforms, personal attribute information of the user, intelligent device information of the user, information of clicking of the user, advertisement conversion, and so on.
And carrying out operations such as denoising and missing value filling on the collected user information. And extracting interest features of corresponding users from user information by using an image processing algorithm, a natural language processing algorithm, a machine learning algorithm and the like, and extracting semantic features from characters and images of advertisements. Finally, converting the interest characteristics of the user and the semantic characteristics of the advertisement into a vector form which can be processed by a machine learning algorithm; generating a binary group for the behavior record of the user on the advertisement by combining the interest characteristics of the user and the semantic characteristics of the advertisement<X，y>Wherein X ═ X1，x2，…，xm) M characteristics including user and advertisement, such as attribute characteristics, behavior characteristics, user interest preference characteristics extracted from behavior, attribute characteristics, image characteristics, text characteristics, etcAnd the like, in the click rate estimation task, y ∈ {1, 0} for indicating whether the user clicked the advertisement, that is, when the user clicked the advertisement, y ═ 1, and when the user did not click the advertisement or closed the advertisement, y ═ 0. In the conversion rate estimation task, y indicates whether the user has converted the advertisement, wherein the conversion is that the user becomes an effective activated, registered or paid user of the strength corresponding to the advertisement by clicking the advertisement, when the user has converted the advertisement, y is 1, when the user has not converted the advertisement, that is, even if a certain user clicks the advertisement, the user has not performed the activation, registration or paid action, the user is still determined to be not converted the advertisement, and y is 0.
Step 320, model training.
By processing the behavior record of the user on the advertisement, the advertisement trading platform generates a mass of binary groups<Xi，yi>I-1, …, n, find an objective function f (x) using machine learning models such as logistic regression, random forests, gradient boosting trees, deep neural networks, and their variants, such that y-f (x). Since we do not know the specific form of f (x) in reality, machine learning algorithms generally find an optimal f (x) by solving the following optimization equation:
Figure BDA0002427630680000131
where L (-) is a loss function used to measure the difference between y and f (x), a general loss function can be defined as a function of logarithmic loss or cross-entropy loss. k is the number of advertisements, niThe number of samples of the binary group of the ith advertisement is selected, and the function which enables the solution of the optimization equation to be minimum is selected as the objective function, namely the multi-level ordering model is obtained.
Step 330, model evaluation.
Also called on-line evaluation, the quality evaluation is carried out on the multi-level sequencing model obtained in the training stage, and the quality evaluation comprises the operation performance, the estimation accuracy and the like.
Step 340, model calibration.
The multi-level ranking model obtained in the training phase after evaluation is calibrated using a validation set of model evaluations, where the validation set is an offline data set.
And step 350, deploying on line.
And further deploying the system to an advertisement trading platform to estimate the click rate and the conversion rate of the combination of the user and the advertisement through a multi-level sequencing model of quality evaluation and calibration.
In the process of training the multi-level ranking model, the model is calibrated by using the verification set of model evaluation during model calibration, so that the method is only suitable for calibrating a single-level ranking model by using the verification data set obtained in advance in the training process, is not suitable for the multi-level ranking model, and cannot change the model in the subsequent model application process, so that the accuracy of the ranking model is poor.
FIG. 4 is a block diagram illustrating an information presentation system in accordance with an exemplary embodiment. The system comprises: an information publisher terminal 420, an information presentation device 440, and a server 460.
The information publisher terminal 420 may be a terminal device with network access capabilities as well as user interface presentation and interaction functions. For example, the information publisher terminal 420 may be a PC (such as a laptop or desktop computer, etc.), a smart phone, a tablet computer, an e-book reader, or the like.
The information display device 440 may be a computer device including or externally connected to an information display platform, for example, the information display device 440 may be a mobile phone, a tablet computer, an electronic book reader, smart glasses, a smart watch, an MP3 player (Moving Picture Experts Group Audio Layer III, motion Picture Experts compression standard Audio Layer 3), an MP4 player (Moving Picture Experts Group Audio Layer IV, motion Picture Experts compression standard Audio Layer 4), a laptop, a desktop computer, a television set-top box, a game console, an outdoor advertisement display screen, a vehicle-mounted advertisement display screen, and so on.
The information distributor terminal 420 and the information presentation apparatus 440 are connected to the server 460 through a communication network, respectively. Optionally, the communication network is a wired network or a wireless network.
The server 460 is a server, or a plurality of servers, or a virtualization platform, or a cloud computing service center.
Server 460 may include, among other things, a resource vending platform 460a, an order management platform 460b, and an information management platform 460 c.
The resource selling platform 460a is configured to interact with the information publisher terminal 420, provide a service for querying orderable information display resources to the information publisher terminal 420 according to a user operation, and provide a service for locking/ordering the information display resources to the information publisher terminal 420.
The order management platform 460b is used for storing and maintaining orders for ordering information display resources by information publishers, for example, creating orders for ordering information display resources by information publishers, or deleting orders that have completed information display tasks (for example, reaching exposure times) from existing orders, and the like.
The information management platform 460c is configured to manage the display of the information published by the information publisher, for example, push the information published by the information publisher to the corresponding information display platform according to the type of the information display resource ordered by the information publisher, and count the exposure of the information in each information display platform.
Optionally, the system may further include a management device (not shown in fig. 4) connected to the server 460 through a communication network. Optionally, the communication network is a wired network or a wireless network.
Optionally, the wireless network or wired network described above uses standard communication techniques and/or protocols. The Network is typically the Internet, but may be any Network including, but not limited to, a Local Area Network (LAN), a Metropolitan Area Network (MAN), a Wide Area Network (WAN), a mobile, wireline or wireless Network, a private Network, or any combination of virtual private networks. In some embodiments, data exchanged over a network is represented using techniques and/or formats including Hypertext Mark-up Language (HTML), Extensible Markup Language (XML), and the like. All or some of the links may also be encrypted using conventional encryption techniques such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec). In other embodiments, custom and/or dedicated data communication techniques may also be used in place of, or in addition to, the data communication techniques described above.
Referring to fig. 5, a flowchart of an information presentation method provided by an exemplary embodiment of the present application is shown, where the information presentation method may be performed by a server, which may be the server shown in fig. 4, and the method may include the following steps:
step 510, acquiring actual display results of all the displayable information; the displayable information is information which is predicted by a displayable information sequencing model to receive the prediction probability of the operation of a specified user and is displayed according to the prediction probability obtained by prediction; the displayable information sequencing model comprises at least two cascaded sub-sequencing models, and the actual display result refers to actual data of specified user operation received after each piece of displayable information is displayed in the display platform.
The information that can be displayed refers to information that is actively recommended to the user by the terminal, where the information that can be displayed can be represented as information such as articles, advertisements, music, and the like.
The thousand-person cost is used for representing the cost which needs to be paid by a corresponding advertiser after the advertisement is displayed on the internet platform to one thousand users who access the internet platform. When the advertisement is released on the Internet of things platform, the advertisement is released in a bidding mode, the higher the cost of thousand people corresponding to the advertisement is, the higher the exposure rate of the advertisement in the Internet of things platform is, and the cost of thousand people is positively correlated with the bidding price, the predicted click rate and the predicted conversion rate, so that the accuracy of prediction of the predicted click rate and the predicted conversion rate influences the calculation result of the cost of thousand people, and further influences the actual display result of the displayable information.
The actual display result of each piece of displayable information refers to actual data of the designated operation of the display platform user, which can be received by each piece of displayable information, in the process of actually displaying each piece of displayable information on the display platform. The actual data can be a statistical result obtained by collecting data information of each piece of displayable information in a specified time period by the display platform and performing statistics on the data information.
Optionally, the displayable information ranking model may be a multi-level ranking model, where the training and the workflow of the multi-level ranking model may refer to related contents in the description contents in the embodiments shown in fig. 1 and fig. 3, and details are not repeated here.
The displayable information sorting model comprises at least two sub-sorting models which are cascaded, wherein the cascading means that hierarchical relation exists among the sub-sorting models, namely a previous sub-sorting model can affect a next sub-sorting model, and the fact that a calibration target and an output result of the next sub-sorting model are changed due to the fact that the calibration target and the output result of the previous sub-sorting model are changed can be understood.
And 520, obtaining a prediction display result of each piece of displayable information corresponding to the at least two sub-ordering models respectively, wherein the prediction display result is used for indicating the prediction probability of the at least two sub-ordering models for predicting the received appointed user operation on each piece of displayable information respectively.
The display prediction result of the at least two sub-ranking models corresponding to each displayable information means that after each displayable information is input into the displayable information ranking models, each sub-ranking model in the displayable information ranking models predicts the click rate and the conversion rate of the displayable information, the ranking result of each displayable information is predicted based on the predicted click rate and the predicted conversion rate and the bid price corresponding to each displayable information, each sub-ranking model can set a plurality of displayable information with the top ranking result to be displayed in the corresponding internet of things platform, and other displayable information can not be displayed.
Step 530, based on the actual display result of each displayable information and the predicted display result of each displayable information corresponding to at least two sub-ordering models, calibration coefficients of each displayable information corresponding to at least two sub-ordering models are obtained.
In the embodiment of the present application, the calibration means that a calibration target of each sub-ranking model is used as an output target of each sub-ranking model, and parameters in each sub-ranking model are adjusted, that is, a calibration coefficient is generated, so that a result of predicting the click rate and/or the conversion rate by each sub-ranking model is as close as possible to the calibration target corresponding to each sub-ranking model.
Because the sub-sequencing models have a cascade relation and have real-time performance, the calibration result corresponding to each sub-sequencing model is the calibration result generated according to the current real-time calibration target, and the unified calibration of the sub-sequencing models can be realized.
And 540, correcting the forecasting probability of each piece of displayable information subsequently by the at least two sub-ranking models based on the calibration coefficients of the at least two sub-ranking models.
Optionally, in a subsequent preset time period, according to calibration coefficients of at least two sub-ranking models corresponding to each displayable information, the click rate and/or the conversion rate output by each sub-ranking model are corrected to obtain corrected prediction probabilities of each displayable information, and based on the corrected prediction probabilities, the cost of thousands of people of each displayable information is calculated and ranked to display each displayable information, so that the display effect of each displayable information is improved.
In summary, in the information display method provided by the embodiment of the application, in the information display process, the displayable information ranking model can be calibrated through the actual display result of each displayable information and the calibration coefficient of each sub-ranking model of the prediction display result of each cascaded sub-ranking model of the displayable information ranking model, each displayable information is displayed based on the ranking result of the calibrated displayable information ranking model, and the displayable information ranking model is updated based on the prediction effect and the actual display effect of the cascaded multistage displayable information ranking model, so that the accuracy of the displayable information ranking model is improved, and the display effect of each displayable information is improved.
Referring to fig. 6, a flowchart of an information presentation method provided by an exemplary embodiment of the present application is shown, where the information presentation method may be performed by a server, which may be the server shown in fig. 4, and the method may include the following steps:
step 610, acquiring actual display results of all the displayable information; the displayable information is information which is predicted by a displayable information sequencing model to receive the prediction probability of the operation of a specified user and is displayed according to the prediction probability obtained by prediction; the displayable information sequencing model comprises at least two cascaded sub-sequencing models, and the actual display result refers to actual data of specified user operation received after each piece of displayable information is displayed in the display platform.
Optionally, the at least two sub-ranking models comprise a coarse ranking model and a fine ranking model cascaded after the coarse ranking model.
In the embodiment of the application, the information presentation method provided by the application is described in the way that the presentable information ranking model is composed of a roughing ranking model and a fine ranking model.
Optionally, the actual display result may include an actual exposure number, an actual click number, and an actual conversion number.
Step 620, obtaining a prediction display result of each piece of displayable information corresponding to the at least two sub-ranking models respectively, where the prediction display result is used to indicate a prediction probability that the at least two sub-ranking models predict and receive the operation of the specified user for each piece of displayable information respectively.
Optionally, the predicted display result may include a predicted click rate and a predicted conversion rate, and the predicted click rate and the predicted conversion rate may be the predicted click rate and the predicted conversion rate respectively corresponding to each sub-ranking model.
Step 630, based on the actual display result of each displayable information and the predicted display result of each displayable information corresponding to the selected ranking model, a calibration coefficient of each displayable information corresponding to the selected ranking model is obtained.
Optionally, the calibration coefficients include a click rate calibration coefficient and a conversion rate calibration coefficient.
Optionally, the process of obtaining the calibration coefficient of each displayable information corresponding to the selected ranking model may be implemented as follows:
step 631, obtaining a first calibration coefficient initial value of the selected ranking model and a calibration coefficient target value of each displayable information corresponding to the selected ranking model based on the actual displayed result of each displayable information and the predicted displayed result of each displayable information corresponding to the selected ranking model.
After the selected ranking model is applied to the rougher ranking model, the target value of the calibration coefficient for the current rougher ranking model is the ranking result of the selected ranking model in the previous time period. Referring to fig. 7, which illustrates a framework diagram of a real-time data stream multi-level calibration algorithm according to an exemplary embodiment of the present application, as shown in fig. 7, after receiving a user request (S71), an advertisement trading platform first performs a rough sorting of advertisements according to user information and advertisement information (S72), selects top N advertisements with the rough sorting of the top real-time CPM, and performs a fine sorting of the top N advertisements with the fine sorting of the top real-time CPM (S73), and selects top M advertisements with the fine sorting of the top real-time CPM to send the top M advertisements to a media host, for example. After the advertisement is pulled by the media owner (S74), exposure of the advertisement is performed (S75), and a click (S76) and conversion (S77) operation performed by the user on the advertisement is responded.
The roughing sorting is completed by a roughing sorting model, the fine sorting is completed by a fine sorting model, when the fine sorting model is calibrated, a calibration coefficient target value is calculated (S78) in real time according to at least one of the real-time exposure rate, the click rate and the conversion rate of the advertisement, and when the roughing sorting model is calibrated, the calibration coefficient target value is calibrated by taking the sorting result of the fine sorting model in the last time period as the calibration coefficient target value.
Optionally, the first calibration coefficient initial value of the refined ranking model may be obtained based on an actual display result of each displayable information in the first time period and a predicted display result of each displayable information in the first time period, which respectively corresponds to the refined ranking model.
The process of obtaining the initial value of the first calibration coefficient of the carefully chosen ranking model is the process of initializing the predicted display result of the displayable information ranking model, and since the predicted display result corresponding to each piece of displayable information is a value that changes with time or the change of the content of the media main platform, the predicted display result corresponding to each piece of displayable information needs to be initialized first in order to realize the calibration algorithm of the carefully chosen ranking model.
Optionally, the process of obtaining the initial value of the first calibration coefficient may be implemented as follows:
s6311, acquiring the sum of the actual number of clicks of each displayable information in the first time period and the sum of the actual number of conversions of each displayable information in the first time period.
Alternatively, the first period may be K hours closest to a point of time at which the sum of the number of hits is obtained, where K is a positive number.
S6312, based on the actual exposure number of each displayable information in the first time period and the estimated click rate and the estimated conversion rate of each displayable information in the first time period corresponding to the selected sorting model, the sum of the first estimated click number of each displayable information in the first time period and the sum of the first estimated conversion number of each displayable information in the first time period are obtained.
The estimated click rate refers to the probability that the displayable information ranking model estimates the click of a certain displayable information after being released under a certain condition, and the estimated conversion rate refers to the probability that the displayable information ranking model estimates the conversion of the certain displayable information after being released under a certain condition.
Optionally, a first sampling period in the first time period may be preset, and the estimated click rate and the estimated conversion number of each piece of displayable information are obtained according to the first sampling period, so as to calculate a total estimated click rate and a total estimated conversion number of each piece of displayable information in the first time period.
For example, if the preset sampling period within K hours is 5 minutes, the estimated click rate and the estimated conversion rate of each message are obtained every 5 minutes, and the estimated number of clicks and the estimated conversion number within 5 minutes are calculated, so that the total estimated number of clicks and the total estimated conversion number within K hours are calculated.
It should be noted that the sampling period may be set according to actual requirements, and the present application is not limited thereto.
S6313, acquiring a first click rate calibration coefficient initial value of the selected sorting model based on the sum of actual clicks of each displayable information in the first time period and the sum of first estimated clicks.
In the calculation process of the estimated click rate and the click rate, the estimated click rate is obtained by calculating the estimated click number divided by the actual exposure number, and the click rate is obtained by calculating the actual click number divided by the actual exposure number, so that when a first click rate calibration coefficient initial value is calculated, the actual exposure number can be divided by the total actual click rate divided by the total estimated click rate, and therefore, the calculation of the first click rate calibration coefficient initial value can be expressed as the sum of the actual click number in a first time period divided by the sum of the estimated click number in the first time period, and the calculation formula can be expressed as:
Figure BDA0002427630680000201
wherein fix _ pctr _ ratio (0) represents the first click rate calibration coefficient initial value, Σ clicki(0) Represents the sum of the actual number of clicks, Σ pctr, over a first time periodi(0) And the sum of the estimated clicks in the first time period is represented, and i represents the ith displayable information.
S6314, obtaining a first conversion rate calibration coefficient initial value of the refined sorting model based on the sum of the actual conversion numbers of the various pieces of displayable information in the first time period and the sum of the first estimated conversion numbers.
In the calculation process of the estimated conversion rate and the conversion rate, the estimated conversion rate is calculated by dividing the estimated conversion number by the actual click number, and the conversion rate is calculated by dividing the actual conversion number by the actual click number, so that when a first conversion rate calibration coefficient initial value is calculated, the actual click number can be divided when the total actual conversion rate is divided by the total estimated conversion rate, and therefore, the calculation of the first conversion rate calibration coefficient initial value can be expressed as the sum of the actual conversion number in the first time period and the estimated conversion number in the first time period, and the calculation formula can be expressed as:
Figure BDA0002427630680000202
wherein fix _ pcvr _ ratio (0) represents the first conversion calibration coefficient initial value, Σ convi(0) Represents the sum of the actual conversion numbers in the first time period, sigma pcvri(0) Representing the sum of the estimated conversions over the first time period, and i represents the ith presentable message.
Optionally, the calibration coefficient target values of the selected ranking model corresponding to the displayable information may be obtained based on the actual display result of the displayable information in the second time period.
Optionally, the second time period may be on the order of minutes to ensure real-time performance of the fine ranking model calibration algorithm, e.g., the first time period may be the last 1 minute.
Optionally, the target value of the calibration coefficient of the fine ranking model may include at least one of exposure rate, click rate and conversion rate of exposable information;
wherein, the exposure rate of the displayable information refers to the probability of the displayable information being displayed to the media master user; the click rate refers to the proportion of users who click the displayable information among the users who receive the push of certain promotion information; the conversion rate refers to the proportion of users who make specific behaviors to displayable information among users who click on the displayable information, and the characteristic behavior can be set according to actual conditions, for example, if the displayable information is an article, the user can click and read the article as the corresponding specific behavior, and the user can click and read the article and is recorded as conversion once; if the exposable information is an advertisement of a mobile APP, the specific behavior may be set as a behavior of downloading the APP for the user, and the APP is downloaded once and then recorded as converted once.
For example, in a period of time, 1000 users of a certain media owner are present, the advertising trading platform pushes a certain displayable information to one of the end users, which is equivalent to that the displayable information is exposed once, assuming that 100 users in the period of time have received the displayable information, the exposure rate of the displayable information is 10%, assuming that 50 end users out of 100 end users browsing the displayable information click the displayable information, the click rate of the displayable information is 50%, assuming that 10 end users out of 50 end users clicking the displayable information click the displayable information, the conversion rate of the displayable information is 20%.
In the embodiment of the present application, the information presentation method provided by the present application is described with the target values of the calibration coefficients of the fine ranking model as the click rate and the conversion rate.
Optionally, the process of obtaining the calibration coefficient target value of each displayable information corresponding to the selected ranking model may be implemented as follows:
and S6315, acquiring the actual number of clicks of each piece of displayable information in the second time period and the actual number of conversions of each piece of displayable information in the second time period.
Optionally, a second sampling period in a second time period may be preset, and an actual number of clicks and an actual number of conversions of each piece of displayable information in each second sampling period of the second time period are obtained according to the second sampling period, so as to calculate the actual number of clicks and the actual number of conversions of each piece of displayable information in the second time period, where a calculation formula may be represented as:
sum_click(t)＝∑clicki(t)
sum_conv(t)＝∑convi(t)
where t represents the second time period, and sum _ click (t) represents the actual number of clicks in the second time period, clicki(t) indicates that the actual hits in each second sampling period represent the actual number of transitions in each second sampling period.
And S6316, acquiring the estimated click rate and the estimated conversion rate of each displayable information in the second time period corresponding to the selected sorting model based on the actual exposure number of each displayable information in the second time period and the estimated click rate and the estimated conversion rate of each displayable information in the second time period.
Optionally, the estimated click rate and the estimated conversion rate of each piece of displayable information in each second sampling period of the second time period are obtained according to the second sampling period, so as to calculate the estimated click rate and the estimated conversion rate in each second sampling period by combining the actual exposure number of each piece of displayable information in each second sampling period in the second time period, thereby calculating the estimated click rate and the estimated conversion rate of each piece of displayable information in the second time period, and the calculation formula can be expressed as:
sum_pctr(t)＝∑pctri(t)
sum_pcvr(t)＝∑pcvri(t)
wherein t represents a second time period, sum _ pctr (t) represents an estimated click number in the second time period, pctri(t) represents the estimated number of hits in each second sampling period, sum _ pcvr (t) tableShowing the predicted number of conversions, pcvr, in the second time periodi(t) represents the estimated number of transitions in each second sampling period.
S6317, based on the actual number of clicks of each displayable information in the second time period and the estimated number of clicks of each displayable information in the second time period corresponding to the selected sorting model, obtaining a target value of the click rate calibration coefficient of each displayable information corresponding to the selected sorting model, where the calculation formula may be represented as:
Figure BDA0002427630680000221
wherein t represents a second time period, update _ pctr _ ratioi(t) represents the click rate calibration coefficient target value for the refined ranking model.
S6318, obtaining a conversion calibration coefficient target value of each displayable information corresponding to the selected ranking model respectively based on the actual conversion number of each displayable information in the second time period and the estimated conversion number of each displayable information in the second time period corresponding to the selected ranking model, and the calculation formula can be expressed as:
Figure BDA0002427630680000222
wherein t represents a second time period, update _ pcvr _ ratioi(t) represents the conversion calibration factor target value for the fine sort model.
Step 632, obtaining calibration coefficients corresponding to the selected ranking model respectively for each piece of displayable information, based on the first calibration coefficient initial value and the calibration coefficient target value corresponding to the selected ranking model respectively for each piece of displayable information.
Optionally, the first calibration coefficient initial value and the calibration coefficient target value of each displayable information corresponding to the carefully chosen ranking model are weighted and summed to obtain the calibration coefficient of each displayable information corresponding to the carefully chosen ranking model, and the calculation formula can be expressed as:
fix_pctr_ratioi(t+1)＝(1-α)*fix_pctr_ratioi(0)+α*update_pctr_ratioi(t)
fix_pcvr_ratioi(t+1)＝(1-α)*fix_pcvr_ratioi(t)+α*update_pcvr_ratioi(t)
where t +1 denotes a next period after the second period, fix _ pctr _ ratioi(t +1) and fix _ pcvr _ ratioi(t +1) represents a calibration coefficient for the refined ranking model, and α represents a smoothing coefficient to balance the first initial value of the click rate calibration coefficient with the target value of the click rate calibration coefficient for the refined ranking model, and to balance the first initial value of the conversion calibration coefficient with the target value of the conversion calibration coefficient for the refined ranking model.
In general, the smoothing coefficient α represents the response speed of the exponential smoothing model to time series changes, and determines the ability of the prediction model to smooth random errors. The value of α can be changed according to the service requirement, and the value thereof is related to the number of samples in the first time, and the more the number of general samples is, the larger the value α can be set.
And step 640, acquiring calibration coefficients of the coarse sorting model respectively corresponding to each displayable information based on the predicted display result of the fine sorting model respectively corresponding to each displayable information and the predicted display result of the coarse sorting model respectively corresponding to each displayable information.
Optionally, the process of obtaining the calibration coefficient of each displayable information corresponding to the coarse sorting model may be implemented as follows:
step 641, obtaining a second calibration coefficient initial value of the rough sorting model and a calibration coefficient target value of the initial sorting model corresponding to each displayable information based on the predicted display result of the fine sorting model corresponding to each displayable information and the predicted display result of the rough sorting model corresponding to each displayable information.
Optionally, the second calibration coefficient initial value of the rough sorting model may be obtained based on that each piece of displayable information corresponds to the predicted display result of the fine sorting model in the third time period, and each piece of displayable information corresponds to the predicted display result of the rough sorting model in the third time period.
Alternatively, the third time period may be the same time period as the first time period.
Optionally, the process of obtaining the first calibration coefficient of the rough sorting model may be implemented as follows:
s6411, acquiring the sum of the second estimated clicks of each displayable information in the third time period and the sum of the second estimated conversions of each displayable information in the third time period corresponding to the selected sorting model based on the actual exposure number of each displayable information in the third time period and the estimated click rate and the estimated conversion rate of each displayable information in the third time period.
S6412, acquiring the sum of the third estimated clicks of each displayable information in the third time period and the sum of the third estimated conversions of each displayable information in the third time period, which correspond to the rougher sorting model, based on the actual exposure number of each displayable information in the third time period and the estimated click rate and the estimated conversion rate of each displayable information in the third time period.
Optionally, the estimated click rate of the roughing sorting model may be referred to as a lightweight estimated click rate, and the estimated conversion rate of the roughing sorting model may be referred to as a lightweight estimated conversion rate.
Optionally, a third sampling period in a third time period may be preset, and the lightweight estimated click rate and the lightweight estimated conversion rate of each piece of displayable information are obtained according to the third sampling period to calculate a total lightweight estimated click count and a total lightweight estimated conversion count of each piece of displayable information in the third time period, where the third sampling period is the same as the first sampling period when the third time period and the first time period are the same time period.
S6413, obtaining a second click rate calibration coefficient initial value of the roughing sequencing model based on the sum of the second estimated click number of each piece of displayable information in the third time period and the sum of the third estimated click number of each piece of displayable information in the third time period.
In the calculation process of the lightweight estimated click rate and the estimated click rate, the lightweight estimated click rate is calculated by dividing the lightweight estimated click number (third estimated click number) by the actual exposure number, the estimated click rate is calculated by dividing the estimated click number by the actual exposure number, when a second click rate calibration coefficient initial value is calculated, the actual exposure number can be divided by dividing the total estimated click rate by the total lightweight estimated click rate, therefore, the calculation of the second click rate calibration coefficient initial value can be expressed as the sum of the estimated click number in a third time period divided by the sum of the lightweight estimated click number (third estimated click number) in the third time period, and the calculation formula can be expressed as:
Figure BDA0002427630680000241
wherein, fix _ lite _ ratio (0) represents the initial value of the second click rate calibration coefficient, Σ pctri(0) Represents the sum of the second estimated clicks, Sigma litctor, in the third time periodi(0) And the sum of the third estimated clicks in the third time period is shown, and i represents the ith displayable information.
S6414, acquiring a second conversion rate calibration coefficient initial value of the roughing sequencing model based on the sum of the second estimated conversion numbers of the displayable information in the third time period and the sum of the third estimated conversion numbers of the displayable information in the third time period.
In the calculation process of the lightweight estimated conversion rate and the estimated conversion rate, the lightweight estimated conversion rate is calculated by dividing a lightweight estimated conversion number (third estimated conversion number) by an actual click number, the estimated conversion rate is calculated by dividing the estimated conversion number by the actual click number, and when a second conversion rate calibration coefficient initial value is calculated, the actual click number can be divided when a total estimated conversion rate is divided by a total lightweight estimated conversion rate, so that the calculation of the second conversion rate calibration coefficient initial value can be expressed as the sum of the second estimated conversion number in a third time period and the sum of the third estimated conversion number in the third time period, and the calculation formula can be expressed as:
Figure BDA0002427630680000251
wherein fix _ litcvr _ ratio (0) represents the initial value of the second conversion calibration coefficient, Σ pcvri(0) And the second estimated conversion in the third time period represents the sum of the third estimated conversion numbers in the third time period, and i represents the ith displayable information.
Optionally, the server may obtain, based on the predicted display result of each displayable information corresponding to the carefully selected ranking model in the fourth time period, a calibration coefficient target value of each displayable information corresponding to the coarsely selected ranking model.
Optionally, the fourth time period may be the same time period as the second time period.
Optionally, the process of obtaining the calibration coefficient target value of each displayable information corresponding to the coarse sorting model may be implemented as follows:
s6415, obtaining the estimated click rate and the estimated conversion rate of each displayable information in the fourth time period, which correspond to the selected sorting model, based on the actual exposure number of each displayable information in the fourth time period and the estimated click rate and the estimated conversion rate of each displayable information in the fourth time period.
Optionally, a fourth sampling period in a fourth time period may be preset, and the estimated click rate and the estimated conversion rate of the selected sorting model of each piece of displayable information in each fourth sampling period in the fourth time period are obtained according to the fourth sampling period, so as to combine the actual exposure number of each piece of displayable information in each fourth sampling period in the fourth time period, thereby calculating the estimated click number of each piece of displayable information in the fourth time period and the estimated conversion number in the fourth time period. When the fourth time period is the same as the second time period, the fourth sampling period should also coincide with the second sampling period.
When the fourth time period is the same time period as the second time period, the calculation formula can be expressed as:
sum_pctri(t)＝∑pctri(t)
sum_pcvri(t)＝∑pcvri(t)
wherein t represents a fourth time period, sum _ pctri(t) represents the estimated number of hits, pctr, for the selected ranking model in the fourth time periodi(t) represents the number of estimated transformations in each sampling period representing the refined ranking model in each sampling period.
And S6416, acquiring the estimated click rate and the estimated conversion rate of each displayable information in the fourth time period, which correspond to the rougher sorting model, based on the actual exposure number of each displayable information in the fourth time period and the estimated click rate and the estimated conversion rate of each displayable information in the fourth time period.
Optionally, the lightweight estimated click rate and the lightweight estimated conversion rate of each displayable information in each fourth sampling period of the fourth time period are obtained according to the fourth sampling period, so as to calculate the lightweight estimated click rate and the lightweight estimated conversion rate in each fourth sampling period in combination with the actual exposure number of each displayable information in each fourth sampling period in the fourth time period, thereby calculating the lightweight estimated click rate and the lightweight estimated conversion rate of each displayable information in the fourth time period, where a calculation formula may be represented as:
sum_litectri(t)＝∑litectri(t)
sum_litecvri(t)＝∑litecvri(t)
wherein t represents a fourth time period, sum _ pctr (t) represents a light weight estimated transformation number in the fourth time period, pctri(t) represents the predicted lightweight transformation number in each fourth sampling period, sum _ pcvr (t) represents the predicted lightweight transformation number in the fourth sampling period, pcvri(t) represents the number of lightweight estimated conversions in each fourth sampling period.
S6417, based on the estimated number of clicks of each displayable information in the fourth time period corresponding to the carefully selected ranking model and the estimated number of clicks of each displayable information in the fourth time period corresponding to the coarsely selected ranking model, obtaining the target value of the click rate calibration coefficient of each displayable information corresponding to the coarsely selected ranking model, where the calculation formula can be expressed as:
Figure BDA0002427630680000261
wherein t represents a fourth time period, and update _ modifier _ ratio (t) represents the target value of the click rate calibration coefficient of the rough sort model.
S6418, based on the estimated conversion numbers of the respective displayable information in the fourth time period corresponding to the carefully selected ranking model and the estimated conversion numbers of the respective displayable information in the fourth time period corresponding to the coarsely selected ranking model, obtaining the conversion calibration coefficient target values of the respective displayable information corresponding to the coarsely selected ranking model, wherein the calculation formula can be expressed as:
Figure BDA0002427630680000262
wherein t represents a fourth time period, and update _ litcvr _ ratio (t) represents the conversion calibration coefficient target value of the coarse sorting model.
And 642, acquiring calibration coefficients of the coarse sorting model respectively corresponding to each piece of displayable information based on the second calibration coefficient initial value and the calibration coefficient target value of the initial sorting model respectively corresponding to each piece of displayable information.
Optionally, the second calibration coefficient initial value and the calibration coefficient target value of each displayable information corresponding to the rough sorting model are subjected to weighted summation to obtain the calibration coefficient of each displayable information corresponding to the rough sorting model, and the calculation formula may be expressed as:
fix_litectr_ratioi(t+1)＝(1-α)*fix_litectr_ratioi(t)+α*update_litectr_ratioi(t)
fix_litecvr_ratioi(t+1)＝(1-α)*fix_litecvr_ratioi(t)+α*update_litecvr_ratioi(t)
where t +1 denotes a next period after the fourth period, fix _ pctr _ ratioi(t +1) and fix _ pcvr _ ratioi(t +1) represents a calibration coefficient of the roughing sort model, α represents a smoothing coefficient for balancing the second initial value of the click rate calibration coefficient with the target value of the click rate calibration coefficient of the roughing sort model, and for balancing the first initial value of the conversion rate calibration coefficient with the target value of the conversion rate calibration coefficient of the roughing sort model.
And 650, correcting the subsequent prediction probability of each displayable information of the at least two sub-ranking models based on the calibration coefficients of the at least two sub-ranking models.
Optionally, after each selected ranking model of the displayable information ranking models is calibrated by combining the corresponding calibration coefficient, the formula for calculating the real-time cost of thousands of people is expressed as:
fine sort real-time CPM ═ bid × pCVR × pCTR × fix _ pCTR _ ratioi(t+1)*fix_pcvr_ratioi(t+1)
Wherein, pCVR XpCTR fix _ pCTR _ ratioi(t+1)*fix_pcvr_ratioi(t +1) represents the prediction probability of the selected sorting model after correction on each displayable information, and the selected sorting real-time thousand-person cost of each displayable information is calculated based on the corrected prediction probability, so that each displayable information is reordered, and the display mode of each displayable information is determined.
And (3) for each roughly selected sorting model capable of displaying the information sorting model, combining the corresponding calibration coefficient to calibrate, and then calculating the real-time cost of thousand people by the formula:
coarse sorting real-time CPM (bid) LiteCVR (LiteCTR) fix _ clipper _ ratioi(t+1)*fix_litecvr_ratioi(t+1)
Wherein LiteCVR LiteCTR fix Litector ratioi(t+1)*fix_litecvr_ratioi(t +1) represents the prediction probability of the corrected rough sorting model to each displayable information, and the rough sorting real-time thousand-person cost of each displayable information is calculated based on the corrected prediction probability, so that each displayable information is reordered, and each displayable information is determinedAnd (5) displaying the information.
Optionally, there is a difference between calibration coefficients of the selected ranking model corresponding to different displayable information, and there is a difference between calibration coefficients of the roughed ranking model corresponding to different displayable information.
In the practical application process of the displayable information sequencing model, the calibration process of the selected sequencing model and the roughing sequencing model is carried out in real time according to real-time data, so that the accuracy of the displayable information sequencing model is ensured.
To sum up, in the information display method provided by the embodiment of the application, in the information display process, the calibration coefficient of each sub-ranking model can be obtained through the actual display result of each displayable information and the predicted display result of each sub-ranking model with the cascade connection of each sub-ranking model of each displayable information ranking model, so as to calibrate each displayable information ranking model, and each displayable information is displayed based on the ranking result of the calibrated displayable information ranking model, so that the displayable information ranking model can be updated based on the prediction effect and the actual display effect of the cascade connection of the multistage displayable information ranking models, and therefore, the accuracy of the displayable information ranking model is improved, and the display effect of each displayable information is improved.
Referring to fig. 8, a block diagram of an information presentation apparatus provided by an exemplary embodiment of the present application is shown, where the information presentation apparatus may be applied to a server, and the server may be implemented as the server shown in fig. 1, and the apparatus may include:
a first obtaining module 810, configured to obtain an actual display result of each piece of displayable information; the displayable information is information which is predicted by a displayable information sequencing model to receive the prediction probability of the specified user operation and is displayed according to the predicted prediction probability; the displayable information sequencing model comprises at least two cascaded sub-sequencing models, and the actual display result refers to actual data of an appointed user operation received after each piece of displayable information is displayed in the display platform;
a second obtaining module 820, configured to obtain a predicted display result that each piece of displayable information corresponds to at least two sub-ranking models, where the predicted display result is used to indicate a predicted probability that the at least two sub-ranking models predict that each piece of displayable information receives a specified user operation;
a third obtaining module 830, configured to obtain, based on an actual display result of each piece of displayable information and a predicted display result of each piece of displayable information corresponding to at least two sub-ranking models, a calibration coefficient of each piece of displayable information corresponding to at least two sub-ranking models, respectively;
the display module 840 is configured to modify the prediction probability of each displayable information subsequently performed by the at least two sub-ranking models based on the calibration coefficients of the at least two sub-ranking models.
Optionally, the at least two sub-ranking models include a coarse ranking model and a fine ranking model cascaded after the coarse ranking model;
the second obtaining module 820 includes:
the first obtaining sub-module is used for obtaining calibration coefficients of various displayable information respectively corresponding to the carefully chosen sequencing model based on the actual display result of the various displayable information and the prediction display result of the carefully chosen sequencing model respectively corresponding to the various displayable information;
and the second obtaining submodule is used for obtaining the calibration coefficients of the various displayable information respectively corresponding to the roughing sequencing model based on the predicted display results of the various displayable information respectively corresponding to the roughing sequencing model and the predicted display results of the various displayable information respectively corresponding to the roughing sequencing model.
Optionally, the first obtaining sub-module includes:
the first obtaining unit is used for obtaining a first calibration coefficient initial value of the carefully chosen sequencing model and a calibration coefficient target value of the carefully chosen sequencing model corresponding to each displayable information respectively based on an actual display result of each displayable information and a prediction display result of each displayable information corresponding to the carefully chosen sequencing model respectively;
and the second obtaining unit is used for obtaining the calibration coefficient of each displayable information corresponding to the selected sorting model respectively based on the first calibration coefficient initial value and the calibration coefficient target value of each displayable information corresponding to the selected sorting model respectively.
Optionally, the first obtaining unit includes:
the first obtaining subunit is configured to obtain a first calibration coefficient initial value of the carefully chosen ranking model based on an actual display result of each piece of displayable information in a first time period and a predicted display result of each piece of displayable information corresponding to the carefully chosen ranking model in the first time period;
and the second obtaining subunit is used for obtaining the calibration coefficient target value of each displayable information corresponding to the selected sorting model respectively based on the actual display result of each displayable information in the second time period.
Optionally, the actual display result includes an actual exposure number, an actual click number, and an actual conversion number; the prediction display result comprises a prediction click rate and a prediction conversion rate; the calibration coefficient comprises a click rate calibration coefficient and a conversion rate calibration coefficient;
the first obtaining subunit is configured to:
acquiring the sum of the actual clicks of each piece of displayable information in a first time period and the sum of the actual conversion numbers of each piece of displayable information in the first time period;
acquiring the sum of the first estimated clicks of each displayable information in the first time period and the sum of the first estimated conversions of each displayable information in the first time period, which correspond to the selected sorting model, based on the actual exposure number of each displayable information in the first time period and the estimated click rate and the estimated conversion rate of each displayable information in the first time period, which correspond to the selected sorting model respectively;
acquiring a first click rate calibration coefficient initial value of the carefully selected sorting model based on the sum of actual click numbers of each piece of displayable information in a first time period and the sum of first estimated click numbers;
and acquiring a first conversion rate calibration coefficient initial value of the selected sorting model based on the sum of the actual conversion numbers of the various pieces of exposable information in the first time period and the sum of the first estimated conversion numbers.
Optionally, the actual display result includes an actual exposure number, an actual click number, and an actual conversion number; the prediction display result comprises a prediction click rate and a prediction conversion rate; the calibration coefficient comprises a click rate calibration coefficient and a conversion rate calibration coefficient;
the second obtaining subunit is configured to:
acquiring the actual number of clicks of each displayable information in a second time period and the actual number of conversions of each displayable information in the second time period;
acquiring the estimated click rate and the estimated conversion rate of each displayable information in the second time period, which correspond to the selected sorting model, based on the actual exposure number of each displayable information in the second time period and the estimated click rate and the estimated conversion rate of each displayable information in the second time period;
acquiring click rate calibration coefficient target values of the displayable information corresponding to the carefully selected sorting model respectively based on the actual click number of the displayable information in the second time period and the estimated click number of the carefully selected sorting model corresponding to the displayable information in the second time period;
and acquiring a conversion rate calibration coefficient target value of each displayable information corresponding to the selected sorting model respectively based on the actual conversion number of each displayable information in the second time period and the estimated conversion number of each displayable information in the second time period corresponding to the selected sorting model.
Optionally, the second obtaining unit is configured to perform weighted summation on the first calibration coefficient initial value and the calibration coefficient target value of each displayable information corresponding to the selected ranking model respectively, so as to obtain a calibration coefficient of each displayable information corresponding to the selected ranking model respectively.
Optionally, the second obtaining sub-module includes:
the third obtaining unit is used for obtaining a second calibration coefficient initial value of the rough sorting model and a calibration coefficient target value of the rough sorting model respectively corresponding to each displayable information based on the predicted display result of the fine sorting model respectively corresponding to each displayable information and the predicted display result of the rough sorting model respectively corresponding to each displayable information;
and the fourth obtaining unit is used for obtaining the calibration coefficient of each displayable information corresponding to the roughing sequencing model respectively based on the second calibration coefficient initial value and the calibration coefficient target value of each displayable information corresponding to the preliminary sequencing model respectively.
Optionally, the third obtaining unit includes:
the third obtaining subunit is configured to obtain a second calibration coefficient initial value of the rough sorting model based on that each piece of displayable information corresponds to the predicted display result of the fine sorting model in the third time period, and each piece of displayable information corresponds to the predicted display result of the rough sorting model in the third time period;
the fourth obtaining subunit is configured to obtain, based on the predicted display result that each piece of displayable information corresponds to the fine sorting model in the fourth time period, a calibration coefficient target value that each piece of displayable information corresponds to the rough sorting model.
Optionally, the prediction display result comprises a prediction click rate and a prediction conversion rate; the calibration coefficient comprises a click rate calibration coefficient and a conversion rate calibration coefficient;
the third obtaining subunit is configured to obtain, based on the actual exposure number of each piece of displayable information in the third time period and the estimated click rate and the estimated conversion rate of each piece of displayable information in the third time period, a sum of second estimated click numbers of each piece of displayable information in the third time period and a sum of second estimated conversion numbers of each piece of displayable information in the third time period, which correspond to the selected sorting model;
acquiring the sum of the third estimated clicks of each displayable information in the third time period and the sum of the third estimated conversions of each displayable information in the third time period, which correspond to the roughing sorting model, based on the actual exposure number of each displayable information in the third time period and the estimated click rate and the estimated conversion rate of each displayable information in the third time period;
acquiring a second click rate calibration coefficient initial value of the roughing sorting model based on the sum of second estimated click numbers of all the displayable information in a third time period and the sum of third estimated click numbers of all the displayable information in the third time period;
and acquiring a second conversion rate calibration coefficient initial value of the roughing sequencing model based on the sum of the second estimated conversion numbers of the displayable information in the third time period and the sum of the third estimated conversion numbers of the displayable information in the third time period.
Optionally, the prediction display result comprises a prediction click rate and a prediction conversion rate; the calibration coefficient comprises a click rate calibration coefficient and a conversion rate calibration coefficient;
the fourth obtaining subunit is configured to obtain, based on the actual exposure number of each piece of displayable information in the fourth time period and the estimated click rate and the estimated conversion rate of each piece of displayable information in the fourth time period, the estimated click number and the estimated conversion number of each piece of displayable information in the fourth time period, which correspond to the selected sorting model;
acquiring the estimated click rate and the estimated conversion rate of each displayable information in the fourth time period, which correspond to the roughing sorting model, based on the actual exposure number of each displayable information in the fourth time period and the estimated click rate and the estimated conversion rate of each displayable information in the fourth time period;
acquiring click rate calibration coefficient target values of the various displayable information respectively corresponding to the rough sorting model based on the estimated click number of the various displayable information in the fourth time period corresponding to the fine sorting model and the estimated click number of the various displayable information in the fourth time period corresponding to the rough sorting model;
and acquiring a conversion rate calibration coefficient target value of each displayable information corresponding to the roughing sequencing model respectively based on the estimated conversion number of each displayable information in the fourth time period corresponding to the fine sequencing model and the estimated conversion number of each displayable information in the fourth time period corresponding to the roughing sequencing model.
Optionally, the fourth obtaining unit is configured to perform weighted summation on the second calibration coefficient initial value and the calibration coefficient target value of each displayable information corresponding to the rough sorting model, so as to obtain a calibration coefficient of each displayable information corresponding to the rough sorting model.
To sum up, the information display device that this application embodiment provided, at the in-process of information display, can obtain the calibration coefficient of each sub-sequencing model through the actual show result of each displayable information and the prediction show result that has cascaded each sub-sequencing model of displayable information sequencing model to can show information sequencing model and calibrate, but show information each based on the sequencing result of the information sequencing model of can showing after the calibration, but realized that the prediction effect and the actual show effect based on cascaded multistage displayable information sequencing model update this displayable information sequencing model, thereby can show information sequencing model's accuracy, and then can show information's the bandwagon effect.
Fig. 9 is a block diagram illustrating the structure of a computer device 900 according to an example embodiment. The computer device may be implemented as a server in the above-mentioned aspects of the present application. The computer apparatus 900 includes a Central Processing Unit (CPU) 901, a system Memory 904 including a Random Access Memory (RAM) 902 and a Read-Only Memory (ROM) 903, and a system bus 905 connecting the system Memory 904 and the CPU 901. The computer device 900 also includes a basic Input/Output system (I/O system) 906, which facilitates the transfer of information between devices within the computer, and a mass storage device 907 for storing an operating system 913, application programs 914, and other program modules 915.
The basic input/output system 906 includes a display 908 for displaying information and an input device 909 such as a mouse, keyboard, etc. for user input of information. Wherein the display 908 and the input device 909 are connected to the central processing unit 901 through an input output controller 910 connected to the system bus 905. The basic input/output system 906 may also include an input/output controller 910 for receiving and processing input from a number of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, input-output controller 910 also provides output to a display screen, a printer, or other type of output device.
The mass storage device 907 is connected to the central processing unit 901 through a mass storage controller (not shown) connected to the system bus 905. The mass storage device 907 and its associated computer-readable media provide non-volatile storage for the computer device 900. That is, the mass storage device 907 may include a computer-readable medium (not shown) such as a hard disk or Compact Disc-Only Memory (CD-ROM) drive.
Without loss of generality, the computer-readable media may comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), flash Memory or other solid state Memory technology, CD-ROM, Digital Versatile Disks (DVD), or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage, or other magnetic storage devices. Of course, those skilled in the art will appreciate that the computer storage media is not limited to the foregoing. The system memory 904 and mass storage device 907 described above may be collectively referred to as memory.
The computer device 900 may also operate as a remote computer connected to a network via a network, such as the internet, in accordance with various embodiments of the present disclosure. That is, the computer device 900 may be connected to the network 912 through the network interface unit 911 coupled to the system bus 905, or the network interface unit 911 may be used to connect to other types of networks or remote computer systems (not shown).
The memory further includes at least one instruction, at least one program, a code set, or a set of instructions, which is stored in the memory, and the central processor 901 implements all or part of the steps in the calibration method for the exposable information ordering model shown in the above embodiments by executing the at least one instruction, the at least one program, the code set, or the set of instructions.
Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
It will be understood that the present application is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims (15)

1. An information presentation method, the method comprising:
acquiring actual display results of all pieces of displayable information; the displayable information is information which is predicted by a displayable information sequencing model to receive the prediction probability of the operation of a specified user and is displayed according to the prediction probability obtained by prediction; the displayable information sequencing model comprises at least two cascaded sub-sequencing models, and the actual display result refers to actual data of the specified user operation received after each piece of displayable information is displayed in the display platform;
obtaining a prediction display result of each piece of displayable information corresponding to the at least two sub-sequencing models respectively, wherein the prediction display result is used for indicating the prediction probability of the at least two sub-sequencing models for predicting the piece of displayable information to receive the appointed user operation respectively;
acquiring calibration coefficients of the at least two sub-sequencing models respectively corresponding to the displayable information based on the actual display result of the displayable information and the predicted display result of the at least two sub-sequencing models respectively corresponding to the displayable information;
and correcting the subsequent prediction probabilities of the at least two sub-ordering models on the various displayable information based on the calibration coefficients of the at least two sub-ordering models.
2. The method of claim 1, wherein the at least two sub-ranking models comprise a coarse ranking model and a fine ranking model cascaded after the coarse ranking model;
the obtaining calibration coefficients of the at least two sub-ranking models respectively corresponding to the each displayable information based on the actual display result of the each displayable information and the predicted display result of the at least two sub-ranking models respectively corresponding to the each displayable information includes:
acquiring calibration coefficients of the various displayable information respectively corresponding to the carefully chosen sorting model based on the actual display result of the various displayable information and the predicted display result of the various displayable information respectively corresponding to the carefully chosen sorting model;
and acquiring calibration coefficients of the various displayable information respectively corresponding to the roughing sequencing model based on the predicted display results of the various displayable information respectively corresponding to the fine sequencing model and the predicted display results of the various displayable information respectively corresponding to the roughing sequencing model.
3. The method according to claim 2, wherein the obtaining calibration coefficients of the respective exposable information corresponding to the selected ranking model based on the actual exhibition result of the respective exposable information and the predicted exhibition result of the respective exposable information corresponding to the selected ranking model comprises:
acquiring a first calibration coefficient initial value of the carefully chosen sorting model and a calibration coefficient target value of the carefully chosen sorting model respectively corresponding to each displayable information based on an actual display result of each displayable information and a predicted display result of each displayable information respectively corresponding to the carefully chosen sorting model;
and acquiring calibration coefficients of the various displayable information respectively corresponding to the selected sorting model based on the first calibration coefficient initial value and the calibration coefficient target value of the various displayable information respectively corresponding to the selected sorting model.
4. The method according to claim 3, wherein the obtaining a first calibration coefficient initial value of the refined ranking model and a calibration coefficient target value of the refined ranking model based on the actual display result of each exposable information and the predicted display result of each exposable information corresponding to the refined ranking model respectively comprises:
acquiring a first calibration coefficient initial value of the selected sorting model based on an actual display result of each piece of displayable information in a first time period and a predicted display result of each piece of displayable information corresponding to the selected sorting model in the first time period respectively;
and acquiring calibration coefficient target values of the various displayable information respectively corresponding to the selected sorting model based on the actual display result of the various displayable information in the second time period.
5. The method of claim 4, wherein the actual show results include an actual exposure number, an actual click number, and an actual conversion number; the prediction display result comprises a prediction click rate and a prediction conversion rate; the calibration coefficients comprise click rate calibration coefficients and conversion rate calibration coefficients;
the obtaining an initial value of a first calibration coefficient of the refined ranking model based on an actual display result of each piece of displayable information in a first time period and a predicted display result of each piece of displayable information in the first time period corresponding to the refined ranking model respectively comprises:
acquiring the sum of the actual clicks of each piece of displayable information in the first time period and the sum of the actual conversion numbers of each piece of displayable information in the first time period;
acquiring the sum of the first estimated clicks of each displayable information in the first time period and the sum of the first estimated conversions of each displayable information in the first time period, which correspond to the selected sorting model, based on the actual exposure number of each displayable information in the first time period and the estimated click rate and the estimated conversion rate of each displayable information in the first time period, which correspond to the selected sorting model respectively;
acquiring a first click rate calibration coefficient initial value of the selected sorting model based on the sum of actual click numbers of the displayable information in the first time period and the sum of the first estimated click numbers;
and acquiring a first conversion rate calibration coefficient initial value of the refined sorting model based on the sum of the actual conversion numbers of the various pieces of displayable information in the first time period and the sum of the first estimated conversion numbers.
6. The method of claim 4, wherein the actual show results include an actual exposure number, an actual click number, and an actual conversion number; the prediction display result comprises a prediction click rate and a prediction conversion rate; the calibration coefficients comprise click rate calibration coefficients and conversion rate calibration coefficients;
the obtaining, based on an actual display result of each piece of displayable information in a second time period, a calibration coefficient target value of each piece of displayable information corresponding to the selected ranking model, includes:
acquiring the actual number of clicks of each piece of displayable information in the second time period and the actual number of conversions of each piece of displayable information in the second time period;
acquiring the estimated click rate and the estimated conversion rate of each displayable information in the second time period, which correspond to the selected sorting model, based on the actual exposure number of each displayable information in the second time period and the estimated click rate and the estimated conversion rate of each displayable information in the second time period;
acquiring click rate calibration coefficient target values of the displayable information corresponding to the carefully selected sorting model respectively based on the actual number of clicks of the displayable information in the second time period and the estimated number of clicks of the displayable information in the second time period corresponding to the carefully selected sorting model;
and acquiring conversion rate calibration coefficient target values of the various displayable information respectively corresponding to the selected sorting model based on the actual conversion numbers of the various displayable information in the second time period and the estimated conversion numbers of the various displayable information in the second time period.
7. The method according to claim 3, wherein obtaining the calibration coefficient of each exposable information corresponding to the selected ranking model based on the initial value of the first calibration coefficient and the target value of the calibration coefficient of each exposable information corresponding to the selected ranking model comprises:
and carrying out weighted summation on the first calibration coefficient initial value and the calibration coefficient target value of each displayable information corresponding to the selected sorting model respectively to obtain the calibration coefficient of each displayable information corresponding to the selected sorting model respectively.
8. The method according to claim 2, wherein the obtaining calibration coefficients of the respective exposable information corresponding to the coarse sorting model based on the predicted exhibition results of the respective exposable information corresponding to the fine sorting model and the predicted exhibition results of the respective exposable information corresponding to the coarse sorting model comprises:
acquiring a second calibration coefficient initial value of the rough sorting model and a calibration coefficient target value of the rough sorting model respectively corresponding to each displayable information based on the predicted display result of the fine sorting model respectively corresponding to each displayable information and the predicted display result of the rough sorting model respectively corresponding to each displayable information;
and acquiring calibration coefficients of the rough sorting model respectively corresponding to the displayable information based on the second calibration coefficient initial value and the calibration coefficient target value of the initial sorting model respectively corresponding to the displayable information.
9. The method of claim 8, wherein obtaining a second calibration coefficient initial value of the coarse sorting model and a calibration coefficient target value of the primary sorting model based on the exposable information respectively corresponding to the predicted exhibition result of the fine sorting model and the exposable information respectively corresponding to the predicted exhibition result of the coarse sorting model comprises:
acquiring a second calibration coefficient initial value of the roughing sequencing model based on the predicted display result of each displayable information corresponding to the carefully-selected sequencing model in a third time period and the predicted display result of each displayable information corresponding to the roughing sequencing model in the third time period;
and acquiring calibration coefficient target values of the various displayable information respectively corresponding to the rough sorting model based on the predicted display results of the various displayable information respectively corresponding to the fine sorting model in a fourth time period.
10. The method of claim 9, wherein the predicted presentation results include a predicted click through rate and a predicted conversion rate; the calibration coefficients comprise click rate calibration coefficients and conversion rate calibration coefficients;
the obtaining a first calibration coefficient value of the rough sorting model based on the predicted display result of each displayable information corresponding to the fine sorting model in a third time period and the predicted display result of each displayable information corresponding to the rough sorting model in the third time period comprises:
acquiring the sum of second estimated clicks of each displayable information in the third time period and the sum of second estimated conversions of each displayable information in the third time period, which correspond to the selected sorting model, based on the actual exposure number of each displayable information in the third time period and the estimated click rate and the estimated conversion rate of each displayable information in the third time period, which correspond to the selected sorting model respectively;
acquiring the sum of the third estimated clicks of each displayable information in the third time period and the sum of the third estimated conversions of each displayable information in the third time period, which correspond to the roughing sorting model, based on the actual exposure number of each displayable information in the third time period and the estimated click rate and the estimated conversion rate of each displayable information in the third time period, which correspond to the roughing sorting model respectively;
acquiring a second click rate calibration coefficient initial value of the roughing sequencing model based on the sum of second estimated click numbers of each piece of displayable information in the third time period and the sum of third estimated click numbers of each piece of displayable information in the third time period;
and acquiring a second conversion rate calibration coefficient initial value of the roughing sequencing model based on the sum of the second estimated conversion numbers of the various displayable information in the third time period and the sum of the third estimated conversion numbers of the various displayable information in the third time period.
11. The method of claim 9, wherein the predicted presentation results include a predicted click through rate and a predicted conversion rate; the calibration coefficients comprise click rate calibration coefficients and conversion rate calibration coefficients;
the obtaining calibration coefficient target values of the various displayable information respectively corresponding to the rough sorting model based on the predicted display results of the various displayable information respectively corresponding to the fine sorting model in a fourth time period includes:
acquiring the estimated click rate and the estimated conversion rate of each displayable information in the fourth time period, which correspond to the selected sorting model, based on the actual exposure number of each displayable information in the fourth time period and the estimated click rate and the estimated conversion rate of each displayable information in the fourth time period, which correspond to the selected sorting model;
acquiring the estimated click rate and the estimated conversion rate of each displayable information in the fourth time period, which correspond to the roughing sorting model, based on the actual exposure number of each displayable information in the fourth time period and the estimated click rate and the estimated conversion rate of each displayable information in the fourth time period, respectively;
acquiring click rate calibration coefficient target values of the displayable information corresponding to the roughing sorting model respectively based on the estimated click number of the displayable information in the fourth time period corresponding to the fine sorting model and the estimated click number of the displayable information in the fourth time period corresponding to the roughing sorting model;
and acquiring conversion rate calibration coefficient target values of the displayable information respectively corresponding to the roughing sequencing model based on the estimated conversion numbers of the displayable information in the fourth time period corresponding to the fine sequencing model and the estimated conversion numbers of the displayable information in the fourth time period corresponding to the roughing sequencing model.
12. The method according to claim 3, wherein the obtaining calibration coefficients of the respective exposable information corresponding to the coarse sorting model based on the second calibration coefficient initial value and the respective exposable information corresponding to the calibration coefficient target value of the primary sorting model comprises:
and carrying out weighted summation on the second calibration coefficient initial value and the calibration coefficient target value of each displayable information corresponding to the roughing sequencing model respectively to obtain the calibration coefficient of each displayable information corresponding to the roughing sequencing model respectively.
13. An information presentation device, the device comprising:
the first acquisition module is used for acquiring the actual display result of each piece of displayable information; the displayable information is information which is predicted by a displayable information sequencing model to receive the prediction probability of the operation of a specified user and is displayed according to the prediction probability obtained by prediction; the displayable information sequencing model comprises at least two cascaded sub-sequencing models, and the actual display result refers to actual data of the specified user operation received after each piece of displayable information is displayed in the display platform;
a second obtaining module, configured to obtain predicted display results of the at least two sub-ranking models respectively corresponding to the displayable information, where the predicted display results are used to indicate prediction probabilities of the at least two sub-ranking models predicting that the displayable information receives a specified user operation;
a third obtaining module, configured to obtain, based on an actual display result of each piece of displayable information and a predicted display result of each piece of displayable information corresponding to the at least two sub-ranking models, calibration coefficients of each piece of displayable information corresponding to the at least two sub-ranking models, respectively;
and the display module is used for correcting the subsequent prediction probability of the at least two sub-sequencing models on each displayable information based on the calibration coefficients of the at least two sub-sequencing models.
14. A computer device comprising a processor and a memory, said memory having stored therein at least one instruction, at least one program, set of codes, or set of instructions, which is loaded and executed by said processor to implement the information presentation method according to any one of claims 1 to 12.
15. A computer-readable storage medium, having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, which is loaded and executed by a processor to implement the information presentation method according to any one of claims 1 to 12.
CN202010225927.0A 2020-03-26 2020-03-26 Information display method and device, computer equipment and storage medium Pending CN113450127A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010225927.0A CN113450127A (en) 2020-03-26 2020-03-26 Information display method and device, computer equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010225927.0A CN113450127A (en) 2020-03-26 2020-03-26 Information display method and device, computer equipment and storage medium

Publications (1)

Publication Number Publication Date
CN113450127A true CN113450127A (en) 2021-09-28

Family

ID=77807350

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010225927.0A Pending CN113450127A (en) 2020-03-26 2020-03-26 Information display method and device, computer equipment and storage medium

Country Status (1)

Country Link
CN (1) CN113450127A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024005712A1 (en) * 2022-06-27 2024-01-04 脸萌有限公司 Return evaluation method and apparatus, and device and storage medium

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024005712A1 (en) * 2022-06-27 2024-01-04 脸萌有限公司 Return evaluation method and apparatus, and device and storage medium

Similar Documents

Publication Publication Date Title
CN106372249B (en) A kind of clicking rate predictor method, device and electronic equipment
CN110796477A (en) Advertisement display method and device, electronic equipment and readable storage medium
CN111401963B (en) Method and device for training user behavior prediction model
CN111798280B (en) Multimedia information recommendation method, device and equipment and storage medium
CN106056398A (en) Method and system for collecting and presenting user feedback information for composite offering
CN111767466A (en) Recommendation information recommendation method and device based on artificial intelligence and electronic equipment
CN110598118A (en) Resource object recommendation method and device and computer readable medium
CN111429214B (en) Transaction data-based buyer and seller matching method and device
CN113011911B (en) Data prediction method and device based on artificial intelligence, medium and electronic equipment
CN112269943B (en) Information recommendation system and method
US20230316106A1 (en) Method and apparatus for training content recommendation model, device, and storage medium
CN113450127A (en) Information display method and device, computer equipment and storage medium
CN113128597B (en) Method and device for extracting user behavior characteristics and classifying and predicting user behavior characteristics
CN117114766A (en) Cost control factor determining method, device, equipment and storage medium
CN114897607A (en) Data processing method and device for product resources, electronic equipment and storage medium
CN116258526A (en) Advertisement putting method, advertisement putting device, computer equipment and storage medium
CN116069959A (en) Method and device for determining display data
CN115829647A (en) Multimedia information profit estimation method and device, electronic equipment and storage medium
CN113095883A (en) Video payment user prediction method and system based on deep cross attention network
CN113011912A (en) Media information processing method, device, electronic equipment and storage medium
CN114996435A (en) Information recommendation method, device, equipment and storage medium based on artificial intelligence
CN111917809B (en) Multimedia data pushing method and device
CN112819500B (en) Information display method, strategy network training equipment and storage medium
CN113723611B (en) Business factor generation method, device, equipment and medium based on causal inference
CN113011921B (en) Information pushing method and device, electronic equipment and readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 40052377

Country of ref document: HK

SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination