CN114547417B

CN114547417B - Media resource ordering method and electronic equipment

Info

Publication number: CN114547417B
Application number: CN202210179843.7A
Authority: CN
Inventors: 戴文恺; 杨钊; 王慕天; 马丽芬; 张瑞昌; 白云龙; 韩艳
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2022-02-25
Filing date: 2022-02-25
Publication date: 2024-08-09
Anticipated expiration: 2042-02-25
Also published as: CN114547417A

Abstract

The disclosure provides a media resource ordering method and electronic equipment, relates to artificial intelligence technology, and particularly relates to the field of big data. The specific scheme is as follows: acquiring the association characteristics of a first user; inputting the associated features into a target click rate prediction model which is obtained in advance to predict to obtain a first predicted click rate vector, and inputting the associated features into a target multi-task learning model which is obtained in advance to predict to obtain a first predicted stay time length vector and a first predicted completion rate vector; for each candidate media resource, a second predicted click rate vector, a second predicted stay time length vector and a second predicted completion rate vector are obtained, and a first score of the candidate media resource is determined based on the first predicted click rate vector, the second predicted click rate vector, the first predicted stay time length vector, the second predicted stay time length vector, the first predicted completion rate vector and the second predicted completion rate vector; the accuracy of the first scoring can be improved, and the resource sorting is performed according to the first scoring, so that the accuracy of the resource sorting is improved.

Description

Media resource ordering method and electronic equipment

Technical Field

The disclosure relates to the technical field of artificial intelligence such as big data, in particular to a media resource ordering method and electronic equipment.

Background

With the development of the internet, it is more and more common to recommend media resources to users, in the recommendation process, media resources of interest are required to be recommended to the users in massive resources, and media resource sorting is an important ring in the recommendation process, so that the recommendation effect is directly affected.

In the current process of sorting media resources, a common method is to sort the media resources by using historical behavior data of users to be recommended.

Disclosure of Invention

The disclosure provides a media resource ordering method and electronic equipment.

In a first aspect, an embodiment of the present disclosure provides a media asset ordering method, the method including:

acquiring the association characteristics of a first user;

inputting the associated features into a target click rate prediction model which is obtained in advance to predict to obtain a first predicted click rate vector, and inputting the associated features into a target multi-task learning model which is obtained in advance to predict to obtain a first predicted stay time length vector and a first predicted completion rate vector;

For each candidate media resource of a plurality of candidate media resources, acquiring a second predicted click rate vector, a second predicted stay time length vector and a second predicted completion rate vector of the candidate media resource; and determining a first score for the candidate media asset based on the first predicted click rate vector, the second predicted click rate vector, the first predicted stay length vector, the second predicted stay length vector, the first predicted completion rate vector, and the second predicted completion rate vector;

the plurality of candidate media assets is ranked based on a first score for the plurality of candidate media assets.

In this embodiment, in determining the first score of the candidate media resource, the association feature of the first user is adopted, and the predicted click rate vector obtained by predicting the target click rate prediction model based on the association feature of the first user and the first predicted stay time length vector and the first predicted completion rate vector obtained by predicting the target multitask learning model based on the association feature of the first user are adopted, and the first score of the candidate media resource is determined based on the first predicted click rate vector, the second predicted click rate vector and the first predicted stay time length vector, so that the accuracy of the first score of the candidate media resource can be improved, and the ranking of the candidate media resources is realized by the first scores of the candidate media resources, so that the accuracy of the ranking of the candidate media resources is improved.

In a second aspect, an embodiment of the present disclosure provides a click rate prediction model training method, the method including:

Screening a plurality of users according to attribute characteristics of the users in the application to determine a first user set;

screening the first user set to obtain a second user set, wherein any user in the second user set has a first action on a first project associated media resource of the application;

Screening a first media resource set with a preset identifier from a plurality of media resources of the application, wherein the preset identifier is used for indicating that the media resources can be distributed in a second project;

Screening the first media resource set to obtain a second media resource set, wherein any media resource in the second media resource set has associated user behavior in a first item of the application;

constructing a first sample data set based on the association features of the second user set, the association features of the second item association user of the application, the resource features of the second media resource set, and the resource features of a third media resource set, wherein the third media resource set comprises media resources of the second item association user of the application subjected to a second action;

And training the initial click rate prediction model by using the first sample data set to obtain a target click rate prediction model.

In this embodiment, a first sample data set may be constructed according to the association feature of the second user set, the association feature of the second item-associated user of the application, the resource feature of the second media resource set, and the resource feature of the third media resource set, and then the initial click rate prediction model may be trained using the first sample data set to obtain the target click rate prediction model, so as to improve the performance of the target click rate prediction model.

In a third aspect, an embodiment of the present disclosure further provides a method for training a multitasking learning model, including:

screening a first media resource set with a preset identifier from a plurality of media resources of the application, wherein the preset identifier is used for indicating that the media resources can be distributed in a second item;

Constructing a second sample data set based on the association features of the second user set, the association features of the second item association user of the application, the resource features of the second media resource set, and the resource features of a third media resource set, wherein the third media resource set comprises media resources of the second item association user of the application subjected to a second action;

and training the initial multi-task learning model by using the second sample data set to obtain a target multi-task learning model.

In this embodiment, a second sample data set may be constructed by using the association feature of the second user set, the association feature of the second item-associated user of the application, the resource feature of the second media resource set, and the resource feature of the third media resource set, and then the initial multi-task learning model may be trained by using the second sample data set to obtain the target multi-task learning model, so as to improve the performance of the target multi-task learning model.

In a fourth aspect, an embodiment of the present disclosure provides a media asset ordering apparatus, the apparatus comprising:

The first acquisition module is used for acquiring the association characteristics of the first user;

The prediction module is used for inputting the associated features into a pre-acquired target click rate prediction model to predict to obtain a first predicted click rate vector, and inputting the associated features into a pre-acquired target multi-task learning model to predict to obtain a first predicted stay time length vector and a first predicted completion rate vector;

the scoring determining module is used for obtaining a second predicted click rate vector, a second predicted stay time length vector and a second predicted completion rate vector of each candidate media resource of the plurality of candidate media resources; and determining a first score for the candidate media asset based on the first predicted click rate vector, the second predicted click rate vector, the first predicted stay length vector, the second predicted stay length vector, the first predicted completion rate vector, and the second predicted completion rate vector;

And a ranking module configured to rank the plurality of candidate media assets based on a first score of the plurality of candidate media assets.

In a fifth aspect, an embodiment of the present disclosure provides a click rate prediction model training apparatus, the apparatus including:

The first user determining module is used for screening a plurality of users according to attribute characteristics of the plurality of users in the application to determine a first user set;

the second user determining module is used for screening the first user set to obtain a second user set, and any user in the second user set has a first action on the first project associated media resource of the application;

A first resource screening module, configured to screen a first media resource set with a preset identifier from a plurality of media resources of the application, where the preset identifier is used to indicate that the media resource can be issued and distributed in a second item;

A second resource screening module, configured to screen the first media resource set to obtain a second media resource set, where any media resource in the second media resource set has an associated user behavior in the first item of the application;

a first sample construction module, configured to construct a first sample data set based on the association feature of the second user set, the association feature of the second item association user of the application, the resource feature of the second media resource set, and the resource feature of a third media resource set, where the third media resource set includes media resources for which the second item association user of the application has performed a second action;

and the first training module is used for training the initial click rate prediction model by using the first sample data set to obtain a target click rate prediction model.

In a sixth aspect, an embodiment of the present disclosure provides a multi-task learning model training device, including:

the third user determining module is used for screening the plurality of users according to the attribute characteristics of the plurality of users in the application and determining a first user set;

A fourth user determining module, configured to screen the first user set to obtain a second user set, where any user in the second user set has a first behavior on a first project-associated media resource of the application;

A third resource screening module, configured to screen a first media resource set with a preset identifier from a plurality of media resources of the application, where the preset identifier is used to indicate that the media resources can be distributed in a second item;

a fourth resource screening module, configured to screen the first media resource set to obtain a second media resource set, where any media resource in the second media resource set has an associated user behavior in the first item of the application;

A second sample construction module, configured to construct a second sample data set based on the association feature of the second user set, the association feature of the second item association user of the application, the resource feature of the second media resource set, and the resource feature of a third media resource set, where the third media resource set includes media resources for which the second item association user of the application has performed a second action;

And the second training module is used for training the initial multi-task learning model by using the second sample data set to obtain a target multi-task learning model.

In a seventh aspect, an embodiment of the present disclosure further provides an electronic device, including:

At least one processor; and

A memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the media asset ordering method as provided in the first aspect or the click-through rate prediction model training method as provided in the second aspect or the multitasking learning model training method as provided in the third aspect of the present disclosure.

In an eighth aspect, an embodiment of the present disclosure further provides a non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the media asset ordering method as provided in the first aspect or the click-through rate prediction model training method as provided in the second aspect or the multitasking learning model training method as provided in the third aspect of the present disclosure.

In a ninth aspect, an embodiment of the present disclosure provides a computer program product comprising a computer program which, when executed by a processor, implements the media asset ordering method as provided in the first aspect or the click-through rate prediction model training method as provided in the second aspect or the multitasking learning model training method as provided in the third aspect of the present disclosure.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.

Drawings

The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

FIG. 1 is a flow diagram of a media asset ordering method according to one embodiment of the present disclosure;

FIG. 2 is a flow chart diagram of a click rate prediction model training method of one embodiment provided by the present disclosure;

FIG. 3 is a flow diagram of a method of training a multi-task learning model in accordance with one embodiment provided by the present disclosure;

FIG. 4 is a block diagram of a media asset ordering device according to one embodiment provided by the present disclosure;

FIG. 5 is a block diagram of a click rate prediction model training device of one embodiment provided by the present disclosure;

FIG. 6 is a block diagram of a multitasking learning model training apparatus of one embodiment provided by the present disclosure;

FIG. 7 is a block diagram of an electronic device used to implement a media asset ordering method, a click rate prediction model training method, or a multitasking learning model training method of embodiments of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

As shown in fig. 1, according to an embodiment of the present disclosure, the present disclosure provides a media resource ranking method, including:

Step S101: and acquiring the association characteristic of the first user.

The first user may understand that the user to be recommended may recommend media resources to the first user after ranking the plurality of candidate resources, and the first user may be a user in a certain application. As one example, the associated features of the user may include, but are not limited to, attribute features, including, but not limited to, age, gender, city, etc., current scene features may be understood as features of the scene in which the user is currently located, including, for example, features of the network currently in use, such as, for example, features of network type and network signal strength. The historical behavioral characteristics may be behavioral characteristics of the user over a preset historical period of time.

Step S102: and inputting the associated features into a pre-acquired target click rate prediction model to predict to obtain a first predicted click rate vector, and inputting the associated features into a pre-acquired target multi-task learning model to predict to obtain a first predicted stay time length vector and a first predicted completion rate vector.

Step S103: for each candidate media resource of the plurality of candidate media resources, a second predicted click rate vector, a second predicted stay time length vector and a second predicted completion rate vector of the candidate media resource are obtained; and determining a first score for the candidate media asset based on the first predicted click rate vector, the second predicted click rate vector, the first predicted stay length vector, the second predicted stay length vector, the first predicted completion rate vector, and the second predicted completion rate vector.

The media resources may include, but are not limited to, videos, graphics, and the like, and the videos may be further classified into a first type of video and a second type of video, where the duration of the first type of video is different from the duration of the second type of video, for example, the first type of video may be understood as a video with a duration greater than a preset duration, and the second type of video may be understood as a video with a duration less than or equal to the preset duration, and may also be referred to as a small video. The plurality of candidate media assets are media assets of the application described above.

As one example, resource characteristics may include, but are not limited to, characteristics of a resource Identification (ID), a resource type (e.g., video type, graphics type, etc.), a resource category (i.e., resource classification, e.g., technology class, entertainment class, educational class, etc.), and a resource size.

It may be understood that the first predicted click rate vector, the first predicted stay time length vector, and the first predicted completion rate vector are predicted vectors measured by the user, the first predicted click rate vector may be a feature vector obtained by feature extracting the associated feature of the first user by the target click rate prediction model, may be a vector representing the possibility of the user behavior of the first user, the second predicted click rate vector, the second predicted stay time length vector, and the second predicted completion rate vector are predicted vectors measured by the resource, the second predicted click rate vector may be a feature vector obtained by feature extracting the associated feature by the target multitask learning model, may represent a vector representing the possibility of the candidate media resource being acted by the user, and may obtain the predicted click rate by integrating the first predicted click rate vector and the second predicted click rate vector, for example, the predicted click rate may include predicted click rate or predicted no click rate.

The first predicted stay length vector may represent a stay length-related vector predicted by the first user after the user action is performed, and the first predicted completion rate vector may represent a completion rate-related vector for the first user after the user action is performed. The second predicted stay length vector may characterize a predicted stay length-related vector of the candidate media asset after being acted upon by the user, and the second predicted completion rate vector may characterize a completion rate-related vector of the candidate media asset after being acted upon by the user. The predicted residence time after the first user performs the user action on the candidate media resource can be obtained through the first predicted time vector and the second predicted time vector, and the predicted residence time can be obtained, for example, the first predicted time vector and the second predicted time vector are subjected to inner product to obtain the predicted residence time. The predicted completion rate after the user action of the first user on the candidate media resource can be obtained through the first predicted completion rate vector and the second predicted completion rate vector, for example, after the user clicks on a certain resource, the user stays for 10 seconds, the duration of the resource is 20 seconds, and the completion rate is 50%. In one example, the first predicted completion rate vector is inner-product with the second predicted completion rate vector to obtain a predicted completion rate.

And determining a first score of the candidate media resource according to the first predicted click rate vector, the second predicted click rate vector, the first predicted stay time length vector, the second predicted stay time length vector, the first predicted completion rate vector and the second predicted completion rate vector of the candidate media resource. In this embodiment, a corresponding first score may be calculated for each candidate media resource, so that a plurality of first scores of candidate media resources may be obtained, i.e., a plurality of first scores may be obtained.

It should be noted that, the first score of the candidate media resource may be used to characterize the matching degree of the candidate media resource to the first user, and the higher the score, the higher the matching degree, and the greater the likelihood that the candidate media resource is subsequently recommended to the first user.

Step S104: the plurality of candidate media assets is ranked based on the first score for the plurality of candidate media assets.

I.e. the ranking of the plurality of candidate media assets is achieved by a first scoring of the plurality of candidate media assets.

In this embodiment, a first predicted click rate vector is obtained by inputting a correlation feature of a first user into a target click rate prediction model obtained in advance, and a first predicted stay time length vector and a first predicted completion rate vector are obtained by inputting the correlation feature into a target multi-task learning model obtained in advance, and a second predicted click rate vector, a second predicted stay time length vector and a second predicted completion rate vector of each candidate media resource of a plurality of candidate media resources are obtained, and the first score of the candidate media resources is determined by using the first predicted click rate vector, the second predicted click rate vector, the first predicted stay time length vector, the second predicted stay time length vector, the first predicted completion rate vector and the second predicted completion rate vector, and then the ranking of the candidate media resources is achieved by the first scores of the plurality of candidate media resources. In the process of determining the first score of the candidate media resource, the association characteristic of the first user is adopted, and the predicted click rate vector obtained by predicting the target click rate prediction model based on the association characteristic of the first user and the first predicted stay time length vector and the first predicted completion rate vector obtained by predicting the target multitask learning model based on the association characteristic of the first user are adopted, and the first score of the candidate media resource is determined based on the first predicted click rate vector, the second predicted stay time length vector and the second predicted completion rate vector of the candidate media resource, so that the accuracy of the first score of the candidate media resource can be improved, the ranking of a plurality of candidate media resources is realized through the first scores of the plurality of candidate media resources, and the accuracy of the ranking of the candidate media resources is improved.

In one embodiment, determining a first score for the candidate media asset based on the first predicted click rate vector, the second predicted click rate vector, the first predicted stay length vector, the second predicted stay length vector, the first predicted completion rate vector, and the second predicted completion rate vector comprises:

Determining a target predicted click rate based on the first predicted click rate vector and the second predicted click rate vector;

determining a target predicted duration based on the first predicted stay duration vector and the second predicted stay duration vector;

determining a target predicted completion rate based on the first predicted completion rate vector and the second predicted completion rate vector;

a first score for the candidate media asset is determined using the target predicted click rate, the target predicted duration, and the target predicted completion rate.

That is, in this embodiment, in the process of determining the first score of the candidate media resource, the target predicted click rate determined by the first predicted click rate vector and the second predicted click rate vector, the target predicted duration determined by the first predicted stay duration vector and the second predicted stay duration vector, and the target predicted completion rate determined by the first predicted completion rate vector and the second predicted completion rate vector are considered, so as to improve the accuracy of the obtained first score, thereby improving the accuracy of sorting the resources.

As one example, the inner product of the first predicted stay length vector and the second predicted stay length vector may be taken as the target predicted length, and the inner product of the first predicted completion rate vector and the second predicted completion rate vector may be taken as the target predicted completion rate. In one example, using the target predicted click rate, the target predicted duration, and the target predicted completion rate, determining a first score for the candidate media asset may include: multiplying the first preset value power of the target predicted click rate, the second preset value power of the target predicted stay time and the third preset value power of the target predicted completion rate to obtain a first score of the candidate media resource. The first preset value, the second preset value and the third preset value can be preset through experience and are all numbers larger than 0, and the first score is obtained through multiplying the first preset value power of the target predicted click rate, the second preset value power of the target predicted stay time length and the third preset value power of the target predicted completion rate, so that the accuracy of the first score can be improved.

In one embodiment, after ranking the plurality of candidate media assets based on the first score for the plurality of candidate media assets, further comprises:

and recommending the media resources with the top N in the ranking among the candidate media resources to the first user, wherein N is a positive integer.

The plurality of candidate media resources are sequentially ranked from large to small in score, N media resources before ranking are N media resources with large scores, and N media resources with large scores are recommended to a first user, so that the recommended media resources are more adaptive to the first user, pertinence is improved, and accuracy of resource recommendation is improved.

In one embodiment, obtaining a second predicted click rate vector, a second predicted dwell time duration vector, and a second predicted completion rate vector for the candidate media asset includes:

Searching a second predicted click rate vector, a second predicted stay time length vector and a second predicted completion rate vector of the candidate media resource from the cache, wherein the second predicted click rate vector of the candidate media resource is obtained by predicting the candidate media resource based on the resource characteristics of the candidate media resource, and the second predicted stay time length vector and the second predicted completion rate vector of the candidate media resource are obtained by predicting the candidate media resource based on the resource characteristics of the candidate media resource.

The resource characteristics of each media resource in the first media resource set with the preset identification can be input into a target click rate prediction model in advance to be predicted so as to obtain a second predicted click rate vector of each media resource in the first media resource set and cached, a plurality of candidate media resources belong to the first media resource set, the resource characteristics of each media resource in the first media resource set can be input into a target multi-task learning model in advance to be predicted so as to obtain a second predicted stay time length vector and a second predicted completion rate vector of each media resource in the first media resource set and cached, in this way, in the process of sorting the candidate media resources, the second predicted click rate vector, the second predicted stay time length vector and the second predicted completion rate vector of the candidate media resources are searched from the cache, so that the speed of obtaining the second predicted click rate vector, the second predicted stay time length vector and the second predicted completion rate vector of the candidate media resources is improved, and the overall resource sorting efficiency is improved.

In one embodiment, the target click rate prediction model is determined by:

Screening a plurality of users according to attribute characteristics of the plurality of users in the application, and determining a first user set;

Screening the first user set to obtain a second user set, wherein any user in the second user set has a first action on the first project associated media resource of the application;

screening a first media resource set with a preset identifier from a plurality of media resources of an application, wherein the preset identifier is used for indicating that the media resources can be distributed in a second project;

Constructing a first sample data set based on the associated features of the second user set, the associated features of the second item associated user of the application, the resource features of the second media resource set, and the resource features of a third media resource set, the third media resource set comprising media resources of the second behavior performed by the second item associated user of the application;

In the process of acquiring the first sample data set, the plurality of users are screened according to the attribute features of the plurality of users in the application, the first user set is determined, the attribute features can include, but are not limited to, at least one of gender, age, city and other features, the first user set with the attribute features meeting the preset attribute conditions can be screened from the plurality of users, the preset attribute conditions can include, but are not limited to, the attribute features and the attribute features of the first user have a matching degree greater than a preset matching degree, namely the attribute features of the first user are similar to the attribute features of the first user, and the second user set can be understood to be a crowd similar to the first user in the first project associated user. In this embodiment, the behaviors of the crowd similar to the current first user in the first item and the behaviors of the user associated with the second item are collected to generate sample data, so as to train the click rate prediction model, improve the model training effect, and improve the performance of the obtained target click rate prediction model.

Because the first user set is selected to meet the attribute condition, the first user set may include users who have no first behavior on the media resources associated with the first item of the application, the first user set is filtered, and the second user set is selected, wherein the first behavior includes at least one of clicking, collecting, sharing, commenting, praying and the like. It should be noted that, in the above application, a plurality of items (channels or topics) may be included, and different types of resources may be displayed under different items, for example, a hotspot item (mainly displaying a hotspot resource), an entertainment item (mainly displaying a resource related to entertainment), a tidal surge of emotion item (mainly displaying a resource such as a trending cross-over), a recommendation item (mainly displaying a resource recommended by the application), a focus item, an education item, a technological item, an international item, a sports item, and the like may be included. The first item may be any one of a plurality of items, and may be a recommended item, for example. The second item may be any one of a plurality of items, for example, tidal surge of emotion items, unlike the first item.

Any user in the second set of users has a first behavior on the first item-associated media asset of the application, which first item-associated media asset may be understood as a asset distributed in the first item or as a asset presented in the first item, i.e. the user in the second set of users is a user in the first set of users having a first behavior on the first item-associated media asset.

In the process of generating the first sample data set, not only a user but also resources need to be considered, a first media resource set with a preset identifier can be screened from a plurality of media resources of an application, the preset identifier is used for indicating that the media resources can be distributed in a second item, namely, the resources in the first media resource set are resources which can be distributed in the second item, then the resources without associated user behaviors in the second item of the application are filtered, the second media resource set is obtained, any media resource in the second media resource set has associated user behaviors in the first item of the application, and the user behaviors can include at least one of clicking, collecting, sharing, commenting, praying and the like.

It should be noted that, the above-mentioned second item associated users may be all users in the second item, and these users have behaved in the application for the resources in the second item. The second behavior may include, but is not limited to, at least one of clicking, collecting, sharing, commenting, and praying, and may be the same as or different from the first behavior.

In this way, the first sample data set can be constructed through the association features of the second user set, the association features of the second item association users of the application, the resource features of the second media resource set and the resource features of the third media resource set, the initial click rate prediction model can be trained by the first sample data set to obtain the target click rate prediction model, so that the performance of the target click rate prediction model is improved, click rate prediction is carried out through the target click rate prediction model, and the accuracy of the predicted click rate can be improved.

In one embodiment, the first sample data set includes an associated feature of the second user set, an associated feature of the second item of the application associated with the user, a resource feature of the second media resource set, and a resource feature of the third media resource set, and any one of the sample data sets includes an associated feature of a user, a resource feature of a media resource, and a corresponding first tag, the first tag including an actual click tag.

The actual click label may be understood as an actual click result of the user on the media resource, for example, one piece of sample data includes an association feature of the user A1 and a resource feature of the media resource M1, and the corresponding actual click label is an actual click result of the user A1 on the media resource M1, for example, the user A1 clicks on the media resource M1, and the actual click label is a click label, for example, the label may be set to 1, and if the user A1 does not click on the media resource M1, the actual click label is a no click label, for example, the label may be set to 0. The associated features of the users in any piece of sample data in the first sample data set belong to the associated features of the total users, the total users comprise a second user set and a second item associated user, the resource features of the media resources in any piece of sample data in the first sample data set belong to the resource features of the total resources, and the total resources comprise a second resource sample set and a third resource sample set.

In this embodiment, by training the initial click rate prediction model by using the first sample data set including the association feature of the second user set, the association feature of the second item-associated user of the application, the resource feature of the second media resource set, and the resource feature of the third media resource set, where the first tag is an actual click tag, the model training effect can be improved, so that the performance of the obtained target click rate prediction model is improved.

In one embodiment, the target multitasking learning model is determined by:

Screening a first media resource set with a preset identifier from a plurality of media resources of an application, wherein the preset identifier is used for indicating that the media resources can be distributed in a second item;

Constructing a second sample data set based on the associated features of the second user set, the associated features of the second item associated user of the application, the resource features of the second media resource set, and the resource features of a third media resource set, the third media resource set comprising media resources of the second behavior performed by the second item associated user of the application;

training the initial multi-task learning model by using the second sample data set to obtain a target multi-task learning model.

It should be noted that, the target multitask learning model may be a target CGC (Customized Gate Control, custom gate control) model, and the construction process of the second sample data set is similar to that of the first sample data set, which is not described herein again, and the feature data between the second sample data set and the first sample data set are the same, except that the label includes an actual stay time label and an actual completion rate label, and the label is an actual click label. By training the initial multi-task learning model by the aid of the second sample data set comprising the association features of the second user set, the association features of the second item-associated user of the application, the resource features of the second media resource set and the resource features of the third media resource set, wherein the second label comprises the actual stay time label and the actual completion rate label, the model training effect can be improved, and accordingly performance of the obtained target multi-task learning model is improved.

In one embodiment, the second sample data set includes an associated feature of the second user set, an associated feature of the second item associated user of the application, a resource feature of the second media resource set, and a resource feature of the third media resource set, and any piece of sample data in the second sample data set includes an associated feature of a user, a resource feature of a media resource, and a corresponding second tag, the second tag including an actual dwell time period tag and an actual completion rate tag.

In this embodiment, the training of the initial multi-task learning model may improve the model training effect by including the association feature of the second user set, the association feature of the second item-associated user of the application, the resource feature of the second media resource set, and the resource feature of the third media resource set, where the second tag includes the second sample data set of the actual stay time tag and the actual completion rate tag, so as to improve the performance of the obtained target multi-task learning model.

As shown in fig. 2, according to an embodiment of the present disclosure, the present disclosure further provides a click rate prediction model training method, including:

Step S201: screening a plurality of users according to attribute characteristics of the plurality of users in the application, and determining a first user set;

Step S202: screening the first user set to obtain a second user set, wherein any user in the second user set has a first action on the first project associated media resource of the application;

step S203: screening a first media resource set with a preset identifier from a plurality of media resources of an application, wherein the preset identifier is used for indicating that the media resources can be distributed in a second project;

Step S204: screening the first media resource set to obtain a second media resource set, wherein any media resource in the second media resource set has associated user behavior in a first item of the application;

step S205: constructing a first sample data set based on the associated features of the second user set, the associated features of the second item-associated user of the application, the resource features of the second media resource set, and the resource features of the third media resource set;

the third media resource set comprises media resources of the second item of the application for which the associated user has performed a second action;

step S206: and training the initial click rate prediction model by using the first sample data set to obtain a target click rate prediction model.

In this embodiment, a first sample data set may be constructed according to the association feature of the second user set, the association feature of the second item-associated user of the application, the resource feature of the second media resource set, and the resource feature of the third media resource set, and then the initial click rate prediction model may be trained by using the first sample data set to obtain the target click rate prediction model, so as to improve the performance of the target click rate prediction model, and then click rate prediction may be performed by the target click rate prediction model, so that accuracy of predicting the click rate may be improved.

As shown in fig. 3, according to an embodiment of the present disclosure, the present disclosure further provides a method for training a multi-task learning model, including:

Step S301: screening a plurality of users according to attribute characteristics of the plurality of users in the application, and determining a first user set;

Step S302: screening the first user set to obtain a second user set, wherein any user in the second user set has a first action on the first project associated media resource of the application;

Step S303: screening a first media resource set with a preset identifier from a plurality of media resources of an application, wherein the preset identifier is used for indicating that the media resources can be distributed in a second item;

Step S304: screening the first media resource set to obtain a second media resource set, wherein any media resource in the second media resource set has associated user behavior in a first item of the application;

Step S305: constructing a second sample data set based on the associated features of the second user set, the associated features of the second item-associated user of the application, the resource features of the second media resource set, and the resource features of the third media resource set;

Step S306: training the initial multi-task learning model by using the second sample data set to obtain a target multi-task learning model.

In this embodiment, a second sample data set may be constructed according to the association feature of the second user set, the association feature of the second item-associated user of the application, the resource feature of the second media resource set, and the resource feature of the third media resource set, and then the initial multi-task learning model may be trained by using the second sample data set to obtain the target multi-task learning model, so as to improve the performance of the target multi-task learning model, and then the residence time and the completion rate are predicted by using the target multi-task learning model, so that the prediction accuracy of the residence time and the completion rate may be improved.

The procedure of the above method is specifically described in the following with reference to an embodiment.

Offline part:

(1) User collection: screening the first user set by user attribute features (gender, age, city, etc.); screening users with first project behaviors from the first user set to obtain a second user set; acquiring all user behaviors associated with the current second item;

(2) And (3) resource collection: screening to obtain a first media resource set through resource characteristics (special identifiers which can be distributed in the current second project); screening resources with first project behaviors from the first media resource set to obtain a second media resource set; acquiring resources of all user behaviors associated with the current second item, namely acquiring a third media resource set;

(3) Sample generation:

click presentation sample: all actual clicks of the resource in (2) by the user in (1) above exhibit behavior. Taking the actual clicking behaviors with too short duration as negative samples, and filtering invalid clicks. Removing the default display which is refreshed for the first time and the sample which is not clicked by the user, and filtering invalid display;

Duration samples: (1) The actual residence time of the user in (2) after all clicks on the resource;

And (3) model building: building an initial click rate prediction model for learning click behaviors of a user on a target resource, and building an initial multi-task learning model for learning duration behaviors of the user on the target resource; for the initial click rate prediction model: in the training process, inputting the associated characteristics of a user, obtaining a click rate vector of the user side through a plurality of layers of fully-connected networks, inputting the resource characteristics, obtaining the click rate vector of the resource side through a plurality of layers of fully-connected networks, fusing (e.g. inner product) the click rate vector of the user side and the click rate vector of the resource side into a predicted click rate, and calculating cross entropy with a label whether the user clicks or not; for a multitasking learning model: inputting the association characteristic of the user, obtaining a duration vector and a completion rate vector of the user side through a CGC structure, inputting the resource characteristic, obtaining the duration vector and the completion rate vector of the resource side through the CGC structure, fusing the duration vector of the user side and the duration vector of the resource side into a predicted stay duration, and calculating a mean square error (mse) loss with the normalized actual stay duration. And fusing the completion rate vector of the user side and the completion rate vector of the resource side into a predicted completion rate and calculating mse with the actual completion rate.

Model training: training by using the generated samples to obtain a target click rate prediction model and a target multi-task learning model, and storing parameters of a bottom embedded layer and an upper layer model (fully connected network) produced by the model into a cluster in a streaming manner. In addition, the white list resource (for example, the first media resource set) can process the resource characteristics of the white list resource through a trained target click rate prediction model and a target multi-task learning model to obtain a second predicted click rate vector of the white list resource, a second predicted stay time length vector of the white list resource and a second predicted completion rate vector of the white list resource, and the second predicted stay time length vector is cached.

On-line part:

after training is completed, a target click rate prediction model and a target multi-task learning model are obtained, in the process of online application, the associated feature target click rate prediction model of the first user can be predicted to obtain a first predicted click rate, and the associated feature of the first user can be input into the target multi-task learning model to be predicted to obtain a first predicted stay time vector and a first predicted completion rate vector; searching a cache of the resource measurement to obtain a second predicted click rate vector, a second predicted stay time length vector and a second predicted completion rate vector of each candidate media resource, and fusing the corresponding second predicted click rate vector, second predicted stay time length vector and second predicted completion rate vector of each candidate media resource through a fusion formula to obtain a first score of the candidate media resource, so that the first scores of a plurality of candidate media resources can be obtained, and the plurality of candidate media resources are ranked by using the first scores of the plurality of candidate media resources.

According to the scheme of the embodiment of the disclosure, the behaviors of the first item associated users, which are similar to the current first user, and the behaviors of the current second item are combined together, a sample is constructed, model training is performed, a model training effect is provided, and the obtained model performance is improved. And the CGC structure is adopted to simultaneously optimize the duration and the completion rate, so that the commonality of two targets with close original relations can be modeled.

As shown in fig. 4, according to an embodiment of the present disclosure, the present disclosure further provides a media resource ordering apparatus 400, including:

a first obtaining module 401, configured to obtain an association feature of a first user;

The prediction module is used for inputting the associated features into a target click rate prediction model which is obtained in advance to predict to obtain a first predicted click rate vector, and inputting the associated features into a target multi-task learning model which is obtained in advance to predict to obtain a first predicted stay time length vector and a first predicted completion rate vector;

a score determining module 402, configured to obtain, for each candidate media resource of the plurality of candidate media resources, a second predicted click rate vector, a second predicted stay time length vector, and a second predicted completion rate vector of the candidate media resource; and determining a first score for the candidate media asset based on the first predicted click rate vector, the second predicted click rate vector, the first predicted stay length vector, the second predicted stay length vector, the first predicted completion rate vector, and the second predicted completion rate vector;

a ranking module 403 is configured to rank the plurality of candidate media assets based on the first scores of the plurality of candidate media assets.

In one embodiment, the scoring module comprises:

the first determining module is used for determining a target predicted click rate based on the first predicted click rate vector and the second predicted click rate vector;

the second determining module is used for determining a target predicted duration based on the first predicted stay duration vector and the second predicted stay duration vector;

a third determining module for determining a target prediction completion rate based on the first prediction completion rate vector and the second prediction completion rate vector;

and the resource score determining module is used for determining a first score of the candidate media resource by using the target predicted click rate, the target predicted time length and the target predicted completion rate.

In one embodiment, the apparatus further comprises:

And the recommending module is used for recommending the media resources with the front N of the candidate media resources to the first user, wherein N is a positive integer.

The media resource sorting device of each embodiment is a device for implementing the media resource sorting method of each embodiment, and the technical features correspond to each other, and the technical effects correspond to each other, which is not described herein again.

As shown in fig. 5, according to an embodiment of the present disclosure, the present disclosure further provides a click rate prediction model training apparatus 500, including:

the first user determining module 501 is configured to screen a plurality of users according to attribute features of the plurality of users in the application, and determine a first user set;

A second user determining module 502, configured to filter the first user set to obtain a second user set, where any user in the second user set has a first behavior on a first project associated media resource of the application;

A first resource screening module 503, configured to screen a first media resource set with a preset identifier from a plurality of media resources of an application, where the preset identifier is used to indicate that the media resource can be issued and distributed in a second item;

A second resource screening module 504, configured to screen the first media resource set to obtain a second media resource set, where any media resource in the second media resource set has an associated user behavior in the first item of the application;

A first sample construction module 505, configured to construct a first sample data set based on the association feature of the second user set, the association feature of the second item association user of the application, the resource feature of the second media resource set, and the resource feature of a third media resource set, where the third media resource set includes media resources of the second item association user of the application that have performed the second action;

The first training module 506 is configured to train the initial click rate prediction model by using the first sample data set, so as to obtain a target click rate prediction model.

The click rate prediction model training device in each embodiment is a device for implementing the click rate prediction model training method in each embodiment, and the technical features and the technical effects are corresponding, and are not described herein.

As shown in fig. 6, according to an embodiment of the present disclosure, the present disclosure further provides a multitasking learning model training apparatus 600, including:

A third user determining module 601, configured to screen a plurality of users according to attribute features of the plurality of users in the application, and determine a first user set;

A fourth user determining module 602, configured to filter the first user set to obtain a second user set, where any user in the second user set has a first behavior on a first project associated media resource of the application;

a third resource screening module 603, configured to screen a first media asset set having a preset identifier from a plurality of media assets of the application, where the preset identifier is used to indicate that the media asset can be distributed in the second item;

A fourth resource screening module 604, configured to screen the first media resource set to obtain a second media resource set, where any media resource in the second media resource set has an associated user behavior in the first item of the application;

A second sample construction module 605, configured to construct a second sample data set based on the association feature of the second user set, the association feature of the second item association user of the application, the resource feature of the second media resource set, and the resource feature of a third media resource set, where the third media resource set includes media resources for which the second item association user of the application performed a second action;

a second training module 606 is configured to train the initial multi-task learning model by using the second sample data set, so as to obtain a target multi-task learning model.

In the technical scheme of the disclosure, the acquisition, storage, application and the like of the related user personal information all conform to the regulations of related laws and regulations, and the public sequence is not violated.

The device for training the multi-task learning model in the above embodiments is a device for implementing the method for training the multi-task learning model in the above embodiments, and the technical features and the technical effects are corresponding, and are not described herein again.

According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

The non-transitory computer readable storage medium of an embodiment of the present disclosure stores computer instructions for causing a computer to perform the media asset ordering method provided by the present disclosure.

The computer program product of the embodiments of the present disclosure includes a computer program for causing a computer to execute the media resource ranking method provided by the embodiments of the present disclosure.

Fig. 7 illustrates a schematic block diagram of an example electronic device 700 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 7, the electronic device 700 includes a computing unit 701 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 702 or a computer program loaded from a storage unit 708 into a Random Access Memory (RAM) 703. In the RAM703, various programs and data required for the operation of the device 700 may also be stored. The computing unit 701, the ROM 702, and the RAM703 are connected to each other through a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.

Various components in the electronic device 700 are connected to the I/O interface 705, including: an input unit 706 such as a keyboard, a mouse, etc.; an output unit 707 such as various types of displays, speakers, and the like; a storage unit 708 such as a magnetic disk, an optical disk, or the like; and a communication unit 709 such as a network card, modem, wireless communication transceiver, etc. The communication unit 709 allows the electronic device 700 to exchange information/data with other devices through a computer network, such as the internet, and/or various telecommunication networks.

The computing unit 701 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 701 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized artificial intelligence (I) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the respective methods and processes described above, such as a media asset ranking method, a click rate prediction model training method, a multitasking learning model training method, and the like. For example, in some embodiments, the media asset ranking method, click rate prediction model training method, or multitasking learning model training method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 700 via ROM 702 and/or communication unit 709. When the computer program is loaded into the RAM703 and executed by the computing unit 701, one or more steps of the media asset ranking method, click rate prediction model training method, or multitasking learning model training method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the media asset ordering method, the click rate prediction model training method, or the multitasking learning model training method in any other suitable manner (e.g., by means of firmware). Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), the internet, and blockchain networks.

The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service ("Virtual PRIVATE SERVER" or simply "VPS") are overcome. The server may also be a server of a distributed system or a server that incorporates a blockchain.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel, sequentially, or in a different order, provided that the desired results of the disclosed aspects are achieved, and are not limited herein.

The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.

Claims

1. A method of media asset ordering, the method comprising:

acquiring the association characteristics of a first user;

Ranking the plurality of candidate media assets based on a first score for the plurality of candidate media assets;

Wherein the determining a first score for the candidate media asset based on the first predicted click rate vector, the second predicted click rate vector, the first predicted stay length vector, the second predicted stay length vector, the first predicted completion rate vector, and the second predicted completion rate vector comprises:

and determining a first score of the candidate media resource by using the target predicted click rate, the target predicted time length and the target predicted completion rate.

2. The method of claim 1, wherein after the ranking the plurality of candidate media assets based on the first score for the plurality of candidate media assets further comprises:

recommending the media resources with the top N of the candidate media resources to the first user, wherein N is a positive integer.

3. The method of claim 1, wherein the obtaining a second predicted click rate vector, a second predicted dwell time vector, and a second predicted completion rate vector for the candidate media asset comprises:

Searching the second predicted click rate vector, the second predicted stay time length vector and the second predicted completion rate vector of the candidate media resource from a cache, wherein the second predicted click rate vector of the candidate media resource is obtained by predicting the target click rate prediction model based on the resource characteristics of the candidate media resource, and the second predicted stay time length vector and the second predicted completion rate vector of the candidate media resource are obtained by predicting the target multitask learning model based on the resource characteristics of the candidate media resource.

4. A click rate prediction model training method comprises the following steps:

5. The method of claim 4, wherein the first sample data set includes an associated feature of the second user set, an associated feature of a second item of the application associated with a user, a resource feature of the second media resource set, and a resource feature of a third media resource set, any of the sample data set including an associated feature of a user, a resource feature of a media resource, and a corresponding first tag, the first tag including an actual click tag.

6. A method of training a multitasking learning model, comprising:

7. The method of claim 6, wherein the second sample data set includes an associated feature of the second user set, an associated feature of a second item of the application associated with a user, a resource feature of the second media resource set, and a resource feature of a third media resource set, any piece of sample data in the second sample data set includes an associated feature of a user, a resource feature of a media resource, and a corresponding second tag, the second tag including an actual stay time period tag and an actual completion rate tag.

8. A media asset ordering apparatus, the apparatus comprising:

a ranking module for ranking the plurality of candidate media assets based on a first score for the plurality of candidate media assets;

wherein the scoring module comprises:

A third determining module configured to determine a target prediction completion rate based on the first prediction completion rate vector and the second prediction completion rate vector;

9. The apparatus of claim 8, wherein the apparatus further comprises:

And the recommending module is used for recommending the media resources with the top N in the ranking among the candidate media resources to the first user, wherein N is a positive integer.

10. The apparatus of claim 8, wherein the obtaining a second predicted click rate vector, a second predicted dwell time vector, and a second predicted completion rate vector for the candidate media asset comprises:

11. A click rate prediction model training device, comprising:

12. The apparatus of claim 11, wherein the first sample data set includes an associated feature of the second user set, an associated feature of a second item of the application associated with a user, a resource feature of the second media resource set, and a resource feature of a third media resource set, any of the sample data set including an associated feature of a user, a resource feature of a media resource, and a corresponding first tag, the first tag including an actual click tag.

13. A multitasking learning model training apparatus comprising:

14. The apparatus of claim 13, wherein the second sample data set includes an associated feature of the second user set, an associated feature of a second item of the application associated with a user, a resource feature of the second media resource set, and a resource feature of a third media resource set, any of the second sample data set including an associated feature of a user, a resource feature of a media resource, and a corresponding second tag, the second tag including an actual stay time period tag and an actual completion rate tag.

15. An electronic device, comprising:

At least one processor; and

A memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the media asset ordering method of any one of claims 1-3 or the click rate prediction model training method of any one of claims 4-5 or the multitasking learning model training method of any one of claims 6-7.

16. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the media asset ordering method of any one of claims 1-3 or the click rate prediction model training method of any one of claims 4-5 or the multitasking learning model training method of any one of claims 6-7.

17. A computer program product comprising a computer program which, when executed by a processor, implements the media asset ordering method of any one of claims 1-3 or the click rate prediction model training method of any one of claims 4-5 or the multitasking learning model training method of any one of claims 6-7.