CN113935554A

CN113935554A - Model training method in delivery system, resource delivery method and device

Info

Publication number: CN113935554A
Application number: CN202111529873.8A
Authority: CN
Inventors: 林伟
Original assignee: Beijing Dajia Internet Information Technology Co Ltd
Current assignee: Beijing Dajia Internet Information Technology Co Ltd
Priority date: 2021-12-15
Filing date: 2021-12-15
Publication date: 2022-01-14
Anticipated expiration: 2041-12-15
Also published as: CN113935554B

Abstract

The disclosure relates to a model training method, a resource delivery method and a resource delivery device in a delivery system. The method comprises the following steps: acquiring a plurality of sample data and corresponding sample labels; inputting the sample structural features and the sample unstructured information into a preset model, and performing prediction processing on the putting parameters of the sample multimedia to obtain a first prediction result and sample unstructured features; inputting the sample structural features and the sample unstructured features into a second prediction network, and performing prediction processing on the input parameters of the sample multimedia to obtain a second prediction result; and training the second prediction network according to the sample label, the first prediction result and the second prediction result to obtain a pre-ordering model. According to the technical scheme provided by the disclosure, the structure of the pre-sequencing model is simpler, and the prediction precision and efficiency of the pre-sequencing model are higher.

Description

Model training method in delivery system, resource delivery method and device

Technical Field

The present disclosure relates to the field of internet application technologies, and in particular, to a model training method, a resource delivery method, and an apparatus in a delivery system.

Background

With the endless application layer, resource delivery (e.g., multimedia delivery) in each application is also concerned, and in order to enable delivered multimedia to match with the interests of users to improve delivery effects, generally, multimedia in applications is filtered through stages of recall, pre-sorting/rough ranking, fine ranking, and the like, so as to obtain multimedia matching with the interests of users.

In the related technology, a pre-sorting stage is used as a key stage of screening, on one hand, in order to ensure the prediction precision of a pre-sorting model, an unstructured feature extraction module is selected to be embedded into the pre-sorting model, so that end-to-end training of unstructured information is realized, and the prediction precision of the pre-sorting model is improved; on the other hand, considering that the multimedia faced by the pre-ordering model is generally thousands of orders, in order to improve the prediction efficiency, the non-structural feature extraction module is not included in the pre-ordering model. Based on the method, training is carried out through two stages, the first stage trains the unstructured feature extraction module, and the second stage utilizes the unstructured feature extraction module to obtain the static unstructured features of the unstructured information, so that the pre-ranking model is trained based on the static unstructured features. However, when the former is applied on line, due to the large multimedia magnitude, the prediction efficiency is low, and the former occupies high resources and has large processing pressure; although the efficiency of the method is high, during training, unstructured information does not participate in learning of labels corresponding to the pre-ranking model, so that deviation exists between unstructured features and a prediction task, the pre-ranking prediction precision is low, and the delivery precision is low.

Disclosure of Invention

The present disclosure provides a model training method, a resource delivery method, and a device in a delivery system, so as to at least solve the problem of how to improve the prediction accuracy and prediction efficiency of a pre-ranking model in the related art. The technical scheme of the disclosure is as follows:

according to a first aspect of the embodiments of the present disclosure, a method for training a model in a delivery system is provided, including:

obtaining a plurality of sample data and corresponding sample labels, wherein the sample data comprises sample structural characteristics and sample unstructured information; the sample structured feature is obtained based on multimedia attribute information of sample multimedia and object attribute information of a sample object; the unstructured information characterizes a plurality of media information of the sample multimedia and interaction information of the sample object;

inputting the sample structural features and the sample unstructured information into a preset model, and performing prediction processing on the putting parameters of the sample multimedia to obtain a first prediction result and sample unstructured features;

inputting the sample structural features and the sample unstructured features into a second prediction network, and performing prediction processing on the input parameters of the sample multimedia to obtain a second prediction result;

and training the second prediction network according to the sample label, the first prediction result and the second prediction result to obtain a pre-ordering model.

In one possible implementation, the preset model includes an unstructured feature extraction network and a first prediction network; the step of inputting the sample structured features and the sample unstructured information into a preset model, and performing prediction processing on the delivery parameters of the sample multimedia to obtain a first prediction result and sample unstructured features includes:

inputting the sample unstructured information into the unstructured feature extraction network, and performing feature extraction processing to obtain the sample unstructured features;

and inputting the sample structural features and the sample unstructured features into the first prediction network, and performing prediction processing on the delivery parameters of the sample multimedia to obtain a first prediction result.

In a possible implementation manner, the training the second prediction network according to the sample label, the first prediction result, and the second prediction result to obtain a pre-ranking model includes:

determining target loss information according to the first prediction result, the second prediction result and the sample label;

and training the second prediction network based on the target loss information to obtain the pre-sequencing model.

In a possible implementation manner, before the step of training the second prediction network based on the target loss information to obtain the pre-ranking model, the method further includes:

determining first loss information according to the first prediction result and the sample label;

updating parameters of the first prediction network and parameters of the non-structural feature extraction network based on the first loss information to obtain an updated preset model;

correspondingly, training the second prediction network based on the target loss information to obtain the pre-ranking model, including:

updating parameters of the second prediction network based on the target loss information to obtain an updated second prediction network;

repeating the step of inputting the sample unstructured information into an unstructured feature extraction network in a preset model according to an updated preset model and an updated second prediction network, and performing feature extraction processing to obtain sample unstructured features, until the step of updating parameters of the second prediction network based on the target loss information to obtain an updated second prediction network;

and acquiring a second prediction network corresponding to preset time as the pre-sequencing model.

In a possible implementation manner, after the step of determining first loss information according to the first prediction result and the sample label, the method further includes:

determining second loss information according to the first loss information and the target loss information;

correspondingly, the step of updating the parameters of the first prediction network and the parameters of the unstructured feature extraction network based on the first loss information to obtain an updated preset model includes:

and updating the parameters of the first prediction network and the parameters of the non-structural feature extraction network based on the second loss information determined according to the first loss information and the target loss information to obtain an updated preset model.

In one possible implementation manner, the multiple media information of the sample multimedia includes sample image information, sample text information and sample audio information, and the unstructured feature extraction network includes a convolutional neural network, a graph neural network and a coding/decoding network; the step of inputting the sample unstructured information into the unstructured feature extraction network for feature extraction processing to obtain the sample unstructured features comprises:

inputting the sample image information into the convolutional neural network, and performing feature extraction processing to obtain sample image features;

inputting the sample text information and the sample audio information into the coding and decoding network, and performing feature extraction processing to obtain sample text features and sample audio features;

inputting the interactive information of the sample object into the graph neural network, and performing feature extraction processing to obtain sample interactive features;

and splicing the sample image characteristic, the sample text characteristic, the sample audio characteristic and the sample interaction characteristic to obtain the unstructured characteristic.

In a possible implementation manner, the step of determining target loss information according to the first prediction result, the second prediction result, and the sample label includes:

determining third loss information according to the second prediction result and the sample label;

determining fourth loss information according to the second prediction result and the first prediction result;

and obtaining the target loss information based on the third loss information and the fourth loss information.

In one possible implementation, the method further includes:

and acquiring a preset model corresponding to the preset time as a target preset model.

According to a second aspect of the embodiments of the present disclosure, there is provided a resource delivery method, including:

acquiring a recall resource matched with a target object and a structural feature and an unstructured feature corresponding to the recall resource, wherein the structural feature is obtained based on a resource attribute feature of the recall resource and an object attribute feature of the target object; the unstructured features are obtained by performing off-line extraction on unstructured information based on a target unstructured feature extraction network;

inputting the structural features and the non-structural features corresponding to the recalled resources into a pre-sorting model and a target preset model, and screening out a resource set to be released;

delivering the resource set to be delivered to the target object;

the pre-ranking model is obtained based on the training method of any one of the first aspect, and the target preset model is a preset launching parameter prediction model or the target preset model obtained based on the first aspect.

In a possible implementation manner, the inputting the structural features and the unstructured features corresponding to the recalled resources into a pre-ranking model and a target preset model, and screening out a resource set to be released includes:

inputting the structural features and the non-structural features corresponding to the recalled resources into the pre-sorting model, and performing delivery parameter prediction processing to obtain first delivery parameter information corresponding to the recalled resources;

screening an initial resource set from the recalled resources according to the first release parameter information;

inputting the structural characteristics and the unstructured information corresponding to each resource in the initial resource set into the target preset model, and performing delivery parameter prediction processing to obtain second delivery parameter information;

and screening the resource set to be launched from the initial resource set according to the second launching parameter information.

According to a third aspect of the embodiments of the present disclosure, there is provided a model training apparatus in a delivery system, including:

a sample acquisition module configured to perform acquisition of a plurality of sample data and corresponding sample labels, the sample data including sample structured features and sample unstructured information; the sample structured feature is obtained based on multimedia attribute information of sample multimedia and object attribute information of a sample object; the unstructured information characterizes a plurality of media information of the sample multimedia and interaction information of the sample object;

the first obtaining module is configured to input the sample structured features and the sample unstructured information into a preset model, and perform prediction processing on the putting parameters of the sample multimedia to obtain a first prediction result and sample unstructured features;

the second obtaining module is configured to input the sample structured features and the sample unstructured features into a second prediction network, and perform prediction processing on the putting parameters of the sample multimedia to obtain a second prediction result;

and the training module is configured to perform training on the second prediction network according to the sample label, the first prediction result and the second prediction result to obtain a pre-ordering model.

In one possible implementation, the preset model includes an unstructured feature extraction network and a first prediction network; the first obtaining module comprises:

the sample unstructured feature extraction unit is configured to input the sample unstructured information into the unstructured feature extraction network, and perform feature extraction processing to obtain the sample unstructured features;

and the first prediction result acquisition unit is configured to input the sample structured features and the sample unstructured features into the first prediction network, and perform prediction processing on the delivery parameters of the sample multimedia to obtain the first prediction result.

In one possible implementation, the training module includes:

a target loss information determination unit configured to perform determining target loss information from the first prediction result, the second prediction result, and the sample label;

a training unit configured to perform training of the second prediction network based on the target loss information, resulting in the pre-ordering model.

In one possible implementation, the apparatus further includes:

a first loss information determination module configured to perform determining first loss information based on the first prediction result and the sample label;

a preset model updating module configured to update parameters of the first prediction network and parameters of the non-structural feature extraction network based on the first loss information to obtain an updated preset model;

accordingly, the training unit comprises:

a second prediction network updating subunit configured to update parameters of the second prediction network based on the target loss information, resulting in an updated second prediction network;

the iterative training subunit is configured to execute the step of repeatedly inputting the sample unstructured information into the unstructured feature extraction network in the preset model according to the updated preset model and the updated second prediction network, and perform feature extraction processing to obtain a sample unstructured feature, until the step of updating the parameters of the second prediction network based on the target loss information to obtain the updated second prediction network;

and the pre-sequencing model acquisition subunit is configured to execute acquisition of a second prediction network corresponding to preset time as the pre-sequencing model.

In one possible implementation manner, the method further includes:

a second loss information determination module configured to perform determining second loss information from the first loss information and the target loss information;

the preset model updating module is further configured to update the parameter of the first prediction network and the parameter of the non-structural feature extraction network based on the second loss information determined according to the first loss information and the target loss information, so as to obtain an updated preset model.

In one possible implementation manner, the multiple media information of the sample multimedia includes sample image information, sample text information and sample audio information, and the unstructured feature extraction network includes a convolutional neural network, a graph neural network and a coding/decoding network; the sample unstructured feature extraction unit includes:

the sample image feature extraction subunit is configured to input the sample image information into the convolutional neural network, and perform feature extraction processing to obtain sample image features;

the sample text and audio feature extraction subunit is configured to input the sample text information and the sample audio information into the coding and decoding network, and perform feature extraction processing to obtain sample text features and sample audio features;

the sample interactive feature extraction subunit is configured to input the interactive information of the sample object into the graph neural network, and perform feature extraction processing to obtain a sample interactive feature;

and the unstructured feature extraction subunit is configured to perform splicing processing on the sample image feature, the sample text feature, the sample audio feature and the sample interaction feature to obtain the unstructured feature.

In one possible implementation manner, the target loss information determining unit includes:

a third loss information determination subunit configured to perform determining third loss information from the second prediction result and the sample label;

a fourth loss information determination subunit configured to perform determination of fourth loss information from the second prediction result and the first prediction result;

a target loss information obtaining subunit configured to perform obtaining the target loss information based on the third loss information and the fourth loss information.

In one possible implementation, the apparatus further includes:

and the target preset model acquisition module is configured to execute acquisition of a preset model corresponding to the preset time as a target preset model.

According to a fourth aspect of the embodiments of the present disclosure, there is provided a resource delivering apparatus, including:

an acquisition module configured to perform acquisition of a recalled resource that matches a target object and structured and unstructured features corresponding to the recalled resource, the structured features being derived based on resource attribute features of the recalled resource and object attribute features of the target object; the unstructured features are obtained by performing off-line extraction on unstructured information based on a target unstructured feature extraction network;

the screening module is configured to input the structural features and the non-structural features corresponding to the recalled resources into a pre-sorting model and a target preset model, and screen out a resource set to be released;

a delivery module configured to perform delivery of the set of resources to be delivered to the target object;

In one possible implementation, the screening module includes:

a first release parameter information obtaining unit, configured to input the structural features and the unstructured features corresponding to the recalled resources into the pre-ranking model, and perform release parameter prediction processing to obtain first release parameter information corresponding to the recalled resources;

a first screening unit configured to perform screening of an initial resource set from the recalled resources according to the first release parameter information;

a second release parameter information obtaining unit, configured to input the structural features and the unstructured information corresponding to each resource in the initial resource set into the target preset model, and perform release parameter prediction processing to obtain second release parameter information;

and the second screening unit is configured to perform screening of the resource set to be released from the initial resource set according to the second releasing parameter information.

According to a fifth aspect of embodiments of the present disclosure, there is provided an electronic apparatus including: a processor; a memory for storing the processor-executable instructions; wherein the processor is configured to execute the instructions to implement the method of any of the first aspects above or the method of any of the second aspects above.

According to a sixth aspect of embodiments of the present disclosure, there is provided a computer-readable storage medium, wherein instructions, when executed by a processor of an electronic device, enable the electronic device to perform the method of any one of the first aspect or the second aspect of the embodiments of the present disclosure.

According to a seventh aspect of embodiments of the present disclosure, there is provided a computer program product comprising computer instructions which, when executed by a processor, cause a computer to perform the method of any one of the first or second aspects of embodiments of the present disclosure.

The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects:

the training efficiency of the pre-sequencing model and the prediction precision when the application is put can be improved by using the preset model with a complex structure to guide the second prediction network with a simple structure to train; and due to the fact that the non-structural feature extraction is carried out by the preset model, the non-structural feature extraction function is embedded in the preset model, the preset model can fully sense the non-structural information, the extracted non-structural features can effectively represent the relevance of the prediction task, and the prediction accuracy of the pre-sequencing model can be further improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure and are not to be construed as limiting the disclosure.

FIG. 1 is a schematic diagram illustrating an application environment in accordance with an exemplary embodiment.

FIG. 2 is a flow diagram illustrating a method of model training in accordance with an exemplary embodiment.

FIG. 3 is an architectural diagram illustrating the use of a pre-set model to guide the training of a pre-ordering model in accordance with an exemplary embodiment.

Fig. 4 is a flowchart illustrating a method for inputting sample structured features and sample unstructured information into a preset model, and performing prediction processing on delivery parameters of sample multimedia to obtain a first prediction result and sample unstructured features according to an exemplary embodiment.

FIG. 5 is a flowchart illustrating a method for training a second prediction network to obtain a pre-ordering model according to a sample label, a first prediction result, and a second prediction result, according to an example embodiment.

FIG. 6 is a flow diagram illustrating another method of model training in accordance with an exemplary embodiment.

FIG. 7 is a flow chart illustrating a method of resource placement in accordance with an exemplary embodiment.

Fig. 8 is a block diagram illustrating a model training apparatus in a delivery system, according to an example embodiment.

Fig. 9 is a block diagram illustrating a resource delivery apparatus according to an example embodiment.

FIG. 10 is a block diagram illustrating an electronic device for model training or resource placement in accordance with an exemplary embodiment.

Detailed Description

In order to make the technical solutions of the present disclosure better understood by those of ordinary skill in the art, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in sequences other than those illustrated or otherwise described herein. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

Artificial Intelligence (AI) is a theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human Intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.

In recent years, with research and development of artificial intelligence technology, the artificial intelligence technology is widely applied in a plurality of fields, and the scheme provided by the embodiment of the application relates to technologies such as machine learning/deep learning, and is specifically described by the following embodiments:

referring to fig. 1, fig. 1 is a schematic diagram illustrating an application environment according to an exemplary embodiment, which may include a server 01 and a terminal 02, as shown in fig. 1.

In an alternative embodiment, the server 01 may be used for pre-ranking the models, training the pre-set models, and delivering multimedia. Specifically, the server 01 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a Network service, cloud communication, a middleware service, a domain name service, a security service, a CDN (Content Delivery Network), a big data and artificial intelligence platform, and the like.

In an alternative embodiment, the terminal 02 may be used to present the delivered resource, e.g. multimedia. Specifically, the terminal 02 may include, but is not limited to, a smart phone, a desktop computer, a tablet computer, a notebook computer, a smart speaker, a digital assistant, an Augmented Reality (AR)/Virtual Reality (VR) device, a smart wearable device, and other types of electronic devices. Optionally, the operating system running on the electronic device may include, but is not limited to, an android system, an IOS system, linux, windows, and the like.

In addition, it should be noted that fig. 1 shows only one application environment of the model training and resource delivery method provided by the present disclosure.

In the embodiment of the present specification, the server 01 and the terminal 02 may be directly or indirectly connected by a wired or wireless communication method, and the present application is not limited herein.

It should be noted that the following figures show a possible sequence of steps, and in fact do not limit the order that must be followed. Some steps may be performed in parallel without being dependent on each other. User information (including but not limited to user device information, user personal information, user behavior information, etc.) and data (including but not limited to data for presentation, training, etc.) to which the present disclosure relates are both information and data that are authorized by the user or sufficiently authorized by various parties.

FIG. 2 is a flow diagram illustrating a method of model training in accordance with an exemplary embodiment. As shown in fig. 2, the following steps may be included.

In step S201, a plurality of sample data and corresponding sample tags are acquired.

In this illustrative embodiment, the sample data may include sample structured features and sample unstructured information; the sample structured feature may be derived based on multimedia attribute information of the sample multimedia and object attribute information of the sample object; the unstructured information may characterize a variety of media information of the sample multimedia as well as interaction information of the sample object.

The sample label may be interaction index information, such as click rate, conversion rate, and the like. The sample multimedia may be multimedia corresponding to the advertisement, where the multimedia may include video, teletext data, etc., and as an example, the sample multimedia may be short video corresponding to the advertisement, etc. The sample object may refer to a user in a multimedia application, which may refer to a user account. The multimedia attribute information may refer to basic information for describing multimedia, such as multimedia identification, content category of the multimedia, and the like. The object attribute information may refer to basic information for describing the object, such as identification, sex, age, etc. of the object. The multimedia information may refer to information in various multimedia forms included in the sample multimedia, for example, when the sample multimedia is a short video, the multimedia information may include image information, text information, and audio information. The interactive information of the sample object may refer to interactive behavior information of the sample object occurring in the multimedia application, such as information concerning the multimedia, information concerning other objects, approval information, and the like, which are not limited in this disclosure.

Optionally, the interaction information of the sample object may refer to interaction information within a preset time from the current time, so that timeliness may be ensured, and the preset time may be one month, which is not limited by the present disclosure.

As an example, one sample data may be represented as follows:

[ sample multimedia 1, user 1; multimedia attribute characteristics of the sample multimedia 1, and object attribute characteristics of the user 1; image information, text information, audio information of the sample multimedia 1, interactive information of the user 1 ].

In practical application, a plurality of sample multimedia can be obtained from multimedia application, and image information, text information and audio information can be extracted from each sample multimedia. A plurality of users in the multimedia application may be obtained, and the plurality of users may include users who have interacted with the plurality of sample multimedia and/or users who have not interacted with the plurality of sample multimedia, which is not limited by the present disclosure. Further, the interaction information of each user within a preset time length from the current time may be obtained, for example, the interaction information of each user within the last month may be obtained. Thus, a plurality of sample multimedia and a plurality of users can be combined to obtain a combination of a sample multimedia and a user, namely, a combination comprises a sample multimedia and a user. And a corresponding structural feature of the combination can be obtained: multimedia attribute characteristics of a sample multimedia in the one combination, user attribute information of a user in the one combination; and corresponding unstructured information: image information, text information, audio information of a sample multimedia in the one combination, and interactive information of a user in the one combination. Thus, one combination and the structured feature and the unstructured information corresponding to the one combination can be used as one sample data. And each sample data can be labeled with a corresponding sample label, and the specific labeling mode is not limited in the disclosure.

In step S203, the sample structured feature and the sample unstructured information are input into a preset model, and a prediction process is performed on the multimedia delivery parameters of the sample, so as to obtain a first prediction result and a sample unstructured feature.

In the embodiment of the present specification, the preset model may refer to a teacher network for training a student network (second prediction network). Based on the method, the sample structured feature and the sample unstructured information can be input into a preset model, the input parameters of the sample multimedia are subjected to prediction processing, and a first prediction result and the sample unstructured feature are output. The sample unstructured features can be obtained by performing feature extraction on sample unstructured information through a preset model. Wherein, relative to the pre-ranking model, the pre-setting model may be a fine ranking model (pre-setting delivery parameter prediction model). The refined model may be a pre-trained model (static, model parameters are unchanged); or the refined model may participate in training to obtain the target preset model, which may be specifically described below.

As an example, as shown in fig. 3, the preset model may include an unstructured feature extraction network and a first prediction network, and accordingly, as shown in fig. 4, the step S203 may include the steps of:

in step S401, the unstructured sample information is input to an unstructured feature extraction network, and feature extraction processing is performed to obtain unstructured sample features.

In practical application, as shown in fig. 3, a preset model with a complex structure can be used as a teacher network to supervise and train a student network with a simple structure: a second predictive network. The second predictive network may refer to a pre-ordered network to be trained. The unstructured sample information can be input into an unstructured feature extraction network in a preset model, and feature extraction processing is carried out to obtain unstructured sample features. The preset model may be a machine learning model trained in advance, which is not limited by the present disclosure. The training efficiency of the pre-sequencing model and the prediction precision when the application is put in can be improved by using the preset model with a complex structure to guide the second prediction network with a simple structure to train, and the pre-sequencing efficiency can be improved due to the simple structure of the pre-sequencing model, so that the multi-media coarse screening with a larger order of magnitude can be effectively carried, and the processing pressure is reduced; furthermore, as the unstructured feature extraction network is embedded in the preset model, the unstructured feature extraction network participates in end-to-end training of the preset model, so that the unstructured feature extraction network can fully sense unstructured information, extracted unstructured features can effectively represent the relevance of a prediction task, and the prediction accuracy of the pre-ranking model can be further improved.

In practical application, the non-structural feature extraction network can be set to include various neural networks adapted to various media information in consideration of the difference of various media information. As an example, the plurality of media information of the sample multimedia may include sample image information, sample text information, and sample audio information, and the unstructured feature extraction network may include a convolutional Neural network cnn (convolutional Neural networks), a graph Neural network, and a codec network. Among them, the graph Neural network may include gnn (graph Neural networks), gcn (graph convergence networks), etc.; the codec network may include a BERT (Bidirectional Encoder retrieval from transforms). Accordingly, in one possible implementation, the step S401 may include the following steps:

inputting sample image information into a convolutional neural network, and performing feature extraction processing to obtain sample image features;

inputting the sample text information and the sample audio information into an encoding and decoding network, and performing feature extraction processing to obtain sample text features and sample audio features; alternatively, the sample text information may be subjected to text feature extraction using a text convolutional neural network textCNN, which is not limited by the present disclosure.

Inputting the interaction information of the sample object into a graph neural network, and performing feature extraction processing to obtain sample interaction features;

Through the multiple neural networks adaptive to the multiple media information, the sample image information, the sample text information and the sample audio information are respectively subjected to feature extraction, the extraction precision of the non-structural features can be improved, and the non-structural features can be more effectively expressed.

In step S403, the sample structured features and the sample unstructured features are input into a first prediction network, and a first prediction result is obtained by performing prediction processing on the delivery parameters of the sample multimedia.

In this embodiment of the present description, the sample structured features and the sample unstructured features may be input into a first prediction network in a preset model, and a first prediction result is obtained by performing prediction processing on an input parameter of a sample multimedia.

In step S205, the sample structured features and the sample unstructured features are input into a second prediction network, and a second prediction result is obtained by performing prediction processing on the delivery parameters of the sample multimedia.

In this embodiment of the present description, the sample structured feature and the sample unstructured feature may be input to a second prediction network, and a second prediction result is obtained by performing prediction processing on the delivery parameters of the sample multimedia.

The first prediction result and the second prediction result may be click rate prediction results, conversion rate prediction results, and the like corresponding to the sample multimedia.

In step S207, a second prediction network is trained according to the sample label, the first prediction result, and the second prediction result, so as to obtain a pre-ranking model.

In this embodiment of the present description, loss information may be determined according to the sample label, the first prediction result, and the second prediction network may be trained based on the loss information to obtain a pre-ranking model. For example, the second prediction network satisfying a preset condition may be determined as the pre-ranking model, where the preset condition may include that the loss is less than the loss threshold, a model extraction event is detected, and the like, which is not limited by the present disclosure.

In one possible implementation, as shown in fig. 5, the step S207 may include:

in step S501, target loss information is determined based on the first prediction result, the second prediction result, and the sample label. As an example, the step S501 may include:

and obtaining target loss information based on the third loss information and the fourth loss information. For example, a weighted sum of the third loss information and the fourth loss information may be used as the target loss information.

The determination of the third loss information may be performed based on a first preset loss function, where the first preset loss function may include a KL Divergence (Kullback-Leibler Divergence) loss function, so as to effectively learn the predicted probability distribution output by the preset model. The determination of the fourth loss information may be performed based on a second preset loss function, and the second preset loss function may include a cross entropy function, which is not limited by the present disclosure. Determining the second by two-part loss informationTarget loss information of the prediction network enables the output of the sample label and the preset model to effectively supervise and guide the learning of the second prediction network. In one example, the target loss information may be obtained using the following formula

：

Wherein:

wherein the content of the first and second substances,

is the third loss information;

is the fourth loss information;

a weight of the third loss information;

a weight of the fourth loss information;

for a first prediction result of the ith sample data,

a second prediction result of the ith sample data;

is a sample label of the ith sample data.

In step S503, a second prediction network is trained based on the target loss information, and a pre-ranking model is obtained.

In practical application, the gradient information can be determined based on the target loss information, so that the adjustment of the model parameters of the second prediction network can be realized based on a gradient reverse transfer method. In the continuous training process, a second prediction network can be periodically acquired as a pre-ranking model to perform screening of online resource delivery. The mode of obtaining the pre-sequencing model through the teacher network training student network training can improve the precision and the training efficiency of the pre-sequencing model.

It should be noted that a fixed preset model may be used to guide the training of the pre-training model, and based on this, in the iterative process of the training, the process may return to S403, and S403 to S503 are repeated, that is, S401 may be executed only in the first iteration, and the sample unstructured features are not changed in the training process of the second prediction network, and may be regarded as static features. Or the training of the fine ranking may be performed while training the second prediction network, that is, the parameters of the second prediction network and the parameters of the preset model are updated simultaneously in the same iteration cycle, which may be specifically referred to as the related content in fig. 6 below.

FIG. 6 is a flow diagram illustrating another method of model training in accordance with an exemplary embodiment. In a possible implementation manner, the preset model may be trained simultaneously with the pre-ranking model, in this case, as shown in fig. 6, before the step S503, the method may further include:

in step S601, determining first loss information according to the first prediction result and the sample label; the first loss information may be determined using a cross-entropy function, which is not limited by this disclosure.

In step S603, updating parameters of the first prediction network and parameters of the non-structural feature extraction network based on the first loss information to obtain an updated preset model; here, the parameter update of the network may be performed based on a gradient reverse transfer method, which is not limited by the present disclosure.

Accordingly, the step S503 may include:

in step S605, updating parameters of the second prediction network based on the target loss information to obtain an updated second prediction network; here, the updating of the parameters of the network may be performed based on a gradient reverse transfer method, which is not limited by the present disclosure.

In step S607, the above steps S301 to S603 are repeated according to the updated preset model and the updated second prediction network.

In practical applications, steps S301 to S603 may be repeated according to the updated preset model and the updated second prediction network. That is, when an iteration cycle is finished, training of the next iteration cycle may be performed, that is, steps S301 to S603 are repeated, and in the next iteration cycle, the preset model and the second prediction network in steps S301 to S603 may be the updated preset model and the updated second prediction network in the previous iteration cycle, so as to implement continuous iterative training of the preset model and the second prediction network.

In step S609, a second prediction network corresponding to a preset time is obtained as the pre-ranking model.

In the embodiment of the present specification, the preset time may be a plurality of time points that are preset and arranged in sequence, and the interval between every two adjacent time points in the plurality of time points may be the same, for example, 8 points, 12:00, and 16:00 of each day, which is not limited by the present disclosure.

Optionally, a preset model corresponding to the preset time may also be obtained as the target preset model. The corresponding preset model and the corresponding pre-sequencing model are obtained at the same time, so that the pre-sequencing model and the pre-sequencing model are high in adaptation degree, and the task can be predicted on line more accurately.

Through setting the preset model and training the pre-sequencing model simultaneously, the unstructured features can fully and dynamically participate in a prediction task (sample label), so that the unstructured features can be more accurately expressed, and the precision of the pre-sequencing model can be further improved.

In a possible implementation manner, target loss information in the pre-ranking model training can be reversely transmitted to the preset model, so that a joint training mode of the pre-ranking model and the preset model is implemented, and the following specific description is provided.

In a possible implementation manner, after the step S601, the method may further include: determining second loss information according to the first loss information and the target loss information;

accordingly, step S603 may include: and updating the parameters of the first prediction network and the parameters of the non-structural feature extraction network based on the second loss information to obtain an updated preset model.

In practical applications, the sum or product of the first loss information and the target loss information may be used as the second loss information. And determining corresponding gradient information based on the second loss information, and further performing gradient reverse transmission based on the gradient information to update the parameters of the first prediction network and the parameters of the non-structural feature extraction network to obtain an updated preset model.

Through the loss of presetting model self and the loss of the model of sequencing in advance, update the network parameter of presetting the model together, realize the joint training of the model of sequencing in advance and presetting the model, when putting in the application, because the multi-media of the model screening of sequencing in advance needs further carry out the secondary screening by presetting the model, consequently this kind of joint training's mode can be so that put in the application, based on the model of sequencing in advance and the joint screening of presetting the model more accurate, promote and put in the effect.

FIG. 7 is a flow chart illustrating a method of resource placement in accordance with an exemplary embodiment. As shown in fig. 7, may include:

in step S701, a recall resource matching the target object and structured features and unstructured features corresponding to the recall resource are acquired.

In this embodiment of the present specification, the structured feature may be obtained based on a resource attribute feature of the recalled resource and an object attribute feature of the target object; the unstructured features can be obtained by performing offline extraction on unstructured information based on a target unstructured feature extraction network, the unstructured information can represent various media information of recalled resources and interaction information of a target object, and the target unstructured feature extraction network can be an unstructured feature extraction network in a target preset model. The recall resource may be obtained by matching the target object tag and the multimedia tags in the multimedia application, which is not limited in this disclosure. The recalled resource may be recalled multimedia and, accordingly, the resource attribute feature may refer to a multimedia attribute feature.

It should be noted that, in the online application of resource delivery, since the pre-ranking model does not include a target unstructured feature extraction network for extracting unstructured features, extraction of unstructured features can be performed in advance under an offline condition, so that delivery efficiency can be ensured.

The step S701 can be realized by referring to the step S201, which is not described herein again.

In step S703, the structural features and the unstructured features corresponding to the recalled resources are input into the pre-ranking model and the target preset model, and a resource set to be released is screened out.

In one example, the structural features and the non-structural features corresponding to the recalled resources may be input into the pre-ranking model and the target preset model, respectively, so as to obtain a first output result of the pre-ranking model and a second output result of the target preset model; in turn, a first set of resources may be filtered from the recalled resources based on the first output result, and a second set of resources may be filtered from the recalled resources based on the second output result. And taking the coincident resources in the first resource set and the second resource set as resource sets to be released. Wherein, the first output result and the second output result can be click rate prediction information, conversion rate prediction information and the like.

In another example, the structured features and the unstructured features corresponding to the recalled resources may be input into the pre-ranking model to obtain a first output result of the pre-ranking model, and further, the initial resource set may be screened from the recalled resources based on the first output result. And the structural characteristics and the non-structural characteristics corresponding to each resource in the initial resource set can be input into a target preset model, and the resource set to be launched is screened out. Specifically, the following steps may be included:

In this embodiment of the present description, the structural features and the unstructured features corresponding to the recalled resources (recalled multimedia) may be input into the pre-ranking model to perform the delivery parameter prediction processing, so as to obtain the first delivery parameter information corresponding to the recalled resources, and the specific implementation manner may be referred to step S205, which is not described herein again. The first delivery parameter information and the second delivery parameter information below may be prediction tasks corresponding to sample labels, such as click rate prediction information and conversion rate prediction information.

In practical application, the number of multimedia included in the recalled resources may be thousands of orders, and based on this, the thousands of recalled resources may be sorted according to the first delivery parameter information, so that hundreds of orders of multimedia may be screened from the recalled resources based on the sorting as the initial multimedia set. For example, 200 multimedia with higher click through rate can be screened from the recalled resources as the initial multimedia set.

In this embodiment of the present description, the unstructured information corresponding to each multimedia in the initial multimedia set may be input to the unstructured feature extraction network of the target preset model, so as to obtain the unstructured features corresponding to the preset model, that is, the unstructured features corresponding to each multimedia in the initial multimedia set. Therefore, the unstructured features and the corresponding structured features corresponding to all the multimedia in the initial multimedia set can be input into the first prediction network of the target preset model, and second release parameter information is obtained.

Further, a multimedia set to be released can be screened from the initial multimedia set according to the second releasing parameter information; the number of resources in the resource set to be released may be set according to a requirement, for example, 9, which is not limited by the present disclosure.

In step S705, the resource set to be released is released to the target object; for example, the 9 resources may be delivered to the terminal of the target object, so as to present the 9 resources, for example, 9 multimedia, to the target object.

By utilizing the pre-ordering model obtained by the training method in resource delivery, the prediction precision of pre-ordering is improved, so that the precision of an initial multimedia set screened from recalled resources can be improved, further, the prediction of a preset model based on the initial multimedia set can be more accurate, and the overall precision of resource delivery is realized; and the network for extracting the non-structural features is not included in the pre-sequencing model, so that the structure is simple, and the prediction efficiency is higher.

Fig. 8 is a block diagram illustrating a model training apparatus in a delivery system, according to an example embodiment. Referring to fig. 8, the apparatus may include:

a sample acquiring module 801 configured to perform acquiring a plurality of sample data and corresponding sample tags, where the sample data includes sample structured features and sample unstructured information; the sample structured feature is obtained based on multimedia attribute information of the sample multimedia and object attribute information of the sample object; the unstructured information represents a plurality of media information of sample multimedia and interaction information of sample objects;

a first obtaining module 803, configured to perform input of the sample structured features and the sample unstructured information into a preset model, and perform prediction processing on the delivery parameters of the sample multimedia to obtain a first prediction result and sample unstructured features;

a second obtaining module 805 configured to perform input of the sample structured features and the sample unstructured features into a second prediction network, and perform prediction processing on the delivery parameters of the sample multimedia to obtain a second prediction result;

a training module 807 configured to perform training the second prediction network according to the sample label, the first prediction result and the second prediction result, so as to obtain a pre-ranking model.

In one possible implementation, the preset model includes an unstructured feature extraction network and a first prediction network; the first obtaining module 803 may include:

the sample unstructured feature extraction unit is configured to input the sample unstructured information into an unstructured feature extraction network, and perform feature extraction processing to obtain sample unstructured features;

and the first prediction result acquisition unit is configured to input the sample structured features and the sample unstructured features into a first prediction network, and perform prediction processing on the release parameters of the sample multimedia to obtain a first prediction result.

In a possible implementation manner, the training module 807 may include:

and the training unit is configured to train the second prediction network based on the target loss information to obtain a pre-ordering model.

In one possible implementation, the apparatus may further include:

accordingly, the training unit may include:

the iterative training subunit is configured to execute the step of repeatedly inputting the sample unstructured information into the unstructured feature extraction network in the preset model according to the updated preset model and the updated second prediction network, and perform feature extraction processing to obtain the sample unstructured feature, until the step of updating the parameters of the second prediction network based on the target loss information to obtain the updated second prediction network;

and the pre-sequencing model acquisition subunit is configured to execute acquisition of a second prediction network corresponding to the preset time as the pre-sequencing model.

In one possible implementation, the apparatus may further include:

and the preset model updating module is also configured to update the parameters of the first prediction network and the parameters of the non-structural feature extraction network based on second loss information determined according to the first loss information and the target loss information to obtain an updated preset model.

In one possible implementation manner, the multiple media information of the sample multimedia comprises sample image information, sample text information and sample audio information, and the non-structural feature extraction network comprises a convolutional neural network, a graph neural network and a coding and decoding network; the sample unstructured feature extraction unit includes:

the sample image feature extraction subunit is configured to input sample image information into the convolutional neural network, and perform feature extraction processing to obtain sample image features;

the sample interactive feature extraction subunit is configured to input the interactive information of the sample object into the neural network of the graph, and perform feature extraction processing to obtain sample interactive features;

and the unstructured feature extraction subunit is configured to perform splicing processing on the sample image features, the sample text features, the sample audio features and the sample interaction features to obtain unstructured features.

In a possible implementation manner, the target loss information determining unit includes:

and a target loss information obtaining subunit configured to perform obtaining the target loss information based on the third loss information and the fourth loss information.

In one possible implementation, the apparatus may further include:

and the target preset model acquisition module is configured to execute acquisition of a preset model corresponding to the preset time as the target preset model.

Fig. 9 is a block diagram illustrating a resource delivery apparatus according to an example embodiment. Referring to fig. 9, the apparatus may include:

an obtaining module 901 configured to perform obtaining a recall resource matched with a target object and a structured feature and an unstructured feature corresponding to the recall resource, where the structured feature is obtained based on a resource attribute feature of the recall resource and an object attribute feature of the target object; the unstructured features are obtained by performing off-line extraction on unstructured information based on a target unstructured feature extraction network;

a screening module 903, configured to perform input of the structural features and the unstructured features corresponding to the recalled resources into a pre-ranking model and a target preset model, and screen out a resource set to be released;

a delivering module 905 configured to perform delivering the resource set to be delivered to the target object;

the pre-ordering model is obtained based on the training method, and the target preset model is a preset putting parameter prediction model or a target preset model obtained based on the training method.

In a possible implementation manner, the screening module may include:

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

FIG. 10 is a block diagram illustrating an electronic device for model training or resource placement, which may be a server, whose internal structure diagram may be as shown in FIG. 10, according to an example embodiment. The electronic device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic equipment comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the electronic device is used for connecting and communicating with an external terminal through a network. The computer program is executed by a processor to implement a method of model training or resource delivery.

Those skilled in the art will appreciate that the architecture shown in fig. 10 is merely a block diagram of some of the structures associated with the disclosed aspects and does not constitute a limitation on the electronic devices to which the disclosed aspects apply, as a particular electronic device may include more or less components than those shown, or combine certain components, or have a different arrangement of components.

In an exemplary embodiment, there is also provided an electronic device including: a processor; a memory for storing the processor-executable instructions; wherein the processor is configured to execute the instructions to implement a method of model training or resource delivery as in embodiments of the present disclosure.

In an exemplary embodiment, a computer-readable storage medium is also provided, and when executed by a processor of an electronic device, the instructions in the computer-readable storage medium enable the electronic device to perform the method of model training or resource delivery in the embodiments of the present disclosure. The computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

In an exemplary embodiment, a computer program product containing instructions is also provided, which when run on a computer, causes the computer to perform the method of model training or resource placement in the embodiments of the present disclosure.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for model training in a delivery system, comprising:

2. The training method according to claim 1, wherein the preset model includes an unstructured feature extraction network and a first prediction network; the step of inputting the sample structured features and the sample unstructured information into a preset model, and performing prediction processing on the delivery parameters of the sample multimedia to obtain a first prediction result and sample unstructured features includes:

3. The training method of claim 2, wherein the training the second prediction network according to the sample label, the first prediction result, and the second prediction result to obtain a pre-ranking model comprises:

4. A training method according to claim 3, wherein before the step of training the second prediction network based on the target loss information to obtain the pre-ordering model, the method further comprises:

5. The training method of claim 4, wherein the step of determining first loss information based on the first prediction and the sample label further comprises:

updating the parameters of the first prediction network and the parameters of the non-structural feature extraction network based on the second loss information determined according to the first loss information and the target loss information to obtain an updated preset model.

6. The training method of claim 2, wherein the plurality of media information of the sample multimedia includes sample image information, sample text information, and sample audio information, and the unstructured feature extraction network includes a convolutional neural network, a graph neural network, and a codec network; the step of inputting the sample unstructured information into the unstructured feature extraction network for feature extraction processing to obtain the sample unstructured features comprises:

7. The training method of claim 3, wherein determining target loss information based on the first prediction, the second prediction, and the sample label comprises:

8. Training method according to claim 4 or 5, characterized in that the method further comprises:

9. A resource delivery method, comprising:

delivering the resource set to be delivered to the target object;

the pre-ranking model is obtained based on the training method of any one of claims 1 to 7, and the target preset model is a preset putting parameter prediction model or the target preset model obtained based on claim 8.

10. The method according to claim 9, wherein the inputting the structured features and the unstructured features corresponding to the recalled resources into a pre-ranking model and a target pre-setting model to screen out a set of resources to be released comprises:

11. A model training device in a delivery system, comprising:

12. The training apparatus of claim 11, wherein the preset model comprises an unstructured feature extraction network and a first prediction network; the first obtaining module comprises:

13. The training device of claim 12, wherein the training module comprises:

14. An exercise device as recited in claim 13, wherein the device further comprises:

accordingly, the training unit comprises:

15. The training device of claim 14, further comprising:

16. The training apparatus of claim 12, wherein the plurality of media information of the sample multimedia includes sample image information, sample text information, and sample audio information, and the unstructured feature extraction network includes a convolutional neural network, a graph neural network, and a codec network; the sample unstructured feature extraction unit includes:

17. The training apparatus according to claim 13, wherein the target loss information determination unit includes:

18. An exercise device as recited in claim 14 or 15, wherein the device further comprises:

19. A resource delivery apparatus, comprising:

20. The apparatus of claim 19, wherein the screening module comprises:

21. An electronic device, comprising:

a processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to execute the instructions to implement the model training method of any one of claims 1 to 8 or to implement the resource placement method of any one of claims 9-10.

22. A computer-readable storage medium, wherein instructions in the computer-readable storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the model training method of any one of claims 1 to 8 or perform the resource placement method of any one of claims 9-10.

23. A computer program product comprising computer instructions, characterized in that the computer instructions, when executed by a processor, implement the model training method of any one of claims 1 to 8 or implement the resource placement method of any one of claims 9-10.