CN114117206B

CN114117206B - Recommendation model processing method and device, electronic equipment and storage medium

Info

Publication number: CN114117206B
Application number: CN202111320914.2A
Authority: CN
Inventors: 谈建超; 陆顺; 李吉祥; 朱文涛; 杨森; 刘霁
Original assignee: Beijing Dajia Internet Information Technology Co Ltd
Current assignee: Beijing Dajia Internet Information Technology Co Ltd
Priority date: 2021-11-09
Filing date: 2021-11-09
Publication date: 2023-05-30
Anticipated expiration: 2041-11-09
Also published as: CN114117206A

Abstract

The disclosure relates to a recommendation model processing method, a recommendation model processing device, electronic equipment and a storage medium, and belongs to the technical field of computers. The method comprises the following steps: acquiring a recommendation model, wherein the recommendation model comprises a plurality of operators arranged in sequence, and the operators sequentially process account data and article data input into the recommendation model in sequence to obtain recommendation parameters; based on the recommendation model, obtaining operator characteristics corresponding to each operator in the recommendation model and structural characteristics of the recommendation model, wherein the structural characteristics represent a connection relationship between any two operators in the plurality of operators; extracting features based on operator features and structural features corresponding to each operator to obtain model features corresponding to the recommended model; and carrying out precision prediction based on the model characteristics to obtain the recommendation precision corresponding to the recommendation model. The model features obtained by the method can more accurately represent the recommendation model, so that recommendation accuracy obtained based on the model feature prediction is more accurate, and accuracy of the recommendation accuracy is improved.

Description

Recommendation model processing method and device, electronic equipment and storage medium

Technical Field

The disclosure relates to the field of computer technology, and in particular relates to a recommendation model processing method, a recommendation model processing device, electronic equipment and a storage medium.

Background

With the development of computer technology, the construction of a network model by adopting a neural architecture search mode has become a common model construction method. Further, after obtaining the network model, it is generally required to determine the model accuracy corresponding to the network model, and determine whether the network model is practically applicable according to the model accuracy. For example, in a recommendation scenario, a recommendation accuracy corresponding to a recommendation model may be determined based on model features corresponding to the recommendation model, where how to obtain the model features corresponding to the recommendation model is a key to determining the recommendation accuracy.

Disclosure of Invention

The disclosure provides a recommendation model processing method, a recommendation model processing device, electronic equipment and a storage medium, and accuracy of determined recommendation precision is improved.

According to an aspect of the embodiments of the present disclosure, there is provided a recommendation model processing method, including:

acquiring a recommendation model, wherein the recommendation model comprises a plurality of operators which are arranged in sequence, and the operators sequentially process account data and article data which are input into the recommendation model in sequence to obtain recommendation parameters, wherein the recommendation parameters indicate whether to recommend the article to the account;

Based on the recommendation model, obtaining operator characteristics corresponding to each operator in the recommendation model and structural characteristics of the recommendation model, wherein the structural characteristics represent connection relations between any two operators in the plurality of operators;

extracting features based on operator features corresponding to each operator and the structural features to obtain model features corresponding to the recommendation model;

and carrying out precision prediction based on the model characteristics to obtain the recommendation precision corresponding to the recommendation model.

According to the method provided by the embodiment of the disclosure, each operator characteristic and each structure characteristic corresponding to the recommendation model are firstly obtained, then the operator characteristic and the structure characteristic are utilized to extract the effective model characteristic, the mode of obtaining the model characteristic can fully obtain the characteristic of the recommendation model, the obtained model characteristic can more accurately represent the recommendation model, and therefore recommendation accuracy obtained based on prediction of the model characteristic is more accurate, and accuracy of the recommendation accuracy is improved.

In some embodiments, the recommendation model further includes an index identifier corresponding to each operator, and the step of obtaining the operator feature corresponding to each operator includes:

Inquiring operator characteristics corresponding to each operator from an operator characteristic table based on the index identifier corresponding to each operator;

the operator feature table comprises index identifiers corresponding to at least one operator and operator features, wherein the operator features are obtained by extracting features of the operators.

In the embodiment of the disclosure, because the operator characteristics are already included in the operator characteristic table, the operator characteristics can be quickly obtained by querying the operator characteristic table, and the operator characteristics corresponding to the operators do not need to be temporarily extracted, so that the efficiency of obtaining recommendation precision is improved.

In some embodiments, the recommendation model includes N operators, N being an integer greater than 1, and the step of obtaining the structural feature includes:

creating a target matrix of N rows and N columns based on the N operators in the recommendation model, wherein the N operators are respectively used as N rows and N columns of the target matrix, the rows and the columns corresponding to the elements on the diagonal line in the target matrix are the same operators, the values of the elements on the diagonal line in the target matrix are equal to the number of other operators connected with the operators corresponding to the elements, the rows and the columns corresponding to the elements on other positions in the target matrix are two operators with different relationships, and the values of the elements on other positions indicate whether the two operators corresponding to the elements have a connection relationship;

And extracting the characteristics of the target matrix to obtain the structural characteristics.

In the embodiment of the disclosure, the target matrix can fully represent the connection relationship between any two operators in the recommendation model, namely, the topological structure information of the recommendation model is fully acquired, so that the structural features obtained based on the target matrix can accurately represent the structure of the recommendation model, and the accuracy of the structural features is improved.

In some embodiments, the extracting the operator features of each operator and the structural features to obtain model features corresponding to the recommendation model includes:

splicing the operator features corresponding to each operator and the structural features to obtain splicing features corresponding to the recommendation model;

and encoding the spliced features to obtain the model features.

In the embodiment of the disclosure, a plurality of operator features and structure features are respectively acquired, the obtained operator features can accurately represent operators in the recommendation model, and the structure features can accurately represent the structure of the recommendation model, so that the recommendation model can be accurately represented by splicing and encoding the operator features and the structure features to obtain the model features, and the accuracy of the model features is improved.

In some embodiments, the recommendation accuracy prediction model includes a feature extraction network and a prediction network,

the feature extraction network is used for extracting features based on operator features corresponding to each operator and the structural features to obtain the model features;

and the prediction network is used for predicting the precision of the model characteristics to obtain the recommended precision.

In some embodiments, the recommendation accuracy prediction model further includes a feature acquisition network,

the feature acquisition network is used for acquiring operator features corresponding to each operator in the recommendation model;

the feature acquisition network is further configured to acquire the structural feature of the recommendation model.

In the embodiment of the disclosure, compared with the mode of training the recommendation model in the related art and then testing the recommendation model by adopting the test data and determining the recommendation precision according to the test result, the recommendation precision is predicted by utilizing the recommendation precision prediction model, so that the recommendation precision can be quickly obtained, and the efficiency of obtaining the recommendation precision is improved.

In some embodiments, the training process of the recommendation accuracy prediction model includes:

acquiring sample operator characteristics, first sample structural characteristics and first sample recommendation precision corresponding to each sample operator in a first sample recommendation model, wherein the first sample structural characteristics represent the connection relationship between any two sample operators in the first sample recommendation model;

Invoking the recommendation precision prediction model, and processing sample operator characteristics and first sample structural characteristics corresponding to each sample operator in the first sample recommendation model to obtain first prediction recommendation precision;

the recommendation accuracy prediction model is trained based on the first prediction recommendation accuracy and the first sample recommendation accuracy.

In some embodiments, the recommendation model processing method further comprises:

inputting the account data and the item data to the recommendation model;

invoking a first operator in the recommendation model, and processing the account data and the article data to obtain output data of the first operator;

and taking the output data of the first operator as the input data of the next operator connected with the first operator, calling the next operator to process the output data until the output data of the last operator in the recommendation model is obtained, and determining the output data of the last operator as the recommendation parameter.

In some embodiments, the acquiring a recommendation model includes:

acquiring a plurality of recommendation models, wherein different operators exist in any two recommendation models, or the connection relations between the operators in any two recommendation models are different;

The method for processing the recommendation model further comprises the following steps of:

selecting a target recommendation model corresponding to the maximum recommendation precision from the plurality of recommendation models based on the recommendation precision corresponding to the plurality of recommendation models;

and calling the target recommendation model to recommend.

In the embodiment of the disclosure, based on the recommendation precision corresponding to the plurality of recommendation models, the most accurate recommendation model can be selected from the plurality of recommendation models, so that when the selected recommendation model is applied to an actual recommendation scene, accurate recommendation can be performed based on the recommendation model, and the recommendation accuracy is improved.

In some embodiments, the recommendation model further includes a plurality of edges, each edge being connected to two operators, the operators being configured to process input data input to the operators to obtain output data; and, the output data of one operator connected by each edge is used as the input data of the other operator connected.

According to still another aspect of the embodiments of the present disclosure, there is provided a recommendation accuracy prediction model training method, the method including:

invoking a recommendation precision prediction model, extracting features of sample operator features and first sample structural features corresponding to each sample operator in the first sample recommendation model to obtain model features corresponding to the first sample recommendation model, and performing precision prediction based on the model features corresponding to the first sample recommendation model to obtain first prediction recommendation precision;

According to the method provided by the embodiment of the disclosure, through training the recommendation precision prediction model for predicting the recommendation precision, each operator characteristic and each structure characteristic corresponding to the recommendation model can be processed by using the recommendation precision prediction model, and the operator characteristic and the structure characteristic are used for extracting effective model characteristics so as to fully acquire the characteristics of the recommendation model, so that the obtained model characteristics can more accurately represent the recommendation model, and the recommendation precision predicted by the recommendation precision prediction model based on the model characteristics is more accurate, and the accuracy of the recommendation precision is improved.

In some embodiments, the recommendation accuracy prediction model training method further comprises:

acquiring sample operator characteristics, second sample structure characteristics and proxy recommendation precision corresponding to each sample operator in a second sample recommendation model, wherein the second sample structure characteristics represent a connection relationship between any two sample operators in the second sample recommendation model, and the proxy recommendation precision is a variable used for indicating the recommendation precision of the second sample recommendation model;

invoking the recommendation precision prediction model, extracting features of sample operator features and second sample structure features corresponding to each sample operator in the second sample recommendation model to obtain model features corresponding to the second sample recommendation model, and performing precision prediction based on the model features corresponding to the second sample recommendation model to obtain second prediction recommendation precision;

the training the recommendation accuracy prediction model based on the first prediction recommendation accuracy and the first sample recommendation accuracy includes:

the recommendation accuracy prediction model is trained based on the first prediction recommendation accuracy, the first sample recommendation accuracy, the second prediction recommendation accuracy, and the proxy recommendation accuracy.

In the embodiment of the disclosure, when the recommendation precision prediction model is trained, not only the first sample recommendation model is adopted to train the recommendation precision prediction model, but also the second sample recommendation model is adopted to train the recommendation precision prediction model, so that more sample models for training are provided, and the accuracy and generalization capability of the recommendation precision prediction model obtained by training are improved.

In some embodiments, the training the recommendation accuracy prediction model based on the first prediction recommendation accuracy, the first sample recommendation accuracy, the second prediction recommendation accuracy, and the proxy recommendation accuracy comprises:

acquiring a first error between the first prediction recommendation precision and the first sample recommendation precision, and a second error between the second prediction recommendation precision and the second sample recommendation precision;

and adjusting model parameters in the recommendation precision prediction model and the proxy recommendation precision based on the first error and the second error, so that the error between the prediction recommendation precision corresponding to the historical moment of the second sample recommendation model and the adjusted proxy recommendation precision approaches to a target error corresponding to the historical moment.

In some embodiments, the training the recommendation accuracy prediction model based on the first prediction recommendation accuracy, the first sample recommendation accuracy, the second prediction recommendation accuracy, and the proxy recommendation accuracy uses the following penalty function:

wherein L represents the loss function, f _θ (. Cndot.) represents the recommended precision prediction model, θ represents model parameters in the recommended precision prediction model, f _θ (x _i ) Representing the first prediction recommendation precision, x, corresponding to the ith first sample recommendation model _i Representing sample operator features and the first sample structural features corresponding to each sample operator in the ith first sample recommendation model, n represents the number of the first sample recommendation models used for training the recommendation accuracy prediction model, y _i Representing the first sample recommendation precision corresponding to the ith first sample recommendation model, f _θ (v _j ) Representing the second prediction recommendation precision, v, corresponding to the j-th second sample recommendation model _j Representing sample operator features and the second sample structure features corresponding to each sample operator in the j-th second sample recommendation model, V representing the number of the second sample recommendation models used for training the recommendation accuracy prediction model,

Representing the agent recommendation accuracy corresponding to the jth second sample recommendation model,/for each agent>

Representing agent recommended essenceDegree, alpha is a reference parameter;

the loss function satisfies the following condition:

wherein V represents the number of the second sample recommendation models used to train the recommendation accuracy prediction model,

Representing the prediction recommendation precision, e, corresponding to the jth second sample recommendation model at the t moment ^(t) The target error corresponding to the T-th time is represented, and T represents T historical times.

In the embodiment of the disclosure, when the recommendation precision prediction model is trained, not only the first sample recommendation model is adopted to train the recommendation precision prediction model, but also the second sample recommendation model is adopted to train the recommendation precision prediction model, and in the training process, the agent recommendation precision corresponding to the second sample recommendation model also needs to meet the condition corresponding to the historical moment, namely, not only the training data in the current training process but also the training data of the historical moment are considered, and the self-evolution optimization framework aiming at the second sample recommendation model is added, so that the training efficiency is improved, and the accuracy of the recommendation precision prediction model obtained by training is also improved.

According to still another aspect of the embodiments of the present disclosure, there is provided a classification model processing method, the method including:

the method comprises the steps that a classification model is obtained, the classification model comprises a plurality of operators which are arranged in sequence, and the operators process data to be classified, which are input into the classification model, in sequence to obtain data types corresponding to the data;

based on the classification model, obtaining operator characteristics corresponding to each operator in the classification model and structural characteristics of the classification model, wherein the structural characteristics represent connection relations between any two operators in the plurality of operators;

extracting features based on operator features corresponding to each operator and the structural features to obtain model features corresponding to the classification model;

and carrying out precision prediction based on the model features to obtain the classification precision corresponding to the classification model.

According to the method provided by the embodiment of the disclosure, each operator characteristic and each structure characteristic corresponding to the classification model are firstly obtained, then the operator characteristic and the structure characteristic are utilized to extract the effective model characteristic, the mode of obtaining the model characteristic can fully obtain the characteristic of the classification model, so that the obtained model characteristic can more accurately represent the classification model, the classification precision obtained based on the model characteristic prediction is more accurate, and the accuracy of the classification precision is improved.

According to still another aspect of the embodiments of the present disclosure, there is provided a model processing apparatus including:

a recommendation model obtaining unit configured to perform obtaining a recommendation model, where the recommendation model includes a plurality of operators arranged in order, and the plurality of operators sequentially process account data and item data input to the recommendation model in order to obtain recommendation parameters, where the recommendation parameters indicate whether to recommend the item to the account;

a first feature acquisition unit configured to perform, based on the recommendation model, acquiring operator features corresponding to each operator in the recommendation model and structural features of the recommendation model, the structural features representing a connection relationship between any two operators in the plurality of operators;

the second feature acquisition unit is configured to perform feature extraction based on the operator features corresponding to each operator and the structural features to obtain model features corresponding to the recommendation model;

and the recommendation precision prediction unit is configured to execute precision prediction based on the model characteristics to obtain recommendation precision corresponding to the recommendation model.

In some embodiments, the recommendation model further includes an index identifier corresponding to each operator, and the first feature obtaining unit includes:

An operator feature obtaining subunit configured to perform querying, from an operator feature table, an operator feature corresponding to each operator based on the index identifier corresponding to each operator;

In some embodiments, the recommendation model includes N operators, N being an integer greater than 1, and the first feature acquisition unit includes:

a structural feature obtaining subunit, configured to execute creating a target matrix of N rows and N columns based on the N operators in the recommendation model, where the N operators are respectively used as N rows and N columns of the target matrix, rows and columns corresponding to elements on a diagonal line in the target matrix are identical operators, values of the elements on the diagonal line in the target matrix are equal to the number of other operators connected with the operators corresponding to the elements, rows and columns corresponding to the elements on other positions in the target matrix are two different operators, and the values of the elements on other positions represent whether the two operators corresponding to the elements have a connection relationship;

The structural feature acquisition subunit is further configured to perform feature extraction on the target matrix to obtain the structural feature.

In some embodiments, the second feature acquisition unit includes:

the feature splicing subunit is configured to splice the operator features corresponding to each operator and the structural features to obtain splicing features corresponding to the recommendation model;

and the coding subunit is configured to perform coding on the spliced characteristic to obtain the model characteristic.

In some embodiments, the recommendation model processing apparatus further comprises:

a model training unit configured to perform obtaining a sample operator feature, a first sample structural feature, and a first sample recommendation precision corresponding to each sample operator in a first sample recommendation model, the first sample structural feature representing a connection relationship between any two sample operators in the first sample recommendation model;

the model training unit is further configured to execute the recommendation precision prediction model, and process sample operator features and first sample structure features corresponding to each sample operator in the first sample recommendation model to obtain a first prediction recommendation precision;

the model training unit is further configured to perform training of the recommendation accuracy prediction model based on the first prediction recommendation accuracy and the first sample recommendation accuracy.

a recommendation unit configured to perform inputting the account data and the item data to the recommendation model;

the recommending unit is further configured to execute calling a first operator in the recommending model, process the account data and the article data, and obtain output data of the first operator;

The recommending unit is further configured to execute that the output data of the first operator is used as the input data of a next operator connected with the first operator, call the next operator to process the output data until the output data of a last operator in the recommending model is obtained, and determine the output data of the last operator as the recommending parameter.

In some embodiments, the obtaining model unit is configured to perform obtaining a plurality of recommendation models, where operators in any two recommendation models exist differently, or a connection relationship between operators in any two recommendation models is different;

the recommendation model processing apparatus further includes:

a model selection unit configured to perform selection of a target recommendation model corresponding to a maximum recommendation precision from the plurality of recommendation models based on recommendation precision corresponding to the plurality of recommendation models;

and the recommending unit is configured to execute the calling of the target recommending model to recommend.

According to still another aspect of the embodiments of the present disclosure, there is provided a recommendation accuracy prediction model training apparatus, the apparatus including:

the sample acquisition unit is configured to acquire sample operator characteristics, first sample structural characteristics and first sample recommendation precision corresponding to each sample operator in a first sample recommendation model, wherein the first sample structural characteristics represent the connection relation between any two sample operators in the first sample recommendation model;

the precision prediction unit is configured to execute calling of a recommendation precision prediction model, perform feature extraction on sample operator features and first sample structural features corresponding to each sample operator in the first sample recommendation model to obtain model features corresponding to the first sample recommendation model, and perform precision prediction based on the model features corresponding to the first sample recommendation model to obtain first prediction recommendation precision;

and a model training unit configured to perform training of the recommendation accuracy prediction model based on the first prediction recommendation accuracy and the first sample recommendation accuracy.

In some embodiments, the sample acquisition unit is further configured to perform: acquiring sample operator characteristics, second sample structure characteristics and proxy recommendation precision corresponding to each sample operator in a second sample recommendation model, wherein the second sample structure characteristics represent a connection relationship between any two sample operators in the second sample recommendation model, and the proxy recommendation precision is a variable used for indicating the recommendation precision of the second sample recommendation model;

The precision prediction unit is configured to execute calling of the recommendation precision prediction model, perform feature extraction on sample operator features and second sample structure features corresponding to each sample operator in the second sample recommendation model to obtain model features corresponding to the second sample recommendation model, and perform precision prediction based on the model features corresponding to the second sample recommendation model to obtain second prediction recommendation precision;

the model training unit is configured to perform training of the recommendation precision prediction model based on the first prediction recommendation precision, the first sample recommendation precision, the second prediction recommendation precision, and the proxy recommendation precision.

In some embodiments, the model training unit is configured to perform:

the recommendation accuracy prediction model is trained based on the first prediction recommendation accuracy, the first sample recommendation accuracy, the second prediction recommendation accuracy and the proxy recommendation accuracy by adopting the following loss function:

/>

Representing agent recommendation precision, wherein alpha is a reference parameter;

the loss function satisfies the following condition:

According to still another aspect of the embodiments of the present disclosure, there is provided a classification model processing apparatus including:

the classification model acquisition unit is configured to execute acquisition of a classification model, wherein the classification model comprises a plurality of operators which are arranged in sequence, and the operators process data to be classified, which are input into the classification model, in sequence to obtain data types corresponding to the data;

a first feature acquisition unit configured to perform, based on the classification model, acquisition of operator features corresponding to each operator in the classification model and structural features of the classification model, the structural features representing a connection relationship between any two operators in the plurality of operators;

The second feature acquisition unit is configured to perform feature extraction based on the operator features corresponding to each operator and the structural features to obtain model features corresponding to the classification model;

and the classification precision prediction unit is configured to perform precision prediction based on the model features to obtain classification precision corresponding to the classification model.

According to still another aspect of the embodiments of the present disclosure, there is provided an electronic device including:

one or more processors;

a memory for storing the one or more processor-executable instructions;

wherein the one or more processors are configured to perform the recommendation model processing method of the above aspect, or to perform the recommendation accuracy prediction model training method of the above aspect, or to perform the classification model processing method of the above aspect.

According to still another aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium, which when executed by a processor of an electronic device, causes the electronic device to perform the recommendation model processing method described in the above aspect, or to perform the recommendation accuracy prediction model training method described in the above aspect, or to perform the classification model processing method described in the above aspect.

According to still another aspect of the embodiments of the present disclosure, there is provided a computer program product including a computer program that is executed by a processor to implement the recommended model processing method described in the above aspect, or to implement the recommended accuracy prediction model training method described in the above aspect, or to implement the classification model processing method described in the above aspect.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.

FIG. 1 is a flowchart illustrating a recommendation model processing method, according to an example embodiment.

FIG. 2 is a flowchart illustrating another recommendation model processing method, according to an example embodiment.

FIG. 3 is a flow chart illustrating a neural architecture search approach, according to an example embodiment.

Fig. 4 is a schematic diagram of a directed acyclic graph, according to an example embodiment.

Fig. 5 is a schematic diagram of a target matrix shown according to an example embodiment.

FIG. 6 is a schematic diagram illustrating a recommendation accuracy prediction model, according to an example embodiment.

FIG. 7 is a schematic diagram illustrating another recommendation accuracy prediction model, according to an example embodiment.

FIG. 8 is a flowchart illustrating yet another recommendation model processing method, according to an example embodiment.

FIG. 9 is a schematic diagram illustrating a recommendation model processing method, according to an example embodiment.

FIG. 10 is a schematic diagram illustrating a recommendation accuracy prediction model training method, according to an example embodiment.

FIG. 11 is a schematic diagram illustrating a recommendation accuracy variation, according to an example embodiment.

FIG. 12 is a flowchart illustrating a classification model processing method according to an exemplary embodiment.

FIG. 13 is a block diagram illustrating a recommendation model processing device, according to an example embodiment.

FIG. 14 is a block diagram illustrating another recommendation model processing device, according to an example embodiment.

FIG. 15 is a block diagram illustrating a recommendation accuracy prediction model training apparatus, according to an example embodiment.

Fig. 16 is a block diagram illustrating a classification model processing apparatus according to an exemplary embodiment.

Fig. 17 is a block diagram illustrating a structure of a terminal according to an exemplary embodiment.

Fig. 18 is a block diagram illustrating a structure of a server according to an exemplary embodiment.

Detailed Description

In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

It should be noted that the terms "first," "second," and the like in the description of the present disclosure and the claims and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the disclosure described herein may be capable of operation in sequences other than those illustrated or described herein. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as detailed in the accompanying claims.

The terms "at least one", "a plurality", "each", "any" and the like as used in this disclosure include one, two or more, and a plurality includes two or more, each referring to each of the corresponding plurality, and any one refers to any one of the plurality. For example, the plurality of operators includes 3 operators, and each operator refers to each operator of the 3 operators, and any operator refers to any operator of the 3 operators, which may be the first operator, the second operator, or the third operator.

The data (including but not limited to user equipment information, user personal information, etc.) related to the present disclosure is information authorized by the user or sufficiently authorized by each party.

The execution subject in the present application is an electronic device. Optionally, the electronic device is a terminal or a server. The terminal can be a mobile phone, a tablet personal computer, a computer and other types of terminals, and the server is a server, or a server cluster formed by a plurality of servers, or a cloud computing service center.

FIG. 1 is a flowchart illustrating a recommendation model processing method, see FIG. 1, according to an exemplary embodiment, the method being performed by an electronic device, comprising the steps of:

in step 101, the electronic device obtains a recommendation model.

The recommendation model comprises a plurality of operators which are arranged in sequence, the operators process account data and article data which are input into the recommendation model in sequence to obtain recommendation parameters, and the recommendation parameters indicate whether the article is recommended to the account. Each operator is used for processing input data input to the operator to obtain output data, the input data of a first operator is account data and article data, and the input data of other operators except the first operator is the output data of the last operator connected with the other operators.

In step 102, the electronic device obtains operator features corresponding to each operator in the recommendation model and structural features of the recommendation model based on the recommendation model.

The operator features are used for describing corresponding operators, and the structural features of the recommendation model represent the connection relation between any two operators in the plurality of operators, namely the topological structure information of the recommendation model.

In step 103, the electronic device performs feature extraction based on the operator features and the structural features corresponding to each operator, so as to obtain model features corresponding to the recommended model.

For a recommendation model, in order to obtain accurate recommendation accuracy, first, a model feature corresponding to the recommendation model needs to be extracted, in the embodiment of the present disclosure, operator features and structure features corresponding to each operator are obtained first, and the operator features and the structure features corresponding to each operator respectively represent part of features of the recommendation model, so in order to obtain accurate features for describing the recommendation model, further feature extraction needs to be performed on the operator features and the structure features corresponding to each operator, so that a plurality of operator features and structure features are fully fused together to obtain model features corresponding to the recommendation model.

In step 104, the electronic device performs accuracy prediction based on the model features, so as to obtain recommendation accuracy corresponding to the recommendation model.

The electronic equipment processes the model characteristics so as to predict the recommendation precision corresponding to the recommendation model, wherein the recommendation precision is used for representing the accuracy of the recommendation model. Then, whether the recommendation model is practically applicable or not can be determined based on the recommendation accuracy.

FIG. 2 is a flowchart illustrating another recommendation model processing method, see FIG. 2, whose subject of execution is an electronic device, according to an exemplary embodiment, comprising the steps of:

in step 201, the electronic device obtains a recommendation model.

The recommendation model comprises a plurality of operators and a plurality of edges which are arranged in sequence, each edge is connected with two operators, each operator is connected with at least one edge, namely, each operator is connected with other operators through at least one edge, namely, one operator can be connected with the other operator through one edge, and can also be respectively connected with the operators through the edges. The operators are used for processing input data input to the operators to obtain output data, and the output data of one operator connected with each edge is used as the input data of the other operator connected with each other. The operator may be a convolution operator, a deconvolution operator, an activation function, a normalization operator, or other operator.

For the recommendation model, the input data of the recommendation model are account data and article data, wherein the account data comprises account information, description information corresponding to the account or other data related to the account, and the article data comprises article names, article description information or other data related to the account. Taking the example of recommending goods to the user, the account data comprises an account corresponding to the user and a user tag, and the goods data comprises goods names, goods description information, goods prices and other information. The output data of the recommendation model is a recommendation parameter, where the recommendation parameter indicates whether to recommend the item to the account, for example, if the recommendation parameter is 1, it indicates that the item is recommended to the account, and if the recommendation parameter is 0, it indicates that the item is not recommended to the account.

And the operators sequentially process the account data and the article data which are input into the recommendation model according to the sequence to obtain recommendation parameters. In some embodiments, the electronic device inputs account data and item data to a recommendation model; invoking a first operator in the recommendation model, and processing account data and article data to obtain output data of the first operator; and taking the output data of the first operator as the input data of the next operator connected with the first operator, calling the next operator to process the output data until the output data of the last operator in the recommendation model is obtained, and determining the output data of the last operator as recommendation parameters. The input data of a first operator in the recommendation model are account data and article data, and the input data of other operators except the first operator are the output data of the last operator connected with the other operators. The operators are sequentially processed, so that each operator can fully play a role, and the characteristics of account data and article data are gradually extracted in the processing process, so that accurate recommendation parameters are obtained, and article recommendation based on the recommendation model is realized.

In some embodiments, the recommendation model further includes an index identification for each operator, the index identification being used to uniquely represent a corresponding one of the operators. For example, the index is identified as an operator name, label, etc.

In some embodiments, the electronic device obtains the recommendation model using a neural architecture search. The neural architecture search (Neural Architecture Search, NAS) refers to automatically finding a recommendation model with excellent effect in a predefined search space. Referring to the neural architecture searching process shown in fig. 3, the neural architecture searching includes three parts, namely a search space, a search strategy and a model effect evaluation, wherein the search space refers to a predefined space containing a plurality of operators, the search strategy is predefined and is used for indicating how to start searching from one operator in the search space to obtain a recommendation model, the corresponding search strategy can be defined according to the recommendation model to be searched, and the model effect evaluation refers to predicting recommendation precision corresponding to the recommendation model obtained by searching to determine recommendation precision corresponding to the recommendation model obtained by searching. The embodiments of the present disclosure do not limit the process of constructing the recommendation model.

In some embodiments, the recommendation model is an artificially designed recommendation model.

In addition, the recommendation model may also be sent to the electronic device by other electronic devices, and the embodiment of the disclosure does not limit the obtaining manner of the recommendation model.

In some embodiments, the recommendation model is regarded as a directed acyclic graph (Directed Acyclic Graph, DAG), the nodes in the directed acyclic graph are operators in the recommendation model, the edges in the directed acyclic graph are connection relations between two operators, namely, edges between two nodes represent connection between two operators corresponding to the two nodes, and no edges between two nodes represent disconnection between two operators corresponding to the two nodes. For example, referring to fig. 2 corresponding to the directed acyclic graph 1 and the directed acyclic graph 2 corresponding to the recommended model 1 shown in fig. 4, the directed acyclic graph 1 includes an input operator, an output operator, four convolution operators 3*3 and two 1*1 convolution operators, each operator is connected according to the sequence shown in fig. 4, edges between the two operators are represented by arrows, a direction indicated by the arrows is a data stream direction, that is, output data of the operator connected by the tail of the arrows is input data of the operator pointed by the head of the arrows, and the directed acyclic graph 2 includes the input operator, the output operator, the four convolution operators 3*3 and one MP (Matching Pursuits, matching approximation) operator. And then, when the structural features of the recommendation model are acquired, acquiring the structural features based on the directed acyclic graph corresponding to the recommendation model.

It should be noted that, in the embodiment of the present disclosure, the number and types of operators included in the recommendation model and the connection relationship between operators are not limited, for example, referring to fig. 4, the number of operators, the types of operators and the connection relationship between operators in the two recommendation models shown in fig. 4 are not the same.

In step 202, the electronic device queries operator features corresponding to each operator from the operator feature table based on the index identifier corresponding to each operator in the recommendation model.

The operator feature table comprises at least one index identifier corresponding to the operator and operator features, namely, a corresponding relation exists between the index identifier and the operator features, and the electronic equipment can inquire the operator features corresponding to the index identifier from the operator feature table based on the index identifier. Operator features are used to describe the corresponding operator, which is obtained by feature extraction of the operator, and the operator features are vectors, matrices or other forms, for example, the operator features are matrices of N rows and M columns. The embodiments of the present disclosure do not limit the form of the operator features.

Because the operator features in the operator feature table are extracted in advance, the operator features can be quickly obtained by inquiring the operator feature table, and the operator features corresponding to the operators do not need to be temporarily extracted, so that the efficiency of obtaining recommendation accuracy is improved.

In some embodiments, the operator feature table is trainable, that is, after the electronic device obtains the operator feature table, the operator features in the operator feature table may be continuously updated, so as to continuously improve the accuracy of each operator feature in the operator feature table, and enable the operator features to accurately describe the corresponding operators.

It should be noted that, in the embodiment of the present disclosure, only the operator features are queried from the operator feature table, and in another embodiment, the electronic device may acquire the operator features corresponding to each operator in other manners. For example, the operator features corresponding to the operators are extracted by using a feature extraction model.

In step 203, the electronic device creates a target matrix based on a plurality of operators in the recommendation model, and performs feature extraction on the target matrix to obtain structural features of the recommendation model.

Taking the recommendation model as an example, the recommendation model comprises N operators, wherein N is an integer greater than 1, and correspondingly, the electronic equipment creates N rows and N columns of target matrixes based on the N operators in the recommendation model. The N operators are respectively used as N rows and N columns of a target matrix, the rows and columns corresponding to the elements on the diagonal lines in the target matrix are the same operators, the values of the elements on the diagonal lines in the target matrix are equal to the number of other operators connected with the operators corresponding to the elements, the rows and the columns corresponding to the elements on other positions in the target matrix are two different operators, and the values of the elements on other positions represent whether the two operators corresponding to the elements have a connection relation or not.

For example, referring to the target matrix shown in fig. 5, the recommendation model includes N operators, a first row in the target matrix corresponds to a first operator, a first column corresponds to the first operator, elements in the first row and the first column in the target matrix are the number of operators connected by the first operator, elements in the first row and the second column in the target matrix are-1, which indicates that the first operator is connected with the second operator, and elements in the first row and the N column are 0, which indicates that the first operator is not connected with the N operator.

In some embodiments, since the direction is between two operators connected in the recommendation model, elements at other locations in the target matrix can also represent the connection direction between the two operators. For example, in the target matrix, the first row and the first column correspond to operator 1, the second row and the second column correspond to operator 2, the element corresponding to the second column of the first row is-1, that is, the operator 1 is represented to be connected with operator 2, and the connection direction is from operator 1 to operator 2, that is, the output of operator 1 is the input of operator 2; the element of the first column of the second row is 0, i.e. it indicates that there is no connection between operator 1 and operator 2, or that the connection direction of operator 1 and operator 2 is not from operator 2 to operator 1.

In some embodiments, when the recommendation model is regarded as a directed acyclic graph, a corresponding degree matrix and an adjacent matrix are obtained based on the recommendation model, the value of an element on a diagonal line in the degree matrix is equal to the number of operators connected with one operator corresponding to the element, the value of the element in the adjacent matrix indicates whether two operators corresponding to the element have a connection relationship, and the difference value between the degree matrix and the adjacent matrix is taken as a target matrix.

In the embodiment of the disclosure, the target matrix only represents the connection relationship between any two operators in the plurality of operators, that is, the target matrix represents the relative position relationship before each operator, and in order to facilitate subsequent processing, feature extraction is performed on the target matrix, so as to obtain structural features. The structural features differ from the target matrix in that the structural features are continuous coding features, whereas the target matrix is only discrete data. Because the target matrix can fully represent the connection relation between any two operators in the recommendation model, namely the topological structure information of the recommendation model is fully acquired, the structure characteristics obtained based on the target matrix can accurately represent the structure of the recommendation model, and the accuracy of the structure characteristics is improved.

It should be noted that, in the embodiment of the disclosure, the step 202 is performed first and then the step 203 is performed, and in another embodiment, the step 203 may be performed first and then the step 202 is performed, or the step 202 and the step 203 are performed simultaneously, and the execution sequence of the step 202 and the step 203 is not limited in the embodiment of the disclosure.

In step 204, the electronic device splices the operator feature and the structural feature corresponding to each operator, so as to obtain a spliced feature corresponding to the recommendation model.

In the embodiment of the present disclosure, in order to ensure that the operator features and the structural features can be spliced, in the step 203, when feature extraction is performed on the target matrix, the dimensions of the obtained structural features are the same as those of the operator features, so that the structural features and the operator features with the same dimensions can be spliced.

In some embodiments, the electronic device adds the structural feature directly after the plurality of operator features to obtain a splice feature, or adds the structural feature to the plurality of operator features to obtain a splice feature; or it is also possible to splice multiple operator features and structural features in other ways.

In step 205, the electronic device encodes the spliced feature to obtain a model feature corresponding to the recommended model.

In the embodiment of the disclosure, the stitching features, although including the features and structural features of each operator of the recommendation model, cannot directly represent the effect generated by connecting or disconnecting each operator in the recommendation model. Therefore, in order to obtain accurate features representing the recommended model, the spliced features are encoded, and operator features corresponding to each operator and structural features of the recommended model are fused in the encoding process to obtain model features. The model features obtained by the method can accurately represent the recommended model, and the accuracy of the model features is improved.

In some embodiments, the stitching features are processed based on an attention mechanism, the weight of each operator is determined, and operator features corresponding to the plurality of operators are adjusted based on the weights of the plurality of operators. For example, based on the weights of the operators, the characteristics corresponding to the operators are weighted, and the weighted operator characteristics are obtained. Alternatively, the attention mechanism is a multi-headed attention mechanism or other type of attention mechanism.

In step 206, the electronic device performs accuracy prediction based on the model features, to obtain a recommendation accuracy corresponding to the recommendation model.

Based on model characteristics, the electronic equipment carries out precision prediction on the recommended model to obtain the recommended precision of the recommended model, wherein the recommended precision represents the accuracy of the recommended model. The higher the recommendation precision is, the more accurate the recommendation model is, and the recommendation model can be practically applied; and the smaller the recommendation precision is, the more inaccurate the recommendation model is, and the recommendation model cannot be practically applied.

In some embodiments, in the context of neural architecture search, after the electronic device predicts the recommendation precision corresponding to the recommendation model, whether the recommendation model can be actually applied or not may be determined according to the recommendation precision, and if the recommendation model cannot be applied, the electronic device needs to continue searching for other recommendation models.

It should be noted that the recommendation model in the embodiment of the present disclosure may be a trained recommendation model or an untrained recommendation model, which is not limited in this embodiment of the present disclosure.

Another point to be described is that the above embodiment only uses processing of one recommendation model as an example, and in another embodiment, the electronic device obtains a plurality of recommendation models, where any two recommendation models in the plurality of recommendation models have different operators, or the connection relationships between the operators in any two recommendation models are different, where the difference in the connection relationship between the operators refers to the difference in the connection relationship between any two operators in the plurality of operators. For example, there are five recommendation models, where different operators exist in the first and second recommendation models, the same operator exists in the third and fourth models, but the connection relationship between operators is different, and different operators exist in the fourth and fifth models.

And then, respectively adopting the recommendation model processing modes shown in the steps 202-206 for each recommendation model to acquire the recommendation precision corresponding to each recommendation model, and then selecting a target recommendation model corresponding to the maximum recommendation precision from a plurality of recommendation models based on the recommendation precision corresponding to the plurality of recommendation models, and calling the target recommendation model to conduct recommendation, namely, applying the target recommendation model to an actual article recommendation scene. In the embodiment of the disclosure, based on the recommendation precision corresponding to the plurality of recommendation models, the most accurate recommendation model can be selected from the plurality of recommendation models, so that when the selected recommendation model is applied to an actual recommendation scene, accurate recommendation can be performed based on the recommendation model, and the recommendation accuracy is improved.

In addition, as the operator features in the operator feature table are extracted in advance, the operator features can be quickly acquired by inquiring the operator feature table, so that the efficiency of acquiring recommendation accuracy is improved. The target matrix can fully represent the connection relation between any two operators in the recommendation model, namely the topological structure information of the recommendation model is fully acquired, so that the structural features obtained based on the target matrix can accurately represent the structure of the recommendation model, and the accuracy of the structural features is improved.

In addition, in the related art, conventional NAS methods such as reinforcement learning and evolutionary learning require a large amount of computing resources to train each recommendation model to determine the recommendation accuracy corresponding to each recommendation model. Recently, the search time based on the NAS approach that can be micro-optimized is greatly reduced compared with the traditional search approach, but the optimization space discontinuity is generally suffered from the problem of optimization performance collapse, so that the optimal model meeting the conditions cannot be searched. Compared with the method that the recommendation model obtained by searching is trained by the related technology and then the recommendation accuracy is tested by adopting test data, the method provided by the embodiment of the disclosure can be used for directly predicting the accuracy of the recommendation model obtained by searching without training the recommendation model, so that the efficiency of obtaining the recommendation accuracy is improved, the time for training the model is saved, and the time waste is avoided for training the model which can not be practically applied.

The embodiment shown in fig. 2 describes the model processing procedure, and in another embodiment, the electronic device can call the recommendation precision prediction model to process the recommendation model, so as to obtain the recommendation precision corresponding to the recommendation model.

Referring to fig. 6, the recommendation accuracy prediction model includes a feature extraction network 601 and a prediction network 602. The feature extraction network 601 is configured to extract model features corresponding to the recommendation model, and the prediction network 602 is configured to predict recommendation accuracy corresponding to the recommendation model.

In some embodiments, referring to fig. 7, the recommendation accuracy prediction model further includes a feature acquisition network 603, where the feature acquisition network 603 is configured to acquire operator features corresponding to each operator in the recommendation model and structural features of the recommendation model.

FIG. 8 is a flowchart illustrating another recommendation model processing method, see FIG. 8, the method being performed by an electronic device, according to an exemplary embodiment, comprising the steps of:

in step 801, an electronic device obtains a recommendation model.

The embodiment of step 801 is the same as the embodiment of step 201 described above, and will not be described again here.

In step 802, the electronic device invokes a feature acquisition network to acquire operator features corresponding to each operator in the recommendation model.

In some embodiments, the index identifier corresponding to each operator in the recommendation model is used as an input of the recommendation precision prediction model, the electronic device invokes the feature acquisition network, and the operator feature corresponding to each operator is queried from the operator feature table based on the index identifier corresponding to each operator.

In step 803, the electronic device invokes the feature retrieval network to retrieve structural features of the recommendation model.

In some embodiments, a target matrix corresponding to the recommendation model is used as an input of a recommendation precision prediction model, and the electronic device calls a feature acquisition network to extract features of the target matrix to obtain structural features. The feature acquisition network may be capable of converting a discrete target matrix into a continuous structural feature, alternatively, the feature acquisition network may be an MLP (Multi-Layer Perceptron), convolutional neural network, or other type of neural network, as the embodiments of the present disclosure are not limited in this respect.

In step 804, the electronic device invokes the feature extraction network, and performs feature extraction based on the operator features and the structural features corresponding to each operator, so as to obtain model features corresponding to the recommendation model.

In some embodiments, the feature extraction network includes a stitching layer and a feature extraction layer, the electronic device calls the stitching layer to stitch operator features and structure features corresponding to each operator to obtain stitching features corresponding to the recommended model, calls the feature extraction layer to encode the stitching features to obtain model features.

Optionally, the feature extraction layer is transformer encoder (transformer encoder), and the feature extraction layer includes an attention unit and an encoding unit, calls the attention unit to process the plurality of operator features to obtain weights of the operators, processes the operator features corresponding to the plurality of operators based on the weights of the plurality of operators to obtain weighted operator features corresponding to each operator, and calls the encoding unit to encode the plurality of weighted operator features and the structural features to obtain model features. Wherein the attention unit is a multi-head attention unit or other types of attention units.

In another embodiment, the feature extraction network may be other types of encoders.

In some embodiments, the recommendation accuracy prediction model includes a plurality of feature extraction networks, for example, including 4 feature extraction networks. The electronic device takes the output of the feature acquisition network as the input of a first feature extraction network, takes the output of the first feature extraction network as the input of a next feature extraction network until the output of a last feature extraction network is obtained, and takes the output as a model feature.

In step 805, the electronic device invokes the prediction network to perform accuracy prediction on the model feature, so as to obtain a recommendation accuracy corresponding to the recommendation model.

Wherein the predictive network is an MLP or other neural network.

In some embodiments, the recommended accuracy prediction model includes a plurality of prediction networks, e.g., including 2 prediction networks. The electronic device takes the output of the feature acquisition network as the input of a first prediction network, takes the output of the first prediction network as the input of a next prediction network until the output of a last prediction network is obtained, and takes the output as the prediction precision.

Optionally, the output of the recommendation accuracy prediction model is a score, and the recommendation accuracy is represented by the score.

See, for example, the schematic diagram of the recommended model process shown in fig. 9. Taking an index identifier corresponding to each operator in the recommendation model and a target matrix corresponding to the recommendation model as input of a recommendation precision prediction model, obtaining operator characteristics corresponding to each operator by inquiring an operator characteristic table based on the index identifier corresponding to each operator, extracting characteristics of the target matrix by MLP to obtain structural characteristics, splicing the operator characteristics corresponding to each operator and the structural characteristics, inputting the spliced characteristics to transformer encoder to obtain model characteristics, and inputting the model characteristics to MLP to obtain final recommendation precision.

In the embodiment of the disclosure, the recommendation precision is predicted by using the recommendation precision prediction model, each operator characteristic and structure characteristic corresponding to the recommendation model are firstly obtained, then the operator characteristic and the structure characteristic are used for extracting effective model characteristics, and the mode of obtaining the model characteristics can fully obtain the characteristics of the recommendation model, so that the obtained model characteristics can more accurately represent the recommendation model, the recommendation precision predicted based on the model characteristics is more accurate, and the accuracy of the recommendation precision is improved.

Compared with the mode of training the recommendation model in the related art and then testing the recommendation model by adopting test data and determining the recommendation accuracy according to the test result, the recommendation accuracy is predicted by utilizing the recommendation accuracy prediction model, so that the recommendation accuracy can be quickly obtained, and the efficiency of obtaining the recommendation accuracy is improved.

Before the recommendation accuracy prediction model is used for predicting the recommendation accuracy, the recommendation accuracy prediction model needs to be trained, and a training process of the recommendation accuracy prediction model is described below by taking a training process as an example.

FIG. 10 is a flowchart of another model training method, see FIG. 10, according to an exemplary embodiment, the method being performed by an electronic device, comprising the steps of:

In step 1001, the electronic device obtains a sample operator feature, a first sample structure feature, and a first sample recommendation accuracy corresponding to each sample operator in the first sample recommendation model.

The first sample structural feature represents a connection relationship between any two sample operators in the first sample recommendation model.

In step 1002, the electronic device invokes a recommendation precision prediction model, and processes a sample operator feature and a first sample structure feature corresponding to each sample operator in the first sample recommendation model to obtain a first prediction recommendation precision.

In some embodiments, the electronic device invokes a recommendation precision prediction model, performs feature extraction on a sample operator feature and a first sample structure feature corresponding to each sample operator in the first sample recommendation model to obtain model features corresponding to the first sample recommendation model, and performs precision prediction based on the model features corresponding to the second sample recommendation model to obtain a first prediction recommendation precision.

In step 1003, the electronic device obtains a sample operator feature, a second sample structure feature, and a proxy recommendation precision corresponding to each sample operator in the second sample recommendation model.

The second sample structure feature represents a connection relationship between any two sample operators in the second sample recommendation model, and the agent recommendation precision is a variable for indicating recommendation precision corresponding to the second sample recommendation model, namely the agent recommendation precision is changed in the training process.

In step 1004, the electronic device invokes the recommendation precision prediction model, and processes the sample operator feature and the second sample structure feature corresponding to each sample operator in the second sample recommendation model to obtain a second prediction recommendation precision.

In some embodiments, the electronic device invokes a recommendation precision prediction model, performs feature extraction on a sample operator feature and a second sample structure feature corresponding to each sample operator in the second sample recommendation model to obtain model features corresponding to the second sample recommendation model, and performs precision prediction based on the model features corresponding to the second sample recommendation model to obtain a second prediction recommendation precision.

In step 1005, the electronic device obtains a first error between the first predicted recommended precision and the first sample recommended precision, and a second error between the second predicted recommended precision and the second sample recommended precision.

In step 1006, the electronic device adjusts the model parameters and the agent recommendation precision in the recommendation precision prediction model based on the first error and the second error, so that an error between the prediction recommendation precision corresponding to the history time and the adjusted agent recommendation precision of the second sample recommendation model approaches to a target error corresponding to the history time.

In an embodiment of the disclosure, the electronic device trains a recommendation precision prediction model based on a first prediction recommendation precision, a first sample recommendation precision, a second prediction recommendation precision, and a proxy recommendation precision. In the training process, the first error is required to be close to 0, the second error is required to be close to 0, and the agent recommendation precision meets the condition that the error between the prediction recommendation precision corresponding to the historical moment and the adjusted agent recommendation precision of the second sample recommendation model is close to the target error corresponding to the historical moment. The target error is an error between the prediction recommendation precision corresponding to the second sample recommendation model at the historical moment and the recommendation precision corresponding to the second sample recommendation model at the historical moment. The historical time is the time when the recommendation accuracy prediction model was previously trained by the second sample recommendation model.

In the training process, taking the operator characteristics and the structure characteristics as the input of the recommendation precision prediction model as an example, in another embodiment, when the index identifiers corresponding to the operators and the target matrixes corresponding to the recommendation models are used as the input of the recommendation precision prediction model, the electronic equipment acquires the sample index identifiers corresponding to each sample operator in the first sample recommendation model, the first sample matrixes and the first sample recommendation precision, calls the recommendation precision prediction model, and processes the sample index identifiers corresponding to each sample operator in the first sample recommendation model and the first sample matrixes to obtain the first prediction recommendation precision; the electronic equipment acquires a sample index identifier corresponding to each sample operator in the second sample recommendation model, a second sample matrix and proxy recommendation precision, calls a recommendation precision prediction model, and processes the sample index identifier corresponding to each sample operator in the second sample recommendation model and the second sample matrix to obtain second prediction recommendation precision. The subsequent processing is the same as in

steps

1005 and 1006 described above.

Another point to be noted is that the above embodiment is described only by taking the example of training the recommendation precision prediction model using the first sample recommendation model and the second sample recommendation model as an example, and in another embodiment, the electronic device can train the recommendation precision prediction model using the first sample recommendation model.

In addition, in some embodiments, the first sample recommendation model is a plurality of the second sample recommendation models. The electronic device trains a recommendation precision prediction model based on the first prediction recommendation precision, the first sample recommendation precision, the second prediction recommendation precision, and the proxy recommendation precision by adopting the following loss function:

wherein L represents a loss function, f _θ (. Cndot.) represents the recommendation accuracy prediction model, θ represents the model parameters in the recommendation accuracy prediction model, f _θ (x _i ) Representing a first predicted recommendation accuracy, x, corresponding to an ith first sample recommendation model _i Representing sample operator features and first sample structure features corresponding to each sample operator in the ith first sample recommendation model, n represents the number of first sample recommendation models used for training the recommendation accuracy prediction model, y _i Representing the first sample recommendation precision corresponding to the ith first sample recommendation model, f _θ (v _j ) Representing a second predicted recommendation accuracy corresponding to a jth second sample recommendation model,v _j representing sample operator features and second sample structure features corresponding to each sample operator in the j-th second sample recommendation model, V representing the number of second sample recommendation models used to train the recommendation accuracy prediction model,

representing the agent recommendation accuracy corresponding to the jth second sample recommendation model, +.>

Representing agent recommendation accuracy, α is a reference parameter, e.g., α is 0.1, 0.2, 0.3, or other value;

the loss function satisfies the following condition:

wherein V represents the number of second sample recommendation models used to train the recommendation accuracy prediction model,

Optionally, the lagrangian multiplier method is used to transform the two formulas into the following formulas, so as to convert the formulas into the minmax optimization problem:

where λ is the lagrange multiplier, and the other definitions are the same as those described above.

Then a gradient is adoptedThe falling and gradient rising modes are used for solving the above formula

Deriving the formula to obtain:

wherein k represents k-order partial derivative, eta _θ 、

and η_λ For reference parameters->

Representation is directed at->

Solving k-order bias of θ, +.>

Representation is directed at->

Ask for->

K-order bias of->

Representation is directed at->

And solving k-order partial derivatives of lambda.

See, for example, the schematic diagram of the training process shown in fig. 11. In the training process adopting the mode, the recommendation precision corresponding to the second sample recommendation model is unchanged, the prediction recommendation precision of a plurality of historical moments corresponding to the second sample recommendation model is continuously changed and gradually approaches to the recommendation precision, and meanwhile, the agent recommendation precision is also continuously changed and gradually approaches to the recommendation precision.

When the recommendation precision prediction model is trained, the recommendation precision prediction model is trained by adopting the first sample recommendation model, the recommendation precision prediction model is trained by adopting the second sample recommendation model, and in the training process, the agent recommendation precision corresponding to the second sample recommendation model also needs to meet the condition corresponding to the historical moment, namely, not only the training data in the current training process but also the training data at the historical moment are considered, and the self-evolution optimization framework aiming at the second sample recommendation model is added, so that the training efficiency is improved, and the accuracy of the recommendation precision prediction model obtained by training is also improved. In addition, the self-evolution optimization framework is adopted, and the recommendation precision prediction model is trained by adopting more sample recommendation models, so that the generalization capability of the recommendation precision prediction model is improved.

Fig. 12 is a flowchart illustrating a classification model processing method according to an exemplary embodiment, referring to fig. 12, the method is performed by an electronic device, and includes the following steps:

in step 1201, the electronic device obtains a classification model.

The classification model comprises a plurality of operators which are arranged in sequence, and the operators process data to be classified, which are input into the classification model, in sequence to obtain data types corresponding to the data. For example, the classification model is used for classifying the image to determine whether the image belongs to a face image, and the type of data output by the classification model is the face image or a non-face image. Or the classification model is used for classifying the articles so as to determine the article category to which the articles belong; or the classification model is used to classify the audio to determine the audio type to which the audio belongs. In the embodiment of the present disclosure, the classification model is not limited in its function and specific structure.

In some embodiments, the electronic device inputs data to be categorized into a recommendation model; invoking a first operator in the classification model, and processing data to be classified to obtain output data of the first operator; and taking the output data of the first operator as the input data of the next operator connected with the first operator, calling the next operator to process the output data until the output data of the last operator in the classification model is obtained, and determining the output data of the last operator as the recommended parameter. The input data of the first operator in the classification model is data to be classified, and the input data of other operators except the first operator are output data of the last operator connected with the other operators.

In addition, the classification model is obtained in the same manner as the recommended model in the above embodiment, for example, the classification model is obtained by using a neural architecture search method, or the classification model is a model designed manually.

In step 1202, the electronic device obtains operator features corresponding to each operator in the classification model and structural features of the classification model based on the classification model.

Wherein the structural feature represents a connection relationship between any two operators of the plurality of operators.

In step 1203, the electronic device performs feature extraction based on the operator features and the structural features corresponding to each operator, to obtain model features corresponding to the classification model.

The implementation of obtaining model features corresponding to the classification model in steps 1202-1203 is the same as the implementation of obtaining model features corresponding to the recommendation model in steps 202-205 of the above embodiment. The difference is that the classification model differs from operators or connections in the recommendation model.

In step 1204, the electronic device performs accuracy prediction based on the model features, to obtain classification accuracy corresponding to the classification model.

The classification accuracy is used for representing the classification accuracy of the classification model. The higher the classification precision is, the more accurate the classification model is, and the classification model can be practically applied; and the smaller the classification accuracy is, the less accurate the classification model is, and the classification model cannot be practically applied.

In the above embodiment, only one classification model is processed as an example, and in another embodiment, the electronic device acquires a plurality of classification models, and a plurality of operators in any two classification models of the plurality of classification models and a connection relationship between any two operators of the plurality of operators are different. And then, aiming at each classification model, respectively adopting the classification model processing modes shown in the steps 1202-1204 to acquire the classification precision corresponding to each classification model, and then, based on the classification precision corresponding to the plurality of classification models, selecting a target classification model corresponding to the maximum classification precision from the plurality of classification models, and calling the target classification model to classify.

In addition, in another embodiment, the electronic device can call the classification precision prediction model to process a plurality of operator features and structural features corresponding to the classification model, so as to obtain classification precision corresponding to the classification model. The structure of the classification accuracy prediction model is the same as that of the recommended accuracy prediction model in the above embodiment, except that the classification accuracy prediction model is used for processing model features corresponding to the classification model to obtain classification accuracy. The training process of the classification accuracy prediction model is the same as that of the recommended accuracy prediction model in the above embodiment, except that the classification accuracy prediction model is trained by using a sample classification model, and the capability of the classification accuracy prediction model to predict the classification accuracy corresponding to the classification model is trained.

In the above embodiment, the recommended model processing procedure and the classifying model processing procedure are described, and in another embodiment, a processing procedure similar to the recommended model processing procedure or the classifying model processing procedure can be used to process a network model such as an image segmentation model, a feature extraction model, an image recognition model, a text processing model, an audio processing model, a video processing model, and the like, so as to obtain model accuracy corresponding to the model.

In some embodiments, taking any recommended model as an example, comparing the recommended model processing method provided by the embodiment of the disclosure with the method for obtaining the recommended precision in the related art, it can be seen that the recommended precision obtained by adopting the method of the embodiment of the disclosure is more accurate.

(1) Reproducible neural network search 101 (NAS-Bench-101)

As can be seen from table 1, the training samples are different, and the effect of the finally obtained precision prediction model is also different, but under the same number of training samples, the effect of the method (TNASP) provided by the embodiment of the present disclosure is significantly higher than that of other prior art.

TABLE 1

(2) Reproducible neural network search 201 (NAS-Bench-201)

As can be seen from table 2, the training samples are different, and the effect of the finally obtained precision prediction model is also different, but under the same number of training samples, the effect of the method (TNASP) provided by the embodiment of the present disclosure is significantly higher than that of other prior art.

TABLE 2

Training sample	780	156	469	781	1563
						Nerve predictor	0.343	0.413	0.584	0.634	0.646
NAO	0.467	0.493	0.470	0.522	0.526
						TNASP	0.539	0.589	0.640	0.689	0.724
Nerve predictor+SE	0.377	0.433	0.620	0.652	0.649
						NAO+SE	0.511	0.511	0.514	0.529	0.528
TNASP+SE	0.565	0.594	0.642	0.690	0.726

(3) DARTS (Differentiable Architecture Search, searching micro-neural network)

As can be seen from table 3, the method (TNASP) provided by the embodiments of the present disclosure is superior to other prior art in terms of accuracy, parameters and cost.

TABLE 3 Table 3

Architecture of architecture	Precision of	Parameters (parameters)	Search cost
				DenseNet-BC	96.54	25.6	-
PyramidNet-BC	96.69	26.0	-
				NASNet-A+cutout	0.539	0.589	0.640
NASNet-B	0.377	0.433	0.620
				NASNet-C	0.511	0.511	0.514
AmoebaNet-A+cutout	96.66	3.2	3150
				AmoebaNet-B+cutout	96.63	2.8	3150
SNAS	97.02	28.6	200
				GHN	97.16	5.7	0.8
PNAS+cutout	97.17	3.2	-
				DARTA+cutout	97.24	3.6	4
CTNAS+cutout	97.41	3.6	0.3
				TNASP+cutout	97.48	3.7	0.3

(4)ProxylessNAS

As can be seen from table 3, the accuracy of the method (TNASP) provided by the embodiments of the present disclosure is superior to other prior art.

TABLE 4 Table 4

Method	Parameters (parameters)	Trigger device	First	1%	First	5%
							FBNet-C	4.4	375	74.9	92.1
Proxyless(GPU)	7.0	457	75.1	92.5
					SPOS	5.4	472	74.8	-
RLNAS	5.3	473	75.6	92.6
					Nerve predictor	6.4	536	74.75	-
NAO	6.5	590	75.5	92.5
					TNASP-A	5.0	433	75.1	92.3
TNASP-B	5.1	478	75.5	92.5
					TNASP-C	5.3	479	75.8	92.7

FIG. 13 is a block diagram illustrating a recommendation model processing device, according to an example embodiment. Referring to fig. 13, the apparatus includes:

a recommendation model obtaining unit 1301 configured to perform obtaining a recommendation model, where the recommendation model includes a plurality of operators arranged in order, and the plurality of operators sequentially process account data and item data input to the recommendation model in order to obtain recommendation parameters, where the recommendation parameters indicate whether to recommend an item to an account;

a first feature obtaining unit 1302 configured to obtain operator features corresponding to each operator in the recommendation model and structural features of the recommendation model based on the recommendation model, the structural features representing a connection relationship between any two operators in the plurality of operators;

a second feature obtaining unit 1303 configured to perform feature extraction based on the operator feature and the structural feature corresponding to each operator, to obtain a model feature corresponding to the recommendation model;

and a recommendation precision prediction unit 1304 configured to perform precision prediction based on the model features, so as to obtain a recommendation precision corresponding to the recommendation model.

According to the device provided by the embodiment of the disclosure, each operator characteristic and each structure characteristic corresponding to the recommendation model are firstly obtained, then the operator characteristic and the structure characteristic are utilized to extract the effective model characteristic, the mode of obtaining the model characteristic can fully obtain the characteristic of the recommendation model, the obtained model characteristic can more accurately represent the recommendation model, and therefore recommendation accuracy obtained based on prediction of the model characteristic is more accurate, and accuracy of the recommendation accuracy is improved.

In some embodiments, the recommendation model further includes an index identifier corresponding to each operator, and the first feature obtaining unit 1302 includes:

the operator characteristic table comprises index identifiers corresponding to at least one operator and operator characteristics, wherein the operator characteristics are obtained by extracting the characteristics of the operators.

In some embodiments, the recommendation model includes N operators, N being an integer greater than 1, the first feature acquisition unit 1302 includes:

the structure characteristic acquisition subunit is configured to execute N operators in the recommendation model, and create N rows and N columns of a target matrix, wherein the N operators are respectively used as N rows and N columns of the target matrix, the rows and the columns corresponding to the elements on the diagonal line in the target matrix are the same operators, the values of the elements on the diagonal line in the target matrix are equal to the number of other operators connected with the operators corresponding to the elements, the rows and the columns corresponding to the elements on other positions in the target matrix are two operators with different values, and the values of the elements on other positions represent whether the two operators corresponding to the elements have a connection relation;

And the structural feature acquisition subunit is further configured to perform feature extraction on the target matrix to obtain structural features.

In some embodiments, the second feature acquisition unit 1303 includes:

the characteristic splicing subunit is configured to splice the operator characteristics and the structure characteristics corresponding to each operator to obtain splicing characteristics corresponding to the recommended model;

and the coding subunit is configured to perform coding on the spliced characteristic to obtain a model characteristic.

the feature extraction network is used for extracting features based on operator features and structural features corresponding to each operator to obtain model features;

and the prediction network is used for predicting the precision of the model characteristics to obtain recommended precision.

the feature acquisition network is also used for acquiring the structural features of the recommendation model.

In some embodiments, referring to fig. 14, the recommendation model processing apparatus further includes:

a model training unit 1305 configured to perform obtaining a sample operator feature, a first sample structural feature, and a first sample recommendation precision corresponding to each sample operator in the first sample recommendation model, the first sample structural feature representing a connection relationship between any two sample operators in the first sample recommendation model;

The model training unit 1305 is further configured to execute a recommendation precision prediction model, and process a sample operator feature corresponding to each sample operator in the first sample recommendation model and a first sample structure feature to obtain a first prediction recommendation precision;

the model training unit 1305 is further configured to perform training of a recommendation accuracy prediction model based on the first prediction recommendation accuracy and the first sample recommendation accuracy.

a recommendation unit 1306 configured to perform inputting of account data and item data to a recommendation model;

the recommending unit 1306 is further configured to execute the first operator in the calling recommending model, and process the account data and the article data to obtain output data of the first operator;

the recommending unit 1306 is further configured to execute the output data of the first operator as the input data of the next operator connected to the first operator, call the next operator to process the output data until the output data of the last operator in the recommendation model is obtained, and determine the output data of the last operator as the recommendation parameter.

In some embodiments, the obtaining model unit 1301 is configured to perform obtaining a plurality of recommendation models, where operators in any two recommendation models exist differently, or a connection relationship between operators in any two recommendation models is different;

Referring to fig. 14, the recommended model processing apparatus further includes:

a model selection unit 1307 configured to perform selection of a target recommendation model corresponding to the maximum recommendation precision from among the plurality of recommendation models based on the recommendation precision corresponding to the plurality of recommendation models;

and a recommendation unit 1306 configured to execute the call target recommendation model to make a recommendation.

In some embodiments, the recommendation model further includes a plurality of edges, each edge connecting two operators, the operators being configured to process input data input to the operators to obtain output data; and, the output data of one operator connected by each edge is used as the input data of the other operator connected.

The specific manner in which the individual units perform the operations in relation to the apparatus of the above embodiments has been described in detail in relation to the embodiments of the method and will not be described in detail here.

FIG. 15 is a block diagram illustrating an accuracy prediction model training apparatus, according to an example embodiment. Referring to fig. 15, the apparatus includes:

a sample acquiring unit 1501 configured to perform acquisition of a sample operator feature, a first sample structural feature, and a first sample recommendation precision corresponding to each sample operator in a first sample recommendation model, the first sample structural feature representing a connection relationship between any two sample operators in the first sample recommendation model;

The precision prediction unit 1502 is configured to execute a recommendation precision prediction model, perform feature extraction on sample operator features and first sample structural features corresponding to each sample operator in the first sample recommendation model to obtain model features corresponding to the first sample recommendation model, and perform precision prediction based on the model features corresponding to the first sample recommendation model to obtain first prediction recommendation precision;

the model training unit 1503 is configured to perform training of a recommendation accuracy prediction model based on the first prediction recommendation accuracy and the first sample recommendation accuracy.

According to the device provided by the embodiment of the disclosure, the recommendation precision prediction model for predicting the recommendation precision is trained, then each operator characteristic and each structure characteristic corresponding to the recommendation model can be processed by using the recommendation precision prediction model, and the operator characteristic and the structure characteristic are used for extracting the effective model characteristic so as to fully acquire the characteristic of the recommendation model, so that the obtained model characteristic can more accurately represent the recommendation model, the recommendation precision predicted by the recommendation precision prediction model based on the model characteristic is more accurate, and the accuracy of the recommendation precision is improved.

In some embodiments, the sample acquisition unit 1501 is further configured to perform: acquiring sample operator characteristics, second sample structure characteristics and proxy recommendation precision corresponding to each sample operator in a second sample recommendation model, wherein the second sample structure characteristics represent the connection relationship between any two sample operators in the second sample recommendation model, and the proxy recommendation precision is a variable for indicating the recommendation precision of the second sample recommendation model;

the precision prediction unit 1502 is configured to execute a recommendation precision prediction model, perform feature extraction on sample operator features and second sample structure features corresponding to each sample operator in the second sample recommendation model, obtain model features corresponding to the second sample recommendation model, and perform precision prediction based on the model features corresponding to the second sample recommendation model, so as to obtain second prediction recommendation precision;

the model training unit 1503 is configured to perform training of a recommendation accuracy prediction model based on the first prediction recommendation accuracy, the first sample recommendation accuracy, the second prediction recommendation accuracy, and the proxy recommendation accuracy.

In some embodiments, model training unit 1503 is configured to perform:

acquiring a first error between a first prediction recommendation precision and a first sample recommendation precision, and acquiring a second error between a second prediction recommendation precision and a second sample recommendation precision;

Based on the first error and the second error, model parameters and agent recommendation precision in the recommendation precision prediction model are adjusted so that errors between the prediction recommendation precision corresponding to the historical moment and the adjusted agent recommendation precision of the second sample recommendation model are close to target errors corresponding to the historical moment.

In some embodiments, model training unit 1503 is configured to perform:

based on the first prediction recommendation accuracy, the first sample recommendation accuracy, the second prediction recommendation accuracy, and the proxy recommendation accuracy, training a recommendation accuracy prediction model using the following loss function:

wherein L represents a loss function, f _θ (. Cndot.) represents the recommendation accuracy prediction model, θ represents the model parameters in the recommendation accuracy prediction model, f _θ (x _i ) Representing a first predicted recommendation accuracy, x, corresponding to an ith first sample recommendation model _i Representing sample operator features and first sample structure features corresponding to each sample operator in the ith first sample recommendation model, n represents the number of first sample recommendation models used for training the recommendation accuracy prediction model, y _i Representing the first sample recommendation precision corresponding to the ith first sample recommendation model, f _θ (v _j ) Representing a second predicted recommendation accuracy, v, corresponding to a jth second sample recommendation model _j Representing sample operator features and second sample structure features corresponding to each sample operator in the j-th second sample recommendation model, V representing the number of second sample recommendation models used to train the recommendation accuracy prediction model,

atmosphere indicating the corresponding second sample recommendation model of jAgent recommendation accuracy->

the loss function satisfies the following condition:

Fig. 16 is a block diagram illustrating a classification model processing apparatus according to an exemplary embodiment. Referring to fig. 16, the apparatus includes:

a classification model obtaining unit 1601 configured to perform obtaining a classification model, where the classification model includes a plurality of operators arranged in order, and the plurality of operators sequentially process data to be classified input to the classification model in order to obtain a data type corresponding to the data;

A first feature obtaining unit 1602 configured to obtain operator features corresponding to each operator in the classification model and structural features of the classification model based on the classification model, the structural features representing a connection relationship between any two operators in the plurality of operators;

a second feature obtaining unit 1603 configured to perform feature extraction based on the operator feature corresponding to each operator and the structural feature, and obtain model features corresponding to the classification model;

and a classification accuracy prediction unit 1604 configured to perform accuracy prediction based on the model features, so as to obtain classification accuracy corresponding to the classification model.

According to the device provided by the embodiment of the disclosure, each operator characteristic and each structure characteristic corresponding to the classification model are firstly obtained, then the operator characteristic and the structure characteristic are utilized to extract the effective model characteristic, the mode of obtaining the model characteristic can fully obtain the characteristic of the classification model, so that the obtained model characteristic can more accurately represent the classification model, the classification precision obtained based on the model characteristic prediction is more accurate, and the accuracy of the classification precision is improved.

In an exemplary embodiment, an electronic device is provided that includes one or more processors and a memory for storing instructions executable by the one or more processors; wherein the one or more processors are configured to perform the recommendation model processing method, the recommendation accuracy prediction model training method, or the classification model processing method in the above embodiments.

In one possible implementation, the electronic device is provided as a terminal. Fig. 17 is a block diagram illustrating a structure of a terminal 1700 according to an exemplary embodiment. The terminal 1700 may be a portable mobile terminal such as: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III, motion picture expert compression standard audio plane 3), an MP4 (Moving Picture Experts Group Audio Layer IV, motion picture expert compression standard audio plane 4) player, a notebook computer, or a desktop computer. Terminal 1700 may also be referred to by other names of user devices, portable terminals, laptop terminals, desktop terminals, etc.

The terminal 1700 includes: a processor 1701 and a memory 1702.

The processor 1701 may include one or more processing cores, such as a 4-core processor, an 8-core processor, or the like. The processor 1701 may be implemented in at least one hardware form of a DSP (Digital Signal Processing ), FPGA (Field-Programmable Gate Array, field programmable gate array), PLA (Programmable Logic Array ). The processor 1701 may also include a main processor and a coprocessor, the main processor being a processor for processing data in an awake state, also referred to as a CPU (Central Processing Unit ); a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 1701 may be integrated with a GPU (Graphics Processing Unit, image processor) for taking care of rendering and drawing of content that the display screen is required to display. In some embodiments, the processor 1701 may also include an AI (Artificial Intelligence ) processor for processing computing operations related to machine learning.

Memory 1702 may include one or more computer-readable storage media, which may be non-transitory. Memory 1702 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 1702 is used to store at least one program code for execution by processor 1701 to implement the recommendation model processing method, recommendation accuracy prediction model training method, or classification model processing method provided by the method embodiments in the present disclosure.

In some embodiments, terminal 1700 may further optionally include: a peripheral interface 1703, and at least one peripheral. The processor 1701, memory 1702, and peripheral interface 1703 may be connected by a bus or signal line. The individual peripheral devices may be connected to the peripheral device interface 1703 by buses, signal lines or a circuit board. Specifically, the peripheral device includes: at least one of radio frequency circuitry 1704, a display screen 1705, a camera assembly 1706, an audio circuit 1707, a positioning assembly 1708, and a power source 1709.

The peripheral interface 1703 may be used to connect at least one Input/Output (I/O) related peripheral to the processor 1701 and the memory 1702. In some embodiments, the processor 1701, the memory 1702, and the peripheral interface 1703 are integrated on the same chip or circuit board; in some other embodiments, either or both of the processor 1701, the memory 1702, and the peripheral interface 1703 may be implemented on separate chips or circuit boards, which is not limited in this embodiment.

The Radio Frequency circuit 1704 is configured to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 1704 communicates with a communication network and other communication devices through electromagnetic signals. The radio frequency circuit 1704 converts an electrical signal into an electromagnetic signal for transmission, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 1704 includes: antenna systems, RF transceivers, one or more amplifiers, tuners, oscillators, digital signal processors, codec chipsets, subscriber identity module cards, and so forth. The radio frequency circuitry 1704 may communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: the world wide web, metropolitan area networks, intranets, generation mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and/or WiFi (Wireless Fidelity ) networks. In some embodiments, the radio frequency circuitry 1704 may also include NFC (Near Field Communication, short range wireless communication) related circuitry, which is not limited by this disclosure.

The display screen 1705 is used to display a UI (User Interface). The UI may include graphics, text, icons, video, and any combination thereof. When the display 1705 is a touch display, the display 1705 also has the ability to collect touch signals at or above the surface of the display 1705. The touch signal may be input as a control signal to the processor 1701 for processing. At this point, the display 1705 may also be used to provide virtual buttons and/or a virtual keyboard, also referred to as soft buttons and/or a soft keyboard. In some embodiments, the display 1705 may be one and disposed on the front panel of the terminal 1700; in other embodiments, the display 1705 may be at least two, respectively disposed on different surfaces of the terminal 1700 or in a folded design; in other embodiments, the display 1705 may be a flexible display disposed on a curved surface or a folded surface of the terminal 1700. Even more, the display 1705 may be arranged in a non-rectangular irregular pattern, i.e. a shaped screen. The display 1705 may be made of LCD (Liquid Crystal Display ), OLED (Organic Light-Emitting Diode) or other materials.

The camera assembly 1706 is used to capture images or video. Optionally, the camera assembly 1706 includes a front camera and a rear camera. The front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, the at least two rear cameras are any one of a main camera, a depth camera, a wide-angle camera and a tele camera, so as to realize that the main camera and the depth camera are fused to realize a background blurring function, and the main camera and the wide-angle camera are fused to realize a panoramic shooting and Virtual Reality (VR) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 1706 may also include a flash. The flash lamp can be a single-color temperature flash lamp or a double-color temperature flash lamp. The dual-color temperature flash lamp refers to a combination of a warm light flash lamp and a cold light flash lamp, and can be used for light compensation under different color temperatures.

The audio circuit 1707 may include a microphone and a speaker. The microphone is used for collecting sound waves of users and the environment, converting the sound waves into electric signals, and inputting the electric signals to the processor 1701 for processing, or inputting the electric signals to the radio frequency circuit 1704 for voice communication. For purposes of stereo acquisition or noise reduction, the microphone may be multiple and separately disposed at different locations of the terminal 1700. The microphone may also be an array microphone or an omni-directional pickup microphone. The speaker is used to convert electrical signals from the processor 1701 or the radio frequency circuit 1704 into sound waves. The speaker may be a conventional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, not only the electric signal can be converted into a sound wave audible to humans, but also the electric signal can be converted into a sound wave inaudible to humans for ranging and other purposes. In some embodiments, the audio circuit 1707 may also include a headphone jack.

The location component 1708 is used to locate the current geographic location of the terminal 1700 to enable navigation or LBS (Location Based Service, location based services). The positioning component 1708 may be a positioning component based on the united states GPS (Global Positioning System ), the beidou system of china, the russian graver positioning system, or the galileo positioning system of the european union.

A power supply 1709 is used to power the various components in the terminal 1700. The power source 1709 may be alternating current, direct current, disposable battery, or rechargeable battery. When the power source 1709 includes a rechargeable battery, the rechargeable battery may be a wired rechargeable battery or a wireless rechargeable battery. The wired rechargeable battery is a battery charged through a wired line, and the wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery may also be used to support fast charge technology.

In some embodiments, terminal 1700 also includes one or more sensors 1710. The one or more sensors 1710 include, but are not limited to: an acceleration sensor 1711, a gyro sensor 1712, a pressure sensor 1713, a fingerprint sensor 1714, an optical sensor 1715, and a proximity sensor 1716.

The acceleration sensor 1711 may detect the magnitudes of accelerations on three coordinate axes of the coordinate system established with the terminal 1700. For example, the acceleration sensor 1711 may be used to detect the components of gravitational acceleration in three coordinate axes. The processor 1701 may control the display 1705 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signal acquired by the acceleration sensor 1711. The acceleration sensor 1711 may also be used for the acquisition of motion data of a game or a user.

The gyro sensor 1712 may detect a body direction and a rotation angle of the terminal 1700, and the gyro sensor 1712 may collect 3D actions of the user on the terminal 1700 in cooperation with the acceleration sensor 1711. The processor 1701 may implement the following functions based on the data collected by the gyro sensor 1712: motion sensing (e.g., changing UI according to a tilting operation by a user), image stabilization at shooting, game control, and inertial navigation.

The pressure sensor 1713 may be disposed at a side frame of the terminal 1700 and/or at a lower layer of the display 1705. When the pressure sensor 1713 is disposed at a side frame of the terminal 1700, a grip signal of the terminal 1700 by a user may be detected, and the processor 1701 performs left-right hand recognition or quick operation according to the grip signal collected by the pressure sensor 1713. When the pressure sensor 1713 is disposed at the lower layer of the display screen 1705, the processor 1701 controls the operability control on the UI interface according to the pressure operation of the user on the display screen 1705. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

The fingerprint sensor 1714 is used to collect a fingerprint of a user, and the processor 1701 identifies the identity of the user based on the fingerprint collected by the fingerprint sensor 1714, or the fingerprint sensor 1714 identifies the identity of the user based on the collected fingerprint. Upon recognizing that the user's identity is a trusted identity, the processor 1701 authorizes the user to perform relevant sensitive operations including unlocking the screen, viewing encrypted information, downloading software, paying for and changing settings, etc. The fingerprint sensor 1714 may be disposed on the front, back, or side of the terminal 1700. When a physical key or vendor Logo is provided on the terminal 1700, the fingerprint sensor 1714 may be integrated with the physical key or vendor Logo.

The optical sensor 1715 is used to collect ambient light intensity. In one embodiment, the processor 1701 may control the display brightness of the display screen 1705 based on the ambient light intensity collected by the optical sensor 1715. Specifically, when the intensity of the ambient light is high, the display brightness of the display screen 1705 is turned up; when the ambient light intensity is low, the display brightness of the display screen 1705 is turned down. In another embodiment, the processor 1701 may also dynamically adjust the shooting parameters of the camera assembly 1706 based on the ambient light intensity collected by the optical sensor 1715.

A proximity sensor 1716, also referred to as a distance sensor, is provided on the front panel of the terminal 1700. The proximity sensor 1716 is used to collect the distance between the user and the front of the terminal 1700. In one embodiment, when the proximity sensor 1716 detects that the distance between the user and the front of the terminal 1700 gradually decreases, the processor 1701 controls the display 1705 to switch from the bright screen state to the off screen state; when the proximity sensor 1716 detects that the distance between the user and the front of the terminal 1700 gradually increases, the processor 1701 controls the display 1705 to switch from the off-screen state to the on-screen state.

Those skilled in the art will appreciate that the structure shown in fig. 17 is not limiting and that terminal 1700 may include more or less components than shown, or may combine certain components, or may employ a different arrangement of components.

In another possible implementation, the electronic device is provided as a server. Fig. 18 is a block diagram illustrating a structure of a server 1800, which may be relatively widely varied according to configuration or performance, may include one or more processors (Central Processing Units, CPU) 1801 and one or more memories 1802, wherein at least one program code is stored in the memories 1802, and the at least one program code is loaded and executed by the processors 1801 to implement the methods provided in the respective method embodiments described above. Of course, the server may also have a wired or wireless network interface, a keyboard, an input/output interface, and other components for implementing the functions of the device, which are not described herein.

In an exemplary embodiment, a computer readable storage medium is also provided, which when instructions in the storage medium are executed by a processor of an electronic device, enable the electronic device to perform the steps performed by the electronic device in the above-described recommendation model processing method, recommendation accuracy prediction model training method, or classification model processing method. Alternatively, the computer readable storage medium may be a ROM (Read Only Memory), a RAM (random access Memory ), a CD-ROM (compact disc Read Only Memory), a magnetic tape, a floppy disk, an optical data storage device, and the like.

In an exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program to be executed by a processor to implement the above-mentioned recommendation model processing method, recommendation accuracy prediction model training method or classification model processing method.

In some embodiments, the computer program related to the embodiments of the present application may be deployed to be executed on one computer device or on multiple computer devices located at one site, or on multiple computer devices distributed across multiple sites and interconnected by a communication network, where the multiple computer devices distributed across multiple sites and interconnected by a communication network may constitute a blockchain system.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure herein. This disclosure is intended to cover any adaptations, uses, or adaptations of the disclosure following the general principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A recommendation model processing method, the method comprising:

invoking a splicing layer in a feature extraction network of a recommendation precision prediction model, and splicing operator features corresponding to each operator and the structural features to obtain splicing features corresponding to the recommendation model; invoking an attention unit in a feature extraction layer of the feature extraction network, processing the spliced features based on an attention mechanism, determining the weight of each operator, and adjusting operator features corresponding to a plurality of operators in the spliced features based on the weights of the operators; invoking an encoding unit in a feature extraction layer of the feature extraction network to encode operator features corresponding to the adjusted operators and structural features of a recommendation model, which are included in the splicing features, so as to obtain model features corresponding to the recommendation model;

invoking a prediction network of the recommendation precision prediction model, and performing precision prediction based on the model characteristics to obtain recommendation precision corresponding to the recommendation model;

The training process of the recommendation precision prediction model comprises the following steps:

determining a first prediction recommendation precision of a first sample recommendation model and a second prediction recommendation precision of a second sample recommendation model based on a recommendation precision prediction model to be trained;

and adjusting parameters of a recommendation precision prediction model to be trained and agent recommendation precision of the second sample recommendation model based on the first prediction recommendation precision, the first sample recommendation precision, the second prediction recommendation precision and the second sample recommendation precision so that errors between the prediction recommendation precision corresponding to the second sample recommendation model at historical moments and the adjusted agent recommendation precision approach target errors corresponding to the historical moments, wherein the agent recommendation precision is a variable for indicating the recommendation precision of the second sample recommendation model.

2. The recommendation model processing method according to claim 1, wherein the recommendation model further includes an index identifier corresponding to each operator, and the step of obtaining operator features corresponding to each operator includes:

3. The recommendation model processing method according to claim 1, wherein the recommendation model includes N operators, N is an integer greater than 1, and the step of acquiring the structural feature includes:

4. The recommended model processing method of claim 1, wherein the recommended accuracy prediction model further comprises a feature acquisition network,

5. The recommended model processing method according to claim 1, wherein the training process of the recommended accuracy prediction model further includes:

acquiring sample operator characteristics, first sample structural characteristics and first sample recommendation precision corresponding to each sample operator in a first sample recommendation model, wherein the first sample structural characteristics represent the connection relationship between any two sample operators in the first sample recommendation model; acquiring sample operator characteristics, second sample structure characteristics and the agent recommendation precision corresponding to each sample operator in the second sample recommendation model, wherein the second sample structure characteristics represent the connection relationship between any two sample operators in the second sample recommendation model;

the determining, based on the recommendation precision prediction model to be trained, the first prediction recommendation precision of the first sample recommendation model and the second prediction recommendation precision of the second sample recommendation model includes: and calling the recommendation precision prediction model, processing sample operator characteristics and first sample structure characteristics corresponding to each sample operator in the first sample recommendation model to obtain first prediction recommendation precision, and processing sample operator characteristics and second sample structure characteristics corresponding to each sample operator in the second sample recommendation model to obtain second prediction recommendation precision.

6. The recommendation model processing method according to claim 1, wherein the adjusting the parameters of the recommendation-precision prediction model to be trained and the agent recommendation precision of the second sample recommendation model based on the first prediction recommendation precision, the first sample recommendation precision, the second prediction recommendation precision, and the second sample recommendation precision so that the error between the prediction recommendation precision of the second sample recommendation model corresponding to the historical time and the adjusted agent recommendation precision approaches the target error corresponding to the historical time includes:

acquiring a first error between the first sample recommendation precision and the first prediction recommendation precision, and acquiring a second error between the second sample recommendation precision and the second prediction recommendation precision;

and adjusting parameters of the recommendation precision prediction model to be trained and proxy recommendation precision of the second sample recommendation model based on the first error and the second error, so that the error between the prediction recommendation precision corresponding to the historical moment and the adjusted proxy recommendation precision of the second sample recommendation model approaches to a target error corresponding to the historical moment.

7. The recommendation model processing method according to claim 1, further comprising:

inputting the account data and the item data to the recommendation model;

8. The recommendation model processing method according to claim 1, wherein the acquiring a recommendation model comprises:

and calling the target recommendation model to recommend.

9. The recommendation model processing method according to any one of claims 1 to 8, wherein the recommendation model further comprises a plurality of edges, each edge being connected to two operators, the operators being used for processing input data input to the operators to obtain output data; and, the output data of one operator connected by each edge is used as the input data of the other operator connected.

10. A recommendation accuracy prediction model training method, the method comprising:

acquiring sample operator characteristics, first sample structure characteristics and first sample recommendation precision corresponding to each sample operator in a first sample recommendation model, wherein the first sample structure characteristics represent a connection relationship between any two sample operators in the first sample recommendation model, each sample operator in the first sample recommendation model is used for sequentially processing account data and article data input into the first sample recommendation model according to a sequence to obtain recommendation parameters, and the recommendation parameters indicate whether to recommend the article to the account;

Invoking a splicing layer in a feature extraction network of a recommendation precision prediction model, and splicing sample operator features corresponding to each sample operator in the first sample recommendation model and first sample structural features to obtain splicing features corresponding to the first sample recommendation model; invoking an attention unit in a feature extraction layer of the feature extraction network, processing the spliced features based on an attention mechanism, determining the weight of each operator, and adjusting sample operator features corresponding to the plurality of sample operators in the spliced features based on the weights of the plurality of sample operators; invoking an encoding unit in a feature extraction layer of the feature extraction network, encoding sample operator features and first sample structure features, which are included in the splicing features and correspond to the adjusted sample operators, to obtain model features corresponding to the first sample recommendation model, and performing precision prediction based on the model features corresponding to the first sample recommendation model to obtain first prediction recommendation precision;

invoking the recommendation precision prediction model, and performing precision prediction on model features corresponding to the second sample recommendation model to obtain second prediction recommendation precision;

And adjusting model parameters in the recommendation precision prediction model and proxy recommendation precision of the second sample recommendation model based on the first prediction recommendation precision, the first sample recommendation precision, the second prediction recommendation precision and the second sample recommendation precision so that errors between the prediction recommendation precision corresponding to the second sample recommendation model at historical moments and the adjusted proxy recommendation precision approach target errors corresponding to the historical moments, wherein the proxy recommendation precision is a variable for indicating the recommendation precision of the second sample recommendation model.

11. The recommended precision prediction model training method of claim 10, further comprising:

acquiring sample operator characteristics, second sample structure characteristics and the agent recommendation precision corresponding to each sample operator in the second sample recommendation model, wherein the second sample structure characteristics represent the connection relationship between any two sample operators in the second sample recommendation model;

and invoking the recommendation precision prediction model to perform precision prediction on model features corresponding to the second sample recommendation model to obtain second prediction recommendation precision, wherein the method comprises the following steps: and calling the recommendation precision prediction model, extracting features of sample operator features and second sample structure features corresponding to each sample operator in the second sample recommendation model to obtain model features corresponding to the second sample recommendation model, and performing precision prediction based on the model features corresponding to the second sample recommendation model to obtain the second prediction recommendation precision.

12. The recommendation precision prediction model training method according to claim 10, wherein the adjusting the model parameters in the recommendation precision prediction model and the proxy recommendation precision of the second sample recommendation model based on the first prediction recommendation precision, the first sample recommendation precision, the second prediction recommendation precision, and the second sample recommendation precision so that the error between the prediction recommendation precision corresponding to the historical time and the adjusted proxy recommendation precision of the second sample recommendation model approaches the target error corresponding to the historical time comprises:

13. The recommendation precision prediction model training method according to claim 11, wherein the recommendation precision prediction model is trained based on the first prediction recommendation precision, the first sample recommendation precision, a second prediction recommendation precision, and the proxy recommendation precision using the following penalty function:

wherein ,

representing the loss function,/->

Representing the recommendation accuracy prediction model, +.>

Representing model parameters in said recommended precision prediction model,/->

Indicate->

The first prediction recommendation precision corresponding to the first sample recommendation model is +.>

Indicate->

Sample operator features corresponding to each sample operator in the first sample recommendation model and the first sample structural features +.>

Representing the number of said first sample recommendation models,/for training said recommendation accuracy prediction model>

Indicate->

The first sample recommendation precision corresponding to the first sample recommendation model is +.>

Indicate->

The second prediction recommendation accuracy corresponding to the second sample recommendation model is +.>

Indicate->

Sample operator features corresponding to each sample operator in the second sample recommendation model and the second sample structural features, ++>

Representing the number of recommended models of said second sample for training said recommended accuracy prediction model,/->

Representing agent recommendation accuracy,/-, and>

indicate->

The agent recommendation accuracy corresponding to the second sample recommendation model,/for each of the agent recommendation models>

Is a reference parameter;

the loss function satisfies the following condition:

wherein ,

Indicate->

Indicate->

The second sample recommendation model is at +.>

Prediction recommendation precision corresponding to each moment, +.>

Indicate->

Target error corresponding to each moment,/-)>

Representation->

A historical time.

14. A classification model processing method, the method comprising:

acquiring a classification model, wherein the classification model comprises a plurality of operators which are arranged in sequence, and when the data to be classified input into the classification model is an image, the operators sequentially process the image data input into the classification model in sequence to determine that the data type is a face image or a non-face image; when the data to be classified are articles, the operators sequentially process the article data input into the classification model according to the sequence, and the article category of the articles is determined; when the data to be classified is audio, the operators sequentially process the audio data input into the classification model according to the sequence, and determine the audio type of the audio;

invoking a splicing layer in a feature extraction network of the classification precision prediction model, and splicing operator features corresponding to each operator and the structural features to obtain splicing features corresponding to the classification model; invoking an attention unit in a feature extraction layer of the feature extraction network, processing the spliced features based on an attention mechanism, determining the weight of each operator, and adjusting operator features corresponding to a plurality of operators in the spliced features based on the weights of the operators; invoking an encoding unit in a feature extraction layer of the feature extraction network to encode operator features corresponding to the adjusted operators and structural features of a classification model, which are included in the spliced features, so as to obtain model features corresponding to the classification model;

invoking a prediction network of the classification precision prediction model, and performing precision prediction based on the model characteristics to obtain classification precision corresponding to the classification model, wherein the classification precision is used for representing the classification accuracy of the classification model, and the classification precision is used for representing the accuracy of the classification model in determining whether the image belongs to a face image or not when the data to be classified is the image; when the data to be classified is an article, the classification precision is used for expressing the accuracy of the classification model in determining the article category to which the article belongs; when the data to be classified is audio, the classification precision is used for expressing the accuracy of the classification model for determining the audio type of the audio;

The training process of the classification precision prediction model comprises the following steps:

determining a first prediction classification precision of the first sample classification model and a second prediction classification precision of the second sample classification model based on the classification precision prediction model to be trained;

and adjusting parameters of a classification precision prediction model to be trained and proxy classification precision of the second sample classification model based on the first prediction classification precision, the first sample classification precision, the second prediction classification precision and the second sample classification precision so that errors between the prediction classification precision corresponding to the historical moment and the adjusted proxy classification precision of the second sample classification model approach to target errors corresponding to the historical moment, wherein the proxy classification precision is a variable for indicating the classification precision of the second sample classification model.

15. A recommendation model processing apparatus, the apparatus comprising:

the second feature acquisition unit is configured to execute a splicing layer in a feature extraction network for calling a recommendation precision prediction model, splice operator features corresponding to each operator and the structural features, and obtain splicing features corresponding to the recommendation model; invoking an attention unit in a feature extraction layer of the feature extraction network, processing the spliced features based on an attention mechanism, determining the weight of each operator, and adjusting operator features corresponding to a plurality of operators in the spliced features based on the weights of the operators; invoking an encoding unit in a feature extraction layer of the feature extraction network to encode operator features corresponding to each operator included in the spliced features and structural features of a recommendation model to obtain model features corresponding to the recommendation model;

the recommendation precision prediction unit is configured to execute a prediction network for calling the recommendation precision prediction model, and performs precision prediction based on the model characteristics to obtain recommendation precision corresponding to the recommendation model;

The apparatus further comprises: a model training unit configured to perform: determining a first prediction recommendation precision of a first sample recommendation model and a second prediction recommendation precision of a second sample recommendation model based on a recommendation precision prediction model to be trained;

16. The apparatus according to claim 15, wherein the recommendation model further includes an index identifier corresponding to each operator, and the first feature obtaining unit includes:

17. The recommendation model processing apparatus according to claim 15, wherein the recommendation model includes N operators, N being an integer greater than 1, the first feature acquisition unit comprising:

18. The recommended model processing device of claim 15, wherein the recommended accuracy prediction model further comprises a feature acquisition network,

19. The recommended model processing device of claim 18, wherein the model training unit is further configured to perform obtaining a sample operator feature, a first sample structural feature, and a first sample recommendation accuracy corresponding to each sample operator in a first sample recommendation model, the first sample structural feature representing a connection relationship between any two sample operators in the first sample recommendation model; acquiring sample operator characteristics, second sample structure characteristics and the agent recommendation precision corresponding to each sample operator in the second sample recommendation model, wherein the second sample structure characteristics represent the connection relationship between any two sample operators in the second sample recommendation model;

the model training unit is configured to execute the recommendation precision prediction model, process the sample operator characteristics and the first sample structure characteristics corresponding to each sample operator in the first sample recommendation model to obtain a first prediction recommendation precision, and process the sample operator characteristics and the second sample structure characteristics corresponding to each sample operator in the second sample recommendation model to obtain a second prediction recommendation precision.

20. The recommendation model processing device according to claim 15, wherein the recommendation model processing device further comprises:

21. The recommended model processing method according to claim 15, wherein the model training unit is configured to perform:

22. The recommendation model processing device according to claim 15, wherein the recommendation model processing device further comprises: the acquiring model unit is configured to execute acquiring a plurality of recommending models, wherein operators which are different exist in any two recommending models, or the connection relationship between the operators in any two recommending models is different;

the recommendation model processing apparatus further includes:

23. The recommendation model processing apparatus according to any one of claims 15 to 22, wherein said recommendation model further comprises a plurality of edges, each edge connecting two operators for processing input data input to said operators to obtain output data; and, the output data of one operator connected by each edge is used as the input data of the other operator connected.

24. A recommendation accuracy prediction model training device, the device comprising:

the sample acquisition unit is configured to acquire sample operator characteristics, first sample structural characteristics and first sample recommendation precision corresponding to each sample operator in a first sample recommendation model, wherein the first sample structural characteristics represent a connection relationship between any two sample operators in the first sample recommendation model, each sample operator in the first sample recommendation model is used for sequentially processing account data and article data input into the first sample recommendation model according to a sequence to obtain recommendation parameters, and the recommendation parameters indicate whether the article is recommended to the account;

the precision prediction unit is configured to execute a splicing layer in a feature extraction network for calling a recommended precision prediction model, splice sample operator features corresponding to each sample operator in the first sample recommendation model and first sample structure features, and obtain splicing features corresponding to the first sample recommendation model; invoking an attention unit in a feature extraction layer of the feature extraction network, processing the spliced features based on an attention mechanism, determining the weight of each operator, and adjusting sample operator features corresponding to the plurality of sample operators in the spliced features based on the weights of the plurality of sample operators; invoking an encoding unit in a feature extraction layer of the feature extraction network, encoding sample operator features and first sample structural features, which are included in the splicing features and correspond to the sample operators, to obtain model features corresponding to the first sample recommendation model, and performing precision prediction based on the model features corresponding to the first sample recommendation model to obtain first prediction recommendation precision; invoking the recommendation precision prediction model, and performing precision prediction on model features corresponding to the second sample recommendation model to obtain second prediction recommendation precision;

And a model training unit configured to perform adjustment of model parameters in the recommendation precision prediction model and proxy recommendation precision of the second sample recommendation model based on the first prediction recommendation precision, the first sample recommendation precision, the second prediction recommendation precision, and the second sample recommendation precision so that an error between the prediction recommendation precision of the second sample recommendation model corresponding to a historical time and the adjusted proxy recommendation precision, which is a variable for indicating the recommendation precision of the second sample recommendation model, approaches a target error corresponding to the historical time.

25. The recommended accuracy prediction model training device of claim 24, wherein the sample acquisition unit is further configured to perform: acquiring sample operator characteristics, second sample structure characteristics and agent recommendation precision corresponding to each sample operator in a second sample recommendation model, wherein the second sample structure characteristics represent the connection relationship between any two sample operators in the second sample recommendation model;

the precision prediction unit is configured to execute invoking the recommendation precision prediction model, perform feature extraction on sample operator features and second sample structure features corresponding to each sample operator in the second sample recommendation model to obtain model features corresponding to the second sample recommendation model, and perform precision prediction based on the model features corresponding to the second sample recommendation model to obtain the second prediction recommendation precision.

26. The recommended precision prediction model training device of claim 24, wherein the model training unit is configured to perform:

27. The recommended precision prediction model training device of claim 25, wherein the model training unit is configured to perform:

training the recommendation accuracy prediction model based on the first prediction recommendation accuracy, the first sample recommendation accuracy, the second prediction recommendation accuracy, and the proxy recommendation accuracy using the following loss function:

wherein ,

representing the loss function,/->

Representing the recommendation accuracy prediction model, +. >

Indicate->

Indicate->

Indicate->

Indicate->

Indicate->

Representing agent recommendation accuracy,/-, and>

indicate->

Is a reference parameter;

the loss function satisfies the following condition:

wherein ,

representing the number of recommended models of said second sample for training said recommended accuracy prediction model,/- >

Indicate->

Indicate->

The second sample recommendation model is at +.>

Prediction recommendation precision corresponding to each moment, +.>

Indicate->

Target error corresponding to each moment,/-)>

Representation->

A historical time.

28. A classification model processing apparatus, the apparatus comprising:

a classification model acquisition unit configured to perform acquisition of a classification model including a plurality of operators arranged in order, the plurality of operators sequentially processing image data input to the classification model in order, in a case where data to be classified input to the classification model is an image, determining that the data type is a face image or a non-face image; when the data to be classified are articles, the operators sequentially process the article data input into the classification model according to the sequence, and the article category of the articles is determined; when the data to be classified is audio, the operators sequentially process the audio data input into the classification model according to the sequence, and determine the audio type of the audio;

the second feature acquisition unit is configured to execute a splicing layer in a feature extraction network for calling the classification precision prediction model, splice operator features corresponding to each operator and the structural features, and obtain splicing features corresponding to the classification model; invoking an attention unit in a feature extraction layer of the feature extraction network, processing the spliced features based on an attention mechanism, determining the weight of each operator, and adjusting operator features corresponding to a plurality of operators in the spliced features based on the weights of the operators; invoking an encoding unit in a feature extraction layer of the feature extraction network to encode operator features corresponding to each operator included in the spliced features and structural features of a classification model to obtain model features corresponding to the classification model;

the classification precision prediction unit is configured to execute a prediction network for calling the classification precision prediction model, perform precision prediction based on the model characteristics, obtain classification precision corresponding to the classification model, and the classification precision is used for representing the classification accuracy of the classification model, and in the case that the data to be classified is an image, the classification precision is used for representing the accuracy of the classification model in determining whether the image belongs to a face image; when the data to be classified is an article, the classification precision is used for expressing the accuracy of the classification model in determining the article category to which the article belongs; when the data to be classified is audio, the classification precision is used for expressing the accuracy of the classification model for determining the audio type of the audio;

29. An electronic device, the electronic device comprising:

one or more processors;

a memory for storing the one or more processor-executable instructions;

wherein the one or more processors are configured to perform the recommendation model processing method of any one of claims 1 to 9, or to perform the recommendation accuracy prediction model training method of any one of claims 10 to 13, or to perform the classification model processing method of claim 14.

30. A computer readable storage medium, wherein instructions in the computer readable storage medium, when executed by a processor of an electronic device, cause the electronic device to perform the recommendation model processing method of any one of claims 1 to 9, or to perform the recommendation accuracy prediction model training method of any one of claims 10 to 13, or to perform the classification model processing method of claim 14.