JP2024507602A - Data processing methods and methods for training predictive models - Google Patents

Abstract

TECHNICAL FIELD The present disclosure provides a data processing method and a method for training a predictive model, and relates to the field of computer technology, and in particular to artificial intelligence technology. As an implementation plan, a target to be processed is determined, a target category to which the target to be processed belongs is determined based on a classification attribute of the target to be processed, and a target category for the target to be processed is determined based on the target category. obtaining a prediction result of the target object by determining a prediction model that is predicted by the target object, and processing at least one prediction feature of the target object using the prediction model; The classification attribute includes variables of the same type. [Selection diagram] Figure 2

Description

This application is based on Chinese Patent Application No. 202210088356 filed on January 25, 2022. X, the entire contents of which are incorporated herein by reference.
TECHNICAL FIELD The present disclosure relates to the field of computer technology, and in particular to artificial intelligence technology, and in particular to data processing methods and methods, apparatus, electronic equipment, computer-readable storage media, and computer program products for training predictive models.

Artificial intelligence is a subject that studies how to make computers imitate some human thought processes and intelligent behaviors (e.g., learning, reasoning, thinking, planning, etc.), and it involves both hardware technology and software technology. There is also surface technology. Artificial intelligence hardware technology generally includes sensors, artificial intelligence dedicated chips, cloud computing, distributed storage, big data processing and other technologies, and artificial intelligence software technology mainly includes computer vision technology, voice recognition technology, It includes several major directions such as natural language processing technology and machine learning/deep learning, big data processing technology, and knowledge graph technology.

Prediction tasks play an important role in various artificial intelligence applications. For example, in the video recommendation scene, predicting how long a user will spend viewing a recommended video resource plays an important role in determining the video recommendation result.

The methods described in this part are not necessarily those previously envisaged or employed. Unless otherwise specified, any method described in this section should not be considered prior art by virtue of its inclusion in this section. Similarly, unless otherwise specified, the subject matter referred to in this section should not be considered to be an admission of prior art.

The present disclosure provides data processing methods and methods, apparatus, electronic devices, computer readable storage media, and computer program products for training predictive models.
According to one aspect of the present disclosure, determining a target to be processed, determining a target category to which the target to be processed belongs based on a classification attribute of the target to be processed, and determining the target category to which the target to be processed belongs based on the target category. obtaining a prediction result of the target object by determining a prediction model to be used for the target object and processing at least one predictive feature of the target object using the prediction model; The present invention provides a data processing method including: the result and the classification attribute are variables of the same type.

According to another aspect of the disclosure, determining a sample set that includes a plurality of sample objects, and determining a first sample subset and a second sample subset in the sample set based on classification attributes of the sample objects. training a first predictive model using a first sample subject in the first sample subset; and training a second predictive model using a second sample subject in the second sample subset. training, wherein the first prediction model and the second prediction model are configured to obtain a prediction result of the processed object by processing at least one prediction feature of the processed object. provides a method for training a prediction model in which the prediction result and the classification attribute are homogeneous variables.

According to another aspect of the present disclosure, a process target determination unit configured to determine a process target, and determining a target category to which the process target belongs based on a classification attribute of the process target. a prediction model determination unit configured to determine a prediction model to be used for the process target based on the target category; and a prediction model determination unit configured to determine a prediction model to be used for the process target based on the target category; A prediction unit configured to obtain a prediction result of the target to be processed by processing at least one prediction feature of the target, the prediction result and the classification attribute being the same type of variable; Provided is a data processing device including:

According to another aspect of the disclosure, a sample determining unit configured to determine a sample set that includes a plurality of sample objects; and a sample determination unit configured to determine a sample set that includes a plurality of sample objects; a classification unit configured to determine a sample subset of 2; and a first predictive model training configured to utilize a first sample subject in the first sample subset to train a first predictive model. a second predictive model training unit configured to utilize a second sample subject in a second sample subset to train a second predictive model; The prediction model and the second prediction model are used to obtain a prediction result of the processed object by processing at least one prediction feature of the processed object, where the prediction result and the classification attribute are An apparatus for training a predictive model on homogeneous variables is provided.

According to another aspect of the disclosure, the invention includes at least one processor and a memory communicatively coupled to the at least one processor, wherein the memory stores instructions executable by the at least one processor. The instructions are stored and executed by the at least one processor to provide an electronic device capable of causing the at least one processor to perform the method described above.

According to another aspect of the present disclosure, a non-transitory computer-readable storage medium having computer instructions stored thereon for causing a computer to perform the methods described above is provided.
According to another aspect of the present disclosure, a computer program product is provided that includes a computer program that, when executed by a processor, implements the method described above.

According to one or more embodiments of the present disclosure, for a single variable prediction problem, the target category to which the processed target belongs is determined based on the value of a classification attribute that is the same type of variable as the single variable to be predicted. A prediction result can be obtained based on a prediction model trained for objects belonging to the object category. Using the above method, predictions can be made using prediction models trained for variable prediction in different intervals, thereby allowing better discrimination of the characteristics of the processed object in different categories.

It should be understood that the content described in this section is not intended to identify key points or important features of the embodiments of the present disclosure, and is not intended to limit the protection scope of the present disclosure. . Other features of the disclosure will be readily understood from the following specification.

The drawings illustratively illustrate embodiments, constitute a part of the specification, and, together with the written description, serve to explain example embodiments of the embodiments. The illustrated embodiments are for illustrative purposes only and are not intended to limit the scope of the claims. In all drawings, the same reference numerals refer to similar, but not necessarily identical, elements.
1 is a schematic diagram illustrating an example system that can implement various methods described herein, according to embodiments of the present disclosure. FIG. 3 is an exemplary flowchart illustrating a data processing method according to an embodiment of the present disclosure. 1 is an example flowchart illustrating a method for training a predictive model according to an embodiment of the present disclosure. FIG. 3 is an exemplary diagram showing regression loss calculated by a loss function according to an embodiment of the present disclosure. FIG. 3 is an exemplary diagram illustrating the slope of a loss function according to an embodiment of the present disclosure. FIG. 2 is an example diagram illustrating a multi-task training frame according to an embodiment of the present disclosure. 1 is an exemplary block diagram illustrating a data processing apparatus according to an embodiment of the present disclosure. FIG. 1 is an example block diagram illustrating an apparatus for training a predictive model according to an embodiment of the present disclosure. FIG. 1 is a configuration block diagram illustrating an example electronic device that can be used to implement embodiments of the present disclosure. FIG.

Hereinafter, exemplary embodiments of the present disclosure will be described in conjunction with the drawings, and various details included therein in the embodiments of the present disclosure will aid in understanding and should therefore be considered as merely illustrative. be. Accordingly, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the disclosure. Similarly, for the sake of clarity and brevity, the following description omits descriptions of well-known features and structures.

In this application, unless otherwise specified, the terms "first," "second," etc. used to describe various elements are not intended to limit the relative position, timing, or importance of these elements. I haven't. These terms are only used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of the element, or in some cases, may refer to different instances based on the description of the context.

The terminology used in the description of various examples of this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context clearly indicates otherwise, there may be one or more elements, unless the number of elements is specifically limited. It should be noted that the term "and/or" as used in this disclosure covers any and all possible combinations of the listed items.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.
FIG. 1 depicts a schematic diagram of an example system 100 in which various methods and apparatus described herein may be implemented, according to embodiments of the present disclosure. Referring to FIG. 1, the system 100 includes one or more client devices 101, 102, 103, 104, 105 and 106, a server 120, and one or more client devices coupling the one or more client devices to the server 120. Includes multiple communication networks 110. Client devices 101, 102, 103, 104, 105 and 106 may be configured to run one or more applications.

In embodiments of the present disclosure, server 120 is operable to execute one or more services or software applications of methods according to embodiments of the present disclosure.
In some examples, server 120 may also provide other services or software applications that may include non-virtual and virtual environments. In some examples, these services may be provided as web-based services or cloud services, such as on client devices 101, 102, 103, 104, 105 and/or in a software-as-a-service (SaaS) model. 106 users.

In the configuration shown in FIG. 1, server 120 may include one or more assemblies that implement the functions performed by server 120. These assemblies may include software assemblies, hardware assemblies, or a combination thereof that can be executed on one or more processors. Users operating client devices 101, 102, 103, 104, 105, and/or 106 interact with server 120 using one or more client applications to take advantage of the services provided by these assemblies. be able to. It should be appreciated that a variety of different system configurations are possible and may differ from system 100. Accordingly, FIG. 1 is one example of a system for implementing the various methods described herein, and is not intended to be limiting.

A user may use client devices 101, 102, 103, 104, 105 and/or 106 to obtain user input and provide the user with processing results obtained by methods according to embodiments of the present disclosure. can. A client device can provide an interface through which a user of the client device interacts with the client device. The client device can also output information to the user via this interface. Although only six client devices are illustrated in FIG. 1, one skilled in the art will appreciate that the present disclosure can support any number of client devices.

Client devices 101, 102, 103, 104, 105 and/or 106 may include portable handheld devices, general purpose computers (e.g., personal computers or laptops), workstation computers, wearable devices, smart screen devices, self-service terminal devices, It may include various types of computing devices, such as service robots, gaming systems, thin clients, various messaging devices, sensors, or other sensing devices. These computing devices may be of various types and versions, such as MICROSOFT Windows, APPLE iOS, similar UNIX operating systems, Linux or similar Linux operating systems (e.g., GOOGLE Chrome OS). It can run software applications and operating systems, and can include a variety of mobile operating systems, such as MICROSOFT Windows Mobile OS, iOS, Windows Phone, and Android. Portable handheld devices may include mobile phones, intelligent phones, tablets, personal digital assistants (PDAs), and the like. Wearable devices may include head-mounted displays (eg, smart glasses) and other devices. The gaming system may include a variety of handheld gaming devices, internet-enabled gaming devices, and the like. A client device can run various applications, such as Internet-related applications, communication applications (eg, email applications), short message service (SMS) applications, and use various communication protocols, for example.

Network 110 may be any type of network known to those skilled in the art, which supports any one of multiple available protocols (TCP/IP, SNA, IPX, etc.) can be used. By way of example, one or more networks 110 may include a local area network (LAN), an Ethernet-based network, a token loop, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet. , a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and/or any combination of these and other networks.

Server 120 may be one or more general purpose computers, dedicated server computers (e.g., PC servers, UNIX servers, midrange servers), blade servers, large scale computers, server clusters, or other suitable servers. may include other arrangements and/or combinations. Server 120 may include one or more virtual machines running a virtual operating system or other computing architecture that involves virtualization (e.g., a set of virtualized logical storage devices to maintain virtual storage for the server). (one or more flexible pools). In various embodiments, server 120 may execute one or more services or software applications that provide the functionality described below.

The computing units in server 120 may run one or more operating systems, including any of the operating systems described above and any commercial server operating system. Server 120 may also run any one of a variety of additional server and/or middle tier applications, such as an HTTP server, an FTP server, a CGI server, a JAVA server, a database server, and the like.

In some embodiments, server 120 includes one or more servers for analyzing and integrating data feeds and/or event updates received from users of client devices 101, 102, 103, 104, 105, and/or 106. Applications may also be included. Server 120 may include one or more applications that display data feeds and/or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105, and/or 106. .

In some embodiments, server 120 may be a distributed system server or a blockchain-embedded server. The server 120 may be a cloud server, an intelligent cloud computing server or an intelligent cloud host equipped with artificial intelligence technology. A cloud server is a host product in a cloud computing service system, and solves the deficiencies of conventional physical host and virtual private server (VPS) services, such as high management difficulty and low business expandability.

System 100 may also include one or more databases 130. In some embodiments, these databases can be used to store data and other information. For example, one or more of databases 130 can be used to store information such as audio files and video files. Database 130 can be located in a variety of locations. For example, the database used by server 120 may be local to server 120 or may be remote from server 120 and communicated with server 120 via a network or dedicated connection. Database 130 may be of various types. In some embodiments, the database used by server 120 may be a relational database. One or more of these databases can store, update, and retrieve data from the databases in response to instructions.

In some embodiments, one or more of databases 130 may also be used by an application to store data for the application. The database used in the application may be of various types, such as a key-value repository, an object repository, or a general purpose repository supported by a file system.

The system 100 of FIG. 1 can be configured and operated in a variety of ways to enable application of the various methods and apparatus described in accordance with this disclosure.
FIG. 2 is an exemplary flowchart illustrating a data processing method according to an embodiment of the present disclosure. A client device as shown in FIG. 1 or a server may be used to perform method 200 as shown in FIG.

As shown in FIG. 2, in step S202, a target to be processed is determined.
In step S204, the target category to which the target object belongs is determined based on the classification attribute of the target target.

In step S206, a prediction model to be used for the object to be processed is determined based on the object category.
In step S208, a prediction result of the object to be processed is obtained by processing at least one prediction feature of the object to be processed using the prediction model, where the prediction result and the classification attribute are variables of the same type.

A method provided by one or more embodiments of the present disclosure, for a single variable prediction problem, a target to which the processed target belongs based on the value of a classification attribute that is the same type of variable as the single variable to be predicted. A category can be determined and a prediction result can be obtained based on a prediction model trained for objects belonging to the target category. Using the above method, predictions can be made using prediction models trained for variable prediction in different intervals, thereby allowing better discrimination of the characteristics of the processed object in different categories.

The data processing method provided by the present disclosure will be described in detail below.
In step S202, a target to be processed can be determined.
In step S204, the target category to which the target object belongs can be determined based on the classification attribute of the target target.

In some embodiments, the processed object may be a video and the classification attribute is video length. By processing a video using the method provided by the present disclosure, a predicted time for a user to view a video can be obtained as a prediction result. Here, the predicted time allows the user to specify a time to view the video when the video is recommended to the user. In another embodiment, the object to be processed may be historical weather data, and the classification attribute may be one of the meteorological parameters included in the historical weather data (eg, temperature, humidity, wind power, precipitation, etc.). Weather historical data can be processed by the methods provided by the present disclosure to obtain forecast results instructing weather forecasts. The prediction result may indicate a prediction result of a meteorological parameter. In other embodiments, any other type of univariate continuous value prediction task may be accomplished by the methods provided by embodiments of the present disclosure.

In the following content, the principle of the present disclosure will be explained by taking as an example that the object to be processed is a moving image.
In some embodiments, step S204 determines the video length section to which the video belongs based on the video length of the video, and sets the indicator of the determined video length section as the target category to which the video belongs. It may include being able to determine.

For the video resources included in the video platform, let us assume that the number of video resources is 1000 and the longest video length is 100 seconds, and classify the video resources based on the number of video resources with different video lengths. can do. For example, assuming that the number of video length sections to be classified is 2, 1000 video resources can be arranged according to the video length, and the video length to which the first half of the video belongs The section is determined as the first video category, and the section of video length to which the latter half of the video belongs is determined as the second video category. For example, a first video category may include video resources within an interval where the video length is 0 to 45 seconds, and a second video category may include video resources within an interval where the video length is 46 to 100 seconds. May include. Video resources on a platform can be divided into two or more categories in a similar manner.

In some examples, if there is a large difference in the video segment lengths obtained in the average classification according to the quantity distribution, the video classification results can be adjusted in various ways. For example, suppose that two video categories are obtained by classifying the longest video length as 100 seconds, and the length of the video resource included in the first video category is in the interval from 0 to 10 seconds. However, if the length of the video resource included in the second video category is in the range of 11 to 100 seconds, the video classification result can be adjusted. For example, the difference in the lengths of video resources corresponding to two video categories can be limited to 50 seconds or less. The video classification results may be adjusted based on such restrictions, but the number of video resources included in the two video categories may not be restricted to be similar (or nearly the same).

In step S206, a prediction model to be used for the object to be processed may be determined based on the object category. For example, the target category includes a first target category (e.g., first video category) and a second target category (e.g., second video category), and the target category is the first target category. in response to determining that the first predictive model is to be used for the target, and in response to determining that the target category is a second target category, It can be determined to use the predictive model for the target to be processed. Here, the first prediction model may be a prediction model obtained by training using video resources belonging to the first video category, and the second prediction model may be a prediction model obtained by training using video resources belonging to the first video category. It may also be a predictive model obtained by training using resources. With such a scheme, different predictive models may be trained to identify different characteristics that objects in different object categories have (e.g., different characteristics of long and short videos), thereby determining whether they belong to different object categories. Different processed objects will obtain more accurate prediction results.

In some examples, the first predictive model and the second predictive model may be similar models with different parameters, such as polynomials, neural networks, or any other applicable mathematical model.

In step S208, a prediction result of the object to be processed is obtained by processing at least one prediction feature of the object to be processed using the prediction model, where the prediction result and the classification attribute are variables of the same type.

Here, the variable value of the prediction result and the variable value of the classification attribute have the same variable unit. Taking prediction of video viewing time as an example, the classification attribute is video time, the prediction result is video viewing time, and the units of the classification attributes and prediction result variables are both units of time. With such a method, it is possible to easily classify univariate continuous value prediction problems and predict variables of different categories.

In some embodiments, step S208 includes obtaining a normalized prediction result for the target object by processing at least one predictive feature of the target object using a predictive model; obtaining a prediction result by processing the normalized prediction result using a normalization parameter determined by the method.

As an example, the processing target is a video, and the classification attribute is video length. When training a model, we calculate the normalization of the actual result of the viewing time by the user of each sample video for videos in different length intervals. , so that when training the model, the predictive effectiveness of the model is not affected by the duration itself. In other words, when training videos in intervals of different lengths, the normalized prediction results are used to obtain model parameters, so that the model learns the user viewing characteristics of videos in intervals of different lengths. be able to.

In some embodiments, the normalization parameter corresponding to a target category may be an interval parameter of a classification variable corresponding to the target category. For example, assuming that the classification variable is video length, the section parameter may be the maximum value (ie, the rightmost end point) of the video length section corresponding to the video category. This allows the actual viewing time by the user of the sample video in the video category to be normalized between 0 and 1. In other examples, the interval parameter may be determined as any other value in the interval of video length corresponding to the video category. In such cases, the actual result after normalization may be greater than one. For example, when the rightmost end point of the interval corresponding to the classification variable includes an infinite term, a value at one of the midpoints in the interval may be selected as the normalization parameter.

A prediction model trained and obtained by such a method outputs a normalized prediction result of the target to be processed. A prediction result can be obtained by performing inverse processing on the normalized prediction result output from the prediction model using the normalization parameter of the target category corresponding to the prediction model. For example, the normalization parameter may be 45 for a video object within a section whose video length is 0 to 45 seconds. During training, the actual viewing time of a sample video by a user is normalized by dividing by 45. A prediction model trained using such a method outputs a normalized prediction result, for example, 0.4, when processing a video to be processed. The actual prediction result can be obtained by multiplying the normalized prediction result by 45 again, that is, 45×0.4=18 seconds. When the normalization parameter is set to another numerical value, the actual prediction result may be obtained by performing inverse processing on the normalized prediction result using a similar method.

FIG. 3 is an example flowchart illustrating a method for training a predictive model according to an embodiment of the present disclosure. A predictive model used in method 200 can be trained by a method such as that shown in FIG. A client device as shown in FIG. 1 or a server may be used to perform method 300 as shown in FIG.

As shown in FIG. 3, in step S302, a sample set including a plurality of sample objects is determined.
In step S304, a first sample subset and a second sample subset in the sample set are determined based on the classification attributes of the sample objects.

In step S306, a first sample object in the first sample subset is utilized to train a first predictive model.
In step S308, a second sample object in the second sample subset is utilized to train a second predictive model.

Here, the first prediction model and the second prediction model are used to obtain a prediction result of the object to be processed by processing at least one prediction feature of the object to be processed, and here, the prediction result is and classification attributes are the same type of variables.

Utilizing the training methods provided by embodiments of the present disclosure, different prediction models may be trained for different objects with different classification attributes, thereby providing more accurate prediction results for objects with different characteristics. be able to.

The method for training a predictive model provided by the present disclosure is described in detail below. Predictive models according to the present disclosure can be implemented using polynomials, neural networks, or any other type of mathematical model.

In step S302, a sample set including multiple sample objects may be determined. It should be explained that the sample objects in this example are derived from public datasets.

In step S304, a first sample subset and a second sample subset in the sample set may be determined based on the classification attributes of the sample objects.
In some embodiments, the sample object and the processed object associated with the predictive model may be a video, and the classification attribute may be video length. In another embodiment, the sample object and the processing object associated with the predictive model may be historical weather data, and the classification attribute is one of the meteorological parameters included in the historical weather data (e.g., temperature, humidity, wind force, precipitation). quantity, etc.). Hereinafter, the principle of the present disclosure will be explained using an example in which the sample target is a moving image. It should be understood that in other embodiments, any other type of univariate continuous value prediction task may be accomplished by the methods provided by embodiments of the present disclosure.

The first sample subset may include at least one first sample object belonging to a first video length interval, and the second sample subset may include at least one second sample object belonging to a second video length interval. may include sample objects. As mentioned above, sample videos can be classified based on the video length of the video resource. The sample videos can be classified based on the longest length of the video resources in the sample set and the distribution of the video resources in intervals of different video lengths, thereby making it possible to classify the sample videos into a first sample subset of the first video category and a second sample subset of the first video category. Obtain a second sample subset of video categories. For example, assuming that the number of video resources is 1000 and the longest video length is 100 seconds, the first sample subset may include video resources within the interval whose video length is 0 to 45 seconds. Often, the second sample subset may include video resources within an interval where the video length is between 46 and 100 seconds. In some examples, the number of sample objects in the first sample subset and the number of sample objects in the second sample subset may be the same. Similar numbers described here may mean that the numbers are exactly the same or that the number of sampled objects in the two sample subsets is approximately the same, i.e., the difference in numbers is a predetermined difference. It means that it is smaller than the threshold value. In another example, when the interval difference obtained by averaging according to the number of samples is too large (for example, the video length interval corresponding to the first sample subset is 0 to 10 seconds, and the second (When the video length section corresponding to the sample subset is 11 to 100 seconds), the difference in video resource length sections corresponding to the two video categories can be limited to 50 seconds or less. The results of the sample video classification may be adjusted based on such constraints and the number of video resources included in the sample subsets of the two video categories may not be restricted to be similar (or nearly the same). .

In step S306, a first sample object in the first sample subset may be used to train a first predictive model.
In step S308, a second sample object in the second sample subset may be used to train a second predictive model.

Here, the first prediction model and the second prediction model are used to obtain a prediction result of the object to be processed by processing at least one prediction feature of the object to be processed, and here, the prediction result is and classification attributes are the same type of variables.

Here, although the sample objects used when training the first and second predictive models belong to different object categories, the parameters in the models can be trained in the same way. Hereinafter, a training method according to an embodiment of the present disclosure will be described using the first prediction model as an example. A similar method can be trained by applying a second subset of samples to obtain a second predictive model.

The first predictive model can be obtained by training in the following manner. determining a first current parameter of a first predictive model and processing at least one first sample feature of the first sample object utilizing the first current parameter; obtaining a first sample prediction result of the sample object, determining a first actual sample result of the first sample, and determining the first sample prediction result of the sample target based on the first sample prediction result and the first actual sample result. Adjusting the first current parameter.

When starting training, the first current parameter of the first predictive model may be a predetermined first initial parameter. After finishing each training session, the first current parameter used for the current training session can be updated. The update can be performed by a variety of optimization methods, including, for example, linear iterative methods, quadratic iterative methods, gradient descent, Newton's methods, and the like. For a prediction model realized by a neural network, the first current parameter may be updated using an error backpropagation method.

In some embodiments, the current parameters may be adjusted based on the loss results obtained with the loss function.
For example, a first loss can be determined based on a first sample prediction result and a first actual sample result, and a first current parameter of the first prediction model is adjusted based on the first loss. do. A first loss may be determined based on a predetermined loss function, wherein the predetermined loss function indicates a difference between a first sample predicted result and a first actual sample result. Can be done.

In some examples, applying different loss functions to sample points with different errors can improve the optimization effect on the current parameters of the prediction model during the training process.

For example, when the difference between the first sample prediction result and the first actual sample result is smaller than the training threshold, the first loss function is used to determine the first loss, and the difference between the first sample prediction result and the first actual sample result is smaller than the training threshold. A second loss function is used to determine the first loss when the difference from the first actual sample result is greater than or equal to a training threshold. Here, when the difference between the first sample prediction result and the first actual sample result is equal to the training threshold, the loss determined by the first loss function and the loss determined by the second loss function are the same. It is. The greater the difference between the first sample prediction result and the first actual sample result, the greater the sample optimization error. The optimization efficiency of the model can be improved by applying different loss functions to sample points with different optimization errors.

In some examples, the training threshold may change as the number of training increases. For example, the training threshold can be attenuated as the number of training increases. Attenuation to the training threshold can be performed using various learning rate decay methods used in machine learning methods, such as exponential decay, fixed step size decay, variable step size decay, cosine annealing decay, etc. Here, as the training threshold increases, the influence and contribution given to the model by sample points with large optimization errors increases. Therefore, by attenuating the training threshold as the number of training increases, sample points with large optimization errors can have a greater influence on model parameter adjustment in the early stages of training, which allows for rapid parameter adjustment. The effect of fine parameter adjustment is achieved by making the influence on the model parameter adjustment larger than that of sample points with small optimization errors in the later stages of training.

In some examples, the first loss function L ₁ can be expressed by the following equation (1).

The second loss function _L2 can be expressed by the following equation (2).

Here, f(x) indicates the first sample prediction result, y indicates the first actual sample result, and β indicates the training threshold. At the initial stage of training, β may be 1 or any other suitable value.

For videos in different length sections, when training the model, users can perform normalization processing on the actual result of the viewing time of each sample video, so that when training the model, the predictive effect of the model will be It is not affected by the duration itself. In other words, when training videos in intervals of different lengths, the normalized prediction results are used to obtain model parameters, so that the model learns the user viewing characteristics of videos in intervals of different lengths. be able to.

In some embodiments, the normalization parameter corresponding to a target category may be an interval parameter of a classification variable corresponding to the target category. For example, the classification variable may be the video length, and the section parameter may be the maximum value of the video length section corresponding to the video category. This allows the actual viewing time by the user of the sample video in the video category to be normalized between 0 and 1. In other examples, the interval parameter may be determined as any other value in the interval of video length corresponding to the video category. In such cases, the actual result after normalization may be greater than one. For example, when the end points of the interval corresponding to the classification variable include an infinite term, the value of one of the midpoints in the interval can be selected as the normalization parameter. During the training process, the normalization parameter may be attenuated as the number of training increases. Such damping may be linear damping, exponential damping or any other possible damping.

FIG. 4A is an exemplary diagram illustrating regression loss calculated by a loss function according to an embodiment of the present disclosure. As can be seen from FIG. 4A, regardless of the value of the training threshold β, the loss calculated by the loss function is small when the regression loss error is small (i.e., the difference between the predicted result and the actual result is small). It can be approximated by the result of L2 loss, and when the regression loss error is large (ie, the difference between the predicted result and the actual result is large), it can be approximated by the result of L1 loss.

FIG. 4B is an exemplary diagram illustrating the slope of a loss function according to an embodiment of the present disclosure. As can be seen from Figure 4B, as the value of the training threshold β is decreasing, the slope value of the loss function corresponding to samples with small regression loss errors (i.e., the difference between the predicted result and the actual result is small) is on the rise. That is, as the training progresses, the attenuated training threshold β can improve the influence and contribution given to the model parameters by sample points with small errors.

In some embodiments, after training utilizing the first sample set and the second sample set to obtain the first predictive model and the second predictive model, respectively, the method 300 generates a multi-task training frame. and performing multi-task training on the first predictive model and the second predictive model, thereby obtaining final parameters of the first predictive model and final parameters of the second predictive model. . The generalization ability of the first predictive model and the second predictive model can be further improved by a multi-task training method. Any conventional multi-task training frame can be used to perform multi-task training on the first predictive model and the second predictive model. For example, a multi-gate mixture-of-experts (MMoe) or progressive layered extraction (PLE) model may be used to implement the multi-task training frame used in the embodiments of the present disclosure.

FIG. 5 is an example diagram illustrating a multi-task training frame according to an embodiment of the present disclosure.
As shown in FIG. 5, the multitask frame 500 includes an input 501, an expert network 502, Gate networks 503-A, 503-B, a first model 504, a second model 505, and the first model 504 and the second model 505, respectively. A first output 506 and a second output 507 corresponding to two models 505 may be included.

Here, the first model 504 and the second model 505 may be a first predictive model and a second predictive model obtained by training by the training method described in conjunction with FIG. 3. For example, a first model 504 may be used to predict the amount of time viewed by a user of a video in a first video category, and a second model 505 may be used to predict the amount of time viewed by a user of a video in a second video category. may be used to predict.

What should be understood is that although the first model 504 and the second model 505 are two different models, they are based on predictions about the viewing time by users for videos in the first video category and for videos in the second video category. Since the feature indicators to be learned for the two tasks of predicting the viewing time by the user are the same, the first model is It is possible to learn common points for predicting the prediction task corresponding to the second model and the task of the second model, thereby further improving the generalization ability of the model.

Although only two models are shown in FIG. 5, it should be understood that a multi-task training frame as shown in FIG. 5 can be used for multi-task training of more models.

In some embodiments, input 501 can correspond to at least one predictive feature of the object being processed. For example, assuming that the object to be processed is a video, the input 501 may be the user characteristics of the user who viewed the video, the characteristics of the video item (item), or the like. Here, the video item characteristics may include a video identifier, video content category, number of likes, number of comments, number of collections, historical click rate, and the like. Furthermore, the input 501 may further include a label indicating the classification attribute of the corresponding processed object. Using such a label, it is possible to determine the target category to which the processed target corresponding to the input 501 belongs, and in the subsequent process, it is possible to determine which model (for example, either the first model or the second model) ) to obtain the final output result.

In the multitask training frame as shown in FIG. 5, the output for the kth task (k=1 or 2) may be expressed by equation (3).

Here, n indicates the number of sub-expert networks in the expert network 502, and g ^k (x) _i indicates that the Gate network of the k-th task is the number of sub-expert networks of the i-th sub-expert network when the input is x. Used for the output, E _i (x) indicates the output of the i-th sub-expert network when the input is x. f ^k indicates the model of the kth task. When k=1, f can represent the first model. When k=2, f can represent the second model.

Here, each sub-expert network can be realized as one fully connected network, and each Gate network can be realized as one combination of linear transformation and softmax. The Gate network may be used to map the input x into n dimensions, and then a softmax function may be applied to the result of each dimension to obtain the weights used for each sub-expert network.

As mentioned above, since the input x includes a label indicating the target category, the first model or the second model can be used to determine the final output result based on the label.
During multitask training, the multitask loss function may be expressed by equation (4).

Here, l _i can represent the loss of the i-th task, α _i can represent the weight used for the i-th task, and N represents the total number of tasks.
Here, the weight of each task may be determined based on the importance of each weight. For example, an enlightened algorithm (eg, reinforcement learning or evolutionary learning) can determine the weight of each task.

FIG. 6 is an exemplary block diagram illustrating a data processing apparatus according to an embodiment of the present disclosure.
As shown in FIG. 6, the data processing device 600 may include a processing target determining unit 610, a target category determining unit 620, a prediction model determining unit 630, and a prediction unit 640.

Here, the processing target determination unit 610 may be configured to determine the processing target. The object category determining unit 620 may be configured to determine an object category to which the object to be processed belongs based on a classification attribute of the object to be processed. Prediction model determination unit 630 may be configured to determine a prediction model to be used for the processed object based on the object category. The prediction unit 640 may be configured to obtain a prediction result of the object by processing at least one prediction feature of the object using the prediction model. Here, the prediction result and the classification attribute are variables of the same type.

Steps S202 to S208 as shown in FIG. 2 can be executed using units 610 to 640 as shown in FIG. 6, and detailed description thereof will be omitted here.
FIG. 7 is an example block diagram illustrating an apparatus for training a predictive model according to an embodiment of the present disclosure.

As shown in FIG. 7, the apparatus 700 may include a sample determination unit 710, a classification unit 720, a first predictive model training unit 730, and a second model training unit 740.

Here, the sample determination unit 710 may be configured to determine a sample set including a plurality of sample objects. Classification unit 720 may be configured to determine a first sample subset and a second sample subset in a sample set based on classification attributes of the sample objects. The first predictive model training unit 730 may be configured to utilize the first sample objects in the first sample subset to train the first predictive model. A second model training unit 740 may be configured to utilize a second sample object in a second sample subset to train a second predictive model. Here, the first prediction model and the second prediction model are used to obtain a prediction result of the object to be processed by processing at least one prediction feature of the object to be processed; The prediction result and the classification attribute are the same type of variables.

Steps S302 to S308 as shown in FIG. 3 can be executed using units 710 to 740 as shown in FIG. 7, and detailed description thereof will be omitted here.
In the technical proposal disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of related user personal information shall comply with the provisions of relevant laws and regulations and will not violate public order and morals.

Embodiments of the present disclosure further provide electronic devices, readable storage media, and computer program products.
As shown in FIG. 8, a configuration block diagram of an electronic device 800 that can be used as a server or a client of the present disclosure, which is an example of a hardware device applicable to various aspects of the present disclosure, will be described here. Electronic equipment refers to various forms of digital electronic computers, such as laptop computers, desktop computers, stages, personal digital assistants, servers, blade servers, large format computers, and other suitable computers. Electronic devices may also refer to various forms of mobile devices, such as personal digital processing, mobile phones, smart phones, wearable devices, and other similar computing devices. The components, their interconnections, and their functions depicted herein are exemplary only and do not limit implementation of the present disclosure as described and/or claimed herein.

As shown in FIG. 8, the electronic device 800 includes a computing unit 801, which is a computer program stored in a read-only memory (ROM) 802 or loaded into a random access memory (RAM) 803 from a storage unit 808. may perform various appropriate operations and processes. The RAM 803 may further store various programs and data necessary to operate the electronic device 800. Computing unit 801, ROM 802 and RAM 803 are connected to each other via bus 804. An input/output (I/O) interface 805 is also connected to bus 804.

A plurality of components in electronic device 800 are connected to I/O interface 805 and include input unit 806, output unit 807, storage unit 808, and communication unit 809. The input unit 806 may be any type of device capable of inputting information into the electronic device 800, and the input unit 806 may input numerical or textual information and user settings of the electronic device and/or Key signal inputs may be generated for function control and may include, but are not limited to, a mouse, keyboard, touch screen, trackboard, trackball, operating lever, microphone, and/or remote control. Output unit 807 may be any type of device capable of presenting information and may include, but is not limited to, a display, a speaker, a video/audio output terminal, a vibrator, and/or a printer. Not done. Storage unit 808 may include, but is not limited to, magnetic disks and optical disks. The communication unit 809 allows the electronic device 800 to exchange information/data with other devices via a computer network and/or various telecommunication networks, for example the Internet, and includes modems, network cards, infrared communication devices. , wireless communication transceivers, and/or chipsets, such as, but not limited to, Bluetooth™ devices, 802.11 devices, WiFi devices, WiMax devices, cellular communication devices, and/or the like. Not done.

Computing unit 801 may be a variety of general purpose and/or special purpose processing components with processing and computing capabilities. Some examples of computational units 801 include central processing units (CPUs), graphics processing units (GPUs), various specialized artificial intelligence (AI) computational chips, various computational units that execute machine learning model algorithms, digital signals, etc. It may include, but is not limited to, a processor (DSP), and any suitable processor, controller, microcontroller, etc. Computing unit 801 executes the methods and processes described above, such as methods 200 and 300. For example, in some embodiments, methods 200, 300 may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 808. In some embodiments, some or all of the computer program may be loaded and/or installed on electronic device 800 via ROM 802 and/or communication unit 809. Once the computer program is loaded into the RAM 803 and executed by the calculation unit 801, one or more steps of the method 200, 300 described above can be performed. Alternatively, in other embodiments, computing unit 801 is configured to perform methods 200, 300 in any other suitable manner (eg, using firmware).

Various embodiments of the systems and techniques described herein above may be implemented as digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products, etc. (ASSP), system on a chip (SOC), complex programmable logic device (CPLD), software/hardware, firmware, software, and/or combinations thereof. These various embodiments may be implemented in one or more computer programs that may be executed and/or interpreted on a programmable system including at least one programmable processor. The programmable processor may be a special purpose or general purpose programmable processor and receives data and instructions from a storage system, at least one input device, and at least one output device, and transmits data and instructions to the storage system, the at least one input device, and the at least one output device. , may be transmitted to the at least one output device.

Program code implementing the methods of this disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing device such that the program codes, when executed by the processor or controller, follow the flowcharts and/or block diagrams set forth in the flowcharts and/or block diagrams. Perform functions/operations. The program code may be executed entirely on a machine, partially executed on a machine, partially executed on a machine and partially executed on a remote machine as an independent software package, or may be executed entirely on a remote machine. Or it can be executed on the server.

In the context of this disclosure, a machine-readable medium may be a tangible medium, comprising or storing a program for use in or coupled to an instruction execution system, apparatus or device. You may do so. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus or devices, or any suitable combination of the above. More specific examples of machine-readable storage media include electrical connection through one or more leads, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable memory, etc. including dedicated memory (EPROM or flash memory), fiber optics, portable compact disc read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the above.

To provide user interaction, a computer may be implemented with the systems and techniques described herein and may include a display device (e.g., a cathode ray tube (CRT) or LCD) for displaying information to the user. (liquid crystal display) surveillance monitor), and a keyboard and pointing device (eg, a mouse or trackball) through which a user may provide input to the computer. Other types of devices may also be for providing interaction with a user. For example, the feedback provided to the user may be any form of sensory feedback (e.g., visual, auditory, or haptic feedback), and any form of feedback provided to the user (including audio, audio, or tactile input) may receive input.

The systems and technologies described here may include computing systems that include backstage components (e.g., as data servers), middleware components (e.g., application servers), and front-end components (e.g., as graphical a user computer having a user interface or web browser that allows a user to interact with those systems or technology embodiments through its graphical user interface or web browser; or backstage or middleware components thereof; A computing system comprising any combination of front end components may be implemented. The components of the system may be interconnected by any form or medium of digital data communication (eg, a communication network). Examples of communication networks include local networks (LANs), wide area networks (WANs), and the Internet.

A computer system may include a client side and a server. The client side and server are typically far apart from each other and typically interact via a communications network. A relationship between a client side and a server is created by running computer programs that have a mutual client side-server relationship on corresponding computers. The server may be a cloud server, a distributed system server, or a server combined with a blockchain.

It should be understood that steps may be re-ranked, added to, or deleted using the various forms of flow described above. For example, each step described in this disclosure may be performed in parallel, sequentially, or in a different order, and the technical solutions disclosed in this disclosure may be performed as desired. The main text is not limited to this, as long as the results can be achieved.

Although embodiments or examples of the present disclosure have been described with reference to the drawings, the methods, systems, and apparatus described above are merely exemplary embodiments or examples, and the scope of the invention is limited by these embodiments or examples. It is to be understood that the invention is not limited, but is limited only by the scope of the following claims and their equivalents. Various elements of the embodiments or examples may be omitted or replaced by equivalent elements thereof. Note that each step may be performed in a different order than described in this disclosure. Furthermore, various elements of the embodiments or examples may be combined in various ways. Importantly, as technology evolves, many of the elements described herein can be replaced with equivalent elements that appear after this disclosure.

Claims (23)

A data processing method, comprising:
Deciding on the target to be processed;
determining a target category to which the processed object belongs based on a classification attribute of the processed object;
determining a prediction model to be used for the target to be processed based on the target category;
obtaining a prediction result of the object to be processed by processing at least one prediction feature of the object to be processed using the prediction model, wherein the prediction result and the classification attribute have the same type of variable; Data processing methods, including. 2. The data processing method according to claim 1, wherein the processing target is a video, the classification attribute is video length, and the prediction result is a predicted time for a user to view the video. Determining the target category to which the processed object belongs based on the classification attributes of the processed object is as follows:
determining a video length section to which the video belongs based on the video length of the video;
3. The data processing method according to claim 2, further comprising determining an indicator of the video length section as the target category. Determining a prediction model to be used for the target to be processed based on the target category includes:
In response to determining that the target category to which the processed target belongs is an interval of a first video length, determining to use a first prediction model for the processed target;
determining that a second predictive model is to be used for the processing target in response to determining that the target category to which the processing target belongs is a second video length section; The data processing method according to claim 3, comprising: 3. The data processing method according to claim 2, wherein the at least one predictive feature includes a user feature of a user who viewed the video and a feature of the video item. Obtaining a prediction result of the object to be processed by processing at least one prediction feature of the object to be processed using the prediction model,
obtaining a normalized prediction result of the target to be processed by processing at least one predictive feature of the target to be processed using the prediction model;
The data processing method according to claim 1, comprising: obtaining the prediction result by processing the normalized prediction result using a normalization parameter used for the target category. 2. The data processing method according to claim 1, wherein the processing target is weather history data, the classification attribute is a weather parameter included in the weather history data, and the prediction result is a predicted weather parameter. A method for training a predictive model, the method comprising:
determining a sample set that includes multiple sample subjects;
determining a first sample subset and a second sample subset in a sample set based on classification attributes of the sample objects;
training a first predictive model utilizing a first sample subject in a first sample subset;
training a second predictive model utilizing a second sample subject in a second sample subset;
Here, the first prediction model and the second prediction model are used to obtain a prediction result of the object to be processed by processing at least one prediction feature of the object to be processed, and the first prediction model and the second prediction model are used to obtain a prediction result of the object to be processed, and A method for training a predictive model, wherein the classification attributes are homogeneous variables. 9. The method of claim 8, wherein the sample object and the processed object are videos, and the classification attribute is video length. The first sample subset includes at least one first sample object belonging to a first video length interval, and the second sample subset includes at least one second sample object belonging to a second video length interval. 10. The method of claim 9, comprising a sample subject of . 11. The method of claim 10, wherein the number of sample objects in the first sample subset is the same as the number of sample objects in the second sample subset. Training a first predictive model utilizing the sample subjects in the first sample subset comprises:
determining a first current parameter of the first predictive model;
obtaining a first sample prediction result for the first sample object by processing at least one first sample feature of the first sample object using the first current parameter;
determining a first actual sample result of the first sample;
A method according to any one of claims 8 to 11, comprising adjusting the first current parameter based on the first sample prediction result and the first actual sample result. adjusting the first current parameter based on the first sample prediction result and the first actual sample result;
determining a first loss based on the first sample prediction result and the first actual sample result;
13. The method of claim 12, comprising: adjusting the first current parameter based on the first loss. Determining a first loss based on the first sample prediction result and the first actual sample result comprises:
determining the first loss using a first loss function when a difference between the first sample prediction result and the first actual sample result is less than a training threshold;
determining the first loss using a second loss function when a difference between the first sample prediction result and the first actual sample result is greater than or equal to a training threshold; 14. The method according to claim 13. 15. The method of claim 14, wherein the training threshold decays with increasing training frequency. a loss determined by the first loss function and a loss determined by the second loss function when the difference between the first sample prediction result and the first actual sample result is equal to the training threshold; 15. The method of claim 14, wherein are the same. The first loss function is expressed by the following formula (1),
The second loss function is expressed by the following formula (2),
15. The method of claim 14, where f(x) denotes a first sample prediction result, y denotes a first actual sample result, and β denotes a training threshold. By performing multitask training on the first predictive model and the second predictive model using a multitask training frame, the final parameters of the first predictive model and the final parameters of the second predictive model can be determined. 9. The method of claim 8, further comprising obtaining. A data processing device,
a target determining unit configured to determine a target;
a target category determination unit configured to determine a target category to which the processed object belongs based on a classification attribute of the processed object;
a predictive model determining unit configured to determine a predictive model to be used for the processed object based on the target category;
A prediction unit configured to obtain a prediction result of the processed object by processing at least one prediction feature of the processed object using the prediction model, wherein the prediction result is and the classification attributes are variables of the same type. An apparatus for training a predictive model, the apparatus comprising:
a sample determining unit configured to determine a sample set including a plurality of sample objects;
a classification unit configured to determine a first sample subset and a second sample subset in a sample set based on classification attributes of the sample objects;
a first predictive model training unit configured to utilize a first sample object in a first sample subset to train a first predictive model;
a second predictive model training unit configured to utilize a second sample object in the second sample subset to train a second predictive model;
Here, the first prediction model and the second prediction model are used to obtain a prediction result of the object to be processed by processing at least one prediction feature of the object to be processed, and the first prediction model and the second prediction model are used to obtain a prediction result of the object to be processed, and An apparatus for training a predictive model, wherein the classification attributes are homogeneous variables. An electronic device,
at least one processor;
a memory communicatively coupled to the at least one processor, wherein:
The memory stores instructions executable by the at least one processor, and when executed by the at least one processor, the instructions cause the at least one processor to execute any one of claims 1 to 18. An electronic device capable of carrying out the method set forth in item 1. A non-transitory computer-readable storage medium having stored thereon computer instructions for causing a computer to perform a method according to any one of claims 1 to 18. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 18.