WO2023011093A1

WO2023011093A1 - Task model training method and apparatus, and electronic device and storage medium

Info

Publication number: WO2023011093A1
Application number: PCT/CN2022/104081
Authority: WO
Inventors: 杨德将
Original assignee: 北京百度网讯科技有限公司
Priority date: 2021-08-04
Filing date: 2022-07-06
Publication date: 2023-02-09
Also published as: CN113807391A

Abstract

The present disclosure relates to the technical field of artificial intelligence such as machine learning and natural language processing. Provided are a task model training method and apparatus, and an electronic device and a storage medium. The specific implementation solution involves: acquiring the similarities between training samples in a training set and a test set; configuring the weights of corresponding training samples according to the similarities between the training samples in the training set and the test set; and training a task model according to the training samples in the training set and the weights of the corresponding training samples. By means of the present disclosure, the accuracy of a trained task model can be effectively improved.

Description

Task model training method, device, electronic equipment and storage medium

This application claims the priority of the Chinese patent application with the application date of August 4, 2021 and the application number 202110891285.2 titled "task model training method, device, electronic equipment and storage medium".

technical field

The present disclosure relates to the field of computer technology, in particular to the field of artificial intelligence technology such as machine learning and natural language processing, and in particular to a task model training method, device, electronic equipment, and storage medium.

Background technique

With the development of artificial intelligence (AI) technology, AI-based neural network models can be applied in various scenarios in various fields, and can achieve certain tasks, which can also be called task models.

Before the existing task model is used, it needs to be trained with the training set and tested with the test set, and it can be put into use only if it meets the usage requirements. Usually, the training set and test set come from time-sliced historical data. Compared with the test set, the training set can use historical data with a longer time. In the scenarios where some task models are applied, it takes 1 to 2 years or even longer to determine the true label of a sample. When the market environment changes, the access strategy is adjusted, etc., due to the large time span, the distribution of samples will shift greatly with the passage of time. At this time, the training set and test set divided by time Inconsistent sample distribution, resulting in a much different effect of the task model on the test set than on the training set.

Contents of the invention

The disclosure provides a task model training method, device, electronic equipment and storage medium.

According to an aspect of the present disclosure, a method for training a task model is provided, wherein the method includes:

Obtain the similarity between each training sample in the training set and the test set;

According to the similarity between each of the training samples in the training set and the test set, configure the weight of the corresponding training samples;

The task model is trained according to each of the training samples in the training set and the corresponding weight of each of the training samples.

According to another aspect of the present disclosure, a task model training device is provided, wherein the device includes:

An acquisition module, configured to acquire the similarity between each training sample in the training set and the test set;

A configuration module, configured to configure the weight of the corresponding training samples according to the similarity between each of the training samples in the training set and the test set;

The training module is configured to train the task model according to the training samples in the training set and the corresponding weights of the training samples.

According to still another aspect of the present disclosure, an electronic device is provided, including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor, the instructions are executed by the at least one processor, so that the at least one processor can perform the above aspects and any possible implementation way of way.

According to yet another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, the computer instructions are used to make the computer execute the method of the above aspect and any possible implementation manner .

According to yet another aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the above aspect and the method of any possible implementation manner are implemented.

According to the technology of the present disclosure, a more efficient task model training scheme can be provided, and the accuracy of the trained task model can be further effectively improved.

It should be understood that what is described in this section is not intended to identify key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will be readily understood through the following description.

Description of drawings

The accompanying drawings are used to better understand the present solution, and do not constitute a limitation to the present disclosure. in:

FIG. 1 is a schematic diagram according to a first embodiment of the present disclosure;

FIG. 2 is a schematic diagram according to a second embodiment of the present disclosure;

Fig. 3 is a schematic diagram according to a third embodiment of the present disclosure;

FIG. 4 is a schematic diagram according to a fourth embodiment of the present disclosure;

Fig. 5 is a block diagram of an electronic device used to implement the task model training method of the embodiment of the present disclosure.

Detailed ways

Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and they should be regarded as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

Apparently, the described embodiments are some of the embodiments of the present disclosure, but not all of them. Based on the embodiments in the present disclosure, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present disclosure.

It should be noted that the terminal devices involved in the embodiments of the present disclosure may include but not limited to mobile phones, personal digital assistants (Personal Digital Assistant, PDA), wireless handheld devices, tablet computers (Tablet Computer) and other smart devices; Including but not limited to personal computers, televisions and other devices with display functions.

In addition, the term "and/or" in this article is only an association relationship describing associated objects, which means that there may be three relationships, for example, A and/or B, which may mean: A exists alone, A and B exist at the same time, There are three cases of B alone. In addition, the character "/" in this article generally indicates that the contextual objects are an "or" relationship.

Fig. 1 is a schematic diagram according to the first embodiment of the present disclosure; as shown in Fig. 1 , this embodiment provides a method for training a task model, which may specifically include the following steps:

S101. Obtain the similarity between each training sample in the training set and the test set;

S102. According to the similarity between each training sample in the training set and the test set, configure the weight of the corresponding training sample;

S103. Train the task model according to each training sample in the training set and the corresponding weight of each training sample.

The task model training method of this embodiment is executed by a task model training device, which may be an electronic entity, or may also be an application using software integration.

The task model of this embodiment can be applied to various scenarios in various fields, such as predicting whether a vehicle will be in danger in the insurance field or predicting whether a user will purchase a certain insurance product, predicting whether a user will be in danger in the financial field Predict the probability of a user taking a designated vehicle, etc. In short, the task model of this embodiment can be mainly used to realize the prediction of binary classification tasks in various scenarios in various fields.

Since the training set and test set are usually divided according to time, the test set is closer to the current time than the training set. However, as time goes by, the characteristics of the samples in the training set and the test set are not consistent, resulting in the task model trained with the training set, the test effect on the test set is not good, and the problem of overfitting occurs. That is, the background of the training method of the task model in this embodiment is the problem that the distribution of the training set and the test set are inconsistent.

In order to overcome the above problems, in this embodiment, the similarity between each training sample in the training set and the test set is obtained first, so that it is possible to distinguish which samples in the training samples are relatively similar to the test set, and which samples are quite different from the test set. In order to effectively train the task model based on the similarity between each training sample and the test set, the weight of the corresponding training sample can be configured according to the similarity between each training sample in the training set and the test set; And the corresponding weights of each training sample, the task model is trained preferentially, which can make the task model more inclined to learn the training samples with high weight, that is, with a large similarity with the test set. With the training method of this embodiment, even if the samples in the training set and the test set are unevenly distributed, the trained task model has a good test effect on the test set, which can well solve the problem of sample distribution deviation.

Wherein, according to the similarity between each training sample in the training set and the test set, when configuring the weight of the corresponding training sample, the similarity between the training sample and the test set can be directly configured as the weight of the corresponding training sample. Or it can also be based on the similarity between the training sample and the test set, combined with other constants or mathematical calculation methods, to be the weight of the corresponding training sample. For example, if the similarity between the training sample and the test set is greater than or equal to the preset similarity threshold, the similarity can be multiplied by a constant greater than 1 as the weight of the corresponding training sample. When the similarity between the training sample and the test set is less than the preset similarity threshold, the similarity can be multiplied by a constant less than 1 as the weight of the corresponding training sample, or the square of the similarity can also be taken at this time , as the weight of the corresponding training sample. In practical applications, based on the similarity between each training sample in the training set and the test set, other methods can be used to configure the weights of the corresponding training samples. In short, the training samples whose similarity is greater than the preset similarity threshold have higher weights, so that the corresponding training samples are more involved in the training of the task model; and the training samples whose similarity is smaller than the preset similarity threshold have Lower weight, so that the corresponding training samples are reduced to participate in the training of the task model.

The training method of the task model of the present embodiment obtains the similarity between each training sample in the training set and the test set; configures the weight of the corresponding training sample according to the similarity between each training sample in the training set and the test set; Each training sample and the corresponding weight of each training sample, the task model is tended to be trained, which can make the task model more inclined to learn the training samples with high weight, that is, the training sample with a large similarity with the test set, so that the sample can be solved. The problem of distribution shift can effectively improve the accuracy of the trained task model.

Fig. 2 is a schematic diagram according to the second embodiment of the present disclosure; as shown in Fig. 1 , this embodiment provides a method for training a task model, which may specifically include the following steps:

S201. Based on the training set and the test set, train a sample classifier;

For example, during specific implementation, this step may specifically include the following steps:

(1) reconfiguring the first label for all training samples in the training set;

(2) reconfigure the second label for all test samples in the test set, the second label is different from the first label;

During specific implementation, the original labels of the training samples in the training set and the test samples in the testing set can be removed, and a first label such as 0 is configured for all training samples in the training set to identify that these training samples are all samples in the training set. Configure a second label such as 1 for all test samples in the test set to identify that these samples are all samples in the test set.

(3) Merge the training set and the test set to obtain the combined sample set;

Specifically, the relabeled training set and sample set in the above step (1) and step (2) are combined to obtain a combined sample set. At this time, the label of each sample in the combined sample set is 0 or 1, indicating that the sample comes from the original training set or test set.

(4) Obtain a new training set and a new test set based on the merged sample set;

Specifically, the combined sample set can be randomly split to obtain a new training set and a new test set.

(5) Construct a sample classifier based on the new training set and the new test set, so that the sample classifier can distinguish the samples in the training set and the test set.

Specifically, the new training set is used to train the sample classifier, so that the sample classifier learns to identify the samples in the training set and the samples in the test set, and after the new test set test, the trained sample classifier has good performance and meets the modeling requirements of the model.

Specifically, during the training process, a training sample is randomly selected from the new training set and input to the sample classifier. The sample classifier predicts whether the training sample comes from the training set or the test set, and then based on the label of the training sample, the label identifies The real source of the training samples, that is, from the training set or the test set, is to check whether the prediction of the sample classifier is correct. If not, adjust the parameters of the sample classifier so that the sample classifier is adjusted in the direction of correct prediction. Use several training samples in the new training set to continuously train the sample classifier, so that the sample classifier can always accurately predict the source of the sample in the continuous preset number of rounds of training, or until the number of training times reaches the maximum threshold , the training is over, at this time, the parameters of the sample classifier are determined, and then the sample classifier is determined.

The sample classifier in this embodiment may be a binary classification model such as random forest, Xgboost, or the like.

S202. Based on the trained sample classifier, detect whether there is a sample distribution offset between the training set and the test set; if yes, perform step S203; otherwise, do not perform the operation temporarily, and end.

For example, in this embodiment, a trained sample classifier is used to detect whether there is a sample distribution offset between the training set and the test set, and the accuracy of detecting whether there is a sample distribution offset between the training set and the test set is very high. Or in practical applications, other methods can also be used to determine that there is a sample distribution offset between the training set and the test set. For example, information of sample distribution shift can be obtained by receiving externally input training set and test set.

For example, based on the trained sample classifier, detecting whether there is a sample distribution shift between the training set and the test set may specifically include the following steps:

(a) Calculate the area under the curve (AUC) index of the trained sample classifier on the new test set;

Among them, the AUC indicator specifically refers to the area indicator under the receiver operating characteristic (Receiver Operating Characteristic; ROC) curve. The AUC indicator is a model evaluation indicator in the field of machine learning.

(b) Detect whether the AUC index is greater than the first preset threshold and less than or equal to the second preset threshold; if so, perform step (c); otherwise, if greater than or equal to the third preset threshold and less than or equal to the first preset threshold value; perform step (d);

For example, in this embodiment, the first preset threshold can be set to 0.6 or other values close to 0.6, such as 0.59, 0.61 and so on. The second preset threshold may be 0.9 or other values close to 0.9, such as 0.89, 0.91 and so on. The third preset threshold may be 0.5 or other values close to 0.5, such as 0.49, 0.51 and so on.

(c) Determine that there is a sample distribution shift between the training set and the test set. This method is very accurate in detecting sample distribution shifts.

(d) It is determined that there is no sample distribution shift between the training set and the test set, that is, the sample classifier cannot effectively distinguish the samples in the training set and the test set at this time, and it is not necessary to perform biased training according to the method of this embodiment.

S203. Use a sample classifier to score each training sample in the training set to identify the similarity between the training sample and the test set;

Specifically, each training sample in the training set is input into the sample classifier, and the sample classifier may output a value greater than 0 and less than 1 to identify whether the input training sample belongs to the test set or the training set. When the sample classifier is trained, the label of the training set is 0, and the label of the test set is 1, so the score output by the sample classifier can also be regarded as the probability that the training sample belongs to the test set, which can identify the training sample and the test set set similarity. The closer the value is to 0, the lower the similarity between the training sample and the test set, and the closer the value is to 1, the higher the similarity between the training sample and the test set. Moreover, the similarity between the training sample and the test set obtained in this way is very accurate.

S204. Configure the similarity between each training sample in the training set and the test set as the weight of the corresponding training sample;

In this step, take directly configuring the similarity between each training sample in the training set and the test set as the weight of the corresponding training sample as an example. The similarity between the training sample and the test set, configure the weight of the corresponding training sample.

S205. Train the task model according to each training sample in the training set and the corresponding weight of each training sample.

For the training method of this embodiment, reference can be made to model weighted samples such as Xgboost and Lightgbm. When constructing the training set, the weight of the training sample obtained above is passed in as the input parameter weight.

For example, the weighted training set constructed by Xgboost can be expressed as:

dtrain=xgb.DMatrix(data=X_train, label=y_train, weight=weight)

For example, the weighted training set constructed by Lightgbm can be expressed as:

dtrain=lgb.Dataset(data=X_train, label=y_train, weight=weight)

In this embodiment, according to each training sample in the training set and the weight of each corresponding training sample, the task model is trained, which may specifically include the following methods:

The first way:

Based on the weight of each training sample, the training samples participating in the training are selected from the training set; based on the selected training samples, the task model is trained.

In this way, during training, training samples with high weights are more likely to be selected to participate in training, which can make the task model more inclined to learn training samples with high weights, that is, training samples that are more similar to the test set. It can overcome the problem of training set and sample set distribution offset.

Based on this method, all training samples whose weight is greater than the preset weight threshold can be directly screened from the training set to form a training subset, and then the task model is trained using the training subset. Since the training samples in the training subset are all samples with high weight and high similarity with the test set, the problem of distribution offset between the training set and the sample set can also be overcome.

The second way:

Randomly select training samples to participate in the training from the training set; based on the selected training samples and the weights of the training samples, the task model is trained.

In this way, the probability of each training sample being selected to participate in training is the same, but in the training process, the weight of each training sample should still be referred to. For example, the corresponding loss function can be calculated according to the weight of each training sample, so that the weight The loss function of a high sample is relatively large, and when the parameters of the task model can be adjusted based on the loss function, the adjustment range is larger, which can make the task model more biased towards training samples with high learning weights, that is, it is biased towards learning and testing. The training samples can overcome the problem of distribution offset between training set and test set.

The training method of the task model of this embodiment, by adopting the above-mentioned technical scheme, can make the task model more inclined to learn samples with higher similarity with the test set, so as to overcome the problem of the distribution offset between the training set and the sample set, and avoid training The obtained task model has the problem of overfitting, which can effectively improve the accuracy of the trained task model.

Fig. 3 is a schematic diagram according to a third embodiment of the present disclosure; as shown in Fig. 3 , this embodiment provides a task model training device 300, including:

Obtaining module 301, used to obtain the similarity between each training sample in the training set and the test set;

The configuration module 302 is used to configure the weight of the corresponding training samples according to the similarity between each training sample in the training set and the test set;

The training module 303 is configured to train the task model according to each training sample in the training set and the corresponding weight of each training sample.

The task model training device 300 of this embodiment uses the above-mentioned modules to realize the realization principle and technical effect of task model training, which is the same as the implementation of the above-mentioned related method embodiments. For details, please refer to the records of the above-mentioned related method embodiments, here No longer.

Fig. 4 is a schematic diagram according to the fourth embodiment of the present disclosure; as shown in Fig. 4, the task model training device 300 of this embodiment is further described in more detail on the basis of the technical solution of the embodiment described in Fig. 3 above The technical scheme of the present application.

As shown in Figure 4, in the training device 300 of the task model of the present embodiment, also include:

The detection module 304 is configured to detect and determine that there is a sample distribution deviation between the training set and the test set.

Further optionally, as shown in FIG. 4, in the task model training device 300 of this embodiment, the acquisition module 301 includes:

A training unit 3011, configured to train a sample classifier based on a training set and a test set;

The scoring unit 3012 is configured to use a sample classifier to score each training sample in the training set, so as to identify the similarity between the training sample and the test set.

Further optionally, the training unit 3011 is used for:

Configure the first label for all training samples in the training set;

Configure a second label for all test samples in the test set, the second label is different from the first label;

Combine the training set and the test set to obtain the combined sample set;

Obtain a new training set and a new test set based on the combined sample set;

Based on the new training set and the new test set, a sample classifier is constructed, so that the sample classifier can distinguish the samples in the training set and the test set.

Further optionally, the detection module 304 is configured to:

Based on the trained sample classifier, detect whether there is a sample distribution shift between the training set and the test set.

Further optionally, the detection module 304 is configured to:

Calculate the area under the curve of the trained sample classifier on the new test set;

Whether the area index under the detection curve is greater than the first preset threshold and less than or equal to the second preset threshold;

If so, it is determined that there is a sample distribution shift between the training set and the test set.

Further optionally, the training module 303 is used for:

Based on the weight of each training sample, select training samples to participate in training from the training set;

Based on the selected training samples, the task model is trained.

Further optionally, the training module 303 is used for:

Randomly select training samples to participate in training from the training set;

Based on the selected training samples and the weights of the training samples, the task model is trained.

In the technical solution of the present disclosure, the acquisition, storage and application of the user's personal information involved are in compliance with relevant laws and regulations, and do not violate public order and good customs.

According to the embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

FIG. 5 shows a schematic block diagram of an example electronic device 500 that may be used to implement embodiments of the present disclosure. Electronic device is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are by way of example only, and are not intended to limit implementations of the disclosure described and/or claimed herein.

As shown in FIG. 5 , the device 500 includes a computing unit 501 that can execute according to a computer program stored in a read-only memory (ROM) 502 or loaded from a storage unit 508 into a random-access memory (RAM) 503. Various appropriate actions and treatments. In the RAM 503, various programs and data necessary for the operation of the device 500 can also be stored. The computing unit 501, ROM 502, and RAM 503 are connected to each other through a bus 504. An input/output (I/O) interface 505 is also connected to the bus 504 .

Multiple components in the device 500 are connected to the I/O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a magnetic disk, an optical disk, etc. ; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 509 allows the device 500 to exchange information/data with other devices over a computer network such as the Internet and/or various telecommunication networks.

The computing unit 501 may be various general-purpose and/or special-purpose processing components having processing and computing capabilities. Some examples of computing units 501 include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processing processor (DSP), and any suitable processor, controller, microcontroller, etc. The calculation unit 501 executes various methods and processes described above, such as a task model training method. For example, in some embodiments, the task model training method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 508 . In some embodiments, part or all of the computer program may be loaded and/or installed on the device 500 via the ROM 502 and/or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the task model training method described above can be performed. Alternatively, in other embodiments, the computing unit 501 may be configured in any other appropriate way (for example, by means of firmware) to execute the task model training method.

Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips Implemented in a system of systems (SOC), load programmable logic device (CPLD), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include being implemented in one or more computer programs executable and/or interpreted on a programmable system including at least one programmable processor, the programmable processor Can be special-purpose or general-purpose programmable processor, can receive data and instruction from storage system, at least one input device, and at least one output device, and transmit data and instruction to this storage system, this at least one input device, and this at least one output device an output device.

Program codes for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special purpose computer, or other programmable data processing devices, so that the program codes, when executed by the processor or controller, make the functions/functions specified in the flow diagrams and/or block diagrams Action is implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer discs, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.

To provide for interaction with the user, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user. ); and a keyboard and pointing device (eg, a mouse or a trackball) through which a user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and can be in any form (including Acoustic input, speech input or, tactile input) to receive input from the user.

The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., as a a user computer having a graphical user interface or web browser through which a user can interact with embodiments of the systems and techniques described herein), or including such backend components, middleware components, Or any combination of front-end components in a computing system. The components of the system can be interconnected by any form or medium of digital data communication, eg, a communication network. Examples of communication networks include: Local Area Network (LAN), Wide Area Network (WAN) and the Internet.

A computer system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

It should be understood that steps may be reordered, added or deleted using the various forms of flow shown above. For example, each step described in the present disclosure may be executed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution disclosed in the present disclosure can be achieved, no limitation is imposed herein.

The specific implementation manners described above do not limit the protection scope of the present disclosure. It should be apparent to those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made depending on design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

A training method for a task model, wherein the method includes:

Obtain the similarity between each training sample in the training set and the test set;

According to the similarity between each of the training samples in the training set and the test set, configure the weight of the corresponding training samples;

The task model is trained according to each of the training samples in the training set and the corresponding weight of each of the training samples.
The method according to claim 1, wherein, before obtaining the similarity between each training sample in the training set and the test set, it also includes:

Detecting and determining that there is a sample distribution shift between the training set and the test set.
The method according to claim 2, wherein obtaining the degree of similarity between each training sample in the training set and the test set comprises:

training a sample classifier based on the training set and the test set;

The sample classifier is used to score each of the training samples in the training set to identify the similarity between the training samples and the test set.
The method according to claim 3, wherein, based on the training set and the test set, training a sample classifier comprises:

configuring a first label for all training samples in the training set;

configuring a second label for all test samples in the test set, the second label being different from the first label;

Merging the training set and the test set to obtain a combined sample set;

Obtain a new training set and a new test set based on the combined sample set;

Based on the new training set and the new test set, construct the sample classifier, so that the sample classifier can distinguish the samples in the training set and the test set.
The method according to claim 4, wherein detecting and determining that there is a sample distribution deviation between the training set and the test set comprises:

Based on the trained sample classifier, it is detected whether there is a sample distribution deviation between the training set and the test set.
The method according to claim 5, based on the trained sample classifier, detecting whether there is a sample distribution deviation in the training set and the test set, comprising:

Calculate the area under the curve ACU index of the trained sample classifier for the new test set;

Detecting whether the ACU index is greater than a first preset threshold and less than or equal to a second preset threshold;

If yes, determine that there is a sample distribution deviation between the training set and the test set.
The method according to any one of claims 1-6, wherein, according to each of the training samples in the training set and the corresponding weights of each of the training samples, training the task model includes:

selecting training samples to participate in training from the training set based on the weight of each of the training samples;

The task model is trained based on the selected training samples.
The method according to any one of claims 1-6, wherein, according to each of the training samples in the training set and the corresponding weights of each of the training samples, training the task model includes:

Randomly select training samples to participate in training from the training set;

The task model is trained based on the selected training samples and the weights of the training samples.
A training device for a task model, wherein the device includes:

An acquisition module, configured to acquire the similarity between each training sample in the training set and the test set;

A configuration module, configured to configure the weight of the corresponding training samples according to the similarity between each of the training samples in the training set and the test set;

The training module is configured to train the task model according to the training samples in the training set and the corresponding weights of the training samples.
The device according to claim 9, wherein the device further comprises:

A detection module, configured to detect and determine that there is a sample distribution deviation between the training set and the test set.
The device according to claim 10, wherein the acquiring module comprises:

a training unit, configured to train a sample classifier based on the training set and the test set;

The scoring unit is configured to use the sample classifier to score each of the training samples in the training set, so as to identify the similarity between the training samples and the test set.
The device according to claim 11, wherein the training unit is configured to:

configuring a first label for all training samples in the training set;

configuring a second label for all test samples in the test set, the second label being different from the first label;

Merging the training set and the test set to obtain a combined sample set;

Obtain a new training set and a new test set based on the combined sample set;

Based on the new training set and the new test set, construct the sample classifier, so that the sample classifier can distinguish the samples in the training set and the test set.
The device according to claim 12, wherein the detection module is configured to:

Based on the trained sample classifier, it is detected whether there is a sample distribution deviation between the training set and the test set.
The device according to claim 13, the detection module is configured to:

Calculate the area under the curve ACU index of the trained sample classifier for the new test set;

Detecting whether the ACU index is greater than a first preset threshold and less than or equal to a second preset threshold;

If yes, determine that there is a sample distribution deviation between the training set and the test set.
The device according to any one of claims 9-14, wherein the training module is configured to:

selecting training samples to participate in training from the training set based on the weight of each of the training samples;

The task model is trained based on the selected training samples.
The device according to any one of claims 9-14, wherein the training module is configured to:

Randomly select training samples to participate in training from the training set;

The task model is trained based on the selected training samples and the weights of the training samples.
An electronic device comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor, the instructions are executed by the at least one processor, so that the at least one processor can perform any one of claims 1-8. Methods.
A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method according to any one of claims 1-8.
A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-8.