CN115238838A - Model updating method, device, equipment and medium for continuous learning - Google Patents

Model updating method, device, equipment and medium for continuous learning Download PDF

Info

Publication number
CN115238838A
CN115238838A CN202211164211.XA CN202211164211A CN115238838A CN 115238838 A CN115238838 A CN 115238838A CN 202211164211 A CN202211164211 A CN 202211164211A CN 115238838 A CN115238838 A CN 115238838A
Authority
CN
China
Prior art keywords
model
sample
loss
updated
historical
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211164211.XA
Other languages
Chinese (zh)
Inventor
陈永红
谢翀
兰鹏
罗伟杰
陈柯树
赵豫陕
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Qianhai Huanrong Lianyi Information Technology Service Co Ltd
Original Assignee
Shenzhen Qianhai Huanrong Lianyi Information Technology Service Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Qianhai Huanrong Lianyi Information Technology Service Co Ltd filed Critical Shenzhen Qianhai Huanrong Lianyi Information Technology Service Co Ltd
Priority to CN202211164211.XA priority Critical patent/CN115238838A/en
Publication of CN115238838A publication Critical patent/CN115238838A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The application relates to a model updating method, a model updating device, model updating equipment and a model updating medium for continuous learning. The method includes the steps of obtaining a historical model of a model to be updated, extracting historical extractor parameters and historical classifier parameters, initializing the model to be updated by using the historical extractor parameters, inputting an updated sample into the historical model and the initialized model to be updated, outputting a first prediction label and a second prediction label, constructing characteristic learning loss, classification learning loss and sample characteristic loss by using a real label, the first prediction label and the second prediction label of the updated sample, calculating total training loss of the model to be updated, updating the initialized model to be updated according to the total training loss, repeatedly calculating until the total training loss is converged to obtain the updated model to be updated, and achieving updating by using only the historical model under the condition that the historical sample is not needed, so that the historical sample is not needed to be maintained, the model can be quickly updated based on the historical model, and updating response efficiency of the model is improved.

Description

Model updating method, device, equipment and medium for continuous learning
Technical Field
The application is applicable to the technical field of artificial intelligence, and particularly relates to a model updating method, device, equipment and medium for continuous learning.
Background
Currently, in a scientific and technological scenario of artificial intelligence, different types of models are usually included in a common artificial intelligence service. However, these models do not meet the needs of all customers, and therefore new needs bring new samples and new identification requirements. New requirements often require new models to identify new classes and not to catastrophically forget old classes. The traditional method retrains the model to meet the requirements of customers through superposition of historical samples and new samples, and the method brings great challenges to sample maintenance and response speed of model iteration and new requirements. Aiming at new requirements of customers, the traditional method mainly mixes old data and new data, labels new and old categories on the mixed data at the same time, and then retrains a new model on the newly labeled data; or to train a new model for a new class on new data only. The conventional method has several problems:
1. historical samples need to be maintained continuously, and after new category extraction requirements are extracted, expanded categories need to be labeled on new and old categories at the same time, so that huge pressure is brought to labeling, and the maintenance of the historical samples becomes too bloated;
2. the historical model cannot be effectively utilized, the model is retrained every time a new requirement comes, and along with the accumulation of samples, the training of the new model becomes very difficult and cannot quickly respond to the requirement of a client.
Therefore, how to avoid depending on the history sample when the model is updated to improve the update response efficiency of the model becomes an urgent problem to be solved.
Disclosure of Invention
In view of this, embodiments of the present application provide a method, an apparatus, a device, and a medium for continuously learning model update, so as to solve the problem of how to avoid relying on a history sample during model update to improve the update response efficiency of a model.
In a first aspect, an embodiment of the present application provides a model updating method for continuous learning, where the model updating method includes:
acquiring a historical model of a model to be updated, extracting historical extractor parameters of a feature extractor in the historical model, initializing the feature extractor of the model to be updated by using the historical extractor parameters to obtain an initialized model to be updated, and taking the initialized model to be updated as a current model;
respectively inputting each sample in the updated samples into the historical model, outputting a first prediction label of the corresponding sample, respectively inputting each sample into the current model, outputting a second prediction label of the corresponding sample, and constructing characteristic learning loss and classification learning loss according to the first prediction label and the second prediction label;
constructing sample characteristic loss by using the real label and the second prediction label of the updated sample, and calculating the total training loss of the current model according to the characteristic learning loss, the classification learning loss and the sample characteristic loss;
and updating the current model according to the total training loss to obtain an updated current model, returning to the step of executing the steps of respectively inputting each sample in the updated samples into the historical model, outputting a first prediction label of the corresponding sample, respectively inputting each sample into the current model, and outputting a second prediction label of the corresponding sample until the total training loss is converged, and determining the model used when the total training loss is converged as the updated model to be updated.
In one embodiment, the constructing the feature learning loss and the classification learning loss according to the first prediction label and the second prediction label comprises:
obtaining classification target classes of classifiers in the historical model;
and for any sample, if the first prediction label corresponding to the sample belongs to the classification target class, constructing a feature learning loss and a classification learning loss by using the first prediction label corresponding to the sample and the second prediction label corresponding to the sample.
In one embodiment, the constructing the feature learning loss and the classification learning loss using the first prediction label corresponding to the sample and the second prediction label corresponding to the sample comprises:
calculating the relative entropy between a first prediction label corresponding to the sample and a second prediction label corresponding to the sample, and determining the calculation result as the characteristic learning loss corresponding to the sample;
and calculating cross entropy between a first prediction label corresponding to the sample and a second prediction label corresponding to the sample, and determining the calculation result as the classification learning loss corresponding to the sample.
In one embodiment, said constructing the sample feature loss using the true label and the second predicted label of the updated sample comprises:
and aiming at any sample, if the first prediction label corresponding to the sample does not belong to the classification target class, constructing sample characteristic loss by using the real label of the sample and the second prediction label corresponding to the sample.
In one embodiment, said constructing a sample feature loss using said true label of said sample and a second predicted label corresponding to said sample comprises:
and calculating the cross entropy between the true label of the sample and a second prediction label corresponding to the sample, and determining the calculation result as the sample characteristic loss corresponding to the sample.
In an embodiment, the calculating a total training loss of the model to be updated according to the feature learning loss, the classification learning loss and the sample feature loss includes:
acquiring a first weight value, a second weight value and a third weight value, wherein the sum of the first weight value, the second weight value and the third weight value is 1;
multiplying the feature learning loss by the first weight value to obtain a first product;
multiplying the classification learning loss by the second weight value to obtain a second product;
multiplying the sample characteristic loss by the third weight value to obtain a third product;
and adding the first product, the second product and the third product, and determining an addition result as the total training loss of the model to be updated.
In an embodiment, before the obtaining the first weight value, the second weight value and the third weight value, the method further includes:
acquiring the preference degree of a user to a historical sample for training the historical model and the preference degree of the updated sample;
and generating a first weight value, a second weight value and a third weight value according to the preference degrees of the historical samples and the preference degrees of the updated samples, wherein if the preference degrees of the historical samples are higher than the preference degrees of the updated samples, the first weight value is larger than the sum of the second weight value and the third weight value.
In a second aspect, an embodiment of the present application provides a model updating apparatus for continuous learning, where the model updating apparatus includes:
the device comprises an initialization module, a feature extraction module and a feature extraction module, wherein the initialization module is used for acquiring a historical model of a model to be updated, extracting historical extractor parameters of a feature extractor in the historical model, initializing the feature extractor of the model to be updated by using the historical extractor parameters to obtain an initialized model to be updated, and taking the initialized model to be updated as a current model;
the label prediction module is used for respectively inputting each sample in the updated samples into the historical model, outputting a first prediction label of the corresponding sample, respectively inputting each sample into the current model, outputting a second prediction label of the corresponding sample, and constructing characteristic learning loss and classification learning loss according to the first prediction label and the second prediction label;
the loss calculation module is used for constructing sample characteristic loss by using the real label of the updated sample and the second prediction label, and calculating the total training loss of the current model according to the characteristic learning loss, the classification learning loss and the sample characteristic loss;
the model updating module is used for updating the current model according to the total training loss to obtain an updated current model, and sending the updated current model to the label predicting module, so that the label predicting module respectively inputs each sample in updated samples into the historical model, outputs a first prediction label of a corresponding sample, respectively inputs each sample into the current model, and outputs a second prediction label of the corresponding sample until the total training loss is converged;
the model updating module is further configured to determine that the model used in the total loss convergence of training is an updated model to be updated.
In one embodiment, the tag prediction module comprises:
the category acquisition unit is used for acquiring a classification target category of the classifier in the historical model;
and the first loss calculation unit is used for constructing the characteristic learning loss and the classification learning loss by using the first prediction label corresponding to the sample and the second prediction label corresponding to the sample if the first prediction label corresponding to the sample belongs to the classification target class aiming at any sample.
In an embodiment, the first loss calculating unit is specifically configured to:
calculating relative entropy between a first prediction label corresponding to the sample and a second prediction label corresponding to the sample, and determining a calculation result as a feature learning loss corresponding to the sample;
and calculating the cross entropy between the first prediction label corresponding to the sample and the second prediction label corresponding to the sample, and determining the calculation result as the classification learning loss corresponding to the sample.
In one embodiment, the loss calculation module comprises:
and the second loss calculation unit is used for constructing sample characteristic loss by using the real label of the sample and the second prediction label corresponding to the sample if the first prediction label corresponding to the sample does not belong to the classification target class.
In an embodiment, the second loss calculating unit is specifically configured to:
and calculating the cross entropy between the true label of the sample and a second prediction label corresponding to the sample, and determining the calculation result as the sample characteristic loss corresponding to the sample.
In one embodiment, the loss calculation module includes:
a weight obtaining unit configured to obtain a first weight value, a second weight value, and a third weight value, where a sum of the first weight value, the second weight value, and the third weight value is 1;
a first product unit, configured to multiply the feature learning loss and the first weight value to obtain a first product;
a second product unit, configured to multiply the classification learning loss and the second weight value to obtain a second product;
a third product unit, configured to multiply the sample feature loss with the third weight value to obtain a third product;
and the total loss calculation unit is used for adding the first product, the second product and the third product and determining an addition result as the total training loss of the model to be updated.
In one embodiment, the loss calculation module further comprises:
the degree acquiring unit is used for acquiring the preference degree of the user on the historical sample for training the historical model and the preference degree of the updated sample before the first weight value, the second weight value and the third weight value are acquired;
and the weight generating unit is used for generating a first weight value, a second weight value and a third weight value according to the preference degree of the historical sample and the preference degree of the updating sample, wherein if the preference degree of the historical sample is higher than the preference degree of the updating sample, the first weight value is larger than the sum of the second weight value and the third weight value.
In a third aspect, an embodiment of the present application provides a computer device, where the computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor implements the model updating method according to the first aspect when executing the computer program.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, where a computer program is stored, and the computer program, when executed by a processor, implements the model updating method according to the first aspect.
Compared with the prior art, the embodiment of the application has the beneficial effects that: the method includes the steps of obtaining a historical model of a model to be updated, extracting historical extractor parameters of a feature extractor in the historical model, initializing the feature extractor of the model to be updated by using the historical extractor parameters, obtaining an initialized model to be updated, inputting each sample in an updated sample into the historical model, outputting a first prediction label of a corresponding sample, inputting each sample into the initialized model to be updated, outputting a second prediction label of the corresponding sample, constructing a feature learning loss and a classification learning loss according to the first prediction label and the second prediction label, constructing a sample feature loss by using a real label and the second prediction label of the updated sample, calculating a training total loss of the model to be updated according to the feature learning loss, the classification learning loss and the sample feature loss, updating the initialized model to be updated according to the training total loss, repeatedly using the updated sample to calculate the training total loss until the training total loss converges, determining the model used when the training total loss converges to be the updated model, and achieving the effect that only the historical model is updated without the historical model, the sample is not required to be maintained, the historical extractor parameters of the model can be updated quickly, and the historical model can be updated based on the historical updating efficiency.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required to be used in the embodiments or the prior art description will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings may be obtained according to these drawings without inventive labor.
FIG. 1 is a schematic diagram of an application environment of a model updating method for continuous learning according to an embodiment of the present disclosure;
FIG. 2 is a flowchart illustrating a model updating method for continuous learning according to a second embodiment of the present disclosure;
fig. 3 is a schematic flowchart of a model updating method for continuous learning according to a third embodiment of the present application;
FIG. 4 is a schematic structural diagram of a model updating apparatus for continuous learning according to a fourth embodiment of the present application;
fig. 5 is a schematic structural diagram of a computer device according to a fifth embodiment of the present application.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It should also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
As used in this specification and the appended claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to a determination" or "in response to a detection". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".
Furthermore, in the description of the present application and the appended claims, the terms "first," "second," "third," and the like are used for distinguishing between descriptions and not necessarily for describing or implying relative importance.
Reference throughout this specification to "one embodiment" or "some embodiments," or the like, means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," or the like, in various places throughout this specification are not necessarily all referring to the same embodiment, but rather "one or more but not all embodiments" unless specifically stated otherwise. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless expressly specified otherwise.
It should be understood that, the sequence numbers of the steps in the following embodiments do not mean the execution sequence, and the execution sequence of each process should be determined by the function and the inherent logic of the process, and should not constitute any limitation to the implementation process of the embodiments of the present application.
In order to explain the technical means of the present application, the following description will be given by way of specific examples.
The model updating method for continuous learning provided by the embodiment of the present application can be applied to the application environment shown in fig. 1, in which a client communicates with a server. The client includes, but is not limited to, a palm top computer, a desktop computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cloud computing device, a Personal Digital Assistant (PDA), and other computing devices. The server can be implemented by an independent server or a server cluster composed of a plurality of servers.
Referring to fig. 2, which is a schematic flow chart of a model updating method for continuous learning provided in the second embodiment of the present application, the model updating method for continuous learning is applied to the server in fig. 1, information related to a history model of a model to be updated is stored in the server, the server is connected to the client, and an update sample and a corresponding real tag that are specified by a user and provided for updating the model are obtained from the client. As shown in fig. 2, the model updating method for continuous learning may include the steps of:
step S201, obtaining a history model of a model to be updated, extracting history extractor parameters of a feature extractor in the history model, initializing the feature extractor of the model to be updated by using the history extractor parameters to obtain an initialized model to be updated, and taking the initialized model to be updated as a current model.
In the present application, the model to be updated may be a model used in any artificial intelligence scene, including but not limited to a neural network model, a deep learning model, a machine learning model, and the like, and specifically includes at least functions of a feature extractor and a classifier in the model, that is, the feature extractor can perform feature encoding and decoding and feature extraction, and the classifier can classify extracted features, and finally implement classification, recommendation, prediction, and the like of a data.
For the model to be updated, at least one version of model exists in the historical model, and further, the current latest version of model is used as the historical model processed by the application, wherein if the user is not satisfied with the current latest version of model, the previous version of model can also be used as the historical model, and a sample for training the current latest version of model can be used as a part of the update sample to train the previous version of model.
The historical model is a trained model, wherein the feature extractor and the classifier are both trained, the parameters of the feature extractor are the parameters of the historical training, namely the parameters of the historical extractor, and the parameters of the classifier are the parameters of the historical training, namely the parameters of the historical classifier.
For the update of the model, the classification category may change, so that the history classifier parameters of the history model cannot be used during the update, and the history extractor parameters are obtained by training historical samples, so that if the model to be updated inherits the history extractor parameters, the model to be updated inherits the training results of the historical samples.
The method has the advantages that the history extractor parameters are used for initializing the feature extractor of the model to be updated, the parameters of the middle feature extractor are initialized into the history extractor parameters in time, history training information is saved, meanwhile, the history samples are not needed to be used for training, the history samples are not needed to be maintained continuously, maintenance cost can be reduced, and training efficiency can be improved without using the history samples.
Step S202, respectively inputting each sample in the updated samples into the historical model, outputting a first prediction label of the corresponding sample, respectively inputting each sample into the current model, outputting a second prediction label of the corresponding sample, and constructing the characteristic learning loss and the classification learning loss according to the first prediction label and the second prediction label.
In the application, a user provides an update sample and a corresponding real label, the update sample comprises N samples, each sample is input into a historical model, a prediction label corresponding to each sample can be obtained, namely, after the characteristics are extracted through a characteristic extractor of the historical model, a classifier is used for classification, and a classification result is the prediction label. For example, for a text sample of "puppy", if the historical model is a classification of a category to which an object belongs, the "puppy" is input into the historical model to obtain a prediction label of "animal".
Similarly, each sample is input into the initialized model to be updated, so that the sample is subjected to feature extraction by using the historical extractor parameters, and then is classified by using the classifier to obtain the prediction label, wherein the classifier at the moment is an initial classifier and only has a basic classification function.
Aiming at two models (namely a historical model and an optimized model to be updated), the difference of two prediction labels obtained by using the same sample represents the contrast loss, and the characteristic learning loss and the classification learning loss can be constructed based on the requirement of the contrast loss, wherein the characteristic learning loss is the loss caused by the difference of the characteristic extractor, and the classification learning loss is the loss caused by the classification difference of the classifier. Further, relative entropy between two prediction labels can be used as feature learning loss, and cross entropy between two prediction labels can be used as classification learning loss.
Optionally, constructing the feature learning loss and the classification learning loss according to the first prediction label and the second prediction label includes:
obtaining classification target categories of classifiers in the historical model;
and aiming at any sample, if the first prediction label of the corresponding sample belongs to the classification target class, constructing the characteristic learning loss and the classification learning loss by using the first prediction label of the corresponding sample and the second prediction label of the corresponding sample.
The classification target category of the classifier in the historical model can be obtained, if the first prediction label of a sample does not belong to the classification target category, the historical model can accurately classify the sample, and errors possibly caused by the fact that the classifier cannot distinguish the sample can be obtained.
Optionally, the constructing the feature learning loss and the classification learning loss by using the first prediction label of the corresponding sample and the second prediction label of the corresponding sample includes:
calculating the relative entropy between the feature extraction value in the first prediction label of the corresponding sample and the feature extraction value in the second prediction label of the corresponding sample, and determining the calculation result as the feature learning loss of the corresponding sample;
and calculating the cross entropy between the first prediction label of the corresponding sample and the second prediction label of the corresponding sample, and determining the calculation result as the classification learning loss of the corresponding sample.
Wherein, the feature extraction value is the value output by the feature extractor in the use model, aiming at the data x belonging to Dt, the Dt is the updating sample, the x is input into the model to be updated and the historical model, the output of the feature extractor is f t+1 (x)，f t (x) The output of the classifier is g t+1 (x)，g t (x) The predicted labels are:
φ t+1 (x)= argmax(g t+1 (x)，f t+1 (x)，x)
φ t (x)= argmax(g t (x)，f t (x)，x)
wherein phi t (x) Is the first predicted label, phi t+1 (x) Is the second predictive label.
If phi is t (x) The method belongs to Ct, the Ct is a classification target category, namely the category predicted by the sample is the information already learned by an old model, and the characteristic extraction capability is learned as follows:
L KL = KL(f t+1 (x)，f t (x))
wherein KL (\8729;) is calculated for KL divergence; meanwhile, the classifier of the new model also needs to fit the prediction label generated by the historical model to learn the capacity of identifying the historical samples:
L CE old =CE(φ t (x)，φ t+1 (x))。
and step S203, constructing sample characteristic loss by using the real label and the second prediction label of the updated sample, and calculating the total training loss of the current model according to the characteristic learning loss, the classification learning loss and the sample characteristic loss.
In the application, for updating samples, each sample can be calculated to obtain corresponding feature learning loss, classification learning loss and sample feature loss, and the total loss of the model can be obtained by calculating the loss of all samples.
And calculating the losses of all samples in the updated sample to obtain the total training loss of the current model, wherein the total training loss is the sum of the feature learning loss, the classification learning loss and the sample feature loss of the sample.
Further, for some scenes, some samples may only contribute to the feature learning loss and the classification learning loss, and some samples only contribute to the sample feature loss, for example, for a scene where the current model and the historical model may have different number and types of classes for classifying the samples, it may be considered that different samples may have different losses, that is, for a sample, if the historical model can be predicted, that is, the historical model learns the information of the sample when it is trained, the loss of the sample is mainly the loss when the feature extractor extracts and the loss when the classifier learns the historical model; for a sample, if the historical model cannot predict, it means that the historical model has not learned the information of the sample when it is trained, and therefore, the loss of the sample is mainly the loss of the classifier when learning a new sample.
Optionally, constructing the sample feature loss by using the true label and the second prediction label of the updated sample comprises:
and aiming at any sample, if the first prediction label corresponding to the sample does not belong to the classification target class, constructing sample characteristic loss by using the real label of the sample and the second prediction label corresponding to the sample.
And if the first prediction label belongs to the classification target class, constructing the characteristic loss of the sample by using the first prediction label of the corresponding sample and the second prediction label of the corresponding sample.
Optionally, constructing the sample feature loss by using the true label of the sample and the second prediction label of the corresponding sample includes:
and calculating the cross entropy between the true label of the sample and the second prediction label of the corresponding sample, and determining the calculation result as the sample characteristic loss of the corresponding sample.
If phi t (x) E, O is a category which does not belong to a classification target category, namely the sample is a category which is not learned by the historical model (namely a new category), and at the moment, the model to be updated learns the characteristics of the updated sample by taking cross entropy as loss:
L CE new =CE(y，φ t+1 (x) Y is the true label corresponding to the sample x.
And S204, updating the current model according to the total training loss to obtain an updated current model, returning to the step of executing the steps of respectively inputting each sample in the updated samples into the historical model, outputting a first prediction label of the corresponding sample, respectively inputting each sample into the current model, and outputting a second prediction label of the corresponding sample until the total training loss is converged, and determining the model used when the total training loss is converged as the updated model to be updated.
In the application, in one iteration of updating the model, parameters in the model of the current iteration need to be updated after the training total loss is obtained, and the parameters are used for the next iteration. The parameters in the model of the current iteration can be updated by adopting a gradient descent method, a gradient back propagation method or the like.
The final purpose of iteration is to make the total loss of training converge when updating the model, and the corresponding model is the updated model to be updated when converging.
The method includes the steps of obtaining a history model of a model to be updated, extracting history extractor parameters of a feature extractor in the history model, initializing the feature extractor of the model to be updated by using the history extractor parameters to obtain an initialized model to be updated, inputting each sample in an updated sample into the history model respectively, outputting a first prediction label of a corresponding sample, inputting each sample into the initialized model to be updated respectively, outputting a second prediction label of the corresponding sample, constructing a feature learning loss and a classification learning loss according to the first prediction label and the second prediction label, constructing a sample feature loss by using a real label and the second prediction label of the updated sample, calculating a total training loss of the model to be updated according to the feature learning loss, the initialized model to be updated and the sample to be updated, repeatedly using the updated sample to calculate a total training loss until the total training loss converges, determining the model used in the total training loss to be updated as the updated model, and realizing that the model to be updated is updated only by using the history model without history model, the model to be updated without maintaining the sample, quickly updating the history model based on the history model and improving the historical updating efficiency of the model.
Referring to fig. 3, a schematic flow chart of a model updating method for continuous learning provided in the third embodiment of the present application is shown, where the model updating method includes the following steps:
step S301, obtaining a history model of the model to be updated, extracting history extractor parameters of a feature extractor in the history model, initializing the feature extractor of the model to be updated by using the history extractor parameters to obtain an initialized model to be updated, and taking the initialized model to be updated as a current model.
Step S302, inputting each sample in the updated samples into the historical model respectively, outputting a first prediction label of the corresponding sample, inputting each sample into the current model respectively, outputting a second prediction label of the corresponding sample, and constructing the characteristic learning loss and the classification learning loss according to the first prediction label and the second prediction label.
Step S303, the real label and the second prediction label of the updated sample are used for constructing sample characteristic loss.
The contents of steps S301 to S303 are the same as the contents of steps S201 to S203, and the descriptions of steps S201 to S203 may be specifically referred to, which are not repeated herein.
Step S304, a first weight value, a second weight value, and a third weight value are obtained.
The three weight values can be set according to requirements, and the user can upload the three weight values to the server, so that the server can acquire the three weight values. The requirement may refer to a preference of a user for a sample, such as a preference for a historical sample or an updated sample, a preference for a model obtained by a historical sample may be used in a processing scenario for historical data, and a preference for a model obtained by an updated sample may be used in a processing scenario for real-time data.
Optionally, before obtaining the first weight value, the second weight value, and the third weight value, the method further includes:
acquiring the preference degree of a user on a historical sample of a training historical model and the preference degree on an updating sample;
and generating a first weight value, a second weight value and a third weight value according to the preference degree of the history sample and the preference degree of the update sample, wherein if the preference degree of the history sample is higher than the preference degree of the update sample, the first weight value is larger than the sum of the second weight value and the third weight value.
The server side can automatically generate the weight values according to the preference degree of the user for the samples, wherein the preference degree of the historical samples is higher, the first weight value is larger, the second weight value and the third weight value are smaller, otherwise, the preference degree of the updated samples is higher, the first weight value is smaller, and the second weight value and the third weight value are larger.
Step S305, the feature learning loss is multiplied by a first weight value to obtain a first product, the classification learning loss is multiplied by a second weight value to obtain a second product, the sample feature loss is multiplied by a third weight value to obtain a third product, the first product, the second product and the third product are added, and the addition result is determined as the total training loss of the current model.
Wherein the sum of the first weight value, the second weight value and the third weight value is 1.
Training the total loss L on the basis of the formula in the second embodiment continue The calculation formula of (c) can be characterized as:
L continue =αL KL +βL CE old +γL CE new
where α, β and γ are weight values of KL (\8729;) and CE (\8729;), respectively, that is, weight values characterizing relative entropy loss and cross entropy loss.
And S306, updating the current model according to the total training loss to obtain an updated current model, returning to execute the steps of respectively inputting each sample in the updated samples into the historical model, outputting a first prediction label of the corresponding sample, respectively inputting each sample into the current model, and outputting a second prediction label of the corresponding sample until the total training loss is converged, and determining the model used when the total training loss is converged as the updated model to be updated.
The content of the step S306 is the same as the content of the step S204, and reference may be specifically made to the description of the step S204, which is not repeated herein.
The method includes the steps of obtaining a historical model of a model to be updated, extracting historical extractor parameters of a feature extractor in the historical model, initializing the feature extractor of the model to be updated by using the historical extractor parameters to obtain an initialized model to be updated, inputting each sample in an updating sample into the historical model respectively, outputting a first prediction label corresponding to the sample, inputting each sample into the initialized model to be updated respectively, outputting a second prediction label corresponding to the sample, building a feature learning loss and a classification learning loss according to the first prediction label and the second prediction label, building a sample feature loss by using a real label and the second prediction label of the updating sample, conducting weighted summation on the feature learning loss, the classification learning loss and the sample feature loss of all samples to obtain a training total loss, updating the initialized model to be updated according to the training total loss, calculating the training total loss by repeatedly using the updating sample until the training total loss converges, determining the model used when the training total loss converges as the updated model to be updated, achieving the effect that the model to be updated is updated only by using the model to be updated without the historical model to be updated under the condition that the historical sample is not needed, updating is not only by using the model to be updated, and the historical model can be updated flexibly and the historical model can be updated based on the user demand of the historical model.
Corresponding to the model updating method for continuous learning in the foregoing embodiment, fig. 4 shows a block diagram of a model updating apparatus for continuous learning provided in the fourth embodiment of the present application, where the model updating apparatus is applied to the server in fig. 1, the server stores information related to a history model of a model to be updated, the server is connected to the client, and obtains a model to be updated specified by a user from the client, and an updated sample and a corresponding real tag provided for updating the model. For ease of illustration, only portions relevant to the embodiments of the present application are shown.
Referring to fig. 4, the model updating apparatus includes:
the initialization module 41 is configured to obtain a history model of the model to be updated, extract history extractor parameters of the feature extractor in the history model, initialize the feature extractor of the model to be updated by using the history extractor parameters, obtain an initialized model to be updated, and use the initialized model to be updated as a current model;
the label prediction module 42 is configured to input each sample in the updated samples into the history model, output a first prediction label of the corresponding sample, input each sample into the current model, output a second prediction label of the corresponding sample, and construct a feature learning loss and a classification learning loss according to the first prediction label and the second prediction label;
a loss calculation module 43, configured to construct a sample feature loss by using the true label and the second prediction label of the updated sample, and calculate a total training loss of the current model according to the feature learning loss, the classification learning loss, and the sample feature loss;
the model updating module 44 is configured to update a current model according to the total training loss to obtain an updated current model, and send the updated current model to the label prediction module, so that the label prediction module inputs each sample in an updated sample into the history model, outputs a first prediction label of a corresponding sample, inputs each sample into the current model, and outputs a second prediction label of the corresponding sample until the total training loss converges;
the model updating module 44 is further configured to determine the model used in the convergence of the total loss of training as the updated model to be updated.
Optionally, the tag prediction module 42 includes:
the classification acquisition unit is used for acquiring classification target classes of the classifiers in the history model;
and the first loss calculation unit is used for constructing the characteristic learning loss and the classification learning loss by using the first prediction label of the corresponding sample and the second prediction label of the corresponding sample if the first prediction label of the corresponding sample belongs to the classification target category aiming at any sample.
Optionally, the first loss calculating unit is specifically configured to:
calculating the relative entropy between the first prediction label of the corresponding sample and the second prediction label of the corresponding sample, and determining the calculation result as the characteristic learning loss of the corresponding sample;
and calculating the cross entropy between the first prediction label of the corresponding sample and the second prediction label of the corresponding sample, and determining the calculation result as the classification learning loss of the corresponding sample.
Optionally, the loss calculating module 43 includes:
and the second loss calculation unit is used for constructing the sample characteristic loss by using the real label of the sample and the second prediction label of the corresponding sample if the first prediction label corresponding to the sample does not belong to the classification target class aiming at any sample.
Optionally, the second loss calculating unit is specifically configured to:
and calculating the cross entropy between the true label of the sample and the second prediction label of the corresponding sample, and determining the calculation result as the sample characteristic loss of the corresponding sample.
Optionally, the loss calculating module 43 includes:
the weight acquiring unit is used for acquiring a first weight value, a second weight value and a third weight value, wherein the sum of the first weight value, the second weight value and the third weight value is 1;
the first product unit is used for multiplying the feature learning loss by the first weight value to obtain a first product;
the second product unit is used for multiplying the classification learning loss by a second weight value to obtain a second product;
the third product unit is used for multiplying the sample characteristic loss by a third weight value to obtain a third product;
and the total loss calculating unit is used for adding the first product, the second product and the third product and determining an addition result as the training total loss of the model to be updated.
Optionally, the loss calculating module 43 further includes:
the degree acquiring unit is used for acquiring the preference degree of the user on the historical sample of the training historical model and the preference degree of the updating sample before acquiring the first weight value, the second weight value and the third weight value;
and the weight generating unit is used for generating a first weight value, a second weight value and a third weight value according to the preference degree of the historical samples and the preference degree of the updating samples, wherein if the preference degree of the historical samples is higher than the preference degree of the updating samples, the first weight value is larger than the sum of the second weight value and the third weight value.
It should be noted that, because the above-mentioned information interaction between the modules, the execution process, and other contents are based on the same concept, specific functions, and technical effects brought by the method embodiment of the present application may be specifically referred to a part of the method embodiment, and are not described herein again.
Fig. 5 is a schematic structural diagram of a computer device according to a fifth embodiment of the present application. As shown in fig. 5, the computer apparatus of this embodiment includes: at least one processor (only one shown in fig. 5), a memory, and a computer program stored in the memory and executable on the at least one processor, the processor when executing the computer program implementing the steps in any of the various continuously learned model update method embodiments described above.
The computer device may include, but is not limited to, a processor, a memory. It will be appreciated by those skilled in the art that fig. 5 is merely an example of a computer device and is not intended to be limiting, and that a computer device may include more or fewer components than those shown, or some components may be combined, or different components may be included, such as a network interface, a display screen, and input devices, etc.
The Processor may be a CPU, or other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory includes readable storage media, internal memory, etc., wherein the internal memory may be the internal memory of the computer device, and the internal memory provides an environment for the operating system and the execution of the computer-readable instructions in the readable storage media. The readable storage medium may be a hard disk of the computer device, and in other embodiments may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, provided on the computer device. Further, the memory may also include both internal and external storage units of the computer device. The memory is used for storing an operating system, application programs, a BootLoader (BootLoader), data, and other programs, such as program codes of a computer program, and the like. The memory may also be used to temporarily store data that has been output or is to be output.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules, so as to perform all or part of the functions described above. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. For the specific working processes of the units and modules in the above-mentioned apparatus, reference may be made to the corresponding processes in the foregoing method embodiments, which are not described herein again. The integrated unit, if implemented in the form of a software functional unit and sold or used as a separate product, may be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the method of the embodiments described above may be implemented by instructing relevant hardware by a computer program, and the computer program may be stored in a computer readable storage medium, and when executed by a processor, the computer program may implement the steps of the embodiments of the method described above. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer readable medium may include at least: any entity or device capable of carrying computer program code, recording medium, computer Memory, read-Only Memory (ROM), random Access Memory (RAM), electrical carrier signals, telecommunications signals, and software distribution media. Such as a usb-disk, a removable hard disk, a magnetic or optical disk, etc. In certain jurisdictions, computer-readable media may not be an electrical carrier signal or a telecommunications signal in accordance with legislative and patent practice.
When the computer program product runs on a computer device, the computer device is enabled to implement the steps in the method embodiments.
In the above embodiments, the description of each embodiment has its own emphasis, and reference may be made to the related description of other embodiments for parts that are not described or recited in any embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the technical solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus/computer device and method may be implemented in other ways. For example, the above-described apparatus/computer device embodiments are merely illustrative, and for example, a module or a unit may be divided into only one logical function, and may be implemented in other ways, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
The above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.

Claims (10)

1. A model update method for continuous learning, the model update method comprising:
acquiring a historical model of a model to be updated, extracting historical extractor parameters of a feature extractor in the historical model, initializing the feature extractor of the model to be updated by using the historical extractor parameters to obtain an initialized model to be updated, and taking the initialized model to be updated as a current model;
respectively inputting each sample in the updated samples into the historical model, outputting a first prediction label of the corresponding sample, respectively inputting each sample into the current model, outputting a second prediction label of the corresponding sample, and constructing characteristic learning loss and classification learning loss according to the first prediction label and the second prediction label;
constructing sample characteristic loss by using the real label and the second prediction label of the updated sample, and calculating the total training loss of the current model according to the characteristic learning loss, the classification learning loss and the sample characteristic loss;
and updating the current model according to the total training loss to obtain an updated current model, returning to the step of executing the steps of respectively inputting each sample in the updated samples into the historical model, outputting a first prediction label of the corresponding sample, respectively inputting each sample into the current model, and outputting a second prediction label of the corresponding sample until the total training loss is converged, and determining the model used when the total training loss is converged as the updated model to be updated.
2. The model updating method of claim 1, wherein said constructing feature learning losses and classification learning losses based on said first prediction label and said second prediction label comprises:
obtaining classification target classes of classifiers in the historical model;
and for any sample, if the first prediction label corresponding to the sample belongs to the classification target class, constructing a feature learning loss and a classification learning loss by using the first prediction label corresponding to the sample and the second prediction label corresponding to the sample.
3. The model updating method of claim 2, wherein said constructing feature learning loss and classification learning loss using a first prediction label corresponding to said sample and a second prediction label corresponding to said sample comprises:
calculating the relative entropy between a first prediction label corresponding to the sample and a second prediction label corresponding to the sample, and determining the calculation result as the characteristic learning loss corresponding to the sample;
and calculating the cross entropy between the first prediction label corresponding to the sample and the second prediction label corresponding to the sample, and determining the calculation result as the classification learning loss corresponding to the sample.
4. The model updating method of claim 2, wherein said constructing a sample feature loss using the true label and the second predicted label of the updated sample comprises:
and aiming at any sample, if the first prediction label corresponding to the sample does not belong to the classification target class, constructing sample characteristic loss by using the real label of the sample and the second prediction label corresponding to the sample.
5. The model updating method of claim 4, wherein said constructing a sample feature loss using the true label of the sample and the second predicted label corresponding to the sample comprises:
and calculating the cross entropy between the true label of the sample and a second prediction label corresponding to the sample, and determining the calculation result as the sample characteristic loss corresponding to the sample.
6. The model updating method according to any one of claims 1 to 5, wherein the calculating the total training loss of the model to be updated according to the feature learning loss, the classification learning loss and the sample feature loss comprises:
acquiring a first weight value, a second weight value and a third weight value, wherein the sum of the first weight value, the second weight value and the third weight value is 1;
multiplying the feature learning loss by the first weight value to obtain a first product;
multiplying the classification learning loss by the second weight value to obtain a second product;
multiplying the sample characteristic loss by the third weight value to obtain a third product;
and adding the first product, the second product and the third product, and determining an addition result as the total training loss of the model to be updated.
7. The model updating method of claim 6, wherein before the obtaining the first weight value, the second weight value and the third weight value, further comprising:
acquiring the preference degree of a user to a historical sample for training the historical model and the preference degree of the updated sample;
and generating a first weight value, a second weight value and a third weight value according to the preference degrees of the historical samples and the preference degrees of the updated samples, wherein if the preference degrees of the historical samples are higher than the preference degrees of the updated samples, the first weight value is larger than the sum of the second weight value and the third weight value.
8. A model updating apparatus for continuous learning, characterized by comprising:
the device comprises an initialization module, a feature extraction module and a feature extraction module, wherein the initialization module is used for acquiring a historical model of a model to be updated, extracting historical extractor parameters of a feature extractor in the historical model, initializing the feature extractor of the model to be updated by using the historical extractor parameters to obtain an initialized model to be updated, and taking the initialized model to be updated as a current model;
the label prediction module is used for respectively inputting each sample in the updated samples into the historical model, outputting a first prediction label of the corresponding sample, respectively inputting each sample into the current model, outputting a second prediction label of the corresponding sample, and constructing characteristic learning loss and classification learning loss according to the first prediction label and the second prediction label;
the loss calculation module is used for constructing sample characteristic loss by using the real label and the second prediction label of the updated sample, and calculating the total training loss of the model to be updated according to the characteristic learning loss, the classification learning loss and the sample characteristic loss;
the model updating module is used for updating the current model according to the total training loss to obtain an updated current model, and sending the updated current model to the label predicting module, so that the label updating module respectively inputs each sample in updated samples into the historical model, outputs a first prediction label of a corresponding sample, respectively inputs each sample into the current model, and outputs a second prediction label of the corresponding sample until the total training loss is converged;
the model updating module is further configured to determine that the model used in the total loss convergence of training is the updated model to be updated.
9. A computer device, characterized in that the computer device comprises a processor, a memory and a computer program stored in the memory and executable on the processor, the processor implementing the model updating method as claimed in any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out a model updating method as claimed in any one of claims 1 to 7.
CN202211164211.XA 2022-09-23 2022-09-23 Model updating method, device, equipment and medium for continuous learning Pending CN115238838A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211164211.XA CN115238838A (en) 2022-09-23 2022-09-23 Model updating method, device, equipment and medium for continuous learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211164211.XA CN115238838A (en) 2022-09-23 2022-09-23 Model updating method, device, equipment and medium for continuous learning

Publications (1)

Publication Number Publication Date
CN115238838A true CN115238838A (en) 2022-10-25

Family

ID=83667361

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211164211.XA Pending CN115238838A (en) 2022-09-23 2022-09-23 Model updating method, device, equipment and medium for continuous learning

Country Status (1)

Country Link
CN (1) CN115238838A (en)

Similar Documents

Publication Publication Date Title
CN112164391A (en) Statement processing method and device, electronic equipment and storage medium
US20220092407A1 (en) Transfer learning with machine learning systems
CN115244587A (en) Efficient ground truth annotation
CN110781818B (en) Video classification method, model training method, device and equipment
CN112883990A (en) Data classification method and device, computer storage medium and electronic equipment
WO2024060684A1 (en) Model training method, image processing method, device, and storage medium
CN111931809A (en) Data processing method and device, storage medium and electronic equipment
CN115063875A (en) Model training method, image processing method, device and electronic equipment
CN107291774B (en) Error sample identification method and device
CN114662006B (en) End cloud collaborative recommendation system and method and electronic equipment
CN114492601A (en) Resource classification model training method and device, electronic equipment and storage medium
CN115034315A (en) Business processing method and device based on artificial intelligence, computer equipment and medium
CN113204698B (en) News subject term generation method, device, equipment and medium
CN110826327A (en) Emotion analysis method and device, computer readable medium and electronic equipment
CN113986674A (en) Method and device for detecting abnormity of time sequence data and electronic equipment
CN114037059A (en) Pre-training model, model generation method, data processing method and data processing device
CN116578925B (en) Behavior prediction method, device and storage medium based on feature images
US20220101120A1 (en) Interpretable visualization system for graph neural network
CN115482436B (en) Training method and device for image screening model and image screening method
CN116257885A (en) Private data communication method, system and computer equipment based on federal learning
CN113989569B (en) Image processing method, device, electronic equipment and storage medium
CN113688232B (en) Method and device for classifying bid-inviting text, storage medium and terminal
CN113591881B (en) Intention recognition method and device based on model fusion, electronic equipment and medium
CN115238838A (en) Model updating method, device, equipment and medium for continuous learning
CN115345669A (en) Method and device for generating file, storage medium and computer equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20221025