WO2025130342A1 - 一种文本分类模型的训练方法、装置、介质及电子设备 - Google Patents

一种文本分类模型的训练方法、装置、介质及电子设备 Download PDF

Info

Publication number
WO2025130342A1
WO2025130342A1 PCT/CN2024/127362 CN2024127362W WO2025130342A1 WO 2025130342 A1 WO2025130342 A1 WO 2025130342A1 CN 2024127362 W CN2024127362 W CN 2024127362W WO 2025130342 A1 WO2025130342 A1 WO 2025130342A1
Authority
WO
WIPO (PCT)
Prior art keywords
model
trained
text
result
teacher
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2024/127362
Other languages
English (en)
French (fr)
Inventor
陆金星
陈欢
赵智源
赵建英
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alipay Hangzhou Information Technology Co Ltd
Original Assignee
Alipay Hangzhou Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alipay Hangzhou Information Technology Co Ltd filed Critical Alipay Hangzhou Information Technology Co Ltd
Publication of WO2025130342A1 publication Critical patent/WO2025130342A1/zh
Anticipated expiration legal-status Critical
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/35—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/096—Transfer learning

Definitions

  • the present invention relates to the field of computer technology, and in particular to a training method, device, medium and electronic device for a text classification model.
  • text classification models are generally trained based on text data and the categories corresponding to the text data.
  • this specification provides a training method for a text classification model.
  • This specification provides a text classification model training method, device, medium and electronic device to partially solve the above-mentioned problems existing in the related technology.
  • the present specification provides a training method for a text classification model, comprising: determining a text sample, and determining a plurality of pre-trained teacher models; wherein the parameter amounts of each teacher model are different; in order of the parameter amounts of each teacher model from small to large, for each teacher model, executing: inputting the text sample into the teacher model to determine a pseudo-labeling result, and inputting the text sample into a student model to be trained to determine a classification result, training the student model to be trained at least according to the pseudo-labeling result obtained based on the teacher model and the classification result; using the trained student model as a text classification model; wherein the text classification model is used to determine the classification result of the text to be classified based on the text to be classified.
  • the student model to be trained includes a feature extraction layer and a classification layer; the text sample is input into the student model to be trained to determine the classification result, specifically comprising: inputting the text sample into the feature extraction layer of the student model to be trained to determine the feature sequence corresponding to the text sample; taking the feature of the position corresponding to the teacher model in the feature sequence as the output feature; inputting the output feature into the classification layer of the student model to be trained to determine the classification result.
  • the student model to be trained is trained at least based on the pseudo-labeling result obtained based on the teacher model and the classification result, specifically including: taking the feature corresponding to the specified position in the feature sequence as the first feature; inputting the first feature into the classification layer of the student model to be trained to determine the first result; determining the annotation corresponding to the text sample; and training the student model to be trained based on the pseudo-labeling result obtained based on the teacher model, the classification result, the first result and the annotation.
  • the student model to be trained is trained according to the pseudo-label result obtained based on the teacher model, the classification result, the first result and the annotation, specifically including: determining a first task loss according to the first result and the annotation; determining a second task loss according to the pseudo-label result obtained based on the teacher model and the classification result; training the student model to be trained according to the first task loss and the second task loss.
  • the student model to be trained is trained at least based on the pseudo-label results obtained based on the teacher model and the classification results, specifically including: determining other teacher models based on the parameter amount of the teacher model; wherein the parameter amount of the other teacher models is smaller than the parameter amount of the teacher model; taking the feature of the position corresponding to the other teacher models in the feature sequence as the second feature; inputting the second feature into the classification layer of the student model to be trained to determine the second result; determining the pseudo-label results corresponding to the other teacher models and taking them as other results; training the student model to be trained at least based on the pseudo-label results obtained based on the teacher model, the classification result, the second result and the other results.
  • the student model to be trained is trained at least based on the pseudo-labeling result, the classification result, the second result and the other results obtained based on the teacher model, specifically including: taking the feature corresponding to the specified position in the feature sequence as the first feature; inputting the first feature into the classification layer of the student model to be trained to determine the first result; determining the annotation corresponding to the text sample; training the student model to be trained based on the pseudo-labeling result, the classification result, the second result, the other results, the first result and the annotation obtained based on the teacher model.
  • the student model to be trained is trained according to the pseudo-label result obtained based on the teacher model, the classification result, the second result, the other results, the first result and the annotation, specifically including: determining the first task loss according to the first result and the annotation; determining the second task loss according to the pseudo-label result obtained based on the teacher model and the classification result; determining the third task loss according to the second result and the other results; training the student model to be trained according to the first task loss, the second task loss and the third task loss.
  • the student model to be trained is trained according to the first task loss, the second task loss and the third task loss, specifically including: weighting the first task loss, the second task loss and the third task loss respectively according to specified weights; training the student model to be trained according to the weighted first task loss, the weighted second task loss and the weighted third task loss.
  • teacher models are pre-trained, specifically including: determining the annotations corresponding to the text samples; for each teacher model to be trained, training the teacher model to be trained based on the text samples and the annotations.
  • the method further includes: determining an initial student model, and determining annotations corresponding to the text sample; and training the initial student model based on the text sample and the annotations to obtain a student model to be trained.
  • the method also includes: determining the user's input text in response to the user's input operation; determining a pre-stored standard text; using the standard text and the input text as text to be classified; inputting the text to be classified into the text classification model to determine the classification result of the text to be classified; when the classification result is similar, determining the reply text corresponding to the standard text and displaying it to the user.
  • the present specification provides a training device for a text classification model, comprising: a first determination module, used to determine a text sample, and to determine a plurality of pre-trained teacher models; wherein the parameter quantities of each teacher model are different; a first training module, used to execute, in order of the parameter quantities of each teacher model from small to large, for each teacher model in turn: inputting the text sample into the teacher model to determine a pseudo-labeling result, and inputting the text sample into a student model to be trained to determine a classification result, and training the student model to be trained at least according to the pseudo-labeling result obtained based on the teacher model and the classification result; a second determination module, used to use the trained student model as a text classification model; wherein the text classification model is used to determine the classification result of the text to be classified based on the text to be classified.
  • the student model to be trained includes a feature extraction layer and a classification layer; the first training module is specifically used to input the text sample into the feature extraction layer of the student model to be trained, and determine the feature sequence corresponding to the text sample; use the feature of the position corresponding to the teacher model in the feature sequence as the output feature; input the output feature into the classification layer of the student model to be trained to determine the classification result.
  • the first training module is specifically used to take the feature corresponding to the specified position in the feature sequence as the first feature; input the first feature into the classification layer of the student model to be trained to determine a first result; determine the annotation corresponding to the text sample; and train the student model to be trained according to the pseudo-label result obtained based on the teacher model, the classification result, the first result and the annotation.
  • the first training module is specifically used to determine a first task loss based on the first result and the annotation; determine a second task loss based on the pseudo-label result obtained based on the teacher model and the classification result; and train the student model to be trained based on the first task loss and the second task loss.
  • the first training module is specifically used to determine other teacher models according to the parameter amount of the teacher model; wherein the parameter amount of the other teacher models is smaller than the parameter amount of the teacher model; taking the feature of the position corresponding to the other teacher model in the feature sequence as the second feature; inputting the second feature into the learning model to be trained;
  • the method comprises the following steps: determining a second result by using the classification layer of the student model; determining a pseudo-label result corresponding to the other teacher models and using it as the other result; and training the student model to be trained based on at least the pseudo-label result, the classification result, the second result and the other result obtained based on the teacher model.
  • the first training module is specifically used to take the feature corresponding to the specified position in the feature sequence as the first feature; input the first feature into the classification layer of the student model to be trained to determine a first result; determine the annotation corresponding to the text sample; and train the student model to be trained according to the pseudo-label result obtained based on the teacher model, the classification result, the second result, the other results, the first result and the annotation.
  • the first training module is specifically used to determine a first task loss based on the first result and the annotation; determine a second task loss based on the pseudo-label result obtained based on the teacher model and the classification result; determine a third task loss based on the second result and the other results; and train the student model to be trained based on the first task loss, the second task loss and the third task loss.
  • the first training module is specifically used to weight the first task loss, the second task loss and the third task loss respectively according to specified weights; and train the student model to be trained according to the weighted first task loss, the weighted second task loss and the weighted third task loss.
  • the device further includes: a second training module, used to determine the annotation corresponding to the text sample; for each teacher model to be trained, training the teacher model to be trained based on the text sample and the annotation.
  • a second training module used to determine the annotation corresponding to the text sample
  • the first determination module is further used to determine an initial student model and determine annotations corresponding to the text sample; based on the text sample and the annotations, the initial student model is trained to obtain a student model to be trained.
  • the device also includes: an application module, used to determine the user's input text in response to the user's input operation; determine a pre-stored standard text; use the standard text and the input text as text to be classified; input the text to be classified into the text classification model to determine the classification result of the text to be classified; when the classification result is similar, determine the reply text corresponding to the standard text and display it to the user.
  • an application module used to determine the user's input text in response to the user's input operation; determine a pre-stored standard text; use the standard text and the input text as text to be classified; input the text to be classified into the text classification model to determine the classification result of the text to be classified; when the classification result is similar, determine the reply text corresponding to the standard text and display it to the user.
  • This specification provides a computer-readable storage medium, which stores a computer program.
  • the computer program is executed by a processor, the training method of the above-mentioned text classification model is implemented.
  • This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned text classification model training method when executing the program.
  • At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:
  • a text sample and a plurality of pre-trained teacher models are determined.
  • the text sample is input into the teacher model in turn.
  • the method comprises the following steps: first, determining a pseudo-labeling result, inputting a text sample into a student model to be trained, determining a classification result, and training the student model to be trained at least based on the pseudo-labeling result and the classification result obtained based on the teacher model.
  • the trained student model is used as a text classification model.
  • the method determines the text samples and several pre-trained teacher models. Then, according to the parameter quantity of each teacher model from small to large, for each teacher model, the text sample is input into the teacher model in turn to determine the pseudo-label result, and the text sample is input into the student model to be trained to determine the classification result, and the student model to be trained is trained at least based on the pseudo-label result and the classification result obtained based on the teacher model. After that, the trained student model is used as the text classification model. Through the training of the student model guided by each teacher, the text representation ability and classification accuracy of the text classification model are improved.
  • FIG1 is a flow chart of a training method for a text classification model provided in this specification
  • FIG2 is a schematic diagram of the structure of a student model provided in this specification.
  • FIG3 is a schematic diagram of a characteristic sequence provided in this specification.
  • FIG4 is a schematic diagram of a training device for a text classification model provided in this specification.
  • FIG. 5 is a schematic diagram of an electronic device corresponding to FIG. 1 provided in this specification.
  • FIG1 is a flowchart of a training method for a text classification model provided in this specification, which specifically includes the following steps.
  • S100 Determine a text sample and determine a number of pre-trained teacher models; wherein the parameters of each teacher model are The quantities vary.
  • the device for training the text classification model can determine a text sample and determine a plurality of pre-trained teacher models.
  • the device for training the text classification model can be a server, or an electronic device such as a desktop computer, a laptop computer, etc.
  • the training method of the text classification model provided in this specification is described below with the server as the execution subject.
  • the above-mentioned text samples can be pre-collected text data, or sample data in any existing text data set.
  • the annotation of text samples is related to the text classification model.
  • the text samples can be pre-collected user input texts.
  • the user input texts can be questions related to the transaction items raised by the user.
  • the user can ask questions about the size, dimensions, color, thickness and usage of the transaction items, and the questions raised by the user can be text samples.
  • the text entered by the user can be "What is the size?"
  • the text entered by the user can also be questions related to the transaction process raised by the user.
  • the user can ask text about the process information such as the entry, starting party, overall process, transaction tools and end party of the transaction process.
  • the text entered by the user can be "How to trade".
  • the text sample includes at least two sentences, and at least one sentence in the text sample corresponds to the reply text.
  • the reply text is the text that pre-marks the text entered by the user.
  • the annotation corresponding to the text sample is similar or dissimilar. When the annotation is similar, it means that the sentences in the text sample are similar. When the annotation is dissimilar, it means that the sentences in the text sample are dissimilar.
  • the text classification model is used to determine the topic of the text sample, the text sample is labeled as various topic types, which are not specifically limited in this specification. For ease of explanation, the following is an example of determining whether the text samples are similar, and the following text samples are labeled as one of similar and dissimilar.
  • the teacher model is a pre-trained model, and can also be any existing text classification model, which is not specifically limited in this specification.
  • Different teacher models have different parameter quantities, that is, the parameter quantities of each teacher model are different, but the purpose of each teacher model is the same.
  • the teacher model includes a first model and a second model, the first model includes 12 network layers, and the second model includes 24 network layers, and the parameter quantities of the first model and the second model are different.
  • each teacher model can be a model with a BERT (Bidirectional Encoder Representation from Transformers) structure, and of course it can also be a model with other structures, which is not specifically limited in this specification.
  • BERT Bidirectional Encoder Representation from Transformers
  • S102 In order of the parameter amounts of the teacher models from small to large, for each teacher model, execute: inputting the text sample into the teacher model to determine the pseudo-labeling result, and inputting the text sample into the student model to be trained to determine the classification result, and training the student model to be trained at least based on the pseudo-labeling result obtained based on the teacher model and the classification result.
  • the server can execute the following process for each teacher model in order of the parameter amount of each teacher model from small to large: input the text sample into the teacher model to determine the pseudo-label result, and input the text sample into the student model to be trained to determine the classification result, and train the student model to be trained at least based on the pseudo-label result and the classification result obtained based on the teacher model.
  • the student model can be a BERT (Bidirectional Encoder Representation from Transformers) structure model.
  • the number of parameters of the student model is smaller than that of all teacher models.
  • the student model includes 4 network layers.
  • the classification result and pseudo-labeling result are used to characterize whether the sentences in the text sample are similar.
  • the classification result can be one of similarity and dissimilarity, and the pseudo-labeling result can be one of similarity and dissimilarity.
  • the server can train the student model to be trained with the goal of at least minimizing the difference between the pseudo-label results and classification results obtained based on the teacher model.
  • the server inputs the text into the teacher model in order of the parameter amount of each teacher model from small to large, and inputs the text sample into the student model to be trained to determine the classification result. Then, the student model to be trained is trained at least according to the pseudo-label result and classification result obtained based on the teacher model.
  • the teacher model includes a first model and a second model, and the parameter amount of the first model is less than the parameter amount of the second model. Therefore, the server first inputs the text into the first model for the first model, determines the pseudo-label result, and inputs the text sample into the student model to be trained to determine the classification result.
  • the student model to be trained is trained at least according to the pseudo-label result and classification result obtained based on the first model.
  • the text is input into the second model for the second model, and the pseudo-label result is determined, and the text sample is input into the student model to be trained after being guided by the first model to determine the classification result.
  • the student model to be trained after being guided by the first model is trained at least according to the pseudo-label result and classification result obtained based on the second model.
  • the server can select a reference model from each teacher model, and initialize the model parameters of the student model to be trained according to the model parameters of the reference model. Then train the initialized student model to be trained.
  • the server can randomly select a reference model from each teacher model, and the server can also select a model with the smallest number of parameters from each teacher model as a reference model, which is not specifically limited in this specification. For example, when the reference model selected by the server includes 12 network layers, and the student model to be trained includes 4 network layers, the server can initialize the model parameters of the student model to be trained according to the parameters of the first 4 network layers of the reference model.
  • S104 Using the trained student model as a text classification model; wherein the text classification model is used to determine the classification result of the text to be classified based on the text to be classified.
  • the server can use the trained student model as a text classification model.
  • the trained student model is the model obtained after the teacher model with increasing parameter values guides the training in step S102, that is, the model obtained after all the teacher models guide the training of the student model to be trained.
  • the text classification model is used to determine the classification result of the text to be classified based on the text to be classified.
  • the server when training a text classification model, can determine the text sample and several pre-trained teacher models. Then, according to the order of the parameter amount of each teacher model from small to large, the text sample is input into the teacher model in turn for each teacher model, the pseudo-label result is determined, and the text sample is input into the teacher model. Enter the student model to be trained, determine the classification result, and train the student model to be trained at least according to the pseudo-label result and the classification result obtained based on the teacher model. After that, the trained student model is used as a text classification model.
  • the student model is trained under the guidance of several teachers, and the guidance is gradually given in the order of the parameter amount of the teacher model from small to large, so that the student model can learn more text representations of the teacher model, and the student model gradually learns the text representation of the teacher model, thereby improving the student model's ability to represent the text and preventing the student model from forgetting the text representation of the teacher model.
  • the trained student model is used as a text classification model to improve the classification accuracy of the text classification model.
  • the student model to be trained includes a feature extraction layer and a classification layer. Therefore, in the above step S102, when the text sample is input into the student model to be trained and the classification result is determined, as shown in FIG2 , FIG2 is a schematic diagram of the structure of a student model provided in this specification, and the server can input the text sample into the feature extraction layer of the student model to be trained to determine the features corresponding to the text sample. Then, the features are input into the classification layer of the student model to be trained to determine the classification result.
  • the server can flexibly align the student model with different teacher models through independent representation, without considering the differences in representation of multiple teacher models. Therefore, when the text sample is input into the student model to be trained in the above step S102 and the classification result is determined, the server can input the text sample into the feature extraction layer of the student model to be trained to determine the feature sequence corresponding to the text sample. The feature of the position corresponding to the teacher model in the feature sequence is used as the output feature. The output feature is input into the classification layer of the student model to be trained to determine the classification result.
  • FIG3 is a schematic diagram of the feature sequence provided in this specification
  • the feature sequence in FIG3 includes features corresponding to the CLS position, the teacher position and the text position respectively
  • the text position is the position corresponding to each word or word in the text
  • the teacher position and the teacher model have a corresponding relationship
  • there are multiple teacher models to guide the student model to train and there are as many teacher positions in the feature sequence.
  • the CLS position is located at the first position of the feature sequence
  • the teacher position can be between the CLS position and the text position (i.e., the T1-Tn position in FIG3)
  • the teacher position can also be the last position of the feature sequence, i.e., behind the text position, which is not specifically limited in this specification.
  • FIG3 only takes the position corresponding to two teacher models (i.e., the first teacher position and the second teacher position in FIG3) as an example that can be between the CLS position and the other text position.
  • the features corresponding to the CLS position and the teacher position are features that characterize the text as a whole.
  • the above classification result is determined based on the features of the position corresponding to the teacher model in the feature sequence, so the student model to be trained is trained at least according to the classification result and the pseudo-label result obtained based on the teacher model.
  • the server can input the text sample into the teacher model, determine the feature sequence corresponding to the text sample, and use the feature corresponding to the specified position in the feature sequence as the text feature. Then, based on the text feature, the pseudo-labeling result is determined.
  • the feature sequence determined based on the teacher model only includes the CLS position and the features corresponding to the current position
  • the specified position is the CLS position
  • the text feature is the feature corresponding to the CLS position.
  • the server can also determine the first result based on the feature corresponding to the specified position in the feature sequence, and then train the student model to be trained based on the first result and the annotation corresponding to the text sample. Specifically, the server can use the feature corresponding to the specified position in the feature sequence as the first feature. Input the first feature into the classification layer of the student model to be trained to determine the first result. Determine the annotation corresponding to the text sample.
  • the student model to be trained is trained.
  • the specified position is the CLS position
  • the first result is used to characterize whether the sentences in the text sample are similar
  • the first result is one of similarity and dissimilarity
  • the annotation corresponding to the text sample is one of similarity and dissimilarity.
  • the server can train the student model to be trained with the goal of minimizing the difference between the pseudo-label results and the classification results obtained based on the teacher model and with the goal of minimizing the difference between the first result and the annotation.
  • the server can also determine the first task loss based on the first result and the annotation. Determine the second task loss based on the pseudo-label results and the classification results obtained based on the teacher model. Train the student model to be trained based on the first task loss and the second task loss.
  • the server when training the student model to be trained according to the first task loss and the second task loss, can weight the first task loss and the second task loss respectively according to the specified weights. Then, the student model to be trained is trained according to the weighted first task loss and the weighted second task loss.
  • the specified weights can be the pre-set weights corresponding to each task loss, for example, the weights corresponding to the first task loss and the second task loss can both be 1.
  • the server may calculate the cross entropy loss of the first result and the annotation and use it as the first task loss.
  • the server may calculate the relative entropy of the pseudo-labeled result and the classification result obtained based on the teacher model, i.e., the KL divergence (Kullback-Leibler divergence), and use it as the second task loss.
  • the server when using the teacher model to guide the student model, can train the student model to be trained according to the pseudo-label results and classification results obtained based on the teacher model.
  • the server can also train the student model to be trained according to the pseudo-label results, classification results, first results and annotations obtained based on the teacher model.
  • the specific process is consistent with the process in the above step S102, and will not be repeated here.
  • the teacher model is not the teacher model with the smallest number of parameters among the teacher models, that is, there is a teacher model with smaller parameters than the teacher model, and the teacher model with smaller parameters than the teacher model has been trained before the teacher model
  • the student model to be trained is trained.
  • the pseudo-label results and classification results obtained by the model, when training the student model to be trained, the server can determine other teacher models according to the parameter amount of the teacher model.
  • the features of the positions corresponding to other teacher models in the feature sequence are used as the second features.
  • the second feature is input into the classification layer of the student model to be trained to determine the second result.
  • the pseudo-label results corresponding to other teacher models are determined and used as other results.
  • the student model to be trained is trained to prevent the student model from forgetting the text representation learned from other teacher models.
  • the parameter amount of other teacher models is less than the parameter amount of the teacher model, so other teacher models train the student model to be trained before the teacher model.
  • the server can directly determine the pseudo-label results of other teacher models and use them as other results.
  • the pseudo-label results of other teacher models are disguised results obtained when other teacher models guide the training of the student model to be trained.
  • the second result is used to characterize whether the sentences in the text sample are similar, and the second result can be one of similarity and dissimilarity.
  • the server can at least train the student model to be trained with the goal of minimizing the difference between the pseudo-label results and classification results obtained based on the teacher model and with the goal of minimizing the difference between the second result and other results.
  • the server can also determine the second task loss based on the pseudo-label results and classification results obtained based on the teacher model. Determine the third task loss based on the second result and other results. Then train the student model to be trained at least based on the second task loss and the third task loss.
  • the process of determining the third task loss based on the second result and other results is similar to the process of determining the second task loss based on the pseudo-label results and classification results obtained based on the teacher model, and will not be repeated here.
  • the server when training the student model to be trained at least based on the second task loss and the third task loss, can weight the second task loss and the third task loss respectively according to the specified weights. Then, the student model to be trained is trained at least based on the weighted second task loss and the weighted third task loss.
  • the server can use the feature corresponding to the specified position in the feature sequence as the first feature. Input the first feature into the classification layer of the student model to be trained to determine the first result. Determine the annotation corresponding to the text sample. Train the student model to be trained based on the pseudo-labeling result, classification result, second result, other results, first result and annotation obtained based on the teacher model.
  • the server can train the student model to be trained with the goal of minimizing the difference between the pseudo-labeled results and the classification results obtained by the teacher model, minimizing the difference between the second results and other results, and minimizing the difference between the first result and the annotation.
  • the server can also determine the first task loss based on the first result and the annotation.
  • the second task loss is determined based on the labeling results and classification results.
  • the third task loss is determined based on the second result and other results.
  • the student model to be trained is then trained based on the first task loss, the second task loss, and the third task loss.
  • the server can weight the first task loss, the second task loss, and the third task loss according to the specified weights.
  • the student model to be trained is trained according to the weighted first task loss, the weighted second task loss, and the weighted third task loss.
  • each of the above teacher models is a model that has been trained with text samples and annotations corresponding to the text samples. Therefore, when pre-training several teacher models, the server determines the annotations corresponding to the text samples, and then, for each teacher model to be trained, the teacher model to be trained is trained based on the text samples and annotations.
  • the server when training the teacher model to be trained based on the text sample and the annotation, can input the text sample into the teacher model to be trained to determine the output result.
  • the teacher model to be trained is trained with the goal of minimizing the difference between the output result and the annotation.
  • the server can determine the initial student model and determine the annotation corresponding to the text sample. Then, based on the text sample and the annotation, the initial student model is trained to obtain the student model to be trained. Among them, the model parameters of the initial student model can be initialized according to the model parameters of the teacher model. The specific process is as in the above step S102, which will not be repeated here.
  • the student model to be trained can be a model trained with text samples and annotations.
  • the server uses the incompletely converged initial student model as the student model to be trained, that is, the incompletely trained initial student model is used as the student model to be trained. Therefore, the server can train the initial student model according to the output results and annotations until the specified number of times is reached, and the initial student model after the last training is used as the student model to be trained.
  • the server may not train the student model to be trained to complete convergence. Specifically, the server may only use the teacher model with the smallest parameter amount to guide the training of the student model to be trained for a preset number of times, and when the preset number of times is reached, the next teacher model of the teacher model with the smallest parameter amount is used to guide the training of the student model to be trained.
  • the preset number of times is the number of trainings pre-set by the server.
  • the server needs to train the student model to be trained to complete convergence. Specifically, how to determine when the student model to be trained is completely converged, the server can set an end condition, and when the student model to be trained meets the end condition, it is determined that the student model to be trained is completely converged. Subsequently, in the above step S104, the server can use the trained student model (that is, the completely converged student model to be trained) as a text classification model.
  • This end condition can The training times of the student model to be trained reaches a preset threshold, which can be a value preset by the server.
  • the end condition can also be that the output results of two consecutive student models to be trained are similar. Of course, it can also be any other existing end condition for determining that the model is fully converged, which is not specifically limited in this specification.
  • the server can determine the text to be classified. Input the text to be classified into the text classification model to determine the classification result of the text to be classified. According to the classification result, the text to be classified is classified.
  • the text to be classified may include at least two sentences, and the classification result is one of similarity and dissimilarity.
  • the server can determine the user's input text in response to the user's input operation. Determine the pre-stored standard text, and use the standard text and the input text as the text to be classified. Then input the text to be classified into the text classification model to determine the classification result of the text to be classified. When the classification result is similar, determine the reply text corresponding to the standard text and display it to the user.
  • the standard text is "What is the size”
  • the reply text corresponding to the standard text can be "The size is 37".
  • the standard text can be the text previously input by the user collected by the server, and the reply text corresponding to the standard text can be pre-marked by the operator or pre-collected by the server, which is not specifically limited in this specification.
  • the server can splice the standard text and the input text, and use the spliced text as the text to be classified.
  • the classification result characterizes whether the input text and the standard text are similar, and the classification result is one of similarity and dissimilarity.
  • the server can input the text to be classified into the feature extraction layer of the text classification model to determine the feature sequence.
  • the feature corresponding to the specified position in the feature sequence is determined as the specified feature, and the feature corresponding to the position of each teacher model in the feature sequence is determined as each teacher feature.
  • Each teacher feature and the specified feature are merged to obtain the classification feature.
  • the classification feature is then input into the classification layer of the text classification model to determine the classification result.
  • the server can redetermine the standard text, and then splice the redetermined standard text with the input text to obtain a new text to be classified, and continue to determine the classification result until the classification result is similar.
  • the model structures of the initial student model, the student model to be trained, and the text classification model are the same, but the models are obtained in different training stages.
  • these student models can also include a global average pooling layer, i.e., a Pooler layer, which is used for global average pooling of features. Therefore, specifically taking the process of inputting the output feature into the classification layer of the student model to be trained in the above-mentioned step S102 and determining the classification result as an example, the server can input the output feature into the Pooler layer of the student model to be trained to obtain the pooled feature. The pooled feature is then input into the classification layer of the student model to be trained to determine the classification result.
  • a Pooler layer which is used for global average pooling of features.
  • this specification also provides a corresponding training device for a text classification model, as shown in FIG4 .
  • FIG4 is a schematic diagram of a training device for a text classification model provided in this specification, which specifically includes:
  • the first determination module 200 is used to determine a text sample and a plurality of pre-trained teacher models, wherein the number of parameters of each teacher model is different;
  • the first training module 202 is used to perform, for each teacher model in order of the parameter amount of each teacher model from small to large, the following steps: inputting the text sample into the teacher model to determine a pseudo-labeling result, and inputting the text sample into a student model to be trained to determine a classification result, and training the student model to be trained at least according to the pseudo-labeling result obtained based on the teacher model and the classification result;
  • the second determination module 204 is used to use the trained student model as a text classification model; wherein the text classification model is used to determine the classification result of the text to be classified based on the text to be classified.
  • the student model to be trained includes a feature extraction layer and a classification layer; the first training module 202 is specifically used to input the text sample into the feature extraction layer of the student model to be trained, determine the feature sequence corresponding to the text sample; use the feature of the position corresponding to the teacher model in the feature sequence as the output feature; input the output feature into the classification layer of the student model to be trained, and determine the classification result.
  • the first training module 202 is specifically used to determine a first task loss based on the first result and the annotation; determine a second task loss based on the pseudo-label result obtained based on the teacher model and the classification result; and train the student model to be trained based on the first task loss and the second task loss.
  • the first training module 202 is specifically used to determine other teacher models based on the parameter quantity of the teacher model; wherein the parameter quantity of the other teacher models is smaller than the parameter quantity of the teacher model; taking the feature of the position corresponding to the other teacher models in the feature sequence as the second feature; inputting the second feature into the classification layer of the student model to be trained to determine the second result; determining the pseudo-label result corresponding to the other teacher models and using it as the other result; and training the student model to be trained at least based on the pseudo-label result obtained based on the teacher model, the classification result, the second result and the other results.
  • the first training module 202 is specifically used to take the feature corresponding to the specified position in the feature sequence as the first feature; input the first feature into the classification layer of the student model to be trained to determine a first result; determine the annotation corresponding to the text sample; and train the student model to be trained according to the pseudo-label result obtained based on the teacher model, the classification result, the second result, the other results, the first result and the annotation.
  • the first training module 202 is specifically configured to determine a first A first task loss; determining a second task loss according to the pseudo-label result obtained based on the teacher model and the classification result; determining a third task loss according to the second result and the other results; training the student model to be trained according to the first task loss, the second task loss and the third task loss.
  • the first training module 202 is specifically used to weight the first task loss, the second task loss and the third task loss respectively according to specified weights; and train the student model to be trained according to the weighted first task loss, the weighted second task loss and the weighted third task loss.
  • the device further includes: a second training module 206, configured to determine the annotation corresponding to the text sample; and for each teacher model to be trained, training the teacher model to be trained based on the text sample and the annotation.
  • a second training module 206 configured to determine the annotation corresponding to the text sample; and for each teacher model to be trained, training the teacher model to be trained based on the text sample and the annotation.
  • the first determination module 200 is further used to determine an initial student model and determine a label corresponding to the text sample; based on the text sample and the label, the initial student model is trained to obtain a student model to be trained.
  • the device also includes: an application module 208, which is used to determine the user's input text in response to the user's input operation; determine a pre-stored standard text; use the standard text and the input text as text to be classified; input the text to be classified into the text classification model to determine the classification result of the text to be classified; when the classification result is similar, determine the reply text corresponding to the standard text and display it to the user.
  • an application module 208 which is used to determine the user's input text in response to the user's input operation; determine a pre-stored standard text; use the standard text and the input text as text to be classified; input the text to be classified into the text classification model to determine the classification result of the text to be classified; when the classification result is similar, determine the reply text corresponding to the standard text and display it to the user.
  • This specification also provides a computer-readable storage medium, which stores a computer program.
  • the computer program can be used to execute the training method of the text classification model shown in FIG. 1 above.
  • FIG5 is a schematic diagram of an electronic device corresponding to FIG1 provided in this specification.
  • the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include hardware required for other services.
  • the processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the training method of the text classification model shown in FIG1 above.
  • a programmable logic device such as a field programmable gate array (FPGA) Programmable Gate Array (FPGA) is such an integrated circuit whose logic function is determined by the user's programming of the device.
  • PLD programmable logic device
  • FPGA field programmable gate array
  • FPGA Programmable Gate Array
  • HDL Hardware Description Language
  • VHDL Very-High-Speed Integrated Circuit Hardware Description Language
  • Verilog Verilog
  • the controller may be implemented in any suitable manner, for example, the controller may take the form of a microprocessor or processor and a computer readable medium storing a computer readable program code (e.g., software or firmware) executable by the (micro)processor, a logic gate, a switch, an application specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, and the memory controller may also be implemented as part of the control logic of the memory.
  • a computer readable program code e.g., software or firmware
  • the controller may be implemented in the form of a logic gate, a switch, an application specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller may be considered as a hardware component, and the means for implementing various functions included therein may also be considered as a structure within the hardware component. Or even, the means for implementing various functions may be considered as both a software module for implementing the method and a structure within the hardware component.
  • a typical implementation device is a computer.
  • the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
  • the embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may be implemented in full hardware, full software, or a combination of software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
  • computer-usable storage media including but not limited to disk storage, CD-ROM, optical storage, etc.
  • each process and/or box in the flowchart and/or block diagram, as well as the combination of the process and/or box in the flowchart and/or block diagram can be implemented by computer program instructions.
  • These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.
  • These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.
  • These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.
  • a computing device includes one or more processors (CPU), input/output interfaces, network interfaces, and memory.
  • processors CPU
  • input/output interfaces network interfaces
  • memory volatile and non-volatile memory
  • Memory may include non-permanent storage in a computer-readable medium, in the form of random access memory (RAM) and/or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
  • RAM random access memory
  • ROM read-only memory
  • flash RAM flash memory
  • Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information.
  • Information can be computer readable instructions, data structures, program modules or other data.
  • Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.
  • computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
  • this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
  • computer-usable storage media including but not limited to disk storage, CD-ROM, optical storage, etc.
  • program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types.
  • This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network.
  • program modules may be located in local and remote computer storage media, including storage devices.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Databases & Information Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

本说明书公开了一种文本分类模型的训练方法、装置、介质及电子设备，所述方法包括：确定文本样本和以及确定预先训练的若干教师模型。再按照各教师模型的参数量从小到大的顺序，依次针对每个教师模型，将文本样本输入该教师模型，确定伪标结果，以及将文本样本输入待训练的学生模型，确定分类结果，至少根据基于该教师模型得到的伪标结果以及分类结果，对待训练的学生模型进行训练。之后，将训练完成的学生模型作为文本分类模型。通过各教师指导学生模型训练，提高文本分类模型的文本表征能力和分类准确率。

Description

一种文本分类模型的训练方法、装置、介质及电子设备 技术领域
本说明书涉及计算机技术领域，尤其涉及一种文本分类模型的训练方法、装置、介质及电子设备。
背景技术
随着信息科技的发展，文本分类模型应用的越来越广泛。同时，隐私数据也受到了大众的关注。
目前，一般是基于文本数据以及文本数据对应的类别，训练文本分类模型。但是，由于文本分类模型的参数量比较大，在模型训练、模型部署以及模型推理时，耗费的时间比较长，故可以采用知识蒸馏的方式，基于已有的大型文本分类模型，训练轻量级的文本分类模型。因此，如何采用知识蒸馏的方式训练文本分类模型是一个非常重要的问题。
基于此，本说明书中提供一种文本分类模型的训练方法。
发明内容
本说明书提供一种文本分类模型的训练方法、装置、介质及电子设备，以部分的解决相关技术存在的上述问题。
本说明书采用下述技术方案。
本说明书提供一种文本分类模型的训练方法，包括：确定文本样本，以及确定预先训练的若干教师模型；其中，各教师模型的参数量不同；按照所述各教师模型的参数量从小到大的顺序，依次针对每个教师模型，执行：将所述文本样本输入该教师模型，确定伪标结果，以及将所述文本样本输入待训练的学生模型，确定分类结果，至少根据基于该教师模型得到的伪标结果以及所述分类结果，对所述待训练的学生模型进行训练；将训练完成的学生模型作为文本分类模型；其中，所述文本分类模型用于根据待分类文本，确定所述待分类文本的分类结果。
可选地，所述待训练的学生模型包括特征提取层和分类层；将所述文本样本输入待训练的学生模型，确定分类结果，具体包括：将所述文本样本输入待训练的学生模型的特征提取层，确定所述文本样本对应的特征序列；将所述特征序列中该教师模型对应的位置的特征作为输出特征；将所述输出特征输入所述待训练的学生模型的分类层，确定分类结果。
可选地，至少根据基于该教师模型得到的伪标结果以及所述分类结果，对所述待训练的学生模型进行训练，具体包括：将所述特征序列中指定位置对应的特征作为第一特征；将所述第一特征输入所述待训练的学生模型的分类层，确定第一结果；确定所述文本样本对应的标注；根据基于该教师模型得到的伪标结果、所述分类结果、所述第一结果和所述标注，对所述待训练的学生模型进行训练。
可选地，根据基于该教师模型得到的伪标结果、所述分类结果、所述第一结果和所述标注，对所述待训练的学生模型进行训练，具体包括：根据所述第一结果和所述标注，确定第一任务损失；根据基于该教师模型得到的伪标结果和所述分类结果，确定第二任务损失；根据所述第一任务损失和所述第二任务损失，对所述待训练的学生模型进行训练。
可选地，至少根据基于该教师模型得到的伪标结果以及所述分类结果，对所述待训练的学生模型进行训练，具体包括：根据该教师模型的参数量，确定其他教师模型；其中，所述其他教师模型的参数量小于该教师模型的参数量；将所述特征序列中所述其他教师模型对应的位置的特征作为第二特征；将所述第二特征输入所述待训练的学生模型的分类层，确定第二结果；确定所述其他教师模型对应的伪标结果，并作为其他结果；至少根据基于该教师模型得到的伪标结果、所述分类结果、所述第二结果和所述其他结果，对所述待训练的学生模型进行训练。
可选地，至少根据基于该教师模型得到的伪标结果、所述分类结果、所述第二结果和所述其他结果，对所述待训练的学生模型进行训练，具体包括：将所述特征序列中指定位置对应的特征作为第一特征；将所述第一特征输入所述待训练的学生模型的分类层，确定第一结果；确定所述文本样本对应的标注；根据基于该教师模型得到的伪标结果、所述分类结果、所述第二结果、所述其他结果、所述第一结果和所述标注，对所述待训练的学生模型进行训练。
可选地，根据基于该教师模型得到的伪标结果、所述分类结果、所述第二结果、所述其他结果、所述第一结果和所述标注，对所述待训练的学生模型进行训练，具体包括：根据所述第一结果和所述标注，确定第一任务损失；根据基于该教师模型得到的伪标结果和所述分类结果，确定第二任务损失；根据所述第二结果和所述其他结果，确定第三任务损失；根据所述第一任务损失、所述第二任务损失和所述第三任务损失，对所述待训练的学生模型进行训练。
可选地，根据所述第一任务损失、所述第二任务损失和所述第三任务损失，对所述待训练的学生模型进行训练，具体包括：按照指定权重，对所述第一任务损失、所述第二任务损失和所述第三任务损失分别进行加权；根据加权后的第一任务损失、加权后的第二任务损失和加权后的第三任务损失，对所述待训练的学生模型进行训练。
可选地，预先训练若干教师模型，具体包括：确定所述文本样本对应的标注；针对每一个待训练的教师模型，基于所述文本样本和所述标注，对该待训练的教师模型进行训练。
可选地，在确定文本样本之后，所述方法还包括：确定初始学生模型，以及确定所述文本样本对应的标注；基于所述文本样本和所述标注，对所述初始学生模型进行训练，得到待训练的学生模型。
可选地，所述方法还包括：响应于用户的输入操作，确定所述用户的输入文本；确定预先存储的标准文本；将所述标准文本和所述输入文本作为待分类文本；将所述待分类文本输入所述文本分类模型，确定所述待分类文本的分类结果；当所述分类结果为相似时，确定所述标准文本对应的答复文本，并显示给用户。
本说明书提供了一种文本分类模型的训练装置，包括：第一确定模块，用于确定文本样本，以及确定预先训练的若干教师模型；其中，各教师模型的参数量不同；第一训练模块，用于按照所述各教师模型的参数量从小到大的顺序，依次针对每个教师模型，执行：将所述文本样本输入该教师模型，确定伪标结果，以及将所述文本样本输入待训练的学生模型，确定分类结果，至少根据基于该教师模型得到的伪标结果以及所述分类结果，对所述待训练的学生模型进行训练；第二确定模块，用于将训练完成的学生模型作为文本分类模型；其中，所述文本分类模型用于根据待分类文本，确定所述待分类文本的分类结果。
可选地，所述待训练的学生模型包括特征提取层和分类层；所述第一训练模块具体用于，将所述文本样本输入待训练的学生模型的特征提取层，确定所述文本样本对应的特征序列；将所述特征序列中该教师模型对应的位置的特征作为输出特征；将所述输出特征输入所述待训练的学生模型的分类层，确定分类结果。
可选地，所述第一训练模块具体用于，将所述特征序列中指定位置对应的特征作为第一特征；将所述第一特征输入所述待训练的学生模型的分类层，确定第一结果；确定所述文本样本对应的标注；根据基于该教师模型得到的伪标结果、所述分类结果、所述第一结果和所述标注，对所述待训练的学生模型进行训练。
可选地，所述第一训练模块具体用于，根据所述第一结果和所述标注，确定第一任务损失；根据基于该教师模型得到的伪标结果和所述分类结果，确定第二任务损失；根据所述第一任务损失和所述第二任务损失，对所述待训练的学生模型进行训练。
可选地，所述第一训练模块具体用于，根据该教师模型的参数量，确定其他教师模型；其中，所述其他教师模型的参数量小于该教师模型的参数量；将所述特征序列中所述其他教师模型对应的位置的特征作为第二特征；将所述第二特征输入所述待训练的学 生模型的分类层，确定第二结果；确定所述其他教师模型对应的伪标结果，并作为其他结果；至少根据基于该教师模型得到的伪标结果、所述分类结果、所述第二结果和所述其他结果，对所述待训练的学生模型进行训练。
可选地，所述第一训练模块具体用于，将所述特征序列中指定位置对应的特征作为第一特征；将所述第一特征输入所述待训练的学生模型的分类层，确定第一结果；确定所述文本样本对应的标注；根据基于该教师模型得到的伪标结果、所述分类结果、所述第二结果、所述其他结果、所述第一结果和所述标注，对所述待训练的学生模型进行训练。
可选地，所述第一训练模块具体用于，根据所述第一结果和所述标注，确定第一任务损失；根据基于该教师模型得到的伪标结果和所述分类结果，确定第二任务损失；根据所述第二结果和所述其他结果，确定第三任务损失；根据所述第一任务损失、所述第二任务损失和所述第三任务损失，对所述待训练的学生模型进行训练。
可选地，所述第一训练模块具体用于，按照指定权重，对所述第一任务损失、所述第二任务损失和所述第三任务损失分别进行加权；根据加权后的第一任务损失、加权后的第二任务损失和加权后的第三任务损失，对所述待训练的学生模型进行训练。
可选地，所述装置还包括：第二训练模块，用于确定所述文本样本对应的标注；针对每一个待训练的教师模型，基于所述文本样本和所述标注，对该待训练的教师模型进行训练。
可选地，所述第一确定模块，在确定文本样本之后，还用于确定初始学生模型，以及确定所述文本样本对应的标注；基于所述文本样本和所述标注，对所述初始学生模型进行训练，得到待训练的学生模型。
可选地，所述装置还包括：应用模块，用于响应于用户的输入操作，确定所述用户的输入文本；确定预先存储的标准文本；将所述标准文本和所述输入文本作为待分类文本；将所述待分类文本输入所述文本分类模型，确定所述待分类文本的分类结果；当所述分类结果为相似时，确定所述标准文本对应的答复文本，并显示给用户。
本说明书提供了一种计算机可读存储介质，所述存储介质存储有计算机程序，所述计算机程序被处理器执行时实现上述文本分类模型的训练方法。
本说明书提供了一种电子设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述程序时实现上述文本分类模型的训练方法。
本说明书采用的上述至少一个技术方案能够达到以下有益效果：本说明书提供的文本分类模型的训练方法中，确定文本样本和以及确定预先训练的若干教师模型。再按照各教师模型的参数量从小到大的顺序，依次针对每个教师模型，将文本样本输入该教师 模型，确定伪标结果，以及将文本样本输入待训练的学生模型，确定分类结果，至少根据基于该教师模型得到的伪标结果以及分类结果，对待训练的学生模型进行训练。之后，将训练完成的学生模型作为文本分类模型。
从上述方法中可以看出，本方法训练文本分类模型时，确定文本样本和以及确定预先训练的若干教师模型。再按照各教师模型的参数量从小到大的顺序，依次针对每个教师模型，将文本样本输入该教师模型，确定伪标结果，以及将文本样本输入待训练的学生模型，确定分类结果，至少根据基于该教师模型得到的伪标结果以及分类结果，对待训练的学生模型进行训练。之后，将训练完成的学生模型作为文本分类模型。通过各教师指导学生模型训练，提高文本分类模型的文本表征能力和分类准确率。
附图说明
此处所说明的附图用来提供对本说明书的进一步理解，构成本说明书的一部分，本说明书的示意性实施例及其说明用于解释本说明书，并不构成对本说明书的不当限定。在附图中：
图1为本说明书中提供的一种文本分类模型的训练方法的流程示意图；
图2为本说明书中提供的一种学生模型的结构的示意图；
图3为本说明书中提供的特征序列的示意图；
图4为本说明书中提供的一种文本分类模型的训练装置的示意图；
图5为本说明书中提供的对应于图1的电子设备示意图。
具体实施方式
为使本说明书的目的、技术方案和优点更加清楚，下面将结合本说明书具体实施例及相应的附图对本说明书技术方案进行清楚、完整地描述。显然，所描述的实施例仅是本说明书一部分实施例，而不是全部的实施例。基于本说明书中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本说明书保护的范围。
本说明书的实施例提供了一种文本分类模型的训练方法、装置、介质及电子设备，以下结合附图，详细说明本说明书各实施例提供的技术方案。
图1为本说明书中提供的一种文本分类模型的训练方法的流程示意图，具体包括以下步骤。
S100：确定文本样本，以及确定预先训练的若干教师模型；其中，各教师模型的参 数量不同。
在本说明书中，用于训练文本分类模型的设备可以确定文本样本，以及确定预先训练的若干教师模型。其中，用于训练文本分类模型的设备可以是服务器，也可以是诸如台式电脑、笔记本电脑等电子设备。为了便于描述，下面仅以服务器为执行主体，对本说明书提供的文本分类模型的训练方法进行说明。
上述文本样本可以为预先收集的文本数据，也可以为任意已有的文本数据集中的样本数据。文本样本的标注与文本分类模型有关，当文本分类模型用于确定文本样本是否相似时，在智能客服回复场景下，文本样本可以为预先收集的用户输入的文本，该用户输入的文本可以为用户对交易物品提出的相关问题，用户可以对交易物品的尺码、尺寸、颜色、厚度和使用方法等信息提出问题，用户提出的问题可以为文本样本。比如用户输入的文本可以为“尺码是多少”，该用户输入的文本还可以为用户对交易流程提出的相关问题，用户可以对交易流程的入口、起始方、总体过程、交易工具以及结束方等过程信息提出文本，比如该用户输入的文本可以为“如何进行交易”。该文本样本至少包括两个语句，该文本样本中至少存在一个语句对应回复文本，该回复文本为预先对用户输入的文本进行标记的文本。文本样本对应的标注为相似或者不相似，标注为相似时，说明文本样本中的语句相似，标注为不相似时，说明文本样本中的语句不相似。另外，当文本分类模型用于确定文本样本的主题时，文本样本对应的标注为各种主题类型，本说明书不做具体限定。为了便于说明，下述以确定文本样本是否相似为例进行说明，下述文本样本的标注为相似和不相似中的一种。
在本说明书中，教师模型为预先训练的模型，也可以为任意已有的文本分类模型，本说明书不做具体限定。不同的教师模型的参数量不同，即各教师模型的参数量不同，但各教师模型的用途相同，比如教师模型包括第一模型和第二模型，第一模型包括12层网络层，第二模型包括24层网络层，第一模型和第二模型的参数量不同。另外，各教师模型均可以为是BERT(Bidirectional Encoder Representation from Transformers)结构的模型，当然还可以为其他结构的模型，本说明书不做具体限定。
S102：按照所述各教师模型的参数量从小到大的顺序，依次针对每个教师模型，执行：将所述文本样本输入该教师模型，确定伪标结果，以及将所述文本样本输入待训练的学生模型，确定分类结果，至少根据基于该教师模型得到的伪标结果以及所述分类结果，对所述待训练的学生模型进行训练。
服务器可以按照各教师模型的参数量从小到大的顺序，依次针对每个教师模型，执行下述过程：将文本样本输入该教师模型，确定伪标结果，以及将文本样本输入待训练的学生模型，确定分类结果，至少根据基于该教师模型得到的伪标结果以及分类结果，对待训练的学生模型进行训练。
其中，学生模型可以是BERT(Bidirectional Encoder Representation from Transformers)结构的模型。学生模型的参数量比所有教师模型的参数量小，比如学生模型包括4层网络层，分类结果和伪标结果均用于表征文本样本中的语句是否相似，分类结果可以为相似和不相似中的一种，伪标结果可以为相似和不相似中的一种。
上述在至少根据基于该教师模型得到的伪标结果以及分类结果，对待训练的学生模型进行训练时，服务器可以至少以基于该教师模型得到的伪标结果以及分类结果之间的差异最小为目标，对待训练的学生模型进行训练。
在本说明书中，服务器是按照各教师模型的参数量从小到大的顺序，依次针对每个教师模型，将文本输入该教师模型，确定伪标结果，以及将文本样本输入待训练的学生模型，确定分类结果。再至少根据基于该教师模型得到的伪标结果以及分类结果，对待训练的学生模型进行训练。例如，教师模型包括第一模型和第二模型，第一模型的参数量小于第二模型的参数量，故服务器先针对第一模型，将文本输入第一模型，确定伪标结果，以及将文本样本输入待训练的学生模型，确定分类结果。再至少根据基于第一模型得到的伪标结果以及分类结果，对待训练的学生模型进行训练。之后，再针对第二模型，将文本输入第二模型，确定伪标结果，以及将文本样本输入经过第一模型指导后的待训练的学生模型，确定分类结果。再至少根据基于第二模型得到的伪标结果以及分类结果，对经过第一模型指导后的待训练的学生模型进行训练。
另外，为了更好地训练学生模型，得到文本分类模型，服务器可以从各教师模型中，选取参照模型，根据参照模型的模型参数，对待训练的学生模型的模型参数进行初始化。再对初始化后的待训练的学生模型进行训练。其中，服务器可以从各教师模型中，随机选取参照模型，服务器也可以从各教师模型中，选择参数量最小的模型作为参照模型，本说明书不做具体限定。例如，当服务器选取到的参照模型包括12层网络层时，而待训练的学生模型包括4层网络层，服务器可以根据参照模型的前4层网络层的参数，对待训练的学生模型的模型参数进行初始化。
S104：将训练完成的学生模型作为文本分类模型；其中，所述文本分类模型用于根据待分类文本，确定所述待分类文本的分类结果。
服务器可以将训练完成的学生模型作为文本分类模型。其中，训练完成的学生模型为上述步骤S102中经过参数量从小到大的教师模型指导训练后得到的模型，即全部教师模型均指导待训练的学生模型训练后得到的模型。文本分类模型用于根据待分类文本，确定待分类文本的分类结果。
从上述方法中可以看出，本申请在训练文本分类模型时，服务器可以确定文本样本和以及确定预先训练的若干教师模型。再按照各教师模型的参数量从小到大的顺序，依次针对每个教师模型，将文本样本输入该教师模型，确定伪标结果，以及将文本样本输 入待训练的学生模型，确定分类结果，至少根据基于该教师模型得到的伪标结果以及分类结果，对待训练的学生模型进行训练。之后，将训练完成的学生模型作为文本分类模型。通过若干教师指导训练学生模型，并且是按照教师模型的参数量从小到大的顺序渐进指导，使得学生模型可以学习到更多的教师模型的文本表征，并且学生模型渐进的学习教师模型的文本表征，从而提高学生模型对文本的表征能力，避免学生模型遗忘教师模型的文本表征。将训练完成的学生模型作为文本分类模型，提高文本分类模型的分类的准确率。
在本说明书中，待训练的学生模型包括特征提取层和分类层。故在上述步骤S102将文本样本输入待训练的学生模型，确定分类结果时，如图2所示，图2为本说明书中提供的一种学生模型的结构的示意图，服务器可以将文本样本输入待训练的学生模型的特征提取层，确定文本样本对应的特征。再将特征输入待训练的学生模型的分类层，确定分类结果。
另外，服务器可以通过独立表征使学生模型与不同的教师模型灵活对齐，无需考虑多个教师模型在表征上的差异。因此，在上述步骤S102将文本样本输入待训练的学生模型，确定分类结果时，服务器可以将文本样本输入待训练的学生模型的特征提取层，确定文本样本对应的特征序列。将特征序列中该教师模型对应的位置的特征作为输出特征。将输出特征输入待训练的学生模型的分类层，确定分类结果。
其中，如图3所示，图3为本说明书中提供的特征序列的示意图，图3中的特征序列包括CLS位置、教师位置以及文本位置分别对应的特征，文本位置为文本中每一个字或者词对应的位置，教师位置与教师模型存在对应关系，有多个教师模型指导学生模型进行训练，特征序列中就存在多少个教师位置。通常CLS位置位于特征序列的首位，教师位置可以在CLS位置与文本位置(即图3中的T1～Tn位置)之间，教师位置还可以特征序列的末位，即文本位置的后面，本说明书不做具体限定。图3仅以两个教师模型对应的位置(即图3中的第一教师位置和第二教师位置)可以在CLS位置与其他文本位置之间为例。另外，CLS位置以及教师位置分别对应的特征均是表征文本整体的特征。上述分类结果是基于该教师模型在特征序列中对应的位置的特征确定的，故后续至少根据该分类结果和基于该教师模型得到的伪标结果，对待训练的学生模型进行训练。
另外，上述步骤S102中将文本样本输入该教师模型，确定伪标结果时，服务器可以将文本样本输入该教师模型，确定文本样本对应的特征序列，将特征序列中指定位置对应的特征作为文本特征。再基于文本特征，确定伪标结果。其中，由于基于教师模型确定出的特征序列中只包括CLS位置以及本位置分别对应的特征，故指定位置为CLS位置，文本特征为CLS位置对应的特征。
基于此，在上述步骤S102至少根据基于该教师模型得到的伪标结果以及分类结果， 对待训练的学生模型进行训练时，服务器除了根据基于该教师模型得到的伪标结果以及分类结果，对待训练的学生模型进行训练外，还可以根据基于特征序列中指定位置对应的特征，确定第一结果，再基于第一结果和文本样本对应的标注，对待训练的学生模型进行训练。具体的，服务器可以将特征序列中指定位置对应的特征作为第一特征。将第一特征输入待训练的学生模型的分类层，确定第一结果。确定文本样本对应的标注。然后，根据基于该教师模型得到的伪标结果、分类结果、第一结果和标注，对待训练的学生模型进行训练。其中，指定位置为CLS位置，第一结果用于表征文本样本中的语句是否相似，该第一结果为相似与不相似中的一种。文本样本对应的标注为相似与不相似中的一种。
上述在根据基于该教师模型得到的伪标结果、分类结果、第一结果和标注，对待训练的学生模型进行训练时，服务器可以以基于该教师模型得到的伪标结果以及分类结果之间的差异最小为目标和以第一结果和标注之间的差异最小为目标，对待训练的学生模型进行训练。服务器还可以根据第一结果和标注，确定第一任务损失。根据基于该教师模型得到的伪标结果和分类结果，确定第二任务损失。根据第一任务损失和第二任务损失，对待训练的学生模型进行训练。
进一步地，为了更好地平衡第一任务损失和第二任务损失，使得学生模型可以渐进的学习教师模型的文本表征，在根据第一任务损失和第二任务损失，对待训练的学生模型进行训练时，服务器可以按照指定权重，对第一任务损失和第二任务损失分别进行加权。再根据加权后的第一任务损失和加权后的第二任务损失，对待训练的学生模型进行训练。其中，指定权重可以为预先设置的各任务损失分别对应的权重，比如第一任务损失和第二任务损失分别对应的权重可以均为1。
上述在根据第一结果和标注，确定第一任务损失时，服务器可以计算第一结果和标注的交叉熵损失，并作为第一任务损失。上述在根据基于该教师模型得到的伪标结果和分类结果，确定第二任务损失时，服务器可以计算基于该教师模型得到的伪标结果和分类结果的相对熵，即KL散度(Kullback-Leibler divergence)，并作为第二任务损失。
在本说明书，若该教师模型为各教师模型中参数量最小的教师模型，那么在利用该教师模型指导学生模型进行时，服务器可以根据基于该教师模型得到的伪标结果以及分类结果，对待训练的学生模型进行训练，服务器还可以根据基于该教师模型得到的伪标结果、分类结果、第一结果和标注，对待训练的学生模型进行训练，具体过程与上述步骤S102中的过程一致，在此就不再赘述。
但是，若该教师模型不为各教师模型中参数量最小的教师模型，也就是存在比该教师模型的参数量小的教师模型，并且比该教师模型的参数量小的教师模型是在该教师模型之前，对待训练的学生模型进行训练过。故在上述步骤S102中至少根据基于该教师 模型得到的伪标结果以及分类结果，对待训练的学生模型进行训练时，服务器可以根据该教师模型的参数量，确定其他教师模型。将特征序列中其他教师模型对应的位置的特征作为第二特征。将第二特征输入待训练的学生模型的分类层，确定第二结果。确定其他教师模型对应的伪标结果，并作为其他结果。至少根据基于该教师模型得到的伪标结果、分类结果、第二结果和其他结果，对待训练的学生模型进行训练，避免学生模型遗忘从其他教师模型学习到的文本表征。其中，其他教师模型的参数量小于该教师模型的参数量，故其他教师模型在该教师模型之前对待训练的学生模型进行训练，服务器可以直接确定出其他教师模型的伪标结果，并作为其他结果，其他教师模型的伪标结果为在其他教师模型指导待训练的学生模型训练时得到的伪装结果。第二结果用于表征文本样本中的语句是否相似，该第二结果可以为相似与不相似中的一种。
上述在至少根据基于该教师模型得到的伪标结果、分类结果、第二结果和其他结果，对待训练的学生模型进行训练时，服务器可以至少以基于该教师模型得到的伪标结果以及分类结果之间的差异最小为目标和以第二结果以及其他结果之间的差异最小为目标，对待训练的学生模型进行训练。服务器还可以根据基于该教师模型得到的伪标结果以及分类结果，确定第二任务损失。根据第二结果以及其他结果，确定第三任务损失。再至少根据第二任务损失和第三任务损失，对待训练的学生模型进行训练。其中，根据第二结果以及其他结果，确定第三任务损失的过程与上述根据基于该教师模型得到的伪标结果和分类结果，确定第二任务损失的过程类似，在此就不再赘述。
进一步地，为了更好地平衡第二任务损失和第三任务损失，使得学生模型可以渐进的学习教师模型的文本表征，在至少根据第二任务损失和第三任务损失，对待训练的学生模型进行训练时，服务器可以按照指定权重，对第二任务损失和第三任务损失分别进行加权。再至少根据加权后的第二任务损失和加权后的第三任务损失，对待训练的学生模型进行训练。
另外，在至少根据基于该教师模型得到的伪标结果、分类结果、第二结果和其他结果，对待训练的学生模型进行训练时，服务器可以将特征序列中指定位置对应的特征作为第一特征。将第一特征输入待训练的学生模型的分类层，确定第一结果。确定文本样本对应的标注。根据基于该教师模型得到的伪标结果、分类结果、第二结果、其他结果、第一结果和标注，对待训练的学生模型进行训练。
其中，在根据基于该教师模型得到的伪标结果、分类结果、第二结果、其他结果、第一结果和标注，对待训练的学生模型进行训练时，服务器可以以该教师模型得到的伪标结果以及分类结果之间的差异最小为目标、以第二结果以及其他结果之间的差异最小为目标和以第一结果以及标注之间的差异最小为目标，对待训练的学生模型进行训练。服务器还可以根据第一结果和标注，确定第一任务损失。根据基于该教师模型得到的伪 标结果和分类结果，确定第二任务损失。根据第二结果和其他结果，确定第三任务损失。再根据第一任务损失、第二任务损失和第三任务损失，对待训练的学生模型进行训练。
进一步地，为了更好地平衡第一任务损失、第二任务损失和第三任务损失，使得学生模型可以渐进的学习各教师模型的文本表征，上述在根据第一任务损失、第二任务损失和第三任务损失，对待训练的学生模型进行训练时，服务器可以按照指定权重，对第一任务损失、第二任务损失和第三任务损失分别进行加权。根据加权后的第一任务损失、加权后的第二任务损失和加权后的第三任务损失，对待训练的学生模型进行训练。
在本说明书中，上述各教师模型均是经过文本样本和文本样本对应的标注训练过的模型。因此，在预先训练若干教师模型时，服务器确定文本样本对应的标注，然后。针对每一个待训练的教师模型，基于文本样本和标注，对该待训练的教师模型进行训练。
具体的，在基于文本样本和标注，对该待训练的教师模型进行训练时，服务器可以将文本样本输入该待训练的教师模型，确定输出结果。以输出结果和标注之间的差异最小为目标，对该待训练的教师模型进行训练。
在本说明书中，在上述步骤S100确定文本样本之后，服务器可以确定初始学生模型，以及确定文本样本对应的标注。然后，基于文本样本和标注，对初始学生模型进行训练，得到待训练的学生模型。其中，初始学生模型的模型参数可以根据教师模型的模型参数进行初始化，具体过程如上述步骤S102，在此就不再赘述。待训练的学生模型可以为经过文本样本和标注训练过的模型，但是，在利用文本样本和标注训练初始学生模型时，服务器是将未完全收敛的初始学生模型作为待训练的学生模型，也就是将未完全训练完成的初始学生模型作为待训练的学生模型。故服务器可以根据输出结果和标注，对初始学生模型进行训练，直到达到指定次数时，将最后一次训练后的初始学生模型作为待训练的学生模型。
另外，在上述步骤S102中每一次利用一个教师模型指导待训练的学生模型进行训练时，即上述步骤S102中每一次针对教师模型时，除了针对参数量最大的教师模型外，以针对参数量最小的教师模型为例，在利用该参数量最小的教师模型指导待训练的学生模型训练时，服务器可以不将待训练的学生模型训练到完全收敛。具体的，服务器可以仅利用该参数量最小的教师模型指导待训练的学生模型训练预设次数，达到预设次数时，利用该参数量最小的教师模型的下一个教师模型指导待训练的学生模型训练。该预设次数为服务器预先设置的训练次数。但是，在利用参数量最大的教师模型指导待训练的学生模型训练时，服务器需要将待训练的学生模型训练到完全收敛。具体如何确定待训练的学生模型何时完全收敛，服务器可以设置结束条件，当待训练的学生模型满足结束条件时，确定待训练的学生模型完全收敛，后续在上述步骤S104中，服务器可以将训练完成的学生模型(即完全收敛的待训练的学生模型)作为文本分类模型。该结束条件可 以是待训练的学生模型的训练次数达到预设阈值，该预设阈值可以为服务器预先设置的数值。该结束条件也可以是连续两次待训练的学生模型的输出结果相似，当然，还可以为其他任意已有的确定模型完全收敛的结束条件，本说明书不做具体限定。
在本说明书中，在得到文本分类模型之后，服务器可以确定待分类文本。将待分类文本输入文本分类模型，确定待分类文本的分类结果。根据分类结果，对待分类文本进行分类。其中，待分类文本可以包括至少两个语句，分类结果为相似与不相似中的一种。
在本说明书中，在智能客服回复场景中，上述在得到文本分类模型之后，服务器可以响应于用户的输入操作，确定用户的输入文本。确定预先存储的标准文本，将标准文本和输入文本作为待分类文本。再将待分类文本输入文本分类模型，确定待分类文本的分类结果。当分类结果为相似时，确定标准文本对应的回复文本，并显示给用户。其中，标准文本存在对应的回复文本，比如标准文本为“尺码是多少”，该标准文本对应的回复文本可以为“尺码是37”。标准文本可以为服务器预先收集的用户历史上输入的文本，标准文本对应的回复文本可以为操作人员预先进行标注的，也可以为服务器预先收集的，本说明书不做具体限定。上述在将标准文本和输入文本作为待分类文本时，服务器可以将标准文本和输入文本进行拼接，并将拼接后的文本作为待分类文本。分类结果表征输入文本和标准文本是否相似，该分类结果为相似和不相似中的一种。
上述在将待分类文本输入文本分类模型，确定待分类文本的分类结果时，服务器可以将待分类文本输入文本分类模型的特征提取层，确定特征序列。确定特征序列中指定位置对应的特征，并作为指定特征，以及确定特征序列中各教师模型对应的位置的特征，并作为各教师特征。将各教师特征和指定特征进行融合，得到分类特征。再将分类特征输入文本分类模型的分类层，确定分类结果。
上述在分类结果为不相似时，服务器可以重新确定标准文本，再将重新确定出的标准文本与上述输入文本进行拼接，得到新的待分类文本，继续确定分类结果，直到分类结果为相似。
在本说明书中，初始学生模型、待训练的学生模型、文本分类模的模型结构相同，只是在不同的训练阶段得到的模型。这些学生模型除了包括特征提取层和分类层外，还可以包括全局平均池化层，即Pooler层，该Pooler层用于特征进行全局平均池化。故具体的以上述步骤S102中将输出特征输入待训练的学生模型的分类层，确定分类结果的过程为例，服务器可以将输出特征输入待训练的学生模型的Pooler层，得到池化后的特征。再将池化后的特征输入待训练的学生模型的分类层，确定分类结果。
以上为本说明书的一个或多个实施例提供的文本分类模型的训练方法，基于同样的思路，本说明书还提供了相应的文本分类模型的训练装置，如图4所示。
图4为本说明书中提供的一种文本分类模型的训练装置示意图，具体包括：
第一确定模块200，用于确定文本样本，以及确定预先训练的若干教师模型；其中，各教师模型的参数量不同；
第一训练模块202，用于按照所述各教师模型的参数量从小到大的顺序，依次针对每个教师模型，执行：将所述文本样本输入该教师模型，确定伪标结果，以及将所述文本样本输入待训练的学生模型，确定分类结果，至少根据基于该教师模型得到的伪标结果以及所述分类结果，对所述待训练的学生模型进行训练；
第二确定模块204，用于将训练完成的学生模型作为文本分类模型；其中，所述文本分类模型用于根据待分类文本，确定所述待分类文本的分类结果。
可选地，所述待训练的学生模型包括特征提取层和分类层；所述第一训练模块202具体用于，将所述文本样本输入待训练的学生模型的特征提取层，确定所述文本样本对应的特征序列；将所述特征序列中该教师模型对应的位置的特征作为输出特征；将所述输出特征输入所述待训练的学生模型的分类层，确定分类结果。
可选地，所述第一训练模块202具体用于，将所述特征序列中指定位置对应的特征作为第一特征；将所述第一特征输入所述待训练的学生模型的分类层，确定第一结果；确定所述文本样本对应的标注；根据基于该教师模型得到的伪标结果、所述分类结果、所述第一结果和所述标注，对所述待训练的学生模型进行训练。
可选地，所述第一训练模块202具体用于，根据所述第一结果和所述标注，确定第一任务损失；根据基于该教师模型得到的伪标结果和所述分类结果，确定第二任务损失；根据所述第一任务损失和所述第二任务损失，对所述待训练的学生模型进行训练。
可选地，所述第一训练模块202具体用于，根据该教师模型的参数量，确定其他教师模型；其中，所述其他教师模型的参数量小于该教师模型的参数量；将所述特征序列中所述其他教师模型对应的位置的特征作为第二特征；将所述第二特征输入所述待训练的学生模型的分类层，确定第二结果；确定所述其他教师模型对应的伪标结果，并作为其他结果；至少根据基于该教师模型得到的伪标结果、所述分类结果、所述第二结果和所述其他结果，对所述待训练的学生模型进行训练。
可选地，所述第一训练模块202具体用于，将所述特征序列中指定位置对应的特征作为第一特征；将所述第一特征输入所述待训练的学生模型的分类层，确定第一结果；确定所述文本样本对应的标注；根据基于该教师模型得到的伪标结果、所述分类结果、所述第二结果、所述其他结果、所述第一结果和所述标注，对所述待训练的学生模型进行训练。
可选地，所述第一训练模块202具体用于，根据所述第一结果和所述标注，确定第 一任务损失；根据基于该教师模型得到的伪标结果和所述分类结果，确定第二任务损失；根据所述第二结果和所述其他结果，确定第三任务损失；根据所述第一任务损失、所述第二任务损失和所述第三任务损失，对所述待训练的学生模型进行训练。
可选地，所述第一训练模块202具体用于，按照指定权重，对所述第一任务损失、所述第二任务损失和所述第三任务损失分别进行加权；根据加权后的第一任务损失、加权后的第二任务损失和加权后的第三任务损失，对所述待训练的学生模型进行训练。
可选地，所述装置还包括：第二训练模块206，用于确定所述文本样本对应的标注；针对每一个待训练的教师模型，基于所述文本样本和所述标注，对该待训练的教师模型进行训练。
可选地，所述第一确定模块200，在确定文本样本之后，还用于确定初始学生模型，以及确定所述文本样本对应的标注；基于所述文本样本和所述标注，对所述初始学生模型进行训练，得到待训练的学生模型。
可选地，所述装置还包括：应用模块208，用于响应于用户的输入操作，确定所述用户的输入文本；确定预先存储的标准文本；将所述标准文本和所述输入文本作为待分类文本；将所述待分类文本输入所述文本分类模型，确定所述待分类文本的分类结果；当所述分类结果为相似时，确定所述标准文本对应的答复文本，并显示给用户。
本说明书还提供了一种计算机可读存储介质，该存储介质存储有计算机程序，计算机程序可用于执行上述图1所示的文本分类模型的训练方法。
本说明书还提供了图5所示的电子设备的示意图。如图5所述，图5为本说明书中提供的对应于图1的电子设备示意图，在硬件层面，该电子设备包括处理器、内部总线、网络接口、内存以及非易失性存储器，当然还可能包括其他业务所需要的硬件。处理器从非易失性存储器中读取对应的计算机程序到内存中然后运行，以实现上述图1所示的文本分类模型的训练方法。
当然，除了软件实现方式之外，本说明书并不排除其他实现方式，比如逻辑器件抑或软硬件结合的方式等等，也就是说以下处理流程的执行主体并不限定于各个逻辑单元，也可以是硬件或逻辑器件。
在20世纪90年代，对于一个技术的改进可以很明显地区分是硬件上的改进(例如，对二极管、晶体管、开关等电路结构的改进)还是软件上的改进(对于方法流程的改进)。然而，随着技术的发展，当今的很多方法流程的改进已经可以视为硬件电路结构的直接改进。设计人员几乎都通过将改进的方法流程编程到硬件电路中来得到相应的硬件电路结构。因此，不能说一个方法流程的改进就不能用硬件实体模块来实现。例如，可编程逻辑器件(Programmable Logic Device,PLD)(例如现场可编程门阵列(Field  Programmable Gate Array，FPGA))就是这样一种集成电路，其逻辑功能由用户对器件编程来确定。由设计人员自行编程来把一个数字系统“集成”在一片PLD上，而不需要请芯片制造厂商来设计和制作专用的集成电路芯片。而且，如今，取代手工地制作集成电路芯片，这种编程也多半改用“逻辑编译器(logic compiler)”软件来实现，它与程序开发撰写时所用的软件编译器相类似，而要编译之前的原始代码也得用特定的编程语言来撰写，此称之为硬件描述语言(Hardware Description Language，HDL)，而HDL也并非仅有一种，而是有许多种，如ABEL(Advanced Boolean Expression Language)、AHDL(Altera Hardware Description Language)、Confluence、CUPL(Cornell University Programming Language)、HDCal、JHDL(Java Hardware Description Language)、Lava、Lola、MyHDL、PALASM、RHDL(Ruby Hardware Description Language)等，目前最普遍使用的是VHDL(Very-High-Speed Integrated Circuit Hardware Description Language)与Verilog。本领域技术人员也应该清楚，只需要将方法流程用上述几种硬件描述语言稍作逻辑编程并编程到集成电路中，就可以很容易得到实现该逻辑方法流程的硬件电路。
控制器可以按任何适当的方式实现，例如，控制器可以采取例如微处理器或处理器以及存储可由该(微)处理器执行的计算机可读程序代码(例如软件或固件)的计算机可读介质、逻辑门、开关、专用集成电路(Application Specific Integrated Circuit，ASIC)、可编程逻辑控制器和嵌入微控制器的形式，控制器的例子包括但不限于以下微控制器：ARC 625D、Atmel AT91SAM、Microchip PIC18F26K20以及Silicone Labs C8051F320，存储器控制器还可以被实现为存储器的控制逻辑的一部分。本领域技术人员也知道，除了以纯计算机可读程序代码方式实现控制器以外，完全可以通过将方法步骤进行逻辑编程来使得控制器以逻辑门、开关、专用集成电路、可编程逻辑控制器和嵌入微控制器等的形式来实现相同功能。因此这种控制器可以被认为是一种硬件部件，而对其内包括的用于实现各种功能的装置也可以视为硬件部件内的结构。或者甚至，可以将用于实现各种功能的装置视为既可以是实现方法的软件模块又可以是硬件部件内的结构。
上述实施例阐明的系统、装置、模块或单元，具体可以由计算机芯片或实体实现，或者由具有某种功能的产品来实现。一种典型的实现设备为计算机。具体的，计算机例如可以为个人计算机、膝上型计算机、蜂窝电话、相机电话、智能电话、个人数字助理、媒体播放器、导航设备、电子邮件设备、游戏控制台、平板计算机、可穿戴设备或者这些设备中的任何设备的组合。
为了描述的方便，描述以上装置时以功能分为各种单元分别描述。当然，在实施本说明书时可以把各单元的功能在同一个或多个软件和/或硬件中实现。
本领域内的技术人员应明白，本发明的实施例可提供为方法、系统、或计算机程序产品。因此，本发明可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方 面的实施例的形式。而且，本发明可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本发明是参照根据本发明实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
在一个典型的配置中，计算设备包括一个或多个处理器(CPU)、输入/输出接口、网络接口和内存。
内存可能包括计算机可读介质中的非永久性存储器，随机存取存储器(RAM)和/或非易失性内存等形式，如只读存储器(ROM)或闪存(flash RAM)。内存是计算机可读介质的示例。
计算机可读介质包括永久性和非永久性、可移动和非可移动媒体可以由任何方法或技术来实现信息存储。信息可以是计算机可读指令、数据结构、程序的模块或其他数据。计算机的存储介质的例子包括，但不限于相变内存(PRAM)、静态随机存取存储器(SRAM)、动态随机存取存储器(DRAM)、其他类型的随机存取存储器(RAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、快闪记忆体或其他内存技术、只读光盘只读存储器(CD-ROM)、数字多功能光盘(DVD)或其他光学存储、磁盒式磁带，磁带磁磁盘存储或其他磁性存储设备或任何其他非传输介质，可用于存储可以被计算设备访问的信息。按照本文中的界定，计算机可读介质不包括暂存电脑可读媒体(transitory media)，如调制的数据信号和载波。
还需要说明的是，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、商品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、商品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、商品或者设备中还存在另外的相同要素。
本领域技术人员应明白，本说明书的实施例可提供为方法、系统或计算机程序产品。因此，本说明书可采用完全硬件实施例、完全软件实施例或结合软件和硬件方面的实施例的形式。而且，本说明书可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本说明书可以在由计算机执行的计算机可执行指令的一般上下文中描述，例如程序模块。一般地，程序模块包括执行特定任务或实现特定抽象数据类型的例程、程序、对象、组件、数据结构等等。也可以在分布式计算环境中实践本说明书，在这些分布式计算环境中，由通过通信网络而被连接的远程处理设备来执行任务。在分布式计算环境中，程序模块可以位于包括存储设备在内的本地和远程计算机存储介质中。
本说明书中的各个实施例均采用递进的方式描述，各个实施例之间相同相似的部分互相参见即可，每个实施例重点说明的都是与其他实施例的不同之处。尤其，对于系统实施例而言，由于其基本相似于方法实施例，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。
以上所述仅为本说明书的实施例而已，并不用于限制本说明书。对于本领域技术人员来说，本说明书可以有各种更改和变化。凡在本说明书的精神和原理之内所作的任何修改、等同替换、改进等，均应包含在本说明书的权利要求范围之内。

Claims (24)

  1. 一种文本分类模型的训练方法，包括：
    确定文本样本，以及确定预先训练的若干教师模型；其中，各教师模型的参数量不同；
    按照所述各教师模型的参数量从小到大的顺序，依次针对每个教师模型，执行：将所述文本样本输入该教师模型，确定伪标结果，以及将所述文本样本输入待训练的学生模型，确定分类结果，至少根据基于该教师模型得到的伪标结果以及所述分类结果，对所述待训练的学生模型进行训练；
    将训练完成的学生模型作为文本分类模型；其中，所述文本分类模型用于根据待分类文本，确定所述待分类文本的分类结果。
  2. 如权利要求1所述的方法，所述待训练的学生模型包括特征提取层和分类层；
    将所述文本样本输入待训练的学生模型，确定分类结果，具体包括：
    将所述文本样本输入待训练的学生模型的特征提取层，确定所述文本样本对应的特征序列；
    将所述特征序列中该教师模型对应的位置的特征作为输出特征；
    将所述输出特征输入所述待训练的学生模型的分类层，确定分类结果。
  3. 如权利要求2所述的方法，至少根据基于该教师模型得到的伪标结果以及所述分类结果，对所述待训练的学生模型进行训练，具体包括：
    将所述特征序列中指定位置对应的特征作为第一特征；
    将所述第一特征输入所述待训练的学生模型的分类层，确定第一结果；
    确定所述文本样本对应的标注；
    根据基于该教师模型得到的伪标结果、所述分类结果、所述第一结果和所述标注，对所述待训练的学生模型进行训练。
  4. 如权利要求3所述的方法，根据基于该教师模型得到的伪标结果、所述分类结果、所述第一结果和所述标注，对所述待训练的学生模型进行训练，具体包括：
    根据所述第一结果和所述标注，确定第一任务损失；
    根据基于该教师模型得到的伪标结果和所述分类结果，确定第二任务损失；
    根据所述第一任务损失和所述第二任务损失，对所述待训练的学生模型进行训练。
  5. 如权利要求2所述的方法，至少根据基于该教师模型得到的伪标结果以及所述分类结果，对所述待训练的学生模型进行训练，具体包括：
    根据该教师模型的参数量，确定其他教师模型；其中，所述其他教师模型的参数量小于该教师模型的参数量；
    将所述特征序列中所述其他教师模型对应的位置的特征作为第二特征；
    将所述第二特征输入所述待训练的学生模型的分类层，确定第二结果；
    确定所述其他教师模型对应的伪标结果，并作为其他结果；
    至少根据基于该教师模型得到的伪标结果、所述分类结果、所述第二结果和所述其他结果，对所述待训练的学生模型进行训练。
  6. 如权利要求5所述的方法，至少根据基于该教师模型得到的伪标结果、所述分类结果、所述第二结果和所述其他结果，对所述待训练的学生模型进行训练，具体包括：
    将所述特征序列中指定位置对应的特征作为第一特征；
    将所述第一特征输入所述待训练的学生模型的分类层，确定第一结果；
    确定所述文本样本对应的标注；
    根据基于该教师模型得到的伪标结果、所述分类结果、所述第二结果、所述其他结果、所述第一结果和所述标注，对所述待训练的学生模型进行训练。
  7. 如权利要求6所述的方法，根据基于该教师模型得到的伪标结果、所述分类结果、所述第二结果、所述其他结果、所述第一结果和所述标注，对所述待训练的学生模型进行训练，具体包括：
    根据所述第一结果和所述标注，确定第一任务损失；
    根据基于该教师模型得到的伪标结果和所述分类结果，确定第二任务损失；
    根据所述第二结果和所述其他结果，确定第三任务损失；
    根据所述第一任务损失、所述第二任务损失和所述第三任务损失，对所述待训练的学生模型进行训练。
  8. 如权利要求7所述的方法，根据所述第一任务损失、所述第二任务损失和所述第三任务损失，对所述待训练的学生模型进行训练，具体包括：
    按照指定权重，对所述第一任务损失、所述第二任务损失和所述第三任务损失分别进行加权；
    根据加权后的第一任务损失、加权后的第二任务损失和加权后的第三任务损失，对所述待训练的学生模型进行训练。
  9. 如权利要求1所述的方法，预先训练若干教师模型，具体包括：
    确定所述文本样本对应的标注；
    针对每一个待训练的教师模型，基于所述文本样本和所述标注，对该待训练的教师模型进行训练。
  10. 如权利要求1所述的方法，在确定文本样本之后，所述方法还包括：
    确定初始学生模型，以及确定所述文本样本对应的标注；
    基于所述文本样本和所述标注，对所述初始学生模型进行训练，得到待训练的学生模型。
  11. 如权利要求1所述的方法，所述方法还包括：
    响应于用户的输入操作，确定所述用户的输入文本；
    确定预先存储的标准文本；
    将所述标准文本和所述输入文本作为待分类文本；
    将所述待分类文本输入所述文本分类模型，确定所述待分类文本的分类结果；
    当所述分类结果为相似时，确定所述标准文本对应的答复文本，并显示给用户。
  12. 一种文本分类模型的训练装置，包括：
    第一确定模块，用于确定文本样本，以及确定预先训练的若干教师模型；其中，各教师模型的参数量不同；
    第一训练模块，用于按照所述各教师模型的参数量从小到大的顺序，依次针对每个教师模型，执行：将所述文本样本输入该教师模型，确定伪标结果，以及将所述文本样本输入待训练的学生模型，确定分类结果，至少根据基于该教师模型得到的伪标结果以及所述分类结果，对所述待训练的学生模型进行训练；
    第二确定模块，用于将训练完成的学生模型作为文本分类模型；其中，所述文本分类模型用于根据待分类文本，确定所述待分类文本的分类结果。
  13. 如权利要求12所述的装置，所述待训练的学生模型包括特征提取层和分类层；
    所述第一训练模块具体用于，将所述文本样本输入待训练的学生模型的特征提取层，确定所述文本样本对应的特征序列；将所述特征序列中该教师模型对应的位置的特征作为输出特征；将所述输出特征输入所述待训练的学生模型的分类层，确定分类结果。
  14. 如权利要求13所述的装置，所述第一训练模块具体用于，将所述特征序列中指定位置对应的特征作为第二特征；将所述第二特征输入所述待训练的学生模型的分类层，确定第二结果；确定所述文本样本对应的标注；根据基于该教师模型得到的伪标结果、所述分类结果、所述第二结果和所述标注，对所述待训练的学生模型进行训练。
  15. 如权利要求14所述的装置，所述第一训练模块具体用于，根据所述第二结果和所述标注，确定第一任务损失；根据基于该教师模型得到的伪标结果和所述分类结果，确定第二任务损失；根据所述第一任务损失和所述第二任务损失，对所述待训练的学生模型进行训练。
  16. 如权利要求13所述的装置，所述第一训练模块具体用于，根据该教师模型的参数量，确定其他教师模型；其中，所述其他教师模型的参数量小于该教师模型的参数量；将所述特征序列中所述其他教师模型对应的位置的特征作为第一特征；将所述第一特征输入所述待训练的学生模型的分类层，确定第一结果；确定所述其他教师模型对应的伪标结果，并作为其他结果；至少根据基于该教师模型得到的伪标结果、所述分类结果、所述第一结果和所述其他结果，对所述待训练的学生模型进行训练。
  17. 如权利要求16所述的装置，所述第一训练模块具体用于，将所述特征序列中指定位置对应的特征作为第二特征；将所述第二特征输入所述待训练的学生模型的分类层，确定第二结果；确定所述文本样本对应的标注；根据基于该教师模型得到的伪标结 果、所述分类结果、所述第一结果、所述其他结果、所述第二结果和所述标注，对所述待训练的学生模型进行训练。
  18. 如权利要求17所述的装置，所述第一训练模块具体用于，根据所述第二结果和所述标注，确定第一任务损失；根据所述第一结果和所述其他结果，确定第二任务损失；根据基于该教师模型得到的伪标结果和所述分类结果，确定第三任务损失；根据所述第一任务损失、所述第二任务损失和所述第三任务损失，对所述待训练的学生模型进行训练。
  19. 如权利要求18所述的装置，所述第一训练模块具体用于，按照指定权重，对所述第一任务损失、所述第二任务损失和所述第三任务损失分别进行加权；根据加权后的第一任务损失、加权后的第二任务损失和加权后的第三任务损失，对所述待训练的学生模型进行训练。
  20. 如权利要求12所述的装置，所述装置还包括：
    第二训练模块，用于确定所述文本样本对应的标注；针对每一个待训练的教师模型，基于所述文本样本和所述标注，对该待训练的教师模型进行训练。
  21. 如权利要求12所述的装置，所述第一确定模块，在确定文本样本之后，还用于确定初始学生模型，以及确定所述文本样本对应的标注；基于所述文本样本和所述标注，对所述初始学生模型进行训练，得到待训练的学生模型。
  22. 如权利要求12所述的装置，所述装置还包括：
    应用模块，用于响应于用户的输入操作，确定所述用户的输入文本；确定预先存储的标准文本；将所述标准文本和所述输入文本作为待分类文本；将所述待分类文本输入所述文本分类模型，确定所述待分类文本的分类结果；当所述分类结果为相似时，确定所述标准文本对应的答复文本，并显示给用户。
  23. 一种计算机可读存储介质，所述存储介质存储有计算机程序，所述计算机程序被处理器执行时实现上述权利要求1～11任一项所述的方法。
  24. 一种电子设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述程序时实现上述权利要求1～11任一项所述的方法。
PCT/CN2024/127362 2023-12-18 2024-10-25 一种文本分类模型的训练方法、装置、介质及电子设备 Pending WO2025130342A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202311754776.8A CN117786107A (zh) 2023-12-18 2023-12-18 一种文本分类模型的训练方法、装置、介质及电子设备
CN202311754776.8 2023-12-18

Publications (1)

Publication Number Publication Date
WO2025130342A1 true WO2025130342A1 (zh) 2025-06-26

Family

ID=90399361

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/127362 Pending WO2025130342A1 (zh) 2023-12-18 2024-10-25 一种文本分类模型的训练方法、装置、介质及电子设备

Country Status (2)

Country Link
CN (1) CN117786107A (zh)
WO (1) WO2025130342A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117786107A (zh) * 2023-12-18 2024-03-29 支付宝(杭州)信息技术有限公司 一种文本分类模型的训练方法、装置、介质及电子设备
CN118503435B (zh) * 2024-07-22 2024-10-11 浙江大学 基于知识融合的多未知领域文本分类方法、设备、介质
CN119274042A (zh) * 2024-11-29 2025-01-07 之江实验室 面向遥感场景非对称跨模态的大模型知识迁移方法和装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114626518A (zh) * 2020-12-09 2022-06-14 国际商业机器公司 使用深度聚类的知识蒸馏
CN114970540A (zh) * 2022-05-11 2022-08-30 北京百度网讯科技有限公司 训练文本审核模型的方法和装置
CN115311605A (zh) * 2022-09-29 2022-11-08 山东大学 基于近邻一致性和对比学习的半监督视频分类方法及系统
CN115546840A (zh) * 2022-10-24 2022-12-30 深圳须弥云图空间科技有限公司 基于半监督知识蒸馏的行人重识别模型训练方法及装置
CN117786107A (zh) * 2023-12-18 2024-03-29 支付宝(杭州)信息技术有限公司 一种文本分类模型的训练方法、装置、介质及电子设备

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114626518A (zh) * 2020-12-09 2022-06-14 国际商业机器公司 使用深度聚类的知识蒸馏
CN114970540A (zh) * 2022-05-11 2022-08-30 北京百度网讯科技有限公司 训练文本审核模型的方法和装置
CN115311605A (zh) * 2022-09-29 2022-11-08 山东大学 基于近邻一致性和对比学习的半监督视频分类方法及系统
CN115546840A (zh) * 2022-10-24 2022-12-30 深圳须弥云图空间科技有限公司 基于半监督知识蒸馏的行人重识别模型训练方法及装置
CN117786107A (zh) * 2023-12-18 2024-03-29 支付宝(杭州)信息技术有限公司 一种文本分类模型的训练方法、装置、介质及电子设备

Also Published As

Publication number Publication date
CN117786107A (zh) 2024-03-29

Similar Documents

Publication Publication Date Title
US11645470B2 (en) Automated testing of dialog systems
WO2025130342A1 (zh) 一种文本分类模型的训练方法、装置、介质及电子设备
CN117331561B (zh) 一种智能低代码页面开发系统及方法
US11003993B1 (en) Training recurrent neural networks to generate sequences
CN110457578B (zh) 一种客服服务需求识别方法及装置
CN113221555A (zh) 一种基于多任务模型的关键词识别方法、装置及设备
CN115545002B (zh) 一种模型训练和业务处理的方法、装置、存储介质及设备
Pan et al. Automatically generating and improving voice command interface from operation sequences on smartphones
CN113887235A (zh) 一种信息推荐的方法及装置
US20250117668A1 (en) Method for model training based on large model, question answering method, and electronic device
CN116662657A (zh) 一种模型训练和信息推荐的方法、装置、存储介质及设备
CN114116816B (zh) 一种推荐方法及装置
CN115618964A (zh) 一种模型训练的方法、装置、存储介质及电子设备
CN117743599B (zh) 一种模型检测方法、装置、介质以及电子设备
CN117422068A (zh) 信息处理方法、装置、电子设备及存储介质
CN116127305A (zh) 一种模型训练的方法、装置、存储介质及电子设备
CN111209277A (zh) 一种数据处理方法、装置、设备及介质
CN115221523A (zh) 数据处理方法、装置及设备
CN108682437B (zh) 信息处理方法、装置、介质和计算设备
CN110895924A (zh) 一种文档内容朗读方法、装置、电子设备及可读存储介质
CN112560431A (zh) 用于生成试题辅导信息的方法、装置、设备、存储介质以及计算机程序产品
CN117216271A (zh) 一种文章文本处理方法、装置以及设备
CN114428746A (zh) 一种坏例识别方法、装置、设备及存储介质
CN116089577A (zh) 一种关键词的标注方法、装置、存储介质及电子设备
CN115658891B (zh) 一种意图识别的方法、装置、存储介质及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24905830

Country of ref document: EP

Kind code of ref document: A1