CN117034926A - Model training method and device for multi-field text classification model - Google Patents

Abstract

The specification discloses a method and a device for model training of a multi-field text classification model. The text classification model can learn potential association under each classification field in the model training process, so that the text classification model can give more accurate classification results, and the text classification model comprises a plurality of sub-classification layers, so that the aim of classifying the text data in each classification field can be fulfilled after the text classification model is trained, and the training cost is greatly saved.

Description

A method and device for model training of multi-domain text classification models

Technical field

This description relates to the field of computer technology and the field of text processing, and in particular, to a method and device for model training of a multi-domain text classification model.

Background technique

With the continuous development of computer technology, artificial intelligence has been widely used in various business scenarios. It uses pre-trained models to input the business data involved in the business scenarios into the model, so as to perform business execution based on the results output by the model. , greatly improving business execution efficiency in various business scenarios.

For example, in text classification scenarios, it is usually necessary to input text data into a pre-trained text classification model. Through the output results of the text classification model, purposes such as information recommendation and security warnings can be achieved.

However, currently, text classification models are only trained using sample text data in a single classification scenario, which results in the trained text classification model usually being able to only provide a classification result in one classification scenario for the input text data. If you want to obtain classification results in multiple classification scenarios, you need to build and train the text classification models used in each classification scenario, which greatly increases the training cost. Moreover, since a piece of text data may contain content under multiple classification fields, text classification models often do not pay attention to the potential connections between various classification fields, resulting in the accuracy of the classification results given by the text classification model. lower.

Contents of the invention

This specification provides a method and device for model training of a multi-domain text classification model to partially solve the above problems existing in the prior art.

This manual adopts the following technical solutions:

This manual provides a method for model training for multi-domain text classification models, including:

Get sample text data;

Input the sample text data into the encoding layer in the text classification model to be trained, so as to encode the sample text data through the encoding layer to obtain the first encoding vector corresponding to the sample text data;

The first encoding vector is input into each expert network layer included in the text classification model, so as to determine, for each expert network layer, the first encoding vector output by the expert network layer. Two encoding vectors;

The second encoding vector output by each expert network layer is input to each sub-classification layer included in the text classification model, so that for each sub-classification layer, each sub-classification layer input to the sub-classification layer is The second coding vector output by each expert network layer is processed to obtain the classification result for the corresponding classification field of the sub-classification layer as the classification result corresponding to the sub-classification layer;

The text classification model is trained with the optimization goal of minimizing the deviation between the classification result corresponding to each sub-classification layer and the actual classification result corresponding to the sample text data in the classification field corresponding to each sub-classification layer.

Optionally, for each sub-classification layer, the text classification model also includes a weight allocation layer for the sub-classification layer;

The second encoding vector output by each expert network layer is input to each sub-classification layer included in the text classification model, so that for each sub-classification layer, each sub-classification layer input to the sub-classification layer is The second coding vector output by the expert network layer is processed to obtain the classification result for the corresponding classification field of the sub-classification layer as the classification result corresponding to the sub-classification layer, specifically including:

The second encoding vector output by each expert network layer is input to each sub-classification layer included in the text classification model, so that for each sub-classification layer, through the weight allocation layer set for the sub-classification layer, each The second coding vectors output by each expert network layer are weighted to obtain the weighted coding vector of each expert network layer, and the weighted coding vector of each expert network layer is input into the sub-classification layer to pass the sub-classification layer. Output the classification results in the corresponding classification field of the sub-classification layer as the classification results corresponding to the sub-classification layer.

Optionally, obtain sample text data, including:

Get initial sample text data;

Identify invalid words contained in the initial sample text data, and remove the invalid words from the initial sample text data to obtain transition sample text data;

The transition sample text data is intercepted according to the specified text length to obtain sample text data input into the text classification model.

Optionally, with the optimization goal of minimizing the deviation between the classification results corresponding to each sub-classification layer and the actual classification results corresponding to the sample text data in the classification field corresponding to each sub-classification layer, classify the text The model is trained, including:

For each round of training, determine some network layers from the text classification model, and fix the network parameters of the partial network layers in this round of training;

With the optimization goal of minimizing the deviation between the classification results corresponding to each sub-classification layer in this round of training and the actual classification results corresponding to the sample text data in the classification field corresponding to each sub-classification layer, the text Network parameters of other network layers included in the classification model except the partial network layer are adjusted to perform this round of training of the text classification model.

This specification provides a device for model training of multi-domain text classification models, including:

Obtain module, used to obtain sample text data;

The first encoding module is used to input the sample text data into the encoding layer in the text classification model to be trained, so as to encode the sample text data through the encoding layer, and obtain the third code corresponding to the sample text data. a coding vector;

A second encoding module, configured to input the first encoding vector into each expert network layer included in the text classification model, so as to determine, for each expert network layer, whether the expert network layer is suitable for the first a second coding vector output from one coding vector;

An output module, configured to input the second encoding vector output by each expert network layer to each sub-classification layer included in the text classification model, so that for each sub-classification layer, the pair of input to the sub-classification layer is input to the sub-classification layer through the sub-classification layer. The second encoding vector output by each expert network layer in the classification layer is processed to obtain the classification result for the corresponding classification field of the sub-classification layer as the classification result corresponding to the sub-classification layer;

The training module is used to optimize the target by minimizing the deviation between the classification results corresponding to each sub-classification layer and the actual classification results corresponding to the sample text data in the classification field corresponding to each sub-classification layer. Classification model is trained.

Optionally, for each sub-classification layer, the text classification model also includes a weight allocation layer for the sub-classification layer;

The output module is specifically configured to input the second encoding vector output by each expert network layer into each sub-classification layer included in the text classification model, so that for each sub-classification layer, by The set weight distribution layer weights the second coding vector output by each expert network layer to obtain the weighted coding vector of each expert network layer, and inputs the weighted coding vector of each expert network layer into the sub-classification layer. , the classification result for the corresponding classification field of the sub-classification layer is output through the sub-classification layer as the classification result corresponding to the sub-classification layer.

Optionally, the acquisition module is specifically configured to obtain initial sample text data; identify invalid words contained in the initial sample text data, and eliminate the invalid words from the initial sample text data, Obtain transition sample text data; intercept the transition sample text data according to the specified text length to obtain sample text data input into the text classification model.

Optionally, the training module is specifically configured to, for each round of training, determine some network layers from the text classification model, and fix the network parameters of the partial network layers in this round of training; to minimize In this round of training, the deviation between the classification results corresponding to each sub-classification layer and the actual classification results corresponding to the sample text data in the classification field corresponding to each sub-classification layer is the optimization target. For the text classification model Network parameters of other network layers included except the partial network layer are adjusted to perform this round of training of the text classification model.

This specification provides a computer-readable storage medium. The storage medium stores a computer program. When the computer program is executed by a processor, the above-mentioned method for model training of a multi-domain text classification model is implemented.

This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the above-mentioned model training for a multi-domain text classification model. Methods.

At least one of the above technical solutions adopted in this manual can achieve the following beneficial effects:

In the model training method for multi-domain text classification models provided in this manual, the text classification model includes a coding layer, multiple expert network layers and multiple sub-classification layers. Each sub-classification layer is used to give a classification respectively. Therefore, after obtaining the sample text data, the sample text data can be input into the encoding layer of the text classification model to obtain the first encoding vector corresponding to the sample text data. Then, the first encoding vector can be A coding vector is input into each expert network layer included in the text classification model, and each expert network layer provides a second coding vector. Further, the second coding vector output by each expert network layer is input into each sub-classification layer included in the text classification model, so that for each sub-classification layer, the pairs of input to the sub-classification layer are input to the sub-classification layer through the sub-classification layer. The second encoding vector output by each expert network layer is processed to obtain the classification result for the corresponding classification field of the sub-classification layer as the classification result corresponding to the sub-classification layer. Finally, the deviation between the classification results corresponding to each sub-classification layer and the actual classification results corresponding to the sample text data in the classification field corresponding to each sub-classification layer can be minimized as the optimization goal to train the text classification model.

It can be seen from the above model training method for multi-domain text classification models that since multiple expert network layers first give their respective second encoding vectors, and then the second encoding vectors given by each expert network layer are The vectors are input into each sub-classification layer respectively, and the text classification model is trained by minimizing the deviation between the classification result given by each sub-classification layer and the corresponding classification label of each sub-classification layer, which makes the During the model training process, the text classification model can learn the potential associations under each classification field, so that the text classification model can give more accurate classification results. Moreover, since the text classification model contains multiple sub-classification layers, This enables the purpose of classifying text data in various classification fields after training the text classification model, thus greatly saving training costs.

Description of the drawings

The drawings described here are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation of this specification. In the attached picture:

Figure 1 is a schematic flowchart of a method for model training of a multi-domain text classification model provided by an embodiment of this specification;

Figure 2 is a schematic network diagram of a text classification model provided by an embodiment of this specification;

Figure 3 is a schematic network diagram of a text classification model provided by an embodiment of this specification;

Figure 4 is a schematic structural diagram of a device for model training of a multi-domain text classification model provided by an embodiment of this specification;

FIG. 5 is a schematic structural diagram of an electronic device provided by an embodiment of this specification.

Detailed ways

In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only some of the embodiments of this specification, but not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of this specification.

The technical solutions provided by each embodiment of this specification will be described in detail below with reference to the accompanying drawings.

Figure 1 is a schematic flowchart of a method for model training of a multi-domain text classification model provided by an embodiment of this specification, which specifically includes the following steps:

S101: Obtain sample text data.

In the embodiments of this specification, the execution subject of the model training method provided in this specification can be a server or a terminal device such as a laptop computer or a desktop computer. For the convenience of description, only the terminal device is used as the execution subject in the following, and the description of this specification is provided. The method of model training is explained.

In order to train a model that can classify text in multiple classification fields, in this specification, the terminal device needs to obtain sample text data, where the sample text data can be obtained from a pre-constructed training sample set. The obtained sample text data can be targeted at a variety of practical application scenarios. For example, in the field of information recommendation, the user's preference classification results in business scenarios such as travel, food, entertainment, etc. can be determined through the obtained user text data. This preference classification result can be used to represent the user's preference for various business objects included in a business scenario. For example, for the business scenario of gourmet food, the preference classification results in this business scenario can be used to indicate which cuisine the user is more interested in. Furthermore, in the subsequent process, information recommendations can be made to users in various business scenarios based on the determined preference classification results. In this scenario, when the user's specific preferences are clearly known, the user's text data obtained can be used as the sample text data required for training.

For another example, in the field of information early warning, users can submit question text for the problems they encounter, and the terminal device can classify the question text through the text classification model to determine the text of the question under different classification standards. Classification results. Wherein, the classification result of the question text under a classification standard is used to indicate which question category under the classification standard the question text should specifically belong to. Furthermore, in the subsequent process, the question text can be forwarded to the staff member who handles the specified problem based on the determined classification result, and the staff member can answer the user's question. In this scenario, when the question category to which the question text belongs is clearly known, the user's question text obtained can be used as the sample text data required for training.

It should be noted that in practical applications, there are many scenarios involving text classification. The above are just examples of two specific application fields involving text classification. Other fields will not be illustrated one by one here.

Furthermore, some grammatical errors or useless words may appear in the sample text data obtained by the terminal device. In order to ensure the training efficiency and training effect of the model, the sample text data obtained can be data cleaned and Use the cleaned sample text data as training samples to train the text classification model.

Specifically, in this specification, the terminal device can obtain the initial sample text data, and then identify invalid words contained in the obtained initial sample text data, and remove the identified invalid words from the initial sample text data. are eliminated to obtain transitional sample text data.

Specifically, the terminal device may use methods such as regular processing and stop word processing to identify and eliminate invalid words contained in the initial sample text data.

Furthermore, since the text classification model to be trained often stipulates the length of the input data, the terminal device needs to intercept the above-mentioned transition sample text data according to the specified text length, thereby obtaining the sample text data input into the text classification model. .

S102: Input the sample text data into the encoding layer in the text classification model to be trained, so as to encode the sample text data through the encoding layer to obtain the first encoding vector corresponding to the sample text data.

S103: Input the first encoding vector into each expert network layer included in the text classification model, so as to determine, for each expert network layer, the output of the expert network layer for the first encoding vector. the second encoding vector.

S104: Input the second encoding vector output by each expert network layer to each sub-classification layer included in the text classification model, so that for each sub-classification layer, the pair of sub-classification layers is input into the sub-classification layer. The second encoding vector output by each expert network layer is processed to obtain the classification result for the corresponding classification field of the sub-classification layer as the classification result corresponding to the sub-classification layer.

In this specification, the text classification model contains a coding layer. The main function of this coding layer is to code the input sample text data to obtain coding vectors that can be processed by other network layers in the text classification model. In addition, the text classification model contains multiple sub-classification layers, and each sub-classification layer is regarded as corresponding to a classification field. Therefore, the text classification model provided in this manual can actually classify text data in multiple classification fields. Compared with the existing technology, only one text classification model can be trained to classify text in multiple fields, thus effectively reducing the training and maintenance costs of the model.

Furthermore, the text classification model in this specification also has several expert network layers. These expert network layers can be understood as giving respective classification suggestions for the text data input into the text classification model, and the expert network layer gives The classification suggestions will be summarized into each sub-classification layer. Each sub-classification layer combines the classification characteristics of its corresponding classification field, refers to the classification suggestions given by each expert network layer, and then outputs the corresponding classification results.

The so-called classification suggestions given by these expert network layers are actually further processed by the encoding vector output by the encoding layer. The data obtained by the processing implies what these expert network layers learned during the model training process. Classification knowledge.

Based on this, in this specification, after the terminal device inputs the obtained sample text data into the encoding layer in the text classification model to be trained, the encoding layer can encode the sample text data to obtain the sample text data corresponding to the The first encoding vector.

Then, the encoding layer will input the output first encoding vector into each expert network layer included in the text classification model, and for each expert network layer, the expert network layer will combine its own training process The learned knowledge (this knowledge is specifically realized by adjusting the network parameters in the expert network layer during the training process), further processes the first encoding vector to determine the expert network layer's response to the first encoding vector. The output second encoding vector.

Each expert network layer can further input the second coding vector output by itself into each sub-classification layer, that is, each sub-classification layer will receive the second coding vectors output by all expert network layers. For each sub-classification layer, the second coding vector output by each expert network layer input to the sub-classification layer is processed through the sub-classification layer, so that the classification under the corresponding classification field of the sub-classification layer can be obtained. The result is used as the classification result corresponding to the sub-classification layer.

As can be seen from the above, each sub-classification layer will give its own classification result based on the second encoding vector output by all expert network layers. Compared with the structure of a single expert network layer, the setting of this model structure is That is, it can ensure that the classification results output by each sub-classification layer are as accurate as possible.

S105: With the optimization goal of minimizing the deviation between the classification results corresponding to each sub-classification layer and the actual classification results corresponding to the sample text data in the classification field corresponding to each sub-classification layer, perform the optimization on the text classification model train.

After each sub-classification layer outputs a classification result, the terminal device can determine the corresponding classification result based on the deviation between the classification result corresponding to each sub-classification layer and the actual classification result corresponding to the sample text data in the classification field corresponding to each sub-classification layer. The loss value corresponding to each sub-classification layer is further added up to obtain the total loss. Finally, the terminal device can train the text classification model in a way that minimizes the total loss, as shown in Figure 2.

Figure 2 is a schematic network diagram of a text classification model provided by an embodiment of this specification.

In the text classification model shown in Figure 2, there is a coding layer, three expert network layers and three sub-classification layers. These three sub-classification layers correspond to three different classification fields. It should be pointed out that there is no strict correspondence between the number of expert network layers and the number of sub-classification layers.

After the terminal device obtains the first encoding vector of the sample text data through the encoding layer, the encoding layer will input the first encoding vector into the three expert network layers respectively. These three expert network layers will then input the second encoding vectors of various outputs into the three sub-classification layers, for example, the second encoding vectors output by expert network layer 1, expert network layer 2 and expert network layer 3 respectively. All are summarized in sub-category layer A.

For each sub-classification layer, the sub-classification layer will provide the classification results under the classification field to which it belongs, and the terminal device will use the classification results given by the sub-classification layer and the predetermined sample text data. Based on the actual classification result under the classification field to which the sub-classification layer belongs (the classification label under the classification field), the loss value corresponding to the sub-classification layer is determined. For example, based on the classification result output by sub-classification layer A, you can Determine the loss value a.

The terminal device finally adds the loss value a, loss value b and loss value c to obtain the total loss, and trains the text classification model with minimizing the total loss as the optimization goal.

It can be seen from the above that, firstly, for any sub-classification layer, the input of the sub-classification layer is given by the output of multiple expert network layers, which can ensure the accuracy of the output results of the sub-classification layer. Secondly, Since the loss values corresponding to all sub-classification layers will be summed in the end, and the text classification model is trained through the loss sum value, the text classification model can learn the differences between various classification fields during the model training process. Potential correlation, so that no matter it is the coding layer, each expert network layer or each sub-classification layer in the text classification model, the results will not be output under a single classification field. In other words, in the text classification model Each network layer will comprehensively consider the characteristics of each classification field and the potential connections between each classification field, giving more accurate and reasonable results.

Furthermore, in order to ensure the training effect of the text classification model and enable the trained text classification model to provide more accurate and reasonable classification results, in this specification, the text classification model can also be equipped with multiple weight distribution layers, and the weight distribution layer The number of layer settings may correspond to the number of sub-classification layers, that is, the same weight distribution layer as the number of sub-classification layers is set in the text classification model.

The main function of the weight distribution layer is to give the sub-classification layer the proportion (i.e. weight) of the output results of each expert network layer that should be referred to in the classification field to which it belongs, so that the sub-classification layer can give a more consistent Classification results under the classification field to which it belongs.

Specifically, after obtaining each second coding vector output by each expert network layer, the second coding vector output by each expert network layer can be input to each sub-classification layer included in the text classification model to target each sub-classification layer. The classification layer weights the second encoding vector output by each expert network layer through the weight allocation layer set for the sub-classification layer to obtain the weighted encoding vector of each expert network layer, and assigns each expert network layer The weighted coding vector is input into the sub-classification layer, so that the classification result for the corresponding classification field of the sub-classification layer is output through the sub-classification layer as the classification result corresponding to the sub-classification layer, as shown in Figure 3.

Figure 3 is a schematic network diagram of a text classification model provided by an embodiment of this specification.

As can be seen from Figure 3, the text classification model is provided with a weight distribution layer A for the sub-classification layer A. The first coding vector output by the coding layer can be input to the weight distribution layer A. The weight distribution layer A then According to the first coding vector, in the classification field corresponding to sub-category layer A, the weights of the second coding vectors output by each expert network layer are given, and then the determined weights are weighted to each The second coding vector output by the expert network layer obtains the weighted coding vector of each expert network layer, so that the sub-classification layer A can provide corresponding classification results based on the weighted coding vector of each expert network layer.

It should be pointed out that Figure 3 only shows the weight distribution layer corresponding to one sub-classification layer. In fact, in the text classification model, there are also weight distribution layers corresponding to sub-classification layer B and sub-classification layer B. The weight distribution layer corresponding to layer C is only not shown in Figure 3, and its specific function is the same as the weight distribution layer A corresponding to sub-classification layer A, so it will not be described in detail here.

Further, for the weight allocation layer under any classification field, the ability of the weight distribution layer output to refer to the weight of the second coding vector output by each expert network layer in the classification field is achieved by performing the text classification model Obtained by training, that is to say, through the above training method, it is actually necessary to continuously adjust the network parameters contained in the weight allocation layer in the text classification model, so that the weights contained in the trained text classification model The distribution layer can give accurate and reasonable weights to ensure the accuracy of the classification results output by each sub-classification layer.

It should also be noted that since the text classification model contains sub-classification layers under multiple classification fields, it may be difficult to achieve faster convergence of network parameters if conventional methods are used. For this reason, in this specification, The network parameters in some network layers can be fixed and the network parameters of the remaining networks can be adjusted to improve the training efficiency of the text classification model.

Specifically, usually, the terminal device will perform multiple rounds of iterative training methods to perform model training tasks. Then, for each round of training, some network layers can be determined from the text classification model, and these network layers can be The network parameters of are fixed in this round of training.

Then, the terminal device can minimize the deviation between the classification results corresponding to each sub-classification layer in this round of training and the actual classification results corresponding to the sample text data in the classification field corresponding to each sub-classification layer as the optimization goal, and the text The network parameters of other network layers included in the classification model except for the above-mentioned partial network layers are adjusted to perform this round of training of the text classification model.

It should be pointed out that the terminal device can fix the network parameters of more network layers in the early training rounds, and as the training rounds increase, the number of network layers that need to fix the network layer parameters in each round of training can be continuously reduced. .

It can be seen from the above method that since multiple expert network layers first provide their respective second encoding vectors, and then the second encoding vectors given by each expert network layer are input into each sub-classification layer, and passed The text classification model is trained with the goal of minimizing the deviation between the classification results given by each sub-classification layer and the corresponding classification labels of each sub-classification layer. This allows the text classification model to learn various aspects during the model training process. Potential correlations under the classification field can enable the text classification model to give more accurate classification results, and since the text classification model contains multiple sub-classification layers, this allows the text classification model to be trained. The purpose of classifying text data in various classification fields, thus greatly saving training costs.

The above is a method for model training of a multi-domain text classification model provided by one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding device for model training of a multi-domain text classification model, such as As shown in Figure 4.

Figure 4 is a schematic structural diagram of a device for model training of a multi-domain text classification model provided by an embodiment of this specification, which specifically includes:

Obtain module 401, used to obtain sample text data;

The first encoding module 402 is used to input the sample text data into the encoding layer in the text classification model to be trained, so as to encode the sample text data through the encoding layer to obtain the sample text data corresponding to first encoding vector;

The second encoding module 403 is used to input the first encoding vector into each expert network layer included in the text classification model, so as to determine, for each expert network layer, whether the expert network layer is a second encoding vector output by the first encoding vector;

The output module 404 is used to input the second encoding vector output by each expert network layer to each sub-classification layer included in the text classification model, so that for each sub-classification layer, the pair of input to the sub-classification layer is input to the The second coding vector output by each expert network layer in the sub-classification layer is processed to obtain the classification result for the corresponding classification field of the sub-classification layer as the classification result corresponding to the sub-classification layer;

The training module 405 is used to optimize the target by minimizing the deviation between the classification results corresponding to each sub-classification layer and the actual classification results corresponding to the sample text data in the classification field corresponding to each sub-classification layer. Text classification model is trained.

Optionally, for each sub-classification layer, the text classification model also includes a weight allocation layer for the sub-classification layer;

The output module 404 is specifically configured to input the second encoding vector output by each expert network layer into each sub-classification layer included in the text classification model, so as to target each sub-classification layer by targeting the sub-classification layer. The set weight distribution layer weights the second coding vector output by each expert network layer to obtain the weighted coding vector of each expert network layer, and inputs the weighted coding vector of each expert network layer into the subcategory. In the layer, the classification result for the corresponding classification field of the sub-classification layer is output through the sub-classification layer as the classification result corresponding to the sub-classification layer.

Optionally, the acquisition module 401 is specifically configured to obtain initial sample text data; identify invalid words contained in the initial sample text data, and eliminate the invalid words from the initial sample text data. , obtain transition sample text data; intercept the transition sample text data according to the specified text length to obtain sample text data input into the text classification model.

Optionally, the training module 405 is specifically configured to, for each round of training, determine some network layers from the text classification model, and fix the network parameters of the partial network layers in this round of training; with a minimum The deviation between the classification results corresponding to each sub-classification layer in this round of training and the actual classification results corresponding to the sample text data in the classification field corresponding to each sub-classification layer is the optimization goal, and the text classification model is The network parameters of other network layers included in the network layer except the partial network layer are adjusted to perform this round of training of the text classification model.

This specification also provides a computer-readable storage medium that stores a computer program. The computer program can be used to execute the method for model training of a multi-domain text classification model provided in Figure 1 above.

This specification also provides a schematic structural diagram of the electronic device shown in Figure 5. As shown in Figure 5, at the hardware level, the electronic device includes a processor, internal bus, network interface, memory and non-volatile memory, and of course may also include other hardware required for business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the method of model training for the multi-domain text classification model provided in Figure 1 above.

Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logical unit, and may also be hardware or logic device.

In the 1990s, improvements in a technology could be clearly distinguished as hardware improvements (for example, improvements in circuit structures such as diodes, transistors, switches, etc.) or software improvements (improvements in method processes). However, with the development of technology, many improvements in today's method processes can be regarded as direct improvements in hardware circuit structures. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement of a method flow cannot be implemented using hardware entity modules. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit, and its logic function is determined by the user programming the device. Designers can program themselves to "integrate" a digital system on a PLD, instead of asking chip manufacturers to design and produce dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented using "logic compiler" software, which is similar to the software compiler used in program development and writing. Before compiling, The original code must also be written in a specific programming language, which is called Hardware Description Language (HDL). There is not only one type of HDL, but many types, such as ABEL (Advanced Boolean Expression Language) , AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., the most commonly used ones currently are VHDL (Very-High-SpeedIntegrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that by simply logically programming the method flow using the above-mentioned hardware description languages and programming it into the integrated circuit, the hardware circuit that implements the logical method flow can be easily obtained.

The controller may be implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer readable medium storing computer readable program code (eg, software or firmware) executable by the (micro)processor. , logic gates, switches, Application Specific Integrated Circuit (ASIC), programmable logic controllers and embedded microcontrollers. Examples of controllers include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, the memory controller can also be implemented as part of the memory control logic. Those skilled in the art also know that in addition to implementing the controller in the form of pure computer-readable program code, the controller can be completely programmed with logic gates, switches, application-specific integrated circuits, programmable logic controllers and embedded logic by logically programming the method steps. Microcontroller, etc. to achieve the same function. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Or even, the means for implementing various functions can be considered as structures within hardware components as well as software modules implementing the methods.

The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or A combination of any of these devices.

For the convenience of description, when describing the above device, the functions are divided into various units and described separately. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and/or hardware.

Those skilled in the art will understand that embodiments of the present specification may be provided as methods, systems, or computer program products. Thus, the present description may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment that combines software and hardware aspects. Furthermore, the present description may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk memory, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The specification is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the specification. It will be understood that each process and/or block in the flowchart illustrations and/or block diagrams, and combinations of processes and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions executed by the processor of the computer or other programmable data processing device produce a use A device for realizing the functions specified in one process or multiple processes of the flowchart and/or one block or multiple blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory that causes a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction means, the instructions The device implements the functions specified in a process or processes of the flowchart and/or a block or blocks of the block diagram.

These computer program instructions may also be loaded onto a computer or other programmable data processing device, causing a series of operating steps to be performed on the computer or other programmable device to produce computer-implemented processing, thereby executing on the computer or other programmable device. Instructions provide steps for implementing the functions specified in a process or processes of a flowchart diagram and/or a block or blocks of a block diagram.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

Memory may include non-permanent storage in computer-readable media, random access memory (RAM), and/or non-volatile memory in the form of read-only memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.

Computer-readable media includes both persistent and non-volatile, removable and non-removable media that can be implemented by any method or technology for storage of information. Information may be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), and read-only memory. (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, Magnetic tape cassettes, tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium can be used to store information that can be accessed by a computing device. As defined in this article, computer-readable media does not include transitory media, such as modulated data signals and carrier waves.

It should also be noted that the terms "comprises," "comprises," or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements not only includes those elements, but also includes Other elements are not expressly listed or are inherent to the process, method, article or equipment. Without further limitation, an element defined by the statement "comprises a..." does not exclude the presence of additional identical elements in a process, method, article, or device that includes the stated element.

Those skilled in the art will appreciate that embodiments of the present specification may be provided as methods, systems, or computer program products. Thus, the present description may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment that combines software and hardware aspects. Furthermore, the present description may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk memory, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

This specification may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present description may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through communications networks. In a distributed computing environment, program modules may be located in both local and remote computer storage media including storage devices.

Each embodiment in this specification is described in a progressive manner. The same and similar parts between the various embodiments can be referred to each other. Each embodiment focuses on its differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For relevant details, please refer to the partial description of the method embodiment.

The above descriptions are only examples of this specification and are not intended to limit this specification. Various modifications and variations may occur to those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification shall be included in the scope of the claims of this specification.

Claims (10)

1. A method of model training for multi-domain text classification models, which is characterized by including: Get sample text data; Input the sample text data into the encoding layer in the text classification model to be trained, so as to encode the sample text data through the encoding layer to obtain the first encoding vector corresponding to the sample text data; The first encoding vector is input into each expert network layer included in the text classification model, so as to determine, for each expert network layer, the first encoding vector output by the expert network layer. Two encoding vectors; The second encoding vector output by each expert network layer is input to each sub-classification layer included in the text classification model, so that for each sub-classification layer, each sub-classification layer input to the sub-classification layer is The second coding vector output by each expert network layer is processed to obtain the classification result for the corresponding classification field of the sub-classification layer as the classification result corresponding to the sub-classification layer; The text classification model is trained with the optimization goal of minimizing the deviation between the classification result corresponding to each sub-classification layer and the actual classification result corresponding to the sample text data in the classification field corresponding to each sub-classification layer. 2. The method according to claim 1, characterized in that, for each sub-classification layer, the text classification model also includes a weight allocation layer for the sub-classification layer; The second encoding vector output by each expert network layer is input to each sub-classification layer included in the text classification model, so that for each sub-classification layer, each sub-classification layer input to the sub-classification layer is The second coding vector output by the expert network layer is processed to obtain the classification result for the corresponding classification field of the sub-classification layer as the classification result corresponding to the sub-classification layer, specifically including: The second encoding vector output by each expert network layer is input to each sub-classification layer included in the text classification model, so that for each sub-classification layer, through the weight allocation layer set for the sub-classification layer, each The second coding vectors output by each expert network layer are weighted to obtain the weighted coding vector of each expert network layer, and the weighted coding vector of each expert network layer is input into the sub-classification layer to pass the sub-classification layer. Output the classification results in the corresponding classification field of the sub-classification layer as the classification results corresponding to the sub-classification layer. 3. The method of claim 1, wherein obtaining sample text data specifically includes: Get initial sample text data; Identify invalid words contained in the initial sample text data, and remove the invalid words from the initial sample text data to obtain transition sample text data; The transition sample text data is intercepted according to the specified text length to obtain sample text data input into the text classification model. 4. The method of claim 1, characterized in that, to minimize the difference between the classification result corresponding to each sub-classification layer and the actual classification result corresponding to the sample text data in the classification field corresponding to each sub-classification layer. The deviation is the optimization target, and the text classification model is trained, specifically including: For each round of training, determine some network layers from the text classification model, and fix the network parameters of the partial network layers in this round of training; With the optimization goal of minimizing the deviation between the classification results corresponding to each sub-classification layer in this round of training and the actual classification results corresponding to the sample text data in the classification field corresponding to each sub-classification layer, the text Network parameters of other network layers included in the classification model except the partial network layer are adjusted to perform this round of training of the text classification model. 5. A device for model training of multi-domain text classification models, characterized by including: Obtain module, used to obtain sample text data; The first encoding module is used to input the sample text data into the encoding layer in the text classification model to be trained, so as to encode the sample text data through the encoding layer, and obtain the third code corresponding to the sample text data. a coding vector; A second encoding module, configured to input the first encoding vector into each expert network layer included in the text classification model, so as to determine, for each expert network layer, whether the expert network layer is suitable for the first a second coding vector output from one coding vector; An output module, configured to input the second encoding vector output by each expert network layer to each sub-classification layer included in the text classification model, so that for each sub-classification layer, the pair of input to the sub-classification layer is input to the sub-classification layer through the sub-classification layer. The second encoding vector output by each expert network layer in the classification layer is processed to obtain the classification result for the corresponding classification field of the sub-classification layer as the classification result corresponding to the sub-classification layer; The training module is used to optimize the target by minimizing the deviation between the classification results corresponding to each sub-classification layer and the actual classification results corresponding to the sample text data in the classification field corresponding to each sub-classification layer. Classification model is trained. 6. The device according to claim 5, wherein for each sub-classification layer, the text classification model also includes a weight allocation layer for the sub-classification layer; The output module is specifically configured to input the second encoding vector output by each expert network layer into each sub-classification layer included in the text classification model, so that for each sub-classification layer, by The set weight distribution layer weights the second coding vector output by each expert network layer to obtain the weighted coding vector of each expert network layer, and inputs the weighted coding vector of each expert network layer into the sub-classification layer. , the classification result for the corresponding classification field of the sub-classification layer is output through the sub-classification layer as the classification result corresponding to the sub-classification layer. 7. The device of claim 5, wherein the acquisition module is specifically configured to acquire initial sample text data; identify invalid words contained in the initial sample text data, and convert the invalid words into Words are eliminated from the initial sample text data to obtain transition sample text data; the transition sample text data is intercepted according to the specified text length to obtain sample text data input into the text classification model. 8. The device of claim 5, wherein the training module is specifically configured to, for each round of training, determine a partial network layer from the text classification model, and convert the network of the partial network layer The parameters are fixed in this round of training; to minimize the deviation between the classification results corresponding to each sub-classification layer in this round of training and the actual classification results corresponding to the sample text data in the classification field corresponding to each sub-classification layer. In order to optimize the target, network parameters of other network layers included in the text classification model except the partial network layer are adjusted to perform this round of training of the text classification model. 9. A computer-readable storage medium, characterized in that the storage medium stores a computer program, and when the computer program is executed by a processor, the method of any one of claims 1 to 4 is implemented. 10. An electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that when the processor executes the program, any one of claims 1 to 4 is realized. method described in the item.