CN115687610A - Text intention classification model training method, recognition device, electronic equipment and storage medium - Google Patents
Text intention classification model training method, recognition device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115687610A CN115687610A CN202211096559.XA CN202211096559A CN115687610A CN 115687610 A CN115687610 A CN 115687610A CN 202211096559 A CN202211096559 A CN 202211096559A CN 115687610 A CN115687610 A CN 115687610A
- Authority
- CN
- China
- Prior art keywords
- label
- training
- text
- intention
- labels
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention provides a text intention classification model training method, a text intention classification model recognition device, an electronic device and a storage medium, and belongs to the field of data processing, wherein the training method comprises the following steps: the method comprises the steps of obtaining a word group of each training text and a label of each training text, obtaining an initial model comprising a graph neural network and a classifier, taking all label labels as iterative input of the graph neural network so that the graph neural network learns the association relation among all intention labels, taking the word group and the association relation of the training texts as iterative input of the classifier, training the initial model, and training a classification model for obtaining the label labels of the training texts, so that the trained classification model can adapt to the application scene of data distribution change, and the classification accuracy of the classification model is greatly improved.
Description
Technical Field
The invention relates to the field of data processing, in particular to a text intention classification model training method, a text intention classification model recognition device, electronic equipment and a storage medium.
Background
Text classification algorithms are the most basic and necessary tasks in natural language processing, and are widely applied to a plurality of fields, such as sentiment analysis, news classification, question and answer system and the like. The intention identification is an important direction in text classification, is mainly applied to an intelligent question answering system, and matches a response case for a user to check by identifying the intention of the user for consulting questions.
With the development of deep learning, the intention recognition mostly adopts a classification model based on the deep learning to predict the intention. At present, there is a hierarchical multi-label text classification method, which applies predetermined association relations between different intention labels to classification model training, and then uses the trained classification model to perform intention recognition. However, in practical applications, the classification bias of the classification model is large because the data distribution varies with the data accumulation in the production environment.
Disclosure of Invention
In view of the above, the present invention provides a text intention classification model training method, a text intention classification model recognition device, an electronic device, and a storage medium, which can solve the problem of large classification deviation of the classification models currently used for intention recognition.
In order to achieve the above purpose, the embodiment of the present invention adopts the following technical solutions:
in a first aspect, an embodiment of the present invention provides a method for training a text intention classification model, where the method includes:
processing the training corpus to obtain corpus data; the corpus data comprises a word group of each training text and a label of each training text; the labeling labels comprise a plurality of label levels with upper and lower hierarchical relations and intention labels under each label level, and the labeling labels of each training text comprise the intention labels to which the training text belongs under each label level;
obtaining an initial model, wherein the initial model comprises a graph neural network and a classifier;
and taking all the labeling labels as iterative input of the graph neural network so that the graph neural network learns the association relation among all the intention labels, taking the word group and the association relation of the training text as iterative input of the classifier, training the initial model, and training a classification model for obtaining the labeling labels of the training text.
Further, the step of training the initial model by using all the label labels as iterative inputs of the graph neural network so that the graph neural network learns the association relations among all the intention labels, and using the word groups and the association relations of the training text as iterative inputs of the classifier includes:
inputting all the label tags into the graph neural network so that the graph neural network learns all the label tags and outputs a relation vector, wherein the relation vector represents the incidence relation among all the intention tags;
multiplying a word group of a training text by the relation vector, and inputting an obtained product result into the classifier so that the classifier classifies according to the product result to obtain a prediction label of the training text output by the classifier;
calculating a loss value between the predicted label and a labeled label of the training text;
if the loss value does not reach the end condition, returning to the step of inputting all the label labels into the graph neural network so that the graph neural network learns all the label labels and outputs a relation vector, wherein the relation vector represents the incidence relation among all the intention labels so as to continue to carry out iterative training on the initial model;
and if the loss value reaches the end condition, stopping iteration to obtain a classification model.
Further, the initial model further comprises a first coding model and a second coding model, the method further comprising:
inputting the word group of each training text into the first coding model to obtain a word vector of the word group;
inputting the standard label of each training text into the second coding model to obtain a label vector of the labeled label, wherein the label vector comprises a vector of each intention label in the labeled label;
the step of inputting all the label tags into the graph neural network comprises the following steps:
inputting the label vectors of all the labeled labels into the graph neural network;
said step of multiplying a set of words of a training text by said relationship vector comprises:
multiplying a word vector of a word group of one training text by the relationship vector.
In a second aspect, an embodiment of the present invention provides a text intent recognition method, where the method includes:
performing word segmentation on a target text to be recognized to obtain a target word group of the target text;
inputting the target word group into a pre-trained classification model; the classification model is obtained by training by adopting the text intention classification model training method in the first aspect;
obtaining a target label to which the target text belongs through the classification model; the target labels comprise intention labels to which the target texts belong under all label hierarchies.
Further, the step of obtaining the target label to which the target text belongs through the classification model includes:
processing the target word group through the classification model to obtain prediction data output by the classification model; wherein the prediction data comprises a probability value for each intention tag under each tag hierarchy;
and acquiring a target label to which the target text belongs from the prediction data based on the upper and lower hierarchical relation among label hierarchies.
Further, the step of obtaining a target tag to which a target text belongs from the prediction data based on a top-bottom hierarchical relationship between tag hierarchies includes:
taking the intention label with the highest probability value as a first label in a plurality of intention labels at the uppermost label level;
and selecting the intention label with the maximum probability value from a plurality of layer labels of the label layer next to the first label as a second label until selecting the intention label with the maximum probability value from the label layer of the lowest layer as a final label to obtain a target label.
In a third aspect, an embodiment of the present invention provides a text intention classification model training device, where the device includes a sample obtaining module, a model obtaining module, and a model training module;
the sample acquisition module is used for processing the training corpus to obtain corpus data; the corpus data comprises a word group of each training text and a label of each training text; the labeling labels comprise a plurality of label levels with upper and lower hierarchical relations and intention labels under each label level, and the labeling labels of each training text comprise the intention labels to which the training text belongs under each label level;
the model obtaining module is used for obtaining an initial model, and the initial model comprises a graph neural network and a classifier;
and the model training module is used for taking all the labeling labels as iterative input of the graph neural network so that the graph neural network learns the association relation among all the intention labels, taking the word group and the association relation of the training text as iterative input of the classifier, training the initial model, and training a classification model for obtaining the labeling labels of the training text.
In a fourth aspect, an embodiment of the present invention provides a text intention recognition apparatus, where the apparatus includes a word segmentation module and a recognition module;
the word segmentation module is used for segmenting a word of a target text to be recognized to obtain a target word group of the target text;
the recognition module is used for inputting the target word group into a classification model trained in advance; the classification model is obtained by training by adopting the text intention classification model training method in the first aspect;
the identification module is further used for obtaining a target label to which the target text belongs through the classification model; and the target label comprises an intention label to which the target text belongs under each label hierarchy.
In a fifth aspect, an embodiment of the present invention provides an electronic device, including a processor and a memory, where the memory stores a computer program executable by the processor, and the processor may execute the computer program to implement the text intention classification model training method according to the first aspect or the text intention recognition method according to the second aspect.
In a sixth aspect, an embodiment of the present invention provides a storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the text intention classification model training method according to the first aspect or the text intention recognition method according to the second aspect.
The text intention classification model training method, the text intention classification model identification device, the electronic equipment and the storage medium provided by the embodiment of the invention are used for processing a training corpus to obtain a word group of each training text and a label of each training text at each label level, namely a level label, so that all the labels are used as iterative input of a graph neural network in an initial model to enable the graph neural network to learn the association relation of all the intention labels, and the learned association relation and the word group of the training text are used as iterative input of a classifier in the initial model to train the initial model, so that the prediction capability of the classifier on the label of the training text is trained according to the association relation of all the intention labels optimized by the graph neural network, the trained classification model can adapt to an application scene of data distribution change, and the classification accuracy of the classification model is greatly improved.
In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.
Fig. 1 is a block diagram illustrating a text intention classification model training system according to an embodiment of the present invention.
Fig. 2 is a schematic structural diagram of a labeling label according to an embodiment of the present invention.
Fig. 3 is a flowchart illustrating one example of a text intention classification model training system according to an embodiment of the present invention.
Fig. 4 shows one of the flow diagrams of a part of the sub-steps of step S15 in fig. 3.
Fig. 5 shows a frame diagram of an initial model provided by an embodiment of the present invention.
Fig. 6 shows a second flowchart of a part of the sub-step of step S15 in fig. 3.
Fig. 7 is a flowchart illustrating a text intent recognition method according to an embodiment of the present invention.
Fig. 8 shows a schematic flow chart of a part of the sub-steps of step S22 in fig. 7.
Fig. 9 is a block diagram illustrating a text intention classification model training apparatus according to an embodiment of the present invention.
Fig. 10 is a block diagram illustrating a text intention recognition apparatus according to an embodiment of the present invention.
Fig. 11 shows a block schematic diagram of an electronic device provided by an embodiment of the present invention.
Reference numerals are as follows: 100-text intention classification model training system; 110-a server; 120-a terminal device; 130-text intention classification model training means; 140-a sample acquisition module; 150-a model acquisition module; 160-a model training module; 170-text intent recognition means; 180-word segmentation module; 190-an identification module; 200-an electronic device.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.
It is noted that relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of additional like elements in a process, method, article, or apparatus that comprises the element.
In an intelligent question-answering scene based on a knowledge graph, user intention identification directly determines whether information returned to a user by a dialogue system meets user requirements, and the information returned to the user based on an incorrect intention identification result can not meet the user requirements at all, so that user experience is very poor. With the development of deep learning, the intention recognition mostly adopts a classification model based on the deep learning to predict the intention.
In the intent classification scenario, there is a hierarchical relationship between different intent category labels. For example, among the intentions of "opening an account", "new stock", "opening material", "opening schedule", "new stock time to market" and "new stock time to pay", the two intentions of "opening material" and "opening schedule" are subordinate to the intention of "opening an account", and the two intentions of "new stock time to market" and "new stock time to pay" are subordinate to the intention of "new stock". Incidence relations such as implications and mutual exclusion exist among different intention labels, the method for obtaining the classification model by applying the incidence relations to model training is similar to a tree structure, and the method is called a hierarchical multi-label text classification method.
However, in the current hierarchical multi-label classification method, when model training is performed, the association relationship between hierarchical labels is represented by data distribution, that is, the number ratio of each label in training data is represented, and the data distribution is used as the transition probability between the labels of the upper and lower levels and is used in the model training process with a fixed transition probability. The classification model obtained by the training mode is only effective for the application scene with unchanged data distribution, and is not suitable for the scene with changed data distribution along with the continuous accumulation of data in the production environment, so that the classification deviation of the trained classification model is larger. In addition, only data distribution is adopted as the incidence relation among the labels, and the information is too single.
Based on the above consideration, embodiments of the present invention provide a text intention classification model training method and a text intention recognition method, where the trained classification model is suitable for both a scene in which data distribution is changed and a scene in which data distribution is not changed, and the classification deviation of the classification model can be reduced, so as to improve the accuracy of text intention recognition. The above method will be described below.
The text intention classification model training method provided by the invention can be applied to the text intention classification model training system 100 as shown in the figure, the text intention classification model training system 100 comprises a server 110 and a terminal device 120, and the server 110 and the terminal device 120 can communicate with each other in a wired connection or wireless connection mode or through a network.
The terminal device 120 is configured to capture the corpus and to create an initial model, and to input the corpus and the created initial model to the server 110.
Wherein the initial model comprises a graph neural network and a classifier.
The server 110 is configured to process the corpus to obtain corpus data.
The corpus data comprises a word group of each training text and a label of each training text. The labeling labels comprise a plurality of label hierarchies with upper and lower hierarchical relations and intention labels under each label hierarchy, and the labeling labels of each training text comprise the intention labels to which the training text belongs under each label hierarchy.
The server 110 is further configured to use all the labeling labels as iterative input of the graph neural network, so that the graph neural network learns association relationships among all the intention labels, use a word group and the association relationships of the training text as iterative input of a classifier, train the initial model, and train a classification model for obtaining the labeling labels of the training text.
It should be understood that the corpus data may also be obtained by processing the corpus through the terminal device 120, and transmitting the corpus data to the server 110.
In this embodiment, the label tag may be in the form of a tree structure as shown in fig. 2, where "opening" and "new stock" are both the intention tags in the first layer of tag hierarchy, and "track and field", "opening material", "opening progress", "new stock marketing time", and "new stock payment time" are the intention tags in the second layer of tag hierarchy and the intention tags in the third layer of tag hierarchy. Wherein the account opening material and the account opening progress are subordinate to the account opening. The 'account opening-account opening material' and 'new stock-new stock time to market' respectively form a label.
Based on the above, in one embodiment, referring to fig. 3, a text intention classification model training method is provided, which can be applied to the server 110 in fig. 1, and includes the following steps.
And S11, processing the training corpus to obtain corpus data.
The training expectation includes a plurality of training texts, and the corpus data includes a word group of each training text and a label tag of each training text. The labeling label comprises a plurality of label hierarchies with a top-bottom hierarchical relationship and an intention label under each label hierarchy. The labeled labels of each training text comprise the intention labels to which the training text belongs under each label level.
The annotation label can be in the form of "account opening-account opening material".
And S13, acquiring an initial model.
Wherein the initial model comprises a graph neural network and a classifier.
And S15, taking all the label labels as iterative input of the graph neural network so that the graph neural network learns the incidence relation among all the intention labels, taking the word group and the incidence relation of the training text as iterative input of a classifier, training the initial model, and training a classification model of the label labels for obtaining the training text.
In this embodiment, the association relationship includes a hierarchical relationship between the intention tags, and a transition probability matrix between tag hierarchies. And performing iterative training on the initial model based on the expected data obtained in the step S11 to obtain a classification model.
Illustratively, after processing the training prediction, the resulting corpus data may be in the form of "(S, l k ,l ki ),[[l 1 ,…,l n ],[l 11 ,…,l 1i ],[l 21 ,…,l 2i ]…]”。
Wherein [ [ l ] 1 ,…,l n ],[l 11 ,…,l 1i ],[l 21 ,…,l 2i ]…]Is a hierarchical label structure of a tree structure consisting of all intention labels, [ l ] 1 ,...,l n ]Indicating a first level of label hierarchy and an intent label therein, [ l 11 ,...,l 1i ]Representing a second level of label hierarchy and the intent labels therein. k is equal to n, S represents training text, l k An intention label, l, representing the training text S under the first level label hierarchy ki And the intention label of the training text S under the k-th label hierarchy is represented.
When the initial model is iterated each time, all the label labels are input into a Graph Neural Network (GNN) together, so that the GNN learns the hierarchical relation among all the intention labels and transition probability matrixes among label layers to construct a hierarchical label representation model. And when the initial model is iterated each time, inputting a word group of a training text and the association relation learned by the neural network of the graph in the iteration into the classifier, and outputting the prediction label of the training text by the classifier.
Through continuous iteration, the graph neural network continuously optimizes the hierarchical relationship between the intention labels and the transition probability matrix (namely the incidence relationship) between label levels, and learns the optimal transition probability matrix as far as possible, so that the prediction capability of the classifier on the labeling labels of the training text, namely the classification accuracy of the classifier can be trained according to the continuously optimized incidence relationship between the intention labels.
Compared with the existing training method of the hierarchical multi-label classification model, the text intention classification model training method can train the prediction capability of the classifier on the label of the training text according to the continuously optimized incidence relation among the intention labels (the hierarchical relation among the intention labels and the transition probability matrix among the label hierarchies), so that the trained classification model can adapt to the application scene of data distribution change, and the classification accuracy of the classification model is greatly improved.
In addition, the incidence relation among the intention labels in the text intention classification model training method provided by the embodiment of the invention comprises the hierarchical relation among the intention labels and the transition probability matrix among label hierarchies, so that the incidence information is richer, and the prediction accuracy of the classification model is improved.
Further, in a possible implementation manner, in order to make the accuracy of the trained classification model reach the required degree, therefore, a loss value can be introduced in the model training to judge the maturity of the model according to the loss value. Specifically, referring to fig. 4, the step S15 includes the following steps.
And S152, inputting all the label labels into the graph neural network so that the graph neural network learns all the label vectors and outputs the relation vectors.
Wherein, the relationship vector represents the incidence relationship among all the intention labels.
And S154, multiplying the word group of the training text by the relation vector, and inputting the obtained product result into the classifier so that the classifier classifies according to the product result to obtain the prediction label of the training text output by the classifier.
And the word group of the training text is randomly extracted from all the word groups of the training text. It should be noted that the multiplication here refers to matrix multiplication.
And S156, calculating a loss value between the predicted label and the labeled label of the training text.
And S158, judging whether the loss value reaches the ending condition. And if so, stopping iteration to obtain a classification model. If not, returning to the step S152 to continue the iterative training of the initial model.
It should be understood that the training end after the loss value reaches the end condition is only one embodiment of the training end, and in other embodiments, the training may be stopped when the number of iterations reaches a preset number, and the like. In the present embodiment, the conditions for ending training are not limited to the only conditions.
In order to enable the graph neural network to learn and establish semantic hierarchical relations among all the intention labels in iteration, the application range of the classification model is further expanded, and the accuracy is improved. In an implementation manner, the initial model provided in the embodiment of the present invention further includes a first coding model and a second coding model, and both the first coding model and the second coding model may be pre-trained AlbertTiny models, in which case, the structure of the initial model may be as shown in fig. 5.
In order to further improve the classification accuracy of the classification model, on the basis of the initial model, semantic information (language vector) of a training text and a label is introduced into model training. In more detail, referring to fig. 6, step S15 further includes the following sub-steps.
S150, inputting the word group of each training text into the first coding model to obtain the word vector of the word group.
And S151, inputting the standard label of each training text into the second coding model to obtain a label vector of the labeled label.
Wherein the tag vector comprises a vector that labels each of the intention tags in the tag.
The word vector is a vector representation of a word group of the training text and the label vector is a labeled vector representation of the training text.
In other embodiments, the first coding model and the second coding model may also be other models that can derive semantic vectors.
Exemplarily, the training text S = (w) 1 ，w 2 ，...w i ) After the first coding model AlbertTiny1 is input, the output word vector is: (e) s1 ，e s2 ，...e si )＝AlbertTiny1(w 1 ，w 2 ，...w i ). Wherein, w i Represents a word obtained by segmenting the training text S into words, (w) 1 ，w 2 ，...w i ) Set of words, e, representing the training text S s1 Meaning word w 1 Is represented by the vector of (e) s1 ，es 2 ，...e si ) Is a word group (w) 1 ，w 2 ，...w i ) I.e. word vectors.
All labeling labels (l) 1 ，l 2 ，...l i ) After inputting the second coding model AlbertTiny2, the output label vector is: (e) l1 ，e l2 ，...e li )＝AlbertTiny2(l 1 ，l 2 ，...l i ). Wherein l 1 Is a label marked, e l1 For labelling with labels l 1 I.e. the tag vector.
On this basis, the above-mentioned step S152 of "inputting all labeling labels into the graph neural network" is further implemented as "inputting all labeling label vectors of labeling labels into the graph neural network". At this time, S132 inputs all the label vectors to the neural network so that the neural network learns all the label vectors and outputs a relationship vector.
Similarly, the above step S154 of multiplying the word group and the relation vector of one training text may be further implemented as "multiplying the word vector and the relation vector of the word group of one training text". At this time, in S154, the word vector and the relation vector of the word group of one training text are multiplied, and the obtained product result is input to the classifier, so that the classifier performs classification according to the product result to obtain the prediction label of the training text output by the classifier.
Illustratively, after all label vectors are input into a Graph Neural Network (GNN), the graph neural network learns the hierarchical relationship among the intention labels and the transition probability matrix among the label hierarchies, and outputs an n-dimensional relationship vector E ln ：E ln ＝GNN(e l1 ，e l2 ，...e li ). And further will train the word vector (e) of the text S s1 ，e s2 ，...e si ) And relation vector E ln Multiplying (i.e., matrix multiplying) by a relationship vector E ln Constructing training text by a classifier as weights for intent classificationThe predicted label of the training text S predicted by the classifier is output according to the relation between the predicted label and the labeled label
Wherein the predicted prediction label of the training text S predicted by the classifier is output
Can be expressed as:
in one embodiment, a binary cross entropy loss function (BCE function) is used to calculate a loss value loss between the predicted labels and the labeled labels of the training text S.
The loss value loss can be expressed as:
After the classification model is trained, a deep learning model optimization framework can be adopted to train the model, and the optimized classification model is obtained.
Through the text intention classification model training method, the finally obtained classification model can comprise a first coding model (AlbertTiny model), a graph neural network and a classifier.
According to the text intention classification model training method provided by the embodiment of the invention, the label vectors of all labeled labels are input into the graph neural network, so that the introduction of the vectorized label vectors (intention label semantic information) of the labeled labels into the establishment process of the incidence relation among the intention labels is realized, and the incidence relation among the intention labels is established more accurately through the propagation of the intention label semantic information from the label hierarchy. Meanwhile, through a Graph Neural Network (GNN), the incidence relation between the intention labels is obtained by adopting a GNN self-learning method, and the transition probability matrix between label layers and the hierarchy relation between the intention labels are learned to obtain the optimal transition probability matrix parameters, so that the optimal incidence relation between the intention labels is learned.
According to the text intention classification model training method provided by the embodiment of the invention, the incidence relation is continuously optimized, and meanwhile, the prediction capability of the classifier on the label of the training text is trained according to the incidence relation of all the intention labels continuously optimized by the graph neural network, so that the trained classification model can adapt to the application scene of data distribution change, and the classification accuracy of the classification model is greatly improved.
In one implementation, referring to fig. 7, an embodiment of the present invention further provides a text intention recognition method, which may be applied to an electronic device, and includes the following steps.
And S21, performing word segmentation on the target text to be recognized to obtain a target word group of the target text.
And S22, inputting the target word group into a classification model trained in advance.
The classification model is obtained by training by adopting the text intention classification model training method.
After the initial model reaches the training end condition by adopting the text intention classification model training method, the server can reserve the model parameters in the last iteration and send the model with the reserved parameters to the electronic equipment. The electronic device receives the model and takes the model as a classification model.
And S23, obtaining a target label to which the target text belongs through a classification model.
The target label comprises an intention label to which the target text belongs under each label hierarchy.
Exemplarily, after the target word group is input into a pre-trained classification model, the AlbertTiny model encodes the target word group to obtain a vector representation of the target word group, wherein the vector representation includes a vector representation of each word, and further multiplies the vector representation of the target word group by an association relationship between intention labels (the association relationship at this time may be a hierarchical label representation model) constructed by a graph neural network, and further, a classifier performs classification and identification according to a product result between the vector representation of the target word group and the association relationship between the intention labels, and outputs prediction labels. And further processing the predicted label to obtain the target label of the target text.
Wherein, the final target label can be in the form of 'account opening-account opening material' and the like.
It should be noted that, when the classification model includes the AlbertTiny model, the word segmentation function of the AlbertTiny model may be directly adopted to segment the target text. At this time, the target text is directly input into the classification model.
Through the steps S21-S23, the intention of the target text is identified by using the classification model which has small classification deviation and is suitable for various scenes, so that the target label deviation is small and more accurate. Meanwhile, the use experience of the user can be improved. Meanwhile, the target label comprises the intention label of the target text under each label level, so that the obtained target label is more accurate.
Further, in order to select an optimal intention tag group from the prediction data as a target tag, a tag hierarchical relationship may be introduced in the acquisition process of the target tag. Referring to fig. 8, the above-described step S22 may include the following sub-steps.
And S221, processing the target word group through the classification model to obtain prediction data output by the classification model.
Wherein the prediction data comprises a probability value for each intention tag under each tag hierarchy.
S222, acquiring a target label of the target text from the prediction data based on the upper and lower hierarchical relation among label hierarchies.
After the predicted data is obtained, the target label can be obtained by processing from the uppermost label layer in the predicted data. Illustratively, among the intention tags of the uppermost tag hierarchy, the intention tag having the highest probability value is used as a first tag, and then, among the intention tags of the next tag hierarchy from the first tag, the intention tag having the highest probability value is selected as a second tag, and the remaining prediction data is processed in the same principle until the intention tag having the highest probability value is selected as a last tag from the lowermost tag hierarchy, and the selected first tag, second tag, \8230;, and last tag are used as target tags.
For example, if the predicted data is [ [ Account-0.9, new stock-0.1, [ Account Material-0.9, account progress-0.1 ], then the final target label is "Account-Account progress".
Based on the above concept of the text intention classification model training method, in one embodiment, referring to fig. 9, a text intention classification model training apparatus 130 is provided, which includes a sample obtaining module 140, a model obtaining module 150 and a model training module 160.
The sample obtaining module 140 is configured to process the corpus to obtain corpus data.
The corpus data comprises a word group of each training text and a label of each training text. The labeling labels comprise a plurality of label hierarchies with upper and lower hierarchical relations and intention labels under each label hierarchy, and the labeling labels of each training text comprise the intention labels to which the training text belongs under each label hierarchy.
And a model obtaining module 150, configured to obtain an initial model.
The initial model includes a graph neural network and a classifier.
And the model training module 160 is configured to use all the labeled labels as iterative inputs of the graph neural network, so that the graph neural network learns the association relationships among all the intention labels, use the word groups and the association relationships of the training text as iterative inputs of a classifier, train the initial model, and train a classification model for obtaining labeled labels of the training text.
Through the text intention classification model training device 130, the relevance relation of all intention labels continuously optimized by a graph neural network and the word group of the training text are trained through the synergistic action of the sample acquisition module 140 and the model training module 160, so that the prediction capability of the classifier on the labeling labels of the training text is trained, the trained classification model can adapt to the application scene of data distribution change, and the classification accuracy of the classification model is greatly improved.
For the specific definition of the text intention classification model training device 130, reference may be made to the above definition of the text intention classification model training method, which is not described herein again. The various modules in the text intention classification model training device 130 described above may be implemented in whole or in part by software, hardware, and combinations thereof. The modules may be embedded in a hardware form or embedded in a processor of the electronic device 200 or stored in a memory of the electronic device 200 in a software form, so that the processor calls and executes operations corresponding to the modules.
Based on the concept of the text intention recognition method, in one embodiment, referring to fig. 10, a text intention recognition apparatus 170 is provided, which includes a word segmentation module 180 and a recognition module 190.
And the word segmentation module 180 is configured to perform word segmentation on the target text to be recognized to obtain a target word group of the target text.
And the recognition module 190 is configured to input the target word group into a pre-trained classification model.
The classification model is obtained by training the text intention classification model training method.
The identifying module 190 is further configured to obtain a target tag to which the target text belongs through the classification model.
The target tags include the intention tags to which the target text belongs under each tag hierarchy.
In the text intention recognition device 170, through the combined action of the classification module and the recognition module 190, the intention recognition is performed on the target text by using the classification model which has small classification deviation and is suitable for various scenes, so that the target label deviation is small and more accurate. Meanwhile, the use experience of the user can be improved. Meanwhile, the target label comprises the intention label of the target text under each label level, so that the obtained target label is more accurate.
The specific definition of the text intent recognition device 170 can be referred to the definition of the text intent recognition method above, and will not be described herein. The various modules in the text intent recognition device 170 described above may be implemented in whole or in part by software, hardware, and combinations thereof. The modules may be embedded in a hardware form or may be independent of a processor in the electronic device, or may be stored in a memory of the electronic device in a software form, so that the processor calls and executes operations corresponding to the modules.
In one embodiment, an electronic device 200 is provided, the electronic device 200 may be the server 110, and the internal structure thereof may be as shown in fig. 11. The electronic device 200 includes a processor, memory, and network interfaces connected by a system bus. Wherein the processor of the electronic device 200 is configured to provide computing and control capabilities. The memory of the electronic device 200 includes a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a database, and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the electronic device 200 is used to store text intention classification model training data or text intention recognition data. The network interface of the electronic device 200 is used for network connection communication with an external terminal. The computer program is executed by a processor to implement a text intent classification model training method, or to implement a text intent recognition method.
It will be understood by those skilled in the art that the structure shown in fig. 11 is a block diagram of only a portion of the structure associated with the inventive arrangements, and does not constitute a limitation on the electronic device 200 to which the inventive arrangements are applied, and that a particular electronic device 200 may include more or fewer components than those shown in fig. 11, or may combine certain components, or have a different arrangement of components.
In one embodiment, the text intention classification model training apparatus 130 provided by the present invention can be implemented in the form of a computer program, and the computer program can be run on the electronic device 200 shown in fig. 11. The memory of the electronic device 200 may store various program modules constituting the text intention classification model training apparatus 130, such as the sample acquisition module 140, the model acquisition module 150, and the model training module 160 shown in fig. 9. The program modules constitute a computer program that causes a processor to perform the steps described in this specification as applied to a text intention classification model training method.
For example, the electronic device 200 shown in fig. 11 may execute step S11 through the sample obtaining module 140 in the text intention classification model training apparatus 130 shown in fig. 9. The electronic device 200 may perform step S13 through the model acquisition module 150. The electronic device 200 may perform step S15 through the model training module 160.
In one embodiment, an electronic device 200 is provided, comprising a memory storing a computer program and a processor implementing the following steps when the processor executes the computer program: processing the training corpus to obtain corpus data; the corpus data comprises a word group of each training text and a label of each training text; the labeling labels comprise a plurality of label levels with upper and lower hierarchical relations and intention labels under each label level, and the labeling labels of each training text comprise the intention labels of the training text under each label level; obtaining an initial model, wherein the initial model comprises a graph neural network and a classifier; and taking all the labeled labels as iterative input of the graph neural network so that the graph neural network learns the incidence relation among all the intention labels, taking a word group and the incidence relation of the training text as iterative input of a classifier, training the initial model, and training a classification model of the labeled labels for obtaining the training text.
In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which when executed by a processor performs the steps of: processing the training corpus to obtain corpus data; the corpus data comprises a word group of each training text and a label of each training text; the labeling labels comprise a plurality of label levels with upper and lower hierarchical relations and intention labels under each label level, and the labeling labels of each training text comprise the intention labels to which the training text belongs under each label level; obtaining an initial model, wherein the initial model comprises a graph neural network and a classifier; and taking all the labeled labels as iterative input of the graph neural network so that the graph neural network learns the incidence relation among all the intention labels, taking the word group and the incidence relation of the training text as iterative input of a classifier, training the initial model, and training a classification model of the labeled labels for obtaining the training text.
In one embodiment, the text intention recognition apparatus 170 provided by the present invention can be implemented in the form of a computer program, and the computer program can be run on the electronic device 200 shown in fig. 11. The memory of the electronic device 200 may store various program modules constituting the text intention classification model training apparatus 130, such as the word segmentation module 180 and the recognition module 190 shown in fig. 10. The respective program modules constitute a computer program that causes a processor to execute the steps described in this specification applied to the text intention recognition method.
For example, the electronic device 200 shown in fig. 11 may perform step S21 through the segmentation module 180 in the text intention recognition apparatus 170 as shown in fig. 10. The electronic device 200 may perform step S22 and step S23 through the recognition module 190.
In one embodiment, an electronic device 200 is provided, comprising a memory storing a computer program and a processor implementing the following steps when the processor executes the computer program: performing word segmentation on a target text to be recognized to obtain a target word group of the target text; inputting the target word group into a pre-trained classification model; the classification model is obtained by training by adopting the text intention classification model training method; obtaining a target label to which the target text belongs through a classification model; the target label comprises an intention label to which the target text belongs under each label hierarchy.
In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which when executed by a processor performs the steps of: performing word segmentation on a target text to be recognized to obtain a target word group of the target text; inputting the target word group into a pre-trained classification model; the classification model is obtained by training by adopting the text intention classification model training method; obtaining a target label to which the target text belongs through a classification model; the target labels comprise intention labels to which the target texts belong under all label hierarchies.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, the functional modules in the embodiments of the present invention may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk, and various media capable of storing program codes.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A text intention classification model training method is characterized by comprising the following steps:
processing the training corpus to obtain corpus data; the corpus data comprises a word group of each training text and a label of each training text; the labeling labels comprise a plurality of label levels with upper and lower hierarchical relations and intention labels under each label level, and the labeling labels of each training text comprise the intention labels of the training texts under each label level;
obtaining an initial model, wherein the initial model comprises a graph neural network and a classifier;
and taking all the labeling labels as iterative input of the graph neural network so that the graph neural network learns the association relation among all the intention labels, taking the word group and the association relation of the training text as iterative input of the classifier, training the initial model, and training a classification model for obtaining the labeling labels of the training text.
2. The method for training the text intention classification model according to claim 1, wherein the step of training the initial model by using all the label labels as iterative inputs of the graph neural network so that the graph neural network learns the association relations among all the intention labels, and using the word groups and the association relations of the training text as iterative inputs of the classifier comprises:
inputting all the label labels into the graph neural network so that the graph neural network learns all the label labels and outputs a relation vector, wherein the relation vector represents the incidence relation among all the intention labels;
multiplying a word group of a training text by the relation vector, and inputting an obtained product result into the classifier so that the classifier classifies according to the product result to obtain a prediction label of the training text output by the classifier;
calculating a loss value between the predicted label and a labeled label of the training text;
if the loss value does not reach the end condition, returning to the step of inputting all the label labels into the graph neural network so that the graph neural network learns all the label labels and outputs a relation vector, wherein the relation vector represents the incidence relation among all the intention labels so as to continue to carry out iterative training on the initial model;
and if the loss value reaches the end condition, stopping iteration to obtain a classification model.
3. The method of claim 2, wherein the initial model further comprises a first coding model and a second coding model, the method further comprising:
inputting the word group of each training text into the first coding model to obtain a word vector of the word group;
inputting the standard label of each training text into the second coding model to obtain a label vector of the labeled label, wherein the label vector comprises a vector of each intention label in the labeled label;
the step of inputting all the label tags into the graph neural network comprises the following steps:
inputting the label vectors of all the labeled labels into the graph neural network;
said step of multiplying a set of words of a training text by said relationship vector comprises:
multiplying a word vector of a word group of a training text with the relationship vector.
4. A method of textual intent recognition, the method comprising:
performing word segmentation on a target text to be recognized to obtain a target word group of the target text;
inputting the target word group into a pre-trained classification model; the classification model is obtained by training by adopting the text intention classification model training method according to any one of claims 1 to 3;
obtaining a target label to which the target text belongs through the classification model; the target labels comprise intention labels to which the target texts belong under all label hierarchies.
5. The method according to claim 4, wherein the step of obtaining the target label to which the target text belongs by the classification model comprises:
processing the target word group through the classification model to obtain prediction data output by the classification model; wherein the prediction data comprises a probability value for each intention tag under each tag hierarchy;
and acquiring a target label to which the target text belongs from the prediction data based on the upper and lower hierarchical relation among label layers.
6. The method according to claim 5, wherein the step of obtaining the target tag to which the target text belongs from the prediction data based on the upper and lower hierarchical relationships between tag hierarchies comprises:
taking the intention label with the highest probability value as a first label in a plurality of intention labels at the uppermost label level;
and selecting the intention label with the maximum probability value from a plurality of label layers of the label layer below the first label as a second label until selecting the intention label with the maximum probability value from the label layer at the lowest layer as a final label to obtain a target label.
7. A text intention classification model training device is characterized by comprising a sample acquisition module, a model acquisition module and a model training module;
the sample acquisition module is used for processing the training corpus to obtain corpus data; the corpus data comprises a word group of each training text and a label of each training text; the labeling labels comprise a plurality of label levels with upper and lower hierarchical relations and intention labels under each label level, and the labeling labels of each training text comprise the intention labels of the training texts under each label level;
the model obtaining module is used for obtaining an initial model, and the initial model comprises a graph neural network and a classifier;
and the model training module is used for taking all the labeling labels as iterative input of the graph neural network so that the graph neural network learns the association relation among all the intention labels, taking the word group and the association relation of the training text as iterative input of the classifier, training the initial model, and training a classification model for obtaining the labeling labels of the training text.
8. The text intention recognition device is characterized by comprising a word segmentation module and a recognition module;
the word segmentation module is used for segmenting a word of a target text to be recognized to obtain a target word group of the target text;
the recognition module is used for inputting the target word group into a classification model trained in advance; wherein the classification model is obtained by training by adopting the text intention classification model training method according to any one of claims 1 to 3;
the identification module is further used for obtaining a target label to which the target text belongs through the classification model; the target labels comprise intention labels to which the target texts belong under all label hierarchies.
9. An electronic device comprising a processor and a memory, the memory storing computer programs executable by the processor, the processor being capable of executing the computer programs to implement the text intention classification model training method of any one of claims 1 to 3 or the text intention recognition method of any one of claims 4 to 6.
10. A storage medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements the text intention classification model training method according to any one of claims 1 to 7 or the text intention recognition method according to any one of claims 4 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211096559.XA CN115687610A (en) | 2022-09-08 | 2022-09-08 | Text intention classification model training method, recognition device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211096559.XA CN115687610A (en) | 2022-09-08 | 2022-09-08 | Text intention classification model training method, recognition device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115687610A true CN115687610A (en) | 2023-02-03 |
Family
ID=85062297
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211096559.XA Pending CN115687610A (en) | 2022-09-08 | 2022-09-08 | Text intention classification model training method, recognition device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115687610A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116028880A (en) * | 2023-02-07 | 2023-04-28 | 支付宝(杭州)信息技术有限公司 | Method for training behavior intention recognition model, behavior intention recognition method and device |
| CN116738343A (en) * | 2023-08-08 | 2023-09-12 | 云筑信息科技(成都)有限公司 | Material data identification method and device for construction industry and electronic equipment |
| CN117496542A (en) * | 2023-12-29 | 2024-02-02 | 恒生电子股份有限公司 | Document information extraction method, device, electronic equipment and storage medium |
-
2022
- 2022-09-08 CN CN202211096559.XA patent/CN115687610A/en active Pending
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116028880A (en) * | 2023-02-07 | 2023-04-28 | 支付宝(杭州)信息技术有限公司 | Method for training behavior intention recognition model, behavior intention recognition method and device |
| CN116028880B (en) * | 2023-02-07 | 2023-07-04 | 支付宝(杭州)信息技术有限公司 | Method for training behavior intention recognition model, behavior intention recognition method and device |
| CN116738343A (en) * | 2023-08-08 | 2023-09-12 | 云筑信息科技(成都)有限公司 | Material data identification method and device for construction industry and electronic equipment |
| CN116738343B (en) * | 2023-08-08 | 2023-10-20 | 云筑信息科技(成都)有限公司 | Material data identification method and device for construction industry and electronic equipment |
| CN117496542A (en) * | 2023-12-29 | 2024-02-02 | 恒生电子股份有限公司 | Document information extraction method, device, electronic equipment and storage medium |
| CN117496542B (en) * | 2023-12-29 | 2024-03-15 | 恒生电子股份有限公司 | Document information extraction method, device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111291185B (en) | Information extraction method, device, electronic equipment and storage medium | |
| CN109493166B (en) | Construction method for task type dialogue system aiming at e-commerce shopping guide scene | |
| US11494647B2 (en) | Slot filling with contextual information | |
| CN110287481B (en) | Named entity corpus labeling training system | |
| CN112699247B (en) | Knowledge representation learning method based on multi-class cross entropy contrast complement coding | |
| CN110263325B (en) | Chinese word segmentation system | |
| CN115687610A (en) | Text intention classification model training method, recognition device, electronic equipment and storage medium | |
| CN113204952B (en) | Multi-intention and semantic slot joint identification method based on cluster pre-analysis | |
| CN111783993A (en) | Intelligent labeling method and device, intelligent platform and storage medium | |
| CN110599324A (en) | Method and device for predicting refund rate | |
| WO2021052137A1 (en) | Emotion vector generation method and apparatus | |
| WO2023137911A1 (en) | Intention classification method and apparatus based on small-sample corpus, and computer device | |
| CN110598869B (en) | Classification method and device based on sequence model and electronic equipment | |
| CN113254675B (en) | Knowledge graph construction method based on self-adaptive few-sample relation extraction | |
| CN111858898A (en) | Text processing method and device based on artificial intelligence and electronic equipment | |
| CN116150367A (en) | Emotion analysis method and system based on aspects | |
| CN111709225A (en) | Event cause and effect relationship judging method and device and computer readable storage medium | |
| US11941360B2 (en) | Acronym definition network | |
| CN113869054A (en) | Deep learning-based electric power field project feature identification method | |
| CN116975743A (en) | Industry information classification method, device, computer equipment and storage medium | |
| CN114898156B (en) | Cross-modal semantic representation learning and fusion-based image classification method and system | |
| CN113886602B (en) | Domain knowledge base entity identification method based on multi-granularity cognition | |
| CN112836482B (en) | Method and device for generating problem by sequence generation model based on template | |
| CN115577283A (en) | Entity classification method and device, electronic equipment and storage medium | |
| CN115600595A (en) | Entity relationship extraction method, system, equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |