CN112860871A

CN112860871A - Natural language understanding model training method, natural language understanding method and device

Info

Publication number: CN112860871A
Application number: CN202110286974.0A
Authority: CN
Inventors: 黄诗磊; 张聪; 范长杰; 胡志鹏
Original assignee: Netease Hangzhou Network Co Ltd
Current assignee: Netease Hangzhou Network Co Ltd
Priority date: 2021-03-17
Filing date: 2021-03-17
Publication date: 2021-05-28
Anticipated expiration: 2041-03-17
Also published as: CN112860871B

Abstract

The application provides a natural language understanding model training method, a natural language understanding method and a device, and relates to the technical field of man-machine conversation, wherein the natural language understanding model training method comprises the following steps: obtaining model training corpuses, extracting the characteristics of the training corpuses of each natural language understanding task, respectively carrying out model training on a preset multilayer perceptron model according to the characteristics of the training corpuses of each natural language understanding task to obtain a natural language understanding sub-model corresponding to each natural language understanding task, and obtaining a natural language understanding model according to the natural language understanding sub-model. According to the method and the device, the characteristics of the training corpora of each natural language understanding task are extracted, and the model training is performed on the preset multilayer perceptron model according to the characteristics of the training corpora of each natural language understanding task, so that the overall scale of the model is reduced, the requirement of GPU resources is greatly reduced, the resource utilization rate is improved, and the system response time is shortened.

Description

Natural language understanding model training method, natural language understanding method and device

Technical Field

The application relates to the technical field of man-machine conversation, in particular to a natural language understanding model training method, a natural language understanding method and a natural language understanding device.

Background

Natural Language Understanding (NLU) is a fundamental and very important module in conversational robots, and refers to extracting meta-information contained in Natural text, including but not limited to: entities, keywords, intentions, emotions, relationships, grammars. Natural language understanding is a generic term for a series of related technologies or modules, where each module may be independent or dependent.

In the prior art, each sub-module of the natural language understanding system is trained independently to obtain a corresponding model, and benefits from the development of relevant research and application of a pre-training language model, the current general mode is to use the labeled corpus of each sub-module to perform fine tuning on the pre-training language model respectively to obtain the model, wherein the pre-training language model is a language model trained based on an automatic supervision method.

However, the pre-training language model has a large scale, which results in that each sub-module has a high requirement for Graphics Processing Unit (GPU) resources during training, which results in a large resource requirement and a reduced resource utilization rate, and the pre-training language model has a large scale, which results in a long system response time.

Disclosure of Invention

An object of the present application is to provide a natural language understanding model training method, a natural language understanding method and a device thereof, aiming at overcoming the defects in the prior art, so as to solve the problems of large resource demand and low resource utilization rate during training in the prior art.

In order to achieve the above purpose, the technical solutions adopted in the embodiments of the present application are as follows:

in a first aspect, an embodiment of the present application provides a natural language understanding model training method, where the method includes:

obtaining a model training corpus, wherein the model training corpus comprises training corpora of at least two natural language understanding tasks, and the training corpus of each natural language understanding task is labeled with a preset keyword corresponding to each natural language understanding task;

extracting the characteristics of the training corpus of each natural language understanding task;

respectively carrying out model training on a preset multilayer perceptron model according to the characteristics of the training corpus of each natural language understanding task to obtain a natural language understanding sub-model corresponding to each natural language understanding task;

and acquiring a natural language understanding model according to the natural language understanding submodel.

In an optional embodiment, the performing model training on a preset multilayer perceptron model according to the characteristics of the training corpus of each natural language understanding task to obtain a natural language understanding submodel corresponding to each natural language understanding task includes:

inputting the characteristics of the training corpus of each natural language understanding task into the multilayer perceptron model to obtain training keywords corresponding to each natural language understanding task;

and performing model training on the multilayer perceptron model according to the training keywords and the preset keywords to obtain the natural language understanding submodel.

In an optional embodiment, the extracting features of the corpus of each natural language understanding task includes:

and extracting the characteristics of the training corpus of each natural language understanding task by adopting a pre-trained characteristic extractor, wherein the characteristic extractor is pre-trained according to a relation type training corpus, the relation type training corpus comprises at least one pair of text samples, and each pair of text samples is labeled with a preset relation type between each pair of text samples.

In an alternative embodiment, the feature extractor is trained by:

obtaining a relation type training corpus;

inputting each pair of text samples into the pre-training language model to obtain the initial characteristics of each text sample in each pair of text samples, wherein the pre-training language model is a language model pre-trained based on an auto-supervision method;

inputting the initial features of each text sample in each pair of text samples into a pooling layer to obtain the pooling features of each text sample, wherein the pooling layer is used for pooling the initial features of each text;

inputting the pooling feature of each text sample into a splicing layer to obtain a splicing feature, wherein the splicing layer is used for splicing the pooling feature of each text sample in each pair of text samples with a preset feature, and the preset feature is an absolute value of a difference between the pooling features of each text sample in each pair of text samples;

inputting the splicing features into a classifier, and acquiring a training relationship type between each pair of text samples, wherein the classifier is used for carrying out relationship classification on each pair of text samples according to the splicing features;

and training the pre-training language model according to the preset relationship type and the training relationship type to obtain the feature extractor.

In an optional implementation manner, the training the pre-training language model according to the preset relationship type and the training relationship type to obtain the feature extractor includes:

training the pre-training language model according to the preset relationship type and the training relationship type to obtain a trained language model;

and acquiring the feature extractor according to the trained language model and the pooling layer.

In an alternative embodiment, the classifier includes a linear transformer and an activator, and the inputting the splicing features into the classifier and obtaining the training relationship type between each pair of text samples includes:

performing linear transformation on the splicing characteristics by using the linear transformer to obtain splicing characteristics after linear transformation;

and carrying out relation classification operation on the linearly transformed splicing features by adopting an activation function corresponding to the activator to obtain the training relation type.

In a second aspect, another embodiment of the present application provides a natural language understanding method, including:

acquiring a target text;

extracting the features of the target text by adopting a pre-trained feature extractor, wherein the feature extractor is pre-trained according to a relation type training corpus, the relation type training corpus comprises at least one pair of text samples, and each pair of text samples is labeled with a preset relation type between each pair of text samples;

inputting the characteristics of the target text into a pre-trained natural language understanding model to obtain an understanding result of the target text under at least two natural language understanding tasks, wherein the natural language understanding model is obtained according to a natural language understanding sub-model corresponding to each natural language understanding task, the natural language understanding sub-model is obtained by training according to a training corpus of each natural language understanding task, and the training corpus of each natural language understanding task is marked with a preset keyword corresponding to each natural language understanding task.

In an alternative embodiment, the at least two natural language understanding tasks include at least one of the following tasks:

a relation extraction task, an emotion analysis task, an intention identification task, a topic analysis task and a slot filling task;

correspondingly, the natural processing models corresponding to the at least two natural language understanding tasks include at least one of the following models:

the system comprises a relation extraction model, an emotion analysis model, an intention identification model, a topic model and a slot filling model.

In an optional implementation manner, if the at least two natural language understanding tasks include a relationship extraction task, an emotion analysis task, an intention identification task, a topic parsing task, and a slot filling task;

inputting the characteristics of the target text into a pre-trained natural language understanding model to obtain an understanding result of the target text under at least two natural language understanding tasks, wherein the understanding result comprises the following steps:

respectively processing the characteristics of the target text according to the relationship extraction model and the emotion analysis model to obtain the relationship attribute characteristics of the target text and the emotional tendency characteristics of the target text;

processing the relationship attribute characteristics and the characteristics of the target text according to the intention identification model and the topic model respectively to obtain intention information characteristics of the target text and topic information characteristics of the target text;

processing the intention information characteristics and the characteristics of the target text according to the slot filling model to obtain slot filling result characteristics of the target text;

acquiring a relation attribute, an emotional tendency, intention information, topic information characteristic and a slot filling result of the target text according to the relation attribute characteristic, the emotional tendency characteristic, the intention information characteristic, the topic information characteristic and the slot filling result characteristic, wherein the understanding result comprises the relation attribute, the emotional tendency, the intention information, the topic information and the slot filling result.

In a third aspect, another embodiment of the present application provides a natural language understanding model training apparatus, including:

the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a model training corpus, the model training corpus comprises training corpora of at least two natural language understanding tasks, and the training corpora of each natural language understanding task are marked with preset keywords corresponding to each natural language understanding task;

the extraction module is used for extracting the characteristics of the training corpus of each natural language understanding task;

the training module is used for respectively carrying out model training on a preset multilayer perceptron model according to the characteristics of the training corpus of each natural language understanding task to obtain a natural language understanding submodel corresponding to each natural language understanding task;

the acquisition module is further used for acquiring the natural language understanding model according to the natural language understanding sub-model.

In an optional embodiment, the training module is specifically configured to:

In an optional implementation manner, the extraction module is specifically configured to:

In an optional implementation manner, the obtaining module is further configured to:

obtaining a relation type training corpus;

In an optional implementation manner, the obtaining module is specifically configured to:

In an optional implementation manner, the classifier includes a linear transformer and an activator, and the obtaining module is specifically configured to:

In a fourth aspect, another embodiment of the present application provides a natural language understanding apparatus, including:

the acquisition module is used for acquiring a target text;

the extraction module is used for extracting the features of the target text by adopting a pre-trained feature extractor, wherein the feature extractor is pre-trained according to a relation type training corpus, the relation type training corpus comprises at least one pair of text samples, and each pair of text samples is labeled with a preset relation type between each pair of text samples;

and the input module is used for inputting the characteristics of the target text into a pre-trained natural language understanding model to obtain an understanding result of the target text under at least two natural language understanding tasks, wherein the natural language understanding model is obtained according to a natural language understanding sub-model corresponding to each natural language understanding task, the natural language understanding sub-model is obtained by training according to a training corpus of each natural language understanding task, and the training corpus of each natural language understanding task is labeled with a preset keyword corresponding to each natural language understanding task.

the input module is specifically configured to:

In a fifth aspect, another embodiment of the present application provides a natural language understanding model training apparatus, including: a processor, a memory and a bus, the memory storing a computer program executable by the processor, the processor and the memory communicating via the bus when the natural language understanding model training device is running, the processor executing the computer program to perform the method according to any of the above first aspects.

In a sixth aspect, another embodiment of the present application provides a natural language understanding apparatus, including: a processor, a memory and a bus, the memory storing a computer program executable by the processor, the processor and the memory communicating via the bus when the natural language understanding apparatus is running, the processor executing the computer program to perform the method of any of the second aspect.

In a seventh aspect, another embodiment of the present application provides a storage medium, on which a computer program is stored, where the computer program is executed by a processor to perform the method according to any one of the first aspect or the second aspect.

The application provides a natural language understanding model training method, a natural language understanding method and a device, wherein the natural language understanding model training method comprises the following steps: obtaining model training corpuses, wherein the model training corpuses comprise training corpuses of at least two natural language understanding tasks, the training corpuses of each natural language understanding task are marked with preset keywords corresponding to each natural language understanding task, the characteristics of the training corpuses of each natural language understanding task are extracted, model training is respectively carried out on a preset multilayer perceptron model according to the characteristics of the training corpuses of each natural language understanding task, a natural language understanding submodel corresponding to each natural language understanding task is obtained, and a natural language understanding model is obtained according to the natural language understanding submodel. According to the method and the device, the characteristics of the training corpora of each natural language understanding task are extracted, and the model training is performed on the preset multilayer perceptron model according to the characteristics of the training corpora of each natural language understanding task, so that the overall scale of the model is reduced, the requirement of GPU resources is greatly reduced, the resource utilization rate is improved, and the system response time is shortened.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.

Fig. 1 is a first flowchart illustrating a natural language understanding model training method provided in an embodiment of the present application;

FIG. 2 is a schematic diagram illustrating a training mode of a natural language understanding model provided by an embodiment of the present application;

FIG. 3 is a flowchart illustrating a second natural language understanding model training method according to an embodiment of the present disclosure;

FIG. 4 is a flowchart illustrating a third method for training a natural language understanding model according to an embodiment of the present disclosure;

FIG. 5 is a schematic diagram illustrating a training process of a feature extractor provided by an embodiment of the present application;

FIG. 6 shows a schematic diagram of a feature extractor provided by an embodiment of the present application;

FIG. 7 is a fourth flowchart illustrating a natural language understanding model training method provided by an embodiment of the present application;

FIG. 8 is a first flowchart illustrating a natural language understanding method according to an embodiment of the present disclosure;

FIG. 9 is a flowchart illustrating a natural language understanding method according to an embodiment of the present application;

FIG. 10 is a schematic diagram illustrating a natural language understanding system inference process provided by an embodiment of the application;

fig. 11 is a schematic structural diagram illustrating a natural language understanding model training apparatus according to an embodiment of the present application;

fig. 12 is a schematic structural diagram of a natural language understanding apparatus provided in an embodiment of the present application;

FIG. 13 is a schematic structural diagram of a natural language understanding model training device provided in an embodiment of the present application;

fig. 14 is a schematic structural diagram of a natural language understanding apparatus provided in an embodiment of the present application.

Detailed Description

In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it should be understood that the drawings in the present application are for illustrative and descriptive purposes only and are not used to limit the scope of protection of the present application. Additionally, it should be understood that the schematic drawings are not necessarily drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of the present application. It should be understood that the operations of the flow diagrams may be performed out of order, and steps without logical context may be performed in reverse order or simultaneously. One skilled in the art, under the guidance of this application, may add one or more other operations to, or remove one or more operations from, the flowchart.

In addition, the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.

It should be noted that in the embodiments of the present application, the term "comprising" is used to indicate the presence of the features stated hereinafter, but does not exclude the addition of further features.

Before the technical solution of the present application is introduced, the terms related to the present application will be described:

natural Language Understanding (NLU): a technology for studying how a computer reads human Language is a generic term for all related technologies that can help a machine understand human Language, including intention recognition, emotion analysis, and the like, and is a part of Natural Language Processing (NLP).

Sentence characterization: vectors that can be used to represent sentence (i.e., text) features.

The language model is as follows: the method is a probability distribution model, and when a context is given, the probability distribution of the words at the specified positions can be obtained.

Pre-training a language model: and (3) a language model trained based on an automatic supervision method.

Fine adjustment: and performing supervised learning on the pre-training language model aiming at downstream tasks, wherein the downstream tasks can comprise an emotion analysis task, a reading and understanding task, an intention recognition task, a named entity recognition task, a semantic similarity matching task and the like.

Multilayer Perceptron (MLP): a supervised learning method.

Reasoning: the process by which the model predicts the inputs.

Natural Language Understanding (NLU) refers to extracting meta information contained in Natural text, including but not limited to: entities, keywords, intentions, emotions, relationships, grammars. Natural language understanding and natural language processing are included relationships, natural language understanding is a part of natural language processing, and natural language understanding is a fundamental and important module in a conversation robot and is used for analyzing and understanding user conversations, and processing unstructured conversation information into a structured form convenient for machine and human understanding.

The natural language understanding is a general term of a series of related technologies or modules, wherein each module can be independent from each other or have a dependency relationship, and with the rapid development of deep learning and pre-training language model related research and application, the natural language understanding is more and more widely applied in NLUs, and the effect is greatly improved. Therefore, it is now common practice to: and (4) fine-tuning the labeled corpus of each sub-module in the NLU on the pre-training language model to obtain a corresponding model. The sub-modules in the NLU are different according to the usage scenario and capability of the platform or system, including but not limited to: the system comprises an intention identification module, a topic identification module, a slot filling module, an emotion analysis module and a relation extraction module. The labeled corpora are data of each submodule (i.e., subtask) used for training the model and include real labels, and the emotion analysis module is taken as an example, and two corpora are taken as examples, namely 'i like the image quality of the game', 'i like the image quality of the game' and 'i' e's BMG is too hard to hear', and the labeled data are taken as negative feelings.

However, the mode of obtaining the final model by fine tuning the pre-training language model by using the labeled data is very dependent on the large-scale pre-training language model, the structure of the natural language understanding model is complex, the natural language understanding model comprises a plurality of sub-modules, and each sub-module obtains the model by training the mode, so that the system has great defects in two aspects of resource utilization rate and response real-time property:

firstly, resource utilization rate: the method mainly comprises two aspects of model training and service deployment. Because the scale (i.e., scale parameter) of the pre-training language model is large, each module has a high requirement on GPU resources during training. In addition, each module is independently deployed, and if N sub-modules are provided, the sub-modules with N pre-trained language models are deployed on the line, so that the requirement on resources is further increased.

Second, response instantaneity: due to the fact that the scale of the pre-training language model is large, time consumption is relatively long in the reasoning process, and the sequence dependency relationship exists among the submodules, the response time of the whole system is increased in multiples, and the pre-training language model is a bottleneck of the response speed of the system.

Based on the above problems, the present application provides a natural language understanding model training method, which performs model training on a preset multilayer perceptron model respectively according to the characteristics of the training corpus of each natural language understanding task by extracting the characteristics of the training corpus of each natural language understanding task, thereby reducing the overall scale of the model, greatly reducing the requirements of GPU resources, improving the resource utilization rate, and shortening the system response time.

The following describes an implementation process of the natural language understanding model training method with reference to the embodiments of fig. 1 to 5.

Fig. 1 is a schematic flowchart illustrating a first flow of a natural language understanding model training method provided in an embodiment of the present application, where an execution subject of the embodiment may be a natural language understanding model training device, for example, a terminal device, a server, and the like.

As shown in fig. 1, the method may include:

s101, obtaining model training corpora.

The model corpus comprises at least two natural language understanding tasks, and the training corpus of each natural language understanding task is marked with preset keywords corresponding to each natural language understanding task.

The training corpus of each natural language understanding task includes a plurality of text samples corresponding to each natural language understanding task, that is, each text sample corresponding to each natural language understanding task is labeled with a preset keyword corresponding to each natural language understanding task, and the preset keyword is an actual keyword corresponding to each text sample.

At least two natural language understanding tasks include, but are not limited to: the method comprises a relation extraction task, an emotion analysis task, an intention identification task, a topic analysis task and a slot filling task.

The relation extraction task is used for extracting a preset relation in the corpus, taking the example that the corpus of the relation extraction task includes a text sample 'when and clearly birth', and the preset keyword corresponding to the text sample can be 'birth date'.

The emotion analysis task is used for judging the emotional tendency of user conversation, and taking the example that the training corpus of the emotion analysis task comprises a text sample 'the operation of the game is not smooth enough', the preset keyword corresponding to the text sample can be 'negative emotion'.

The intention recognition task is used for judging the intention of the user conversation, and taking the training language including a text sample of 'what name you call' as an example, the preset keyword corresponding to the text sample can be 'inquiry name'.

The topic analysis task is used for analyzing topics related to user conversation, and taking the example that the topic analysis task comprises a text sample 'you like to eat chicken feet', the preset keyword corresponding to the text sample can be 'food'.

The slot filling task is used for extracting a specified slot position in a task type conversation, such as an air ticket booking task and a slot position destination, a training corpus of the slot filling task comprises a text sample 'I want to specify an air ticket to Beijing', the aim is to extract the 'Beijing' to be matched with the destination slot position, namely, a preset keyword corresponding to the text sample can be 'Beijing'.

And S102, extracting the characteristics of the training corpus of each natural language understanding task.

S103, respectively carrying out model training on a preset multilayer perceptron model according to the characteristics of the training corpus of each natural language understanding task to obtain a natural language understanding sub-model corresponding to each natural language understanding task.

Extracting the characteristics of the training corpus of each natural language understanding task, then respectively carrying out model training on a preset multilayer perceptron model according to the characteristics of the training corpus of each natural language understanding task to obtain a natural language understanding submodel corresponding to each natural language understanding task, namely extracting the characteristics of each text sample in the training corpus of each natural language understanding task, inputting the characteristics of each text sample in the training corpus of each natural language understanding task into the preset multilayer perceptron model, and training the multilayer perceptron model to obtain the natural language understanding submodel corresponding to each natural language understanding task.

The number of the multilayer perceptron models is consistent with the number of the natural language understanding tasks, that is, if there are N natural language understanding tasks, the characteristics of the training corpus of each natural language understanding task can be respectively input into the multilayer perceptron models for model training, so as to obtain the natural language understanding submodels corresponding to each natural language understanding task.

It should be noted that the preset multilayer perceptron model may be a neural network model, and the model scale of the multilayer perceptron model is far smaller than that of the pre-trained language model compared with that of the pre-trained language model, so that the requirement of GPU resources can be greatly reduced and the resource utilization rate is improved in the model training process.

Understanding tasks in at least two natural languages includes: for example, the relationship extraction task, the emotion analysis task, the intention recognition task, the topic parsing task, and the slot filling task, the natural language understanding submodel corresponding to each natural language understanding task may include: the system comprises a relation extraction model, an emotion analysis model, an intention identification model, a topic analysis model and a slot filling model.

In an alternative embodiment, step S102 may include:

and extracting the characteristics of the training corpus of each natural language understanding task by adopting a pre-trained characteristic extractor.

The feature extractor is trained in advance according to a relation type training corpus, the relation type training corpus comprises at least one pair of text samples, and each pair of text samples is marked with a preset relation type between each pair of text samples.

The preset relationship type between each pair of text samples is an actual relationship type between each pair of text samples, and the preset relationship type may include three types: neutral, intrinsic and contradictory.

Where neutral refers to the independence between each pair of text samples, such as the text "a car is on the highway," the text "today is really good weather"; implications mean that each pair of text samples is similar, e.g., the text "today's weather is really good"; contradiction means that each pair of text samples is contradictory, such as the text "good weather today" and "bad weather today".

In particular, the feature extractor is used to extract a feature representation of the natural language text, typically in the form of a fixed length vector, such as [0.1,0.06,0.23,0.11 ]. The training corpus of each natural language understanding task is respectively input into the feature extractor, so that the features of the training corpus of each natural language understanding task can be obtained, namely, the features of each text sample in the training corpus of each natural language understanding task can be extracted and obtained by adopting the pre-trained feature extractor.

Referring to fig. 2, fig. 2 is a schematic diagram illustrating a training mode of a natural language understanding model according to an embodiment of the present application, and as shown in fig. 2, a model corpus includes at least two training corpora of natural language understanding tasks, where the training corpus of each natural language understanding task is labeled with a preset keyword corresponding to each natural language understanding task, and the training corpora of the at least two natural language understanding tasks are respectively labeled as: the method comprises the following steps of intent recognition linguistic data, relation extraction marking linguistic data, groove filling marking linguistic data, topic marking linguistic data and emotion analysis linguistic data, wherein the intent recognition linguistic data, the relation extraction marking linguistic data, the groove filling marking linguistic data, the topic marking linguistic data and the emotion analysis linguistic data are input into a feature extractor respectively to obtain the features of each training linguistic data, then the features of each training linguistic data are input into N multilayer perceptron models respectively to conduct model training, and an intent recognition model, a relation extraction model, a groove filling model, a topic model and an emotion analysis model are obtained.

The key point of this embodiment is to provide a natural language understanding system based on reusable sentence representation, by constructing a unified feature extractor for each sub-module of the NLU system to use, each sub-module model can be simplified into a multi-layer perceptron model, the corpus text features obtained by the feature extractor are used as the input of the perceptron model, and finally, a corresponding model is obtained by training, and this design reduces the overall scale of the model of each sub-module. In the model training phase, the demands of GPU resources are greatly reduced, the resource utilization rate is improved, and the system response time is shortened.

And S104, acquiring a natural language understanding model according to the natural language understanding sub-model.

The natural language understanding submodel corresponding to each natural language understanding task is obtained through training, and the natural language understanding model is obtained according to the natural language understanding submodel, namely, the natural language understanding model comprises the natural language understanding submodel, so that the text input by the user can be input into the natural language understanding model in the model prediction process, the processing result obtained by processing each natural language understanding submodel can be obtained, and the processing result of all the natural language understanding submodels is used as the understanding result of the text input by the user.

The natural language understanding model training method of the embodiment includes: obtaining model training corpuses, extracting the characteristics of the training corpuses of each natural language understanding task, respectively carrying out model training on a preset multilayer perceptron model according to the characteristics of the training corpuses of each natural language understanding task to obtain a natural language understanding sub-model corresponding to each natural language understanding task, and obtaining a natural language understanding model according to the natural language understanding sub-model. In this embodiment, the characteristics of the corpus of each natural language understanding task are extracted, and the preset multilayer perceptron model is subjected to model training according to the characteristics of the corpus of each natural language understanding task, so that the overall scale of the model is reduced, the requirement of GPU resources is greatly reduced, the resource utilization rate is improved, and the system response time is shortened.

Exemplarily, the following describes a training process of the natural language understanding sub-model with reference to the embodiment of fig. 3, where fig. 3 shows a second flowchart of the natural language understanding model training method provided in the embodiment of the present application, and as shown in fig. 3, step S103 may include:

and S1031, inputting the characteristics of the training corpus of each natural language understanding task into the multilayer perceptron model to obtain training keywords corresponding to each natural language understanding task.

S1032, performing model training on the multilayer perceptron model according to the training keywords and the preset keywords to obtain a natural language understanding submodel.

Inputting the characteristics of the training corpus of each resource language processing task into the multilayer perceptron model to obtain training keywords corresponding to each natural language understanding task, and then performing model training on the multilayer perceptron model according to the training keywords and preset keywords to obtain a natural language understanding sub-model, wherein the preset keywords can be actual keywords.

In the natural language understanding model training method of this embodiment, the characteristics of the training corpus of each natural language understanding task are input into the multilayer perceptron model to obtain the training keywords corresponding to each natural language understanding task, and the multilayer perceptron model is subjected to model training according to the training keywords and the preset keywords to obtain the natural language understanding submodel. Compared with the prior art, model training is carried out based on the multilayer perceptron model, a pre-training language model is not needed to carry out model training, the overall scale of the model is reduced, the requirement of GPU resources is greatly reduced, and the resource utilization rate is improved.

The training process of the feature extractor will be described with reference to the embodiments of fig. 4 and 5. Fig. 4 is a flowchart illustrating a third schematic view of a process of training a natural language understanding model according to an embodiment of the present application, where an execution subject of the embodiment may be a feature extractor training device, for example, a terminal device or a server.

As shown in fig. 4, the feature extractor is trained as follows:

s201, obtaining the relation type training corpus.

S202, inputting each pair of text samples into a pre-training language model to obtain the initial characteristics of each text sample in each pair of text samples.

The relation type training corpus comprises at least one pair of text samples, and each pair of text samples are marked with a preset relation type between each pair of text samples.

The preset relationship type includes an actual relationship type between each pair of text samples, and the preset relationship type may include three types: neutral, intrinsic and contradictory.

The pre-training language model is a language model pre-trained based on an auto-supervision method, and may be, for example, a pre-training model BERT (english language full name: Bidirectional Encoder retrieval from transforms) trained on dialog corpus.

In the present embodiment, Natural Language Inference (NLI) is adopted as a training task of the feature extractor. Specifically, each pair of text samples in the relation type training corpus is input into the pre-training language model, so as to obtain the initial characteristics of each text sample in each pair of text samples, that is, the pre-training language model is used to obtain the characteristic representation of two text samples.

S203, inputting the initial characteristics of each text sample in each pair of text samples into a pooling layer to obtain the pooling characteristics of each text sample.

The pooling layer is used to pool the initial features of each text. Inputting the initial features of each text sample in each pair of text samples into a pooling layer, and performing pooling operation on the initial features of each text sample by using the pooling layer to obtain the pooled features of each text sample, wherein the pooling operation refers to averaging.

For example, the pre-trained language model is a BERT model, a text sample with a length of L is obtained, after the pre-trained language model is passed through, a matrix with a dimension of (L +2, hidden _ size) is obtained, where 2 is added that BERT has two special mark symbols for representing the beginning and the end of the text sample, and hidden _ size is the length of a preset vector, then the initial features of each text sample in each pair of text samples are input to the pooling layer, so that the pooled features of each text sample, that is, the pooled features with a length of hidden _ size, can be obtained, then each pair of text samples are input to the pooling layer, and two pooled features with a length of hidden _ size are obtained.

And S204, inputting the pooling characteristic of each text sample into a splicing layer to obtain the splicing characteristic.

The splicing layer is used for splicing the pooling feature of each text sample in each pair of text samples and a preset feature, wherein the preset feature is an absolute value of a difference between the pooling features of each text sample in each pair of text samples.

Inputting the pooling feature of each text sample into a splicing layer, calculating a preset feature according to the pooling feature of each text sample in each pair of text samples, wherein the preset feature is an absolute value of a difference between the pooling features of each text sample in each pair of text samples, for example, the pooling feature of each text sample in each pair of text samples is respectively marked as x and y, so that the preset feature z is | x-y |, and then the pooling feature of each text sample and the preset feature are combined and spliced to obtain a splicing feature, namely (x, y, | x-y |).

For example, each text sample in each pair of text samples is respectively marked as "a black racing car starts up in front of a group of people" and "a person drives on a quiet road", two vectors with dimensions being the hidden _ size features are respectively obtained through a pre-training language model, and the vectors can also be understood as a list with the length being the hidden _ size, wherein each element in the list is a number, the pooled features obtained after pooling operation are marked as x and y, the x subtracts y, and the absolute value is obtained as a result to obtain the preset feature | x-y |, the three features are spliced, and the length of the obtained spliced feature is 3 | _ hidden _ size, and the spliced feature can also be understood as a list with the length being 3 | _ hidden _ size.

And S205, inputting the splicing characteristics into a classifier, and acquiring the training relationship type between each pair of text samples.

And S206, training the pre-training language model according to the preset relation type and the training relation type to obtain the feature extractor.

And the classifier is used for carrying out relation classification on each pair of text samples according to the splicing characteristics. Inputting the splicing characteristics into a classifier to obtain a training relationship type between each pair of text samples, namely a training relationship type between each text sample in each pair of text samples, and then training the pre-training language model according to a preset relationship type and the training relationship type to obtain a characteristic extractor, namely, the required characteristic extractor is obtained after the pre-training language model is trained by an NLI task.

Referring to fig. 5, fig. 5 shows a schematic diagram of a training process of a feature extractor provided in this embodiment, as shown in fig. 5, a text sample 1 and a text sample 2 are respectively input to a pre-training language model to obtain an initial feature of the text sample 1 and an initial feature of the text sample 2, the initial feature of the text sample 1 and the initial feature of the text sample 2 are input to a pooling layer for pooling operation to obtain a pooled feature x of the text sample 1 and a pooled feature y of the text sample 2, then the pooled feature x of the text sample 1 and the pooled feature y of the text sample 2 are input to a concatenation layer to obtain a concatenated feature (x, y, | x-y |), the concatenated feature (x, y, | x-y |) is input to a classifier to obtain a training relationship type of the text sample 1 and the text sample 2, the pre-training language model is trained according to the training relationship type and a preset relationship type, a feature extractor is obtained.

In an alternative embodiment, step S206 may include:

and training the pre-training language model according to the preset relation type and the training relation type to obtain the trained language model.

And acquiring a feature extractor according to the trained language model and the pooling layer.

And performing model training on the pre-training language model according to the preset relation type and the training relation type to obtain a trained language model, and then acquiring a feature extractor according to the trained language model and the pooling layer. Based on the embodiment of fig. 5, fig. 6 shows a schematic diagram of a feature extractor provided in the embodiment of the present application, as shown in fig. 6, the feature extractor includes a trained language model and a pooling layer, and in an application process, a target text is input into the feature extractor, so that a feature representation of the target text, that is, a feature of the target text, can be obtained.

The natural language understanding model training method of the embodiment includes obtaining relationship type training corpora, inputting each pair of text samples into a pre-training language model, obtaining initial characteristics of each text sample in each pair of text samples, inputting the initial characteristics of each text sample in each pair of text samples into a pooling layer, obtaining pooling characteristics of each text sample, inputting the pooling characteristics of each text sample into a splicing layer, obtaining splicing characteristics, inputting the splicing characteristics into a classifier, obtaining a training relationship type between each pair of text samples, and training the pre-training language model according to a preset relationship type and a training relationship type, so as to obtain a characteristic extractor. In this embodiment, a unified feature extractor is constructed based on the pre-training language model, so that the overall scale of the model of each sub-module can be reduced while the advantages of the pre-training language model are utilized, the requirements of GPU resources are greatly reduced in the natural language understanding model training stage, and the resource utilization rate is improved.

In an alternative embodiment, the classifier includes a linear transformer and an activator, and the training relationship type obtaining process is described below with reference to the embodiment of fig. 7. Fig. 7 shows a fourth flowchart of the natural language understanding model training method provided in the embodiment of the present application, and as shown in fig. 7, step S205 may include:

and S2051, performing linear transformation on the splicing characteristics by adopting a linear transformer to obtain the splicing characteristics after the linear transformation.

And S2052, performing relation classification operation on the linearly transformed splicing features by adopting an activation function corresponding to the activator to obtain a training relation type.

The linear variator is used for linearly varying the splicing characteristics, and the purpose of linear transformation is not analysis but combination of the characteristics. Inputting the splicing characteristics into a classifier, performing linear transformation on the splicing characteristics by using a linear transformer to obtain the splicing characteristics after the linear transformation, and performing relation classification operation on the splicing characteristics after the linear transformation by using an activation function corresponding to an activator to obtain a training relation type.

It should be noted that the parameters of the linear transformation include w and b, the splicing feature is x, the linear transformation on the splicing feature may be to calculate a splicing feature wx + b after the linear transformation, and then perform a relationship classification operation on the splicing feature after the linear transformation by using an activation function H corresponding to an activator, that is, calculate H (wx + b), where the matrix size of H (wx + b) is (3hidden _ size, 3).

For example, the preset relationship types may include three types: neutral, intrinsic, contradictory, the matrix size of x is (3 x 5, 1), x is denoted ([0.1652], [ -1.0379], [0.2642], [0.7698], [1.3985], [ -0.3933], [ -0.2142], [1.3650], [1.3375], [ -0.0350], [ -0.7475], [1.1432], [1.4882], [ -0.1322], [ -0.1525 ]); w has a matrix size of (3, 3 × 5), w is denoted ([1.1478, -1.3863, -0.5489, 2.4907, 0.2426, 1.2681, 0.3901, 0.0547, -0.3516, -0.5536, -1.0327, -3.2929, -1.5016, -0.9504, 0.2205], [ -1.5363,0.7316,0.5526,0.300, -0.1739, -1.0257,0.8219, -0.7030,0.9676,1.4635, -0.5628, 0.8349, -0.9413, -0.3485, -0.3343], [ -0.9402, -0.8493, -0.1146,0.2488, -0.4301, 0.1202, 0.7642,1.2753, -0.4647,0.4976,0.4875, -0.0399, -2.4998, -0.9739, 1.7172 ]); b has a matrix size of (3, 1), and b is denoted as ([ -1.8844], [ -4.3207], [ -0.9502 ]).

The matrix size of w x is then (3, 1), denoted ([ -1.8844], [ -4.3207], [ -0.9802]), the H function is: the result for softmax, H (wx + b) is: ([0.1848], [0.0411], [0.7742]), namely a matrix with 1 row and 3 columns, wherein the values of the 3 columns respectively represent three types of neutrality, inclusion and contradiction, and the value corresponding to the value of the value training relationship type in the matrix is the maximum, so that the training relationship type is contradictory.

In the training method for the natural language understanding model of this embodiment, the linear transformer is used to perform linear transformation on the splicing features to obtain splicing features after linear transformation, and the activation function corresponding to the activator is used to perform relationship classification operation on the splicing features after linear transformation to obtain a training relationship type. In this embodiment, the stitching features are processed through a linear transformer and an activator, such that the type of training relationship between each pair of text samples can be determined.

The following describes an implementation process of the natural language understanding method with reference to the embodiment of fig. 8.

Fig. 8 is a flowchart illustrating a first natural language understanding method provided in an embodiment of the present application, where an execution subject of the embodiment may be a natural language understanding device, for example, a terminal device and a server.

As shown in fig. 8, the method may include:

s301, acquiring a target text.

And S302, extracting the features of the target text by adopting a pre-trained feature extractor.

The target text can be any text to be subjected to natural language understanding, the target text is obtained and is input into a pre-trained feature extractor, and the features of the target text can be extracted by adopting the pre-trained feature extractor.

It should be noted that, for the training process of the feature extractor, reference may be made to the related description in the embodiments of fig. 4 to fig. 7, and details are not repeated here.

And S303, inputting the characteristics of the target text into a pre-trained natural language understanding model to obtain an understanding result of the target text under at least two natural language understanding tasks.

The natural language understanding sub-model is obtained according to a natural language understanding sub-model corresponding to each natural language understanding task, the natural language understanding sub-model is obtained according to training corpora of each natural language understanding task, and the training corpora of each natural language understanding task are marked with preset keywords corresponding to each natural language understanding task.

In an alternative embodiment, the at least two natural language understanding tasks include at least one of the following:

accordingly, the natural processing models corresponding to the at least two natural language understanding tasks include at least one of the following models:

The relation extraction model is used for extracting relation attributes of the target text, the relation attributes can be 'birth date', the emotion analysis model analyzes emotion tendencies of the target text and can comprise 'negative emotion' and 'positive emotion', the intention identification model is used for identifying the intention of the target text and can comprise 'inquiry name', the topic model is used for analyzing topics related to the target text and can comprise 'gourmet', and the slot filling model is used for extracting slot filling results in the target text and can fill 'Beijing' in the target text into a destination slot.

Inputting the characteristics of the target text into a pre-trained natural language understanding model to obtain understanding results of the target text under at least two natural language understanding tasks, that is, obtaining understanding results under each natural language understanding task, taking at least two natural language understanding tasks including a relationship extraction task, an emotion analysis task, an intention recognition task, a topic parsing task and a slot filling task as examples, and understanding results of the target text under at least two natural language understanding tasks may include: relationship attributes, emotional tendencies, intent information, topic information, and slot filling results.

It should be noted that, for the training process of the natural language understanding model, reference may be made to the related description in the embodiments of fig. 1 to fig. 3, and details are not described herein again.

The natural language understanding method of the embodiment acquires a target text, extracts features of the target text by adopting a pre-trained feature extractor, inputs the features of the target text into a pre-trained natural language understanding model, and obtains an understanding result of the target text under at least two natural language understanding tasks. In the embodiment, a unified feature extractor is constructed based on a pre-training language model, only one sentence representation needs to be uniformly calculated for user input in the overall system inference process, each submodule of an NLU can perform inference of a corresponding task by multiplexing the feature, the overall model scale of each submodule can be reduced while the advantages of the pre-training language model are utilized, the requirement of GPU resources is greatly reduced in the model deployment stage, the resource utilization rate is improved, the inference speed of each submodule is accelerated due to the reduction of the model scale, and the overall response time of the natural language understanding system is reduced.

In an alternative embodiment, if the at least two natural language understanding tasks include: a relation extraction task, an emotion analysis task, an intention identification task, a topic analysis task and a slot filling task; accordingly, the natural processing models corresponding to the at least two natural language understanding tasks include: the system comprises a relation extraction model, an emotion analysis model, an intention identification model, a topic model and a slot filling model.

Step S303 is described below with reference to the embodiments of fig. 9 to 10, where fig. 9 shows a second flowchart of the natural language understanding method provided in the embodiment of the present application, and as shown in fig. 9, step S303 may include:

s3031, the characteristics of the target text are respectively processed according to the relation extraction model and the emotion analysis model, and the relation attribute characteristics of the target text and the emotion tendency characteristics of the target text are obtained.

Inputting the characteristics of the target text into a pre-trained relationship extraction model to obtain the relationship attribute characteristics of the target text, wherein the relationship attribute characteristics are used for ensuring the relationship attributes of the target text, and the relationship attributes can be the birth date, for example.

Inputting the characteristics of the target text into an emotion analysis model to obtain the emotional tendency characteristics of the target text, wherein the emotional tendency characteristics are used for representing the emotional tendency of the target text, and the emotional tendency can be negative emotion or positive emotion.

And S3032, processing the relationship attribute characteristics and the characteristics of the target text respectively according to the intention identification model and the topic model to obtain intention information characteristics of the target text and topic information characteristics of the target text.

And inputting the relationship attribute features and the features of the target text into an intention recognition model to obtain intention recognition features of the target text, wherein the intention recognition features are used for representing the intention of the target text and can be 'inquiry name'.

And inputting the relationship attribute features and the features of the target text into a topic model to obtain topic information features of the target text, wherein the topic information features are used for representing topics related to the target text and can comprise 'food', for example.

And S3033, processing the intention information characteristics and the characteristics of the target text according to the slot filling model to obtain slot filling result characteristics of the target text.

And inputting the intention information characteristic and the characteristic of the target text into a slot filling model to obtain a slot filling result characteristic of the target text, wherein the slot filling result characteristic of the target text is used for representing the slot filling result of the target text, and for example, "Beijing" in the target question can be filled into a destination slot.

S3034, acquiring the relation attribute, the emotional tendency, the intention information, the topic information characteristic and the slot filling result of the target text according to the relation attribute characteristic, the emotional tendency characteristic, the intention information characteristic, the topic information characteristic and the slot filling result characteristic.

And acquiring the relation attribute, the emotional tendency, the intention information, the topic information characteristic and the slot filling result of the target text respectively according to the relation attribute characteristic, the emotional tendency characteristic, the intention information characteristic, the topic information characteristic and the slot filling result characteristic, wherein the understanding result of the target text under at least two natural language understanding tasks comprises the relation attribute, the emotional tendency, the intention information, the topic information and the slot filling result.

It should be noted that the understanding results of the target text under at least two natural language understanding tasks may be integrated and output by the system, for example, each understanding result record may be saved in a preset table for being called by a subsequent natural language processing module, taking the understanding result including emotion tendencies as an example, in an actual human-computer conversation scenario, if it is recognized that the emotion tendencies of the conversation text spoken by the user are negative emotions, the conversation robot may reply to the conversation with positive emotions to the user, and if the conversation text spoken by the user is "happy today", the reply text of the conversation robot may be "keep good mood every day".

Fig. 10 is a schematic diagram illustrating an inference process of a natural language understanding system according to an embodiment of the present application, and as shown in fig. 10, in a first step, a target text input by a user is input to a feature extractor to obtain a feature of the target text; the second step, inputting the characteristic of the target text into the relation extraction model and the emotion analysis model respectively to obtain a relation attribute characteristic of the target text and an emotion tendency characteristic of the target text; inputting the relationship attribute characteristic of the target text and the characteristic of the target text into an intention recognition model to obtain intention information characteristics of the target text, and inputting the relationship attribute characteristic of the target text and the characteristic of the target text into a topic model to obtain a topic information characteristic of the target text; and fourthly, inputting the intention information characteristic first and the characteristic first of the target text into a groove filling model to obtain a groove filling result of the target text.

The natural language understanding method of the embodiment includes respectively processing features of a target text according to a relationship extraction model and an emotion analysis model to obtain relationship attribute features of the target text and emotion tendency features of the target text, respectively processing the relationship attribute features and the features of the target text according to an intention identification model and a topic model to obtain intention information features of the target text and topic information features of the target text, processing the intention information features and the features of the target text according to a slot filling model to obtain slot filling result features of the target text, and obtaining relationship attributes, emotion tendency, intention information, topic information features and slot filling results of the target text according to the relationship attribute features, the emotion tendency features, the intention information features, the topic information features and the slot filling result features. In this embodiment, because some sub-modules in the natural language understanding system have a sequential dependency relationship, by providing a uniform feature extractor, the processing speed on the dependency path is increased, that is, the bottleneck of the system response time is broken through.

Fig. 11 shows a schematic structural diagram of a natural language understanding model training device provided in an embodiment of the present application, where the natural language understanding model training device may be integrated in a natural language understanding model training device. As shown in fig. 11, the natural language understanding model training device 40 includes:

an obtaining module 401, configured to obtain a model corpus, where the model corpus includes training corpora of at least two natural language understanding tasks, and a preset keyword corresponding to each natural language understanding task is labeled in the training corpus of each natural language understanding task;

an extraction module 402, configured to extract features of the corpus of each natural language understanding task;

a training module 403, configured to perform model training on a preset multilayer perceptron model according to features of a training corpus of each natural language understanding task, to obtain a natural language understanding sub-model corresponding to each natural language understanding task;

the obtaining module 401 is further configured to obtain a natural language understanding model according to the natural language understanding sub-model.

In an optional implementation manner, the training module 403 is specifically configured to:

In an optional implementation manner, the extracting module 402 is specifically configured to:

In an optional implementation manner, the obtaining module 401 is further configured to:

obtaining a relation type training corpus;

In an optional implementation manner, the obtaining module 401 is specifically configured to:

In an optional implementation manner, the classifier includes a linear transformer and an activator, and the obtaining module 401 is specifically configured to:

The implementation process and the implementation principle of the natural language understanding model training device of this embodiment may refer to the description related to the natural language understanding model training method in the foregoing method embodiments, and are not described herein again.

Fig. 12 is a schematic structural diagram of a natural language understanding apparatus provided in an embodiment of the present application, and the natural language understanding apparatus may be integrated into a natural language understanding device. As shown in fig. 12, the natural language understanding apparatus 50 includes:

an obtaining module 501, configured to obtain a target text;

an extraction module 502, configured to extract features of the target text by using a pre-trained feature extractor, where the feature extractor is pre-trained according to a relationship type corpus, the relationship type corpus includes at least one pair of text samples, and each pair of text samples is labeled with a preset relationship type between each pair of text samples;

an input module 503, configured to input a feature of the target text into a pre-trained natural language understanding model, so as to obtain an understanding result of the target text under at least two natural language understanding tasks, where the natural language understanding model is obtained according to a natural language understanding sub-model corresponding to each natural language understanding task, the natural language understanding sub-model is obtained by training according to a training corpus of each natural language understanding task, and a preset keyword corresponding to each natural language understanding task is labeled in the training corpus of each natural language understanding task.

the input module 503 is specifically configured to:

The natural language understanding apparatus, implementation process and implementation principle of this embodiment may refer to the description related to the natural language understanding method in the foregoing method embodiments, and are not described herein again.

Fig. 13 is a schematic structural diagram of a natural language understanding model training device provided in an embodiment of the present application, and as shown in fig. 13, the natural language understanding model training device 60 includes: a processor 601, a memory 602 and a bus 603, wherein the memory 602 stores a computer program executable by the processor 601, when the natural language understanding model training device 60 runs, the processor 601 and the memory 602 communicate with each other through the bus 603, and the processor 601 executes the computer program to execute the natural language understanding model training method.

Fig. 14 is a schematic structural diagram of a natural language understanding apparatus provided in an embodiment of the present application, and as shown in fig. 14, the natural language understanding apparatus 70 includes: a processor 701, a memory 702 and a bus 703, wherein the memory 702 stores a computer program executable by the processor 701, when the natural language understanding apparatus 70 runs, the processor 701 communicates with the memory 702 through the bus 703, and the processor 701 executes the computer program to perform the natural language understanding method.

The embodiment of the present application further provides a storage medium, where a computer program is stored on the storage medium, and the computer program is executed by a processor to perform the above method embodiments.

It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to corresponding processes in the method embodiments, and are not described in detail in this application. In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. The above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is merely a logical division, and there may be other divisions in a predetermined implementation, and for example, a plurality of modules or components may be combined or integrated with another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or modules through some communication interfaces, and may be in an electrical, mechanical or other form.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application.

Claims

1. A natural language understanding model training method, comprising:

2. The method according to claim 1, wherein the performing model training on a preset multilayer perceptron model according to the features of the corpus of each natural language understanding task to obtain the natural language understanding submodel corresponding to each natural language understanding task comprises:

3. The method according to claim 1 or 2, wherein said extracting the features of the corpus of each natural language understanding task comprises:

4. The method of claim 3, wherein the feature extractor is trained by:

obtaining a relation type training corpus;

5. The method according to claim 4, wherein the training the pre-trained language model according to the preset relationship type and the training relationship type to obtain the feature extractor comprises:

6. The method of claim 4, wherein the classifier comprises a linear transformer and an activator, and wherein inputting the stitching features into the classifier to obtain the training relationship type between each pair of text samples comprises:

7. A natural language understanding method, comprising:

acquiring a target text;

8. The method of claim 7, wherein the at least two natural language understanding tasks include at least one of:

9. The method of claim 8, wherein if the at least two natural language understanding tasks include a relationship extraction task, an emotion analysis task, an intent recognition task, a topic parsing task, a slot filling task;

10. A natural language understanding model training apparatus, comprising:

11. A natural language understanding apparatus, comprising:

the acquisition module is used for acquiring a target text;

12. A natural language understanding model training apparatus, comprising: a processor, a memory and a bus, the memory storing a computer program executable by the processor, the processor and the memory communicating via the bus when the natural language understanding model training device is running, the processor executing the computer program to perform the method of any one of claims 1-6.

13. A natural language understanding apparatus, comprising: a processor, a memory and a bus, the memory storing a computer program executable by the processor, the processor and the memory communicating via the bus when the natural language understanding apparatus is running, the processor executing the computer program to perform the method of any one of claims 7-9.

14. A storage medium, characterized in that the storage medium has stored thereon a computer program which, when being executed by a processor, performs the method of any one of claims 1-9.