CN117951515A

CN117951515A - Model training method and device

Info

Publication number: CN117951515A
Application number: CN202311017099.1A
Authority: CN
Inventors: 白安琪; 蒋宁; 陆全; 夏粉; 吴海英; 肖冰
Original assignee: Mashang Xiaofei Finance Co Ltd
Current assignee: Mashang Xiaofei Finance Co Ltd
Priority date: 2023-08-11
Filing date: 2023-08-11
Publication date: 2024-04-30

Abstract

The embodiment of the specification provides a model training method and device, wherein the model training method comprises the following steps: determining the type of the participatory word of the first sentence sample; performing the virtual word transformation on the first sentence sample according to the virtual word transformation mode corresponding to the virtual word type to obtain a second sentence sample; model training is carried out on the grammar recognition model to be trained based on the second sentence sample to obtain a trained grammar recognition model, and model training is carried out on the semantic recognition model to be trained based on the first sentence sample and the second sentence sample to obtain a trained semantic recognition model; the trained grammar recognition model is used for carrying out grammar recognition on the sentences to be processed to obtain grammar types, and the trained semantic recognition model is used for carrying out semantic recognition on the sentences to be processed to obtain semantic information. By adopting the embodiment of the application, the comprehensiveness and accuracy of model training can be improved.

Description

Model training method and device

Technical Field

The present application relates to the field of data processing technologies, and in particular, to a model training method and apparatus.

Background

With the rapid development of artificial intelligence technology, sentence recognition is widely applied, for example, in the process of dialogue between intelligent customer service and users, the user sentences are subjected to intention recognition and the like, in the process of sentence recognition, the sentence features capable of representing sentence information are required to be recognized, the sentence features can reflect the whole situation of the sentence, the sentence is roughly described, and in the process, the generation of the sentence features gradually becomes the focus of more and more researchers.

Disclosure of Invention

In a first aspect, an embodiment of the present application provides a model training method, where the method includes:

determining the type of the participatory word of the first sentence sample;

Performing the virtual word transformation on the first sentence sample according to the virtual word transformation mode corresponding to the virtual word type to obtain a second sentence sample;

Model training is carried out on the grammar recognition model to be trained based on the second sentence sample to obtain a trained grammar recognition model, and model training is carried out on the semantic recognition model to be trained based on the first sentence sample and the second sentence sample to obtain a trained semantic recognition model;

The training grammar recognition model is used for carrying out grammar recognition on the sentence to be processed to obtain a grammar type, and the training semantic recognition model is used for carrying out semantic recognition on the sentence to be processed to obtain semantic information so as to carry out sentence recognition on the sentence to be processed according to sentence characteristics obtained by fusion of the grammar type and the semantic information.

It can be seen that, in the embodiment of the present application, a virtual word transformation manner is determined by using a virtual word type of a first sentence sample, and virtual word transformation is performed on the first sentence sample according to the virtual word transformation manner, so as to obtain a second sentence sample, model training is performed on a grammar recognition model to be trained by using the second sentence sample, so as to obtain a trained grammar recognition model, the trained grammar recognition model is obtained by combining the first sentence sample and the second sentence sample, grammar recognition is performed on a sentence to be processed by using the trained grammar recognition model, semantic information is obtained by using the trained grammar recognition model to be processed, and sentence recognition is performed by using sentence features obtained by fusing the grammar type and the semantic information, so that model training is performed on the grammar recognition model by using the first sentence sample before virtual word transformation and the second sentence sample after virtual word transformation, so as to improve the attention degree of virtual word in the sentence to be processed by using the trained semantic recognition model, and further improve the accuracy of semantic recognition, and simultaneously, by combining the information output after training and the grammar recognition model in the process of obtaining sentence features, thereby improving the accuracy of the grammar recognition model.

In a second aspect, an embodiment of the present application provides a sentence processing method, where the method includes:

Acquiring a statement to be processed;

inputting the sentence to be processed into a grammar recognition model for grammar recognition, obtaining the grammar type of the sentence to be processed, and generating grammar characteristics of the grammar type; the grammar type is determined based on real word characteristics and imaginary word characteristics of the sentence to be processed;

inputting the sentence to be processed into a semantic recognition model for semantic recognition to obtain semantic information of the sentence to be processed, and generating semantic features of the semantic information;

and carrying out feature fusion on the grammar features and the semantic features to obtain sentence features of the sentence to be processed, wherein the sentence features are used for carrying out sentence recognition of the sentence to be processed.

It can be seen that, in the embodiment of the application, on one hand, grammar recognition is performed on the basis of a sentence to be processed through a grammar recognition model to obtain a grammar type of the sentence to be processed, in the process of grammar recognition, the grammar type is determined on the basis of real word features and imaginary word features of the sentence to be processed, and grammar features of the grammar type are generated, on the other hand, semantic recognition is performed on the basis of the sentence to be processed through a semantic recognition model to obtain semantic information of the sentence to be processed and generate semantic features of the semantic information, on the basis, feature fusion is performed on the grammar features and the semantic features to obtain the sentence features of the sentence to be processed for sentence recognition of the sentence to be processed, so that the sentence features are determined by combining the grammar features and the semantic features, the comprehensiveness and the effectiveness of the sentence features are improved, the accuracy of sentence recognition is further improved, and meanwhile, the grammar type is determined by combining the real word features and the imaginary word features, and the accuracy of the grammar types is improved.

In a third aspect, an embodiment of the present application provides a model training apparatus, including:

the type determining module is used for determining the type of the virtual word of the first sentence sample;

the virtual word transformation module is used for carrying out virtual word transformation on the first sentence sample according to a virtual word transformation mode corresponding to the virtual word type to obtain a second sentence sample;

The model training module is used for carrying out model training on the grammar recognition model to be trained based on the second sentence sample to obtain a trained grammar recognition model, and carrying out model training on the semantic recognition model to be trained based on the first sentence sample and the second sentence sample to obtain a trained semantic recognition model;

In a fourth aspect, an embodiment of the present application provides a sentence processing apparatus, including:

The sentence acquisition module is used for acquiring sentences to be processed;

The grammar characteristic generation module is used for inputting the sentence to be processed into a grammar recognition model for grammar recognition, obtaining the grammar type of the sentence to be processed and generating the grammar characteristic of the grammar type; the grammar type is determined based on real word characteristics and imaginary word characteristics of the sentence to be processed;

The semantic feature generation module is used for inputting the sentence to be processed into a semantic recognition model for semantic recognition to obtain semantic information of the sentence to be processed, and generating semantic features of the semantic information;

And the feature fusion module is used for carrying out feature fusion on the grammar features and the semantic features to obtain sentence features of the sentences to be processed, so as to be used for carrying out sentence recognition of the sentences to be processed.

In a fifth aspect, an embodiment of the present application provides a computer apparatus, including: a processor; and a memory configured to store computer-executable instructions that, when executed, cause the processor to perform the model training method of the first aspect.

In a sixth aspect, an embodiment of the present application provides another computer apparatus, including: a processor; and a memory configured to store computer-executable instructions that, when executed, cause the processor to perform the sentence processing method of the second aspect.

In a seventh aspect, embodiments of the present application provide a computer readable storage medium storing computer executable instructions that, when executed by a processor, implement the model training method of the first aspect.

In an eighth aspect, embodiments of the present application provide another computer-readable storage medium storing computer-executable instructions which, when executed by a processor, implement the sentence processing method according to the second aspect.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, it will be obvious that the drawings in the following description are only some embodiments described in the present specification, and that other drawings can be obtained according to these drawings without inventive effort to a person skilled in the art;

FIG. 1 is a process flow diagram of a model training method according to an embodiment of the present application;

FIG. 2 is a schematic diagram of a model training process according to an embodiment of the present application;

FIG. 3 is a process flow diagram of a model training method applied to a resource processing scenario according to an embodiment of the present application;

FIG. 4 is a flowchart illustrating a sentence processing method according to an embodiment of the present application;

FIG. 5 is a schematic diagram of a model training apparatus according to an embodiment of the present application;

FIG. 6 is a schematic diagram of a sentence processing device according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of a computer device according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of another computer device according to an embodiment of the present application.

Detailed Description

In order to enable those skilled in the art to better understand the technical solutions of the embodiments of the present application, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present specification, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present application without making any inventive effort, shall fall within the scope of the present application.

The embodiment of a model training method provided in the specification is as follows:

in practical application, in the training process of the semantic recognition model, the degree of attention to real words in sentence samples is higher, the degree of attention to virtual words is insufficient or the virtual words are directly ignored, and under the condition that virtual words are added into sentences and virtual words are not added into sentences, the influence on the semantics is larger, namely, under the condition that virtual words are added into sentences and virtual words are not added into sentences, the semantics are different, the accuracy of the trained semantic recognition model is low, in addition, in the process of acquiring sentence characteristics for carrying out sentence recognition, only semantic characteristics are adopted, and the training of the grammar recognition model is not carried out.

In view of this, in the model training method provided in this embodiment, first, the first sentence sample is subjected to the virtual word transformation according to the virtual word transformation mode corresponding to the word type of the first sentence sample, so as to obtain the second sentence sample; secondly, on one hand, training a grammar recognition model to be trained by means of a second sentence sample to obtain a trained grammar recognition model, on the other hand, training the grammar recognition model to be trained by means of a first sentence sample and a second sentence sample to obtain a trained semantic recognition model, carrying out grammar recognition on sentences to be processed by means of the trained grammar recognition model to obtain grammar types, carrying out semantic recognition on sentences to be processed by means of the trained semantic recognition model to obtain semantic information, carrying out sentence recognition on sentence characteristics obtained by means of fusion of the grammar types and the semantic information, and carrying out model training on the semantic recognition model by means of the first sentence sample before the change of the virtual words and the second sentence sample after the change of the virtual words, so that the attention degree of the virtual words in the sentences to be processed by the trained semantic recognition model is improved, and further the accuracy of the semantic recognition is improved.

Fig. 1 is a process flow diagram of a model training method according to an embodiment of the present application. The model training method of fig. 1 may be performed by an electronic device, which may be a terminal device, such as a mobile phone, a notebook computer, an intelligent interaction device, etc.; or the electronic device may also be a server, such as a stand-alone physical server, a cluster of servers, or a cloud server capable of cloud computing. Referring to fig. 1, the model training method provided in this embodiment specifically includes steps S102 to S106.

Step S102, determining the type of the participatory word of the first sentence sample.

The first sentence sample in this embodiment includes sentence samples constructed by one or more historical sentences, that is, the first sentence sample may include one or more historical sentences, and the form of the historical sentences may be text; the history statement may be a statement in any scenario, such as the history statement "you or your manager" in the resource processing scenario; the sentences contained in the first sentence sample can also be historical user sentences corresponding to the user voice data generated in the dialogue process of the intelligent customer service. The first sentence sample can be obtained according to one or more sentence construction extracted from a grammar library; the grammar library may have stored therein one or more grammar types and one or more statements under the grammar types.

It should be noted that the first sentence sample may be a sentence sample of any language, for example, the first sentence sample may be a first chinese sentence sample, a first Tibetan sentence sample, a first english sentence sample, and the first sentence sample may also be a sentence sample of another language type.

The term type refers to whether the first sentence sample contains a term or not; the types of the works include a first type including works and/or a second type not including works.

The term in this embodiment refers to a word having no complete meaning but a grammatical meaning or function, and the term is attached to a real word or sentence to represent a grammar, cannot be separately sentence-formed, cannot be separately used as a grammatical component, for example, a place, or a place.

In specific implementation, in the process of determining the type of the article of the first sentence sample, the following operations may be performed:

detecting whether the first sentence sample contains an imaginary term or not;

If yes, determining that the type of the participle of the first sentence sample is a first type containing the participle;

if not, determining that the type of the participle of the first sentence sample is a second type which does not contain the participle.

In addition, in order to improve the efficiency of determining the type of the participatory words, a participatory word detection algorithm can be introduced, and on the basis, in the process of determining the type of the participatory words of the first sentence sample, the first sentence sample can be input into the participatory word detection algorithm to perform the participatory word detection, so that the type of the participatory words of the first sentence sample can be obtained; the term detection algorithm may be a regularization algorithm, and in particular may be a regularized expression.

And step S104, performing the virtual word transformation on the first sentence sample according to the virtual word transformation mode corresponding to the virtual word type to obtain a second sentence sample.

In the step of determining the type of the virtual word of the first sentence sample, in order to obtain the sentence sample after the virtual word transformation to perform model training on the semantic recognition model to be trained, the virtual word transformation can be performed on the first sentence sample according to the virtual word transformation mode corresponding to the type of the virtual word to obtain the second sentence sample.

The term transformation method in this embodiment refers to a method of performing term transformation on a first sentence sample; the virtual word transformation mode comprises a first transformation mode and/or a second transformation mode; the first transformation mode may be a term deletion mode or a term replacement mode, and the second transformation mode may be a term filling mode.

In particular implementation, in order to improve flexibility and convenience of the term transformation, in an alternative implementation provided in this embodiment, in a process of performing the term transformation on the first sentence sample according to the term transformation mode corresponding to the term type, the following operations are performed:

If the type of the virtual word is a first type containing the virtual word, deleting or replacing the virtual word contained in the first sentence sample according to a first transformation mode corresponding to the first type to obtain the second sentence sample;

And if the type of the virtual word is a second type which does not contain the virtual word, filling the virtual word into the first sentence sample according to a second transformation mode corresponding to the second type, so as to obtain the second sentence sample.

Specifically, in the process of deleting or replacing the virtual words contained in the first sentence sample, deleting or replacing each virtual word contained in the first sentence sample to obtain a second sentence sample, namely, performing virtual word transformation on each virtual word contained in the first sentence sample in a deleting mode or a replacing mode to obtain the second sentence sample;

In the process of replacing the works included in the first sentence sample, the works included in the first sentence sample may be replaced based on the target works, where the target works refer to works different from the works included in the first sentence sample.

In the process of filling the first sentence sample with the dummy word, the filling bit in the first sentence sample can be determined, the filling dummy word of the filling bit is randomly determined in the dummy word library, and the filling dummy word is filled into the filling bit. Wherein the padding bits may be one or more; the filling articles of each filling bit can be the same or different. One or more articles may be stored in the article library.

For example, if the first sentence sample s1 is "a jin of fish", and the type of the works of the first sentence sample s1 is the first type including works, deleting the works of the works included in the first sentence sample s1 "a jin of fish" to obtain a second sentence sample "a jin of fish"; for another example, if the first sentence sample s2 is "you or your manager", and the type of the article of the first sentence sample s2 is a second type that does not include the article, then the first sentence sample s2 "you or your manager" is filled with the article, and "you or your manager" can be filled with "to obtain a second sentence sample" your or your manager "; it should be noted that, the method of transforming the virtual word of the first sentence sample is merely illustrative, and the method of transforming the virtual word of the first sentence sample may be flexibly selected, and determined according to the actual application scenario.

Step S106, model training is carried out on the grammar recognition model to be trained based on the second sentence sample, a trained grammar recognition model is obtained, model training is carried out on the semantic recognition model to be trained based on the first sentence sample and the second sentence sample, and a trained semantic recognition model is obtained.

In the step, in order to improve the use effects of the grammar recognition model and the semantic recognition model, model training can be performed on the grammar recognition model to be trained based on the first sentence sample to obtain a trained grammar recognition model, and model training is performed on the semantic recognition model to be trained based on the first sentence sample and the second sentence sample to obtain a trained semantic recognition model. Optionally, the trained grammar recognition model is used for carrying out grammar recognition on the sentence to be processed to obtain a grammar type, the trained semantic recognition model is used for carrying out semantic recognition on the sentence to be processed to obtain semantic information, and sentence recognition is carried out on the sentence to be processed according to sentence characteristics obtained by fusion of the grammar type and the semantic information.

Optionally, the sentence feature is obtained after feature fusion is performed on the grammar feature of the grammar type and the semantic feature of the semantic information. The grammar characteristics can be obtained by the following ways: searching the grammar characteristics mapped by the grammar types in a grammar characteristic mapping table; the semantic features can be obtained by searching the semantic features mapped by the semantic information in a semantic feature mapping table.

The grammar recognition model in this embodiment refers to a model for grammar recognition; the grammar recognition model may employ any one of a number of language models, such as Bart, roberta, bert, mengzi or Ernie models. The semantic recognition model refers to a model for semantic recognition, and the semantic recognition model can be any language model, such as Bart, roberta, bert, mengzi or Ernie models.

In practical application, in the process of grammar recognition, the grammar recognition model has higher attention to real words in sentences and insufficient attention to virtual words possibly, so that the grammar recognition accuracy is low, and for the purpose of improving the grammar recognition accuracy of the grammar recognition model, the grammar recognition model can be trained through a second sentence sample after virtual word transformation; in an optional implementation manner provided in this embodiment, in a process of performing model training on a grammar recognition model to be trained based on the second sentence sample to obtain a trained grammar recognition model, the following operations are performed:

Inputting the second sentence sample into the grammar recognition model to be trained for grammar recognition to obtain a predicted grammar type;

And calculating training loss based on the grammar type label corresponding to the predicted grammar type and the second sentence sample, and performing parameter adjustment on the grammar recognition model to be trained based on the training loss to obtain the trained grammar recognition model.

The prediction grammar type refers to a grammar type of a sentence obtained by grammar recognition, for example, the prediction grammar type comprises a main predicate grammar type, a moving object grammar type, a middle complement grammar type, a continuous predicate grammar type or a preposition grammar type, and the prediction grammar type is only illustrative and can be other grammar types. The grammar type label corresponding to the second sentence sample refers to a label of the second sentence sample aiming at the grammar type; the grammar type tag can be obtained by manually marking.

Referring to the training process of the grammar recognition model to be trained, iterative training is performed on the grammar recognition model to be trained, that is, after parameter adjustment is performed on the grammar recognition model to be trained based on training loss, the process of inputting the second sentence sample into the grammar recognition model to be trained for grammar recognition, calculating training loss and performing parameter adjustment can be returned until the grammar recognition model to be trained converges, and the trained grammar recognition model is obtained.

On this basis, in an alternative implementation manner provided in this embodiment, in order to improve accuracy of grammar recognition performed by the grammar recognition model to be trained, grammar recognition is performed by combining two dimensions of real words and imaginary words, so as to improve comprehensiveness and integrity of grammar recognition, and the grammar recognition model to be trained performs grammar recognition in the following manner:

Performing feature extraction processing based on the second sentence sample to obtain real word features and virtual word features of the second sentence sample;

And determining the prediction grammar type according to the real word characteristics and the imaginary word characteristics.

The real word characteristics of the second sentence sample refer to real word characteristics of real words contained in the second sentence sample; the term characteristics of the second sentence sample refer to term characteristics of terms contained in the second sentence sample; the real word feature and the imaginary word feature herein may be real word feature vectors and imaginary word feature vectors.

Further, in order to improve accuracy of the prediction grammar type, in an alternative implementation manner provided in this embodiment, in determining the prediction grammar type according to the real word feature and the imaginary word feature of the second sentence sample, the following operations are performed:

Performing similarity calculation based on the real word characteristics and real word characteristics of a plurality of sentences in a grammar library to obtain a plurality of real word similarities, and performing similarity calculation based on the virtual word characteristics and virtual word characteristics of the plurality of sentences to obtain a plurality of virtual word similarities;

calculating the similarity of each sentence according to the similarity of each real word in the plurality of real word similarities and the similarity of the virtual word corresponding to the similarity of each real word in the plurality of virtual word similarities;

And determining target sentence similarity in the sentence similarity, and taking a target grammar type corresponding to the target sentence similarity as the prediction grammar type.

The grammar library comprises a grammar library, a plurality of grammar types and a plurality of grammar types, wherein the grammar library comprises a plurality of grammar types; the real word characteristics of the multiple sentences can be obtained by extracting the real word characteristics of the multiple sentences; the characteristic of the virtual word of the plurality of sentences can be obtained by extracting the characteristic of the virtual word of the plurality of sentences. The similarity calculation comprises Euclidean distance calculation or cosine distance calculation; the similarity comprises Euclidean distance or cosine distance of real words; the plurality of the article similarity includes a euclidean distance or a cosine distance of the plurality of articles.

Specifically, in the process of performing feature extraction processing based on the second sentence sample to obtain the real word feature and the imaginary word feature of the second sentence sample, the following operations may be performed: performing word segmentation processing based on the second sentence sample to obtain a plurality of word segments of the second sentence sample; clustering the plurality of word fragments according to the part of speech of the plurality of word fragments to obtain a real word sequence and an imaginary word sequence of the second sentence sample; extracting real word characteristics from real word sequences and extracting imaginary word characteristics from imaginary word sequences.

In the process of calculating the sentence similarity according to the real word similarity in the real word similarities and the virtual word similarity corresponding to the real word similarity in the virtual word similarities, the sum of the real word similarity and the corresponding virtual word similarity in the real word similarities can be directly calculated to be used as the sentence similarity; the corresponding term similarity is among the plurality of term similarities.

In the process of determining the target sentence similarity in the sentence similarity, sorting processing can be performed on the sentence similarity, and the sentence similarity with the sorting order before the preset order is used as the target sentence similarity.

In addition, in the process of calculating the sentence similarity according to the real word similarity in the real word similarities and the virtual word similarity corresponding to the real word similarity in the virtual word similarities, weighting calculation can be performed according to the real word similarity in the real word similarities and the virtual word similarity corresponding to the real word similarity in the virtual word similarities to obtain the sentence similarity; specifically, a first product of each real word similarity and real word weight in the plurality of real word similarities can be calculated, a second product of the real word similarity and the real word weight corresponding to each real word similarity in the plurality of real word similarities is calculated, and each sentence similarity is calculated based on the first product of each real word similarity and the real word weight and the second product of the real word similarity and the real word weight corresponding to each real word similarity; specifically, the sum of the first product and the second product can be calculated as the similarity of each sentence.

In addition, optionally, the grammar recognition model to be trained comprises a feature extraction module and a grammar recognition module; the feature extraction module is used for carrying out feature extraction processing based on the second sentence sample to obtain real word features and virtual word features of the second sentence sample; the grammar recognition module is used for determining a prediction grammar type according to real word characteristics and virtual word characteristics of the second sentence sample; the specific implementation process of the feature extraction process and the determination of the prediction grammar type is described in detail above, and will not be described herein. In this case, in the process of inputting the second sentence sample into the grammar recognition model to be trained to perform grammar recognition to obtain the predicted grammar type, the feature extraction module in the grammar recognition model to be trained may input the second sentence sample into the feature extraction module to perform feature extraction processing to obtain the real word feature and the virtual word feature of the second sentence sample, and the real word feature and the virtual word feature of the second sentence sample are input into the grammar recognition module in the grammar recognition model to be trained to perform grammar type determination to obtain the predicted grammar type.

In addition, after the second sentence sample is input into the grammar recognition model to be trained, the grammar recognition model to be trained can also perform grammar recognition in the following manner: extracting features based on a second sentence sample to obtain sentence features of the second sentence sample, performing similarity calculation on the sentence features and the sentence features in the grammar library, determining target similarity in the calculated similarities, and taking a grammar type corresponding to the target similarity as the predicted grammar type; in determining the target similarity among the similarities, the target similarity whose ranking order is before the preset order may be determined among the similarities.

In practical application, in the process of carrying out sentence recognition on a sentence to be processed, sentence characteristics are needed, so that the sentence characteristics can more completely represent the characteristics of the sentence to be processed in order to improve the comprehensiveness and the effectiveness of the sentence characteristics, and the accuracy of sentence recognition is further improved; for this, semantic features and grammar features can be introduced and combined to obtain sentence features, in the process of obtaining grammar features, the process of training the grammar recognition model in advance is described in detail, and in the process of obtaining semantic features, in order to improve the convenience of obtaining semantic features, the process of training in advance can also be used for obtaining the semantic recognition model; in an optional implementation manner provided in this embodiment, in a process of performing model training on a semantic recognition model to be trained based on a first sentence sample and a second sentence sample to obtain a trained semantic recognition model, the following operations are performed:

Inputting the first sentence sample and the second sentence sample into the semantic recognition model to be trained for semantic recognition to obtain first semantic information and second semantic information;

Calculating a loss value based on the first semantic information, the second semantic information and the semantic comparison label corresponding to the second sentence sample, and performing parameter adjustment on the semantic recognition model to be trained based on the loss value to obtain the trained semantic recognition model.

The first semantic information refers to semantic information of a first sentence sample; the second semantic information refers to semantic information of a second sentence sample; the semantic comparison label corresponding to the second sentence sample is a comparison result label for performing semantic comparison on the semantic information of the second sentence sample and the semantic information of the first sentence sample; the semantic comparison tag comprises a first semantic comparison tag, a second semantic comparison tag and/or a third semantic comparison tag; the first semantic comparison label comprises a semantic non-compliance label, the second semantic comparison label comprises a semantic offset label, and the third semantic comparison label comprises a semantic unclear label. The semantic comparison label can be obtained through manual labeling.

The semantic recognition model to be trained can carry out semantic recognition in the following mode: calculating first real word characteristics and first virtual word characteristics of a first sentence sample, calculating second real word characteristics and second virtual word characteristics of a second sentence sample, performing characteristic splicing on the first real word characteristics and the first virtual word characteristics to obtain first splicing characteristics, performing characteristic splicing on the second real word characteristics and the second virtual word characteristics to obtain second splicing characteristics, and determining first semantic information corresponding to the first splicing characteristics and second semantic information corresponding to the second splicing characteristics.

On the basis of the above, in an optional implementation manner provided in this embodiment, in a process of calculating a loss value based on the first semantic information, the second semantic information and the semantic comparison tag corresponding to the second sentence sample, the following operations are performed:

Performing fusion processing based on the first semantic information and the second semantic information to obtain semantic fusion characteristics;

And carrying out semantic comparison analysis on the semantic fusion features to obtain semantic comparison information, and calculating the loss value according to the semantic comparison information and the semantic comparison label.

The semantic comparison information is comparison information obtained by performing semantic comparison on the second semantic information of the second sentence sample and the first semantic information of the first sentence sample; the semantic comparison information may be semantic variation information of second semantic information of the second sentence sample compared with first semantic information of the first sentence sample, and the semantic comparison information may be semantic non-compliance, semantic offset or semantic unclear, for example, the first sentence sample is "one jin of fish", the second sentence sample is "one jin of fish", and the second semantic information of the second sentence sample is semantic offset (semantic deviation) compared with the first semantic information of the first sentence sample.

Specifically, in the process of obtaining the semantic fusion feature based on the first semantic information and the second semantic information through fusion processing, the first semantic feature of the first semantic information and the second semantic feature of the second semantic information can be generated, and the product of the first semantic feature and the second semantic feature is calculated to be used as the semantic fusion feature; in the process of carrying out semantic comparison analysis on the semantic fusion features to obtain semantic comparison information, feature conversion can be carried out on the semantic fusion features to obtain target semantic features, similarity is calculated according to the target semantic features and the comparison features of the target semantic features and the reference semantic comparison information, the target similarity is determined in the similarity, and the reference semantic comparison information corresponding to the target similarity is used as the semantic comparison information.

In addition, in the process of training the semantic recognition model to be trained based on the first sentence sample and the second sentence sample to obtain a trained semantic recognition model, the first sentence sample and the second sentence sample can be input into an attention module in the semantic recognition model to be trained to perform attention calculation to obtain a first attention feature and a second attention feature, and the first attention feature and the second attention feature are input into the semantic recognition module in the semantic recognition model to be trained to obtain first semantic information and second semantic information; and calculating a loss value based on the semantic comparison information and the semantic comparison label corresponding to the second sentence sample, and carrying out parameter adjustment on the attention module and the semantic recognition module based on the loss value to obtain the trained semantic recognition model.

The semantic comparison information can be obtained by inputting semantic fusion features into a semantic comparison module in a semantic recognition model to be trained for semantic comparison analysis; the semantic fusion feature can be obtained by inputting the first semantic information and the second semantic information into a fusion module in a semantic recognition model to be trained and carrying out feature fusion on the first semantic feature of the first semantic information and the second semantic feature of the second semantic information.

Optionally, the semantic recognition model to be trained includes an attention calculation module, a semantic recognition module, a fusion module and a semantic comparison module; the trained semantic recognition model comprises an attention calculation module and a semantic recognition module. The attention calculating module is used for carrying out weight calculation on a first word segmentation vector in the first sentence sample and a second word segmentation vector in the second sentence sample to obtain a first attention characteristic and a second attention characteristic; the first attention characteristic is used for representing the similarity (cosine similarity) between each word segment in the first sentence sample and the first sentence sample, namely the importance degree of each word segment in the first sentence sample; the second attention characteristic is used for representing the similarity between each word segment in the second sentence sample and the second sentence sample.

In the specific implementation, after the trained grammar recognition model and the trained semantic recognition model are obtained, the sentence to be processed can be processed, specifically, the sentence to be processed can be input into the grammar recognition model for grammar recognition, the grammar type of the sentence to be processed is obtained, and the grammar characteristics of the grammar type are generated; inputting the sentence to be processed into a semantic recognition model for semantic recognition to obtain semantic information of the sentence to be processed and generating semantic features of the semantic information; and carrying out feature fusion on the semantic features and the semantic features to obtain sentence features of the sentences to be processed, so as to be used for carrying out sentence recognition of the sentences to be processed.

Wherein the statement to be processed refers to a statement to be processed; the statement to be processed can be any statement; the form of the sentence to be processed may be a text form; the statement to be processed may be one or more statements; in addition, the statement to be processed can also be a statement in any scene, for example, the statement to be processed is a statement "day and night is repayment day" in a resource processing scene; the sentence to be processed can also be a user sentence corresponding to the user voice data generated in the dialogue process of the user and the intelligent customer service. The grammar type is similar to the prediction grammar type, and refers to the grammar type of the sentence to be processed, which is obtained by carrying out grammar recognition on the sentence to be processed, for example, the grammar type comprises a main predicate method type, a moving object method type, a middle complement method type, a continuous predicate method type or a preposition grammar type, the example of the grammar type is only schematic, and in addition, the grammar type can be other grammar types. The semantic information is information representing meaning or meaning of a sentence to be processed; the grammar features refer to feature vectors that characterize the grammar types; the semantic features refer to feature vectors that characterize the semantic information.

Optionally, the grammar type is determined based on real word characteristics and imaginary word characteristics of the sentence to be processed. In a specific execution process, the trained grammar recognition model can carry out grammar recognition on the sentence to be processed in the following manner:

Carrying out feature extraction processing on the sentence to be processed to obtain real word features and virtual word features of the sentence to be processed;

and determining the grammar type of the sentence to be processed according to the real word characteristics and the imaginary word characteristics of the sentence to be processed.

The real word characteristics of the sentence to be processed refer to real word characteristics of real words contained in the sentence to be processed; the real word feature may be a real word feature vector; the term characteristics of the term to be processed refer to term characteristics of the term contained in the term to be processed, and the term characteristics can be term characteristic vectors.

Specifically, in the process of performing feature extraction processing on a sentence to be processed to obtain real word features and imaginary term features of the sentence to be processed, word segmentation processing can be performed on the sentence to be processed to obtain a plurality of words of the sentence to be processed, clustering processing is performed on a plurality of words of the sentence to be processed according to the part of speech of the plurality of words of the sentence to be processed to obtain a real word sequence and an imaginary term sequence of the sentence to be processed, the real word features are extracted from the real word sequence of the sentence to be processed, and the imaginary term features are extracted from the imaginary term sequence of the sentence to be processed.

In the process of determining the grammar type of the sentence to be processed according to the real word characteristics and the imaginary term characteristics of the sentence to be processed, similarity calculation can be performed based on the real word characteristics of the sentence to be processed and the real word characteristics of a plurality of reference sentences in a grammar library to obtain a plurality of real word similarities, and similarity calculation is performed based on the imaginary term characteristics of the sentence to be processed and the imaginary term characteristics of a plurality of reference sentences to obtain a plurality of imaginary term similarities; calculating the similarity of each sentence according to the similarity of each real word in the plurality of real word similarities and the similarity of the virtual word corresponding to the similarity of each real word in the plurality of real word similarities, determining the similarity of the target sentence in the similarity of each sentence, and taking the target grammar type corresponding to the similarity of the target sentence as the grammar type. The reading can be referred to here similarly to the process of performing grammar recognition based on the second sentence sample by the grammar recognition model to be trained in the model training of the grammar recognition model to be trained.

The trained semantic recognition model can carry out semantic recognition on the statement to be processed in the following mode to obtain semantic information:

Word segmentation processing is carried out on sentences to be processed to obtain a plurality of words, and attention calculation is carried out on the words to obtain attention characteristics;

And performing feature conversion on the attention features to obtain conversion features, and determining semantic information corresponding to the conversion features as semantic information of the sentences to be processed.

In addition, in order to more completely express the semantic information of the sentence to be processed, besides the whole consideration of the sentence to be processed, the semantic information can be determined by combining the two angles of the real word and the virtual word of the sentence to be processed, so that the comprehensiveness and the effectiveness of the semantic information can be improved; for this, the trained semantic recognition model may also perform semantic recognition on the statement to be processed to obtain semantic information in the following manner:

Calculating real word characteristics and imaginary word characteristics of the sentence to be processed based on the sentence to be processed;

and performing feature splicing on the real word features and the imaginary word features of the sentences to be processed to obtain spliced features, and determining semantic information corresponding to the spliced features as semantic information of the sentences to be processed.

Specifically, in the process of calculating real word characteristics and imaginary term characteristics of the to-be-processed sentence based on the to-be-processed sentence, word segmentation processing can be performed on the to-be-processed sentence, real words and imaginary words in a plurality of segmented words after word segmentation are determined, and a real word sequence and an imaginary word sequence of the to-be-processed sentence are obtained; extracting real word characteristics from real word sequences of sentences to be processed and extracting imaginary word characteristics from imaginary word sequences of the sentences to be processed.

In a specific execution process, after the grammar type of the sentence to be processed is obtained, grammar characteristics of the grammar type can be generated, and after semantic information of the sentence to be processed is obtained, semantic characteristics of the semantic information can be generated; in this case, in order to promote the comprehensiveness and effectiveness of the sentence features, in the process of performing sentence recognition on the sentence to be processed according to the sentence features obtained by fusing the grammar type and the semantic information, the following operations may be performed:

calculating a first probability of the sentence to be processed under each intention according to the sentence characteristics, and determining the sentence intention of the sentence to be processed in each intention based on the first probability;

And/or the number of the groups of groups,

And calculating a second probability of the sentence to be processed under each emotion polarity according to the sentence characteristics, and determining the emotion polarity of the sentence to be processed in each emotion polarity based on the second probability.

The statement intention refers to the intention of a statement to be processed, for example, the statement to be processed is "I want to loan back", and the statement intention of the statement to be processed is loan back; the emotion polarity includes negative and positive, for example, the emotion polarity of the sentence to be processed is negative, namely, how not to issue loans.

In the specific implementation, in order to promote the persistence of model training, grammar type label updating and/or semantic comparison label updating can be performed on the target sentence sample, so that the model training of the next grammar recognition model and/or semantic recognition model can be performed according to the updated target sentence sample; the target sentence sample can be a second sentence sample or a reconstructed sentence sample; specifically, in the process of updating the grammar type tag of the second sentence sample, the following operations may be performed:

Obtaining a target sentence sample carrying grammar type tags and/or semantic comparison tags; the target sentence sample comprises a plurality of sample sentences;

inputting the plurality of sample sentences into a sentence pair extraction model to extract sentence pairs, so as to obtain sentence pairs with the same real words and different virtual words;

carrying out grammar recognition on the sentence pair input trained grammar recognition model to obtain grammar types of each sentence in the sentence pair;

If the grammar types of the sentences in the sentence pair are different and the grammar type labels of the sentences are the same, marking or updating the grammar type labels of the sentences in the sentence pair to obtain an updated target sentence sample for model training of the grammar recognition model.

After the sentence pairs with the same real words and different virtual words are obtained, semantic recognition can be carried out on the sentence pairs after the input training of the semantic recognition model to obtain semantic information of each sentence in the sentence pairs; if the semantic information of each sentence in the sentence pair is different and the semantic comparison labels of each sentence are the same, marking or updating the semantic comparison labels of each sentence in the sentence pair to obtain an updated target sentence sample for carrying out model training of the semantic recognition model.

The sentence pair extraction model may be any language model, such as Roberta, bert, or Mengzi.

The sentence pair extraction model performs sentence pair extraction based on a plurality of sample sentences in the following manner:

performing word segmentation processing on the plurality of sample sentences to obtain word segmentation results of the plurality of sample sentences;

determining real word sequences and imaginary word sequences of all sample sentences in the sample sentences according to word segmentation results of the sample sentences;

And determining two sample sentences with the same real word sequence and different imaginary word sequences in the real word sequence and the imaginary word sequence of each sample sentence, and constructing sentence pairs according to the two sample sentences.

The fact sequence is the same, and the fact sequence comprises the same words in the fact sequence and the same arrangement sequence of the words; the different sequences of the works include different works and different orders of the works.

In order to improve training effects of a grammar recognition model and a semantic recognition model, in a process of training the grammar recognition model and the semantic recognition model, firstly, performing the virtual word transformation on a first sentence sample according to a virtual word transformation mode corresponding to a virtual word type of the first sentence sample to obtain a second sentence sample, secondly, training the grammar recognition model to be trained by means of the second sentence sample to obtain a trained grammar recognition model, and training the semantic recognition model to be trained by combining the first sentence sample and the second sentence sample to obtain a trained semantic recognition model, performing grammar recognition on the grammar recognition model to be processed by means of the trained grammar recognition model to obtain a grammar type, performing semantic recognition on the sentence to be processed by means of the trained semantic recognition model to obtain semantic information, performing the sentence recognition on sentence features obtained by fusion of the grammar type and the semantic information, so as to train the semantic recognition model by means of the first sentence sample before virtual word transformation and the second sentence sample after virtual word transformation, improving the degree of the semantic recognition model to be processed by means of the trained semantic recognition model, further improving the accuracy of the semantic recognition model to be processed by means of the trained semantic recognition model, and further improving the accuracy of recognition, and simultaneously, and comprehensively outputting the grammar recognition model and the feature after the training by combining the grammar recognition model with the training accuracy.

The following further describes, by taking an example of application of the model training method provided in this embodiment to a resource processing scenario, the model training method provided in this embodiment applied to a resource processing scenario with reference to fig. 2 and 3, and referring to fig. 3, the model training method applied to a resource processing scenario specifically includes the following steps.

Step S302, determining the type of the virtual word of the first resource sentence sample.

The resource processing scenario in this embodiment may also be a financial scenario, that is, the first resource statement sample may be a first financial statement sample, and the second resource statement sample may be a second financial statement sample.

Step S304, if the type of the virtual word is a first type containing the virtual word, deleting or replacing the virtual word contained in the first resource sentence sample according to a first transformation mode corresponding to the first type, so as to obtain a second resource sentence sample.

And step S306, if the type of the virtual word is a second type which does not contain the virtual word, performing virtual word filling on the first resource sentence sample according to a second transformation mode corresponding to the second type to obtain a second resource sentence sample.

As shown in fig. 2, the first resource sentence sample is subjected to the participle transformation to obtain a second resource sentence sample, and specifically, the first resource sentence sample is subjected to the participle transformation according to the participle transformation mode corresponding to the participle type of the first resource sentence sample to obtain the second resource sentence sample.

Step S308, inputting the second resource sentence sample into a grammar recognition model to be trained, performing feature extraction processing, and determining grammar types according to real word features and imaginary word features of the second resource sentence sample obtained by the feature extraction processing to obtain a predicted grammar type.

Step S310, calculating training loss based on the grammar type label corresponding to the predicted grammar type and the second resource sentence sample, and performing parameter adjustment on the grammar recognition model to be trained based on the training loss to obtain the trained grammar recognition model.

As shown in fig. 2, the grammar recognition model to be trained performs feature extraction processing based on the second resource sentence sample to obtain real word features and virtual word features, determines the predicted grammar type of the second resource sentence sample based on the real word features and the virtual word features, calculates training loss based on grammar type labels corresponding to the predicted grammar type and the second resource sentence sample, and returns to perform parameter adjustment on the grammar recognition model to be trained.

Step S312, inputting the first resource sentence sample and the second resource sentence sample into a semantic recognition model to be trained for semantic recognition, and obtaining first semantic information and second semantic information.

Step S314, fusion processing is carried out based on the first semantic information and the second semantic information, and semantic fusion characteristics are obtained.

And S316, carrying out semantic comparison analysis on the semantic fusion features to obtain semantic comparison information, and calculating a loss value according to the semantic comparison information and the semantic comparison label corresponding to the second resource statement sample.

Step S318, parameter adjustment is carried out on the semantic recognition model to be trained based on the loss value, so as to obtain the trained semantic recognition model.

As shown in fig. 2, the semantic recognition model to be trained performs semantic recognition based on the first resource sentence sample and the second resource sentence sample to obtain first semantic information and second semantic information, performs fusion processing based on the first semantic information and the second semantic information to obtain semantic fusion features, and performs semantic comparison analysis based on the semantic fusion features to obtain semantic comparison information; and calculating a loss value according to the semantic comparison information and the semantic comparison label corresponding to the second resource statement sample, and returning to perform parameter adjustment on the semantic recognition model to be trained based on the loss value.

Optionally, the trained grammar recognition model is used for carrying out grammar recognition on the sentence to be processed to obtain a grammar type, the trained semantic recognition model is used for carrying out semantic recognition on the sentence to be processed to obtain semantic information, and sentence recognition is carried out on the sentence to be processed according to sentence characteristics obtained by fusion of the grammar type and the semantic information.

It should be noted that, the execution sequence between the steps S308 to S310 and the steps S312 to S318 is not particularly limited, and the steps S308 to S310 may be executed first, the steps S312 to S318 may be executed first, the steps S308 to S310 may be executed again, or both may be executed simultaneously.

An embodiment of a sentence processing method provided in the present specification is as follows:

The sentence processing method provided by the embodiment can be executed by computer equipment, and the computer equipment can be a terminal or a server, wherein the terminal can comprise a mobile phone, a notebook computer, intelligent interaction equipment and the like, and the server can comprise an independent physical server, a server cluster formed by a plurality of servers or a cloud server capable of performing cloud computing.

Referring to fig. 4, the sentence processing method provided in the present embodiment specifically includes steps S402 to S408.

Step S402, a statement to be processed is acquired.

The statement to be processed in this embodiment refers to a statement to be processed; the statement to be processed can be any statement; the form of the sentence to be processed may be a text form; the statement to be processed may be one or more statements; in addition, the statement to be processed can also be a statement in any scene, for example, the statement to be processed is a statement "day and night is repayment day" in a resource processing scene; the sentence to be processed can also be a user sentence corresponding to the user voice data generated in the dialogue process of the user and the intelligent customer service.

The statement to be processed can be a resource statement to be processed or a financial statement to be processed or a user statement to be processed in an intelligent customer service scene.

Step S404, inputting the sentence to be processed into a grammar recognition model for grammar recognition, obtaining the grammar type of the sentence to be processed, and generating the grammar characteristics of the grammar type.

Optionally, the grammar type is determined based on real word characteristics and imaginary word characteristics of the sentence to be processed. Optionally, the grammar recognition model is obtained after model training is performed on the grammar recognition model to be trained based on the second sentence sample; the second sentence sample is obtained after the first sentence sample is subjected to the virtual word transformation according to the virtual word transformation mode corresponding to the virtual word type of the first sentence sample. The grammar characteristics can be obtained by the following ways: and searching the grammar characteristics mapped by the grammar type in a grammar characteristic mapping table.

The grammar types in this embodiment are similar to the predicted grammar types, and refer to the grammar types of the to-be-processed sentences obtained by carrying out grammar recognition on the to-be-processed sentences, for example, the grammar types include a main predicate type, a moving object type, a middle complement type, a conjunctive predicate type or a preposition grammar type, and the examples of the grammar types are only illustrative, and in addition, the grammar types can be other grammar types. The grammar characteristics refer to characteristic vectors characterizing the grammar types.

In a specific implementation, in an alternative implementation manner provided in this embodiment, the grammar recognition model may perform grammar recognition on the statement to be processed in the following manner:

Specifically, in an optional implementation manner provided in this embodiment, in a process of performing feature extraction processing on a sentence to be processed to obtain a real word feature and an imaginary word feature of the sentence to be processed, the following operations are performed:

Performing word segmentation processing on the sentence to be processed to obtain a plurality of segmented words, and performing clustering processing on the segmented words according to the part of speech of the segmented words to obtain a real word sequence and an imaginary word sequence of the sentence to be processed;

extracting the real word characteristics from the real word sequence, and extracting the imaginary word characteristics from the imaginary word sequence.

In the process of determining the grammar type of the sentence to be processed according to the real word characteristics and the imaginary term characteristics of the sentence to be processed, similarity calculation can be performed based on the real word characteristics of the sentence to be processed and the real word characteristics of a plurality of reference sentences in a grammar library to obtain a plurality of real word similarities, and similarity calculation is performed based on the imaginary term characteristics of the sentence to be processed and the imaginary term characteristics of a plurality of reference sentences to obtain a plurality of imaginary term similarities; according to each real word similarity in the plurality of real word similarities and the corresponding virtual word similarity in the plurality of virtual word similarities, calculating each sentence similarity, determining target sentence similarity in each sentence similarity, and taking a target grammar type corresponding to the target sentence similarity as the grammar type.

Step S406, inputting the sentence to be processed into a semantic recognition model for semantic recognition, obtaining semantic information of the sentence to be processed, and generating semantic features of the semantic information.

And inputting the sentence to be processed into a grammar recognition model for grammar recognition to obtain the grammar type of the sentence to be processed and generating grammar characteristics of the grammar type. Optionally, the semantic recognition model is obtained after model training is performed on the semantic recognition model to be trained based on the first sentence sample and the second sentence sample; the second sentence sample is obtained after the first sentence sample is subjected to the virtual word transformation according to the virtual word transformation mode corresponding to the virtual word type of the first sentence sample.

The semantic information in this embodiment refers to meaning or meaning information of a sentence to be processed. The semantic features refer to feature vectors that characterize the semantic information. The semantic features can be obtained by: searching the semantic features mapped by the semantic information in a semantic feature mapping table.

In specific implementation, the semantic recognition model can perform semantic recognition on the statement to be processed in the following manner:

In addition, in order to more completely express the semantic information of the sentence to be processed, besides the whole consideration of the sentence to be processed, the semantic information can be determined by combining the two angles of the real word and the virtual word of the sentence to be processed, so that the comprehensiveness and the effectiveness of the semantic information can be improved; for this, the semantic recognition model may also perform semantic recognition on the statement to be processed to obtain semantic information in the following manner:

It should be noted that, there is no specific limitation on the execution sequence between the step S404 and the step S406, that is, the step S404 may be executed first, then the step S406 may be executed first, then the step S404 may be executed, or both may be executed simultaneously.

Step S408, feature fusion is carried out on the grammar features and the semantic features to obtain sentence features of the to-be-processed sentences for carrying out sentence recognition of the to-be-processed sentences.

In a specific execution process, after the grammar characteristics and the semantic characteristics of the semantic information of the grammar type are obtained, in order to improve the comprehensiveness and the effectiveness of the sentence characteristics, the grammar characteristics and the semantic characteristics can be subjected to characteristic fusion to obtain sentence characteristics of the sentence to be processed, and the grammar characteristics and the semantic characteristics can be subjected to characteristic splicing to obtain the sentence characteristics of the sentence to be processed, wherein the sentence characteristics can be used for carrying out sentence identification of the sentence to be processed.

In the above process of performing sentence recognition, the following operations may be performed:

And/or the number of the groups of groups,

It should be noted that, the processes of model training and the like of the grammar recognition model and the semantic recognition model related to the embodiment of the present application also relate to the previous embodiment of the present application, so that the specific implementation of model training in this embodiment may refer to the implementation of the foregoing model training method, and the repetition is not repeated; the processing procedure of the statement to be processed according to this embodiment of the present application is also related to the previous embodiment, and reference may be made to the content related to the previous embodiment.

The sentence processing method provided by the embodiment of the application can be used for various scenes needing sentence processing, including but not limited to the following application scenes: and carrying out sentence processing and the like on user sentences corresponding to the user voice data in the intelligent customer service scene. For example, when performing the sentence processing of the user sentence in the intelligent customer service scenario, the user sentence corresponds to the sentence to be processed in the present embodiment, and the sentence processing is performed on the user sentence by using steps S402 to S408 provided in the present embodiment.

The sentence processing process of the sentence processing method provided by the embodiment of the present application in other application scenarios is similar to the sentence processing process in the intelligent customer service scenario, and is not described in detail.

An embodiment of a model training device provided in the present specification is as follows:

in the above-described embodiments, a model training method and a model training apparatus corresponding thereto are provided, and the description is given below with reference to the accompanying drawings.

Referring to fig. 5, a schematic diagram of a model training apparatus according to the present embodiment is shown.

Since the apparatus embodiments correspond to the method embodiments, the description is relatively simple, and the relevant portions should be referred to the corresponding descriptions of the method embodiments provided above. The device embodiments described below are merely illustrative.

The embodiment provides a model training device, which is characterized in that the device comprises:

a type determining module 502, configured to determine an article type of the first sentence sample;

the term transformation module 504 is configured to perform term transformation on the first sentence sample according to a term transformation manner corresponding to the term type, so as to obtain a second sentence sample;

The model training module 506 is configured to perform model training on the grammar recognition model to be trained based on the second sentence sample to obtain a trained grammar recognition model, and perform model training on the semantic recognition model to be trained based on the first sentence sample and the second sentence sample to obtain a trained semantic recognition model;

An embodiment of a sentence processing device provided in the present specification is as follows:

in the above-described embodiments, a sentence processing method is provided, and a sentence processing apparatus is provided corresponding thereto, which will be described below with reference to the accompanying drawings.

Referring to fig. 6, a schematic diagram of a sentence processing device according to the present embodiment is shown.

The embodiment provides a sentence processing device, which is characterized in that the device includes:

a sentence acquisition module 602, configured to acquire a sentence to be processed;

A grammar feature generation module 604, configured to input the sentence to be processed into a grammar recognition model for grammar recognition, obtain a grammar type of the sentence to be processed, and generate a grammar feature of the grammar type; the grammar type is determined based on real word characteristics and imaginary word characteristics of the sentence to be processed;

the semantic feature generation module 606 is configured to input the sentence to be processed into a semantic recognition model for semantic recognition, obtain semantic information of the sentence to be processed, and generate a semantic feature of the semantic information;

And the feature fusion module 608 is configured to perform feature fusion on the grammar feature and the semantic feature to obtain a sentence feature of the sentence to be processed, so as to perform sentence recognition of the sentence to be processed.

An embodiment of a computer device provided in the present specification is as follows:

corresponding to the above-described model training method, based on the same technical concept, the embodiment of the present application further provides a computer device, where the computer device is configured to execute the above-provided model training method, and fig. 7 is a schematic structural diagram of the computer device provided by the embodiment of the present application.

The embodiment provides a computer device, including:

As shown in fig. 7, the computer device may have a relatively large difference due to different configurations or performances, and may include one or more processors 701 and a memory 702, where the memory 702 may store one or more storage applications or data. Wherein the memory 702 may be transient storage or persistent storage. The application programs stored in the memory 702 may include one or more modules (not shown) each of which may include a series of computer-executable instructions in a computer device. Still further, the processor 701 may be arranged to communicate with the memory 702 and execute a series of computer executable instructions in the memory 702 on a computer device. The computer device may also include one or more power supplies 703, one or more wired or wireless network interfaces 704, one or more input/output interfaces 705, one or more keyboards 706, and the like.

In a particular embodiment, a computer device includes a memory, and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for the computer device, and configured to be executed by one or more processors, the one or more programs comprising computer-executable instructions for:

determining the type of the participatory word of the first sentence sample;

Another embodiment of a computer device provided in this specification is as follows:

Corresponding to the sentence processing method described above, based on the same technical concept, the embodiment of the present application further provides another computer device, where the computer device is configured to execute the sentence processing method provided above, and fig. 8 is a schematic structural diagram of the computer device provided by the embodiment of the present application.

The embodiment provides a computer device, including:

As shown in fig. 8, the computer device may have a relatively large difference due to different configurations or performances, and may include one or more processors 801 and a memory 802, where the memory 802 may store one or more storage applications or data. Wherein the memory 802 may be transient storage or persistent storage. The application programs stored in memory 802 may include one or more modules (not shown in the figures), each of which may include a series of computer-executable instructions in a computer device. Still further, the processor 801 may be configured to communicate with a memory 802 and execute a series of computer executable instructions in the memory 802 on a computer device. The computer device may also include one or more power supplies 803, one or more wired or wireless network interfaces 804, one or more input/output interfaces 805, one or more keyboards 806, and the like.

Acquiring a statement to be processed;

An embodiment of a computer-readable storage medium provided in the present specification is as follows:

Corresponding to the model training method described above, the embodiment of the application further provides a computer readable storage medium based on the same technical concept.

The present embodiment provides a computer-readable storage medium for storing computer-executable instructions that, when executed by a processor, implement the following flow:

determining the type of the participatory word of the first sentence sample;

It should be noted that, in the present specification, an embodiment of a computer readable storage medium and an embodiment of a model training method in the present specification are based on the same inventive concept, so that a specific implementation of the embodiment may refer to the implementation of the foregoing corresponding method, and the repetition is omitted.

Another computer-readable storage medium embodiment provided in this specification is as follows:

Corresponding to one sentence processing method described above, another computer readable storage medium is provided according to an embodiment of the present application based on the same technical concept.

Acquiring a statement to be processed;

It should be noted that, in the present specification, an embodiment of another computer readable storage medium and an embodiment of a sentence processing method in the present specification are based on the same inventive concept, so that a specific implementation of the embodiment may refer to an implementation of the foregoing corresponding method, and a repetition is omitted.

The foregoing describes specific embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, embodiments of the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present description can take the form of a computer program product on one or more computer-readable storage media (including, but not limited to, magnetic disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present description is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the specification. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable test apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable test apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable test apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable test apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.

Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Disks (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.

Embodiments of the application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. One or more embodiments of the specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.

The foregoing description is by way of example only and is not intended to limit the present disclosure. Various modifications and changes may occur to those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. that fall within the spirit and principles of the present document are intended to be included within the scope of the claims of the present document.

Claims

1. A method of model training, the method comprising:

determining the type of the participatory word of the first sentence sample;

2. The method of claim 1, wherein performing the term transformation on the first sentence sample according to the term transformation mode corresponding to the term type to obtain a second sentence sample includes:

3. The method according to claim 1, wherein the training the grammar recognition model to be trained based on the second sentence sample to obtain a trained grammar recognition model includes:

4. A method according to claim 3, wherein the grammar recognition model to be trained performs grammar recognition by:

5. The method of claim 4, wherein said determining said predicted grammar type from said real word features and said imaginary word features comprises:

6. The method according to claim 1, wherein the training the semantic recognition model to be trained based on the first sentence sample and the second sentence sample to obtain a trained semantic recognition model includes:

7. The method of claim 6, wherein the calculating a penalty value based on the first semantic information, the second semantic information, and the semantic contrast labels corresponding to the second sentence samples comprises:

8. A sentence processing method, the method comprising:

Acquiring a statement to be processed;

9. The method of claim 8, wherein said performing grammar recognition comprises:

Performing feature extraction processing on the sentence to be processed to obtain real word features and virtual word features of the sentence to be processed;

10. The method of claim 9, wherein the performing feature extraction processing on the sentence to be processed to obtain real word features and imaginary word features of the sentence to be processed includes:

11. A model training apparatus, the apparatus comprising:

12. A sentence processing apparatus, the apparatus comprising:

13. A computer device, the device comprising:

A processor; and a memory configured to store computer-executable instructions that, when executed, cause the processor to perform the model training method of any of claims 1 to 7; or performs a sentence processing method as claimed in any one of claims 8 to 10.

14. A computer readable storage medium for storing computer executable instructions that when executed by a processor implement the model training method of any of claims 1 to 7; or implement a sentence processing method as claimed in any one of claims 8 to 10.