CN115438149A - End-to-end model training method and device, computer equipment and storage medium - Google Patents

Abstract

The embodiment of the application belongs to the technical field of artificial intelligence natural language processing, and relates to an end-to-end model training method and device suitable for multi-Chinese medical language processing tasks, computer equipment and a storage medium. In addition, the present application also relates to a block chain technology, and a target sequence model of a user can be stored in the block chain. According to the method and the device, an initial sequence model is established according to a mT5-small model of a Seq2Seq framework, and pre-training is performed on an entity recognition task and a tail prediction task through a large amount of medical corpus data, so that the pre-trained sequence model can learn medical knowledge hidden in other tasks, and the accuracy of multi-Chinese medical language processing tasks is effectively improved.

Description

An end-to-end model training method, device, computer equipment and storage medium

technical field

The present application relates to the technical field of natural language processing of artificial intelligence, in particular to an end-to-end model training method, device, computer equipment and storage medium suitable for multi-Chinese medical language processing tasks.

Background technique

For different NLP tasks of medical knowledge, the existing training schemes are trained separately for different tasks. For example, NLU tasks are trained using Bert-like models, text generation tasks are trained using GPT, and NER tasks are trained based on LSTM-related models.

However, the applicant found that traditional training methods cannot use information from other tasks in the same field, and cannot learn medical knowledge hidden in other tasks. For some tasks with small sample sizes, it is easy to cause overfitting if large models such as Bert are used. Traditional multi-Chinese medical language processing models have low predictive accuracy.

Contents of the invention

The purpose of the embodiment of the present application is to propose an end-to-end model training method, device, computer equipment and storage medium suitable for multi-Chinese medical language processing tasks, so as to solve the low prediction accuracy of traditional multi-Chinese medical language processing models The problem.

In order to solve the above technical problems, the embodiment of the present application provides an end-to-end model training method suitable for multi-Chinese medical language processing tasks, adopting the following technical solutions:

Obtain medical corpus data corresponding to the medical field;

Preprocessing the medical corpus data to obtain training corpus data;

Carrying out an entity matching operation on the training corpus data to obtain a training corpus entity, wherein the training corpus entity includes a head entity, an entity relationship, and a tail entity;

Create an initial sequence model based on the mT5-small model of the Seq2seq framework;

Constructing entity recognition training data according to the training corpus data, entity recognition soft prompts and entity recognition hard prompts;

Using the entity recognition training data as input data and the training corpus entities as label information to perform entity recognition training operations on the initial sequence model;

Constructing tail prediction training data with the head entity, the entity relationship, tail entity prediction soft hints and tail entity prediction hard hints;

Using the tail prediction training data as input data and the tail entity as label information to perform a tail prediction training operation on the initial sequence model;

The original sequence model after completing the entity recognition training operation and the tail prediction training operation is used as the target sequence model.

In order to solve the above technical problems, the embodiment of the present application also provides an end-to-end model training device suitable for multi-Chinese medical language processing tasks, which adopts the following technical solutions:

A data acquisition module, configured to acquire medical corpus data corresponding to the medical field;

A preprocessing module, configured to perform a preprocessing operation on the medical corpus data to obtain training corpus data;

An entity matching module is used to perform an entity matching operation on the training corpus data to obtain a training corpus entity, wherein the training corpus entity includes a head entity, an entity relationship, and a tail entity;

A model creation module for creating an initial sequence model according to the mT5-small model of the Seq2seq framework;

Entity recognition data construction module, for constructing entity recognition training data according to described training corpus data, entity recognition soft prompt and entity recognition hard prompt;

An entity recognition training module, configured to use the entity recognition training data as input data and the training corpus entity as label information to perform entity recognition training operations on the initial sequence model;

Tail prediction data construction module, used for constructing tail prediction training data with the head entity, the entity relationship, tail entity prediction soft prompt and tail entity prediction hard prompt;

A tail prediction training module, configured to use the tail prediction training data as input data and the tail entity as label information to perform a tail prediction training operation on the initial sequence model;

The model confirmation module is used to use the original sequence model after the entity recognition training operation and the tail prediction training operation as the target sequence model.

In order to solve the above technical problems, the embodiment of the present application also provides a computer device, which adopts the following technical solution:

Comprising a memory and a processor, computer-readable instructions are stored in the memory, and when the processor executes the computer-readable instructions, the above-mentioned end-to-end model training method applicable to multi-Chinese medical language processing tasks is implemented step.

In order to solve the above technical problems, the embodiment of the present application also provides a computer-readable storage medium, which adopts the following technical solution:

Computer-readable instructions are stored on the computer-readable storage medium, and when the computer-readable instructions are executed by a processor, the steps of the above-mentioned end-to-end model training method applicable to multi-Chinese medical language processing tasks are implemented.

This application provides an end-to-end model training method suitable for multi-Chinese medical language processing tasks, including: obtaining medical corpus data corresponding to the medical field; performing preprocessing operations on the medical corpus data to obtain training corpus data Carry out entity matching operation to described training corpus data, obtain training corpus entity, wherein, described training corpus entity comprises head entity, entity relation and tail entity; Create initial sequence model according to the mT5-small model of Seq2seq framework; According to all The training corpus data, the entity recognition soft prompt and the entity recognition hard prompt construct the entity recognition training data; the entity recognition training data is used as the input data, and the training corpus entity is used as the label information to carry out the entity recognition training operation on the initial sequence model ; The head entity, the entity relationship, the tail entity prediction soft prompt and the tail entity prediction hard prompt are used to construct the tail prediction training data; the tail prediction training data is used as input data, and the tail entity is used as label information for all The tail prediction training operation is performed on the initial sequence model; the original sequence model after the entity recognition training operation and the tail prediction training operation is completed is used as the target sequence model. Compared with the existing technology, this application creates an initial sequence model based on the mT5-small model of the Seq2seq framework, and pre-trains entity recognition tasks and tail prediction tasks through a large amount of medical corpus data, so that the pre-trained sequence model can learn The medical knowledge hidden in other tasks can effectively improve the accuracy of multi-Chinese medical language processing tasks.

Description of drawings

In order to illustrate the solution in this application more clearly, a brief introduction will be given below to the accompanying drawings that need to be used in the description of the embodiments of the application. Obviously, the accompanying drawings in the following description are some embodiments of the application. Ordinary technicians can also obtain other drawings based on these drawings on the premise of not paying creative work.

FIG. 1 is an exemplary system architecture diagram to which the present application can be applied;

Fig. 2 is the implementation flowchart of the end-to-end model training method applicable to multi-Chinese medical language processing tasks provided by Embodiment 1 of the present application;

FIG. 3 is a flow chart of another specific implementation of the end-to-end model training method provided in Embodiment 1 of the present application;

FIG. 4 is a flowchart of a specific implementation of step S202 in FIG. 2;

FIG. 5 is a flow chart of yet another specific implementation of the end-to-end model training method provided in Embodiment 1 of the present application;

FIG. 6 is a flowchart of a specific implementation of step S202 in FIG. 5;

FIG. 7 is a flow chart of a specific implementation of acquiring a semantic analysis model provided in Embodiment 1 of the present application;

8 is a schematic structural diagram of an end-to-end model training device suitable for multi-Chinese medical language processing tasks provided by Embodiment 2 of the present application;

FIG. 9 is a schematic structural diagram of another specific embodiment of an end-to-end model training device suitable for multi-Chinese medical language processing tasks provided by Embodiment 2 of the present application;

Fig. 10 is a schematic structural diagram of an embodiment of a computer device according to the present application.

detailed description

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field of the application; the terms used herein in the description of the application are only to describe specific embodiments The purpose is not to limit the present application; the terms "comprising" and "having" and any variations thereof in the specification and claims of the present application and the description of the above drawings are intended to cover non-exclusive inclusion. The terms "first", "second" and the like in the description and claims of the present application or the above drawings are used to distinguish different objects, rather than to describe a specific order.

Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The occurrences of this phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is understood explicitly and implicitly by those skilled in the art that the embodiments described herein can be combined with other embodiments.

In order to enable those skilled in the art to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

As shown in FIG. 1 , a system architecture 100 may include terminal devices 101 , 102 , 103 , a network 104 and a server 105 . The network 104 is used as a medium for providing communication links between the terminal devices 101 , 102 , 103 and the server 105 . Network 104 may include various connection types, such as wires, wireless communication links, or fiber optic cables, among others.

Users can use terminal devices 101 , 102 , 103 to interact with server 105 via network 104 to receive or send messages and the like. Various communication client applications can be installed on the terminal devices 101, 102, 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, and the like.

Terminal devices 101, 102, 103 can be various electronic devices with display screens and support web browsing, including but not limited to smartphones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III, moving picture Expert Compression Standard Audio Layer 3), MP4 (Moving PictureExperts Group Audio Layer IV, Moving Picture Experts Compression Standard Audio Layer 4) Players, Laptop Portable Computers and Desktop Computers, etc.

The server 105 may be a server that provides various services, such as a background server that provides support for pages displayed on the terminal devices 101 , 102 , 103 .

It should be noted that the end-to-end model training method suitable for multi-Chinese medical language processing tasks provided by the embodiment of the present application is generally executed by a server/terminal device, and correspondingly, it is suitable for end-to-end multi-Chinese medical language processing tasks The model training device is generally set in the server/terminal device.

It should be understood that the numbers of terminal devices, networks and servers in Fig. 1 are only illustrative. According to the implementation needs, there can be any number of terminal devices, networks and servers.

Embodiment one

Continuing to refer to FIG. 2 , it shows the implementation flow chart of the end-to-end model training method suitable for multi-Chinese medical language processing tasks provided by Embodiment 1 of the present application. For the convenience of description, only the parts related to the present application are shown.

The above-mentioned end-to-end model training method applicable to multi-Chinese medical language processing tasks includes the following steps:

Step S201: Obtain medical corpus data corresponding to the medical field;

Step S202: Preprocessing the medical corpus data to obtain training corpus data;

Step S203: Perform entity matching operation on the training corpus data to obtain the training corpus entity, wherein the training corpus entity includes head entity, entity relationship and tail entity;

Step S204: Create an initial sequence model according to the mT5-small model of the Seq2seq framework;

Step S205: Construct entity recognition training data according to the training corpus data, entity recognition soft prompts and entity recognition hard prompts;

In the embodiment of this application, a special token is used as a soft prompt (soft prompt), and the task description is used as a hard prompt (a prompt composed of specific Chinese or English words, which is a human-readable hard prompt) Creation of the above training data.

Step S206: Using entity recognition training data as input data and training corpus entities as label information, perform entity recognition training operations on the initial sequence model;

Step S207: Construct tail prediction training data with the head entity, entity relationship, tail entity prediction soft prompt and tail entity prediction hard prompt;

Step S208: Using the tail prediction training data as input data and tail entities as label information, perform tail prediction training operations on the initial sequence model;

Step S209: use the original sequence model after the entity recognition training operation and the tail prediction training operation as the target sequence model.

In some optional implementations of this embodiment, in order to enhance the accuracy of the model in generating medical texts, we introduce external knowledge, and we design two steps for the model to enhance the ability of the model to utilize knowledge. 1: Knowledge selection (knowledge selection) training model, the input is text, and the output is the triplet related to the text in the knowledge map (KG); 2: Knowledge indorsation (knowledge indorsation), the relevant knowledge obtained from the knowledge map and Dialogue stitching together serves as input to the model to jointly generate responses.

In the embodiment of this application, an end-to-end model training method suitable for multi-Chinese medical language processing tasks is provided, including: obtaining medical corpus data corresponding to the medical field; performing preprocessing operations on the medical corpus data to obtain Training corpus data; perform entity matching operation on the training corpus data to obtain training corpus entities, wherein the training corpus entities include head entities, entity relations and tail entities; create an initial sequence model according to the mT5-small model of the Seq2seq framework; according to the training corpus Data, entity recognition soft prompts and entity recognition hard prompts construct entity recognition training data; use entity recognition training data as input data and training corpus entities as label information to perform entity recognition training operations on the initial sequence model; head entities, entity relationships, Tail entity prediction soft hints and tail entity prediction hard hints construct tail prediction training data; use tail prediction training data as input data and tail entities as label information to perform tail prediction training operations on the initial sequence model; entity recognition training operations and tail predictions will be completed The original sequence model after the training operation is used as the target sequence model. Compared with the existing technology, this application creates an initial sequence model based on the mT5-small model of the Seq2seq framework, and pre-trains entity recognition tasks and tail prediction tasks through a large amount of medical corpus data, so that the pre-trained sequence model can learn The medical knowledge hidden in other tasks can effectively improve the accuracy of multi-Chinese medical language processing tasks.

Continuing to refer to FIG. 3 , it shows a flow chart of another specific implementation of the end-to-end model training method provided in Embodiment 1 of the present application. For the convenience of description, only the parts relevant to the present application are shown.

In some optional implementation manners of this embodiment, after step S204 and before step S209, further include: step S301 and step S302, and step S209 includes: step S303.

Step S301: Construct article summary training data according to article content, article summary soft prompts, and article summary hard prompts.

Step S302: Using article summary training data as input data and article titles as label information to perform article summary training on the initial sequence model.

Step S303: The original sequence model after the entity recognition training operation, the tail prediction training operation and the article summary training operation is completed is used as the target sequence model.

Continuing to refer to FIG. 4 , it shows a flow chart of a specific implementation of step S202 in FIG. 2 , and for ease of description, only parts relevant to the present application are shown.

In some optional implementation manners of this embodiment, step S202 specifically includes: step S401 and/or step S402, wherein:

Step S401: Perform similar text deduplication operation on medical corpus data according to Jaccard similarity algorithm.

In the embodiment of the present application, the Jaccard similarity algorithm is used to compare the similarity and difference between limited sample sets. The larger the Jaccard coefficient value, the higher the sample similarity.

Step S402: According to the regularization matching algorithm, the text with relatively large noise in the medical corpus data is deleted to obtain the training corpus data.

Continuing to refer to FIG. 5 , it shows a flow chart of yet another specific implementation of the end-to-end model training method provided in Embodiment 1 of the present application. For ease of description, only the parts relevant to the present application are shown.

In some optional implementations of this embodiment, the medical corpus data includes medical question and answer information carrying medical question information and medical answer information. After step S204 and before step S209, it also includes: step S501 and step S502, step S209 includes: step S503.

Step S501: Construct medical question-and-answer training data according to medical question information, medical question-and-answer soft prompts, and medical question-and-answer hard prompts.

Step S502: Using medical question-answer training data as input data and medical answer information as label information, perform medical question-answer training on the initial sequence model.

Step S503: The original sequence model after the entity recognition training operation, the tail prediction training operation and the medical question answering training operation is completed is used as the target sequence model.

Continuing to refer to FIG. 6 , it shows a flow chart of a specific implementation of step S202 in FIG. 5 , and for ease of description, only the parts relevant to the present application are shown.

In some optional implementations of this embodiment, step S202 specifically includes: step S601, step S602, step S603, step S604, and step S605, wherein:

Step S601: judging whether there are ambiguous words in the medical question-and-answer information;

Step S602: If there is no ambiguous vocabulary, use the medical corpus data as the training corpus data;

Step S603: If there is an ambiguous vocabulary, obtain associated text information associated with the context of the ambiguous vocabulary;

Step S604: Input the associated text information into the semantic analysis model to perform word meaning recognition operation, and obtain the real word meaning information of ambiguous words;

Step S605: Replace the ambiguous words in the medical question-and-answer information with real word meaning information to obtain training corpus data.

Continuing to refer to FIG. 7 , it shows a flow chart of a specific implementation manner of acquiring a semantic analysis model provided by Embodiment 1 of the present application. For ease of description, only the parts relevant to the present application are shown.

In some optional implementations of this embodiment, before step S604, further include: step S701, step S702, step S703, step S704, step S705 and step S706, wherein:

Step S701: Obtain a sample text in a local database, and determine each participle included in the sample text.

In the embodiment of the present application, a plurality of texts may be acquired from the above local database first, and a training set composed of the acquired plurality of texts may be determined, then, for each text in the training set, the text may be used as a sample text.

In the embodiment of the present application, when the word segmentation contained in the sample text is determined, word segmentation processing may be performed on the sample text first, so as to obtain each word segmentation contained in the sample text. When performing word segmentation processing on the sample text, any word segmentation method can be used. Of course, each character in the sample text can also be processed as a word segmentation. It should be understood that the example of word segmentation processing here is only for convenience of understanding. It is not intended to limit this application.

Step S702: Determine the word vector corresponding to each word segment based on the semantic analysis model to be trained.

In the embodiment of the present application, the semantic analysis model may include at least four layers, namely: a semantic representation layer, an attribute representation layer, an attribute correlation representation layer, and a classification layer.

In the embodiment of the present application, the semantic representation layer includes at least a sub-model for outputting a bidirectional semantic representation vector, such as a BERT (Bidirectional Encoder Representations from Transformers) model. Each word segment can be input into the semantic representation layer in the semantic analysis model, and a bidirectional semantic representation vector corresponding to each word segment output by the semantic representation layer can be obtained as a word vector corresponding to each word segment. It should be understood that the model for outputting the bidirectional semantic representation vector includes other models besides the above-mentioned BERT model. The examples of the model for outputting the bidirectional semantic representation vector here are only for convenience of understanding and are not intended to limit the present application.

Step S703: Obtain the semantic attributes in the local database, and determine the first feature representation vector of the sample text related to the semantic attributes according to the attention matrix corresponding to the semantic attributes contained in the semantic analysis model to be trained and the word vector corresponding to each word segment.

In the embodiment of the present application, the word vector corresponding to each word segment can be input into the attribute representation layer in the semantic analysis model, and the word vector corresponding to each word segment can be calculated through the attention matrix corresponding to the semantic attribute contained in the attribute representation layer. Perform attention weighting, and determine the first feature representation vector of the sample text related to the semantic attribute according to the word vector corresponding to each word segment after attention weighting.

Step S704: According to the self-attention matrix used to represent the correlation between different semantic attributes contained in the semantic analysis model to be trained, and the first feature representation vector, determine the second feature representation vector related to the semantic attribute of the sample text.

In the embodiment of the present application, the first feature representation vector of the sample text related to each semantic attribute can be input into the attribute correlation representation layer in the speech analysis model, and through the above-mentioned self-attention matrix contained in the attribute correlation representation layer, the The first feature representation vectors of the sample text related to each semantic attribute are weighted by self-attention, and the second feature representation vectors of the sample text related to each semantic attribute are determined according to the first feature representation vectors weighted by self-attention.

Step S705: According to the semantic analysis model to be trained and the second feature representation vector, determine the classification result output by the semantic training model to be trained, the classification result includes the semantic attribute to which the sample text belongs and the emotional polarity corresponding to the semantic attribute to which the sample text belongs .

In the embodiment of the present application, the classification layer includes at least a hidden layer, a fully connected layer and a softmax layer.

In the embodiment of the present application, the second feature representation vector of the sample text related to each semantic attribute can be sequentially input into the hidden layer, fully connected layer and softmax layer in the classification layer, and according to each second feature representation vector and the classification layer The classification parameters corresponding to each semantic attribute contained in the hidden layer, the fully connected layer and the softmax layer classify the sample text and obtain the classification result output by the classification layer.

In the embodiment of the present application, the classification result of at least includes the semantic attribute to which the sample text belongs and the emotional polarity corresponding to the semantic attribute to which the sample text belongs.

In the embodiment of this application, the emotional polarity can be quantified by numerical value. For example, the closer the numerical value is to 1, the more positive the emotional polarity is, and the closer the numerical value is to -1, the more inclined the emotional polarity is. Negative, with a value close to 0, indicates that the emotional polarity tends to be neutral.

Step S706: According to the classification result and the preset annotation of the sample text, adjust the model parameters in the semantic analysis model to obtain the semantic analysis model.

In the embodiment of the present application, the model parameters to be adjusted include at least the above-mentioned classification parameters, and may also include the above-mentioned attention matrix and self-attention matrix. The model parameters in the semantic analysis model can be adjusted using traditional training methods. That is, directly according to the classification result obtained in step S108 and the preset label for the sample text, the loss corresponding to the classification result (hereinafter referred to as the first loss) is determined, and the first loss is minimized as the training target pair semantic The model parameters in the analysis model are adjusted to complete the training of the semantic analysis model.

In the embodiment of the present application, since the self-attention matrix used to represent the correlation between different semantic attributes has been added to the above-mentioned semantic analysis model, the semantic analysis model trained by the above-mentioned traditional training method can be more accurate Analyze the semantics of the text to be analyzed.

It should be emphasized that, in order to further ensure the privacy and security of the above-mentioned target sequence model, the above-mentioned target sequence model can also be stored in a node of a block chain.

The blockchain referred to in this application is a new application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm. Blockchain (Blockchain), essentially a decentralized database, is a series of data blocks associated with each other using cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify its Validity of information (anti-counterfeiting) and generation of the next block. The blockchain can include the underlying platform of the blockchain, the platform product service layer, and the application service layer.

The application can be used in numerous general purpose or special purpose computer system environments or configurations. Examples: personal computers, server computers, handheld or portable devices, tablet-type devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, including A distributed computing environment for any of the above systems or devices, etc. This application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including storage devices.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. , when the computer-readable instructions are executed, they may include the processes of the embodiments of the above-mentioned methods. Wherein, the aforementioned storage medium may be a nonvolatile storage medium such as a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), or a random access memory (Random Access Memory, RAM).

It should be understood that although the various steps in the flow chart of the accompanying drawings are displayed sequentially according to the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but may be executed at different times, and the order of execution is also It is not necessarily performed sequentially, but may be performed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

Embodiment two

Further referring to FIG. 8 , as an implementation of the method shown in FIG. 2 above, the present application provides an embodiment of an end-to-end model training device suitable for multi-Chinese medical language processing tasks. This device embodiment is the same as that shown in FIG. 2 Corresponding to the method embodiments shown, the apparatus can be specifically applied to various electronic devices.

As shown in Figure 8, the end-to-end model training device 200 applicable to multi-Chinese medical language processing tasks in this embodiment includes: data acquisition module 201, preprocessing module 202, entity matching module 203, model creation module 204, entity recognition Data construction module 205 , entity recognition training module 206 , tail prediction data construction module 207 , tail prediction training module 208 and model confirmation module 209 . in:

A data acquisition module 201, configured to acquire medical corpus data corresponding to the medical field;

A preprocessing module 202, configured to perform preprocessing operations on medical corpus data to obtain training corpus data;

Entity matching module 203, for carrying out entity matching operation to training corpus data, obtains training corpus entity, wherein, training corpus entity comprises head entity, entity relation and tail entity;

A model creation module 204, configured to create an initial sequence model according to the mT5-small model of the Seq2seq framework;

Entity recognition data construction module 205, for constructing entity recognition training data according to training corpus data, entity recognition soft prompt and entity recognition hard prompt;

Entity recognition training module 206, is used for carrying out entity recognition training operation to initial sequence model with entity recognition training data as input data, training corpus entity as label information;

Tail prediction data construction module 207, used for constructing tail prediction training data with head entity, entity relationship, tail entity prediction soft prompt and tail entity prediction hard prompt;

The tail prediction training module 208 is used to use the tail prediction training data as the input data and the tail entity as the label information to carry out the tail prediction training operation on the initial sequence model;

The model confirmation module 209 is configured to use the original sequence model after the entity recognition training operation and the tail prediction training operation as the target sequence model.

In the embodiment of this application, a special token is used as a soft prompt (soft prompt), and the task description is used as a hard prompt (a prompt composed of specific Chinese or English words, which is a human-readable hard prompt) Creation of the above training data.

In some optional implementations of this embodiment, in order to enhance the accuracy of the model in generating medical texts, we introduce external knowledge, and we design two steps for the model to enhance the ability of the model to utilize knowledge. 1: Knowledge selection (knowledge selection) training model, the input is text, and the output is the triplet related to the text in the knowledge map (KG); 2: Knowledge indorsation (knowledge indorsation), the relevant knowledge obtained from the knowledge map and Dialogue stitching together serves as input to the model to jointly generate responses.

In the embodiment of the present application, an end-to-end model training device 200 suitable for multi-Chinese medical language processing tasks is provided, including: a data acquisition module 201 for acquiring medical corpus data corresponding to the medical field; preprocessing Module 202, is used for carrying out preprocessing operation to medical corpus data, obtains training corpus data; Entity matching module 203, is used for carrying out entity matching operation to training corpus data, obtains training corpus entity, wherein, training corpus entity comprises head entity, Entity relationship and tail entity; model creation module 204, for creating initial sequence model according to the mT5-small model of Seq2seq framework; entity recognition data construction module 205, for building according to training corpus data, entity recognition soft prompt and entity recognition hard prompt Entity recognition training data; Entity recognition training module 206, is used for using entity recognition training data as input data, and training corpus entity carries out entity recognition training operation to initial sequence model as label information; Tail prediction data construction module 207, is used for head Entities, entity relationships, tail entity prediction soft hints and tail entity prediction hard hints construct tail prediction training data; tail prediction training module 208 is used to use tail prediction training data as input data and tail entities as label information to perform tail prediction on the initial sequence model Predictive training operation; the model confirmation module 209 is used to use the original sequence model after completing the entity recognition training operation and the tail prediction training operation as the target sequence model. Compared with the existing technology, this application creates an initial sequence model based on the mT5-small model of the Seq2seq framework, and pre-trains entity recognition tasks and tail prediction tasks through a large amount of medical corpus data, so that the pre-trained sequence model can learn The medical knowledge hidden in other tasks can effectively improve the accuracy of multi-Chinese medical language processing tasks.

Continue to refer to FIG. 9 , which shows a structural schematic diagram of another specific implementation of an end-to-end model training device suitable for multi-Chinese medical language processing tasks provided by Embodiment 2 of the present application. Apply for relevant parts.

In some optional implementations of this embodiment, the above-mentioned end-to-end model training device 200 suitable for multi-Chinese medical language processing tasks also includes: article summary data construction module 210 and article summary training module 211, model confirmation module 209 Including: the first model confirmation sub-module 2091, wherein:

Article summary data construction module 210, for constructing article summary training data according to article content, article summary soft prompt and article summary hard prompt;

The article summary training module 211 is used to use the article summary training data as the input data and the article title as the label information to carry out the article summary training operation to the initial sequence model;

The first model confirmation sub-module 2091 is used to use the original sequence model after the entity recognition training operation, the tail prediction training operation and the article summary training operation as the target sequence model.

In some optional implementations of this embodiment, the above-mentioned preprocessing module 202 includes: a deduplication submodule and a deletion submodule, wherein:

The deduplication submodule is used to perform similar text deduplication operations on the medical corpus data according to the Jaccard similarity algorithm;

The deletion sub-module is used to delete the noisy text in the medical corpus data according to the regular matching algorithm to obtain the training corpus data.

In some optional implementations of this embodiment, the above-mentioned end-to-end model training device 200 applicable to multi-Chinese medical language processing tasks also includes: a medical question-and-answer data construction module and a medical question-and-answer training module, and the above-mentioned model confirmation module 209 includes : The second model determines the submodule, where:

A medical question-and-answer data building module, used to construct medical question-and-answer training data according to the medical question-and-answer information, medical question-and-answer soft prompts and medical question-and-answer hard prompts;

A medical question-and-answer training module, configured to use the medical question-and-answer training data as input data and the medical answer information as label information to perform medical question-and-answer training operations on the initial sequence model;

The second model determining sub-module is used to use the original sequence model after completing the entity recognition training operation, the tail prediction training operation and the medical question answering training operation as the target sequence model.

In some optional implementations of this embodiment, the above-mentioned preprocessing module 202 includes: an ambiguous vocabulary judging submodule, an ambiguous denying submodule, an ambiguous confirming submodule, a real word meaning acquiring submodule, and a vocabulary replacing submodule, wherein:

An ambiguous vocabulary judging submodule, used to judge whether there are ambiguous vocabulary in the medical question-and-answer information;

An ambiguity denying submodule, configured to use the medical corpus data as the training corpus data if the ambiguous vocabulary does not exist;

The ambiguity confirmation submodule is used to obtain associated text information associated with the context of the ambiguous vocabulary if the ambiguous vocabulary exists;

The real word meaning acquisition sub-module is used to input the associated text information into the semantic analysis model to perform word meaning recognition operation, and obtain the real word meaning information of the ambiguous vocabulary;

The vocabulary replacement submodule is used to replace the ambiguous vocabulary in the medical question-and-answer information with the real word meaning information to obtain the training corpus data.

In some optional implementations of this embodiment, the preprocessing module 202 further includes: a word segmentation determination module, a word vector determination module, a first feature representation vector determination module, a second feature representation vector determination module, and a classification result determination module and the model acquisition module. in:

The participle determination module is used to obtain the sample text in the local database, and determine each participle contained in the sample text;

The word vector determination module is used to determine the word vector corresponding to each word segmentation based on the semantic analysis model to be trained;

The first feature representation vector determination module is used to obtain semantic attributes in the local database, and determine that the sample text involves semantic attributes according to the attention matrix corresponding to the semantic attributes included in the semantic analysis model to be trained, and the word vector corresponding to each word segment The first feature representation vector of ;

The second feature representation vector determination module is used to determine that the sample text involves a semantic attribute according to the self-attention matrix used to represent the correlation between different semantic attributes contained in the semantic analysis model to be trained, and the first feature representation vector The second feature representation vector of ;

The classification result determination module is used to determine the classification result output by the semantic training model to be trained according to the semantic analysis model to be trained and the second feature representation vector. The classification result includes the semantic attribute to which the sample text belongs and the corresponding semantic attribute to which the sample text belongs. emotional polarity;

The model acquisition module is used to adjust the model parameters in the semantic analysis model according to the classification result and the preset annotation of the sample text to obtain the semantic analysis model.

In order to solve the above technical problems, the embodiment of the present application further provides computer equipment. Please refer to FIG. 10 for details. FIG. 10 is a block diagram of the basic structure of the computer device in this embodiment.

The computer device 300 includes a memory 310 , a processor 320 , and a network interface 330 connected to each other through a system bus for communication. It should be noted that only computer device 300 is shown with components 310-330, but it should be understood that implementing all of the illustrated components is not required and that more or fewer components may instead be implemented. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculation and/or information processing according to preset or stored instructions, and its hardware includes but is not limited to microprocessors, dedicated Integrated circuit (Application Specific Integrated Circuit, ASIC), programmable gate array (Field-Programmable Gate Array, FPGA), digital processor (Digital Signal Processor, DSP), embedded devices, etc.

The computer equipment may be computing equipment such as a desktop computer, a notebook, a palmtop computer, and a cloud server. The computer device can perform human-computer interaction with the user through keyboard, mouse, remote controller, touch panel or voice control device.

The memory 310 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static Random Access Memory (SRAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Programmable Read Only Memory (PROM), Magnetic Memory, Magnetic Disk, Optical Disk, etc. In some embodiments, the storage 310 may be an internal storage unit of the computer device 300 , such as a hard disk or memory of the computer device 300 . In some other embodiments, the memory 310 may also be an external storage device of the computer device 300, such as a plug-in hard disk equipped on the computer device 300, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, flash memory card (Flash Card), etc. Certainly, the memory 310 may also include both an internal storage unit of the computer device 300 and an external storage device thereof. In this embodiment, the memory 310 is usually used to store the operating system and various application software installed on the computer device 300, such as computer-readable instructions applicable to the end-to-end model training method for multi-Chinese medical language processing tasks Wait. In addition, the memory 310 can also be used to temporarily store various types of data that have been output or will be output.

The processor 320 may be a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor, or other data processing chips in some embodiments. The processor 320 is generally used to control the overall operation of the computer device 300 . In this embodiment, the processor 320 is configured to run computer-readable instructions stored in the memory 310 or process data, for example, a computer running the end-to-end model training method applicable to multi-Chinese medical language processing tasks can read command.

The network interface 330 may include a wireless network interface or a wired network interface, and the network interface 330 is generally used to establish a communication connection between the computer device 300 and other electronic devices.

The computer equipment provided by this application creates an initial sequence model based on the mT5-small model of the Seq2seq framework, and pre-trains entity recognition tasks and tail prediction tasks through a large amount of medical corpus data, so that the pre-trained sequence model can learn to hide The medical knowledge of other tasks can effectively improve the accuracy of multi-Chinese medical language processing tasks.

The present application also provides another implementation manner, which is to provide a computer-readable storage medium, the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to The at least one processor is made to execute the steps of the above-mentioned end-to-end model training method applicable to multi-Chinese medical language processing tasks.

The computer-readable storage medium provided by this application creates an initial sequence model according to the mT5-small model of the Seq2seq framework, and performs pre-training for entity recognition tasks and tail prediction tasks through a large amount of medical corpus data, so that the sequence model after pre-training can be Learn medical knowledge hidden in other tasks, effectively improve the accuracy of multi-Chinese medical language processing tasks.

Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product in essence or the part that contributes to the prior art, and the computer software product is stored in a storage medium (such as ROM/RAM, disk, CD) contains several instructions to make a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the methods described in the various embodiments of the present application.

Apparently, the embodiments described above are only some of the embodiments of the present application, not all of them. The drawings show preferred embodiments of the present application, but do not limit the patent scope of the present application. The present application can be implemented in many different forms, on the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or perform equivalent replacements for some of the technical features . All equivalent structures made using the contents of the description and drawings of this application, directly or indirectly used in other related technical fields, are also within the scope of protection of this application.

Claims (10)

1. An end-to-end model training method suitable for multiple Chinese medical language processing tasks is characterized by comprising the following steps:

acquiring medical corpus data corresponding to the medical field;

preprocessing the medical corpus data to obtain training corpus data;

performing entity matching operation on the corpus data to obtain a corpus entity, wherein the corpus entity comprises a head entity, an entity relationship and a tail entity;

establishing an initial sequence model according to a mT5-small model of a Seq2Seq framework;

constructing entity recognition training data according to the training corpus data, the entity recognition soft prompt and the entity recognition hard prompt;

taking the entity recognition training data as input data and the corpus entity as label information to perform entity recognition training operation on the initial sequence model;

constructing tail prediction training data by using the head entity, the entity relation, the tail entity prediction soft prompt and the tail entity prediction hard prompt;

taking the tail prediction training data as input data and the tail entity as label information to carry out tail prediction training operation on the initial sequence model;

and taking the original sequence model after the entity recognition training operation and the tail prediction training operation as a target sequence model.

2. The method of claim 1, wherein the corpus data further includes medical article data carrying article titles and article contents, and wherein after the step of creating an initial sequence model according to the mT5-small model of the Seq2Seq framework, and before the step of using the initial sequence model after the entity recognition training operation and the tail prediction training operation as a target sequence model, the method further includes the steps of:

constructing article summary training data according to the article content, the article summary soft prompt and the article summary hard prompt;

performing article summarization training operation on the initial sequence model by using the article summarization training data as input data and the article titles as label information;

the step of using the original sequence model after the entity recognition training operation and the tail prediction training operation are completed as a target sequence model specifically includes the following steps:

and taking the original sequence model after the entity recognition training operation, the tail prediction training operation and the article summary training operation as the target sequence model.

3. The end-to-end model training method applicable to multiple chinese medical language processing tasks according to claim 1, wherein the step of preprocessing the medical corpus data to obtain training corpus data specifically comprises the steps of:

carrying out similar text duplication elimination operation on the medical corpus data according to a Jaccard similarity algorithm;

and deleting the text with larger noise in the medical corpus data according to a regular matching algorithm to obtain the training corpus data.

4. The end-to-end model training method applicable to multiple chinese medical language processing tasks according to claim 1, wherein the medical corpus data includes medical question and answer information carrying medical question and answer information, and after the step of creating an initial sequence model according to the mT5-small model of the Seq2Seq framework, and before the step of using the initial sequence model after the entity recognition training operation and the tail prediction training operation as a target sequence model, the method further comprises the steps of:

constructing medical question and answer training data according to the medical question and answer information, the medical question and answer soft prompt and the medical question and answer hard prompt;

taking the medical question-answer training data as input data and the medical answer information as label information to carry out medical question-answer training operation on the initial sequence model;

the step of using the original sequence model after the entity recognition training operation and the tail prediction training operation are completed as a target sequence model specifically includes the following steps:

and taking the original sequence model after the entity recognition training operation, the tail prediction training operation and the medical question-answer training operation as the target sequence model.

5. The end-to-end model training method applicable to multiple chinese medical language processing tasks according to claim 4, wherein the step of preprocessing the medical corpus data to obtain training corpus data specifically comprises the steps of:

judging whether the medical question-answer information has ambiguous vocabularies or not;

if the ambiguous vocabulary does not exist, taking the medical corpus data as the training corpus data;

if the ambiguous vocabulary exists, acquiring associated text information associated with the ambiguous vocabulary context;

inputting the associated text information into a semantic analysis model to perform word sense recognition operation to obtain real word sense information of the ambiguous vocabulary;

and replacing the ambiguous vocabulary in the medical question-answering information with the real word meaning information to obtain the training corpus data.

6. The method of claim 5, wherein before the step of inputting the associated text information into a semantic analysis model for word sense recognition to obtain the real word sense information of the ambiguous vocabulary, the method further comprises:

obtaining a sample text from the local database, and determining each participle contained in the sample text;

determining a word vector corresponding to each participle based on a semantic analysis model to be trained;

obtaining semantic attributes from the local database, and determining a first feature expression vector of the sample text related to the semantic attributes according to an attention matrix corresponding to the semantic attributes and a word vector corresponding to each participle in the semantic analysis model to be trained;

determining a second feature representation vector of the sample text related to the semantic attributes according to a self-attention matrix which is contained in the semantic analysis model to be trained and used for representing correlation among different semantic attributes and the first feature representation vector;

determining a classification result output by the semantic training model to be trained according to the semantic analysis model to be trained and the second feature expression vector, wherein the classification result comprises a semantic attribute to which the sample text belongs and an emotion polarity corresponding to the semantic attribute to which the sample text belongs;

and adjusting model parameters in the semantic analysis model according to the classification result and the preset label of the sample text to obtain the semantic analysis model.

7. An end-to-end model training device suitable for multiple chinese medical language processing tasks, comprising:

the data acquisition module is used for acquiring medical corpus data corresponding to the medical field;

the preprocessing module is used for preprocessing the medical corpus data to obtain training corpus data;

the entity matching module is used for carrying out entity matching operation on the corpus data to obtain a corpus entity, wherein the corpus entity comprises a head entity, an entity relation and a tail entity;

the model creating module is used for creating an initial sequence model according to the mT5-small model of the Seq2Seq framework;

the entity identification data construction module is used for constructing entity identification training data according to the training corpus data, the entity identification soft prompt and the entity identification hard prompt;

the entity recognition training module is used for carrying out entity recognition training operation on the initial sequence model by taking the entity recognition training data as input data and the training corpus entity as label information;

the tail prediction data construction module is used for constructing tail prediction training data by the head entity, the entity relation, the tail entity prediction soft prompt and the tail entity prediction hard prompt;

the tail prediction training module is used for carrying out tail prediction training operation on the initial sequence model by taking the tail prediction training data as input data and the tail entity as label information;

and the model confirmation module is used for taking the original sequence model after the entity recognition training operation and the tail prediction training operation are finished as a target sequence model.

8. The apparatus for end-to-end model training for multiple chinese medical language processing tasks according to claim 7, further comprising: the model confirmation module comprises: a first model validation submodule, wherein:

the article summary data construction module is used for constructing article summary training data according to the article content, the article summary soft prompt and the article summary hard prompt;

the article summarization training module is used for performing article summarization training operation on the initial sequence model by using the article summarization training data as input data and the article titles as label information;

and the first model confirming submodule is used for taking the original sequence model after the entity recognition training operation, the tail prediction training operation and the article summary training operation are finished as the target sequence model.

9. A computer device comprising a memory having computer readable instructions stored therein and a processor which when executed implements the steps of the method for end-to-end model training for multiple chinese medical language processing tasks of any one of claims 1 to 6.

10. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon computer readable instructions, which when executed by a processor, implement the steps of the end-to-end model training method for multiple chinese medical language processing tasks according to any one of claims 1 to 6.

Images