CN112270184A - Natural language processing method, device and storage medium - Google Patents
Natural language processing method, device and storage medium Download PDFInfo
- Publication number
- CN112270184A CN112270184A CN202011152152.5A CN202011152152A CN112270184A CN 112270184 A CN112270184 A CN 112270184A CN 202011152152 A CN202011152152 A CN 202011152152A CN 112270184 A CN112270184 A CN 112270184A
- Authority
- CN
- China
- Prior art keywords
- word
- text
- network model
- medical
- corrected
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000003058 natural language processing Methods 0.000 title claims abstract description 55
- 238000012549 training Methods 0.000 claims abstract description 49
- 230000011218 segmentation Effects 0.000 claims abstract description 8
- 239000013598 vector Substances 0.000 claims description 40
- 238000012545 processing Methods 0.000 claims description 29
- 230000015654 memory Effects 0.000 claims description 17
- 238000012937 correction Methods 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 9
- 238000011478 gradient descent method Methods 0.000 claims description 7
- 238000004891 communication Methods 0.000 claims description 5
- 230000000903 blocking effect Effects 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 abstract description 3
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 16
- 238000010586 diagram Methods 0.000 description 7
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000003745 diagnosis Methods 0.000 description 2
- 239000003814 drug Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000000306 recurrent effect Effects 0.000 description 2
- 241001522296 Erithacus rubecula Species 0.000 description 1
- 241000238558 Eucarida Species 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 229940079593 drug Drugs 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 229960004329 metformin hydrochloride Drugs 0.000 description 1
- OETHQSJEHLVLGH-UHFFFAOYSA-N metformin hydrochloride Chemical compound Cl.CN(C)C(=N)N=C(N)N OETHQSJEHLVLGH-UHFFFAOYSA-N 0.000 description 1
- XZWYZXLIPXDOLR-UHFFFAOYSA-N metformin hydrochloride Natural products CN(C)C(=N)NC(N)=N XZWYZXLIPXDOLR-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3344—Query execution using natural language analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biomedical Technology (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Databases & Information Systems (AREA)
- Machine Translation (AREA)
Abstract
The application relates to the technical field of medical science and technology, and particularly discloses a natural language processing method, a natural language processing device and a storage medium. The method comprises the following steps: acquiring a text sample; performing word segmentation on the text sample to obtain at least one word; acquiring morphemes corresponding to each word in the at least one word from a pre-constructed semantic knowledge base, and taking the morphemes corresponding to each word as supervision labels of each word in each word; inputting the text sample into a network model to obtain a first morpheme of each word in the text sample; adjusting network parameters of the network model according to the supervision label and the first morpheme of each word in the text sample to obtain a pre-training network model; and performing natural language processing by using the pre-training network model. The method and the device are beneficial to improving the accuracy of natural language processing.
Description
Technical Field
The application relates to the technical field of artificial intelligence, in particular to a natural language processing method, a natural language processing device and a storage medium.
Background
With the great development of artificial intelligence technology, better processing effect is obtained in the field of natural language processing, and great convenience is brought to the life of people. For example, the trained language processing model is used to correct the text input by the user in the dialog box, so as to correctly express the intention of the user, and for example, in the man-machine dialog, such as siri speech, the spoken language of the user is understood through the trained language processing, so as to execute the intention of the user.
Although different trained language processing models can execute different natural language processing tasks, training is performed only depending on the literal semantics of the language in the training process, and the potential semantics of the language cannot be mined, so that the processing precision in the natural language processing process is low.
Disclosure of Invention
The embodiment of the application provides a natural language processing method, a natural language processing device and a storage medium. By integrating morpheme information of each word, the potential semantics of the language can be mined, and the processing progress of the natural language is improved.
In a first aspect, an embodiment of the present application provides a natural language processing method, including:
acquiring a text sample;
performing word segmentation on the text sample to obtain at least one word;
acquiring morphemes corresponding to each word in the at least one word from a pre-constructed semantic knowledge base, and taking the morphemes corresponding to each word as supervision labels of each word in each word;
inputting the text sample into a network model to obtain a first morpheme of each word in the text sample;
adjusting network parameters of the network model according to the supervision label and the first morpheme of each word in the text sample to obtain a pre-training network model;
and performing natural language processing by using the pre-training network model.
In a second aspect, an embodiment of the present application provides a natural language processing apparatus, including:
the acquisition unit is used for acquiring a text sample;
the processing unit is used for segmenting words of the text sample to obtain at least one word;
the processing unit is further configured to acquire a morpheme corresponding to each word in the at least one word from a pre-constructed semantic knowledge base, and use the morpheme corresponding to each word as a supervision tag of each word in each word;
the processing unit is further configured to input the text sample into a network model to obtain a first morpheme of each word in the text sample;
the processing unit is further used for adjusting network parameters of the network model according to the supervision label and the first morpheme of each word in the text sample to obtain a pre-training network model;
the processing unit is further configured to perform natural language processing using the pre-training network model.
In a third aspect, an embodiment of the present application provides a natural language processing apparatus, including a processor, a memory, a communication interface, and one or more programs, where the one or more programs are stored in the memory and configured to be executed by the processor, and the program includes instructions for executing steps in the method according to the first aspect.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, which stores a computer program, where the computer program makes a computer execute the method according to the first aspect.
In a fifth aspect, embodiments of the present application provide a computer program product comprising a non-transitory computer-readable storage medium storing a computer program, the computer being operable to cause a computer to perform the method according to the first aspect.
The embodiment of the application has the following beneficial effects:
it can be seen that in the process of pre-training the network model, the morpheme information of each word needs to be aligned, i.e. model training is performed using the implicit semantics of each word. In this way, after the network model is iterated for multiple times, the pre-training network model obtained in the subsequent natural language processing process contains morpheme information (implicit semantic information) corresponding to each word in the word vector obtained by coding each word, so that the word vector contains more semantic information, and the accuracy of the natural language processing is further improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic flowchart of a natural language processing method according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of a network model according to an embodiment of the present application;
fig. 3 is a schematic flowchart of a method for constructing a text sample according to an embodiment of the present application;
fig. 4 is a schematic diagram of a medical text correction provided in an embodiment of the present application;
fig. 5 is a schematic structural diagram of a natural language processing apparatus according to an embodiment of the present application;
fig. 6 is a block diagram illustrating functional units of a natural language processing apparatus according to an embodiment of the present disclosure.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The terms "first," "second," "third," and "fourth," etc. in the description and claims of this application and in the accompanying drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.
Reference herein to "an embodiment" means that a particular feature, result, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
Referring to fig. 1, fig. 1 is a schematic flowchart of a natural language processing method according to an embodiment of the present application. The method is applied to a natural language processing device. The method comprises the following steps:
101: a natural language processing device obtains a text sample.
The text sample is pre-constructed, and the construction process of the text sample will be described in detail later, which will not be described herein too much.
102: and the natural language processing device divides words of the text sample to obtain at least one word.
Illustratively, the text sample may be segmented by an existing segmentation network, such as a round robin network, a long-short term memory network, or the like. The word segmentation process is the prior art and is not described.
103: and the natural language processing device acquires the morpheme corresponding to each word in the at least one word from a pre-constructed semantic knowledge base, and takes the morpheme corresponding to each word as a supervision label of each word in each word.
Illustratively, the semantic knowledge base is a knowledge base composed of morphemes of each word constructed in advance. Therefore, dictionary matching can be performed on each word of at least one word in the text sample and the semantic knowledge base respectively to obtain a morpheme corresponding to each word in the text sample, and the morpheme corresponding to each word is used as a supervision label of each word to further obtain a supervision label corresponding to each word in the text sample.
104: and the natural language processing device inputs the text sample into a network model to obtain a first morpheme of each word in the text sample.
Illustratively, the text sample is input into a network model, and each word in at least one word in the text sample is encoded, such as word embedding, to obtain a word vector corresponding to each word; and then performing morpheme prediction according to the word vector corresponding to each word to obtain a first morpheme corresponding to each word, and taking the first morpheme corresponding to each word as the morpheme corresponding to each word in each word to further obtain the first morpheme of each word in the text sample.
105: and the natural language processing device adjusts the network parameters of the network model according to the supervision label and the first morpheme of each word in the text sample to obtain a pre-training network model.
For example, a first loss of each word is determined according to the corresponding supervision tag of each word and the first morpheme, for example, the euclidean distance between the supervision tag of each word and the first morpheme may be used as the first loss; then, network parameters of the network model are adjusted according to the first loss of each word and a gradient descent method. For example, determining cross entropy loss of each word to obtain a first loss corresponding to each word, then taking an average value of the first losses of all words in the text sample as a target loss, and adjusting network parameters of the network model according to the target loss and a gradient descent method until the network model converges to obtain a pre-training network model.
Illustratively, the target loss can be represented by equation (1):
wherein L ismFor target loss, Cross _ Encopy is the Cross Entropy loss, N is the number of words in the text sample, and N is an integer greater than or equal to 1, θiIs the surveillance tag for the ith word,
is the first morpheme of the ith word.
106: and the natural language processing device uses the pre-training network model to perform natural language processing.
Illustratively, the pre-trained network model may be used for text correction, intent recognition, spoken language understanding, human-computer interaction, and so forth.
It can be seen that in the process of pre-training the network model, the morpheme information of each word needs to be aligned, i.e. model training is performed using the implicit semantics of each word. In this way, after the network model is iterated for multiple times, the pre-training network model obtained in the subsequent natural language processing process contains morpheme information (implicit semantic information) corresponding to each word in the word vector obtained by coding each word, so that the word vector contains more semantic information, and the accuracy of the natural language processing is further improved.
In one embodiment of the present application, the natural language processing method of the present application may be applied to the field of smart medicine. For example, the pre-training model can be finely tuned, and when a doctor searches for a historical case, the finely tuned network model can be used for correcting the medical text input by the doctor, so that the medical text input by the doctor is correct, the historical case can be accurately searched, case reference is provided for the current diagnosis of the doctor, the diagnosis efficiency of the doctor is improved, and the development of medical science and technology is further promoted.
The process of training the network model to obtain the pre-trained network model is illustrated below by referring to a schematic structural diagram of the network model.
As shown in fig. 2, the network model includes an embedding layer, an encoding layer, which may be an Albert encoder, and a classification layer.
Performing word segmentation on the text sample to obtain at least one word [ X1, X2, X3, … …, Xn ], matching morphemes corresponding to each word in the at least one word in a semantic knowledge base, and taking the morphemes corresponding to each word as supervision labels of each word in the word;
then, carrying out word embedding processing on each word through an embedding layer to obtain a word vector of each word; then, the word vectors of each word are encoded through an encoding layer to obtain a target feature vector of each word, for example, the word vectors of the words can be fused through an attention mechanism; finally, performing classification prediction on each word through a classification layer and a target characteristic vector of each word to obtain a first morpheme corresponding to each word, and taking the first morpheme corresponding to each word as the first morpheme of each word in the word; finally, obtaining loss according to the first morpheme of each word and the supervision label; and adjusting the network parameters of the network model according to the loss and gradient descent method until the model converges to obtain a pre-training network model.
Referring to fig. 3, fig. 3 is a schematic flowchart of a method for constructing a text sample according to an embodiment of the present disclosure. The method comprises the following steps:
301: a first text sequence is obtained.
Wherein the first text sequence is an original text sequence.
302: and replacing the target words in the first text sequence to obtain at least one second text sequence.
The target word can be other words except the stop word, the entity word and the vertical keywords in the first text sequence. For example, a first text sequence "i want to take a drug, metformin hydrochloride tablet", may be replaced with "want", or "want", etc. for "want" in the first text sequence. The intention of the first text sequence is not changed by replacing the target word, but the expression mode of the first text sequence is changed, so that the first text sequence can be expanded into a plurality of second text sequences with the same intention, and a plurality of abundant corpora corresponding to the intention are obtained, so that the fact that the morphemes of partial words, such as unchanged words in the first text sequence, can be recognized by the network model in different expression modes is achieved, and the generalization capability of the network model is improved.
303: and replacing the entity of each second text sequence in the at least one second text sequence to obtain at least one third text sequence corresponding to each second text sequence.
For example, an entity of each of the at least one second text sequence may be determined, wherein the entity of each second text sequence may be implemented by a recurrent neural network, a long-short term memory network, and will not be described; then, at least one candidate entity corresponding to each entity of the second text sequence is obtained, wherein the entity type of each candidate entity in the at least one candidate entity is the same as the entity type of each entity of the second text sequence. And finally, replacing the entities in the second text sequence by using each entity in the at least one candidate entity to obtain at least one third text sequence corresponding to each second text sequence.
It should be understood that the replacement of the entities in each second text sequence mainly expands the richness of the text samples in each entity field, so that after the network model is trained by using such text samples, the morphemes of each entity in each field can be encoded, thereby improving the generalization capability of the network model.
304: and taking each third text sequence in the at least one third text sequence corresponding to each second text sequence as the training text.
The following illustrates an application scenario of the pre-training network.
Scene 1: and (3) using the pre-training network model to carry out medical text error correction.
Illustratively, the medical text may be obtained first; fine-tuning (fine-tuning) the pre-trained network model using the medical text; and correcting the medical text to be corrected by using the fine-tuned network model.
For example, medical text may be read from a medical database and used as the correct medical text, i.e. a supervision tag; then, randomly selecting a first word from the medical text as a word to be replaced, and acquiring a candidate word corresponding to the first word from a dictionary library, wherein the candidate word is an error-prone word corresponding to the first word, such as a shape word, a pronunciation word and the like of the first word; then, replacing the candidate word with a first word in the medical text to obtain a training sample; finally, inputting the training sample into the pre-training network to obtain an error correction result, and obtaining loss according to the error correction result and the medical text (supervision label); the pre-trained network is fine-tuned using the loss and gradient descent method. Then, after the fine tuning is completed, the fine-tuned network model can be used to correct the medical text to be corrected.
Illustratively, as shown in fig. 4, the entities in the medical text to be corrected are determined, for example, the entities in the medical text to be corrected can be identified by the recurrent neural network RNN or the long-short term memory network LSTM; then, acquiring a medical knowledge map corresponding to the entity from a pre-constructed medical knowledge map library, and coding the medical knowledge map to obtain a map vector; coding each word in the medical text to be corrected to obtain a word vector corresponding to each word, and splicing the word vector of each word with the map vector to obtain a target characteristic vector corresponding to each word; and correcting the medical text to be corrected according to the target characteristic vector corresponding to each word to obtain the corrected medical text.
Exemplarily, the score corresponding to each word can be determined according to the target feature vector corresponding to each word, and the word with the score smaller than the threshold value is taken as the word to be corrected; then, at least one candidate word corresponding to the word to be corrected is obtained from the dictionary library; and finally, determining the score of each candidate word in the at least one candidate word, and replacing the word to be corrected with the candidate word with the largest score to obtain the corrected medical text, wherein the score of each candidate word is determined to be similar to the score of each word in the corrected medical text, for example, replacing the word to be corrected with each candidate word in sequence to obtain the replaced medical text, and obtaining the score of each candidate word through the replaced medical text without description.
It can be seen that, in the process of coding each word in the text to be corrected, the network model after fine tuning can code the morpheme information of each word into the word vector corresponding to the word, so that the morpheme information of each word can be used in the process of correcting the text to be corrected, and the correction precision is improved. And medical map knowledge is combined in the error correction process, so that the error correction precision is further improved.
Scene 2: intent recognition in the process of spoken language understanding.
Illustratively, a training sample and a training label are obtained, wherein the training label is an intention labeling result of the training sample; fine-tuning the pre-training model by using the training sample to obtain a fine-tuned network model, wherein the fine-tuning of the pre-training model is to fine-tune the loss between the prediction result of the training sample and the training label without detailed description; and performing intention recognition on the text to be recognized by using the finely tuned network model to obtain the intention corresponding to the text to be recognized, wherein the text to be recognized is obtained by performing voice conversion on the words of the user. Coding each word in the text to be recognized to obtain a word vector of each word, and filling slot positions according to the word vector of each word; and determining the intention of the text to be recognized according to the slot filling result of each word.
It can be seen that, in the present embodiment, in the process of understanding spoken language, when the text to be recognized is subjected to intent recognition, morpheme information of each word may be encoded into a word vector of each word, so that the slot filling accuracy of the word may be improved, and then the accuracy of intent recognition may be improved, thereby better performing spoken language understanding.
Referring to fig. 5, fig. 5 is a schematic structural diagram of a natural language processing device according to an embodiment of the present disclosure. As shown in fig. 4, the natural language processing apparatus 400 includes a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, the programs including instructions for performing the steps of:
acquiring a text sample;
performing word segmentation on the text sample to obtain at least one word;
acquiring morphemes corresponding to each word in the at least one word from a pre-constructed semantic knowledge base, and taking the morphemes corresponding to each word as supervision labels of each word in each word;
inputting the text sample into a network model to obtain a first morpheme of each word in the text sample;
adjusting network parameters of the network model according to the supervision label and the first morpheme of each word in the text sample to obtain a pre-training network model;
and performing natural language processing by using the pre-training network model.
In some possible embodiments, the program is specifically configured to, in adjusting the network parameters of the network model according to the supervised label and the first morpheme of each word in the text sample to obtain a pre-trained network model, execute the following steps:
determining a first loss corresponding to each word in the text sample according to the supervision label and the first morpheme of each word in the text sample;
and adjusting the network parameters of the network model according to the first loss corresponding to each word in the text sample and a gradient descent method to obtain a pre-training network model.
In some possible embodiments, the program is further for instructions to, prior to obtaining the text sample, perform the steps of:
acquiring a first text sequence;
replacing target words in the first text sequence to obtain at least one second text sequence;
replacing an entity of each second text sequence in the at least one second text sequence to obtain at least one third text sequence corresponding to each second text sequence;
and taking each third text sequence in the at least one third text sequence corresponding to each second text sequence as the training text.
In some possible embodiments, the program is specifically configured to, in terms of replacing a part of words in the first text sequence to obtain at least one second text sequence, execute the following steps:
blocking a target word in the first text sequence;
predicting the target words by using a Bert model to obtain at least one word to be replaced;
and replacing the target words by using each word to be replaced in the at least one word to be replaced to obtain at least one second text sequence.
In some possible embodiments, in the case of medical text correction using the pre-trained network model, the program is specifically for instructions to perform the following steps in terms of natural language processing using the pre-trained network model:
acquiring a medical text;
fine-tuning the pre-trained network model using the medical text;
and correcting the medical text to be corrected by using the finely adjusted network model.
In some possible embodiments, in the aspect of correcting the medical text to be corrected using the fine-tuned network model, the program is specifically for executing the following steps:
determining an entity in the medical text to be corrected;
acquiring a medical knowledge map corresponding to the entity from a pre-constructed medical knowledge map library;
and correcting the medical text to be corrected according to the medical knowledge map corresponding to the entity to obtain the corrected medical text.
In some possible embodiments, in terms of performing error correction on the medical text to be corrected according to the medical knowledge graph corresponding to the entity to obtain an error-corrected medical text, the above-mentioned program is specifically used to execute the following instructions:
coding each word in the medical text to be corrected to obtain a word vector corresponding to each word in the medical text to be corrected;
coding the medical knowledge map corresponding to the entity to obtain a map vector;
splicing the word vector corresponding to each word in the medical text to be corrected with the map vector to obtain a target characteristic vector corresponding to each word in the medical text to be corrected;
and correcting the medical text to be corrected according to the target characteristic vector corresponding to each word in the medical text to be corrected to obtain the corrected medical text.
Referring to fig. 6, fig. 6 is a block diagram illustrating functional units of a natural language processing apparatus according to an embodiment of the present disclosure. The natural language processing apparatus 600 includes: an acquisition unit 601 and a processing unit 602, wherein:
an obtaining unit 601, configured to obtain a text sample;
the processing unit 602 is configured to perform word segmentation on the text sample to obtain at least one word;
the processing unit 602 is further configured to obtain a morpheme corresponding to each word in the at least one word from a pre-constructed semantic knowledge base, and use the morpheme corresponding to each word as a supervision tag of each word in each word;
the processing unit 602 is further configured to input the text sample into a network model, so as to obtain a first morpheme of each word in the text sample;
the processing unit 602 is further configured to adjust a network parameter of the network model according to the supervision label and the first morpheme of each word in the text sample, so as to obtain a pre-training network model;
the processing unit 602 is further configured to perform natural language processing using the pre-training network model.
In some possible embodiments, in terms of adjusting the network parameters of the network model according to the supervised label and the first morpheme of each word in the text sample to obtain a pre-trained network model, the processing unit 602 is specifically configured to:
determining a first loss corresponding to each word in the text sample according to the supervision label and the first morpheme of each word in the text sample;
and adjusting the network parameters of the network model according to the first loss corresponding to each word in the text sample and a gradient descent method to obtain a pre-training network model.
In some possible embodiments, before acquiring the text sample, the acquiring unit 601 is further configured to: acquiring a first text sequence;
the processing unit 602 is further configured to replace a target word in the first text sequence to obtain at least one second text sequence;
replacing an entity of each second text sequence in the at least one second text sequence to obtain at least one third text sequence corresponding to each second text sequence;
and taking each third text sequence in the at least one third text sequence corresponding to each second text sequence as the training text.
In some possible embodiments, in terms of replacing a part of words in the first text sequence to obtain at least one second text sequence, the processing unit 602 is specifically configured to:
blocking a target word in the first text sequence;
predicting the target words by using a Bert model to obtain at least one word to be replaced;
and replacing the target words by using each word to be replaced in the at least one word to be replaced to obtain at least one second text sequence.
In some possible embodiments, in the case of medical text correction using the pre-trained network model, the program is specifically for instructions to perform the following steps in terms of natural language processing using the pre-trained network model:
acquiring a medical text;
fine-tuning the pre-trained network model using the medical text;
and correcting the medical text to be corrected by using the finely adjusted network model.
In some possible embodiments, in terms of performing error correction on the medical text to be corrected by using the fine-tuned network model, the processing unit 602 is specifically configured to:
determining an entity in the medical text to be corrected;
acquiring a medical knowledge map corresponding to the entity from a pre-constructed medical knowledge map library;
and correcting the medical text to be corrected according to the medical knowledge map corresponding to the entity to obtain the corrected medical text.
In some possible embodiments, in terms of performing error correction on the medical text to be corrected according to the medical knowledge graph corresponding to the entity to obtain an error-corrected medical text, the processing unit 602 is specifically configured to:
coding each word in the medical text to be corrected to obtain a word vector corresponding to each word in the medical text to be corrected;
coding the medical knowledge map corresponding to the entity to obtain a map vector;
splicing the word vector corresponding to each word in the medical text to be corrected with the map vector to obtain a target characteristic vector corresponding to each word in the medical text to be corrected;
and correcting the medical text to be corrected according to the target characteristic vector corresponding to each word in the medical text to be corrected to obtain the corrected medical text.
It should be understood that the natural language processing device in the present application may include a smart Phone (e.g., an Android Phone, an iOS Phone, a Windows Phone, etc.), a tablet computer, a palm computer, a notebook computer, a Mobile Internet device MID (MID), a wearable device, or the like. The above-mentioned natural language processing device is only an example, not an exhaustive list, and includes but is not limited to the above-mentioned natural language processing device. In practical applications, the natural language processing apparatus may further include: intelligent vehicle terminals, computer equipment, etc.
Embodiments of the present application also provide a computer storage medium, which stores a computer program, where the computer program is executed by a processor to implement part or all of the steps of any one of the natural language processing methods as described in the above method embodiments.
Embodiments of the present application also provide a computer program product comprising a non-transitory computer readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any one of the natural language processing methods as set forth in the above method embodiments.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are exemplary embodiments and that the acts and modules referred to are not necessarily required in this application.
In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implementing, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of some interfaces, devices or units, and may be an electric or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit may be implemented in the form of hardware, or may be implemented in the form of a software program module.
The integrated units, if implemented in the form of software program modules and sold or used as stand-alone products, may be stored in a computer readable memory. Based on such understanding, the technical solution of the present application may be substantially implemented or a part of or all or part of the technical solution contributing to the prior art may be embodied in the form of a software product stored in a memory, and including several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method described in the embodiments of the present application. And the aforementioned memory comprises: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by associated hardware instructed by a program, which may be stored in a computer-readable memory, which may include: flash Memory disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.
The foregoing detailed description of the embodiments of the present application has been presented to illustrate the principles and implementations of the present application, and the above description of the embodiments is only provided to help understand the method and the core concept of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.
Claims (10)
1. A natural language processing method, comprising:
acquiring a text sample;
performing word segmentation on the text sample to obtain at least one word;
acquiring morphemes corresponding to each word in the at least one word from a pre-constructed semantic knowledge base, and taking the morphemes corresponding to each word as supervision labels of each word in each word;
inputting the text sample into a network model to obtain a first morpheme of each word in the text sample;
adjusting network parameters of the network model according to the supervision label and the first morpheme of each word in the text sample to obtain a pre-training network model;
and performing natural language processing by using the pre-training network model.
2. The method of claim 1, wherein adjusting network parameters of the network model according to the supervised label and the first morpheme of each word in the text sample to obtain a pre-trained network model comprises:
determining a first loss corresponding to each word in the text sample according to the supervision label and the first morpheme of each word in the text sample;
and adjusting the network parameters of the network model according to the first loss corresponding to each word in the text sample and a gradient descent method to obtain a pre-training network model.
3. The method of claim 1 or 2, wherein prior to obtaining the text sample, the method further comprises:
acquiring a first text sequence;
replacing target words in the first text sequence to obtain at least one second text sequence;
replacing an entity of each second text sequence in the at least one second text sequence to obtain at least one third text sequence corresponding to each second text sequence;
and taking each third text sequence in the at least one third text sequence corresponding to each second text sequence as the training text.
4. The method of claim 3, wherein the replacing partial words in the first text sequence to obtain at least one second text sequence comprises:
blocking a target word in the first text sequence;
predicting the target words by using a Bert model to obtain at least one word to be replaced;
and replacing the target words by using each word to be replaced in the at least one word to be replaced to obtain at least one second text sequence.
5. The method according to any one of claims 1-4, wherein in the case of medical text correction using the pre-trained network model, the performing natural language processing using the pre-trained network model comprises:
acquiring a medical text;
fine-tuning the pre-trained network model using the medical text;
and correcting the medical text to be corrected by using the finely adjusted network model.
6. The method of claim 5, wherein the correcting the medical text to be corrected using the refined network model comprises:
determining an entity in the medical text to be corrected;
acquiring a medical knowledge map corresponding to the entity from a pre-constructed medical knowledge map library;
and correcting the medical text to be corrected according to the medical knowledge map corresponding to the entity to obtain the corrected medical text.
7. The method according to claim 6, wherein the correcting the medical text to be corrected according to the medical knowledge map corresponding to the entity to obtain a corrected medical text comprises:
coding each word in the medical text to be corrected to obtain a word vector corresponding to each word in the medical text to be corrected;
coding the medical knowledge map corresponding to the entity to obtain a map vector;
splicing the word vector corresponding to each word in the medical text to be corrected with the map vector to obtain a target characteristic vector corresponding to each word in the medical text to be corrected;
and correcting the medical text to be corrected according to the target characteristic vector corresponding to each word in the medical text to be corrected to obtain the corrected medical text.
8. A natural language processing apparatus, comprising:
the acquisition unit is used for acquiring a text sample;
the processing unit is used for segmenting words of the text sample to obtain at least one word;
the processing unit is further configured to acquire a morpheme corresponding to each word in the at least one word from a pre-constructed semantic knowledge base, and use the morpheme corresponding to each word as a supervision tag of each word in each word;
the processing unit is further configured to input the text sample into a network model to obtain a first morpheme of each word in the text sample;
the processing unit is further used for adjusting network parameters of the network model according to the supervision label and the first morpheme of each word in the text sample to obtain a pre-training network model;
the processing unit is further configured to perform natural language processing using the pre-training network model.
9. A natural language processing apparatus comprising a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, the programs comprising instructions for performing the steps in the method of any of claims 1-7.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program which is executed by a processor to implement the method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011152152.5A CN112270184B (en) | 2020-10-23 | 2020-10-23 | Natural language processing method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011152152.5A CN112270184B (en) | 2020-10-23 | 2020-10-23 | Natural language processing method, device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112270184A true CN112270184A (en) | 2021-01-26 |
| CN112270184B CN112270184B (en) | 2023-11-14 |
Family
ID=74341694
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011152152.5A Active CN112270184B (en) | 2020-10-23 | 2020-10-23 | Natural language processing method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112270184B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112860896A (en) * | 2021-03-05 | 2021-05-28 | 三一重工股份有限公司 | Corpus generalization method and man-machine conversation emotion analysis method for industrial field |
| CN113569974A (en) * | 2021-08-04 | 2021-10-29 | 网易（杭州）网络有限公司 | Error correction method and device for programming statement, electronic equipment and storage medium |
| CN114048321A (en) * | 2021-08-12 | 2022-02-15 | 湖南达德曼宁信息技术有限公司 | Multi-granularity text error correction data set generation method, device and equipment |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080275694A1 (en) * | 2007-05-04 | 2008-11-06 | Expert System S.P.A. | Method and system for automatically extracting relations between concepts included in text |
| US20190220749A1 (en) * | 2018-01-17 | 2019-07-18 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Text processing method and device based on ambiguous entity words |
| CN110442870A (en) * | 2019-08-02 | 2019-11-12 | 深圳市珍爱捷云信息技术有限公司 | Text error correction method, device, computer equipment and storage medium |
| CN110598213A (en) * | 2019-09-06 | 2019-12-20 | 腾讯科技（深圳）有限公司 | Keyword extraction method, device, equipment and storage medium |
| CN111062217A (en) * | 2019-12-19 | 2020-04-24 | 江苏满运软件科技有限公司 | Language information processing method and device, storage medium and electronic equipment |
| CN111507104A (en) * | 2020-03-19 | 2020-08-07 | 北京百度网讯科技有限公司 | Method and device for establishing label labeling model, electronic equipment and readable storage medium |
| CN111783451A (en) * | 2020-06-30 | 2020-10-16 | 北京百度网讯科技有限公司 | Method and apparatus for enhancing text samples |
-
2020
- 2020-10-23 CN CN202011152152.5A patent/CN112270184B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080275694A1 (en) * | 2007-05-04 | 2008-11-06 | Expert System S.P.A. | Method and system for automatically extracting relations between concepts included in text |
| US20190220749A1 (en) * | 2018-01-17 | 2019-07-18 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Text processing method and device based on ambiguous entity words |
| CN110442870A (en) * | 2019-08-02 | 2019-11-12 | 深圳市珍爱捷云信息技术有限公司 | Text error correction method, device, computer equipment and storage medium |
| CN110598213A (en) * | 2019-09-06 | 2019-12-20 | 腾讯科技（深圳）有限公司 | Keyword extraction method, device, equipment and storage medium |
| CN111062217A (en) * | 2019-12-19 | 2020-04-24 | 江苏满运软件科技有限公司 | Language information processing method and device, storage medium and electronic equipment |
| CN111507104A (en) * | 2020-03-19 | 2020-08-07 | 北京百度网讯科技有限公司 | Method and device for establishing label labeling model, electronic equipment and readable storage medium |
| CN111783451A (en) * | 2020-06-30 | 2020-10-16 | 北京百度网讯科技有限公司 | Method and apparatus for enhancing text samples |
Non-Patent Citations (1)
| Title |
|---|
| HOANG NGUYEN等: "Dynamic Semantic Matching and Aggregation Network for Few-shot Intent Detection", HTTPS://ARXIV.ORG/ABS/2010.02481, pages 1 - 10 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112860896A (en) * | 2021-03-05 | 2021-05-28 | 三一重工股份有限公司 | Corpus generalization method and man-machine conversation emotion analysis method for industrial field |
| CN113569974A (en) * | 2021-08-04 | 2021-10-29 | 网易（杭州）网络有限公司 | Error correction method and device for programming statement, electronic equipment and storage medium |
| CN113569974B (en) * | 2021-08-04 | 2023-07-18 | 网易（杭州）网络有限公司 | Programming statement error correction method, device, electronic equipment and storage medium |
| CN114048321A (en) * | 2021-08-12 | 2022-02-15 | 湖南达德曼宁信息技术有限公司 | Multi-granularity text error correction data set generation method, device and equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112270184B (en) | 2023-11-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111931490B (en) | Text error correction method, device and storage medium | |
| CN112528637B (en) | Text processing model training method, device, computer equipment and storage medium | |
| CN110163181B (en) | Sign language identification method and device | |
| CN112270184A (en) | Natural language processing method, device and storage medium | |
| CN113869044A (en) | Keyword automatic extraction method, device, equipment and storage medium | |
| EP4131076A1 (en) | Serialized data processing method and device, and text processing method and device | |
| CN111444715B (en) | Entity relationship identification method and device, computer equipment and storage medium | |
| CN112188311B (en) | Method and apparatus for determining video material of news | |
| CN111950303B (en) | Medical text translation method, device and storage medium | |
| CN114676234A (en) | Model training method and related equipment | |
| CN108304376B (en) | Text vector determination method and device, storage medium and electronic device | |
| US10915756B2 (en) | Method and apparatus for determining (raw) video materials for news | |
| CN114298121A (en) | Multi-mode-based text generation method, model training method and device | |
| CN113408287B (en) | Entity identification method and device, electronic equipment and storage medium | |
| CN115062134B (en) | Knowledge question-answering model training and knowledge question-answering method, device and computer equipment | |
| CN113657105A (en) | Medical entity extraction method, device, equipment and medium based on vocabulary enhancement | |
| CN111145914B (en) | Method and device for determining text entity of lung cancer clinical disease seed bank | |
| CN110659392B (en) | Retrieval method and device, and storage medium | |
| CN116341651A (en) | Entity recognition model training method and device, electronic equipment and storage medium | |
| CN111368531B (en) | Translation text processing method and device, computer equipment and storage medium | |
| CN112307749A (en) | Text error detection method and device, computer equipment and storage medium | |
| CN110852066B (en) | Multi-language entity relation extraction method and system based on confrontation training mechanism | |
| CN112016281B (en) | Method and device for generating wrong medical text and storage medium | |
| CN114758330A (en) | Text recognition method and device, electronic equipment and storage medium | |
| CN114417891A (en) | Reply sentence determination method and device based on rough semantics and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |