CN113177108A - Semantic role labeling method and device, computer equipment and storage medium - Google Patents
Semantic role labeling method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN113177108A CN113177108A CN202110587690.5A CN202110587690A CN113177108A CN 113177108 A CN113177108 A CN 113177108A CN 202110587690 A CN202110587690 A CN 202110587690A CN 113177108 A CN113177108 A CN 113177108A
- Authority
- CN
- China
- Prior art keywords
- probability
- input sample
- role
- word
- labeling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000002372 labelling Methods 0.000 title claims abstract description 113
- 238000000034 method Methods 0.000 claims abstract description 26
- 238000012545 processing Methods 0.000 claims abstract description 20
- 230000011218 segmentation Effects 0.000 claims abstract description 20
- 230000015654 memory Effects 0.000 claims description 16
- 230000006870 function Effects 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 12
- 238000004364 calculation method Methods 0.000 claims description 11
- 238000012549 training Methods 0.000 claims description 9
- 238000013473 artificial intelligence Methods 0.000 abstract description 2
- 229920001098 polystyrene-block-poly(ethylene/propylene) Polymers 0.000 description 11
- 230000008569 process Effects 0.000 description 9
- 238000007726 management method Methods 0.000 description 8
- 102100021723 Arginase-1 Human genes 0.000 description 5
- 101000752037 Homo sapiens Arginase-1 Proteins 0.000 description 5
- 101000800287 Homo sapiens Tubulointerstitial nephritis antigen-like Proteins 0.000 description 5
- 102100030356 Arginase-2, mitochondrial Human genes 0.000 description 4
- 101000792835 Homo sapiens Arginase-2, mitochondrial Proteins 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000012544 monitoring process Methods 0.000 description 4
- 230000006978 adaptation Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000012550 audit Methods 0.000 description 2
- 230000002457 bidirectional effect Effects 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000003058 natural language processing Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 101150005709 ARG4 gene Proteins 0.000 description 1
- 108010001267 Protein Subunits Proteins 0.000 description 1
- 101100004044 Vigna radiata var. radiata AUX22B gene Proteins 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013475 authorization Methods 0.000 description 1
- 238000010923 batch production Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000003862 health status Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000012954 risk control Methods 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3344—Query execution using natural language analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3346—Query execution using probabilistic model
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/103—Formatting, i.e. changing of presentation of documents
- G06F40/117—Tagging; Marking up; Designating a block; Setting of attributes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Evolutionary Computation (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biophysics (AREA)
- Databases & Information Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Probability & Statistics with Applications (AREA)
- Machine Translation (AREA)
Abstract
The application relates to the technical field of artificial intelligence, and provides a semantic role labeling method, a semantic role labeling device, computer equipment and a storage medium, wherein the method comprises the following steps: obtaining a sentence to be annotated, and performing word segmentation processing and part-of-speech annotation on the sentence to be annotated; determining verbs in the sentences to be labeled according to the result of part-of-speech labeling; acquiring role labels preset by each verb, and constructing input samples corresponding to the role labels according to the role labels and the word segmentation processing result; inputting each input sample into a semantic role labeling model; in the semantic role labeling model, a first probability when each character in an input sample is used as the starting position and a second probability when the character label corresponding to the input sample is at the ending position are calculated, and role labeling is carried out according to the first probability and the second probability. The semantic role labeling method, the semantic role labeling device, the computer equipment and the storage medium provided by the application can be used for providing the semantic role labeling accuracy.
Description
Technical Field
The present application relates to the field of artificial intelligence technologies, and in particular, to a semantic role labeling method and apparatus, a computer device, and a storage medium.
Background
Semantic role labeling is an important ring of semantic parsing in the field of natural language processing. Usually, our expression is centered on verbs (also called predicates), for example, "go" in "i'm tomorrow go to beijing" in the evening, which is the core of the meaning of the whole sentence expression. Semantic role labeling, which is a technology for recognizing elements of verbs in sentences, is generally labeled as "ARG 1" in the foregoing example as a subject; the "tomorrow evening" is generally labeled "TMP", representing time; "Beijing," generally labeled "LOC," represents a place. The symbol of the role mark is not important, and we can completely define a set of role label system belonging to the role according to the needs, but the meaning behind the role is important, and generally comprises the identification of contents such as 'main and predicate guest, time, place, mode', and the like, and the colloquial expression is that 'who (or what) does what to whom (or what), at what time and place, and in what mode, and the like'. It can be seen that, with semantic role labeling, the meaning of the whole sentence expression can be obtained from the shallow layer, which is helpful for the subsequent language understanding.
The traditional semantic character labeling adopts a BIO (begin, inside, outside) sequence labeling method, and the whole method performs one-time labeling on the input verbs, for example, "I go to Beijing at night tomorrow" is labeled as "B _ ARG0, B _ TMP, I _ TMP, I _ TMP, I _ TMP, O, B _ LOC, I _ LOC". Such a labeling method is directed at the verb "go" at a time, and marks all semantic roles that it governs, so that the accuracy is low.
Disclosure of Invention
The application mainly aims to provide a semantic role labeling method, a semantic role labeling device, computer equipment and a storage medium, and aims to solve the technical problem of low semantic role labeling accuracy.
In order to achieve the above object, the present application provides a semantic role labeling method, which includes the following steps:
obtaining a sentence to be labeled, and performing word segmentation processing and part-of-speech labeling on the sentence to be labeled;
determining verbs in the sentences to be labeled according to the result of the part-of-speech labeling;
acquiring role labels preset by each verb, and constructing input samples corresponding to the role labels according to the role labels and the word segmentation processing result;
inputting each input sample into a semantic role labeling model; wherein the semantic role labeling model is obtained by training based on a BERT, BilSTM, RoBERT or XLNE model;
in the semantic role labeling model, calculating a first probability when each word in the input sample is used as the starting position of the role label corresponding to the input sample and a second probability when each word in the input sample is used as the ending position of the role label corresponding to the input sample, and labeling the role according to the first probability and the second probability.
Further, before the step of calculating a first probability when each word in the input sample is used as the start position of the character label corresponding to the input sample and a second probability when each word in the input sample is used as the end position of the character label corresponding to the input sample, the method includes:
calculating a third probability that a field corresponding to the role label exists in each input sample and a fourth probability that the field corresponding to the role label does not exist;
comparing the third probability to the fourth probability;
if the fourth probability is greater than the third probability, not performing a step of calculating a first probability when each word in the input sample is used as the start position of the role label corresponding to the input sample and a second probability when each word in the input sample is used as the end position of the role label corresponding to the input sample;
if the fourth probability is less than or equal to the third probability, a step of calculating a first probability when each word in the input sample is used as the start position of the role label corresponding to the input sample and a second probability when each word in the input sample is used as the end position of the role label corresponding to the input sample is performed.
Further, the step of labeling the role according to the first probability and the second probability includes:
determining the position of the word corresponding to the maximum first probability in each input sample as the starting position of the role label, and determining the position of the word corresponding to the maximum second probability in each input sample as the ending position of the role label;
and extracting a field between the starting position and the ending position, and marking the role label at the field.
Further, the step of labeling the role according to the first probability and the second probability includes:
comparing the maximum first probability in the input samples with a preset first probability, and if the maximum first probability is greater than the preset first probability, taking the position of the word corresponding to the maximum first probability as the initial position of the character label;
comparing the maximum second probability in the input sample with a preset second probability, and if the maximum second probability is greater than the preset second probability, taking the position of the word corresponding to the maximum second probability as the end position of the role label;
and extracting a field between the starting position and the ending position, and marking the role label at the field.
Further, the step of calculating a first probability when each word in the input sample is used as the start position of the character label corresponding to the input sample and a second probability when each word in the input sample is used as the end position of the character label corresponding to the input sample includes:
calculating a first probability when each word is used as the starting position of the role label corresponding to the input sample and a second probability when each word in the input sample is used as the ending position of the role label corresponding to the input sample by using all input samples corresponding to each verb through a batch calculation mode.
Further, the semantic role labeling model comprises a first fully-connected layer and a second fully-connected layer, and the first fully-connected layer and the second fully-connected layer both comprise a softmax function; the step of calculating a first probability when each word in the input sample is used as the start position of the role label corresponding to the input sample and a second probability when each word in the input sample is used as the end position of the role label corresponding to the input sample includes:
calculating a first probability when each word in the input sample is used as a starting position of the role label corresponding to the input sample through a softmax function of the first full connection layer;
and calculating each word in the input sample as a second probability when the end position of the role label corresponding to the input sample is the second probability through the softmax function of the second full connection layer.
The present application further provides a semantic role labeling apparatus, including:
the system comprises an acquisition unit, a word segmentation unit and a word property tagging unit, wherein the acquisition unit is used for acquiring a sentence to be tagged and carrying out word segmentation processing and part-of-speech tagging on the sentence to be tagged;
the first determining unit is used for determining verbs in the sentences to be annotated according to the result of part-of-speech annotation;
a second determining unit, configured to obtain a role label preset by each verb, and construct an input sample corresponding to each role label according to the role label and a result of the participle processing;
the input unit is used for inputting each input sample to a semantic role labeling model; wherein the semantic role labeling model is obtained by training based on a BERT, BilSTM, RoBERT or XLNE model;
and the calculation unit is used for calculating a first probability when each word in the input sample is used as the starting position of the role label corresponding to the input sample and a second probability when each word in the input sample is used as the ending position of the role label corresponding to the input sample in the semantic role labeling model, and performing role labeling according to the first probability and the second probability.
Further, the calculation unit includes:
the first calculating subunit is configured to calculate a third probability that a field corresponding to the role label exists in each input sample and a fourth probability that the field corresponding to the role label does not exist;
a comparison subunit configured to compare the third probability with the fourth probability;
a non-entry subunit, configured to, if the fourth probability is greater than the third probability, not perform a step of calculating a first probability when each word in the input sample is used as the start position of the character label corresponding to the input sample and a second probability when each word in the input sample is used as the end position of the character label corresponding to the input sample;
and an entering subunit, configured to enter a step of calculating a first probability when each word in the input sample is used as the start position of the character label corresponding to the input sample and a second probability when each word in the input sample is used as the end position of the character label corresponding to the input sample, if the fourth probability is less than or equal to the third probability.
The application also provides a computer device, which comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the semantic role labeling method described in any one of the above items when executing the computer program.
The present application further provides a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the steps of the semantic role labeling method according to any of the above.
The semantic role labeling method, the semantic role labeling device, the computer equipment and the storage medium provided by the application input the input sample into the semantic role labeling model by constructing the input sample of each role label, calculating a first probability when each word in each input sample is used as the initial position of the corresponding role label and a second probability when each word in the input samples is used as the end position of the role label corresponding to the input sample, and then carrying out role labeling according to the first probability and the second probability without labeling each word, only needing to label the initial position and the end position, reducing the number of paths needing decision making of a semantic role labeling model, and reducing the search space of semantic role labeling, meanwhile, only one role label is predicted each time, and the semantic role labeling performance is improved and the semantic role labeling accuracy is improved by reducing the search space of each decision.
Drawings
FIG. 1 is a schematic diagram illustrating steps of a semantic role labeling method according to an embodiment of the present application;
FIG. 2 is a block diagram of a semantic role labeling apparatus according to an embodiment of the present application;
fig. 3 is a block diagram illustrating a structure of a computer device according to an embodiment of the present application.
The implementation, functional features and advantages of the objectives of the present application will be further explained with reference to the accompanying drawings.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
Referring to fig. 1, an embodiment of the present application provides a semantic role labeling method, including the following steps:
step S1, obtaining a sentence to be annotated, and performing word segmentation processing and part-of-speech annotation on the sentence to be annotated;
step S2, determining verbs in the sentences to be annotated according to the result of part-of-speech annotation;
step S3, obtaining role labels preset by each verb, and constructing input samples corresponding to the role labels according to the role labels and the word segmentation processing results;
step S4, inputting each input sample into a semantic role labeling model; wherein the semantic role labeling model is obtained by training based on a BERT, BilSTM, RoBERT or XLNE model;
step S5, in the semantic role labeling model, calculating a first probability when each word in the input sample is used as the start position of the role label corresponding to the input sample and a second probability when each word in the input sample is used as the end position of the role label corresponding to the input sample, and performing role labeling according to the first probability and the second probability.
In this embodiment, as described in the foregoing steps S1-S2, the sentence to be annotated is a sentence that is to be annotated with a semantic character, and the sentence to be annotated may be obtained by the terminal from the received voice information or text content information. The method includes the steps that word segmentation processing and part-of-speech tagging are conducted on a to-be-tagged sentence, specifically, word segmentation processing and part-of-speech tagging can be conducted through a Stanford NLP tool, a jieba word segmentation tool and other tools, and if the to-be-tagged sentence is ' Telnopu yesterday visits China, the fact that China is visited by the to-be-tagged sentence means that China ' Med-Med ' interaction can be increased is shown. ". The result of the segmentation process is "terlang/yesterday/visit/china/,/and/show/meet/enlarge/china/collaborate/exchange/. ". According to the result of part-of-speech tagging, wherein the verbs of access, representation and increase are verbs (the verb part-of-speech tag is generally V), the verbs of the tagged sentences are access, representation and increase.
As described in step S3 above, the role labels can be determined by the user himself which of the 19 labels are used, such as ARG0/ARG1/ARG2/ARG3/ARG4/ARGM-ADV/ARGM-TMP/ARGM-DIS/ARGM-LOC/ARGM-MNR/ARGM-PRP/ARGM-TPC/ARGM-DIR/ARGM-EXT/ARGM-BNF/ARGM-DGR/ARGM-FRQ/C-ARG0/C-ARG1, the user can only label both the ARG0 and the ARG1 role labels, but not all the role labels, and different verbs can label, such as "visit" which can label both the ARG0 and the ARG1 label, and "enlarge" this role can label both the ARG2 and the ARG3 role labels.
According to the result of the word segmentation process and the role labels, corresponding input samples are determined, for example, for the verb "visit", two role labels of ARG0 and ARG1 are required to be marked, and the constructed input samples are respectively "CLS/Te/Lang/Pu/yesterday/< s >/visit/</s >/visited/China/,/and/table/show/meet/add/big/Central/United/doing/AC/streaming/. /SEP/ARG0/SEP "and" CLS/te/lang/pu/yesterday/day/< s >/visit/</s >/visited/china/,/and/show/meet/add/big/me/merge/do/cross/stream/. /SEP/ARG1/SEP "; for the verb "enlarge" to want to label the two role labels ARG2 and ARG3, the resulting input samples are constructed as "CLS/tex/lang/pu/yesterday/day/visit/visited/home/country/,/and/table/show/meet/< s >/add/big/</s >/medium/america/meet/trade/stream/, respectively. /SEP/ARG2/SEP "and" CLS/te/lang/pu/yesterday/day/visit/go/medium/country/,/and/table/show/meet/< s >/add/big/</s >/medium/america/merge/do/cross/flow/. /SEP/ARG 3/SEP'. In each input sample, the verb to which the input sample is focused is determined by two < s >, the role label to which the input sample is focused is determined by two SEPs, and for the input samples "CLS/te/lang/yesterday/< s >/visit/</s >/go/center/state/,/and/table/show/meet/add/up/center/me/co/do/cross/stream/. the/SEP/ARG 0/SEP ", it is understood that the input sample is intended to be labeled with the role label of the ARG0 (subject) of the verb" visit ", with one input sample corresponding to each role label of each verb.
As described in the above steps S4-S5, the semantic role labeling model can be trained based on any one of the BERT (Bidirectional Encoder responses from the Bidirectional attention neural network model), the BilSTM (Bi-directional Long Short-Term Memory), the RoBERTA and the XLNET model.
Taking the BERT model as an example, the BERT model performs one-hot encoding on all words in a whole input sample to obtain respective vector representations [ r ]1，r2，…，rn]. Each input sample represents a role label corresponding to the role label to be labeled, a first probability and a second probability of an end position of each word of the input sample are calculated and used as the start position and the end position of the role label, and the corresponding start position and the end position are determined by calculating the first probability and the second probability, so that a field corresponding to the role label is obtained.
Specifically, the semantic role labeling model obtained by training the BERT model can be expressed as:
R＝BERT(Input)
Vstart＝Wstart·[r1,r2,…,rn]
Vend＝Wend·[r1,r2,…,rn]
YhasAns＝WhasAns·rCLS
xstart＝argmax(Vstart)
xend＝argmax(Vend)
wherein R ═ rcls, R1,r2,…,rn]The representation is a row representation of the R matrix, including a representation of each position of the input, in one-to-one correspondence with the input, which may also be referred to as a context representation, i.e., the output via the BERT, XLNet, etc. encoder. VstartRepresenting the first probability calculated for each input sample, the starting position x is obtained by arg max (taking the maximum score position)start。VendIs the first probability calculated for each input sample, and the end position x is obtained by arg max (taking the maximum score position)endAnd W denotes a parameter matrix.
For example, in the above example, the probabilities of the words as the start position and the end position are obtained by calculation, and if the probability of the first word as the start position is the highest and the probability of the fourth word as the end position is the highest, it is determined that the field corresponding to the role label of the ARG0 of the verb "visit" is "terlangpu", and the role label of the ARG0 is labeled at "terlangpu". When the role labels in the input samples of the same verb all determine corresponding fields, unified labeling can be performed in the sentences to be labeled.
In the implementation, each character is not required to be labeled, only the initial position and the end position are required to be labeled, the number of paths needing to be decided by a semantic character labeling model is reduced, the search space of semantic character labeling is reduced, only one character label is predicted at a time, the performance of semantic character labeling is improved by reducing the search space of decision at each time, and the accuracy of semantic character labeling is improved. Evaluation of semantic roles is generally evaluated using P, R, F1, published as precision, recall, and F1 values, where
On two data sets of public evaluation tasks CoNLL-2005 and CoNLL-2012, compared with the prior art, the technical scheme provided by the application has the advantages that the P value is absolutely improved by 2 percentage points, the R value is slightly reduced by 1 percentage point, and the comprehensive F1 value is 0.8-1 percentage point higher than that of the prior art. When the technical scheme provided by the application is applied to actual services, such as a customer service system, an electric marketing system and the like, the semantic role marking effect is improved, the accuracy of semantic understanding in the customer service system and the electric marketing system can be directly improved, so that the system can acquire accurate semantic roles related to a principal and a predicate guest, time and place from a natural language spoken or input by a user, thereby helping customer service and enterprises to improve the description and understanding capacity of the user, and improving the customer satisfaction degree.
In an embodiment, before the step S5 of calculating a first probability when each word in the input sample is used as the start position of the character label corresponding to the input sample and a second probability when each word in the input sample is used as the end position of the character label corresponding to the input sample, the method includes:
step S5a, calculating a third probability that the field corresponding to the role label exists in each input sample and a fourth probability that the field corresponding to the role label does not exist;
step S5b, comparing the third probability with the fourth probability;
step S5c, if the fourth probability is greater than the third probability, not performing the step of calculating a first probability when each word in the input sample is used as the start position of the character label corresponding to the input sample and a second probability when each word in the input sample is used as the end position of the character label corresponding to the input sample;
step S5d, if the fourth probability is less than or equal to the third probability, performing a step of calculating a first probability when each word in the input sample is used as the start position of the character label corresponding to the input sample and a second probability when each word in the input sample is used as the end position of the character label corresponding to the input sample.
In this embodiment, a full connection layer is disposed in the semantic role labeling model, the input samples are input into the full connection layer, the full connection layer can calculate to obtain a classification result, that is, a third probability and a fourth probability of each input sample, when the fourth probability is greater than the third probability, it is indicated that a field corresponding to a corresponding role label does not exist in the input sample, it is not necessary to subsequently calculate the first probability and the second probability, the amount of calculation is reduced, and it is avoided that when a field corresponding to a role label does not exist, a field is still determined according to the third probability and the fourth probability, and at this time, the field is wrong with respect to the corresponding role label.
In an embodiment, the step S5 of labeling the character according to the first probability and the second probability includes:
step S51, determining a position of a word corresponding to the maximum first probability in each input sample as a start position of the character tag, and determining a position of a word corresponding to the maximum second probability in each input sample as an end position of the character tag;
step S52, extracting a field between the start position and the end position, and labeling the role label at the field.
In this embodiment, each word in each input sample calculates a first probability and a second probability, the first probability indicates a probability that the word is used as an initial position of a character tag in the input sample, the greater the first probability, the more likely the word is to be the initial position, the greater the accuracy, the same reason is that the position of the word corresponding to the maximum first probability is selected as the initial position, and the position of the word corresponding to the maximum second probability is selected as the end position. By selecting the words corresponding to the maximum first probability and the maximum second probability as the starting position and the ending position, semantic role labeling can be more accurately carried out.
In an embodiment, the step S5 of labeling the character according to the first probability and the second probability includes:
step S5A, comparing the maximum first probability in the input samples with a preset first probability, and if the maximum first probability is greater than the preset first probability, using the position of the word corresponding to the maximum first probability as the start position of the character label;
step S5B, comparing the maximum second probability in the input sample with a preset second probability, and if the maximum second probability is greater than the preset second probability, taking a position of a word corresponding to the maximum second probability as an end position of the role label;
and step S5C, extracting a field between the starting position and the ending position, and marking the role label at the field.
In this embodiment, the preset first probability and the preset second probability may be equal, after the first probability and the second probability of each character are calculated in each input sample, when the maximum first probability needs to be greater than the preset first probability and the maximum second probability needs to be greater than the preset second probability, the corresponding position of the character can be used as the starting position and the ending position of the character tag, so as to avoid a situation that the value of the first probability is relatively small even if the maximum first probability is the maximum first probability, in which case, the obtained result is inaccurate, and only when the maximum first probability needs to be greater than the preset first probability and the maximum second probability needs to be greater than the preset second probability, the corresponding starting position and the corresponding ending position are determined.
In an embodiment, in the semantic character labeling model, the step of calculating a first probability when each word in the input sample is used as the start position of the character label corresponding to the input sample and a second probability when each word in the input sample is used as the end position of the character label corresponding to the input sample includes:
and calculating a first probability when each word is used as the starting position of the role label corresponding to the input sample and a second probability when each word in the input sample is used as the ending position of the role label corresponding to the input sample by using all input samples corresponding to the verbs through a batch calculation mode.
In this embodiment, the batch is a hyper-parameter, and is used to define the number of samples to be processed by the semantic role labeling. The first probability and the second probability are calculated in a batch calculation mode, an input sample can be processed more quickly, semantic role labeling is carried out, and one semantic role labeling result can be output by the same verb. In other embodiments, different verbs may output a semantic role annotation result. In another embodiment, in model training, the value of batch is set, and the batch is treated as a loop to iterate through one or more samples and make predictions. At the end of the batch process, the prediction is compared to the expected output variables and the error is calculated. From this error, the update algorithm is used to improve the semantic character annotation model.
In an embodiment, the semantic role labeling model comprises a first fully-connected layer and a second fully-connected layer, each comprising a softmax function; the step of calculating a first probability when each word in the input sample is used as the start position of the role label corresponding to the input sample and a second probability when each word in the input sample is used as the end position of the role label corresponding to the input sample includes:
calculating a first probability when each word in the input sample is used as a starting position of the role label corresponding to the input sample through a softmax function of the first full connection layer;
and calculating each word in the input sample as a second probability when the end position of the role label corresponding to the input sample is the second probability through the softmax function of the second full connection layer.
In this embodiment, specifically, the semantic role model includes two full connection layers, which are a first full connection layer and a second full connection layer, each full connection layer includes a softmax function, and the first probability and the second probability can be calculated through softmax, softmax can map an arbitrary real vector of one K-dimension to a real vector of another K-dimension, where a value of each element in the vector is between (0, 1), and a function expression of softmax is:
and respectively taking the position of the word corresponding to the maximum probability value as the starting position and the ending position of the role label by calculating the first probability and the second probability of each word as the starting position and the ending position.
The semantic role marking method can be applied to the field of block chains, trained semantic role marking models are stored in a block chain network, and the block chains are novel application modes of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism and an encryption algorithm. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product services layer, and an application services layer.
The block chain underlying platform can comprise processing modules such as user management, basic service, intelligent contract and operation monitoring. The user management module is responsible for identity information management of all blockchain participants, and comprises public and private key generation maintenance (account management), key management, user real identity and blockchain address corresponding relation maintenance (authority management) and the like, and under the authorization condition, the user management module supervises and audits the transaction condition of certain real identities and provides rule configuration (wind control audit) of risk control; the basic service module is deployed on all block chain node equipment and used for verifying the validity of the service request, recording the service request to storage after consensus on the valid request is completed, for a new service request, the basic service firstly performs interface adaptation analysis and authentication processing (interface adaptation), then encrypts service information (consensus management) through a consensus algorithm, transmits the service information to a shared account (network communication) completely and consistently after encryption, and performs recording and storage; the intelligent contract module is responsible for registering and issuing contracts, triggering the contracts and executing the contracts, developers can define contract logics through a certain programming language, issue the contract logics to a block chain (contract registration), call keys or other event triggering and executing according to the logics of contract clauses, complete the contract logics and simultaneously provide the function of upgrading and canceling the contracts; the operation monitoring module is mainly responsible for deployment, configuration modification, contract setting, cloud adaptation in the product release process and visual output of real-time states in product operation, such as: alarm, monitoring network conditions, monitoring node equipment health status, and the like.
Referring to fig. 2, an embodiment of the present application further provides a semantic role labeling apparatus, including:
the acquiring unit 10 is configured to acquire a sentence to be annotated, and perform word segmentation processing and part-of-speech annotation on the sentence to be annotated;
a first determining unit 20, configured to determine a verb in the to-be-annotated sentence according to the result of the part-of-speech annotation;
a second determining unit 30, configured to obtain a role label preset by each verb, and construct an input sample corresponding to each role label according to the role label and the result of the participle processing;
an input unit 40, configured to input each input sample to a semantic character labeling model; wherein the semantic role labeling model is obtained by training based on a BERT, BilSTM, RoBERT or XLNE model;
a calculating unit 50, configured to calculate, in the semantic character labeling model, a first probability when each word in the input sample is used as a start position of the character tag corresponding to the input sample and a second probability when each word in the input sample is used as an end position of the character tag corresponding to the input sample, and perform character labeling according to the first probability and the second probability.
In one embodiment, the computing unit 50 includes:
the first calculating subunit is configured to calculate a third probability that a field corresponding to the role label exists in each input sample and a fourth probability that the field corresponding to the role label does not exist;
a first comparing subunit, configured to compare the third probability with the fourth probability;
a non-entry subunit, configured to, if the fourth probability is greater than the third probability, not perform a step of calculating a first probability when each word in the input sample is used as the start position of the character label corresponding to the input sample and a second probability when each word in the input sample is used as the end position of the character label corresponding to the input sample;
and an entering subunit, configured to enter a step of calculating a first probability when each word in the input sample is used as the start position of the character label corresponding to the input sample and a second probability when each word in the input sample is used as the end position of the character label corresponding to the input sample, if the fourth probability is less than or equal to the third probability.
In one embodiment, the computing unit 50 includes:
a determining subunit, configured to determine, as a start position of the role label, a position of a word corresponding to a maximum first probability in each input sample, and determine, as an end position of the role label, a position of a word corresponding to a maximum second probability in each input sample;
and the first extraction subunit is used for extracting a field between the starting position and the ending position, and labeling the role label at the field.
In one embodiment, the computing unit 50 includes:
a second comparing subunit, configured to compare the maximum first probability in the input samples with a preset first probability, and if the maximum first probability is greater than the preset first probability, use a position of a word corresponding to the maximum first probability as a starting position of the character tag;
the second comparing subunit is configured to compare the maximum second probability in the input sample with a preset second probability, and if the maximum second probability is greater than the preset second probability, use a position of a word corresponding to the maximum second probability as an end position of the role label;
and the second extraction subunit is used for extracting a field between the starting position and the ending position, and labeling the role label at the field.
In one embodiment, the computing unit 50 includes:
and the second calculating subunit is used for calculating a first probability when each word is used as the starting position of the role label corresponding to the input sample and a second probability when each word in the input sample is used as the ending position of the role label corresponding to the input sample in a way that all input samples corresponding to each verb are subjected to batch calculation.
In an embodiment, the semantic role labeling model comprises a first fully-connected layer and a second fully-connected layer, each comprising a softmax function; the calculation unit 50 includes:
a third calculating subunit, configured to calculate, through the softmax function of the first full connection layer, a first probability that each word in the input sample is used as a start position of the role label corresponding to the input sample;
a fourth calculating subunit, configured to calculate, through the softmax function of the second fully-connected layer, a second probability that each word in the input sample is used as the end position of the role label corresponding to the input sample.
In this embodiment, please refer to the above method embodiment for the specific implementation of each unit and sub-unit, which is not described herein again.
Referring to fig. 3, a computer device, which may be a server and whose internal structure may be as shown in fig. 3, is also provided in the embodiment of the present application. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the computer designed processor is used to provide computational and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer equipment is used for storing sentences to be annotated, role labels and the like. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a semantic role tagging method.
Those skilled in the art will appreciate that the architecture shown in fig. 3 is only a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects may be applied.
An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements a semantic role labeling method.
In summary, for the semantic role labeling method, apparatus, computer device and storage medium provided in this embodiment of the present application, the method includes: obtaining a sentence to be labeled, and performing word segmentation processing and part-of-speech labeling on the sentence to be labeled; determining verbs in the sentences to be labeled according to the result of the part-of-speech labeling; acquiring role labels preset by each verb, and constructing input samples corresponding to the role labels according to the role labels and the word segmentation processing result; inputting each input sample into a semantic role labeling model; wherein the semantic role labeling model is obtained by training based on a BERT, BilSTM, RoBERT or XLNE model; in the semantic role labeling model, calculating a first probability when each word in the input sample is used as the starting position of the role label corresponding to the input sample and a second probability when each word in the input sample is used as the ending position of the role label corresponding to the input sample, and labeling the role according to the first probability and the second probability. Through the semantic role labeling method, the semantic role labeling device, the computer equipment and the storage medium, all characters do not need to be labeled, only the initial position and the end position need to be labeled, the number of paths needing to be decided by a semantic role labeling model is reduced, the search space of semantic role labeling is reduced, only one role label is predicted at a time, the performance of semantic role labeling is improved and the accuracy of semantic role labeling is improved by reducing the search space of decision at a time.
It will be understood by those skilled in the art that all or part of the processes of the methods of the above embodiments may be implemented by hardware associated with instructions of a computer program, which may be stored on a non-volatile computer-readable storage medium, and when executed, may include processes of the above embodiments of the methods. Any reference to memory, storage, database, or other medium provided herein and used in the examples may include non-volatile and/or volatile memory. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), double-rate SDRAM (SSRSDRAM), Enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, apparatus, article, or method that includes the element.
The above description is only for the preferred embodiment of the present application and not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the specification and the drawings of the present application, or which are directly or indirectly applied to other related technical fields, are intended to be included within the scope of the present application.
Claims (10)
1. A semantic role labeling method is characterized by comprising the following steps:
obtaining a sentence to be labeled, and performing word segmentation processing and part-of-speech labeling on the sentence to be labeled;
determining verbs in the sentences to be labeled according to the result of the part-of-speech labeling;
acquiring role labels preset by each verb, and constructing input samples corresponding to the role labels according to the role labels and the word segmentation processing result;
inputting each input sample into a semantic role labeling model; wherein the semantic role labeling model is obtained by training based on a BERT, BilSTM, RoBERT or XLNE model;
in the semantic role labeling model, calculating a first probability when each word in the input sample is used as the starting position of the role label corresponding to the input sample and a second probability when each word in the input sample is used as the ending position of the role label corresponding to the input sample, and labeling the role according to the first probability and the second probability.
2. The semantic character labeling method according to claim 1, wherein the step of calculating a first probability of each word in the input sample as a starting position of the character label corresponding to the input sample and a second probability of each word in the input sample as an ending position of the character label corresponding to the input sample comprises:
calculating a third probability that a field corresponding to the role label exists in each input sample and a fourth probability that the field corresponding to the role label does not exist;
comparing the third probability to the fourth probability;
if the fourth probability is greater than the third probability, not performing a step of calculating a first probability when each word in the input sample is used as the start position of the role label corresponding to the input sample and a second probability when each word in the input sample is used as the end position of the role label corresponding to the input sample;
if the fourth probability is less than or equal to the third probability, a step of calculating a first probability when each word in the input sample is used as the start position of the role label corresponding to the input sample and a second probability when each word in the input sample is used as the end position of the role label corresponding to the input sample is performed.
3. The semantic role labeling method according to claim 1, wherein the step of performing role labeling according to the first probability and the second probability comprises:
determining the position of the word corresponding to the maximum first probability in each input sample as the starting position of the role label, and determining the position of the word corresponding to the maximum second probability in each input sample as the ending position of the role label;
and extracting a field between the starting position and the ending position, and marking the role label at the field.
4. The semantic role labeling method according to claim 1, wherein the step of performing role labeling according to the first probability and the second probability comprises:
comparing the maximum first probability in the input samples with a preset first probability, and if the maximum first probability is greater than the preset first probability, taking the position of the word corresponding to the maximum first probability as the initial position of the character label;
comparing the maximum second probability in the input sample with a preset second probability, and if the maximum second probability is greater than the preset second probability, taking the position of the word corresponding to the maximum second probability as the end position of the role label;
and extracting a field between the starting position and the ending position, and marking the role label at the field.
5. The semantic character labeling method according to claim 1, wherein the step of calculating a first probability when each word in the input sample is used as the start position of the character label corresponding to the input sample and a second probability when each word in the input sample is used as the end position of the character label corresponding to the input sample comprises:
calculating a first probability when each word is used as the starting position of the role label corresponding to the input sample and a second probability when each word in the input sample is used as the ending position of the role label corresponding to the input sample by using all input samples corresponding to each verb through a batch calculation mode.
6. The semantic role tagging method of claim 1, wherein the semantic role tagging model comprises a first fully connected layer and a second fully connected layer, the first fully connected layer and the second fully connected layer each comprising a softmax function; the step of calculating a first probability when each word in the input sample is used as the start position of the role label corresponding to the input sample and a second probability when each word in the input sample is used as the end position of the role label corresponding to the input sample includes:
calculating a first probability when each word in the input sample is used as a starting position of the role label corresponding to the input sample through a softmax function of the first full connection layer;
and calculating each word in the input sample as a second probability when the end position of the role label corresponding to the input sample is the second probability through the softmax function of the second full connection layer.
7. A semantic role labeling apparatus, comprising:
the system comprises an acquisition unit, a word segmentation unit and a word property tagging unit, wherein the acquisition unit is used for acquiring a sentence to be tagged and carrying out word segmentation processing and part-of-speech tagging on the sentence to be tagged;
the first determining unit is used for determining verbs in the sentences to be annotated according to the result of part-of-speech annotation;
a second determining unit, configured to obtain a role label preset by each verb, and construct an input sample corresponding to each role label according to the role label and a result of the participle processing;
the input unit is used for inputting each input sample to a semantic role labeling model; wherein the semantic role labeling model is obtained by training based on a BERT, BilSTM, RoBERT or XLNE model;
and the calculation unit is used for calculating a first probability when each word in the input sample is used as the starting position of the role label corresponding to the input sample and a second probability when each word in the input sample is used as the ending position of the role label corresponding to the input sample in the semantic role labeling model, and performing role labeling according to the first probability and the second probability.
8. The semantic character tagging apparatus of claim 7, the computing unit comprising:
the first calculating subunit is configured to calculate a third probability that a field corresponding to the role label exists in each input sample and a fourth probability that the field corresponding to the role label does not exist;
a comparison subunit configured to compare the third probability with the fourth probability;
a non-entry subunit, configured to, if the fourth probability is greater than the third probability, not perform a step of calculating a first probability when each word in the input sample is used as the start position of the character label corresponding to the input sample and a second probability when each word in the input sample is used as the end position of the character label corresponding to the input sample;
and an entering subunit, configured to enter a step of calculating a first probability when each word in the input sample is used as the start position of the character label corresponding to the input sample and a second probability when each word in the input sample is used as the end position of the character label corresponding to the input sample, if the fourth probability is less than or equal to the third probability.
9. A computer device comprising a memory and a processor, the memory having stored therein a computer program, characterized in that the processor, when executing the computer program, implements the steps of the semantic character tagging method of any one of claims 1 to 6.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the semantic role labeling method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110587690.5A CN113177108A (en) | 2021-05-27 | 2021-05-27 | Semantic role labeling method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110587690.5A CN113177108A (en) | 2021-05-27 | 2021-05-27 | Semantic role labeling method and device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113177108A true CN113177108A (en) | 2021-07-27 |
Family
ID=76927539
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110587690.5A Pending CN113177108A (en) | 2021-05-27 | 2021-05-27 | Semantic role labeling method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113177108A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080221878A1 (en) * | 2007-03-08 | 2008-09-11 | Nec Laboratories America, Inc. | Fast semantic extraction using a neural network architecture |
| CN101908042A (en) * | 2010-08-09 | 2010-12-08 | 中国科学院自动化研究所 | Tagging method of bilingual combination semantic role |
| CN104462066A (en) * | 2014-12-24 | 2015-03-25 | 北京百度网讯科技有限公司 | Method and device for labeling semantic role |
| CN107729309A (en) * | 2016-08-11 | 2018-02-23 | 中兴通讯股份有限公司 | A kind of method and device of the Chinese semantic analysis based on deep learning |
| CN111859968A (en) * | 2020-06-15 | 2020-10-30 | 深圳航天科创实业有限公司 | Text structuring method, text structuring device and terminal equipment |
| CN112269862A (en) * | 2020-10-14 | 2021-01-26 | 北京百度网讯科技有限公司 | Text role labeling method and device, electronic equipment and storage medium |
-
2021
- 2021-05-27 CN CN202110587690.5A patent/CN113177108A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080221878A1 (en) * | 2007-03-08 | 2008-09-11 | Nec Laboratories America, Inc. | Fast semantic extraction using a neural network architecture |
| CN101908042A (en) * | 2010-08-09 | 2010-12-08 | 中国科学院自动化研究所 | Tagging method of bilingual combination semantic role |
| CN104462066A (en) * | 2014-12-24 | 2015-03-25 | 北京百度网讯科技有限公司 | Method and device for labeling semantic role |
| CN107729309A (en) * | 2016-08-11 | 2018-02-23 | 中兴通讯股份有限公司 | A kind of method and device of the Chinese semantic analysis based on deep learning |
| CN111859968A (en) * | 2020-06-15 | 2020-10-30 | 深圳航天科创实业有限公司 | Text structuring method, text structuring device and terminal equipment |
| CN112269862A (en) * | 2020-10-14 | 2021-01-26 | 北京百度网讯科技有限公司 | Text role labeling method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021139283A1 (en) | Knowledge graph question-answer method and apparatus based on deep learning technology, and device | |
| CN112612894B (en) | Method and device for training intention recognition model, computer equipment and storage medium | |
| CN109992664B (en) | Dispute focus label classification method and device, computer equipment and storage medium | |
| CN112016279B (en) | Method, device, computer equipment and storage medium for structuring electronic medical record | |
| CN111651992A (en) | Named entity labeling method and device, computer equipment and storage medium | |
| CN112131888B (en) | Method, device, equipment and storage medium for analyzing semantic emotion | |
| CN112347310A (en) | Event processing information query method and device, computer equipment and storage medium | |
| CN109857846B (en) | Method and device for matching user question and knowledge point | |
| CN112329865B (en) | Data anomaly identification method and device based on self-encoder and computer equipment | |
| CN112836061A (en) | Intelligent recommendation method and device and computer equipment | |
| CN113204618A (en) | Information identification method, device and equipment based on semantic enhancement and storage medium | |
| CN111831908A (en) | Medical field knowledge graph construction method, device, equipment and storage medium | |
| CN110597956A (en) | Searching method, searching device and storage medium | |
| CN111091004A (en) | Training method and training device for sentence entity labeling model and electronic equipment | |
| CN114756667A (en) | Dialog generation method, device, equipment and storage medium based on artificial intelligence | |
| CN111901554A (en) | Call channel construction method and device based on semantic clustering and computer equipment | |
| CN112036172A (en) | Entity identification method and device based on abbreviated data of model and computer equipment | |
| CN113204619B (en) | Sentence completion method, device, equipment and medium based on natural language processing | |
| CN113343711B (en) | Work order generation method, device, equipment and storage medium | |
| CN113821587B (en) | Text relevance determining method, model training method, device and storage medium | |
| CN112036151A (en) | Method and device for constructing gene disease relation knowledge base and computer equipment | |
| CN116402630B (en) | Financial risk prediction method and system based on characterization learning | |
| CN113360644B (en) | Text model retraining method, device, equipment and storage medium | |
| CN113064997B (en) | Intention analysis method, device, equipment and medium based on BERT model | |
| CN113177108A (en) | Semantic role labeling method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |