CN116186241A - Event element extraction method and device based on semantic analysis and prompt learning, electronic equipment and storage medium - Google Patents
Event element extraction method and device based on semantic analysis and prompt learning, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN116186241A CN116186241A CN202211664406.0A CN202211664406A CN116186241A CN 116186241 A CN116186241 A CN 116186241A CN 202211664406 A CN202211664406 A CN 202211664406A CN 116186241 A CN116186241 A CN 116186241A
- Authority
- CN
- China
- Prior art keywords
- event
- event element
- semantic
- prompt
- text
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/34—Browsing; Visualisation therefor
- G06F16/345—Summarisation for human users
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3344—Query execution using natural language analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
- G06F16/355—Class or cluster creation or modification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- General Health & Medical Sciences (AREA)
- Machine Translation (AREA)
Abstract
The invention discloses a semantic-based method, a semantic-based device, a semantic-based equipment and a semantic-based storage medium for extracting prompt learning event elements, which are used for carrying out linguistic analysis on the event elements to form a traditional event element label-event element semantic label bidirectional conversion module; building an event element classification model, and training the event element classification model by using an event element data set based on event element semantic tags; inputting the event text into a prompt learning event element abstract model in combination with an event prompt template, and extracting abstract text containing the event elements; and then decoding the abstract text by combining with the event element classification model bidirectional conversion module to finish the extraction of the event elements. According to the method, potential knowledge is mined from the pre-training language model by utilizing prompt learning through combining the linguistic characteristics of the event elements, so that the accuracy of event element extraction under the condition of few samples is improved.
Description
Technical Field
The invention relates to the field of natural language processing in the technical field of information, in particular to the technical fields of deep learning, information extraction and the like, and particularly provides an event element extraction method, device, electronic equipment and storage medium based on semantic analysis and prompt learning.
Background
With the rapid development of information technology, the internet is generating massive text data every day, and most of the text data are unstructured text data which are difficult to directly utilize. As an important form of information presentation, the fact that an event describes the interaction of some people or people at a certain time and place can help us to learn the world. The task of event element extraction is to extract key information such as time, place, person, influence, reason and the like from unstructured text containing event information, so that computer processing is facilitated, and meanwhile, efficient information acquisition by a user is facilitated. Event element extraction is the basis for automatic construction of a matter map, is widely applied to the fields of finance, intelligence, news and the like, and has wide research and application prospects.
However, the event element extraction also faces great challenges, the first challenge is that data acquisition is difficult, the existing public data sets are fewer, and the event element extraction has the characteristics of high labeling difficulty, high professional knowledge requirement, high manual labeling cost and the like, so that the data acquisition difficulty is further improved. The second challenge is that the domain migration is difficult, different domains pay attention to different event types, event elements of different event types are different, for example, a visit event in the politics domain can comprise 5 event elements of time, place, person, place and visit destination, a bankruptcy event in the finance domain pay attention to 3 event elements of time, company and bankruptcy reason, the two event elements are not completely corresponding, and the corresponding syntax dependency structures are also greatly different, so that the algorithm is more difficult to migrate. Therefore, how to use a small amount of labeling samples to obtain an event element extraction model with good field mobility is a key problem to be solved in the current urgent need.
In recent years, prompt learning becomes a new model of natural language processing, and only a small number of labeling samples are needed to obtain excellent effects on tasks such as text classification, knowledge mining, machine translation, information extraction and the like. The prompt learning is to add a prompt text into an input text to enable the model to generate a corresponding answer text according to a complete filling form, and then obtain a target result after decoding. In addition, the event element extraction task has obvious linguistic features, similar semantic features are arranged in the event elements, and specific syntactic dependency relation exists among the event elements, so that linguistic basis is provided for prompt learning. The prompt learning is combined with linguistic characteristics of event element extraction, so that implicit knowledge of a pre-training language model can be fully mined, and the problems of difficult acquisition of event element extraction data and difficult field migration can be effectively solved.
Disclosure of Invention
The invention provides an event element extraction method and device based on linguistic analysis and prompt learning, electronic equipment and a storage medium. Carrying out linguistic analysis on the event elements to form a bidirectional conversion module of a traditional event element label-event element semantic label; building an event element classification model, and training the event element classification model by using an event element data set based on event element semantic tags; inputting the event text into a prompt learning event element abstract model in combination with an event prompt template, and extracting abstract text containing the event elements; and then decoding the abstract text by combining with the event element classification model bidirectional conversion module to finish the extraction of the event elements. According to the method, potential knowledge is mined from the pre-training language model by utilizing prompt learning through combining the linguistic characteristics of the event elements, so that the accuracy of event element extraction under the condition of few samples is improved.
The invention discloses an event element extraction method based on linguistic analysis and prompt learning, which comprises the following steps:
a. forming a bi-directional label conversion of a traditional event element label-event element semantic label based on linguistic analysis;
b. constructing an event element semantic data set: firstly, acquiring an event element extraction data set based on a traditional event element label, and re-labeling the data set by using a label conversion method in the step a so as to form an event element semantic data set;
c. constructing an event element semantic classification model: the model comprises an event text coding model and an event element classification model, so that semantic classification of the event elements is realized;
d. constructing an event prompt template according to a natural language expression method aiming at each event type;
e. constructing a prompt learning event element abstract model: d, taking the event prompt template and the event text generated in the step d as a model input, and outputting a natural language sequence with the event elements connected in series;
f. event element summary text decoding: firstly, word segmentation and prefiltering are carried out on the event element series text output in the step e to form a group of event elements, and semantic classification is carried out on the group of event elements by using the event element semantic classification model in the step c;
g. And d, converting the event element semantic classification result in the step f by using the bidirectional label conversion method of the traditional event element label-event element semantic label in the step a, thereby realizing the event element extraction based on the original event labeling rule.
The bidirectional label conversion from the traditional event element label to the event element semantic label in the step a is specifically as follows: the linguistic analysis means is event element semantic analysis and event element dependency relationship analysis, event element types of all event types are classified to form reasons, event element semantic tags related to time, place, event subjects, event objects and results weakly with the event types are obtained, and the bidirectional mapping relationship between the original traditional event element labeling tags and the new event element semantic tags is the bidirectional tag conversion of the traditional event element tags and the event element semantic tags.
The event element semantic data set in the step b is a data set marked by adopting an event element semantic tag rule, the event element semantic tag is the event element semantic tag in the claim 2, the tag of the event element extraction data set based on the traditional event element tag is remarked, and the remark is to map the original traditional event element extraction tag by using bidirectional tag conversion of the traditional event element tag and the event element semantic tag.
The event element semantic classification model in the step c is formed by serially connecting an event text coding model and an event element classification model and is trained together, the event text coding model is a pre-trained language model for removing a classification layer, the event element classification model comprises an event element feature conversion module and an event element classification module, the event element feature conversion module firstly carries out space conversion of event element features, and then an event element classification result is output through the event element classification module.
The event prompt template in the step d refers to a complete blank filling pre-training task of a self-pre-training language model, is composed of prompt texts and blank prompts based on a natural language expression mode and an event element dependency structure, the number of the blank prompts is consistent with the type number of the event elements, and each blank prompt corresponds to one event element.
And e, the prompt learning event element abstract model is a pre-training language model, the event text is encoded and decoded by the pre-training language model, the event prompt template can be input in series with the event text or input from a pre-training language model decoder, and the model output is continuous natural event element abstract text which is formed by serially connecting the event elements and other texts in sequence.
F, decoding the abstract text of the event element, wherein the pre-filtering comprises stopping word filtering, part-of-speech filtering and triggering word filtering; the semantic classification method is that after the abstract text of the event element is preprocessed according to the requirement of the semantic classification model of the event element, the abstract text of the event element is sent into the semantic classification model of the event element for classification prediction, and the semantic label of the predicted event element is the semantic classification result of the event element.
An event element extraction device based on linguistic analysis and prompt learning is composed of an event prompt template module, a prompt learning event element abstract module and an event element decoding module, wherein
Event prompt template module: the method comprises the steps of providing a corresponding event prompt template for each event type, and providing the corresponding event prompt template for each event type based on a natural language expression mode and an event element dependency structure so as to prompt a pre-training language model to generate event elements.
Prompting a learning event element summary module: the method comprises the steps of abstracting event elements corresponding to event types from event texts, outputting natural language texts with the event elements connected in series, taking an event prompt template and the event texts output by an event element prompt template module as input, utilizing text generation capacity obtained by a pre-training language model in a pre-training stage, abstracting the event elements corresponding to the event types from the event texts, and outputting the natural language texts with the event elements connected in series.
An event element decoding module: the event element extraction method comprises the steps that an event element is decoded from an event element series text output by a prompt learning event element abstract module, the event element series text comprises an event abstract word segmentation unit, an event element classification unit and an event element conversion unit, the event element series text output by the prompt learning event element abstract module is segmented and pre-filtered by the event abstract word segmentation unit, then the event element classification unit is used for classifying event elements output by the event abstract word segmentation unit based on event semantic tags, and finally the event element conversion unit is used for converting the event semantic tags of the event elements into traditional event element tags, so that the event element extraction based on the traditional event element tags is realized;
an electronic device comprising at least one processor; at least one GPU computing card; and a memory communicatively coupled to the processor; wherein: the memory stores instructions for execution by at least one processor or by at least one GPU computing card to enable the at least one processor or the at least one GPU computing card to perform the method of claims 1-7.
A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method recited in claims 1-7.
A processor;
at least one GPU computing card; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor or by the at least one GPU computing card to enable the at least one processor or the at least one GPU computing card to perform the method of any of the examples of this application.
According to the technology, the event element extraction task under the condition of few samples can be completed, and the accuracy of event element extraction under the condition of few samples is improved.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description and drawings. The technical scheme of the invention is further described in detail through the drawings and the embodiments.
Drawings
FIG. 1 is a flow chart of an event element extraction method based on linguistic analysis and prompt learning provided by the invention;
FIG. 2 is a flow chart of a bi-directional label conversion method for forming a conventional event element label-event element semantic label based on linguistic analysis;
FIG. 3 is a flow chart of a method for semantic data sets of event elements provided by the present invention;
FIG. 4 is a flow chart of a method for semantic classification of event elements according to the present invention;
FIG. 5 is a block diagram of an event element semantic classification model provided by the invention;
FIG. 6 is a predictive flow chart of an event element semantic classification model provided by the invention;
FIG. 7 is a block diagram of a method for prompting element abstracts of learning events provided by the invention;
FIG. 8 is a block diagram of a summary model for prompting learning event elements provided by the invention;
FIG. 9 is a block diagram of a summary model for prompting learning event elements provided by the invention;
FIG. 10 is a flow chart of summary text decoding for event elements provided by the present invention;
FIG. 11 is a block diagram of a learning event element extraction device according to the present invention;
fig. 12 is a block diagram of an electronic device of the present invention.
Detailed Description
In order to more clearly describe the technical scheme of the embodiment of the invention, the invention is further described in detail below with reference to the accompanying drawings. Various details of the embodiments of the present application are included to facilitate understanding, and they should be considered merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Examples
The invention discloses an event element extraction method based on linguistic analysis and prompt learning, which comprises the following steps:
a. the two-way label conversion of the traditional event element label-event element semantic label is formed based on linguistic analysis, and specifically comprises the following steps: the linguistic analysis means is event element semantic analysis and element dependency relation analysis, event element types of all event types are classified to form event element semantic tags which are weakly related to the event types, such as time, place, main subject, event object, result and the like, and the mapping relation between the original traditional event element labeling tags and the new event element semantic tags is the conversion of the traditional event element tags and the event element semantic tags;
b. Constructing an event element semantic data set, firstly acquiring an event element extraction data set based on a traditional event element label, and re-labeling the data set by using a label conversion method in the step a so as to form the event element semantic data set; the event element semantic data set is a data set marked by adopting an event element semantic tag rule, the event element semantic tag is the event element semantic tag in claim 2, the tag of the event element extraction data set based on the traditional event element tag is remarked, namely the original traditional event element extraction tag is mapped by using the conversion of the traditional event element tag and the event element semantic tag;
c. constructing an event element semantic classification model, wherein the model comprises an event text coding model and an event element classification model, and realizing semantic classification of the event elements; the event element semantic classification model is a pre-training language model (such as ERNIE, T5 and BART) for removing classification layers, the event element semantic classification model comprises an event element feature conversion module and an event element classification module, the event element feature conversion module performs space conversion of event element features, and then outputs an event element classification result, the event element feature conversion can effectively reserve implicit knowledge of the pre-training language model, and the implicit knowledge is helpful for improving the effect of few samples extracted by the event elements;
d. Constructing an event prompt template according to a natural language expression method aiming at each event type; the event prompt template refers to a complete gap filling pre-training task (MLM) of a self-pre-training language model, is composed of prompt texts and blank prompts ([ MASK ]) based on a natural language expression mode and an event element dependency structure, and can be designed as time [ MASK ], an assisted person [ MASK ] assists an assisted person [ MASK ] supplies [ MASK ], wherein [ MASK ] is a blank prompt mark (different pre-training language models can be different), and the pre-training language model can generate corresponding texts at the [ MASK ];
e. d, constructing a prompt learning event element abstract model, taking the event prompt template and the event text generated in the step d as model input, and outputting a natural language sequence of the event element series connection; the prompt learning event element abstract model is a pre-training language model (such as ERNIE, T5 and BART), event texts are encoded and decoded by the pre-training language model, an event prompt template can be input in series with the event texts or input from a pre-training language model decoder, the model output is continuous natural event element abstract texts which are serially connected by the event elements and other texts according to a certain sequence, the event element sequence corresponding to blank prompts in the event prompt template is output, and if certain event elements are absent from the event texts, the event element abstract texts also do not contain the event element type texts. The other texts comprise trigger words and necessary punctuation marks which enable the event abstract text to keep continuous and natural, and preposition texts, the pre-training language model is usually pre-trained by adopting a complete blank filling pre-training task (MLM), the complete blank filling pre-training task (MLM) is a main stream pre-training task of the pre-training language model, partial texts are replaced by [ MASK ] in input texts, the model has the capability of predicting the texts at the [ MASK ] through training, namely the complete blank filling capability, the event text is completely filled by an event prompt template, and blank prompts [ MASK ] are completely filled by the pre-training language model, so that the complete blank filling capability of the pre-training language model can be exerted, and the effect of few samples extracted by event elements is improved;
f. E, decoding an event element abstract text, firstly, performing word segmentation and prefiltering on the event element series text output in the step E to form a group of event elements, and performing semantic classification on the group of event elements by using an event element semantic classification model; the event element abstract text is decoded, wherein the pre-filtering comprises stop words, part-of-speech filtering and trigger word filtering; the event element semantic classification model is the event element semantic classification model, and the semantic classification method is that after an event text or an event element abstract text is preprocessed according to the requirement of the event element semantic classification model, the event element semantic classification model is sent into the model for classification prediction, and a predicted event element semantic label is the semantic classification result of the event element;
g. and d, converting the event element semantic classification result in the step f by using the bidirectional label conversion method of the traditional event element label-event element semantic label in the step a, thereby realizing the event element extraction based on the original event labeling rule.
An event element extraction device based on linguistic analysis and prompt learning is composed of an event prompt template module, a prompt learning event element abstract module and an event element decoding module, wherein
Event prompt template module: the method comprises the steps of providing a corresponding event prompt template for each event type to prompt a pre-training language model to generate event elements, providing a corresponding event prompt template for each event type based on a natural language expression mode and an event element dependency structure to prompt the pre-training language model to generate event elements, so as to better mine hidden knowledge in the pre-training language model, wherein the hidden knowledge in the pre-training language model is non-obvious knowledge obtained after the pre-training language model uses a large amount of texts to perform pre-training, and comprises objective world knowledge, linguistic knowledge and other knowledge, thereby being beneficial to improving the effect of few samples extracted by the event elements;
prompting a learning event element summary module: the method comprises the steps of abstracting event elements corresponding to event types from event texts, outputting a natural language sequence of the event element series connection, taking an event prompt template and the event texts output by an event element prompt template module as input, utilizing text generation capacity of a pre-training language model obtained in a pre-training stage, abstracting the event elements corresponding to the event types from the event texts, and outputting the natural language sequence of the event element series connection, wherein the text generation capacity of the pre-training language model is beneficial to improving the effect of few samples extracted from the event elements;
An event element decoding module: the event element extraction method comprises the steps of decoding event elements from event element series texts output by a prompt learning event element summary module, wherein the event element series texts comprise an event summary word segmentation unit, an event element classification unit and an event element conversion unit;
an electronic device comprising at least one processor; at least one GPU computing card; and a memory communicatively coupled to the processor; wherein: the memory storing instructions for execution by at least one processor or by at least one GPU computing card to enable the at least one processor or the at least one GPU computing card to perform the method of claims 1-7;
fig. 1 is a flowchart of an event element extraction method based on linguistic analysis and prompt learning provided in an embodiment of the present application, where the embodiment is applicable to the event element extraction case with a small number of labeling samples, and the method may be performed by a prompt learning event element extraction device, and the device may be implemented by software and/or hardware, and referring to fig. 1, the event element extraction method provided in the embodiment of the present application includes:
A two-way label conversion method for forming a traditional event element label-event element semantic label based on linguistic analysis;
the specific method for constructing the bidirectional label switching method, referring to fig. 2, specifically includes the steps of:
acquiring an event element set containing all event types; for example, event elements corresponding to the assistance event and the marketing event are added into an event element set to form { assistance time, assistance person, assisted person, assistance article, marketing time, marketing company, marketing place;
semantic analysis and classification of event elements;
analyzing and classifying the event element dependency relationship;
b, forming a bidirectional label conversion method of a traditional event element label-event element semantic label, forming a bidirectional mapping conversion relation between the traditional event element label and the classified event element semantic label according to the classification result of the traditional event element label formed after the linguistic analysis step b and c, wherein a traditional event element label assistor is analyzed and classified into an event element semantic label applying subject, and the two labels form the bidirectional mapping conversion relation;
wherein, the traditional event element labels correspond to a group of event elements for each event type, and the event elements of different events are different;
For example, the event element sets corresponding to the assistance event and the marketing event are { assistance time, assistance person, assisted person, assistance article } and { marketing time, marketing company, marketing place }, respectively, and the assistance time and the marketing time are respectively represented by events but different labels;
wherein the linguistic analysis includes an event element semantic analysis and an event element dependency analysis;
the event element semantic analysis is to analyze the semantics of event element sets included in all event types and classify the event element types with similar semantics into one type;
illustratively, the assistance time element and the time element on the market have similar semantics, categorized as time elements;
the event element dependency relationship analysis is to analyze the dependency relationship among event elements of the same event type and classify the event element types with similar dependency relationship in different event types into one type;
illustratively, the helper element and the marketer element are typically subjects in the element, have similar subject dependency characteristics, and are categorized as subject of a subject.
The event element semantic tags are event element semantic tags generated by event element semantic analysis and event element dependency analysis and are weakly related to event types, and the event element semantic tags comprise reasons, time, places, event subjects and results;
Constructing an event element semantic data set, firstly acquiring an event element extraction data set, and re-labeling the data set according to a bidirectional label conversion method so as to form the event element semantic data set;
the event element extraction data set is a data set adopting a traditional event element extraction labeling method, and the labeling method specifically comprises the following steps: according to the event type and trigger word of the event text, marking event element texts corresponding to each event element type from the event text in turn aiming at a group of event element types corresponding to the event type; wherein the event element type is characterized by: a set of event elements for each event type is generally different, but includes event element types for time, place;
the specific method of the event element semanteme data set method, referring to fig. 3, specifically comprises the following steps:
the event element extraction data set is obtained, and in one embodiment, the disclosed event element extraction data set may be collected first, and then the subset of data may be selected as desired. In another embodiment, an event text data set containing the requirement event type can be collected first, and then the event text is marked with event elements manually;
The event element extraction data set is a data set adopting a traditional event element extraction labeling method, and the labeling method specifically comprises the following steps: according to the event type and trigger word of the event text, marking event element texts corresponding to each event element type from the event text in turn aiming at a group of event element types corresponding to the event type;
for example, for 8 months and 20 days of event text, the first emergency grain assistance in China is carried out by using a Businessland card, the event type is an assistance event, the trigger word is assistance, and the event element types corresponding to the assistance event comprise: time, an assisted person and an assisted article, wherein event elements marked in sequence correspond to 8 months and 20 days, china, spearmint and grains in an event text;
re-labeling the data set according to a bidirectional label conversion method; the re-labeling method is to adopt a bidirectional label conversion method, map the event element labels of the event element extraction dataset by adopting the traditional event element labels and replace the event element labels with event element semantic labels;
constructing an event element semantic classification model, wherein the model comprises an event text coding model and an event element classification model, and realizing semantic classification of the event elements;
In one embodiment, the specific method for constructing the event element semantic classification model, see fig. 4, specifically includes the steps of:
building an event element semantic classification model, wherein a model structure diagram is shown in fig. 5, and the event element classification model is randomly initialized by using parameters of a pre-training language model in the event text coding model;
preprocessing an event text of an event element semantic data set, namely firstly encoding the event text into a token sequence, then adding a special mark [ CLS ] at the beginning position of the token sequence, adding a special mark [ SEP ] at the end position, and if the length of the event text after the addition of the mark is greater than the maximum field length allowed by a pre-training language model, cutting the event text until the field length requirement of the pre-training language model is met;
the event element semantic classification model is trained and updated, firstly, a preprocessed event text is input into the event element semantic classification model, a loss function is used for calculating a predicted loss value between event element semantic class probability distribution and labels, loss is propagated in a reverse gradient mode through an optimizer algorithm, model parameters are updated, the loss function can adopt cross entropy, focal loss and the like, and the optimizer algorithm can adopt Adam and SGD;
The structure diagram of the event element semantic classification model is shown in fig. 5; the event element semantic classification model consists of an event text coding model and an event element classification model which are connected in series, and the gradient of the event element semantic classification model can be transmitted and obtained through common training; the event text coding model is a pre-training language model (such as ERNIE, T5 and BART) with a classification layer removed; the event element classification model comprises an event element feature conversion module and an event element classification module;
the prediction process of the event element semantic classification model is shown in fig. 6, firstly, preprocessing an original event text, inputting the original event text into an event text coding model, sequentially selecting field features corresponding to event elements after coding, then converting the event element features into another feature space by an event element feature conversion module, and finally predicting event element semantic categories of the event element features after conversion by using an event element classification network; the event element feature conversion module converts the feature space of the event text coding model, and then the event element classification network classifies the feature space of the event text coding model and the feature space of the event element classification model; the event text coding model is a pre-training language model, and the pre-training language model is pre-trained on a large number of texts, so that hidden knowledge such as potential objective world knowledge, grammar knowledge and the like is provided, the improvement of the event element semantic classification effect is facilitated, and the event element semantic classification effect with less sample is improved; therefore, the event element feature conversion module realizes the feature space separation, further reserves the implicit knowledge of an event text coding model, namely a pre-training language model, and is beneficial to improving the accuracy and the effect of few samples of the event element semantic classification model;
In one embodiment, the event element feature conversion module consists of a global pooling layer, a plurality of full connection layers, a batch standardization layer and an activation function layer; the global pooling layer is used for converting the field dimension of the field characteristics of the event elements into 1; the event element classification module is a full-connection layer, the input feature dimension of the full-connection layer is equal to the feature dimension of the event element features after the event element feature conversion module, the output feature dimension is the number Z of tags of event element semantic tags, and the probability distribution on the predicted event element semantic tags is represented;
for example, the text of an event is 8 months and 20 days, the text is preprocessed to be [ CLS ]8 months and 20 days, the text of the first emergency grain assistance is conveyed to [ SEP ], field features corresponding to the Chinese of the event element of the main subject of the event are selected after the text of the event is encoded into a model, the dimensions are 2 x K, K are feature encoding dimensions in the text of the event, the feature dimensions are changed into K after the global pooling layer, and Z-dimensional vectors are output after the event element classification model. After the event element semantic classification model is trained, the prediction probability of the semantic tags of the subjects on the central element is larger than that of other semantic tags (such as time and place);
Constructing an event prompt template according to a natural language expression method aiming at each event type; the prompt template refers to a complete blank filling pre-training task (MLM) of a self-pre-training language model, and consists of prompt texts and blank prompts (MASK) based on a natural language expression mode and an event element dependency structure; for example, the alert template for an assistance event may be designed as time [ MASK ], the assistance person [ MASK ] assists the assisted person [ MASK ] material [ MASK ], where [ MASK ] is a blank alert mark (different pre-training language models may be different), at which the pre-training language model may generate corresponding text;
constructing a prompt learning event element abstract model, taking the generated event prompt template and event text as model input, and outputting a natural language sequence of the event element series connection;
in one embodiment, the specific method for constructing the summary of the prompt learning event element, referring to fig. 7, specifically includes the steps of:
constructing a prompt learning event element abstract data set, combining event elements in the event element extraction data set with other texts according to a certain rule to form continuous texts as tags of event texts, namely tag event element abstract texts; the rules comprise rules of continuous nature of texts and consistent sequence of prompt templates; the text is continuous and natural, namely the text is consistent along grammar, the sequence of the prompt templates is consistent, namely the arrangement sequence of the event elements is consistent with the types of the event elements corresponding to blank prompt marks in the prompt templates; if the input event text lacks certain event elements, the tag event element abstract text does not contain the event element type text; the other texts comprise trigger words and necessary texts such as punctuation marks, prepositions and the like which keep the text of the event abstract continuous and natural;
For example, for 8 months and 20 days of event text, the first emergency grain assistance in China carries out against Styland card, the event type is assistance, event elements comprise {8 months and 20 days, china, grain and Styland }, if a prompt template is designed as time [ MASK ], an assisted person [ MASK ] assists an assisted person [ MASK ] and supplies [ MASK ], and according to the rule of consistent template sequence, labels {8 months and 20 days, china, styland and grain } are consistent with the prompt template, and blank prompt marks in the prompt template can be sequentially filled; according to the principle of continuous nature of texts, the Chinese Spearmint grains are more natural than the Chinese Spearmint grains for 8 months and 20 days, and are suitable for being used as the tags;
constructing a prompt learning event element abstract model; the prompt learning event element abstract model is a pre-training language model (such as ERNIE, T5 and BART), in one embodiment, the structure of the prompt learning event element abstract model is shown in fig. 8, the event trigger words, the event prompt templates and the event text are connected in series, the series is not limited to the sequence shown in fig. 8, and the event element abstract text is output after being input into the pre-training language model for encoding and decoding; in another embodiment, the event text and the prompt template are separately input, the structure is shown in fig. 9, the event text and the event trigger words are input together into the encoder of the pre-training language model, the sequence of the event trigger words and the event text can be reversed, the event trigger words can be removed, the event prompt template is input from the decoder of the pre-training language model, the event trigger words can be added in series to the front or the back of the event prompt template, and finally the abstract text of the event element is output;
Prompting training and updating of element abstract models of learning events; according to the input requirement of the model, firstly, carrying out text preprocessing on an event text and an event prompt template, then inputting a prompt learning event element abstract model, calculating a loss value between the abstract text predicted by the model and a generated tag abstract text by using a loss function, and finally carrying out inverse gradient propagation on the loss by using an optimizer algorithm and updating model parameters; the loss function can adopt text abstract loss functions and the like, and the optimizer algorithm can adopt Adam and SGD;
the pre-training language model is usually pre-trained by adopting a complete gap filling pre-training task (MLM), wherein the complete gap filling pre-training task (MLM) is a main stream pre-training task of the pre-training language model, and the model has the capability of predicting texts at the position of [ MASK ] through training by replacing part of texts in input texts as [ MASK ]; the event prompt template generated in the step D of claim 5 is added to the event text, and blank prompt [ MASK ] is completely filled by the pre-training language model, so that the complete filling capacity of the pre-training language model can be exerted, and the effect of few samples extracted by the event elements is improved;
Optionally, before training and updating the model, pre-training the event element abstract model, wherein the pre-training adopts a text abstract task, the input of the text abstract task is a piece of text (not limited to an event text), and the text abstract task is output as an abstract of the piece of text; although the input text of the text summarization task is not limited to an event text, the effect of the event element summarization model can be improved because of certain similarity between the text summarization and the event summarization. The text summarization task can adopt a published text summarization data set or a self-collected text summarization data set;
c, word segmentation and pre-filtering are carried out on the event element abstract text output in the step e to form a group of event elements, and the event text or the event element abstract text is input into an event element semantic classification model to realize semantic classification of the group of event elements;
in one embodiment, the specific method for constructing the semantic classification of the event element abstract, see fig. 10, specifically includes the steps of:
e, using a word segmentation tool to segment the event element abstract text generated in the step E into a plurality of phrases; the word segmentation tool can use an open source word segmentation tool such as jieba word segmentation;
Pre-filtering word groups after word segmentation; the pre-filtering includes stop words (e.g., in), part-of-speech filtering (e.g., verbs, prepositions, adjectives, etc.), trigger word filtering (e.g., trigger word assistance for assistance events, support, subsidized, etc.);
predicting the set of event element semantic tags using an input event element semantic classification model; the semantic classification method of the event element comprises the steps of preprocessing an event text or an event element abstract text according to a model requirement, and then sending the event element abstract text into the model for classification prediction, wherein a predicted event element semantic label is a semantic classification result of the event element;
converting the event element semantic classification result by using a bidirectional label conversion method of a traditional event element label-event element semantic label, thereby realizing event element extraction based on an original event labeling rule;
FIG. 11 is a schematic diagram showing the structure of a linguistic-based learning-prompting event element extraction device, which consists of an event prompting template module, a learning-prompting event element summary module and an event element decoding module, wherein
Event prompt template module: providing a corresponding event prompt template for each event type to prompt the pre-training language model to generate event elements;
Prompting a learning event element summary module: the method comprises the steps of abstracting event elements corresponding to event types from event texts, and outputting natural language sequences of the event elements in series;
an event element decoding module: the method is used for word segmentation and segmentation of the event element series text, semantic classification of the event element and traditional event element label conversion;
further, the event prompt template module includes:
event prompt template unit: providing a corresponding event prompt template for each event type based on the natural language expression mode and the event element dependency structure so as to prompt the pre-training language model to generate event elements;
further, the prompt learning event element summary module includes:
prompting a learning event element abstract unit: the method comprises the steps of taking an event prompt template and an event text output by an event element prompt template module as input, abstracting event elements corresponding to event types from event text by utilizing text generating capability obtained by a pre-training language model in a pre-training stage, and outputting a natural language sequence of the event elements in series;
further, the event element decoding module includes:
event element decoding means: the event element extraction module is used for carrying out word segmentation on the event element serial text output by the prompt learning event element abstract module, then carrying out event semantic label-based classification on the event element by combining with the event element classification module, and finally converting the event semantic label into a traditional event element label by combining with the event element conversion module, thereby realizing the event element extraction based on the traditional event element label;
An event element classification unit: the method comprises the steps of carrying out event element semantic classification on 1 or more event elements contained in an event text, using the event text as input, classifying the 1 or more event elements contained in the event text, wherein labels generated by classification are labels based on event element semantics, and training a model by using a small amount of event element semantic data sets;
the method comprises the steps that an event element semantic data set is obtained, an event element extraction data set based on a traditional event element label is firstly obtained, and then an event element conversion unit is used for mapping and replacing the event element label of the event element extraction data set by the traditional event element label;
event element conversion means: the event element semantic label processing method comprises the steps of realizing bidirectional label conversion of a traditional event element label-event element semantic label, converting the event element semantic label into the traditional event element label, and re-labeling a data set based on the traditional event element label to convert the data set into an event element semantic data set based on the event element semantic label, wherein the data set can be used for training an event element classification model;
the invention provides an electronic device and a readable storage medium;
As shown in fig. 12, which is a block diagram of the electronic device of the present invention, the electronic device refers to a wide variety of modern electronic digital computers, including, for example: personal computers, portable computers, various server devices; the components shown herein and their interconnection and function are by way of example only;
as shown in fig. 12, the electronic device includes: one or more multi-core processors, one or more GPU computing cards, memory, for causing interactions to occur with an electronic device, further comprising: input equipment and output equipment. The devices are interconnected and communicated through buses;
a memory is a non-transitory computer readable storage medium provided herein, where the memory stores instructions executable by the at least one processor or by the at least one GPU computing card to enable the at least one processor or the at least one GPU computing card to perform the method of any of the embodiments herein;
an input device for providing and receiving control signals input into the electronic device by a user, including a keyboard for generating numeric or character information and a mouse for controlling the device to generate other key signals. The output device provides feedback information from the consumer electronic device including a display of the print execution results or processes.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is to be understood that the invention is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof.
Claims (10)
1. The event element extraction method based on linguistic analysis and prompt learning is characterized by comprising the following steps:
a. forming a bi-directional label conversion of a traditional event element label-event element semantic label based on linguistic analysis;
b. constructing an event element semantic data set: firstly, acquiring an event element extraction data set based on a traditional event element label, and re-labeling the data set by using a label conversion method in the step a so as to form an event element semantic data set;
c. constructing an event element semantic classification model: the model comprises an event text coding model and an event element classification model, so that semantic classification of the event elements is realized;
d. Constructing an event prompt template according to a natural language expression method aiming at each event type;
e. constructing a prompt learning event element abstract model: d, taking the event prompt template and the event text generated in the step d as a model input, and outputting a natural language sequence with the event elements connected in series;
f. event element summary text decoding: firstly, word segmentation and prefiltering are carried out on the event element series text output in the step e to form a group of event elements, and semantic classification is carried out on the group of event elements by using the event element semantic classification model in the step c;
g. and d, converting the event element semantic classification result in the step f by using the bidirectional label conversion method of the traditional event element label-event element semantic label in the step a, thereby realizing the event element extraction based on the original event labeling rule.
2. The method for extracting event elements based on linguistic analysis and prompt learning according to claim 1, wherein the bi-directional label conversion of the conventional event element label-event element semantic label in the step a is specifically as follows: the linguistic analysis means is event element semantic analysis and event element dependency relationship analysis, event element types of all event types are classified to form reasons, event element semantic tags related to time, place, event subjects, event objects and results weakly with the event types are obtained, and the bidirectional mapping relationship between the original traditional event element labeling tags and the new event element semantic tags is the bidirectional tag conversion of the traditional event element tags and the event element semantic tags.
3. The method for extracting event elements based on linguistic analysis and prompt learning according to claim 1, wherein the event element semantic data set in the step b is a data set marked by using an event element semantic tag rule, the event element semantic tag is an event element semantic tag according to claim 2, the tag of the event element extraction data set based on a traditional event element tag is remarked, and the remarking is to map an original traditional event element extraction tag by using bidirectional tag conversion of the traditional event element tag and the event element semantic tag.
4. The method for extracting event elements based on linguistic analysis and prompt learning according to claim 1, wherein the event element semantic classification model in the step c is formed by serially connecting an event text coding model and an event element classification model and is trained together, the event text coding model is a pre-trained language model with a classification layer removed, the event element classification model comprises an event element feature conversion module and an event element classification module, the event element feature conversion module firstly carries out space conversion of event element features, and then the event element classification module outputs event element classification results.
5. The method for extracting event elements based on linguistic analysis and prompt learning according to claim 1, wherein the event prompt template in the step d refers to a complete blank filling pre-training task of a self-pre-training language model, and based on a natural language expression mode and an event element dependency structure, the event prompt template consists of prompt texts and blank prompts, the number of the blank prompts is consistent with the number of types of the event elements, and each blank prompt corresponds to one event element.
6. The method for extracting event elements based on linguistic analysis and prompt learning according to claim 1, wherein the prompt learning event element abstract model in step e is a pre-training language model, the event text is encoded and decoded by the pre-training language model, the event prompt template can be input in series with the event text or from a pre-training language model decoder, and the model output is continuous natural event element abstract text which is formed by sequentially connecting the event elements and other texts in series.
7. The method for extracting event elements based on linguistic analysis and prompt learning according to claim 1, wherein the event element summary text decoding in step f, wherein the pre-filtering includes disabling word filtering, part-of-speech filtering, and trigger word filtering; the semantic classification method is that after the abstract text of the event element is preprocessed according to the requirement of the semantic classification model of the event element, the abstract text of the event element is sent into the semantic classification model of the event element for classification prediction, and the semantic label of the predicted event element is the semantic classification result of the event element.
8. The device is characterized by comprising an event prompt template module, a prompt learning event element abstract module and an event element decoding module, wherein:
event prompt template module: providing a corresponding event prompt template for each event type based on a natural language expression mode and an event element dependency structure so as to prompt the pre-training language model to generate event elements;
prompting a learning event element summary module: the method comprises the steps of abstracting event elements corresponding to event types from event texts, outputting natural language texts with the event elements connected in series, taking an event prompt template and the event texts output by an event element prompt template module as input, utilizing text generation capacity obtained by a pre-training language model in a pre-training stage, abstracting the event elements corresponding to the event types from the event texts, and outputting natural language texts with the event elements connected in series;
an event element decoding module: the method comprises the steps of decoding event elements from event element series texts output by a prompt learning event element abstract module, wherein the event element series texts comprise an event abstract word segmentation unit, an event element classification unit and an event element conversion unit, the event abstract word segmentation unit is used for segmenting and pre-filtering the event element series texts output by the prompt learning event element abstract module, then the event element classification unit is used for classifying the event elements output by the event abstract word segmentation unit based on event semantic tags, and finally the event element conversion unit is used for converting the event semantic tags of the event elements into traditional event element tags, so that event element extraction based on the traditional event element tags is realized.
9. An electronic device comprising at least one processor; at least one GPU computing card; and a memory communicatively coupled to the processor; wherein: the memory stores instructions for execution by at least one processor or by at least one GPU computing card to enable the at least one processor or the at least one GPU computing card to perform the method of claims 1-7.
10. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the methods recited in claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211664406.0A CN116186241A (en) | 2022-12-23 | 2022-12-23 | Event element extraction method and device based on semantic analysis and prompt learning, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211664406.0A CN116186241A (en) | 2022-12-23 | 2022-12-23 | Event element extraction method and device based on semantic analysis and prompt learning, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116186241A true CN116186241A (en) | 2023-05-30 |
Family
ID=86439382
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211664406.0A Pending CN116186241A (en) | 2022-12-23 | 2022-12-23 | Event element extraction method and device based on semantic analysis and prompt learning, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116186241A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116861901A (en) * | 2023-07-04 | 2023-10-10 | 广东外语外贸大学 | Chinese event detection method and system based on multitask learning and electronic equipment |
-
2022
- 2022-12-23 CN CN202211664406.0A patent/CN116186241A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116861901A (en) * | 2023-07-04 | 2023-10-10 | 广东外语外贸大学 | Chinese event detection method and system based on multitask learning and electronic equipment |
| CN116861901B (en) * | 2023-07-04 | 2024-04-09 | 广东外语外贸大学 | Chinese event detection method and system based on multitask learning and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108363743B (en) | Intelligent problem generation method and device and computer readable storage medium | |
| CN111858944B (en) | Entity aspect level emotion analysis method based on attention mechanism | |
| CN110688854B (en) | Named entity recognition method, device and computer readable storage medium | |
| CN111897908A (en) | Event extraction method and system fusing dependency information and pre-training language model | |
| CN110489555A (en) | A kind of language model pre-training method of combination class word information | |
| CN113761936A (en) | Multi-task chapter-level event extraction method based on multi-head self-attention mechanism | |
| CN112989841A (en) | Semi-supervised learning method for emergency news identification and classification | |
| CN112818698B (en) | Fine-grained user comment sentiment analysis method based on dual-channel model | |
| CN111145914B (en) | Method and device for determining text entity of lung cancer clinical disease seed bank | |
| CN113987187A (en) | Multi-label embedding-based public opinion text classification method, system, terminal and medium | |
| CN116661805B (en) | Code representation generation method and device, storage medium and electronic equipment | |
| CN115952791A (en) | Chapter-level event extraction method, device and equipment based on machine reading understanding and storage medium | |
| CN113934909A (en) | Financial event extraction method based on pre-training language and deep learning model | |
| González-Gallardo et al. | Sentence boundary detection for French with subword-level information vectors and convolutional neural networks | |
| CN112349294A (en) | Voice processing method and device, computer readable medium and electronic equipment | |
| CN116341519A (en) | Event causal relation extraction method, device and storage medium based on background knowledge | |
| CN115587184A (en) | Method and device for training key information extraction model and storage medium thereof | |
| CN113947087B (en) | Label-based relation construction method and device, electronic equipment and storage medium | |
| CN116186241A (en) | Event element extraction method and device based on semantic analysis and prompt learning, electronic equipment and storage medium | |
| Rudra Murthy et al. | A deep learning solution to named entity recognition | |
| CN117725211A (en) | Text classification method and system based on self-constructed prompt template | |
| CN114691836B (en) | Text emotion tendentiousness analysis method, device, equipment and medium | |
| CN116483314A (en) | Automatic intelligent activity diagram generation method | |
| CN114936564A (en) | Multi-language semantic matching method and system based on alignment variational self-coding | |
| Kirsch et al. | Noise reduction in distant supervision for relation extraction using probabilistic soft logic |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |