CN113326371A - Event extraction method fusing pre-training language model and anti-noise interference remote monitoring information - Google Patents
Event extraction method fusing pre-training language model and anti-noise interference remote monitoring information Download PDFInfo
- Publication number
- CN113326371A CN113326371A CN202110480675.0A CN202110480675A CN113326371A CN 113326371 A CN113326371 A CN 113326371A CN 202110480675 A CN202110480675 A CN 202110480675A CN 113326371 A CN113326371 A CN 113326371A
- Authority
- CN
- China
- Prior art keywords
- model
- event
- training
- data
- text
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012549 training Methods 0.000 title claims abstract description 58
- 238000000605 extraction Methods 0.000 title claims abstract description 46
- 238000012544 monitoring process Methods 0.000 title claims abstract description 22
- 238000000034 method Methods 0.000 claims abstract description 19
- 238000007781 pre-processing Methods 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 10
- 238000013507 mapping Methods 0.000 claims description 8
- 238000010606 normalization Methods 0.000 claims description 8
- 238000012360 testing method Methods 0.000 claims description 8
- 230000009466 transformation Effects 0.000 claims description 7
- 230000007246 mechanism Effects 0.000 claims description 5
- 238000012545 processing Methods 0.000 claims description 5
- 230000004913 activation Effects 0.000 claims description 4
- 238000013528 artificial neural network Methods 0.000 claims description 4
- 230000006870 function Effects 0.000 claims description 4
- 238000002372 labelling Methods 0.000 claims description 4
- 230000001502 supplementing effect Effects 0.000 claims description 4
- 238000004140 cleaning Methods 0.000 claims description 3
- 238000001514 detection method Methods 0.000 claims description 3
- 238000010200 validation analysis Methods 0.000 claims description 3
- 238000012795 verification Methods 0.000 claims 1
- 239000011159 matrix material Substances 0.000 description 7
- 238000010586 diagram Methods 0.000 description 5
- 239000013598 vector Substances 0.000 description 5
- 238000010801 machine learning Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 239000000284 extract Substances 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 150000001875 compounds Chemical class 0.000 description 2
- 239000000126 substance Substances 0.000 description 2
- 238000010276 construction Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000001902 propagating effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/38—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/383—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Library & Information Science (AREA)
- Machine Translation (AREA)
Abstract
The invention provides an event extraction method for fusing a pre-training language model and anti-noise interference remote monitoring information, and belongs to the technical field of computers. The method uses an integrated knowledge auxiliary model for judgment, is formed by introducing massive text pre-training, takes a pre-training language model containing a large amount of semantic grammar knowledge information as a network structure unit of an event extraction model, uses a model algorithm of a remote supervision characteristic of mixed anti-noise interference, and adds gradient direction anti-interference training under a circular constraint condition.
Description
Technical Field
The invention relates to an event extraction method for fusing a pre-training language model and anti-noise interference remote monitoring information, in particular to the technical field of computer data processing.
Background
With the continuous deepening of information construction in the internet era, mass internet information shows explosive growth, and how to utilize the internet information to assist the making of industry decisions becomes the key point of current attention of enterprises and even government countries. Information from the internet often appears in the form of characters, usually comes from channels such as news manuscripts, forum replies and the like, generally presents the characteristics of no structure and multiple redundancies, and needs to be read, understood and positioned to key information in the characters, and irrelevant contents are filtered. The event extraction is to present the data content of the non-structured text in a structured form, extract the expression key intention in the text by taking the event as a unit, convert the unstructured text information into the structured event information, and further be used in a series of subsequent work such as trend analysis, establishment of a physical knowledge graph, public opinion message early warning and the like, which is an important ring in the information extraction engineering.
The traditional event extraction usually depends on manual overall participation, and in the face of massive internet information, the traditional event analysis reads and searches related information data in huge article reports by means of manual work, and arranges and records the information data, so that a large amount of human resources are consumed. In order to solve the problem of large manpower consumption in the information structuring process, in recent years, a machine learning mode is proposed to identify and extract event patterns. The machine learning mode extracts the text segments which accord with the event pattern in a structured mode by identifying the event pattern in the characters, can realize batched machine text processing, and greatly improves the efficiency problem of extracting character structured information by manual reading. However, the traditional machine learning event pattern template still needs to rely on the knowledge of experts in the field, and corresponding event pattern features are automatically learned through labeled data by means of deep learning, so that the method becomes a new direction for structured extraction of events in recent years. Considering that the internet information is huge, the content is complex and various, the migration and generalization capability of the deep learning model among different events is improved, and the method becomes a difficult problem of the extraction of the internet event information. It is common practice to introduce external knowledge to assist the prediction of the model using remote supervision. The remote supervision algorithm assumes: for a structured event in an existing knowledge graph, it is assumed that any sentence in the external knowledge base containing the entity therein reflects this relationship to some extent. Based on the assumption, the remote supervision algorithm can label the relation labels for the sentences in the external document library based on a labeled small-sized knowledge map, which is equivalent to automatic labeling of samples, so that the remote supervision algorithm is a semi-supervision algorithm. However, the remote supervision brings wrong guidance information besides external knowledge information, and introduces noise interference to influence the accuracy of judgment of the model. The shortfalls of RNN and CNN in text representation capability also affect the prediction extraction of events. Therefore, it is a problem to be considered to study how to use a neural network model with stronger expression ability and use an external knowledge assisted depth model for event structured extraction, and simultaneously reduce error noise interference.
Disclosure of Invention
The purpose of the invention is as follows: an object is to provide an event extraction method for fusing a pre-training language model and anti-noise interference remote monitoring information, so as to solve the above problems in the prior art, enrich text information, and increase the resistance of the model to noise errors through anti-interference training.
The technical scheme is as follows: in a first aspect, an event extraction method for fusing a pre-training language model and anti-noise interference remote supervision information is provided, and the method includes the following steps:
And 2, preprocessing the marked data, namely removing html tags and special symbols, and segmenting the text into short texts in the form of sentences or paragraphs.
Step 3, marking event trigger words, subjects, objects, time, places and event types of the text with events according to the event definitions, supplementing marked data into a remote monitoring knowledge base, and finishing marking the data; matching the marked data with a remote supervision knowledge base, adding the successfully matched trigger word into the remote supervision information of the sample, and performing the following steps: 1: the scale of 2 is divided into a training set, a validation set, and a test set.
And 4, respectively constructing models in two stages of event detection and event participation element extraction of event extraction.
And 5, training the event extraction model by using the training data, evaluating the quality of the training by verifying the data set and testing the data set, and selecting the model with the optimal performance for use by multiple rounds of iteration.
And 6, predicting and extracting events of the new unlabelled Internet text data by using the trained model, matching the new text with a remote supervision knowledge base after the new text is subjected to data preprocessing and cleaning, adding trigger words appearing in the remote knowledge base and the new text to be predicted into remote characteristics of the text, and inputting the trigger type of the model prediction event and related event participation elements.
In some implementations of the first aspect, the specific steps of constructing the event extraction model are as follows:
performing language modeling through a self-attention mechanism, and capturing multi-angle characteristic information in a text by using multi-head attention;
performing feature transformation extraction through a double-layer neural network FFN, adopting a ReLU as an activation function layer, and performing feature normalization processing by using layer normalization;
combining all layers of the pre-training model by utilizing a residual connection mode, and obtaining the characteristics extracted by the pre-training model through loop iteration;
marking the position where the remote supervision trigger word appears by using the type number of the trigger word in a remote supervision library to obtain a discrete sequence characteristic, and mapping the discrete sequence characteristic to a low-dimensional space through a remote supervision embedding layer;
splicing the features extracted by the pre-training model and the remote supervision features;
in the process of learning and parameter updating of the remote supervision embedded layer, adding disturbance in the gradient direction, and calculating the gradient of the model in the remote supervision characteristic embedded layer in a back propagation manner;
original parameters of a remote supervision characteristic embedding layer are reserved, the obtained disturbance step length is used for adding counterdisturbance, and forward and backward propagation is carried out again to obtain a new gradient;
restoring the original parameters of the remote supervision characteristic embedded layer, and updating the overall parameters of the model by using the new gradient after disturbance is added;
repeating the training process for multiple times, and selecting the model with the best result for use;
inputting new internet text data into a model through preprocessing;
and predicting whether an event occurs or not, and extracting the structured event elements when the event occurs is detected.
Has the advantages that: the invention provides an event extraction method fusing a pre-training language model and anti-noise interference remote monitoring information, which is characterized in that automatic event structured extraction is carried out by combining the pre-training language model and the remote monitoring information, and disturbance countermeasure training is added, so that the error noise information caused by remote monitoring is effectively relieved while the model effect is improved by introducing external knowledge, and the event extraction effect is improved. The comprehensive knowledge auxiliary model is used for judging, and is formed by introducing massive text pre-training, a pre-training language model containing a large amount of semantic grammar knowledge information is used as a network structure unit of an event extraction model, and a model algorithm of a remote supervision characteristic of mixed anti-noise interference is used.
Drawings
FIG. 1 is a model structure diagram of an event extraction method for fusing a pre-training language model and anti-noise interference remote monitoring information.
FIG. 2 is a schematic diagram of an event extraction method attention mechanism feature extraction that fuses a pre-training language model and anti-noise interference remote supervision information.
FIG. 3 is a diagram of an event extraction method remote monitoring feature layer structure that fuses a pre-trained language model with anti-noise interference remote monitoring information.
FIG. 4 is a diagram of experimental effects of an event extraction method that combines a pre-training language model with anti-noise interference remote monitoring information.
FIG. 5 is a schematic diagram of an event extraction method incorporating a pre-trained language model and anti-noise jamming remote supervisory information with circular constraints added.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
In one embodiment, the present invention provides an event extraction method for fusing a pre-training language model and anti-noise interference remote monitoring information, which includes the following steps:
And 2, preprocessing the marked data, namely removing html tags and special symbols, and segmenting the text into short texts in the form of sentences or paragraphs.
Step 3, marking event trigger words, subjects, objects, time, places and event types of the text with events according to the event definitions, supplementing marked data into a remote monitoring knowledge base, and finishing marking the data; matching the marked data with a remote supervision knowledge base, adding the successfully matched trigger word into the remote supervision information of the sample, and performing the following steps: 1: the scale of 2 is divided into a training set, a validation set, and a test set.
And 4, respectively constructing models in two stages of event detection and event participation element extraction of event extraction.
Specifically, an event extraction model combining a pre-training model and remote supervision is established, as shown in fig. 1, the specific process is as follows: firstly, a pre-training model structure based on massive text training is used for coding a text, and as shown in fig. 2, language modeling is performed through a self-attention mechanism:
q, K, V is a calculation matrix of attention, which is obtained by matrix operation of input data and corresponding parameters;
and representing the dimension of the text vector after the text is compressed by the embedded matrix.
Secondly, capturing feature information of multiple angles in the text by using multi-head attention as follows:
in the formula (I), the compound is shown in the specification,
representing a linear transformation matrix and participating in the updating of model training parameters; wherein:
in the formula (I), the compound is shown in the specification,
a transformation mapping matrix representing a Query vector;
a transformation mapping matrix representing Key vectors;
representing Value x vectorsTransforming the mapping matrix;
thirdly, performing feature transformation extraction through a double-layer neural network FFN, and taking ReLU as an activation function layer:
and using layer normalization for feature normalization:
wherein the content of the first and second substances,
a calculated parameter representing a first fully-connected network in the feedforward network layer;
a calculated parameter representing a second fully-connected network in the feedforward network layer; indicating that each word position of the sample represents the average of the vectors,
representing the variance of each word position representation vector for the sample,
is a learnable weight parameter;
a smaller value to prevent divide by zero errors;
indicating that the Element-wise Product is multiplied position by position.
From time to time, the layers of the pre-training model are combined in a residual connection mode:
x represents an input variable, corresponding to cross-layer addition;
representing the original input of the layer network; and y is the output characteristic of the network layer, and the characteristic extracted by the pre-training model is obtained by superposing the processes for multiple times.
And finally, splicing the features extracted by the pre-training model with the remote monitoring features, labeling the positions where the remote monitoring trigger words appear by using the type numbers of the trigger words in a remote monitoring library to obtain discrete sequence features, mapping the discrete sequence features to a low-dimensional space through a remote monitoring embedding layer, splicing the discrete sequence features with the pre-training extracted features and sending the discrete sequence features and the pre-training extracted features into a classifier for event prediction as shown in figure 3, and performing classification prediction on the position of each character to determine whether the trigger words are the remote monitoring trigger words.
And 5, training the event extraction model by using the training data, evaluating the quality of the training by verifying the data set and testing the data set, and selecting the model with the optimal performance for use by multiple rounds of iteration.
And 6, predicting and extracting events of the new unlabelled Internet text data by using the trained model, matching the new text with a remote supervision knowledge base after the new text is subjected to data preprocessing and cleaning, adding trigger words appearing in the remote knowledge base and the new text to be predicted into remote characteristics of the text, and inputting the trigger type of the model prediction event and related event participation elements.
In a further embodiment, the specific steps of constructing the event extraction model include:
language modeling is carried out through a self-attention mechanism, and multi-head attention is used for capturing feature information of multiple angles in a text.
And performing feature transformation extraction through a double-layer neural network FFN, adopting the ReLU as an activation function layer, and performing feature normalization processing by using layer normalization.
And combining all layers of the pre-training model by utilizing a residual error connection mode, and obtaining the characteristics extracted by the pre-training model through loop iteration.
And marking the position where the remote supervision trigger word appears by using the type number of the trigger word in a remote supervision library to obtain a discrete sequence characteristic, and mapping the discrete sequence characteristic to a low-dimensional space through a remote supervision embedding layer.
And splicing the features extracted by the pre-training model and the remote supervision features.
And in the process of learning and parameter updating of the remote supervision embedded layer, adding disturbance in the gradient direction, and reversely propagating the gradient of the remote supervision characteristic embedded layer by the calculation model.
And (4) retaining the original parameters of the remote supervision characteristic embedded layer, adding the anti-disturbance by using the obtained disturbance step length, and performing forward and backward propagation again to obtain a new gradient.
And recovering the original parameters of the remote supervision characteristic embedded layer, and updating the overall parameters of the model by using the new gradient after disturbance is added.
And repeating the training process for multiple times, and selecting the model with the best result for use.
And inputting new Internet text data into the model through preprocessing.
And predicting whether an event occurs or not, and extracting the structured event elements when the event occurs is detected.
In a further embodiment, in order to alleviate false trigger word information brought by remote supervision, a counterstudy strategy is adopted in the patent, and disturbance in the gradient direction is added by adopting a counterstudy method in the process of learning and parameter updating of a remote supervision embedded layer. The anti-noise interference capability of the model is improved, and the method specifically comprises the following steps:
calculating the gradient of the model back propagation in the remote supervision feature embedding layer:
setting disturbance radius by preventing optimal point in disturbance deviation constraint in spherical mapping mode
Representing the size of the constraint range of the disturbance, and obtaining a disturbance step length:
wherein the content of the first and second substances,
in order to be a constraint space for the perturbation,
step size of a small step.
And (4) retaining the original parameters of the remote supervision characteristic embedded layer, adding the anti-disturbance by using the obtained disturbance step length, and performing forward and backward propagation again to obtain a new gradient.
And recovering the original parameters of the remote supervision characteristic embedded layer, and updating the overall parameters of the model by using the new gradient after disturbance is added.
The event element extraction network structure is similar to the event prediction, only the remote supervision feature embedding layer is removed, and each type of element is predicted on the output layer. Tests prove that the method is superior to other event extraction methods of machine learning in accuracy, recall rate and f1 scores, and is shown in figure 4.
As noted above, while the present invention has been shown and described with reference to certain preferred embodiments, it is not to be construed as limited thereto. Various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (4)
1. An event extraction method for fusing a pre-training language model and anti-noise interference remote supervision information is characterized by comprising the following steps:
step 1, collecting training data corpora, and storing data in a text form in a txt file form through a crawler according to internet text data obtained by the crawler;
step 2, preprocessing the marked data;
and 3, labeling the text according to the event definition, supplementing the labeled data into a remote monitoring knowledge base to finish labeling the data, and according to the following steps of 7: 1: 2, dividing the proportion into a training set, a verification set and a test set;
step 4, respectively constructing models in two stages of event detection and event participation element extraction of event extraction;
step 5, training an event extraction model by using training data, evaluating the quality of the training by verifying a data set and testing the data set, and selecting the model with the optimal performance for use by multiple rounds of iteration;
and 6, predicting and extracting events of the new unlabelled Internet text data by using the trained model, matching the new text with a remote supervision knowledge base after the new text is subjected to data preprocessing and cleaning, adding trigger words appearing in the remote knowledge base and the new text to be predicted into remote characteristics of the text, and inputting the trigger type of the model prediction event and related event participation elements.
2. The method for extracting events fusing a pre-trained language model and anti-noise interference remote supervision information according to claim 1, wherein the event extraction model is constructed by the following specific steps:
performing language modeling through a self-attention mechanism, and capturing multi-angle characteristic information in a text by using multi-head attention;
performing feature transformation extraction through a double-layer neural network FFN, adopting a ReLU as an activation function layer, and performing feature normalization processing by using layer normalization;
combining all layers of the pre-training model by utilizing a residual connection mode, and obtaining the characteristics extracted by the pre-training model through loop iteration;
marking the position where the remote supervision trigger word appears by using the type number of the trigger word in a remote supervision library to obtain a discrete sequence characteristic, and mapping the discrete sequence characteristic to a low-dimensional space through a remote supervision embedding layer;
splicing the features extracted by the pre-training model and the remote supervision features;
in the process of learning and parameter updating of the remote supervision embedded layer, adding disturbance in the gradient direction, and calculating the gradient of the model in the remote supervision characteristic embedded layer in a back propagation manner;
original parameters of a remote supervision characteristic embedding layer are reserved, the obtained disturbance step length is used for adding counterdisturbance, and forward and backward propagation is carried out again to obtain a new gradient;
restoring the original parameters of the remote supervision characteristic embedded layer, and updating the overall parameters of the model by using the new gradient after disturbance is added;
repeating the training process for multiple times, and selecting a model with the optimal result for use;
inputting new internet text data into a model through preprocessing;
and predicting whether an event occurs or not, and extracting the structured event elements when the event occurs is detected.
3. The event extraction method for fusing the pre-training language model and the anti-noise interference remote supervision information according to claim 1, wherein the pre-processing operation comprises removing html tags and special symbols, and segmenting the text into short texts in the form of sentences or paragraphs.
4. The method for extracting events fusing a pre-trained language model with anti-noise interference remote supervision information according to claim 1, wherein the step 3 is further as follows: marking event trigger words, subjects, objects, time, places and event types of the text with events according to the event definitions, supplementing marked data into a remote monitoring knowledge base, and finishing marking of the data; matching the marked data with a remote supervision knowledge base, adding the successfully matched trigger words into the remote supervision information of the current sample, and performing the following steps: 1: the scale of 2 is divided into a training set, a validation set, and a test set.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110480675.0A CN113326371B (en) | 2021-04-30 | 2021-04-30 | Event extraction method integrating pre-training language model and anti-noise interference remote supervision information |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110480675.0A CN113326371B (en) | 2021-04-30 | 2021-04-30 | Event extraction method integrating pre-training language model and anti-noise interference remote supervision information |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113326371A true CN113326371A (en) | 2021-08-31 |
| CN113326371B CN113326371B (en) | 2023-12-29 |
Family
ID=77414011
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110480675.0A Active CN113326371B (en) | 2021-04-30 | 2021-04-30 | Event extraction method integrating pre-training language model and anti-noise interference remote supervision information |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113326371B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113762482A (en) * | 2021-09-15 | 2021-12-07 | 智道网联科技(北京)有限公司 | Training method of neural network model for automatic driving and related device |
Citations (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106874410A (en) * | 2017-01-22 | 2017-06-20 | 清华大学 | Chinese microblogging text mood sorting technique and its system based on convolutional neural networks |
| CN108182295A (en) * | 2018-02-09 | 2018-06-19 | 重庆誉存大数据科技有限公司 | A kind of Company Knowledge collection of illustrative plates attribute extraction method and system |
| CN109063185A (en) * | 2018-08-27 | 2018-12-21 | 电子科技大学 | Social networks short text data filter method towards event detection |
| CN109582949A (en) * | 2018-09-14 | 2019-04-05 | 阿里巴巴集团控股有限公司 | Event element abstracting method, calculates equipment and storage medium at device |
| CN110188172A (en) * | 2019-05-31 | 2019-08-30 | 清华大学 | Text based event detecting method, device, computer equipment and storage medium |
| CN111125370A (en) * | 2019-12-06 | 2020-05-08 | 南京中新赛克科技有限责任公司 | Relation extraction method suitable for small samples |
| CN111339774A (en) * | 2020-02-07 | 2020-06-26 | 腾讯科技（深圳）有限公司 | Text entity relation extraction method and model training method |
| CN111694924A (en) * | 2020-06-17 | 2020-09-22 | 合肥中科类脑智能技术有限公司 | Event extraction method and system |
| CN111897908A (en) * | 2020-05-12 | 2020-11-06 | 中国科学院计算技术研究所 | Event extraction method and system fusing dependency information and pre-training language model |
| CN111914558A (en) * | 2020-07-31 | 2020-11-10 | 湖北工业大学 | Course knowledge relation extraction method and system based on sentence bag attention remote supervision |
| CN112016293A (en) * | 2020-10-22 | 2020-12-01 | 浙江大学 | Remote supervision relation extraction method based on multi-instance collaborative confrontation training |
| CN112052665A (en) * | 2020-09-12 | 2020-12-08 | 广东工业大学 | Remote monitoring event extraction method and application thereof |
| WO2020247616A1 (en) * | 2019-06-07 | 2020-12-10 | Raytheon Bbn Technologies Corp. | Linguistically rich cross-lingual text event embeddings |
| CN112307130A (en) * | 2020-10-21 | 2021-02-02 | 清华大学 | Document-level remote supervision relation extraction method and system |
| AU2020103905A4 (en) * | 2020-12-04 | 2021-02-11 | Chongqing Normal University | Unsupervised cross-domain self-adaptive medical image segmentation method based on deep adversarial learning |
| WO2021042503A1 (en) * | 2019-09-06 | 2021-03-11 | 平安科技（深圳）有限公司 | Information classification extraction method, apparatus, computer device and storage medium |
| CN112487203A (en) * | 2019-01-25 | 2021-03-12 | 中译语通科技股份有限公司 | Relation extraction system integrated with dynamic word vectors |
| CN112580328A (en) * | 2020-12-11 | 2021-03-30 | 上海明略人工智能(集团)有限公司 | Event information extraction method and device, storage medium and electronic equipment |
-
2021
- 2021-04-30 CN CN202110480675.0A patent/CN113326371B/en active Active
Patent Citations (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106874410A (en) * | 2017-01-22 | 2017-06-20 | 清华大学 | Chinese microblogging text mood sorting technique and its system based on convolutional neural networks |
| CN108182295A (en) * | 2018-02-09 | 2018-06-19 | 重庆誉存大数据科技有限公司 | A kind of Company Knowledge collection of illustrative plates attribute extraction method and system |
| CN109063185A (en) * | 2018-08-27 | 2018-12-21 | 电子科技大学 | Social networks short text data filter method towards event detection |
| CN109582949A (en) * | 2018-09-14 | 2019-04-05 | 阿里巴巴集团控股有限公司 | Event element abstracting method, calculates equipment and storage medium at device |
| CN112487203A (en) * | 2019-01-25 | 2021-03-12 | 中译语通科技股份有限公司 | Relation extraction system integrated with dynamic word vectors |
| WO2020239061A1 (en) * | 2019-05-31 | 2020-12-03 | 腾讯科技（深圳）有限公司 | Text-based event detection method and apparatus, computer device and storage medium |
| CN110188172A (en) * | 2019-05-31 | 2019-08-30 | 清华大学 | Text based event detecting method, device, computer equipment and storage medium |
| WO2020247616A1 (en) * | 2019-06-07 | 2020-12-10 | Raytheon Bbn Technologies Corp. | Linguistically rich cross-lingual text event embeddings |
| WO2021042503A1 (en) * | 2019-09-06 | 2021-03-11 | 平安科技（深圳）有限公司 | Information classification extraction method, apparatus, computer device and storage medium |
| CN111125370A (en) * | 2019-12-06 | 2020-05-08 | 南京中新赛克科技有限责任公司 | Relation extraction method suitable for small samples |
| CN111339774A (en) * | 2020-02-07 | 2020-06-26 | 腾讯科技（深圳）有限公司 | Text entity relation extraction method and model training method |
| CN111897908A (en) * | 2020-05-12 | 2020-11-06 | 中国科学院计算技术研究所 | Event extraction method and system fusing dependency information and pre-training language model |
| CN111694924A (en) * | 2020-06-17 | 2020-09-22 | 合肥中科类脑智能技术有限公司 | Event extraction method and system |
| CN111914558A (en) * | 2020-07-31 | 2020-11-10 | 湖北工业大学 | Course knowledge relation extraction method and system based on sentence bag attention remote supervision |
| CN112052665A (en) * | 2020-09-12 | 2020-12-08 | 广东工业大学 | Remote monitoring event extraction method and application thereof |
| CN112307130A (en) * | 2020-10-21 | 2021-02-02 | 清华大学 | Document-level remote supervision relation extraction method and system |
| CN112016293A (en) * | 2020-10-22 | 2020-12-01 | 浙江大学 | Remote supervision relation extraction method based on multi-instance collaborative confrontation training |
| AU2020103905A4 (en) * | 2020-12-04 | 2021-02-11 | Chongqing Normal University | Unsupervised cross-domain self-adaptive medical image segmentation method based on deep adversarial learning |
| CN112580328A (en) * | 2020-12-11 | 2021-03-30 | 上海明略人工智能(集团)有限公司 | Event information extraction method and device, storage medium and electronic equipment |
Non-Patent Citations (4)
| Title |
|---|
| QUANZHI LI ET AL.: "A Unified Model for Financial Event Classification, Detection and Summarization", 《PROCEEDINGS OF THE TWENTY-NINTH INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE 》, pages 4668 - 4674 * |
| QUANZHI LI等: "A Unified Model for Financial Event Classification, Detection and Summarization", PROCEEDINGS OF THE TWENTY-NINTH INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE (IJCAI-20), pages 4668 - 4674 * |
| 陈星月等: "基于ELECTRA模型与词性特征的金融事件抽取方法研究", 《数据分析与知识发现》, vol. 5, no. 7, pages 36 - 47 * |
| 陈星月等: "基于ELECTRA模型与词性特征的金融事件抽取方法研究", 数据分析与知识发现, vol. 5, no. 07, pages 36 - 47 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113762482A (en) * | 2021-09-15 | 2021-12-07 | 智道网联科技(北京)有限公司 | Training method of neural network model for automatic driving and related device |
| CN113762482B (en) * | 2021-09-15 | 2024-04-16 | 智道网联科技(北京)有限公司 | Training method and related device for neural network model for automatic driving |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113326371B (en) | 2023-12-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110134757B (en) | Event argument role extraction method based on multi-head attention mechanism | |
| CN110598005A (en) | Public safety event-oriented multi-source heterogeneous data knowledge graph construction method | |
| CN113191148B (en) | Rail transit entity identification method based on semi-supervised learning and clustering | |
| CN109918505B (en) | Network security event visualization method based on text processing | |
| CN113742733B (en) | Method and device for extracting trigger words of reading and understanding vulnerability event and identifying vulnerability type | |
| CN111597328B (en) | New event theme extraction method | |
| CN115292568B (en) | Civil news event extraction method based on joint model | |
| Suyanto | Synonyms-based augmentation to improve fake news detection using bidirectional LSTM | |
| CN113988075A (en) | Network security field text data entity relation extraction method based on multi-task learning | |
| CN115203338A (en) | Label and label example recommendation method | |
| CN114881173A (en) | Resume classification method and device based on self-attention mechanism | |
| CN113378024B (en) | Deep learning-oriented public inspection method field-based related event identification method | |
| CN117151222B (en) | Domain knowledge guided emergency case entity attribute and relation extraction method thereof, electronic equipment and storage medium | |
| CN111786999B (en) | Intrusion behavior detection method, device, equipment and storage medium | |
| CN113326371A (en) | Event extraction method fusing pre-training language model and anti-noise interference remote monitoring information | |
| CN114969334B (en) | Abnormal log detection method and device, electronic equipment and readable storage medium | |
| CN115757062A (en) | Log anomaly detection method based on sentence embedding and Transformer-XL | |
| CN116089605A (en) | Text emotion analysis method based on transfer learning and improved word bag model | |
| CN115994531A (en) | Multi-dimensional text comprehensive identification method | |
| CN112765940B (en) | Webpage deduplication method based on theme features and content semantics | |
| CN114648029A (en) | Electric power field named entity identification method based on BiLSTM-CRF model | |
| CN114298041A (en) | Network security named entity identification method and identification device | |
| CN114491033A (en) | Method for building user interest model based on word vector and topic model | |
| CN116821349B (en) | Literature analysis method and management system based on big data | |
| Jony et al. | Domain specific fine tuning of pre-trained language model in NLP |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |