CN113255321B - Financial field chapter-level event extraction method based on article entity word dependency relationship - Google Patents

Financial field chapter-level event extraction method based on article entity word dependency relationship Download PDF

Info

Publication number
CN113255321B
CN113255321B CN202110648901.1A CN202110648901A CN113255321B CN 113255321 B CN113255321 B CN 113255321B CN 202110648901 A CN202110648901 A CN 202110648901A CN 113255321 B CN113255321 B CN 113255321B
Authority
CN
China
Prior art keywords
entity
event
candidate
word
financial
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110648901.1A
Other languages
Chinese (zh)
Other versions
CN113255321A (en
Inventor
王海涛
许浩
刘智
周丹
孙婉琪
马雪环
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Lab
Original Assignee
Zhejiang Lab
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang Lab filed Critical Zhejiang Lab
Priority to CN202110648901.1A priority Critical patent/CN113255321B/en
Publication of CN113255321A publication Critical patent/CN113255321A/en
Application granted granted Critical
Publication of CN113255321B publication Critical patent/CN113255321B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/205—Parsing
    • G06F40/211—Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/205—Parsing
    • G06F40/216—Parsing using statistical methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
    • G06F40/295—Named entity recognition

Abstract

The invention discloses a financial field chapter-level event extraction method based on article entity word dependency, and designs a structured dependency self-attention mechanism module, wherein the module takes the entity word dependency in an article as input data of an event extraction deep learning model, and the input data is combined with language features of word level and sentence level to improve the prediction precision of event trigger words and event arguments when the deep learning model extracts financial events. In addition, in the Chinese financial event extraction task, the invention firstly provides 8 different types of entity relationships for uniformly representing entity dependency relationships in articles. The invention simultaneously constructs a set of hierarchical event relations in the financial field, and is used for distinguishing similar event types by a model. The invention has clear logic structure, distinct hierarchy and detailed system implementation details from system input to system output, realizes an end-to-end system closed working mode, and is easy to fall on the ground and applied in a large scale.

Description

Financial field chapter-level event extraction method based on article entity word dependency relationship
Technical Field
The invention belongs to the cross field of artificial intelligence and finance, and particularly relates to a method for extracting chapter-level events in the financial field based on article entity word dependency.
Background
Artificial intelligence is gradually involved in various industry fields at the present stage, and new possibility is provided for the development of the artificial intelligence. The financial field, one of the largest industries in which real-time data is generated, is also stepping into the "AI + finance" era. As one of the important tasks in the financial field, it is very significant and valuable to extract valuable financial events from important financial announcements/news. The currently known chapter-level financial event extraction method is an event extraction system based on the traditional rules and the traditional machine learning paradigm, and has the following defects:
1) most of the existing financial event extraction methods are based on single-statement event extraction, and cannot process chapter-level and multi-statement financial event extraction;
2) the existing chapter-level financial event extraction method does not consider the relationship information between long dependencies among entities; for example, the meaning of the representation appears in different sentences for the same entity, the semantic information represented in the same sentence appears in different entities at the same time, and the like. The meaning represented by the entity appearing in different contexts or the same context constitutes rich semantic information of a certain entity, and is an essential semantic feature for the task of event extraction.
Disclosure of Invention
The invention aims to provide a financial field chapter-level event extraction method based on article entity word dependency relationship aiming at the defects of the prior art. The invention applies the artificial intelligence method to the financial field, and autonomously extracts major events and forms a structural event representation aiming at major notices and news released by listed companies.
The purpose of the invention is realized by the following technical scheme: a financial field chapter-level event extraction method based on article entity word dependency relationship comprises the following steps:
(1) and (3) entity word extraction: converting a Chinese article into an entity set comprising n entitiesE={e 1,e 2,e 3,……,e n }；
(2) Entity word vector mapping module: entity set using Embedding mapping method
Figure 81738DEST_PATH_IMAGE001
The Chinese entity is mapped into an entity word vector set of a vector space, and the entity word vector is as follows:
Figure 733299DEST_PATH_IMAGE002
wherein the content of the first and second substances,i=1~n，e i is as followsiThe number of the individual entities,LMin order to be a function of the vector mapping,W e to generate a physical word vector
Figure 857244DEST_PATH_IMAGE003
Trainable parameters of (a);
(3) extracting candidate argument sets: candidate argument set of article extracted by sequence labeling method based on pre-training language modelA={a 1,a 2,……,a K }；
(4) Building an entity dependency and structured self-attention module: according to entity words in entity setE={e 1,e 2,e 3,……,e n Constructing entity dependency relationships of different types according to the positions of the entity dependency relationships in the articles; the structured self-attention module constructs a structured entity dependence characteristic and outputs an entity set fused with the structured entity dependence characteristic;
(5) extracting a candidate trigger word set: taking an entity set fused with the dependency characteristics of the structured entities as input, and extracting a candidate trigger word set of an article by using a sequence tagging method based on a pre-training language modelT={T 1,T 2,……,T u }；
(6) A hierarchical financial event attention module: generating a hierarchical financial event characteristic by utilizing a predefined hierarchical financial event structure and combining an attention mechanism model;
(7) event trigger word and event argument joint prediction based on the Peer Attention mechanism: the Peer Attention mechanism module obtains semantic relationship characteristics among the entities based on the entity word vector set obtained in the step (2); the argument extraction module is used for obtaining a probability set corresponding to candidate arguments based on the candidate argument set, the candidate trigger word set, the hierarchical financial event characteristics and the semantic relation characteristics among the entities; the trigger word extraction module obtains a probability set corresponding to the candidate trigger words based on the candidate trigger word set, the hierarchical financial event characteristics and the semantic relation characteristics among the entities; joint prediction module predicts trigger words of events in articles based on two probability setsT’And constituting an eventSet of theory elementsA’。
Further, the step (3) is specifically as follows:
(3.1) carrying out entity annotation by using a BIO annotation strategy;
(3.2) extracting candidate arguments by using a sequence tagging model based on a pre-training language model BERT;
(3.3) outputting the candidate theory element setA={a 1,a 2,……,a K }。
Further, the step (4) is specifically as follows:
(4.1) constructing different types of entity dependency relationships, and defining the entity dependency relationships of the financial chapter-level text to obtain an entity dependency relationship matrixsThe method comprises the following steps:
(a) the trigger words of the event appear in the same sentence for a plurality of times and are marked as 'common fingers in the trigger words and sentences';
(b) trigger words of events appear in different sentences and are marked as 'common reference between trigger words and sentences';
(c) one argument of an event appears in the same sentence for multiple times and is marked as 'common reference in an argument sentence';
(d) one argument of an event appears in different sentences, marked as "intersentence coreference";
(e) the trigger words and arguments of the event appear in the same sentence and are marked as 'related in entity sentence';
(f) trigger words and arguments of events occur in different sentences, marked as "inter-sentence correlation of entities";
(g) the dependency relationship of the non-entity word and the trigger word/argument in the same sentence is marked as 'other correlations in the sentence';
(h) the relationship of other types of non-entity words to trigger words/arguments, labeled "no dependency";
(4.2) constructing a structural entity dependence characteristic by a structural self-attention module, and outputting an entity set fused with the structural entity dependence characteristic; the structured self-attention module is a computing unit and a module which can be repeatedly used in an overlapping wayThe input of the block is an entity vector characteristic and an entity dependency relationship matrixsAnd 1 is firstl-the output of 1 structured self-attention module islThe entity vector characteristics input by the structured self-attention module; and (3) the first structured entity vector feature input from the attention module is the entity word vector set obtained in the step (2).
Further, the step (4.2) is specifically as follows:
parameter instantiation is carried out on the entity dependence relationship:
Figure 721295DEST_PATH_IMAGE004
wherein, the matrixsElement (1) ofs i j,Representing entitiesiAndjj = 1-n;
Figure DEST_PATH_IMAGE005
is shown aslGenerated by a structured self-attention modules i j,Vectorization features that satisfy certain entity dependencies;
Figure 129142DEST_PATH_IMAGE006
is the firstlA structured self-attention module pairs i j,Training parameters for the transformation;
generating vector characteristics integrating entity dependency relationships by utilizing bidirectional affine transformation:
Figure 420446DEST_PATH_IMAGE007
wherein the content of the first and second substances,
Figure 421900DEST_PATH_IMAGE008
is the firstlThe structured entity dependency characteristics of each structured self-attention module,
Figure DEST_PATH_IMAGE009
and
Figure 699429DEST_PATH_IMAGE010
are respectively input tolAn entity of a structured self-attention moduleiAndjthe vectorization feature of (a) is,
Figure 102729DEST_PATH_IMAGE011
is as followslTrainable parameters of a structured self-attention module;
combining the entity vector characteristics and the structured entity dependency relationship characteristics input by the current structured self-attention module, sequentially passing through the characteristic merging layer of the backward normalization layer and the full connection layer of the backward normalization layer to obtain the output of the current structured self-attention module, inputting the output into the next structured self-attention module, and finally outputting the last structured self-attention module as the vector characteristics of the entity set fused with the structured entity dependency characteristics.
Further, the step (5) is specifically as follows: taking the entity set which is output in the step (4.2.2) and fused with the dependency characteristics of the structural entities as input, adopting a BIO (building information organization) labeling strategy and combining a sequence labeling model based on a pre-training language model, extracting candidate trigger word entities from the labeled articles, and outputting a candidate trigger word setT={T 1,T 2,……,T u }。
Further, the step (6) is specifically as follows:
the hierarchical financial event structure divides a plurality of subdivided events in the financial events into different large events, and specifically comprises the following steps: the major events include financing, trading, stock increase and decrease, financial index change, multi-party cooperation, personnel change, marketing correlation and law enforcement; the financing comprises the quality assurance, the quality assurance and the enterprise borrowing, the transaction comprises the share repurchase and the enterprise mergence, the share increase and decrease support comprises the share decrease support and the share increase support, the financial index change comprises the loss, the multi-party cooperation comprises the bid-winning, the personnel change comprises the high-level manager change, the marketing correlation comprises the company marketing and the bankruptcy clearing, and the law enforcement comprises the contracted negotiation and the punishment;
based on the step (5)The extracted candidate trigger word set respectively processes the large event characteristics and the subdivided event characteristics, and the large event characteristics and the subdivided event characteristics respectively generate importance weighted values for all candidate arguments through a hierarchical event attention mechanism
Figure 689568DEST_PATH_IMAGE012
And
Figure 178318DEST_PATH_IMAGE013
wherein, in the step (A),
Figure 649750DEST_PATH_IMAGE012
is a candidate trigger word
Figure 517343DEST_PATH_IMAGE014
Candidate argument importance weight values generated by events belonging to the broad category,
Figure 681609DEST_PATH_IMAGE013
is a candidate trigger word
Figure 657655DEST_PATH_IMAGE014
Candidate argument importance weighted values generated by the belonged subdivision events; obtaining the ith financial event structure based on the hierarchical financial event structuretAttention weight corresponding to each candidate trigger word
Figure 526254DEST_PATH_IMAGE015
：
Figure 904145DEST_PATH_IMAGE016
IthtHierarchical financial event features corresponding to candidate trigger words
Figure 583520DEST_PATH_IMAGE017
Obtained by the following calculation:
Figure 46862DEST_PATH_IMAGE018
wherein the content of the first and second substances,
Figure DEST_PATH_IMAGE019
as candidate argument
Figure 594518DEST_PATH_IMAGE020
Is used to represent the vector of (a),
Figure 685971DEST_PATH_IMAGE021
is that
Figure 192038DEST_PATH_IMAGE015
Middle candidate trigger wordi t And candidate argumentj t The weight relationship of (a) to (b),M e andb e are trainable parameters of the hierarchical financial event attention module.
Further, in step (7), the Peer Attention mechanism module is embodied as an entityiSet of contiguous entities ofN i And entityiAndN i set of edges ofD i As input and output entitiesiAnd entitiesjSemantic relationship features between
Figure 877098DEST_PATH_IMAGE022
：
Figure 103811DEST_PATH_IMAGE023
Wherein the content of the first and second substances,Multi_headis a functional representation of a multi-headed self-attentive module,
Figure 190715DEST_PATH_IMAGE024
is an entityjA vector representation of (a);
Figure 726739DEST_PATH_IMAGE025
is thatN i To middlepThe number of the individual entities,
Figure 899094DEST_PATH_IMAGE026
is that
Figure 788553DEST_PATH_IMAGE025
Is used to represent the vector of (a),
Figure 870910DEST_PATH_IMAGE027
is an entityjAnd
Figure 718780DEST_PATH_IMAGE025
vector representation of edges in between;p=1~q。
further, in the step (7), the argument extraction module and the trigger word extraction module are two independent full connection layers.
Further, in step (7), the joint prediction module defines the joint prediction probability of the event trigger and the argumentP(event|D) The following were used:
Figure 378431DEST_PATH_IMAGE028
wherein D represents a Chinese article, event represents a financial event,
Figure 196215DEST_PATH_IMAGE029
in the representative article D
Figure 992132DEST_PATH_IMAGE030
The probability of an event occurring;
Figure 745325DEST_PATH_IMAGE031
as candidate trigger word
Figure 767639DEST_PATH_IMAGE014
K corresponds to the type of subdivision event,
Figure 530058DEST_PATH_IMAGE032
representing candidate triggers in article D
Figure 446062DEST_PATH_IMAGE014
Belong to the firstkProbability of class subdivision events;
Figure 229210DEST_PATH_IMAGE033
is a candidate trigger word
Figure 863453DEST_PATH_IMAGE014
A set of contiguous entities; representing the candidate trigger words by a correlation probability matrix constructed based on the candidate trigger word probability set and the candidate argument probability set
Figure 898406DEST_PATH_IMAGE014
The corresponding event type contains argument type
Figure 809861DEST_PATH_IMAGE034
The probability of (d);
Figure 904856DEST_PATH_IMAGE034
the type of the argument is represented and,
Figure 760816DEST_PATH_IMAGE035
to be in a candidate trigger word
Figure 989672DEST_PATH_IMAGE014
And candidate arguments
Figure 614689DEST_PATH_IMAGE020
Article D of (1), candidate arguments
Figure 146164DEST_PATH_IMAGE020
Is of the type
Figure 99208DEST_PATH_IMAGE034
The probability of (d); finally, the trigger words of events in the text are predicted through two parallel output layersT’And argument set constituting eventA’。
Further, parameters of a structured self-Attention module, an entity word vector mapping module, a hierarchical financial event Attention module, a Peer Attention mechanism module, an argument extraction module, a trigger word extraction module and a joint prediction module are trained by utilizing a back propagation and Adam optimization algorithm.
The invention has the beneficial effects that:
(1) the method defines the entity relationship types existing in 8 chapter-level financial long texts, and can greatly improve the modeling of a mechanistic/deep learning model on entity relationship information in the model;
(2) according to the method, a set of hierarchical financial event structure is defined according to the extracted categories of financial events, so that the machine learning/deep learning model can be helped to better construct the relationship between the categories of the events, and similar events can be distinguished more accurately;
(3) the hierarchical financial event attention mechanism and the structured entity dependence self-attention mechanism designed by the invention integrate the two information into the entity semantics and entity relationship modeling of the model, and the model can carry out higher precision to obtain the trigger words and type prediction and argument and type prediction of the financial event in a joint training mode;
(4) the invention provides a novel method for extracting a financial long text event from end to end (from a financial long text end to a structured event end), which can effectively improve the extraction efficiency of a financial institution on real-time long text core elements and improve the processing capacity of the financial institution on massive financial text information.
Drawings
FIG. 1 is a general structure diagram of a financial field chapter-level event extraction method based on article entity word dependency;
FIG. 2 is an entity dependency matrixsA schematic diagram;
fig. 3 is a schematic diagram of a structured self-attention module.
Detailed Description
The invention relates to a financial field chapter-level event extraction method based on article entity word dependency, which starts with the analysis of actual financial field chapter-level announcements, summarizes the actual entity distribution conditions of the financial field chapter-level announcements, and summarizes and defines the entity dependency types of 8 chapter-level texts; vectorizing a chapter-level text by using a pre-training language model based on deep learning to obtain text features fused with bidirectional semantics; fusing predefined hierarchical event types by using a hierarchical financial event attention mechanism to generate an importance weight for each candidate argument in a candidate argument set, and finally generating a hierarchical financial event characteristic; converting entity dependence information corresponding to the chapter-level text into entity relationship characteristics which can be fused with vectorized text characteristics by utilizing a structured entity dependence self-attention mechanism; generating text features integrating the long text semantic relationship by using a Peer Attention mechanism and taking chapter-level vectorized text features as input; the financial time elements of the long text at chapter level are simultaneously output by utilizing the global constraint condition defined by the invention and two parallel prediction modules: the financial event triggers a collection of arguments associated with the financial event. As shown in fig. 1, the method specifically comprises the following steps:
(1) and (3) entity word extraction: converting a Chinese article to contain n entitiese i Entity set ofE={e 1,e 2,e 3,……,e n }，e i Is as followsiThe number of the individual entities,i=1,2,3,…,n. This step results in a string of unlabeled entity sequences.
(2) Entity word vector mapping module: using Embedding mapping method to collect entityEThe Chinese entity is mapped into an entity word vector set of a vector space, and the entity word vector
Figure 7121DEST_PATH_IMAGE003
Comprises the following steps:
Figure 486644DEST_PATH_IMAGE036
wherein the content of the first and second substances,LMin order to be a function of the vector mapping,W e to generate a physical word vector
Figure 313655DEST_PATH_IMAGE003
The trainable parameters of (a).
(3) Extracting a candidate theory element set: before the event extraction calculation, the invention utilizes a sequence labeling method based on a pre-training language model to extract a candidate element set for the article. Due to any unstructured financial articles (bulletins/news), etc., there is no meaningful set of candidate arguments; therefore, before the event extraction, the candidate argument extraction is carried out, and the method comprises the following sub-steps:
and (3.1) carrying out entity tagging on the target article by using a BIO tagging strategy, wherein the entity tagging comprises a trigger word, an argument and a non-entity word.
And (3.2) performing candidate argument extraction on the article (entity word vector set) labeled in the step (3.1) by using a sequence labeling model based on a pre-training language model BERT.
(3.3) outputting a candidate argument set based on a sequence tagging model of a pre-training language model BERTA={a 1,a 2,……,a K }; wherein the content of the first and second substances,
Figure 144207DEST_PATH_IMAGE020
in order to be a candidate argument of argument,j t =1~K；Ais a subset of E.
(4) And obtaining an entity set fused with the dependency features of the structured entities based on the structured self-attention module.
And (4.1) constructing entity dependence relations.
The entity set obtained according to the step (1)E={e 1,e 2,e 3,……,e n And their location in the article, the present invention defines an entity dependency setSIncluding eight different types of entity dependencies.
The invention is based on the analysis of the text of the financial chapters of the Chinese language to obtain the following three conclusions:
(i) the trigger of an event may appear in multiple sentences in an article;
(ii) the same argument of an event may appear in multiple sentences in an article;
(iii) different arguments of an event may appear in one sentence at the same time or be distributed in different sentences.
Based on the analysis results and according to the positions of the entities in the article, the invention divides eight entity dependencies as follows:
(a) the trigger words of the event appear in the same sentence for a plurality of times and are marked as 'common fingers in the trigger words and sentences';
(b) trigger words of events appear in different sentences and are marked as 'common reference between trigger words and sentences';
(c) one argument of an event appears in the same sentence for multiple times and is marked as 'common reference in an argument sentence';
(d) one argument of an event appears in different sentences, marked as "intersentence coreference";
(e) the trigger words and the arguments of the events appear in the same statement, and are marked as 'related in an entity sentence', representing that the pair of trigger words and the argument set are related by certain predicate semantics;
(f) trigger words and arguments of events occur in different sentences, marked as "inter-sentence correlation of entities";
(g) the dependency relationship of the non-entity word and the trigger word/argument in the same sentence is defined as 'other correlations in the sentence';
(h) other types of relationships between non-entity words and trigger words/arguments are labeled as "independent" since they do not contain critical semantic and dependency information.
Thus, the present invention formalizes the structured entity dependencies as eight entity-centric matrices whose elements originate from a finite set of entity relationships，I.e. entity dependency setSThe phrase "trigger words and sentences co-refer", "argument sentences co-refer", "entity sentences are related", "sentence is related to other", "no dependency" }.
Defining entity dependence relationship on financial chapter-level text to obtain two-dimensional matrixsThe method specifically comprises the following steps: dependent on entity dependency setSFor each sentenceConstruct a two-dimensional matrix for the pairs (sentences S1 and S2)sAccording to the eight entity dependency relationships defined by the invention and the positions of the different types of entities in the sentence, the entities at different positions are subjected to relationship marking. Matrix arraysThe specific structure of (1) is shown in fig. 2, wherein "N T1N A2 a 1N A2T A3" is two sentences composed of different entities; wherein, "N T1N A2A 1N" is the sentence S1, and "A2T A3" is the sentence S2; n is a non-entity word, T1 is a trigger word of a certain class, and A1 and A2 are two different types of arguments. For example, the entity T1 in the second row and the entity N in the first column satisfy the entity dependency g of "other intra-sentence related"; the entity T1 in the second row satisfies the entity dependency a of "co-referent within trigger sentence" with itself (the entity T1 in the second column).
(4.2) after the entity dependency relationship of the financial chapter-level text is defined, the structured self-attention module constructs a structured entity dependency feature and outputs an entity set fused with the structured entity dependency feature, as shown in fig. 3. The structured self-attention module is a computing unit which can be repeatedly used by superposition, and the input of the module is entity vector characteristics and an entity dependency relationship matrixsAnd 1 is firstl-the output of 1 structured self-attention module islThe entity vector characteristics input by the structured self-attention module; and (3) the first structured entity vector feature input from the attention module is the entity word vector set obtained in the step (2). The structured self-attention module can be combined with any language model.
(4.2.1) integrating the entity i, j and the entity dependency relationship to obtain the vector feature fusing the entity word feature and the relationship dependency feature
Figure 324653DEST_PATH_IMAGE008
。
(4.2.1.1) to matrix entity dependenciessIntegrated into the end-to-end financial event extraction model processing process, the invention performs specific parameter instantiation on eight types of entity dependency relations:
Figure 799628DEST_PATH_IMAGE037
wherein the two-dimensional matrixsElement (1) ofs i j,Representing entitiesiAndjj = 1-n;
Figure 672906DEST_PATH_IMAGE005
is shown aslGenerated by a structured self-attention modules i j,Vectorization features that satisfy certain entity dependencies;
Figure 990755DEST_PATH_IMAGE006
is the firstlA structured self-attention module pairs i j,Training parameters for the transformation are performed.
(4.2.1.2) the invention utilizes bidirectional affine transformation to transform vector representation of discourse level text, and generates structural entity dependency relationship characteristics through calculation of the following formula
Figure 833946DEST_PATH_IMAGE008
：
Figure 553640DEST_PATH_IMAGE038
Wherein the content of the first and second substances,
Figure 332240DEST_PATH_IMAGE008
is the firstlThe structured entity dependency characteristics of each structured self-attention module,
Figure 12751DEST_PATH_IMAGE009
and
Figure 800579DEST_PATH_IMAGE010
are respectively input tolAn entity of a structured self-attention moduleiAndjthe vectorization feature of (a) is,
Figure 374780DEST_PATH_IMAGE011
is as followslTrainable parameters of a structured self-attention module.
(4.2.2) through a feature merging layer, the jth structured entity vector feature input by the attention module corresponding to the entity i and the structured entity dependency relationship feature obtained in the step (4.2.1.2) are combined
Figure 448915DEST_PATH_IMAGE008
Merging; then, performing first normalization through a normalization layer based on the entity vector characteristics input by the jth structured self-attention module; and finally, carrying out second normalization through a normalization layer based on a result of the first normalization to obtain the output of the jth structured self-attention module, wherein the output of the last structured self-attention module is the vector characteristic of the entity set fused with the dependency characteristic of the structured entities.
(5) Extracting a candidate trigger word set: the same principle as the extraction of the candidate argument set in the step (3) is carried out, the entity set which is output in the step (4.2.2) and fused with the dependency characteristics of the structured entities is taken as input, a BIO labeling strategy is adopted to be combined with a sequence labeling model based on a pre-training language model, candidate trigger word entities are extracted from the labeled articles, and a candidate trigger word set is outputT={T 1,T 2,……,T u Therein ofTIs a subset of E.
(6) The hierarchical financial event attention module is used for constructing a hierarchical financial event feature.
A specific financial event representation incorporating a broad range of financial event characteristics is generated using a predefined hierarchical financial event structure in conjunction with an attention mechanism model. The invention divides fourteen types of subdivided financial events into eight types of financial events, and the hierarchical financial event structure is as follows:
Figure 475777DEST_PATH_IMAGE039
the present invention utilizes linear transformation functions for eight major classes of events and fourteen subdivided events in stepsAnd (5) taking the extracted candidate trigger word set as input, generating transformed vector features, and respectively corresponding to each large event and each subdivided event to obtain the large event features and the subdivided event features. Then, the major event features and the subdivided event features generate importance weighted values for all candidate arguments through a hierarchical event attention mechanism respectively
Figure 801716DEST_PATH_IMAGE012
And
Figure 105789DEST_PATH_IMAGE013
(ii) a Wherein the content of the first and second substances,
Figure 491771DEST_PATH_IMAGE012
is a candidate trigger word
Figure 271509DEST_PATH_IMAGE014
Candidate argument importance weight values generated by events belonging to the broad category,
Figure 260193DEST_PATH_IMAGE013
is a candidate trigger word
Figure 543407DEST_PATH_IMAGE014
Candidate argument importance weighted values generated by the belonged subdivision events; finally, the following method is adopted to obtain the attention weight of the hierarchical financial event characteristics
Figure 365869DEST_PATH_IMAGE015
：
Figure 988829DEST_PATH_IMAGE040
Wherein the content of the first and second substances,
Figure 922150DEST_PATH_IMAGE014
is the ithtVector characteristics of individual candidate trigger words, it=1~u。
To obtain the ithtAttention weight corresponding to each candidate trigger word
Figure 59870DEST_PATH_IMAGE015
Later, hierarchical financial event features
Figure 912289DEST_PATH_IMAGE017
Obtained by the following calculation:
Figure 401039DEST_PATH_IMAGE041
wherein the content of the first and second substances,
Figure 872472DEST_PATH_IMAGE017
is the ithtThe hierarchical financial event characteristics corresponding to the candidate trigger words,
Figure 740065DEST_PATH_IMAGE019
as candidate argument
Figure 904330DEST_PATH_IMAGE020
Is used to represent the vector of (a),
Figure 614797DEST_PATH_IMAGE021
is that
Figure 748975DEST_PATH_IMAGE015
Middle candidate trigger wordi t And candidate argumentj t The weight relationship of (a) to (b),M e andb e are trainable parameters of the hierarchical financial event attention module.
(7) And (3) based on the entity word vector set obtained in the step (2), jointly predicting event trigger words and event arguments based on a Peer Attention mechanism.
(7.1) the pendant authorization mechanism module: in order to solve the problem that the deep learning model is difficult to capture the semantic relationship between semantic units in long text under space-chapter long text, the invention adopts the Peer Attention mechanism and uses an entityiSet of contiguous entities ofN i And entityiAndN i set of edges ofD i As input and output entitiesiAnd entitiesjSemantic relationship features between
Figure 126867DEST_PATH_IMAGE022
：
Figure 196454DEST_PATH_IMAGE042
Wherein the content of the first and second substances,Multi_headis a functional representation of a multi-headed self-attentive module,
Figure 269583DEST_PATH_IMAGE024
is an entityjA vector representation of (a);
Figure 82818DEST_PATH_IMAGE025
is thatN i To middlepThe number of the individual entities,
Figure 315216DEST_PATH_IMAGE026
is that
Figure 414760DEST_PATH_IMAGE025
Is used to represent the vector of (a),
Figure 365398DEST_PATH_IMAGE027
is an entityjAnd
Figure 716745DEST_PATH_IMAGE025
vector representation of edges in between;p=1~q。
(7.2) argument extraction module and trigger word extraction module: the invention combines the candidate argument set generated by the above stepsACandidate trigger word setTHierarchical financial event features
Figure 679016DEST_PATH_IMAGE017
And semantic relationship features between entities
Figure 355985DEST_PATH_IMAGE022
And two independent Full-connected modules (Full connection layers) are adopted to construct a trigger word extraction module and an argument extraction module. Wherein, the argument is extracted from the module
Figure 262761DEST_PATH_IMAGE017
、
Figure 542433DEST_PATH_IMAGE022
、A、TGenerating a probability set corresponding to the candidate argument for input; trigger the word extraction module to
Figure 749423DEST_PATH_IMAGE017
、
Figure 66135DEST_PATH_IMAGE022
AndTfor input, a set of probabilities corresponding to the candidate trigger words is generated.
(7.3) a joint prediction module: the invention constructs a correlation probability matrix based on a candidate trigger word probability set and a candidate argument probability setW e2a Joint prediction probability of event trigger and argument of final modelP(event|D) The definition is as follows:
Figure DEST_PATH_IMAGE043
wherein D represents a Chinese financial article, event represents a financial event,
Figure 335573DEST_PATH_IMAGE029
in the representative article D
Figure 28723DEST_PATH_IMAGE030
The probability of an event occurring;
Figure 214853DEST_PATH_IMAGE031
as candidate trigger word
Figure 233625DEST_PATH_IMAGE014
K corresponds to a subdivided financial event type,
Figure 114993DEST_PATH_IMAGE032
representing candidate triggers in article D
Figure 487200DEST_PATH_IMAGE014
Belong to the firstkProbability of class-breaking financial events;
Figure 403203DEST_PATH_IMAGE033
is a candidate trigger word
Figure 592876DEST_PATH_IMAGE014
A set of contiguous entities;
Figure 86175DEST_PATH_IMAGE044
representing candidate trigger words
Figure 386706DEST_PATH_IMAGE014
The corresponding event type contains argument type
Figure 891637DEST_PATH_IMAGE034
The probability of (d);
Figure 127577DEST_PATH_IMAGE034
the type of the argument is represented and,
Figure 249117DEST_PATH_IMAGE035
to be in a candidate trigger word
Figure 87760DEST_PATH_IMAGE014
And candidate arguments
Figure 837410DEST_PATH_IMAGE020
Article D of (1), candidate arguments
Figure 368885DEST_PATH_IMAGE020
Is of the type
Figure 446563DEST_PATH_IMAGE034
The probability of (c).
Finally, through two parallel output layers, a trigger word output module and an argument output module, the trigger words of events in the article are predicted simultaneouslyT’And argument set constituting eventA’。
(8) The invention trains the model by using a supervised machine learning training mode, a training sample consists of announcements and news published by various listed companies, errors of an event structure predicted by the model and a real event structure are adopted in the training process, and trainable parameters involved in the steps (2) to (7) are updated by combining an error back propagation mechanism. The method comprises the following steps:
and performing model training by taking the pre-collected financial document data as a training sample. Wherein, all text contents of the financial document are taken as input data and are sequentially input into the steps (1) - (3); and (4) calculating through the steps (4) to (7), and finally outputting a trigger word forming the financial event and an argument set related to the event simultaneously by the combined prediction module. Comparing the structured event generated by the model with the real event contained in the text to generate an error, applying the information contained in the error value to all the involved trainable parameters in the steps (2) - (7) by using a back propagation algorithm and an optimization algorithm, and updating and optimizing the model parameters. And (3) repeatedly executing the steps (2) to (7) by utilizing the financial document data to finally obtain parameters of a structured Attention module, an entity word vector mapping module, a hierarchical financial event Attention module, a Peer Attention mechanism module, an argument extraction module, a trigger word extraction module and a joint prediction module.
(9) Inputting a financial document to be identified, and predicting to obtain a trigger word forming a financial event and an argument set related to the event through the processing of the steps (1) to (7).

Claims (8)

1. A financial field chapter-level event extraction method based on article entity word dependency relationship is characterized by comprising the following steps:
(1) and (3) entity word extraction: chinese articleConverting to an entity set E ═ E containing n entities1,e2,e3,……,en}；
(2) Entity word vector mapping module: mapping Chinese entities of the entity set E into an entity word vector set of a vector space by using an Embedding mapping method, wherein the entity word vectors are as follows:
Figure FDA0003224194890000011
wherein i is 1 to n, eiFor the ith entity, LM is the vector mapping function, WeTo generate a physical word vector
Figure FDA0003224194890000012
Trainable parameters of (a);
(3) extracting candidate argument sets: extracting candidate theorem set A ═ a of article by sequence marking method based on pre-training language model1,a2,……,aK}；
(4) Building an entity dependency and structured self-attention module: according to entity word E ═ { E ═ E in entity set1,e2,e3,……,enConstructing entity dependency relationships of different types according to the positions of the entity dependency relationships in the articles; the structured self-attention module constructs a structured entity dependence characteristic and outputs an entity set fused with the structured entity dependence characteristic;
(5) extracting a candidate trigger word set: taking an entity set fused with the dependency characteristics of the structured entities as input, and extracting a candidate trigger word set T ═ T of an article by using a sequence tagging method based on a pre-training language model1,T2,……,Tu}；
(6) A hierarchical financial event attention module: generating a hierarchical financial event characteristic by utilizing a predefined hierarchical financial event structure and combining an attention mechanism model;
(7) event trigger word and event argument joint prediction based on the Peer Attention mechanism: the Peer Attention mechanism module obtains semantic relationship characteristics among the entities based on the entity word vector set obtained in the step (2); the argument extraction module is used for obtaining a probability set corresponding to candidate arguments based on the candidate argument set, the candidate trigger word set, the hierarchical financial event characteristics and the semantic relation characteristics among the entities; the trigger word extraction module obtains a probability set corresponding to the candidate trigger words based on the candidate trigger word set, the hierarchical financial event characteristics and the semantic relation characteristics among the entities; the joint prediction module predicts a trigger word T 'of an event in an article and an argument set A' forming the event based on the two probability sets;
the step (3) is specifically as follows:
(3.1) carrying out entity annotation by using a BIO annotation strategy;
(3.2) extracting candidate arguments by using a sequence tagging model based on a pre-training language model BERT;
(3.3) outputting candidate theorem set A ═ { a ═ a1,a2,……,aK}；
The step (6) is specifically as follows:
the hierarchical financial event structure divides a plurality of subdivided events in the financial events into different large events, and specifically comprises the following steps: the major events include financing, trading, stock increase and decrease, financial index change, multi-party cooperation, personnel change, marketing correlation and law enforcement; the financing comprises the quality assurance, the quality assurance and the enterprise borrowing, the transaction comprises the share repurchase and the enterprise mergence, the share increase and decrease support comprises the share decrease support and the share increase support, the financial index change comprises the loss, the multi-party cooperation comprises the bid-winning, the personnel change comprises the high-level manager change, the marketing correlation comprises the company marketing and the bankruptcy clearing, and the law enforcement comprises the contracted negotiation and the punishment;
based on the candidate trigger word set extracted in the step (5), respectively processing the large event features and the subdivided event features, and respectively generating importance weighted values for all candidate arguments through a hierarchical event attention mechanism
Figure FDA0003224194890000021
And
Figure FDA0003224194890000022
wherein the content of the first and second substances,
Figure FDA0003224194890000023
is a candidate trigger word
Figure FDA0003224194890000024
Candidate argument importance weight values generated by events belonging to the broad category,
Figure FDA0003224194890000025
is a candidate trigger word
Figure FDA0003224194890000026
Candidate argument importance weighted values generated by the belonged subdivision events; obtaining the ith financial event structure based on the hierarchical financial event structuretAttention weight corresponding to each candidate trigger word
Figure FDA0003224194890000027
Figure FDA0003224194890000028
IthtHierarchical financial event features corresponding to candidate trigger words
Figure FDA0003224194890000029
Obtained by the following calculation:
Figure FDA00032241948900000210
wherein the content of the first and second substances,
Figure FDA00032241948900000211
as candidate argument
Figure FDA00032241948900000212
Is used to represent the vector of (a),
Figure FDA00032241948900000213
is that
Figure FDA00032241948900000214
Middle candidate trigger word itAnd candidate argument jtWeight relationship of, MeAnd beAre trainable parameters of the hierarchical financial event attention module.
2. The method for extracting events at the chapter level in the financial field based on the article entity word dependency relationship as claimed in claim 1, wherein the step (4) is as follows:
(4.1) constructing different types of entity dependency relationships, and defining the entity dependency relationships of the financial chapter-level text to obtain an entity dependency relationship matrix s, wherein the entity dependency relationship matrix s comprises the following steps:
(a) the trigger words of the event appear in the same sentence for a plurality of times and are marked as 'common fingers in the trigger words and sentences';
(b) trigger words of events appear in different sentences and are marked as 'common reference between trigger words and sentences';
(c) one argument of an event appears in the same sentence for multiple times and is marked as 'common reference in an argument sentence';
(d) one argument of an event appears in different sentences, marked as "intersentence coreference";
(e) the trigger words and arguments of the event appear in the same sentence and are marked as 'related in entity sentence';
(f) trigger words and arguments of events occur in different sentences, marked as "inter-sentence correlation of entities";
(g) the dependency relationship of the non-entity word and the trigger word/argument in the same sentence is marked as 'other correlations in the sentence';
(h) the relationship of other types of non-entity words to trigger words/arguments, labeled "no dependency";
(4.2) constructing a structural entity dependence characteristic by a structural self-attention module, and outputting an entity set fused with the structural entity dependence characteristic; the structured self-attention module is a computing unit which can be repeatedly used in a superposition manner, the input of the module is entity vector characteristics and an entity dependency relationship matrix s, and the output of the l-1 th structured self-attention module is the entity vector characteristics input by the l-th structured self-attention module; and (3) the first structured entity vector feature input from the attention module is the entity word vector set obtained in the step (2).
3. The method for extracting events at the chapter level in the financial field based on the article entity word dependency relationship as claimed in claim 2, wherein the step (4.2) is as follows:
parameter instantiation is carried out on the entity dependence relationship:
Figure FDA0003224194890000031
wherein the elements s in the matrix si,jRepresenting entity dependency relationship between entities i and j, j being 1 to n;
Figure FDA0003224194890000032
representing the l structured self-attention Module generated si,jVectorization features that satisfy certain entity dependencies;
Figure FDA0003224194890000033
is the l th pair of structured self-attention modules si,jTraining parameters for the transformation;
generating vector characteristics integrating entity dependency relationships by utilizing bidirectional affine transformation:
Figure FDA0003224194890000034
wherein the content of the first and second substances,
Figure FDA0003224194890000035
is the structured entity dependency feature of the ith structured self-attention module,
Figure FDA0003224194890000036
and
Figure FDA0003224194890000037
the vectorized features of entities i and j respectively that are input into the ith structured self-attention module,
Figure FDA0003224194890000038
trainable parameters for the first structured self-attention module;
combining the entity vector characteristics and the structured entity dependency relationship characteristics input by the current structured self-attention module, sequentially passing through the characteristic merging layer of the backward normalization layer and the full connection layer of the backward normalization layer to obtain the output of the current structured self-attention module, inputting the output into the next structured self-attention module, and finally outputting the last structured self-attention module as the vector characteristics of the entity set fused with the structured entity dependency characteristics.
4. The method for extracting events at the chapter level in the financial field based on the article entity word dependency relationship as claimed in claim 1, wherein the step (5) is specifically as follows: taking the entity set which is output in the step (4.2.2) and fused with the dependency characteristics of the structural entities as input, adopting a BIO labeling strategy and combining a sequence labeling model based on a pre-training language model, extracting candidate trigger word entities from the labeled articles, and outputting a candidate trigger word set T ═ T { (T {)1,T2,……,Tu}。
5. The method for extracting events at the chapter level in the financial field based on the word dependency relationship of the article entities as claimed in claim 1, wherein in step (7), the Peer Attention mechanism module uses the adjacent entity set N of the entity iiAnd entities i and NiSet of edges D ofiOutputting semantic relations between entity i and entity j for inputIs characterized by
Figure FDA0003224194890000039
Figure FDA00032241948900000310
Where Multi _ head is a functional representation of a Multi-head self-attention module,
Figure FDA00032241948900000311
is a vector representation of entity j;
Figure FDA00032241948900000312
is NiThe p-th entity in (1),
Figure FDA00032241948900000313
is that
Figure FDA00032241948900000314
Is used to represent the vector of (a),
Figure FDA00032241948900000315
is entity j and
Figure FDA00032241948900000316
vector representation of edges in between; p is 1 to q.
6. The method for extracting events at the chapter level of the financial field sections based on the article entity word dependency relationship as claimed in claim 1, wherein in the step (7), the argument extraction module and the trigger word extraction module are two independent full connection layers.
7. The method for extracting events at the chapter level in the financial field based on the article entity word dependency relationship as claimed in claim 1, wherein in step (7), the joint prediction module defines a joint prediction probability P (event | D) of event trigger words and arguments as follows:
Figure FDA0003224194890000041
wherein D represents a Chinese article, event represents a financial event, and P (event | D) represents the probability of occurrence of the event in the article D;
Figure FDA0003224194890000042
as candidate trigger word
Figure FDA0003224194890000043
K corresponds to the type of subdivision event,
Figure FDA0003224194890000044
representing candidate triggers in article D
Figure FDA0003224194890000045
Probability of belonging to class k subdivision events;
Figure FDA0003224194890000046
is a candidate trigger word
Figure FDA0003224194890000047
A set of contiguous entities;
Figure FDA0003224194890000048
representing the candidate trigger words by a correlation probability matrix constructed based on the candidate trigger word probability set and the candidate argument probability set
Figure FDA0003224194890000049
The corresponding event type contains argument type
Figure FDA00032241948900000410
The probability of (d);
Figure FDA00032241948900000411
the type of the argument is represented and,
Figure FDA00032241948900000412
to be in a candidate trigger word
Figure FDA00032241948900000413
And candidate arguments
Figure FDA00032241948900000414
Article D of (1), candidate arguments
Figure FDA00032241948900000415
Is of the type
Figure FDA00032241948900000416
The probability of (d); finally, the trigger words T 'of the events in the text and the argument sets A' forming the events are predicted through two parallel output layers.
8. The method for extracting events at the chapter level in the financial field based on article entity word dependency relationship as claimed in claim 1, wherein parameters of a structured self-Attention module, an entity word vector mapping module, a hierarchical financial event Attention module, a Peer Attention mechanism module, an argument extraction module, a trigger word extraction module, and a joint prediction module are trained by using a back propagation and Adam optimization algorithm.
CN202110648901.1A 2021-06-10 2021-06-10 Financial field chapter-level event extraction method based on article entity word dependency relationship Active CN113255321B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110648901.1A CN113255321B (en) 2021-06-10 2021-06-10 Financial field chapter-level event extraction method based on article entity word dependency relationship

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110648901.1A CN113255321B (en) 2021-06-10 2021-06-10 Financial field chapter-level event extraction method based on article entity word dependency relationship

Publications (2)

Publication Number Publication Date
CN113255321A CN113255321A (en) 2021-08-13
CN113255321B true CN113255321B (en) 2021-10-29

Family

ID=77187501

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110648901.1A Active CN113255321B (en) 2021-06-10 2021-06-10 Financial field chapter-level event extraction method based on article entity word dependency relationship

Country Status (1)

Country Link
CN (1) CN113255321B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113761936B (en) * 2021-08-19 2023-04-07 哈尔滨工业大学（威海） Multi-task chapter-level event extraction method based on multi-head self-attention mechanism
CN113779227B (en) * 2021-11-12 2022-01-25 成都数之联科技有限公司 Case fact extraction method, system, device and medium
CN115841105B (en) * 2022-09-21 2024-02-27 中国人民解放军海军工程大学 Event extraction method, system and medium based on event type hierarchical relationship
CN115983268B (en) * 2023-03-17 2023-06-30 北京澜舟科技有限公司 Financial event chain analysis method and computer readable storage medium

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111859935B (en) * 2020-07-03 2022-09-20 大连理工大学 Method for constructing cancer-related biomedical event database based on literature
CN111783394B (en) * 2020-08-11 2024-03-08 深圳市北科瑞声科技股份有限公司 Training method of event extraction model, event extraction method, system and equipment
CN112163416B (en) * 2020-10-09 2021-11-02 北京理工大学 Event joint extraction method for merging syntactic and entity relation graph convolution network

Also Published As

Publication number Publication date
CN113255321A (en) 2021-08-13

Similar Documents

Publication Publication Date Title
Li et al. Incorporating stock prices and news sentiments for stock market prediction: A case of Hong Kong
CN113255321B (en) Financial field chapter-level event extraction method based on article entity word dependency relationship
CN109446338B (en) Neural network-based drug disease relation classification method
Fu et al. Listening to the investors: A novel framework for online lending default prediction using deep learning neural networks
CN112434535B (en) Element extraction method, device, equipment and storage medium based on multiple models
CN107688870B (en) Text stream input-based hierarchical factor visualization analysis method and device for deep neural network
Zhang et al. Aspect-based sentiment analysis for user reviews
CN112256866A (en) Text fine-grained emotion analysis method based on deep learning
Sadr et al. ACNN-TL: attention-based convolutional neural network coupling with transfer learning and contextualized word representation for enhancing the performance of sentiment classification
CN116383399A (en) Event public opinion risk prediction method and system
CN115329088B (en) Robustness analysis method of graph neural network event detection model
CN114896386A (en) Film comment semantic emotion analysis method and system based on BilSTM
Fang et al. Patent2Vec: Multi-view representation learning on patent-graphs for patent classification
Subbalakshmi et al. A Gravitational Search Algorithm Study on Text Summarization Using NLP
US20080301082A1 (en) Knowledge base comprising executable stories
CN116245107A (en) Electric power audit text entity identification method, device, equipment and storage medium
CN111259106A (en) Relation extraction method combining neural network and feature calculation
Wang et al. Simplified representation learning model based on parameter-sharing for knowledge graph completion
Chen Semantic matching efficiency of supply and demand text on cross-border E-commerce online technology trading platforms
Cheng et al. BHONEM: Binary high-order network embedding methods for networked-guarantee loans
Khashabi On the recursive neural networks for relation extraction and entity recognition
Kaur et al. Performance Analysis of Terrain Classifiers Using Different Packages
Irawan et al. State of the art of machine learning: An overview of the past, current, and the future research trends in the era of quantum computing
Shi et al. Prompted and integrated textual information enhancing aspect-based sentiment analysis
Song et al. Prior-guided multi-scale fusion transformer for face attribute recognition

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant