CN116578665A - Method and equipment for jointly extracting extensible text information based on prompt learning - Google Patents

Abstract

The invention discloses a method and equipment for extracting extensible text information based on prompt learning, belonging to the technical field of text information extraction in natural language processing, and comprising the following steps: constructing a unified representation frame of text information extraction annotation information; constructing a unified prompt template based on the original data set and the extended data set; constructing and training a combined extraction model based on prompt learning text information; and extracting joint information from the input text by using the trained model. The invention solves the technical bottleneck brought by data expansion or cross-domain, and improves the accuracy and the robustness of joint information extraction.

Description

Method and equipment for jointly extracting extensible text information based on prompt learning

Technical Field

The invention relates to the technical field of text information extraction in natural language processing, in particular to a method and equipment for extracting extensible text information jointly based on prompt learning.

Background

In recent years, with the popularization of information transmission software of the internet and mobile terminals, necessary text information is extracted from massive text data, so that text resources can be better managed, and downstream services such as information recommendation, knowledge graph construction, text data recording and the like can be assisted. However, the current text information extraction method has various problems, among which the following are highlighted:

(1) The joint extraction of different types of text information (such as entities, relations, events and the like) cannot be performed;

(2) When data defined by different text information structures is faced, especially cross-domain data occurs or new text information structures are added, unified information extraction cannot be performed.

The above two problems greatly limit the performance of information extraction in practical application scenarios.

Disclosure of Invention

The invention aims to overcome the defects of the prior art, provides an expandable text information joint extraction method and equipment based on prompt learning, solves the technical bottleneck brought by data expansion or cross-field, and improves the accuracy, the robustness and the like of joint information extraction.

The invention aims at realizing the following scheme:

a scalable text information joint extraction method based on prompt learning comprises the following steps:

A. constructing a unified representation frame of text information extraction annotation information;

B. constructing a unified prompt template based on the original data set and the extended data set;

C. constructing and training a combined extraction model based on prompt learning text information;

D. and extracting joint information from the input text by using the trained model.

Further, in the step a, the step of constructing a unified representation framework of the text information extraction annotation information includes the following sub-steps:

A1. aiming at a named entity recognition task, constructing a labeling information representation framework as follows: (entity_type: span_e), wherein the entity_type represents a selected one of the Entity type names, and span_e represents a text segment in the text belonging to the Entity of the Entity type;

A2. aiming at entity relation extraction tasks, constructing a labeling information representation framework as follows: (EntityType: span_e (relationship_type_i: span_r_i)), wherein relationship_type_i represents an i-th Entity relationship type name associated with the Entity type of the Entity type, span_r_i represents a tail Entity text segment to which span_e corresponds under relationship_type_i;

A3. aiming at event extraction tasks, constructing a labeling information representation framework as follows: (event_type: span_v (rule_type_i: span_o_i)), wherein event_type represents a selected one of Event type names, span_v represents a text segment of an Event trigger word belonging to the event_type Event type in the text, rule_type_i represents an i-th Event Role name associated with the event_type Event, span_o_i represents a corresponding Event Role text segment of span_v under rule_type_i;

A4. And merging named entity identification, entity relation extraction and event extraction annotation information representation frames to form a unified representation frame of text information extraction annotation information.

Further, in the step B, the constructing a unified hint template based on the original data set and the extended data set includes the following sub-steps:

B1. defining the original data set as D _ori ＝{(S _i ，L _i )，i＝1，...，N _ori}, wherein ,S_i Text representing the ith data of the original dataset, L _i Annotation information representing the ith data of the original dataset, N _ori Representing the number of original data samples;

B2. defining an extended dataset as D _ext ＝{(S _j ，L _j )，j＝1，...，N _ext}, wherein ,S_j Text representing the j-th data of the expanded dataset, L _j Annotation information representing the j-th data of the extended data set, N _ext Representing the number of extended data samples;

B3. and fusing the original data set and the extended data set, and constructing a unified prompt template for each piece of labeling information.

Further, in the step C, the building and training of the joint extraction model based on prompt learning text information includes the following sub-steps:

C1. setting an encoder based on a transducer network architecture, wherein the output dimension is d _E Denoted as E _model ；

C2. Constructing a full-connection layer network module aiming at the initial position of a text fragment, which is marked as FC _start Constructing a full connection layer network module aiming at a text fragment termination position, which is marked as FC _end ，FC _start With FC _end Is d _E The output dimension is 1;

C3. for the specified input triplet data (S _k ，P _k ，L _k ) According to P _k S _k Performing prompt and text splicing, and marking the length of the spliced text as d _k ；

C4. Completing the blank characters of the spliced text, namely adding a nonsensical blank character sequence Q after splicing the text _k So that P _k S _k Q _k The character length reaches d _* ＝max(d _k ，k＝1，...，K)；

C5. Construction PkS _k Q _k Mask vector T _k ＝(t ₁ ，...，t _d*), wherein ,

C6. use E _model Pair PkS _k Q _k Coding, and marking the coded characteristic matrix as

C7. Based on L _k At S _k The starting position and the ending position in the sequence, and constructing a reference result vector GT _{k_start} 、 wherein ,GT_{k_start} Remove L _k At S _k The starting position of the catalyst is 1, the rest are 0, GT _{k_endt} Remove L _k At S _k The rest of the terminal positions are 0 except 1;

C8. setting the maximum iteration number EP, and recording the current completed iteration number as EP;

C9. calculating the minimum information entropy threshold value rho=exp [ EP/(EP+1) acceptable by the model under the current iteration number] ^-1 ；

C10. Will F _k Input FC respectively _start and FC_end Activating by adopting sigmoid to obtain prediction vectors P respectively _start and P_end ；

C11. Respectively calculate P _start and P_end Information entropy, denoted as θ _start and θ_end ；

C12. If theta is _start < ρ and θ _end < ρ, then based on P _start and P_end Respectively with GT _{k_start} and GT_{k_end} Calculating cross entropy loss and performing model training by adopting back propagation; otherwise, skipping the current data;

C13. C3 to C12 are executed on all the K groups of data, and after one round of training of the K groups of data is completed, the ep value is increased by 1;

C14. and repeatedly executing C13 until ep=EP, and marking the trained text information joint extraction Model as Model.

Further, in step D, the step of extracting joint information from the input text by using the trained model includes the following sub-steps:

D1. setting an output threshold delta epsilon (0, 1);

D2. traversing all Entity class names { entity_type (Q), q=1,..once, Q }, which is the number of all Entity class names, contained in the trained data for the input Text; the entity_type (q) is spliced with the Text and then input into a Model to obtain a named Entity recognition extraction prediction vector under the entity_type (q) so as to obtain P _start >Delta is used as a starting point, and the nearest P is used after the position _end >The delta position is the end point, and the text segment between the delta position and the delta position is used as the pre-preparation of the entity_type (q)Measuring an entity text fragment;

D3. traversing the input Text and the Entity Text fragments predicted in the step D2, and recording the Entity relationship type name { relation_type (G), g=1, G } associated with the Entity type to which the predicted Entity Text fragments belong, wherein G is the number of all Entity relationship types associated with the Entity type to which the entity_span belongs; the relation_type (g) of the' Entity_span is spliced with Text and then input into a Model, and a tail Entity associated with the Entity_span under the relation_type (g) is obtained to extract a prediction vector so as to obtain P _dtart >Delta is used as a starting point, and the nearest P is used after the position _end >The delta position is an ending point, and the text segment between the delta position and the ending point is used as a predicted tail Entity text segment of the entity_span under the relation_type (g);

D4. traversing all Event category names { event_type (H), h=1,..once, H }, H being the number of all Event category names, contained in the trained data, for the input Text; the event_type (h) and Text are spliced and then input into a Model to obtain an Event trigger word extraction prediction vector under the event_type (h) so as to obtain P _dtart >Delta is used as a starting point, and the nearest P is used after the position _end >The delta position is an ending point, and a text segment between the delta position and the ending point is used as a predicted Event trigger word text segment of event_type (h);

D5. traversing Event Role names { rotor_type (W), w=1, and W } associated with Event types to which event_span belongs for the input Text and the Event trigger word Text fragments predicted in the step D4, wherein W is the number of all Event roles associated with the Event types to which the event_span belongs; the rule_type (w) of the' event_span is spliced with Text and then input into a Model, so that an Event Role extraction prediction vector associated with the event_span under the rule_type (w) is obtained, and the Event Role extraction prediction vector is used as P _dtart >Delta is used as a starting point, and the nearest P is used after the position _end >The delta position is an ending point, and the text segment between the delta position and the ending point is used as an Event character text segment of event_span under the rule_type (w);

D6. and D2, fusing the predicted results of the steps D2 to D5 according to the unified characterization framework constructed in the step A as the result of the text information joint extraction.

Further, in the step B3, the following sub-steps are included:

and B3-1, constructing a named entity recognition task prompt template as follows:

{

'text' S

'hint' Entity_type

'result' span_e

}

S is text in an original data set or an extended data set, the entity_type is an Entity type name selected from the labeling information of the data set to which the S belongs, and span_e is an Entity text segment belonging to the entity_type in the S;

and B3-2, constructing an entity relation extraction task prompt template as follows:

{

'text' S

'hint' match_e relation_type

'result' span_r

}

S is a text in an original data set or an expanded data set, span_e is a text segment serving as a head entity of a certain Relation type in S, relation_type is an entity Relation type name selected in labeling information of the data set to which S belongs, span_r is a tail entity text segment with a selected Relation relation_type in S and span_e;

And B3-3, constructing an entity relation extraction task prompt template as follows:

{

'text' S

'prompt': event_type

'results' span_v

}

{

'text' S

'hint' Role_type of span_v

'result' span_o

}

The method comprises the steps that S is a text in an original data set or an extended data set, event_type is a specific Event type name in data set marking information to which S belongs, span_v is a text segment serving as a trigger word of a certain Event type in S, role_type is a specific Event Role name in data set marking information to which S belongs, span_o is a text segment of a role_type Role marked as event_type Event in S;

b3-4, reconstructing the data according to the steps B3-1 to B3-3 for all the labeling information in the fusion data, and randomly sampling text fragments in the labeling data and constructing negative sample data in all entity type names, relation type names, event type names and event role names, namely, a sample corresponding to a result cannot be queried; the data set after fusing, reconstructing and adding negative samples is denoted as D _mix ＝{(S _k ,P _k ,L _k ) K=1,.. _k Representing text, P _k Representation prompt, L _k The result is represented by K, the number of all triples of the dataset.

Further, in step B3-1, if there is no Entity text segment belonging to the entity_type in S, span_e is made empty string.

Further, in step B3-2, if there is no tail entity text segment of the relationship_type with span_e in S, span_r is made to be an empty string.

Further, in step B3-3, the two 'result' fields are made empty when no corresponding text segment is queried.

An expandable text information joint extraction device based on prompt learning, based on the method as set forth in any one of the above, further comprising:

the first construction module is used for constructing a unified representation frame of the text information extraction annotation information;

the second construction module is used for constructing a unified prompt template based on the original data set and the extended data set;

the third construction module is used for constructing and training a text information joint extraction model based on prompt learning;

and the extraction module is used for extracting the joint information of the input text by using the trained model.

The beneficial effects of the invention include:

according to the invention, a unified text information characterization framework is constructed on the basis of prompt learning, different types of information extraction tasks can be fused into a unified framework, and meanwhile, data sets under different definitions can be subjected to joint training, so that the technical bottleneck brought by data expansion or cross-field is solved; on the basis, an information entropy filtering mode is adopted to avoid negative influence of noise data on a model, meanwhile, data overfitting is relieved to a certain extent, and accuracy and robustness of joint information extraction are improved.

Drawings

In order to more clearly illustrate the embodiments of the invention or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the description below are only some embodiments of the invention, and that other drawings can be obtained according to these drawings without inventive faculty for a person skilled in the art.

Fig. 1 is a flowchart of an expandable text information joint extraction method based on prompt learning according to an embodiment of the present invention.

Detailed Description

All of the features disclosed in all of the embodiments of this specification, or all of the steps in any method or process disclosed implicitly, except for the mutually exclusive features and/or steps, may be combined and/or expanded and substituted in any way.

As shown in fig. 1, this embodiment proposes an expandable text information joint extraction method based on prompt learning, which includes the following steps:

A. constructing a unified representation frame of text information extraction annotation information;

B. constructing a unified prompt template based on the original data set and the extended data set;

C. constructing and training a combined extraction model based on prompt learning text information;

D. And extracting joint information from the input text by using the trained model.

In some embodiments, step a constructs a unified presentation framework for text information extraction annotation information comprising the sub-steps of:

A1. aiming at a named entity recognition task, constructing a labeling information representation framework as follows: (entity_type: span_e), wherein entity_type represents a specific Entity type name, span_e represents a text segment in the text belonging to the Entity of the Entity type;

A2. aiming at entity relation extraction tasks, constructing a labeling information representation framework as follows: (EntityType: span_e (relationship_type_i: span_r_i)), wherein relationship_type_i represents an i-th Entity relationship type name associated with the Entity type of the Entity type, span_r_i represents a tail Entity text segment to which span_e corresponds under relationship_type_i;

A3. aiming at event extraction tasks, constructing a labeling information representation framework as follows: (event_type: span_v (rule_type_i: span_o_i)), wherein event_type represents a specific Event type name, span_v represents a text segment of an Event trigger word belonging to the event_type Event type in the text, rule_type_i represents an i-th Event Role name associated with the event_type Event, span_o_i represents a corresponding Event Role text segment of span_v under rule_type_i;

A4. Fusion of named entity recognition, entity relation extraction and event extraction annotation information representation frames forms a unified representation frame of text information extraction annotation information, and one example form is as follows:

in some embodiments, step B of constructing a unified hint template based on the original dataset and the expanded dataset includes the sub-steps of:

B1. defining the original data set as D _ori ＝{(S _i ,L _i ),i＝1,...,N _ori}, wherein ,S_i Text representing the ith data of the original dataset, L _i Annotation information (unified representation framework organization structure according to step A) representing ith data of original data set, N _ori Representing the number of original data samples;

B2. defining an extended dataset as D _ext ＝{(S _j ,L _j ),j＝1,...,N _ext}, wherein ,S_j Text representing the j-th data of the expanded dataset, L _j Annotation information (unified frame organization according to step A) representing jth data of extended data set, N _ext Representing the number of extended data samples;

B3. and fusing the original data set and the extended data set, and constructing a unified prompt template for each piece of labeling information.

In some embodiments, step B3 comprises the sub-steps of:

and B3-1, constructing a named entity recognition task prompt template as follows:

{

'text' S

'hint' Entity_type

'result' span_e

}

Wherein S is a text in the original dataset or the extended dataset, the entity_type is a specific Entity type name in the dataset label information in S, and span_e is an Entity text segment belonging to the entity_type in S (in particular, if there is no Entity text segment belonging to the entity_type in S, let span_e be an empty string);

and B3-2, constructing an entity relation extraction task prompt template as follows:

{

'text' S

'hint' match_e relation_type

'result' span_r

}

Wherein S is a text in the original dataset or the extended dataset, span_e is a text segment in S as a header entity of a certain Relation type, relationship_type is a specific entity Relation type name in the dataset annotation information in S, span_r is a tail entity text segment in S having a specific Relation relationship relationship_type with span_e (particularly, if no tail entity text segment in S having a specific Relation relationship relationship_type with span_e, span_r is an empty string);

and B3-3, constructing an entity relation extraction task prompt template as follows:

{

'text' S

'prompt': event_type

'results' span_v

}

{

'text' S

'hint' Role_type of span_v

'result' span_o

}

Wherein S is a text in the original dataset or the extended dataset, event_type is a specific Event type name in the dataset label information to which S belongs, span_v is a text segment in S that is a trigger word for a certain Event type, role_type is a specific Event Role name in the dataset label information to which S belongs, span_o is a text segment in S that is labeled as a role_type Role for an event_type Event (in particular, two 'result' fields allow for blank character strings when no corresponding text segment is queried).

B3-4, reconstructing the data according to the steps B3-1 to B3-3 for all the labeling information in the fusion data, and randomly sampling text fragments in the labeling data and constructing negative sample data in all entity type names, relation type names, event type names and event role names, namely, a sample corresponding to a result cannot be queried; the data set after fusing, reconstructing and adding negative samples is denoted as D _mix ＝{(S _k ,P _k ,L _k ) K=1,.. _k Representing text, P _k Representation prompt, L _k The result is represented by K, the number of all triples of the dataset.

In some embodiments, step C of constructing and training a prompt-learning text-based information joint extraction model includes the sub-steps of:

C1. Setting an encoder based on a transducer network architecture, wherein the output dimension is d _E Denoted as E _model ；

C2. Constructing a full-connection layer network module aiming at the initial position of a text fragment, which is marked as FC _start Constructing a full connection layer network module aiming at a text fragment termination position, which is marked as FC _end ，FC _start With FC _end Is d _E The output dimension is 1;

C3. for the specified input triplet data (S _k ，P _k ，L _k ) According to P _k S _k Performing prompt and text splicing, and marking the length of the spliced text as d _k ；

C4. Completing the blank characters of the spliced text, namely adding a nonsensical blank character sequence Q after splicing the text _k So that PkS _k Q _k The character length reaches d _* ＝max(d _k ，k＝1，...，K)；

C5. Construction of P _k S _k Q _k Mask vector T _k ＝(t ₁ ，...，t _d*), wherein ,

C6. use E _model P pair P _k S _k Q _k Coding, and marking the coded characteristic matrix as

C7. Based on L _k At S _k The starting position and the ending position in the sequence, and constructing a reference result vector GT _{k_start} 、 wherein ,GT_{k_start} Remove L _k At S _k The starting position of the catalyst is 1, the rest are 0, GT _{k_endt} Remove L _k At S _k The rest of the terminal positions are 0 except 1;

C8. setting the maximum iteration number EP, and recording the current completed iteration number as EP;

C9. calculating the minimum information entropy threshold value rho=exp [ EP/(EP+1) acceptable by the model under the current iteration number] ^-1 ；

C10. Will F _k Input FC respectively _start and FC_end Activating by adopting sigmoid to obtain prediction vectors P respectively _start and P_end ；

C11. Respectively calculate P _start and P_end Information entropy, denoted as θ _start and θ_end ；

C12. If theta is _start < ρ and θ _end < ρ, then based on P _start and P_end Respectively with GT _{k_start} and GT_{k_end} Calculating cross entropy loss and performing model training by adopting back propagation; otherwise, skipping the current data;

C13. c3 to C12 are executed on all the K groups of data, and after one round of training of the K groups of data is completed, the ep value is increased by 1;

C14. and repeatedly executing C13 until ep=EP, and marking the trained text information joint extraction Model as Model.

In some embodiments, step D of extracting joint information for the input text using the trained model comprises the sub-steps of:

D1. setting an output threshold delta epsilon (0, 1)

D2. Traversing all Entity class names { entity_type (Q), q=1,..once, Q }, which is the number of all Entity class names, contained in the trained data for the input Text; the entity_type (q) is spliced with the Text and then input into a Model to obtain a named Entity recognition extraction prediction vector under the entity_type (q) so as to obtain P _start The position of > delta is taken as a starting point, and the nearest P behind the position is taken as a starting point _end The position of delta is an ending point, and the text segment between the two positions is used as a predicted Entity text segment of the entity_type (q);

D3. Traversing and predicting Entity class of Entity Text segment (denoted as entity_span) to which the input Text and the Entity Text segment predicted in step D2 belongEntity relationship type name { relation_type (G), g=1,..g }, G being the number of all Entity relationship types associated by the Entity type to which the entity_span belongs; the relation_type (g) of the' Entity_span is spliced with Text and then input into a Model, and a tail Entity associated with the Entity_span under the relation_type (g) is obtained to extract a prediction vector so as to obtain P _start >Delta is used as a starting point, and the nearest P is used after the position _end >The delta position is an ending point, and the text segment between the delta position and the ending point is used as a predicted tail Entity text segment of the entity_span under the relation_type (g);

D4. traversing all Event category names { event_type (H), h=1,..once, H }, H being the number of all Event category names, contained in the trained data, for the input Text; the event_type (h) and Text are spliced and then input into a Model to obtain an Event trigger word extraction prediction vector under the event_type (h) so as to obtain P _start >Delta is used as a starting point, and the nearest P is used after the position _end >The delta position is an ending point, and a text segment between the delta position and the ending point is used as a predicted Event trigger word text segment of event_type (h);

D5. Traversing Event Role names { rule_type (W) associated with Event types to which the predicted Event trigger word Text fragments (denoted as event_span) belong, w=1, & gt, W }, W being the number of all Event roles associated with the Event types to which the event_span belongs, for the input Text and the Event trigger word Text fragments predicted in step D4; the rule_type (w) of the' event_span is spliced with Text and then input into a Model, so that an Event Role extraction prediction vector associated with the event_span under the rule_type (w) is obtained, and the Event Role extraction prediction vector is used as P _start >Delta is used as a starting point, and the nearest P is used after the position _end >The delta position is an ending point, and the text segment between the delta position and the ending point is used as an Event character text segment of event_span under the rule_type (w);

D6. and D2, fusing the predicted results of the steps D2 to D5 according to the unified characterization framework constructed in the step A as the result of the text information joint extraction.

To sum up steps a to D, corresponding text information may be jointly extracted from the given text, for example: naming an entity, entity relationship, and event. According to the invention, a unified text information characterization architecture is constructed, different types of information extraction tasks can be fused into a unified framework, and meanwhile, data sets under different definitions can be subjected to joint training, so that the technical bottleneck brought by data expansion or cross-field is solved; on the basis, an information entropy filtering mode is adopted to avoid negative influence of noise data on a model, meanwhile, data overfitting is relieved to a certain extent, and accuracy and robustness of joint information extraction are improved.

It should be noted that, within the scope of protection defined in the claims of the present invention, the following embodiments may be combined and/or expanded, and replaced in any manner that is logical from the above specific embodiments, such as the disclosed technical principles, the disclosed technical features or the implicitly disclosed technical features, etc.

Example 1

A scalable text information joint extraction method based on prompt learning comprises the following steps:

A. constructing a unified representation frame of text information extraction annotation information;

B. constructing a unified prompt template based on the original data set and the extended data set;

C. constructing and training a combined extraction model based on prompt learning text information;

D. and extracting joint information from the input text by using the trained model.

Example 2

On the basis of embodiment 1, in step a, the construction of the unified representation framework of the text information extraction annotation information includes the following sub-steps:

A1. aiming at a named entity recognition task, constructing a labeling information representation framework as follows: (entity_type: span_e), wherein the entity_type represents a selected one of the Entity type names, and span_e represents a text segment in the text belonging to the Entity of the Entity type;

A2. Aiming at entity relation extraction tasks, constructing a labeling information representation framework as follows: (EntityType: span_e (relationship_type_i: span_r_i)), wherein relationship_type_i represents an i-th Entity relationship type name associated with the Entity type of the Entity type, span_r_i represents a tail Entity text segment to which span_e corresponds under relationship_type_i;

A3. aiming at event extraction tasks, constructing a labeling information representation framework as follows: (event_type: span_v (rule_type_i: span_o_i)), wherein event_type represents a selected one of Event type names, span_v represents a text segment of an Event trigger word belonging to the event_type Event type in the text, rule_type_i represents an i-th Event Role name associated with the event_type Event, span_o_i represents a corresponding Event Role text segment of span_v under rule_type_i;

A4. fusion of named entity recognition, entity relation extraction and event extraction annotation information representation frames forms a unified representation frame of text information extraction annotation information, and one example form is as follows:

/>

example 3

On the basis of embodiment 1, in step B, the constructing a unified hint template based on the original dataset and the extended dataset includes the following sub-steps:

B1. Defining the original data set as D _ori ＝{(S _i ,L _i ),i＝1,...,N _ori}, wherein ,S_i Text representing the ith data of the original dataset, L _i Annotation information representing the ith data of the original dataset, N _ori Representing the number of original data samples;

B2. defining an extended dataset as D _ext ＝{(S _j ,L _j ),j＝1,...,N _ext}, wherein ,S_j Text representing the j-th data of the expanded dataset, L _j Annotation information representing the j-th data of the extended data set, N _ext Representing the number of extended data samples;

B3. and fusing the original data set and the extended data set, and constructing a unified prompt template for each piece of labeling information.

Example 4

On the basis of embodiment 1, in step C, the constructing and training the joint extraction model based on prompt learning text information includes the following sub-steps:

C1. setting an encoder based on a transducer network architecture, wherein the output dimension is d _E Denoted as E _model ；

C2. Constructing a full-connection layer network module aiming at the initial position of a text fragment, which is marked as FC _start Constructing a full connection layer network module aiming at a text fragment termination position, which is marked as FC _end ，FC _start With FC _end Is d _E The output dimension is 1;

C3. for the specified input triplet data (S _k ，P _k ，L _k ) According to P _k S _k Performing prompt and text splicing, and marking the length of the spliced text as d _k ；

C4. Completing the blank characters of the spliced text, namely adding a nonsensical blank character sequence Q after splicing the text _k So that PkS _k Q _k The character length reaches d _* ＝max(d _k ，k＝1，...，K)；

C5. Construction of P _k S _k Q _k Mask vector wherein ,

C6. use E _model P pair P _k S _k Q _k Coding, and marking the coded characteristic matrix as

C7. Based on L _k At S _k The starting position and the ending position in the sequence, and constructing a reference result vector GT _{k_start} 、 wherein ,GT_{k_start} Remove L _k At S _k The starting position of the catalyst is 1, the rest are 0, GT _{k_endt} Remove L _k At S _k The rest of the terminal positions are 0 except 1;

C8. setting the maximum iteration number EP, and recording the current completed iteration number as EP;

C9. calculating the minimum information entropy threshold value rho=exp [ EP/(EP+1) acceptable by the model under the current iteration number] ^-1 ；

C10. Will F _k Input FC respectively _start and FC_end Activating by adopting sigmoid to obtain prediction vectors P respectively _start and P_end ；

C11. Respectively calculate P _start and P_end Information entropy, denoted as θ _start and θ_end ；

C12. If theta is _start < ρ and θ _end < ρ, then based on P _start and P_end Respectively with GT _{k_start} and GT_{k_end} Calculating cross entropy loss and adopting back propagation to carry out model training, otherwise, skipping current data;

C13. c3 to C12 are executed on all the K groups of data, and after one round of training of the K groups of data is completed, the ep value is increased by 1;

C14. and repeatedly executing C13 until ep=EP, and marking the trained text information joint extraction Model as Model.

Example 5

On the basis of embodiment 1, in step D, the joint information extraction of the input text using the trained model includes the following sub-steps:

D1. Setting an output threshold delta epsilon (0, 1);

D2. traversing all Entity class names { entity_type (Q), q=1,..once, Q }, which is the number of all Entity class names, contained in the trained data for the input Text; the entity_type (q) is spliced with the Text and then input into a Model, and a named Entity recognition extraction under the entity_type (q) is obtainedTaking the predictive vector, let P _start >Delta is used as a starting point, and the nearest P is used after the position _end >The delta position is an ending point, and the text segment between the delta position and the ending point is used as a predicted Entity text segment of the entity_type (q);

D3. traversing the input Text and the Entity Text fragments predicted in the step D2, and recording the Entity relationship type name { relation_type (G), g=1, G } associated with the Entity type to which the predicted Entity Text fragments belong, wherein G is the number of all Entity relationship types associated with the Entity type to which the entity_span belongs; the relation_type (g) of the' Entity_span is spliced with Text and then input into a Model, and a tail Entity associated with the Entity_span under the relation_type (g) is obtained to extract a prediction vector so as to obtain P _start >Delta is used as a starting point, and the nearest P is used after the position _end >The delta position is an ending point, and the text segment between the delta position and the ending point is used as a predicted tail Entity text segment of the entity_span under the relation_type (g);

D4. Traversing all Event category names { event_type (H), h=1,..once, H }, H being the number of all Event category names, contained in the trained data, for the input Text; the event_type (h) and Text are spliced and then input into a Model to obtain an Event trigger word extraction prediction vector under the event_type (h) so as to obtain P _start >Delta is used as a starting point, and the nearest P is used after the position _end >The delta position is an ending point, and a text segment between the delta position and the ending point is used as a predicted Event trigger word text segment of event_type (h);

D5. traversing Event Role names { rotor_type (W), w=1, and W } associated with Event types to which event_span belongs for the input Text and the Event trigger word Text fragments predicted in the step D4, wherein W is the number of all Event roles associated with the Event types to which the event_span belongs; the rule_type (w) of the' event_span is spliced with Text and then input into a Model, so that an Event Role extraction prediction vector associated with the event_span under the rule_type (w) is obtained, and the Event Role extraction prediction vector is used as P _start >Delta is used as a starting point, and the nearest P is used after the position _end >The delta position is the end point, and the text segment between the delta position and the delta position is used as Event_span is an event character text segment under roller_type (w);

D6. And D2, fusing the predicted results of the steps D2 to D5 according to the unified characterization framework constructed in the step A as the result of the text information joint extraction.

Example 6

On the basis of example 3, step B3 comprises the following sub-steps:

and B3-1, constructing a named entity recognition task prompt template as follows:

{

'text' S

'hint' Entity_type

'result' span_e

}

S is text in an original data set or an extended data set, the entity_type is an Entity type name selected from the labeling information of the data set to which the S belongs, and span_e is an Entity text segment belonging to the entity_type in the S;

and B3-2, constructing an entity relation extraction task prompt template as follows:

{

'text' S

'hint' match_e relation_type

'result' span_r

}

S is a text in an original data set or an expanded data set, span_e is a text segment serving as a head entity of a certain Relation type in S, relation_type is an entity Relation type name selected in labeling information of the data set to which S belongs, span_r is a tail entity text segment with a selected Relation relation_type in S and span_e;

and B3-3, constructing an entity relation extraction task prompt template as follows:

{

'text' S

'prompt': event_type

'results' span_v

}

{

'text' S

'hint' Role_type of span_v

'result' span_o

}

The method comprises the steps that S is a text in an original data set or an extended data set, event_type is a specific Event type name in data set marking information to which S belongs, span_v is a text segment serving as a trigger word of a certain Event type in S, role_type is a specific Event Role name in data set marking information to which S belongs, span_o is a text segment of a role_type Role marked as event_type Event in S;

b3-4, reconstructing the data according to the steps B3-1 to B3-3 for all the labeling information in the fusion data, and randomly sampling text fragments in the labeling data and constructing negative sample data in all entity type names, relation type names, event type names and event role names, namely, a sample corresponding to a result cannot be queried; the data set after fusing, reconstructing and adding negative samples is denoted as D _mix ＝{(S _k ,P _k ,L _k ) K=1,.. _k Representing text, P _k Representation prompt, L _k The result is represented by K, the number of all triples of the dataset.

Example 7

On the basis of embodiment 6, in step B3-1, span_e is made to be an empty string if there is no Entity text segment belonging to the entity_type type in S.

Example 8

Based on embodiment 6, in step B3-2, span_r is made to be an empty string if there is no tail entity text segment of relationship_type in S with span_e.

Example 9

Based on embodiment 6, in step B3-3, the two 'result' fields are made empty strings when no corresponding text segment is queried.

Example 10

An expandable text information joint extraction device based on prompt learning, based on the method of any one of embodiments 1 to 9, further comprising:

the first construction module is used for constructing a unified representation frame of the text information extraction annotation information;

the second construction module is used for constructing a unified prompt template based on the original data set and the extended data set;

the third construction module is used for constructing and training a text information joint extraction model based on prompt learning;

and the extraction module is used for extracting the joint information of the input text by using the trained model.

The units involved in the embodiments of the present invention may be implemented by software, or may be implemented by hardware, and the described units may also be provided in a processor. Wherein the names of the units do not constitute a limitation of the units themselves in some cases.

According to an aspect of embodiments of the present invention, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The computer instructions are read from the computer-readable storage medium by a processor of a computer device, and executed by the processor, cause the computer device to perform the methods provided in the various alternative implementations described above.

As another aspect, the embodiment of the present invention also provides a computer-readable medium that may be contained in the electronic device described in the above embodiment; or may exist alone without being incorporated into the electronic device. The computer-readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to implement the methods described in the above embodiments.

The invention is not related in part to the same as or can be practiced with the prior art.

The foregoing technical solution is only one embodiment of the present invention, and various modifications and variations can be easily made by those skilled in the art based on the application methods and principles disclosed in the present invention, not limited to the methods described in the foregoing specific embodiments of the present invention, so that the foregoing description is only preferred and not in a limiting sense.

In addition to the foregoing examples, those skilled in the art will recognize from the foregoing disclosure that other embodiments can be made and in which various features of the embodiments can be interchanged or substituted, and that such modifications and changes can be made without departing from the spirit and scope of the invention as defined in the appended claims.

Claims (10)

1. The expandable text information joint extraction method based on prompt learning is characterized by comprising the following steps of:

A. constructing a unified representation frame of text information extraction annotation information;

B. constructing a unified prompt template based on the original data set and the extended data set;

C. constructing and training a combined extraction model based on prompt learning text information;

D. and extracting joint information from the input text by using the trained model.

2. The method for extracting extensible text information based on prompt learning according to claim 1, wherein in the step a, the constructing a unified representation framework of extracting annotation information of the text information comprises the following sub-steps:

A1. aiming at a named entity recognition task, constructing a labeling information representation framework as follows: (entity_type: span_e), wherein the entity_type represents a selected one of the Entity type names, and span_e represents a text segment in the text belonging to the Entity of the Entity type;

A2. Aiming at entity relation extraction tasks, constructing a labeling information representation framework as follows: (EntityType: span_e (relationship_type_i: span_r_i)), wherein relationship_type_i represents an i-th Entity relationship type name associated with the Entity type of the Entity type, span_r_i represents a tail Entity text segment to which span_e corresponds under relationship_type_i;

A3. aiming at event extraction tasks, constructing a labeling information representation framework as follows: (event_type: span_v (rule_type_i: span_o_i)), wherein event_type represents a selected one of Event type names, span_v represents a text segment of an Event trigger word belonging to the event_type Event type in the text, rule_type_i represents an i-th Event Role name associated with the event_type Event, span_o_i represents a corresponding Event Role text segment of span_v under rule_type_i;

A4. and merging named entity identification, entity relation extraction and event extraction annotation information representation frames to form a unified representation frame of text information extraction annotation information.

3. The method for extracting extensible text information in combination based on prompt learning according to claim 1, wherein in step B, the constructing a unified prompt template based on the original data set and the extended data set includes the following sub-steps:

B1. Defining the original data set as D _ori ＝{(S _i ，L _i )，i＝1，...，N _ori}, wherein ,S_i Text representing the ith data of the original dataset, L _i Annotation information representing ith data of original dataset, no _ri Representing the number of original data samples;

B2. defining an extended dataset as D _ext ＝{(S _j ，L _j )，j＝1，...，N _ext}, wherein ,S_j Text representing the j-th data of the expanded dataset, L _j Annotation information representing the j-th data of the extended data set, N _ext Representing the number of extended data samples;

B3. and fusing the original data set and the extended data set, and constructing a unified prompt template for each piece of labeling information.

4. The method for joint extraction of expandable text information based on prompt learning according to claim 1, wherein in step C, the constructing and training of the joint extraction model of text information based on prompt learning includes the following sub-steps:

C1. setting an encoder based on a transducer network architecture, wherein the output dimension is d _E Denoted as E _model ；

C2. Constructing a full-connection layer network module aiming at the initial position of a text fragment, which is marked as FC _start Constructing a full connection layer network module aiming at a text fragment termination position, which is marked as FC _end ，FC _start With FC _end Is d _E The output dimension is 1;

C3. for the specified input triplet data (S _k ，P _k ，L _k ) According to P _k S _k Performing prompt and text splicing, and marking the length of the spliced text as d _k ；

C4. Completing the blank characters of the spliced text, namely adding a nonsensical blank character sequence Q after splicing the text _k So that P _k S _k Q _k The character length reaches d _* ＝max(d _k ，k＝1，...，K)；

C5. Construction of P _k S _k Q _k Mask vector wherein ,

C6. use E _model P pair P _k S _k Q _k Coding, and marking the coded characteristic matrix as

C7. Based on L _k At S _k The starting position and the ending position in the sequence, and constructing a reference result vector GT _{k_start} 、 wherein ,GT_{k_start} Remove L _k At S _k The starting position of the catalyst is 1, the rest are 0, GT _{k_endt} Remove L _k At S _k The rest of the terminal positions are 0 except 1;

C8. setting the maximum iteration number EP, and recording the current completed iteration number as EP;

C9. calculating the minimum information entropy threshold value rho=exp [ EP/(EP+1) acceptable by the model under the current iteration number] ^-1 ；

C10. Will F _k Input FC respectively _start and FC_end Activating by adopting sigmoid to obtain prediction vectors P respectively _start and P_end ；

C11. Respectively calculate P _start and P_end Information entropy, denoted as θ _start and θ_end ；

C12. If theta is _start < ρ and θ _end < ρ, then based on P _start and P_end Respectively with GT _{k_start} and GT_{k_end} Calculating cross entropy loss and performing model training by adopting back propagation; otherwise, skipping the current data;

C13. c3 to C12 are executed on all the K groups of data, and after one round of training of the K groups of data is completed, the ep value is increased by 1;

C14. and repeatedly executing C13 until ep=EP, and marking the trained text information joint extraction Model as Model.

5. The method for extracting the combined information of the expandable text information based on prompt learning according to claim 1, wherein in the step D, the step of extracting the combined information of the input text by using the trained model comprises the following sub-steps:

D1. setting an output threshold delta epsilon (0, 1);

D2. traversing all Entity class names { entity_type (Q), q=1,..once, Q }, which is the number of all Entity class names, contained in the trained data for the input Text; the entity_type (q) is spliced with the Text and then input into a Model to obtain a named Entity recognition extraction prediction vector under the entity_type (q) so as to obtain P _start The position of > delta is taken as a starting point, and the nearest P behind the position is taken as a starting point _end The position of delta is an ending point, and the text segment between the two positions is used as a predicted Entity text segment of the entity_type (q);

D3. traversing the input Text and the Entity Text fragments predicted in the step D2, and recording the Entity relationship type name { relation_type (G), g=1, G } associated with the Entity type to which the predicted Entity Text fragments belong, wherein G is the number of all Entity relationship types associated with the Entity type to which the entity_span belongs; the relation_type (g) of the' Entity_span is spliced with Text and then input into a Model, and a tail Entity associated with the Entity_span under the relation_type (g) is obtained to extract a prediction vector so as to obtain P _start The position of > delta is taken as a starting point, and the nearest P behind the position is taken as a starting point _end The position of delta is an ending point, and the text segment between the two positions is used as a predicted tail Entity text segment of the entity_span under the relation_type (g);

D4. traversing all Event category names { event_type (H), h=1,..once, H }, H being the number of all Event category names, contained in the trained data, for the input Text; the event_type (h) and Text are spliced and then input into a Model to obtain an Event trigger word extraction prediction vector under the event_type (h) so as to obtain P _start The position of > delta is taken as a starting point, and the nearest P behind the position is taken as a starting point _end The position of delta is an ending point, and a text segment between the two positions is used as a predicted Event trigger word text segment of event_type (h);

D5. traversing Event Role names { rotor_type (W), w=1, and W } associated with Event types to which event_span belongs for the input Text and the Event trigger word Text fragments predicted in the step D4, wherein W is the number of all Event roles associated with the Event types to which the event_span belongs; the rule_type (w) of the' event_span is spliced with Text and then input into a Model, so that an Event Role extraction prediction vector associated with the event_span under the rule_type (w) is obtained, and the Event Role extraction prediction vector is used as P _start The position of > delta is taken as a starting point, and the nearest P behind the position is taken as a starting point _end The position of delta is an ending point, and the text segment between the two positions is used as an Event character text segment of event_span under the rule_type (w);

D6. and D2, fusing the predicted results of the steps D2 to D5 according to the unified characterization framework constructed in the step A as the result of the text information joint extraction.

6. The method for extracting extensible text information in combination based on prompt learning according to claim 3, wherein the step B3 comprises the following sub-steps:

and B3-1, constructing a named entity recognition task prompt template as follows:

{

'text': s is S

'hint': entityType

'result' span_e

}

S is text in an original data set or an extended data set, the entity_type is an Entity type name selected from the labeling information of the data set to which S belongs, and span_e is an Entity text segment belonging to the entity_type in S;

and B3-2, constructing an entity relation extraction task prompt template as follows:

{

'text': s is S

'hint' match_e relation_type

'result' span_r

}

S is a text in an original data set or an expanded data set, span_e is a text segment serving as a head entity of a certain Relation type in S, relation_type is an entity Relation type name selected in labeling information of the data set to which S belongs, span_r is a tail entity text segment with a selected Relation relation_type in S and span_e;

And B3-3, constructing an entity relation extraction task prompt template as follows:

{

'text': s is S

'hint': event_type

'results': span_v

}

{

'text': s is S

'hint': span_v's roller_type

'result' span_0

}

The method comprises the steps that S is a text in an original data set or an extended data set, event_type is a specific Event type name in data set marking information to which S belongs, span_v is a text segment serving as a trigger word of a certain Event type in S, role_type is a specific Event Role name in data set marking information to which S belongs, span_0 is a text segment of a role_type Role marked as an event_type Event in S;

b3-4, reconstructing the data according to the steps B3-1 to B3-3 for all the labeling information in the fusion data, and randomly sampling text fragments in the labeling data and constructing negative sample data in all entity type names, relation type names, event type names and event role names, namely, a sample corresponding to a result cannot be queried; the data set after fusing, reconstructing and adding negative samples is denoted as D _mix ＝{(S _k ，P _k ，L _k ) K=1,.. _k Representing text, P _k Representation prompt, L _k The result is represented by K, the number of all triples of the dataset.

7. The method for joint extraction of text information based on prompt learning according to claim 6, wherein in step B3-1, span_e is made empty string if there is no solid text segment belonging to the entity_type type in S.

8. The method for joint extraction of text information based on prompt learning according to claim 6, wherein in step B3-2, span_r is made empty string if there is no tail text segment of relationship_type with span_e in S.

9. The method of claim 6, wherein in step B3-3, the two 'result' fields are empty strings when no corresponding text segment is queried.

10. A scalable text message joint extraction device based on prompt learning, characterized by further comprising, based on the method of any one of claims 1 to 9:

the first construction module is used for constructing a unified representation frame of the text information extraction annotation information;

the second construction module is used for constructing a unified prompt template based on the original data set and the extended data set;

the third construction module is used for constructing and training a text information joint extraction model based on prompt learning;

And the extraction module is used for extracting the joint information of the input text by using the trained model.