CN115017906A - Method, device and storage medium for identifying entities in text - Google Patents
Method, device and storage medium for identifying entities in text Download PDFInfo
- Publication number
- CN115017906A CN115017906A CN202210614127.7A CN202210614127A CN115017906A CN 115017906 A CN115017906 A CN 115017906A CN 202210614127 A CN202210614127 A CN 202210614127A CN 115017906 A CN115017906 A CN 115017906A
- Authority
- CN
- China
- Prior art keywords
- vector
- text
- entity
- recognized
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
- G06F40/295—Named entity recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Abstract
The application provides a method, a device and a storage medium for identifying entities in texts, wherein the method comprises the following steps: determining an attention vector of a text to be recognized; determining an entity characteristic vector and at least one first knowledge representation vector corresponding to a target vocabulary in a text to be recognized based on a preset knowledge graph; the entity knowledge representation vector is used for representing the incidence relation between the target vocabulary and each entity in the preset knowledge map; inputting entity feature vectors corresponding to target words in a text to be recognized and first knowledge expression vectors of each entity into a first multi-head attention layer in a pre-trained entity class recognition model, and determining at least one second knowledge expression vector corresponding to each target word; and inputting a second knowledge representation vector and an attention vector corresponding to the target vocabulary of the text to be recognized into the aggregation layer, and determining the entity category corresponding to the target vocabulary in the text to be recognized. Therefore, the accuracy of determining the entity category corresponding to the target vocabulary in the text is improved.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a method and an apparatus for identifying an entity in a text, and a storage medium.
Background
The identification of the entity classes in the text and classification of the entity classes are important links in tasks such as text classification, data resource protection, data desensitization and the like. For entity recognition, the existing methods which are applied more are a rule and dictionary-based method, a statistical machine learning method or a fusion of the two methods. The existing method for identifying the entity in the unstructured text document depends on formulation of a large number of rules, and the manual formulation of the rules consumes a large amount of manpower, so that the cost of the identification and induction process is high, and the identification efficiency is generally low. And based on a machine learning method, a word2vec or n-gram is often adopted as a word vector generation method, the method cannot represent word ambiguity in Chinese, the generated word vector does not refer to context information of a text, and the requirement on text extraction characteristics is high. In addition, the complex data types in the text and the entity nesting problem cause the entity recognition task to be more complex and deeper, and the existing text recognition method and model are not flexible and efficient enough for named entities with randomness, complexity, variability and nesting, and are difficult to solve the problem of recognizing the categories of the complex data type entities.
Disclosure of Invention
In view of this, an object of the present application is to provide a method, an apparatus, and a storage medium for recognizing an entity in a text, so that a preset knowledge graph and an entity class recognition model are utilized, prior knowledge information is introduced at a vector representation stage of the text, so that vector representation of the text has more semantic information and common sense information, a feature space of a text vector is expanded, and accuracy of determining an entity class corresponding to a target word in the text is improved.
The embodiment of the application provides an identification method for an entity in a text, and the identification method comprises the following steps:
determining an attention vector of a text to be recognized;
determining an entity characteristic vector and at least one first knowledge representation vector corresponding to a target vocabulary in the text to be recognized based on a preset knowledge graph;
inputting the entity feature vectors corresponding to the target vocabularies in the text to be recognized and each first knowledge representation vector into a first multi-head attention layer in a pre-trained entity category recognition model, and determining second knowledge representation vectors corresponding to the target vocabularies;
and inputting a second knowledge representation vector corresponding to the target vocabulary of the text to be recognized and the attention vector into an aggregation layer in the entity category recognition model, and determining an entity category corresponding to the target vocabulary in the text to be recognized.
In one possible embodiment, the attention vector of the text to be recognized is determined by the following steps;
performing word segmentation processing on the acquired text to be recognized to obtain the text to be recognized after word segmentation;
inputting the segmented text to be recognized into a second multi-head attention layer of the entity category recognition model, and determining the attention vector of the text to be recognized.
In a possible implementation manner, the determining, based on a preset knowledge graph, an entity feature vector and at least one first knowledge representation vector corresponding to a target word in the text to be recognized includes:
acquiring an entity feature vector of a target node corresponding to the target vocabulary and at least one adjacent node associated with the target node from the preset knowledge graph;
detecting a node pointing direction between the target node and the adjacent node; if the pointing direction is a forward direction, subtracting the entity feature vector of the adjacent node from the node relation vector of the adjacent node to determine a first vector result, and determining the first vector result as the first knowledge representation vector;
and if the pointing direction is a reverse direction, adding the entity feature vector of the adjacent node and the node relation vector of the adjacent node to determine a second vector result, and determining the first vector result as the first knowledge representation vector.
In one possible implementation, the inputting the second knowledge representation vector corresponding to the target vocabulary of the text to be recognized and the attention vector into an aggregation layer in the entity class recognition model, and determining the entity class corresponding to the target vocabulary of the text to be recognized includes:
determining a first value by multiplying the first weight by the attention vector;
determining a second value by multiplying the second weight by the second knowledge representation vector;
the sum of the first numerical value and the second numerical value determines a third numerical value;
and determining the entity category corresponding to the target vocabulary in the text to be recognized by the product of the third numerical value and the activation function.
In one possible embodiment, the entity class identification model is trained by:
acquiring a plurality of sample vocabularies and sample entity category information corresponding to each sample vocabulary;
and performing iterative training processing on an initial neural network model based on the plurality of sample vocabularies and the corresponding sample entity category information to determine the entity category recognition model.
The embodiment of the present application further provides an identification apparatus for an entity in a text, where the identification apparatus includes:
the first determination module is used for determining the attention vector of the text to be recognized;
the second determination module is used for determining an entity feature vector and at least one first knowledge representation vector corresponding to a target vocabulary in the text to be recognized based on a preset knowledge graph;
a third determining module, configured to input the entity feature vectors and each of the first knowledge representation vectors corresponding to the target vocabulary in the text to be recognized to a first multi-head attention layer in a pre-trained entity category recognition model, and determine a second knowledge representation vector corresponding to the target vocabulary;
and the entity category determining module is used for inputting the second knowledge representation vector corresponding to the target vocabulary of the text to be recognized and the attention vector into an aggregation layer in the entity category recognition model, and determining the entity category corresponding to the target vocabulary in the text to be recognized.
In one possible embodiment, the first determination module determines the attention vector of the text to be recognized by the following steps;
performing word segmentation processing on the acquired text to be recognized to obtain the text to be recognized after word segmentation;
inputting the segmented text to be recognized into a second multi-head attention layer of the entity category recognition model, and determining the attention vector of the text to be recognized.
In a possible implementation manner, when the second determining module is configured to determine, based on the preset knowledge graph, an entity feature vector and at least one first knowledge representation vector corresponding to a target vocabulary in the text to be recognized, the second determining module is specifically configured to:
acquiring an entity feature vector of a target node corresponding to the target vocabulary and at least one adjacent node associated with the target node from the preset knowledge graph;
detecting a node pointing direction between the target node and the adjacent node; if the pointing direction is a forward direction, subtracting the entity feature vector of the adjacent node from the node relation vector of the adjacent node to determine a first vector result, and determining the first vector result as the first knowledge representation vector;
and if the pointing direction is a reverse direction, adding the entity feature vector corresponding to the target vocabulary and the node relation vector of the adjacent node to determine a second vector result, and determining the first vector result as the first knowledge representation vector.
An embodiment of the present application further provides an electronic device, including: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating via the bus when the electronic device is running, the machine-readable instructions when executed by the processor performing the steps of the method for identifying an entity in text as described above.
Embodiments of the present application also provide a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to perform the steps of the above-mentioned identification method for an entity in a text.
The embodiment of the application provides a method, a device and a storage medium for identifying entities in texts, which comprises the following steps: determining an attention vector of a text to be recognized; determining an entity characteristic vector and at least one first knowledge representation vector corresponding to a target vocabulary in a text to be recognized based on a preset knowledge graph; the entity knowledge representation vector is used for representing the incidence relation between the target vocabulary and each entity in the preset knowledge map; inputting entity feature vectors corresponding to target words in a text to be recognized and first knowledge expression vectors of each entity into a first multi-head attention layer in a pre-trained entity class recognition model, and determining at least one second knowledge expression vector corresponding to each target word; and inputting a second knowledge representation vector and an attention vector corresponding to the target vocabulary of the text to be recognized into the aggregation layer, and determining the entity category corresponding to the target vocabulary in the text to be recognized. Therefore, the method and the device realize that the priori knowledge information is introduced in the vector representation stage of the text by using the preset knowledge map and the entity category identification model, so that the vector representation of the text has more semantic information and common sense information, the feature space of the text vector is expanded, and the accuracy of determining the entity category corresponding to the target vocabulary in the text is improved.
In order to make the aforementioned objects, features and advantages of the present application more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
To more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.
Fig. 1 is a flowchart of a method for identifying entities in a text according to an embodiment of the present application;
fig. 2 is a schematic diagram of a node relationship in a preset knowledge graph according to an embodiment of the present application;
FIG. 3 is a schematic structural diagram of an entity class identification model according to an embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of an apparatus for recognizing an entity in a text according to an embodiment of the present application;
fig. 5 is a second schematic structural diagram of an apparatus for recognizing entities in text according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
To make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it should be understood that the drawings in the present application are for illustrative and descriptive purposes only and are not used to limit the scope of protection of the present application. Additionally, it should be understood that the schematic drawings are not necessarily drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of the present application. It should be understood that the operations of the flow diagrams may be performed out of order, and that steps without logical context may be performed in reverse order or concurrently. One skilled in the art, under the guidance of this application, may add one or more other operations to, or remove one or more operations from, the flowchart.
In addition, the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, as presented in the figures, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be obtained by a person skilled in the art without making any inventive step based on the embodiments of the present application, fall within the scope of protection of the present application.
To enable those skilled in the art to utilize the present disclosure, in connection with certain application scenarios "identify entities in the text," the following embodiments are presented to enable those skilled in the art to apply the general principles defined herein to other embodiments and application scenarios without departing from the spirit and scope of the present disclosure.
The method, the apparatus, the electronic device, or the computer-readable storage medium described in the embodiments of the present application may be applied to any scenario in which an entity in a text needs to be identified, and the embodiments of the present application do not limit a specific application scenario, and any scheme that uses the method, the apparatus, and the storage medium for identifying an entity in a text provided in the embodiments of the present application is within the protection scope of the present application.
First, an application scenario to which the present application is applicable will be described. The method and the device can be applied to the technical field of data processing.
Research shows that the identification of the entity category in the text and the classification of the entity category are important links in tasks such as text classification, data resource protection, data desensitization and the like. For entity recognition, the existing methods which are applied more are methods based on rules and dictionaries, methods of statistical machine learning, or a combination of the two methods. The existing method for identifying the entity in the unstructured text document depends on formulation of a large number of rules, and the manual formulation of the rules consumes a large amount of manpower, so that the cost of the identification and induction process is high, and the identification efficiency is generally low. And based on a machine learning method, a word2vec or n-gram is often adopted as a word vector generation method, the method cannot represent word ambiguity in Chinese, the generated word vector does not refer to context information of a text, and the requirement on text extraction characteristics is high. In addition, the complex data types in the text and the entity nesting problem cause the entity recognition task to be more complex and deeper, and the existing text recognition method and model are not flexible and efficient enough for named entities with randomness, complexity, variability and nesting, and are difficult to solve the problem of recognizing the categories of the complex data type entities.
Based on this, the embodiment of the application provides a method for identifying an entity in a text, so that the prior knowledge information is introduced in a vector representation stage of the text by using a preset knowledge map and an entity category identification model, so that the vector representation of the text has more semantic information and common sense information, the feature space of the text vector is expanded, and the accuracy of determining the entity category corresponding to a target vocabulary in the text is improved.
Referring to fig. 1, fig. 1 is a flowchart illustrating a method for recognizing an entity in a text according to an embodiment of the present disclosure. As shown in fig. 1, the identification method provided in the embodiment of the present application includes:
s101: an attention vector of the text to be recognized is determined.
In the step, a text to be recognized is obtained first, and the attention vector of the text to be recognized is determined according to the text to be recognized.
Here, the text to be recognized may be a word, an article, or the like.
In one possible embodiment, the attention vector of the text to be recognized is determined by the following steps:
(1): and performing word segmentation processing on the acquired text to be recognized to obtain the text to be recognized after word segmentation.
After the text to be recognized is obtained, word segmentation processing is performed according to semantic information of the text to be recognized, and the text to be recognized after word segmentation is obtained.
For example, the text to be recognized after the word segmentation processing is performed to "i sit in a train to get to beijing" is "today", "i", "sit", "train", "go", and "beijing". Each word in the segmented text to be recognized carries a corresponding text serial number according to semantic information, if the text serial number of the segmented text is 01, the text serial number of the segmented text is 02, and the like is performed in the following steps.
(2): inputting the segmented text to be recognized into a second multi-head attention layer of the entity category recognition model, and determining the attention vector of the text to be recognized.
And inputting the segmented text to be recognized into a second multi-head attention layer of the entity type recognition model according to the text sequence, and performing attention processing on the segmented text to be recognized in the second multi-head attention layer to determine the attention vector of the text to be recognized.
Wherein, the input of attention function in the second multi-head attention layer is changed from original Q, K and V into QW (upper mark is Q, subscript is i), KW (upper mark is K, subscript is i) and VW (upper mark is V, subscript is i); i.e. all 3W are different; changing the original 512 dimensionality of Q, K and V into 64 dimensionality; then splicing the two into 512 dimensions, and performing linear conversion through W (marked as O); and obtaining the final attention vector of the text to be recognized.
In one possible embodiment, the entity class identification model is trained by:
acquiring a plurality of sample vocabularies and sample entity category information corresponding to each sample vocabulary; and performing iterative training processing on an initial neural network model based on the plurality of sample vocabularies and the corresponding sample entity category information to determine the entity category recognition model.
Here, the initial neural network model is iteratively trained by using a plurality of sample vocabularies and sample entity class information corresponding to the sample vocabularies, and the entity class recognition model is determined.
The iterative training process is to stop the training when the number of iterations reaches a certain threshold value to determine the entity type recognition model, or to stop the training when the loss function is small to determine the entity type recognition model.
S102: and determining an entity characteristic vector and at least one first knowledge representation vector corresponding to a target vocabulary in the text to be recognized based on a preset knowledge graph.
In the step, an entity characteristic vector and at least one first knowledge representation vector corresponding to a target vocabulary in a text to be recognized are determined by using a preset knowledge graph.
The preset knowledge graph is preset, a large amount of entity type information, entity types and relations between the entities and other entities are stored in the preset knowledge graph, characteristics of the entities and relations in the knowledge graph and semantic information contained in the characteristics can be summarized to the greatest extent, the preset knowledge graph can be used for modeling semantic relations between known entities/relations and new entities/relations, and richer external information and common sense information are brought to the entities/relations.
The first knowledge representation vector introduces prior knowledge information for the target vocabulary in the vector representation stage of the text, so that the vector representation of the target vocabulary has more semantic information.
Here, a first knowledge representation vector introduces a priori knowledge information for the target vocabulary during the vector representation phase of the text.
In a possible implementation manner, the determining, based on a preset knowledge graph, an entity feature vector and at least one first knowledge representation vector corresponding to a target word in the text to be recognized includes:
a: and acquiring an entity feature vector of a target node corresponding to the target vocabulary and at least one adjacent node associated with the target node from the preset knowledge graph.
Here, the entity feature vector of the target node corresponding to the target vocabulary and at least one adjacent node associated with the target node are obtained in the preset knowledge graph.
Wherein the target vocabulary is determined by the following steps: determining a plurality of candidate vocabularies in the files to be recognized after word segmentation; screening the target vocabulary from a plurality of candidate vocabularies based on the preset knowledge graph; the target vocabulary is an entity in the preset knowledge graph; non-target words in the candidate words are non-entities.
The entity feature vector is a representation vector representing the entity feature of the target node corresponding to the target vocabulary in the preset knowledge graph, wherein the entity feature vector is preset, and the representation form of the entity feature vector can be
The adjacent nodes are other nodes associated with the target node in the preset knowledge graph, and the vocabulary corresponding to the adjacent nodes is also an entity. For example, the target node is 'milk', and other nodes adjacent to the target node are 'mongolian cattle', 'drinks', and the like.
B: detecting a node pointing direction between the target node and the adjacent node; and if the pointing direction is a forward direction, subtracting the entity feature vector of the adjacent node from the node relation vector of the adjacent node to determine a first vector result, and determining the first vector result as the first knowledge representation vector.
Here, the node pointing direction between the target node and the adjacent node is detected, if the pointing direction is the forward direction, the entity feature vector of the adjacent node is subtracted from the node relation vector of the adjacent node to determine a first vector result, and the first vector result is determined as a first knowledge representation vector.
Wherein, the node relation of the adjacent nodes is the entity relation between the target node and the adjacent nodes, if the target node is 'milk' and the adjacent nodes are 'Mongolian', the node relation between the adjacent nodes and the target node is 'brand', and the node relation vector can be used
Representing, the node relation vector is known in the presetSearching in the identification graph, wherein the entity characteristic vector of the adjacent node is
Wherein, the forward direction is that the target node points to the adjacent node according to the node relation, if the forward direction is, the target node e i By node relation r ij To a neighboring node e ij Neighboring node e ij The entity feature vector of (2) minus the node relation r ij The node relation vector of (1) determines a first vector result, and determines the first vector result as a first knowledge representation vector.
C: and if the pointing direction is a reverse direction, adding the entity feature vector of the adjacent node and the node relation vector of the adjacent node to determine a second vector result, and determining the first vector result as the first knowledge representation vector.
Here, the node pointing direction between the target node and the adjacent node is detected, if the pointing direction is the reverse direction, the entity feature vector of the adjacent node and the node relation vector of the adjacent node are added to determine a second vector result, and the second vector result is determined as the first knowledge representation vector.
Wherein, the reverse direction is that the adjacent node points to the target node according to the node relation, if the reverse direction is, the adjacent node e ij By node relation r ij To a target node e i Neighboring node e ij The entity feature vector plus node relation r ij The second vector result is determined by the node relation vector, and the second vector result is determined as the first knowledge representation vector.
In a possible implementation, the first knowledge representation vector may be further determined by: according to the destination node e i The direction relationship with the adjacent node is obtained about a target node e i The first knowledge of (a) represents a set of vectors E i . If the direction is reverse, the adjacent node e ij By node relation r ij To a target node e i Neighboring node e ij The entity feature vector plus node relation r ij The result of the node relation vector is added into the set to be E ij (ii) a If the direction is forward, the target node e i By node relation r ij To a neighboring node e ij Neighboring node e ij The entity feature vector of (2) minus the node relation r ij The result of the node relation vector of (a) is added into the set as E ij Until each adjacent node is added into the set E after the calculation is carried out ij 。
Further, referring to fig. 2, fig. 2 is a schematic diagram of a node relationship in a preset knowledge graph according to an embodiment of the present application, as shown in fig. 2. The node of the city is adjacent to the node of the country, the node of the airport and the node of the hotel, the node of the city points to the node of the country as a forward direction, the node relation of the city and the country is that a city belongs to a certain country, the node of the airport points to the node of the city as a reverse direction, and the node relation of the airport and the city is that a certain airport is located in a certain city.
S103: and inputting the entity characteristic vectors corresponding to the target vocabularies in the text to be recognized and each first knowledge representation vector into a first multi-head attention layer in a pre-trained entity category recognition model, and determining second knowledge representation vectors corresponding to the target vocabularies.
In the step, the entity characteristic vectors corresponding to the target vocabularies in the text to be recognized and each first knowledge representation vector are input to a first multi-head attention layer in a pre-trained entity category recognition model, and second knowledge representation vectors corresponding to the target vocabularies are determined.
Here, the second knowledge representation vector is determined by the entity feature vector and each of the first knowledge representation vectors together, the second knowledge representation vector being represented by K i And (4) showing.
In the first multi-head attention layer
K＝E i And calculating to determine a second knowledge representation vector. Wherein Q is a first matrix and Q is a second matrix,
is an entity feature vector, K is a second knowledge representation vector, E i A set of vectors is represented for a plurality of first knowledge.
S104: and inputting a second knowledge representation vector corresponding to the target vocabulary of the text to be recognized and the attention vector into an aggregation layer in the entity category recognition model, and determining an entity category corresponding to the target vocabulary in the text to be recognized.
In the step, a second knowledge representation vector and an attention vector corresponding to a target vocabulary of the text to be recognized are input into an aggregation layer in an entity category recognition model, and an entity category corresponding to the target vocabulary in the text to be recognized is determined.
In one possible implementation, the inputting the second knowledge representation vector corresponding to the target vocabulary of the text to be recognized and the attention vector into an aggregation layer in the entity class recognition model, and determining the entity class corresponding to the target vocabulary of the text to be recognized includes:
determining a first value by multiplying the first weight by the attention vector; determining a second value by multiplying the second weight by the second knowledge representation vector; the sum of the first numerical value and the second numerical value determines a third numerical value; and determining the entity category corresponding to the target vocabulary in the text to be recognized by the product of the third numerical value and the activation function.
Here, O is used in the polymerization layer i ＝σ(W 1 H i +W 2 E i ) And calculating the entity category corresponding to the target vocabulary in the text to be recognized. Wherein, O i For the entity class corresponding to the target vocabulary, W 1 Is a first weight, W 2 Is a second weight, H i As attention vector, E i For the second knowledge representation vector, σ is the activation function. Here, the first weight and the second weight are set in advance.
In the specific embodiment, an attention vector of 'I sit on the train to get to Beijing' today 'is determined, word segmentation processing is carried out on' I sit on the train to get to Beijing 'today' to determine a plurality of candidate words, entity information corresponding to each candidate word is screened out in a preset knowledge map, if the entity information corresponding to the candidate words exists, the candidate words are determined to be target words, the target words are determined to be 'train' and 'Beijing', an entity feature vector of the 'train' and an entity feature vector of the 'Beijing' are determined according to the preset knowledge map, an adjacent node of a target node corresponding to the 'train' is determined to be 'harmony number' in the preset knowledge map, a node relation is a train name, a node pointing direction is 'train' pointing to 'harmony number', a first knowledge vector of the 'train' is determined, and the first knowledge vector and the entity feature vector of the 'train' are input to a first multi-head attention layer, determining a second knowledge representation vector corresponding to the train; determining that an adjacent node of a target node corresponding to the Beijing is a city, the node relation is a place, the node pointing direction is that the city points to the Beijing, determining a first knowledge vector of the Beijing, inputting the first knowledge vector and an entity feature vector of the Beijing to a first multi-head attention layer, and determining a second knowledge representation vector corresponding to the Beijing; inputting the attention vector of ' I sit in the train to get to Beijing ' today ' and the second knowledge representation vector corresponding to ' train ' and the second knowledge representation vector corresponding to ' Beijing ' into a polymerization layer together, wherein the determined ' train ' is an entity, the entity type is the name of the train, ' Beijing ' is the entity, and the entity type is the city.
Further, please refer to fig. 3, wherein fig. 3 is a schematic structural diagram of an entity class identification model according to an embodiment of the present application. As shown in fig. 3, the entity class recognition model includes a first multi-head attention layer, a second multi-head attention layer, a preset knowledge graph, and an aggregation layer, respectively. The second multi-head attention layer is used for determining an attention vector of a text to be recognized, the preset knowledge map is used for determining an entity feature vector and at least one first knowledge representation vector corresponding to a target word of the text to be recognized, the first multi-head attention layer is used for determining a second knowledge representation vector corresponding to the target word, and the aggregation layer is used for aggregating the second knowledge representation vector and the attention vector to determine an entity category corresponding to the target word in the text to be recognized.
In a specific embodiment, the obtained text to be recognized is subjected to word segmentation to determine the text to be recognized after word segmentation, the text to be recognized after word segmentation is input into a second multi-head attention layer in an entity type recognition model according to a text sequence to obtain an attention vector of the text to be recognized, candidate words in the text to be recognized after word segmentation are utilized to assemble a preset knowledge graph to screen out corresponding entities, candidate words with the corresponding entities in the preset knowledge graph are assembled to be determined as target words, and after target nodes corresponding to the target words and the target words are determined, a plurality of adjacent nodes adjacent to the target nodes are determined. Determining a first knowledge expression vector of a target vocabulary according to the node pointing relation between the target node and an adjacent node and the entity characteristic vector of the adjacent node, inputting the entity characteristic vector and the first knowledge expression vector corresponding to the target vocabulary into a first multi-head attention layer, determining a second knowledge expression vector corresponding to the target vocabulary by using a multi-head attention calculation formula, inputting the second knowledge expression vector and the attention vector into an aggregation layer in an entity category recognition model together, and determining the entity category corresponding to the target vocabulary in a text to be recognized by using the calculation formula.
An identification method for an entity in a text provided by the embodiment of the application comprises the following steps: determining an attention vector of a text to be recognized; determining an entity feature vector and at least one first knowledge representation vector corresponding to a target vocabulary in the text to be recognized based on a preset knowledge graph; inputting the entity feature vectors corresponding to the target vocabularies in the text to be recognized and each first knowledge representation vector into a first multi-head attention layer in a pre-trained entity category recognition model, and determining second knowledge representation vectors corresponding to the target vocabularies; and inputting a second knowledge representation vector corresponding to the target vocabulary of the text to be recognized and the attention vector into an aggregation layer in the entity category recognition model, and determining an entity category corresponding to the target vocabulary in the text to be recognized. Therefore, the method and the device realize that the priori knowledge information is introduced in the vector representation stage of the text by using the preset knowledge map and the entity category identification model, so that the vector representation of the text has more semantic information and common sense information, the feature space of the text vector is expanded, and the accuracy of determining the entity category corresponding to the target vocabulary in the text is improved.
Referring to fig. 4 and 5, fig. 4 is a first schematic structural diagram of an apparatus for recognizing an entity in a text according to an embodiment of the present application, and fig. 5 is a second schematic structural diagram of the apparatus for recognizing an entity in a text according to the embodiment of the present application. As shown in fig. 4, the recognition apparatus 400 includes:
a first determining module 410, configured to determine an attention vector of a text to be recognized;
a second determining module 420, configured to determine, based on a preset knowledge graph, an entity feature vector and at least one first knowledge representation vector corresponding to a target vocabulary in the text to be recognized;
a third determining module 430, configured to input the entity feature vectors corresponding to the target vocabulary in the text to be recognized and each of the first knowledge representation vectors into a first multi-head attention layer in a pre-trained entity category recognition model, and determine a second knowledge representation vector corresponding to the target vocabulary;
and an entity category determining module 440, configured to input the second knowledge representation vector corresponding to the target vocabulary of the text to be recognized and the attention vector into an aggregation layer in the entity category recognition model, and determine an entity category corresponding to the target vocabulary in the text to be recognized.
Further, the first determining module 410 determines the attention vector of the text to be recognized by the following steps;
performing word segmentation processing on the acquired text to be recognized to obtain the text to be recognized after word segmentation;
inputting the segmented text to be recognized into a second multi-head attention layer of the entity category recognition model, and determining the attention vector of the text to be recognized.
Further, when the second determining module 420 is configured to determine, based on the preset knowledge graph, an entity feature vector and at least one first knowledge representation vector corresponding to a target vocabulary in the text to be recognized, the second determining module 420 is specifically configured to:
acquiring an entity feature vector of a target node corresponding to the target vocabulary and at least one adjacent node associated with the target node from the preset knowledge graph;
detecting a node pointing direction between the target node and the adjacent node; if the pointing direction is a forward direction, subtracting the entity feature vector of the adjacent node from the node relation vector of the adjacent node to determine a first vector result, and determining the first vector result as the first knowledge representation vector;
and if the pointing direction is a reverse direction, adding the entity feature vector corresponding to the target vocabulary and the node relation vector of the adjacent node to determine a second vector result, and determining the first vector result as the first knowledge representation vector.
Further, when the second determining module 430 is configured to determine, based on the preset knowledge graph, an entity feature vector and at least one first knowledge representation vector corresponding to a target vocabulary in the text to be recognized, the second determining module 430 is specifically configured to:
acquiring an entity feature vector of a target node corresponding to the target vocabulary and at least one adjacent node associated with the target node from the preset knowledge graph;
detecting a node pointing direction between the target node and the adjacent node; if the pointing direction is a forward direction, subtracting the entity feature vector of the adjacent node from the node relation vector of the adjacent node to determine a first vector result, and determining the first vector result as the first knowledge representation vector;
and if the pointing direction is a reverse direction, adding the entity feature vector of the adjacent node and the node relation vector of the adjacent node to determine a second vector result, and determining the first vector result as the first knowledge representation vector.
Further, the entity category determining module 440 inputs the second knowledge representation vector corresponding to the target vocabulary of the text to be recognized and the attention vector into an aggregation layer in the entity category recognition model, and determines that the entity category corresponding to the target vocabulary of the text to be recognized is used, where the entity category determining module 440 is specifically configured to:
determining a first value by multiplying the first weight by the attention vector;
determining a second value by multiplying the second weight by the second knowledge representation vector;
the sum of the first numerical value and the second numerical value determines a third numerical value;
and determining the entity category corresponding to the target vocabulary in the text to be recognized by the product of the third numerical value and the activation function.
Further, as shown in fig. 5, the recognition apparatus further includes a model training module 450 for training the entity class recognition model by:
acquiring a plurality of sample vocabularies and sample entity category information corresponding to each sample vocabulary;
and performing iterative training processing on an initial neural network model based on the plurality of sample vocabularies and the corresponding sample entity category information to determine the entity category recognition model.
The embodiment of the application provides a recognition device for entities in texts, which comprises: the first determination module is used for determining the attention vector of the text to be recognized; the second determination module is used for determining an entity feature vector and at least one first knowledge representation vector corresponding to a target vocabulary in the text to be recognized based on a preset knowledge graph; a third determining module, configured to input the entity feature vectors and each of the first knowledge representation vectors corresponding to the target vocabulary in the text to be recognized to a first multi-head attention layer in a pre-trained entity category recognition model, and determine a second knowledge representation vector corresponding to the target vocabulary; and the entity category determining module is used for inputting the second knowledge representation vector corresponding to the target vocabulary of the text to be recognized and the attention vector into an aggregation layer in the entity category recognition model, and determining the entity category corresponding to the target vocabulary in the text to be recognized. Therefore, the method and the device realize that the priori knowledge information is introduced in the vector representation stage of the text by using the preset knowledge map and the entity category identification model, so that the vector representation of the text has more semantic information and common sense information, the feature space of the text vector is expanded, and the accuracy of determining the entity category corresponding to the target vocabulary in the text is improved.
Referring to fig. 6, fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure. As shown in fig. 6, the electronic device 600 includes a processor 610, a memory 620, and a bus 630.
The memory 620 stores machine-readable instructions executable by the processor 610, when the electronic device 600 runs, the processor 610 communicates with the memory 620 through the bus 630, and when the machine-readable instructions are executed by the processor 610, the step of the method for identifying an entity in a text in the embodiment of the method shown in fig. 1 may be performed.
An embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the step of the method for identifying an entity in a text in the method embodiment shown in fig. 1 may be executed.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in software functional units and sold or used as a stand-alone product, may be stored in a non-transitory computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present application, and are used for illustrating the technical solutions of the present application, but not limiting the same, and the scope of the present application is not limited thereto, and although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope disclosed in the present application; such modifications, changes or substitutions do not depart from the spirit and scope of the exemplary embodiments of the present application, and are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A recognition method for an entity in text, the recognition method comprising:
determining an attention vector of a text to be recognized;
determining an entity characteristic vector and at least one first knowledge representation vector corresponding to a target vocabulary in the text to be recognized based on a preset knowledge graph;
inputting the entity feature vectors corresponding to the target vocabularies in the text to be recognized and each first knowledge representation vector into a first multi-head attention layer in a pre-trained entity category recognition model, and determining second knowledge representation vectors corresponding to the target vocabularies;
and inputting a second knowledge representation vector corresponding to the target vocabulary of the text to be recognized and the attention vector into an aggregation layer in the entity category recognition model, and determining an entity category corresponding to the target vocabulary in the text to be recognized.
2. The recognition method according to claim 1, characterized in that the attention vector of the text to be recognized is determined by the following steps:
performing word segmentation processing on the acquired text to be recognized to obtain the text to be recognized after word segmentation;
inputting the segmented text to be recognized into a second multi-head attention layer of the entity category recognition model, and determining the attention vector of the text to be recognized.
3. The recognition method according to claim 1, wherein the determining, based on the preset knowledge graph, an entity feature vector and at least one first knowledge representation vector corresponding to a target vocabulary in the text to be recognized comprises:
acquiring an entity feature vector of a target node corresponding to the target vocabulary and at least one adjacent node associated with the target node from the preset knowledge graph;
detecting a node pointing direction between the target node and the adjacent node; if the pointing direction is a forward direction, subtracting the entity feature vector of the adjacent node from the node relation vector of the adjacent node to determine a first vector result, and determining the first vector result as the first knowledge representation vector;
and if the pointing direction is a reverse direction, adding the entity feature vector of the adjacent node and the node relation vector of the adjacent node to determine a second vector result, and determining the first vector result as the first knowledge representation vector.
4. The recognition method according to claim 1, wherein the inputting the second knowledge representation vector corresponding to the target vocabulary of the text to be recognized and the attention vector into an aggregation layer in the entity class recognition model, and determining the entity class corresponding to the target vocabulary of the text to be recognized comprises:
determining a first value by multiplying the first weight by the attention vector;
determining a second value by multiplying the second weight by the second knowledge representation vector;
the sum of the first numerical value and the second numerical value determines a third numerical value;
and determining the entity category corresponding to the target vocabulary in the text to be recognized by the product of the third numerical value and the activation function.
5. The recognition method of claim 1, wherein the entity class recognition model is trained by:
acquiring a plurality of sample vocabularies and sample entity category information corresponding to each sample vocabulary;
and performing iterative training processing on an initial neural network model based on the plurality of sample vocabularies and the corresponding sample entity category information to determine the entity category recognition model.
6. An apparatus for recognizing an entity in a text, the apparatus comprising:
the first determination module is used for determining the attention vector of the text to be recognized;
the second determination module is used for determining an entity feature vector and at least one first knowledge representation vector corresponding to a target vocabulary in the text to be recognized based on a preset knowledge graph;
a third determining module, configured to input the entity feature vectors and each of the first knowledge representation vectors corresponding to the target vocabulary in the text to be recognized to a first multi-head attention layer in a pre-trained entity category recognition model, and determine a second knowledge representation vector corresponding to the target vocabulary;
and the entity category determining module is used for inputting the second knowledge representation vector corresponding to the target vocabulary of the text to be recognized and the attention vector into an aggregation layer in the entity category recognition model, and determining the entity category corresponding to the target vocabulary in the text to be recognized.
7. The recognition apparatus according to claim 6, wherein the first determination module determines the attention vector of the text to be recognized by;
performing word segmentation processing on the acquired text to be recognized to obtain the text to be recognized after word segmentation;
inputting the segmented text to be recognized into a second multi-head attention layer of the entity category recognition model, and determining the attention vector of the text to be recognized.
8. The recognition apparatus according to claim 6, wherein when the second determining module is configured to determine the entity feature vector and the at least one first knowledge representation vector corresponding to the target vocabulary in the text to be recognized based on the preset knowledge graph, the second determining module is specifically configured to:
acquiring an entity feature vector of a target node corresponding to the target vocabulary and at least one adjacent node associated with the target node from the preset knowledge graph;
detecting a node pointing direction between the target node and the adjacent node; if the pointing direction is a forward direction, subtracting the entity feature vector of the adjacent node from the node relation vector of the adjacent node to determine a first vector result, and determining the first vector result as the first knowledge representation vector;
and if the pointing direction is a reverse direction, adding the entity feature vector corresponding to the target vocabulary and the node relation vector of the adjacent node to determine a second vector result, and determining the first vector result as the first knowledge representation vector.
9. An electronic device, comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating via the bus when the electronic device is running, the machine-readable instructions being executed by the processor to perform the steps of the method for identifying an entity in text as claimed in any one of claims 1 to 5.
10. A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, carries out the steps of the method for identifying an entity in text according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210614127.7A CN115017906A (en) | 2022-05-31 | 2022-05-31 | Method, device and storage medium for identifying entities in text |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210614127.7A CN115017906A (en) | 2022-05-31 | 2022-05-31 | Method, device and storage medium for identifying entities in text |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115017906A true CN115017906A (en) | 2022-09-06 |
Family
ID=83071466
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210614127.7A Pending CN115017906A (en) | 2022-05-31 | 2022-05-31 | Method, device and storage medium for identifying entities in text |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115017906A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115757774A (en) * | 2022-11-03 | 2023-03-07 | 中国电子技术标准化研究院 | Industry field identification method and device, electronic equipment and storage medium |
-
2022
- 2022-05-31 CN CN202210614127.7A patent/CN115017906A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115757774A (en) * | 2022-11-03 | 2023-03-07 | 中国电子技术标准化研究院 | Industry field identification method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110222178B (en) | Text emotion classification method and device, electronic equipment and readable storage medium | |
| CN107480143A (en) | Dialogue topic dividing method and system based on context dependence | |
| CN106778878B (en) | Character relation classification method and device | |
| CN110222330B (en) | Semantic recognition method and device, storage medium and computer equipment | |
| JP6291443B2 (en) | Connection relationship estimation apparatus, method, and program | |
| CN112784009B (en) | Method and device for mining subject term, electronic equipment and storage medium | |
| CN113392209A (en) | Text clustering method based on artificial intelligence, related equipment and storage medium | |
| CN114969275A (en) | Conversation method and system based on bank knowledge graph | |
| CN111753082A (en) | Text classification method and device based on comment data, equipment and medium | |
| CN112860896A (en) | Corpus generalization method and man-machine conversation emotion analysis method for industrial field | |
| CN111695358A (en) | Method and device for generating word vector, computer storage medium and electronic equipment | |
| Hasan et al. | Sentiment analysis using out of core learning | |
| CN113360654B (en) | Text classification method, apparatus, electronic device and readable storage medium | |
| CN115017906A (en) | Method, device and storage medium for identifying entities in text | |
| CN114416976A (en) | Text labeling method and device and electronic equipment | |
| CN113934834A (en) | Question matching method, device, equipment and storage medium | |
| CN113609865A (en) | Text emotion recognition method and device, electronic equipment and readable storage medium | |
| CN115035890B (en) | Training method and device of voice recognition model, electronic equipment and storage medium | |
| US20230130662A1 (en) | Method and apparatus for analyzing multimodal data | |
| CN114398482A (en) | Dictionary construction method and device, electronic equipment and storage medium | |
| CN115357720A (en) | Multi-task news classification method and device based on BERT | |
| CN112651590B (en) | Instruction processing flow recommending method | |
| CN115309899A (en) | Method and system for identifying and storing specific content in text | |
| CN115130475A (en) | Extensible universal end-to-end named entity identification method | |
| CN110909546B (en) | Text data processing method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |