CN115221869A

CN115221869A - Entity identification method, medium, device and computing equipment

Info

Publication number: CN115221869A
Application number: CN202210848667.1A
Authority: CN
Inventors: 李家诚; 胡光龙; 侯同鹏; 沙雨辰; 袁威强; 肖康; 卢睿轩
Original assignee: Netease Hangzhou Network Co Ltd
Current assignee: Netease Hangzhou Network Co Ltd
Priority date: 2022-07-19
Filing date: 2022-07-19
Publication date: 2022-10-21

Abstract

The embodiment of the disclosure provides an entity identification method, medium, device and computing equipment. The method comprises the following steps: inputting the text to be recognized into an entity prediction model to predict potential entities contained in the text to be recognized and potential entity categories of the corresponding potential entities; determining a first entry in a corresponding knowledge base based on the potential entity and the potential entity category, the first entry further comprising a corresponding first entry introduction; and inputting the text to be recognized, the first vocabulary entry and the first vocabulary entry brief introduction into the entity recognition model, and determining a target entity in the text to be recognized and a target entity category of the corresponding target entity. The method and the device solve the problem that the entity identification accuracy and reliability are poor when the text data features are insufficient in the related technology, and obviously improve the accuracy and reliability of the entity identification.

Description

Entity identification method, medium, device and computing equipment

Technical Field

Embodiments of the present disclosure relate to the field of computer technologies, and in particular, to an entity identification method, medium, apparatus, and computing device.

Background

This section is intended to provide a background or context to the embodiments of the disclosure recited in the claims. The description herein is not admitted to be prior art by inclusion in this section.

In the fields of man-machine conversation, language translation, and the like, it is important to accurately and unmistakably recognize each type of corresponding text entity contained in given text information. By quickly finding out the corresponding entity under the context in the given text, the corresponding function of the entity can be conveniently called, and the method and the device are widely applied to scenes such as medicine, automobiles, e-commerce and the like.

In the related art, the text entity is generally identified by converting the text into a feature vector and analyzing the feature vector to identify the entity in the text. However, when the text length is short, the corresponding feature vectors are few, and the accuracy of the obtained data features is insufficient, so that the accuracy and the reliability of the obtained recognition result are poor.

Disclosure of Invention

The present disclosure provides an entity identification method, medium, apparatus, and computing device, to solve the problem of poor accuracy and reliability of entity identification when text data features are insufficient in the related art.

In a first aspect of embodiments of the present disclosure, there is provided an entity identification method, including:

inputting the text to be recognized into an entity prediction model to predict potential entities contained in the text to be recognized and potential entity categories of the corresponding potential entities;

determining a first entry in a corresponding knowledge base based on the potential entity and the potential entity category, the first entry further comprising a corresponding first entry introduction;

and inputting the text to be recognized, the first vocabulary entry and the first vocabulary entry brief introduction into an entity recognition model, and determining a target entity in the text to be recognized and a target entity category of the corresponding target entity.

In a second aspect of the disclosed embodiments, there is provided a method for entity prediction model training, comprising:

inputting the training text labeled with the character category label into an entity prediction model for training, outputting a prediction entity contained in the training text and a prediction entity category of the corresponding prediction entity,

the character category label includes:

the first character and entity category label of the entity;

a non-initial character and an entity category label of the entity;

a non-entity tag.

In a third aspect of the disclosed embodiments, there is provided an entity recognition model training method, including:

determining corresponding entries and entry blurbs of training texts in a knowledge base, wherein the training texts are marked with character type labels;

inputting the training text and the corresponding terms and term blurbs of the training text in the entity library into an entity recognition model for training to output final entities contained in the training text and final entity classes of the corresponding final entities,

the character category labels include:

the first character and entity category label of the entity;

a non-initial character and an entity category label of the entity;

a non-entity tag.

In a fourth aspect of embodiments of the present disclosure, there is provided a computer-readable storage medium comprising:

the computer-readable storage medium has stored therein computer-executable instructions for implementing the entity identification method as in the first aspect of the present disclosure when executed by the processor; and/or, the computer executable instructions when executed by the processor are for implementing a method of entity prediction model training as in the second aspect of the present disclosure; and/or the computer executable instructions, when executed by the processor, are for implementing the entity recognition model training method as in the third aspect of the present disclosure.

In a fifth aspect of embodiments of the present disclosure, there is provided an entity identifying apparatus comprising: the prediction module is used for inputting the text to be recognized into the entity prediction model so as to predict potential entities contained in the text to be recognized and potential entity categories of the corresponding potential entities;

the enhancement module is used for determining a first vocabulary entry in a corresponding knowledge base based on the potential entity and the potential entity category, wherein the first vocabulary entry also comprises a corresponding first vocabulary entry brief introduction;

and the recognition module is used for inputting the text to be recognized, the first vocabulary entry and the first vocabulary entry brief introduction into the entity recognition model, and determining a target entity in the text to be recognized and a target entity category of the corresponding target entity.

In a sixth aspect of the disclosed embodiments, there is provided an entity prediction model training apparatus comprising:

a training module used for inputting the training text labeled with the character category label into the entity prediction model for training and outputting the prediction entity contained in the training text and the prediction entity category of the corresponding prediction entity,

the character category labels include:

the first character of the entity and the entity category label;

a non-initial character and an entity category label of the entity;

a non-entity tag.

In a seventh aspect of the disclosed embodiments, there is provided an entity recognition model training apparatus, comprising:

the determining module is used for determining corresponding entries and entry blurbs of the training texts in the knowledge base, and the training texts are labeled with character category labels;

a training module for inputting the training text and the corresponding entries and entry blurs of the training text in the entity library into the entity recognition model for training to output the final entities contained in the training text and the final entity categories of the corresponding final entities,

the character category labels include:

the first character and entity category label of the entity;

a non-initial character and an entity category label of the entity;

a non-entity tag.

In an eighth aspect of embodiments of the present disclosure, there is provided a computing device comprising: at least one processor;

and a memory communicatively coupled to the at least one processor;

wherein the memory stores instructions executable by the at least one processor to cause the computing device to perform the entity identification method as in the first aspect of the disclosure; and/or to cause a computing device to perform a method of entity prediction model training as in the second aspect of the disclosure; and/or to cause a computing device to perform a method of entity recognition model training as in the third aspect of the disclosure.

According to the entity identification method, the medium, the device and the computing equipment of the embodiment of the disclosure, the text to be identified is input into the entity prediction model, the potential entities contained in the text to be identified and the potential entity categories of the corresponding potential entities are predicted, then the first vocabulary entry in the corresponding knowledge base is determined based on the potential entities and the potential entity categories, and the text to be identified, the first vocabulary entry and the first vocabulary entry brief description are input into the entity identification model, so that the target entities in the text to be identified and the target entity categories of the corresponding target entities are determined. Therefore, the text to be recognized can be combined through the entries and the entry blurb in the knowledge base, and the information which can be used for recognition in the combined text is increased, so that the accuracy and reliability of recognizing the entities in the text to be recognized can be obviously improved.

Drawings

The above and other objects, features and advantages of exemplary embodiments of the present disclosure will become readily apparent from the following detailed description read in conjunction with the accompanying drawings. Several embodiments of the present disclosure are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which:

FIG. 1 schematically illustrates an application scenario diagram according to an embodiment of the present disclosure;

fig. 2 schematically shows a flow chart of an entity identification method according to another embodiment of the present disclosure;

FIG. 3 schematically illustrates a flow chart of an entity identification method according to yet another embodiment of the present disclosure;

FIG. 4 schematically illustrates a flow diagram of a method of entity prediction model training in accordance with yet another embodiment of the present disclosure;

FIG. 5a schematically illustrates a flow chart of a method of entity prediction model training in accordance with yet another embodiment of the present disclosure;

FIG. 5b schematically illustrates a flow chart for translating a foreign language training text into a first language training text according to the embodiment illustrated in FIG. 5 a;

FIG. 6 schematically illustrates a flow diagram of a method of entity recognition model training in accordance with yet another embodiment of the present disclosure;

FIG. 7 schematically illustrates a flow diagram of a method of entity recognition model training in accordance with yet another embodiment of the present disclosure;

FIG. 8 schematically shows a structural diagram of a computer-readable storage medium according to yet another embodiment of the present disclosure;

fig. 9 schematically shows a structural diagram of an entity identifying apparatus according to still another embodiment of the present disclosure;

FIG. 10 schematically shows a structural diagram of an entity prediction model training apparatus according to still another embodiment of the present disclosure;

FIG. 11 is a schematic diagram illustrating an architecture of an entity recognition model training apparatus according to still another embodiment of the present disclosure;

fig. 12 schematically shows a structural schematic of a computing device according to yet another embodiment of the present disclosure.

In the drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

Detailed Description

The principles and spirit of the present disclosure will be described with reference to a number of exemplary embodiments. It is understood that these embodiments are given solely for the purpose of enabling those skilled in the art to better understand and to practice the present disclosure, and are not intended to limit the scope of the present disclosure in any way. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

As will be appreciated by one skilled in the art, embodiments of the present disclosure may be embodied as a system, apparatus, device, method, or computer program product. Accordingly, the present disclosure may be embodied in the form of: entirely hardware, entirely software (including firmware, resident software, micro-code, etc.), or a combination of hardware and software.

According to an embodiment of the disclosure, an entity identification method, a medium, an apparatus and a computing device are provided.

In this context, it is to be understood that the terminology which has been used is intended to be in the nature of words of description rather than of limitation. Moreover, any number of elements in the drawings are by way of example and not by way of limitation, and any nomenclature is used solely for differentiation and not by way of limitation.

The following is a description of terms involved in this disclosure:

entity: the term "entity" refers to a specific person, thing, object or concept that can exist objectively and can be distinguished from each other, and is used in this disclosure to mean a word having an actual meaning.

Entity identification: the full name is Named after Entity Recognition, NER for short, and corresponding Chinese is Named Entity Recognition, namely an information extraction technology for acquiring entities from texts, character strings and the like.

The principles and spirit of the present disclosure are explained in detail below with reference to several representative embodiments thereof.

Summary of The Invention

The inventor finds that entity recognition tasks widely exist in the fields of man-machine conversation, translation and the like, or scenes of medicine, automobiles, e-commerce and the like, and it is very important to accurately recognize corresponding entities in various types of texts (including texts extracted from voice and images). The existing entity recognition method mainly converts a text to be recognized into a feature vector through a recognition model, and then determines an entity (and a category thereof) and a non-entity contained in the text to be recognized according to different data features corresponding to the feature vectors of the entity and the non-entity (learned by the recognition model through pre-training), wherein when the text to be recognized is short (the number of the feature vectors available for analysis is small), the text to be recognized contains a plurality of entities to be recognized (at the moment, the data features obtained based on the text to be recognized possibly only contain information of the entity, the feature vectors of the non-entity are small, and the data features of the non-entity are difficult to recognize, so that the entity to be recognized and the non-entity are difficult to distinguish, and the specific category of the entity to be recognized is difficult to determine), or in a scene with complex context and professional field (because the recognition model learns less data features of such context or field, the recognition accuracy is poor), when the data features are analyzed directly through the feature vectors, the credibility is poor, the entity in the scene is easy to occur, and the entity cannot be recognized accurately, or the accuracy or the reliability when the entity is poor.

In the scheme, the potential entities contained in the text to be recognized are determined by using the entity prediction model, the entries corresponding to the potential entities are found in the knowledge base, and after the entries are combined with the text to be recognized, the entries are recognized again through the entity recognition model, so that the characteristic quantity of available data in the text to be recognized is increased, and the accuracy and the reliability of entity recognition are obviously improved.

Having described the basic principles of the present disclosure, various non-limiting embodiments of the present disclosure are described in detail below.

Application scene overview

Referring first to fig. 1, after a user inputs data (which may be text, voice or video) 110 containing text to be recognized into the terminal device 100, the terminal device 100 determines an entity and an entity category 120 contained therein based on the text to be recognized, so as to implement an entity recognition process.

It should be noted that, in the scenario shown in fig. 1, the terminal device, the data including the text to be recognized, and the entity category thereof are illustrated as an example, but the disclosure is not limited thereto, that is, the number of the terminal device, the data including the text to be recognized, the entity and the entity category thereof may be any.

Exemplary method

In the following, in connection with the application scenario of fig. 1, a method for entity identification according to an exemplary embodiment of the present disclosure is described with reference to fig. 2 to 5. It should be noted that the above application scenarios are merely illustrated for the convenience of understanding the spirit and principles of the present disclosure, and the embodiments of the present disclosure are not limited in this respect. Rather, embodiments of the present disclosure may be applied to any scenario where applicable.

Fig. 2 is a flowchart of an entity identification method according to an embodiment of the present disclosure. The entity identification method provided by the disclosure is applied to terminal equipment or a server. As shown in fig. 2, the entity identification method provided in this embodiment includes the following steps:

step S201, inputting the text to be recognized into the entity prediction model to predict the potential entities contained in the text to be recognized and the potential entity categories of the corresponding potential entities.

Specifically, the text to be recognized may be text included in characters input by the user, may also be text extracted from a picture or video (picture), and may also be text extracted from audio or video (sound).

The potential entity is an entity contained in the text to be recognized, and the category of the potential entity is a potential entity category, for example, the potential entity is Zhang III, and the potential entity category is a name of a person. The potential entities may be one or more, and the category of the potential entities of each potential entity in the text to be recognized may also be one or more, such as "apple" which is a potential entity in "apple on table", and the category of the potential entities may be "electronic equipment" or "food".

The entity prediction model is a pre-trained neural network model capable of executing an entity recognition task, a text to be recognized is input into the entity prediction model, each character in the text to be recognized is converted into a feature vector by the entity prediction model, then a potential entity, a category of the potential entity and a score of each character in the potential entity which are determined based on the text to be recognized are output by analyzing the data feature of each feature vector, and the score is calculated and determined based on the data feature corresponding to the feature vector of each character.

The entity prediction model comprises an encoder part and a decoder part, wherein the encoder part is used for converting the text to be recognized into a feature vector, the decoder part determines the evaluation score (or score) of each character according to the feature vector, and the entity in the text can be determined according to the evaluation score.

In an exemplary embodiment of the present disclosure, there may be one or more decoders in the entity prediction model, for example, multiple decoders respectively determine evaluation scores corresponding to feature vectors, and then combine the evaluation scores to jointly determine an entity therein, so as to improve the accuracy of the determination result.

In an exemplary embodiment of the disclosure, the text to be recognized that is input into the entity prediction model includes a label of each character, for example, an entity type, and a non-entity are all labeled, at this time, an encoder portion of the entity prediction model obtains a corresponding feature vector according to each character and the label corresponding to the character, and a decoder portion determines a corresponding evaluation score based on the feature vector.

The potential entities determined by the entity prediction model may be entities in the text to be recognized, and recognition errors may occur, which are inconsistent with the entities or entity classes actually contained in the text to be recognized. If the text to be recognized is "apple really good, the entity prediction model may recognize the entity category of the entity" apple "as an electronic product, but actually the entity category should be food or fruit.

Step S202, determining a first entry in a corresponding knowledge base based on the potential entity and the potential entity category.

Wherein the first entry further comprises a corresponding first entry introduction.

Specifically, the knowledge base may be a database provided by a network service provider, such as various encyclopedia databases or thesis databases; or may be a database established in an internal server, such as an internal description information database.

The knowledge base includes entries and entry blurbs corresponding to the entries, and may further include other different types of contents such as specific descriptions, application examples, and corresponding names in other languages, but in this embodiment, only the entries and the entry blurbs corresponding to the entries are needed.

The different kinds of content in the knowledge base can be directly determined by the label of each kind of content. And determining a first entry corresponding to the potential entity, comparing the information labeled as the entry and the entry introduction with the categories of the potential entity and the potential entity, and determining the entry and the entry introduction with the same comparison result as the first entry and the first entry introduction.

Illustratively, if the potential entity and the potential entity category thereof are ' plane ' and ' vehicle ', four entries with the entry name ' plane ' are obtained by querying the knowledge base, keywords contained in the profile information of the entries are ' vehicle ', ' book ', ' song ', ' role ' and the like, and the entry profile containing the keywords same as the potential entity category is determined as a first entry profile by comparing the keywords in the entry profile with the potential entity category (if the entry profile is ' plane ' which has one or more engines 82308230; 8230; at atmospheric layer 8230; at the traffic vehicle for flying '), the corresponding entry is determined as the first entry).

In an exemplary embodiment of the present disclosure, the first vocabulary entry and the first vocabulary entry blurb corresponding to the potential entity and the potential entity category may be obtained by directly obtaining information labeled as vocabulary entry and vocabulary entry blurb, comparing the vocabulary entry blurb part in the obtained information with the potential entity category, and comparing the vocabulary entry blurb part with the potential entity category.

In an exemplary embodiment of the present disclosure, the knowledge base corresponding to the underlying entity is predetermined and generally remains unchanged during use (e.g., only an administrator modifying the configuration will change the knowledge base corresponding to the underlying entity of the text to be recognized).

There may be one or more knowledge bases corresponding to the potential entities, one or more knowledge bases, such as an encyclopedia database and an internal description information database.

In an exemplary embodiment of the disclosure, there are multiple knowledge bases corresponding to the potential entities, and the server or the processor determines the first entry in each knowledge base corresponding to the potential entity and the potential entity category, respectively. If each knowledge base comprises the first entry, the first entry uniquely corresponding to the potential entity and the category of the potential entity can be determined based on the priority of the knowledge base and/or the searching degree of the first entry in the plurality of first entries.

In an exemplary embodiment of the present disclosure, if the first entry corresponding to the potential entity does not exist in the knowledge base, the potential entity is removed and no further processing is performed.

Step S203, inputting the text to be recognized, the first vocabulary entry and the first vocabulary entry brief introduction into an entity recognition model, and determining a target entity in the text to be recognized and a target entity category of the corresponding target entity.

Specifically, the text to be recognized, the first vocabulary entry and the first vocabulary entry introduction may be spliced together in a fixed order (for example, fixed as the text to be recognized, the first vocabulary entry and the first vocabulary entry introduction, or the first vocabulary entry, the text to be recognized and the first vocabulary entry introduction), and then input into the entity recognition model; or directly inputting the text to be recognized, the first entry and the introduction of the first entry into the entity recognition model in sequence according to a fixed sequence (corresponding to different input modes, and different training modes of the entity recognition model), wherein the data results are the target entity and the target entity category contained in the text to be recognized.

The entity recognition model is similar to the entity prediction model and is a pre-trained neural network model capable of executing an entity recognition task, but a first vocabulary entry and a first vocabulary entry brief introduction of a text to be recognized are input into the entity recognition model, and a target entity, a category of the target entity and a score of each character in the target entity in the text to be recognized are output. The entity recognition model does not recognize the input first entry and the entities contained in the first entry brief introduction, because the first entry brief introduction may also contain other entities per se so as to better explain the first entry, but these entities may not exist in the text to be recognized, for example, the entry brief introduction of the entry "puer tea" contains entities for explaining the entry, such as "Yunnan province" and "arbor", but the corresponding text to be recognized is "want to drink puer tea", and only contains the entity "puer tea", and does not contain two entities, namely "Yunnan province" and "arbor".

Furthermore, because the first vocabulary entry and the first vocabulary entry blurb have strong relevance with the target entity in the text to be recognized, the purpose is to help the entity recognition model to utilize the feature vectors corresponding to the characters in the first vocabulary entry and the first vocabulary entry blurb, to assist in judging the data features of the entity in the text to be recognized (the more feature vectors, the more accurate and reliable the data features determined according to the feature vectors), and further to better determine the target entity and the target entity category in the text to be recognized.

In an exemplary embodiment of the present disclosure, each character in the text to be recognized input into the entity recognition model includes a label (e.g., a character type label), while the first term and the first term profile do not include labels, whereby the entity recognition model may determine only the target entity and the target entity type corresponding to the portion including the label.

In an exemplary embodiment of the present disclosure, the first term and the first term profile may also be labeled separately (to indicate that the target entity does not need to be determined for this part of the content), and in this case, the entity recognition model only determines the corresponding target entity and the target entity category that do not include the labeled part.

In an exemplary embodiment of the present disclosure, the text to be recognized, the first vocabulary entry and the first vocabulary entry brief introduction may also respectively adopt different labels to indicate differences, and at this time, the entity recognition model only determines the target entity and the target entity category corresponding to the part of the content of the text to be recognized, which is indicated by the label.

Potential entities contained in the text to be recognized and potential entity categories of the corresponding potential entities are predicted by inputting the text to be recognized into an entity prediction model, then a first vocabulary entry in a corresponding knowledge base is determined based on the potential entities and the potential entity categories, and the text to be recognized, the first vocabulary entry and the introduction of the first vocabulary entry are input into an entity recognition model, so that a target entity in the text to be recognized and a target entity category of the corresponding target entity are determined. Therefore, the text to be recognized can be combined through the entries and the entry introduction in the knowledge base, the information which can be used for recognition in the combined text is increased, the number of the feature vectors which can be used for analysis is increased, the accuracy of the data features determined based on the feature vectors is improved, and therefore the accuracy of the entities and the entity categories in the text to be recognized determined based on the data features is improved.

Fig. 3 is a flowchart of an entity identification method according to an embodiment of the present disclosure. As shown in fig. 3, the entity identification method provided in this embodiment includes the following steps:

step S301, inputting the text to be recognized into the entity prediction model to predict the potential entities contained in the text to be recognized and the potential entity categories of the corresponding potential entities.

The entity prediction model comprises a first coder and a first decoder, wherein the first coder is used for identifying and coding the text to be identified, and the first decoder is used for determining potential entities in the text to be identified and potential entity classes of the corresponding potential entities based on the output result of the first coder.

Specifically, after a text to be recognized is recognized and encoded by the first encoder, a feature vector corresponding to the text to be recognized is obtained, and the first decoder directly determines potential entities and potential entity classes in the text to be recognized based on the feature vector.

Because the entity prediction model mainly plays a role in finding out potential entities in the text to be recognized, the accuracy of the potential entities does not need to be ensured, and therefore, only one first decoder can be included in the entity prediction model.

In an exemplary embodiment of the present disclosure, the first encoder may be a BERT model or a blstm model, and the first decoder may be a CRF model or a SPAN model. These models are all neural network models that can be used in the related art for entity recognition tasks.

Step S302, determining the entry with the same name as the potential entity in the knowledge base as a first entry.

Wherein, the knowledge base stores the vocabulary entry, the vocabulary entry brief introduction and the vocabulary entry hot value in a triple mode.

Specifically, the knowledge base in this embodiment necessarily includes the term, the term introduction, and the term heat value, and these three parameters are stored in the form of a triple (or ternary array), for example, (term, term introduction, term heat value), where the term heat value is used to reflect the number of times the term is searched/viewed/used.

Therefore, when the first entry is determined, the triples in the knowledge base are directly read, matching is performed on the entries and the potential entities based on the entries, the triples with the entries identical to the potential entities are determined, the entries in the triples are used as the first entries, and entry blurs in the triples are used as the first entry blurs.

Step S303, in response to the fact that the entries with the same name as the potential entity are not unique, uniquely determining a first entry based on the entry heat value of each corresponding entry with the same name.

Specifically, if at least two entries with the same name as the potential entity can be obtained based on the matching between the entries and the potential entity (the triples corresponding to the entries are different, including entry introduction difference and entry hot value difference), one entry with the same name needs to be selected as the first entry.

Specifically, the entry with the highest rank hot value determined by the ranking may be determined as the first entry based on the reverse rank ranking of the entry hot values.

In an exemplary embodiment of the present disclosure, the entry having the highest entry popularity value for each entry of the same name is uniquely determined as the first entry.

Specifically, the entry corresponding to the highest degree of heat value among the entry degree of heat values corresponding to the plurality of entries of the same name may be directly obtained without sorting, and the entry is determined as the first entry.

In an exemplary embodiment of the present disclosure, the entry with the highest lexical item heat value in the plurality of entry profiles including the first lexical item category in the keyword (or synonym of the keyword) may be determined as the first entry based on the matching of the keyword and the first entry category in the entry profiles and the reverse order of the entry heat values.

Specifically, if the text to be recognized is ' Pu ' er tea ', the potential entity category corresponding to the potential entity ' Pu ' er tea ' is ' book ', the corresponding entry is Pu ' er tea, but if the word entries corresponding to the ' Pu ' er tea ' are arranged in reverse order according to the heat value of each entry, the key word corresponding to the highest heat degree is ' tea, and obviously does not correspond to the potential entity category, so that the entry with the highest heat degree in the entry introduction of the entry introduction (such as a popular science book published in 2004) containing the ' book ' (or synonyms: book and book) in the key words of the entry introduction is determined as a first entry, and the corresponding entry introduction is determined as a first entry introduction.

And S304, splicing the text to be recognized with the first entry and the first entry brief introduction to obtain an information enhanced text corresponding to the text to be recognized.

Specifically, unlike the case where the text to be recognized is directly input in the entity prediction model, the information enhanced text including the text to be recognized, the first vocabulary entry, and the first vocabulary entry introduction is input.

And the splicing sequence of the text to be recognized and the first vocabulary entry introduction is fixed, and if the text to be recognized is followed by the first vocabulary entry and the first vocabulary entry introduction in turn. Therefore, the entity recognition model can be conveniently trained, and the accuracy of entity recognition is ensured.

Step S305, inputting the information enhanced text into the entity recognition model to obtain a target entity in the text to be recognized and a target entity category of the corresponding target entity.

Specifically, the text portion to be recognized, the first entry and the first entry in the information enhanced text may have different labels (described in reference to step S203 in the embodiment shown in fig. 2), so that the entity recognition model can perform recognition based on the information enhanced text, and only the target entity and the target entity category corresponding to the text portion to be recognized are output.

The entity recognition model comprises a second encoder and a second decoder, the second encoder is used for recognizing and encoding the information enhancement text, and the second decoder is used for determining target entities in the text to be recognized and target entity classes of the corresponding target entities based on output results of the second encoder.

Specifically, the entity recognition model also comprises a second encoder and a second decoder respectively, and because the feature vectors contained in the information enhancement texts input in the second encoder are far more than those of the texts to be recognized independently, the accuracy of the output target entities and the accuracy of the types of the target entities can be effectively ensured even if only one second decoder is provided.

According to the entity identification method disclosed by the embodiment of the disclosure, potential entities contained in the text to be identified and potential entity categories of the corresponding potential entities are predicted by inputting the text to be identified into an entity prediction model; and then determining that the entries with the same name as the potential entities in the knowledge base are first entries, uniquely determining the first entries based on the entry heat values of the corresponding entries with the same name when responding to the non-uniqueness of the entries with the same name as the potential entities, splicing the text to be recognized with the first entries and the brief introduction of the first entries to obtain information enhanced texts corresponding to the text to be recognized, and finally inputting the information enhanced texts into an entity recognition model to obtain target entities in the text to be recognized and target entity categories of the corresponding target entities. Therefore, the first entry and the first entry brief introduction corresponding to the potential entity are accurately and effectively determined, the relevance between the content in the information enhanced text and the entity and entity category in the text to be identified is further improved, and the accuracy and reliability of the target entity identified by the entity identification model are further enhanced.

Fig. 4 is a flowchart of a method for training an entity prediction model according to an embodiment of the present disclosure. The entity prediction model training method is applied to the embodiments shown in fig. 2 and 3. As shown in fig. 4, the entity prediction model training method provided in this embodiment includes the following steps:

step S401, inputting the training text labeled with the character type label into an entity prediction model for training, and outputting a prediction entity contained in the training text and a prediction entity type of the corresponding prediction entity.

Wherein the character category label includes: the first character and entity category label of the entity; a non-initial character and an entity category label of the entity; a non-entity tag.

Specifically, the training text used for training the entity prediction model is a text labeled with a character type label in advance.

The character category label is a label for respectively marking the category of each character in the training text, and the category comprises a first character and an entity category label of the entity, a non-first character and an entity category label of the entity and a non-entity label.

Illustratively, the training text is "chocolate good eating", wherein "chocolate" is an entity, the entity category is food, and "true good eating" is non-entity, and thus, the word "chocolate" may be added with the entity first character and the entity category is a character category label for food (e.g., B-food, wherein B represents the entity first character and-food represents the entity category is food), both "gram" and "strength" may be added with the entity non-first character and the entity category is a character category label for food (e.g., I-food, wherein I represents the entity non-first character), and then "true", "good", "eating" may be added with the non-entity label (e.g., O, for representing non-entity). Then there are: chocolate (I-food) chocolate (I) food) true (O) good (O) eat (O).

The training text may select a text to which a character category label has been added, or may select a text that does not include a character category label, and then manually add a character category label, or select a text to which an entity has been identified but to which a character category label has not been added, and then automatically add a character category label based on the identification result (the automatic addition of a character category label may be implemented based on an existing algorithm, and is not described here any more since it is not a protection content of the present scheme).

According to the entity prediction model training method disclosed by the embodiment of the disclosure, the training text labeled with the character category label is input into the entity prediction model for training, and the prediction entity contained in the training text and the prediction entity category of the corresponding prediction entity are output. Therefore, the entity prediction model can be focused on the tasks of entity and entity type identification, and the effectiveness of the identification result can be ensured.

Fig. 5a is a flowchart of a method for training a solid prediction model according to an embodiment of the disclosure.

As shown in fig. 5a, the entity prediction model training method provided in this embodiment includes the following steps:

step S501, translating the foreign language training text into a training translation corresponding to the first language.

Wherein the entries and entry blunts determined for the predicted entity and the predicted entity category in the respective knowledge bases correspond to the first language.

Specifically, since the text to be recognized by the entity prediction model may be obtained based on different languages, when the entity prediction model is trained, translation of the different languages is involved, and the translation is used for training the entity prediction model, so as to improve prediction accuracy and robustness of the entity prediction model.

Further, as shown in fig. 5b, which is a flowchart for translating the foreign language training text into the first language training text, the translation process specifically includes the following steps:

step S5011, determining entity vocabularies in the foreign language training texts based on the character category labels.

Specifically, the foreign language training text is also a text containing character type labels, wherein the character type labels in the foreign language text do not necessarily correspond to characters, and may also correspond to words with actual meanings. Therefore, a previously labeled foreign language text is usually selected as the foreign language training text.

Illustratively, if the foreign language training text is "apple is tasty" (Chinese translation: apple is very delicious), then a, p will not be labeled separately, but will only be labeled: applet (B-food) is (O) tasty (O), wherein "applet" is an entity; correspondingly, if the foreign language training text is "1245083\\12362123561235635: \1245083 (B-food) 12503 (I-food) 12523 (I-food) 12362 (O) 1235675 (O) wherein (O) 12450831250312523 (O) is an entity.

And S5012, replacing the entity vocabulary in the foreign language training text with the place-occupying character, and translating the replaced foreign language training text to correspond to the first language.

Specifically, in the process of translating the foreign language training text into the training translation, the position of the entity may move, and therefore, each character type label and the corresponding text (character or vocabulary) need to move along with each other, so as to avoid the situation that the character type label does not correspond to the training translation.

Illustratively, the entity "key" in "where is the key" is at the end of the sentence, and the translation into Chinese is "where is the key", and the entity "key" corresponding to "key" is at the beginning of the sentence.

Because the translation of each sentence is mainly based on sentence patterns and virtual words rather than entities, the solid words are replaced by placeholder characters (generally, characters which cannot be translated, such as @ symbols), and then the translation is performed based on other parts in the foreign language training text.

Exemplary, "where is the key? "middle entity" key "can be replaced with @, then the sentence becomes: "where is the @? "to be directly translated: "@ where? ", i.e., obtaining a translation of a portion outside the physical vocabulary.

And S5013, determining the position of the placeholder character in the translation result, and restoring the placeholder character into an entity vocabulary.

Specifically, after the foreign language training text of the part except the entity vocabulary is translated, the entity vocabulary is translated independently, so that the space-occupying characters are reduced into the entity vocabulary firstly, and the entity vocabulary is translated conveniently. Such as where is "@? Where is "restore before" key? ".

And S5014, translating the entity vocabulary obtained by reduction in the translation result to correspond to the first language to obtain a training translation corresponding to the first language.

Specifically, after the entity vocabulary is translated independently, the entity vocabulary is replaced into a translation result, and then the corresponding training translation can be obtained. If "key" is translated into a key, then "key is there? "where can be found" is the key? "as a training version.

Step S502, inputting the training translation into the entity prediction model for training so that the entity prediction model outputs a prediction entity corresponding to the training translation and a prediction entity category of the corresponding prediction entity.

Specifically, the training translation is substituted into the entity prediction model to perform first-stage training, the robustness and the effectiveness of the entity prediction model on the translated text are improved, and then the training text with the first language is substituted into the entity prediction model to perform second-stage training, so that the reliability and the effectiveness of the recognition result of the entity prediction model are improved.

Step S503, inputting the training text labeled with the character type label into an entity prediction model for training, and outputting the prediction entity contained in the training text and the prediction entity type of the corresponding prediction entity.

Specifically, this step is a second stage of training, that is, a training text labeled with a character category label in the first language is input into the entity prediction model for training, and specific contents may refer to corresponding descriptions in the embodiment shown in fig. 4, and are not described here again.

According to the entity prediction model training method of the embodiment of the disclosure, a training translation is obtained after foreign language training texts with different languages from the vocabulary entries and the vocabulary entry brief introduction in the knowledge base are translated, the entity prediction model is trained by the training translation, and after the training is finished, the entity prediction model is trained by the training texts with the same languages as the vocabulary entries and the vocabulary entry brief introduction in the knowledge base.

Fig. 6 is a flowchart of an entity recognition model training method according to an embodiment of the present disclosure. The entity recognition model training method is applied to the embodiments shown in fig. 2 and 3. As shown in fig. 6, the entity recognition model training method provided in this embodiment includes the following steps:

step S601, determining corresponding entries and entry blurbs of the training texts in the knowledge base, wherein the training texts are marked with character type labels.

Specifically, the training text itself is a text including a character category label, and the method for obtaining the training text may refer to the embodiment shown in fig. 4, which is not described herein again.

After the training text is obtained, the corresponding entries and entry blurbs of the entities in the training text in the knowledge base need to be determined. Entities in the training text can be obtained through an entity prediction model; or directly selecting a text of a predetermined entity as a training text; the entity recognition model may also be trained (at a first stage) using the training text to obtain the entity and entity class output by the entity recognition model (and then the entity-corresponding terms and term introduction are combined with the training text to perform a second stage of training on the entity recognition model).

The specific steps of determining the corresponding entries and entry blurbs in the knowledge base through the entities in the training text may refer to the embodiment shown in fig. 3, which is not described herein again.

Step S602, inputting the training text labeled with the character category label, the corresponding entry and entry brief introduction of the entity in the training text in the knowledge base into an entity recognition model for training, and outputting the final entity contained in the training text and the final entity category of the corresponding final entity.

Wherein the character category label includes: the first character and entity category label of the entity, the non-first character and entity category label of the entity, and the non-entity label.

Specifically, since the training text input to the entity recognition model includes character category labels, and the vocabulary entry and vocabulary entry introduction do not include character category labels, the encoder of the entity recognition model encodes all the contents input to the entity recognition model to obtain feature vectors, but the decoder of the entity recognition model calculates evaluation scores of the characters based on the training text including the character category labels (the evaluation scores are determined based on all the feature vectors), and determines the entities in the training text according to the evaluation scores, the entity recognition model can determine only the final entities in the training text, and does not output the entities in the vocabulary entry and vocabulary entry introduction.

According to the entity recognition model training method of the embodiment of the disclosure, the corresponding vocabulary entry and vocabulary entry brief introduction of the training text in the knowledge base are determined, then the training text labeled with the character category label and the corresponding vocabulary entry and vocabulary entry brief introduction of the entity in the training text in the knowledge base are input into the entity recognition model for training, and the final entity contained in the training text and the final entity category of the corresponding final entity are output, so that the entity recognition model can recognize the final entity and the final entity type in the training text according to the training text, the vocabulary entry and the vocabulary entry brief introduction, and the accuracy of the entity recognition model recognition is remarkably improved.

Fig. 7 is a flowchart of an entity recognition model training method according to an embodiment of the present disclosure. As shown in fig. 7, the entity recognition model training method provided in this embodiment includes the following steps:

step S701, translating the foreign language training text into a training translation corresponding to the first language.

The first vocabulary entry and the brief introduction of the first vocabulary entry in the knowledge base correspond to a first language, and the training text comprises foreign language training text corresponding to a second language.

The translation process specifically comprises the following steps:

step one (not shown), based on the character category labels, entity vocabularies in the foreign language training texts are determined.

And step two (not shown), replacing the entity vocabulary in the foreign language training text with the placeholder characters, and translating the replaced foreign language training text to correspond to the first language.

And step three (not shown), determining the position of the placeholder character in the translation result, and restoring the placeholder character into an entity vocabulary.

And step four (not shown), translating the entity vocabulary obtained by reduction in the translation result to correspond to the first language to obtain a training translation corresponding to the first language.

Step S702, inputting the training translation into the entity recognition model for training, so that the entity recognition model outputs a final entity corresponding to the training translation and a final entity category of the corresponding final entity.

Specifically, the process of training the entity recognition model through the foreign language training text and translating the foreign language training text to obtain the training translation is the same as the content of the corresponding step in the training process of the entity prediction model in the embodiment shown in fig. 5, and is not repeated here.

And step S703, determining the entries in the knowledge base with the same name as the entities in the training texts as corresponding entries.

Step S704, in response to the fact that the entries with the same name as the entity in the training text are not unique, uniquely determining the corresponding entries and entry blurb of the corresponding entries based on the entry heat values of the corresponding entries with the same name.

The entry with the highest entry hot value corresponding to each entry with the same name is uniquely determined as the corresponding entry and the entry introduction of the corresponding entry.

Specifically, the determination manner of the entity in the training text may refer to the corresponding description in the embodiment of fig. 6; the method for determining the corresponding entries and entry blurbs of the entities in the training text through the knowledge base may refer to the corresponding contents in the embodiment shown in fig. 3, which are not described herein again.

The execution sequence of steps S701 to S702 and steps S703 to S704 in practical application may be changed arbitrarily, that is, the entity recognition model may be trained by the foreign language training text (the training translation thereof), and then the vocabulary entry and the vocabulary entry introduction of the entity corresponding to the training text are determined; or determining the entries and entry blurbs of entities corresponding to the training texts, and then training the entity recognition model by using the foreign language training texts.

Step S705, inputting the training text and the corresponding vocabulary entry and vocabulary entry blurb of the training text in the entity library into the entity recognition model for training, so as to output the final entity contained in the training text and the final entity category of the corresponding final entity.

Specifically, after the entity recognition model is trained (in the first stage) with the foreign language training text (training version thereof), the entity recognition model is trained (in the second stage) with the training text, the vocabulary entry and the vocabulary entry introduction, for which specific contents refer to corresponding descriptions in the embodiment shown in fig. 6 and are not repeated here.

According to the entity prediction model training method of the embodiment of the disclosure, a training translation is obtained by translating foreign language training texts of different languages from the terms and the term introductions in the knowledge base, the entity recognition model is trained by using the training translation, the terms and the term introductions corresponding to the entities in the training texts in the knowledge base are determined, and then the training texts, the terms and the term introductions are input into the entity recognition model for training.

Exemplary Medium

Having described the method of the exemplary embodiment of the present disclosure, next, the storage medium of the exemplary embodiment of the present disclosure will be described with reference to fig. 8.

Referring to fig. 8, a storage medium 80 stores therein a program product for implementing the above method according to an embodiment of the present disclosure, which may employ a portable compact disc read only memory (CD-ROM) and includes computer-executable instructions for causing a computing device to perform the entity identification method provided by the present disclosure. However, the program product of the present disclosure is not limited thereto.

The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

A readable signal medium may include a propagated data signal with computer-executable instructions embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. The readable signal medium may also be any readable medium other than a readable storage medium.

Computer-executable instructions for carrying out operations of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer executable instructions may execute entirely on the user computing device, partly on the user device, partly on a remote computing device, or entirely on the remote computing device or server. In situations involving remote computing devices, the remote computing devices may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN).

Exemplary devices

After introducing the medium of the exemplary embodiment of the present disclosure, an entity identification apparatus of the exemplary embodiment of the present disclosure is described below with reference to fig. 9 for implementing an entity identification method in any one of the method embodiments, an entity prediction model training apparatus of the exemplary embodiment of the present disclosure is described below with reference to fig. 10 for implementing an entity prediction model training method in any one of the method embodiments, and an entity identification model training apparatus of the exemplary embodiment of the present disclosure is described below with reference to fig. 11 for implementing an entity identification model training method in any one of the method embodiments.

The present disclosure provides an entity identifying apparatus 900, including:

the prediction module 910 is configured to input the text to be recognized into the entity prediction model to predict potential entities included in the text to be recognized and potential entity categories of the corresponding potential entities;

an enhancement module 920 configured to determine a first term in the corresponding knowledge base based on the potential entity and the potential entity category, the first term further including a corresponding first term introduction;

the recognition module 930 is configured to input the text to be recognized, the first term and the first term introduction into the entity recognition model, and determine a target entity in the text to be recognized and a target entity category of the corresponding target entity.

In an exemplary embodiment of the disclosure, the enhancing module 920 is specifically configured to: when the vocabulary entry, the vocabulary entry introduction and the vocabulary entry heat value are stored in the knowledge base in a triple mode, determining the vocabulary entry with the same name as the potential entity in the knowledge base as a first vocabulary entry; in response to the entry being non-unique with respect to the potential entity, a first entry is uniquely determined based on the entry-heat values of the respective entries.

In an exemplary embodiment of the disclosure, the enhancing module 920 is specifically configured to: and in response to the non-uniqueness with the homonym entry of the potential entity, determining the entry with the highest entry hot value of the corresponding homonym entry as the first entry.

In an exemplary embodiment of the disclosure, the identifying module 930 is specifically configured to: splicing the text to be recognized with the first entry and the first entry brief introduction to obtain an information enhanced text corresponding to the text to be recognized; and inputting the information enhanced text into the entity recognition model to obtain a target entity in the text to be recognized and a target entity category of the corresponding target entity.

In an exemplary embodiment of the disclosure, the entity prediction model includes a first encoder and a first decoder, the first encoder is used for identifying and encoding a text to be identified, and the first decoder is used for determining potential entities in the text to be identified and potential entity classes of the corresponding potential entities based on an output result of the first encoder; the entity recognition model comprises a second encoder and a second decoder, wherein the second encoder is used for recognizing and encoding the information enhancement text, and the second decoder is used for determining target entities in the text to be recognized and target entity classes of the corresponding target entities based on output results of the second encoder.

In an exemplary embodiment of the present disclosure, the prediction module 910 includes: the entity prediction model is obtained by training in the following way: inputting the training text marked with the character category label into an entity prediction model for training, and outputting a prediction entity contained in the training text and a prediction entity category of the corresponding prediction entity, wherein the character category label comprises: the first character of the entity and the entity category label; a non-initial character and an entity category label of the entity; a non-entity tag.

In an exemplary embodiment of the present disclosure, the prediction module 910 includes: responding to a first vocabulary entry and a first vocabulary entry brief introduction in a knowledge base corresponding to a first language, and a foreign language training text corresponding to a second language included in the training text, before inputting the training text labeled with a character category label into an entity prediction model for training and outputting a predicted entity included in the training text and a predicted entity category of a corresponding predicted entity: translating the foreign language training text into a training translation corresponding to the first language; and inputting the training translation into the entity prediction model for training so that the entity prediction model outputs a prediction entity corresponding to the training translation and a prediction entity category of the corresponding prediction entity.

In an exemplary embodiment of the present disclosure, the prediction module 910 includes: determining entity vocabularies in the foreign language training text based on the character category labels; replacing the entity vocabulary in the foreign language training text with place-occupying characters, and translating the replaced foreign language training text to correspond to the first language; determining the position of the placeholder character in the translation result, and reducing the placeholder character into an entity vocabulary; and translating the entity vocabulary obtained by reduction in the translation result to correspond to the first language to obtain a training translation corresponding to the first language.

In an exemplary embodiment of the present disclosure, the identifying module 930 includes: inputting training texts marked with character type labels, corresponding terms and term blurbs of entities in the training texts in a knowledge base into an entity recognition model for training, and outputting final entities contained in the training texts and final entity types of the corresponding final entities, wherein the character type labels comprise: the first character and entity category label of the entity; a non-initial character and an entity category label of the entity; a non-entity tag.

In an exemplary embodiment of the present disclosure, the identifying module 930 includes: in response to the vocabulary entry and the vocabulary entry introduction in the knowledge base corresponding to the first language, the training text including a foreign language training text corresponding to the second language, translating the foreign language training text into a training translation corresponding to the first language; and inputting the training translation into the entity recognition model for training so that the entity recognition model outputs a final entity corresponding to the training translation and a final entity category of the corresponding final entity.

The entity prediction model training apparatus 1000 provided by the present disclosure includes:

a training module 1010, configured to input a training text labeled with a character category label into an entity prediction model for training, and output a predicted entity included in the training text and a predicted entity category of a corresponding predicted entity, where the character category label includes:

the first character of the entity and the entity category label;

a non-initial character and an entity category label of the entity;

a non-entity tag.

In an exemplary embodiment of the present disclosure, the training module 1010 is further configured to: responding to the predicted entity and the predicted entity category, determining that the vocabulary entry and the vocabulary entry brief introduction which correspond to the first language in the corresponding knowledge base, wherein the training text comprises foreign language training text corresponding to the second language, and translating the foreign language training text into a training translation corresponding to the first language before inputting the training text labeled with the character category label into an entity prediction model for training and outputting the predicted entity contained in the training text and the predicted entity category of the corresponding predicted entity; and inputting the training translation into the entity prediction model for training so that the entity prediction model outputs a prediction entity corresponding to the training translation and a prediction entity category of the corresponding prediction entity.

In an exemplary embodiment of the present disclosure, the training module 1010 is specifically configured to: determining entity vocabularies in the foreign language training text based on the character category labels; replacing the entity vocabulary in the foreign language training text with place-occupying characters, and translating the replaced foreign language training text to correspond to the first language; determining the position of the placeholder character in the translation result, and reducing the placeholder character into an entity vocabulary; and translating the entity vocabulary obtained by reduction in the translation result to correspond to the first language to obtain a training translation corresponding to the first language.

The present disclosure provides an entity recognition model training apparatus 1100, comprising:

a determining module 1110, configured to determine corresponding entries and entry blurbs of training texts in a knowledge base, where the training texts are labeled with character category labels;

a training module 1120, for inputting the training text and the corresponding vocabulary entry and vocabulary entry introduction of the training text in the entity library into the entity recognition model for training, so as to output the final entity contained in the training text and the final entity category of the corresponding final entity,

the character category labels include:

the first character and entity category label of the entity;

a non-initial character and an entity category label of the entity;

a non-entity tag.

In an exemplary embodiment of the disclosure, the determining module 1110 is specifically configured to: in response to the fact that entries, entry blurbs and entry hot values are stored in the knowledge base in a triple mode, determining the entries with the same names as entities in the training texts in the knowledge base as corresponding entries;

and in response to the fact that the entries with the same names as the entities in the training text are not unique, uniquely determining the corresponding entries and entry blurb of the corresponding entries based on the entry heat values of the corresponding entries with the same names.

In an exemplary embodiment of the disclosure, the determining module 1110 is specifically configured to: and uniquely determining the entry with the highest entry heat value corresponding to each entry with the same name as the corresponding entry and the entry brief introduction of the corresponding entry.

In an exemplary embodiment of the disclosure, training module 1120 is specifically configured to: in response to the vocabulary entry and the vocabulary entry introduction in the knowledge base corresponding to the first language, the training text including a foreign language training text corresponding to the second language, translating the foreign language training text into a training translation corresponding to the first language before inputting the training text and the vocabulary entry and vocabulary entry introduction corresponding to the training text in the entity base into the entity recognition model for training to output a final entity contained in the training text and a final entity class of the corresponding final entity; and inputting the training translation into the entity recognition model for training so that the entity recognition model outputs a final entity corresponding to the training translation and a final entity category of the corresponding final entity.

In an exemplary embodiment of the disclosure, training module 1120 is specifically configured to: determining an entity vocabulary in the foreign language training text based on the character category label; replacing the entity vocabulary in the foreign language training text with place-occupying characters, and translating the replaced foreign language training text to correspond to the first language; determining the position of the placeholder character in the translation result, and reducing the placeholder character into an entity vocabulary; and translating the entity vocabulary obtained by reduction in the translation result to correspond to the first language to obtain a training translation corresponding to the first language.

Exemplary computing device

Having described the methods, media, and apparatus of the exemplary embodiments of the present disclosure, a computing device of the exemplary embodiments of the present disclosure is described next with reference to fig. 12.

The computing device 1200 shown in fig. 12 is only one example and should not impose any limitations on the functionality or scope of use of embodiments of the disclosure.

As shown in fig. 12, computing device 1200 is embodied in the form of a general purpose computing device. Components of computing device 1200 may include, but are not limited to: at least one processing unit 1201, at least one memory unit 1202, and a bus 1203 connecting the various system components including the processing unit 1201 and the memory unit 1202. Wherein the at least one memory unit 1202 has stored therein computer executable instructions; the at least one processing unit 1201 includes a processor that executes the computer-executable instructions to implement the methods described above.

The bus 1203 includes a data bus, a control bus, and an address bus.

The storage unit 1202 may include readable media in the form of volatile memory, such as Random Access Memory (RAM) 12021 and/or cache memory 12022, and may further include readable media in the form of non-volatile memory, such as Read Only Memory (ROM) 12023.

Storage unit 1202 may also include programs having a set (at least one) of program modules 12024, such program modules 12024 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination thereof may comprise an implementation of a network environment.

Computing device 1200 may also communicate with one or more external devices 1204, such as a keyboard, pointing device, etc. Such communication may occur via input/output (I/O) interfaces 1205. Moreover, computing device 1200 may also communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) through network adapter 1206. As shown in FIG. 12, a network adapter 1206 communicates with the other modules of the computing device 1200 via a bus 1203. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with computing device 1200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.

It should be noted that although in the above detailed description several units/modules or sub-units/modules of the entity identification means are mentioned, such a division is merely exemplary and not mandatory. Indeed, the features and functions of two or more units/modules described above may be embodied in one unit/module according to embodiments of the present disclosure. Conversely, the features and functions of one unit/module described above may be further divided into embodiments by a plurality of units/modules.

Further, while the operations of the disclosed methods are depicted in the drawings in a particular order, this does not require or imply that these operations must be performed in this particular order, or that all of the illustrated operations must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions.

While the spirit and principles of the present disclosure have been described with reference to several particular embodiments, it is to be understood that the present disclosure is not limited to the particular embodiments disclosed, nor is the division of aspects which is intended to be construed to be merely illustrative of the fact that features of the aspects may be combined to advantage. The disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. An entity identification method, comprising:

inputting a text to be recognized into an entity prediction model to predict potential entities contained in the text to be recognized and potential entity categories of the corresponding potential entities;

determining a first entry in a respective knowledge base based on the potential entity and the potential entity category, the first entry further comprising a corresponding first entry introduction;

2. The entity recognition method of claim 1, wherein the knowledge base stores entries, entry blurbs, and entry heat values in a triplet manner;

the determining a first entry in a respective knowledge base based on the potential entity and the potential entity category includes:

determining the entries with the same name as the potential entities in the knowledge base as first entries;

in response to the entry being non-unique with respect to the potential entity, uniquely determining the first entry based on the entry heating value of the respective entry.

3. The entity identification method of claim 2, wherein the uniquely determining the first term based on the term heat value of the respective term in response to being non-unique to the potential entity term comprises:

and uniquely determining the entry with the highest entry heat value corresponding to each entry with the same name as the first entry.

4. The entity recognition method of claim 1, wherein the inputting the text to be recognized, the first vocabulary entry and the first vocabulary entry introduction into an entity recognition model, and the determining the target entities in the text to be recognized and the target entity categories of the corresponding target entities comprise:

splicing the text to be recognized with the first entry and the first entry brief introduction to obtain an information enhanced text corresponding to the text to be recognized;

and inputting the information enhanced text into the entity recognition model to obtain a target entity in the text to be recognized and a target entity category of the corresponding target entity.

5. The entity identification method of claim 4,

the entity prediction model comprises a first encoder and a first decoder, wherein the first encoder is used for identifying and encoding the text to be identified, and the first decoder is used for determining potential entities in the text to be identified and potential entity classes of the corresponding potential entities based on the output result of the first encoder;

6. A method for training a solid prediction model, comprising:

the character category label includes:

the first character and entity category label of the entity;

a non-initial character and an entity category label of the entity;

a non-entity tag.

7. An entity recognition model training method is characterized by comprising the following steps:

determining corresponding entries and entry blurbs of training texts in a knowledge base, wherein the training texts are labeled with character type labels;

inputting the training texts and the corresponding terms and term blurbs of the training texts in an entity library into the entity recognition model for training so as to output final entities contained in the training texts and final entity categories of the corresponding final entities,

the character category label includes:

the first character and entity category label of the entity;

a non-initial character and an entity category label of the entity;

a non-entity tag.

8. A computer-readable storage medium, comprising: the computer-readable storage medium having stored therein computer-executable instructions for implementing the entity identification method of any one of claims 1 to 5 when executed by a processor; and/or, the computer executable instructions when executed by a processor are for implementing the entity prediction model training method of claim 6; and/or the computer executable instructions when executed by a processor are for implementing the entity recognition model training method of claim 7.

9. An entity identification apparatus comprising:

the prediction module is used for inputting the text to be recognized into the entity prediction model so as to predict potential entities contained in the text to be recognized and potential entity categories of the corresponding potential entities;

an enhancement module to determine a first entry in a respective knowledge base based on the potential entity and the potential entity category, the first entry further comprising a corresponding first entry introduction;

and the recognition module is used for inputting the text to be recognized, the first vocabulary entry and the first vocabulary entry brief introduction into an entity recognition model, and determining a target entity in the text to be recognized and a target entity category of the corresponding target entity.

10. A computing device, comprising: at least one processor;

and a memory communicatively coupled to the at least one processor;

wherein the memory stores instructions executable by the at least one processor to cause the computing device to perform the entity identification method of any of claims 1 to 5; and/or to cause a computing device to perform the entity prediction model training method of claim 6; and/or to cause a computing device to perform the entity recognition model training method of claim 7.