CN115828915A - Entity disambiguation method, apparatus, electronic device and storage medium - Google Patents

Entity disambiguation method, apparatus, electronic device and storage medium Download PDF

Info

Publication number
CN115828915A
CN115828915A CN202211092241.4A CN202211092241A CN115828915A CN 115828915 A CN115828915 A CN 115828915A CN 202211092241 A CN202211092241 A CN 202211092241A CN 115828915 A CN115828915 A CN 115828915A
Authority
CN
China
Prior art keywords
entity
text
determining
candidate
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202211092241.4A
Other languages
Chinese (zh)
Other versions
CN115828915B (en
Inventor
陈子平
朱嘉琪
冯知凡
柴春光
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to CN202211092241.4A priority Critical patent/CN115828915B/en
Publication of CN115828915A publication Critical patent/CN115828915A/en
Application granted granted Critical
Publication of CN115828915B publication Critical patent/CN115828915B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Machine Translation (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention provides an entity disambiguation method, an entity disambiguation device, electronic equipment and a storage medium, relates to the field of artificial intelligence, particularly relates to the technical fields of natural language processing, knowledge maps and the like, and can be applied to scenes such as intelligent industry, intelligent government affairs, search engines, intelligent customer service and the like. The concrete implementation scheme of the entity disambiguation method is as follows: identifying a query text to obtain entity words to be disambiguated in the query text; determining a target text in the text obtained by query according to the query text; determining candidate entities corresponding to the entity words and entity information of the candidate entities according to the entity words; wherein the candidate entities comprise at least two entities indicated by the same entity word and ambiguous with respect to each other; and determining the target entity indicated by the entity words in the candidate entity according to the target text and the entity information of the candidate entity.

Description

Entity disambiguation method, apparatus, electronic device and storage medium
Technical Field
The utility model relates to an artificial intelligence field, concretely relates to technical fields such as natural language processing and knowledge map can be applied to scenes such as intelligent industry, wisdom government affairs, search engine and intelligent customer service.
Background
With the development of computer technology and network technology, deep learning technology has been widely used in many fields. For example, deep learning techniques can be employed to perform natural language processing on Query text (Query) to improve the search quality and use experience of a search engine. In the process of natural language processing of the query text, the text semantics need to be understood and the intention requirement of the query text needs to be judged. However, since the query text often includes ambiguous entities, there are situations where the determined intent requirement is inaccurate or even incorrect.
Disclosure of Invention
The present disclosure is directed to providing an entity disambiguation method, apparatus, device, and medium to improve accuracy of understanding text semantics of a query text and thus precision of a judged intent requirement of the text through disambiguation of an entity.
According to an aspect of the present disclosure, there is provided an entity disambiguation method comprising: identifying a query text to obtain entity words to be disambiguated in the query text; determining a target text in a text obtained by query according to a query text; determining candidate entities corresponding to the entity words and entity information of the candidate entities according to the entity words; wherein the candidate entities comprise at least two entities indicated by the same entity word and ambiguous with respect to each other; and determining the target entity indicated by the entity words in the candidate entity according to the target text and the entity information of the candidate entity.
According to another aspect of the present disclosure, there is provided an entity disambiguation apparatus comprising: the entity identification module is used for identifying the query text to obtain the entity words of the entity to be disambiguated in the query text; the target text determining module is used for determining a target text in the text obtained by inquiring according to the query text; the candidate entity determining module is used for determining a candidate entity corresponding to the entity word and entity information of the candidate entity according to the entity word; wherein the candidate entities comprise at least two entities indicated by the same entity word and ambiguous with respect to each other; and the target entity determining module is used for determining the target entity indicated by the entity words in the candidate entity according to the target text and the entity information of the candidate entity.
According to another aspect of the present disclosure, there is provided an electronic device including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the entity disambiguation method provided by the present disclosure.
According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the entity disambiguation method provided by the present disclosure.
According to another aspect of the present disclosure, there is provided a computer program product comprising computer programs/instructions which, when executed by a processor, implement the entity disambiguation method provided by the present disclosure.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
FIG. 1 is a schematic diagram of an application scenario of an entity disambiguation method and apparatus according to an embodiment of the present disclosure;
FIG. 2 is a flow diagram of an entity disambiguation method in accordance with an embodiment of the disclosure;
FIG. 3 is a schematic diagram of an entity disambiguation method according to an embodiment of the present disclosure;
FIG. 4 is a schematic diagram illustrating a principle of determining an entity type of an entity indicated by an entity word to be disambiguated in a query text according to an embodiment of the present disclosure;
FIG. 5 is a schematic diagram illustrating a principle of determining similarity between a text title of a target text and entity information of a candidate entity according to an embodiment of the present disclosure;
FIG. 6 is a schematic diagram illustrating a principle of determining similarity between a text title of a target text and entity information of a candidate entity according to another embodiment of the present disclosure;
FIG. 7 is a block diagram of an entity disambiguation apparatus according to an embodiment of the present disclosure; and
FIG. 8 is a block diagram of an electronic device for implementing the entity disambiguation method of an embodiment of the disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Query text (Query) received by search engines is ever-changing, representing a diverse need for users. The requirement intent expressed by the query text may include a number of different intents due to the presence of a word-ambiguous condition. To accurately determine the required intent expressed by the query text, entity words representing multiple entities may be disambiguated. For example, the entity indicated by the entity word in the query text can be determined based on the context information of the query text, so as to determine the requirement intention expressed by the query text. However, in the case that the query text is a short text or the expressed context information is limited, the disambiguation effect is poor, and the entity indicated by the entity word determined after the disambiguation still has an ambiguous situation. It should be understood that the search engine may be a commonly used web page version search engine, may also be a search engine used when searching for the reply text in an intelligent customer service scenario, and may also be a search engine used when searching for the text in an intelligent industrial or intelligent government affair scenario, which is not limited in this disclosure.
To avoid this situation, the present disclosure provides an entity disambiguation method, apparatus, electronic device, and storage medium.
An application scenario of the entity disambiguation method and apparatus provided by the present disclosure will be schematically described below with reference to fig. 1.
As shown in fig. 1, the application scenario 100 of this embodiment may include an electronic device 110, and the electronic device 110 may be various electronic devices with processing functionality, including but not limited to a smartphone, a tablet, a laptop, a desktop computer, a server, and so on.
The electronic device 110 may, for example, have installed thereon various types of client applications, such as a web browser, a search-type application, an instant messaging-type application, and/or a shopping-type application, to name a few examples. The electronic device 110 can, for example, in response to receiving the query text 120, perform entity recognition on the query text 120 to determine entity words in the query text that need disambiguation.
In an embodiment, the electronic device 110 may also disambiguate the entity words that need to be disambiguated by semantically understanding the query text 120 to determine the entities indicated by the entity words in the query text. After disambiguation, the electronic device 110 may determine and present query results 130 obtained in response to the query text based on the semantics of the disambiguated query text.
In an embodiment, the electronic device 110 may also send the query text 120 to the server 140, for example, the server 140 obtains information related to the query text 120 from the database 150 according to the query text 120, and feeds back the obtained information as an initial query result to the electronic device 110. The electronic device 110 can disambiguate the entity words to be disambiguated from the received initial query results and the query text 120, and determine the entity indicated by the entity words in the query text. Subsequently, the electronic device 110 may send the disambiguated query text to the server 140, so that the server 140 obtains the information related to the disambiguated query text from the database 150 according to the disambiguated query text, and feeds back the information related to the disambiguated query text as a query result to the electronic device 110 for presentation by the electronic device 110.
Wherein the electronic device 110 may be communicatively coupled to the server 140 via a network. The network may include wired or wireless communication links. The server 140 may be a background management server supporting the running of a client application provided in the electronic device 110.
In one embodiment, the electronic device 110 may further send the query text 120 to the server 140, perform entity recognition on the query text 120 by the server 140, and determine the entity words in the query text 120 that need to be disambiguated. At the same time, the query text 120 may be semantically understood by the server 140 to disambiguate the entity words that need to be disambiguated.
It is noted that the entity disambiguation methods provided by the present disclosure may be performed by the electronic device 110, or may be performed by the server 140. Accordingly, the entity disambiguation apparatus provided by the present disclosure may be disposed in the electronic device 110, and may also be disposed in the server 140.
It should be understood that the number and types of electronic devices 110, servers 140, and databases 150 in fig. 1 are merely illustrative. There may be any number and type of electronic devices 110, servers 140, and databases 150, as desired for an implementation.
The entity disambiguation method provided by the present disclosure will be described in detail below with reference to fig. 2 to 6.
Fig. 2 is a flow diagram of an entity disambiguation method in accordance with an embodiment of the disclosure.
As shown in FIG. 2, the entity disambiguation method 200 of this embodiment may include operations S210-S240.
In operation S210, the query text is identified, and the entity words to be disambiguated in the query text are obtained.
According to the embodiment of the disclosure, a deep learning method can be adopted to identify the query text. For example, entity words in the query text can be identified by using an entity identification model to obtain at least one entity word. The embodiment may select the entity word that needs to be disambiguated from the at least one entity word. The entity recognition model may be a model composed of a Long-Short Term Memory network (LSTM) and a conditional random field, or may be a model composed of an expanded convolutional network (IDCNN) and a conditional random field. It is to be understood that the structure of the entity recognition model described above is merely an example to facilitate understanding of the present disclosure, and the present disclosure is not limited thereto.
For example, the present disclosure may maintain an entity library comprised of ambiguous entity words. The embodiment may query the entity library according to the at least one entity word after obtaining the at least one entity word, determine an entity word belonging to the entity library in the at least one entity word, and use the determined entity word as the entity word to be disambiguated.
In one embodiment, the embodiment may determine the entity words to disambiguate based on the grammatical composition of the query text. For example, entity words that are subjects in the query text may be treated as entity words to be disambiguated.
For example, the embodiment may employ a sequence annotation model to annotate entity words to be disambiguated in the query text. The embodiment can firstly perform word segmentation processing on the query text to obtain a word sequence. Then, the character sequence is input into a sequence labeling model, and labeling information for each character in the character sequence is output by the sequence labeling model. The label information may include, for example, a label indicating a B-I-O (Begin-Inside-out) category, and the label information may indicate which category each word in the word sequence is a starting word of the entity word to be disambiguated (labeled Begin), other words of the entity word to be disambiguated (labeled insert) except the starting word, and other words of the entity word to be disambiguated (labeled out) except the word included in the entity word to be disambiguated. This embodiment may use the word resulting from the concatenation of the words labeled B and I as the entity word to be disambiguated. The sequence labeling model may be, for example, a model constructed based on a transform-based bidirectional encoder characterization network (BERT), or a model constructed based on a chinese character center model (ERNIE), which is not limited in this disclosure.
In operation S220, a target text among texts queried according to the query text is determined.
According to the embodiment of the disclosure, the texts in the predetermined database can be queried according to the query texts, and the predetermined number of texts with the highest correlation degree with the query texts in the queried texts are used as the target texts. The predetermined database may be, for example, a knowledge database maintained for a search engine. The text in the predetermined database may comprise plain text, text in a teletext format, text containing video, etc. The target text in the text obtained by the query may be represented by a text title, for example, which is usually in a plain text format. The embodiment may represent the degree of correlation between the text in the predetermined database and the query text by the similarity between the text title of the query text and the text in the predetermined database.
In operation S230, a candidate entity corresponding to the entity word and entity information of the candidate entity are determined according to the entity word.
In the above-described entity library configured by the ambiguous entity words, for example, entity information of the entity represented by the ambiguous entity words may also be maintained, and the entity information may be, for example, attribute information of the entity and/or description information of the entity, and the like. For example, the entity library may maintain a mapping relationship between the entity identifier and the entity word, and also maintain a mapping relationship between the entity identifier and the entity information. Different entities represented by the same entity word have different entity identities. The embodiment may search the entity library according to the entity word, determine an entity identifier having a mapping relationship with the entity word, use the entity having the determined entity identifier as a candidate entity, and use the entity information having the mapping relationship with the determined entity identifier as the entity information of the candidate entity.
According to an embodiment of the present disclosure, when an entity word indicates two different entities, the entity word is an entity word that needs disambiguation. If the entity identifications determined by searching the entity library are at least two, it may be determined that the entity word in the query text is an entity word that needs to be disambiguated, and operation S240 continues. If only one entity identifier is determined by searching the entity library, it may be determined that the target entity indicated by the entity word having the determined entity identifier may be obtained without disambiguating the entity word in the query text.
In operation S240, a target entity indicated by the entity word in the candidate entity is determined according to the target text and the entity information of the candidate entity.
According to an embodiment of the present disclosure, the target entity may be determined according to a similarity between the target text and the entity information of the candidate entity. For example, the embodiment may take the candidate entity having the highest similarity with the target text as the target entity.
In one embodiment, the target text and the query text may be combined into one text, and then the target entity may be determined according to the similarity between the combined text and the entity information of the candidate entity.
When determining the similarity between the text and the entity information, for example, a semantic feature extraction network may be used to extract semantic features of the text and the entity information, and the similarity between the semantic features of the text and the semantic features of the entity information is used as the similarity between the text and the entity information. The semantic feature extraction network may be, for example, a recurrent neural network, a transform encoder, or a pre-training characterization network (e.g., BERT or ERNIE), which is not limited in this disclosure.
When disambiguating the entity words in the query text, the embodiment of the disclosure firstly obtains the target text according to the query text. And then disambiguation of the entity words is carried out according to the target text and the entity information of the candidate entities represented by the entity words, so that the characteristics related to the query text can be added to the disambiguation process. Particularly, under the condition that the query text is a short text or the information expressed by the query text is insufficient, the target text is introduced, so that the context information of the query text is made up, the expression capability of the query text in the disambiguation process is improved, the complete disambiguation of the ambiguous entity words is realized, and the disambiguation precision is improved. This is because the target text is usually an extension of the query text, and the target text can provide richer information on the basis of representing the query text.
It is to be understood that the operation of identifying the query text in operation S210 may be understood as an operation of identifying Entity mentions, that is, obtaining Entity words indicating to be disambiguated through Entity mentions identification (EMD). Wherein, the entity refers to a reference Expression (refering Expression) used for expressing a certain entity in the text. The object of the entity mention identification task is to identify all mentions of entities of a specified type, including naming, noun and also pronoun mentions, and to take into account the nested structure between entity mentions.
FIG. 3 is a schematic diagram of a method of entity disambiguation according to an embodiment of the present disclosure.
According to the embodiment of the disclosure, before determining the candidate entity, for example, the entity type of the entity indicated by the entity word in the query text may be determined, and then the candidate entity may be determined according to the entity type and the entity word. For example, entities that are indicated by the entity word to be disambiguated in the query text and that have the same type as the entity indicated by the entity word in the query text may be considered candidate entities. Therefore, the number of the determined candidate entities can be reduced, the determined candidate entities can be more fit to the query text, a more reliable basis is provided for the determination of the target entities, and the entity disambiguation efficiency is accelerated.
For example, the entity type of the entity indicated by the entity word in the query text can be determined according to the context information of the entity word to be disambiguated in the query text. The embodiment may input the query text into an entity type prediction model, and determine the entity type from the output of the entity type prediction model. The entity type prediction model may be, for example, a model composed of a semantic feature extraction network and a full connection layer.
In one embodiment, the target text and the entity word may be combined to determine an entity type of the entity indicated by the entity word in the query text. For example, the entity type of the entity indicated by the entity word in the query text can be determined according to the context information of the target text. The target text is usually an extension of the query text, and the target text can provide richer information on the basis of representing the query text, so that the accuracy of the determined entity type can be improved.
On this basis, as shown in fig. 3, the implementation principle of the entity disambiguation method of this embodiment 300 may be: the sequence annotation model 310 is employed to annotate entity words to be disambiguated in the query text 301. Meanwhile, the knowledge database 320 may be queried according to the query text 301 to obtain the target text 302. Subsequently, an entity type prediction model 330 may be employed to determine an entity type 304 of the entity indicated by the entity word 303 to be disambiguated, based on the target text 302 and the entity word 303 to be disambiguated. Subsequently, the entity library 340 may be queried according to the entity type 304 and the entity word 303 to obtain the candidate entity 305 and the entity information 306 of the candidate entity 305. Finally, a target entity 307 of the candidate entities 305 is determined based on the entity information 306 and the target text 302.
Taking the query text 301 as "XY is that of the country" as an example, the sequence annotation result output by the sequence annotation model may be "B I O", and eight annotations in the sequence annotation result correspond to eight characters in the query text 301 one to one, where B represents a starting character of an entity word to be disambiguated, I represents a character of the entity word to be disambiguated other than the starting character, and O represents another character of the entity word to be disambiguated other than the character included in the entity word to be disambiguated. The entity word 303 to be disambiguated can be determined to be the word "XY" from the labeling result. It is understood that the candidate entities 305 obtained by querying the entity library 340 have a one-to-one correspondence with the entity information 306, and when the entity word indicates at least two entities, the candidate entities 305 include at least two entities.
When the entity type 304 is determined according to the target text 302 and the entity word 303 to be disambiguated, for example, the position of the entity word 303 to be disambiguated in the target text 302 may be labeled, the target text added with the label is input into the entity type prediction model 330, and the entity type 304 is determined according to the output of the entity type prediction model 330. For example, for the target text "XY, belonging to country a, with a total population of aaa and a floor area of bbb", by adding a label to the entity word "XY" to be disambiguated, the text "# XY #, XY, belonging to country a, with a total population of aaa and a floor area of bbb" can be obtained.
After the entity information 306 and the target text 302 are obtained, the candidate entity with the highest similarity between the entity information and the target text 302 may be determined as the target entity 307 according to the similarity between the entity information 306 and the target text 302 of each candidate entity. For example, a similarity prediction model 350 may be employed to determine the similarity between entity information and the target text 302. The similarity prediction model 350 may include the semantic feature extraction network described above, among other things, to extract the semantic features of the entity information and the target text. The similarity prediction model 350 may be, for example, a single tower model or a double tower model, which is not limited by the present disclosure.
According to an embodiment of the present disclosure, for example, a mapping relationship between an entity word and an entity identifier may be maintained in the entity library 340, and the entity identifier uniquely corresponds to one entity. And the entity library 340 may further maintain a mapping relationship between the entity identifier and the entity information of the entity corresponding to the entity identifier. In the embodiment, when querying the entity library 340, an entity word to be disambiguated may be used as a query basis to determine an entity indicated by the entity identifier having a mapping relationship with the entity word 303 in the entity library 340, and the entity is used as a candidate entity. It is understood that each entity identifier in the entity library may also be added with, for example, annotation information of the type of the entity represented by the entity identifier. After determining the entity identifier having a mapping relationship with the entity word 303 in the entity library 340, the embodiment may determine the type of the entity corresponding to the entity identifier having a mapping relationship according to the label information of the entity identifier having a mapping relationship. The entity with the mapping relationship is then identified as a candidate entity 305, which is an entity with a type consistent with the entity type 304 in the corresponding entity. Subsequently, after determining the entity identifier having a mapping relationship with the entity word, the entity information having a mapping relationship with the entity identifier having a mapping relationship in the entity library 340 may be determined as the entity information of the entity indicated by the entity identifier having a mapping relationship.
According to an embodiment of the present disclosure, the mapping relationship in the entity library 340 may be updated in real time, for example, and specifically may be updated according to the update of the entity in the upstream application. Therefore, when the entity words in the online query text are disambiguated, the entity words can be disambiguated by combining the latest information provided by the upstream application, so that the disambiguation effect can be improved, and the accuracy of the recommended information obtained by query can be improved. For example, if the query text is text input by a search engine, the upstream application may be an encyclopedia application or the like, which is not limited by this disclosure. Wherein, the updating of the entity in the upstream application may include at least one of the following: addition of entities, deletion of entities, modification of entity information.
For example, the embodiment may employ a streaming computing framework to listen for updates of entities in upstream applications. The streaming framework may include Strorm, spark, flink, etc. For example, the streaming framework may be used to listen to a message queue (e.g., kafka messages, etc.) where messages in the message queue are messages that have been altered by an entity in an upstream application.
In this embodiment, based on rpc/http service, a mainstream frame gRPC, flash, or springboot, etc. may be used to modify mapping information in the entity library 340 according to an entity-changed message in a monitored message queue, for example, the mapping relationship of an entity identifier in the entity library 340 may be added or deleted, or the entity information having a mapping relationship with the entity identifier may be modified.
Illustratively, the entity repository 340 may be a Key-value database. For example, the entity library 340 may be any one of the databases such as Redis, rawbase, levelDB, rocksDB, and the like, which is not limited in this disclosure.
FIG. 4 is a schematic diagram illustrating a principle of determining an entity type of an entity indicated by an entity word to be disambiguated in a query text according to an embodiment of the present disclosure.
According to an embodiment of the present disclosure, the entity type of the entity indicated by the entity word to be disambiguated may be determined according to the text title of the target text. This is because the title of the text can usually express the text more completely, and the entity type is determined only according to the text title, which can reduce the amount of calculation.
In an embodiment, the first text feature may be obtained according to a text title of the target text. For example, the text feature extraction network may be used to extract the text feature of the text title, thereby obtaining the first text feature. The first text feature may then be processed using a fully-connected layer that outputs a probability vector comprising a plurality of probability values corresponding to a plurality of predetermined types. The embodiment may take a predetermined type corresponding to a maximum probability value among the plurality of probability values as the entity type.
As shown in fig. 4, in one embodiment 400, the text title of the target text is set to "XY (one of the major cities in country a)", and the entity word to be disambiguated is set to "XY". In this embodiment, a mark may be added to a position where an entity word to be disambiguated is located in a text title, so as to obtain an input text 401"# XY # (one of major cities in country a)". The embodiment 400 may perform a word segmentation process on the input text 401 to obtain a word sequence 402. The embedded features 403 of the word sequence 402 are then input into a text feature extraction network 410 comprised by the entity type prediction model, and the initial text features 404 represented by the feature sequence are output by the text feature extraction network 410. Wherein the features in the feature sequence may correspond one-to-one with the words in the word sequence 402.
Then, the embodiment may extract a text sub-feature corresponding to the entity word from the initial text feature 404 according to the position of the entity word in the target text, obtain a first text feature 405, and then determine the entity type of the entity indicated by the entity word in the query text according to the first text feature 405. For example, the embodiment may concatenate features corresponding to entity words in the initial text features 404 to obtain the first text features 405. The first textual feature 405 may then be input into a fully-connected layer 420 included in the entity type prediction model, and a probability vector may be output by the fully-connected layer 420 from which the entity type 406 may be determined.
When the entity type is determined, the embodiment extracts the corresponding text sub-features from the initial text features according to the position of the entity word to be disambiguated in the text title of the target text, and then determines the entity type according to the text sub-features only, so that the calculation amount when the entity type is determined can be reduced as much as possible on the basis of ensuring that the text sub-features can fully express the semantics of the entity word to be disambiguated.
In one embodiment, not only the text title, but also the query text may be considered in obtaining the first text feature 405, for example. Therefore, the expression capability of the extracted first text feature 405 can be improved, and richer information can be provided for determining the entity type.
For example, the embodiment may first concatenate the text titles of the query text and the target text to obtain a first concatenated text. And then, extracting the text features of the first spliced text to obtain first text features. The principle of extracting the text feature of the first concatenated text to obtain the first text feature is similar to the principle of obtaining the first text feature 405 according to the text title of the target text in embodiment 400, and is not described herein again. Where, for example, the query text is "XY is for that country", and the text title of the target text is "XY (one of the major cities in country a)", the first spliced text obtained by splicing may be "XY is XY (one of the major cities in country a)". In order to distinguish the query text from the text title, a separation character may be added between the query text and the text title, or an identifier representing the query text may be added before the query text, and an identifier representing the text title may be added between the text title and the query text, so as to obtain the first concatenated text.
According to the embodiment of the disclosure, when the entity type is determined, the type information of the entity indicated by the entity word to be disambiguated can be determined according to each text in the plurality of texts, and then the plurality of types of information can be obtained according to the plurality of texts. Finally, the embodiment may determine the entity type from the plurality of types of information. The type information may be, for example, a probability vector including a plurality of probability values corresponding to a plurality of predetermined types.
For example, in this embodiment, an entity type may be determined according to each type information, and a predetermined type corresponding to the maximum probability value in the probability vector may be used as the entity type determined according to each type information, and a plurality of entity types may be obtained in total. The embodiment can take the type with the most value in the entity types as the entity type of the entity indicated by the entity word in the query text.
In an embodiment, predetermined weights may also be assigned in advance to a plurality of texts included in the target text, respectively. Then, an entity type of an entity indicated by the entity word in the query text is determined according to a plurality of predetermined weights and a plurality of types of information corresponding to the plurality of texts. For example, a plurality of probability vectors representing a plurality of types of information may be weighted according to a plurality of predetermined weights to obtain a weighted probability vector. And finally, taking the preset type corresponding to the maximum probability value in the weighted probability vector as the entity type of the entity indicated by the entity word in the query text.
For example, a predetermined weight may be assigned to each text in the target text according to the degree of correlation between the text and the query text, and the assigned predetermined weight may be positively correlated with the degree of correlation, for example. Alternatively, each text in the target text may be assigned a predetermined weight, for example, according to the access data of the text. The access data may include, for example, reading amount, collection amount, and/or approval amount. For example, the predetermined weight may be positively correlated with reading, collection, and/or approval. Alternatively, for example, each text may be assigned a predetermined weight according to an arrangement order of the text in a plurality of texts included in the target text, and the earlier the arrangement order, the higher the assigned predetermined weight.
According to the embodiment, the preset weight is distributed to the texts, and the entity type of the entity indicated by the entity word in the query text is determined by combining the preset weight, so that the type information determined according to the text with high reliability can be more concerned when the entity type is determined, the accuracy of the determined entity type can be improved, and the disambiguation effect and the disambiguation efficiency can be improved.
According to an embodiment of the present disclosure, similar to the principle of determining the entity type of the entity indicated by the entity word to be disambiguated, the embodiment may consider the text title of the target text when determining the target entity, thereby reducing the amount of computation. For example, the embodiment may determine the similarity between the text title of the target text and the entity information of the candidate entities first, and then determine the entity with the highest similarity between the entity information and the text title of the target text in the candidate entities as the target entity.
The operation of determining the similarity between the text title of the target text and the entity information will be further expanded and defined below with reference to fig. 5 to 6.
Fig. 5 is a schematic diagram of determining similarity between a text title of a target text and entity information of a candidate entity according to an embodiment of the present disclosure.
According to the embodiment of the present disclosure, the similarity between the text title of the target text and the entity information may be expressed by using the similarity of the text features between the text title of the target text and the entity information.
For example, the embodiment may first obtain the second text feature according to the text title of the target text. When the candidate entity includes a plurality of entities, the text feature of the entity information of each entity in the candidate entity may be extracted while the second text feature is obtained, so as to obtain a third text feature. Then, according to the second text feature and the third text feature, the similarity between the text title of the target text and the entity information of each entity is determined. For example, the similarity between the second text feature and the third text feature may be taken as the similarity between the text title of the target text and the entity information of each entity. Then, for a plurality of entities included in the candidate entity, a plurality of similarities corresponding to the plurality of entities one to one may be obtained. The embodiment may take an entity corresponding to the maximum similarity among the plurality of similarities as the target entity.
According to an embodiment of the present disclosure, as shown in fig. 5, a double tower model may be employed to determine a similarity between a text title of a target text and entity information of a candidate entity. The two-tower model may include a first text feature extraction network 510 and a second text feature extraction network 520 arranged in parallel, a stitching network 530, and a full connectivity layer 540.
This embodiment may obtain the input to the first text feature extraction network 510 by performing word segmentation and embedded representation on the text title 501 of the target text. Similarly, the entity information 502 of each entity is divided into words and embedded to obtain the input of the second text feature extraction network 520. The second text feature 503 is output by the first text feature extraction network 510 and the third text feature 504 is output by the second text feature extraction network 520. It is understood that the first text feature extraction network 510 and the second text feature extraction network 520 may be two networks with the same network structure but different network parameters, or may be any two networks with similar network structures. The two feature extraction networks are similar to the text feature extraction network in the entity type prediction model described above, and are not described herein again.
The embodiment 500 may use the second text feature 503 and the third text feature 504 as input for the concatenation network 530, and the concatenation network 530 concatenates the second text feature 503 and the third text feature 504 to obtain the feature 505. Subsequently, the features 505 are input into the fully-connected layer 540, and the similarity 506 between the text header 501 and the entity information 502 of each entity is output by the fully-connected layer 540. It will be appreciated that the functionality of the splice network 530 may also be integrated into the fully-connected layer 540, for example.
Wherein, when stitching the second textual feature 503 and the third textual feature 504, the feature vector representing the third textual feature 504 may be stitched directly after the feature vector representing the second textual feature 503. Alternatively, in order to distinguish the second text feature 503 and the third text feature 504 in the spliced feature, the segmentation identification feature may be added after the feature vector representing the second text feature 503, and then the feature vector representing the third text feature 504 may be spliced after the segmentation identification feature. The segmentation identification feature may be any identifier, which is not limited in this disclosure.
In an embodiment, as described above, the principle of obtaining the first text feature is similar, and when the second text feature is obtained, the text titles of the query text and the target text may be first spliced to obtain the first spliced text. Then, the first text feature extraction network 510 is used to extract the text features of the first stitched text, so as to obtain the second text features. Therefore, the expression capability of the extracted second text features can be improved, and richer information is provided for determining the similarity.
In one embodiment, the text feature obtained from the text title of the target text may be used as the complete text feature. That is, the text feature output by the first text feature extraction network 510 is taken as the complete text feature. And then, according to the position of the entity word to be disambiguated in the target text, determining a sub-feature corresponding to the entity word in the complete text feature, and taking the sub-feature as a second text feature. It can be understood that the principle of determining the sub-features corresponding to the entity words in the complete text features is similar to the above-described principle of extracting the text sub-features corresponding to the entity words from the first text features, and is not described herein again. In this way, the embodiment can reduce the calculation amount when determining the similarity as much as possible on the basis that the second text feature for determining the similarity can ensure that the semantics of the entity word to be disambiguated are fully expressed.
According to an embodiment of the present disclosure, in the case that the target text includes a plurality of texts, when determining the similarity, the embodiment may first determine the similarity between the text title of each text in the plurality of texts and the entity information of the candidate entity, for example, by using a principle similar to the above principle of determining the entity type according to the predetermined weight assigned to the plurality of texts, resulting in a plurality of initial similarities corresponding to the plurality of texts. Then, a similarity between the text title of the target text and the entity information of the candidate entity is determined according to a plurality of predetermined weights and a plurality of initial similarities corresponding to the plurality of texts. The distribution of the predetermined weight is similar to the distribution principle described above, and is not described herein again. In this embodiment, the plurality of initial similarities may be weighted according to a plurality of predetermined weights, and the weighted similarity may be used as the similarity between the text title of the target text and the entity information of the candidate entity. In the case where the candidate entity includes a plurality of entities, the operation of the present embodiment may be performed for each entity, thereby obtaining a similarity between the text title of the target text and the entity information of each entity.
Fig. 6 is a schematic diagram of the principle of determining similarity between a text title of a target text and entity information of a candidate entity according to another embodiment of the present disclosure.
As shown in fig. 6, in embodiment 600, a single tower model may be employed to determine similarity between the text title of the target text and the entity information of each entity. The single tower model may include, among other things, a text feature extraction network 610 and a fully connected layer 620.
In the embodiment, when determining the similarity, the text header 601 of the target text and the entity information 602 of each entity may be first spliced to obtain a second spliced text 603. Subsequently, the text feature extraction network 610 is used to extract the text feature of the second stitched text 603, so as to obtain a fourth text feature 604. Subsequently, a similarity 605 between the text title 601 of the target text and the entity information 602 of each entity is determined according to the fourth text feature 604 using the full connection layer 620.
Here, when the text header 601 and the entity information 602 are concatenated, identification information indicating a single text may be added before the text header 601 and the entity information 602, respectively. For example, the identification information "text _ a" may be added before the text header 601, and the identification information "text _ b" may be added before the entity information 602. Subsequently, the entity information 602 to which the identification information is added is spliced after the text header 601 to which the identification information is added, thereby obtaining a second spliced text 603.
It will be appreciated that the textual feature extraction network 610 may be identical or similar in structure to the textual feature extraction networks 610 described above. The text feature extraction networks in the sequence annotation model, the entity type prediction model, and the similarity prediction model may have the same structure. In an end-to-end system, the text feature extraction networks in the three models can share network parameters.
In an embodiment, for example, a model formed by the pre-training network Bert and the full connection layer may be used as a base model, and the base model is trained for the sequence labeling task, the entity type prediction task, and the similarity prediction task, so as to obtain a sequence labeling model, an entity type prediction model, and a similarity prediction model, respectively.
In an embodiment, in the process of obtaining the fourth text feature, for example, the text feature of the second concatenated text extracted by the text feature extraction network may be used as the initial feature, then, according to the position of the entity word to be disambiguated in the second concatenated text, the sub-feature corresponding to the entity word to be disambiguated in the initial feature is determined, and the extracted sub-feature is used as the fourth text feature. It is understood that the principle of extracting the sub-features to obtain the fourth text features is similar to the principle of obtaining the first text features from the extracted sub-features, and the description thereof is omitted here. Therefore, the calculation amount for determining the similarity can be reduced as much as possible on the basis of ensuring that the fourth text feature can fully express the semantics of the entity words to be disambiguated.
Based on the entity disambiguation method provided by the disclosure, the disclosure also provides an entity disambiguation device. The apparatus will be described in detail below with reference to fig. 7.
Fig. 7 is a block diagram of an entity disambiguation apparatus according to an embodiment of the present disclosure.
As shown in fig. 7, the entity disambiguation apparatus 700 of this embodiment may include an entity identification module 710, a target text determination module 720, a candidate entity determination module 730, and a target entity determination module 740.
The entity identification module 710 is configured to identify the query text to obtain the entity word to be disambiguated in the query text. In an embodiment, the entity identifying module 710 may be configured to perform the operation S210 described above, which is not described herein again.
The target text determination module 720 is configured to determine a target text in the text obtained according to the query text query. In an embodiment, the target text determining module 720 may be configured to perform the operation S220 described above, which is not described herein again.
The candidate entity determining module 730 is configured to determine a candidate entity corresponding to the entity word and entity information of the candidate entity according to the entity word. Wherein the candidate entities include at least two entities that are indicated by the same entity word and ambiguous with respect to each other. In an embodiment, the candidate entity determining module 730 may be configured to perform the operation S230 described above, which is not described herein again.
The target entity determining module 740 is configured to determine a target entity indicated by the entity word in the candidate entity according to the target text and the entity information of the candidate entity. In an embodiment, the target entity determining module 740 may be configured to perform the operation S240 described above, which is not described herein again.
According to an embodiment of the present disclosure, the apparatus 700 may further include, for example, a type determination module configured to determine an entity type of an entity indicated by an entity word in the query text. The candidate entity determining module 730 may be specifically configured to determine a candidate entity corresponding to the entity word and entity information of the candidate entity according to the entity word and the entity type, where the entity type of the candidate entity is the same as the entity type of the entity indicated by the entity word.
According to an embodiment of the present disclosure, the type determining module may be specifically configured to determine, according to the target text and the entity word, an entity type of an entity indicated by the entity word in the query text.
According to an embodiment of the present disclosure, the type determining module may include: the initial characteristic determining submodule is used for obtaining initial text characteristics according to the text title of the target text; the feature extraction sub-module is used for extracting text sub-features corresponding to the entity words from the initial text features according to the positions of the entity words in the text titles to obtain first text features; and the first type determining sub-module is used for determining the entity type of the entity indicated by the entity words in the query text according to the first text characteristics.
According to an embodiment of the present disclosure, the initial feature determination submodule includes: the first text splicing unit is used for splicing the text titles of the query text and the target text to obtain a first spliced text; and the first feature extraction unit is used for extracting the text features of the first spliced text to obtain the initial text features.
According to an embodiment of the present disclosure, the target text includes a plurality of texts, and the type determining module may include: the type information obtaining sub-module is used for determining the type information of the entity indicated by the entity word in the query text according to each text and the entity word in the plurality of texts to obtain a plurality of types of information corresponding to the plurality of texts; and a second type determining sub-module, configured to determine an entity type of an entity indicated by the entity word in the query text according to a plurality of predetermined weights corresponding to the plurality of texts and a plurality of type information, where the predetermined weights are related to at least one of the following information: the relevance between each text and the query text; access data for each text.
According to an embodiment of the present disclosure, the target entity determining module 740 may include: the similarity determining submodule is used for determining the similarity between the text title of the target text and the entity information of the candidate entity; and the entity determining submodule is used for determining the entity with the highest similarity between the entity information and the text title of the target text in the candidate entities as the target entity.
According to an embodiment of the present disclosure, the similarity determination submodule includes: the second feature extraction unit is used for obtaining a second text feature according to the text title of the target text; the third feature extraction unit is used for extracting the text features of the entity information of each entity in the candidate entities to obtain third text features; and a first similarity determining unit for determining similarity between the text title of the target text and the entity information of each entity according to the second text feature and the third text feature.
According to an embodiment of the present disclosure, the second feature extraction unit includes: the text splicing subunit is used for splicing the text titles of the query text and the target text to obtain a first spliced text; and the first feature extraction subunit is used for extracting the text features of the first spliced text to obtain second text features.
According to an embodiment of the present disclosure, the second feature extraction unit includes: the complete characteristic determining subunit is used for obtaining complete text characteristics according to the text title of the target text; and the second feature extraction sub-unit is used for extracting the text sub-features corresponding to the entity words in the complete text features as second text features according to the positions of the entity words in the target text.
According to an embodiment of the present disclosure, the similarity determination submodule may include: the second text splicing unit is used for splicing the text title of the target text and the entity information of each entity in the candidate entities to obtain a second spliced text; the fourth feature extraction unit is used for extracting the text features of the second spliced text to obtain fourth text features; and a second similarity determining unit for determining a similarity between the text title of the target text and the entity information of each entity according to the fourth text feature.
According to an embodiment of the present disclosure, the fourth feature extraction unit is specifically configured to: and determining sub-features corresponding to the entity words in the extracted text features of the second spliced text as fourth text features according to the positions of the entity words in the second spliced text.
According to an embodiment of the present disclosure, the target text includes a plurality of texts. The similarity determination submodule may include: the first similarity determining unit is used for determining the similarity between the text title of each text in the plurality of texts and the entity information of the candidate entity to obtain a plurality of initial similarities corresponding to the plurality of texts; and a second similarity determination unit configured to determine a similarity between the text title of the target text and the entity information of the candidate entity according to a plurality of predetermined weights and a plurality of initial similarities corresponding to the plurality of texts, wherein the predetermined weights are related to at least one of the following information: the relevance between each text and the query text; access data for each text.
According to an embodiment of the present disclosure, the candidate entity determining module 730 may include a candidate entity determining sub-module, configured to determine, as a candidate entity, an entity indicated by an entity identifier having a mapping relationship with an entity word in a predetermined entity library; the entity information determining submodule is used for determining entity information which has a mapping relation with the entity identification in a preset entity library and is used as the entity information of the entity indicated by the entity identification, wherein the mapping relation in the entity library is updated by monitoring the updating of the entity in the upstream application; the updating of the entity in the upstream application includes at least one of: addition of entities, deletion of entities, modification of entity information.
In the technical scheme of the present disclosure, the processes of collecting, storing, using, processing, transmitting, providing, disclosing and applying the personal information of the user all conform to the regulations of the relevant laws and regulations, and necessary security measures are taken without violating the customs of the public order. In the technical scheme of the disclosure, before the personal information of the user is acquired or collected, the authorization or the consent of the user is acquired.
The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.
Fig. 8 shows a schematic block diagram of an example electronic device 800 that may be used to implement the entity disambiguation method of an embodiment of the disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 8, the apparatus 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 802 or a computer program loaded from a storage unit 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The calculation unit 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.
A number of components in the device 800 are connected to the I/O interface 805, including: an input unit 806, such as a keyboard, a mouse, or the like; an output unit 807 such as various types of displays, speakers, and the like; a storage unit 808, such as a magnetic disk, optical disk, or the like; and a communication unit 809 such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
Computing unit 801 may be a variety of general and/or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and the like. The computing unit 801 performs the various methods and processes described above, such as the entity disambiguation method. For example, in some embodiments, the entity disambiguation method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and/or installed onto device 800 via ROM 802 and/or communications unit 809. When loaded into RAM 803 and executed by computing unit 801, a computer program may perform one or more steps of the entity disambiguation method described above. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the entity disambiguation method by any other suitable means (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The Server may be a cloud Server, which is also called a cloud computing Server or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of high management difficulty and weak service extensibility in a traditional physical host and a VPS service ("Virtual Private Server", or simply "VPS"). The server may also be a server of a distributed system, or a server incorporating a blockchain.
It should be understood that various forms of the flows shown above, reordering, adding or deleting steps, may be used. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and the present disclosure is not limited herein.
The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

Claims (18)

1. A method of entity disambiguation comprising:
identifying a query text to obtain entity words to be disambiguated in the query text;
determining a target text in the text obtained by querying according to the query text;
determining a candidate entity corresponding to the entity word and entity information of the candidate entity according to the entity word; wherein the candidate entities comprise at least two entities indicated by the same entity word and ambiguous with respect to each other; and
and determining a target entity indicated by the entity word in the candidate entity according to the target text and the entity information of the candidate entity.
2. The method of claim 1, further comprising:
determining an entity type of an entity indicated by the entity word in the query text;
wherein, the determining, according to the entity word, a candidate entity corresponding to the entity word and entity information of the candidate entity includes:
determining a candidate entity corresponding to the entity word and entity information of the candidate entity according to the entity word and the entity type,
wherein the entity type of the candidate entity is the same as the entity type of the entity indicated by the entity word.
3. The method of claim 2, wherein the determining an entity type of an entity indicated by the entity word in the query text comprises:
and determining the entity type of the entity indicated by the entity word in the query text according to the target text and the entity word.
4. The method of claim 3, wherein the determining, from the target text and the entity word, an entity type of an entity indicated by the entity word in the query text comprises:
obtaining initial text characteristics according to the text title of the target text;
extracting text sub-features corresponding to the entity words from the initial text features according to the positions of the entity words in the text titles to obtain first text features; and
and determining the entity type of the entity indicated by the entity word in the query text according to the first text characteristic.
5. The method of claim 4, wherein the obtaining initial text features according to the text title of the target text comprises:
splicing the text titles of the query text and the target text to obtain a first spliced text; and
and extracting text features of the first spliced text to obtain the initial text features.
6. The method of any of claims 3-5, wherein the target text comprises a plurality of texts; the determining, according to the target text and the entity word, an entity type of an entity indicated by the entity word in the query text includes:
determining type information of an entity indicated by the entity word in the query text according to each text and the entity word in the texts to obtain a plurality of types of information corresponding to the texts; and
determining an entity type of an entity indicated by the entity word in the query text according to a plurality of predetermined weights corresponding to the plurality of texts and the plurality of type information,
wherein the predetermined weight is related to at least one of the following information: a degree of correlation between each text and the query text; access data for each text.
7. The method of claim 1, wherein the determining, according to the target text and the entity information of the candidate entity, the target entity indicated by the entity word in the candidate entity comprises:
determining similarity between a text title of the target text and the entity information of the candidate entity; and
and determining the entity with the highest similarity between the entity information and the text title of the target text in the candidate entities as the target entity.
8. The method of claim 7, wherein the determining a similarity between the text title of the target text and the entity information of the candidate entity comprises:
obtaining a second text characteristic according to the text title of the target text;
extracting text features of entity information of each entity in the candidate entities to obtain third text features; and
and determining the similarity between the text title of the target text and the entity information of each entity according to the second text characteristic and the third text characteristic.
9. The method of claim 8, wherein the deriving a second text feature from a text title of the target text comprises:
splicing the text titles of the query text and the target text to obtain a first spliced text; and
and extracting the text features of the first spliced text to obtain the second text features.
10. The method according to claim 8 or 9, wherein the deriving a second text feature from the text title of the target text comprises:
obtaining complete text characteristics according to the text title of the target text; and
and extracting text sub-features corresponding to the entity words in the complete text features as the second text features according to the positions of the entity words in the target text.
11. The method of claim 7, wherein the determining a similarity between a text title of the target text and entity information of the candidate entity comprises:
splicing the text title of the target text and the entity information of each entity in the candidate entities to obtain a second spliced text;
extracting text features of the second spliced text to obtain fourth text features; and
and according to the fourth text characteristics, determining the similarity between the text title of the target text and the entity information of each entity.
12. The method of claim 11, wherein the extracting the text feature of the second stitched text to obtain a fourth text feature comprises:
and determining sub-features corresponding to the entity words in the extracted text features of the second spliced text as the fourth text features according to the positions of the entity words in the second spliced text.
13. The method of claim 7, wherein the target text comprises a plurality of texts; the determining the similarity between the text title of the target text and the entity information of the candidate entity comprises:
determining similarity between a text title of each text in the plurality of texts and the entity information of the candidate entity to obtain a plurality of initial similarities corresponding to the plurality of texts; and
determining a similarity between a text title of the target text and the entity information of the candidate entity according to a plurality of predetermined weights corresponding to the plurality of texts and the plurality of initial similarities,
wherein the predetermined weight is related to at least one of the following information: a degree of correlation between each text and the query text; access data for each text.
14. The method of claim 1, wherein the determining, according to the entity word, a candidate entity corresponding to the entity word and entity information of the candidate entity comprises:
determining an entity indicated by an entity identifier having a mapping relation with the entity word in a predetermined entity library as the candidate entity; and
determining entity information in the predetermined entity library having a mapping relation with the entity identifier as entity information of the entity indicated by the entity identifier,
wherein the mapping relation in the entity library is updated by monitoring the update of the entity in the upstream application; the updating of the entity in the upstream application comprises at least one of: addition of entities, deletion of entities, modification of entity information.
15. An entity disambiguation apparatus comprising:
the entity identification module is used for identifying the query text to obtain the entity words to be disambiguated in the query text;
the target text determining module is used for determining a target text in the text obtained by inquiring according to the inquiry text;
the candidate entity determining module is used for determining a candidate entity corresponding to the entity word and entity information of the candidate entity according to the entity word; wherein the candidate entities comprise at least two entities indicated by the same entity word and ambiguous with respect to each other; and
and the target entity determining module is used for determining the target entity indicated by the entity word in the candidate entity according to the target text and the entity information of the candidate entity.
16. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 14.
17. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-14.
18. A computer program product comprising computer program/instructions stored on at least one of a readable storage medium and an electronic device, which when executed by a processor implement the steps of the method according to any one of claims 1 to 14.
CN202211092241.4A 2022-09-07 2022-09-07 Entity disambiguation method, device, electronic equipment and storage medium Active CN115828915B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211092241.4A CN115828915B (en) 2022-09-07 2022-09-07 Entity disambiguation method, device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211092241.4A CN115828915B (en) 2022-09-07 2022-09-07 Entity disambiguation method, device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN115828915A true CN115828915A (en) 2023-03-21
CN115828915B CN115828915B (en) 2023-08-22

Family

ID=85523446

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211092241.4A Active CN115828915B (en) 2022-09-07 2022-09-07 Entity disambiguation method, device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN115828915B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108280061A (en) * 2018-01-17 2018-07-13 北京百度网讯科技有限公司 Text handling method based on ambiguity entity word and device
CN110555208A (en) * 2018-06-04 2019-12-10 北京三快在线科技有限公司 ambiguity elimination method and device in information query and electronic equipment
CN114021570A (en) * 2021-11-05 2022-02-08 平安普惠企业管理有限公司 Entity disambiguation method, apparatus, device and storage medium
CN114625889A (en) * 2022-03-25 2022-06-14 达而观信息科技(上海)有限公司 Semantic disambiguation method and device, electronic equipment and storage medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108280061A (en) * 2018-01-17 2018-07-13 北京百度网讯科技有限公司 Text handling method based on ambiguity entity word and device
US20190220749A1 (en) * 2018-01-17 2019-07-18 Beijing Baidu Netcom Science And Technology Co., Ltd. Text processing method and device based on ambiguous entity words
CN110555208A (en) * 2018-06-04 2019-12-10 北京三快在线科技有限公司 ambiguity elimination method and device in information query and electronic equipment
CN114021570A (en) * 2021-11-05 2022-02-08 平安普惠企业管理有限公司 Entity disambiguation method, apparatus, device and storage medium
CN114625889A (en) * 2022-03-25 2022-06-14 达而观信息科技(上海)有限公司 Semantic disambiguation method and device, electronic equipment and storage medium

Also Published As

Publication number Publication date
CN115828915B (en) 2023-08-22

Similar Documents

Publication Publication Date Title
WO2020108063A1 (en) Feature word determining method, apparatus, and server
US20220318275A1 (en) Search method, electronic device and storage medium
CN114861889B (en) Deep learning model training method, target object detection method and device
CN113836314B (en) Knowledge graph construction method, device, equipment and storage medium
CN112528641A (en) Method and device for establishing information extraction model, electronic equipment and readable storage medium
CN113836316B (en) Processing method, training method, device, equipment and medium for ternary group data
CN114116997A (en) Knowledge question answering method, knowledge question answering device, electronic equipment and storage medium
CN116955856A (en) Information display method, device, electronic equipment and storage medium
CN114818736B (en) Text processing method, chain finger method and device for short text and storage medium
CN115186163B (en) Training of search result ranking model and search result ranking method and device
CN116049370A (en) Information query method and training method and device of information generation model
CN114201607B (en) Information processing method and device
CN113868508B (en) Writing material query method and device, electronic equipment and storage medium
CN114417862A (en) Text matching method, and training method and device of text matching model
CN115098729A (en) Video processing method, sample generation method, model training method and device
CN115292506A (en) Knowledge graph ontology construction method and device applied to office field
CN114048315A (en) Method and device for determining document tag, electronic equipment and storage medium
CN114647739A (en) Entity chain finger method, device, electronic equipment and storage medium
CN115828915B (en) Entity disambiguation method, device, electronic equipment and storage medium
CN112989066A (en) Data processing method and device, electronic equipment and computer readable medium
CN112784600A (en) Information sorting method and device, electronic equipment and storage medium
CN113051926A (en) Text extraction method, equipment and storage medium
CN113377922B (en) Method, device, electronic equipment and medium for matching information
CN116069914B (en) Training data generation method, model training method and device
CN113377921B (en) Method, device, electronic equipment and medium for matching information

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant