CN115828915B - Entity disambiguation method, device, electronic equipment and storage medium - Google Patents

Entity disambiguation method, device, electronic equipment and storage medium Download PDF

Info

Publication number
CN115828915B
CN115828915B CN202211092241.4A CN202211092241A CN115828915B CN 115828915 B CN115828915 B CN 115828915B CN 202211092241 A CN202211092241 A CN 202211092241A CN 115828915 B CN115828915 B CN 115828915B
Authority
CN
China
Prior art keywords
entity
text
determining
word
candidate
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202211092241.4A
Other languages
Chinese (zh)
Other versions
CN115828915A (en
Inventor
陈子平
朱嘉琪
冯知凡
柴春光
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to CN202211092241.4A priority Critical patent/CN115828915B/en
Publication of CN115828915A publication Critical patent/CN115828915A/en
Application granted granted Critical
Publication of CN115828915B publication Critical patent/CN115828915B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Abstract

The disclosure provides an entity disambiguation method, an entity disambiguation device, electronic equipment and a storage medium, relates to the field of artificial intelligence, in particular to the technical fields of natural language processing, knowledge graph and the like, and can be applied to scenes such as intelligent industry, intelligent government affairs, search engines, intelligent customer service and the like. The specific implementation scheme of the entity disambiguation method is as follows: identifying a query text to obtain entity words to be disambiguated in the query text; determining a target text in a text obtained according to the query text; determining a candidate entity corresponding to the entity word and entity information of the candidate entity according to the entity word; wherein the candidate entities include at least two entities indicated by the same entity word and ambiguous to each other; and determining the target entity indicated by the entity word in the candidate entity according to the target text and the entity information of the candidate entity.

Description

Entity disambiguation method, device, electronic equipment and storage medium
Technical Field
The disclosure relates to the field of artificial intelligence, in particular to the technical fields of natural language processing, knowledge graph and the like, and can be applied to scenes such as intelligent industry, intelligent government affairs, search engines, intelligent customer service and the like.
Background
With the development of computer technology and network technology, deep learning technology is widely used in a plurality of fields. For example, deep learning techniques may be employed to perform natural language processing on Query text (Query) to enhance the search quality and use experience of a search engine. In the process of natural language processing of query text, it is necessary to understand text semantics and determine the intended needs of the query text. However, since query text often includes ambiguous entities, there are situations where the determined intent needs are inaccurate or even erroneous.
Disclosure of Invention
The present disclosure is directed to providing an entity disambiguation method, apparatus, device, and medium to improve accuracy of understanding text semantics of query text, and thus accuracy of intent requirements of the determined text, through disambiguation of entities.
According to one aspect of the present disclosure, there is provided an entity disambiguation method comprising: identifying a query text to obtain entity words to be disambiguated in the query text; determining a target text in a text obtained according to the query text; determining a candidate entity corresponding to the entity word and entity information of the candidate entity according to the entity word; wherein the candidate entities include at least two entities indicated by the same entity word and ambiguous to each other; and determining the target entity indicated by the entity word in the candidate entity according to the target text and the entity information of the candidate entity.
According to another aspect of the present disclosure, there is provided an entity disambiguation apparatus comprising: the entity recognition module is used for recognizing the query text to obtain entity words of the entity to be disambiguated in the query text; the target text determining module is used for determining target texts in texts obtained according to the query text; the candidate entity determining module is used for determining a candidate entity corresponding to the entity word and entity information of the candidate entity according to the entity word; wherein the candidate entities include at least two entities indicated by the same entity word and ambiguous to each other; and the target entity determining module is used for determining the target entity indicated by the entity word in the candidate entity according to the target text and the entity information of the candidate entity.
According to another aspect of the present disclosure, there is provided an electronic device including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the entity disambiguation methods provided by the present disclosure.
According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the entity disambiguation method provided by the present disclosure.
According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program/instruction which, when executed by a processor, implements the entity disambiguation method provided by the present disclosure.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.
Drawings
The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
fig. 1 is an application scenario schematic diagram of an entity disambiguation method and apparatus according to an embodiment of the present disclosure;
FIG. 2 is a flow diagram of a method of entity disambiguation, according to an embodiment of the present disclosure;
FIG. 3 is a schematic diagram of an entity disambiguation method according to an embodiment of the present disclosure;
FIG. 4 is a schematic diagram of determining entity types of entities indicated by entity words to be disambiguated in query text, in accordance with an embodiment of the present disclosure;
FIG. 5 is a schematic diagram of determining similarity between a text heading of a target text and entity information of a candidate entity according to an embodiment of the disclosure;
FIG. 6 is a schematic diagram of determining similarity between a text title of a target text and entity information of a candidate entity according to another embodiment of the present disclosure;
Fig. 7 is a block diagram of the physical disambiguation device according to an embodiment of the present disclosure; and
fig. 8 is a block diagram of an electronic device used to implement an entity disambiguation method of an embodiment of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Query text (Query) received by search engines is a vast array of changes, representing the diverse needs of users. The demand intent expressed by the query text may include a number of different intents due to the presence of the word ambiguous case. To accurately determine the demand intent expressed by the query text, entity words representing multiple entities may be disambiguated. For example, the entity indicated by the entity word in the query text may be determined based on the contextual information of the query text, thereby determining the demand intent expressed by the query text. However, in the case where the query text is short text or the expressed context information is limited, disambiguation results in poor effect, and there is still a case where the entity indicated by the entity word determined after disambiguation is ambiguous. It can be appreciated that the search engine may be a web page version search engine that is commonly used, a search engine that is used when searching for a reply text in an intelligent customer service scenario, or a search engine that is used when searching for a text in a smart industry or smart government scenario, which is not limited in this disclosure.
To avoid this situation, the present disclosure provides an entity disambiguation method, apparatus, electronic device, and storage medium.
An application scenario of the method and apparatus for entity disambiguation provided by the present disclosure will be schematically described below with reference to fig. 1.
As shown in fig. 1, the application scenario 100 of this embodiment may include an electronic device 110, and the electronic device 110 may be various electronic devices with processing functions, including but not limited to a smart phone, a tablet computer, a laptop computer, a desktop computer, a server, and the like.
The electronic device 110 may be equipped with, for example, a web browser, a search class application, an instant messaging class application, and/or a shopping class application, among many types of client applications, to name a few. The electronic device 110 may, for example, in response to receiving the query text 120, perform entity recognition on the query text 120, and determine the entity words in the query text that need to be disambiguated.
In an embodiment, the electronic device 110 may further disambiguate the entity word to be disambiguated by performing semantic understanding on the query text 120, and determine the entity indicated by the entity word in the query text. After disambiguation, the electronic device 110 may determine query results 130 derived in response to the query text based on the semantics of the disambiguated query text and display the query results.
In an embodiment, the electronic device 110 may also send the query text 120 to the server 140, and the server 140 obtains, from the database 150, information related to the query text 120 according to the query text 120, and feeds back the obtained information to the electronic device 110 as an initial query result. The electronic device 110 may disambiguate the entity word to be disambiguated based on the received initial query result and the query text 120, and determine an entity indicated by the entity word in the query text. The electronic device 110 may then send the post-disambiguation query text to the server 140, such that the server 140 obtains information related to the post-disambiguation query text from the database 150 according to the post-disambiguation query text, and feeds back the information related to the post-disambiguation query text to the electronic device 110 as a query result for presentation by the electronic device 110.
Wherein the electronic device 110 may be communicatively coupled to the server 140 via a network. The network may include wired or wireless communication links. Server 140 may be a background management server that supports the running of client applications provided in electronic device 110.
In an embodiment, the electronic device 110 may also send the query text 120 to the server 140, and the server 140 performs entity recognition on the query text 120 to determine entity words in the query text 120 that need to be disambiguated. At the same time, the query text 120 may be semantically understood by the server 140 to disambiguate the entity terms that need to be disambiguated.
It should be noted that, the entity disambiguation method provided by the present disclosure may be performed by the electronic device 110 or may be performed by the server 140. Accordingly, the entity disambiguation apparatus provided by the present disclosure may be disposed in the electronic device 110 or in the server 140.
It should be understood that the number and types of electronic devices 110, servers 140, and databases 150 in fig. 1 are merely illustrative. There may be any number and type of electronic devices 110, servers 140, and databases 150 as desired for an implementation.
The entity disambiguation method provided by the present disclosure will be described in detail below in connection with fig. 2-6.
Fig. 2 is a flow diagram of an entity disambiguation method according to an embodiment of the present disclosure.
As shown in fig. 2, the entity disambiguation method 200 of this embodiment may include operations S210 through S240.
In operation S210, the query text is recognized, and the entity word to be disambiguated in the query text is obtained.
In accordance with embodiments of the present disclosure, deep learning methods may be employed to identify query text. For example, an entity recognition model may be used to recognize the entity words in the query text to obtain at least one entity word. The embodiment may choose the entity word from the at least one entity word for which disambiguation is required. The entity recognition model may be a model composed of a Long-short-term memory network (Long-Short Term Memory, LSTM) and a conditional random field, or a model composed of an expanded convolution network (expanded related CNN, IDCNN) and a conditional random field. It will be appreciated that the above-described structure of the entity recognition model is merely an example to facilitate understanding of the present disclosure, which is not limited thereto.
For example, the present disclosure may maintain an entity library of ambiguous entity words. The embodiment can query the entity library according to the at least one entity word after obtaining the at least one entity word, determine the entity word belonging to the entity library in the at least one entity word, and take the determined entity word as the entity word to be disambiguated.
In an embodiment, the embodiment may determine the entity words to be disambiguated based on the grammatical composition of the query text. For example, the entity word in the query text that is the subject may be used as the entity word to be disambiguated.
For example, the embodiment may employ a sequence annotation model to annotate entity words to be disambiguated in query text. The embodiment can firstly perform word segmentation processing on the query text to obtain a word sequence. Then, the word sequence is input into a sequence labeling model, and labeling information for each word in the word sequence is output by the sequence labeling model. The labeling information may include, for example, a label indicating a B-I-O (Begin-side) classification, and the labeling information may indicate which category of each word in the word sequence is a start word (labeled Begin) of an entity word to be disambiguated, other words (labeled side) of the entity word to be disambiguated other than the start word, and other words (labeled side) other than the word included in the entity word to be disambiguated. The embodiment can take the word spliced by the words marked as B and the word marked as I as the entity word to be disambiguated. The sequence labeling model may be a model constructed based on a bidirectional encoder characterization network (BERT) of a transducer, or may be a model constructed based on a centroid model (ERNIE), which is not limited in this disclosure.
In operation S220, a target text among texts obtained according to the query text is determined.
According to the embodiment of the disclosure, the text in the predetermined database can be queried based on the query text, and a predetermined number of texts with highest correlation degree with the query text in the queried text are used as target texts. Wherein the predefined database may be, for example, a knowledge database maintained for a search engine, or the like. The text in the predetermined database may include plain text, text in teletext format, text containing video, and the like. The target text in the text resulting from the query may be represented, for example, by a text title, typically in plain text format. This embodiment may represent the correlation between the text in the predetermined database and the query text by the similarity between the query text and the text titles of the text in the predetermined database.
In operation S230, a candidate entity corresponding to the entity word and entity information of the candidate entity are determined according to the entity word.
In the entity library formed by the ambiguous entity words, for example, entity information of the entity represented by the ambiguous entity words can be maintained, and the entity information can be attribute information of the entity and/or description information of the entity, for example. For example, the entity library may maintain a mapping relationship between entity identifiers and entity words, and also maintain a mapping relationship between entity identifiers and entity information. Different entities represented by the same entity word have different entity identities. The embodiment can search the entity library according to the entity words, determine the entity identifications with the mapping relation with the entity words, take the entity with the determined entity identifications as candidate entities, and take the entity information with the mapping relation with the determined entity identifications as the entity information of the candidate entities.
According to the embodiment of the disclosure, when the entity word indicates two different entities, the entity word is the entity word needing disambiguation. If the entity identification determined by searching the entity library is at least two, it may be determined that the entity word in the query text is the entity word to be disambiguated, and operation S240 is continued. If the entity identification determined by searching the entity library is only one, it can be determined that the entity word in the query text does not need to be disambiguated, and the target entity indicated by the entity word with the determined entity identification can be determined.
In operation S240, a target entity indicated by the entity word in the candidate entity is determined according to the target text and entity information of the candidate entity.
According to embodiments of the present disclosure, a target entity may be determined according to a similarity between a target text and entity information of a candidate entity. For example, the embodiment may use the candidate entity having the highest similarity with the target text as the target entity.
In one embodiment, the target text and the query text may be combined into one text, and then the target entity may be determined based on the similarity between the combined text and the entity information of the candidate entity.
When determining the similarity between the text and the entity information, for example, a semantic feature extraction network may be used to extract semantic features of the text and the entity information, and the similarity between the semantic features of the text and the semantic features of the entity information may be used as the similarity between the text and the entity information. The semantic feature extraction network may be, for example, a recurrent neural network, an encoder of a transducer, a pre-training characterization network (e.g., BERT or ERNIE, etc.), etc., which is not limited in this disclosure.
When the embodiment of the disclosure disambiguates entity words in the query text, the target text is obtained according to the query text. And then, according to the entity information of the candidate entity represented by the target text and the entity word, performing disambiguation of the entity word, and adding the characteristics related to the query text to the disambiguation process. Especially, under the condition that the query text is short text or the information expressed by the query text is insufficient, by introducing the target text, less context information of the query text can be made up, the expression capability of the query text in the disambiguation process is improved, the complete disambiguation of ambiguous entity words is realized, and the disambiguation precision is improved. This is because the target text is typically an extension of the query text, which may also provide more information than can represent the query text.
It is understood that the operation of identifying the query text in operation S210 may be understood as an operation of identifying the entity mention, i.e., obtaining the entity word indicating the to be disambiguated through the entity mention identification (Entity Mention Detection, EMD). Wherein an entity reference refers to a reference expression (Referring Expression) in the text that is used to express an entity. The purpose of the entity-mention identification task is to identify all references, including naming references, noun references, and also pronoun references, of an entity of a specified type and to consider the nested structure between entity references.
Fig. 3 is a schematic diagram of an entity disambiguation method according to an embodiment of the present disclosure.
According to embodiments of the present disclosure, before determining the candidate entity, for example, an entity type of the entity indicated by the entity word in the query text may also be determined, and then the candidate entity may be determined according to the entity type and the entity word. For example, an entity indicated by an entity word to be disambiguated in the query text and having the same type as the entity indicated by the entity word in the query text may be used as the candidate entity. Therefore, the number of the determined candidate entities can be reduced, the determined candidate entities can be more fit with the query text, a more reliable basis is provided for determining the target entity, and the entity disambiguation efficiency is improved.
For example, the entity type of the entity indicated by the entity word in the query text may be determined from the contextual information of the entity word to be disambiguated in the query text. The embodiment may input query text into the entity type prediction model, and determine the entity type from the output of the entity type prediction model. The entity type prediction model may be, for example, a model formed by a semantic feature extraction network and a full connection layer.
In one embodiment, the target text and the entity words may be combined to determine the entity type of the entity indicated by the entity word in the query text. For example, the entity type of the entity indicated by the entity word in the query text may be determined based on the context information of the target text. The target text can also provide more rich information on the basis of representing the query text, so that the accuracy of determining the entity type can be improved, as the target text is usually an extension of the query text.
On this basis, as shown in fig. 3, the implementation principle of the entity disambiguation method of this embodiment 300 may be: the sequence annotation model 310 is employed to annotate the entity words to be disambiguated in the query text 301. Meanwhile, knowledge database 320 may be queried from query text 301 to obtain target text 302. The entity type prediction model 330 may then be employed to determine the entity type 304 of the entity indicated by the entity word 303 to be disambiguated from the target text 302 and the entity word 303 to be disambiguated. The entity library 340 may then be queried based on the entity type 304 and the entity words 303 to obtain candidate entities 305 and entity information 306 for the candidate entities 305. Finally, a target entity 307 of the candidate entities 305 is determined from the entity information 306 and the target text 302.
Taking the query text 301 as "XY is that country" as an example, the sequence labeling result output by the sequence labeling model can be B I O O O O O O, eight labels in the sequence of labeling results correspond one-to-one with eight words in query text 301, wherein B represents a start word of the entity word to be disambiguated, I represents a word except the start word in the entity word to be disambiguated, and O represents other words except the word included in the entity word to be disambiguated. The entity word 303 to be disambiguated may be determined to be the word "XY" based on the labeling result. It will be appreciated that candidate entities 305 obtained by querying the entity library 340 are in one-to-one correspondence with the entity information 306, and that when the entity word indicates at least two entities, the candidate entities 305 include at least two entities.
When determining the entity type 304 according to the target text 302 and the entity word 303 to be disambiguated, for example, the location of the entity word 303 to be disambiguated in the target text 302 may be marked first, the marked target text is input into the entity type prediction model 330, and the entity type 304 is determined according to the output of the entity type prediction model 330. For example, for the target text "XY", belonging to country a, the population is aaa and the floor space is bbb ", the text" # xy#, XY ", belonging to country a, the population is aaa and the floor space is bbb" can be obtained by adding a label at the entity word "XY" to be disambiguated.
After obtaining the entity information 306 and the target text 302, the embodiment may determine, as the target entity 307, the candidate entity with the highest similarity between the entity information and the target text 302 according to the similarity between the entity information 306 and the target text 302 of each candidate entity. For example, a similarity prediction model 350 may be employed to determine the similarity between the entity information and the target text 302. The similarity prediction model 350 may include, among other things, the semantic feature extraction network described above to extract semantic features of the entity information and the target text. The similarity prediction model 350 may be, for example, a single tower model or a double tower model, which is not limited by the present disclosure.
In accordance with an embodiment of the present disclosure, the entity library 340 may, for example, maintain a mapping relationship between entity words and entity identifiers, where an entity identifier uniquely corresponds to one entity. And the entity library 340 may also maintain a mapping relationship between entity identifiers and entity information of entities corresponding to the entity identifiers. In this embodiment, when querying the entity library 340, the entity indicated by the entity identifier having the mapping relationship with the entity word 303 in the entity library 340 may be determined by using the entity word to be disambiguated as the query basis, and the entity is used as the candidate entity. It will be appreciated that each entity identifier in the entity library may also be added, for example, with labeling information for the type of entity represented by the each entity identifier. After determining the entity identifier having a mapping relationship with the entity word 303 in the entity library 340, the embodiment can determine the type of the entity corresponding to the entity identifier having the mapping relationship according to the labeling information of the entity identifier having the mapping relationship. The entity with the mapping relationship is then identified as a candidate entity 305, where the entity type corresponds to the entity with the entity type 304. Then, after determining the entity identifier having a mapping relationship with the entity word, entity information having a mapping relationship with the entity identifier having a mapping relationship in the entity library 340 may be determined as entity information of the entity indicated by the entity identifier having a mapping relationship.
According to embodiments of the present disclosure, the mappings in the entity library 340 may be updated in real-time, for example, and may specifically be updated according to updates of entities in the upstream application. Therefore, when the entity words in the online query text are disambiguated, the entity words can be disambiguated by combining the latest information provided by the upstream application, so that the disambiguation effect can be improved, and the accuracy of the recommended information obtained by the query can be improved. For example, if the query text is text entered through a search engine, the upstream application may be an encyclopedia application or the like, which is not limited by the present disclosure. Wherein the updating of the entity in the upstream application may for example comprise at least one of: an addition of an entity, a deletion of an entity, and a modification of entity information.
For example, this embodiment may employ a streaming computing framework to monitor for updates of entities in upstream applications. The streaming computing framework may include Strorm, spark, flink, etc. For example, the streaming framework may be used to snoop a message queue (e.g., kafka message, etc.) for messages that are changed by entities in the upstream application.
The embodiment can adopt mainstream frame gRPC, flash or springboot, etc. to modify the mapping information in entity library 340 according to the monitored information of entity change in the message queue based on rpc/http service, for example, the mapping relation of entity identifier in entity library 340 can be added and deleted, or the entity information with mapping relation with entity identifier can be modified.
Illustratively, the entity library 340 may be a Key-value database. For example, the entity library 340 may be any one of databases such as Redis, rawbase, levelDB, rocksDB, which is not limited in this disclosure.
Fig. 4 is a schematic diagram of determining an entity type of an entity indicated by an entity word to be disambiguated in query text, according to an embodiment of the present disclosure.
According to embodiments of the present disclosure, the entity type of the entity indicated by the entity word to be disambiguated may be determined from the text title of the target text. This is because the title of the text can usually express the text more completely, and the entity type is determined only according to the text title, so that the calculation amount can be reduced.
In one embodiment, the first text feature may be derived from a text title of the target text. For example, a text feature extraction network may be employed to extract text features of a text title, resulting in a first text feature. The first text feature may then be processed using the full join layer, the full join layer outputting a probability vector of a plurality of probability values corresponding to a plurality of predetermined types. The embodiment may use a predetermined type corresponding to a maximum probability value among the plurality of probability values as the entity type.
As shown in fig. 4, in an embodiment 400, the text label of the target text is set to "XY (one of the main cities of country a)", and the entity word to be disambiguated is set to "XY". The embodiment may first add an identifier at the location of the entity word to be disambiguated in the text header, to obtain the input text 401"#xy# (one of the main cities of country a)". The embodiment 400 may perform word segmentation processing on the input text 401 to obtain a word sequence 402. The embedded features 403 of the word sequence 402 are then input into a text feature extraction network 410 comprised by the entity type prediction model, and the initial text features 404 represented by the feature sequence are output by the text feature extraction network 410. Wherein features in the feature sequence may correspond one-to-one with words in the word sequence 402.
The embodiment may then extract text sub-features corresponding to the entity words from the initial text features 404 according to the locations of the entity words in the target text, resulting in first text features 405, and then determine the entity type of the entity indicated by the entity words in the query text according to the first text features 405. For example, the embodiment may splice features corresponding to entity words in the initial text features 404, thereby obtaining the first text features 405. The first text feature 405 may then be input into a fully connected layer 420 comprised by the entity type prediction model, and a probability vector may be output by the fully connected layer 420 from which the entity type 406 may be determined.
In the embodiment, when determining the entity type, by extracting the corresponding text sub-feature from the initial text feature according to the position of the entity word to be disambiguated in the text title of the target text and then determining the entity type according to the text sub-feature only, the calculation amount in determining the entity type can be reduced as much as possible on the basis of ensuring that the text sub-feature can fully express the semantics of the entity word to be disambiguated.
In an embodiment, not only the text title but also the query text may be considered, for example, when obtaining the first text feature 405. In this way, the expression capability of the extracted first text feature 405 can be improved, and more abundant information is provided for determining the entity type.
For example, the embodiment may first splice text titles of the query text and the target text to obtain the first spliced text. And then, extracting text features of the first spliced text to obtain first text features. The principle of extracting the text features of the first spliced text to obtain the first text features is similar to that of the embodiment 400 of obtaining the first text features 405 according to the text title of the target text, and will not be described herein. Where, for example, the query text is "XY is one of the main cities of the country", and the text of the target text is "XY (one of the main cities of the country)", the first spliced text spliced may be "XY is one of the main cities of the country". To distinguish between the query text and the text title, a separation character may also be added between the query text and the text title, or an identification representing the query text may be added before the query text, and an identification representing the text title may be added between the text title and the query text, thereby obtaining the first spliced text.
According to the embodiment of the disclosure, in general, the target text includes a plurality of texts, and in the embodiment, when determining the entity type, the type information of the entity indicated by the entity word to be disambiguated can be determined according to each text in the plurality of texts, and then the plurality of types of information can be obtained according to the plurality of texts. Finally, the embodiment may determine the entity type based on the plurality of type information. The type information may be, for example, a probability vector formed by a plurality of probability values corresponding to a plurality of predetermined types.
For example, the embodiment may determine an entity type according to each type information, and may aggregate a predetermined type corresponding to a maximum probability value in the probability vector as the entity type determined according to each type information to obtain a plurality of entity types. The embodiment can take the most valued type of the entity types as the entity type of the entity indicated by the entity word in the query text.
In an embodiment, a predetermined weight may also be assigned in advance to each of the plurality of texts included in the target text. Then, the entity type of the entity indicated by the entity word in the query text is determined according to a plurality of predetermined weights and a plurality of types of information corresponding to the plurality of texts. For example, a plurality of probability vectors representing a plurality of types of information may be weighted according to a plurality of predetermined weights, resulting in weighted probability vectors. And finally, taking the preset type corresponding to the maximum probability value in the weighted probability vector as the entity type of the entity indicated by the entity word in the query text.
Wherein each text of the target text may be assigned a predetermined weight, for example, based on a degree of relevance between the text and the query text, the assigned predetermined weight may be positively correlated with the degree of relevance, for example. Alternatively, each text in the target text may be assigned a predetermined weight, for example, based on the access data for that text. The access data may include, for example, a reading amount, a collection amount, and/or a praise amount, among others. For example, the predetermined weight may be positively correlated with the reading amount, the collection amount, and/or the praise amount. Alternatively, for example, each text may be assigned a predetermined weight according to the arrangement order of the text in a plurality of texts included in the target text, the earlier the arrangement order, the higher the assigned predetermined weight.
According to the embodiment, the predetermined weight is distributed to the texts, and the entity type of the entity indicated by the entity word in the query text is determined by combining the predetermined weight, so that the type information determined according to the text with high reliability can be more focused when the entity type is determined, the accuracy of the determined entity type can be improved, and the disambiguation effect and disambiguation efficiency can be improved.
According to the embodiment of the present disclosure, similar to the principle of determining the entity type of the entity indicated by the entity word to be disambiguated, the embodiment may consider the text title of the target text when determining the target entity, thereby reducing the amount of calculation. For example, the embodiment may first determine the similarity between the text header of the target text and the entity information of the candidate entity, and then determine the entity with the highest similarity between the entity information in the candidate entity and the text header of the target text as the target entity.
The operation of determining the similarity between the text header of the target text and the entity information will be further expanded and defined in connection with fig. 5 to 6.
Fig. 5 is a schematic diagram of determining similarity between a text title of a target text and entity information of a candidate entity according to an embodiment of the present disclosure.
According to embodiments of the present disclosure, the similarity of text features between the text title of the target text and the entity information may be employed to represent the similarity between the text title of the target text and the entity information.
For example, the embodiment may first derive the second text feature based on the text title of the target text. When the candidate entity includes a plurality of entities, the text feature of the entity information of each entity in the candidate entity can be extracted while the second text feature is obtained, so as to obtain a third text feature. Then, a similarity between the text title of the target text and the entity information of each entity is determined based on the second text feature and the third text feature. For example, the similarity between the second text feature and the third text feature may be regarded as the similarity between the text title of the target text and the entity information of each entity. Then multiple similarities can be obtained for the multiple entities included in the candidate entity, one-to-one correspondence with the multiple entities. The embodiment can take the entity corresponding to the maximum similarity in the multiple similarities as the target entity.
According to embodiments of the present disclosure, as shown in fig. 5, a double-tower model may be employed to determine the similarity between the text title of the target text and the entity information of the candidate entity. The dual-tower model may include a first text feature extraction network 510 and a second text feature extraction network 520, a stitching network 530, and a full connectivity layer 540, arranged in parallel.
This embodiment may obtain input to the first text feature extraction network 510 by word-splitting and embedding the text header 501 of the target text. Similarly, the entity information 502 of each entity is word-processed and represented embedded to obtain the input of the second text feature extraction network 520. The second text feature 503 is output by the first text feature extraction network 510 and the third text feature 504 is output by the second text feature extraction network 520. It is understood that the first text feature extraction network 510 and the second text feature extraction network 520 may be two networks with the same network structure but different network parameters, or any two networks with similar network structures. The two feature extraction networks are similar to the text feature extraction networks in the entity type prediction model described above and will not be described in detail here.
The embodiment 500 may take the second text feature 503 and the third text feature 504 as inputs to a stitching network 530, and stitch the second text feature 503 and the third text feature 504 by the stitching network 530 to obtain the feature 505. The feature 505 is then input into the fully connected layer 540, and the similarity 506 between the text header 501 and the entity information 502 of each entity is output by the fully connected layer 540. It is to be appreciated that the functionality of the spliced network 530 may also be integrated into the fully connected layer 540, for example.
Wherein, when stitching the second text feature 503 and the third text feature 504, the feature vector representing the third text feature 504 may be stitched directly after the feature vector representing the second text feature 503. Alternatively, to distinguish the second text feature 503 from the third text feature 504 in the stitching feature, a segmentation identification feature may be added after the feature vector representing the second text feature 503, and then the feature vector representing the third text feature 504 may be stitched after the segmentation identification feature. The segmentation identification feature may be any identifier, which is not limited by the present disclosure.
In an embodiment, the principle of obtaining the first text feature is similar to that described above, and when the second text feature is obtained, the text titles of the query text and the target text may be spliced first to obtain the first spliced text. Subsequently, the text features of the first spliced text are extracted by using the first text feature extraction network 510, and the second text features are obtained. Therefore, the expression capability of the extracted second text feature can be improved, and richer information is provided for the determination of the similarity.
In one embodiment, the text feature obtained from the text header of the target text may be used as the complete text feature. That is, the text feature output by the first text feature extraction network 510 is taken as a complete text feature. And then, determining a sub-feature corresponding to the entity word in the complete text feature according to the position of the entity word to be disambiguated in the target text, and taking the sub-feature as a second text feature. It will be appreciated that the principle of determining the sub-feature corresponding to the entity word in the complete text feature is similar to the principle of extracting the text sub-feature corresponding to the entity word from the first text feature described above, and will not be described herein. In this way, the embodiment can reduce the calculation amount in determining the similarity as much as possible on the basis that the second text feature for determining the similarity can ensure that the semantics of the entity word to be disambiguated are sufficiently expressed.
According to the embodiment of the present disclosure, in the case where the target text includes a plurality of texts, in determining the similarity, the embodiment may first determine the similarity between the text title of each text in the plurality of texts and the entity information of the candidate entity, and obtain a plurality of initial similarities corresponding to the plurality of texts, for example, using a principle similar to the principle of determining the entity type according to the predetermined weight assigned to the plurality of texts. Then, a similarity between the text title of the target text and the entity information of the candidate entity is determined based on a plurality of predetermined weights corresponding to the plurality of texts and a plurality of initial similarities. The allocation of the predetermined weights is similar to the allocation principle described above, and will not be described here again. The embodiment may weight a plurality of initial similarities according to a plurality of predetermined weights, and the weighted similarities are used as the similarities between the text titles of the target text and the entity information of the candidate entity. In the case where the candidate entity includes a plurality of entities, the operations of the present embodiment may be performed for each entity, thereby obtaining the similarity between the text title of the target text and the entity information of each entity.
Fig. 6 is a schematic diagram of determining a similarity between a text title of a target text and entity information of a candidate entity according to another embodiment of the present disclosure.
As shown in fig. 6, in an embodiment 600, a single tower model may be employed to determine the similarity between the text title of the target text and the entity information for each entity. Wherein the single tower model may include a text feature extraction network 610 and a full connectivity layer 620.
In this embodiment, when determining the similarity, the text header 601 of the target text and the entity information 602 of each entity may be spliced first to obtain the second spliced text 603. Subsequently, the text feature extraction network 610 is used to extract text features of the second stitched text 603, resulting in fourth text features 604. Then, using the full connection layer 620, a similarity 605 between the text header 601 of the target text and the entity information 602 of each entity is determined according to the fourth text feature 604.
In the case of splicing the text header 601 and the entity information 602, identification information indicating a single text may be added before the text header 601 and the entity information 602, respectively. For example, the identification information "text_a" may be added before the text title 601, and the identification information "text_b" may be added before the entity information 602. Subsequently, the entity information 602 added with the identification information is spliced after the text header 601 added with the identification information, thereby obtaining a second spliced text 603.
It is understood that the text feature extraction network 610 is the same or similar in structure to each of the text feature extraction networks 610 described above. The text feature extraction networks in the sequence annotation model, the entity type prediction model, and the similarity prediction model may have the same structure. In an end-to-end system, the text feature extraction network in the three models may share network parameters.
In an embodiment, for example, a model formed by the pretrained network Bert and the full connection layer may be used as a basic model, and the basic model is respectively trained for the sequence labeling task, the entity type prediction task and the similarity prediction task, so as to obtain the sequence labeling model, the entity type prediction model and the similarity prediction model respectively.
In an embodiment, in the process of obtaining the fourth text feature, for example, the text feature of the second spliced text extracted by the text feature extraction network may be used as an initial feature, and then, according to the position of the entity word to be disambiguated in the second spliced text, a sub-feature corresponding to the entity word to be disambiguated in the initial feature is determined, and the extracted sub-feature is used as the fourth text feature. It will be appreciated that the principle of extracting the sub-feature to obtain the fourth text feature is similar to that of extracting the sub-feature to obtain the first text feature, and will not be described in detail herein. Therefore, the calculation amount in the process of determining the similarity can be reduced as much as possible on the basis of ensuring that the fourth text feature can fully express the semantics of the entity word to be disambiguated.
Based on the entity disambiguation method provided by the disclosure, the disclosure also provides an entity disambiguation device. The device will be described in detail below in connection with fig. 7.
Fig. 7 is a block diagram of the structure of an entity disambiguation device according to an embodiment of the present disclosure.
As shown in fig. 7, the entity disambiguation apparatus 700 of this embodiment may include an entity identification module 710, a target text determination module 720, a candidate entity determination module 730, and a target entity determination module 740.
The entity recognition module 710 is configured to recognize the query text, and obtain entity words to be disambiguated in the query text. In an embodiment, the entity identification module 710 may be configured to perform the operation S210 described above, which is not described herein.
The target text determination module 720 is configured to determine target text in text obtained according to the query text. In an embodiment, the target text determining module 720 may be configured to perform the operation S220 described above, which is not described herein.
The candidate entity determining module 730 is configured to determine a candidate entity corresponding to the entity word and entity information of the candidate entity according to the entity word. Wherein the candidate entities comprise at least two entities indicated by the same entity word and ambiguous to each other. In an embodiment, the candidate entity determining module 730 may be configured to perform the operation S230 described above, which is not described herein.
The target entity determining module 740 is configured to determine, according to the target text and entity information of the candidate entity, a target entity indicated by the entity word in the candidate entity. In an embodiment, the target entity determining module 740 may be configured to perform the operation S240 described above, which is not described herein.
According to an embodiment of the present disclosure, the apparatus 700 may further comprise a type determination module for determining an entity type of an entity indicated by the entity word in the query text, for example. The above-mentioned candidate entity determining module 730 may specifically be configured to determine, according to the entity word and the entity type, a candidate entity corresponding to the entity word and entity information of the candidate entity, where the entity type of the candidate entity is the same as the entity type of the entity indicated by the entity word.
According to an embodiment of the disclosure, the type determining module may be specifically configured to determine, according to the target text and the entity word, an entity type of the entity indicated by the entity word in the query text.
According to an embodiment of the present disclosure, the above-described type determining module may include: the initial characteristic determining sub-module is used for obtaining initial text characteristics according to the text title of the target text; the feature extraction sub-module is used for extracting text sub-features corresponding to the entity words from the initial text features according to the positions of the entity words in the text titles to obtain first text features; and the first type determining submodule is used for determining the entity type of the entity indicated by the entity word in the query text according to the first text characteristic.
According to an embodiment of the present disclosure, the initial feature determination submodule includes: the first text splicing unit is used for splicing the text titles of the query text and the target text to obtain a first spliced text; and the first feature extraction unit is used for extracting text features of the first spliced text to obtain initial text features.
According to an embodiment of the present disclosure, the target text includes a plurality of texts, and the above-described type determining module may include: the type information obtaining sub-module is used for determining type information of an entity indicated by the entity word in the query text according to each text and the entity word in the plurality of texts to obtain a plurality of type information corresponding to the plurality of texts; and a second type determining sub-module for determining an entity type of an entity indicated by the entity word in the query text according to a plurality of predetermined weights and a plurality of type information corresponding to the plurality of texts, wherein the predetermined weights are related to at least one of the following information: correlation between each text and the query text; access data for each text.
According to an embodiment of the present disclosure, the target entity determining module 740 may include: the similarity determination submodule is used for determining similarity between a text title of the target text and entity information of the candidate entity; and the entity determining submodule is used for determining the entity with the highest similarity between the entity information in the candidate entity and the text title of the target text as the target entity.
According to an embodiment of the present disclosure, the similarity determination submodule includes: the second feature extraction unit is used for obtaining second text features according to the text titles of the target text; a third feature extraction unit, configured to extract text features of entity information of each entity in the candidate entities, to obtain third text features; and a first similarity determining unit configured to determine a similarity between a text title of the target text and entity information of each entity based on the second text feature and the third text feature.
According to an embodiment of the present disclosure, the above-described second feature extraction unit includes: the text splicing subunit is used for splicing the text titles of the query text and the target text to obtain a first spliced text; and the first characteristic extraction subunit is used for extracting text characteristics of the first spliced text to obtain second text characteristics.
According to an embodiment of the present disclosure, the above-described second feature extraction unit includes: a complete characteristic determining subunit, configured to obtain a complete text characteristic according to a text title of the target text; and a second feature extraction subunit, configured to extract, according to the position of the entity word in the target text, a text sub-feature corresponding to the entity word in the complete text feature, as a second text feature.
According to an embodiment of the present disclosure, the similarity determination submodule may include: the second text splicing unit is used for splicing the text title of the target text and the entity information of each entity in the candidate entities to obtain a second spliced text; the fourth feature extraction unit is used for extracting text features of the second spliced text to obtain fourth text features; and a second similarity determination unit configured to determine a similarity between the text title of the target text and the entity information of each entity, based on the fourth text feature.
According to an embodiment of the present disclosure, the fourth feature extraction unit is specifically configured to: and determining sub-features corresponding to the entity words in the extracted text features of the second spliced text as fourth text features according to the positions of the entity words in the second spliced text.
According to an embodiment of the present disclosure, the target text includes a plurality of texts. The similarity determination submodule may include: a first similarity determining unit, configured to determine a similarity between a text title of each text in the plurality of texts and entity information of the candidate entity, to obtain a plurality of initial similarities corresponding to the plurality of texts; and a second similarity determining unit configured to determine a similarity between a text title of the target text and entity information of the candidate entity based on a plurality of predetermined weights corresponding to the plurality of texts and a plurality of initial similarities, wherein the predetermined weights are related to at least one of: correlation between each text and the query text; access data for each text.
According to an embodiment of the present disclosure, the candidate entity determining module 730 may include a candidate entity determining submodule, configured to determine, as a candidate entity, an entity indicated by an entity identifier having a mapping relationship with an entity word in a predetermined entity library; the entity information determining submodule is used for determining entity information with a mapping relation with the entity identifier in a preset entity library as entity information of the entity indicated by the entity identifier, wherein the mapping relation in the entity library is updated by monitoring the update of the entity in the upstream application; the updating of the entity in the upstream application comprises at least one of: an addition of an entity, a deletion of an entity, and a modification of entity information.
In the technical scheme of the disclosure, the related processes of collecting, storing, using, processing, transmitting, providing, disclosing and applying personal information of the user all conform to the regulations of related laws and regulations, necessary security measures are adopted, and the public welcome is not violated. In the technical scheme of the disclosure, the authorization or consent of the user is obtained before the personal information of the user is obtained or acquired.
According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
Fig. 8 shows a schematic block diagram of an example electronic device 800 that may be used to implement the entity disambiguation method of embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 8, the apparatus 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 802 or a computer program loaded from a storage unit 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to the bus 804.
Various components in device 800 are connected to I/O interface 805, including: an input unit 806 such as a keyboard, mouse, etc.; an output unit 807 such as various types of displays, speakers, and the like; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, or the like. The communication unit 809 allows the device 800 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
The computing unit 801 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 801 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the entity disambiguation method. For example, in some embodiments, the entity disambiguation method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 800 via ROM 802 and/or communication unit 809. When the computer program is loaded into RAM 803 and executed by computing unit 801, one or more steps of the entity disambiguation method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the entity disambiguation method by any other suitable means (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service ("Virtual Private Server" or simply "VPS"). The server may also be a server of a distributed system or a server that incorporates a blockchain.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel or sequentially or in a different order, provided that the desired results of the technical solutions of the present disclosure are achieved, and are not limited herein.
The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.

Claims (17)

1. A method of entity disambiguation, comprising:
identifying a query text to obtain entity words to be disambiguated in the query text;
determining target text in the text obtained according to the query text; the target text is an expanded text of the query text;
determining a candidate entity corresponding to the entity word and entity information of the candidate entity according to the entity word; wherein the candidate entities include at least two entities indicated by the same entity word and ambiguous to each other; and
Determining a target entity indicated by the entity word in the candidate entity according to the target text and the entity information of the candidate entity;
wherein the method further comprises:
determining the entity type of the entity indicated by the entity word in the query text;
wherein, according to the entity word, determining the candidate entity corresponding to the entity word and the entity information of the candidate entity includes:
determining a candidate entity corresponding to the entity word and entity information of the candidate entity according to the entity word and the entity type,
and the entity type of the candidate entity is the same as the entity type of the entity indicated by the entity word.
2. The method of claim 1, wherein the determining the entity type of the entity indicated by the entity word in the query text comprises:
and determining the entity type of the entity indicated by the entity word in the query text according to the target text and the entity word.
3. The method of claim 2, wherein the determining, from the target text and the entity words, an entity type of an entity indicated by the entity word in the query text comprises:
Obtaining initial text characteristics according to the text title of the target text;
extracting text sub-features corresponding to the entity words from the initial text features according to the positions of the entity words in the text titles, and obtaining first text features; and
and determining the entity type of the entity indicated by the entity word in the query text according to the first text feature.
4. The method of claim 3, wherein the deriving initial text features from the text title of the target text comprises:
splicing the text titles of the query text and the target text to obtain a first spliced text; and
and extracting text characteristics of the first spliced text to obtain the initial text characteristics.
5. The method of any of claims 2-4, wherein the target text comprises a plurality of texts; the determining the entity type of the entity indicated by the entity word in the query text according to the target text and the entity word comprises the following steps:
determining type information of an entity indicated by the entity word in the query text according to each text in the plurality of texts and the entity word, and obtaining a plurality of types of information corresponding to the plurality of texts; and
Determining the entity type of the entity indicated by the entity word in the query text according to a plurality of preset weights corresponding to the plurality of texts and the plurality of type information,
wherein the predetermined weight is associated with at least one of the following information: a degree of relatedness between each text and the query text; access data for each text.
6. The method of claim 1, wherein the determining, from the target text and entity information of the candidate entities, a target entity indicated by the entity word in the candidate entities comprises:
determining the similarity between the text title of the target text and the entity information of the candidate entity; and
and determining the entity with the highest similarity between the entity information in the candidate entity and the text title of the target text as the target entity.
7. The method of claim 6, wherein the determining the similarity between the text title of the target text and the entity information of the candidate entity comprises:
obtaining a second text feature according to the text title of the target text;
extracting text features of entity information of each entity in the candidate entities to obtain third text features; and
And determining the similarity between the text title of the target text and the entity information of each entity according to the second text characteristic and the third text characteristic.
8. The method of claim 7, wherein the deriving a second text feature from the text title of the target text comprises:
splicing the text titles of the query text and the target text to obtain a first spliced text; and
and extracting text features of the first spliced text to obtain the second text features.
9. The method according to claim 7 or 8, wherein the deriving a second text feature from the text title of the target text comprises:
obtaining complete text characteristics according to the text title of the target text; and
and extracting text sub-features corresponding to the entity words in the complete text features according to the positions of the entity words in the target text, and taking the text sub-features as the second text features.
10. The method of claim 6, wherein the determining the similarity between the text title of the target text and the entity information of the candidate entity comprises:
splicing the text title of the target text and the entity information of each entity in the candidate entities to obtain a second spliced text;
Extracting text features of the second spliced text to obtain fourth text features; and
and according to the fourth text characteristic, determining the similarity between the text title of the target text and the entity information of each entity.
11. The method of claim 10, wherein the extracting text features of the second stitched text to obtain fourth text features comprises:
and determining sub-features corresponding to the entity words in the extracted text features of the second spliced text as the fourth text features according to the positions of the entity words in the second spliced text.
12. The method of claim 6, wherein the target text comprises a plurality of texts; the determining the similarity between the text title of the target text and the entity information of the candidate entity comprises:
determining the similarity between the text title of each text in the plurality of texts and the entity information of the candidate entity to obtain a plurality of initial similarities corresponding to the plurality of texts; and
determining a similarity between a text title of the target text and entity information of the candidate entity based on a plurality of predetermined weights corresponding to the plurality of texts and the plurality of initial similarities,
Wherein the predetermined weight is associated with at least one of the following information: a degree of relatedness between each text and the query text; access data for each text.
13. The method of claim 1, wherein the determining, from the entity word, a candidate entity corresponding to the entity word and entity information of the candidate entity comprises:
determining an entity indicated by an entity identifier with a mapping relation with the entity word in a preset entity library as the candidate entity; and
determining entity information with a mapping relation with the entity identifier in the preset entity library, taking the entity information as entity information of the entity indicated by the entity identifier,
wherein the mapping relationship in the entity library is updated by listening for updates of entities in the upstream application; the updating of the entity in the upstream application comprises at least one of: an addition of an entity, a deletion of an entity, and a modification of entity information.
14. An entity disambiguation apparatus, comprising:
the entity recognition module is used for recognizing the query text and obtaining entity words to be disambiguated in the query text;
the target text determining module is used for determining target texts in texts obtained according to the query text query; the target text is an expanded text of the query text;
The candidate entity determining module is used for determining a candidate entity corresponding to the entity word and entity information of the candidate entity according to the entity word; wherein the candidate entities include at least two entities indicated by the same entity word and ambiguous to each other; and
the target entity determining module is used for determining a target entity indicated by the entity word in the candidate entity according to the target text and the entity information of the candidate entity;
the device also comprises a type determining module, a type determining module and a type determining module, wherein the type determining module is used for determining the entity type of the entity indicated by the entity word in the query text;
the candidate entity determining module is configured to: determining a candidate entity corresponding to the entity word and entity information of the candidate entity according to the entity word and the entity type,
and the entity type of the candidate entity is the same as the entity type of the entity indicated by the entity word.
15. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein, the liquid crystal display device comprises a liquid crystal display device,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 13.
16. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-13.
17. A computer program product comprising computer programs/instructions stored on at least one of a readable storage medium and an electronic device, which when executed by a processor, implement the steps of the method according to any one of claims 1 to 13.
CN202211092241.4A 2022-09-07 2022-09-07 Entity disambiguation method, device, electronic equipment and storage medium Active CN115828915B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211092241.4A CN115828915B (en) 2022-09-07 2022-09-07 Entity disambiguation method, device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211092241.4A CN115828915B (en) 2022-09-07 2022-09-07 Entity disambiguation method, device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN115828915A CN115828915A (en) 2023-03-21
CN115828915B true CN115828915B (en) 2023-08-22

Family

ID=85523446

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211092241.4A Active CN115828915B (en) 2022-09-07 2022-09-07 Entity disambiguation method, device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN115828915B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108280061A (en) * 2018-01-17 2018-07-13 北京百度网讯科技有限公司 Text handling method based on ambiguity entity word and device
CN110555208A (en) * 2018-06-04 2019-12-10 北京三快在线科技有限公司 ambiguity elimination method and device in information query and electronic equipment
CN114021570A (en) * 2021-11-05 2022-02-08 平安普惠企业管理有限公司 Entity disambiguation method, apparatus, device and storage medium
CN114625889A (en) * 2022-03-25 2022-06-14 达而观信息科技(上海)有限公司 Semantic disambiguation method and device, electronic equipment and storage medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108280061A (en) * 2018-01-17 2018-07-13 北京百度网讯科技有限公司 Text handling method based on ambiguity entity word and device
CN110555208A (en) * 2018-06-04 2019-12-10 北京三快在线科技有限公司 ambiguity elimination method and device in information query and electronic equipment
CN114021570A (en) * 2021-11-05 2022-02-08 平安普惠企业管理有限公司 Entity disambiguation method, apparatus, device and storage medium
CN114625889A (en) * 2022-03-25 2022-06-14 达而观信息科技(上海)有限公司 Semantic disambiguation method and device, electronic equipment and storage medium

Also Published As

Publication number Publication date
CN115828915A (en) 2023-03-21

Similar Documents

Publication Publication Date Title
TWI729472B (en) Method, device and server for determining feature words
CN107992585B (en) Universal label mining method, device, server and medium
US10796224B2 (en) Image processing engine component generation method, search method, terminal, and system
EP3851975A1 (en) Method and apparatus for generating text topics, and electronic device
EP3933657A1 (en) Conference minutes generation method and apparatus, electronic device, and computer-readable storage medium
US20220318275A1 (en) Search method, electronic device and storage medium
CN114861889B (en) Deep learning model training method, target object detection method and device
JP2022191412A (en) Method for training multi-target image-text matching model and image-text retrieval method and apparatus
KR20210091076A (en) Method and apparatus for processing video, electronic device, medium and computer program
CN111639228B (en) Video retrieval method, device, equipment and storage medium
CN113660541B (en) Method and device for generating abstract of news video
CN112948573B (en) Text label extraction method, device, equipment and computer storage medium
CN114116997A (en) Knowledge question answering method, knowledge question answering device, electronic equipment and storage medium
CN116955856A (en) Information display method, device, electronic equipment and storage medium
CN116597443A (en) Material tag processing method and device, electronic equipment and medium
CN116049370A (en) Information query method and training method and device of information generation model
CN114444514B (en) Semantic matching model training method, semantic matching method and related device
CN115828915B (en) Entity disambiguation method, device, electronic equipment and storage medium
CN112784600B (en) Information ordering method, device, electronic equipment and storage medium
CN115098729A (en) Video processing method, sample generation method, model training method and device
CN112989066A (en) Data processing method and device, electronic equipment and computer readable medium
CN112926297A (en) Method, apparatus, device and storage medium for processing information
CN113361280B (en) Model training method, prediction method, apparatus, electronic device and storage medium
CN115392389B (en) Cross-modal information matching and processing method and device, electronic equipment and storage medium
CN113377921B (en) Method, device, electronic equipment and medium for matching information

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant