CN113536742A - Method and device for generating description text based on knowledge graph and electronic equipment - Google Patents
Method and device for generating description text based on knowledge graph and electronic equipment Download PDFInfo
- Publication number
- CN113536742A CN113536742A CN202010310669.6A CN202010310669A CN113536742A CN 113536742 A CN113536742 A CN 113536742A CN 202010310669 A CN202010310669 A CN 202010310669A CN 113536742 A CN113536742 A CN 113536742A
- Authority
- CN
- China
- Prior art keywords
- graph
- knowledge
- embedded data
- points
- type
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/166—Editing, e.g. inserting or deleting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Animal Behavior & Ethology (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention provides a method, a device and electronic equipment for generating a description text based on a knowledge graph, wherein the method comprises the following steps: acquiring a knowledge graph; extracting graph structure relation characteristics of a plurality of different angles from the knowledge graph to generate first graph embedded data; and decoding the first graph embedded data based on natural language to generate a description text. The embodiment of the invention takes the knowledge graph as input data, extracts graph structure relation characteristics in the knowledge graph, and generates text description based on natural language through coding and decoding processing of a machine learning model. In the natural language conversion process, the encoding and decoding processing is driven based on the graph structure relationship in the knowledge graph, so that the content points in the knowledge graph can be fully captured, and natural language description capable of more fully representing the main content of the knowledge graph is generated.
Description
Technical Field
The application relates to a method and a device for generating a description text based on a knowledge graph and electronic equipment, and belongs to the technical field of computers.
Background
In real-life and commercial practical applications, it is often necessary to describe some structured data in natural language so that people can quickly know the meaning of the data representation. With the development of knowledge graph technology, a large amount of information is stored in a database in the form of a knowledge graph, and when some information content needs to be known, the data in the form of the knowledge graph needs to be converted into a description document of a natural language, so that a user can conveniently and quickly know the information content.
In the prior art, in the process of converting knowledge graph data into natural language, information in a knowledge graph is generally converted into an information sequence, and then a text generation model from sequence to sequence is used to generate a description text. Such a generation method may lose much information and may not obtain an ideal text description.
Disclosure of Invention
The embodiment of the invention provides a method and a device for generating a description text based on a knowledge graph and electronic equipment, which are used for generating richer text description contents based on the knowledge graph.
In order to achieve the above object, an embodiment of the present invention provides a method for generating a description text based on a knowledge graph, including:
acquiring a knowledge graph;
extracting graph structure relation characteristics of a plurality of different angles from the knowledge graph to generate first graph embedded data;
and decoding the first graph embedded data based on natural language to generate a description text.
The embodiment of the invention also provides a device for generating the description text based on the knowledge graph, which comprises the following steps:
the map data acquisition module is used for acquiring a knowledge map;
the encoder module is used for extracting graph structure relation characteristics of a plurality of different angles from the knowledge graph to generate first graph embedded data;
and the decoder module is used for carrying out decoding processing based on natural language on the first graph embedded data to generate a description text.
An embodiment of the present invention further provides an electronic device, including:
a memory for storing a program;
a processor for executing the program stored in the memory to perform the aforementioned method for generating a descriptive text based on a knowledge-graph.
The embodiment of the invention takes the knowledge graph as input data, extracts graph structure relation characteristics in the knowledge graph, and generates text description based on natural language through coding and decoding processing of a machine learning model. In the natural language conversion process, the encoding and decoding processing is driven based on the graph structure relationship in the knowledge graph, so that the content points in the knowledge graph can be fully captured, and natural language description capable of more fully representing the main content of the knowledge graph is generated.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
Drawings
FIG. 1 is a schematic diagram of an exemplary model framework architecture in accordance with an embodiment of the present invention;
FIG. 2 is a schematic diagram of a model structure of a multi-map convolutional neural network layer according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating graph structure relationship feature processing of a graph encoder according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating an example of knowledge-graph retrieval in accordance with embodiments of the present invention;
FIG. 5 is a flowchart illustrating a method for generating a descriptive text based on a knowledge-graph according to an embodiment of the present invention;
FIG. 6 is a schematic structural diagram of a device for generating a descriptive text based on a knowledge-graph according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
With the development of computer technology, a great deal of structured information and data are accumulated in various industries, wherein the structured data in the form of a knowledge graph is widely applied. A knowledge graph is essentially a semantic network, a graph-based data structure, consisting of points (points) and edges (edges). In a knowledge graph, each point represents an entity existing in the real world, and each edge is a relationship between the entities. Knowledge-graphs are the most efficient way to represent relationships. The knowledge graph can be a relational network obtained by connecting various kinds of Information (Heterogeneous Information), and the knowledge graph provides the capability of analyzing problems from the perspective of relations, so that the knowledge graph is also adopted by many search engines or big data analysis platforms.
Since the knowledge graph belongs to a structured data form, in practical application, most users often know information more conveniently and easily in a natural language mode, and therefore, the requirement of converting the knowledge graph into natural language description exists.
The techniques of embodiments of the present invention provide techniques for converting structured data, such as a knowledge graph, to descriptions in natural language. In the embodiment of the invention, the knowledge graph is used as input data, the graph structure relation characteristics in the knowledge graph are extracted, and then the text description based on natural language is generated through the coding and decoding processing of a machine learning model. In the natural language conversion process, the encoding and decoding processing is driven based on the graph structure relationship in the knowledge graph, so that the content points in the knowledge graph can be fully captured, and natural language description capable of more fully representing the main content of the knowledge graph is generated.
Fig. 1 is a schematic diagram of an exemplary model framework structure according to an embodiment of the present invention. In this example, the entire model adopts an encoding-decoding architecture (encoder-decoder), as shown in fig. 1, the left part is an encoder model, which is mainly used for performing feature extraction and encoding processing on an input knowledge graph, generating graph embedding data containing graph structure relationship features, and then outputting the graph embedding data to the right decoder model. The decoder model of the right part is mainly used for decoding processing according to the graph embedded data to generate and output the description text.
Encoder section
In the embodiment of the present invention, a model structure of a multi-graph convolutional neural network (MGCN) is adopted as an example encoder model, where a structure of each layer may be further as shown in fig. 2, and fig. 2 is a schematic diagram of the model structure of the multi-graph convolutional neural network layer according to the embodiment of the present invention, as shown in the figure, each multi-graph convolutional neural network layer includes a plurality of graph encoders (graph encoders), and graph embedding data after encoding processing by the plurality of graph encoders is aggregated by an aggregation layer (aggregation layer) and then is used as output graph embedding data. By way of example, in an embodiment of the present invention, the multi-map convolutional neural network layer employs 6 map encoders, the function of which corresponds to that shown in FIG. 3.
FIG. 3 is a diagram illustrating a graph structure relationship feature process of a graph encoder according to an embodiment of the present invention. The leftmost input map corresponds to a knowledge map, which includes four points E1, E2, E3, and E4 and three edges R1, R2, and R3 as shown in the figure, and is input to the map encoders 1 to 6 shown in fig. 2, respectively, and the processes corresponding to the respective map encoders as shown in fig. 3 are performed, respectively. In the embodiment of the invention, in the processes of feature extraction and encoding, edges in the knowledge graph are treated as points, so that structural relationship information in the knowledge graph is more reserved, and the number of model parameters can be reduced. For convenience of description, an original point in the knowledge graph, that is, a point corresponding to an entity, is referred to as a first type point, and an edge in the knowledge graph, that is, a relationship between entities, is referred to as a first type point. In fig. 3, points E1 through E4 correspond to a first type of point, points R1 through R3 are a second type of point, and point G corresponds to a global point. The respective picture encoders respectively perform the following processing:
the processing performed by the encoder 1 of fig. is: each first type point and each second type point of the knowledge graph are obtained, and a self-circulation edge is added to each first type point, each second type point and each global point, as shown in fig. 3 of an encoder 1, each point has an edge with a starting point and an end point as the self edge.
The processing performed by the encoder 2 of fig. is: and acquiring the forward association relation between each first type point and each second type point as the edge from the first type point to the second type point. As shown in the encoder 2 in fig. 3, based on the structural relationship of the knowledge-graph, positive edges between the points E1 to E4 and the points R1 to R3 are formed, and these edges represent the positive associations in the knowledge-graph between the points E1 to E4 and the points R1 to R3. For example, since the edge R1 starts from the point E1 in the knowledge graph, the encoder 2, which extracts the forward point-edge correlation, generates an edge starting from the point E1 and pointing to the point R1 in the generated graph-embedded data.
The processing performed by the encoder 3 of fig. is: and acquiring the reverse association relation between each first type point and each second type point as the edge from the second type point to the first type point. As shown in the encoder 3 in fig. 3, based on the structural relationship of the knowledge-graph, the reverse edges between the points E1 to E4 and the points R1 to R3 are formed, and these edges represent the reverse association relationship in the knowledge-graph between the points E1 to E4 and the points R1 to R3. For example, in the knowledge graph, since the edge R1 starts from the point E1, the encoder 3, which extracts the inverse point-edge correlation, generates an edge starting from the point R1 and pointing to the point E1 in the generated graph embedding data.
The processing performed by the fig. encoder 4 is: and acquiring the forward association relation between the first type points as a forward edge between the first type points. As shown in the encoder 4 in fig. 3, based on the structure relationship of the knowledge-graph, forward edges between the points E1 to E4 are formed, and these edges represent the forward association relationship in the knowledge-graph between the points E1 to E4. For example, in the knowledge graph, there are edges that point from the point E1 to the point E2 and the point E4, and therefore, as the encoder 4 that extracts the association between the forward point and the point, in the generated map embedded data thereof, one edge that points from the point E1 to the point E2 and the point E4 is generated.
The processing performed by the fig. encoder 5 is: and acquiring the reverse association relation among the first type points as reverse edges among the first type points. As shown in the encoder 5 in fig. 3, based on the structure relationship of the knowledge-graph, reverse edges between the points E1 to E4 are formed, and these edges represent the reverse association relationship in the knowledge-graph between the points E1 to E4. For example, in the knowledge graph, there are edges pointing to the point E2 and the point E4 from the point E1, and therefore, as the encoder 5 that extracts the association between the reverse point and the point, in the generated map embedded data thereof, an edge pointing to the point E1 from both the point E2 and the point E4 is generated.
The processing performed by the encoder 6 of fig. is: and acquiring the association relation between the global point and the first type point and the second type point as the edge between the global point and the first type point and the second type point. The encoder 6 is used to form an association relationship of the global point with all the first type points and the second type points, and therefore, in its generation map embedding data, edges pointing from the global point G to the points E1 to E4 and R1 to R3, respectively, are generated.
The multiple graph encoders extract and encode the graph structure relationship features of the knowledge graph from different angles, so that a simple and clear conversion process plays a great help role in learning in the multi-graph convolutional neural network, and the information content contained in the knowledge graph can be deeply and comprehensively mined.
The various multi-map convolutional neural network layers shown in fig. 2 may use the same model structure for the processing shown in fig. 3. Firstly, carrying out graph embedding coding processing on an input knowledge graph to generate initial graph embedding data, inputting the initial graph embedding data to a multi-graph convolution neural network of a first layer, and further processing the graph embedding data generated after the multi-graph convolution neural network of the first layer is processed in the multi-graph convolution neural network provided for a next layer until the multi-graph convolution neural network of an nth layer. As shown in fig. 2, the input graph-embedded data may be the initial graph-embedded data or the graph-embedded data from an upper layer, while the output graph-embedded data also serves as the input graph-embedded data of a lower layer. The graph embedding data output by each layer of multi-graph convolution neural network processing is provided to the decoder model shown in fig. 1 after being overlapped.
The graph feature coding is carried out through the model structure of the multilayer multi-graph convolution neural network, deeper abstract information can be captured, and the specific number n of layers can be determined according to the training result of the actual model so as to obtain a more appropriate text description result.
Decoder part
The decoder module in this embodiment may adopt a decoder model based on natural language processing, so that the graph embedding information output by the encoder model may be converted into natural language encoding for output. As an example, as shown in the right part of fig. 1, the decoder model section mainly includes a plurality of LSTM layers and a Cross-attention layer (Cross-attention layer). The output word embedding vector is input into the LSTM layer of the first layer, then the image embedding data after the processing of the multi-layer LSTM layer and the output superposition of the encoder model are input into the cross attention processing layer together for processing, finally the encoding of the description text is output, and finally the description text can be converted into the description text. The number m of the LSTM layers in the figure can be determined according to actual needs and model training effects. The decoder model can adopt a model which does not limit the length of the output text, so that the output text can be more flexible, and the key content information contained in the knowledge graph can be more accurately described.
The model adopts a text generation model of an encoding-decoding architecture (encoder-decoder), wherein a multi-graph convolutional neural network is adopted to extract and encode the characteristics of graph structure relationship characteristics of the knowledge graph, so that the problems of information loss and excessive parameters are effectively avoided, important information in the multi-knowledge graph is selectively captured and effectively aggregated, and the generated description text can more effectively describe the key information contained in the knowledge graph. The embodiment of the invention provides the model processing without requiring strict correspondence of input and output, so that richer description contents can be generated by assistance of a knowledge graph.
Knowledge graph acquisition and model application
In the embodiment of the present invention, the knowledge graph input as the model may be derived from a search result performed in a database of knowledge graphs. With the development of big data technology, a great deal of knowledge graph data is accumulated in various industries and fields, and the knowledge graph data can provide information inquiry based on key words for users. In some application scenarios, a user may carry query keywords through a query request, and based on the keywords, retrieve the query keywords from a knowledge graph database to obtain a related knowledge graph, input the retrieved knowledge graph into the model provided by the embodiment of the present invention, output a description text, and present the description text to the user.
In the process of acquiring the knowledge graph, the acquisition of the knowledge graph can be performed based on a main entity (main entity) and an affiliated subject entity (topic-related entity), and further, a text description of the topic word is generated by using the model provided by the embodiment of the invention. Specifically, the main entity refers to the most core execution of the information that is desired to be acquired, and the subordinate subject entity refers to some subject content or the like associated with the main entity. The main entity and the subject matter entity correspond to keywords in a specific search process, and are explained below by way of an example.
Fig. 4 is a schematic diagram of an example of knowledge-graph retrieval according to an embodiment of the present invention. As the keywords for the search, the user provides the name "AAAA" of a certain singer as the keyword corresponding to the main entity, and "antique wind (retro style)", "loc music (funk)", "rhythm blues music (rhythm and blue)", and the like. In the embodiment of the present invention, the retrieval process based on the main entity and the subordinate subject entity may follow the following strategies: a first triple directly associated with a primary entity and a second triple associating the primary entity with an affiliated subject entity are obtained. A knowledge graph may generally be recorded in the form of triples, with a more typical triplet form being entity-relationship-entity, with entities corresponding to points in the knowledge graph and relationships corresponding to edges in the knowledge graph. In the actual retrieval process, the main entity and the auxiliary subject entity are located in the knowledge map database, and then the corresponding triple is extracted according to the strategy, so that the knowledge map serving as the retrieval result is formed. The data stored in the knowledge-graph database may be considered as one or more larger knowledge graphs, and the knowledge graph as a retrieval result is only a part of the large knowledge graph in the database.
As shown in fig. 4, the acquired knowledge-graph includes a main entity located at the center and a plurality of subordinate entities 1 to 4 and other entities 5 to 10 other than the main entity and the subordinate entities. Wherein, the entities 2, 3, 5, 6, 7, 8, 9 are directly related entities with the main entity, and the entities 2 and 3 are also affiliated subject entities. The triples formed by these entities and the main entity are the first triples described above. In addition, the entities 1, 3, and 4 are subject entities, which are not directly associated with the main entity, and are associated with the main entity through the entities 9 and 10, the entity 8, and the entity 7, respectively, and the triples formed on these association paths are the second triples described above, and there are triples in the second triples that are partially overlapped with the first triples, for example, the triples formed between the entity 8, the entity 7, and the entity 9 and the main entity, and since we finally obtain a knowledge graph, there is no problem of duplication, and the division of the triples into the first triples and the second triples is merely for convenience of description.
Returning to the foregoing example of using the name "AAAA" of a singer as an affiliated subject entity related to a main entity, a knowledge graph such as that shown in fig. 4 is obtained by searching in a knowledge graph database, and then the knowledge graph can be input into the foregoing model shown in fig. 1 for processing, so that a textual description based on the main entity and the affiliated subject entity related to the singer can be generated, for example, the obtained textual description is that "the singer AAAA is born 10 months and 10 days 1985, and is an authored singer, a music producer, and a dancer. He is well known about his stage performance, he can perform a very broad range of music styles including recovering from the ancient style, crazy music, rhythmic blues music, R & B, hip pop, rock, etc.
The method and the machine learning model provided by the embodiment of the invention can be applied or deployed on a cloud platform providing data services, such as a search engine providing retrieval services, a power provider service platform and the like, and combine information retrieval with text description of natural language. Of course, the machine learning model described above may also be deployed on a local computer, mobile terminal, or server, providing a conversion process from a knowledge graph to a natural language description.
In the aspect of application scenes, the method can be applied to various aspects such as business, life, information acquisition and the like. Taking the e-commerce platform as an example, the e-commerce platform can search in a knowledge map database related to the product according to keywords provided by the user, and after acquiring the knowledge map, the knowledge map is converted into introduction description of the commodity in natural language and provided for the user. In the keywords provided by the user, the commodity name can be used as a main entity, other attribute keywords can be used as auxiliary subject entities, and the energized e-commerce knowledge base is used for retrieving and acquiring the knowledge graph. In addition, the method can be applied to data officer projects, and can search the knowledge graph according to the knowledge graph which is accumulated in a certain field and is related to the operation data, and generate text description of the operation state or the current sale state of the field.
The machine learning model can use the data in the existing knowledge map database and the existing corresponding text description as training data. For example, the knowledge-graph data accumulated in the knowledge-encyclopedia database and the corresponding textual introductions may be used as training data. In addition, for some specialized fields, such as the e-commerce industry, accumulated knowledge-graph data of various products sold by e-commerce and corresponding introductions on merchandise pages can also be used as training data.
The technical solution of the present invention is further illustrated by some specific examples.
Example one
As shown in fig. 5, it is a schematic flowchart of a method for generating a descriptive text based on a knowledge-graph according to an embodiment of the present invention, where the method includes:
s101: and acquiring a knowledge graph. The knowledge graph can be obtained from a knowledge graph database, specifically, after a user query request is obtained, the process of obtaining the knowledge graph can obtain a first keyword corresponding to a main entity and a second keyword corresponding to an auxiliary subject entity from the query request, and then, the first keyword and the second keyword are searched in the knowledge graph database to obtain the knowledge graph, wherein the knowledge graph comprises a first triple directly associated with the main entity and a second triple associated between the main entity and the auxiliary subject entity.
S102: and extracting graph structure relation characteristics of a plurality of different angles from the knowledge graph to generate first graph embedded data. The specific feature extraction method may be as follows: and taking the edges in the knowledge graph as points, extracting graph structure relationship features of the knowledge graph to generate second graph embedded data of a plurality of different angles, and then aggregating the second graph embedded data to generate first graph embedded data.
Specifically, the above-mentioned extraction of the graph structure relationship features from a plurality of different angles may include one or more combinations of the following aspect feature extraction and encoding processes, and the detailed example of this part may be according to the processing mechanism shown in fig. 3 above:
1) and acquiring each first type point and each second type point of the knowledge graph, and adding a self-circulation edge for each first type point, each second type point and the global point to generate second graph embedded data.
2) And acquiring the forward association relation between each first type point and each second type point, and generating second graph embedded data by taking the forward association relation as the edge from the first type point to the second type point.
3) And acquiring the reverse association relation between each first type point and each second type point, and generating second graph embedded data by taking the reverse association relation as the edge from each second type point to each first type point.
4) And acquiring the forward association relation between the first type points, and generating second graph embedded data by taking the forward association relation as a forward edge between the first type points.
5) And acquiring reverse association relations among the first type points, and generating second graph embedded data by taking the reverse association relations as reverse edges among the first type points.
6) And acquiring the association relation between the global point and the first type point and the second type point, and generating second graph embedded data by taking the association relation as the edge between the global point and the first type point and the second type point.
Specifically, the processing performed in this step may be implemented by using a multi-layer multi-map convolutional neural network, and accordingly, this step may specifically include:
carrying out graph embedding coding processing on the knowledge graph to generate initial graph embedding data, and executing the following layer-by-layer processing through a multilayer multi-graph convolution neural network:
taking edges in the knowledge graph as points, extracting graph structure relationship features of the knowledge graph, and generating second graph embedded data of a plurality of different angles;
aggregating the plurality of second graph embedded data to generate third graph embedded data, and repeatedly executing the processing as initial graph embedded data of the next layer of multi-graph convolutional neural network until the last layer of multi-graph convolutional neural network is reached;
and overlapping the third graph embedded data output by each layer of multi-graph convolution neural network to obtain the first graph embedded data.
The multi-map convolutional neural networks of each layer may include a plurality of map encoders shown in fig. 2, and each map encoder performs the encoding processes 1) to 6) described above. The model structure of the multi-layer multi-map convolutional neural network can be referred to fig. 1 and 2 described above.
S103: the first graph-embedded data is subjected to decoding processing based on natural language, and a description text is generated. The processing of the decoding part can be executed by adopting a long-time memory network and a cross attention model, and the graph embedded data is subjected to decoding processing based on natural language without limiting the length so as to generate the descriptive text.
The embodiment of the invention takes the knowledge graph as input data, extracts graph structure relation characteristics in the knowledge graph, and generates text description based on natural language through coding and decoding processing of a machine learning model. In the natural language conversion process, the encoding and decoding processing is driven based on the graph structure relationship in the knowledge graph, so that the content points in the knowledge graph can be fully captured, and natural language description capable of more fully representing the main content of the knowledge graph is generated.
Example two
As shown in fig. 6, it is a schematic structural diagram of a description text generation apparatus based on a knowledge-graph according to an embodiment of the present invention, the apparatus includes:
and the map data acquisition module 11 is used for acquiring the knowledge map. The knowledge graph can be obtained from a knowledge graph database, specifically, after a user query request is obtained, the process of obtaining the knowledge graph can obtain a first keyword corresponding to a main entity and a second keyword corresponding to an auxiliary subject entity from the query request, and then, the first keyword and the second keyword are searched in the knowledge graph database to obtain the knowledge graph, wherein the knowledge graph comprises a first triple directly associated with the main entity and a second triple associated between the main entity and the auxiliary subject entity.
And the encoder module 12 is used for extracting the graph structure relationship characteristics of a plurality of different angles from the knowledge graph to generate first graph embedded data. The specific feature extraction method may be as follows: and taking the edges in the knowledge graph as points, extracting graph structure relationship features of the knowledge graph to generate second graph embedded data of a plurality of different angles, and then aggregating the second graph embedded data to generate first graph embedded data.
Specifically, the above-mentioned extraction of the graph structure relationship features of a plurality of different angles may include one or more combinations of the feature extraction and encoding processes shown in the first introduction 1) to 6).
Further, the processing performed by the encoder module 12 may be implemented by using a multi-layer multi-map convolutional neural network, which may specifically include:
carrying out graph embedding coding processing on the knowledge graph to generate initial graph embedding data, and executing the following layer-by-layer processing through a multilayer multi-graph convolution neural network:
taking edges in the knowledge graph as points, extracting graph structure relationship features of the knowledge graph, and generating second graph embedded data of a plurality of different angles;
aggregating the plurality of second graph embedded data to generate third graph embedded data, and repeatedly executing the processing as initial graph embedded data of the next layer of multi-graph convolutional neural network until the last layer of multi-graph convolutional neural network is reached;
and overlapping the third graph embedded data output by each layer of multi-graph convolution neural network to obtain the first graph embedded data.
The multi-map convolutional neural networks of each layer may include a plurality of map encoders shown in fig. 2, and each map encoder performs the encoding processes 1) to 6) described above. The model structure of the multi-layer multi-map convolutional neural network can be referred to fig. 1 and 2 described above.
And a decoder module 13, configured to perform decoding processing based on natural language on the first graph-embedded data to generate a description text. The processing of the decoding part can be executed by adopting a long-time memory network and a cross attention model, and the graph embedded data is subjected to decoding processing based on natural language without limiting the length so as to generate the descriptive text.
The detailed description of the above processing procedure, the detailed description of the technical principle, and the detailed analysis of the technical effect are described in the foregoing embodiments, and are not repeated herein.
EXAMPLE III
The foregoing embodiment describes a flow process and an apparatus structure of a method for generating a description text based on a knowledge graph according to an embodiment of the present invention, and functions of the method and the apparatus may be implemented by an electronic device, as shown in fig. 7, which is a schematic structural diagram of the electronic device according to an embodiment of the present invention, and specifically includes: a memory 110 and a processor 120.
And a memory 110 for storing a program.
In addition to the programs described above, the memory 110 may also be configured to store other various data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phonebook data, messages, pictures, videos, and so forth.
The memory 110 may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.
A processor 120, coupled to the memory 110, for executing the program in the memory 110 to perform the operation steps of the knowledge-graph based description text generation method described in the foregoing embodiments.
Further, the processor 120 may also include various modules described in the foregoing embodiments to perform processes of description text generation based on a knowledge-graph, and the memory 110 may be used, for example, to store data required for the modules to perform operations and/or output data.
The detailed description of the above processing procedure, the detailed description of the technical principle, and the detailed analysis of the technical effect are described in the foregoing embodiments, and are not repeated herein.
Further, as shown, the electronic device may further include: communication components 130, power components 140, audio components 150, display 160, and other components. Only some of the components are schematically shown in the figure and it is not meant that the electronic device comprises only the components shown in the figure.
The communication component 130 is configured to facilitate wired or wireless communication between the electronic device and other devices. The electronic device may access a wireless network based on a communication standard, such as WiFi, a mobile communication network, such as 2G, 3G, 4G/LTE, 5G, or a combination thereof. In an exemplary embodiment, the communication component 130 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 130 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
The power supply component 140 provides power to the various components of the electronic device. The power components 140 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for an electronic device.
The audio component 150 is configured to output and/or input audio signals. For example, the audio component 150 includes a Microphone (MIC) configured to receive external audio signals when the electronic device is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal may further be stored in the memory 110 or transmitted via the communication component 130. In some embodiments, audio assembly 150 also includes a speaker for outputting audio signals.
The display 160 includes a screen, which may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.
Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The aforementioned program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (11)
1. A description text generation method based on a knowledge graph comprises the following steps:
acquiring a knowledge graph;
extracting graph structure relation characteristics of a plurality of different angles from the knowledge graph to generate first graph embedded data;
and decoding the first graph embedded data based on natural language to generate a description text.
2. The method of claim 1, wherein extracting graph structure relationship features from the knowledge-graph for a plurality of different angles, generating first graph-embedded data comprises:
taking the edges in the knowledge graph as points, and extracting graph structure relationship features of the knowledge graph to generate second graph embedded data of a plurality of different angles;
and performing aggregation processing on the plurality of second graph embedded data to generate the first graph embedded data.
3. The method of claim 1, wherein in the second graph embedding data, regarding edges in the knowledge-graph as points, regarding points corresponding to entities as first type points, regarding points corresponding to edges as first type points, and adding global points,
accordingly, extracting graph structure relationship features of a plurality of different angles from the knowledge-graph, generating first graph embedding data comprises:
acquiring each first type point and each second type point of the knowledge graph, and adding a self-circulation edge for each first type point, each second type point and each global point to generate second graph embedded data;
and/or the presence of a gas in the gas,
acquiring a forward association relation between each first type point and each second type point, and generating second graph embedded data by taking the forward association relation as an edge from the first type point to the second type point;
and/or the presence of a gas in the gas,
acquiring a reverse association relation between each first type point and each second type point, taking the reverse association relation as an edge from the second type point to the first type point, and generating second graph embedded data;
and/or the presence of a gas in the gas,
acquiring a forward incidence relation between the first type points, and generating second graph embedded data by taking the forward incidence relation as a forward edge between the first type points;
and/or the presence of a gas in the gas,
acquiring reverse association relations among the first type points, and generating second graph embedded data by taking the reverse association relations as reverse edges among the first type points;
and/or the presence of a gas in the gas,
and acquiring the incidence relation between the global point and the first type point and the second type point, and generating the second graph embedded data by taking the incidence relation as the edge between the global point and the first type point and the second type point.
4. The method of claim 1, wherein the knowledge-graph comprises a first triplet and a second triplet, the obtaining the knowledge-graph comprising:
acquiring a query request, wherein the query request comprises a first keyword corresponding to a main entity and a second keyword corresponding to an affiliated subject entity;
and retrieving in a knowledge-graph database according to the first keyword and the second keyword to obtain the knowledge graph, wherein the knowledge graph comprises a first triple directly associated with a main entity and a second triple associating the main entity with the affiliated subject entity.
5. The method of claim 1, wherein extracting graph structure relationship features from the knowledge-graph for a plurality of different angles, generating first graph-embedded data comprises:
carrying out graph embedding coding processing on the knowledge graph to generate initial graph embedding data, and executing the following layer-by-layer processing through a multilayer multi-graph convolution neural network:
taking the edges in the knowledge graph as points, and extracting graph structure relationship features of the knowledge graph to generate second graph embedded data of a plurality of different angles;
performing aggregation processing on the plurality of second graph embedded data to generate third graph embedded data, and repeatedly executing the processing as initial graph embedded data of a next layer of multi-graph convolutional neural network until the last layer of multi-graph convolutional neural network is reached;
and overlapping the third graph embedded data output by each layer of multi-graph convolutional neural network to obtain the first graph embedded data.
6. The method of claim 1, wherein the performing a natural language based decoding process on the first graph-embedded data to generate a description text comprises:
and performing decoding processing without limiting the length on the graph embedded data based on natural language by adopting a long-time memory network and a cross attention model to generate a description text.
7. A knowledge-graph-based descriptive text generation apparatus comprising:
the map data acquisition module is used for acquiring a knowledge map;
the encoder module is used for extracting graph structure relation characteristics of a plurality of different angles from the knowledge graph to generate first graph embedded data;
and the decoder module is used for carrying out decoding processing based on natural language on the first graph embedded data to generate a description text.
8. The apparatus of claim 7, wherein the knowledge-graph comprises a first triplet and a second triplet, the obtaining the knowledge-graph comprising:
acquiring a query request, wherein the query request comprises a first keyword corresponding to a main entity and a second keyword corresponding to an affiliated subject entity;
and retrieving in a knowledge-graph database according to the first keyword and the second keyword to obtain the knowledge graph, wherein the knowledge graph comprises a first triple directly associated with a main entity and a second triple associating the main entity with the affiliated subject entity.
9. The apparatus of claim 7, wherein in the second graph embedding data, edges in the knowledge graph are considered as points, points corresponding to entities are considered as first type points, points corresponding to edges are considered as first type points, and global points are added,
accordingly, extracting graph structure relationship features of a plurality of different angles from the knowledge-graph, generating first graph embedding data comprises:
acquiring each first type point and each second type point of the knowledge graph, and adding a self-circulation edge for each first type point, each second type point and each global point to generate second graph embedded data;
and/or the presence of a gas in the gas,
acquiring a forward association relation between each first type point and each second type point, and generating second graph embedded data by taking the forward association relation as an edge from the first type point to the second type point;
and/or the presence of a gas in the gas,
acquiring a reverse association relation between each first type point and each second type point, taking the reverse association relation as an edge from the second type point to the first type point, and generating second graph embedded data;
and/or the presence of a gas in the gas,
acquiring a forward incidence relation between the first type points, and generating second graph embedded data by taking the forward incidence relation as a forward edge between the first type points;
and/or the presence of a gas in the gas,
acquiring reverse association relations among the first type points, and generating second graph embedded data by taking the reverse association relations as reverse edges among the first type points;
and/or the presence of a gas in the gas,
and acquiring the incidence relation between the global point and the first type point and the second type point, and generating the second graph embedded data by taking the incidence relation as the edge between the global point and the first type point and the second type point.
10. The apparatus of claim 7, wherein extracting graph structure relationship features from the knowledge-graph for a plurality of different angles, generating first graph embedding data comprises:
carrying out graph embedding coding processing on the knowledge graph to generate initial graph embedding data, and executing the following layer-by-layer processing through a multilayer multi-graph convolution neural network:
taking the edges in the knowledge graph as points, and extracting graph structure relationship features of the knowledge graph to generate second graph embedded data of a plurality of different angles;
performing aggregation processing on the plurality of second graph embedded data to generate third graph embedded data, and repeatedly executing the processing as initial graph embedded data of a next layer of multi-graph convolutional neural network until the last layer of multi-graph convolutional neural network is reached;
and overlapping the third graph embedded data output by each layer of multi-graph convolutional neural network to obtain the first graph embedded data.
11. An electronic device, comprising:
a memory for storing a program;
a processor for executing the program stored in the memory to perform the method of generating a knowledge-graph based description text according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010310669.6A CN113536742A (en) | 2020-04-20 | 2020-04-20 | Method and device for generating description text based on knowledge graph and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010310669.6A CN113536742A (en) | 2020-04-20 | 2020-04-20 | Method and device for generating description text based on knowledge graph and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113536742A true CN113536742A (en) | 2021-10-22 |
Family
ID=78123552
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010310669.6A Pending CN113536742A (en) | 2020-04-20 | 2020-04-20 | Method and device for generating description text based on knowledge graph and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113536742A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117195913A (en) * | 2023-11-08 | 2023-12-08 | 腾讯科技（深圳）有限公司 | Text processing method, text processing device, electronic equipment, storage medium and program product |
-
2020
- 2020-04-20 CN CN202010310669.6A patent/CN113536742A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117195913A (en) * | 2023-11-08 | 2023-12-08 | 腾讯科技（深圳）有限公司 | Text processing method, text processing device, electronic equipment, storage medium and program product |
| CN117195913B (en) * | 2023-11-08 | 2024-02-27 | 腾讯科技（深圳）有限公司 | Text processing method, text processing device, electronic equipment, storage medium and program product |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220214775A1 (en) | Method for extracting salient dialog usage from live data | |
| EP3320490B1 (en) | Transfer learning techniques for disparate label sets | |
| US10521189B1 (en) | Voice assistant with user data context | |
| US9299342B2 (en) | User query history expansion for improving language model adaptation | |
| US9064006B2 (en) | Translating natural language utterances to keyword search queries | |
| US20100100371A1 (en) | Method, System, and Apparatus for Message Generation | |
| CN109565621B (en) | Method, system and computer storage medium for implementing video management | |
| CN111694965B (en) | Image scene retrieval system and method based on multi-mode knowledge graph | |
| WO2017151400A1 (en) | Interpreting and resolving conditional natural language queries | |
| CN101241537A (en) | Voice interface to nfc applications | |
| US20200342856A1 (en) | Multi-modal interface in a voice-activated network | |
| US11790953B2 (en) | Smart summarization, indexing, and post-processing for recorded document presentation | |
| CN115982376A (en) | Method and apparatus for training models based on text, multimodal data and knowledge | |
| CN111078849A (en) | Method and apparatus for outputting information | |
| CN110246494A (en) | Service request method, device and computer equipment based on speech recognition | |
| KR101651963B1 (en) | Method of generating time and space associated data, time and space associated data generation server performing the same and storage medium storing the same | |
| CN113536742A (en) | Method and device for generating description text based on knowledge graph and electronic equipment | |
| US11314793B2 (en) | Query processing | |
| CN116977701A (en) | Video classification model training method, video classification method and device | |
| CN108509059B (en) | Information processing method, electronic equipment and computer storage medium | |
| US20160232231A1 (en) | System and method for document and/or message document and/or message content suggestion, user rating and user reward | |
| CN115115984A (en) | Video data processing method, apparatus, program product, computer device, and medium | |
| CN111222011B (en) | Video vector determining method and device | |
| CN115018548A (en) | Advertisement case prediction method and device, equipment, medium and product thereof | |
| CN114925206A (en) | Artificial intelligence body, voice information recognition method, storage medium and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |