CN116843028A - Multi-mode knowledge graph construction method, system, storage medium and electronic equipment - Google Patents
Multi-mode knowledge graph construction method, system, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN116843028A CN116843028A CN202310817273.4A CN202310817273A CN116843028A CN 116843028 A CN116843028 A CN 116843028A CN 202310817273 A CN202310817273 A CN 202310817273A CN 116843028 A CN116843028 A CN 116843028A
- Authority
- CN
- China
- Prior art keywords
- data
- knowledge graph
- knowledge
- entity
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000010276 construction Methods 0.000 title claims abstract description 42
- 238000003860 storage Methods 0.000 title claims abstract description 21
- 238000000034 method Methods 0.000 claims abstract description 55
- 238000007781 pre-processing Methods 0.000 claims abstract description 24
- 238000012800 visualization Methods 0.000 claims abstract description 18
- 238000005065 mining Methods 0.000 claims abstract description 16
- 230000000007 visual effect Effects 0.000 claims abstract description 15
- 238000012545 processing Methods 0.000 claims description 33
- 238000004458 analytical method Methods 0.000 claims description 25
- 230000010354 integration Effects 0.000 claims description 10
- 238000004140 cleaning Methods 0.000 claims description 9
- 230000014509 gene expression Effects 0.000 claims description 8
- 238000010801 machine learning Methods 0.000 claims description 7
- 238000013135 deep learning Methods 0.000 claims description 5
- 230000002159 abnormal effect Effects 0.000 claims description 3
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 22
- 238000000605 extraction Methods 0.000 description 15
- 238000010586 diagram Methods 0.000 description 13
- 238000004590 computer program Methods 0.000 description 11
- 238000005516 engineering process Methods 0.000 description 11
- 239000000463 material Substances 0.000 description 9
- 238000011960 computer-aided design Methods 0.000 description 8
- 238000013461 design Methods 0.000 description 8
- 238000003058 natural language processing Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 7
- 239000011150 reinforced concrete Substances 0.000 description 6
- 238000010606 normalization Methods 0.000 description 5
- 238000013480 data collection Methods 0.000 description 4
- 238000006731 degradation reaction Methods 0.000 description 4
- 238000005457 optimization Methods 0.000 description 4
- 230000015556 catabolic process Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 230000008520 organization Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 239000004567 concrete Substances 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 238000007418 data mining Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 238000009472 formulation Methods 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 108090000623 proteins and genes Proteins 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 229910000831 Steel Inorganic materials 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 239000004566 building material Substances 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000010924 continuous production Methods 0.000 description 1
- 238000013075 data extraction Methods 0.000 description 1
- 238000013499 data model Methods 0.000 description 1
- 230000009849 deactivation Effects 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 238000001035 drying Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 102000004169 proteins and genes Human genes 0.000 description 1
- 238000007670 refining Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 239000010959 steel Substances 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/02—Knowledge representation; Symbolic representation
- G06N5/022—Knowledge engineering; Knowledge acquisition
- G06N5/025—Extracting rules from data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3344—Query execution using natural language analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/04—Inference or reasoning models
- G06N5/041—Abduction
Abstract
The invention discloses a multi-mode knowledge graph construction method, a system, a storage medium and electronic equipment, wherein the method comprises the following steps: preprocessing the collected engineering field original data into structured content collaboration data, and obtaining a knowledge graph dominant network through the entity, attribute and relationship among the entities of the content collaboration data; the knowledge graph dominant network is stored in a database, dominant relations and invisible relations among data are mined, and the knowledge graph in the engineering field is established through a visualization tool; deep mining and understanding of knowledge are realized, so that the intelligent degree and the application effect of the knowledge graph are improved; the understanding and the use of the knowledge graph are more visual, and the overall grasp and decision making of a user on engineering projects are facilitated; the system can be adjusted and optimized according to different application scenes and requirements, and customized knowledge management and service are provided.
Description
Technical Field
The invention relates to the technical field of knowledge graphs, in particular to a multi-mode knowledge graph construction method, a multi-mode knowledge graph construction system, a storage medium and electronic equipment.
Background
The multi-mode knowledge graph is an emerging knowledge representation method, integrates various types of information (such as text, images, video, audio and the like), expresses the entity, attribute and relation of knowledge in the form of graph, and realizes the depth fusion of different types of data and the unified representation of knowledge.
The multi-mode knowledge graph has important application value in the engineering field. Conventional engineering information typically exists in the form of drawings, tables, reports, etc., which are scattered across various documents and systems, making information retrieval, understanding, and utilization difficult. The multi-modal knowledge graph can integrate the scattered information together to form a structured knowledge representation, thereby greatly improving the availability and usability of the information.
However, the prior art mainly remains in the stage of processing structured data, and processing power for unstructured data such as text, images, video, etc. is weak. This means that knowledge contained in a large amount of unstructured data is ignored and the construction effect of the knowledge graph is limited.
In addition, the prior art often suffers from inefficiency in storage and querying when dealing with large-scale data. These problems have prevented the application of knowledge maps in large-scale, real-time engineering fields.
Based on the above problems, constructing a multi-modal knowledge graph in the engineering field that is comprehensive and can be used efficiently is a problem that needs to be solved.
Disclosure of Invention
Aiming at the problem that the engineering field lacks a multi-modal knowledge graph, the invention provides a multi-modal knowledge graph construction method, a multi-modal knowledge graph construction system, a storage medium and electronic equipment, wherein the constructed multi-modal knowledge graph can effectively process large-scale data, and meanwhile, the multi-modal knowledge graph has good expansibility and adaptability in consideration of the processing of cross-modal data, so that the blank of the prior art is filled.
The multi-mode knowledge graph construction method comprises the following steps:
preprocessing the collected engineering field original data into structured content collaboration data, and obtaining a knowledge graph dominant network through the entity, attribute and relationship among the entities of the content collaboration data;
and storing the knowledge graph dominant network in a database, mining dominant relations and invisible relations between data, and establishing a knowledge graph in the engineering field through a visualization tool.
Preferably, the pretreatment includes:
cleaning, namely identifying and processing noise, abnormal values and missing values in the data;
standardization, converting the data into a unified standard form; and
integration integrates data from different sources.
Preferably, the engineering field original data comprises structured data and unstructured data, the unstructured data is subjected to structured conversion through preprocessing, and all the structured data are used as content collaboration data;
wherein the unstructured data comprises: text files, images, video, audio, BIM models, and CAD models.
Preferably, the entity is obtained by analyzing the content collaboration data;
analyzing the entity to obtain the attribute of the entity; and
and analyzing the relation between the entities to obtain the relation between the entities.
Preferably, the entity, attribute and relationship among the entities of the content collaboration data perfects the knowledge graph through entity disambiguation, entity classification, knowledge base search question and answer and complex relationship reasoning, and stores the knowledge graph in an explicit network to obtain the knowledge graph explicit network.
Preferably, the explicit relation and the invisible relation between the data of the knowledge graph explicit network are mined through a graph algorithm, and the visual expression is performed on the mined structure through a visual tool, so that the establishment of the knowledge graph in the engineering field is completed.
Preferably, the method further comprises: and introducing priori knowledge, establishing a relation between the cross-modal data, and optimizing the knowledge graph in the engineering field.
A multimodal knowledge graph construction system comprising:
the data preprocessing module is used for executing data cleaning, standardization and integration on the collected original data in the engineering field;
an unstructured data processing module for structuring unstructured data;
the entity identification module is used for analyzing the content collaboration data and identifying and obtaining an entity;
the attribute and relationship analysis module is used for analyzing the entities to obtain the attributes of the entities and the relationship among the entities;
the knowledge graph construction module is used for constructing a knowledge graph dominant network according to the entity, the attribute and the relation information among the entities and storing the knowledge graph dominant network in a database;
the system comprises an entity disambiguation module, an entity classification module, a knowledge base searching question-answering module and a complex relation reasoning module, wherein the entity disambiguation module, the entity classification module, the knowledge base searching question-answering module and the complex relation reasoning module are used for optimizing and perfecting a knowledge map through a machine learning and deep learning method;
the graph algorithm module is used for carrying out graph algorithm analysis on the knowledge graph dominant network and mining dominant relations and recessive relations among data;
the visualization module is used for carrying out visual expression on the knowledge graph dominant network and the mining result to obtain a knowledge graph in the engineering field;
The priori knowledge introduction module is used for introducing priori knowledge and optimizing the knowledge graph;
and the cross-modal data processing module is used for processing the cross-modal data and establishing a relation between the cross-modal data.
A computer readable storage medium storing at least one instruction that when executed by a processor implements a multi-modal knowledge graph construction method.
An electronic device comprising a memory for storing at least one instruction and a processor for executing the at least one instruction to implement a multi-modal knowledge graph construction method.
Compared with the prior art, the invention has the advantages that:
(1) Comprehensively: the proposal not only processes the structured data, but also processes the unstructured data, covers the omnibearing processing and analysis of the original data in the engineering field, and ensures that the multi-mode data such as texts, images, audios, videos and the like can be effectively utilized;
(2) Systematic: the method organically connects the steps of data preprocessing, entity identification, attribute and relation analysis, knowledge graph construction, graph algorithm analysis and the like together in series to form a complete flow, and can finally obtain a visualized knowledge graph from original data;
(3) Intelligent: the scheme adopts advanced machine learning and deep learning technology, and further optimizes and perfects the knowledge graph by means of entity disambiguation, entity classification, knowledge base search question-answering, complex relation reasoning and the like;
(4) And (3) visualization: the complex knowledge graph is presented in an intuitive mode through the visualization tool, so that the user can understand and utilize the complex knowledge graph conveniently;
(5) Optimizing: the scheme introduces priori knowledge and cross-modal data processing, so that the knowledge graph can be better expressed and utilized, and the accuracy and usability of the knowledge graph are improved.
(6) Scalability: the design of the scheme considers large-scale data processing and query and cross-modal data processing, and has good expansibility and adaptability.
Drawings
FIG. 1 is a schematic flow chart of the present invention;
FIG. 2 is a schematic diagram of a pretreatment process according to the present invention;
FIG. 3 is a schematic diagram of a process flow of raw data in the engineering field according to the present invention;
FIG. 4 is a flowchart of acquiring entities, attributes and relationships between entities according to the present invention;
fig. 5 is a block diagram of the system of the present invention.
Detailed Description
Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is only exemplary and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the concepts of the present disclosure.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and/or the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be construed to have meanings consistent with the context of the present specification and should not be construed in an idealized or overly formal manner.
Where expressions like at least one of "A, B and C, etc. are used, the expressions should generally be interpreted in accordance with the meaning as commonly understood by those skilled in the art (e.g.," a system having at least one of A, B and C "shall include, but not be limited to, a system having a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.). Where a formulation similar to at least one of "A, B or C, etc." is used, in general such a formulation should be interpreted in accordance with the ordinary understanding of one skilled in the art (e.g. "a system with at least one of A, B or C" would include but not be limited to systems with a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).
Some of the block diagrams and/or flowchart illustrations are shown in the figures. It will be understood that some blocks of the block diagrams and/or flowchart illustrations, or combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the instructions, when executed by the processor, create means for implementing the functions/acts specified in the block diagrams and/or flowchart. The techniques of this disclosure may be implemented in hardware and/or software (including firmware, microcode, etc.). Additionally, the techniques of this disclosure may take the form of a computer program product on a computer-readable storage medium having instructions stored thereon, the computer program product being for use by or in connection with an instruction execution system.
As shown in fig. 1, the multi-modal knowledge graph construction method includes the following steps:
preprocessing the collected engineering field original data into structured content collaboration data, and obtaining a knowledge graph dominant network through the entity, attribute and relationship among the entities of the content collaboration data;
and storing the knowledge graph dominant network in a database, mining dominant relations and invisible relations between data, and establishing a knowledge graph in the engineering field through a visualization tool.
The method preprocesses the collected original data in the engineering field into structured content collaboration data: raw, unstructured engineering data is converted into structured content collaboration data by preprocessing technology (such as data cleaning, standardization, integration and the like), which is the basis for constructing a knowledge graph.
Obtaining a knowledge graph explicit network through the relationship among the entities, the attributes and the entities of the content collaboration data: the principle of the step is that the entity, the attribute and the relation among the entities are identified by analyzing the preprocessed data, so that an explicit network of the knowledge graph is constructed. This step is the core of building a knowledge graph.
Storing the knowledge-graph dominant network in a database: the principle of this step is mainly to store the constructed knowledge graph dominant network by utilizing the technology of a database management system so as to facilitate subsequent inquiry and analysis.
Digging dominant relations and invisible relations between data, and establishing an engineering field knowledge graph through a visualization tool: the principle of the step is mainly that dominant and recessive relations between data are mined through technologies such as a graph algorithm and the like, and then the relations are displayed through a visualization technology to form a knowledge graph.
Preprocessing original data: original unstructured engineering data are converted into content collaboration data with a certain structure, so that subsequent entity identification, attribute extraction and relationship analysis are facilitated.
Acquiring a knowledge graph dominant network: the content collaboration data is converted into a dominant network of the knowledge graph, namely scattered and complex data information is effectively integrated in a structured form.
And (3) storing a database: the method realizes the persistent storage of the knowledge graph and facilitates the subsequent data query and analysis.
Excavating relations and establishing a knowledge graph: dominant and recessive relations between data are mined, and the relations are displayed in a visual mode, so that knowledge graphs can be known to provide visual and convenient knowledge acquisition and understanding services for users.
According to the scheme, the original data is preprocessed and converted into the structured content collaboration data, the entity, the attribute and the relation are further identified, the explicit network of the knowledge graph is constructed, the explicit and implicit relation is stored and mined through technologies such as a graph algorithm, the knowledge graph is formed through a visualization technology, and the efficient acquisition and utilization of knowledge are achieved.
The primary goal in performing the method of the present solution is to collect multi-source heterogeneous data in the engineering field. Such data sources may be structured, such as databases or CSV files, or unstructured, such as text files, images, video and audio. The data in the engineering field may come from a variety of sources including, but not limited to, architectural design documents, CAD drawings, BIM models, engineering specifications and standards, equipment manuals, and the like. For non-text data, such as images, video and audio, it may come from monitoring systems, drone inspection, live recording, etc. In addition, data specific to the engineering field, such as building information models (BI M), computer Aided Design (CAD), etc., are also collected.
The purpose of collecting this data is to understand and grasp knowledge systems and information structures in the engineering field from multiple angles, multiple dimensions, and to lay a solid foundation for subsequent structuring, correlation, and analysis work. Notably, due to the data nature and diversity of the engineering arts, the process of data collection requires special attention to data quality, integrity, and updatability. At the same time, to address the processing and storage requirements of large-scale data, there is also a need for efficient data collection and storage tools and systems.
The process of data collection is not done once, but rather is a continuous iterative and updated process. New data can be continuously collected and updated to ensure the timeliness and the accuracy of the knowledge graph. The data sources are also continually optimized and adjusted during data collection to gradually improve the quality and coverage of the data.
As shown in fig. 2, preferably, the preprocessing includes:
cleaning, namely identifying and processing noise, abnormal values and missing values in the data;
the purpose of the cleaning task is to identify and process noise, outliers and missing values in the data. For example, missing values need to be filled in according to specific rules or policies, or outliers need to be identified and processed using anomaly detection algorithms;
standardization, converting the data into a unified standard form;
the purpose of the normalization task is to convert the data into a unified standard form. For example, it is necessary to unify dates of different formats to ISO 8601 format, or to convert all text data to lowercase. The purpose of the transformation task is then to transform the data into a form more suitable for analysis or modeling. For example, it is necessary to perform processing such as word segmentation, stem extraction, part-of-speech tagging, etc. on text data, or performing processing such as size adjustment, graying, normalization, etc. on image data; and
Integration, integrating data from different sources;
the purpose of the integration task is to integrate data from different sources together. For example, it may be necessary to fuse data from different databases or to merge data from different files together.
Preprocessing the collected multi-source heterogeneous data so as to prepare for the following steps of entity extraction, relationship establishment and the like. Preprocessing is an important step in data mining and analysis because raw data is generally not directly used for further analysis or modeling. The purpose of the preprocessing is to convert the raw data into a form more suitable for subsequent steps.
Preprocessing is an iterative and continuous process. With the continued depth of needs and understanding of knowledge maps in the engineering field, there is a need for methods and strategies to continuously adjust and optimize the pretreatment. At the same time, attention is also paid to privacy and security issues of the data.
As shown in fig. 3, preferably, the engineering field original data includes structured data and unstructured data, the unstructured data is subjected to structural conversion through preprocessing, and all the structured data is used as content collaboration data.
Unstructured data is more difficult to process and analyze than structured data because they are generally not compliant with predefined data models nor are they easy to machine read. Preprocessing is the process of converting unstructured data into structured data, which may include steps of data cleaning, normalization, feature extraction, and the like. For example, text data may be processed by natural language processing techniques (e.g., part-of-speech tagging, named entity recognition, etc.), picture data may be processed by computer vision techniques (e.g., object detection, image segmentation, etc.), and 3D model and point cloud data may be processed by three-dimensional data processing techniques. The preprocessed data is more suitable for subsequent analysis and processing.
After preprocessing, all data (including the original structured data and the converted unstructured data) will exist in structured form, which can be used as content collaboration data. Structured data is typically organized in a predefined pattern or model, which allows the data to be more efficiently queried and analyzed. At this stage, various types and sources of data will be integrated together, providing a rich, diverse source of data for subsequent knowledge-graph construction.
Through the process, information in the original data can be utilized to the maximum extent, valuable knowledge is extracted, and a foundation is provided for subsequent data collaboration and knowledge graph construction. Unstructured data preprocessing and structured data integration allow for better understanding and use of the data, enhancing the usability and operability of the data.
Wherein the unstructured data comprises: text files, images, video, audio, BIM models, and CAD models.
In the engineering field there is a large amount of unstructured data, such as text files, images, video and audio. The data in the engineering field may come from a variety of sources including, but not limited to, architectural design documents, CAD drawings, BI M models, engineering specifications and standards, equipment manuals, and the like. For non-text data, such as images, video and audio, it may come from monitoring systems, drone inspection, live recording, etc. In addition, data specific to the engineering field, such as Building Information Models (BIM), computer Aided Design (CAD), etc., are also collected.
In the scheme, an NLP technology (such as a BERT model) and other machine learning technologies suitable for processing multi-mode data are utilized, and basic entities, attributes and relations are extracted from the structured content collaboration data obtained through the steps. In the process, entity disambiguation, entity classification, knowledge base search question and answer and complex relation reasoning based on priori knowledge are also carried out, so that a richer and more accurate knowledge map dominant network in the engineering field is obtained.
After the data preprocessing is completed, basic entities, attributes and relationships need to be acquired in the data. This is a key step in building knowledge maps, requiring the extraction of key information from the original structured and unstructured data using deep learning and Natural Language Processing (NLP) techniques, such as BERT model algorithms.
As shown in fig. 4, the acquisition entity is preferably identified by analyzing the content collaboration data;
first, entity extraction is required, which includes analyzing all data content to identify major entities therein, such as person names, place names, organization names, professional terms, etc. These entities would constitute nodes of the knowledge graph;
analyzing the entity to obtain the attribute of the entity;
What is needed then is attribute extraction, i.e., obtaining the attributes of each entity. Such as a person's age, sex, occupation, etc., or an organization's size, address, date of establishment, etc. These attributes may help to better understand and categorize entities; and
the relation between the entities is obtained by analyzing the relation between the entities;
all that is required is that the relationship extraction be performed to identify the relationship between the entities. This may involve identifying explicit semantic relationships in the text, such as "XX is the author of YY", "ZZ is the subsidiary of AA", and so on.
In the knowledge graph of the engineering field, the "entity" may include various specific engineering elements, such as a building (e.g., a specific skyscraper, bridge, tunnel, etc.), engineering materials (e.g., concrete, steel, masonry, etc.), and engineering equipment (e.g., a crane, a concrete mixer truck, a pile driver, etc.). These entities all have a series of related "attributes," such as building may have attributes of altitude, design style, construction date, geographic location, etc.; engineering materials may have density, strength, durability, etc.; engineering equipment may have manufacturer, model, specification, etc. attributes.
And "relationships between entities" represents the relationship of these entities to each other. For example, a building is "used" with a certain engineering material, or a certain engineering device is "produced" at a certain manufacturer.
In this staged approach, these entities, attributes and relationships have been extracted from the multi-source heterogeneous data and a preliminary knowledge graph has been formed. However, this preliminary knowledge graph may have the following problems:
physical disambiguation problem: the same entity may have different representations in different data sources, for example, one building may be referred to as "wall street 30" in one data source and "new york securities exchange building" in another data source. How to confirm these two names refers to the same entity is a challenge.
Incomplete attributes and relationships: due to data source limitations, all entity attributes and relationships cannot be obtained. For example, design style and construction date for a certain building may not be in the collected data.
Resolution problem of unstructured data: for unstructured data, such as pictures, audio, video, etc., how to extract useful entities, attributes and relationships from it is also a big problem.
Next, these problems need to be solved to optimize the knowledge-graph.
Preferably, the entity, attribute and relationship among the entities of the content collaboration data perfects the knowledge graph through entity disambiguation, entity classification, knowledge base search question and answer and complex relationship reasoning, and stores the knowledge graph in an explicit network to obtain the knowledge graph explicit network.
Problems of entity disambiguation may be encountered during entity extraction, for example, the same name may refer to a plurality of different entities. In this case, disambiguation is required by context information, or with a priori knowledge.
Entity disambiguation: the same entity may have different representations in different data sources. Taking the building as an example, the "Huai street 30" and the "New York securities exchange building" may both refer to the same building. By entity disambiguation, it can be determined that the two names actually refer to the same entity, namely "new york securities exchange building".
After the entity, the attribute and the relationship among the entities are obtained, entity classification and knowledge base retrieval questions and answers are performed. The purpose of this step is to further refine the knowledge graph. By means of entity classification, similar entities can be classified into one type, and convenience is provided for subsequent analysis. By searching the question and answer through the knowledge base, the missing relation can be searched in the existing knowledge base or used for verifying the acquired relation.
Entity classification: grouping similar entities into a class facilitates better understanding and utilization of data. For example, all buildings can be divided into different categories of houses, office buildings, factories, etc.; engineering equipment may also be categorized by function, such as transportation equipment, construction equipment, and the like.
The knowledge base retrieves questions and answers: missing relationships may be found in existing knowledge bases or used to verify relationships that have been acquired. For example, for a previously undetermined construction date of the "new york securities exchange building," this information can be found in the knowledge base by retrieving questions and answers, thereby refining the attributes of this building entity.
And finally, carrying out complex relation reasoning based on priori knowledge, and perfecting a knowledge graph. By inference, deeper relationships can be found, such as deriving implicit relationships, or predicting relationships that may occur in the future.
Complicated relation reasoning: implicit relationships may be derived based on a priori knowledge and existing relationships, and even relationships that may occur in the future may be predicted. For example, if a particular engineering material is known to degrade structural performance at high temperatures, and a building is constructed using that material, it can be inferred that the building may have a safety hazard at high temperatures.
Through the steps, a more perfect knowledge graph can be obtained, wherein the knowledge graph contains rich entities, attributes and relations. Then, the knowledge graph is stored in an explicit network, and the knowledge graph explicit network is formed. This explicit network can clearly exhibit various relationships between entities, providing a basis for subsequent analysis and applications.
Storing the perfect knowledge graph in a proper database, selecting a database supporting large-scale graph data storage and query so as to facilitate subsequent graph query and analysis, providing basic data for subsequent steps, and obtaining the knowledge graph dominant network.
Preferably, the explicit relation and the invisible relation between the data of the knowledge graph explicit network are mined through a graph algorithm, and the visual expression is performed on the mined structure through a visual tool, so that the establishment of the knowledge graph in the engineering field is completed.
Dominant relationships generally refer to relationships that can be observed directly from data, such as where one paper references another, which is a dominant relationship. Whereas implicit relationships need to be discovered through some analysis and reasoning, for example, two papers have no direct reference, but they all refer to the third, then it can be inferred that there is some correlation between the two papers.
These dominant and recessive relationships will be represented by visualization tools, forming a knowledge graph in the engineering field. Visualization of knowledge maps may help not only to better understand the data, but may also help discover patterns, trends, and anomalies that may be present in the data.
The mining of explicit and implicit relationships is typically implemented using a graph algorithm. In engineering domain knowledge graphs, the relationships between data may be considered edges in the graph, and entities may be considered nodes in the graph. Dominant and recessive relationships between data can be mined through a graph algorithm.
Dominant relation mining: an explicit relationship is typically one that can be directly observed from the data, e.g., a manager of a certain construction project is a person, and this relationship can be directly derived from a personnel record. The mining of such relationships is generally relatively straightforward, primarily through data extraction and cleansing.
Recessive relation mining: implicit relationships are those relationships that are not easily observed directly from the data, but can be found through some analysis and reasoning. For example, if the managers of two construction projects are the same person, it may be inferred that the two projects may have some association in some way. Mining of implicit relationships typically requires the use of graph algorithms, such as PageRank, deepWalk, node Vec, etc., to analyze patterns and trends in the data.
After the explicit and implicit relationships are mined, the relationships may be graphically presented using visualization tools, such as Gephi, cytoscape, etc., so that the data may be more intuitively understood and analyzed. In the construction of knowledge maps in the engineering field, such visual presentation can help to better understand various aspects of engineering projects, such as personnel configuration, material use, equipment conditions, and the like.
Preferably, the method further comprises: and introducing priori knowledge, establishing a relation between the cross-modal data, and optimizing the knowledge graph in the engineering field.
After the engineering domain knowledge graph is established, the engineering domain knowledge graph is further extended, priori knowledge is introduced, and cross-modal data cross-reference is realized.
A priori knowledge is introduced: there is a great deal of recognized knowledge in the engineering field, including theory, theorem, rules, etc., known as a priori knowledge. The prior knowledge can effectively guide the establishment and reasoning process of the knowledge graph in the engineering field. This is a priori knowledge, for example, if it is known that a particular engineering material will suffer from performance degradation at high temperatures. This a priori knowledge can be encoded as relationships in the engineering domain knowledge graph, which will help better understand and interpret the data.
In engineering-domain knowledge graphs, a priori knowledge may be encoded in a variety of forms, often expressed as entities, attributes, or relationships.
Entity: the prior knowledge may be added as an entity to the engineering domain knowledge graph. For example, it is assumed that there is a knowledge about the building material, which may exist as an entity, such as "reinforced concrete".
Attributes: the a priori knowledge may also be used as an attribute of the entity. For example, it is known that reinforced concrete has reduced performance under high temperature conditions. This information may be used as an attribute of a "reinforced concrete" entity, such as "high_temperature_performance_degradation".
Relationship: in addition, the prior knowledge can also be expressed as a relationship between entities in the knowledge graph of the engineering field. For example, if it is known that "reinforced concrete" is "degraded" in a "high temperature" environment, this is a relationship. In the engineering domain knowledge graph, it may be expressed in the form of a triplet (object), such as ("reinforced concrete", "in the environment", "high temperature") and ("reinforced concrete", "performance", "degradation").
The process of encoding a priori knowledge into knowledge maps in the engineering field typically requires some expertise to ensure that such information is properly understood and represented. In addition, some natural language processing and information extraction techniques may be required to automatically extract and encode this a priori knowledge from the text.
Cross-modal data cross-reference: in the previous step, data of multiple modalities, including text, pictures, video, audio, etc., have been collected, which are structured and stored in the engineering domain knowledge graph. It is necessary to establish relationships between these cross-modal data so that they can be referenced to each other. For example, a paper may reference a piece of video as evidence, or an audio may reference a paper as its theoretical basis. By establishing these cross-modal reference relationships, the data may be more fully understood and utilized.
In the engineering field knowledge graph, the reference relation among different modal data can be realized by adding a specific type of edge (namely relation). These relationships may directly connect different types of entities, forming links between data. For example, a "reference" relationship may be established between a paper entity and its referenced video entity. The following are some specific steps:
and (3) data identification: first, a unique identification needs to be created for the data of the various modalities. This may be an ID or a URI that can be used to uniquely identify data in the engineering domain knowledge graph. For example, a paper may have a DOI as its unique identifier and a video may have a URL or other identifier.
Entity creation: and then, adding the data of each mode as an entity into the engineering field knowledge graph. The types of these entities may be related to the modality of the data, such as "text", "image", "video", etc. Each entity may contain some attributes such as its content, metadata, etc.
And (3) establishing a relation: finally, a relationship is established between the related entities. If one entity (e.g., a paper) references another entity (e.g., a video), a "referencing" relationship is added between them. In this way, a connection is created, indicating that the paper references video.
It is noted that this process may require the use of some natural language processing and information extraction techniques to automatically extract and identify the reference relationships from the data. In addition, normalization and disambiguation of the data is also an important step to ensure consistency and accuracy of the data.
The engineering field knowledge graph optimization aims are specifically as follows:
accuracy is improved: an important goal of optimizing knowledge maps in the engineering field is to improve the accuracy of the knowledge maps and ensure that entities, attributes and relationships in the maps are correct. This may involve using more accurate data sources, better entity recognition and disambiguation algorithms, and more accurate relation extraction methods.
Enlarging coverage: another optimization goal of engineering domain knowledge maps is to expand their coverage, including more entities and relationships. This may involve collecting data from more data sources and developing new methods to discover and extract entities and relationships.
Enhanced intelligibility: the knowledge graph in the engineering field needs to contain not only correct information, but also easy understanding and use. Therefore, one of the goals of optimizing knowledge maps in the engineering field is to enhance its understandability, enabling users to more easily understand and utilize the information in the maps. This may involve improving the structure of the map, introducing more intuitive entities and relationship types, and providing better visualization tools.
The robustness is improved: knowledge maps in the engineering field need to be able to handle uncertainties and errors, so improving their robustness is an important goal for optimization. This may involve developing more powerful error handling and anomaly detection mechanisms, as well as better redundancy and backup strategies.
Efficiency is improved: knowledge graph construction and query need to be efficient, so that one of the targets of optimization is to improve the efficiency. This may involve optimizing storage and indexing policies, and developing faster query algorithms.
Enhancing dynamics: as data sources continue to update and increase, knowledge maps also need to be able to dynamically update and expand. Therefore, one goal of optimizing knowledge maps is to enhance their dynamics, develop efficient incremental updates and online learning algorithms.
In practical applications, data of various modalities need to be processed. Such data may include, but is not limited to, the following:
3D model data: 3D model data is a very common type of data in the construction, design, manufacturing, etc. industries. Such data may visually represent information about the geometry, size, position, etc. of the object.
And (3) point cloud data: in the fields of remote sensing, geographic information, unmanned driving and the like, point cloud data is a common data type. Point cloud data is typically composed of a large number of 3D points, each of which may contain information of location, color, intensity, etc. Such data may be used to represent information such as the shape, texture, etc. of the object.
Sensor data: sensor data is also a very important data source in many engineering fields, such as aerospace, automotive, industrial manufacturing, etc. Such data may include measurements of various physical quantities such as temperature, pressure, humidity, velocity, acceleration, etc.
Time series data: time series data is a common data type in the fields of finance, electricity, traffic, climate, etc. This data represents a series of values over time that can be used to analyze and predict trends and patterns of time series.
Network data: network data is a common data type in the fields of social networks, communication networks, the internet, etc. Such data represents a set of entities and their relationships that can be used to analyze and mine the structure, community, propagation, etc. attributes of the network.
Bioinformatic data: bioinformatic data is an important data type in the fields of biology, medicine, etc. The data may contain information such as gene sequences, protein structures, medical images, etc., and can be used to study problems such as biological structures, functions, diseases, etc.
So far, the engineering domain knowledge graph not only contains a great amount of engineering domain knowledge, but also contains rich prior knowledge and cross-modal data reference relation.
As shown in fig. 5, the multi-modal knowledge graph construction system includes:
the data preprocessing module is used for executing data cleaning, standardization and integration on the collected original data in the engineering field;
The module is mainly responsible for processing the collected engineering field original data for the subsequent module to use. First, the data cleansing part removes invalid, erroneous, duplicate data and processes missing values and outliers. The normalization portion will then convert the data into a commonly used, easily handled format. For example, the numbers may need to be normalized, the dates may need to be converted to a uniform format, etc. Finally, the data integration portion will merge data from different sources together for subsequent analysis and processing.
An unstructured data processing module for structuring unstructured data;
this module is mainly responsible for converting unstructured data, such as text, pictures, audio, video, etc., into structured data. For text data, pre-processing steps such as vocabulary analysis, word drying, word deactivation and the like may need to be performed, and then the text is converted into numerical vectors by using methods such as a word bag model, TF-I DF, word embedding and the like. For unstructured data such as images, audio, video, etc., it may be necessary to convert it into a vector of values using methods such as feature extraction, feature selection, etc.
The entity identification module is used for analyzing the content collaboration data and identifying and obtaining an entity;
This module is mainly responsible for identifying entities from the content collaboration data. An entity may be a person, place, organization, event, object, or the like having a particular meaning. Entity recognition typically requires techniques using natural language processing and machine learning, such as sequence annotation models, deep learning models, and the like.
The attribute and relationship analysis module is used for analyzing the entities to obtain the attributes of the entities and the relationship among the entities;
the module is mainly responsible for analyzing and acquiring the attributes of the entities and the relationship among the entities from the content collaboration data. Attributes are descriptions of entities such as the age of a person, the latitude and longitude of a place, etc. A relationship is a line connecting two entities, representing some kind of connection between the two entities, such as a "living" relationship between a person and a place. Acquiring attributes and relationships typically requires techniques using natural language processing and machine learning such as information extraction, relationship extraction, and the like.
The knowledge graph construction module is used for constructing a knowledge graph dominant network according to the entity, the attribute and the relation information among the entities and storing the knowledge graph dominant network in a database;
the module is mainly responsible for constructing a knowledge graph explicit network according to the entity, the attribute and the relation information among the entities. The process of constructing a knowledge graph explicit network generally comprises the steps of entity linkage, relationship linkage, knowledge reasoning and the like. The constructed knowledge-graph is then stored in a database for subsequent query and analysis.
The system comprises an entity disambiguation module, an entity classification module, a knowledge base searching question-answering module and a complex relation reasoning module, wherein the entity disambiguation module, the entity classification module, the knowledge base searching question-answering module and the complex relation reasoning module are used for optimizing and perfecting a knowledge map through a machine learning and deep learning method;
the graph algorithm module is used for carrying out graph algorithm analysis on the knowledge graph dominant network and mining dominant relations and recessive relations among data;
the module is mainly responsible for carrying out graph algorithm analysis on the knowledge graph dominant network. Dominant and recessive relations between data can be mined by using a graph algorithm. An explicit relationship is a relationship directly observed in data, e.g., a relationship between a constructor and a designer of an engineering project. Whereas implicit relations are relations that can only be found by data mining and analysis, e.g. by analyzing data of a plurality of engineering projects, we may find that there is a relation between a certain type of material and engineering quality. The graph algorithm module may use various graph algorithms such as graph traversal, shortest path, community discovery, graph embedding, etc.
The visualization module is used for carrying out visual expression on the knowledge graph dominant network and the mining result to obtain a knowledge graph in the engineering field;
the module is mainly responsible for carrying out visual representation on the knowledge graph dominant network and the mining result. Visualization is a powerful tool that can help people understand and interpret complex data and models. For example, through a visualization module, we can present the knowledge graph explicit network as a graph, with nodes representing entities and edges representing relationships. Furthermore, we can use visual elements of various colors, sizes, shapes, etc. to represent different types of entities and relationships, as well as their importance.
The priori knowledge introduction module is used for introducing priori knowledge and optimizing the knowledge graph;
the module is mainly responsible for introducing priori knowledge and is used for optimizing the knowledge graph. The prior knowledge is the knowledge of theory, theorem, rules and the like which are accepted in the engineering field. By introducing a priori knowledge, we can improve the quality and accuracy of the knowledge graph. For example, if we know that a certain type of material will suffer from performance degradation at high temperatures, we can add this information as a relationship to the knowledge-graph.
And the cross-modal data processing module is used for processing the cross-modal data and establishing a relation between the cross-modal data.
The module is mainly responsible for processing the cross-modal data and establishing the relation between the cross-modal data. Data in the engineering field typically includes multiple modalities such as text, pictures, audio, video, 3D models, point cloud data, and the like. In order to make efficient use of this data, we need to establish a relationship between these data. For example, a design document may reference a 3D model, and we can build a link in the knowledge graph to connect the design document and the 3D model.
A computer readable storage medium storing at least one instruction that when executed by a processor implements a multi-modal knowledge graph construction method.
An electronic device comprising a memory for storing at least one instruction and a processor for executing the at least one instruction to implement a multi-modal knowledge graph construction method.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, etc., such as Read Only Memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises an element.
The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and variations of the present application will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. which come within the spirit and principles of the application are to be included in the scope of the claims of the present application.
Claims (10)
1. The multi-mode knowledge graph construction method is characterized by comprising the following steps of:
preprocessing the collected engineering field original data into structured content collaboration data, and obtaining a knowledge graph dominant network through the entity, attribute and relationship among the entities of the content collaboration data;
and storing the knowledge graph dominant network in a database, mining dominant relations and invisible relations between data, and establishing a knowledge graph in the engineering field through a visualization tool.
2. The multi-modal knowledge graph construction method according to claim 1, wherein the preprocessing includes:
cleaning, namely identifying and processing noise, abnormal values and missing values in the data;
standardization, converting the data into a unified standard form; and
integration integrates data from different sources.
3. The multi-modal knowledge graph construction method according to claim 1, wherein the engineering field original data includes structured data and unstructured data, the unstructured data is subjected to structured conversion by preprocessing, and all the structured data is used as content collaboration data;
Wherein the unstructured data comprises: text files, images, video, audio, BIM models, and CAD models.
4. The multi-modal knowledge graph construction method according to claim 1, wherein the entity is identified by analyzing the content collaboration data;
analyzing the entity to obtain the attribute of the entity; and
and analyzing the relation between the entities to obtain the relation between the entities.
5. The method for constructing a multi-modal knowledge graph according to claim 1, wherein the entities, attributes and relationships among the entities of the content collaboration data are used for perfecting the knowledge graph through entity disambiguation, entity classification, knowledge base search question-answering and complex relationship reasoning, and storing the knowledge graph in an explicit network to obtain the knowledge graph explicit network.
6. The multi-mode knowledge graph construction method according to claim 1, wherein the knowledge graph construction method is characterized in that the explicit relation and the invisible relation between the knowledge graph explicit network are mined through a graph algorithm, the visual expression is performed on the mined structure through a visual tool, and the construction of the knowledge graph in the engineering field is completed.
7. The multi-modal knowledge graph construction method of claim 1, further comprising: and introducing priori knowledge, establishing a relation between the cross-modal data, and optimizing the knowledge graph in the engineering field.
8. The multi-mode knowledge graph construction system is characterized by comprising:
the data preprocessing module is used for executing data cleaning, standardization and integration on the collected original data in the engineering field;
an unstructured data processing module for structuring unstructured data;
the entity identification module is used for analyzing the content collaboration data and identifying and obtaining an entity;
the attribute and relationship analysis module is used for analyzing the entities to obtain the attributes of the entities and the relationship among the entities;
the knowledge graph construction module is used for constructing a knowledge graph dominant network according to the entity, the attribute and the relation information among the entities and storing the knowledge graph dominant network in a database;
the system comprises an entity disambiguation module, an entity classification module, a knowledge base searching question-answering module and a complex relation reasoning module, wherein the entity disambiguation module, the entity classification module, the knowledge base searching question-answering module and the complex relation reasoning module are used for optimizing and perfecting a knowledge map through a machine learning and deep learning method;
the graph algorithm module is used for carrying out graph algorithm analysis on the knowledge graph dominant network and mining dominant relations and recessive relations among data;
the visualization module is used for carrying out visual expression on the knowledge graph dominant network and the mining result to obtain a knowledge graph in the engineering field;
The priori knowledge introduction module is used for introducing priori knowledge and optimizing the knowledge graph;
and the cross-modal data processing module is used for processing the cross-modal data and establishing a relation between the cross-modal data.
9. A computer readable storage medium storing at least one instruction that when executed by a processor implements the multi-modal knowledge graph construction method of any one of claims 1 to 7.
10. An electronic device, characterized in that the electronic device comprises a memory for storing at least one instruction and a processor for executing the at least one instruction to implement the multi-modal knowledge-graph construction method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310817273.4A CN116843028A (en) | 2023-07-05 | 2023-07-05 | Multi-mode knowledge graph construction method, system, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310817273.4A CN116843028A (en) | 2023-07-05 | 2023-07-05 | Multi-mode knowledge graph construction method, system, storage medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116843028A true CN116843028A (en) | 2023-10-03 |
Family
ID=88166663
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310817273.4A Pending CN116843028A (en) | 2023-07-05 | 2023-07-05 | Multi-mode knowledge graph construction method, system, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116843028A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117131938A (en) * | 2023-10-26 | 2023-11-28 | 合肥工业大学 | Dynamic implicit relation mining method and system based on graph deep learning |
| CN117271460A (en) * | 2023-11-22 | 2023-12-22 | 北京大学 | Scientific research digital networking service method and system based on scientific research digital object language relation |
| CN117575579A (en) * | 2024-01-17 | 2024-02-20 | 长江水利委员会长江科学院 | Hydraulic engineering perspective inspection method and related device based on BIM and knowledge graph |
-
2023
- 2023-07-05 CN CN202310817273.4A patent/CN116843028A/en active Pending
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117131938A (en) * | 2023-10-26 | 2023-11-28 | 合肥工业大学 | Dynamic implicit relation mining method and system based on graph deep learning |
| CN117131938B (en) * | 2023-10-26 | 2024-01-19 | 合肥工业大学 | Dynamic implicit relation mining method and system based on graph deep learning |
| CN117271460A (en) * | 2023-11-22 | 2023-12-22 | 北京大学 | Scientific research digital networking service method and system based on scientific research digital object language relation |
| CN117271460B (en) * | 2023-11-22 | 2024-02-20 | 北京大学 | Scientific research digital networking service method and system based on scientific research digital object language relation |
| CN117575579A (en) * | 2024-01-17 | 2024-02-20 | 长江水利委员会长江科学院 | Hydraulic engineering perspective inspection method and related device based on BIM and knowledge graph |
| CN117575579B (en) * | 2024-01-17 | 2024-04-09 | 长江水利委员会长江科学院 | Hydraulic engineering perspective inspection method and related device based on BIM and knowledge graph |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Karan et al. | BIM and GIS integration and interoperability based on semantic web technology | |
| Soibelman et al. | Management and analysis of unstructured construction data types | |
| Malik et al. | A methodology for real-time data sustainability in smart city: Towards inferencing and analytics for big-data | |
| CN116843028A (en) | Multi-mode knowledge graph construction method, system, storage medium and electronic equipment | |
| Hamilton et al. | Urban information model for city planning | |
| CN107679221B (en) | Time-space data acquisition and service combination scheme generation method for disaster reduction task | |
| CN113434623B (en) | Fusion method based on multi-source heterogeneous space planning data | |
| Karam et al. | A terminology service supporting semantic annotation, integration, discovery and analysis of interdisciplinary research data | |
| EP3732587B1 (en) | Systems and methods for context-independent database search paths | |
| CN106294520A (en) | The information extracted from document is used to carry out identified relationships | |
| CN113901231A (en) | Method, device and medium for evolution analysis of knowledge graph | |
| Sagi et al. | Artificial intelligence for ocean science data integration: current state, gaps, and way forward | |
| Sadeghineko et al. | Development of semantically rich 3D retrofit models | |
| Gopalakrishnan et al. | Big Data in building information modeling research: survey and exploratory text mining | |
| CN110737779A (en) | Knowledge graph construction method and device, storage medium and electronic equipment | |
| Saeed et al. | Automatic integration and querying of semantic rich heterogeneous data: Laying the foundations for semantic web of things | |
| CN113656647A (en) | Intelligent operation and maintenance oriented engineering archive data management platform, method and system | |
| Cheng et al. | Using digital humanity approaches to visualize and evaluate the cultural heritage ontology | |
| Sadeghineko et al. | A semantic web-based approach for generating parametric models using RDF | |
| Lee et al. | Heuristics for assessing Computational Archival Science (CAS) research: The case of the human face of big data project | |
| CN114880483A (en) | Metadata knowledge graph construction method, storage medium and system | |
| WO2022154906A1 (en) | Graph neural network for signal processing | |
| Mirarchi et al. | Impact of Industry 4.0 in Architecture and Cultural Heritage: Artificial Intelligence and Semantic Web Technologies to Empower Interoperability and Data Usage | |
| Portugal et al. | Towards a provenance-aware spatial-temporal architectural framework for massive data integration and analysis | |
| Yu et al. | A linked data approach for information integration between BIM and sensor data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |