CN116701663A - Method for constructing knowledge graph based on digital retina system - Google Patents

Method for constructing knowledge graph based on digital retina system Download PDF

Info

Publication number
CN116701663A
CN116701663A CN202310982255.1A CN202310982255A CN116701663A CN 116701663 A CN116701663 A CN 116701663A CN 202310982255 A CN202310982255 A CN 202310982255A CN 116701663 A CN116701663 A CN 116701663A
Authority
CN
China
Prior art keywords
data
entity
constructing
picture
map
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202310982255.1A
Other languages
Chinese (zh)
Other versions
CN116701663B (en
Inventor
王耀威
李潘
田永鸿
山其本
高文
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Peng Cheng Laboratory
Original Assignee
Peng Cheng Laboratory
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Peng Cheng Laboratory filed Critical Peng Cheng Laboratory
Priority to CN202310982255.1A priority Critical patent/CN116701663B/en
Publication of CN116701663A publication Critical patent/CN116701663A/en
Application granted granted Critical
Publication of CN116701663B publication Critical patent/CN116701663B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/36Creation of semantic tools, e.g. ontology or thesauri
    • G06F16/367Ontology
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33Querying
    • G06F16/3331Query processing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/34Browsing; Visualisation therefor
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/35Clustering; Classification
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • G06F16/953Querying, e.g. by the use of web search engines
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Animal Behavior & Ethology (AREA)
  • Processing Or Creating Images (AREA)

Abstract

The invention discloses a method for constructing a knowledge graph based on a digital retina system, which comprises the following steps: creating an entity and relationship map conforming to the end-edge cloud form based on the architecture of the digital retina; carrying out data classification treatment and storage on service data covered by a control stream, a model stream, a feature stream and a video stream of the digital retina system; constructing an entity and a relation by combining an actual application system, and carrying out entity extraction and knowledge fusion on the entity and the relation and the feature data of the digital retina to enrich a map model; constructing map mapping relations of the map database and other databases according to the data types and the storage modes, and importing the map mapping relations into the map database; text and atlas search engines are built based on the drawing data, the picture video data, and the drawing vector data, providing visual retrieval capabilities and API services. The invention can provide visual text and picture data searching and relation expanding service and provide capability support for the application system for information analysis, intelligent question and answer and auxiliary decision making.

Description

Method for constructing knowledge graph based on digital retina system
Technical Field
The invention relates to the technical field of knowledge maps, in particular to a method for constructing a knowledge map based on a digital retina system.
Background
Today, with the development of video security industry and public security demands, monitoring cameras have become more and more popular. The traditional camera only compresses shot video data and then uploads the compressed video data to the cloud for storage, and then analysis and identification processing are carried out. The digital retina requires high-quality video coding and visual feature extraction coding for video shooting at a camera end, locally stores the compressed and coded video stream and uploads the video stream to a cloud end as required, and all the compact feature streams are synchronously synchronized to the cloud end in real time, so that high-efficiency storage can be ensured, large data query analysis can be conveniently supported, and meanwhile, adaptive migration, compression, updating and conversion of a deep learning model facing intelligent video coding and feature analysis are supported between the end-side-cloud. In short, the digital retina is a scalable end Bian Yun collaborative visual computing architecture that contains a video encoding stream, a feature encoding stream, and a model update stream.
In recent years, knowledge patterns are used as a very popular technology in the field of artificial intelligence, and have achieved a lot of successful landing cases in a plurality of fields. In the retrieval of information, a knowledge graph-based search engine can structure heterogeneous knowledge in the field to construct a knowledge-to-knowledge association. The method mainly solves the problems that data are scattered in a plurality of systems in the field, the data are various and complex, the island is formed, and the single data have low value. It is clear that structured knowledge, natural, explicitly precipitates and correlates domain knowledge. A graph is constructed, and the characteristics of the original graph can be utilized to support mining and analysis of data.
In summary, knowledge maps provide a better ability to present, manage, and understand vast amounts of information by representing the information in a form that more closely approximates human cognition. However, in the traditional video security field, video and pictures are mostly analyzed, and a video monitoring system is rarely seen to be visually displayed by constructing a knowledge graph. Along with the combination of the landing of the digital retina system and the actual application scene, the problems of data multi-source heterogeneous phenomenon, data redundancy, end cloud edge coordination, complex data association degree, data resource sharing, multi-application-crossing data combination and the like are increased. Meanwhile, because the application programs of different systems are inconsistent with the underlying databases, the data of different functional systems are not communicated, and the requirements of big data analysis and potential relation exploitation cannot be supported. Therefore, research on the fusion of the knowledge graph implementation based on the big data technology and the digital retina system architecture is urgently needed, and clues for hiding the data behind are mined.
Disclosure of Invention
The invention aims to solve the technical problems that aiming at the defects in the prior art, a method for constructing a knowledge graph based on a digital retina system is provided, and aims to solve the problems that in the prior art, the digital retina is only applied to inquiring videos and pictures, big data integration analysis and application are not carried out on multi-source heterogeneous data, the requirements of big data analysis and relation mining cannot be supported, and a video monitoring system is visually displayed by constructing the knowledge graph.
In order to solve the technical problems, the technical scheme adopted by the invention is as follows:
in a first aspect, the present invention provides a method for constructing a knowledge graph based on a digital retinal system, wherein the method comprises:
analyzing the system architecture of a digital retina system, constructing a map model by combining the end-edge cloud form with business data extraction entities and relations covered by a control stream, a model stream, a feature stream and a video stream, and classifying, managing and storing the data of the control stream, the model stream, the feature stream and the video stream;
carrying out data management on service data of the application system according to people, places, objects, events and organizations to construct entities and relations, carrying out entity extraction and knowledge fusion processing on the service data and the feature data, and associating the feature data with the service data of the application system;
constructing a map mapping relation between the map model and a relational database, a picture video MINIO object library and an Mlivus vector library according to the data type and the storage mode;
based on the constructed map mapping relation, the data is imported into a map database in real time through the FlinkCDC technology configuration;
and constructing a text and picture search engine based on the picture data, the picture video data and the vector data, and quickly searching by the search engine to obtain visual entity-relation visual display data.
In one implementation, analyzing an end-edge cloud form of a digital retina system, and acquiring main service scenes and relations of a cloud side, an edge side and an end side to obtain end-edge cloud form data;
acquiring an entity and a relation according to the end-edge cloud form data, the control flow, the model flow, the characteristic flow and the video flow, and obtaining a knowledge graph based on a digital retina system; wherein the control stream, the model stream, the feature stream, and the video stream are primary data streams that intersect end-edge cloud aspects.
The construction of the map mapping relation between the map model and a relational database, a picture video MINIO object library and a Mlivus vector library according to the data type and the storage mode comprises the following steps:
in one implementation, mapping configuration is performed on the relational database of data storage and the map model;
associating the picture data of the feature stream with the picture entity of the map model through the MINIO object library, and associating the video data stored in the video stream with the video entity of the map model through the MINIO object library;
and associating the picture vectorization data of the feature stream with a picture entity of the map model through the Mlivus vector library.
In one implementation, after constructing the map mapping relationship between the map model and the relational database, the picture video MINIO object library and the Mlivus vector library according to the data type and the storage mode, the method includes:
acquiring a camera characteristic stream of a camera in the digital retina system, storing data of the entity and the relation identified in the camera characteristic stream in a relation database, and importing the data into a graph database;
and acquiring picture data related to the camera feature stream, and importing the picture data into a vector database, wherein the vector database carries out vectorization storage on the picture.
In one implementation, the method further comprises:
and monitoring the characteristic data in real time, and updating the information of the entity and the relation and the map mapping relation in time.
In one implementation, the method further comprises:
and storing the entity data and the characteristic data in the knowledge graph into an elastic search to establish a text search engine, and rapidly associating the entity data with the entity data of the graph database through the entity ID.
In one implementation, the method further comprises:
and establishing a retrieval service of the knowledge graph based on a standard Restful API service, and establishing a unified API interface for providing capability output.
In a second aspect, an embodiment of the present invention further provides an apparatus for constructing a knowledge graph based on a digital retinal system, where the apparatus includes:
the map construction module is used for analyzing the system architecture of the digital retina system, combining the end-to-side cloud form with the business data extraction entity and relation covered by the control flow, the model flow, the characteristic flow and the video flow to construct a map model, and classifying, managing and storing the data of the control flow, the model flow, the characteristic flow and the video flow;
the data association module is used for carrying out data management on the service data of the application system according to people, places, things, events and organizations to construct entities and relations, carrying out entity extraction and knowledge fusion processing on the service data and the feature data, and associating the feature data with the service data of the application system;
the data mapping module is used for constructing a map mapping relation between the map model and a relational database, a picture video MINIO object library and an Mlivus vector library according to the data type and the storage mode;
the data importing module is used for importing data into a graph database in real time based on the constructed map mapping relation and through the FlinkCDC technology configuration;
And the search engine module is used for constructing a search engine of texts and pictures based on the picture data, the picture video data and the vector data, and visual entity-relation visual display data is obtained through quick search of the search engine.
In a third aspect, an embodiment of the present invention further provides a terminal device, where the terminal device includes a memory, a processor, and a digital retinal system-based knowledge graph construction program stored in the memory and capable of running on the processor, and when the processor executes the digital retinal system-based knowledge graph construction program, the processor implements the step of the digital retinal system-based knowledge graph construction method according to any one of the above schemes.
In a fourth aspect, an embodiment of the present invention further provides a computer readable storage medium, where the computer readable storage medium stores a digital retinal system-based knowledge graph construction program, where the digital retinal system-based knowledge graph construction program, when executed by a processor, implements the steps of the digital retinal system-based knowledge graph construction method according to any one of the above schemes.
The beneficial effects are that: compared with the prior art, the invention can construct a knowledge graph based on a digital retina based on a business architecture of the digital retina system, then carries out one-to-one mapping association configuration on the entity and the relation of the knowledge graph, and fuses the end-edge cloud form and four stream data in the digital retina system into the knowledge graph to realize the function of searching through texts and pictures. On one hand, the invention provides a process for constructing a knowledge graph through digital retina business, so that multi-source heterogeneous data in a digital retina system can be fused into the knowledge graph, and on the other hand, clues hidden behind big data analysis and mining data are supported through text picture retrieval, graph algorithm and visualized data display. Therefore, the method for constructing the knowledge graph based on the digital retina system can enrich the digital retina system and support more application landings.
Drawings
Fig. 1 is a flowchart of a specific implementation of a method for constructing a knowledge graph based on a digital retinal system according to an embodiment of the present invention.
Fig. 2 is a functional schematic diagram of a device for constructing a knowledge graph based on a digital retina system according to an embodiment of the present invention.
Fig. 3 is a schematic block diagram of a terminal device according to an embodiment of the present invention.
Fig. 4 is a flowchart of a method for constructing a knowledge graph based on a digital retina system according to an embodiment of the present invention.
Fig. 5 is a conversion structure diagram based on a digital retinal system construction map according to an embodiment of the present invention.
Fig. 6 is a block diagram of a digital retinal system-based data map construction according to an embodiment of the present invention.
Fig. 7 is a diagram of a search engine for constructing a knowledge graph based on a digital retina system according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and effects of the present invention clearer and more specific, the present invention will be described in further detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
In a specific implementation, the embodiment first analyzes a system architecture of a digital retina system, constructs a map model by combining a terminal edge cloud form, a control flow, a model flow, a feature flow and a business data extraction entity and a relation covered by a video flow, and constructs the entity, the relation and the feature data by combining the business data of an application system. And then, constructing a map mapping relation between a map model and a relational database, a picture video MINIO object library and an Mlivus vector library according to the data types and the storage modes, and importing the data into the map database. And finally, constructing a text and picture search engine, obtaining visual entity-relation visual display data through quick retrieval, performing relation expansion and algorithm analysis on a query result, previewing pictures and videos, and providing API interface capability. On one hand, the invention provides a process for constructing a knowledge graph through digital retina business, so that multi-source heterogeneous data in a digital retina system can be fused into the knowledge graph, and on the other hand, clues hidden behind big data analysis and mining data are supported through text picture retrieval, graph algorithm and visualized data display. Therefore, the method for constructing the knowledge graph based on the digital retina system can enrich the digital retina system and support more application landings.
Exemplary method
The method for constructing the knowledge graph based on the digital retina system can be applied to terminal equipment, wherein the terminal equipment can be intelligent product terminals such as mobile phones, tablets and computers. In this embodiment, the terminal device may be an external device connected to a device for constructing a knowledge graph based on a digital retinal system, or may be a device built in the device for constructing a knowledge graph based on a digital retinal system. As shown in fig. 1, the method for constructing a knowledge graph based on a digital retinal system according to the present embodiment includes the steps of:
and S100, analyzing the architecture of the digital retina system, constructing a map model by combining the end-edge cloud form with the business data extraction entity and relation covered by the control flow, the model flow, the feature flow and the video flow, and classifying, managing and storing the data of the control flow, the model flow, the feature flow and the video flow.
In specific implementation, the embodiment is based on the digital retina to construct a knowledge graph for reasoning, so that the embodiment firstly needs to analyze the business architecture of the digital retina system to determine the relevance between the knowledge graph and the digital retina, and secondly, the embodiment can construct the knowledge graph based on the digital retina according to the relevance between the knowledge graph and the digital retina.
In one implementation manner, the method analyzes a service architecture of a digital retina system, creates an entity and a relationship, and when obtaining a knowledge graph based on a digital retina, includes the following steps:
step S101, according to the end-to-end morphology of the digital retina system, analyzing cloud-to-end and end-to-end service characteristics to refine the included entities, such as: cloud system, side system, end system, task, algorithm model bin, algorithm, camera, picture, intelligent box, etc., each entity is equivalent to a set of data tables. The cloud system entity is described as an entity of a cloud side system, and usually only one data value is needed, such as a traffic cloud system, but the cloud system entity and the end system entity can be one or more, and can be connected with the end system or directly associated with the end system without the existence of the end system to form an end cloud form. The following entity types can be obtained according to the service composition of the end edge cloud, but are not limited to: cloud system, side system, end system, task, control instruction, algorithm model bin, algorithm, intelligent box, camera, picture, video, equipment manufacturer, geographic position, characteristic data, etc.;
And step S102, analyzing the connection relation among entities according to the business relation of the cloud-to-edge, edge-to-end control flow, the model flow, the feature flow and the video flow.
Specifically, as shown in the conversion structure diagram based on the digital retina business architecture construction map provided in fig. 5, the control flow, the model flow, the feature flow and the video flow of the digital retina system are analyzed first. The control flow mainly comprises information of system operation layers such as equipment control, configuration parameters, function definition and the like, the information is sent to the side through the cloud side, and the side is sent to the end side, for example: the cloud system issues instructions for shutting down, restarting, adjusting the focal length and angle of the camera and the like of the terminal equipment to the side system, and the side system identifies a camera of the specific side system according to the instructions to issue the instructions. The model flow is an algorithm model data sequence transmitted according to a certain format and protocol, and mainly comprises an algorithm file, a model file, an algorithm model pulling update issuing instruction and the like. The model flow is mainly managed based on an algorithm model warehouse at the cloud side. The algorithm model warehouse aims at establishing an algorithm model management, scheduling and operation mechanism, issuing an algorithm model to the side end side through the cloud side, pulling the algorithm from the side end side to the cloud side, updating the algorithm from the cloud side to the side end side and the like. The video stream is a video data sequence transmitted according to a certain format and protocol, and mainly comprises an online real-time video stream and an offline video segment. The video stream is mainly associated with a camera at the side, the camera at the side shoots and transmits video data to the side for storage, and the cloud side calls the video from the side according to the service requirement. The feature stream is a feature data sequence transmitted according to a certain format and protocol after the operation of an algorithm task at the side or the end side, and mainly comprises structured feature text data information and unstructured picture and video information. For example, a camera at the end side of the digital retina system can acquire a picture and a video, then the picture and the video are identified through an algorithm task, and people, places, things, events, organizations and the like are analyzed to obtain feature result information related to the picture and the video. The feature result information is transmitted to an edge system through an end system, the edge system transmits the picture and the feature data to a cloud system, and the edge system stores video data; in addition, the feature data identified by the end side is mainly the result data obtained by an image identification algorithm, such as: the face recognition can recognize the face shot and recognize the information of the person, the license plate recognition algorithm can recognize the license plate number of the vehicle of the intercepted picture, and the like, and the recognition is that the data information is stored in a MYSQL service library or a CLICKHOUSE database, and in the embodiment, MYSQL is taken as a service data storage example.
Further, the embodiment determines the entity according to the feature data, where the entity includes: characteristic information is combed in digital retina systems such as people, places, things, events, tissues and the like. Specifically, the present embodiment needs to determine the relationships between the entities, and create the relationships between the entities, for example: and determining a corresponding relation among places, people and vehicles in the pictures acquired by the cameras in the same identification time as the relation among entities of the time. In this embodiment, the feature data is converted into entity nodes through the entity and the relationship, and then the entity nodes are associated with the identified video screenshot pictures to form edges, and the edges have the identification time attribute, so that a knowledge graph based on the digital retina is obtained.
And step 200, carrying out data management on service data of the application system according to people, places, things, events and organizations to construct entities and relations, carrying out entity extraction and knowledge fusion processing on the service data and the feature data, and associating the feature data with the service data of the application system.
In one implementation manner, the embodiment performs data management by combining service data of an application system, constructs a fusion association between a person, a ground, an object, an event and an organization entity and data of a digital retina map feature data row, and comprises the following steps:
Step S201, carrying out data classification treatment on the application system data according to people, places, objects, events and organizations. The application system refers to an actual landing project using a digital retina architecture, and can be intelligent security, digital city, intelligent traffic, intelligent manufacturing and the like. And carrying out operations such as data classification, data cleaning, data deduplication, data merging and the like on service data of the application system according to people, places, objects, events and organizations, and creating five types of data entities such as the people, the places, the objects, the events and the organizations according to service conditions to construct a knowledge graph of the data.
Step S202, the application system entity and the relation data are associated with the digital retina characteristic entity data. The relationship between the feature data and the people, the ground, the objects, the events and the organization entities in the application system is analyzed through knowledge extraction and knowledge fusion technology, and relationship connection is established, for example: the license plate number of the license plate feature data, namely Yue B12345, is the same as the name of the entity of the vehicle, so that the association relation is established.
And step S300, constructing a map mapping relation between the map model and a relational database, a picture video MINIO object library and an Mlivus vector library according to the data type and the storage mode. Specifically, as shown in fig. 6, a block diagram of a data map constructed based on a digital retinal system according to an embodiment of the present invention is shown;
In one implementation manner, the present embodiment constructs a map mapping relationship with a map database according to a storage manner of service data, control flow data, model flow data, feature flow data and video flow data, and includes the following steps:
step S301, structured data mapping. The service data, control flow data, structured data of model flow and structured data of characteristic flow of the digital retina system are stored in relational databases MYSQL and CLICKHOUSE. The data table which can be mapped one-to-one with the entity and relation of the map is obtained by carrying out the processes of data cleaning, data deduplication, data integration and the like on the structured data. And carrying out association mapping on tables and fields of the relational database and the entity relationship and attributes through mapping configuration of the entity relationship in the map. Such as: the camera information is stored in a camera table in the MYSQL database, a camera entity in the knowledge graph is associated with the camera table in the MYSQL database, and each attribute of the camera entity is set to be associated with field information of the camera table one to one;
step S302, mapping of picture and video data. The pictures and video data in the digital retina system are stored in a MINIO object library, wherein a cloud side MINIO object library stores information of all pictures and partial video information, and a side MINIO object library stores video information of cameras which are docked on the side. Cloud side and side MINIO object library pictures and video addresses are associated by "additional attributes" fields in the "pictures" and "video" entities of the atlas. Such as: when the side algorithm task runs and intercepts license plate pictures with the obtained feature data, synchronizing license plate picture data into a cloud side MINIO object database, and carrying out mapping association with picture URL addresses in the cloud side MINIO object database through the additional attribute of the picture of the knowledge graph;
Step S303, mapping the cloud side Mlivus vector data. And acquiring picture data related to the feature stream, and importing the picture data into a vector database, wherein the vector database carries out vectorization storage on the picture. The pictures of the feature streams need to be stored in a vectorization mode to meet the requirement of picture searching, so that the pictures identified by each feature stream need to be stored in a cloud side Mlivus vector database besides being stored in a MINIO object database. The cloud side Mlivus vector stores vectorization data of pictures, and the vector library ID of the pictures is associated with the picture ID of the MINIO object library and stored in the CLICKHOUSE database. The vector IDs of the pictures in the clikhouse database are associated by a "vector ID" attribute of the "picture" entity of the atlas.
In one implementation manner, the data importing method according to the embodiment includes the following steps:
step S304, importing the full data. After the map model and the data association mapping are established, full data import is carried out, and the import sequence is carried out according to entity mapping data, relation mapping data, picture video mapping data and vector mapping data in sequence.
On the one hand, in the embodiment, through the entities and the relations in the knowledge graph, data of the entities, the relations and the relational database are preferably imported into the graph database, wherein the feature stream data of the imported camera is the identified picture, video and related structured text data. Specifically, the feature data imported into the map database includes time information of the camera shooting the picture and geographical location information of the entity displayed on the picture, so that the map database of the embodiment can embody space-time data information, that is, the embodiment relates the entity of the digital retina to the relationship, the time information, the geographical location and other space-time data information to the knowledge graph for importing, thereby also embodying the space-time feature of the digital retina.
On the other hand, after the relational data with the built knowledge graph is imported into the graph database, the pictures and videos acquired by the camera cannot be directly stored in the graph database, so that in the embodiment, the pictures and videos are stored in the MINIO object library, and the pictures are matched and stored with the object id of the MINIO object library through the 'additional attribute' field of the picture entity. For example, when the image feature data in the digital retina system is imported, first, parameter information such as the name and the image size of the image is identified and obtained, for example, the obtained parameter information such as the name and the image size of the image is "automobile image license plate Guangdong B12345". Then according to the entity description of 'car', the entity type can be judged to be 'car'; according to the time information of the license plate, the automobile entity can be judged to be Yue B12345. Finally, the pictures are stored in a MINIO object library, and object storage addresses such as: http://192.168.X.x:9000/shuyi/00079d5e941b4ed095aadf6 and saving this address in the "additional attribute" field of the picture entity, thereby completing the association of the picture address with the picture entity;
Step S305, updating incremental data. After the map data is imported, the data is updated in an incremental updating mode according to the change condition of the service data, and the incremental data is imported through the FlinkCDC technology to ensure the instantaneity of the map data. The FlinkCDC is used for obtaining the adding and deleting operations on the data table by reading the log information of the relational database, and the data updating module is used for converting the adding and deleting operations on the data table into the graph database statement to update the graph database in real time, so that the performance of the service database is not affected, and the real-time updating of the graph database is ensured.
In one implementation manner, the embodiment is based on the constructed map mapping relationship and the data is imported into a map database in real time through the flankcdc technology configuration, a search engine for constructing texts and pictures based on the map data, the picture video data and the vector data, and visual entity-relationship visual display data is obtained through quick search by the search engine, and the method comprises the following steps:
step S401, text and picture search engine construction. The text and picture search engine obtains visual entity-relation visual display data through quick search, and the visual entity-relation search engine comprises the following steps:
Text searching is achieved by creating information such as names, IDs, descriptions and the like of map entities and relations into an elastomer search distributed search analysis engine, quickly locating specific data names and IDs through the retrieval capability of elastomer search, and querying a map database. The graph database accelerates the search of graph data by indexing entity names and relationship names.
Searching pictures by using an Mlivus vector database, filtering by setting a similarity value, comparing the returned 'vector ID' with a picture entity 'vector ID' in the picture database, and acquiring a picture address from a Minio database for display;
step S402, integrating and using a graph algorithm. And based on the graph algorithm analysis and display of the retrieved data, integrating a centrality algorithm, a path search and a community algorithm. The centrality algorithm mainly comprises the application and configuration of K-hop, pagerank, personalrank, a degree association algorithm, a tight centrality algorithm, K-core, centrality and other algorithms. The path search algorithm mainly comprises the application and configuration of the association path, the shortest path, the single-source shortest path and the full shortest path algorithm. The community algorithm mainly comprises the application and configuration of the algorithms such as triangle counting, label propagation, connected components, N-degree friends, louvain algorithm, common neighbors and the like. The association relation between the entities is better found and potential relations are mined by utilizing an integrated graph algorithm;
Step S403, visual interactive display. And providing a unified entity-relation for data display through knowledge graph construction, and providing a visual interactive service on the basis. The front-end system web page adopts visual tools such as VUE, echarts, D3.Js and the like, and searches entities through keywords in the map query page, performs relationship selection expansion, relationship hiding, picture preview, video preview, shortest path algorithm use and the like for interaction, and intuitively displays digital retina system data through a time axis and relationship topology, so that a set of knowledge maps with high portability and popularization and application to the digital retina system are constructed;
step S404, API interface capability. The retrieval capability based on the map query encapsulates the standard Restful API service, and provides the API interface capability to the outside. The RESTful API adopts HTTP as a transmission protocol REST style API, the front end calls the API to initiate an HTTP request to the background, and the background responds to the request to feed back the processing result to the front end. And combining the business functions of the knowledge graph to provide business capability interfaces such as entity name query, relationship name query, graph information query, entity statistics, relationship statistics, text retrieval, picture retrieval and the like.
In summary, when the method for constructing a knowledge graph based on a digital retina system is implemented, firstly, the architecture of the digital retina system is analyzed, a graph model is constructed by combining a terminal edge cloud form, a control flow, a model flow, a feature flow and a business data extraction entity and a relation covered by a video flow, and the relationship is associated with the feature data by combining the business data construction entity and the relation of an application system. And then, constructing a map mapping relation between a map model and a relational database, a picture video MINIO object library and an Mlivus vector library according to the data types and the storage modes, and importing the data into the map database. And finally, constructing a text and picture search engine, obtaining visual entity-relation visual display data through quick retrieval, performing relation expansion and algorithm analysis on a query result, previewing pictures and videos, and providing API interface capability. On one hand, the invention provides a process for constructing a knowledge graph through digital retina business, so that multi-source heterogeneous data in a digital retina system can be fused into the knowledge graph, and on the other hand, clues hidden behind big data analysis and mining data are supported through text picture retrieval, graph algorithm and visualized data display. Therefore, the method for constructing the knowledge graph based on the digital retina system can enrich the digital retina system and support more application landings.
Exemplary apparatus
Based on the above embodiment, the present invention further provides an apparatus for constructing a knowledge graph based on a digital retinal system, as shown in fig. 2, the apparatus for constructing a knowledge graph based on a digital retinal system includes: a map construction module 201, a data association module 202, a data mapping module 203, a data importing module 204 and a search engine module 205. Specifically, the map construction module 201 is configured to analyze an architecture of a digital retina system, combine a terminal edge cloud form with a business data extraction entity and a relationship covered by a control flow, a model flow, a feature flow and a video flow to construct a map model, and classify, manage and store data of the control flow, the model flow, the feature flow and the video flow. The data association module 202 is configured to administer data of the service data of the application system according to people, places, things, events and organizations, so as to construct entities and relationships, and perform entity extraction and knowledge fusion processing with the feature data, so as to associate the feature data with the service data of the application system. The data mapping module 203 is configured to construct a mapping relationship between the mapping model and a relational database, a picture video MINIO object library, and an Mlivus vector library according to the data type and the storage mode. The data importing module 204 is configured to import data into a graph database in real time based on the constructed map mapping relation and through the flankcdc technology configuration. The search engine module 205 is configured to construct a search engine for text and pictures based on the graph data, the picture video data and the vector data, and the visual presentation data of the intuitive entity-relationship is quickly retrieved by the search engine.
In one implementation, the map construction module 201 includes:
the knowledge map entity construction unit is used for analyzing the end-edge cloud form of the digital retina system and acquiring service entities contained in the end-edge cloud form;
in one implementation, the data association module 202 includes:
and the knowledge graph relation construction unit is used for determining the connection relation between the entities according to the control flow, the model flow, the feature flow and the video flow data relation.
And the knowledge graph attribute construction unit is used for constructing attribute information of the entity according to the entity of the service data, and the attribute is associated with the field of the table of the structured data.
In one implementation, the data mapping module 203 includes:
the atlas and cloud side Mlivus vector data association unit is used for importing pictures related to feature streams in the digital retina system into a vector database to be associated with the 'vector ID' attribute of a 'picture' entity of the atlas.
The map and structured data mapping unit is used for mapping the business data, the control flow data, the structured data of the model flow and the structured data of the characteristic flow of the digital retina system with the entity and the relation of the map in a one-to-one mode.
And the map and picture and video data mapping unit is used for storing pictures and video data in the digital retina system in a MINIO object library, wherein the cloud side MINIO object library stores information of all pictures and partial video information, and the side MINIO object library stores video information of cameras which are butted on the side.
In one implementation, the data import module 204 includes:
and the full data importing unit is used for importing the full data after the map model and data association mapping is established, and the importing sequence is sequentially carried out according to the entity mapping data, the relation mapping data, the picture video mapping data and the vector mapping data.
And the incremental data updating unit is used for updating the data by adopting an incremental updating mode according to the service data change condition after the full data is imported into the map, and importing the incremental data through the FlinkCDC technology to ensure the instantaneity of the map data.
In one implementation, the search engine module 205 includes:
the centrality algorithm application unit is used for configuring related parameters to perform data filtering by using K-hop, pagerank, personalrank, a degree association algorithm, a tight centrality algorithm, K-core, centrality and other algorithms when entity relation query is performed;
And the path search algorithm application unit is used for configuring the application and configuration of related parameters by using the associated path, the shortest path, the single-source shortest path and the full shortest path algorithm when the entity relation query is carried out.
When the community algorithm is used for inquiring entity relations, relevant parameters are configured to carry out data filtering by utilizing algorithms such as triangle count, label propagation, connected components, N-degree friends, louvain algorithm, common neighbors and the like.
And the elastic search engine construction unit is used for storing the entity and relation related information in the graph database to the elastic search to realize quick text data retrieval.
The visual construction unit is used for carrying out entity and relation data display on the front-end WEB page and can carry out interactive operations such as relation selection expansion, relation hiding, picture preview, video preview, shortest path algorithm use and the like.
And the picture searching and constructing unit is used for carrying out similarity setting and comparison on the pictures, and rapidly searching and displaying the pictures.
The working principle of each module in the device for constructing a knowledge graph based on the digital retina system in this embodiment is the same as that of each step in the above method embodiment, and will not be described here again.
Based on the above embodiment, the present invention also provides a terminal device, and a schematic block diagram of the terminal device may be shown in fig. 3. The terminal device may include one or more processors 100 (only one shown in fig. 3), a memory 101, and a computer program 102 stored in the memory 101 and executable on the one or more processors 100, for example, a program for constructing a knowledge-graph based on a digital retinal system. The execution of the computer program 102 by the one or more processors 100 may implement various steps in an embodiment of a method for building a knowledge-graph based on a digital retinal system. Alternatively, the one or more processors 100, when executing the computer program 102, may implement the functions of the modules/units in the embodiment of the apparatus for building a knowledge-graph based on a digital retinal system, which is not limited herein.
In one embodiment, the processor 100 may be a central processing unit (Central Processing Unit, CPU), but may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
In one embodiment, the memory 101 may be an internal storage unit of the electronic device, such as a hard disk or a memory of the electronic device. The memory 101 may also be an external storage device of the electronic device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash card (flash card) or the like, which are provided on the electronic device. Further, the memory 101 may also include both an internal storage unit and an external storage device of the electronic device. The memory 101 is used to store computer programs and other programs and data required by the terminal device. The memory 101 may also be used to temporarily store data that has been output or is to be output.
It will be appreciated by persons skilled in the art that the functional block diagram shown in fig. 3 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the terminal device to which the present inventive arrangements are applied, and that a particular terminal device may include more or fewer components than shown, or may combine some of the components, or may have a different arrangement of components.
Those skilled in the art will appreciate that implementing all or part of the above-described methods may be accomplished by way of a computer program, which may be stored on a non-transitory computer readable storage medium, that when executed may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, operational database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual operation data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
In summary, the invention discloses a method, a device and terminal equipment for constructing a knowledge graph based on a digital retina system, wherein the method comprises the following steps: firstly, analyzing the architecture of a digital retina system, constructing a map model by combining the end-edge cloud form with a business data extraction entity and a relation covered by a control flow, a model flow, a feature flow and a video flow, and associating the entity and the relation with the feature data by combining the business data construction entity and the relation of an application system. And then, constructing a map mapping relation between a map model and a relational database, a picture video MINIO object library and an Mlivus vector library according to the data types and the storage modes, and importing the data into the map database. And finally, constructing a text and picture search engine, obtaining visual entity-relation visual display data through quick retrieval, performing relation expansion and algorithm analysis on a query result, previewing pictures and videos, and providing API interface capability. The invention provides a process for constructing a knowledge graph through digital retina business, so that multi-source heterogeneous data in a digital retina system can be fused into the knowledge graph, and clues hidden behind big data analysis and mining data are supported through text picture retrieval, a graph algorithm and visualized data display. Therefore, the method for constructing the knowledge graph based on the digital retina system can enrich the digital retina system and support more application landings.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. A method for constructing a knowledge graph based on a digital retinal system, the method comprising:
analyzing the system architecture of a digital retina system, constructing a map model by combining the end-edge cloud form with business data extraction entities and relations covered by a control stream, a model stream, a feature stream and a video stream, and classifying, managing and storing the data of the control stream, the model stream, the feature stream and the video stream;
carrying out data management on service data of the application system according to people, places, objects, events and organizations to construct entities and relations, carrying out entity extraction and knowledge fusion processing on the service data and the feature data, and associating the feature data with the service data of the application system;
Constructing a map mapping relation between the map model and a relational database, a picture video MINIO object library and an Mlivus vector library according to the data type and the storage mode;
based on the constructed map mapping relation, the data is imported into a map database in real time through the FlinkCDC technology configuration;
and constructing a text and picture search engine based on the picture data, the picture video data and the vector data, and quickly searching by the search engine to obtain visual entity-relation visual display data.
2. The method of constructing a knowledge-graph based on a digital retinal system of claim 1, further comprising:
analyzing the end-edge cloud form of the digital retina system, and acquiring main service scenes and relations of cloud sides, edge sides and end sides to obtain end-edge cloud form data;
acquiring an entity and a relation according to the end-edge cloud form data, the control flow, the model flow, the characteristic flow and the video flow, and obtaining a knowledge graph based on a digital retina system; wherein the control stream, the model stream, the feature stream, and the video stream are primary data streams that intersect end-edge cloud aspects.
3. The method for constructing a knowledge graph based on a digital retina system according to claim 1, wherein constructing a graph mapping relationship between the graph model and a relational database, a picture video MINIO object library, and an Mlivus vector library according to a data type and a storage mode comprises:
Mapping configuration is carried out on the relational database stored in data and the map model;
associating the picture data of the feature stream with the picture entity of the map model through the MINIO object library, and associating the video data stored in the video stream with the video entity of the map model through the MINIO object library;
and associating the picture vectorization data of the feature stream with a picture entity of the map model through the Mlivus vector library.
4. The method for constructing a knowledge graph based on a digital retina system according to claim 1, wherein after constructing a graph mapping relation between the graph model and a relational database, a photo video MINIO object library, and an Mlivus vector library according to data types and storage modes, the method comprises:
acquiring a camera characteristic stream of a camera in the digital retina system, storing data of the entity and the relation identified in the camera characteristic stream in a relation database, and importing the data into a graph database;
and acquiring picture data related to the camera feature stream, and importing the picture data into a vector database, wherein the vector database carries out vectorization storage on the picture.
5. The method of constructing a knowledge-graph based on a digital retinal system of claim 1, further comprising:
and monitoring the characteristic data in real time, and updating the information of the entity and the relation and the map mapping relation in time.
6. The method of constructing a knowledge-graph based on a digital retinal system of claim 2, further comprising:
and storing the entity data and the characteristic data in the knowledge graph into an elastic search to establish a text search engine, and rapidly associating the entity data with the entity data of the graph database through the entity ID.
7. The method of constructing a knowledge-graph based on a digital retinal system of claim 2, further comprising:
and establishing a retrieval service of the knowledge graph based on a standard Restful API service, and establishing a unified API interface for providing capability output.
8. An apparatus for constructing a knowledge-graph based on a digital retina, the apparatus comprising:
the map construction module is used for analyzing the system architecture of the digital retina system, combining the end-to-side cloud form with the business data extraction entity and relation covered by the control flow, the model flow, the characteristic flow and the video flow to construct a map model, and classifying, managing and storing the data of the control flow, the model flow, the characteristic flow and the video flow;
The data association module is used for carrying out data management on the service data of the application system according to people, places, things, events and organizations to construct entities and relations, carrying out entity extraction and knowledge fusion processing on the service data and the feature data, and associating the feature data with the service data of the application system;
the data mapping module is used for constructing a map mapping relation between the map model and a relational database, a picture video MINIO object library and an Mlivus vector library according to the data type and the storage mode;
the data importing module is used for importing data into a graph database in real time based on the constructed map mapping relation and through the FlinkCDC technology configuration;
and the search engine module is used for constructing a search engine of texts and pictures based on the picture data, the picture video data and the vector data, and visual entity-relation visual display data is obtained through quick search of the search engine.
9. A terminal device, characterized in that it comprises a memory, a processor and a digital retina system based knowledge graph construction program stored in the memory and operable on the processor, the processor implementing the steps of the digital retina system based knowledge graph construction method according to any of claims 1-7 when executing the digital retina system based knowledge graph construction program.
10. A computer-readable storage medium, wherein a program for constructing a knowledge-graph based on a digital retinal system is stored on the computer-readable storage medium, and when the program for constructing a knowledge-graph based on a digital retinal system is executed by a processor, the steps of the method for constructing a knowledge-graph based on a digital retinal system according to any one of claims 1 to 7 are implemented.
CN202310982255.1A 2023-08-07 2023-08-07 Method for constructing knowledge graph based on digital retina system Active CN116701663B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310982255.1A CN116701663B (en) 2023-08-07 2023-08-07 Method for constructing knowledge graph based on digital retina system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310982255.1A CN116701663B (en) 2023-08-07 2023-08-07 Method for constructing knowledge graph based on digital retina system

Publications (2)

Publication Number Publication Date
CN116701663A true CN116701663A (en) 2023-09-05
CN116701663B CN116701663B (en) 2024-01-09

Family

ID=87831561

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310982255.1A Active CN116701663B (en) 2023-08-07 2023-08-07 Method for constructing knowledge graph based on digital retina system

Country Status (1)

Country Link
CN (1) CN116701663B (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110009665A (en) * 2019-03-12 2019-07-12 华中科技大学 A kind of target detection tracking method blocked under environment
CN112884120A (en) * 2021-01-29 2021-06-01 北京大学 Graph neural network representation architecture
US20210241050A1 (en) * 2020-02-03 2021-08-05 Samsung Electronics Co., Ltd. System and method for efficient multi-relational entity understanding and retreival
CN113609305A (en) * 2021-07-27 2021-11-05 三峡大学 Method and system for building geographical knowledge graph of film and television works based on BERT
CN115269877A (en) * 2022-08-10 2022-11-01 深圳市网联安瑞网络科技有限公司 Method, system and equipment for constructing domain entity and event double-center knowledge graph
CN115620512A (en) * 2022-09-20 2023-01-17 鹏城实验室 Traffic prediction method, device, terminal and storage medium based on knowledge graph
CN116028653A (en) * 2023-03-29 2023-04-28 鹏城实验室 Method and system for constructing map by visually configuring multi-source heterogeneous data

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110009665A (en) * 2019-03-12 2019-07-12 华中科技大学 A kind of target detection tracking method blocked under environment
US20210241050A1 (en) * 2020-02-03 2021-08-05 Samsung Electronics Co., Ltd. System and method for efficient multi-relational entity understanding and retreival
CN112884120A (en) * 2021-01-29 2021-06-01 北京大学 Graph neural network representation architecture
CN113609305A (en) * 2021-07-27 2021-11-05 三峡大学 Method and system for building geographical knowledge graph of film and television works based on BERT
CN115269877A (en) * 2022-08-10 2022-11-01 深圳市网联安瑞网络科技有限公司 Method, system and equipment for constructing domain entity and event double-center knowledge graph
CN115620512A (en) * 2022-09-20 2023-01-17 鹏城实验室 Traffic prediction method, device, terminal and storage medium based on knowledge graph
CN116028653A (en) * 2023-03-29 2023-04-28 鹏城实验室 Method and system for constructing map by visually configuring multi-source heterogeneous data

Also Published As

Publication number Publication date
CN116701663B (en) 2024-01-09

Similar Documents

Publication Publication Date Title
CN108780463B (en) Social graph generation and management
CN111782872B (en) Query method, query device, electronic equipment and storage medium
US10599313B2 (en) System for high volume data analytic integration and channel-independent advertisement generation
CN110309264B (en) Method and device for acquiring geographic product data based on knowledge graph
US11188586B2 (en) Organization, retrieval, annotation and presentation of media data files using signals captured from a viewing environment
US20160364770A1 (en) System for high volume data analytic integration and channel-independent advertisement generation
US10097888B2 (en) Determining audience engagement
CN109033109B (en) Data processing method and system
US20150201030A1 (en) Systems and methods for providing geographically delineated content
CN111125344B (en) Related word recommendation method and device
JP7084691B2 (en) How to process and present real-time social data on a map
CN114244595B (en) Authority information acquisition method and device, computer equipment and storage medium
US20200186869A1 (en) Method and apparatus for referencing, filtering, and combining content
JP7242994B2 (en) Video event identification method, apparatus, electronic device and storage medium
US20230342663A1 (en) Machine learning application method, device, electronic apparatus, and storage medium
KR20100120191A (en) N-way multimedia collaboration system
CN112307264A (en) Data query method and device, storage medium and electronic equipment
CN112328658B (en) User profile data processing method, device, equipment and storage medium
CN116701663B (en) Method for constructing knowledge graph based on digital retina system
Alam et al. Intellibvr-intelligent large-scale video retrieval for objects and events utilizing distributed deep-learning and semantic approaches
WO2022178238A1 (en) Live updates in a networked remote collaboration session
US11244019B2 (en) Enrichment of user specific information
CN114117192A (en) Object query method, device, server and storage medium
CN111353067A (en) Unified search method, search device and system
CN111506737B (en) Graph data processing method, searching method, device and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant