CN111680036A - Method and device for realizing configuration management library based on graph storage - Google Patents
Method and device for realizing configuration management library based on graph storage Download PDFInfo
- Publication number
- CN111680036A CN111680036A CN202010396069.6A CN202010396069A CN111680036A CN 111680036 A CN111680036 A CN 111680036A CN 202010396069 A CN202010396069 A CN 202010396069A CN 111680036 A CN111680036 A CN 111680036A
- Authority
- CN
- China
- Prior art keywords
- node
- module
- graph
- configuration management
- gql
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Abstract
The invention discloses a configuration management database based on graph storage, which comprises a Node file Node Files storage module; a Relation file storage module; an Index file Index Files storage module; a Log file Log Files storage module; a Graph Buffer module of the Graph; the indexed cache Index Buffer; connecting a Pool Connection Pool module; a Web View Web View module; a GQL Parser module supporting graph Query language GQL (graph Query language), including but not limited to Cypher, Gremlin, nGQL; a GQL Optimizer module; a GQL Executor module; adapting node data structures, inter-node relationship data structures, and execution logic of a graph query cut (create Update Retrieve delete) of a graph of a configuration management database.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a method and a device for managing a configuration management database based on graph storage.
Background
With the rapid development of information technology and the arrival of the big data era, various enterprises and institutions such as electric power, banks, tax, securities, public security and the like increasingly rely on the support of information technology on business, the IT infrastructure is continuously perfected, and various application systems, servers and network equipment are increased explosively. Therefore, under the guidance of development targets of a visual, quantifiable, intelligent, automatic and controllable modern data center, how to effectively manage system operation, equipment operation states and the relationship among the system operation and the equipment operation states is of great importance in constructing a CMDB (configuration Management database) configuration Management library system facing an IT infrastructure.
In the early stage of informatization, the IT infrastructure is generally managed by document management (such as Excel spreadsheet) and configuration management of the IT infrastructure is carried out in a manual entry mode. With the development of informatization, the types and the quantity of equipment are rapidly increased, and the configuration management requirements of an IT infrastructure cannot be met based on a document mode. Therefore, a configuration management library based on a relational database is largely constructed, and most of the configuration management library is based on a BS architecture (Browser Server), IT infrastructure data is stored in the relational database, and a Server side accesses the database to acquire asset data and displays the asset data on the Browser side.
However, with the rapid development of application services of enterprises and organizations, the operations and maintenance of the enterprises and organizations are built around the configuration management library, which puts higher requirements on the configuration management library: 1, the method can flexibly expand and add new types of assets; 2, IT can manage IT infrastructure with finer granularity, for example, because IT needs to monitor the state and flow of each port on the switch, the port of the switch also needs to be managed; 3, quick and flexible association query can be performed on the IT infrastructure, for example, statistics can be performed on how many devices are in a computer room, which managers belong to which departments the devices belong to, the operation states and alarm states of the devices, and the like. The existing configuration management library system based on the relational database is difficult to meet the requirements, and the main reasons are as follows: the relational database stores data according to a structured method, each data table has to define each field (namely, the structure of the table is defined firstly), and then the data is stored according to the structure of the table, so that the advantage is that the reliability and the stability of the whole data table are higher because the form and the content of the data are defined before the data are stored, but the problem is that once the data are stored, the structure of the data table is difficult to modify if necessary. 2, the relational database stores the data in the form of the minimum relational table in order to avoid duplicating and normalizing the data and fully utilize the storage space, so that the data management can become clear and clear at a glance, which is the case of a data table. If the situation of a plurality of tables is different, the data relates to a plurality of data tables, complex relations exist among the data tables, and data management becomes more and more complex as the number of the data tables increases. 3, because the relational database stores data in the data tables, the bottleneck of data operation occurs in the operation of a plurality of data tables, and the problem is more serious if the number of data tables is more, if the problem is to be alleviated, only the processing capacity can be improved, namely, a computer with higher speed and higher performance is selected.
Therefore, the configuration relational database constructed based on the relational database cannot meet the demand of rapid development of informatization of enterprises and institutions.
Therefore, the invention provides a management method and a device for managing a database based on graph storage, and the method and the device are used for performing CURD (Create, Update, Retrieve, Delete) operation on data of the configuration management database based on a traversal algorithm in graph theory. The graph is composed of a set of nodes and a set of relationships between the nodes. The graph may be implemented as an adjacency matrix, an adjacency table, an orthogonal list, an adjacency multilist, and an edge set array, or otherwise.
Disclosure of Invention
In order to realize the purpose of the invention, the following technical scheme is adopted for realizing the purpose:
a configuration management database stored based on a graph, wherein: the database comprises a Node file Node Files storage module; a Relation file storage module; an Index file Index Files storage module; a log files storage module; a Graph Buffer module of the Graph; an indexed cache Index Buffer module; connecting a Pool Connection Pool module; a Web View Web View module; a GQL Parser module; a GQL Optimizer module; GQL Executor module.
The database, wherein:
the Node Files module is used for storing the Node data in the configuration management database in a JSON mode in a fragmentation mode;
the relationship Files module is used for storing the relationship data between the nodes in the configuration management database in a JSON mode in a fragmentation mode in a file;
and the Index Files module is used for storing Index information of the attributes in the node data.
The Log Files module is used for recording the GQL operation of the Log file on the configuration management database according to the date slices;
the Graph Buffer module is used for loading Node data and relationship data among nodes stored in a file into a memory in a Graph mode after a configuration management library is started, providing queryable data for the GQL Executor module, wherein the queryable data comprises the Node data and the relationship data among the nodes, and when the GQL Executor module executes the operations of adding, deleting and modifying the Node and the relationship data, firstly changing the data in the Graph Buffer module, changing the Graph structure, and then triggering the data change of the Node Files module, the relationship Files module and the Index Files module by the Graph Buffer module;
the Connection Pool module is used for providing a Connection Pool for the configuration management library to the outside, allowing a user application program to be connected to the configuration management library in a TCP/IP mode or an Http mode, and performing authorization verification on the Connection;
the Web View module is used for providing a Web View for configuration management when a user is connected to the configuration management library in an Http mode;
the GQL Parser module is used for analyzing a GQL statement or script input by a user and generating structural data which can be understood by a configuration management library;
the GQL Optimizer module is used for optimizing the structured data output by the GQLGQL Parser module and outputting executable structured data;
the GQL analyzer module is used for executing executable structured data output by the GQL Optimizer module based on a traversal algorithm of a graph theory and adapting to service logic of an application layer of a configuration management library, and returning an execution result to a user in a JSON format.
The database, wherein:
in the database, the data of the node is defined and described in a key-value structure mode, and the key-value structure comprises four parts: < label > < index > < head > < body >, wherein the head part fixedly comprises the five-tuple: name, uuid, gmt _ create, gmt _ modify, gmt _ life, respectively: the method comprises the following steps of (1) node name, universal unique identification code of the node, creation time of the node, modification time of the node and life cycle of the node;
in the database, the data of the relationship between the nodes is defined and described in a key-value structure mode, and the key-value structure comprises three parts: < label >, < head >, < body >, wherein the head part fixedly comprises six tuples: name, start, end, gmt _ create, gmt _ modify, gmt _ life, respectively: name of relationship, start node, end node, creation time, modification time, life cycle.
A management method for a configuration management database based on graph storage at least comprises the following steps:
create node operation of the graph of the configuration management database: implementing code logic to be implemented at an application layer in a configuration management database, including: searching for repeated nodes according to the labels and the names; generating a uuid for the node; searching opposite end nodes of the relation, if the node does not exist, automatically creating the node, and creating the relation between the nodes;
delete node operation of the graph of the configuration management database: implementing code logic to be implemented at an application layer in a configuration management database, including: searching nodes according to the index; judging whether a relation can be deleted or not, and if so, automatically deleting;
modify node operation of a graph of a configuration management database: implementing code logic to be implemented at an application layer in a configuration management database, including: searching nodes according to the index; judging whether the relation to be deleted exists or not, and if so, automatically deleting the relation; judging whether the relationship to be created exists or not, and if so, automatically creating;
lookup node operation of the graph of the configuration management database: implementing code logic to be implemented at an application layer in a configuration management database, including: and judging whether the associated node exists, and if so, automatically searching the relationship between the associated node and the node.
The management method, wherein the operation of creating the node comprises the following steps:
1) a user inputs GQL of a creating node through a client, the GQL is processed by a GQL Parser module and a GQLOptimizer module to obtain GQL optimized structured data of the creating node, the GQL optimized structured data is input to a GQL Executor module, and the creating of the node is realized by the GQL Executor module;
2) inquiring whether nodes with the same type and name exist in a Graph Buffer module according to the type and name of the nodes;
3) if the same type and name entity already exists in the Graph Buffer, returning that the node already exists;
4) if the same type and name entity do not exist in the Graph Buffer, creating a node in the Graph Buffer according to the attribute information of the node, and updating data in the Index Buffer;
5) if the input information contains the relationship information, inquiring whether the associated node exists in the Graph Buffer;
6) if the associated node exists, establishing a relationship with the associated node in the Graph Buffer according to the relationship information;
7) if the associated node does not exist, but if the associated node needs to be created at the same time, the associated node is created in the GraphBuffer first, then the relationship is created, and the Index Buffer is updated; if the associated node needs to exist, prompting the user that the relationship establishment fails;
8) updating Node Files, relationship Files and Index Files;
9) a unique index of the created node is returned.
The management method, wherein the operation of deleting the node comprises the following steps:
1) the user inputs the GQL of the deleted node through the client, the GQL is processed by the GQL Parser module and the GQLOptimizer module to obtain GQL optimized structured data of the deleted node, the GQL optimized structured data is input to the GQL Executor module, and the deletion of the node is realized by the GQL Executor module;
2) inquiring whether the node exists in the Graph Buffer according to the unique index of the node;
3) if the node does not exist, returning the node deletion success;
4) if the node exists, inquiring whether the node has a relationship with other nodes in the Graph Buffer;
5) if the relationship exists and can be deleted, deleting the relationship, then deleting the node, and updating the Index Buffer; if the entity of the associated node map is part of the entity of the node map, then the associated node is also deleted;
6) if the relationship exists, but the relationship cannot be deleted, returning deletion failure;
7) returning deletion success;
the management method, wherein the operation of modifying the node comprises the following steps:
1) a user inputs the GQL of a modification node through a client, the GQL is processed by a GQL Parser module and a GQLOptimizer module to obtain GQL optimized structured data of the modification node, the GQL optimized structured data is input to a GQL Executor module, and the modification of the node is realized by the GQL Executor module;
2) inquiring whether the node exists in the Graph Buffer according to the unique index of the node;
3) if not, returning that the node does not exist;
4) if so, modifying the node in the Graph Buffer, and updating the Index Buffer;
5) inquiring whether the node has a relation to be deleted in the Graph Buffer;
6) if the relation which needs to be deleted exists, deleting the relation in the Graph Buffer;
7) if the relation which needs to be deleted does not exist or is deleted, judging whether the relation needs to be established with other nodes or not;
8) if the relationship with other nodes is not required to be established, the modification success is returned;
9) if a relationship is required to be established with other nodes, whether the associated node exists is inquired in the Graph Buffer;
10) if the associated node exists, establishing a relationship with the associated node according to the relationship information;
11) if the associated node does not exist, but if the associated node needs to be created at the same time, the associated node is created first, then the relationship is created, and the Index Buffer is updated; if the associated node needs to exist, finishing the node modification process and prompting the user that the relationship creation fails;
12) and returning the success of modifying the node.
The management method, wherein the searching node operation comprises the following steps:
1) a user inputs the GQL of a query node through a client, the GQL is processed by a GQL Parser module and a GQLOptimizer module to obtain GQL optimized structured data of the query node, the GQL optimized structured data is input to a GQL extractor module, and the query of the node is realized by the GQL extractor module;
2) inquiring whether the node exists in the Graph Buffer according to the unique index of the node;
3) if the node meeting the query condition does not exist, returning a query result as null;
if the nodes meeting the query conditions exist, querying whether the Graph Buffer has associated nodes;
4) if the associated node exists, jumping to the step (4), and inquiring whether the associated node has other associated nodes in the Graph Buffer;
5) if no associated node exists, all nodes and relationships are returned.
The management method, wherein the management method is applied to the database as described in one of the above.
Drawings
FIG. 1 is a block diagram of a configuration management database based on graph storage according to the present invention;
FIG. 2 is a schematic diagram of a process for creating nodes and relationships between nodes according to the present invention;
FIG. 3 is a schematic diagram of a process for deleting graph nodes and relationships between nodes according to the present invention;
FIG. 4 is a schematic diagram of a process for modifying nodes and relationships between nodes according to the present invention;
FIG. 5 is a schematic diagram of a process for querying nodes and relationships between nodes according to the present invention;
FIG. 6 is a schematic diagram of adding asset types to a configuration management library based on a relational database according to the present invention;
FIG. 7 is a schematic diagram of adding asset types based on graph store configuration management of the present invention.
Detailed Description
The following detailed description of the present invention will be made with reference to the accompanying drawings 1-7.
Fig. 1 is a configuration management database framework based on graph storage, which is specifically as follows: the database comprises a Node Files (Node Files) storage module; a Relation Files (relationship Files) storage module; an Index Files (Index Files) storage module; a Log Files (Log Files) storage module; a Graph-based cache (Graph Buffer) module; an Index cache (Index Buffer) module; a Connection Pool (Connection Pool) module; a Web View (Web View) module; a graph query language Parser (GQL Parser) module; a graph query language Optimizer (GQL Optimizer) module; a graph query language Executor (GQL Executor) module.
Wherein:
the Node Files module is used for storing Node data in a file in a JSON (JavaScript object Notation) mode in a configuration management database in a fragmentation mode, wherein the lower limit of each fragmentation is 64Kbits, the upper limit is 64Mbits, and the lower limit can be adjusted through a configuration file;
the nodes of the graph database represent configuration items for configuring the management database, mapping entities in the real world. A node may be a complete entity, for example: machine rooms, cabinets, network equipment, and the like. The nodes may also be part of an entity that needs separate model management, such as: ports of switches, hard disks of servers, etc. A node may also be a logically meaningful group containing multiple entities, such as: MySQL cluster, xx business system, etc. The data of the node is defined and described by adopting a Key-value structure, and the Key of the node comprises: < label > < index > < head > < body >, as defined below:
the data of the nodes has two expression modes of strict expression and loose expression:
the head is defined as follows:
and 2, the relationship Files module is used for storing the relationship data between the nodes in the configuration management database in a JSON mode in a file in a segmented manner, the lower limit of each segment is 64Kbits, the upper limit is 64Mbits, and the configuration management database can be adjusted through the configuration file. The definition and paradigm of the relationships between nodes are designed as follows:
the relationship between the nodes represents the relationship between the configuration items of the configuration management database, and maps the relationship between the entities in the real world. The accuracy of the relationship is important for the configuration management library, and besides the fact that the nodes can be associated together for management, the relationship has many important applications in an intelligent operation and maintenance scene built around the configuration management library, for example: alarm correlation analysis, fault root cause analysis and the like all depend on the relationship between nodes. In the figure, the relationship may or may not have a direction. In the configuration management library application scenario, most relationships are directional, and the meaning of the representation includes but is not limited to: dependency, superordinate, inclusion, affiliation, ownership, operation, management, and the like. The relationship may also be directionless, simply indicating that two nodes have a simple associative relationship, e.g., a co-worker relationship between people. There may be multiple relationships between two nodes, each representing a different meaning. The data of the relationship between the nodes is defined and described in a key-value structure mode, and the key-value structure comprises four parts: < label >, < head >, < body >, which are specifically defined as follows:
the relational data model has two expression modes of strict expression and loose expression:
the head is defined as follows:
and 3, the Index Files module is used for storing Index information of the attributes in the node data. The index data adopts the definition of a key-value structure, and the specific definition is as follows:
log Files module: and the system is used for recording the log file in a graph query of the configuration management database according to the date slices. The specific format is as follows:
and 5, the Graph Buffer module is used for loading the Node data and the relationship data between the nodes stored in the Node Files and the relationships Files into the memory in a Graph mode after the configuration management library is started. And providing queryable data for the GQLExcitor module, wherein the queryable data comprises node data and relationship data among nodes. And when the GQL interpolator module executes the operations of adding, deleting and modifying the Node and the relationship data, firstly changing the data in the Graph Buffer module, changing the Graph structure, and then triggering the data change of the Node Files module, the relationship Files module and the Indexfiles module by the Graph Buffer module. The graph may be implemented as an adjacency matrix, an adjacency table, a cross-linked list, an adjacency multilist, and an edge-set array, or otherwise;
and 6, the Index Buffer module is used for loading the data stored in the Index Files to the memory after the configuration management library is started and the Graph Buffer module is loaded, so that the efficiency of querying according to the node attribute names is improved. Index data is stored in memory in a key-value structure. The specific format is as follows:
and 7, the Connection Pool module is used for providing a Connection Pool for the configuration management library to the outside, allowing the user application program to be connected to the configuration management library in a TCP/IP mode or an Http mode, and performing authorization verification on the Connection. The working principle is that TCP service monitoring or Http service monitoring is started on a designated port, and the TCP service monitoring port is not consistent with the Http service monitoring port. When a TCP connection request is monitored, a long TCP connection is established with the opposite terminal for subsequent communication. After the connection is established, the authorization information (user name and password) in the connection is checked, if the authorization information fails to be verified, the connection is disconnected, otherwise, the connection is maintained. When an Http service request is monitored, authorization information verification is carried out, if the authorization information verification fails, an error code is returned, and if the authorization information verification fails, other modules are switched to process the error code;
the role of the Web View module is: when the user is connected to the configuration management library in an Http mode, the module provides a Web view of configuration management, and the view content comprises: the system comprises a login view, a basic information view of a configuration management library, a relationship view between nodes and the like, a command view and a help user to manage the configuration management library in a Web mode;
the GQL Parser module is used for: and analyzing the GQL statement or script input by the user to generate structured data which can be understood by the configuration management library. Examples are as follows:
the GQL Optimizer module is to: and converting the structured data output by the GQL Parser module into structured data which can be executed by the GQL Parser module according to the optimization rule. The results of different GQL conversions may be the same, examples are as follows;
a GQL Executor module for: and executing the executable structured data output by the GQL Optimizer module based on a graph theory traversal algorithm, and returning an execution result to a user in a JSON format.
Fig. 2 is a flowchart of creating nodes and relationships between nodes based on the configuration management library stored in the graph, which is specifically as follows:
1. a user inputs GQL of a creating node through a client, the GQL is processed by a GQL Parser module and a GQLOptimizer module to obtain GQL optimized structured data of the creating node, the GQL optimized structured data is input to a GQL Executor module, and the creating of the node is realized by the GQL Executor module;
2. inquiring whether nodes with the same type and name exist in a Graph Buffer module according to the type and name of the nodes;
3. if the same type and name entity already exists in the Graph Buffer, returning that the node already exists;
4. if the same type and name entity do not exist in the Graph Buffer, creating a node in the Graph Buffer according to the attribute information of the node, and updating data in the Index Buffer;
5. if the input information contains the relationship information, inquiring whether the associated node exists in the Graph Buffer;
6. if the associated node exists, establishing a relationship with the associated node in the Graph Buffer according to the relationship information;
7. if the associated node does not exist, but if the associated node needs to be created at the same time, the associated node needs to be created in the graph Buffer first, then the relationship needs to be created, and the Index Buffer needs to be updated. For example, if the ports of the switch depend on the switch, creating the switch node requires creating the switch port node first and then creating the relationship between the switch and the switch port. However, if the associated node needs to already exist, the user is prompted for a failed relationship creation because the associated node does not exist. For example, when a switch is created, the cabinet node where the switch is located must already exist;
8. updating Node Files, relationship Files and Index Files;
9. a unique index of the created node is returned.
Fig. 3 is a flow of deleting nodes and relationships between nodes in the configuration management library based on graph storage, which is specifically as follows:
1. the user inputs the GQL of the deleted node through the client, the GQL is processed by the GQL Parser module and the GQLOptimizer module to obtain GQL optimized structured data of the deleted node, the GQL optimized structured data is input to the GQL Executor module, and the deletion of the node is realized by the GQL Executor module;
2. inquiring whether the node exists in the Graph Buffer according to the unique index of the node;
3. if the node does not exist, returning the node deletion success;
4. if the node exists, inquiring whether the node has a relationship with other nodes in the Graph Buffer;
5. if a relationship exists and can be deleted, the relationship is deleted, the node is deleted, and the Index Buffer is updated. For example, where switches are deployed in cabinets, the relationship between the switch and the current switch may be deleted directly when the switch is to be migrated to other cabinets. In addition, if the entity of the associated node map is part of the entity of the node map, the associated node also needs to be deleted. For example, when a switch node needs to be deleted, a node of a port on the switch also needs to be deleted synchronously; if the associated nodes have associated nodes with similar relations, the associated nodes also need to be deleted;
6. if the relationship exists, but the relationship cannot be deleted, returning deletion failure because the node has the relationship with other nodes;
7. returning deletion success;
fig. 4 is a flow of modifying nodes and relationships between nodes based on a configuration management library stored in a graph, specifically as follows:
1. a user inputs the GQL of a modification node through a client, the GQL is processed by a GQL Parser module and a GQLOptimizer module to obtain GQL optimized structured data of the modification node, the GQL optimized structured data is input to a GQL Executor module, and the modification of the node is realized by the GQL Executor module;
2. inquiring whether the node exists in the Graph Buffer according to the unique index of the node;
3. if not, returning that the node does not exist;
4. if so, modifying the node in the Graph Buffer, and updating the Index Buffer;
5. and inquiring whether the node has a relation to be deleted in the Graph Buffer. For example, when a switch is migrated from one cabinet to another cabinet, the relationship with the original cabinet needs to be deleted, and a relationship with a new cabinet needs to be created;
6. if the relation which needs to be deleted exists, deleting the relation in the Graph Buffer;
7. if the relation which needs to be deleted does not exist or is deleted, judging whether the relation needs to be established with other nodes or not;
8. if the relationship with other nodes is not required to be established, the modification success is returned;
9. if a relationship is required to be established with other nodes, whether the associated node exists is inquired in the Graph Buffer;
10. if the associated node exists, establishing a relationship with the associated node according to the relationship information;
11. if the associated node does not exist, but if the associated node needs to be created at the same time, the associated node needs to be created first, then the relationship needs to be created, and the Index Buffer needs to be updated. For example, if the ports of the switch depend on the switch, creating the switch node requires creating the switch port node first and then creating the relationship between the switch and the switch port. However, if the associated node needs to already exist, the node modification process is ended, the graph database rolls over the modified attributes, and the user is prompted for a failed relationship creation because the associated node does not exist. For example, when a switch node is modified, the cabinet node in which the switch resides must already exist.
12. Returning the node modification success;
fig. 5 is a flow of querying a relationship between nodes and a configuration management library node based on graph storage, and the specific operations are as follows:
1. a user inputs the GQL of a query node through a client, the GQL is processed by a GQL Parser module and a GQLOptimizer module to obtain GQL optimized structured data of the query node, the GQL optimized structured data is input to a GQL extractor module, and the query of the node is realized by the GQL extractor module;
2. inquiring whether the node exists in the Graph Buffer according to the unique index of the node;
3. if the node meeting the query condition does not exist, returning a query result as null;
if the nodes meeting the query conditions exist, querying whether the Graph Buffer has associated nodes;
4. if the associated node exists, jumping to the step (4), and inquiring whether the associated node has other associated nodes in the Graph Buffer;
5. if no associated node exists, all nodes and relationships are returned.
Relational databases are typically stored in a tree structure, such as a B + tree, a B-tree. Relational database queries are highly efficient with a small amount of data and few tables, but cross-table queries are less efficient when there are a large number of associated tables. Furthermore, the maintenance of the association table is complicated, and adding or deleting the table involves adding or deleting the association table, or changing the table structure affects the association table. In an enterprise or organization application, a configuration management library may involve storing dozens or even hundreds of types of assets, including: hardware, software, auxiliary equipment, and various association relations exist among the assets, so a large number of association tables are required. In addition, the association relationship between the assets and the personnel and between the assets and the departments is also stored. This results in a large number of tables in the relational database, and when performing multi-table joint query, the join operation needs to perform cartesian product operation on all data, which may drastically reduce the performance of the database cut (create Update Retrieve delete) operation.
However, for the present invention, the graph-based configuration management library has no table concept, each node of the graph represents a record, the edges represent the relationships between the nodes, and the time efficiency of the graph depth traversal and the breadth traversal can be always maintained at O (n + e), where n is the number of nodes and e is the number of edges. In some scenarios, the configuration management library may reach billions of levels of nodes and edges, but graph-based queries tend to be in subgraphs where nodes and edges are limited. Therefore, in a scenario where a large number of relationships exist, there is also stable high-performance cut execution efficiency. In addition, the atomic operation of the CURD of the graph is specific to a single node and a single edge, so that the CURD operation of a normal form on data can be realized through the design of a data model of the normal form, the requirement that the configuration management library is not required to be upgraded and reissued after the data structure is released by the configuration management library is met, and the operation stability and reliability of the configuration management library are improved.
Taking fig. 6 as an example, in the configuration management library based on the relational database, when a Switch type asset is newly added, since the Switch asset needs to associate Frame and User, a Switch table, an association table of Switch and Frame, and an association table of Switch and User need to be newly added. However, in the configuration management library based on graph storage, adding a new type of asset is to add a new node and a corresponding edge relation in the way, and both the Browser side and the Server side can be realized by a paradigm code without upgrading the configuration management library, and a graph structure is formed as shown in fig. 7.
Taking fig. 6 as an example, in the configuration management library based on the relational data, if the switches managed by the user are to be queried, the SQL that is required is as follows:
the Inner join operation requires Cartesian product operations on all data, which can drastically reduce the performance of the database CURD (Create Update Retrieve delete) operation. Meanwhile, in order to support the cut operation of the Switch table, both the Browser side and the Server side need to upgrade programs.
However, in the configuration management library based on graph storage, if the switches under the users are to be queried, the canonical graph query statement based on the Cypher language is as follows:
when label1 is User and label2 is Switch, the User managed Switch can be queried. In the graph structure, if the User node and the Switch node have an edge relationship, the User node and the Switch node become adjacent points, and the query efficiency is O (1).
By the invention, enterprises and organizations can quickly construct the configuration management database, and the configuration management database based on graph storage has great improvement on the aspects of the associated query efficiency and the attribute expansibility of the configuration items compared with the configuration management database based on a relational database, and the CURD operation specially integrated for the configuration management database can reduce the development workload and the system function expansibility of other business systems based on the configuration management database.
Claims (3)
1. A configuration management database based on graph storage, characterized by: the database comprises a node file NodeFiles storage module; a Relation file storage module; an Index file Index Files storage module; a Log file Log Files storage module; a Graph Buffer module of the Graph; an indexed cache Index Buffer module; connecting a Pool Connection Pool module; a Web View Web View module; a GQL Parser module; a GQL Optimizer module; GQL Executor module.
2. The database of claim 1, wherein:
in the database, the data of the node is defined and described in a key-value structure mode, and the key-value structure comprises four parts: < label > < index > < head > < body >, wherein the head part fixedly comprises the five-tuple: name, uuid, gmt _ create, gmt _ modify, gmt _ life, respectively: the method comprises the following steps of node name, universal unique identification code of the node, creation time of the node, modification time of the node and life cycle of the node.
3. A method for managing a configuration management database based on graph storage, comprising at least one of the following steps:
create node operation of the graph of the configuration management database: implementing code logic to be implemented at an application layer in a configuration management database, including: searching for repeated nodes according to the labels and the names; generating a uuid for the node; searching opposite end nodes of the relation, if the node does not exist, automatically creating the node, and creating the relation between the nodes;
delete node operation of the graph of the configuration management database: implementing code logic to be implemented at an application layer in a configuration management database, including: searching nodes according to the index; judging whether a relation can be deleted or not, and if so, automatically deleting;
modify node operation of a graph of a configuration management database: implementing code logic to be implemented at an application layer in a configuration management database, including: searching nodes according to the index; judging whether the relation to be deleted exists or not, and if so, automatically deleting the relation; judging whether the relationship to be created exists or not, and if so, automatically creating;
lookup node operation of the graph of the configuration management database: implementing code logic to be implemented at an application layer in a configuration management database, including: and judging whether the associated node exists, and if so, automatically searching the relationship between the associated node and the node.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010396069.6A CN111680036B (en) | 2020-05-12 | 2020-05-12 | Configuration management database based on graph storage |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010396069.6A CN111680036B (en) | 2020-05-12 | 2020-05-12 | Configuration management database based on graph storage |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111680036A true CN111680036A (en) | 2020-09-18 |
| CN111680036B CN111680036B (en) | 2022-09-27 |
Family
ID=72434171
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010396069.6A Active CN111680036B (en) | 2020-05-12 | 2020-05-12 | Configuration management database based on graph storage |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111680036B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114385866A (en) * | 2022-03-25 | 2022-04-22 | 湖南高至科技有限公司 | Graph database based executable model storage system and method |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040123048A1 (en) * | 2002-07-22 | 2004-06-24 | Ward Mullins | Dynamic object-driven database manipulation and mapping system having a simple global interface and an optional multiple user need only caching system with disable and notify features |
| CN101169716A (en) * | 2007-11-30 | 2008-04-30 | 清华大学 | Emulated procedure information modeling and maintenance method based on product structural tree |
| US20120096043A1 (en) * | 2010-10-19 | 2012-04-19 | 7 Degrees, Inc. | Data graph cloud system and method |
| CN102855332A (en) * | 2012-09-24 | 2013-01-02 | 上海天玑科技股份有限公司 | Graphic configuration management database based on graphic database |
| CN103380421A (en) * | 2010-12-30 | 2013-10-30 | 脸谱公司 | Distributed cache for graph data |
| CN103823865A (en) * | 2014-02-25 | 2014-05-28 | 南京航空航天大学 | Database primary memory indexing method |
| CN104063507A (en) * | 2014-07-09 | 2014-09-24 | 时趣互动（北京）科技有限公司 | Graph computation method and engine |
| CN104123369A (en) * | 2014-07-24 | 2014-10-29 | 中国移动通信集团广东有限公司 | CMDB system based on graphic data base and implementation method |
| CN104572856A (en) * | 2014-12-17 | 2015-04-29 | 武汉科技大学 | Converged storage method of service source data |
| CN107733894A (en) * | 2017-10-19 | 2018-02-23 | 携程旅游网络技术（上海）有限公司 | Comparison method, system, equipment and the storage medium of logic interfacing message |
| CN108027818A (en) * | 2015-09-18 | 2018-05-11 | 微软技术许可有限责任公司 | Inquiry based on figure |
-
2020
- 2020-05-12 CN CN202010396069.6A patent/CN111680036B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040123048A1 (en) * | 2002-07-22 | 2004-06-24 | Ward Mullins | Dynamic object-driven database manipulation and mapping system having a simple global interface and an optional multiple user need only caching system with disable and notify features |
| CN101169716A (en) * | 2007-11-30 | 2008-04-30 | 清华大学 | Emulated procedure information modeling and maintenance method based on product structural tree |
| US20120096043A1 (en) * | 2010-10-19 | 2012-04-19 | 7 Degrees, Inc. | Data graph cloud system and method |
| CN103380421A (en) * | 2010-12-30 | 2013-10-30 | 脸谱公司 | Distributed cache for graph data |
| CN102855332A (en) * | 2012-09-24 | 2013-01-02 | 上海天玑科技股份有限公司 | Graphic configuration management database based on graphic database |
| CN103823865A (en) * | 2014-02-25 | 2014-05-28 | 南京航空航天大学 | Database primary memory indexing method |
| CN104063507A (en) * | 2014-07-09 | 2014-09-24 | 时趣互动（北京）科技有限公司 | Graph computation method and engine |
| CN104123369A (en) * | 2014-07-24 | 2014-10-29 | 中国移动通信集团广东有限公司 | CMDB system based on graphic data base and implementation method |
| CN104572856A (en) * | 2014-12-17 | 2015-04-29 | 武汉科技大学 | Converged storage method of service source data |
| CN108027818A (en) * | 2015-09-18 | 2018-05-11 | 微软技术许可有限责任公司 | Inquiry based on figure |
| CN107733894A (en) * | 2017-10-19 | 2018-02-23 | 携程旅游网络技术（上海）有限公司 | Comparison method, system, equipment and the storage medium of logic interfacing message |
Non-Patent Citations (1)
| Title |
|---|
| 董玉坤等: "基于Java技术的OA系统的设计", 《办公自动化》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114385866A (en) * | 2022-03-25 | 2022-04-22 | 湖南高至科技有限公司 | Graph database based executable model storage system and method |
| CN114385866B (en) * | 2022-03-25 | 2022-06-14 | 湖南高至科技有限公司 | Executable model storage system and method based on graph database |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111680036B (en) | 2022-09-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11768848B1 (en) | Retrieving, modifying, and depositing shared search configuration into a shared data store | |
| US11748394B1 (en) | Using indexers from multiple systems | |
| US9582585B2 (en) | Discovering fields to filter data returned in response to a search | |
| US7730056B2 (en) | Software and method for utilizing a common database layout | |
| CN109144994A (en) | Index updating method, system and relevant apparatus | |
| US20130024484A1 (en) | System management in datacenter using a non-relational database | |
| US10909114B1 (en) | Predicting partitions of a database table for processing a database query | |
| GB2534374A (en) | Distributed System with accelerator-created containers | |
| US10949409B2 (en) | On-demand, dynamic and optimized indexing in natural language processing | |
| CN113760847A (en) | Log data processing method, device, equipment and storage medium | |
| CN111680036B (en) | Configuration management database based on graph storage | |
| US11636124B1 (en) | Integrating query optimization with machine learning model prediction | |
| US11294901B1 (en) | Isolating the performance of functions included in queries | |
| CN111708895B (en) | Knowledge graph system construction method and device | |
| Arputhamary et al. | A review on big data integration | |
| US11657069B1 (en) | Dynamic compilation of machine learning models based on hardware configurations | |
| US11836125B1 (en) | Scalable database dependency monitoring and visualization system | |
| US11574217B2 (en) | Machine learning based identification and classification of database commands | |
| US20210141796A1 (en) | System and methods for performing updated query requests in a system of multiple database engine | |
| US11500874B2 (en) | Systems and methods for linking metric data to resources | |
| US11188228B1 (en) | Graphing transaction operations for transaction compliance analysis | |
| US20240095225A1 (en) | Intelligently storing data in a fault tolerant system using robotic process automation and blockchain | |
| US11586608B1 (en) | Handling requests to access separately stored items in a non-relational database | |
| US11914610B1 (en) | Multi-way data store conversion | |
| US11797521B1 (en) | Associating a function with a table in a database system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |