CN117785871A

CN117785871A - Service model, software model and storage model highly-consistent storage system

Info

Publication number: CN117785871A
Application number: CN202311689710.5A
Authority: CN
Inventors: 杜辉; 程建芳; 张�浩; 孙欣; 张岳; 韩浩泽; 陈理凡; 钱程; 郑思思
Original assignee: China Nuclear Huawei Engineering Design And Research Co ltd
Current assignee: China Nuclear Huawei Engineering Design And Research Co ltd
Priority date: 2023-12-11
Filing date: 2023-12-11
Publication date: 2024-03-29

Abstract

The invention belongs to the field of computers, and particularly discloses a storage system with highly consistent service models, software models and storage models, which comprises the following steps: s1, combining a B+ tree disk data structure of a relational database management system with a graph storage structure of a graph database; s2, providing standard SQL statement support for complex query and relation modes, and not creating brand-new SQL dialects and database concepts randomly; s3, providing relevant operation support of adjacent query, depth-first traversal and breadth-first traversal based on a primary graph database for graph data query; s4, providing MYSQL, POSTGRESQL, ORACLE NET, TDS and other protocol support for complex query and relation modes so as to adapt to the common database protocol in the market, and not creating a brand new protocol randomly; s5, providing built-in RESTful protocol or HTTP protocol support for all queries, and describing formats of query and return data by JSON. The domestic industrial software (database) prepared by the storage system can be widely applied to various scientific and technical fields.

Description

Service model, software model and storage model highly-consistent storage system

Technical Field

The invention relates to the field of computers, in particular to a storage system, and specifically relates to a storage system with highly consistent business models, software models and storage models.

Background

The evolution of memory systems is an important development in the field of information technology, the history of which can be traced back to the end of the last 50 s and the beginning of the 60 s of the last century. Initially, data processing was primarily dependent on a decentralized file system, which had problems with data redundancy and consistency that were difficult to maintain. To solve these problems, a hierarchical database model has been developed that organizes data in a tree structure, but still suffers from the disadvantages of a relatively complex structure and difficulty in expansion.

Until the proposal of relational database models, the field of storage systems underwent a real revolution. The relational database adopts a table structure to store data, and queries through relational algebra and relational algorithm, so that a more flexible and standard mode is provided for data management. At this stage, the SQL language has been created as a standard query language, becoming a de facto standard for database operations. The push for commercialization and standardization has prompted the rise of large commercial database systems like Oracle, IBM DB2, microsoft SQL Server, mySQL, postgreSQL, etc., which provide reliable, high performance data management solutions for enterprises.

However, with the continuous development of information technology and the diversification of application scenarios, the demand for storage systems is also evolving. Object-oriented databases attempt to introduce object-oriented ideas into database systems to better meet the needs of object-oriented programming. Then, noSQL databases have been developed to make up for the deficiencies of traditional relational databases in handling large-scale distributed systems and unstructured data. The NoSQL database adopts various data models, including document type, key value pair, column family type and graph databases, and provides more flexible selection for different application scenes.

In recent years, with the rise of emerging technologies such as big data, artificial intelligence, and internet of things, database systems are facing new challenges again. NewSQL databases attempt to balance between traditional relational databases and NoSQL databases to meet the needs of high performance transaction processing and distributed systems. The time sequence database is used for processing time sequence data and is suitable for scenes such as the Internet of things and monitoring.

The current storage system is mainly divided into the following aspects according to the characteristics of the data model, the storage structure, the query language, the application scene and the like: the invention aims at the categories of a relational database management system (RDBMS), a NoSQL database, an object-oriented database, a NewSQL database, a time sequence database, a graph database, a column database, a memory database, a cloud database service, a blockchain database and the like.

The above-mentioned existing storage system has the following disadvantages: relational database management systems may be limited in performance in handling large-scale and high concurrency, and require tightly defined patterns and fixed table structures, are inflexible, and may suffer from reduced query efficiency in handling complex relationships and large numbers of connections. Although the object-oriented database supports visual object model mapping, the lack of a general query language results in higher learning and use costs, and is not applicable to all data types, limiting its wide application. Graph databases perform well when dealing with very large-scale graph data, but may present performance bottlenecks for complex queries and relational schemas, and are relatively inefficient when dealing with non-graph data, unlike specially designed database types. In combination with a service scene, the service model can be well matched with the software model, but the software model cannot be well matched with the storage model, and the matching is carried out by adopting O/R Mapping provided by a third party manufacturer. Most of industrial software (database) is a non-domestic product, and the contradiction between the increasing demands of domestic chemical industrial software (database) and the unbalanced development of domestic chemical industrial software (database) is increasingly remarkable, especially in the fields of national defense, military, nuclear industry and the like.

Disclosure of Invention

The invention aims to overcome the defects in the prior art and provides a storage system with highly consistent service models, software models and storage models.

In order to achieve the above purpose, the technical scheme adopted by the invention is as follows:

a storage system with highly consistent business model, software model and storage model comprises the following steps:

s1, combining a B+ tree disk data structure of a relational database management system with a graph storage structure of a graph database;

s2, providing standard SQL statement support for complex query and relation modes, and not creating brand-new SQL dialects and database concepts randomly;

s3, providing relevant operation support of adjacent query, depth-first traversal and breadth-first traversal based on a primary graph database for graph data query;

s4, providing MYSQL, POSTGRESQL, ORACLE NET, TDS and other protocol support for complex query and relation modes so as to adapt to the common database protocol in the market, and not creating a brand new protocol randomly;

s5, providing built-in RESTful protocol or HTTP protocol support for all queries, and describing formats of query and return data by JSON.

Further, all queries in S5 include complex queries, relational schema and graph data queries.

Further, in the step S1, different types of devices may be respectively designated to optimally utilize server resources.

Further, the B+ tree disk data structure sequentially selects equipment with good storage effect, and the graph storage structure selects equipment with good random reading performance.

Further, in the step S2, the standard SQL statement is limited to SQL-86, SQL-89, SQL-92, SQL-1999, SQL-2003, SQL-2008, SQL-2011, SQL-2016, SQL-2019.

Further, in the S3, the protogram database refers to a graph query language Graph Query Language.

Further, in the step S4, the protocol is used to support a common database protocol, and the storage system is used as any database familiar to the user.

Further, in the S4, the protocol performs RESTful protocol conversion on the query and return data in the S2, S3, and S4, and performs data exchange through JSON.

The beneficial effects of the invention are as follows: 1. the method has the advantages of a relational database management system and a graph database, is used for spatial exchange of performance and rich functional support, and provides access mode adaptation of the object database.

2. And (3) supporting the execution standard, adapting to a common database protocol, and realizing the user 0 learning cost switching and the application low-cost switching.

3. The original provided objectification access interface is supported in a RESTful protocol form, so that the service model, the software model and the storage model are highly consistent, the object model and the database model are seamlessly integrated, a user of the database does not depend on an O/R Mapping technology of a third party manufacturer any more, and the performance cost of secondary Mapping conversion is reduced.

The domestic industrial software (database) prepared by the storage system can be widely applied to various scientific and technical fields, and particularly applied to the research and development of software systems stored by depending on the database.

Drawings

FIG. 1 is a schematic diagram of a storage system according to the present invention;

FIG. 2 is a flow chart of query processing of the storage system of the present invention;

FIG. 3 is a timing diagram of a client query embodiment of a memory system of the present invention;

FIG. 4 is a timing diagram of a JDBC query embodiment of a memory system according to the present invention;

FIG. 5 is a timing diagram of an embodiment of an O/R Mapping query of the memory system of the present invention;

FIG. 6 is a timing diagram of an embodiment of an object query of the memory system of the present invention.

Detailed Description

Referring to fig. 1, a storage system with highly consistent service model, software model and storage model comprises the following steps:

The invention is further described below with reference to the drawings and detailed description.

Example 1

The embodiment describes an architecture model of the storage system of the present invention, as shown in fig. 1, where the storage system of the present invention is composed of four parts, i.e., a management service tool, a physical memory, a storage engine, and a file storage, the management service tool provides a connection manager, a restful adapter, a query analyzer, an optimizer, a transaction manager, a structured query algorithm, a graph query algorithm, and a task scheduler, the physical memory mainly provides a data buffer, a log buffer, and a control buffer, the storage engine is used to link the physical memory with the file storage, and the file storage is composed of a data file, a log file, and a control file, and is divided into a relational data storage area and a graph data storage area according to different data structures.

Compared with the traditional relational database management system and the graph database, the architecture model adds a RESTful adapter in a management service tool, and achieves object query adaptation.

Compared with the traditional relational database management system, the architecture model adds a graph query algorithm in a management service tool, and increases a graph data storage area in file storage.

Compared with a graph database, the architecture model adds a structured query algorithm in a management service tool, and increases a relational data storage area in file storage.

The embodiment describes a query processing flow of the storage system of the present invention, as shown in fig. 2, the storage system of the present invention provides query for a conventional client, JDBC, and an O/R Mapping manner based on JDBC, and additionally provides object query support based on RESTful protocol, and after a query request is issued, the object query is distinguished by a connection manager, or the conventional query is respectively queried through a RESTful adapter or directly entering a query analyzer, and data processing is performed by using a structured query algorithm and a graph query algorithm according to different queries.

The client adopted in the processing flow can be any client supporting protocols such as MYSQL, POSTGRESQL, ORACLE NET, TDS and the like, and is fully adapted to the active database protocol of the user, so that the learning cost switching of the user 0 is realized.

The JDBC adopted by the processing flow can be any JDBC of a database, and the application 0 adaptation cost switching based on the JDBC is realized.

The RESTful protocol mentioned in this process flow exchanges data via JSON.

This embodiment describes a timing diagram of a client query of a storage system of the present invention, as shown in fig. 3, where the storage system of the present invention provides support for the client query. The specific query flow is as follows:

step 1, a user (usually a user) initiates a query to a client, and the client can be any client supporting protocols such as MYSQL, POSTGRESQL, ORACLE NET, TDS and the like as described in the second embodiment;

step 2, the client calls a connection manager of the storage system to acquire connection and sends a query instruction;

step 3, the connection manager sends a query instruction to a query analyzer;

step 4, the query analyzer performs query analysis;

step 5, the query analyzer sends the analysis result to the optimizer;

step 6, the optimizer performs query optimization;

step 7, the optimizer sends an optimization result to the transaction manager;

step 8, the transaction manager starts the transaction;

step 9, the transaction manager sends a query instruction to data processing according to the data structure difference, and the transaction manager specifically comprises the following steps:

step 9.1, sending a relational data query instruction to relational data processing;

step 9.2, sending a graph data query instruction to graph data processing;

step 10, data processing returns a query result to the transaction manager, and the method specifically comprises the following steps:

step 10.1, the relational data processing returns a relational data query result to the transaction manager;

step 10.2, the graph data processing returns the graph data query result to the transaction manager;

step 11, closing the transaction by the transaction manager;

step 12, the transaction manager returns the query result to the connection manager;

step 13, the connection manager returns the query result to the client;

step 14, the client returns the query result to the user.

This embodiment describes a timing diagram of a JDBC query of the memory system of the present invention, as shown in fig. 4, which is directed to a memory system that provides support for JDBC queries. The specific query flow is as follows:

step 1, a user (usually an application) initiates a query to JDBC, and the JDBC can be any JDBC of a database as described in the second embodiment;

step 2, the JDBC calls a connection manager of the storage system to acquire connection and sends a query instruction;

step 3, the connection manager sends a query instruction to a query analyzer;

step 4, the query analyzer performs query analysis;

step 5, the query analyzer sends the analysis result to the optimizer;

step 6, the optimizer performs query optimization;

step 7, the optimizer sends an optimization result to the transaction manager;

step 8, the transaction manager starts the transaction;

step 9.2, sending a graph data query instruction to graph data processing;

step 11, closing the transaction by the transaction manager;

step 13, the connection manager returns the query result to the JDBC;

jdbc returns the query results to the user.

The embodiment describes a timing diagram of an O/R Mapping query of the storage system of the present invention, as shown in fig. 5, where the storage system of the present invention provides support for the O/R Mapping query. The specific query flow is as follows:

step 1, a user (usually an application) initiates a query to O/R Mapping;

step 2, O/R Mapping is converted into a query instruction;

step 3.O/R Mapping sends a query to JDBC, which may be any database as described in embodiment two;

step 4, the JDBC calls a connection manager of the storage system to acquire connection and sends a query instruction;

step 5, the connection manager sends a query instruction to a query analyzer;

step 6, the query analyzer performs query analysis;

step 7, the query analyzer sends the analysis result to the optimizer;

step 8, the optimizer performs query optimization;

step 9, the optimizer sends an optimization result to the transaction manager;

step 10, a transaction manager starts a transaction;

step 11, the transaction manager sends a query instruction to data processing according to the data structure difference, and the transaction manager specifically comprises the following steps:

step 11.1, a relational data query instruction is sent to relational data processing;

step 11.2, sending a graph data query instruction to graph data processing;

step 12, the data processing returns the query result to the transaction manager, specifically as follows:

step 12.1, the relational data processing returns a relational data query result to the transaction manager;

step 12.2, the graph data processing returns the graph data query result to the transaction manager;

step 13, closing the transaction by the transaction manager;

step 14, the transaction manager returns the query result to the connection manager;

step 15, the connection manager returns the query result to the JDBC;

step 16, JDBC returns the query result to O/R Mapping;

step 17, O/R Mapping is carried out to carry out object Mapping, and the query result is converted into an object;

and step 18. The O/R Mapping returns the object query result to the user.

This embodiment describes a timing diagram of an object query of a storage system of the present invention, as shown in fig. 6, which provides support for the object query. The specific query flow is as follows:

step 1, a user (usually an application) initiates an object query to a RESTful client, wherein the RESTful is the same as that in the second embodiment, and the RESTful protocol exchanges data through JSON;

step 2, the RESTful client calls a connection manager of the storage system to acquire connection and sends a query instruction;

step 3, the connection manager sends a query instruction to the RESTful adapter;

step 4, the RESTful adapter performs protocol conversion to convert the object query instruction into a query instruction;

step 5, the RESTful adapter sends a query instruction to a query analyzer;

step 6, the query analyzer performs query analysis;

step 7, the query analyzer sends the analysis result to the optimizer;

step 8, the optimizer performs query optimization;

step 9, the optimizer sends an optimization result to the transaction manager;

step 10, a transaction manager starts a transaction;

step 11.2, sending a graph data query instruction to graph data processing;

step 13, closing the transaction by the transaction manager;

step 14, the transaction manager returns the query result to the RESTful adapter;

step 15, the RESTful adapter carries out protocol conversion, and converts the query result into an object query result;

step 16, the RESTful adapter sends the object query result to the connection manager;

step 17, the connection manager returns the object query result to the RESTful client;

the restful client returns the object query result to the user.

In summary, the invention has the advantages of both the relational database management system and the graph database, uses space to exchange performance and rich functional support, and provides access mode adaptation of the object database.

And (3) supporting the execution standard, adapting to a common database protocol, and realizing the user 0 learning cost switching and the application low-cost switching.

The original provided objectification access interface is supported in a RESTful protocol form, so that the service model, the software model and the storage model are highly consistent, the object model and the database model are seamlessly integrated, a user of the database does not depend on an O/R Mapping technology of a third party manufacturer any more, and the performance cost of secondary Mapping conversion is reduced.

The foregoing has shown and described the basic principles, principal features and advantages of the invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, and that the above embodiments and descriptions are merely illustrative of the principles of the present invention, and various changes and modifications may be made therein without departing from the spirit and scope of the invention, which is defined by the appended claims. The scope of the invention is defined by the appended claims and equivalents thereof.

Claims

1. A storage system with highly consistent business models, software models and storage models, which is characterized by comprising the following steps:

2. The system of claim 1, wherein all queries in S5 comprise complex queries, relational schema and graph data queries.

3. A business model, software model, storage system with high consistency of storage model according to claim 1 or 2, characterized in that in S1, different types of devices can be respectively designated to optimize the utilization of server resources.

4. A storage system with highly consistent service models, software models and storage models according to claim 3, wherein the b+ tree disk data structure is a device with good storage effect, and the graph storage structure is a device with good random reading performance.

5. The storage system of claim 2, wherein in S2, the standard SQL statement is limited to SQL-86, SQL-89, SQL-92, SQL-1999, SQL-2003, SQL-2008, SQL-2011, SQL-2016, and SQL-2019.

6. The system of claim 2, wherein in S3, the protogram database references a graph query language Graph Query Language.

7. A business model, software model, storage model highly consistent storage system as claimed in claim 3 wherein in S4 said protocol is used to support a common database protocol, this storage system being used as any database familiar to the user.

8. A service model, software model, storage model highly consistent storage system as claimed in claim 3 wherein in S4 said protocol performs RESTful protocol conversion on the query and return data in S2, S3, S4 and data exchange via JSON.