CN111708761A - Partitioned autonomous distributed heterogeneous database management system and method - Google Patents
Partitioned autonomous distributed heterogeneous database management system and method Download PDFInfo
- Publication number
- CN111708761A CN111708761A CN202010557254.9A CN202010557254A CN111708761A CN 111708761 A CN111708761 A CN 111708761A CN 202010557254 A CN202010557254 A CN 202010557254A CN 111708761 A CN111708761 A CN 111708761A
- Authority
- CN
- China
- Prior art keywords
- partition
- data
- identification information
- source
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000005192 partition Methods 0.000 claims abstract description 290
- 238000007726 management method Methods 0.000 claims abstract description 87
- 238000012545 processing Methods 0.000 claims abstract description 62
- 238000012546 transfer Methods 0.000 claims abstract description 54
- 230000003993 interaction Effects 0.000 claims abstract description 16
- 230000002452 interceptive effect Effects 0.000 claims abstract description 4
- 230000008569 process Effects 0.000 claims description 22
- 238000006243 chemical reaction Methods 0.000 claims description 13
- 238000000638 solvent extraction Methods 0.000 claims description 3
- 230000006870 function Effects 0.000 description 11
- 230000005540 biological transmission Effects 0.000 description 10
- 238000013523 data management Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 4
- 230000010354 integration Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 230000002567 autonomic effect Effects 0.000 description 2
- 239000012634 fragment Substances 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 241001178520 Stomatepia mongo Species 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002688 persistence Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2471—Distributed queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/283—Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Fuzzy Systems (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a partitioned autonomous distributed heterogeneous database management system and a partitioned autonomous distributed heterogeneous database management method, which relate to the field of distributed database management; dividing different database systems into different partitions, respectively allocating different partition identification information to the different partitions, and performing data interactive management among the partitions according to the partition identification information: the method comprises the steps that a source partition sends a service command to be executed, source partition identification information and target partition identification information to a transfer processing layer, the transfer processing layer converts data of the service command into a data format which can be identified by a target partition and sends the data to the target partition, the target partition executes the service command to obtain related data, the source partition identification information and the target partition identification information are returned to the transfer processing layer, the transfer processing layer converts the related data into the data format which can be identified by the source partition and sends the data to the source partition, and data interaction is completed.
Description
Technical Field
The invention discloses a management system and a management method, relates to the field of distributed database management, and particularly relates to a partitioned autonomous distributed heterogeneous database management system and a partitioned autonomous distributed heterogeneous database management method.
Background
Under the background of the current technical development, a series of databases represented by Oracle, MySQL, SQLServer, MongoDB and DB2 emerge from the database industry, each database has its own excellence field, advantages and user group, different databases are extremely difficult to communicate due to different database protocols and different storage structures and processing logics among different databases, and the development of enterprise business is affected.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a partitioned autonomous distributed heterogeneous database management system and a partitioned autonomous distributed heterogeneous database management method, which are matched with a certain algorithm to realize unified processing of various heterogeneous databases in a heterogeneous interconnection scene, realize partitioned autonomous and integrate heterogeneous partitions into a complete generalized database for management.
The specific scheme provided by the invention is as follows:
a method for managing the partitioned and autonomous distributed heterogeneous database includes dividing different database systems into different partitions, assigning different partition ID information to different partitions,
according to the partition identification information, performing data interactive management among partitions:
the source partition sends the service command to be executed and the identification information of the source partition and the identification information of the destination partition to the transit processing layer,
the transfer processing layer converts the data of the service command into a data format which can be identified by the target subarea, sends the data to the target subarea,
the target subarea executes the service command to obtain related data, the source subarea identification information and the target subarea identification information are returned to the transfer processing layer,
and the transfer processing layer converts the related data into a data format which can be identified by the source partition, and sends the data format to the source partition to complete data interaction.
The partition identification information in the partition autonomous distributed heterogeneous database management method comprises a partition identifier and a database type identifier, and according to the database type identifier of a source partition and the database type identifier of a target partition, a transfer processing layer selects a proper data format conversion rule and converts data needing to be converted into a data format capable of being identified.
In the method for managing the partitioned autonomous distributed heterogeneous database, when a source partition interacts associated data with more than one target partition, one of the target partitions is selected to integrate all the obtained related data, and the integrated related data is sent to the source partition through a transfer processing layer.
In the management method of the partitioned autonomous distributed heterogeneous database, a global index of a partition is established, and information of the partition is searched through the global index.
The process of searching the information of the partitions through the global index in the partition autonomous distributed heterogeneous database management method is as follows:
the transfer processing layer receives the command of inquiring the partition information, searches the global index according to the provided information, searches the corresponding partition information,
if the search is successful, the corresponding operation is executed according to the target partition searched by the global index,
and if the search fails, sequentially inquiring the information of each partition.
A partitioned autonomous distributed heterogeneous database management system comprises a partitioning module and a transit management module,
the dividing module divides different database systems into different partitions, respectively allocates different partition identification information to the different partitions,
according to the partition identification information, the data interaction management is carried out between the partitions by using a transfer management module:
the source partition sends the service command to be executed and the identification information of the source partition and the identification information of the destination partition to the transfer processing module,
the transfer processing module converts the data of the service command into a data format which can be identified by the target subarea, sends the data format to the target subarea,
the target subarea executes the service command to obtain related data, the source subarea identification information and the target subarea identification information are returned to the transfer processing module,
and the transfer processing module converts the related data into a data format which can be identified by the source partition, and sends the data format to the source partition to complete data interaction.
A partitioned autonomous distributed heterogeneous database management apparatus, comprising: at least one memory and at least one processor;
the at least one memory to store a machine readable program;
the at least one processor is configured to invoke the machine readable program to perform the partitioned autonomic distributed heterogeneous database management method.
A computer readable medium having stored thereon computer instructions which, when executed by a processor, cause the processor to perform the method of partitioned autonomous distributed heterogeneous database management.
The invention has the advantages that:
the invention provides a partitioned self-governing distributed heterogeneous database management system and a method thereof, which allows users to use different databases under different scenes to enable each small scene to realize the optimal processing of data, the scenes are different database system partitions which can be applied in a distributed system, in one partition, the self management is carried out through a specific database management system, and the data transmission and interaction among different partitions are carried out through a transfer management layer in each partition, the instruction of each partition is sent to other partitions through the transfer management layer, thereby not only enabling each partition to process the data in the partition with the highest efficiency, but also interconnecting the advantages of each partition through the transfer management system, realizing the infinitely-expanded efficient partitioned self-governing heterogeneous distributed database management among different partitions, the processing level of the existing database system can be achieved for different operations.
Drawings
FIG. 1 is a schematic diagram of the application framework of the method of the present invention;
FIG. 2 is a schematic diagram of a data format conversion process in the method of the present invention;
FIG. 3 is a schematic diagram of data interaction between a source partition and a target partition associated in the method of the present invention.
Detailed Description
The Oracle database system may allow users to easily deploy new OLTPs and data warehouses at the cloud, or migrate existing OLTPs and data warehouses to the cloud. And a safe, intelligent and highly available cloud database is utilized to obtain more values from the data, so that the service development is accelerated.
MySQL is a relational database management system developed by MySQL AB, Sweden, and currently belongs to the product under Oracle flag. MySQL is one of the most popular relational database management systems (RDBMS).
SQL Server is a relational database management system introduced by Microsoft corporation. The method has the advantages of convenient use, good scalability, high integration degree of related software and the like, and can be used across a plurality of platforms from a laptop running Microsoft Windows 98 to a server running a large-scale multiprocessor of Microsoft Windows 2012.
Mongodb, a distributed document storage database, aims to provide an extensible, high-performance data storage solution for WEB applications. May be used in many scenarios to replace traditional relational databases or key/value storage approaches. Mongo was developed using C + +.
DB2 is a series of relational database management systems produced by IBM, each serving on a different operating system platform.
The above is merely an example of the database systems in common use at present, and the database systems described in the present invention are not limited to the types and numbers of the above database systems.
CRUD refers to operations of adding (Create), reading query (Retrieve), updating (Update) and deleting (Delete) when performing calculation processing, and is mainly used for describing basic operation functions of a DataBase or a persistence layer in a software system.
The present invention is further described below in conjunction with the following figures and specific examples so that those skilled in the art may better understand the present invention and practice it, but the examples are not intended to limit the present invention.
The invention provides a partition autonomous distributed heterogeneous database management method, which divides different database systems into different partitions, respectively allocates different partition identification information to the different partitions,
according to the partition identification information, performing data interactive management among partitions:
the source partition sends the service command to be executed and the identification information of the source partition and the identification information of the destination partition to the transit processing layer,
the transfer processing layer converts the data of the service command into a data format which can be identified by the target subarea, sends the data to the target subarea,
the target subarea executes the service command to obtain related data, the source subarea identification information and the target subarea identification information are returned to the transfer processing layer,
and the transfer processing layer converts the related data into a data format which can be identified by the source partition, and sends the data format to the source partition to complete data interaction.
The method of the invention can be used for utilizing the database to the data most efficiently under different scenes, and different databases are used according to specific conditions, for example, a user selects a most suitable database management system according to actual conditions, for example: MongoDB is used for processing file-intensive data, Oracle is used for better support of transactions, and the like. These scenarios are different partitions throughout the distributed system, within a region, using the optimal database management system for the scenario, within a partition, the management of itself by its particular database management system, the different database management systems running on different partitions, through a transfer management layer, the function of a middle data management system is implemented, heterogeneous partitions are integrated into a complete generalized database management system, various databases under heterogeneous interconnection scenes can be enabled, the transaction unified processing of all or part of functions of the databases is realized, the integration of data and information fragments is more efficient, the management and the use of the whole database system are more convenient due to a unified abstract interface provided by a data management terminal, and the possibility of infinite expansion is provided at the same time, so that the application framework shown in the figure 1 is referred.
And the generalized database system is provided with a physical terminal and an abstract database, and corresponding intelligent or electronic equipment can be embedded into the physical terminal, so that the physical terminal can realize data management in a self range. Communication between different partitions also requires certain transmission paths, such as wired twisted pair transmission, optical fiber transmission, etc., and wireless digital microwave and analog microwave transmission, etc.
In an embodiment of the present invention, the partition identification information of an explicit partition includes a partition identifier and a database type identifier, when a source partition requests and transmits data to a destination partition, its own partition identifier and database type identifier, and a partition identifier and database type identifier of a destination partition are attached, after the data of the source partition reaches a relay processing layer, the relay processing layer needs to select a suitable data format conversion rule according to the database type identifier and the destination partition database type identifier of the source partition, correctly convert the data into a data format that can be recognized by the destination partition, and transmit the data related to the CRUD to the destination partition for processing, after the destination partition receives the data transmitted by the intermediate data management system, the destination partition makes a corresponding response or CRUD, if necessary, and then transmits the execution result to a new destination partition in a manner of transmitting the source data, refer to fig. 2;
for example, the MySQL database now located on partition 1 would execute a query command to the DB2 database located on partition 3 and display it on the MySQL terminal, as follows,
s1: the MySQL terminal enters the query statement to query the DB2 database of partition 3, partition 1 sends this command to the transit processing layer along with the necessary partition information,
s2: the transfer processing layer receives the command and the corresponding partition information, selects a proper data conversion function according to the partition information, converts the original MySQL format data into DB2 database format data,
s3: the relay processing layer sends the format-converted command to the DB2 database system of the partition 3, the DB2 database system executes the query operation to obtain the required data,
s4: after obtaining the data, the DB2 database system of partition 3 returns the data and the changed partition information to the transit processing layer,
s5: similar to the step S2, the intermediate data management system performs format conversion on the data according to the partition information, and sends the data transmitted from the partition 3 to the MySQL database of the partition 1,
s6: and the subarea 1 receives the query result, displays the query result and finishes the query.
The above embodiments specifically describe the inter-partition distinguishing identifier and the basis for the transfer processing layer to convert the data format, and exemplify a specific data interaction process.
In a general application scenario, if two tables of two different dbms are combined and written into three dbms, it is difficult to implement or impossible to perform such operation. Yet this can be easily achieved by another embodiment of the invention. Namely, when the source partition performs associated data interaction with more than one destination partition, one of the destination partitions is selected to integrate all the obtained related data, and the integrated related data is sent to the source partition through the transfer processing layer, for example, a specific process of requesting for a partition a to query (partition B, partition C) and writing a query result into a partition D:
the partition A sends a query request (B unit C), the request is converted into data which can be correctly processed by the partition B after being processed by a transfer processing layer and is forwarded to the partition B, namely the indicating processes of the first and the second in the figure 3;
after receiving the request, the partition B finds that the request needs not only the data inside the partition B but also the data of the partition C, and before replying to the request of the partition a, the partition B first sends a request of the unit C to the partition C, namely, an indicating process of the third step in fig. 3;
the request sent by B is received by the relay processing layer, and the relay processing layer also performs format conversion or translation on the data sent by B according to the partition information, and then sends the uneon request to C, i.e. the indicating process of the fourth in fig. 3;
c, after receiving the union request, searching required data in the partition C according to the request information and the conditions;
c returns the inquired data in the partition C to B through the conversion of the transfer processing layer according to the previous partition information, namely indicating processes of (fifth) and (sixth) in FIG. 3;
b, after receiving the union data screened by the partition C, executing union combined operation of the partition data and the partition C data in the partition to obtain a combined data result of the partition B and the partition C;
the result of B unit C is returned to A through the partition information when A sends (B unit C) request at the beginning, during which, the data of B unit is converted into the data format of partition A through the transfer processing layer, namely the indication process of (C) and (B) in FIG. 3;
after receiving the combined data result of B unit C, A sends the data to the partition D through the transfer processing layer, and performs the disk-dropping storage in the partition D, namely the indication process of ninc and the R in FIG. 3;
d returns a success status to A after successful storage, and informs A that the instruction is successfully executed, namely, the instruction is successfully executed in FIG. 3
And
the indicating process of (1).
So far, command execution ends where A queries (B units C) and writes the result to D.
In another embodiment of the present invention, a global index of a partition is established, information of the partition is searched through the global index, for a case that a user does not specify a sub-database of a certain partition to be operated, but only provides some information of the operation, such as specifying a table name that is desired to be deleted and modified, etc., a transit processing layer may perform processing through the global index, for example, a MySQL database located on partition 1 now needs to query a table Student, and the Student table is in a DB2 database of partition 2, and the steps are as follows:
the relay management layer receives the query command, looks up the global index according to the information provided by the user of the partition 1, such as the table name, tries to find the destination partition, i.e. the partition where the table is located,
if the search is successful, selecting a target partition according to the global index, generating an SQL statement in a target partition format, executing the operation, if the information of the database of the target partition is modified after the operation is completed, updating the global index,
and if the search fails, sequentially inquiring the table information of the database of each partition, judging which partition or partitions the object of the operation is, and updating the global index after the operation is finished.
In addition, the partition mentioned in the present invention is not limited in scope, and a partition may be only a single machine, or may be a set composed of a plurality of nodes of the same type, as long as they are managed by a database system of the same type, and the nodes in the entire distributed database system may be suitably partitioned according to the partition identifier and the partition database type identifier, so as to achieve the purpose of more optimal integration of cost and efficiency.
The invention also provides a partitioned autonomous distributed heterogeneous database management system, which comprises a partitioning module and a transfer management module,
the dividing module divides different database systems into different partitions, respectively allocates different partition identification information to the different partitions,
according to the partition identification information, the data interaction management is carried out between the partitions by using a transfer management module:
the source partition sends the service command to be executed and the identification information of the source partition and the identification information of the destination partition to the transfer processing module,
the transfer processing module converts the data of the service command into a data format which can be identified by the target subarea, sends the data format to the target subarea,
the target subarea executes the service command to obtain related data, the source subarea identification information and the target subarea identification information are returned to the transfer processing module,
and the transfer processing module converts the related data into a data format which can be identified by the source partition, and sends the data format to the source partition to complete data interaction.
The system of the invention can be used for utilizing the database to the data most efficiently under different scenes, and different databases are used according to specific conditions, for example, a user selects the most appropriate database management system according to actual conditions, for example: MongoDB is used for processing file-intensive data, Oracle is used for better support of transactions, and the like. These scenarios are different partitions throughout the distributed system, within a region, using the optimal database management system for the scenario, within a partition, the management of itself by its particular database management system, the different database management systems running on different partitions, through a transfer management layer, the function of a middle data management system is implemented, heterogeneous partitions are integrated into a complete generalized database management system, various databases under heterogeneous interconnection scenes can be enabled, the transaction unified processing of all or part of functions of the databases is realized, the integration of data and information fragments is more efficient, the management and the use of the whole database system are more convenient due to a unified abstract interface provided by a data management terminal, and the possibility of infinite expansion is provided at the same time, so that the application framework shown in the figure 1 is referred.
And the generalized database system is provided with a physical terminal and an abstract database, and corresponding intelligent or electronic equipment can be embedded into the physical terminal, so that the physical terminal can realize data management in a self range. Communication between different partitions also requires certain transmission paths, such as wired twisted pair transmission, optical fiber transmission, etc., and wireless digital microwave and analog microwave transmission, etc.
In an embodiment of the system of the present invention, the partition identification information of the partition module for specifying the partition includes a partition identifier and a database type identifier, when a source partition requests and transmits data to a destination partition, the source partition attaches its partition identifier and database type identifier and the partition identifier and database type identifier of the destination partition, after the data of the source partition reaches the transit management module, the transit management module needs to select a suitable data format conversion rule according to the database type identifier of the source partition and the database type identifier of the destination partition, correctly convert the data into a data format that can be recognized by the destination partition, and transmit the data related to the CRUD to the destination partition for processing, after the destination partition receives the data transmitted by the transit management module, the destination partition makes a corresponding response or CRUD, if necessary, then, the executed result is sent to the new destination partition in a manner of source data transmission, referring to fig. 2;
for example, the MySQL database now located on partition 1 would execute a query command to the DB2 database located on partition 3 and display it on the MySQL terminal, as follows,
s1: the MySQL terminal enters the query statement to query the DB2 database of partition 3, partition 1 sends this command to the transit management module along with the necessary partition information,
s2: the transfer management module receives the command and the corresponding partition information, selects a proper data conversion function according to the partition information, converts the original MySQL format data into DB2 database format data,
s3: the transit management module sends the format-converted command to the DB2 database system of the partition 3, the DB2 database system performs a query operation, obtains required data,
s4: after obtaining the data, the DB2 database system of partition 3 returns the data and the changed partition information to the transit management module,
s5: similar to the step S2, the relay management module performs format conversion on the data according to the partition information, and sends the data transmitted from the partition 3 to the MySQL database of the partition 1,
s6: and the subarea 1 receives the query result, displays the query result and finishes the query.
The above embodiments specifically describe the partition distinguishing identifier and the basis for the transfer management module to convert the data format, and exemplify a specific data interaction process.
In a general application scenario, if two tables of two different dbms are combined and written into three dbms, it is difficult to implement or impossible to perform such operation. Yet this can be easily achieved by another embodiment of the invention. Namely, when the source partition performs associated data interaction with more than one destination partition, one of the destination partitions is selected to integrate all the obtained related data, and the integrated related data is sent to the source partition through the transfer management module, for example, a specific process of requesting for a partition a to query (partition B, partition C) and writing a query result into a partition D is performed:
the partition A sends a query request (B unit C), the request is converted into data which can be correctly processed by the partition B after being processed by the transfer management module and is forwarded to the partition B, namely the indicating processes of the first and the second in the figure 3;
after receiving the request, the partition B finds that the request needs not only the data inside the partition B but also the data of the partition C, and before replying to the request of the partition a, the partition B first sends a request of the unit C to the partition C, namely, an indicating process of the third step in fig. 3;
the request sent by B is received by the transit management module, and the transit management module also performs format conversion or translation on the data sent by B according to the partition information, and then sends the uneon request to C, i.e. the indicating process of the fourth in fig. 3;
c, after receiving the union request, searching required data in the partition C according to the request information and the conditions;
c returns the inquired data in the partition C to B through the conversion of the transfer management module according to the previous partition information, namely the indication processes of the fifth step and the sixth step in the figure 3;
b, after receiving the union data screened by the partition C, executing union combined operation of the partition data and the partition C data in the partition to obtain a combined data result of the partition B and the partition C;
the result of B unit C is returned to A through the partition information when A sends (B unit C) request at the beginning, during the period, the data of B partition is converted into the data format of A partition through the transfer management module, namely the indication process of (C) and (B) in FIG. 3;
after receiving the combined data result of B unit C, A sends the data to the partition D through the transfer management module, and performs the disk-dropping storage in the partition D, namely the indication process of ninc and the R in FIG. 3;
d returns a success status to A after successful storage, and informs A that the instruction is successfully executed, namely, the instruction is successfully executed in FIG. 3
And
the indicating process of (1).
So far, command execution ends where A queries (B units C) and writes the result to D.
In another embodiment of the present invention, a global index of a partition is established, information of the partition is searched through the global index, for a case that a user does not specify a sub-database of a certain partition to be operated, but only provides some information of the operation, such as specifying a table name that is desired to be deleted and modified, etc., the transit management module may perform processing through the global index, for example, a MySQL database located on partition 1 now queries a table Student, and the Student table is in a DB2 database of partition 2, and the steps are:
the transit management module receives the query command, looks up the global index according to the information provided by the user of partition 1, such as the table name, tries to find the destination partition, i.e. the partition where the table is located,
if the search is successful, selecting a target partition according to the global index, generating an SQL statement in a target partition format, executing the operation, if the information of the database of the target partition is modified after the operation is completed, updating the global index,
and if the search fails, sequentially inquiring the table information of the database of each partition, judging which partition or partitions the object of the operation is, and updating the global index after the operation is finished.
The invention also provides a partitioned autonomous distributed heterogeneous database management device, which comprises: at least one memory and at least one processor;
the at least one memory to store a machine readable program;
the at least one processor is configured to invoke the machine readable program to perform the partitioned autonomic distributed heterogeneous database management method.
And a computer readable medium having stored thereon computer instructions that, when executed by a processor, cause the processor to perform the method of partitioned autonomous distributed heterogeneous database management. Specifically, a system or an apparatus equipped with a storage medium on which software program codes that realize the functions of any of the above-described embodiments are stored may be provided, and a computer (or a CPU or MPU) of the system or the apparatus is caused to read out and execute the program codes stored in the storage medium.
In this case, the program code itself read from the storage medium can realize the functions of any of the above-described embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.
Examples of the storage medium for supplying the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (e.g., CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD + RW), a magnetic tape, a nonvolatile memory card, and a ROM. Alternatively, the program code may be downloaded from a server computer via a communications network.
Further, it should be clear that the functions of any one of the above-described embodiments may be implemented not only by executing the program code read out by the computer, but also by causing an operating system or the like operating on the computer to perform a part or all of the actual operations based on instructions of the program code.
Further, it is to be understood that the program code read out from the storage medium is written to a memory provided in an expansion board inserted into the computer or to a memory provided in an expansion unit connected to the computer, and then causes a CPU or the like mounted on the expansion board or the expansion unit to perform part or all of the actual operations based on instructions of the program code, thereby realizing the functions of any of the above-described embodiments.
It should be noted that not all steps and modules in the above flows and system structure diagrams are necessary, and some steps or modules may be omitted according to actual needs. The execution order of the steps is not fixed and can be adjusted as required. The system structure described in the above embodiments may be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by a plurality of physical entities, or some components in a plurality of independent devices may be implemented together.
In the above embodiments, the hardware unit may be implemented mechanically or electrically. For example, a hardware element may comprise permanently dedicated circuitry or logic (such as a dedicated processor, FPGA or ASIC) to perform the corresponding operations. The hardware elements may also comprise programmable logic or circuitry, such as a general purpose processor or other programmable processor, that may be temporarily configured by software to perform the corresponding operations. The specific implementation (mechanical, or dedicated permanent, or temporarily set) may be determined based on cost and time considerations.
The above-mentioned embodiments are merely preferred embodiments for fully illustrating the present invention, and the scope of the present invention is not limited thereto. The equivalent substitution or change made by the technical personnel in the technical field on the basis of the invention is all within the protection scope of the invention. The protection scope of the invention is subject to the claims.
Claims (8)
1. A method for managing the partitioned and autonomous distributed heterogeneous database includes dividing different database systems into different partitions, assigning different partition ID information to different partitions,
according to the partition identification information, performing data interactive management among partitions:
the source partition sends the service command to be executed and the identification information of the source partition and the identification information of the destination partition to the transit processing layer,
the transfer processing layer converts the data of the service command into a data format which can be identified by the target subarea, sends the data to the target subarea,
the target subarea executes the service command to obtain related data, the source subarea identification information and the target subarea identification information are returned to the transfer processing layer,
and the transfer processing layer converts the related data into a data format which can be identified by the source partition, and sends the data format to the source partition to complete data interaction.
2. The method as claimed in claim 1, wherein the partition identification information includes partition identifiers and database type identifiers, and the relay processing layer selects an appropriate data format conversion rule according to the database type identifier of the source partition and the database type identifier of the destination partition, so as to convert the data to be converted into the recognizable data format.
3. The method as claimed in claim 1 or 2, wherein when the source partition interacts with the associated data of more than one destination partition, one of the destination partitions is selected to integrate all the obtained associated data, and the integrated associated data is sent to the source partition through the transfer processing layer.
4. The method as claimed in claim 3, wherein a global index of the partitions is established, and the information of the partitions is searched by the global index.
5. The method as claimed in claim 4, wherein the process of searching the partition information through the global index is as follows:
the transfer processing layer receives the command of inquiring the partition information, searches the global index according to the provided information, searches the corresponding partition information,
if the search is successful, the corresponding operation is executed according to the target partition searched by the global index,
and if the search fails, sequentially inquiring the information of each partition.
6. A partition autonomous distributed heterogeneous database management system is characterized by comprising a partitioning module and a transfer management module,
the dividing module divides different database systems into different partitions, respectively allocates different partition identification information to the different partitions,
according to the partition identification information, the data interaction management is carried out between the partitions by using a transfer management module:
the source partition sends the service command to be executed and the identification information of the source partition and the identification information of the destination partition to the transfer processing module,
the transfer processing module converts the data of the service command into a data format which can be identified by the target subarea, sends the data format to the target subarea,
the target subarea executes the service command to obtain related data, the source subarea identification information and the target subarea identification information are returned to the transfer processing module,
and the transfer processing module converts the related data into a data format which can be identified by the source partition, and sends the data format to the source partition to complete data interaction.
7. A partition autonomous distributed heterogeneous database management device is characterized by comprising: at least one memory and at least one processor;
the at least one memory to store a machine readable program;
the at least one processor, configured to invoke the machine readable program to perform a partitioned autonomous distributed heterogeneous database management method according to any of claims 1 to 5.
8. Computer readable medium characterized in that said computer readable medium has stored thereon computer instructions which, when executed by a processor, cause said processor to perform a method of partitioned autonomous distributed heterogeneous database management according to any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010557254.9A CN111708761A (en) | 2020-06-18 | 2020-06-18 | Partitioned autonomous distributed heterogeneous database management system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010557254.9A CN111708761A (en) | 2020-06-18 | 2020-06-18 | Partitioned autonomous distributed heterogeneous database management system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111708761A true CN111708761A (en) | 2020-09-25 |
Family
ID=72541008
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010557254.9A Pending CN111708761A (en) | 2020-06-18 | 2020-06-18 | Partitioned autonomous distributed heterogeneous database management system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111708761A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112328704A (en) * | 2020-11-03 | 2021-02-05 | 成都中科大旗软件股份有限公司 | Method, system, computer equipment and storage medium for realizing multi-data source combined query |
| CN114866619A (en) * | 2022-04-29 | 2022-08-05 | 苏州浪潮智能科技有限公司 | Heterogeneous acceleration method, device and system and computer readable storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102841927A (en) * | 2012-07-18 | 2012-12-26 | 福建省新泽尔资讯科技有限公司 | Method and device for data interaction among a plurality of databases and distributed data interaction system |
| CN104462269A (en) * | 2014-11-24 | 2015-03-25 | 中国联合网络通信集团有限公司 | Isomerous database data exchange method and system |
| CN105808776A (en) * | 2016-03-29 | 2016-07-27 | 中国建设银行股份有限公司 | Data management system and method of distributed database |
| CN107656980A (en) * | 2017-09-07 | 2018-02-02 | 北京神州绿盟信息安全科技股份有限公司 | Applied to the method and distributed data base system in distributed data base system |
| US20190095485A1 (en) * | 2017-09-27 | 2019-03-28 | Sap Se | Local identifiers for database objects |
| CN110134683A (en) * | 2019-04-23 | 2019-08-16 | 中国地质大学（武汉） | The partition zone optimizing research method and system that magnanimity element stores in relational database |
-
2020
- 2020-06-18 CN CN202010557254.9A patent/CN111708761A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102841927A (en) * | 2012-07-18 | 2012-12-26 | 福建省新泽尔资讯科技有限公司 | Method and device for data interaction among a plurality of databases and distributed data interaction system |
| CN104462269A (en) * | 2014-11-24 | 2015-03-25 | 中国联合网络通信集团有限公司 | Isomerous database data exchange method and system |
| CN105808776A (en) * | 2016-03-29 | 2016-07-27 | 中国建设银行股份有限公司 | Data management system and method of distributed database |
| CN107656980A (en) * | 2017-09-07 | 2018-02-02 | 北京神州绿盟信息安全科技股份有限公司 | Applied to the method and distributed data base system in distributed data base system |
| US20190095485A1 (en) * | 2017-09-27 | 2019-03-28 | Sap Se | Local identifiers for database objects |
| CN110134683A (en) * | 2019-04-23 | 2019-08-16 | 中国地质大学（武汉） | The partition zone optimizing research method and system that magnanimity element stores in relational database |
Non-Patent Citations (3)
| Title |
|---|
| 李瑞轩,卢正鼎,吴炜,文珠穆: "一种多自治域层次互操作模型的系统框架" * |
| 赵卫东;刘永红;鄢涛;于曦;: "Oracle分区表和分区索引在VLDB中的研究" * |
| 马新娜,牛存良,石玉晶,马建华: "异构数据库互访中间件的设计和应用" * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112328704A (en) * | 2020-11-03 | 2021-02-05 | 成都中科大旗软件股份有限公司 | Method, system, computer equipment and storage medium for realizing multi-data source combined query |
| CN112328704B (en) * | 2020-11-03 | 2024-02-23 | 成都中科大旗软件股份有限公司 | Method, system, computer device and storage medium for implementing multiple data source joint inquiry |
| CN114866619A (en) * | 2022-04-29 | 2022-08-05 | 苏州浪潮智能科技有限公司 | Heterogeneous acceleration method, device and system and computer readable storage medium |
| CN114866619B (en) * | 2022-04-29 | 2023-08-08 | 苏州浪潮智能科技有限公司 | Heterogeneous acceleration method, device and system and computer readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109408551B (en) | Data query method and system, consensus method and system, device and storage medium | |
| CN109299110B (en) | Data query method and device, storage medium and electronic equipment | |
| CN108228817B (en) | Data processing method, device and system | |
| US10055509B2 (en) | Constructing an in-memory representation of a graph | |
| CN110309196A (en) | Block chain data storage and query method, apparatus, equipment and storage medium | |
| US7606813B1 (en) | Model consolidation in a database schema | |
| US20150161207A1 (en) | Assisting query and querying | |
| CN103150394A (en) | Distributed file system metadata management method facing to high-performance calculation | |
| CN105335403A (en) | Database access method and device, and database system | |
| US8209440B2 (en) | Device-configuration-information optimum arrangement method and device-configuration-information optimum arrangement system | |
| CN116955361A (en) | Method and system for searching key range in memory | |
| CN106547646B (en) | Data backup and recovery method and data backup and recovery device | |
| CN112699151B (en) | Data processing method, device, equipment and medium | |
| CN111708761A (en) | Partitioned autonomous distributed heterogeneous database management system and method | |
| US8756205B2 (en) | System and method for rule-based data object matching | |
| CN111241108A (en) | Key value pair-based KV system indexing method and device, electronic equipment and medium | |
| CN110036381B (en) | In-memory data search technique | |
| CN116521956A (en) | Graph database query method and device, electronic equipment and storage medium | |
| CN113722600B (en) | Data query method, device, equipment and product applied to big data | |
| US11188594B2 (en) | Wildcard searches using numeric string hash | |
| CN117453980A (en) | Metadata management, configuration page generation method, server and storage medium | |
| CN110716933B (en) | Novel urban rail train big data-oriented high-flexibility distributed index method | |
| CN111290714A (en) | Data reading method and device | |
| US20050091128A1 (en) | System and method for querying inventory | |
| CN111309932B (en) | Comment data query method, comment data query device, comment data query equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200925 |