CN105404638A - Method for solving correlated query of distributed cross-database fragment table - Google Patents
Method for solving correlated query of distributed cross-database fragment table Download PDFInfo
- Publication number
- CN105404638A CN105404638A CN201510625406.3A CN201510625406A CN105404638A CN 105404638 A CN105404638 A CN 105404638A CN 201510625406 A CN201510625406 A CN 201510625406A CN 105404638 A CN105404638 A CN 105404638A
- Authority
- CN
- China
- Prior art keywords
- burst
- customers
- burst table
- orders
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24553—Query execution of query operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24553—Query execution of query operations
- G06F16/24554—Unary operations; Data partitioning operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2471—Distributed queries
Abstract
The invention discloses a method for solving the correlated query of a distributed cross-database fragment table. The method comprises the following steps: constructing a correlated relationship between a CUSTOMERS fragment table and an ORDERS fragment table; appointing the table name, all distributed nodes, the fragment field and the fragment rule of the CUSTOMERS fragment table; meanwhile, appointing the table name, the foreign key listing and the listing of a list quoted by a foreign key of the ORDERS fragment table; and causing a storage operation of the CUSTOMERS fragment table to be the normal storage operation, and causing the storage operation of the ORDERS fragment table to be executed according to a node where associated CUSTOMERS fragment table data is positioned, wherein the CUSTOMERS fragment table is a conventional fragment table, and the CUSTOMERS fragment table needs to be associated with the conventional fragment table.
Description
Technical field
The present invention relates to network communication technology field, be specifically related to a kind of method solving distributed inter-library burst table correlation inquiry.
Background technology
1, solving at present the scheme of distributed inter-library burst table association, is the mode by data redundancy, by the whole redundancy of table data that is associated on all node databases, with this solve distributed in the difficult problem of inter-library burst correlation inquiry.As shown in Figure 1: if the Data distribution8 of CUSTOMERS burst table is in two databases dn1, dn2, and CUSTOMERS burst table needs to carry out correlation inquiry with ORDERS burst table, so just the total data of ORDERS burst table is stored in respectively in those two databases (dn1, dn2), now ORDERS burst table is called global table, and data must be consistent in distributed lower all child nodes for ORDERS burst table.Although the problem of distributed inter-library burst table association can be solved by the mode of redundant data, need to sacrifice a lot of data space, and there is performance issue, illustrate as follows:
If CUSTOMERS burst table is distributed to the database of N platform machine and needs to associate with M table, so this M table must be configured to global table, namely need all necessary redundancy of all data shown this M in the correspondence database of N platform machine, this needs to sacrifice a large amount of machine disk space; Secondly, for the data consistent managerial demand at substantial energy and time of this M table under distributed environment; Moreover if the data volume of some table in this M table is very large, the performance of meeting antithetical phrase node database causes certain influence, and the query performance of whole distributed type assemblies also phase strain differential.
Summary of the invention
The object of the invention is to address the deficiencies of the prior art, one is provided not waste extra data storage space, too large impact can not be caused to the system performance of each child node, also ensure that whole distributed type assemblies can have the method for the distributed inter-library burst table correlation inquiry of solution showing correlation inquiry performance comparatively efficiently, the technical scheme of employing is as follows simultaneously:
Solve a method for distributed inter-library burst table correlation inquiry, comprising:
Build the incidence relation between CUSTOMERS burst table and ORDERS burst table: specifies the table name of CUSTOMERS burst table, all nodes of distribution, burst field and burst regular; The row name of the row of the table of simultaneously specifying the table name of ORDERS burst table, foreign key column name and external key to quote;
The in-stockroom operation of CUSTOMERS burst table is made to be normal in-stockroom operation, and the in-stockroom operation of ORDERS burst table need perform in-stockroom operation according to the node at associated CUSTOMERS burst table data place, wherein CUSTOMERS burst table is conventional burst table, and ORDERS burst table and conventional burst table need the burst table associated.
By above setting, make the present invention without the need to being realized distributed inter-library burst table correlation inquiry by the mode of data redundancy, and when the table data volume involved by ORDERS burst table is larger, method CUSTOMERS burst table all contingency tables data being stored in same child node that the present invention adopts contributes to improving child node search efficiency, thus improves the search efficiency of whole distributed type assemblies.
As preferably, configured by XML file and represent the incidence relation between CUSTOMERS burst table and ORDERS burst table.
The incidence relation between distributed data base burst table and burst table is built by XML file, very clear, specific as follows:
<RootTablename="CUSTOMERS"shardNode="dn1,dn2"shardKey="id"shardRule="hash">
<ChildTablename="ORDERS"foreignKey="cid"referenceKey="id">
</ChildTable>
</RootTable>
Wherein, RootTable represents CUSTOMERS burst table, and name specifies the table name of CUSTOMERS burst table, shardNode and shardKey specifies all nodes and the burst field of the distribution of CUSTOMERS burst table, and shardRule specifies burst rule to be Hash.ChildTable represents ORDERS burst table, and name specifies the table name of ORDERS burst table, and foreignKey specifies foreign key column name, and referenceKey specifies the row name of which row of which table of foreign key reference.
As preferably, the in-stockroom operation of ORDERS burst table need perform in-stockroom operation according to the node at associated CUSTOMERS burst table data place and be specially: when relating to the in-stockroom operation of ORDERS burst table, first the sql statement that is searched the id of CUSTOMERS burst table is generated, and be issued to all node execution, when the result set of certain node feeding back has data, then carry out the in-stockroom operation of ORDERS burst table on this node.
By this method, in time performing burst table and burst table correlation inquiry at distributed type assemblies, burst table operation associated all localized, namely can fulfil ahead of schedule in each child node database, thus avoid the burst table association of cross-node database in distributed type assemblies, merged by this method and the data returned also be final accurately.
Compared with prior art, beneficial effect of the present invention: the present invention is without the need to realizing distributed inter-library burst table correlation inquiry by the mode of data redundancy, without the need to wasting extra storage space, and when the table data volume involved by ORDERS burst table is larger, method CUSTOMERS burst table all contingency tables data being stored in same child node that the present invention adopts contributes to improving child node search efficiency, thus improves the search efficiency of whole distributed type assemblies.
Accompanying drawing explanation
Fig. 1 is the schematic diagram that prior art solves the method for distributed inter-library burst table correlation inquiry;
Fig. 2 is the logical framework figure of the embodiment of the present invention;
Fig. 3 is the schematic diagram that the embodiment of the present invention solves the method for distributed inter-library burst table correlation inquiry.
Embodiment
Below in conjunction with drawings and Examples, the present invention is described in further detail.
Embodiment: a kind of method solving distributed inter-library burst table correlation inquiry, comprising:
Build the incidence relation between CUSTOMERS burst table and ORDERS burst table: specifies the table name of CUSTOMERS burst table, all nodes of distribution, burst field and burst regular; The row name of the row of the table of simultaneously specifying the table name of ORDERS burst table, foreign key column name and external key to quote;
The in-stockroom operation of CUSTOMERS burst table is made to be normal in-stockroom operation, and the in-stockroom operation of ORDERS burst table need perform in-stockroom operation according to the node at associated CUSTOMERS burst table data place, wherein CUSTOMERS burst table is conventional burst table, and ORDERS burst table and conventional burst table need the burst table associated.
In the present embodiment, configured by XML file and represent the incidence relation between CUSTOMERS burst table and ORDERS burst table.
The incidence relation between distributed data base burst table and burst table is built by XML file, very clear, specific as follows:
<RootTablename="CUSTOMERS"shardNode="dn1,dn2"shardKey="id"shardRule="hash">
<ChildTablename="ORDERS"foreignKey="cid"referenceKey="id">
</ChildTable>
</RootTable>
Wherein, RootTable represents CUSTOMERS burst table, and name specifies the table name of CUSTOMERS burst table, shardNode and shardKey specifies all nodes and the burst field of the distribution of CUSTOMERS burst table, and shardRule specifies burst rule to be Hash.ChildTable represents ORDERS burst table, and name specifies the table name of ORDERS burst table, and foreignKey specifies foreign key column name, and referenceKey specifies the row name of which row of which table of foreign key reference.
In the present embodiment, the in-stockroom operation of ORDERS burst table need perform in-stockroom operation according to the node at associated CUSTOMERS burst table data place and be specially: when relating to the in-stockroom operation of ORDERS burst table, first the sql statement that is searched the id of CUSTOMERS burst table is generated, and be issued to all node execution, when the result set of certain node feeding back has data, then carry out the in-stockroom operation of ORDERS burst table.
As shown in Figure 2, when relating to the in-stockroom operation of CUSTOMERS burst table and ORDERS burst table, warehouse-in treatment scheme is:
Carry out route processing by the burst table association process module of system to in-stockroom operation C1, C2, it is conventional burst table that system recognizes CUSTOMERS by the related information that XML configures, and according to routing rule, C1 is routed to node dn1, C2 is routed to node dn2;
By the burst table association process module of system to in-stockroom operation O1, O2 carries out route processing, it is the table be associated with CUSTOMERS that system recognizes ORDERS by the related information that XML configures, at this time system generates the statement that is searched CUSTOMERS burst table, search CUSTOMERS burst table record according to CID field and be routed on corresponding node, finally, the record of O1(CID=1 is stored in node dn1) be routed to node dn1, and the record of O2(CID=2 is stored in node dn2) be routed to node dn2.
As shown in Figure 3, by above setting, make the present invention without the need to being realized distributed inter-library burst table correlation inquiry by the mode of data redundancy, and when the table data volume involved by ORDERS burst table is larger, method CUSTOMERS burst table all contingency tables data being stored in same child node that the present invention adopts contributes to improving child node search efficiency, thus improves the search efficiency of whole distributed type assemblies.By this method, in time performing burst table and burst table correlation inquiry at distributed type assemblies, burst table operation associated all localized, namely can fulfil ahead of schedule in each child node database, thus avoid the burst table association of cross-node database in distributed type assemblies, merged by this method and the data returned also be final accurately.
Claims (3)
1. solve a method for distributed inter-library burst table correlation inquiry, it is characterized in that, comprising:
Build the incidence relation between CUSTOMERS burst table and ORDERS burst table: specifies the table name of CUSTOMERS burst table, all nodes of distribution, burst field and burst regular; The row name of the row of the table of simultaneously specifying the table name of ORDERS burst table, foreign key column name and external key to quote;
The in-stockroom operation of CUSTOMERS burst table is made to be normal in-stockroom operation, and the in-stockroom operation of ORDERS burst table need perform in-stockroom operation according to the node at associated CUSTOMERS burst table data place, wherein CUSTOMERS burst table is conventional burst table, and ORDERS burst table and conventional burst table need the burst table associated.
2. a kind of method solving distributed inter-library burst table correlation inquiry according to claim 1, be is characterized in that, configured and represent the incidence relation between CUSTOMERS burst table and ORDERS burst table by XML file.
3. a kind of method solving distributed inter-library burst table correlation inquiry according to claim 1, it is characterized in that, the in-stockroom operation of described ORDERS burst table need perform in-stockroom operation according to the node at associated CUSTOMERS burst table data place and be specially: when relating to the in-stockroom operation of ORDERS burst table, first the sql statement that is searched the id of CUSTOMERS burst table is generated, and be issued to all node execution, when the result set of certain node feeding back has data, then carry out the in-stockroom operation of ORDERS burst table on this node.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510625406.3A CN105404638A (en) | 2015-09-28 | 2015-09-28 | Method for solving correlated query of distributed cross-database fragment table |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510625406.3A CN105404638A (en) | 2015-09-28 | 2015-09-28 | Method for solving correlated query of distributed cross-database fragment table |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105404638A true CN105404638A (en) | 2016-03-16 |
Family
ID=55470128
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510625406.3A Pending CN105404638A (en) | 2015-09-28 | 2015-09-28 | Method for solving correlated query of distributed cross-database fragment table |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105404638A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105930407A (en) * | 2016-04-18 | 2016-09-07 | 北京思特奇信息技术股份有限公司 | Cross-database associated query method and system for distributed database |
| CN107644060A (en) * | 2017-08-25 | 2018-01-30 | 国网辽宁省电力有限公司 | A kind of intelligent grid cross-node join methods based on E R stripping strategies |
| CN107729370A (en) * | 2017-09-12 | 2018-02-23 | 上海艾融软件股份有限公司 | Micro services multi-data source connects implementation method |
| CN108021578A (en) * | 2016-11-03 | 2018-05-11 | 北京国双科技有限公司 | The relation query method and device of data file |
| CN109165262A (en) * | 2018-10-16 | 2019-01-08 | 成都索贝数码科技股份有限公司 | Fragmentation clustering system and fragmentation method of relational large table |
| CN113297250A (en) * | 2021-05-28 | 2021-08-24 | 北京思特奇信息技术股份有限公司 | Method and system for multi-table association query of distributed database |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120030246A1 (en) * | 2008-01-07 | 2012-02-02 | Akiban Technologies, Inc. | Multiple dimensioned database architecture supporting operations on table groups |
| CN102402586A (en) * | 2011-10-24 | 2012-04-04 | 深圳华强电子交易网络有限公司 | Distributed data storage method |
| CN102467570A (en) * | 2010-11-17 | 2012-05-23 | 日电（中国）有限公司 | Connection query system and method for distributed data warehouse |
| CN102831120A (en) * | 2011-06-15 | 2012-12-19 | 腾讯科技（深圳）有限公司 | Data processing method and system |
| CN103902614A (en) * | 2012-12-28 | 2014-07-02 | 中国移动通信集团公司 | Data processing method, device and system |
-
2015
- 2015-09-28 CN CN201510625406.3A patent/CN105404638A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120030246A1 (en) * | 2008-01-07 | 2012-02-02 | Akiban Technologies, Inc. | Multiple dimensioned database architecture supporting operations on table groups |
| CN102467570A (en) * | 2010-11-17 | 2012-05-23 | 日电（中国）有限公司 | Connection query system and method for distributed data warehouse |
| CN102831120A (en) * | 2011-06-15 | 2012-12-19 | 腾讯科技（深圳）有限公司 | Data processing method and system |
| CN102402586A (en) * | 2011-10-24 | 2012-04-04 | 深圳华强电子交易网络有限公司 | Distributed data storage method |
| CN103902614A (en) * | 2012-12-28 | 2014-07-02 | 中国移动通信集团公司 | Data processing method, device and system |
Non-Patent Citations (2)
| Title |
|---|
| 孙惠: ""基于Hadoop框架的大数据集连接优化算法"", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 陈慧英: ""基于NoSQL数据库的海量天文图像分布存储研究"", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105930407A (en) * | 2016-04-18 | 2016-09-07 | 北京思特奇信息技术股份有限公司 | Cross-database associated query method and system for distributed database |
| CN105930407B (en) * | 2016-04-18 | 2019-05-17 | 北京思特奇信息技术股份有限公司 | A kind of inter-library relation query method of distributed data base and system |
| CN108021578A (en) * | 2016-11-03 | 2018-05-11 | 北京国双科技有限公司 | The relation query method and device of data file |
| CN107644060A (en) * | 2017-08-25 | 2018-01-30 | 国网辽宁省电力有限公司 | A kind of intelligent grid cross-node join methods based on E R stripping strategies |
| CN107729370A (en) * | 2017-09-12 | 2018-02-23 | 上海艾融软件股份有限公司 | Micro services multi-data source connects implementation method |
| CN109165262A (en) * | 2018-10-16 | 2019-01-08 | 成都索贝数码科技股份有限公司 | Fragmentation clustering system and fragmentation method of relational large table |
| CN109165262B (en) * | 2018-10-16 | 2022-05-10 | 成都索贝数码科技股份有限公司 | Fragmentation clustering system and fragmentation method of relational large table |
| CN113297250A (en) * | 2021-05-28 | 2021-08-24 | 北京思特奇信息技术股份有限公司 | Method and system for multi-table association query of distributed database |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105404638A (en) | Method for solving correlated query of distributed cross-database fragment table | |
| US10127279B2 (en) | Eigenvalue-based data query | |
| CN106547796B (en) | Database execution method and device | |
| CN102609451B (en) | SQL (structured query language) query plan generation method oriented to streaming data processing | |
| CN103020301B (en) | A kind of multidimensional data query and storage means and system | |
| Zhao et al. | Modeling MongoDB with relational model | |
| CN105824957A (en) | Query engine system and query method of distributive memory column-oriented database | |
| CN107085570B (en) | Data processing method, application server and router | |
| US20110314027A1 (en) | Index building, querying method, device, and system for distributed columnar database | |
| CN104376017A (en) | Method and system for inter-database data synchronization | |
| CN104714972B (en) | Database divides table foundation and querying method | |
| CN102201010A (en) | Distributed database system without sharing structure and realizing method thereof | |
| WO2018040722A1 (en) | Table data query method and device | |
| CN104899295B (en) | A kind of heterogeneous data source data relation analysis method | |
| CN104885054A (en) | System and method for performing a transaction in a massively parallel processing database | |
| CN101246500B (en) | Retrieval system and method for implementing data fast indexing | |
| CN104391948A (en) | Data standardization construction method and system of data warehouse | |
| CN107330098B (en) | Query method, computing node and query system for custom report | |
| CN108241632B (en) | Data verification method oriented to database data migration | |
| CN103019728A (en) | Effective complex report parsing engine and parsing method thereof | |
| CN104008199A (en) | Data inquiring method | |
| CN103258036A (en) | Distributed real-time search engine based on p2p | |
| CN104504137A (en) | Data storage method and system | |
| CN102426587A (en) | Method for customizing and inquiring heterogeneous BOM (Bill of Materiel) based on complex product | |
| CN103927346A (en) | Query connection method on basis of data volumes |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160316 |