CN104239417A - Dynamic adjustment method and dynamic adjustment device after data fragmentation in distributed database - Google Patents
Dynamic adjustment method and dynamic adjustment device after data fragmentation in distributed database Download PDFInfo
- Publication number
- CN104239417A CN104239417A CN201410409582.9A CN201410409582A CN104239417A CN 104239417 A CN104239417 A CN 104239417A CN 201410409582 A CN201410409582 A CN 201410409582A CN 104239417 A CN104239417 A CN 104239417A
- Authority
- CN
- China
- Prior art keywords
- burst
- rule
- data
- burst rule
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2379—Updates performed during online database operations; commit processing
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a dynamic adjustment method after data fragmentation in a distributed database. The dynamic adjustment method comprises the following steps: generating new fragmentation rules, and meanwhile, keeping old fragmentation rules; fetching all related fragmented data in each node in the distributed database according to the old fragmentation rules, and then, redistributing the fragmented data to all the nodes according to the new fragmentation rules; deleting the old fragmentation rules after the fragmented data are redistributed to all the nodes, and only keeping the new fragmentation rules, wherein performing read operation according to all the fragmentation rules, and performing write operation according to the newest fragmentation rules. The dynamic adjustment method provided by the invention has the positive effects that high availability and expansibility of the distributed database are ensured, and the distributed database can continue to provide services when node adjustment is performed; meanwhile, the performance of the distributed database is not influenced after background arrangement is finished, and the use ratio, on cluster nodes, of the distributed database is effectively ensured.
Description
Technical field
The invention belongs to distributed data base field, especially relate to dynamic adjusting method and device after a kind of distributed data base data fragmentation.
Background technology
Along with the fast development of informationization technology, large-scale database system needs the data volume of process and storage increasing, calculating becomes increasingly complex, challenge for performance is also increasing, performance, reliability, the demand of extensibility will be more and more stronger, and this time one, centralized database obviously can not meet demand.In order to adapt to the development need of applied business, distributed data base system is by Data distribution8 on the different nodes of computer network, and these data logically belong to same system.
In order to ensure higher performance and larger extendability, distributed data base have employed data fragmentation technology.Application data according to certain burst regular distribution on different back end, when application system is accurately inquired about or is inquired about among a small circle, according to burst rule, this inquiry only affects the burst node related to, and other node can continue externally to provide service, make the Concurrency Access performance that distributed data base externally can provide higher.Meanwhile, close data are deployed on a node according to unified rule, also make the maintenance and management of application data more convenient.
In traditional sharding method, there is a problem to need to solve: due to the needs of operation system, when burst rule occurs to change, need to arrange all data related to, owing to now relating to a large amount of data carrying work, the service of database can stop service.Meanwhile, due to these data because data volume is excessive, cannot transaction guarantee be provided during arrangement, once break down in arrangement process, likely cause the damage of all data.
As Fig. 1, in distributed data base system, tables of data is distribution table, and wherein the data type of field 1 is positive integer.The system cloud gray model initial stage only comprises two nodes, and using the odd even of field 1 as burst, tables of data is distributed on node 1 and node 2 tables of data by rule, as field 1 for singular value then the row data be stored on node 1, otherwise to be stored on node 2.Along with the development of business, data scale is increasing, original two nodes cannot meet the needs of operation system for data storage and access, so determine interpolation node, new burst rule is to field one numerical value 3 delivery, if delivery value is 1, then this row is stored on node 1, if delivery value is 2, then this row is stored on node 2, otherwise is stored on node.
In order to data after making the adjustment of data fragmentation can be queried to, distributed data base also needs the following process shown in Fig. 2 that performs:
Step 201, distributed data base system stops externally service, avoids the accumulation due to new data inserting to cause the generation of the inconsistent situation of data;
Data in node 1 and node 2 are all derived by step 202.Because legacy data in node is according to old burst rule burst, so old data cannot arrive according to new burst rule query, need these statistical conversion;
Step 203, data previous step derived merge, and eliminate the impact that old burst rule causes;
Step 204, deletes the legacy data on node 1 and node 2;
Step 205, data step 203 merged, import within node 1, node 2, node 3 according to new burst rule;
Step 206, Data Update is complete, and distributed data base system externally can provide service;
In above step, have a very long time, distributed data base system externally cannot provide service, and meanwhile, the derivation of data needs to take a large amount of physical storage devices.And once above step generation abnormal conditions, user data likely causes the loss that cannot retrieve.
In sum, the sharding method of existing distributed data base cannot ensure the high availability of distributed data base system when data fragmentation adjusts and high reliability, also cannot provide the node dynamic retractility required for distributed data base system.
Summary of the invention
The problem to be solved in the present invention is, after being particularly suitable for data fragmentation, distributed data base system can continue when not withdrawing business correctly to provide service, perform the performance that data preparation does not affect distributed data base system, for the dynamic expansion of distributed data base node provides solution by backstage simultaneously.
In order to solve the problems of the technologies described above, the design philosophy that the present invention adopts is: distributed data base system uses the burst rule of multiple version, and node changes or after the change of burst rule, old burst rule is not deleted immediately.Write operation uses the burst rule of redaction, and read operation uses the burst rule of all versions.Distributed data base system performs data housekeeping operation for new burst rule automatically on backstage, just deletes the burst rule of legacy version after data preparation.
For solving the problems of the technologies described above, the technical solution used in the present invention is:
Dynamic adjusting method after a kind of distributed data base data fragmentation, comprising:
Generate new burst rule, retain old burst rule simultaneously;
Take out the fragment data in all each nodes related to of distributed data base according to old burst rule, arrive each node according to the redistribution of new burst rule;
After redistribution to each node, delete old burst rule, only stay new burst rule;
Wherein, the data operation request that distributed data base receives is associated with the burst rule in the queue of burst rule, carries out read operation, carry out write operation according to up-to-date burst rule according to all burst rules.
Further, after described distributed data base data fragmentation, dynamic adjusting method comprises:
The burst rule of same data is added the queue of a burst rule, when the burst rule that generation one is new, and joined in corresponding burst rule queue, the multiple burst rules wherein in the queue of burst rule arrange according to time sequencing;
According to the time sequencing of burst rule in the queue of burst rule, the fragment data in each node related to is taken out according to burst initial in it rule, each node is arrived according to the redistribution of up-to-date burst rule, the data of all nodes related to when initial burst rule all according to up-to-date burst regular distribution in each node, delete initial burst regular;
Repeat last step until after only remaining up-to-date burst rule in the queue of burst rule, complete the dynamic conditioning of data.
Further, if the data operation request read operation that described distributed data base receives, obtain all burst rules in the queue of corresponding burst rule, gather according to the nodal information of all burst Rules; If write operation, get burst rule up-to-date in the queue of corresponding burst rule, according to up-to-date burst rule executable operations.
Further, the data in the burst rule of described same data and the logical relation between burst rule refer to, data set up corresponding relation according to feature name or a certain Property Name and burst rule.
Further, when after the burst rule that generation one is new, Data Dynamic adjustment can be carried out immediately, or formulate task scheduling at the appointed time in perform, or manually performed by user and operate.
Further, when described distributed data base receives and creates fragment request and amendment fragment request, new burst rule can all be generated.
Further, described burst rule comprises: Hash burst, or list burst, or scope burst.
Another problem that the present invention solves is to provide dynamic adjusting device after a kind of distributed data base data fragmentation, comprises:
Burst rule management, can carry out interpolation, the deletion of burst rule;
Burst Rule Builder, for generating new burst rule, and is added burst rule management;
Data point reuse actuator, takes out the fragment data in all each nodes related to of distributed data base according to old burst rule, arrive each node according to the redistribution of new burst rule;
First receiver, receives read data operation requests, carries out read operation according to all burst rules, and summarized results;
Second receiver, receives data writing operation request, carries out write operation according to up-to-date burst rule.
Further, described burst rule management is the list manager storing burst rule, sorts in chronological order, can carry out interpolation, the deletion of burst rule to burst rule.
The advantage that the present invention has and good effect are: the high availability and the extendability that have ensured distributed data base, and distributed data base can be made to continue to provide service when knot adjustment; Meanwhile, after backstage has arranged, do not affect the performance of distributed data base, effectively ensure the utilization factor of distributed data base to clustered node.
Accompanying drawing explanation
Fig. 1 is burst adjustment process flow diagram after existing distributed data base data fragmentation;
Fig. 2 is burst adjustment example schematic diagram after distributed data base system data fragmentation of the present invention;
Fig. 3 is burst rule queue schematic diagram in distributed data base in one embodiment of the invention;
Fig. 4 is distributed data base write operation process flow diagram in one embodiment of the invention;
Fig. 5 is distributed data base read operation process flow diagram in one embodiment of the invention;
Fig. 6 is that in one embodiment of the invention, distributed data base back-end data arranges process flow diagram.
Embodiment
After distributed data base data fragmentation of the present invention, the design philosophy of dynamic adjusting method is: distributed data base system uses the burst rule of multiple version, and node changes or after the change of burst rule, old burst rule is not deleted immediately; Distributed data base system performs data housekeeping operation for new burst rule automatically on backstage, just deletes the burst rule of legacy version after data preparation.The data operation request that in process, distributed data base receives is associated with the burst rule in the queue of burst rule, and write operation uses the burst rule of redaction, and read operation uses the burst rule of all versions.
Be described in detail below in conjunction with the method for accompanying drawing 2 to 6 to invention.One embodiment of the invention changes to example with the burst rule shown in Fig. 2: the initial burst rule of tables of data of distributed data base is the row of odd number for storing all field 1 values in node 1, and it is the row of even number that node 2 stores all field 1 values.After change, node 1 store all field 1 values except 3 deliverys be the row of 1, node 2 store all field 1 values except 3 deliverys be the row of 2, node 3 store all field 1 values except 3 deliverys be the value of 0.
First by the burst of distributed data base rule as except 2 deliverys, store with the form of queue, a queue correspond to the burst rule of same data, when adjusting the carrying out of data fragmentation rule, all generate a new burst rule as except 3 deliverys, and add in this burst queue, the multiple burst rules in the queue of burst rule arrange according to time sequencing; Wherein initial except 2 deliverys, the new burst rale store except 3 deliverys are inside the queue of a burst rule, so the change of the burst rule queue of field 1 as shown in Figure 3, after change, the original burst rule of this burst rule queue is not deleted, and just with the addition of new burst rule.When the establishment fragment request that distributed data base receives and amendment fragment request, be all that the carrying out of data fragmentation rule is adjusted, a new burst rule can be generated.
The logical relation that in distributed data base system of the present invention, queue correspond between the data of the burst rule of same data and burst rule refers to, data set up corresponding relation according to feature name or a certain Property Name and burst rule; And this is sent out burst rule adoptable and can be Hash burst, or list burst, or scope burst, thus logical relation be such as according to ' file name ' field by All Files according to the uniform burst of Hash mode in 3 nodes; Or fetch data table ' creation-time ' field according to time range burst in 5 nodes.
According to the time sequencing of burst rule in the queue of burst rule, the fragment data in each node related to is taken out according to burst initial in it rule, each node is arrived according to the redistribution of up-to-date burst rule, the data of all nodes related to when initial burst rule all according to up-to-date burst regular distribution in each node, delete initial burst regular;
Repeat last step until after only remaining up-to-date burst rule in the queue of burst rule, complete the dynamic conditioning of data.
Appoint and change to example with the burst rule shown in Fig. 2: the fragment data in each node that the burst rule taking-up needing basis to remove 2 deliverys relates to, each node is arrived according to the burst rule redistribution except 3 deliverys, the rear deletion that distributed, except the burst rule of 2 deliverys, is only left the burst rule except 3 deliverys in burst queue.
In order to make distributed data base performance best, data point reuse can be carried out at reasonable time, such as, after the burst rule that generation one is new, Data Dynamic adjustment can be carried out immediately, or formulate task scheduling at the appointed time interior execution, or manually to be performed by user and operate, determine according to configuration mode.
During the specific implementation of above-mentioned steps of the present invention, the Data Dynamic adjustment process of distributed data base as shown in Figure 6:
Step 601, distributed data base system starts back-end data and arranges;
Step 602, distributed data base, for the different pieces of information stored, judges that whether all field 1 data preparation be complete according to above-mentioned steps update all in this example, if upgrade complete execution step 611, otherwise execution step 603;
Step 603, takes out the queue of a data fragmentation rule, in this example the data fragmentation rule queue of unpack field 1 correspondence;
Step 604, judges whether only comprise a burst rule in the queue of burst rule, if only there is a burst rule, represents data point reuse complete execution step 602, otherwise perform step 605;
Step 605, in the queue of burst rule, takes out the oldest burst rule, in this example for removing 2 deliverys;
Step 606, each relate to node in of data point reuse in distributed data base, performs, the scope of node by previous step or the burst rule of getting obtain.In this example, the node related to is node 1, node 2; Judge whether that the data point reuse of all nodes is complete, if completed, perform step 610, otherwise perform step 607;
Step 607, according to the node listing that previous step obtains, chooses a unenforced node successively;
Step 608, in node, extract data according to the burst rule that step 605 obtains, such as burst rule is in this example " if (' field 1 ' %2==1) node=1; Else node=2; ", then node 1 filtercondition of data that should extract for " if (' field 1 ' %2==1) ";
The data that previous step derives are imported in distributed data base system according to new burst rule by step 609 again; In this example for again importing in 3 nodes of distributed data base system according to the burst rule except 3 deliverys;
Step 610, completes the arrangement of this burst rule, deletes burst rule; In this example for delete in burst list except 2 deliverys;
Step 611, completes whole fast resampling and arranges process.
Appoint and change to example with the burst rule shown in Fig. 2: how distributed data base system tackles data operation request is described in detail:
When the data operation request that distributed data base receives is write operation, to the write operation access process of fragment data as shown in Figure 4:
Database performs inquiry in a distributed manner " INSERT INTO ' tables of data ' (' field 1 ', ' field 2 ') VALUES (' 275', 1); " be example.
Step 401, according to access object, the burst rule queue that data query ' tables of data ' is corresponding;
Step 402, according to statement access type, ' INSERT ' operation is judged to be write operation;
Step 403, owing to being operating as write operation, so obtain up-to-date burst rule in the queue of burst rule; This burst rule is: " if (' field 1 ' %3==1) node=1; Else if (' field 1 ' %3==2) node=2; Else node=3; ";
Step 404, according to access queries condition, ' field 1 ' value is ' 275 ', by burst rule, judges that the row data answer burst on node 2;
Step 405, performs this query statement on node 2.
When the data operation request that distributed data base receives is read operation, distributed data base to the read operation access process of fragment data as shown in Figure 5:
Database performs inquiry in a distributed manner " SELECT*FROM ' tables of data ' WHERE ' field 1 '=' 275'; " be example.
Step 501, according to access object, the burst rule queue that data query ' tables of data ' is corresponding;
Step 502, according to statement access type, ' SELECT ' operation is judged to be read operation;
Step 503, owing to being operating as read operation, so obtain all burst rules in the queue of burst rule; Two the burst rules got are: " if (' field 1 ' %3==1) node=1; Elseif (' field 1 ' %3==2) node=2; Else node=3; " and " if (' field 1 ' %2==1) node=1; Else node=2; ";
Step 504, according to querying condition, ' field 1 ' value is ' 275 ', by burst rule 1, judges that qualified data may be positioned at node 1, judges that qualified data may be positioned on node 2 according to burst rule 2;
Step 505, being gathered by the node that previous step obtains, is node 1 and node 2;
Step 506, performs respectively by this inquiry on node 1 and node 2;
Step 507, obtains query results by previous step and merges, and return to user.
One embodiment of the invention, dynamic adjusting device after distributed data base data fragmentation, comprises:
Burst rule management, can carry out interpolation, the deletion of burst rule;
Burst Rule Builder, for generating new burst rule, and is added burst rule management;
Data point reuse actuator, takes out the fragment data in all each nodes related to of distributed data base according to old burst rule, arrive each node according to the redistribution of new burst rule;
First receiver, receives read data operation requests, carries out read operation according to all burst rules, and summarized results;
Second receiver, receives data writing operation request, carries out write operation according to up-to-date burst rule.
An alternative embodiment of the invention described burst rule management is defined as the list manager storing burst rule on the basis of said apparatus, sorts in chronological order, can carry out interpolation, the deletion of burst rule to burst rule.
Above one embodiment of the present of invention have been described in detail, but described content being only preferred embodiment of the present invention, can not being considered to for limiting practical range of the present invention.All equalizations done according to the present patent application scope change and improve, and all should still belong within patent covering scope of the present invention.
Claims (9)
1. a dynamic adjusting method after distributed data base data fragmentation, is characterized in that comprising:
Generate new burst rule, retain old burst rule simultaneously;
Take out the fragment data in all each nodes related to of distributed data base according to old burst rule, arrive each node according to the redistribution of new burst rule;
After redistribution to each node, delete old burst rule, only stay new burst rule;
Wherein, the data operation request that distributed data base receives is associated with the burst rule in the queue of burst rule, carries out read operation, carry out write operation according to up-to-date burst rule according to all burst rules.
2. dynamic adjusting method after distributed data base data fragmentation according to claim 1, is characterized in that comprising:
The burst rule of same data is added the queue of a burst rule, when the burst rule that generation one is new, and joined in corresponding burst rule queue, the multiple burst rules wherein in the queue of burst rule arrange according to time sequencing;
According to the time sequencing of burst rule in the queue of burst rule, the fragment data in each node related to is taken out according to burst initial in it rule, each node is arrived according to the redistribution of up-to-date burst rule, the data of all nodes related to when initial burst rule all according to up-to-date burst regular distribution in each node, delete initial burst regular;
Repeat last step until after only remaining up-to-date burst rule in the queue of burst rule, complete the dynamic conditioning of data.
3. dynamic adjusting method after distributed data base data fragmentation according to claim 2, it is characterized in that: if the data operation request read operation that described distributed data base receives, obtain all burst rules in the queue of corresponding burst rule, gather according to the nodal information of all burst Rules; If write operation, get burst rule up-to-date in the queue of corresponding burst rule, according to up-to-date burst rule executable operations.
4. dynamic adjusting method after distributed data base data fragmentation according to claim 2, it is characterized in that: the data in the burst rule of described same data and the logical relation between burst rule refer to, data set up corresponding relation according to feature name or a certain Property Name and burst rule.
5. dynamic adjusting method after distributed data base data fragmentation according to claim 1 and 2, it is characterized in that: when after the burst rule that generation one is new, Data Dynamic adjustment can be carried out immediately, or formulate task scheduling at the appointed time in perform, or manually performed by user and operate.
6. dynamic adjusting method after distributed data base data fragmentation according to claim 1, is characterized in that: when described distributed data base receives and creates fragment request and amendment fragment request, all can generate new burst rule.
7., according to dynamic adjusting method after the arbitrary described distributed data base data fragmentation of Claims 1-4, it is characterized in that: described burst rule comprises: Hash burst, or list burst, or scope burst.
8. a dynamic adjusting device after distributed data base data fragmentation, is characterized in that comprising:
Burst rule management, can carry out interpolation, the deletion of burst rule;
Burst Rule Builder, for generating new burst rule, and is added burst rule management;
Data point reuse actuator, takes out the fragment data in all each nodes related to of distributed data base according to old burst rule, arrive each node according to the redistribution of new burst rule;
First receiver, receives read data operation requests, carries out read operation according to all burst rules, and summarized results;
Second receiver, receives data writing operation request, carries out write operation according to up-to-date burst rule.
9. dynamic adjusting device after distributed data base data fragmentation according to claim 8, it is characterized in that: described burst rule management is the list manager storing burst rule, in chronological order burst rule is sorted, interpolation, the deletion of burst rule can be carried out.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410409582.9A CN104239417B (en) | 2014-08-19 | 2014-08-19 | Dynamic adjusting method and device after a kind of distributed data base data fragmentation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410409582.9A CN104239417B (en) | 2014-08-19 | 2014-08-19 | Dynamic adjusting method and device after a kind of distributed data base data fragmentation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104239417A true CN104239417A (en) | 2014-12-24 |
| CN104239417B CN104239417B (en) | 2017-06-09 |
Family
ID=52227477
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410409582.9A Active CN104239417B (en) | 2014-08-19 | 2014-08-19 | Dynamic adjusting method and device after a kind of distributed data base data fragmentation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104239417B (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104537078A (en) * | 2014-12-31 | 2015-04-22 | 天津南大通用数据技术股份有限公司 | Catalogue index optimizing method based on slider |
| WO2016116052A1 (en) * | 2015-01-21 | 2016-07-28 | Huawei Technologies Co., Ltd. | System and method for massively parallel processor database |
| CN106649418A (en) * | 2015-11-04 | 2017-05-10 | 江苏引跑网络科技有限公司 | High-performance method for importing data into distributed database through direct connection of fragments in driver |
| WO2017097059A1 (en) * | 2015-12-07 | 2017-06-15 | 中兴通讯股份有限公司 | Distributed database system and self-adaptation method therefor |
| CN107153680A (en) * | 2017-04-18 | 2017-09-12 | 北京思特奇信息技术股份有限公司 | A kind of method and system of the online expanding node of distributed memory database |
| CN107480185A (en) * | 2017-07-06 | 2017-12-15 | 联动优势科技有限公司 | A kind of data access method and equipment |
| CN107704554A (en) * | 2017-09-27 | 2018-02-16 | 广东亿迅科技有限公司 | The method and its system that data high-speed based on MPP databases imports |
| CN107943832A (en) * | 2017-10-23 | 2018-04-20 | 中国联合网络通信集团有限公司 | Method for processing business and device |
| CN109960573A (en) * | 2018-12-29 | 2019-07-02 | 天津南大通用数据技术股份有限公司 | A kind of cross-domain calculating task dispatching method and system based on Intellisense |
| CN110297859A (en) * | 2019-06-10 | 2019-10-01 | 恩亿科(北京)数据科技有限公司 | The method and apparatus for handling real time data |
| CN111290700A (en) * | 2018-12-10 | 2020-06-16 | 北京京东尚科信息技术有限公司 | Distributed data reading and writing method and system |
| WO2020224023A1 (en) * | 2019-05-08 | 2020-11-12 | 厦门网宿有限公司 | Reshard method and system in distributed storage system |
| CN112100160A (en) * | 2020-11-05 | 2020-12-18 | 四川新网银行股份有限公司 | Elastic Search based double-activity real-time data warehouse construction method |
| CN112860654A (en) * | 2019-11-27 | 2021-05-28 | 中国电信股份有限公司 | Data fragment processing method and device and storage medium |
| CN113609226A (en) * | 2021-08-09 | 2021-11-05 | 平安国际智慧城市科技股份有限公司 | Data export method and device, computer equipment and storage medium |
| CN113687846A (en) * | 2021-06-30 | 2021-11-23 | 北京百度网讯科技有限公司 | Method, apparatus, device and readable storage medium for processing data |
| WO2022001883A1 (en) * | 2020-06-28 | 2022-01-06 | 中兴通讯股份有限公司 | Data redistribution method and apparatus |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1549512A (en) * | 2003-05-13 | 2004-11-24 | ��Ϊ��������˾ | Method of frame relay partition strategy dynamic regulation along with speech |
| US20080082516A1 (en) * | 2006-09-28 | 2008-04-03 | Hiroshi Niina | System for and method of searching distributed data base, and information management device |
| CN102110154A (en) * | 2011-02-21 | 2011-06-29 | 中国科学院计算技术研究所 | File redundancy storage method in cluster file system |
| CN102402586A (en) * | 2011-10-24 | 2012-04-04 | 深圳华强电子交易网络有限公司 | Distributed data storage method |
| CN103399943A (en) * | 2013-08-14 | 2013-11-20 | 曙光信息产业(北京)有限公司 | Communication method and communication device for parallel query of clustered databases |
-
2014
- 2014-08-19 CN CN201410409582.9A patent/CN104239417B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1549512A (en) * | 2003-05-13 | 2004-11-24 | ��Ϊ��������˾ | Method of frame relay partition strategy dynamic regulation along with speech |
| US20080082516A1 (en) * | 2006-09-28 | 2008-04-03 | Hiroshi Niina | System for and method of searching distributed data base, and information management device |
| CN102110154A (en) * | 2011-02-21 | 2011-06-29 | 中国科学院计算技术研究所 | File redundancy storage method in cluster file system |
| CN102402586A (en) * | 2011-10-24 | 2012-04-04 | 深圳华强电子交易网络有限公司 | Distributed data storage method |
| CN103399943A (en) * | 2013-08-14 | 2013-11-20 | 曙光信息产业(北京)有限公司 | Communication method and communication device for parallel query of clustered databases |
Cited By (26)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104537078A (en) * | 2014-12-31 | 2015-04-22 | 天津南大通用数据技术股份有限公司 | Catalogue index optimizing method based on slider |
| CN104537078B (en) * | 2014-12-31 | 2018-02-27 | 天津南大通用数据技术股份有限公司 | A kind of directory index optimization method based on sliding block |
| WO2016116052A1 (en) * | 2015-01-21 | 2016-07-28 | Huawei Technologies Co., Ltd. | System and method for massively parallel processor database |
| US9959332B2 (en) | 2015-01-21 | 2018-05-01 | Futurewei Technologies, Inc. | System and method for massively parallel processor database |
| CN106649418A (en) * | 2015-11-04 | 2017-05-10 | 江苏引跑网络科技有限公司 | High-performance method for importing data into distributed database through direct connection of fragments in driver |
| WO2017097059A1 (en) * | 2015-12-07 | 2017-06-15 | 中兴通讯股份有限公司 | Distributed database system and self-adaptation method therefor |
| CN107153680A (en) * | 2017-04-18 | 2017-09-12 | 北京思特奇信息技术股份有限公司 | A kind of method and system of the online expanding node of distributed memory database |
| CN107480185A (en) * | 2017-07-06 | 2017-12-15 | 联动优势科技有限公司 | A kind of data access method and equipment |
| CN107704554A (en) * | 2017-09-27 | 2018-02-16 | 广东亿迅科技有限公司 | The method and its system that data high-speed based on MPP databases imports |
| CN107943832A (en) * | 2017-10-23 | 2018-04-20 | 中国联合网络通信集团有限公司 | Method for processing business and device |
| CN111290700A (en) * | 2018-12-10 | 2020-06-16 | 北京京东尚科信息技术有限公司 | Distributed data reading and writing method and system |
| CN109960573A (en) * | 2018-12-29 | 2019-07-02 | 天津南大通用数据技术股份有限公司 | A kind of cross-domain calculating task dispatching method and system based on Intellisense |
| CN109960573B (en) * | 2018-12-29 | 2021-01-08 | 天津南大通用数据技术股份有限公司 | Cross-domain computing task scheduling method and system based on intelligent perception |
| WO2020224023A1 (en) * | 2019-05-08 | 2020-11-12 | 厦门网宿有限公司 | Reshard method and system in distributed storage system |
| US11314701B2 (en) | 2019-05-08 | 2022-04-26 | Xiamen Wangsu Co. Ltd. | Resharding method and system for a distributed storage system |
| CN110297859A (en) * | 2019-06-10 | 2019-10-01 | 恩亿科(北京)数据科技有限公司 | The method and apparatus for handling real time data |
| CN110297859B (en) * | 2019-06-10 | 2021-07-06 | 恩亿科(北京)数据科技有限公司 | Method and apparatus for processing real-time data |
| CN112860654A (en) * | 2019-11-27 | 2021-05-28 | 中国电信股份有限公司 | Data fragment processing method and device and storage medium |
| CN112860654B (en) * | 2019-11-27 | 2024-01-30 | 中国电信股份有限公司 | Data slicing processing method, device and storage medium |
| WO2022001883A1 (en) * | 2020-06-28 | 2022-01-06 | 中兴通讯股份有限公司 | Data redistribution method and apparatus |
| CN112100160A (en) * | 2020-11-05 | 2020-12-18 | 四川新网银行股份有限公司 | Elastic Search based double-activity real-time data warehouse construction method |
| CN112100160B (en) * | 2020-11-05 | 2021-09-07 | 四川新网银行股份有限公司 | Elastic Search based double-activity real-time data warehouse construction method |
| CN113687846A (en) * | 2021-06-30 | 2021-11-23 | 北京百度网讯科技有限公司 | Method, apparatus, device and readable storage medium for processing data |
| CN113687846B (en) * | 2021-06-30 | 2023-07-18 | 北京百度网讯科技有限公司 | Method, apparatus, device and readable storage medium for processing data |
| CN113609226A (en) * | 2021-08-09 | 2021-11-05 | 平安国际智慧城市科技股份有限公司 | Data export method and device, computer equipment and storage medium |
| CN113609226B (en) * | 2021-08-09 | 2024-05-14 | 深圳平安智慧医健科技有限公司 | Data export method and device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104239417B (en) | 2017-06-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104239417A (en) | Dynamic adjustment method and dynamic adjustment device after data fragmentation in distributed database | |
| US11494407B2 (en) | Query plans for analytic SQL constructs | |
| CN108431804B (en) | Ability to group multiple container databases into a single container database cluster | |
| US11023448B2 (en) | Data scrubbing method and apparatus, and computer readable storage medium | |
| CN104123374B (en) | The method and device of aggregate query in distributed data base | |
| US10140351B2 (en) | Method and apparatus for processing database data in distributed database system | |
| CN109189852B (en) | Data synchronization method and device for data synchronization | |
| CN108259562B (en) | Data synchronization method and device based on multiple endpoints | |
| WO2017067117A1 (en) | Data query method and device | |
| US20150032695A1 (en) | Client and server integration for replicating data | |
| CN104361113B (en) | A kind of OLAP query optimization method under internal memory flash memory mixing memory module | |
| CN104881466B (en) | The processing of data fragmentation and the delet method of garbage files and device | |
| CN107247778A (en) | System and method for implementing expansible data storage service | |
| CN103605698A (en) | Cloud database system used for distributed heterogeneous data resource integration | |
| CN104657387B (en) | A kind of data query method and device | |
| WO2012119449A1 (en) | Method and system for configuring storage devices under hybrid storage environment | |
| CN105843933B (en) | The index establishing method of distributed memory columnar database | |
| US9959301B2 (en) | Distributing and processing streams over one or more networks for on-the-fly schema evolution | |
| CN110245134B (en) | Increment synchronization method applied to search service | |
| KR20190063835A (en) | System for processing real-time data modification of in-memory database | |
| CN106991190A (en) | A kind of database automatically creates subdata base system | |
| CN106547584A (en) | A kind of backing method and device of application program | |
| CN103365987A (en) | Clustered database system and data processing method based on shared-disk framework | |
| CN108073641A (en) | The method and apparatus for inquiring about tables of data | |
| CN111259062B (en) | Method and device capable of guaranteeing sequence of statement result set of full-table query of distributed database |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |