CN105159925A - Database cluster data distribution method and system - Google Patents
Database cluster data distribution method and system Download PDFInfo
- Publication number
- CN105159925A CN105159925A CN201510472474.0A CN201510472474A CN105159925A CN 105159925 A CN105159925 A CN 105159925A CN 201510472474 A CN201510472474 A CN 201510472474A CN 105159925 A CN105159925 A CN 105159925A
- Authority
- CN
- China
- Prior art keywords
- data
- database
- cluster
- write
- link
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2219—Large Object storage; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/80—Information retrieval; Database structures therefor; File system structures therefor of semi-structured data, e.g. markup language structured data such as SGML, XML or HTML
- G06F16/83—Querying
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a database cluster data distribution method and system. The method comprises: regularly detecting data storage states of all database sets in a database cluster, obtaining the database set with the data storage quantity exceeding a threshold value as a to-be-segmented database set, and setting the to-be-segmented database set to be in a segmentation state; transferring data in the to-be-segmented database set to a transfer database set; in response to a data write-in request of a client, if the current database set is in the segmentation state, obtaining the transfer database set corresponding to the current database set according to a segmentation rule, and writing data written by the client into the current database set and the transfer database set; and after the data in the to-be-segmented database set is transferred to the transfer database set, uninstalling the to-be-segmented database set from the database cluster. According to the database cluster data distribution method and system, the data is uniformly distributed and the influence on the data written by the client in segmentation and transfer processes is avoided at the same time.
Description
Technical field
The present invention relates to database-related techniques field, particularly a kind of data-base cluster data distributing method and system.
Background technology
Along with the variation to querying condition, and query performance requires more and more higher, and a lot of business is all with data-base cluster now, such as: SolrCloud, as query engine.Solr is an independently enterprise-level search application server, and be similar to database, it externally provides the api interface being similar to Web-service.User can pass through http request, submits the XML file of certain format to, generating indexes to search engine server; Also can propose search request by HttpGet operation, and obtain returning results of XML format.SolrCloud applies to several solr a set of Deployment System For Clusters provided.
But along with the growth of data volume, data just need to do burst and store, and the burst mechanism that data-base cluster provides well can not meet business demand, such as self-defined burst key data can not be uniformly distributed.
Summary of the invention
Based on this, be necessary that the burst mechanism for the data-base cluster of prior art well can not meet the technical matters of business demand, a kind of data-base cluster data distributing method and system are provided.
A kind of data-base cluster data distributing method, comprising:
Timing detecting step, comprising: the data storage condition of all database collections in timing Test database cluster, obtains memory data output and exceed the database collection of threshold values as database collection to be split, database collection to be split is set to cutting state;
Transfer step, comprise: treat the set of partition data storehouse and carry out pre-segmentation according to presetting segmentation rule, obtain the new collective number after splitting, create the transferring data storehouse set of new collective number, determine the multiple transferring data storehouses set corresponding to database collection to be split according to segmentation rule, the data batchmove in database collection to be split is gathered to transferring data storehouse;
Write step, comprise: client obtains the link of database collection to be written from the routing rule server of cluster, according to link to cluster write data, the router of cluster is in response to the write data message of client, using the database collection specified by the link of write data message as current database set, judge whether current database set is cutting state, if be cutting state, transferring data storehouse set then corresponding to the set of segmentation Rule current database, data client write write current database set and the set of transferring data storehouse simultaneously,
Database collection unloading step, comprising: after by the data batchmove in database collection to be split to transferring data storehouse set, unload database collection to be split from data-base cluster
A kind of data-base cluster data distribution system, comprising:
Timing detection module, for: the data storage condition of all database collections in timing Test database cluster, obtains memory data output and exceeds the database collection of threshold values as database collection to be split, database collection to be split is set to cutting state;
Shift module, for: treat the set of partition data storehouse and carry out pre-segmentation according to presetting segmentation rule, obtain the new collective number after splitting, create the transferring data storehouse set of new collective number, determine the multiple transferring data storehouses set corresponding to database collection to be split according to segmentation rule, the data batchmove in database collection to be split is gathered to transferring data storehouse;
Writing module, for: client obtains the link of database collection to be written from the routing rule server of cluster, according to link to cluster write data, the router of cluster is in response to the write data message of client, using the database collection specified by the link of write data message as current database set, judge whether current database set is cutting state, if be cutting state, transferring data storehouse set then corresponding to the set of segmentation Rule current database, data client write write current database set and the set of transferring data storehouse simultaneously,
Database collection Unload module, for: after by the data batchmove in database collection to be split to transferring data storehouse set, from data-base cluster, unload database collection to be split.
The present invention is by detecting data-base cluster timing, thus database collection memory data output being exceeded threshold values is divided into multiple database, be uniformly distributed to realize data, simultaneously for the client write request of data in segmentation transfer process, by simultaneously to the mode of two database writes, avoid impacting client write.
Accompanying drawing explanation
Fig. 1 is the workflow diagram of a kind of data-base cluster data distributing method of the present invention;
Fig. 2 is a most preferred embodiment of the present invention;
Fig. 3 is the construction module figure of a kind of data-base cluster data distribution system of the present invention.
Embodiment
Below in conjunction with the drawings and specific embodiments, the present invention will be further described in detail.
Be illustrated in figure 1 the workflow diagram of a kind of data-base cluster data distributing method of the present invention, comprise:
Step S101, comprising: the data storage condition of all database collections in timing Test database cluster, obtains memory data output and exceed the database collection of threshold values as database collection to be split, database collection to be split is set to cutting state;
Step S102, comprise: treat the set of partition data storehouse and carry out pre-segmentation according to presetting segmentation rule, obtain the new collective number after splitting, create the transferring data storehouse set of new collective number, determine the multiple transferring data storehouses set corresponding to database collection to be split according to segmentation rule, the data batchmove in database collection to be split is gathered to transferring data storehouse;
Step S103, comprise: client obtains the link of database collection to be written from the routing rule server of cluster, according to link to cluster write data, the router of cluster is in response to the write data message of client, using the database collection specified by the link of write data message as current database set, judge whether current database set is cutting state, if be cutting state, transferring data storehouse set then corresponding to the set of segmentation Rule current database, data client write write current database set and the set of transferring data storehouse simultaneously,
Step S104, comprising: after by the data batchmove in database collection to be split to transferring data storehouse set, unload database collection to be split from data-base cluster.
Step S101, carries out timing to the situation of data-base cluster and detects, and specifically can adopt a server disposing timed task specially, adopt timed task, each hour is detected once.Then segmentation and transfer is performed by step S102.Wherein, segmentation rule is preset by the keeper of cluster.In the process performing step S102, client writes data to data-base cluster, then triggered step S103, because cluster comprises multiple database collection, client can not determine the final database collection that will write data, therefore client is to the link that first can obtain database collection to be written during data-base cluster write data from routing rule server, then to data-base cluster write data, write data message first can be delivered to router, then judge whether the current database set that client will be accessed is cutting state by router, if cutting state, the data then client write write current database set and the set of transferring data storehouse simultaneously, therefore client can't need to upgrade its write target, data also can be kept in the set of transferring data storehouse simultaneously, if and the state of current database set is not cutting state, then adopt normal mode, the set of write current database.After transfer completes, database collection to be split unloads by execution step S104, then follow-up all inquiries to database or write, processes being gathered by new transferring data storehouse.
Be illustrated in figure 2 the workflow diagram of a most preferred embodiment of the present invention, data base set is combined into the SolrCloud comprising multiple solr databases, specifically comprises:
Step S201, timed task at set intervals, detects the Data distribution8 situation of all solr database collections, obtains the database collection that data exceed threshold values, this data base set is set and is combined into cutting state, then by the division of tasks function of message trigger step S202;
Step S202: division of tasks function, according to regular pre-segmentation database collection, obtains the quantity being divided into subdata base set.Create the new database set of corresponding quantity according to the quantity obtained, the data base set being set to cutting state in step S201 is combined into old database collection, by the Data dissemination before current point in time on old database collection in new database set;
Step S203, when client is toward solr server data inserting, by self-defining burst field, is routed to the current database set of actual insertion by algorithmic rule.Now judge that current database set is cutting state, if cutting state, obtain the name of new database set, carry out two write operation to current database set and new database set;
Step S204, after step S202 is complete, amendment routing rule, uninstalls old database collection.
The present invention is by detecting data-base cluster timing, thus database collection memory data output being exceeded threshold values is divided into multiple database collection, be uniformly distributed to realize data, data-base cluster can be allowed when data persistence increases, can be good at expansion, and allow data keep as much as possible being uniformly distributed.Simultaneously for the client write request of data in segmentation transfer process, by simultaneously to the mode of two database collection writes, avoid impacting client write.
Wherein in an embodiment, described step S104, specifically comprises:
From the routing rule server of cluster, the link pointing to database collection to be split replaced with and point to the set of transferring data storehouse, described client, when carrying out inquiry and write operation, obtains link from described routing rule server.
The mode of the present embodiment, makes the segmentation displacement behavior of client to data-base cluster transparent.
Wherein in an embodiment, described step S103, specifically comprises:
Client obtains the link of database collection to be written from the routing rule server of cluster, according to link to cluster write data, the router of cluster is in response to the write data message of client, using the database collection specified by the link of write data message as current database set, the router of cluster judges whether current database set is cutting state, if be cutting state, transferring data storehouse set then corresponding to the set of segmentation Rule current database, data client write are forwarded to current database set and the set of transferring data storehouse simultaneously.
Wherein in an embodiment, described client obtains the link of database collection to be written from the routing rule server of cluster, specifically comprise: client sends to the routing rule server of cluster the write request of data comprising burst field, obtains the link of the database collection to be written about described burst field from routing rule server.
Burst field is specified voluntarily by keeper or user, such as, different commodity can be specified to use different burst fields.It is freer that the present embodiment makes data distribute, and can tackle the sustainable growth of data.
Wherein in an embodiment, in described step S102, described in treat partition data storehouse set according to preset segmentation rule carry out pre-segmentation, obtain split after new collective number, specifically comprise:
Obtain the memory data output m of database collection to be split, and the preferred memory space k of the individual data storehouse set preset, the then new collective number n>m ÷ k after segmentation.
Preferably, a thread can be had to go the set that monitor data amount is large, if exceed the threshold values of needs segmentation, just separation calculation can be carried out, if what arrange is gather the data of all recommending within maintenance 5,000,000, just the calculating of the data of this set being divided into several new set can allow each collective data below 5,000,000, segmentation flow process is carried out after calculating completes, namely determine the multiple transferring data storehouses set corresponding to database collection to be split according to segmentation rule, the data batchmove in database collection to be split is gathered to transferring data storehouse.
Be illustrated in figure 3 the construction module figure of a kind of data-base cluster data distribution system of the present invention, comprise:
Timing detection module 301, for: the data storage condition of all database collections in timing Test database cluster, obtain memory data output and exceed the database collection of threshold values as database collection to be split, database collection to be split is set to cutting state;
Shift module 302, for: treat the set of partition data storehouse and carry out pre-segmentation according to presetting segmentation rule, obtain the new collective number after splitting, create the transferring data storehouse set of new collective number, determine the multiple transferring data storehouses set corresponding to database collection to be split according to segmentation rule, the data batchmove in database collection to be split is gathered to transferring data storehouse;
Writing module 303, for: client obtains the link of database collection to be written from the routing rule server of cluster, according to link to cluster write data, the router of cluster is in response to the write data message of client, using the database collection specified by the link of write data message as current database set, judge whether current database set is cutting state, if be cutting state, transferring data storehouse set then corresponding to the set of segmentation Rule current database, data client write write current database set and the set of transferring data storehouse simultaneously,
Database collection Unload module 304, for: after by the data batchmove in database collection to be split to transferring data storehouse set, from data-base cluster, unload database collection to be split.
Wherein in an embodiment, described database collection Unload module, specifically for:
From the routing rule server of cluster, the link pointing to database collection to be split replaced with and point to the set of transferring data storehouse, described client, when carrying out inquiry and write operation, obtains link from described routing rule server.
Wherein in an embodiment, said write module, specifically for:
Client obtains the link of database collection to be written from the routing rule server of cluster, according to link to cluster write data, the router of cluster is in response to the write data message of client, using the database collection specified by the link of write data message as current database set, the router of cluster judges whether current database set is cutting state, if be cutting state, transferring data storehouse set then corresponding to the set of segmentation Rule current database, data client write are forwarded to current database set and the set of transferring data storehouse simultaneously.
Wherein in an embodiment, described client obtains the link of database collection to be written from the routing rule server of cluster, specifically comprise: client sends to the routing rule server of cluster the write request of data comprising burst field, obtains the link of the database collection to be written about described burst field from routing rule server.
Wherein in an embodiment, in described shift module, described in treat partition data storehouse set according to preset segmentation rule carry out pre-segmentation, obtain split after new collective number, specifically comprise:
Obtain the memory data output m of database collection to be split, and the preferred memory space k of the individual data storehouse set preset, the then new collective number n>m ÷ k after segmentation.
The above embodiment only have expressed several embodiment of the present invention, and it describes comparatively concrete and detailed, but therefore can not be interpreted as the restriction to the scope of the claims of the present invention.It should be pointed out that for the person of ordinary skill of the art, without departing from the inventive concept of the premise, can also make some distortion and improvement, these all belong to protection scope of the present invention.Therefore, the protection domain of patent of the present invention should be as the criterion with claims.
Claims (10)
1. a data-base cluster data distributing method, is characterized in that, comprising:
Timing detecting step, comprising: the data storage condition of all database collections in timing Test database cluster, obtains memory data output and exceed the database collection of threshold values as database collection to be split, database collection to be split is set to cutting state;
Transfer step, comprise: treat the set of partition data storehouse and carry out pre-segmentation according to presetting segmentation rule, obtain the new collective number after splitting, create the transferring data storehouse set of new collective number, determine the multiple transferring data storehouses set corresponding to database collection to be split according to segmentation rule, the data batchmove in database collection to be split is gathered to transferring data storehouse;
Write step, comprise: client obtains the link of database collection to be written from the routing rule server of cluster, according to link to cluster write data, the router of cluster is in response to the write data message of client, using the database collection specified by the link of write data message as current database set, judge whether current database set is cutting state, if be cutting state, transferring data storehouse set then corresponding to the set of segmentation Rule current database, data client write write current database set and the set of transferring data storehouse simultaneously,
Database collection unloading step, comprising: after by the data batchmove in database collection to be split to transferring data storehouse set, unload database collection to be split from data-base cluster.
2. data-base cluster data distributing method according to claim 1, is characterized in that, described database collection unloading step, specifically comprises:
From the routing rule server of cluster, the link pointing to database collection to be split replaced with and point to the set of transferring data storehouse, described client, when carrying out inquiry and write operation, obtains link from described routing rule server.
3. data-base cluster data distributing method according to claim 1, is characterized in that, said write step, specifically comprises:
Client obtains the link of database collection to be written from the routing rule server of cluster, according to link to cluster write data, the router of cluster is in response to the write data message of client, using the database collection specified by the link of write data message as current database set, the router of cluster judges whether current database set is cutting state, if be cutting state, transferring data storehouse set then corresponding to the set of segmentation Rule current database, data client write are forwarded to current database set and the set of transferring data storehouse simultaneously.
4. data-base cluster data distributing method according to claim 1, it is characterized in that, described client obtains the link of database collection to be written from the routing rule server of cluster, specifically comprise: client sends to the routing rule server of cluster the write request of data comprising burst field, obtains the link of the database collection to be written about described burst field from routing rule server.
5. data-base cluster data distributing method according to claim 1, is characterized in that, in described transfer step, described in treat partition data storehouse set according to preset segmentation rule carry out pre-segmentation, obtain split after new collective number, specifically comprise:
Obtain the memory data output m of database collection to be split, and the preferred memory space k of the individual data storehouse set preset, the then new collective number n>m ÷ k after segmentation.
6. a data-base cluster data distribution system, is characterized in that, comprising:
Timing detection module, for: the data storage condition of all database collections in timing Test database cluster, obtains memory data output and exceeds the database collection of threshold values as database collection to be split, database collection to be split is set to cutting state;
Shift module, for: treat the set of partition data storehouse and carry out pre-segmentation according to presetting segmentation rule, obtain the new collective number after splitting, create the transferring data storehouse set of new collective number, determine the multiple transferring data storehouses set corresponding to database collection to be split according to segmentation rule, the data batchmove in database collection to be split is gathered to transferring data storehouse;
Writing module, for: client obtains the link of database collection to be written from the routing rule server of cluster, according to link to cluster write data, the router of cluster is in response to the write data message of client, using the database collection specified by the link of write data message as current database set, judge whether current database set is cutting state, if be cutting state, transferring data storehouse set then corresponding to the set of segmentation Rule current database, data client write write current database set and the set of transferring data storehouse simultaneously,
Database collection Unload module, for: after by the data batchmove in database collection to be split to transferring data storehouse set, from data-base cluster, unload database collection to be split.
7. data-base cluster data distribution system according to claim 6, is characterized in that, described database collection Unload module, specifically for:
From the routing rule server of cluster, the link pointing to database collection to be split replaced with and point to the set of transferring data storehouse, described client, when carrying out inquiry and write operation, obtains link from described routing rule server.
8. data-base cluster data distribution system according to claim 6, is characterized in that, said write module, specifically for:
Client obtains the link of database collection to be written from the routing rule server of cluster, according to link to cluster write data, the router of cluster is in response to the write data message of client, using the database collection specified by the link of write data message as current database set, the router of cluster judges whether current database set is cutting state, if be cutting state, transferring data storehouse set then corresponding to the set of segmentation Rule current database, data client write are forwarded to current database set and the set of transferring data storehouse simultaneously.
9. data-base cluster data distribution system according to claim 6, it is characterized in that, described client obtains the link of database collection to be written from the routing rule server of cluster, specifically comprise: client sends to the routing rule server of cluster the write request of data comprising burst field, obtains the link of the database collection to be written about described burst field from routing rule server.
10. data-base cluster data distribution system according to claim 6, is characterized in that, in described shift module, described in treat partition data storehouse set according to preset segmentation rule carry out pre-segmentation, obtain split after new collective number, specifically comprise:
Obtain the memory data output m of database collection to be split, and the preferred memory space k of the individual data storehouse set preset, the then new collective number n>m ÷ k after segmentation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510472474.0A CN105159925B (en) | 2015-08-04 | 2015-08-04 | A kind of data-base cluster data distributing method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510472474.0A CN105159925B (en) | 2015-08-04 | 2015-08-04 | A kind of data-base cluster data distributing method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105159925A true CN105159925A (en) | 2015-12-16 |
| CN105159925B CN105159925B (en) | 2019-08-30 |
Family
ID=54800782
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510472474.0A Active CN105159925B (en) | 2015-08-04 | 2015-08-04 | A kind of data-base cluster data distributing method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105159925B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106959951A (en) * | 2016-01-08 | 2017-07-18 | 北京国双科技有限公司 | The treating method and apparatus of database |
| CN107665246A (en) * | 2017-09-13 | 2018-02-06 | 北京知道未来信息技术有限公司 | Dynamic date migration method and chart database cluster based on chart database |
| CN107967135A (en) * | 2017-10-31 | 2018-04-27 | 平安科技（深圳）有限公司 | Computing engines implementation method, electronic device and storage medium |
| CN109344198A (en) * | 2018-09-19 | 2019-02-15 | 国网浙江省电力有限公司嘉兴供电公司 | Log system and sharding method based on MongoDB distributed type assemblies framework |
| CN110442565A (en) * | 2019-08-08 | 2019-11-12 | 深圳市今天国际物流技术股份有限公司 | A kind of data processing method, device, computer equipment and storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1870526A (en) * | 2005-12-20 | 2006-11-29 | 华为技术有限公司 | Method of internodal loading transfer in network accounting |
| US20100130733A1 (en) * | 2002-10-21 | 2010-05-27 | Biosearch Technologies, Inc. | Luminescent metal ion complexes |
| CN102867035A (en) * | 2012-08-28 | 2013-01-09 | 浪潮(北京)电子信息产业有限公司 | High-availability method and device of distributed document system cluster |
| CN102938001A (en) * | 2012-12-10 | 2013-02-20 | 曙光信息产业(北京)有限公司 | Data loading device and data loading method |
| CN102955845A (en) * | 2012-10-23 | 2013-03-06 | 北京亿赞普网络技术有限公司 | Data access method and device as well as distributed database system |
| CN103095804A (en) * | 2011-12-13 | 2013-05-08 | 微软公司 | Load Balancing In Cluster Storage Systems |
| CN103902735A (en) * | 2014-04-18 | 2014-07-02 | 中国人民解放军理工大学 | Application perception data routing method oriented to large-scale cluster deduplication and system |
| CN103914516A (en) * | 2014-02-25 | 2014-07-09 | 深圳市中博科创信息技术有限公司 | Method and system for layer-management of storage system |
| CN103973583A (en) * | 2013-01-28 | 2014-08-06 | 中兴通讯股份有限公司 | Load balancing optimizing method and system for distributed file system |
-
2015
- 2015-08-04 CN CN201510472474.0A patent/CN105159925B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100130733A1 (en) * | 2002-10-21 | 2010-05-27 | Biosearch Technologies, Inc. | Luminescent metal ion complexes |
| CN1870526A (en) * | 2005-12-20 | 2006-11-29 | 华为技术有限公司 | Method of internodal loading transfer in network accounting |
| CN103095804A (en) * | 2011-12-13 | 2013-05-08 | 微软公司 | Load Balancing In Cluster Storage Systems |
| CN102867035A (en) * | 2012-08-28 | 2013-01-09 | 浪潮(北京)电子信息产业有限公司 | High-availability method and device of distributed document system cluster |
| CN102955845A (en) * | 2012-10-23 | 2013-03-06 | 北京亿赞普网络技术有限公司 | Data access method and device as well as distributed database system |
| CN102938001A (en) * | 2012-12-10 | 2013-02-20 | 曙光信息产业(北京)有限公司 | Data loading device and data loading method |
| CN103973583A (en) * | 2013-01-28 | 2014-08-06 | 中兴通讯股份有限公司 | Load balancing optimizing method and system for distributed file system |
| CN103914516A (en) * | 2014-02-25 | 2014-07-09 | 深圳市中博科创信息技术有限公司 | Method and system for layer-management of storage system |
| CN103902735A (en) * | 2014-04-18 | 2014-07-02 | 中国人民解放军理工大学 | Application perception data routing method oriented to large-scale cluster deduplication and system |
Non-Patent Citations (1)
| Title |
|---|
| 孙亮，等: "《现代网络工程设计与应用研究》", 31 January 2014 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106959951A (en) * | 2016-01-08 | 2017-07-18 | 北京国双科技有限公司 | The treating method and apparatus of database |
| CN107665246A (en) * | 2017-09-13 | 2018-02-06 | 北京知道未来信息技术有限公司 | Dynamic date migration method and chart database cluster based on chart database |
| CN107665246B (en) * | 2017-09-13 | 2021-03-30 | 北京知道未来信息技术有限公司 | Dynamic data migration method based on graph database and graph database cluster |
| CN107967135A (en) * | 2017-10-31 | 2018-04-27 | 平安科技（深圳）有限公司 | Computing engines implementation method, electronic device and storage medium |
| CN109344198A (en) * | 2018-09-19 | 2019-02-15 | 国网浙江省电力有限公司嘉兴供电公司 | Log system and sharding method based on MongoDB distributed type assemblies framework |
| CN110442565A (en) * | 2019-08-08 | 2019-11-12 | 深圳市今天国际物流技术股份有限公司 | A kind of data processing method, device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105159925B (en) | 2019-08-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9020802B1 (en) | Worldwide distributed architecture model and management | |
| US9280381B1 (en) | Execution framework for a distributed file system | |
| CN105159925A (en) | Database cluster data distribution method and system | |
| JP6716727B2 (en) | Streaming data distributed processing method and apparatus | |
| CN101944124A (en) | Distributed file system management method, device and corresponding file system | |
| CN103095758B (en) | A kind of method processing file data in distributed file system and this system | |
| CN105160039A (en) | Query method based on big data | |
| CN103812939A (en) | Big data storage system | |
| CN103597474A (en) | Efficient indexing and searching of access control listed documents | |
| CN103246549B (en) | A kind of method and system of data conversion storage | |
| CN102236580B (en) | Method for distributing node to ETL (Extraction-Transformation-Loading) task and dispatching system | |
| CN103218404A (en) | Multi-dimensional metadata management method and system based on association characteristics | |
| CN102890682A (en) | Method for creating index, searching method, device and system | |
| CN105677904B (en) | Small documents storage method and device based on distributed file system | |
| CN102202087A (en) | Method for identifying storage equipment and system thereof | |
| CN107888666A (en) | A kind of cross-region data-storage system and method for data synchronization and device | |
| CN104881466A (en) | Method and device for processing data fragments and deleting garbage files | |
| CN105900093A (en) | Keyvalue database data table updating method and data table updating device | |
| CN103823846A (en) | Method for storing and querying big data on basis of graph theories | |
| JP6975153B2 (en) | Data storage service processing method and equipment | |
| CN106960011A (en) | Metadata of distributed type file system management system and method | |
| CN108681866B (en) | Waybill processing method, system, device and storage medium | |
| CN104166661A (en) | Data storage system and method | |
| CN103279489A (en) | Method and device for storing metadata | |
| CN103326925A (en) | Message push method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |