CN103064914A - Data processing system and method - Google Patents
Data processing system and method Download PDFInfo
- Publication number
- CN103064914A CN103064914A CN2012105567194A CN201210556719A CN103064914A CN 103064914 A CN103064914 A CN 103064914A CN 2012105567194 A CN2012105567194 A CN 2012105567194A CN 201210556719 A CN201210556719 A CN 201210556719A CN 103064914 A CN103064914 A CN 103064914A
- Authority
- CN
- China
- Prior art keywords
- data
- memory node
- level
- order
- memory
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention discloses a data distribution device which comprises a data classification unit and a data migration unit, wherein the data classification unit is used for classifying data stored in a plurality of storage nodes of a data processing system to obtain data of various stages according to a set data classification rule, wherein the data of various stages at least comprise first stage data and second stage data, the plurality of storage nodes at least comprise a first stage storage node and a second storage node; and the data migration unit is used for migrating the first stage data to the first storage node when the first stage data are not stored in the first storage node of the plurality of storage nodes as well as migrating the second stage data to the second storage node when the second stage data are not stored in the second storage node of the plurality of storage nodes. The invention also discloses a data processing method and system. Therefore, the efficiency of accessing data from mass data is improved.
Description
Technical field
The present invention relates to the microcomputer data processing field, more specifically, relate to data handling system, data processing method and data distribution device under a kind of large-scale data environment.
Background technology
The typical classification storage policy that existing storage system adopts is for being stored in frequent accessed data in the internal memory, and the algorithm that adopts different caching replacements updates stored in the data in the internal memory, because the restriction of memory headroom, not often accessed then being stored on the disk.Be convenient to like this locate fast in data query and the analytic process, thereby can reduce the expense of safeguarding and processing.
In addition, existing a lot of distributed memory systems, according to certain Distribution Strategy with the mass data distributed storage on a plurality of memory nodes.Yet along with the increase of data volume, interstitial content is constant, and internal memory will become larger bottleneck, and data location expense is also increasing.Carry out data query and analysis in such system, be bound to so that performance reduces along with the increase of data volume.Present solution is to increase memory node, expands.Can increase amount of ram after the expansion, thus can be in the to a certain degree lower randomly located hit rate of data that improves.But expanding node must consider to expand the efficient that can not produce too much Data Migration and not reduce the data processing simultaneously.
Summary of the invention
The problem that exists for solving prior art the present invention is directed to mass data processing a kind of data distribution device, data handling system and disposal route that can improve data access efficiency is provided.
According to an aspect of the present invention, provide a kind of data distribution device, having comprised:
The data staging unit, thereby be used for carrying out the data that classification obtains different stage according to the data that the data staging rule that arranges is stored a plurality of memory nodes of data disposal system, the data of described different stage comprise first order data and second level data at least, wherein, described a plurality of memory node comprises first order memory node and second level memory node at least;
Data migration unit is used for when a memory node of described a plurality of memory nodes does not store first order data for first order memory node, and the first order Data Migration of storing is arrived first order memory node; And, when a memory node in described a plurality of memory nodes does not store second level data for second level memory node, with the second level Data Migration stored to second level memory node.
In optional embodiment, described data staging unit, the data based described data staging rule that will load that also is used for receiving is carried out classification, obtains the described rank that will load data.Described data distribution device also comprises the data loading unit, also is used for will loading the memory node that data are loaded into appropriate level according to described rank with described.
In optional embodiment, described data migration unit also is used for will being stored in the described first order data of described first order memory node and/or being stored in described second level data in the memory node of the described second level according to the predefined time backuping to back-up storage node in described a plurality of memory node.
In optional embodiment, described data loading unit, also be used for according to described rank with described to load the memory node that data are loaded into appropriate level in, store the described data that will load into described back-up storage node.
In optional embodiment, described data staging rule comprises the access frequency of data and/or the importance of data.
In optional embodiment, described data migration unit and described data loading unit can be merged into a unit.
According to another aspect of the present invention, also provide a kind of data handling system, having comprised:
A plurality of memory nodes, described a plurality of memory nodes comprise first order memory node and second level memory node at least; And,
Data distribution device is used for carrying out following functions:
According to the data staging rule that arranges thereby the data of storing in described a plurality of memory nodes are carried out the data that classification obtains different stage, the data of described different stage comprise first order data and second level data at least;
When a memory node in described a plurality of memory nodes does not store first order data for first order memory node, the first order Data Migration of storing is arrived first order memory node; And,
When a memory node in described a plurality of memory nodes does not store second level data for second level memory node, with the second level Data Migration stored to second level memory node.
In optional embodiment, described data handling system also comprises data loading device, and the data based described data staging rule that will load that is used for receiving is carried out classification, obtains the described rank that will load data; And, to load in the memory node that data are loaded into appropriate level described according to described rank.
In optional embodiment, described data distribution device also is used for will being stored in the described first order data of described first order memory node and/or being stored in described second level data in the memory node of the described second level according to the predefined time backuping to back-up storage node in described a plurality of memory node.
In optional embodiment, described data loading device, also be used for according to described rank with described to load the memory node that data are loaded into appropriate level in, store the described data that will load into described back-up storage node.
In optional embodiment, described first order memory node is the on-line storage node, and described second level memory node is the offline storage node.
In optional embodiment, the described data staging rule of stating comprises the access frequency of data and/or the importance of data.
In optional embodiment, described data loading device and described data distribution device can be merged into a device.
In optional embodiment, described data handling system also comprises: the data query device, be used for definite data rank of wanting data query, and from the memory node of appropriate level, inquire data by querying condition according to determined data rank.
According to another aspect of the invention, also provide a kind of data processing method, having comprised:
Thereby carry out the data that classification obtains different stage according to the data of storing in a plurality of memory nodes of data staging rule to the data disposal system that arrange, the data of described different stage comprise first order data and second level data at least, and wherein said a plurality of memory nodes comprise first order memory node and second level memory node at least;
When a memory node in described a plurality of memory nodes does not store first order data for first order memory node, the first order Data Migration of storing is arrived first order memory node; And
When a memory node in described a plurality of memory nodes does not store second level data for second level memory node, with the second level Data Migration stored to second level memory node.
In optional embodiment, described data processing method also comprises: the data based described data staging rule that load that will receive is carried out classification, obtains the described rank that will load data; And, to load in the memory node that data are loaded into appropriate level described according to described rank.
In optional embodiment, described data staging rule comprises the access frequency of data and/or the importance of data; Perhaps,
In optional embodiment, described data processing method also comprises: will store the first order data in the described first order memory node and/or be stored in second level data in the memory node of the described second level according to the predefined time and backup to back-up storage node in described a plurality of memory node.
In optional embodiment, described data processing method also comprises: according to described rank with described to load in the memory node that data are loaded into appropriate level in, store the described data that will load into described back-up storage node.
The embodiment of the invention by large-scale data is carried out classification then adopt classification storage with the data of different stage in the memory node of different stage, can improve the treatment effeciency to mass data, especially improve efficient and the speed of visit data from mass data.
Description of drawings
Fig. 1 is the structural representation of data handling system according to an embodiment of the invention.
Fig. 2 is the structural representation of data handling system according to another embodiment of the present invention.
Fig. 3 is the structural representation of an embodiment that is used for according to an embodiment of the invention the data distribution device of data handling system.
Fig. 4 is the schematic flow sheet of data processing method according to an embodiment of the invention.
Embodiment
Below in conjunction with accompanying drawing the present invention is described in detail further.
Fig. 1 is the structural representation of data handling system according to an embodiment of the invention.As shown in Figure 1, described data handling system comprises a plurality of memory nodes.Wherein these a plurality of memory nodes can be divided into a plurality of ranks, for example comprise at least first order memory node 20A and second level memory node 20B.Every one-level memory node can comprise one or more memory node.
Data handling system also comprises data distribution device 40, is used for carrying out classification according to the data that the data staging rule is stored each memory node, thereby obtains the data of different stage.Data can be divided into first order data, second level data, third level data etc.
The data of having stored for each memory node, if comprise first order data and second level data after the data of storing in the memory node are graded, yet this memory node is divided into first order memory node, data distribution device 40 will be from this memory node with second level Data Migration to second level memory node.Similar ground, if for having first order data on the memory node of first order memory node, being not to have third level data on the memory node of third level memory node, data distribution device also can not move respectively first order memory node, third level memory node from the memory node at place with these first order data, third level data.
In optional embodiment, the data based described data staging rule that will load that data distribution device 40 also is used for newly receiving is carried out classification, obtains the described rank that will load data.For example to load data and can be the data that receive from client or other data handling systems.Then, data distribution device will load in the memory node that data are loaded into appropriate level described according to described rank.For example, load data based data staging rule and be classified as first order data, then these first order data are loaded into first order memory node.
In the optional embodiment of the present invention, described data staging rule comprises the access frequency of data and/or the importance of data.For example, data that can access frequency is high and most important data are stored in the cluster of first order memory node or first order memory node composition, the data that access frequency is lower and not too important data are stored in the cluster of second level memory node or second level memory node composition, by that analogy.
In an optional embodiment, first order memory node can be the on-line storage node, and second level memory node can be the offline storage node.Can carry out separated maintenance to on-line storage node and offline storage node like this.
Fig. 2 is the structural representation of data handling system according to another embodiment of the present invention.As shown in Figure 2, data handling system can comprise data distribution device 40, data loading device 50 and data query device 70.For improving the reliability of data handling system, after preventing the data memory node collapse of online and off-line, loss of data except first order memory node 20A, second level memory node 20B, also comprises back-up storage node 20C in a plurality of memory nodes of this data handling system.The back-up storage node can adopt the cheap lower storage medium of throughput, and data adopt the mode of compression storage more.Data on the back-up storage node can be by random access, just is used for the on-line storage node that recovers to lose and the data of offline storage node.
Data distribution device 40, the function of the data distribution device in can execution graph 1, also be used for to be stored in the described first order data of described first order memory node and/or to be stored in described second level data in the memory node of the described second level according to the predefined time backuping to back-up storage node 20C in described a plurality of memory node.In another optional embodiment, data distribution device 40 also be used for according to described rank with described to load the memory node that data are loaded into appropriate level in, store the described data that will load into back-up storage node 20C.The performed data that a plurality of memory nodes have been stored of data distribution device are carried out the operation of classification and migration and can be carried out by the Service Management node in the data handling system.
In embodiments of the present invention, when carrying out Data Migration, at first can be with Data Migration that for example important and/or access frequency is high in high level memory node, then the Data Migration that access frequency is low is in low-level memory node.The data that certainly, also can adopt inverted order or other modes to move different stage.
In the mass data processing system, mostly data are to become magnanimity along with passage of time increases gradually, so the importance of data and access frequency can be the time correlations that produces with data.For example, the data importance of general nearest generation is high, so data are newer, and general importance is higher and access frequency is also higher.In an example, the data of recent months even several years can be classified as importantly, this can be according to the Location of requirement time restriction of practical business.Then, can unify to safeguard to data important and/or that access frequency is high.As time goes on, can be that expired data are moved on the low-level node from high level memory node.And, can be to move in the not busy time interruption of business every day, and can not affect the normal operation of data handling system.In addition, Data Migration can realize that the data of a plurality of memory nodes move to a memory node.
In optional embodiment, the data based described data staging rule that will load that data loading device 50 is used for newly receiving is carried out classification, obtains the described rank that will load data.For example to load data and can be the data that receive from client or other data handling systems.Then, data distribution device will load in the memory node that data are loaded into appropriate level described according to described rank.For example, load data based data staging rule and be classified as first order data, then these first order data are loaded into first order memory node.In optional embodiment, data loading device 50, also be used for according to described rank with described to load the memory node that data are loaded into appropriate level in, store the described data that will load into described back-up storage node.
In optional embodiment, data handling system can also comprise data query device 70, is used for determining to want the data rank of data query, inquires data by querying condition according to determined data rank from the memory node of appropriate level.In the prior art, when visit data, need to the memory node of all rank be conducted interviews, and in embodiments of the present invention, data can be inquired about or have access to the memory node that only needs directly to access appropriate level according to the data rank, obviously improved search efficiency.
In an alternate embodiment of the present invention, can also according to the data load balance requirement, dispose a plurality of data distribution devices and data loading device.In addition, can also require to arrange according to data load balance the quantity of first order memory node and second level memory node, namely the quantity of the quantity of first order memory node possibility second level memory node is identical, also can be different.
Fig. 3 is the structural representation of an embodiment that is used for according to an embodiment of the invention the data distribution device of data handling system.As shown in Figure 3, data distribution device 40 comprises data staging unit 401, thereby be used for carrying out the data that classification obtains different stage according to the data that the data staging rule that arranges is stored a plurality of memory nodes of data disposal system, the data of described different stage comprise first order data and second level data at least.A plurality of memory nodes comprise first order memory node and second level memory node at least.
Data distribution device 40 also comprises data migration unit 403, is used for when a memory node of a plurality of memory nodes does not store first order data for first order memory node, and the first order Data Migration of storing is arrived first order memory node; And, when a memory node in described a plurality of memory nodes does not store second level data for second level memory node, with the second level Data Migration stored to second level memory node.
In optional embodiment, data staging unit 401, the data based described data staging rule that will load that also is used for receiving is carried out classification, obtains the described rank that will load data.Data distribution device 40 can also comprise data loading unit 405, also is used for will loading the memory node that data are loaded into appropriate level according to the rank that will load data that obtains.
In an optional embodiment, data migration unit 403 also is used for will being stored in the described first order data of described first order memory node and/or being stored in described second level data in the memory node of the described second level according to the predefined time backuping to back-up storage node in described a plurality of memory node.In an optional embodiment, data loading unit 405 also is used for will loading data and storing the back-up storage node into will load the memory node that data are loaded into appropriate level according to the rank that will load data when.
In embodiments of the present invention, described data staging rule comprises the access frequency of data and/or the importance of data.In embodiments of the present invention, data migration unit and data loading unit can be merged into a unit.
Need to prove that the structure of the data distribution device in the above-described data handling system only is some embodiments, be not intended to also should not be understood as the restriction to the structure of data distributor.The invention technician can understand, all should be included within the scope of the invention so long as can realize parts, device or the equipment of the corresponding function of data distribution device described here, even its inner structure is different from the structure of data distribution device 40 described here.
Fig. 4 is a kind of according to an embodiment of the invention data processing method.As shown in Figure 4, this data processing method comprises:
S401, thus carry out the data that classification obtains different stage according to the data of storing in a plurality of memory nodes of data staging rule to the data disposal system that arrange, and the data of described different stage comprise first order data and second level data at least.Wherein, described a plurality of memory node comprises first order memory node and second level memory node at least.
S402 when a memory node in described a plurality of memory nodes does not store first order data for first order memory node, arrives first order memory node with the first order Data Migration of storing.
S403, when a memory node in described a plurality of memory nodes does not store second level data for second level memory node, with the second level Data Migration stored to second level memory node.
Need to prove that the order of step S402 and S403 can be exchanged.
In optional embodiment, described data processing method also comprises: the data based described data staging rule that load that will receive is carried out classification, obtains the described rank that will load data; And, to load in the memory node that data are loaded into appropriate level described according to described rank.
In embodiments of the present invention, described data staging rule comprises the access frequency of data and/or the importance of data.
In optional embodiment, described data processing method can also comprise: will store the first order data in the described first order memory node and/or be stored in second level data in the memory node of the described second level according to the predefined time and backup to back-up storage node in described a plurality of memory node.
In optional embodiment, described data processing method can also comprise: according to described rank with described to load in the memory node that data are loaded into appropriate level in, store the described data that will load into described back-up storage node.
The embodiment of the invention obviously can be raised the efficiency for the processing of mass data by large-scale data being adopted the mode of classification storage.And, compare with simple node expansion scheme of the prior art, dwindled the scope that data are processed, to change to the improved efficiency that data service brings.In addition, because the differentiated control data so that the data of different stage can be treated with a certain discrimination, can adopt different hardware and softwares to satisfy the data handling utility of different stage based on this, so that on the whole can raising efficiency the time, have also reduced the cost of whole system.
Be not limited to any specific computing machine, virtual system or miscellaneous equipment at this algorithm that provides and realization.According to top description, it is apparent constructing the desired structure of this type systematic.In addition, the present invention is not also for any certain programmed language.Should be understood that and to utilize various programming languages to realize content of the present invention described here, and the top description that language-specific is done is in order to disclose preferred forms of the present invention.
Those skilled in the art are appreciated that and can adaptively change and they are arranged in one or more equipment different from this embodiment the module in the equipment among the embodiment.Can be combined into a module or unit or parts to the module among the embodiment or unit or assembly, and can be divided into a plurality of submodules or subelement or subassembly to them in addition.In such feature and/or process or unit at least some are mutually repelling, and can adopt any combination to disclosed all features in this instructions (comprising claim, summary and the accompanying drawing followed) and so all processes or the unit of disclosed any method or equipment make up.Unless in addition clearly statement, disclosed each feature can be by providing identical, being equal to or the alternative features of similar purpose replaces in this instructions (comprising claim, summary and the accompanying drawing followed).
In addition, although a large amount of details of the embodiment of the invention have been described in the application's the instructions, yet, can understand, the embodiment of the invention is not could implement in all detail situations.In some instances, be not shown specifically known method, structure and technology, so that be expressly understood inventive concept of the present invention.
The above only is preferred embodiment of the present invention, and is in order to limit the present invention, within the spirit and principles in the present invention not all, any modification of doing, is equal to replacement, improvement etc., all should be included within protection scope of the present invention.
Claims (12)
1. data distribution device comprises:
The data staging unit, thereby be used for carrying out the data that classification obtains different stage according to the data that the data staging rule that arranges is stored a plurality of memory nodes of data disposal system, the data of described different stage comprise first order data and second level data at least, wherein, described a plurality of memory node comprises first order memory node and second level memory node at least;
Data migration unit is used for when a memory node of described a plurality of memory nodes does not store first order data for first order memory node, and the first order Data Migration of storing is arrived first order memory node; And,
When a memory node in described a plurality of memory nodes does not store second level data for second level memory node, with the second level Data Migration stored to second level memory node.
2. according to claim 1 data distribution device is characterized in that, described data staging unit, and the data based described data staging rule that will load that also is used for receiving is carried out classification, obtains the described rank that will load data; And,
The data loading unit also is used for will loading the memory node that data are loaded into appropriate level according to described rank with described.
3. data distribution device according to claim 1, it is characterized in that, described data migration unit also is used for will being stored in the described first order data of described first order memory node and/or being stored in described second level data in the memory node of the described second level according to the predefined time backuping to back-up storage node in described a plurality of memory node; And/or,
Described data loading unit, also be used for according to described rank with described to load the memory node that data are loaded into appropriate level in, store the described data that will load into described back-up storage node.
4. according to claim 1 to 3 each described data distribution devices, it is characterized in that described data staging rule comprises the access frequency of data and/or the importance of data; Perhaps,
Described data migration unit and described data loading unit are merged into a unit.
5. data handling system comprises:
A plurality of memory nodes, described a plurality of memory nodes comprise first order memory node and second level memory node at least; And,
Data distribution device is used for carrying out following functions:
According to the data staging rule that arranges thereby the data of storing in described a plurality of memory nodes are carried out the data that classification obtains different stage, the data of described different stage comprise first order data and second level data at least;
When a memory node in described a plurality of memory nodes does not store first order data for first order memory node, the first order Data Migration of storing is arrived first order memory node; And,
When a memory node in described a plurality of memory nodes does not store second level data for second level memory node, with the second level Data Migration stored to second level memory node.
6. data handling system according to claim 5 is characterized in that, described data handling system also comprises data loading device, and the data based described data staging rule that will load that is used for receiving is carried out classification, obtains the described rank that will load data; And, to load in the memory node that data are loaded into appropriate level described according to described rank.
7. it is characterized in that according to claim 5 or 6 described data handling systems:
Described data distribution device also is used for will being stored in the described first order data of described first order memory node and/or being stored in described second level data in the memory node of the described second level according to the predefined time backuping to back-up storage node in described a plurality of memory node; And/or
Described data loading device, also be used for according to described rank with described to load the memory node that data are loaded into appropriate level in, store the described data that will load into described back-up storage node.
8. it is characterized in that according to claim 5 or 6 described data handling systems:
Described first order memory node is the on-line storage node, and described second level memory node is the offline storage node; Perhaps,
The described data staging rule of stating comprises the access frequency of data and/or the importance of data; Perhaps,
Described data loading device and described data distribution device are merged into a device.
9. data handling system according to claim 5 is characterized in that, described data handling system also comprises:
The data query device is used for definite data rank of wanting data query, inquires data by querying condition according to determined data rank from the memory node of appropriate level.
10. data processing method comprises:
Thereby carry out the data that classification obtains different stage according to the data of storing in a plurality of memory nodes of data staging rule to the data disposal system that arrange, the data of described different stage comprise first order data and second level data at least, and wherein said a plurality of memory nodes comprise first order memory node and second level memory node at least;
When a memory node in described a plurality of memory nodes does not store first order data for first order memory node, the first order Data Migration of storing is arrived first order memory node; And
When a memory node in described a plurality of memory nodes does not store second level data for second level memory node, with the second level Data Migration stored to second level memory node.
11. data processing method according to claim 10 is characterized in that, described data processing method also comprises:
The data based described data staging rule that will load that receives is carried out classification, obtain the described rank that will load data; And
To load in the memory node that data are loaded into appropriate level described according to described rank.
12. according to claim 10 or 11 described data processing methods, it is characterized in that described data processing method also comprises:
Described data staging rule comprises the access frequency of data and/or the importance of data; Perhaps,
To store the first order data in the described first order memory node and/or be stored in second level data in the memory node of the described second level according to the predefined time and backup to back-up storage node in described a plurality of memory node; Perhaps,
According to described rank with described to load in the memory node that data are loaded into appropriate level in, store the described data that will load into described back-up storage node.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012105567194A CN103064914A (en) | 2012-12-20 | 2012-12-20 | Data processing system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012105567194A CN103064914A (en) | 2012-12-20 | 2012-12-20 | Data processing system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103064914A true CN103064914A (en) | 2013-04-24 |
Family
ID=48107544
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2012105567194A Pending CN103064914A (en) | 2012-12-20 | 2012-12-20 | Data processing system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103064914A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103544285A (en) * | 2013-10-28 | 2014-01-29 | 华为技术有限公司 | Data loading method and device |
| CN103731505A (en) * | 2014-01-17 | 2014-04-16 | 中国联合网络通信集团有限公司 | Data distributed storage method and system |
| CN104021092A (en) * | 2014-06-17 | 2014-09-03 | 上海华兴数字科技有限公司 | Working condition data hierarchical storage structure and method |
| CN104636327A (en) * | 2013-11-06 | 2015-05-20 | 上海语镜汽车信息技术有限公司 | Distributed type stream data system based on incremental computation |
| CN105262740A (en) * | 2015-09-25 | 2016-01-20 | 瑞斯康达科技发展股份有限公司 | Big data transmission method and system |
| CN106095333A (en) * | 2016-06-02 | 2016-11-09 | 腾讯科技（深圳）有限公司 | A kind of data-storage system, date storage method and data storage center |
| CN106294671A (en) * | 2016-08-03 | 2017-01-04 | 杭州华三通信技术有限公司 | A kind of data migration method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101079902A (en) * | 2007-06-29 | 2007-11-28 | 清华大学 | A great magnitude of data hierarchical storage method |
| CN101361048A (en) * | 2006-02-03 | 2009-02-04 | 国际商业机器公司 | Restoring a file to its proper storage tier in an information lifecycle management environment |
| CN102508789A (en) * | 2011-10-14 | 2012-06-20 | 浪潮电子信息产业股份有限公司 | Grading storage method for system |
| CN102520882A (en) * | 2011-12-08 | 2012-06-27 | 浪潮电子信息产业股份有限公司 | Static random memory and implementation method thereof |
-
2012
- 2012-12-20 CN CN2012105567194A patent/CN103064914A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101361048A (en) * | 2006-02-03 | 2009-02-04 | 国际商业机器公司 | Restoring a file to its proper storage tier in an information lifecycle management environment |
| CN101079902A (en) * | 2007-06-29 | 2007-11-28 | 清华大学 | A great magnitude of data hierarchical storage method |
| CN102508789A (en) * | 2011-10-14 | 2012-06-20 | 浪潮电子信息产业股份有限公司 | Grading storage method for system |
| CN102520882A (en) * | 2011-12-08 | 2012-06-27 | 浪潮电子信息产业股份有限公司 | Static random memory and implementation method thereof |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103544285A (en) * | 2013-10-28 | 2014-01-29 | 华为技术有限公司 | Data loading method and device |
| CN104636327A (en) * | 2013-11-06 | 2015-05-20 | 上海语镜汽车信息技术有限公司 | Distributed type stream data system based on incremental computation |
| CN103731505A (en) * | 2014-01-17 | 2014-04-16 | 中国联合网络通信集团有限公司 | Data distributed storage method and system |
| CN104021092A (en) * | 2014-06-17 | 2014-09-03 | 上海华兴数字科技有限公司 | Working condition data hierarchical storage structure and method |
| CN105262740A (en) * | 2015-09-25 | 2016-01-20 | 瑞斯康达科技发展股份有限公司 | Big data transmission method and system |
| CN106095333A (en) * | 2016-06-02 | 2016-11-09 | 腾讯科技（深圳）有限公司 | A kind of data-storage system, date storage method and data storage center |
| CN106095333B (en) * | 2016-06-02 | 2023-09-26 | 腾讯科技（深圳）有限公司 | Data storage system, data storage method and data storage center |
| CN106294671A (en) * | 2016-08-03 | 2017-01-04 | 杭州华三通信技术有限公司 | A kind of data migration method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103064914A (en) | Data processing system and method | |
| CN100571281C (en) | Great magnitude of data hierarchical storage method | |
| CN103677654B (en) | The method and electronic equipment of a kind of data storage | |
| US8984085B2 (en) | Apparatus and method for controlling distributed memory cluster | |
| US8412739B2 (en) | Optimization of an upgrade process of an original system | |
| US10356150B1 (en) | Automated repartitioning of streaming data | |
| CN101944124A (en) | Distributed file system management method, device and corresponding file system | |
| CN103036994A (en) | Cloud storage system capable of achieving load balance | |
| US20170344546A1 (en) | Code dispersion hash table-based map-reduce system and method | |
| CN102938001A (en) | Data loading device and data loading method | |
| CN105701257A (en) | Data processing method and device | |
| CN111813805A (en) | Data processing method and device | |
| US20170124148A1 (en) | Index table based routing for query resource optimization | |
| US11308066B1 (en) | Optimized database partitioning | |
| CN109196807B (en) | Network node and method of operating a network node for resource distribution | |
| CN105528454A (en) | Log treatment method and distributed cluster computing device | |
| CN109885642A (en) | Classification storage method and device towards full-text search | |
| US11138178B2 (en) | Separation of computation from storage in database for better elasticity | |
| CN104573112B (en) | Page interrogation method and data processing node in OLTP Cluster Databases | |
| CN111159140A (en) | Data processing method and device, electronic equipment and storage medium | |
| Chang et al. | Integration and optimization of multiple big data processing platforms | |
| CN105117175B (en) | A kind of variable resistance type memory abrasion equilibrium method based on software translating layer | |
| US8990524B2 (en) | Management of data elements of subgroups | |
| Xu et al. | Enhancing HDFS with a full-text search system for massive small files | |
| US20130159961A1 (en) | Method and Arrangement for Planning and/or Programming a Project of an Industrial Automation Arrangement |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20130424 |
|
| RJ01 | Rejection of invention patent application after publication |